site speed

Advanced Mobile Optimization: How does it work? How do we measure success? [slides]

It’s been a busy couple of weeks, but I finally got around to posting the slides from my talk about advanced mobile optimization at the San Francisco & Silicon Valley Web Performance Meetup.

I always enjoy coming to these Meetups, and this time was no exception. Thanks again to Aaron Kulick for inviting me, to LinkedIn for hosting, and to the extremely keen and knowledgeable crowd who turned out. :)

Related posts:

New report: What we’ve learned from two years of watching the top 2,000 e-commerce websites

Yesterday was a busy day for us here at Strangeloop. We released our second annual “state of the union” report on the page speed and performance of 2,000 leading e-commerce websites.

Infographics: 2012 State of the Union for E-Commerce Page Speed and Website Performance

We came up with the idea of conducting these reports over a year ago, when we realized there was no way to track performance changes and trends — from a real user’s perspective — on an ongoing basis. As I mentioned to CNet’s Stephen Shankland, when we talked about the report the other day, we can’t rely on benchmarking tests run out of data centers to give us any relevant sense of how our sites are actually performing out in the real world. They’ve got ridiculous bandwidth, and they’re parked right next to the content delivery machines, so they’ve got zero latency. By using WebPagetest — which simulates how fast a site loads in a real user’s browser — to measure the performance of the same set of 2,000 sites year after year, we can compare real-world performance snapshots and see what’s changed, and what’s caused the changes.

I don’t want to give away all our findings, the highlights of which are illustrated in the infographics above, but a couple of interesting things we discovered:

  • Pages have gotten bigger – from an average of 86 requests last year to 98 requests this year. This kind of growth is consistent with trends identified via the HTTP Archive.
  • Repeat views are a whopping 20% slower than they were last year, probably due to the the number of page objects.
  • Page Speed scores have gotten significantly worse, from 83% last year to 75% now. I was just talking to Pat Meenan, who created WebPagetest, and he pointed out that a probable reason for the drop is because the newest version of Page Speed checks for more optimizations, which has resulted in lower scores across the board. (Side note: It’s important to know that Page Speed doesn’t take into account advanced front-end content optimization techniques — nor could it, because there are way too many of them to track and measure. As a for-instance, we’ve had experiences here at Strangeloop where we’ve taken a page with a perfect Page Speed score, accelerated it with Site Optimizer, cut load time in half, and ended up with a new Page Speed score of 74%.)

While the findings are a mixed bag of good and bad news, to me the most exciting takeaway is the  attention this report has received from major media outlets like CBS News. It goes to confirm what became evident over Black Friday: site speed has become a mainstream issue.

Download the report: 2012 State of the Union: E-Commerce Page Speed and Website Performance

Download a high-res version of the infographics above (feel free to re-post): Poster: E-Commerce Page Speed and Website Performance (2012)

Related links:

Interesting new findings about page views, time on site, and bounce rate across desktop and mobile browsers

Last month, I talked with Mac Slocum at O’Reilly Radar about mobile performance, and he asked me a couple of interesting question:

  • Are mobile users more or less tolerant of delays than desktop users?
  • Are users of one type of system more accepting of delays than users of another?

These questions are a gateway to a fascinating area of research, because they lead into a topic that we all have pet theories about (i.e. Chrome users are more tech savvy than the average person, while Internet Explorer users are less) but have little statistical evidence to back up.

I told Mac that I planned to do more digging and report back, so here I am.

Methodology

  1. I took five e-commerce sites (full sites, not mobile versions) that Strangeloop is currently accelerating and pulled their entire transaction volume over the past month — totaling hundreds of millions of unique visits via desktop and mobile. While desktop transactions outnumbered mobile transactions, the mobile numbers were still statistically significant: the smallest set of mobile numbers comprised around 200,000 unique visits and the largest set comprised about 20 million unique visits.
  2. I extracted the following data: page views, time on site, and bounce rate.
  3. I sorted the data into the following browser/OS groups: Internet Explorer, Firefox, Chrome, iPad, iPhone, and Android (phone).
  4. I calculated the averages for each metric and browser.
  5. I graphed the numbers and looked for trends. Some interesting patterns emerged:

Desktop vs. mobile performance: average pageviewsDesktop vs mobile performance: average time on siteDesktop vs mobile browser performance: average bounce rate

Three Key Findings

Finding #1: Internet Explorer users consistently view more pages, spend more time on site, and have a lower bounce rate than Firefox and Chrome users.

The average number of page views for IE users was 6.134, as opposed to 5.17 for Firefox users and 5.14 for Chrome users. IE users spent between 30-45 seconds longer on the site than other users, and their bounce rate was lower by 5 or 6 percentage points — a pretty significant difference. Could all of this substantiate the belief (among non-IE users, at least) that IE fans are less tech savvy and therefore slower and more ponderous web users than the rest of us? Or perhaps it’s a hardware issue — are IE users more likely to be using older systems with less processing power?

But what about that lower bounce rate? At around 35%, it’s a pretty strong number, especially compared to 41% for Firefox users and 42% for Chrome. A lower bounce rate generally signifies that people who come to your site find it relevant and worth sticking around to check out. Are IE users better searchers and more likely to arrive at the right destination, or are they simply more easily satisfied than other users?

Finding #2: iPad users are more similar to desktop users than they are to smartphone users.

While iPad users view somewhat fewer pages per visit than desktop users (4.54 versus 5.14, 5.17, and 6.13), their average time on site and bounce rate were commensurate with the desktop crowd. This isn’t a huge surprise. We know that most iPad users are browsing in the comfort of their home, and they consider their iPad to be more like a small laptop than an oversized phone. What’s interesting here is that, even though iPad performance lags behind desktop (it is a mobile device, after all, and it suffers from many of the same performance constraints as a smartphone: from low processor power to touchscreen lag), iPad users seem willing to stick around for a longer desktop-like experience.

Finding #3: iPhone users consistently view fewer pages, spend less time on site, and have a higher bounce rate.

At the opposite end of the spectrum from Internet Explorer users we find iPhone users. In every sample group, iPhone users, on average, spent significantly less time on site (2:31 vs 3:20) and viewed fewer pages (2.41 vs 3.1) than Android, and had a higher bounce rate (60.76% vs 57.17%). The shorter time spent on site could be attributed to the (arguable) fact that iPhones are better-powered than other devices, but that doesn’t account for the page views and bounce rate. Do these validate all the stereotypes about iPhone users: that they — or should I say, we — are impatient, savvy web users who will bounce from a site if we can’t find what we want right away and aggressively search elsewhere? Or that we know what we want and can expedite a transaction faster and more efficiently than other users?

Takeaways

This research doesn’t directly answer Mac’s questions, but it does come at them sideways and raise some interesting — to me, anyway — questions about the types of people who use different technologies, and how and why they use them. We can’t answer these questions today, but these findings are food for thought and debate.

Your thoughts?

Related posts:

O’Reilly Radar interview: You can’t get away with a bad mobile experience anymore

Last week, Mac Slocum invited me to chat with him about mobile web performance. He had some great questions about apps versus mobile sites, common optimization mistakes, and whether or not different device owners have different expectations when it comes to performance. I had a great time answering them, and it gave me some good food for thought for future areas of research. Win-win. :)

You can read the full interview on O’Reilly Radar: You can’t get away with a bad mobile experience anymore

Link roundup: Black Friday / Cyber Monday in review

By all accounts, this holiday season got off to a rocking start. Forbes estimates that over the Black Friday weekend, consumers set at least two new records: total amount spent ($11.4 billion) and increase in online purchasing (up 24.3% over last year).

For my own purposes, I wanted to gather all the e-commerce data from Black Friday weekend in one place. At the end of this post, I’ve also included a couple of links showing how sites fared from a performance perspective.

GigaOM: This holiday season shopping has gone mobile in a big way
“The number of consumers using a mobile device to visit a retailer’s site is 17.37% with almost 9.73% of consumers using mobile device to make a purchase.”

Internet Retailer: Thanksgiving Day online sales rise 39% over last year
“Heavy use of mobile devices, including smartphones and tablets like the iPad, were a big part of Thanksgiving Day online shopping. 15.2% of traffic came from mobile devices, up from 6.45% last Thanksgiving, and mobile devices accounted for 11.09% of purchases, well ahead of 4.25% last year, IBM says.”

Forbes: Implications of Black Friday: Apple, Amazon, eBay
“Mobile traffic was an increasing percentage of online traffic: 14.3% of total, versus 5.6% a year ago. Mobile purchasing as a percent of total more than tripled: 9.8% of online sales, versus 3.2% last year.”

Mobile Commerce Daily: PayPal global mobile payment volume increased 516% on Black Friday
“The company also saw a 371% increase in the number of customers shopping through mobile on Black Friday 2011 compared to last year.”

Akamai: Couch Commerce Takes a Cut at Midnight Doorbusters
“While nowhere near as explosive as Thanksgiving Day, the traffic growth continues to trend at much higher levels than 2010, on average 30% higher. The open question from Thanksgiving was whether Thanksgiving Day was just a shift of shopping activity from Black Friday, or a new shopping day in its own right. With Black Friday growth data so strong, it appears to be the latter.”

Internet Retailer: E-commerce shines on Black Friday
“Apple and Android devices account for virtually all of the mobile shopping activity, IBM says. 5.4% of traffic to retail web sites Friday came came from Apple iPhones, 4.8% from Apple iPads and 4.1% from devices that use Google’s Android software. That leaves less than 1% of traffic to retail sites coming from all other mobile devices. Shoppers using iPad converted at a much higher rate than other mobile consumers, 4.6% vs. 2.8% for users of all other mobile devices.”

Washington Post: Sales trackers say ‘Cyber Monday’ was biggest online shopping day in history
“Online sales rose 22% to $1.25 billion on Cyber Monday, when retailers ramp up online promotions, according to research firm comScore Inc. That makes it the biggest online shopping day ever, the research firm said. A year ago, Cyber Monday sales topped $1 billion for the first time.”

While all this online activity was going on, how did sites fare performance-wise?

Good, obviously, but performance monitoring data suggests that they could have been better:

Compuware: Countdown to Cyber Monday: Black Friday Retail Web Performance Results
“Some of the leading retailers are leaving shoppers frustrated with slow websites and periodic outages. For the Top 50 retailers, page load times spiked by 21 percent and 11 percent for the Top 10 retailers. What does this mean for online shoppers? They are experiencing less than satisfying web and mobile site performance when compared to a similar non-peak period.”

Catchpoint: CyberMonday 2011 – The results are in!
Catchpoint monitored 55 top e-commerce sites and found that the performance winners were Macy’s (with a start render time of 414ms) and JC Penney (with a load time of 695ms). In terms of availability, 10% of the sites they monitored suffered downtime of an hour or longer.

Related posts: