Early findings: 97% of mobile end-user response time happens at the front end

20 Apr 2011

49 Comments

Yesterday I was standing around talking with some of my fellow Strangeloopians, and someone brought up a couple of interesting questions:

  1. Everyone goes around quoting Steve Souders’s stat that 80% of performance issues happen at the front end, but this quote is almost four years old, dating back to the publication of Steve’s book High Performance Web Sites. Is this number still accurate?
  2. And what about mobile performance? To the best of our collective knowledge, this area has never been studied.

We’re in the fortunate position of being able to get to the bottom of these questions. We gather and store a lot of data for our customers, so it only took me about five minutes to look at the last five million desktop and mobile transactions (about a week’s worth) in our beacon database. (I stopped at five million because my pivot table in Excel would not handle more and my SQL query skills are non-existent.) These transactions occurred on a dozen or so sites.

Methodology

To do my calculation, I included all the reported pages, both landing and non-landing, which had document complete load times of less than 45 seconds. (We see huge outliers in some of the mobile browsers due to how multitasking works.*)

I conducted each calculation twice for verification. I also performed calculations in which I removed large outliers. In all cases, the results were consistent.

Results: Desktop vs mobile

Here’s the breakdown of where response time was spent, on average, for desktop versus mobile users:


 Back end Front end
Desktop 15% 85%
Mobile 3% 97%

This exercise confirms that the proportion of front end to back end time with desktops has remained about the same. What’s more interesting to me is the fact that the front end — shorthand for “everything after the HTML arrives” — was where 97% of mobile response time happens. I expected this number to be higher than the desktop number, but not this high.

I then did a quick analysis based on a few of the mobile browsers:


 Back end Front end
Android2 4% 96%
BB5 1% 99%
IEMobile7 1% 99%
iPad3 10% 90%
iPad4 15% 85%
iPhone3 2% 98%
iPhone4 1% 99%
iPod4 5% 95%

This raises new questions

This quick analysis raises a number of additional questions for the mobile data set:

  • What is the breakdown in the mobile space based on Wifi vs 3/4G networks?
  • How does the carrier affect these numbers?
  • What is the cache hit rate for the mobile browsers?
  • What is the typical latency for the mobile customer?

Lately, we’ve been stepping up our efforts to push key web performance stats out there to a mainstream audience. It’s crucial to keep revisiting these stats with real world data to make sure we’re pushing the right numbers.

Final caveat: I don’t think this exercise is the last word on this subject. We need a more controlled study and significantly more analysis.

*I always chuckle when I see major outliers in the data. Here are a few fun examples. If you’re wondering, 75190442ms equals roughly 21 days.

The front end is where 97% of mobile response time happens.

Related posts:

By: Tammy Everts Category: Tags: ,