End-of-year web performance report: Top retail sites are slower, not faster, than the rest of the pack

Benchmarking is an endless numbers game. Just when you accept one number, another comes along to burst your bubble. But sometimes not in the way you expected.

Case in point: Everyone wants to know how the big players measure up, hence the countless performance studies on everything from the Fortune 500 to the Internet Retail 500. We here at Strangeloop have done a few of our own. When we undertook our most recent study — a look at the top 2,000 retail sites according to Alexa — we got a few surprises.

Methodology

We used Webpagetest to test each site’s home page, via the server in Dulles, VA San Jose, CA, as it would appear to a visitor using IE7 on DSL. (Wondering “Why IE7?” Please read this.)

For the purposes of this study, we focused on these performance criteria:

  • Time to first byte, start render, document load time, and full load times for first and repeat views
  • Page Speed scores: Keep-alive, compress text, cache static content, combine JS/CSS, and use CDN

What we expected to find

We went into this study expecting to see three patterns emerge:

1. Average page load time of 7 seconds or less.
Last spring, Aptimize released a report that benchmarked the average Fortune 500 website at 7.066 seconds, following a virtually identical methodology. We expected that our results in this area would mirror theirs, or possibly be faster, given the fact that performance has become such a hot topic over the course of the year, especially in the retail sector.

2. Top-ranked sites load faster than lower-ranked sites.
We assumed that top-ranked sites would invest more heavily in performance optimization. As a result, we expected that the top 100 sites would perform better, on average, than the overall average for all 2,000 sites.

3. Generally poor Page Speed scores.
Based on our recent look at the Alexa Retail 1000, which revealed that almost half of these sites don’t follow two of the easiest performance best practices — enabling keep-alives and text compression — we anticipated that this trend would prevail.

What we actually found

Turns out, we got two out of three wrong. Or one out of three right, if you’re one of those half-full-glass types.

1. Average page load time was 11.21 seconds.

Site speed as indicated by time to first byte, start render, document complete time, and time to fully load

That’s right: the average page took more than 4 seconds longer to fully load than the widely accepted Fortune 500 benchmark of 7 seconds.

2. The Alexa-ranked top 100 sites were slower than the overall average, not faster.

Site speed as indicated by time to first byte, start render, document complete time, and time to fully load

Not only were the top 100 sites slower, they were much slower. The average load time was 14.39 seconds, 3.18 seconds slower than the overall average. (This kind of blows my mind.)

3. Page Speed scores were as poor as expected.

Performance grades: keep-alives, compress text, cache static content, combine JS/CSS, use CDN

While it’s heartening to see a significant number of sites enabling keep-alives — one of the easiest ways of immediately improving page speed — this histogram demonstrates that most sites had failing Page Speed grades.

Who was slowest and who was fastest

I was surprised at how many sites in the top 100 took 20 seconds or more to fully load. Here they are:

Slowest-loading sites in the Alexa Retail Top 100
Site Fully loaded (seconds)
autotrader.com 40.284
babycenter.com 33.251
kohls.com 33.237
nike.com 31.643
dickssportinggoods.com 27.044
barnesandnoble.com 24.925
sears.com 22.623
bhphotovideo.com 22.588
allposters.com 21.791
cabelas.com 21.539
macys.com 21.186
stubhub.com 20.466
landsend.com 20.448

At the opposite end of the spectrum, these were the sites from the top 100 that loaded in less than 5 seconds:

Fastest-loading sites in the Alexa Retail Top 100
Site Fully loaded (seconds)
ecrater.com 2.270
futureshop.ca 2.768
amazon.com 2.939
emusic.com 3.131
wellsfargo.com 3.146
etsy.com 3.354
6pm.com 3.571
bodybuilding.com 3.998
bestbuy.com 4.402
shopbop.com 4.411
netflix.com 4.445
audible.com 4.529
ebay.com 4.991

Some clear takeaways

Just because the performance community has made gains in some areas, this battle is still moving in an uphill direction. It’s too early to assume that (a) every online retailer even has performance as a stated goal, and (b) retailers have a clear sense of how to go about making their sites faster.

2010 may be the year that performance finally got on the map, but 2011 is the year that the map needs to get some serious eyeballs.

Related posts: