Firefox

More new findings: Top ecommerce sites are 22% slower than they were last year

27 Mar 2013

8 Comments

Let me say that again, because this is a staggering fact: The world’s top ecommerce sites are 22% slower than they were last year.

In December 2011, the median load time for a site in the Alexa Retail 2000 was 5.94 seconds. Just twelve months later, the median was 7.25 seconds. At this rate of growth, this number could hit almost 9 seconds by the end of this year.

Web page load time changes: December 2011 to December 2012

This was the key finding of our brand-new quarterly report (yes, a new report, not to be confused with last week’s report about web performance in the EU) on ecommerce web performance. If you’re new to these reports, since 2010 we’ve been measuring the load time, page composition, and best practice implementation of the same set of 2,000 leading online retailers, as ranked by Alexa. The goal is to learn how pages are changing over time and what impact, if any, these changes have on per-page performance. The results have been eye-opening.

I have to confess that I frequently feel like that tiresome guy at the party who keeps saying the same things over and over again. Pages are getting slower… pages are getting bigger… the gap between load times and user expectations is getting wider almost by the week. If you’re reading this out there and saying to yourself, “I can’t believe that Bixby guy is going on about this AGAIN,” then forward the next part of this post to five people you think could really benefit from it. If enough people internalize this message, maybe I’ll shut up.*

Three performance myths I would give anything to permanently bust:

Myth #1. Pages are, de facto, getting faster.

What with our better systems, networks, and browsers, pages must be getting faster, right? Everyone believes this instinctively, because most of us seem to be hardwired to believe that technology solves problems rather than creating new ones. But as I said at the top of this post, the quickly emerging fact is that pages seem to not just be getting slower, they’re getting slower at an alarming rate. (Optional: You may choose to take this finding as proof that we shouldn’t always trust our instincts. :) )

Myth #2. Users are more or less satisfied with the status quo.

“People are used to pages that take 5-8 seconds to load. They don’t mind that much.” I still hear this on a regular basis. Site owners rationalize that, because they’re not hearing a lot of complaints, their visitors are happy… or at least happy enough. But as numerous case studies have shown, people talk with their wallets. Faster sites earn more. And user surveys over the years are telling us that people’s expectations for a speedy online experience are continually growing:

Web Page Load Time: User expectations 200-2012

Myth #3. Browser development is more than capable of mitigating the factors, such as page size and complexity, that are causing pages to slow down.

This belief is widely held, even among technical folks. Again, looking to our findings, we saw that for all three browsers, median load times slowed down by anywhere from 3% to 12% in just six months. This downward trend isn’t a browser development issue. Instead, it’s an indicator that despite browser vendors’ huge commitment to speed, development can’t keep pace with the demands of bigger and increasingly complex web pages.
Browser performance: 2011 to 2012

Takeaway: Pass it on.

I am extremely happy that Radware is committed to continuing the tradition of releasing these quarterly  “state of the union” reports. As time passes, we’re gaining some invaluable insights. Based on these latest results, I’m very curious to see what our Summer 2013 report will hold.

I urge you to download this report (and the infographics, too). And I was only slightly kidding when I suggested that you forward this post to people who need to have a few performance myths dispelled. Our community does a lot of preaching to the choir. What seems basic to us is not necessarily basic to the rest of the world. We need to get out there and make sure these simple messages are being heard.

*Maybe.

Related posts:

By: Tammy Everts Category: Tags: , , , , , , ,

New findings: Ecommerce sites are 9% slower than in 2011

18 Sep 2012

12 Comments

Two years ago, when Strangeloop started tracking the load times of 2,000 top North American ecommerce sites, we had a hunch we’d spot some interesting trends over time. We did not expect, however, to see that pages are continuing to get slower rather than faster. Yet according to the Fall 2012 release of our quarterly Ecommerce Page Speed and Web Performance State of the Union, which came out today, that’s exactly what’s happening.

Not only are pages slower, they’re dramatically slower. Since November 2011, when we last tested these sites, the median home page has taken a 9% performance hit, with load time increasing from 5.94 seconds to 6.5 seconds. This flies in the face of conventional belief that, thanks to faster browsers, networks, and devices, the average end user is enjoying a premium online experience. This is clearly not the case. As user expectations continue to grow, the gap between expectations and reality continues to widen.

Ecommerce Page Speed and Web Performance State of the Union [Fall 2012]

These graphics (higher res version here) illustrate a few of our  findings. I encourage you to download the report to read the rest. Without giving it all away, here are a few highlights:

  • Internet Explorer 10 served pages faster than other browsers, most notably 8% faster than Chrome 20. We tested each page across a number of browsers, including the latest versions of IE, Chrome, and Firefox. It’s important to bear in mind that these were simple tests that didn’t take into account the many nuances of browser performance (which is discussed further in the report), but we considered these results interesting enough to share.
  • Top sites are 10% slower than the pack as a whole. While the median site took 6.5 seconds to load, we saw even poorer results when we looked at the top 100 sites (ranked by revenue and profitability), with the median Alexa 100 home page having a load time of 7.14 seconds. Check out the report for our thoughts on this.
  • Many sites are still not following core performance best practices. We found that 30% of sites tested did not use compression, and 12% did not use keep-alives. As I’ve talked about elsewhere, these two fairly simple techniques can yield big results, including up to 52% improvement in start render time.

Why you should care about these findings

To my knowledge, Strangeloop’s state of the union reports (which we’re now releasing on a quarterly basis) are the only ongoing surveys that measure performance from the perspective of real users. By using WebPagetest, we can simulate performance across browsers and realistic latencies, and get a real-world look at how websites actually behave. It’s easy for site owners to fall into the trap of thinking that their sites are fast for everyone, because site owners are typically seeing benchmark tests run out of datacenters.

I want to emphasize that reports like this one are not a substitute for the real user monitoring you should be performing on your site on an ongoing basis. Instead, consider it a snapshot that we can collectively hold up as a mirror of big-picture ecommerce performance.

As always, I welcome your feedback and questions.

Download the report: State of the Union: Ecommerce Page Speed and Website Performance [Fall 2012]

Download a high-res version of the infographics above (and feel free to re-post): Poster: Ecommerce Page Speed and Website Performance [Fall 2012]

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , , , , , ,

New findings: How does browser usage vary throughout the day and week?

8 Aug 2012

4 Comments

Last month, I questioned the validity of using Internet Explorer 8 as the default browser for performance testing. In the comments on that post, Stephen Pierzchala raised a good point:

I would be really interested to see a little more granularity in both the Akamai IO and Strangeloop data that shows browser usage by time of day and day of week. While it doesn’t change the aggregate results, it may indicate the type of browser that people choose to use, rather than the browser they are told to use.

We all tend to have semi-educated guesses about the browsers that people choose to use at work/school versus those they use at home, but I wasn’t able to find any studies that answer Stephen’s question. So I took a look through our beacon data to see if I could spot any trends. I got some interesting results that confirmed a few of my assumptions.

Approach

  1. Gathered data for one month of traffic for two large North America-based ecommerce sites.
  2. Pared traffic down to four browsers: Chrome, Firefox, Internet Explorer, and Safari.
  3. Aggregated data for each day of the week and plotted this on a graph for each site.
  4. Aggregated data for each hour of the day and plotted this on a graph for each site.

The results were interesting.

Observations

Browser usage throughout the week

As you can see on this pair of graphs:

  • Internet Explorer was the dominant browser for these sites.
  • Traffic for every browser, with the exception of Safari, either flatlined or decreased over the weekend.
  • Safari traffic spiked dramatically over the weekend.
  • Friday is a big browsing/shopping day across all browsers.

Browser usage throughout the day

These graphs bear a striking similarity to the daily breakdown graphs, in that IE, Firefox, and Chrome use begins to trend downward at the end of the workday, just when Safari use spikes.

Conclusions (aka “more educated guesses”)

While this data represents only two websites, I think there are enough similarities to make these findings a compelling argument for a couple of statements:

  1. Internet Explorer should still be the go-to browser for testing ecommerce sites in North America. The primary reason for this, in my opinion, is that many people are browsing/shopping while at work.
  2. Safari is a serious contender as the browser of choice for home shoppers, particularly those in desirable demographics (in other words, the Apple crowd). This is more of a hunch than an educated guess, but my feeling is that this partially based on the massive recent growth of the iPad market. I say this because everyone I know who uses a MacBook uses Chrome or Firefox instead of the default safari browser, whereas everyone I know who owns an iPad uses Safari. (Like I said, it’s just a hunch.)

I definitely don’t consider this the last word on the subject. The browser wars are far from over, and there’s enough flux in the industry that it’s anyone’s guess as to what the next year might hold. If you know of any comparable studies or statistics, let me know.

Related posts:

By: Tammy Everts Category: Tags: , , , ,

The 33 best web performance links of Q2 2012

31 Jul 2012

4 Comments

For some reason, I thought that the past few months had been kind of quiet on the research front, so when I started this post, I thought it would be one of my shortest roundups yet. I was pleasantly surprised to watch it grow to become one of the longest!

There are some great case studies here, of both large and small sites, which I love to see. There’s also some truly excellent debate about responsive design and the mobile web, sparked by a post from Jakob Nielsen last spring, as well as some good stuff about the browser wars and third-party content. So enough with the intro. Let’s get into it.

Case studies

Optimizing Retr-O-Mat’s Web Performance

A casual performance optimizer details her efforts to get Retr-O-Mat’s average load times under 2 seconds. Good information for anyone interested in the nuts and bolts of front-end optimization (FEO).

Web performance can be beautiful too

After performing poorly in a 2011 web performance comparison of leading retailers, Crate and Barrel made WPO a priority moving forward. This blog post from Catchpoint shows just how “beautiful” their performance has been in 2012.

How the Post is improving site performance

Responding to a flood of user frustrations with their website, the IT team at the Washington Post rolled out a number of performance upgrades to their site over the past year. Find out what they did to improve their page speed by 32.4%.

Tips and how-tos

Building a faster web: Tools, tips, and lessons

If “faster connectivity and more bandwidth won’t save us,” then what will? Google’s Ilya Grigorik shares his insight on making the web faster in this in-depth slide deck, and he draws some very interesting conclusions.

How to Make Progress Bars Feel Faster to Users

The human perception of time is anything but linear, and with just minor visual tricks, it gets even more skewed. After reading this post, you may never trust a progress bar again. :)

The 3 white lies behind Instagram’s lightning speed

More cool perceptual tricks. The “secret sauce” behind Instagram’s stellar user experience is rooted in a combination of coding tricks aimed at giving users a feeling of constant responsiveness. Find out how their site “always pretends to work.”

Mobile

The web only works thanks to reload (and why the mobile web fails)

As Mike Belshe points out, web page resources routinely fail, but thanks to the ever-handy reload/refresh button, we can often solve these problems ourselves. With mobile browsing, however, the rules are different. Find out what this means for the future of HTML 5.

Web first for mobile

Performance evangelist Steve Souders focuses his performance research strictly on the mobile web – not on native apps. Why? He’s got more than a few good reasons.

A taste test of mobile website development

A solid webcast on the complex world of mobile development, touching on topics including Responsive Web Design (RWD), server-side device detection, and HTML5 performance on mobile.

Jakob Nielsen on mobile sites vs. full sites

Jakob Nielsen believes that mobile and full sites should be entirely different entities. Summarizing his argument, he states that “good mobile user experience requires a different design than what’s needed to satisfy desktop users. Two designs, two sites, and cross-linking to make it all work.”

Is Nielsen wrong on mobile? Arguments abound

From Net magazine: Jakob Nielsen’s assertion that “good mobile user experience requires a different design” is being challenged by a noted mobile expert, who argues that rather than stripping down for mobile, companies should be doing more.

Why we shouldn’t make separate mobile websites

More counterpoint to Nielsen’s post. Smashing Magazine’s Bruce Lawson argues that mobile redirection is unreliable, and excluding features for mobile browsers “perpetuates the digital divide.”

Responsive web design: Missing the point

Still more Nielsen backlash: Brad Frost states that, though mobile browsers are getting better at rendering full websites, creating adaptive sites for mobile users is essential to improving the user experience.

HTML5 features increase mobile usage by 28%

Interesting piece explaining how static pages needing an upgrade can vastly improve mobile user engagement through the addition of HTML5. The new release features interactive galleries, overlays, and expandable/collapsible boxes, driving up pageviews and decreasing bounce rates.

Tools

More ways to measure web performance with User Timings

Google Analytics has expanded its collection of Site Speed reports with a new feature called User Timings. The feature enables tracking of specific load times for discrete hits, images, and other user interactions.

New mod_spdy release supports Apache servers

More from Google. The latest version of mod_spdy – an Apache module that adds SPDY server support – is intended to fix bugs found in the original release.

“Speed Index” introduced as new performance metric

The Speed Index metric has been added to WebPagetest, helping measure the speed at which page contents are visually populated. The tool is especially useful for comparing page experience before and after optimization.

Browsers

Browser Speed Tests: Chrome 19, Firefox 13, Internet Explorer 9, and Opera 12

Lifehacker conducts it’s semi-regular browser speed tests, pitting the four titans of desktop browsing against each other in races for startup speed, tab loading times, and other performance indicators.

Which Browsers are the Fastest?

An interesting companion read to the Lifehacker piece, New Relic’s “Speed Wars” study shows that, while IE 9 speeds past other browsers on Windows, Chrome 13 on Mac was overall the fastest experience. In mobile speed tests, the fastest experience was delivered by Blackberry Opera Mini at 2.6 seconds, twice as fast as Safari 5.1 on iPad.

How the Chrome Predictor hides latency from users

Ilya Grigorik demonstrates how Google Chrome hides latency from users. Interesting stuff here.

Internet Explorer market share surges, as IE 9 wins hearts and minds

From Ars Tecnica: “The browser wars are back on in earnest. For the second time in three months, Internet Explorer made large gains, picking up almost 1 point of market share. Chrome, Firefox, and Safari all lost out, as Internet Explorer 9 won over new users.”

CDNs

A one-size-fits-all CDN solution isn’t always best

Server configurations come in all shapes and sizes, which means a one-size-fits-all CDN is seldom effective. Find out which Level 3 customer was the beneficiary of a custom CDN solution and how it worked out.

Third-party content

10 Golden Rules for 3rd Party Providers

Murphy’s Law reigned supreme throughout June, with a flood of large-scale outages taking down some of the world’s most popular websites. Given the inevitability of online failures, third-party providers must be prepared to deal with the worst. The folks at Catchpoint outline the 10 Golden Rules by which all third-party providers should live by.

The vendor who flunked the web performance test

Are third-party vendors ignorant to the consequences of slow web performance? According to Catchpoint they are, as they detail a story of one such vendor who was completely unaware of the performance impact of their product.

Average UK website has 14 trackers per page

Interesting findings from TRUSTe: Despite the prevalence of privacy policies, over two-thirds of trackers on UK websites originate from third-party companies, and almost half embed themselves permanently.

Google releases +1 button preview – loads 20% faster

Google announced that they’ve improved performance of the +1 button and Google+ badge. By reducing the size of the js/plusone.js loader and making the code smarter, page elements now load 20% faster.

Third-party JavaScript should be loaded asynchronously

Old news to some, but still worth mentioning: Third-party JavaScript should be loaded asynchronously, as it helps avoid slowdowns and can speed up page loads.

Third-party front-end performance, Act 1

Application provider Bazaarvoice is delving into the realm of front-end performance, and provides an interesting third-party perspective.

Opinions and analysis

Performance Nightmare: Nasdaq & the Facebook IPO

When Facebook began trading on May 18, 2012, a series of performance failures on Nasdaq.com caused a huge headache for the company. This article from Intechnica asks how much these badly timed hiccups cost investors.

More, better, faster: Steve Souders on WPO

Steve Souders kicked off O’Reilly’s Velocity video podcast series with an in-depth discussion of the state of web performance optimization. Key topics included measuring slowness, performance monitoring tools, and whether mobile disrupts performance.

Other research

How complex systems fail

As a complex and interdependent system, the web is prone to catastrophe at the highest levels. In this fascinating paper on resilience engineering, presented at Velocity 2012, Dr. Richard Cook outlines the reasons why all complex systems are intrinsically hazardous, why disaster is always just around the corner, and how failure-free operations still require experience with failure.

The growing epidemic of page bloat

I don’t usually pimp my own writing here, but this information is too important not to share. I wrote a piece for GigaOM showing that the average page size is now over 1MB, according to the HTTP Archive. At current growth rates, the average page could hit 2MB by 2015, which is a really big deal, especially for mobile users.

How fast are websites around the world?

Some fascinating findings here. Google’s Site Speed Reports provides detailed latency data for page load times by separating data according to device, location, and industry.

These links were all sourced from Strangeloop’s Web Performance Hub, which contains hundreds (and by now, possibly even thousands) of industry-wide links, organized by topic, source, research type, and industry. It’s a pretty good resource, if I do say so. If you have any new links to recommend, let me know.

http://t.co/EHbRdT6r
10 Golden Rules for 3rd Party Providers [article]
Catchpoint – June 26, 2012
Summary: Murphy’s Law reigned supreme throughout June, with a flood of large-scale outages taking down some of the world’s most popular websites. Given the inevitability of online failures, third-party providers must be prepared to deal with the worst. Find out the 10 Golden Rules by which all third-party providers should operate by.
http://t.co/LGYUpGn1
End-to-end optimization: Taking content delivery to the next level [blog post]
Web Performance Today – June 27, 2012
Summary: Strangeloop Networks is thrilled to announce the launch of our latest product, Network Accelerator. Learn all about how this product works, what it does – and most importantly – why it’s a major step forward for content delivery networks.
http://t.co/T4z69s7k
How complex systems fail [research paper]
CTALab.org – June 26, 2012
Summary: As a complex and interdependent system, the web is prone to failure and catastrophe at the highest levels. In this fascinating paper on resilience engineering, Dr. Richard Cook outlines the reasons why all complex systems are intrinsically hazardous, why catastrophe is always just around the corner, and how failure-free operations require experience with failure.
http://bit.ly/LzGPqN
Mobile optimization starts with mindset: Hooman Beheshti interviewed at Velocity 2012
O’Reilly Media – June 25, 2012
Summary: Where are we in the mobile optimization life-cycle? What mindset should site owners adopt when boosting mobile performance? Are performance measurements improving? In this video, Strangeloop Technology VP Hooman Beheshti offers his unique insight on the current state of mobile.
http://t.co/Vnced8tq
The 90-Minute Mobile Optimization Life Cycle [slides]
Strangeloop Networks – June 25, 2012
Summary: Strangeloop Technology VP Hooman Beheshti wowed attendees at this year’s Velocity Conference with a presentation on the mobile optimization life cycle. For those who missed it, be sure to check to check out these fascinating slides.
http://bit.ly/Nu1gCi
Ghosts of Velocities Past: 9 presentations that are still relevant today [blog post]
Web Performance Today – June 20, 2012
Summary: Velocity’s short (yet incredibly important) history is filled with memorable moments, and these 9 presentations from past conferences remain relevant today. Perhaps not trendsetting anymore, but certainly trend affirming, which may just be better.
http://bit.ly/MFuxMR
My recent post on SEOMoz: 13 Questions (and Answers) About Google, Site Speed, and SEO [article]
SEOmoz – June 18, 2012
Summary: In this article, Strangeloop president Joshua Bixby breaks down how site speed and performance metrics affect Google page ranks. For anyone who has ever wondered how Google manages to make performance metrics affect SEO, this article is a must-read.
http://mz.cm/M24fGc
Introducing: New Browser Tax feature for our ecommerce customers [blog post]
Web Performance Today – June 14, 2012
Summary: Ever wish you could arbitrarily tax your customer base for failing to stay current, with zero repercussions? With the new Strangeloop Browser tax, your wish is now a reality!
http://bit.ly/NC5Q65
Optimizing Retr-O-Mat’s Web Performance [blog post]
Finding Marbles – June 9, 2012
Summary: For the “WPO guy’s wife,” average load times just aren’t good enough. In this post, a blogger and casual performance optimizer details her efforts to get Retr-O-Mat’s average load times under 2 seconds. Great information for anyone interested in the nuts and bolts of WPO.
http://t.co/k0lXkniG
Browser Speed Tests: Chrome 19, Firefox 13, Internet Explorer 9, and Opera 12 [article]
Lifehacker – June 12, 2012
Summary: It’s a battle of startup times, tab loading times and other KPIs between the four titans of Windows browsing. Lifehacker’s speed tests are always entertaining for what they’re not afraid to say, and this article is no exception.
http://bit.ly/Nb2ibS
Marrying CDNs with front-end optimization for maximum acceleration [blog post]
Web Performance Today – June 12, 2012
Summary: Front-end optimization (FEO) has been weaving its way further into content delivery networks (CDNs) over the past two years, and the dynamic between these two technologies continues to evolve. In this video presentation, Strangeloop’s Joshua Bixby breaks down the benefits of combining these performance solutions.
http://bit.ly/L3aatz
How the Chrome Predictor hides latency from users [article]
Igvita.com – June 4, 2012
Summary: Google Chrome features countless tools for supercharging load times, but when those aren’t enough, the browser can hide latency from users. Find out how!
http://bit.ly/Nw6ZMh
Building a faster web: tools, tips, and lessons [slides]
Igvita.com – June 3, 2012
Summary: If “faster connectivity and more bandwidth won’t save us,” then what will? Google’s Ilya Grigorik shares his insight on making the web faster in this in-depth slide deck, and draws some very interesting conclusions.
http://bit.ly/L0eERH
The “performance poverty line”: What is it and why does it matter? [blog post]
Web Performance Today – June , 2012
Summary: The “performance poverty line” is the point at which business metrics have sunk so low, load times cease to matter. But how is this line measured? Does it differ between industries? And most importantly: is there hope?
http://bit.ly/NJAqs2
A one-size-fits-all CDN solution isn’t always best [article]
Level 3 – June , 2012
Summary: Server configurations come in all shapes and sizes, which means a one-size-fits-all CDN is seldom effective. Find out which Level 3 customer was the beneficiary of a custom CDN solution.
http://bit.ly/M9P5xt
Why the Facebook outage is (yet another) wakeup call for site owners [blog post]
Web Performance Today – June , 2012
Summary: The hazards of running third-party scripts are well documented, but the May 31st Facebook outage was another stern reminder. In this post, Strangeloop’s Joshua Bixby discusses all things third-party, including rogue content and common performance pitfalls caused by third-party content.
http://bit.ly/JQj7GX
The web only works thanks to reload (and why the mobile web fails) [article]
Belshe.com – June , 2012
Summary: Web page resources routinely fail, but thanks to the ever-handy reload/refresh button, we can often solve these problems ourselves. With mobile browsing, however, the rules are different. Find out what this means for the future of HTML 5.
http://bit.ly/NAAQ3O
How to Make Progress Bars Feel Faster to Users
UXMovement – June , 2012
Summary: The human perception of time is anything but linear, and with just minor visual tricks, it gets even more skewed. After reading this post, you’ll never trust a progress bar again!
http://bit.ly/NcsVfg
Does the average web user waste two days a year waiting for pages to load? [blog post]
Strangeloop Networks – June , 2012
Summary: It may not be true, but in web performance, perception is reality. Web users in the UK are less than pleased about their online experience, but just how cranky are they?
http://bit.ly/LSFIlv

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , , , , , , , , , , ,

Is it time to give up Internet Explorer 8 as our default test browser?

10 Jul 2012

9 Comments

If you’re a regular user of WebPagetest, you depend on it to give you a reliable sense of how fast pages load across a variety of browsers and simulated latencies. I’m a huge fan and applaud Pat Meenan and the rest of his team in their efforts to maintain its relevance in the face of constantly shifting browser trends.

Knowing how responsive the WebPagetest team is to changing demands, I recently started wondering about why WebPagetest still uses Internet Explorer 8 as its default browser, when it’s commonly believed that Chrome is more widely used and Firefox comes in at a respectable second place. I also wondered if there are new gaps in terms of which browsers we should be testing on. So I started to do a bit of digging.

WebPagetest browsers

Why is this an important question?

While modern browsers are more similar than older browsers when it comes to embracing common standards, all browsers are still not created equal. As I’ve talked about in the past, as web pages become increasingly complex, data-intensive, and dynamic, front-end optimization (FEO) techniques that can make pages faster in some browsers can slow them down, or even break them, in another browser.

If you’re focusing your testing efforts on a single browser, such as IE8, you might just be getting a pinhole view of your site’s performance. And if the majority of your visitors use Chrome and Firefox, you might be getting an irrelevant pinhole view of your site’s performance.

How many people use Webpagetest’s default settings?

But before spending too much time contemplating the question of whether or not IE8 should be the default test browser, first I wanted to establish that this is a relevant question. If most people are using their own custom settings on WebPagetest, the defaults aren’t a huge issue.

To examine the use of default settings, last Friday afternoon I looked at roughly 24,000 public tests on WebPagetest (the sum of a half day’s testing for that day).

As this graph shows, most users are indeed using the default settings:

Global browser usage

Next, I went to Statcounter and grabbed the global browser numbers for July 1-5, and created a comparable graph:

WebPagetest versus global browser stats

Looking at both charts, there are some striking observations about these two sets of data:

  • While just over 13% of global internet users use IE8, roughly two-thirds of WebPagetests are done on IE8.
  • Chrome 20 is the dominant global browser, with a 23% share. IE9 is second, at 17%, and Firefox 13 is third, at almost 15%.
  • While WebPagetests are split almost equally between IE9 and IE7, this doesn’t mirror the purported real world data offered by Statcounter, which shows IE9 to be vastly more popular than IE7.
  • In fact, further to the above point, the number of Safari iPad users is almost double the number of IE7 users, at 2.52% and 1.38% respectively.

Here’s a graph that illustrates the differences more starkly:

Browser stats: WebPagetest use vs. global browser stats

You can see there’s a significant disconnect between the reported global browser usage and the browsers that people in our industry test on.

Based on this data, if we were to create an ideal set of testing options for WebPagetest, we would:

  • Add the capability to test on Chrome 20, and make this the default browser.
  • Add Firefox 13, Safari 5.1, and Safari iPad.
  • Question why so many of us are bothering to test on Firefox 6.

Or would we?

Some of these global usage numbers weren’t sitting well with me, because they don’t reflect the real-world data that I see every day here at Strangeloop. I gathered traffic data for the same time period, July 1-5, for a handful of Strangeloop customers using desktop browsers, representing approximately 350,000 unique visits. I broke the data down by browser version:

Out of curiosity, I compared these numbers to the data gathered by Akamai IO between July 1-5:

Akamai IO browser stats - July 1-5, 2012

While Akamai’s results aren’t identical to Strangeloop’s, they’re similar enough — specifically when it comes to the popularity of IE, Chrome, and Firefox — to validate my belief that both samples are fairly representative of typical internet users in our market.

WebPagetest versus Strangeloop browser stats

So I took Strangeloop’s browser usage numbers and overlaid them on WebPagetest’s numbers:

Browser stats: WebPagetest vs Strangeloop data

While the results don’t dovetail perfectly, you can see that IE8 usage correlates pretty decently. However, there’s a significant gap when it comes to IE9. And Chrome 20, Firefox 13, and Safari — though widely used according to both Strangeloop’s and Akamai’s data — are not yet on WebPagetest’s map.

Conclusion #1: IE8 is still relevant as the default test browser, but we should keep an eye toward using IE9 in the near future.

It would also be great if WebPagetest could add the ability to test for Chrome 20, Firefox 13, and Safari.

Conclusion #2: It could be an extremely useful feature if WebPagetest offered different defaults based on country, or even based on country plus user/application type.

This research raises some interesting new ideas. It’s pretty clear that Strangeloop and Akamai’s data is North American-centric with a bias toward ecommerce customers, and this no doubt accounts for some of the discrepancy noted above. This opens up the question of whether it would be useful to default to different browsers for different countries, and even to different browsers for different types of applications. For example, selecting “Ecommerce Dulles” would test on IE8, while a content-driven site in India should default to Chrome 20.

Takeaway: Know your users… and their browsers.

While this test validates that Internet Explorer 8 is a sound default for the mainstream North American ecommerce market, it may not be the most-used browser for your visitors.

One of our customers, Wine.com, discovered that their highest-spending customers work in the financial services sector, and they were placing orders from work, where they were restricted to IE6 and IE7, which had serious usability implications. And last year, L.L. Bean realized that 4-5% of their affluent older customers were still using IE6.

Before you spend a lot of time running tests, look into your analytics and learn who your users are.

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , ,
1 2 3