How to perform a 5-minute page speed/revenue analysis of your ecommerce site

• Tweet

[This was such a popular post that we turned it into a brief (11-minute) webinar. Check it out.]

I proselytize the value of performance for a living. I am a member of the tribe that believes, beyond a shadow of a doubt, that improving the performance of your website will make you more money. I have dozens of stats at my disposal to convince you, and I feel an evangelical need to save non-believers from their errant ways.

Sometimes I’m confronted by non-believers who have the audacity to question my army of statistics and feel that the results at Amazon, Bing, Microsoft, Ebay, Shopzilla, AutoAnything, etc. do not apply to them.

I was confronted with one of these non-believers last week. In this case, he was a non-believer who was open enough to spend time brainstorming how the data could be presented in a way that would be more convincing. I am proud to say that he came out the other side as a serious performance convert. I want to share his conversion story.

“These stats don’t apply to me.”

I hear this a lot and I get it. It’s difficult for mortal companies to see themselves in relation to ecommerce mega-giants like Amazon and Ebay. This is why we put together stats on mortal companies.

But even those stats didn’t feel comparable for my skeptical comrade. He wanted to get an idea for how his site speed affects his users on his site with his content. Fair enough. My challenge was to figure out how to do this.

The obvious answer was to just implement Site Optimizer, speed things up, then check out conversion rate changes using a segmentation test, but this wasn’t an option. He needed to get buy-in before we could run a test implementation.

It was a chicken-or-the-egg conundrum. After much discussion, we decided to find proxies in his current analytics program that might convince him that performance would make him more money.

Using Google Analytics to find proxies for performance

After reading my post on Internet Explorer performance, which showed that IE8 is about 25% faster than IE7 across almost 200 websites, we decided to explore the different conversion behaviour by browser version.

The first thing we did was perform a Webpagetest in IE7 and IE8. We found that his site was 30% faster in IE8.

After he was convinced that his site performed faster in IE8, we went to Google Analytics and explored the numbers for his site. We found a remarkable trend. The value per visitor on his site was 29% higher for IE8 than it was for IE7 — almost exactly matching the speed difference we noted in the Webpagetest numbers. (Click image to enlarge.)

Change connection speed = change order value

We then explored different connection speeds within IE8. First, we performed Webpagetests on the different connection speeds, and then we compared them to the results in Google Analytics.

Again we found a remarkable relationship between connection speed and order value. (Click image to enlarge.)

On average, online shoppers using T1 connections spent about 11% more than shoppers with DSL connections. And shoppers with T1 connections spent roughly 32% more than those using dialup.

Calculating the benefits and caveats

We then explored the performance benefits his site would gain on IE7 and IE8 with Site Optimizer, and then compared them to the current performance of his site. After correlating the performance gains with the graphs from his analytics tool,  my non-believer converted. He was convinced that, at minimum, the proposed performance enhancements would increase his per visitor value across the board to at least the same levels as IE8, if not higher.

Obviously, other mitigating factors are at play here. (I’ll say it before someone else does: Correlation does not imply causation.) The choice of browser and connection speed might imply something about the buyer that is totally unrelated to the effects of speed on shopping choices. To pass any hardcore statistical muster, a much more in-depth regression test would need to occur.

What amazed me, however, was that these two screens of his own data was enough to convince my non-believer, on the spot, that performance mattered and that he should invest in it.

This trend held across other sites I sampled

After observing these results, I had to see if this trend held. I explored the analytics data for three other Strangeloop clients and was very excited to see the results and the overall trend. These validate the fact that, across the board, page speed improvements seem to correlate to greater order value.

Value per visit by browser:

Value per visit by connection speed in IE8:

This trend held in the real world, too

I needed to apply this methodology to one last test to determine if this back-of-the-napkin calculation had any validity in the real world.

I took a Strangeloop customer who had been through a rigorous month-long 50/50 test. In this particular case, with all other variables accounted for, the optimized site outperformed the unoptimized site. On average, order value for the optimized site was 20% greater than for the unoptimized site:

Takeaway: How to perform a 5-minute analysis of your ecommerce site

Using two simple tools you probably already have at hand — Webpagetest and Google Analytics — you can quickly calculate how a faster user experience correlates to greater order value from your customers.

1. Using Webpagetest, test your site’s speed on IE7 and IE8. Calculate the difference as a percentage.
2. Test your site to see what performance gains you could gain from front-end performance enhancements.
3. Look into Google Analytics (or whatever analytics package you use) to find out your current order value per visit for IE7 and IE8.
4. Use your analytics to find out your order value per visit by connection speed.*
5. Correlate the results in simple graphs, as I’ve done above.
6. Look at the performance gains from front end performance enhancements and determine what lift you anticipate in value per visit.
7. Share with your team.

If you’ve been having a tough time convincing your company to invest in performance, these results could provide your tipping point.

*If you’re using Google Analytics, I think the easiest way to get this data is with a custom report that looks something like this:

Related posts:

• Is it time to give up Internet Explorer 7 as our default test scenario?
• O’Reilly interview: Web performance for mortal companies
• Cheat Sheet: Everything you wanted to know about web performance but were afraid to ask