Ecommerce

4 awesome slides showing how page speed correlates to business metrics at Walmart.com

28 Feb 2012

42 Comments

When one of the first slides in a presentation offers this level of candor, you know you’re in for a treat:

Walmart web performance optimization case study

File this presentation under “I wish I’d been there in person.” Cliff Crocker, Aaron Kulick, and Balaji Ram joined forces at February’s SF Web Performance Meetup to tell a RUM (real user monitoring) story through the lens of three job functions at Walmart.com: the performance data analyst, the developer, and the business analyst.

You can check out the full slide deck here (you need to be a Minus member to view it online, but anyone can download it), but I want to highlight my favourite slides.

First, some back story.

The problem: Folks at Walmart knew their pages were slow. As a for instance, initial measurement showed that an item page took about 24 seconds to load for the slowest 5% of users. Why? The usual culprits: too many page elements, slow third-party scripts, multiple hosts (25% of page content is served by partners, affiliates, and Marketplace), and various best practice no-nos.

The goal: Create and meet a performance SLA that would see Walmart’s 95th percentile traffic hit 20 seconds.

The approach: To get to that goal, they dedicated a scrum team to one sprint of performance optimization. At the start of the process, the team performed some baseline measurements in which they used their RUM data to look at the load times of key pages and look for patterns. Then the team would create targets for page performance and at the end of the sprint measure the impact of optimization on key metrics.

The following slides are a great use of historical RUM data to make a powerful case (embodied in compelling graphs) for investing in performance. Throughout this set, there’s a clear, consistent correlation  between load time and conversion.

Awesome slide #1: Overall, converted shoppers received pages that loaded 2X faster than non-converted shoppers.

This should be of interest to anyone who’s asked themselves how to set targets for their own performance.

Walmart web performance optimization case study

Awesome slide #2: The above trend persists, even on individual pages that experience greater load times.

An example is show here, in which a specific page loaded almost 2 seconds faster for buyers than it did for non-buyers. (There’s nothing I love more than seeing parallel lines on a graph like this.)

Walmart web performance optimization case study

Awesome slide #3: Non-buyers were served category pages that were 2-3 seconds slower than buyers.

I really appreciate the comprehensiveness of this set of slides. It’s a good proxy for measuring page flows, which is a much trickier task, due to the massive number of permutations involved. (Learn how you can do something similar using WebPagetest.)

Walmart web performance optimization case study

Awesome slide #4: Bounce rate strongly correlates to page speed.

No huge surprise here, but great to see this validated.

Walmart web performance optimization case study

Conclusions: Optimization resulted in improved conversions, revenue, and SEO

You may be happy to know that the Walmart team not only hit their SLA at the end of their optimization sprint, they actually overreached it by almost 3 seconds. They note the following benefits:

  • For every 1 second of improvement they experienced up to a 2% increase in conversions
  • For every 100 ms of improvement, they grew incremental revenue by up to 1%
  • SEO benefits for entry pages and reduce bounces

Do it yourself

Check out the original slide deck to learn how Walmart used RUM tools, first to baseline performance and its benefits, and then to measure the results of optimization.

If RUM isn’t an option for you, then read this post to learn how to identify and measure the performance of sample page flows using WebPagetest or HTTPwatch. And then watch this short video to learn how to use Google Analytics to perform a page speed/revenue analysis of the pages in your flows.

Before I sign off, I want to applaud the team at Walmart for providing this level of disclosure. Case studies like this are a massive boon for our industry. It’s always inspiring to see e-commerce giants joining the likes of Amazon and Shopzilla in their willingness to share data.

Related posts:

By: Tammy Everts Category: Tags: , , , , , ,

Top e-commerce sites suffer on Valentine’s Day [INFOGRAPHIC]

14 Feb 2012

0 Comment

In honour of Valentine’s Day, our marketing team here at Strangeloop put together this cute set of infographics about how people shop, online and offline, for their special someone(s). (News to me: Until I read this, I didn’t know that annual condom sales spike just before February 14th, and home pregnancy test sales spike in March.)

Of course, since we are a web performance company, naturally there’s a performance angle here. Some interesting factoids:

  • Last year, the average load time of 10 leading Valentine’s Day retail sites was 21.5 seconds in the week leading up to Valentine’s Day.
  • Florist sites get hit hardest on Valentine’s Day morning (probably due to the fact that something like 30% of us — myself included — waited till today to shop). Traffic volume can make pages up to 400% slower or cause them to totally crash.

Enjoy, and feel free to re-post these on your own site (high-res version here). If you have any new holiday performance stats, I’d love to hear them.

Web performance and e-commerce facts and statistics

Related posts:

By: Tammy Everts Category: Tags: , , ,

New report: What we’ve learned from two years of watching the top 2,000 e-commerce websites

26 Jan 2012

11 Comments

Yesterday was a busy day for us here at Strangeloop. We released our second annual “state of the union” report on the page speed and performance of 2,000 leading e-commerce websites.

Infographics: 2012 State of the Union for E-Commerce Page Speed and Website Performance

We came up with the idea of conducting these reports over a year ago, when we realized there was no way to track performance changes and trends — from a real user’s perspective — on an ongoing basis. As I mentioned to CNet’s Stephen Shankland, when we talked about the report the other day, we can’t rely on benchmarking tests run out of data centers to give us any relevant sense of how our sites are actually performing out in the real world. They’ve got ridiculous bandwidth, and they’re parked right next to the content delivery machines, so they’ve got zero latency. By using WebPagetest — which simulates how fast a site loads in a real user’s browser — to measure the performance of the same set of 2,000 sites year after year, we can compare real-world performance snapshots and see what’s changed, and what’s caused the changes.

I don’t want to give away all our findings, the highlights of which are illustrated in the infographics above, but a couple of interesting things we discovered:

  • Pages have gotten bigger – from an average of 86 requests last year to 98 requests this year. This kind of growth is consistent with trends identified via the HTTP Archive.
  • Repeat views are a whopping 20% slower than they were last year, probably due to the the number of page objects.
  • Page Speed scores have gotten significantly worse, from 83% last year to 75% now. I was just talking to Pat Meenan, who created WebPagetest, and he pointed out that a probable reason for the drop is because the newest version of Page Speed checks for more optimizations, which has resulted in lower scores across the board. (Side note: It’s important to know that Page Speed doesn’t take into account advanced front-end content optimization techniques — nor could it, because there are way too many of them to track and measure. As a for-instance, we’ve had experiences here at Strangeloop where we’ve taken a page with a perfect Page Speed score, accelerated it with Site Optimizer, cut load time in half, and ended up with a new Page Speed score of 74%.)

While the findings are a mixed bag of good and bad news, to me the most exciting takeaway is the  attention this report has received from major media outlets like CBS News. It goes to confirm what became evident over Black Friday: site speed has become a mainstream issue.

Download the report: 2012 State of the Union: E-Commerce Page Speed and Website Performance

Download a high-res version of the infographics above (feel free to re-post): Poster: E-Commerce Page Speed and Website Performance (2012)

Related links:

By: Tammy Everts Category: Tags: , , , , , , , , , , , ,

Interesting new findings about page views, time on site, and bounce rate across desktop and mobile browsers

20 Jan 2012

11 Comments

Last month, I talked with Mac Slocum at O’Reilly Radar about mobile performance, and he asked me a couple of interesting question:

  • Are mobile users more or less tolerant of delays than desktop users?
  • Are users of one type of system more accepting of delays than users of another?

These questions are a gateway to a fascinating area of research, because they lead into a topic that we all have pet theories about (i.e. Chrome users are more tech savvy than the average person, while Internet Explorer users are less) but have little statistical evidence to back up.

I told Mac that I planned to do more digging and report back, so here I am.

Methodology

  1. I took five e-commerce sites (full sites, not mobile versions) that Strangeloop is currently accelerating and pulled their entire transaction volume over the past month — totaling hundreds of millions of unique visits via desktop and mobile. While desktop transactions outnumbered mobile transactions, the mobile numbers were still statistically significant: the smallest set of mobile numbers comprised around 200,000 unique visits and the largest set comprised about 20 million unique visits.
  2. I extracted the following data: page views, time on site, and bounce rate.
  3. I sorted the data into the following browser/OS groups: Internet Explorer, Firefox, Chrome, iPad, iPhone, and Android (phone).
  4. I calculated the averages for each metric and browser.
  5. I graphed the numbers and looked for trends. Some interesting patterns emerged:

Desktop vs. mobile performance: average pageviewsDesktop vs mobile performance: average time on siteDesktop vs mobile browser performance: average bounce rate

Three Key Findings

Finding #1: Internet Explorer users consistently view more pages, spend more time on site, and have a lower bounce rate than Firefox and Chrome users.

The average number of page views for IE users was 6.134, as opposed to 5.17 for Firefox users and 5.14 for Chrome users. IE users spent between 30-45 seconds longer on the site than other users, and their bounce rate was lower by 5 or 6 percentage points — a pretty significant difference. Could all of this substantiate the belief (among non-IE users, at least) that IE fans are less tech savvy and therefore slower and more ponderous web users than the rest of us? Or perhaps it’s a hardware issue — are IE users more likely to be using older systems with less processing power?

But what about that lower bounce rate? At around 35%, it’s a pretty strong number, especially compared to 41% for Firefox users and 42% for Chrome. A lower bounce rate generally signifies that people who come to your site find it relevant and worth sticking around to check out. Are IE users better searchers and more likely to arrive at the right destination, or are they simply more easily satisfied than other users?

Finding #2: iPad users are more similar to desktop users than they are to smartphone users.

While iPad users view somewhat fewer pages per visit than desktop users (4.54 versus 5.14, 5.17, and 6.13), their average time on site and bounce rate were commensurate with the desktop crowd. This isn’t a huge surprise. We know that most iPad users are browsing in the comfort of their home, and they consider their iPad to be more like a small laptop than an oversized phone. What’s interesting here is that, even though iPad performance lags behind desktop (it is a mobile device, after all, and it suffers from many of the same performance constraints as a smartphone: from low processor power to touchscreen lag), iPad users seem willing to stick around for a longer desktop-like experience.

Finding #3: iPhone users consistently view fewer pages, spend less time on site, and have a higher bounce rate.

At the opposite end of the spectrum from Internet Explorer users we find iPhone users. In every sample group, iPhone users, on average, spent significantly less time on site (2:31 vs 3:20) and viewed fewer pages (2.41 vs 3.1) than Android, and had a higher bounce rate (60.76% vs 57.17%). The shorter time spent on site could be attributed to the (arguable) fact that iPhones are better-powered than other devices, but that doesn’t account for the page views and bounce rate. Do these validate all the stereotypes about iPhone users: that they — or should I say, we — are impatient, savvy web users who will bounce from a site if we can’t find what we want right away and aggressively search elsewhere? Or that we know what we want and can expedite a transaction faster and more efficiently than other users?

Takeaways

This research doesn’t directly answer Mac’s questions, but it does come at them sideways and raise some interesting — to me, anyway — questions about the types of people who use different technologies, and how and why they use them. We can’t answer these questions today, but these findings are food for thought and debate.

Your thoughts?

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , , , ,

The 20 best web performance links of Q4

15 Dec 2011

1 Comment

Every time I write one of these posts, I’m impressed by the volume and quality of writing that happens in our industry on an ongoing basis. It’s truly an exciting time for web performance. I feel endlessly engaged by the dialogue that happens every day, and honoured to be part of it it.

This roundup (which includes links pulled from the Strangeloop WPO Hub), includes some increasingly refined thinking about mobile optimization, a handful of excellent tutorials and case studies (including some great new presentations from Velocity EU), and some revolutionary browser developments.

But my favourite link is this first one…

The best link of Q4

Retailers need for tech speed
Does it tell us anything new? No. But I’ve lost count of how many people I’ve forwarded this two-minute segment on CNBC’s “Power Lunch” — which discusses the importance of speed for e-commerce sites, particularly for mobile users, during the holiday shopping season. For me, this shows that site speed has finally jumped into the mainstream. I’m excited to see how this attention snowballs in 2012.

Mobile

Mobile UI Performance
This slide deck from Estelle Weyl’s excellent presentation at Velocity EU gives an overview of mobile performance challenges, why we need to address them differently than we deal with desktop sites, and detailed tips on how to do just that.

Performance Automation 101
This slide deck from Jeroen Tjepkema’s Velocity EU presentation explains what performance automation is, how it works, and why it’s the only viable solution for dealing with the challenges of mobile device/browser fragmentation.

HTML5 Techniques for Optimizing Mobile Performance
Great post on HTML5 Rocks: ”In this article, we will discuss the bare minimum of what it takes to create a mobile HTML5 web app. The main point is to unmask the hidden complexities which today’s mobile frameworks try to hide. You will see a minimalistic approach (using core HTML5 APIs) and basic fundamentals that will empower you to write your own framework or contribute to the one you currently use.”

Mobile Performance Manifesto
Love this post from David Calhoun itemizing and describing mobile performance best practices.

Tools

How WebPagetest works
If you’ve ever wondered how exactly WebPagetest gathers performance data from the various browsers it simulates, this is great post from Pat Meenan in which he cracks the hood of WebPagetest and explains all that.

Mobile Perf Bookmarklet
Steve Souders offers one mobile bookmarklet to rule them all: a new “master bookmarklet” that lets you install a handful of common debugger and profiler bookmarklets in your mobile broswer in one step.

Is Synthetic Monitoring Really Going to Die?
Alois Reitbauer asks: “Will User Experience Management using JavaScript agents eventually replace synthetic monitoring or will there be a coexistence of both approaches in the end?” As you might guess, the answer is not cut and dried.

Case studies, how-tos, and other research

Diagnosing Slow Web Servers with Time to First Byte
Much as it pains me to admit it, from time to time performance pains aren’t caused at the front end. Performance expert Andy King gives some good tips on how to use the time to first byte metric, as displayed on a waterfall chart, to help diagnose a slow server.

The art and craft of the async snippet
Stoyan Stefanov examines the topic of asynchronous code “from the perspective of a third party – when you’re the third party, providing a snippet for other developers to include on their pages, be it an ad, a plugin, widget, visits counter, analytics, or anything else.”

Why loading third party scripts async is not good enough
We talk about asynchronously loading third-party snippets as if that’s the sole cure for performance pains, but in this case study, Aaron Peters reminds us that sometimes it’s okay to defer their loading until after onload.

Fast Loading JavaScript
Slide deck from performance consultant Aaron Peters’ great Velocity EU presentation: “A walk-through of several JavaScript loading techniques with a characteristics table for each and at the end a decision tree to help you decide which technique to use.”

How Downtime Financially Impacts Top Ecommerce Websites
Compelling infographic showing how downtime affected the Internet Retailer 500 in 2010. Includes the estimate that downtime resulted in more than $300 million in lost revenue for the IR 500.

Testing for Frontend SPOF
Excellent post from Pat Meenan in which he simulates third-party outage with a blackhole server in order to demonstrate — via WebPagetest-generated video — how that outage slows down or disrupts page load.

Browsers and connectivity

SPDY of the Future Might Blow Your Mind Today
Great post (“definitely for protocol geeks”) by Google software engineer Mike Belshe on SPDY’s evolution and how Kindle Silk is taking it beyond other browsers.

Chrome Fast
Slides from Google software engineer Tony Gentilcore’s excellent presentation at Velocity EU, in which he gives an overview of the Chrome platform and explains what makes Chrome fast.

Report reveals drop between peak and off-peak surfing
No big surprise, but a good reminder (especially if you rely on synthetic testing) that real-world performance is a nebulous thing: UK study finds that web speed is up to 69% slower during evening peak time.

The end of an era: Internet Explorer drops below 50% of Web usage
Mark the month and year. November 2011 was the first time in more than a decade that Internet Explorer’s share of global browser usage dropped below 50%.

Opinion pieces

Your CDN is not a silver bullet for web performance
In the e-commerce and SaaS world, the two most common causes of poor web performance are third-party content and server-side processing. Neither of these bottlenecks are addressed by loading static content from a closer location via a content delivery network.

Why you have less than a second to deliver exceptional performance
dynaTrace’s Alois Reitbauer writes: “Being exceptionally fast is becoming the dogma for developing web applications. But what is exceptionally fast and how hard is it to build a top performing web site?” I like posts like this because they remind us what the fundamental questions are that our industry is trying to address.

If you have any other great links to share, let me know in the comments. And if you’re looking for more great links, we have hundreds — sorted by topic, industry, and type — over in the Strangeloop WPO Hub.

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , , , , , , ,
« 3 4 5 6 7 8 9 10 »