New findings: Typical leading ecommerce site takes 5.3 seconds to become interactive

15 Oct 2013

0 Comment

It’s fall, which means it’s time for Radware’s quarterly State of the Union for Ecommerce Performance! If you’re not familiar with these reports, here’s a quick backgrounder:

Ecommerce Web Performance State Of The Union Fall 2013 InfographicEvery three months, we analyze the performance of the top 500 retail websites, as ranked by Alexa. Using WebPagetest, a synthetic tool that simulates page load from a real user’s perspective across various browser types, we test the home page of each site and record a number of metrics, including size, composition, load time, start render time, and adoption of performance best practices.

The goal of this research is to gain ongoing visibility into the real-world performance of leading ecommerce sites – to learn how these sites perform for visitors sitting at home using the internet under normal browsing conditions – and to provide strategies and best practices that will help site owners serve their pages faster.

What I expected we’d find in our latest round of tests…

Prior to this latest round of tests, I made two big predictions. As it turns out, I’m only batting 0.500 on this one. Fortunately, I like surprises.

Assumption #1: Pages would be bigger and slower.

Since 2010, when we initiated these performance SOTUs (at Strangeloop, now part of Radware), we’ve consistently found that pages are getting bigger (in terms of number of objects and total bytes in) and slower (in terms of total load time). Assuming that this trend would continue in our latest report was kind of a no-brainer.

Assumption #2: Time to interact (TTI) would plateau or improve.

Our last report, which came out three months ago, marked the first time we included time to interact (TTI) as a performance metric. TTI is the moment that a page’s primary content — in the case of ecommerce sites, this is usually some sort of feature banner or carousel — renders in the browser and becomes interactive, i.e. has a usable call-to-action button or link.

Last summer, I was surprised (and borderline shocked) to learn that the median page took 4.9 seconds to become interactive. Ideally, pages should be interactive in 3 seconds or less. Like many folks in the performance community, I had assumed — wrongly, as it turns out — that most pages serve their key content within the first couple of seconds.

Given the fact that TTI surprised me once, you’d think that would make me somewhat jaded about my later assumptions. You would be wrong. I assumed that the slow TTI was an unfortunate performance blip, and that TTI would have improved, or at least plateaued, in our newest round of tests.

What we actually found…

Finding #1: Pages are bigger and slower in terms of total load time.

Ecommerce Web Performance State Of The Union Fall 2013 Infographic

No surprise here: The median load time was 8.56 seconds, a 14% slowdown from the median of 7.48 seconds we recorded three months ago. This is due, in part, to the fact that the median page is now 1258 KB in size and contains 92 resources, meaning that pages today are 24% bigger than they were back in 2011.

Finding #2: Pages are slower in terms of time to interact.

TTI State Of The Union Fall 2013 Infographic

Here’s the surprise: The median page took 5.3 seconds to load primary content, an 8% slowdown from the median of 4.9 seconds we recorded last quarter. The median number isn’t the only noteworthy finding here. We also discovered that only 18 of the top 100 sites had a TTI of 3 seconds or less, while 26 of the top 100 took 8 seconds or longer — with some nearing the 20-second mark — to become interactive.

While the TTI slowdown is disheartening, there’s a bright side…

Time to interact (TTI) is taking hold as the performance metric to watch.

If you care about user experience, then time to interact is arguably the metric you need to care about the most. It’s been extremely heartening to see TTI — along with its kissing cousin, above-the-fold time (AFT) — starting to get attention inside and outside the performance community as the metric to watch.

The great thing about TTI: It pinpoints the most critical moment in a page load, from the end user’s perspective.

The not-so-great thing about TTI: Right now, there’s no reliable way to automatically measure it. For our research, we identified TTI the old-school way: human eyeballs. Using WebPagetest, we downloaded the filmstrip version of each site’s median load, then analyzed each filmstrip and pinpointed the moment at which the primary page element (almost always a feature banner/carousel) loaded and became interactive (the main CTA button loaded). If anyone develops a tool that can consistently, reliably measure TTI, let me know!

But wait! There’s more…

I’ll be using this space to discuss our other findings. If you can’t wait, you can download the report and infographics here:

DOWNLOAD: State of the Union: Ecommerce Page Speed & Web Performance [Fall 2013]

Related posts:

By: Tammy Everts Category: Ecommerce, Page speed tests, Research Tags: , , ,

Are top VC-backed sites getting slower? New findings suggest they are.

9 Oct 2013

0 Comment

First and foremost, we believe that speed is more than a feature. Speed is the most important feature. If your application is slow, people won’t use it…

We think that the application has to be fast, and if it’s not, you can see what happens. We have every single one of our portfolio company services on Pingdom, and we take a look at that every week. When we see some of our portfolio company’s applications getting bogged down, we also note that they don’t grow as quickly. There is real empirical evidence that substantiates the fact that speed is more than a feature. It’s a requirement.

In the spring of 2010, tech VC Fred Wilson said this about the web applications that his company, Union Square Ventures, supports. Considering the fact that Union Square backs household-name winners like Etsy, Meetup, and SoundCloud, it’s safe to assume Fred knows what he’s talking about.

Three years ago, Joshua Bixby wrote this great post that validated Fred’s statement. Joshua randomly selected five sites backed by Union Square and ran them through WebPagetest, and found that the median site took 4.5 seconds to fully load. Pretty good, especially compared to the then-benchmark of around 7 seconds for Fortune 500 sites.

Recently, someone asked me if VC-backed sites still outperform other sites, and I thought it was high time to revisit this question.

Methodology

1. Per Joshua’s study, randomly select five sites currently backed by Union Square. (I’ve anonymized these, so as to avoid embarrassing anyone.)

2. Using WebPagetest, test each site’s home page nine times — first as the page would load over DSL and then over cable — via the test server in Dulles, VA. (While cable has overtaken DSL in usage, I feel like it’s too early to disregard DSL completely. And as we’ll find out in a moment, the DSL results are pretty compelling.)

3. Take the median load time for each page and plot it next to Joshua’s 2010 test results.

4. New in 2013: Using the frame-by-frame filmstrip view of the median load time for each page, identify the median time to interact. (TTI is the point at which the primary page content appears and becomes interactive, e.g. a feature banner that contains a call-to-action button.)

Results

This is where I’ll let you in on my under-the-vest hypothesis: based on the recent spate of findings that pages are getting bigger and slower, I somewhat-more-than-half-expected to learn that pages had slowed down somewhat, even in this set of sites. But the results were more striking than I’d expected:

  1. The current median load times over both DSL (11.44s) and cable (5.15s) are slower than the 2010 load time (4.54).
  2. The median time to interact over DSL (5.6s) is slower than the 2010 load time (4.54s).
  3. While the median TTI over cable is relatively fast at 3.5 seconds, it’s not optimal. Ideally, pages should be interactive in less than 3 seconds.
  4. The median page was 1520KB and contained 100 resource requests. Unfortunately, we don’t have these numbers from 2010, but it’s a fairly safe assumption that this is an increase over three years ago, and that this parallels the growth we’ve seen elsewhere in the Alexa Retail 500 and in the sites tracked by the HTTP Archive.
Load time – DSL (2010) Load time – DSL (2013) Load time – Cable (2013) TTI – DSL (2013) TTI – Cable (2013)
Site 1 3.30 4.37 2.45 4.3 1.3
Site 2 3.98 7.59 4.50 5.3 3.1
Site 3 4.54 11.44 5.15 5.6 3.5
Site 4 4.91 12.33 5.32 8.2 4.6
Site 5 5.87 16.82 16.74 10.1 4.6
Median 4.54 11.44 5.15 5.6 3.5

If tables aren’t your thing, check out the graph versions, first showing the difference between the 2010 load time and the 2013 load times for both DSL and cable:

VC performance tests: Load time

You can see that, across the board, load times over DSL are dramatically slower. Load times over cable vary between being somewhat faster and significantly slower.

And now, for interest’s sake, let’s look at how the current TTIs compare to the 2010 load time:

VC performance tests: Time to interactIt’s interesting to note that, while in theory TTI should be significantly faster than load time, in these tests the TTI over DSL was still markedly slower, while the TTI over cable was only marginally faster in most cases.

Caveat and conclusions

This is obviously a pretty simple ad-hoc set of tests. We’re looking at just the home pages, so we can’t measure flow or see how pages perform deeper into the visit. And we’re only looking at five sites. But it’s still telling to look at these simple numbers side-by-side with the numbers from five years ago.

I’m drawing attention to this, not to shame site owners or VCs, but to highlight the persistent performance challenges presented by page growth and complexity. Even hot household-name sites aren’t immune to these challenges.

Related posts:

By: Tammy Everts Category: Page speed tests, Research Tags: , ,

iPhone speed test findings: Faster processor doesn’t equal huge performance boost

2 Oct 2013

0 Comment

I came across this video from Everything Apple Pro yesterday and had to share it here because it illustrates something I talk about all the time: the fact that upgrading your mobile device doesn’t necessarily deliver the massive performance boost that some users believe it will.

For my purposes, you can start watching at around the 2:00 point. It’s a side-by-side speed test showing how quickly the same Reddit page loads across every iPhone ever made. Each device — from 2G up to 5S — has had the same firmware upgrade, meaning that each is using the latest version of Safari.

The conclusion after the test: While the iPhone 5s is “the king of this test” because it loads the page marginally faster than the other devices, the “spike in power” you get from upgrading “isn’t even that noticeable in day-to-day use”.

But watch for yourself and see:

These findings are consistent with our earlier findings that, for iOS, 98-99% of response time happens after the HTML arrives. In other words, when it comes to delivering faster pages to mobile devices, your browser is king.

If you’re upgrading your OS, and therefore upgrading your browser, then you’re taking your phone most of the way there in terms of what it’s capable of delivering performance-wise. There is arguably more performance to be gained by using a more optimized browser (e.g., smarter request ordering, optimized JavaScript engine, etc.) than by adding more horsepower.

Buying a brand-new device isn’t a magic bullet for performance. But they sure do look pretty. ;)

Related posts:

By: Tammy Everts Category: Mobile, Page speed tests Tags: , ,

Sneak peek: This is your brain on the mobile web

19 Sep 2013

0 Comment

I don’t think I’ve mentioned it yet in this space, but I’m extremely excited to have been chosen by O’Reilly to speak at Velocity Europe this November.

I’m also extremely excited about my topic: the neurological impact of slow load times on mobile users. I’ll be unveiling findings from some research that we’ve been engaged in here at Radware to measure and quantify “web stress” in mobile users. I’ve desperately wanted to dig into this untapped area of study for more than two years, ever since reading this quietly groundbreaking piece of research that studied web stress (as well as coining this phrase, which I love) in desktop users.

We’ve been working with the great people at Seren — with an assist from the folks at NeuroStrata — to collect the data, and I want to take a moment to show you what that process looks like:

Mobile web stress

Mobile web stress

Pretty cool, right?

I hope to see you at Velocity this fall. If you can’t make it, I’ll be posting our findings here after the conference.

Related posts:

By: Tammy Everts Category: Human factors, Mobile, Research Tags:

The Slow Web: Why fast is important… and why it isn’t

11 Sep 2013

0 Comment

Ever since I was a kid, this has always been one of the most exciting times of the year. Back to school, back to work… new projects, new people, new ideas, new rhythms. Because my world touches the ecommerce world, Black Friday looms large on my mental calendar, as it may also do for you, and with that comes a huge sense of urgency. While I honour the excitement and energy that surrounds this time of year, I want to take a few minutes to slow down and think about why fast is important — and why it isn’t.

A couple of months back, I read this essay by Jack Cheng, and it’s been kicking around in the back alleys of my brain ever since. Before plunging into the bustle of fall, this seems like a good time to think aloud about it.

Cheng writes about a new movement — the Slow Web — which he compares to the Slow Food Movement that launched more than twenty years ago as a protest against the growing global dominance of fast food chains. Just as the Slow Food Movement was a response to the rise of fast food, the Slow Web Movement is a reaction against our accelerating and pervasive use of the web. While Cheng acknowledges that “fast food” and “fast web” are blanket terms that can be interpreted in many ways, the general gist is that both are low in nutrients, high in filler, devoured quickly (and some might also say compulsively and thoughtlessly) in overlarge portions, and yet strangely unsatisfying. The overarching sentiment is that the fast web, like fast food, is bad for us.

Some excerpts from the essay that illustrate these points:

“Fast Web is destination-based. Slow Web is interaction-based. Fast Web is built around homepages, inboxes, and dashboards. Slow Web is built around timely notifications. Fast Web companies often try to rack up pageviews, since pageviews mean ad impressions. Slow Web companies tend to put effectiveness first.”

“Fast Web is about information. Slow Web is about knowledge. Information passes through you; knowledge dissolves into you. And timeliness, rhythm, and moderation are all essential for memory and learning.”

I research and write about speed, yet when I first read this essay, I found myself agreeing with much of what Cheng has to say about Fast versus Slow Web. Since then, I’ve been thinking about why I’m able to reconcile what I do with the Slow Web manifesto.

Here’s how accelerating the user experience ultimately serves the Slow Web Movement:

  • The Slow Web isn’t about slowing down pages and choking us all back to 1993-era response times. It’s about introducing integrity to pages, and elegance and meaningfulness to how we interact with them.
  • It’s about acknowledging the excessive carbs, fats, and unpronounceable additives — in the form of third-party scripts, unnecessary JavaScript, and unoptimized images — that get in the way of this integrity, elegance, and meaningfulness, and if not cutting them out entirely, then serving them on the side, where they don’t taint the flavour of the main dish.
  • It’s about making online experiences not just seamless, but consistently seamless — introducing a predictable cadence from the moment you log on until the moment you shut down.
  • It’s about delivering information quickly, allowing the people who are engaging with that information to internalize it and use it in the most optimal way for their individual needs.
  • It’s about recognizing that you don’t always need tons of pageviews to validate your site’s existence. You need to deliver flawless experiences. Sometimes fewer pageviews and fewer return visits means you’re doing it right, not wrong.

I’ll close with one more inspiring quote from Cheng’s essay:

“Timely not real-time. Rhythm not random. Moderation not excess. Knowledge not information. These are a few of the many characteristics of the Slow Web. It’s not so much a checklist as a feeling, one of being at greater ease with the web-enabled products and services in our lives.”

How you use the web is your business, whether you’re a compulsive Twitter-checker or one of those mythical highly disciplined people who only checks their email twice a day. As somebody who veers toward the compulsive end of the spectrum, I’m not in a place to judge. But regardless of what type of internet user you are, I believe the Slow Web Movement can benefit us all.

Related posts:

By: Tammy Everts Category: Human factors Tags:
1 2 3 4 5 6 7 8 »