CDN

Case study: How effective are CDNs for mobile visitors?

9 May 2013

9 Comments

Substitute “measuring mobile performance” for “herding cats” in this video, and you’ve pretty much nailed the challenge we’re up against every day.

Fortunately, we like cats. :)

Experiment: Measuring the impact of CDN deployment on 3G performance

As we continue to evolve our mobile treatments, we also monitor their effectiveness alongside other optimization solutions. Today I want to call out some interesting results we noted when, as a fun little in-house exercise, we took the O’Reilly website, de-optimized it, and then iterated through a handful of core performance best practices using our FEO service. The goal was to demonstrate the acceleration benefit (in terms of bytes in, start render time, document complete time, connections, and resources) of each practice for a typical 3G mobile user.

While we saw predictable results for step 1 — enabling keep-alives and compression — we were somewhat surprised by what we saw when we added a content delivery network.

Step 1: Apply fundamental best practices

First we added keep-alive connections:

  • What it does: Lessens the impact of TCP connection setup
  • Benefit: Addresses the problem of having too many TCP connections

Then we added HTTP compression:

  • What it does: Compresses text-based content (HTML, CSS, JavaScript, etc.)
  • Benefit: Easy way to reduce bytes/payload

We got the results we expected: faster start render, and about a 25% reduction in doc complete time. This is fantastic, even more so because both these treatments are really easy to do — usually it’s just a matter of a single configuration option on your server, proxy, or load balancer. However, these two treatments aren’t enough to give you the acceleration you need.

Step 2: Use a content delivery network (CDN)

Here’s the elevator-ride explanation of how a CDN works (for an excellent detailed explainer, go here): Static page assets (images, CSS, etc.) are served from locations near the requesting client (mobile or otherwise). The shorter distance between client and content means smaller time to first byte (TTFB) and, ostensibly, faster start render. This means users start to see content in their browser faster.

How a content delivery network works

In our experiment, we expected that adding a CDN would result in faster average time to first byte, start render, and doc complete time. Here’s what we saw:

Before and after: Document complete (aka load time)

Before and after content delivery network deployment

Before and after: Time to first byte (TTFB)

Before-and-after CDN: Time to first byte (TTFB)What we helped

CDN before and after: What we helpedFindings

With a CDN, the page got faster, but not by much. For the unoptimized page, we forced requests to travel from west coast to east coast. After, we let the CDN naturally select the closest edge. As a result, we saw that:

  • Doc complete time decreased by 10%, compared to a 20% improvement we noted in a similar experiment in desktop optimization.
  • We shaved less than a second off start render time, taking it from 7.059 seconds down to 6.245 seconds.
  • True, we cut time to first byte by 39%, but from an end-user perspective, TTFB doesn’t really mean anything because the user still isn’t seeing anything in the browser.

Five questions about CDNs and mobile acceleration

I’m not making any nutty claims like “CDNs aren’t effective for mobile devices.” (Heresy!) But these results do raise a few questions:

  1. Is edge selection for mobile devices not as effective as for desktop?
  2. Some, if not all, CDNs probably deploy servers near cell network exit points. But what if most of the latency occurs inside the cell network? (I’ll admit I’m not an expert on what happens inside a 3G network. I’m ready to be enlightened.)
  3. Does the meaning of “closeness” change for mobile?
  4. Acknowledging the existence of mobile-specific CDNs, how much more effective are they? How do they compare when it comes to 3G versus WiFi? I’ve been trying to dig up case studies, with no luck.
  5. High CDN costs may be justifiable when you see significant benefit for your desktop traffic, but do they deliver sufficient benefit/ROI for mobile users?

Takeaway

LTE is grabbing a lot of attention, but it’s a mistake to sweep 3G under the rug. There are 256 million 3G subscribers in the US, representing 81% penetration, so 3G performance is still a big deal. We need more research. If you have findings to share, please do.

Related posts:

By: Tammy Everts Category: Tags: , , , ,

Lori MacVittie (F5): “Fast is an incredibly, incredibly stressful thing to try and do.” [PODCAST]

16 Apr 2013

2 Comments

I am so incredibly tempted to call Lori MacVittie the First Lady of Web Performance, if only because I know how hard it would make her laugh. But seriously, in the intro to my latest podcast, I said that Lori’s forgotten more about the history of our industry than most of us ever knew, and I meant it.

Performance has become a hot topic in the past three or four years, but it’s been around since the inception of the web, and it’s meant different things at different times. Up till six or seven years ago, the focus was on just keeping sites up under load stress, no matter whether or not this meant losing a few seconds. Fast forward to today, and things have done a complete turnabout. As most of us know, optimal performance is not only about supporting millions of users, but also ensuring that each of these users is blazing fast. As Lori points out, “it’s an incredibly, incredibly stressful thing to try and do, because you’ve got even more moving parts now than you did then.”

Lori and I covered a lot of ground in our chat, from the early acceleration pioneers who helped birth modern-day FEO, to ADCs and SDNs and a bunch of other cool acronyms. Lori also made a heroic attempt to convince me that the cloud is interesting. You’ll have to listen to the podcast to see if she succeeded. :)

(Note: I want to apologize in advance for the audio quality in this podcast. We’re working on making it better.)

Listen to the podcast: Lori MacVittie

Related posts:

By: Tammy Everts Category: Tags: , , , , , ,

More new findings: Top ecommerce sites are 22% slower than they were last year

27 Mar 2013

8 Comments

Let me say that again, because this is a staggering fact: The world’s top ecommerce sites are 22% slower than they were last year.

In December 2011, the median load time for a site in the Alexa Retail 2000 was 5.94 seconds. Just twelve months later, the median was 7.25 seconds. At this rate of growth, this number could hit almost 9 seconds by the end of this year.

Web page load time changes: December 2011 to December 2012

This was the key finding of our brand-new quarterly report (yes, a new report, not to be confused with last week’s report about web performance in the EU) on ecommerce web performance. If you’re new to these reports, since 2010 we’ve been measuring the load time, page composition, and best practice implementation of the same set of 2,000 leading online retailers, as ranked by Alexa. The goal is to learn how pages are changing over time and what impact, if any, these changes have on per-page performance. The results have been eye-opening.

I have to confess that I frequently feel like that tiresome guy at the party who keeps saying the same things over and over again. Pages are getting slower… pages are getting bigger… the gap between load times and user expectations is getting wider almost by the week. If you’re reading this out there and saying to yourself, “I can’t believe that Bixby guy is going on about this AGAIN,” then forward the next part of this post to five people you think could really benefit from it. If enough people internalize this message, maybe I’ll shut up.*

Three performance myths I would give anything to permanently bust:

Myth #1. Pages are, de facto, getting faster.

What with our better systems, networks, and browsers, pages must be getting faster, right? Everyone believes this instinctively, because most of us seem to be hardwired to believe that technology solves problems rather than creating new ones. But as I said at the top of this post, the quickly emerging fact is that pages seem to not just be getting slower, they’re getting slower at an alarming rate. (Optional: You may choose to take this finding as proof that we shouldn’t always trust our instincts. :) )

Myth #2. Users are more or less satisfied with the status quo.

“People are used to pages that take 5-8 seconds to load. They don’t mind that much.” I still hear this on a regular basis. Site owners rationalize that, because they’re not hearing a lot of complaints, their visitors are happy… or at least happy enough. But as numerous case studies have shown, people talk with their wallets. Faster sites earn more. And user surveys over the years are telling us that people’s expectations for a speedy online experience are continually growing:

Web Page Load Time: User expectations 200-2012

Myth #3. Browser development is more than capable of mitigating the factors, such as page size and complexity, that are causing pages to slow down.

This belief is widely held, even among technical folks. Again, looking to our findings, we saw that for all three browsers, median load times slowed down by anywhere from 3% to 12% in just six months. This downward trend isn’t a browser development issue. Instead, it’s an indicator that despite browser vendors’ huge commitment to speed, development can’t keep pace with the demands of bigger and increasingly complex web pages.
Browser performance: 2011 to 2012

Takeaway: Pass it on.

I am extremely happy that Radware is committed to continuing the tradition of releasing these quarterly  “state of the union” reports. As time passes, we’re gaining some invaluable insights. Based on these latest results, I’m very curious to see what our Summer 2013 report will hold.

I urge you to download this report (and the infographics, too). And I was only slightly kidding when I suggested that you forward this post to people who need to have a few performance myths dispelled. Our community does a lot of preaching to the choir. What seems basic to us is not necessarily basic to the rest of the world. We need to get out there and make sure these simple messages are being heard.

*Maybe.

Related posts:

By: Tammy Everts Category: Tags: , , , , , , ,

New findings: Typical leading European commerce site takes 7.04 seconds to load

19 Mar 2013

3 Comments

Last fall, at Velocity London, I had a really great talk with Stephen Thair, who is a UK-based web performance consultant, Velocity committee member, WebPerfDays organizer, and all-around knowledgeable guy. Among other things, we talked about how frustrating it can be for performance pros based in Europe to preach outside their community.

As Stephen said:

“I guess it’s a bit frustrating in the UK at the moment. One of the things that I found is that we haven’t yet got that killer web performance case study in one of the big major retailers. So we are still, I think, a bit in the evangelical stage. We are still trying to get the message out there. There are still a lot of websites in the UK that don’t even have gzip turned on.”

So we set out to help fill that gap. In December of 2012, working with Radware (our soon-to-be parent company) in conjunction with our partners at Level 3 in Europe, we studied the page speed and composition of 400 top European retailers, as ranked by Internet Retailer magazine, to see how these sites would load for visitors using Chrome 23 (the most popular browser in the EU at the time of testing) via the test server in Amsterdam. (We chose the Amsterdam location because it allowed us to test across all major browsers.) The report was released today.

While the results may not be shocking if you’ve been paying close attention to this space, they may come as an eye-opener to online retailers in the EU. Our chief finding was this:

The median page took more than 7 seconds to load.

Depending on whom you ask, the average internet user expects web pages to load in less than 3 seconds, 2 seconds, or even 400 milliseconds. The last time the average person reported being cool with 7-second load times was around 2001.

The survey also found that:

  • 1 out of 4 pages took more than 10 seconds to load.
  • 1 out of 3 pages contained more than 100 resources.
  • 79% of sites don’t use a recognized CDN. (A “recognized CDN” refers to any CDN listed in the extensive directory of CDNs maintained by WebPagetest.)
  • Speaking to Stephen’s point about gzip at the top of this post, 1 out of 5 sites failed to implement text compression, a relatively simple technique that delivers easy, significant performance gains.

Why you should care about these findings

I may be pointing out the obvious, but it may need to be pointed out: these expectations are universal. Internet users in the EU do not have lower performance expectations than users in North America. These findings should be a wake-up call for European site owners. (Not that North American site owners should be resting easy. Last fall, we found that the median leading US commerce home page took almost 7 seconds to load.)

Download the report: State of the Union: European Ecommerce Page Speed and Web Performance

Related posts:

By: Tammy Everts Category: Tags: , , , , , , ,

Aaron Peters (Turbobytes): Why all CDNs are not created equal [PODCAST]

22 Feb 2013

2 Comments

This week’s podcast is, for me, a fascinating convergence of two hot-button topics: content delivery and real user measurement (RUM).

This is a really exciting time in the CDN space. If you haven’t spent a lot of time in the content delivery world, you need to know that it’s an intensely competitive space. In just a few short years, it’s shifted from being a virtual monopoly to a much more competitive landscape, with new contenders arriving almost weekly, it seems. Content delivery can be expensive. It can also be really hard to know which CDN will actually deliver the goods the fastest, since this can fluctuate over time and geographical locations.

Over in its own parallel world, RUM has really evolved over the past few years — from an arcane topic discussed by only the geekiest geeks to a subject that’s on pretty much every site owner’s lips (or at least should be). It’s a no-brainer that any content owner wants to get as much insight as possible into how their content moves out in the real world. The funny thing is that it took so long for our tools to catch up to this need.

It was only a matter of time before someone saw an opportunity to apply RUM to pick the best CDN by region/time. Pioneered by my good friends at Cedexis, this practice has been adapted for the SME market by my guest this week, Aaron Peters, along with his partner Sajal Kayan. Their company, Turbobytes, which launched last spring, monitors the real-world performance of six global CDNs, and then ensures that site owners’ content is always delivered by the fastest one. They buy buckets of CDN capacity and resell it to their customers, thus creating a one-stop shop for CDN measurement, dynamic CDN selection, and small object caching. And as the icing on the cake, Turbobytes wants to monetizes all the RUM data it gathers by selling it back to the CDNs.

Aaron was kind enough to sit down and share with me how he and Sajal came to this business model.

Listen to the podcast: Aaron Peters

Related posts:

By: Tammy Everts Category: Tags: , , , , ,
1 2 3 4 5 6 7