• On one hand, site owners are pouring massive amounts of money and energy into site development and delivery.
• On the other hand, these same site owners are ignoring the out-of-control proliferation of third-party scripts on their sites.

This raises the question: What’s the rationale behind running up a $100K+ monthly CDN tab — ostensibly to shave a couple of seconds off your load times — if you’re going to then implement a single line of unoptimized JavaScript that could not only negate those precious seconds, but take down your site completely?

The answer, more often than not, is this: There is no rationale. Most site owners are either unaware of this issue, or they don’t realize its seriousness.

If the issue of third-party performance is new to you, this post will help you:

1. Understand the impact of all those harmless-looking little snippets of code.
2. Regain control over rogue third-party content.

Background: Third-party scripts are everywhere. Some are fine. Most aren’t.

A few months ago, the folks at New Relic did some digging into the most popular third-party APIs used by the 200,000+ applications the company monitors and took a look at which ones performed the best. No surprise, these are all familiar names:

• Amazon Web Services (response time: 432ms)
• Twitter (response time: 832ms)
• Facebook (response time: 918ms)
• PayPal (response time: 1.788s)

This is good news for site owners… assuming these are the only four scripts you’re running. But third-party content — analytics, ads, trackers, social sharing widgets, etc. — are on the rise.

As I wrote here several months ago, the average top ecommerce site contains 7 third-party scripts, with some sites containing up to 25 scripts. Cumulatively, these can have a massive impact on page performance. Unoptimized scripts can slow down page load by several seconds, or even stall it completely. Third-party scripts are one of the most common points of failure for sites: just a single line of JavaScript can take down your entire site. Despite this, measuring the impact of third-party content on a site’s usability is often an afterthought — if it even gets thought about at all.

If you want to scare yourself, run a simulation using WebPagetest to see how your site would perform if one of its third-party providers went down. As a for-instance, here’s a simulation I ran, which shows what would happen to Staples.com if its third-party providers went down:

In this case, the main culprit is Omniture, which stalls the site for a full 30 seconds when it goes down. Note that about 50% of top ecommerce sites use Omniture. (I say this not to pick on Omniture, but to point out that if they ever go down, they’re going to take down a lot of sites with them.)

Outages are inevitable. As site owners and managers, our job is to figure out a strategy to mitigate these outages.

I dream of a world where all third-party providers offer clear service level agreements to users.

In an ideal world, all third-party providers would offer a clear SLA that, at the very least:

• Expresses their annual uptime guarantee as a percentage (ideally, as close to 100% as possible).
• Describes the process for reimbursing site owners (if site owners are paying for the service provided by the script) if uptime drops below the SLA guarantee.

Right now, most third-party providers don’t offer real-time monitoring of their scripts, nor do they offer meaningful service level agreements (SLAs). My hope is that, as site owners become more educated about the importance of page speed, they’re going to start demanding properly optimized scripts, as well as better monitoring, reporting, and accountability.

In the meantime, what can site owners do to take control of third-party performance?

Until recently, adding third-party code to your site meant giving up a lot of control over how your pages load. Not any more (sort of). Here are four options at your disposal:

1. Deferral

In simplest terms, deferral is a front-end optimization technique that delays the execution of non-critical scripts until the rest of the page has loaded and rendered on the browser.

Pro: It’s a relatively easy fix.

Con: Deferral won’t work for all content. If your site hosts ads, your advertisers won’t be happy to have their ads show up last. Save deferral for third-party scripts like analytics beacons, tracking pixels, and social widgets.

2. Asynchronous loading

With asynchronous loading, third-party scripts load in parallel with the crucial page content. Async code can be tricky to program, which is all the more reason why it’s been gratifying to note its increasing rate of adoption among third-party providers. All the social buttons on this blog are async versions. (You can read about why I removed the non-async StumbleUpon button.)

Pro: Lets you display ads and other business-critical third-party content without blocking primary content.

Con: Slow third-party scripts will prevent onLoad event from firing. This post will give you a detailed understanding of how the onLoad event works. But the short answer is that a page’s onLoad determines its load time as measured by performance measurement tools. Too many delayed onLoads will mess up your results. If you’re tracking thousands of pages over extended periods of time, these messed-up results will make it a pain to pinpoint other potential performance problems.

3. Third-part timing and script killing

Also known as “tag management”, this technique involves establishing an allotted time for scripts to load. Then, if a script fails to load in that time, it’s either killed or deferred. Strangeloop has been an early pioneer of this technique in both our desktop and mobile FEO solutions, and it’s been gratifying to see it catching on.

Pro: Gives site owners the most control over third-party content.

Cons: Doesn’t lend itself to hand-coding. Best performed by an automated system.

4. Just say no

Ask yourself if you really need that widget. Perform a cost-benefit analysis of what a proposed new third-party tool offers your site and determine if it’s really worth the performance hit. Because make no mistake about it: there’s always a performance hit. And too many hits, no matter how small, add up.

Pro: Status quo is easy to maintain.

Con: Can be incredibly difficult to resist the siren song of the latest widget du jour.

And don’t forget to audit your third-party scripts.

It doesn’t matter how well you optimize the rest of your site if a single line of external JavaScript can take out the whole thing. If you aren’t already conducting regular audits of your site’s third-party content, you should be. The audit should identify:

• All the third-party scripts your site is running, and on which pages they appear.
• Which of the above performance practices (load asynchronously, deferral, timing/killing) each script follows.
• Does the third-party provider offer a service level agreement? If so, what are the terms?

What’s your experience with third-party snippets? Which ones are worth the performance hit? Which ones should send us running to the hills?

Related posts:

• The day StumbleUpon stumbled: Why we removed the SU button from our sites
• How vulnerable is your site to third-party failure?
• Do you really need that widget?

If you’ve ever looked for hard data about the combined benefits of a CDN+FEO solution, you’ll be interested in the slides from my session. It includes a case study that clearly shows the acceleration gains using just a CDN, then the additional gains of adding FEO to the mix:

As I mentioned in my talk, this year is going to mark a lot of major innovation in desktop and mobile acceleration, including aggressive advances in CDN and FEO technology. A few other trends I’ve noticed from the trenches:

• 200+ of the top ecommerce sites have been running automated FEO for two or more years.
• 5 out of 10 internet retailers have an automated FEO strategy and plan to implement it in 2012.
• FEO-accelerated sites are 30-50% faster than non-FEO sites.
• CDNs are increasing their MRR with customers by 40-50% on top of existing acceleration solutions like DSA.

Many thinks to Dan Rayburn, who organized the summit and invited me to speak. It was a great opportunity to speak with a new audience, alongside an excellent roster of speakers.

Related posts:

• When good front-end optimization goes bad: How to make sure your site tests well AND looks good
• 2012 predictions: The average web page will hit 1 MB, Google and Siri will face off, and Chrome, Windows 7, and RUM will rise
• Why the performance measurement island you trust is sinking

But there’s a new problem: Now that we’re seeing wide-scale adoption of front-end optimization best practices, misapplication of these best practices could be delivering “false positives” — sites that test well but look bad for real users.

There are a lot of ways to optimize a page at the front end, and while some of these techniques can give you great-looking RUM and WebPagetest results on paper, they can also mess with how your page actually looks and performs. These days, I’m seeing more and more sites that test well but perform poorly.

I see the same pair of issues over and over. When you’re reviewing your RUM or WebPagetest results, keep an eye on these two things to ensure that how your site looks on paper matches how it looks in the real world:

1. Find out what is being deferred, and make sure that this level of deferral works for your business.

One of the things I’m seeing more and more is something called “blanket deferral”, in which almost every page resource is deemed a candidate for deferral after the onload event. Moving things after the onload event is a great technique for loading useful content first, but it has to be used strategically. If you move everything after the onload event, then your load time numbers become meaningless.

I recently had the people at an ad-driven site tell me they were shown a 50% performance gain through FEO, but when they reviewed the results, the ads didn’t show up until 2 seconds after onload. For some types of sites that level of deferral might work, but for these guys it would’ve meant a huge drop in clicks and revenue.

The best way to see if your test results are based on blanket deferral is to capture a video, using a tool like WebPagetest, to confirm that the onload event actually corresponds to how the site loads. After you capture the video (it’s easy to do: here’s a quick how-to), then scroll across the frames. This will also scroll across the waterfall, like this:

Cross referencing the waterfall and the pictures will help you sniff out tests that just don’t feel right. When you’re looking at them, ask yourself:

• Are key sections blank when I hit the blue line?
• Does something actually render at the red line?

If the answer to either of these questions is “no”, then your test may be showing you that someone has implemented blanket deferral on this page.

2. How do your pages look within a flow?

Up to 96% of your page views will not be seen as either a first view or repeat view, as characterized by WebPagetest. Most of the page traffic on your site is “flow” traffic, in other words, pages that are seen after the visitor has already visited at least one other page on your site. If you’re focusing solely on how your pages look for first-time and repeat views, you’re not getting a realistic sense of how your pages load for most of your traffic.

Take, for example, a product page on a typical ecommerce site. That page may be accessed directly from a search engine, but in many cases that page will be accessed after first visiting at least one or two pages on your site.

Why do you need to care about flow traffic? Because badly implemented FEO can slow down a flow.

How to measure flow timing via RUM: Try to extract a graph like this to show you how fast pages are based on the session page:

How to measure flow timing via WebPagetest: Using WebPagetest, run a script to mimic a typical flow. Go to the site and click on the “Script” tab under “Advanced Settings”:

Enter a script to navigate to the page in question — something like this:

Either of these tests will give you a more insightful look into how your pages perform for most visitors than a simple test that shows “first view” and “repeat view” results.

Closing thoughts

One of the main tenets of our industry is that we can only manage what we measure. Another main tenet is that real-user testing is crucial, because the entire point of what we’re doing is to deliver the best possible online experience.

We need to make sure that we’re not gaming our measurements — however unintentionally — by some of the very front-end optimization techniques we tout. If we’re optimizing pages to make them faster but ultimately delivering a poorer user experience, then we’re not doing anybody a service.

Related posts:

• A non-geeky guide to understanding performance measurement terms
• Why the performance measurement island you trust is sinking
• Page Test Tools: Which results should you trust?

It’s a pretty short video, but if you’re in a rush, you can skip the first five or six minutes where I talk about how we launched Strangeloop as a performance pioneer six years ago, and get right into the juicy stuff about the business value of performance at around the seven-and-a-half-minute mark.

Last week, O’Reilly also ran a great interview with our VP of Technology, Hooman Beheshti, which you should check out. Hooman will be talking about advanced front-end optimization for mobile at Velocity this June, and even if he didn’t work here, I’d still encourage you to attend his session. When it comes to FEO, and to performance in general, he’s one of the most knowledgeable people you could hope to find, and he has an amazing gift for explaining complex material in an interesting, entertaining way.

Related posts:

• Four important web performance lessons for SMBs
• Interesting new findings about page views, time on site, and bounce rate across desktop and mobile browsers
• How to perform a 5-minute page speed/revenue analysis of your ecommerce site

Lesson #1: Sub-3-second page loads are a serious (and attainable) goal, even for SMBs.

While LuckyVitamin.com and Wine.com are good-sized companies, they’re not Amazon or eBay. But they have Amazon- and eBay-sized performance goals. According to Wine.com CTO Geoffrey Smalling: “A four-second download was acceptable at one time, but today it’s far too slow, given advancements in computer technology and growing consumer expectations. Getting closer to two seconds was an ambitious goal, but we had to reach it or lose potential customers, especially first-time visitors.”

Lesson #2: Conversion/revenue gains are a given.

In the past, our case studies have focused heavily on highlighting the relationship between page speed and metrics like conversions. It’s very interesting to me that, these days, these gains are a given. “We’ve known for a long time that faster pages equal more revenue for our business,” says Sam Wolf, President of LuckyVitamin.com.

So, if we’re not focusing on KPIs during our proof-of-concept phase with customers, what are we focusing on? These days, site owners who “get” performance are much more focused on finding solutions that:

1. deliver,
2. deliver consistently and reliably, and
3. deliver consistently and reliably across massive sets of complex and extremely dynamic pages.

This brings us to the next lesson…

Lesson #3: Optimization is complex.

Geoffrey at Wine.com told us that “Our site is probably more complicated than most e-commerce sites because of the extremely complex regulations that govern where and how we can ship our products. These regulations are constantly changing, which has an effect on every page of our site. A cookie-cutter solution would never work for us.”

I talk to a lot of site owners, and almost all of them tell me their own version of the “here’s why my site is exceptionally complicated” story. Sam at LuckyVitamin.com told us this: “The more complex a site is, the greater the risk of breaking pages. It’s not enough to optimize on a simple per-page basis. Pages don’t function as standalone destinations. Each page is just one part of a customer’s path through the site. An optimization technique that works on one page can break the next page in that path.”

The takeaway here: Most successful ecommerce sites are complicated. Complicated sites have complicated performance requirements.

Lesson #4: “There’s no single magic bullet for making pages faster.”

This is a great quote from Geoffrey: “Anyone who tells you there is [a single magic bullet] does not have a deep understanding of the myriad issues that affect performance. Performance tuning requires a multi-tiered approach.” (It should be noted that Wine.com uses Strangeloop alongside its Akamai DSA and in conjunction with the efforts of its small in-house performance engineering team. Most of our customers use a combination of FEO, CDN, ADC, and in-house engineering.)

What are your hard-won lessons? Let me know what you’d add to this list.

Check out the full case studies: Wine.com and LuckyVitamin.com.

Related links:

• 4 awesome slides showing how page speed correlates to business metrics at Walmart.com
• Case study: The impact of HTML delay on mobile business metrics
• Case study: PrintingForLess.com proves that mortal companies are no longer settling for “fast enough”

A few things I noted in the first quarter of 2012:

• We saw a lot of jockeying for performance dominance and audience dominance among browsers, indicating that the war isn’t over yet. This is great news for all of us — from users to site owners.
• Our understanding of issues that affect mobile performance and third-party performance — topics that most of us were ignorant of just a couple of years ago — is increasingly nuanced.
• More and more, we’re seeing mainstream coverage of web performance, signalling that this topic has permanently entered the public spotlight. It definitely makes my job easier.

Mainstream coverage

For Impatient Web Users, an Eye Blink Is Just Too Long to Wait
Really great piece in the New York Times about Google’s research into the neuroscience behind perceived page speed. Excerpt: ”Remember when you were willing to wait a few seconds for a computer to respond to a click on a Web site or a tap on a keyboard? These days, even 400 milliseconds — literally the blink of an eye — is too long, as Google engineers have discovered. That barely perceptible delay causes people to search less.”

How One Second Could Cost Amazon $1.6 Billion in Sales
Fast Company published this piece about our (lack of) tolerance for wait times in general, and about page speed in particular. Among other things, they cite the stat that one in four people abandons a page if it takes longer than four seconds to load.

Webpages showing sharp growth in girth
This was a fun example of the trickle-up process in action. On December 20th, I predicted that the average web page would hit 1MB in 2012. On December 21st, GigaOM picked it up. On December 22nd, it showed up on BBC News. I love the internet.

Think Quarterly: The Speed Issue
Google’s “Think Quarterly” isn’t exactly a mainstream publication, though it’s pretty widely read in the tech community, but I’m including this link here because I love the broad approach — ranging from technical to philosophical — of this particular issue, which is dedicated to the idea of speed. It includes articles by Urs Hoelzle, Kristen Gil, Jeff Jarvis, and Tjaco Walvis. Definitely a must-read.

Browsers

Which Browsers are the Fastest?
Interesting study from New Relic. Key findings were that, while IE 9 outperforms other browsers on Windows, Chrome 13 on Mac was overall the fastest experience (2.4 seconds). In mobile speed tests, the fastest experience was on Blackberry Opera Mini (2.6 seconds), twice as fast as Safari 5.1 on iPad.

Internet Explorer market share surges, as IE 9 wins hearts and minds
Summary: ”The browser wars are back on in earnest. For the second time in three months, Internet Explorer made large gains, picking up almost 1 point of market share. Chrome, Firefox, and Safari all lost out, as Internet Explorer 9 won over new users.”

Browser Speed Tests: Chrome 17, Firefox 10, Internet Explorer 9, and Opera 11.61
As always, the latest round of Lifehacker browser speed tests generated some interesting results. Overall, Chrome was the winner, but the tests crossed a lot of parameters, and each browser had its strengths. Definitely worth checking out, especially if you’re new to understanding the nuances of browser performance.

Internet Explorer Performance Lab: reliably measuring browser performance
The MSDN blog offers a detailed look behind the scenes at the IE Performance Lab. Be sure to check out the comments, too.

Chrome Fast for Android
A must-read if you care about mobile web performance: Google engineer Tony Gentilcore’s top 10 Chrome for Android speed features.

Chrome 16 – World’s Most Popular Browser
Interesting observation from Google engineer Mike Belshe: In January 2012, Chrome 16 quietly became the world’s most popular browser.

Case studies and other research

Just One Second Delay In Page-Load Can Cause 7% Loss In Customer Conversions
The folks at TagMan partnered with the UK’s leading glasses e-tailor, Glasses Direct, to study page speed and conversion behavior. Their findings substantiated Aberdeen’s 2009 findings, in which slower page load times correlated with a 7% drop in conversion rate.

Real User Monitoring at Walmart.com
Cliff Crocker, Aaron Kulick, and Balaji Ram joined forces at a meeting of the SF Web Performance Meetup to tell a RUM (real user monitoring) story through the lens of three job functions at Walmart.com: the performance data analyst, the developer, and the business analyst. Some excellent slides demonstrating the business value of performance. (I featured my four favourite slides in this post.)

The Performance Golden Rule
Excellent post from Steve Souders. He revisits his six-year-old stat about where end-user response time happens and confirms that 76-92% still happens at the front end. I really love this kind of post. In our industry, it’s crucial to keep revisiting these kinds of numbers and testing our assumptions.

The Need for Speed
Great case study for web performance geeks: How Chemeo delivers “pure bounce with pure value”.

No framework needed
Nice case study from 37signals about how they shaved 500ms from a page and got a 5% conversion bump.

January 2012 Site Performance Report
It’s great to trumpet success stories, but we learn as much by talking about what’s not working as by talking about what is.  This report card — in which the team at Wayfair shares how they suffered page slowdowns of up to 370% —  is a good reminder that optimization is a neverending task.

When good back-ends go bad
In this post for the annual Performance Advent Calendar, Pat Meenan talks about the fact that, while the back end is only responsible for 10-20% of response time, when it goes wrong, it can really go wrong. Among other things, Pat shares his findings that ”It is not uncommon to see 8-20s back-end times from virtually all of the different CMS systems.” Ouch.

Mobile

M-commerce site performance hampered by Verizon
Not exactly news, but still a good reminder from Internet Retailer that you’re at the mercy of networks when it comes to mobile performance: “The load times for virtually all of the 30 retailers on the Keynote Mobile Commerce Performance Index for the week ending Feb. 12 were negatively affected by a dip in performance on the Verizon wireless network.”

Mobile UI performance considerations
More goodness from the Performance Advent Calendar. If you’re just starting out with mobile web performance and don’t know where to begin, this post by Estelle Weyl is a good place to start. You should also check out her session slides from Velocity EU.

Tips and how-tos

Web Font Performance: Weighing @font-face Options and Alternatives
Nice collection of tips for making sure your web fonts aren’t slowing down your pages.

Timing the Web
Still more Advent Calendar goodness. dynaTrace technology strategist Alois Reitbauer explains how to use navigation timing to measure page load time.

Third parties

3rd Party Providers and DNS Poisoning Risks
Here’s a topic that doesn’t get talked about enough, via the Catchpoint blog: ”DNS cache poisoning refers to data breach, whereby new DNS records are introduced in the DNS Cache of a resolver or computer and divert traffic to a different IP address. The cache poisoning can be caused by hackers trying to divert traffic to a phishing/malware sites or it could be a configuration mistake. In either case the end result is not good for end users – they will end up accessing the wrong site or not access your site at all.”

Are External Services Slowing You Down? New Relic Infographic Reveals the Fastest and Most Popular External APIs
More research (with infographics) from New Relic, this time showing the four fastest 3rd-party APIs among the 200,000+ applications the company monitors.

Tools

Welcome YSlow Open Source
Yahoo has released the source code for YSlow under the BSD Open Source license. They’re encouraging devs to “use the source code, learn how it works, fork it to make your own projects and enhance it with new rules, features, and whatever will improve this tool we all love.”

Google: SPDY Gaining Adoption
Article in Web Pro News about the fact that SPDY is gaining support among browsers, developers, and vendors: “Recently, Firefox has added SPDY support, which means that soon half of the browsers in use will support SPDY. On the server front, nginx has announced plans to implement SPDY, and we’re actively working on a full featuredmod-spdy for Apache. In addition, Strangeloop, Amazon, and Cotendo have all announced that they’ve been using SPDY.”

Introducing mod_spdy, a SPDY module for the Apache HTTP server
Google engineers Matthew Steele and Bryan McQuade break down how Google’s mod_spdy for Apache works.

The Biggest Misconception about Google Page Speed
Really interesting findings from Catchpoint: “There is no correlation between Page Speed Score and how fast a page loads.” This corroborates our own findings here at Strangeloop. A while back, I was talking about this with Pat Meenan, and a probable reason for the drop is because the newest version of Page Speed checks for more optimizations, which has resulted in lower scores across the board. It’s also important to know that Page Speed doesn’t take into account advanced front-end content optimization techniques — nor could it, because there are way too many of them to track and measure. As a for-instance, we’ve had experiences here at Strangeloop where we’ve taken a page with a perfect Page Speed score, accelerated it with Site Optimizer, cut load time in half, and ended up with a new Page Speed score of 74%.

And finally, a shameless plug for Strangeloop’s annual performance state of the union, which came out in January. We tested the top 2,000 Alexa-ranked retail sites and analyzed the results against our findings from the previous year. Among other things, we found that the average e-commerce website takes 10 seconds to load, web pages are getting bigger, and Internet Explorer 9 outperforms other browsers.

If you have any other links to share, let me know in the comments. And if you’re looking for more great links, we have hundreds — sorted by topic, industry, and type — over in the Strangeloop WPO Hub.

Related links:

• 2012 predictions: The average web page will hit 1 MB, Google and Siri will face off, and Chrome, Windows 7, and RUM will rise
• 4 awesome slides showing how page speed correlates to business metrics at Walmart.com
• New report: What we’ve learned from two years of watching the top 2,000 e-commerce websites

At the time, I mentioned that I planned to conduct a similar project to identify typical latency for mobile users versus desktop users. We recently completed our research, and the results were surprising. While we expected mobile latency to be high (last fall, some ad hoc research led to the realization that latency can hit 350ms on 3G), we also expected latency for desktop users to be relatively low — and this was not the case.

(If you’re a newcomer to the web performance scene, here’s a short primer describing what latency is, why it’s a problem, and a few common solutions.)

Methodology

1. We took latency measurements for three US-based Strangeloop customers for discrete periods of time between February 15, 2012 and March 15, 2012. Each site uses a different content delivery network (CDN). Latency measurements were performed once per visit. We measured a total of three million unique visits.
2. While we tested across all mobile and desktop browsers, the results for Internet Explorer are not included, due to some incompatibilities with the latency measurement process. The browsers included in the results are Firefox, Chrome, Safari, and any others that were able to run the test JavaScript.
3. Desktop and mobile results were each aggregated and graphed for each site.

Results

Check out the histograms depicting the latency spread for each site. Medians are highlighted.

Site #1: Desktop browsers

Site #1: Mobile browsers

Site #2: Desktop browsers

Site #2: Mobile browsers

Site #3: Desktop browsers

Site #3: Mobile browsers

Observations

1. Desktop latency was worse than expected.

Median desktop latency ranged from 65ms to 145ms. My experience is that people tend to lowball their latency predictions for their own sites, estimating numbers in the 20-30ms range. If you do this, these numbers should make you think again.

I can’t stress this often enough: If you rely on your backbone tests to understand your site’s performance, you’re not getting an accurate view of your site’s true latency issues. Even 20-30ms is on the incredibly optimistic end of the real-world spectrum. Numbers like 76ms and 138ms are a lot more realistic. If you’re being told that your site experiences numbers below 20ms, you’re getting bad information.

2. Median mobile latency was between 26% to 96% poorer than desktop.

There’s been a spate of reports lately citing the fact that mobile users are using devices over their wifi connections at home. As a result, I’ve heard more than one web dev/design person say that this takes some of the heat off of optimizing for mobile — a dangerous opinion, in my mind. Median latency of 190ms is a serious performance hit.

3. The CDN you pick really matters.

Further to the above, the CDN you choose makes a big difference. Each of the sites we measured is on a different CDN, and as you can see, desktop latency was almost doubled from one site to another.

4. Outliers for mobile had double the latency of desktop outliers.

Check out the long tail of outliers for mobile. Individually, outliers aren’t hugely significant, but when you see a tail this long, it represents a large number of people who are experiencing painful latency – in this case up to 990ms. That’s almost a full second for EVERY page resource. Even if you’re maxing out the browser’s ability to make concurrent connections, those nigh-seconds add up fast.

5. Mobile traffic accounted for about 10% of total traffic.

As an interesting aside, we found this number held across all three sites. Note that these were mobile users who were visiting the full site, not an m.site.

Conclusions

While the sample size of three sites isn’t large enough to draw a sweeping, industry-wide conclusion, the number of visits measured — three million in total — is still significant enough to draw some meaningful conclusions.

Latency is hard to get a bead on. It’s all too easy to ignore your site’s latency issues if you assume that your site is at the fastest end of the latency spectrum, or if you assume your CDN is a catch-all solution, or if you rely on backbone tests for your performance measurement. If you’re in the habit of doing any of these things, these findings should give you pause and encourage you to perform this kind of testing on your own pages.

As a further area of study, I’m interested in looking at the wifi versus 3/4G latency numbers, as well as the latency breakdown by device. If you have any suggestions for ways to cut and slice this data, let me know.

If you have any questions about how you can replicate our latency measurement tests on your own site, ask away. I’ll be happy to help.

Related links:

• How I learned that 3G mobile performance is up to 10X slower than throttled broadband service
• Case study: How to use network quality as a proxy for measuring mobile performance
• Early findings: 97% of mobile end-user response time happens at the front end

In web performance circles, “latency” is the amount of time it takes for the host server to receive and process a request for a page object. The amount of latency depends largely on how far away the user is from the server.

To put this in real-world terms, say you visit a web page and that page contains 100 objects — things like images, CSS files, etc. Your browser has to make 100 individual requests to the site’s host server(s) in order to retrieve those objects. Each of those requests experiences at least 20-30ms of latency. (More typically, latency is in the 75-140ms range, even for sites that use a CDN.) This adds up to 2 or 3 seconds, which is pretty significant when you consider it as just one factor that can slow your pages down.

When you also consider that a page can have upwards of 300 or 400 objects, and that latency can reach a full second for some mobile users, you can easily see where latency becomes a major problem. If your goal is to have your entire page load in less than 3 seconds (and if that’s not your goal, it should be), then latency can kill you right out of the gate.

For obvious reasons, tackling latency is a top priority for the performance industry. There are several ways to do this:

• Allow more requests to happen concurrently.
• Shorten the server round trips by bringing content closer to users.
• Reduce the number of round trips.
• Improve the browser cache, so that it can (1) store files and serve them where relevant on subsequent pages in a visit and (2) store and serve files for repeat visits.

Browser vendors work around this problem by using multiple connections, which allows the browser to make simultaneous requests to the host server. Since 2008, most browsers have finally moved from 2 connections per domain to 6. Vendors also focus on improving the browser cache.

Google’s SPDY protocol extends what the browser can do by adding a session layer atop of SSL that allows for multiple concurrent streams over a single connection.

Content delivery networks (CDNs) cache content in distributed servers across a region or worldwide, thereby bringing content closer to users and reducing the round trip time. Important to note: While CDNs help with desktop performance, they don’t help mobile latency.

Front-end optimization (either manual or automated) alleviates latency by consolidating page objects into bundles. Fewer bundles means fewer trips to the server, so the total latency hit is greatly reduced. For example, using Strangeloop’s Site Optimizer, a page that starts  with 63 objects could see those objects consolidated into 9 resource requests. FEO also leverages the browser cache and allows it to do a better job of storing files and serving them again where relevant, so that the browser doesn’t have to make repeat calls to the server.

Solving latency is an ongoing, spare-no-expense effort. Just last week, it was announced that this summer marks the start of a $4.5 billion fiber-optic cable project that will connect the UK and Japan — with the sole purpose of shaving 60ms of latency. The reason why:

The massive drop in latency is expected to supercharge algorithmic stock market trading, where a difference of a few milliseconds can gain (or lose) millions of dollars.

Related posts:

• A non-geeky guide to understanding performance measurement terms
• Why you’re probably reading your performance measurement results wrong (but at least you’re in good company)
• Waterfalls 101: How to understand your website’s performance via waterfall chart

To summarize, this summer marks the start of a massive project to lay the first ever trans-Arctic Ocean submarine fiber optic cables connecting the United Kingdom and Japan. In total, the three cables are expected to cost between $1.8 and $4.5 billion. Why? To reduce latency by a mere 60 milliseconds.

This small latency boost is expected to yield big results:

The massive drop in latency is expected to supercharge algorithmic stock market trading, where a difference of a few milliseconds can gain (or lose) millions of dollars.

These kinds of results are the reason why a similar cable is currently being laid between the UK and the US:

“[The UK-US cable] will cost $300 million and shave “just” six milliseconds off the fastest link currently available.”

At Strangeloop, we’ve recently conducted some research into mobile and desktop latency. I’m working on a post about it, which should be ready next week.

Related posts:

• How I learned that 3G mobile performance is up to 10X slower than throttled broadband service
• Case study: The impact of HTML delay on mobile business metrics
• Use of content delivery networks doesn’t correlate to faster websites for the Alexa Retail 1000

Methodology

1. Targeted 9 ticketing sites, ranging from smaller agents like FindTicketsFast.com to Ticketmaster and the official MLB.com site.
2. Used Webpagetest.org to test the load time of each page in a transaction (ordering tickets for the same Rays vs. Red Sox game) up to, but not including, completing the purchase, as it would appear to a visitor using Internet Explorer 8.
3. Calculated the total transaction time for each site.
4. Took a deeper look at the numbers and individual test results to see if there were any trends.

Results

Observation #1: Most transactions were slow.

While a couple of sites – Cheap MLB Tickets and Find Tickets Fast – had relatively fast total transaction times (12.08 and 15.03 seconds, respectively), the rest were much slower, with transaction times ranging from 19.01 seconds to 43.37 seconds.

The average load time for all transaction pages across all the sites was 5.85 seconds. This might not sound like much, but as I regularly remind people, more than half of online shoppers will abandon a page after 3 seconds.

Observation #2: The official MLB site and Ticketmaster offered the slowest user experience — by a large margin.

I assumed these two sites would be among the fastest, but in fact Ticketmaster and MLB.com were by far the slowest, with total transaction times of 39.67 and 43.37 seconds respectively.

Looking at the test results, it’s easy to see why these transactions took so long. Over the course of the MLB.com transaction, there were a total of 636 page requests – content like images and third-party scripts that have to travel from the servers to the user before the page can render. MLB.com is pretty graphics intensive, so a lot of that content is individual image files. But when you look at waterfall charts for the site, you can see dozens of JavaScript and CSS files that are preventing visible content from loading. This is the kind of content you should defer or have load asynchronously to give your pages a huge performance boost right out of the gate.

Observation #3: The average transaction took five pages, not including checkout.

The number of pages in each transaction varied from four to six. From a usability perspective, this is a significant amount of friction. We know that the fewer pages in a transaction, the higher the number of visitors who convert (make a purchase, download an app, etc.).

There’s a reason why Amazon initiated one-click checkout a while back. They know that getting shoppers through the sales funnel faster leads to higher conversion rates, greater order value, and better customer satisfaction. To illustrate, there’s a case study that came out a few years ago that showed how one e-commerce site eliminated just one page in an online transaction and dropped their transaction abandonment rate from 45% to 33%.

Observation #4: For a fast ticket-buying experience, eBay is a solid bet.

I chatted with Sam Laird at Mashable about these results (read his post here), and he wondered how fast it would be to buy the same tickets on eBay. A few tests later and we have the number: 12.2 seconds, making eBay among the fastest options.

Observation #5: If these transactions are slow for desktop users, they’re even worse for mobile users.

These findings have huge ramifications for mobile consumers. A page that takes 15 seconds to load on your desktop could take a minute or longer to load on your smartphone, depending on your device and connection. Or worse, the transaction could end in an error message, which is what happened when I tried to time the MLB.com transaction on my iPhone:

Mobile ticket buying is already a big industry, and it’s only going to get bigger. According to comScore, 35% of smartphone owners have bought event tickets online. According to StubHub.com, mobile ticket sales could hit a 10% of total sales this year, up from 5% last year. And Juniper Research predicts huge growth in the next four years: worldwide mobile ticketing transactions are set to quadruple to 23 billion by 2016.

Should giants like Ticketmaster be running scared from smaller, faster ticketing agents like Cheap MLB Tickets?

In the short run, no. But in the long run, as the mobile ticketing market matures and consumers conduct more and more transactions via smartphone, those people aren’t going to wait minutes for each page in a 5-page transaction to load.

Some would argue that apps are the answer to these problems, and this is somewhat true, but only for power users. Mobile shoppers still use the web more than apps, according to a recent Nielsen study. While apps will continue to be popular, it’s incredibly short-sighted to think they’re a cure-all for performance.

Related posts:

• A non-geeky guide to understanding performance measurement terms
• Web performance and the 2012 US election: Is site speed an early indicator of future success?
• New report: What we’ve learned from two years of watching the top 2,000 e-commerce websites