site speed

Case Study: How Symantec is showing its visitors the wrong content first

In my post on how to read waterfall charts, I touched on the importance of knowing how quickly a page starts to render, because this can have a powerful effect on a visitor’s perception of your site’s overall speed.

This is all true, but here’s where it gets a bit tricky. Yes, it’s important to know when a page starts to render and how long it takes to fully load. But this information isn’t enough. You also need to look at which elements are rendering first and make sure those are the elements you most want your users to see.

Sounds elementary, right? And yet time after time I see sites that make the same mistake: having their most important content load dead last.

Case study: Symantec

Take for example this case study provided by Jakob Nielsen. His team recently conducted an eyetracking study in which they monitored how people viewed a landing page on the Symantec website.

This graphic shows all the places a visitor’s eyes fell when viewing a page that they perceived as loading instantaneously:

Norton: Visitor eye gaze patterns for page that downloads instantly

Notice how the visitor’s gaze spends a significant amount of time on the yellow slideshow widget, the most critical content on the page.

Now compare that gaze pattern to this graphic, which shows all the places a visitor’s eyes fell on the same page when the slideshow widget took 8 seconds to load:

Norton: Visitor eye gaze patterns for page that loads in 8 seconds

Notice how the visitor looked at the empty yellow box a few times while waiting for something to appear, then gave up and spent the rest of their time looking at the rest of the page.

This slideshow widget was the most important piece of content on the entire page, and yet it was the last thing visitors saw. And even after the slideshow fully loaded, the visitor never even glanced at it.

As Jakob observes in his post:

The user who had to endure the download delay spent only 1% of her total viewing time within this space. In contrast, the user who in effect received instantaneous page rendering (because he didn’t look until it was done), spent 20% of his viewing time within the slideshow area.

Let’s look at this problem another way

I wanted to see if Symantec has addressed this usability problem yet, so I ran this page of their site through Webpagetest.* (Jakob’s test took place a while back, on a page that is no longer on the site, so I chose a landing page as similar as possible to the one he used.)

Here’s the waterfall of the results:

The page started to render in just over 4 seconds, and fully loaded in 7.3 seconds — an okay, if not awesome, performance. But let’s look a little bit closer, first in video:

It’s apparent that the featured content widget is still one of the last page elements to load. Now let’s slow it all down and take a frame-by-frame look at the video so that we can see exactly what loads when.

First up, we can see that page elements don’t start to appear until 4.5 seconds:

Webpagetest: Symantec at 4.5 seconds

Next, we see the yellow box that houses the promo widget appears at 5.6 seconds:

And finally, the promo content shows up at 7.1 seconds:

So what to do with this information?

Before reading about Jakob’s eyetracking study, it would be easy to ignore these seemingly tiny lag times. But armed with this information, and knowing that when it comes to web performance and conversion rates, every millisecond counts, it’s clear that this is a performance problem that needs to be fixed. That widget needs to be one of the first, not one of the last, elements to download.

Your tasks

  1. Identify which are the most important pages on your site.
  2. On those pages, prioritize the most important pieces of content.
  3. Analyze those pages using a third-party tool that lets you see the order in which the page elements load, and how long it takes for each of them to load.
  4. Using either a manual or automated solution, tweak your content so that it loads according to its priority.
  5. Repeat as pages change, as new pages are added, and as new browser versions are released.

*Test conducted on 07/28/10 at 11:05:19 from Dulles, VA. Simulated results on IE7 via DSL. See full test results here.

Related posts:

Mobile Web Performance: Desperately seeking data

At some point you’ve probably asked yourself some variation of this question:

If my site takes X seconds to load now, and generates Y dollars, will speeding it up by Z seconds lead to more dollars? And if so, how many dollars?

Substitute “conversions” or “downloads” for “dollars” – whatever word you choose, that’s what it boils down to, right? It’s a fundamental question. The mainstream web performance community is rapidly amassing reams of data in this area. But when it comes to the mobile web, we’re pretty much at square one.

These days, you can’t scan the tech news without reading headline after headline trumpeting the growth of the mobile market. Anyone can tell you that there are 75 million American adults using the mobile web and that by 2013 that number is projected to hit 134 million. And you’ve probably already read the study stating that more than half of mobile device users expect sites to download as quickly on their mobile devices as they do on their home computers.

But you know what kinds of numbers are hard to come by? Real data about mobile web performance. If you look at the Analytics section of Mobile Commerce Daily, you’ll see all of three articles. None of these articles touch on mobile performance and key business metrics such as conversion rates, downloads and revenues.

What are the challenges in gathering mobile performance data?

The same challenges that faced those of us in the web performance industry a few years ago:

  • Lack of tools for measuring performance. We need tools that are the mobile equivalent of Webpagetest that can give us data about how individual mobile sites perform in the real world.
  • Need for large-scale A/B testing by major mobile sites. Interest in web performance began to soar after online monoliths like Microsoft, Amazon and Shopzilla conducted hardcore long-term studies of how performance changes affected user behavior and business metrics.
  • Lack of information sharing. Further to my point above, I’m sure that large companies are already analyzing their mobile performance, but they’re keeping these numbers close to their vest. At some point down the road – Velocity 2011, I hope? – this information will start to trickle down to the general public. This will be what drives companies to finally focus on improving the speed of their mobile sites.

Wouldn’t it be great to have charts and graphs like these?

Business impact of improved performance on order size and conversion rate

Conversion rate falloff by landing page slowdown

This is all the information I’ve ever managed to corral about how improving a mobile website’s performance may have affected conversion and cart size. Please note these words: May have. This is data gathered over the course of eight months of A/B testing on Strangeloop client AutoAnything‘s mobile site. It appeared that accelerating the site by 40% caused the average order size to increase by 3% and conversions to increase by 5%. As the second graph shows, we also saw that slowing down the mobile site caused a conversion decrease that paralleled the decrease on the regular site.

Again, I want to emphasize that this is very preliminary stuff (read: I don’t want to read these numbers being touted as universal truths on some other blog a month from now), but it goes to show what kind of numbers we should be trying to gather and share. The information is out there. We just need to get our mitts on it. It took our community almost ten years to generate meaningful data around regular web performance. We don’t have that luxury with the mobile internet.

Do you have any numbers or methodologies to share? Ideas to suggest? Get in touch with me by email (joshua /at/ webperformancetoday /dot/ com) or in the comments.

Related posts:

Announcing the Strangeloop Site Optimizer

Announcing the Strangeloop Site Optimizer. Have a cigar!I didn’t start this blog to shill product, but it would be kind of weird if I didn’t at least mention that my company, Strangeloop, recently released Site Optimizer, a new family of site acceleration products and services that is already yielding pretty incredible results for companies like O’Reilly and Sodexo. Consider this my “proud papa” birth announcement. Your cigar is in the mail.

If you’re curious to learn more about Site Optimizer, David Hamilton at Web Host Industry Review was kind enough to interview me about it last week.

Friday Four: Page load races, Future Day, and surprising data on how people are using the iPad

Which Loads Faster
I’ve been having a lot of fun with this little tool, which pits sites against each other in head-to-head speed trials. See who’s faster – Google versus Bing, Amazon versus Shopzilla, TMZ versus OMG! – or try your own matchup. I asked its creator, Ryan Witt, how Which Loads Faster works, and he told me that the “pages are loading in real time in your browser and I’m measuring the time it takes using javascript.” Pretty nifty.

“Future Day” Mistake Spreads Like Wildfire Online
Earlier this week, when Total Film magazine mistakenly posted that July 5, 2010 was the future date that Marty and the Doc travel to at the start of the second Back to the Future movie, it highlighted how quickly information now travels online. Total Film corrected their mistake by posting a Photoshopped image of the time machine with the wrong date in it, as a joke, but the joke image got picked up as real news and became a trending topic on Twitter. It goes to show two things. One: People seem willing to believe anything they see. Two: Once incorrect information starts spreading, it’s impossible to control it. [via Human 2.0]

Whatever Happened to Voice Recognition?
Speaking of back to the future, remember voice recognition? Back in the ’80s and early ’90s, it was going to be the wave of the computing future. This post by developer and human factors thinker Jeff Atwood explains why voice recognition will probably never pan out.

An In-Depth Look at How People Are Using the iPad
And back to the present, Mashable just posted an iPad usage survey, which revealed a few surprises. For instance, while the iPad was never touted as a portable gaming device, more than one-third of respondents said that after owning an iPad, they would not buy a separate gaming device: “The size of the device and its accelerometer really make for an immersive gaming experience.” That said, more than half of all current and prospective iPad owners owners said they see it as a non-essential luxury item, not a necessity.

Waterfalls 101: How to understand your website’s performance via waterfall chart

[This post later served as a jumping-off point for a 30-minute webinar: Waterfalls 101: How to read a waterfall chart]

I keep running into executives who are being presented waterfall charts and don’t know how to interpret them. Today I thought it would be a good idea to take a bird’s eye view of a typical performance waterfall – pre- and post-acceleration – that you can take away for your own reference or pass along to anyone you think could benefit from having this information.

(If you’re technical, live and breath waterfalls, and never plan to speak with an executive, stop reading now. :) )

Waterfall charts are diagrams that let you visualize data that is generated cumulatively and sequentially across a process. In the case of performance waterfalls, they let you see the series of actions that occur between a user and your server in order for that user to view a specific page of your site.

Here’s what a waterfall looks like for a site* that has not been optimized in any way:

Waterfall chart: Before web performance optimization

To the uninitiated, this is a mess. For now, it’s sufficient to understand that each of the rows represents a different object – including text, image, Javascript files, etc. – that the page contains. Depending on whom you ask, the average web page contains anywhere between 44 and 75 objects (including text, images, and Javascript). Each of these objects makes its own roundtrip between the server and the browser.

Now let’s go in for a closer look:

Close-up look at waterfall

Each of the colored bars represents a different activity that happens as the object is delivered to the user’s browser:

Dark green = DNS lookup

This is when the browser looks up the domain of the object being requested by the browser. Think of this as asking the “phone book” of the internet to find someone’s phone number using their first and last name. You can’t do much about the dark greens bars and they should not be a problem on most sites.

Orange = TCP connection

Also called the “three-way handshake,” this is the process by which both the user and the server send and receive acknowledgment that a connection has been made and data can begin to be transferred. It’s sort of like how CB radio users start a conversation:

“Breaker 1-9. This is Bluebeard. Anyone out there read me? Over.”
“I read you, Bluebeard. This is Jolly Roger. Over.”
“Okay, let’s QSO.”

(Important note: Everything I know about CB slang I learned from watching The Dukes of Hazzard.)

It’s not easy to speed up TCP connection, but you can control how many times the TCP connection takes place. Too many connections will slow down site performance.

Bright green = Time to first byte

This is the window of time between asking the server for content and starting to get the first bit back. Continuing our CB radio analogy, think of this as the interval between asking a question to a friend over a radio and waiting to hear the beginning of their response.

The user’s internet connection is a factor here, but there are other things that can slow things down: the amount of time it takes your servers to “think” of what content to send, and the distance between your servers and the user.

Blue = Content download

This is the time it takes for every piece of content to be sent from the server to the browser. Think of this as the time it takes from the first sound you hear on the radio to the end of the speech from your mother after she broke in on your chat and told you to get off that walky-talky and come down for dinner.

You can speed up content download by making the total amount of stuff you need to send smaller. More on that in a minute.

For now, let’s go back to our mondo chart:

Waterfall chart: Before web performance optimization

I want you to observe four things:

  1. The number of orange bars
  2. The number of bright green bars
  3. And the length of the bright green bars
  4. The number of blue bars

These four things show where this site performs poorly. Let’s break it down:

1. Too many orange bars

You can address this problem by having your developers use something called “keep-alives” to reduce the number of TCP connections. (I’m not going to go into keep-alives here, because this post is long enough already. If you want more info, feel free to email me or ask in the comments.)

2. Too many bright green bars

Remember the 75 page objects I mentioned earlier, each one making its own roundtrip? That’s what’s causing all the green and blue bars. This problem can be fixed by implementing performance best practices, such as those identified by YSlow and Google. Among other things, these best practices include consolidating page objects into fewer bundles.

3. Bright green bars are too big

You can fix this problem with a CDN, which will bring your content closer to your users. Chances are, you’re already doing this. Hopefully, this post will help show why CDNs – while a great tool in your toolset – don’t address all aspects of the performance problem.

4. Too many blue bars

Not only are there too many page objects, each of these objects is too large. The average web page is between 320 and 420 kb. In other words, too big. This problem can be partially fixed through implementing the performance best practices suggested above, and partially fixed using something called compression. (Again, contact me if you want to know more about compression.)

So how does a waterfall chart look for a site that has been optimized to follow some of the best practices?** Take a look at these before-and-after charts for the same site:

Waterfall: Before and after web performance optimization

If you take nothing else away from all this…

Remember that when you’re looking at performance reports for your site, you want to see these five things:

  1. As few rows as possible.
  2. As few orange bars as possible.
  3. Bright green bars that are as few and as short as possible.
  4. As little blue as possible.
  5. The “start render” and “document complete” vertical lines to occur as early as possible, and be as close together as possible.

*These charts were done by testing unoptimized and optimized versions of the Velocity 2010 website.
**Site optimization was done using Strangeloop’s cloud-based Site Optimizer service.

Related posts: