Browsers

New findings: Ecommerce sites are 9% slower than in 2011

18 Sep 2012

12 Comments

Two years ago, when Strangeloop started tracking the load times of 2,000 top North American ecommerce sites, we had a hunch we’d spot some interesting trends over time. We did not expect, however, to see that pages are continuing to get slower rather than faster. Yet according to the Fall 2012 release of our quarterly Ecommerce Page Speed and Web Performance State of the Union, which came out today, that’s exactly what’s happening.

Not only are pages slower, they’re dramatically slower. Since November 2011, when we last tested these sites, the median home page has taken a 9% performance hit, with load time increasing from 5.94 seconds to 6.5 seconds. This flies in the face of conventional belief that, thanks to faster browsers, networks, and devices, the average end user is enjoying a premium online experience. This is clearly not the case. As user expectations continue to grow, the gap between expectations and reality continues to widen.

Ecommerce Page Speed and Web Performance State of the Union [Fall 2012]

These graphics (higher res version here) illustrate a few of our  findings. I encourage you to download the report to read the rest. Without giving it all away, here are a few highlights:

  • Internet Explorer 10 served pages faster than other browsers, most notably 8% faster than Chrome 20. We tested each page across a number of browsers, including the latest versions of IE, Chrome, and Firefox. It’s important to bear in mind that these were simple tests that didn’t take into account the many nuances of browser performance (which is discussed further in the report), but we considered these results interesting enough to share.
  • Top sites are 10% slower than the pack as a whole. While the median site took 6.5 seconds to load, we saw even poorer results when we looked at the top 100 sites (ranked by revenue and profitability), with the median Alexa 100 home page having a load time of 7.14 seconds. Check out the report for our thoughts on this.
  • Many sites are still not following core performance best practices. We found that 30% of sites tested did not use compression, and 12% did not use keep-alives. As I’ve talked about elsewhere, these two fairly simple techniques can yield big results, including up to 52% improvement in start render time.

Why you should care about these findings

To my knowledge, Strangeloop’s state of the union reports (which we’re now releasing on a quarterly basis) are the only ongoing surveys that measure performance from the perspective of real users. By using WebPagetest, we can simulate performance across browsers and realistic latencies, and get a real-world look at how websites actually behave. It’s easy for site owners to fall into the trap of thinking that their sites are fast for everyone, because site owners are typically seeing benchmark tests run out of datacenters.

I want to emphasize that reports like this one are not a substitute for the real user monitoring you should be performing on your site on an ongoing basis. Instead, consider it a snapshot that we can collectively hold up as a mirror of big-picture ecommerce performance.

As always, I welcome your feedback and questions.

Download the report: State of the Union: Ecommerce Page Speed and Website Performance [Fall 2012]

Download a high-res version of the infographics above (and feel free to re-post): Poster: Ecommerce Page Speed and Website Performance [Fall 2012]

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , , , , , ,

Speed vs. security: Four ways that security solutions can cause performance problems (and what you can do about it)

14 Aug 2012

3 Comments

Security-related performance problems can happen in a few key areas — some of which you have control over, and some of which you don’t. While each of these areas by itself may not amount to much, the cumulative performance penalty can be significant. And while you may not be able to make performance-improving changes to your (or your visitors’) security features, this makes a compelling argument for why you need to optimize all the other areas that are within your control.

Here are four key ways that security measures can hurt page load:

1. TLS/SSL

Transport Layer Security (TLS) — and its predecessor, Secure Sockets Layer (SSL) — are protocols for encrypting information over the Internet. You know you’re on a site that uses TLS/SSL when you see “https” as the URL prefix.

HTTPS requires an initial “handshake” to create a connection between the browser and the server, and this handshake can be be very slow. This diagram (courtesy of IBM) gives a good idea of how the SSL handshake works:

SSL handshake

The problem

It takes at least four TCP roundtrips just to open a single SSL connection between the client and the server — and this doesn’t happen until after the initial TCP connection has been set up. The amount of data transferred as part of the handshake isn’t huge (under 5 kB typically), but for very small requests this can be quite a bit of overhead.

Important to know: The SSL handshake is actually even more process intensive than the actual data encryption that happens over the connection after the handshake is successfully complete.

How to know if this problem affects you

No one can give you a meaningful answer to this question without some information about the nature of your web site, hardware, software, and network configuration.

There are a couple of ways to look into this problem for your own site:

1. Profile the performance of your web server. There are several tools out there (such as JMeter and Visual Studio) to compare the performance of an HTTP vs HTTPS server.

2. Run WebPagetests of key secure pages on your site. Look at the purple bars on the waterfall chart that indicate SSL negotiation. You don’t want to see:

  • too many purple bars
  • long purple bars
  • purple bars associated with non-essential page resources

Here are a couple of examples showing how you don’t want your waterfalls to look:

SSL: bad waterfall 1

SSL: bad waterfall 2

Solution 1: Fewer handshakes.

Re-use TCP connections. Take extra care when it comes to HTTP keep-alives, in order to create longer-lasting SSL connections.

Solution 2: Offload SSL processing to another device.

You can mitigate the performance concern by offloading the SSL processing to another device, such as a dedicated load balancer or firewall. There is also dedicated hardware for handling SSL, which not only unburdens your servers from dealing with it, but also speeds up the SSL handshakes and encryption, overall.

Solution 3: Make sure you’re not using SSL content on pages that don’t need it.

Some third-party tags, particularly those for website monitoring products (i.e., Alertsite and Verisign), are secure by default, so they add SSL load to every page they appear on — even pages that are otherwise non-secure. Here’s how this looks on a waterfall:

SSL bad waterfall 3

 

Using unnecessarily secure tags is a simple enough mistake to make. Site devs will include the default tag in page templates, so that it’s automatically used on every page of a site, even the non-SSL pages. There are non-SSL versions of these tags. Ideally, devs should use the SSL version on secure pages, and the non-SSL version on the rest of their pages.

2. Security products (e.g., firewalls, IDS/Intrusion Detection Systems, IPS/Intrusion Protection Systems, WAF/Web App Firewalls)

Security products track traffic behaviour at a granular level in order to prevent attacks. They can be used either offline or inline. Products that function offline obviously don’t affect performance, but inline products — also known as “bumps in the wire” — look inside pages in realtime, watching at a much deeper level than load balancers/ADCs.

The problem

It’s difficult to quantify the exact performance penalty caused by security products, but because they’re performing deep packet inspection, we know that they incur extra latency at the packet level.

Solution 1: Ask your security product vendor for details.

Talk to your vendor about the performance impact of their products, and ask them for benchmark tests.

Solution 2: Perform your own A/B tests on your security products.

To test performance impact for yourself, you could set up split tests with the products turned on and off.

3. Browser extensions

As internet users become more savvy about security and privacy, there’s a growing market for plugins that add security support at the browser level. (Here are some industry-recommended plugins for Chrome, Firefox, Internet Explorer, and Safari.) Some of these addons govern your passwords or block ads, which are pretty innocuous from a performance perspective. Others, however, can cause a noticeably performance hit.

The problem

This entire post was inspired by my experience with installing HTTPS Everywhere, an extension for Firefox and Chrome that automatically identifies when you’re visiting a site that supports HTTPS and then serves you a secure version of the page, following the protocol described above in the section on TLS/SSL. After I installed HTTPS Everywhere, I perceived a noticeable slowdown in many pages I visit regularly — most of which don’t really need to be secure, in my opinion.

Some might disagree with this. Google’s motto — “Secure the world” — is a great goal, but right now, given the fact that the average page still takes 8 seconds to load, I can’t rationalize the additional performance hit created by universal HTTPS.

Browser extensions can have other performance implications, ranging from reportedly slower load times due to redirects (KB SSL Enforcer) to preventing pages from working properly (NotScripts and NoScript).

The solution

As a site owner, you have no control over what your users are doing at the browser level. You should, however, have the foresight to anticipate what they’re doing. I recommend that site owners test drive leading security extensions and measure their performance impact from the perspective of a real user

4. Desktop antivirus software

Antivirus software scans incoming files to identify and eliminate computer viruses and other malware such as adware, spyware, trojan horses, etc. It does this by analyzing all files coming through your browser. Like firewalls and security products, antivirus software operates in realtime, meaning that files are paused for inspection before being permitted to download.

The problem

As with security hardware, it’s hard to quantify the performance impact of antivirus software, but a certain penalty is inevitable, depending on the software and on the composition of the page/resource being rendered in the browser.

A few years ago there was an issue with some antivirus software breaking a fundamental performance best practice called compression. This practice involves using gzip to compress content so there are fewer bytes to send. Fewer bytes equal faster pages. The antivirus software ripped out the gzip header in the requests that would have indicated to the site that the browser could compress content. As a result, sites wouldn’t compress content, even if the browser supported compression. So sites kept sending uncompressed content to the browsers. (Steve Souders wrote about this issue here and here.)

The solution

As with browser extensions, you have no control over potential problems caused by AV software if you’re a site owner. But that doesn’t mean you shouldn’t arm yourself with the knowledge that problems are occurring.

One way to do this is by examining your server logs – where you can see if a browser that you know can compress (pretty much any modern browser) showed up without the Accept-Encoding header that would’ve indicated that it supports compression. This will at least tell you if you may have a problem.

Takeaway

In case it needs to be stated, I am emphatically not advocating that people sacrifice security for the sake of speed. That would be ridiculous. Security is an increasingly critical issue, and site owners and internet users need to protect themselves online. The question is: how to protect yourself while reducing the performance penalty caused by your security measures.

While there are some areas, outlined above, in which site owners can improve performance without compromising security, in other areas we have to accept certain performance hits. The only solution then is to focus our efforts on optimizing performance elsewhere.

Related posts:

By: Tammy Everts Category: Tags: , , , ,

New findings: How does browser usage vary throughout the day and week?

8 Aug 2012

4 Comments

Last month, I questioned the validity of using Internet Explorer 8 as the default browser for performance testing. In the comments on that post, Stephen Pierzchala raised a good point:

I would be really interested to see a little more granularity in both the Akamai IO and Strangeloop data that shows browser usage by time of day and day of week. While it doesn’t change the aggregate results, it may indicate the type of browser that people choose to use, rather than the browser they are told to use.

We all tend to have semi-educated guesses about the browsers that people choose to use at work/school versus those they use at home, but I wasn’t able to find any studies that answer Stephen’s question. So I took a look through our beacon data to see if I could spot any trends. I got some interesting results that confirmed a few of my assumptions.

Approach

  1. Gathered data for one month of traffic for two large North America-based ecommerce sites.
  2. Pared traffic down to four browsers: Chrome, Firefox, Internet Explorer, and Safari.
  3. Aggregated data for each day of the week and plotted this on a graph for each site.
  4. Aggregated data for each hour of the day and plotted this on a graph for each site.

The results were interesting.

Observations

Browser usage throughout the week

As you can see on this pair of graphs:

  • Internet Explorer was the dominant browser for these sites.
  • Traffic for every browser, with the exception of Safari, either flatlined or decreased over the weekend.
  • Safari traffic spiked dramatically over the weekend.
  • Friday is a big browsing/shopping day across all browsers.

Browser usage throughout the day

These graphs bear a striking similarity to the daily breakdown graphs, in that IE, Firefox, and Chrome use begins to trend downward at the end of the workday, just when Safari use spikes.

Conclusions (aka “more educated guesses”)

While this data represents only two websites, I think there are enough similarities to make these findings a compelling argument for a couple of statements:

  1. Internet Explorer should still be the go-to browser for testing ecommerce sites in North America. The primary reason for this, in my opinion, is that many people are browsing/shopping while at work.
  2. Safari is a serious contender as the browser of choice for home shoppers, particularly those in desirable demographics (in other words, the Apple crowd). This is more of a hunch than an educated guess, but my feeling is that this partially based on the massive recent growth of the iPad market. I say this because everyone I know who uses a MacBook uses Chrome or Firefox instead of the default safari browser, whereas everyone I know who owns an iPad uses Safari. (Like I said, it’s just a hunch.)

I definitely don’t consider this the last word on the subject. The browser wars are far from over, and there’s enough flux in the industry that it’s anyone’s guess as to what the next year might hold. If you know of any comparable studies or statistics, let me know.

Related posts:

By: Tammy Everts Category: Tags: , , , ,

The 33 best web performance links of Q2 2012

31 Jul 2012

4 Comments

For some reason, I thought that the past few months had been kind of quiet on the research front, so when I started this post, I thought it would be one of my shortest roundups yet. I was pleasantly surprised to watch it grow to become one of the longest!

There are some great case studies here, of both large and small sites, which I love to see. There’s also some truly excellent debate about responsive design and the mobile web, sparked by a post from Jakob Nielsen last spring, as well as some good stuff about the browser wars and third-party content. So enough with the intro. Let’s get into it.

Case studies

Optimizing Retr-O-Mat’s Web Performance

A casual performance optimizer details her efforts to get Retr-O-Mat’s average load times under 2 seconds. Good information for anyone interested in the nuts and bolts of front-end optimization (FEO).

Web performance can be beautiful too

After performing poorly in a 2011 web performance comparison of leading retailers, Crate and Barrel made WPO a priority moving forward. This blog post from Catchpoint shows just how “beautiful” their performance has been in 2012.

How the Post is improving site performance

Responding to a flood of user frustrations with their website, the IT team at the Washington Post rolled out a number of performance upgrades to their site over the past year. Find out what they did to improve their page speed by 32.4%.

Tips and how-tos

Building a faster web: Tools, tips, and lessons

If “faster connectivity and more bandwidth won’t save us,” then what will? Google’s Ilya Grigorik shares his insight on making the web faster in this in-depth slide deck, and he draws some very interesting conclusions.

How to Make Progress Bars Feel Faster to Users

The human perception of time is anything but linear, and with just minor visual tricks, it gets even more skewed. After reading this post, you may never trust a progress bar again. :)

The 3 white lies behind Instagram’s lightning speed

More cool perceptual tricks. The “secret sauce” behind Instagram’s stellar user experience is rooted in a combination of coding tricks aimed at giving users a feeling of constant responsiveness. Find out how their site “always pretends to work.”

Mobile

The web only works thanks to reload (and why the mobile web fails)

As Mike Belshe points out, web page resources routinely fail, but thanks to the ever-handy reload/refresh button, we can often solve these problems ourselves. With mobile browsing, however, the rules are different. Find out what this means for the future of HTML 5.

Web first for mobile

Performance evangelist Steve Souders focuses his performance research strictly on the mobile web – not on native apps. Why? He’s got more than a few good reasons.

A taste test of mobile website development

A solid webcast on the complex world of mobile development, touching on topics including Responsive Web Design (RWD), server-side device detection, and HTML5 performance on mobile.

Jakob Nielsen on mobile sites vs. full sites

Jakob Nielsen believes that mobile and full sites should be entirely different entities. Summarizing his argument, he states that “good mobile user experience requires a different design than what’s needed to satisfy desktop users. Two designs, two sites, and cross-linking to make it all work.”

Is Nielsen wrong on mobile? Arguments abound

From Net magazine: Jakob Nielsen’s assertion that “good mobile user experience requires a different design” is being challenged by a noted mobile expert, who argues that rather than stripping down for mobile, companies should be doing more.

Why we shouldn’t make separate mobile websites

More counterpoint to Nielsen’s post. Smashing Magazine’s Bruce Lawson argues that mobile redirection is unreliable, and excluding features for mobile browsers “perpetuates the digital divide.”

Responsive web design: Missing the point

Still more Nielsen backlash: Brad Frost states that, though mobile browsers are getting better at rendering full websites, creating adaptive sites for mobile users is essential to improving the user experience.

HTML5 features increase mobile usage by 28%

Interesting piece explaining how static pages needing an upgrade can vastly improve mobile user engagement through the addition of HTML5. The new release features interactive galleries, overlays, and expandable/collapsible boxes, driving up pageviews and decreasing bounce rates.

Tools

More ways to measure web performance with User Timings

Google Analytics has expanded its collection of Site Speed reports with a new feature called User Timings. The feature enables tracking of specific load times for discrete hits, images, and other user interactions.

New mod_spdy release supports Apache servers

More from Google. The latest version of mod_spdy – an Apache module that adds SPDY server support – is intended to fix bugs found in the original release.

“Speed Index” introduced as new performance metric

The Speed Index metric has been added to WebPagetest, helping measure the speed at which page contents are visually populated. The tool is especially useful for comparing page experience before and after optimization.

Browsers

Browser Speed Tests: Chrome 19, Firefox 13, Internet Explorer 9, and Opera 12

Lifehacker conducts it’s semi-regular browser speed tests, pitting the four titans of desktop browsing against each other in races for startup speed, tab loading times, and other performance indicators.

Which Browsers are the Fastest?

An interesting companion read to the Lifehacker piece, New Relic’s “Speed Wars” study shows that, while IE 9 speeds past other browsers on Windows, Chrome 13 on Mac was overall the fastest experience. In mobile speed tests, the fastest experience was delivered by Blackberry Opera Mini at 2.6 seconds, twice as fast as Safari 5.1 on iPad.

How the Chrome Predictor hides latency from users

Ilya Grigorik demonstrates how Google Chrome hides latency from users. Interesting stuff here.

Internet Explorer market share surges, as IE 9 wins hearts and minds

From Ars Tecnica: “The browser wars are back on in earnest. For the second time in three months, Internet Explorer made large gains, picking up almost 1 point of market share. Chrome, Firefox, and Safari all lost out, as Internet Explorer 9 won over new users.”

CDNs

A one-size-fits-all CDN solution isn’t always best

Server configurations come in all shapes and sizes, which means a one-size-fits-all CDN is seldom effective. Find out which Level 3 customer was the beneficiary of a custom CDN solution and how it worked out.

Third-party content

10 Golden Rules for 3rd Party Providers

Murphy’s Law reigned supreme throughout June, with a flood of large-scale outages taking down some of the world’s most popular websites. Given the inevitability of online failures, third-party providers must be prepared to deal with the worst. The folks at Catchpoint outline the 10 Golden Rules by which all third-party providers should live by.

The vendor who flunked the web performance test

Are third-party vendors ignorant to the consequences of slow web performance? According to Catchpoint they are, as they detail a story of one such vendor who was completely unaware of the performance impact of their product.

Average UK website has 14 trackers per page

Interesting findings from TRUSTe: Despite the prevalence of privacy policies, over two-thirds of trackers on UK websites originate from third-party companies, and almost half embed themselves permanently.

Google releases +1 button preview – loads 20% faster

Google announced that they’ve improved performance of the +1 button and Google+ badge. By reducing the size of the js/plusone.js loader and making the code smarter, page elements now load 20% faster.

Third-party JavaScript should be loaded asynchronously

Old news to some, but still worth mentioning: Third-party JavaScript should be loaded asynchronously, as it helps avoid slowdowns and can speed up page loads.

Third-party front-end performance, Act 1

Application provider Bazaarvoice is delving into the realm of front-end performance, and provides an interesting third-party perspective.

Opinions and analysis

Performance Nightmare: Nasdaq & the Facebook IPO

When Facebook began trading on May 18, 2012, a series of performance failures on Nasdaq.com caused a huge headache for the company. This article from Intechnica asks how much these badly timed hiccups cost investors.

More, better, faster: Steve Souders on WPO

Steve Souders kicked off O’Reilly’s Velocity video podcast series with an in-depth discussion of the state of web performance optimization. Key topics included measuring slowness, performance monitoring tools, and whether mobile disrupts performance.

Other research

How complex systems fail

As a complex and interdependent system, the web is prone to catastrophe at the highest levels. In this fascinating paper on resilience engineering, presented at Velocity 2012, Dr. Richard Cook outlines the reasons why all complex systems are intrinsically hazardous, why disaster is always just around the corner, and how failure-free operations still require experience with failure.

The growing epidemic of page bloat

I don’t usually pimp my own writing here, but this information is too important not to share. I wrote a piece for GigaOM showing that the average page size is now over 1MB, according to the HTTP Archive. At current growth rates, the average page could hit 2MB by 2015, which is a really big deal, especially for mobile users.

How fast are websites around the world?

Some fascinating findings here. Google’s Site Speed Reports provides detailed latency data for page load times by separating data according to device, location, and industry.

These links were all sourced from Strangeloop’s Web Performance Hub, which contains hundreds (and by now, possibly even thousands) of industry-wide links, organized by topic, source, research type, and industry. It’s a pretty good resource, if I do say so. If you have any new links to recommend, let me know.

http://t.co/EHbRdT6r
10 Golden Rules for 3rd Party Providers [article]
Catchpoint – June 26, 2012
Summary: Murphy’s Law reigned supreme throughout June, with a flood of large-scale outages taking down some of the world’s most popular websites. Given the inevitability of online failures, third-party providers must be prepared to deal with the worst. Find out the 10 Golden Rules by which all third-party providers should operate by.
http://t.co/LGYUpGn1
End-to-end optimization: Taking content delivery to the next level [blog post]
Web Performance Today – June 27, 2012
Summary: Strangeloop Networks is thrilled to announce the launch of our latest product, Network Accelerator. Learn all about how this product works, what it does – and most importantly – why it’s a major step forward for content delivery networks.
http://t.co/T4z69s7k
How complex systems fail [research paper]
CTALab.org – June 26, 2012
Summary: As a complex and interdependent system, the web is prone to failure and catastrophe at the highest levels. In this fascinating paper on resilience engineering, Dr. Richard Cook outlines the reasons why all complex systems are intrinsically hazardous, why catastrophe is always just around the corner, and how failure-free operations require experience with failure.
http://bit.ly/LzGPqN
Mobile optimization starts with mindset: Hooman Beheshti interviewed at Velocity 2012
O’Reilly Media – June 25, 2012
Summary: Where are we in the mobile optimization life-cycle? What mindset should site owners adopt when boosting mobile performance? Are performance measurements improving? In this video, Strangeloop Technology VP Hooman Beheshti offers his unique insight on the current state of mobile.
http://t.co/Vnced8tq
The 90-Minute Mobile Optimization Life Cycle [slides]
Strangeloop Networks – June 25, 2012
Summary: Strangeloop Technology VP Hooman Beheshti wowed attendees at this year’s Velocity Conference with a presentation on the mobile optimization life cycle. For those who missed it, be sure to check to check out these fascinating slides.
http://bit.ly/Nu1gCi
Ghosts of Velocities Past: 9 presentations that are still relevant today [blog post]
Web Performance Today – June 20, 2012
Summary: Velocity’s short (yet incredibly important) history is filled with memorable moments, and these 9 presentations from past conferences remain relevant today. Perhaps not trendsetting anymore, but certainly trend affirming, which may just be better.
http://bit.ly/MFuxMR
My recent post on SEOMoz: 13 Questions (and Answers) About Google, Site Speed, and SEO [article]
SEOmoz – June 18, 2012
Summary: In this article, Strangeloop president Joshua Bixby breaks down how site speed and performance metrics affect Google page ranks. For anyone who has ever wondered how Google manages to make performance metrics affect SEO, this article is a must-read.
http://mz.cm/M24fGc
Introducing: New Browser Tax feature for our ecommerce customers [blog post]
Web Performance Today – June 14, 2012
Summary: Ever wish you could arbitrarily tax your customer base for failing to stay current, with zero repercussions? With the new Strangeloop Browser tax, your wish is now a reality!
http://bit.ly/NC5Q65
Optimizing Retr-O-Mat’s Web Performance [blog post]
Finding Marbles – June 9, 2012
Summary: For the “WPO guy’s wife,” average load times just aren’t good enough. In this post, a blogger and casual performance optimizer details her efforts to get Retr-O-Mat’s average load times under 2 seconds. Great information for anyone interested in the nuts and bolts of WPO.
http://t.co/k0lXkniG
Browser Speed Tests: Chrome 19, Firefox 13, Internet Explorer 9, and Opera 12 [article]
Lifehacker – June 12, 2012
Summary: It’s a battle of startup times, tab loading times and other KPIs between the four titans of Windows browsing. Lifehacker’s speed tests are always entertaining for what they’re not afraid to say, and this article is no exception.
http://bit.ly/Nb2ibS
Marrying CDNs with front-end optimization for maximum acceleration [blog post]
Web Performance Today – June 12, 2012
Summary: Front-end optimization (FEO) has been weaving its way further into content delivery networks (CDNs) over the past two years, and the dynamic between these two technologies continues to evolve. In this video presentation, Strangeloop’s Joshua Bixby breaks down the benefits of combining these performance solutions.
http://bit.ly/L3aatz
How the Chrome Predictor hides latency from users [article]
Igvita.com – June 4, 2012
Summary: Google Chrome features countless tools for supercharging load times, but when those aren’t enough, the browser can hide latency from users. Find out how!
http://bit.ly/Nw6ZMh
Building a faster web: tools, tips, and lessons [slides]
Igvita.com – June 3, 2012
Summary: If “faster connectivity and more bandwidth won’t save us,” then what will? Google’s Ilya Grigorik shares his insight on making the web faster in this in-depth slide deck, and draws some very interesting conclusions.
http://bit.ly/L0eERH
The “performance poverty line”: What is it and why does it matter? [blog post]
Web Performance Today – June , 2012
Summary: The “performance poverty line” is the point at which business metrics have sunk so low, load times cease to matter. But how is this line measured? Does it differ between industries? And most importantly: is there hope?
http://bit.ly/NJAqs2
A one-size-fits-all CDN solution isn’t always best [article]
Level 3 – June , 2012
Summary: Server configurations come in all shapes and sizes, which means a one-size-fits-all CDN is seldom effective. Find out which Level 3 customer was the beneficiary of a custom CDN solution.
http://bit.ly/M9P5xt
Why the Facebook outage is (yet another) wakeup call for site owners [blog post]
Web Performance Today – June , 2012
Summary: The hazards of running third-party scripts are well documented, but the May 31st Facebook outage was another stern reminder. In this post, Strangeloop’s Joshua Bixby discusses all things third-party, including rogue content and common performance pitfalls caused by third-party content.
http://bit.ly/JQj7GX
The web only works thanks to reload (and why the mobile web fails) [article]
Belshe.com – June , 2012
Summary: Web page resources routinely fail, but thanks to the ever-handy reload/refresh button, we can often solve these problems ourselves. With mobile browsing, however, the rules are different. Find out what this means for the future of HTML 5.
http://bit.ly/NAAQ3O
How to Make Progress Bars Feel Faster to Users
UXMovement – June , 2012
Summary: The human perception of time is anything but linear, and with just minor visual tricks, it gets even more skewed. After reading this post, you’ll never trust a progress bar again!
http://bit.ly/NcsVfg
Does the average web user waste two days a year waiting for pages to load? [blog post]
Strangeloop Networks – June , 2012
Summary: It may not be true, but in web performance, perception is reality. Web users in the UK are less than pleased about their online experience, but just how cranky are they?
http://bit.ly/LSFIlv

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , , , , , , , , , , ,

Early findings: Mobile browser cache persistence and behaviour

12 Jul 2012

1 Comment

If you’ve spent any time in the mobile space, you know that the browser cache is a critical limiting factor for performance. I often talk about:

  • how unreliable the mobile browser cache is,
  • how poorly it persists data, and
  • how poorly it observes basic caching headers.

You may also know that one of the key benefits that front-end optimization (FEO) can bring to the mobile world is a more useful browser cache. Our team, led by Hooman Beheshti and Jarrod Connolly, has been working for years to perfect mobile acceleration, and they’ve spent a huge amount of time digging into the browser cache conundrum. Hooman recently presented some very interesting findings at Velocity, which I want to expand on today.

Background:

Much has been written about how poor mobile caching is. For a thorough overview, read more here, here, here, here, and — my favourite link — here. But for our purposes today, here’s the Reader’s Digest version:

1. Mobile cache sizes are far too small.

The stock browser in Android (2.x) has a cache size limit of around 5.7MB. iOS is much larger (>50MB).

2. When we think about mobile cache sizes, we need to take into account a bunch of variables.

We need to consider the total cache size (which is a shared cache for all our browsing), individual object size limits for caching, and the fact that HTML pages may be treated slightly differently.

But we had more questions about mobile caches, so Hooman set out to get some answers. Below is what he found.

Question 1: How do non-stock browsers cache on Android?

Further to this question, Hooman wanted to see if anything has changed with Android ICS (Ice Cream Sandwich, version 4.0 of Android). His methodology was pretty similar to what others have done to determine mobile cache sizes (see the links above).

Here’s what came out of it:

  • On Gingerbread (Android 2.3.x), Firefox and Opera both had cache sizes that were slightly larger than the stock browser. They both came out at around 10MB.
  • The stock ICS browser had an HTTP cache that was much bigger than its predecessor. Hooman’s tests showed a limit of 25MB.

He was also going to test Chrome on ICS (Chrome is not available for Gingerbread), but then found this great article from Tony Gentilcore from Google that outlines how the Chrome cache is large and persistent. Better yet, the cache size is dynamic and based on the amount of disk space available, which is great news.

To put things into perspective, it’s good to compare mobile cache sizes to those of desktop browsers, which are usually in the range of 100s of MB:

  • This article from IE tells us that in IE6/7/8 the IE cache size was 1/32 of the disk, with a default cap of 50MB. IE9 is 1/256 of disk size (not a bad thing since disks are much bigger these days) and has a default cap of 250MB. Cache size is user configurable in IE.
  • Spot checking our own Chrome and Firefox browsers showed a cache size of >300MB on Chrome and >700MB on Firefox.
  • Steve has a great post on cache sizes with some crowdsourced info on what’s out there now.

Conclusion: Mobile caches are lacking when it comes to how much content they can hold.

This is especially true when you consider that an HTTP object cache is a shared resource for a browser. If you do even minimal browsing on your smartphone, you’re bound to fill up the cache rather quickly. But things are getting better. ICS is a good example of that. And even though progress may not be occurring at the pace we’d like, any progress is good. This also highlights the value of using localStorage as an object cache, which mobile acceleration solutions like Strangeloop’s Mobile Optimizer automatically enable a site to do.

Question 2: How do mobile browser caches behave with various user actions (e.g., closing the browser, locking the phone, shutting down the phone)?

This is an extremely important question that’s never been satisfactorily answered. There is very little data on the persistence of the cache and how (and if) objects are fetched from cache. Here’s a breakdown of Hooman’s suite of experiments in this area.

Methodology

Considering the hundreds of permutations that something like this could end up with, we tried to keep things simple. We put an Android Nexus S (version 2.3.7) and an older iPhone 3GS (iOS version 5.1.1) through the following paces:

1. We used a single page with ~2MB of images on it to prime the cache.

2. In priming the cache, we used four different sets of cache control headers (including none at all) on the images to see their effect on caching:

  • No cache headers at all
  • A Last-Modified header only
  • A Cache-Control: max-age header, together with a Last-Modified header
  • A Cache-Control: max-age header only

3. Then, after priming the cache, we performed some user actions and revisited the page.

4. Through network traces, we determined if, when, and how the browser went back to the server to fetch any of the images.

Results

The table below shows all the permutations and how each phone behaved in various circumstances.

It’s not an easy table to read, so here’s what’s happening:

  • In all the green cells, all the images were fetched from the cache. No requests for those images went over the network to the server.
  • In all the orange cells, nothing was fetched from the cache. All the images were re-fetched from the server.
  • In the blue cells, the requests were made from the server, but with some relevant caching headers in the requests. If you want to know exactly what these headers mean, it’s best to consult RFC2616.
User action Device Primed with no cache headers Primed with Last-Modified Primed with Last-Modified and max-age Primed with max-age
New window or tab Android All from cache All from cache All from cache All from cache
iOS None from cache All from cache All from cache All from cache
Exit browser, lock phone, unlock phone, open browser Android All from cache All from cache All from cache All from cache
iOS None from cache All from cache All from cache All from cache
Kill browser, launch browser Android All from cache All from cache All from cache All from cache
iOS None from cache All from cache All from cache All from cache
Reboot phone Android All from cache All from cache All from cache All from cache
iOS None from cache All from cache All from cache All from cache
After page loads, click URL and hit “Enter” Android All from cache If-Modified-Since and Cache-Control:max-age=0 If-Modified-Since and Cache-Control:max-age=0 All from cache
iOS None from cache All from cache All from cache All from cache
Refresh after page loads Android Cache-Control:no-cache Cache-Control:no-cache Cache-Control:no-cache Cache-Control:no-cache
iOS Cache-Control:max-age=0 If-Modified-Since and Cache-Control:max-age=0 If-Modified-Since and Cache-Control:max-age=0 Cache-Control:max-age=0

Observations and conclusions

Most of the cells show behavior that is expected from a well-behaving browser. But there are a few areas Hooman highlighted as particularly interesting:

Primed with No Cache Headers

It’s interesting that Android was so aggressive and chose to cache the images even if they didn’t have caching headers on them. This is particularly aggressive.  But that’s not necessarily a bad thing, especially if this behavior is specific to images. But it makes it that much more likely that the cache will fill up quickly. iOS, by contrast, doesn’t cache any images if they don’t have any caching headers at all.

After page loads, click URL and hit “Enter”

This whole row was interesting because, to the Android, this is very much like a refresh of the page. For situations where the original images were put into the cache with the Last-Modified validator, the browser wants to validate all the way to the origin server. (Intermediate caches can’t validate on their own because of the max-age=0 directive.) This is true even if there was a max-age put on the image when it originally went into the cache.

Primed with Last-Modified

When the image went into the cache with just a Last-Modified validator, caching was rather aggressive form both Android and iOS. This is actually probably a good thing. It’s also possible that this behavior would be different if we had revisited the page much later than when the images originally went into the cache.

A few caveats

As Hooman points out, even though this is all very telling, it’s important to consider that we didn’t test the hundreds of permutations that could have included how full the cache was and combinations of different content types (images, scripts, CSS, HTML, etc). So these observations apply just to images. It’s possible that some of the behaviours could have been different if other file types were introduced into the mix.

Also, while some research out there suggests that the iOS cache isn’t persistent (it reportedly gets wiped after the device shuts down), we didn’t find this in our research.  However, the fact that we used a small amount of images could explain this discrepancy.

Final thoughts: Mobile browser vendors need to offer more visibility into their products

This is all great info and we’ll continue to test and get insight into how mobile browsers work. This is not only important for the performance community, but also vital to our own product development here at Strangeloop.

But we’d also like to re-iterate what others have said, which is to ask browser vendors to provide more visibility and documentation into what their browsers are doing (kudos to Chrome and Tony for starting to walk that path). As mobile browsing is poised to rival and overtake desktop browsing in the near future, the more we know, the better we can design our web applications and the products that help optimize them.

For a greater understanding of mobile performance challenges, I encourage you to download Strangeloop’s mobile optimization whitepaper (not a product shill, I promise).

Related posts:

By: Tammy Everts Category: Tags: , , , , ,
1 2 3 4 5 6