13 Oct 2011
After reading Pat Meenan’s excellent post on third-party SPOF (single points of failure), in which he used WebPagetest to simulate how your site will perform if one of its third-party providers goes down, I was inspired to try it out for myself. (I wasn’t the only one. Steve Souders wrote this great post today, as well.)
After running my tests, I revisited some research we did here at Strangeloop into third-party scripts and how they’re used by 200 top ecommerce sites. Some very interesting stuff came up, which I’ll get into later in this post.
I wanted to see how vulnerable the top five ecommerce sites (according to Internet Retailer) are to third-party SPOF. I did the exact test that Patrick did, but decided to blackhole all third-party domains except the CDN domain (i.e. if a site was using Akamai or Level 3 for small objects, I did not blackhole that domain).
I generated a series of side-by-side videos for each site, so you can get a vivid sense of the impact of third-party failure on page load. I’ve also included the waterfalls so you can see what the major culprits are. (You can see the scripts I used by clicking on the waterfall for each test and clicking “strip”.)
- Site still performs
- Site looks fine, except for the fact that the ads don’t come in.
- When Omniture dies (see line 18 of the waterfall), the entire site stalls for almost 30 seconds.
- This site is entirely reliant on Omniture.
- Caveat: Right now, Apple is still running its Steve Jobs memorial home page. Practically zero content, meaning the page is incredibly small.
- Very few third-party calls.
- Nothing blocks. The entire internet could go down and this site would still work.
- As with Staples, Omniture blocks the page request (see line 11 of the waterfall), this time for almost 20 seconds.
- This waterfall is the worst of the bunch.
- The same behaviour happens when testing this page with HTTPWatch in IE9, Firefox 7, and Chrome 16.
How are top ecommerce sites using third-party scripts?
A few months ago, I wanted a to get a sense of how ecommerce sites were implementing third-party scripts. We did an audit of the top 200 Internet Retailer sites to see who is using what. Here’s some of what we found.
Average number of third-party scripts
|Average # of 3rd-party scripts|
|Top 200 sites
|Top 20 sites
6.7 actually isn’t that bad, but some sites use many more than that…
Top sites, in terms of the number of third-party scripts used
|Site||# of 3rd-party scripts|
|The Sports Authority
When you look at the sheer volume of widgets and third-party tools out there, the numbers above are not too surprising. This next table represents just the tip of the iceberg…
Most-used third-party scripts
|3rd-party script provider
||Appearance in top 200 sites
|Google AdWords Conversion
Many people would guess Facebook because of its visibility, but this is a good reminder that “invisible” scripts are actually much more widely used than obvious content like social buttons.
Omniture doesn’t come off well, obviously. In 2 out of 5 of the sites I tested up top, it was clear that if Omniture goes down, the site goes down. And as our survey shows, almost half of the top 200 sites use Omniture.
The odds that all your third-party scripts are going to fail simultaneously? Pretty close to nil. The odds that some of them going to fail sometimes? Pretty much guaranteed. You need to ask yourself these questions:
- What do you know about the third-party scripts on your site? Do you know how many scripts your site contains. Do you know who all of your providers are?
- Is your third-party content optimized? You can’t optimize the script itself, obviously, but you can make sure that it’s implemented well in your pages. (Here are some good tips and how-tos.)
- Are all those scripts adding value? And is that value significant enough to outweigh any performance losses? (See my blog post and accompanying webinar to figure out how to calculate this.)
- What are your SLAs with your third-party providers? Do you even have SLAs with your third-party providers? (Not to turn this into a product shill, but third-party SLAs are a feature in our new Mobile Site Optimizer. If you want to learn how this feature works, read more here.)