How vulnerable is your site to third-party failure?

Third-party scripts are the single most common point of failure for sites: just a single line of JavaScript can take down your entire site. Despite this, measuring the impact of third-party content on a site’s usability is often an afterthought — if it even gets thought about at all.

After reading Pat Meenan’s excellent post on third-party SPOF (single points of failure), in which he used WebPagetest to simulate how your site will perform if one of its third-party providers goes down, I was inspired to try it out for myself. (I wasn’t the only one. Steve Souders wrote this great post today, as well.)

After running my tests, I revisited some research we did here at Strangeloop into third-party scripts and how they’re used by 200 top ecommerce sites. Some very interesting stuff came up, which I’ll get into later in this post.

Methodology

I wanted to see how vulnerable the top five ecommerce sites (according to Internet Retailer) are to third-party SPOF. I did the exact test that Patrick did, but decided to blackhole all third-party domains except the CDN domain (i.e. if a site was using Akamai or Level 3 for small objects, I did not blackhole that domain).

I generated a series of side-by-side videos for each site, so you can get a vivid sense of the impact of third-party failure on page load. I’ve also included the waterfalls so you can see what the major culprits are. (You can see the scripts I used by clicking on the waterfall for each test and clicking “strip”.)

Amazon

Waterfalls: normal vs. broken

Observations:

• Site still performs
• Site looks fine, except for the fact that the ads don’t come in.

Staples

Waterfalls: normal vs. broken

Observations:

• When Omniture dies (see line 18 of the waterfall), the entire site stalls for almost 30 seconds.
• This site is entirely reliant on Omniture.

Apple

Waterfalls: normal vs. broken

Observations:

• Caveat: Right now, Apple is still running its Steve Jobs memorial home page. Practically zero content, meaning the page is incredibly small.
• Very few third-party calls.
• Nothing blocks. The entire internet could go down and this site would still work.

Dell

Waterfalls: normal vs. broken

Observations:

• As with Staples, Omniture blocks the page request (see line 11 of the waterfall), this time for almost 20 seconds.

Office Depot

Waterfalls: normal vs. broken

Observations:

• This waterfall is the worst of the bunch.
• The issue here seems to be one that Steve described in this 2010 blog post about SPOF. In this case the file is an external JavaScript file very near the top of the page, which is why the effect is so bad. The page is white until it times out trying to connect to the broken domain.
• The same behaviour happens when testing this page with HTTPWatch in IE9, Firefox 7, and Chrome 16.

How are top ecommerce sites using third-party scripts?

A few months ago, I wanted a to get a sense of how ecommerce sites were implementing third-party scripts. We did an audit of the top 200 Internet Retailer sites to see who is using what. Here’s some of what we found.

Average number of third-party scripts

6.7 actually isn’t that bad, but some sites use many more than that…

Top sites, in terms of the number of third-party scripts used

When you look at the sheer volume of widgets and third-party tools out there, the numbers above are not too surprising. This next table represents just the tip of the iceberg…

Most-used third-party scripts

Many people would guess Facebook because of its visibility, but this is a good reminder that “invisible” scripts are actually much more widely used than obvious content like social buttons.

Conclusions

Omniture doesn’t come off well, obviously. In 2 out of 5 of the sites I tested up top, it was clear that if Omniture goes down, the site goes down. And as our survey shows, almost half of the top 200 sites use Omniture.

But I don’t want to over-focus on Omniture. The key issue here is that site owners are implementing more and more third-party scripts, possibly improperly, and with little to no analysis of how these scripts affect their sites. It doesn’t matter how well you optimize the rest of your site if a single line of external JavaScript can take out the whole thing.

Takeaways

The odds that all your third-party scripts are going to fail simultaneously? Pretty close to nil. The odds that some of them going to fail sometimes? Pretty much guaranteed. You need to ask yourself these questions:

• What do you know about the third-party scripts on your site? Do you know how many scripts your site contains. Do you know who all of your providers are?
• Is your third-party content optimized? You can’t optimize the script itself, obviously, but you can make sure that it’s implemented well in your pages. (Here are some good tips and how-tos.)
• Are all those scripts adding value? And is that value significant enough to outweigh any performance losses? (See my blog post and accompanying webinar to figure out how to calculate this.)
• What are your SLAs with your third-party providers? Do you even have SLAs with your third-party providers? (Not to turn this into a product shill, but third-party SLAs are a feature in our new Mobile Site Optimizer. If you want to learn how this feature works, read more here.)

Related posts:

• The three biggest performance problems with third-party content (and how to fix them)
• Fourth-party calls: What you don’t know can hurt your site… and your visitors
• Do you really need that widget?