Research

Speed tests: How fast are China’s top websites?

28 Oct 2011

2 Comments

A couple of months back, Strangeloop released a report about the load times of North American and European sites in China — arguably the fastest growing e-commerce sector in the world. We found that the average load time for these sites was just over 16 seconds.

In my post about the report, Steve Souders asked a great question: To make the case that luxury sites need to be faster, it would be great if you could compare their load time (16.2 seconds) with the load time of popular Chinese sites.

Steve’s question has been percolating on a back burner (I have a lot of back burners), and I was reminded of it when looking at Mary Meeker’s slides from her presentation on internet trends at the recent Web 2.0 Summit.

This slide in particular jumped out:

Of the top 25 websites, six are based in China. While none of these sites come even close to the top four U.S. sites in terms of profitability and valuation, it’s important to remember that this is an online marketplace that is still in its babyhood. By 2015, China’s e-commerce base is expected to increase by almost 400%. It’ll be interesting to see what this table looks like then.

But getting back to Steve’s question, I thought this would be a good time to run some tests and see how China’s top sites compare to the sites we tested in our study. (I used the same parameters: Testing via WebPagetest‘s location in Jiangsu, China, using Internet Explorer 7.)

Page load times of leading Chinese websites

Website Load time (seconds)
 CDN detected?
Sina.com 20.818 No
Ctrip.com 2.554 Yes
Alibaba.com 4.877 No
163.com 21.139 No
Tencent.com 27.468 No
Baidu.com 1.706 No
Average Chinese site
 13.102 17%
Average NA/EU site
 16.2
 29%

A few thoughts

  • This is a tiny sample size, of course, but it’s still an interesting snapshot of how China’s top websites fare when it comes to performance.
  • There’s a clear performance divide between verticals, and these correlate closely to the same performance divide I’ve observed in western-based sites. The news sites — Sina and 163.com — experienced 20+ second load times. The e-commerce sites — Ctrip (travel) and Alibaba (retail) — were much faster, with sub-5-second load times. The search site Baidu was the fastest of the bunch, while, ironically, the internet service provider Tencent was the worst. (I suspect that competition isn’t a concern for them.)
  • Only one site, Ctrip, uses a content delivery network.
  • And returning again to Steve’s question, while the overall average load time of just over 13 seconds wasn’t much lower than the 16 seconds we recorded for western sites, it should be noted again that the top e-commerce sites did load in under 5 seconds. From this, we can deduce that leading e-tailers in China are taking performance seriously — which means that western e-tailers need to do the same in order not to get left behind in the global marketplace.

Related posts:

By: Tammy Everts Category: Tags: , , , ,

The 25 best new web performance links of Q3

5 Oct 2011

6 Comments

A few people have asked me why I no longer do these monthly link roundups, and I didn’t have a good answer. (Not-good answer: I got busy.) But there have been so many great reads over the past three months that it feels like they really deserve the spotlight.

Not surprisingly, mobile is an even bigger deal than ever. There have been some great new presentations and studies. I was surprised, when digging through my bookmarks, to note just how many case studies there are, including under-the-hood reports from companies like Twitter and Facebook. To me, this is a really inspiring indicator of how much openness and excitement there is in our industry.

We’ve added these to our WPO Hub, which I encourage you to check out. It contains hundreds of links — organized by topic, source, research type, and industry — to the best performance-related content on the web.

Mobile

Mobile HTML5
For hardcore HTML5 and/or mobile geeks, this is an awesome table, created by mobile performance guru Maximiliano Firtman, which illustrates HTML5 compatibility across major mobile and tablet browsers.

Steve Souders: High Performance Mobile
A lot of people in our community (including myself) were kicking themselves for not being able to attend Steve’s talk at the San Francisco/Silicon Valley Web Performance Meetup. Next best thing: watching the video of his talk.

Mobile WPO
In June, I had the privilege of being a speaker at the Web Performance Summit. Fellow speaker Tim Kadlec gave a fantastic overview of the current mobile state of the union at the Web Performance Summit in June, which is a must-see.

Measuring Mobile Performance
Another great presentation, this one from front-end performance consultant Stephen Thair’s presentation to the London Web Performance Meetup Group. It’s packed with useful tips and how-tos.

What Mobile Users Want
Gomez revisited their two-year-old survey of mobile user expectations with this excellent report, which compares new data with those earlier benchmarks. Gomez found that mobile users are even more impatient than ever. 74% say they will abandon a site after waiting 5 seconds or less for it to load, up from 20% just two years ago. (If you want to see another take on this data, I created this set of graphs showing just how dramatic these changes are.)

Mobile website optimization now factors into mobile search ads quality
Google’s official announcement that mobile-optimized sites will factor into landing page quality and perform better in AdWords. This didn’t get a ton of media attention, but I think it should have. This algorithm change has implications beyond just AdWords, and I wrote about this here.

Tools

The Complete List of End User Experience Monitoring Tools
This is helpful list of tools for RUM and other user monitoring is a work in progress. Send your suggestions for additions to the folks at CorrelSense.

Yahoo! YSlow (Mobile)
YSlow for Mobile is now available as a bookmarklet. Users can run the equivalent of regular YSlow in Mobile browsers as well as any bookmarklet-enabled desktop browsers. [Note that this is a beta version.]

How-tos, case studies, and other research

Social button BFFs
Good post from Stoyan Stefanov on how to make your social buttons load asynchronously.

How-To: Optimize Social Plugin Performance
Facebook developers share some best practices, such as asynchronous loading, that can improve the performance of social plugins on your website.

Twitter’s mobile web app delivers performance
In-depth look at how Twitter developed its mobile app to maximize speed and performance.

“And that is why you need to speed up your site!”
I’m biased. I like this case study from performance consultant André Scholten because it validates my theory that, by using browser type and connection speed as proxies, you can use Google Analytics to simply demonstrate how faster sites make more money. Personal bias aside, it’s a pretty nifty speed-revenue analysis.

How We Improved Page Speed By Cleaning CSS, HTML and Images
Front-end developer Lara Swanson walks through a detailed case study showing how Dyn tackled performance optimization on its site.

How case-sensitivity for ID and ClassName can kill your page load time
Interesting findings from Andreas Grabner, showing how case-sensitivity causes a huge execution time difference in Internet Explorer 7, Internet Explorer 8, and Firefox 6.

Creating a Performance SLA with your customers – Betfair’s Customer Charter
Stephen Thair analyzes Betfair’s performance commitment in its customer charter, and discusses what a performance SLA should contain. Some thought-provoking stuff here.

Google +1 Button Performance Review and Google Triples the Speed of the +1 button
After Aaron Peters did a thorough audit of the +1 button’s performance and found some issues that could slow down page load by up to 2 seconds, Google responded by making some fixes to make the button faster. What I like about this pair of links is that it illustrates the level of transparency and mutual respect in our industry.

New performance findings

HTTP Archive: nine months
Nine months after the launch of the archive, Steve Souders compares performance data from November 2010 and August 2015 and discovers increases in total transfer sizes, requests per page, redirects, and page errors.

Website performance drop threatens top retailers
Report from Site Confidence analyses average download speeds for the UK’s top e-tailer websites and found an increase from 12 seconds in the first quarter of the year to 12.5 seconds in quarter two. This is a pretty steep increase from the average 10 second load time from the 2010 holiday season. This slowdown comes at a time when ecommerce is hurting in the UK.

Browsers and connectivity

Introducing Amazon Silk
You already know about Amazon Silk, but I just wanted to mention that I really like the short explainer video on the official blog. As someone who’s been working on “how it works” videos for my own company’s products lately, let me tell you: it isn’t easy.

Browser Market Pollution: IE[x] is the new IE6
Google Chrome team member Paul Irish explains why developers may soon need to develop with 76 different browsers in mind.

Browser Speed Tests: Firefox 7, Chrome 14, Internet Explorer 9, and More
Lifehacker tested browsers across a variety of functions — from cold boot-up to memory use — and Opera emerged as the clear winner.

The way carriers manage networks can hurt phone performance
Among other things, CNN writer Amy Gahran says that ”One of the largest U.S. carriers (unnamed in the study, since the data was made anonymous for legal reasons) appears to be slowing its network speed by as much as 50%.”

Industry news

Israel Web Performance Meetup Group
It’s always exciting news when a performance meetup group springs up.  Say hi to the new group based in Herzeliyya, Israel.

Opinion pieces

Why I think automation is the future of #webperf
Stephen Thair costs out manual versus automated performance optimization. I’m posting this link here, so you can guess that I like his conclusions. :)

Did I miss any great new links? Do you have other suggestions for the Hub? Let me know!

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , , ,

Trendwatching on the HTTP Archive: Interesting findings about payload, social content, and core best practices

30 Sep 2011

3 Comments

Steve put up a great post about the HTTP Archive last month that I’ve been meaning to pile onto. As one of the archive’s financial supporters, Strangeloop is obviously a big fan and I’m always talking it up with our customers.  (I was on the phone last week, pimping our mobile product with one of my favourite analysts — another data geek — who didn’t know it existed. She was very interested when I pointed out the incredibly exciting database we are creating.)

A few trends jumped out at me when I compared the first run in November 2010 to the latest run September 15, 2011.

As Steve pointed out in his post, payload is going up… and up… and up.

When I dug into this, I focused my attention on the top 100 sites because these guys represent my customers and I am very familiar with them. I wasn’t surprised to see total payload going up by 26% in just under a year — a pretty amazing number when you think about it. Images grew by close to 30% and scripts by close to 26%. It is tough to make pages fast when they grow this quickly.

HTTP Archive: Page size changes - November 2010 to September 2011

When I see payload going up, my first instinct is to blame the unconverted — the big guys who just don’t get it yet. To test my assumption, I took a look at the players who do really get it. I was surprised by my findings:

HTTP Archive: growth in payload for Amazon, Google, Gmail, Microsoft & Yahoo

On the one hand, Amazon decreased total payload by almost 15%. But they were the only site that showed improvement. Every other major player I checked increased their total payload: Google by 34.5%, Gmail by 25%, Yahoo by 18%, and Microsoft by 30%.

Not surprisingly, the number of requests increased across the board as well:

HTTP Archive: Increase in number of requests for Amazon, Google, Gmail, Microsoft & Yahoo

Overall, I was really surprised to see the big guys not practicing what they preach.

Social content is growing, and Google+ is neck-and-neck with Facebook.

I was also surprised at the growth in social on the top 100 sites. I was most surprised by the growth in Google+ and the fact that it is equal to Facebook. See below:

Popularity of JavaScript libraries in 2010:

HTTP Archive: Popular Javascript libraries (2010)

Popularity of JavaScript libraries in 2011:

HTTP Archive: Popular Javascript libraries (2010)Twitter has pulled ahead, from 2% to 8%. Facebook has grown from 2% to 5%. And right out of the blocks, Google+ has surged to a tie with Facebook. Some people say Google+ is a flash in the pan, others say it’s a serious contender. I’ll be very interested in seeing where these numbers are at next year.

1 out of 4 of the top 100 sites still don’t use cache headers.

HTTP Archive: 2010-2011: Zero change in use of cache-control headers

This is a core best practice, but about 1 out of 4 of the top 100 sites still don’t use it. This is a humbling reminder that, despite the great strides front-end optimization has made in the past couple of years, we can’t assume everyone is on the same page.

Correlations to render time and load time have inverted.

HTTP Archive: Changes in highest correlation to render time - 2010-2011HTTP Archive: Changes in highest correlation to load time - 2010-2011

Both of these sets of graphs intrigued me. It’s interesting to see the decrease across the board in all of the items as a contributor to render time. At the same time, we see an increase in correlation to load time. The fact that these two graphs seem inverted makes me wonder if there’s a connection between them.

I asked Hooman Beheshti, our VP of Product, about this, and here are his thoughts:

Round trips correlate to load time a lot more this year, and are in front. With all the 3rd party and social networking tags, this matches what we see with our customers. Round trips continue to be a massive contributor to load time, maybe now more than ever.

Transfer size may be second, which may fool us into thinking we’re getting things from point A to point B faster, but their impact on total load time has gone up. So, it may not have as big an impact as roundtrips, but it matters more now than it did before.

The fact that domains used on a page is a new big-boy contributor to load time (and leads the charge now in render time) may point to the fact that, collectively, we may not be doing as well as we thought with modern browsers and parallelism. And by that, I don’t mean concurrent connections to the same domain – just concurrent connections, period. Either that, or the domains-per-page is increasing (by 30% according to this, and by 20%+ for the top 100) and so is its impact on performance. Third-party tags could also be a contributor to this.

That’s all I can think of. I don’t have general theories on why the numbers are bigger for one and smaller for the other. It’s interesting, though, that the trend for render and load times themselves is not a part of the comparison and analysis. It would be interesting to see if these metrics are going up or down on average.

I had a blast digging into the HTTP Archive, and I strongly encourage you to do the same, if you haven’t already. And if you have any theories about my findings, or findings of your own, I’d love to hear them.

Related posts:

By: Tammy Everts Category:

The average luxury site takes 16.2s to load in urban China. Here’s why this should matter to you.

2 Aug 2011

7 Comments

One of the biggest differences between Chinese consumers and the rest of us is this: they’re spending, and we’re not. In 2009, when the global recession was at an all-time low, China was one of the few places where spending actually increased. The sale of luxury goods alone grew by 16%.

Infographic: Chinese consumers are going online in unprecedented numbers

What’s more, people in China are going online at a faster rate every year. This — coupled with the knowledge that international luxury brands are hot-ticket items in mainland China — made it clear to us at Strangeloop that if we wanted to get some understanding of what kind of user experience Chinese consumers are getting from western sites, it would be useful to start by looking at the web performance of leading luxury brands.

The results are available in a report we’ve just released: Why Luxury Websites Are Disappointing Chinese Consumers (And Why This Matters to You).

What we found: Sites are way too slow

Among other things, we found that only 36% of sites offered a Chinese-language version, and only 3 out of 10 had a China-specific URL. But our main finding was that the average site took 16.2 seconds to load, with some taking much longer — up to 58 seconds.

Infographic: Luxury brand website performance

What makes this finding even more compelling is how it bangs up against a recent stat from Gomez, which found that internet users in China are some of the most demanding in the world: 75% say that they will abandon a mobile site after waiting just 3 seconds for it to load.

But at the same time, up to 60% of online shoppers in China reportedly still use Internet Explorer 6, which basically sucks. So there’s a huge gap between user expectations and the capabilities of the platform they’re browsing on. This gap presents a formidable challenge for site owners.

Infographic: Web browser usage in China

We also learned this: Your CDN won’t necessarily solve your problems

Much like the Alexa retail report we released last year, we found that CDN use didn’t correlate to page speed. Only 29% of the sites we tested used a content delivery network. You might think this explains why sites performed so badly overall, until you look at the five fastest sites. They all loaded in 3 seconds or less, and none of them used a CDN.

Why does the performance of luxury sites matter to all site owners?

By 2015, there will be 520 million online shoppers in China, compared to just 200 million in the U.S. There are huge opportunities for companies to differentiate themselves early by appealing to Chinese consumers. Site owners who get in early, deliver a premium online experience, and grab this market as it’s just beginning to explode, are in an excellent position to emerge as leaders.

Infographic: Growth of the Chinese ecommerce user base

(If you like the infographics in this post, you can go here to see a high-res version of the complete set of graphics we created to accompany the report.)

Related posts:

By: Tammy Everts Category: Tags: , , , , , , ,

Mobile performance: Don’t panic, but it’s worse than we thought.

22 Jun 2011

7 Comments

Earlier today, I was part of Neustar’s online panel about mobile performance. We all had a great time bandying around our favourite stats and research, and when it was over I was inspired to do a little more digging. I’ve found a couple of things that have kind of blown my mind, which I want to add to the dialogue now:

  • In just a few months, the load time for the average mobile retail website has almost doubled, from 5.47 seconds to almost 10 seconds. (I’m being a bit disingenuous here. More on this below.)
  • Mobile user expectations have grown radically. In 2009, 58% of mobile users expected a site to load as fast on their mobile device as on their desktop. In 2011, that number has shot up to 85%.

Mobile sites are even slower than we thought.

A while back, I wrote about indicators that mobile sites seem to be getting slower rather than faster. I cited research from Keynote and Gomez illustrating that the average average m-commerce site (according to their respective indices) loads in 5.47 seconds, up from 4.73 seconds the previous year. As it turns out, these estimates were way off the mark.

Here at Strangeloop, we’ve been tracking Keynote’s mobile e-commerce index since 2009, and we recently noted this dramatic change in load time:

Growth in page load time

At the end of March, the index stood at around 5.3 seconds. The latest index, for June 19, is 9.86 seconds.

It didn’t take long to figure out the cause. A couple of months ago, Keynote quietly announced that they were doubling their mobile index list and changing their performance monitoring methodology. Before, they focused solely on measuring mobile domains (i.e. m.URL.com). Now, according to Keynote’s mobile performance evangelist, Herman Ng:

“We now capture additional page load time as a result of URL redirection. Best practices would call for no more than two URL redirects. Each URL redirection requires an HTTP round-trip request between the mobile device and the retailer web server, all of which adds extra time to load the page. Retail sites with three or more redirects should definitely fine-tune the page setup to reduce unnecessary page load latency. Whenever possible retailers should make sure they use the smallest number of URL redirections, as this is an important area for optimizing performance.”

This is a promising step forward, and it reflects what many of us probably felt in our guts: that the 2-4 second load times being reported for leading sites were just not believable. If you look at Keynote’s latest index results, for June 19, you can see that the fastest site, Dell.com, loads in a much more realistic 5.18 seconds. The slowest site is HSN.com, at an equally realistic 18.18 seconds.

What’s very important to note here: It’s easy to dismiss this jump as being solely due to the change in methodology. But if you isolate just the past two months on the graph, you can see that load time is still trending upward. Between April 17 and June 19, the index has increased by 1.63 seconds. In other words, in just two months the load time of the average mobile site in the index has ballooned by 20%. There’s more going on here than just a change in methodology.

But wait, there’s more: Mobile users are more demanding than ever.

In 2009, Equation Research announced that 58% of mobile users expect a site to load at least as quickly on their device as it does on their desktop.  Users’ expectations have grown hugely since then. In a report that came out in March of this year, Tealeaf stated that:

  • 47% of consumers who have conducted a mobile transaction in the past year expect the experience on their phones to be better than the experience in-store.
  • 80% expect the experience to be better than or equal to in-store.
  • 85% expect the experience to be better than or equal to online using a laptop or desktop computer.

In conclusion: Don’t panic. (Okay, maybe panic a little bit.)

I’m taking the glass-is-half-full approach to these findings. Now that we have more accurate numbers to work with, it’ll be that much easier to make site owners aware of the urgency of fixing their mobile performance issues.

Related posts:

By: Tammy Everts Category: Tags: , , , ,
« 3 4 5 6 7 8 9 10 »