metrics

The 20 best web performance links of Q4

15 Dec 2011

1 Comment

Every time I write one of these posts, I’m impressed by the volume and quality of writing that happens in our industry on an ongoing basis. It’s truly an exciting time for web performance. I feel endlessly engaged by the dialogue that happens every day, and honoured to be part of it it.

This roundup (which includes links pulled from the Strangeloop WPO Hub), includes some increasingly refined thinking about mobile optimization, a handful of excellent tutorials and case studies (including some great new presentations from Velocity EU), and some revolutionary browser developments.

But my favourite link is this first one…

The best link of Q4

Retailers need for tech speed
Does it tell us anything new? No. But I’ve lost count of how many people I’ve forwarded this two-minute segment on CNBC’s “Power Lunch” — which discusses the importance of speed for e-commerce sites, particularly for mobile users, during the holiday shopping season. For me, this shows that site speed has finally jumped into the mainstream. I’m excited to see how this attention snowballs in 2012.

Mobile

Mobile UI Performance
This slide deck from Estelle Weyl’s excellent presentation at Velocity EU gives an overview of mobile performance challenges, why we need to address them differently than we deal with desktop sites, and detailed tips on how to do just that.

Performance Automation 101
This slide deck from Jeroen Tjepkema’s Velocity EU presentation explains what performance automation is, how it works, and why it’s the only viable solution for dealing with the challenges of mobile device/browser fragmentation.

HTML5 Techniques for Optimizing Mobile Performance
Great post on HTML5 Rocks: ”In this article, we will discuss the bare minimum of what it takes to create a mobile HTML5 web app. The main point is to unmask the hidden complexities which today’s mobile frameworks try to hide. You will see a minimalistic approach (using core HTML5 APIs) and basic fundamentals that will empower you to write your own framework or contribute to the one you currently use.”

Mobile Performance Manifesto
Love this post from David Calhoun itemizing and describing mobile performance best practices.

Tools

How WebPagetest works
If you’ve ever wondered how exactly WebPagetest gathers performance data from the various browsers it simulates, this is great post from Pat Meenan in which he cracks the hood of WebPagetest and explains all that.

Mobile Perf Bookmarklet
Steve Souders offers one mobile bookmarklet to rule them all: a new “master bookmarklet” that lets you install a handful of common debugger and profiler bookmarklets in your mobile broswer in one step.

Is Synthetic Monitoring Really Going to Die?
Alois Reitbauer asks: “Will User Experience Management using JavaScript agents eventually replace synthetic monitoring or will there be a coexistence of both approaches in the end?” As you might guess, the answer is not cut and dried.

Case studies, how-tos, and other research

Diagnosing Slow Web Servers with Time to First Byte
Much as it pains me to admit it, from time to time performance pains aren’t caused at the front end. Performance expert Andy King gives some good tips on how to use the time to first byte metric, as displayed on a waterfall chart, to help diagnose a slow server.

The art and craft of the async snippet
Stoyan Stefanov examines the topic of asynchronous code “from the perspective of a third party – when you’re the third party, providing a snippet for other developers to include on their pages, be it an ad, a plugin, widget, visits counter, analytics, or anything else.”

Why loading third party scripts async is not good enough
We talk about asynchronously loading third-party snippets as if that’s the sole cure for performance pains, but in this case study, Aaron Peters reminds us that sometimes it’s okay to defer their loading until after onload.

Fast Loading JavaScript
Slide deck from performance consultant Aaron Peters’ great Velocity EU presentation: “A walk-through of several JavaScript loading techniques with a characteristics table for each and at the end a decision tree to help you decide which technique to use.”

How Downtime Financially Impacts Top Ecommerce Websites
Compelling infographic showing how downtime affected the Internet Retailer 500 in 2010. Includes the estimate that downtime resulted in more than $300 million in lost revenue for the IR 500.

Testing for Frontend SPOF
Excellent post from Pat Meenan in which he simulates third-party outage with a blackhole server in order to demonstrate — via WebPagetest-generated video — how that outage slows down or disrupts page load.

Browsers and connectivity

SPDY of the Future Might Blow Your Mind Today
Great post (“definitely for protocol geeks”) by Google software engineer Mike Belshe on SPDY’s evolution and how Kindle Silk is taking it beyond other browsers.

Chrome Fast
Slides from Google software engineer Tony Gentilcore’s excellent presentation at Velocity EU, in which he gives an overview of the Chrome platform and explains what makes Chrome fast.

Report reveals drop between peak and off-peak surfing
No big surprise, but a good reminder (especially if you rely on synthetic testing) that real-world performance is a nebulous thing: UK study finds that web speed is up to 69% slower during evening peak time.

The end of an era: Internet Explorer drops below 50% of Web usage
Mark the month and year. November 2011 was the first time in more than a decade that Internet Explorer’s share of global browser usage dropped below 50%.

Opinion pieces

Your CDN is not a silver bullet for web performance
In the e-commerce and SaaS world, the two most common causes of poor web performance are third-party content and server-side processing. Neither of these bottlenecks are addressed by loading static content from a closer location via a content delivery network.

Why you have less than a second to deliver exceptional performance
dynaTrace’s Alois Reitbauer writes: “Being exceptionally fast is becoming the dogma for developing web applications. But what is exceptionally fast and how hard is it to build a top performing web site?” I like posts like this because they remind us what the fundamental questions are that our industry is trying to address.

If you have any other great links to share, let me know in the comments. And if you’re looking for more great links, we have hundreds — sorted by topic, industry, and type — over in the Strangeloop WPO Hub.

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , , , , , , ,

Case study: The impact of HTML delay on mobile business metrics

23 Nov 2011

19 Comments

When I gave my talk about mobile performance and business KPIs at Velocity Berlin a couple weeks back, one of the areas I got the most questions about later was the experiments we were able to do in which we delayed HTML on a customer’s site and tracked the results over a 12-week period. I thought it might be useful to break some of this out into its own post.

As I mentioned at Velocity, I was insanely jealous of Google and Bing a couple years ago, when they revealed their own in-house experiments with HTML delay. Most of us in the performance community would kill for that kind of experimentation platform. So I was extremely happy and grateful when one of our customers at Strangeloop expressed an interest in figuring out the value of time for their business.

Methodology

We conducted a split test over the course of 12 weeks, in which we segmented mobile traffic into four groups: fully optimized, 200 ms delay, 500 ms delay, and 1000 ms delay. We monitored four metrics: bounce rate, conversion rate, cart size, and page views. We also monitored and analyzed user behavior for 6 weeks after the test ended, to gauge the long-term impact, if any, of slow performance even after users begin to receive an accelerated site.

Results

The results of the 200 ms delay weren’t significant, but the customer and I were both taken by the dramatic impact of the 500 ms and 1000 ms slowdown. Our customer was blown away that they were losing 3.5% percent of their conversions when their site was delayed by just one second on a mobile device. This was a major epiphany for them, and it’s already helped them change their business and how they view mobile.

I’m including this next graph to reinforce the connection between load time and user behavior.

Over the 12 weeks, you can see that, while the HTML delays were constant — 500 ms and 1000 ms — users’ reaction to these delays fluctuated. The drop in bounce rate ranged from around -2% and -12% for users who experienced a 1000 ms delay, and 0% and -6% for those who experienced a 500 ms delay. While the bounce rate may have varied, the one thing that was constant was the fact that the behavior trends are strongly linear for both groups, and the bounce rate for the 1000 ms group was consistently worse.

Finally, and most interestingly to me, we wanted to look beyond just the effect of delay in the timeframe of the experiment. We know that slower pages have an immediate impact on user behavior and customer satisfaction. We wanted to find out if there was any long-term impact on customer satisfaction.

So we looked at our traffic data for the 6 weeks immediately after the experiment — specifically at the behavior of returning visitors. As any site owner will tell you, repeat customers are the bread and butter of an e-commerce vendor. These are the people you need to keep happy. If you look at the graph below, you can see that, even after the experiment was over and the shoppers in the 500 ms and 1000 ms group started to be served the same accelerated site as the baseline group, they were significantly less likely to return to the site. By the end of the 6-week period, you can see that return traffic is slowly improving as visitors seem to finally be recovering from their poor experience.

Conclusions

As I’ve already mentioned, these findings have been a huge revelation to the company that owns this site. It’s had a major impact on how they’re tackling performance on their mobile platform. The most important over-arching takeaways from this experiment were:

  • Mobile shoppers are now fully engaging with e-commerce sites in significant enough numbers that we can analyze their behavior as a group.
  • Even a 500 ms delay has a major impact on metrics. For mobile sites, which can suffer excruciating load times (the latest Keynote index is about 12 seconds), this is a wake-up call that they need to take a hard-line approach to optimizing their pages.
  • The damage of poor performance is lasting.

See the full presentation: Case Studies from the Mobile Frontier: The Relationship Between Faster Mobile Sites and Business KPIs

Related posts:

By: Tammy Everts Category: Tags: , , , , , , , ,

Why doesn’t our industry talk more about performance and productivity?

18 Oct 2011

6 Comments

Ever wondered how much time people in your company spend waiting for internal apps to load? One of our clients did some math on this, and they got some eye-opening results:

They calculated that, in just one small department of 20 people, those people spent a total of 130 hours per month waiting for the pages of a single internal web-based application to load. That’s 130 hours (in other words, almost an entire month’s worth of work hours for one person) that could have been spent making sales, responding to clients, fighting with the photocopier – basically doing anything more productive than staring at a screen.

In 2008, Aberdeen released a benchmark report called Application Performance Management: The Lifecycle Approach Brings IT and Business Together. This report stated that application performance issues — including internal app performance — could hurt overall corporate revenues by up to 9%.

At the same time, Aberdeen also found that the average organization was using six business-critical applications, with plans to rollout four more, bringing the total up to ten applications by 2010.

Let’s imagine a not-so-crazy scenario

Now let’s apply Aberdeen’s findings to my customer findings at the top of this post. Imagine that this same department of 20 people isn’t using just one application, but instead is using ten. Imagine that four out of ten of these apps are experiencing significant performance problems.

The results:

  • Instead of waiting 130 hours, the people in this department spend a total of 520 hours a month waiting for pages to load.
  • That’s the equivalent of the total number of work hours for 3.5 employees.
  • In monetary terms: that wait time equals about $9,500 per month, or $114,000 a year (based on an average annual salary of $32,700, according to  U.S. Bureau of Labor Statistics, January 2010).
  • Instead of a 20-person department, extrapolate these numbers throughout a larger enterprise, such as Amazon. With more than 33,000 employees in total, let’s assume that half are desktop workers. Following the rationale above, poor app performance could cost Amazon upward of $7.8 million a month, or $94 million a year.

Crazy? No. A bit facile? Yes. But only a bit. While employees are waiting for apps to load, they could be doing other things, such as making calls and multitasking with other slow apps. But bear in mind usability expert Jakob Nielsen’s findings about response time and human behavior: 10 seconds is about the limit for keeping a user’s attention focused on a non-responsive dialogue. After 10 seconds, even the most efficient worker has to struggle to re-focus on the task at hand. Repeated interruptions are a huge detriment to productivity.

We need more performance + productivity case studies.

We all know about the impact that faster page load had on revenue for Amazon and Shopzilla. But there are some inspiring lesser-known case studies that demonstrate the relationship between improving internal app performance and employee productivity.

In one case study, Hydro-Québec was experiencing some serious performance pains with a shared CAD app, with people in remote offices suffering 30- to 60-second delays between every mouse click. Not surprisingly: “Response time was ugly,” according to Daniel Brisebois, Hydro-Québec’s IT advisor. After application response time was improved by 10- to 25-fold, Hydro-Québec reported benefits such as fewer errors, a faster engineering cycle, and enhanced data integrity.

In another case study, this one for the Hilton Grand Vacations Club, a division of the company that sells timeshares to international customers, the company was experiencing major latency issues that caused delays of up to 30 minutes for a vital contract-processing app. Hilton’s senior director of technology applications, Rich Jackson, said that “As you can imagine, with the customer is sitting in front of you while you are waiting on a computer process, it’s not an ideal situation. When you’re processing a contract, time is of the essence. You don’t want delays due to technology issues.” After accelerating the app, Jackson said that “The contract process has been reduced from more than 30 minutes to just a minute or two. It has a huge effect on customer and employee satisfaction.”

These are good stories, but we need more.

Why don’t we WCO folks talk more about performance and productivity?

Off the top of my head:

  • In the early days of our industry, we distanced ourselves from the application acceleration folks in order to carve out our own niche.
  • We share information using tools like WebPagetest and the HTTP Archive, which are only relevant to the public web.
  • Talking about money is a lot sexier than talking about productivity.

In the months to come, I’m going to do my part to make productivity as sexy as money. If you have productivity stories to share, I’d love to hear about them.

Related posts:

By: Tammy Everts Category: Tags: , , , ,

Case study: Faster pages lead to 6% revenue increase for Artbeads.com

26 May 2011

0 Comment

There are a bunch of reasons why I’m really excited by the newest case study — for Artbeads.com — that we’ve added to our customer success stories:

  • While it’s a successful ecommerce site with a good-sized inventory (27,000-plus items), Artbeads isn’t a mega-giant. It’s what I call a “mortal company“, and it makes a great case for how smaller companies can justify automating their content optimization efforts.
  • As a Yahoo store, Artbeads presented some interesting new challenges for us. The site’s development and hosting platforms presented several potential optimization limitations, which are explained in the case study.
  • And my favourite aspect of this case study: Despite the fact that it’s a mid-sized site with hosting challenges, and despite the fact that its customers had no complaints about the site’s speed, the people at Artbeads still made it their mission to improve performance. As they told us, they had a hunch that performance optimization would pay off, and their hunch paid off. A/B testing revealed that the optimized site delivered a revenue increase of 6%.

Performance optimization for smaller ecommerce websites

You can read the case study here.

Related posts:

By: Tammy Everts Category: Tags: , ,

Why we need to be more than just page speed advocates. We need to be user experience advocates.

24 May 2011

1 Comment

The other day I was thinking about the many parallels between the early days of the web and the early days of performance:

Web dev circa 1996 Web performance circa 2011
Developers leading the charge Developers leading the charge
Designers starting to get interested Designers starting to get interested
Websites barely on execs’ radar Performance barely on execs’ radar
Formal user experience testing just on horizon Formal user experience testing just on horizon

We’re at a really interesting juncture. We’re about to enter a time of heavy transition, and while it’s a necessary transition, it’s not going to be easy for everyone. (Warning: There will be crabbiness.)

The early days: The rise of the developer

In the beginning (back when text was left-justified, all gifs were animated, and you could have a website in any colour you wanted, so long as the colour you wanted was grey), the developer was king. He or she was master of the web domain, keeper of the code. And that worked fine until…

Designers arrived on the scene

Whether they were print designers looking to make a career move or self-taught folks with a good eye, designers saw an opportunity and took an interest in the web. Not coincidentally, this happened at around the time that HTML got fancier. At the same time, business owners started taking an interest in this nutty internet fad.

And then the turf wars began

Sometimes the developer was also the designer, but more and more, these roles were split. These two factions tussled regularly over the right way to build sites. Business owners threw in their two cents as well. There were a lot of heated debates where the phrases “Our users want…” and “Our users need…” got bandied about pretty loosely. Interestingly, “user needs” generally coincided with the desire of the person who was making the argument.

Usability testing came along to set us all straight

While usability engineering had been an active practice in software development for years, it was new to the web. When it made the crossover, some devs and designers were all for it, while others were all against it (and some were really REALLY against it). The upshot, however, was that we all realized that usability was something that could be measured:

“Do I get more clicks when I put that menu here or over there?” “What happens to my conversion rate if I change the wording on the main call-to-action button?” “Which banner graphic gets the most click-throughs?”

Today: Performance seems to be following a similar evolution

Up till recently, developers have been almost solely driving performance. And “performance” has been defined as a pure speed play, hence our industry’s unofficial mantras: “Faster is better” and “Every second counts.”

But as I’ve written about recently, performance is a complex and nuanced challenge that demands an equally nuanced solution. Cranking out pages that seem to be faster according to simple tests is not the ultimate solution. Here’s why:

Everyone in our industry needs to be more than just a page speed advocate. We need to be user experience advocates.

I know I risk sounding like a broken record here, but saying “We sped up pages by 150%” isn’t enough any more. Those faster pages could be killing conversion in some unanticipated way. Just like usability engineers use usability tests to test sites in the real world, we need to run A/B and multivariate tests on real users. We need to test the impact of various optimization treatments on different pages and flows, and we need to focus on measuring metrics that matter: revenue, conversion, cart size, and page views.

And just like usability engineers trust real-world data over gut instinct when deciding if a website works or not, we need to rely on these metrics to make our decisions for us, not just our gut feeling that faster pages = better.

Related posts:

By: Tammy Everts Category: Tags: ,
1 2 3 4 5 6 7