13 Aug 2013
I don’t usually pimp product on this blog, but this video is too good not to share. The laugh-out-loud moment happens at 1:10.
I don’t usually pimp product on this blog, but this video is too good not to share. The laugh-out-loud moment happens at 1:10.
It’s ironic that I’m writing this post at a time when my own flow has been seriously compromised by the arrival of my third child, who arrived a couple of weeks ago. But I’m going to do my best. Stick with me.
I’ve written about flow in the past. It’s a fascinating topic — one that I keep coming back to as I discover new (or new to me) research into it. Last year–
Sorry. Crying baby. What was I talking about? Oh yeah. Flow. What was that about irony?
Last week, I was up late (again: baby) and decided to watch a documentary called Happy, which, as you might guess, is an exploration of the factors that truly make us truly happy. I won’t give away all the secrets to happiness here, but I want to share one line from the movie that jumped out:
“People who experience flow on a regular basis
are happier than people who don’t.”
This was really interesting to me. In the past I’ve talked about research that links flow, in web terms, to conversions, pageviews, and revenue, but I’ve never explored the blunt statement that flow = happiness. So I decided to do some digging with this angle in mind.
First, let’s back up for a minute and define our terms. In web performance, when I talk about flow, I’m talking about one of two things:
A. Flow as a user’s path through a site or application.
B. Flow as a descriptor for the seamlessness of a sequence of actions.
Ideally, you want to experience as much as possible of flow B while you’re experiencing flow A.
Mihaly Csíkszentmihályi (considered by many to be pioneer of the concept of flow) lists the ideal components of flow in his 1998 book Finding Flow: The Psychology of Engagement with Everyday Life:
If, like most people, you live a busy urban life — particularly one that involves new babies — achieving flow has countless challenges from the time you get up until you go to bed:
This list was disturbingly easy to write, and I’m sure I could go on. To summarize, achieving a state of flow is hard work.
As I’ve written in the past, we humans are hard-wired to perform tasks seamlessly. That’s because for hundreds of thousands of years, our brains have evolved to help us carry out day-to-day tasks — from building a fire to planting a field — that are comprised of a series of minute actions that flow virtually without interruption from one to the next.
It’s only in the past forty years, with the advent of computers, that we’ve imposed a new set of demands on our brains. As most of us are painfully aware, instead of offering a series of smoothly sequential actions, human-computer interaction is characterized by lag, downtime, and restarts.
In my travels, I encounter people who are skeptical about the impact of lag, downtime, and restarts on productivity and other key performance indicators. The argument I hear is that most people do, in fact, adjust to poor performance.
As it turns out, these people may be somewhat correct, but they may also be focusing on the wrong part of the picture.
I recently came across a really interesting study into workplace interruptions: Temporal factors in mental work: Effects of interrupted activities (Fred R. H. Zijlstra and Robert A. Roe, 1999). In it, groups of workers were subjected to various disruptions in the course of their day-to-day responsibilities, and then were measured in terms of both their productivity and their self-reported state of mind. While this study focused on general workplace interruptions, with only some attention given to human-computer interaction, there were some fascinating findings that are arguably relevant to web performance.
This research suggests that, at least for some workers in some environments, not only do they learn how to cope with interruptions, they may even strive to overcompensate for their potential performance decline.
Cumulatively, interruptions had a negative impact on emotion and well-being. Participants ultimately needed to increase the amount of effort required to perform the same tasks.
Interruptions seemed to have a cumulative effect. When the number of interruptions grew, the resumption time (i.e. the time needed to re-start the task) became disproportionally longer. The participants seemed to lose motivation and develop mental fatigue.
It’s possible that people can develop coping strategies for dealing with application delays, and that these coping strategies can allow them to maintain productivity in the short term. But the missing ingredient here is flow. And without flow, eventually our sense of motivation and well-being suffers.
It’s also eye-opening to think about our small world of application performance as just one part of a bigger world. As I mentioned at the top of this post, our days are filled with challenges to flow. Poor web performance is just one factor, but it is a significant factor. Consider the cumulative effects of lack of flow in the fast-paced world most of us live in.
As countless studies have proven, human beings are really good at convincing ourselves that we understand what makes us happy, and we’re really bad at actually realizing what makes us happy. Because of this, it’s easy to kid ourselves that, because our productivity is more or less the same — and because we place a great deal of value on productivity — somehow this equates to happiness.
In other words, we can convince ourselves that we’re fine — or if you’re an employer, you can convince yourself that your workers are fine — when perhaps we’re not.
I’d love to close with something like “if you’re in the web performance business, you’re in the happiness business”, but I can sense the imminent mockery of my colleagues here at Strangeloop. And in all truth, I don’t believe that human beings have that much power to make each other happy.
But I do believe that, if you’re in the performance business, you’re in the flow business. By improving flow, we’re helping to remove an obstacle, and in doing this, in our own small way we’re allowing people to find their own happiness. I do believe this. Let the mockery commence.
Last week, we were preparing for a meeting with a customer and, during the process, our sales team asked me this question:
Customer X just asked if we had any analysis of cost savings for the amount of manpower they would save with Site Optimizer. I haven’t seen anything like this, but I thought I’d ask. A blog post, case study, or anything along the lines of cost savings would be great.
The question of ROI from automating front-end performance from a developer perspective is something I haven’t written about. Part of the reason it’s taken so long to get to this topic is because I feel like the answer is obvious, but at the same time I dread doing calculations that might cost someone their job. (More on that later.)
But when I reflect on it, the answer is really only obvious to those of us who care about performance and have tried optimizing by hand. If you’ve never done this, you’d never suspect the pain that lurks behind the performance doors.
In lieu of simply believing that many of the performance best practices should be automated just because industry gurus say they should, let’s get specific on the cost savings.
Understanding this problem starts with some basic assumptions about the type of client interested in developer ROI:
A good example of this type of customer is Sears.com. Their site matters, it’s dynamic, and they care about performance.
Trying to frame this problem is hard because you can look at it from many different perspectives. A good way to start is to look at a couple of key front-end performance rules as described by Steve Souders in the context of different perspectives. I’ll cherry pick a few to provide a picture of the complexity the problems and the ROI benefits.
As I’ve already mentioned, the costs associated with performance tuning are significantly reduced if the site never changes… but good luck with employing that strategy in modern ecommerce warfare.
Let’s take a look at this data from two perspectives:
Visual: Looking at the video below, you can see how much the site has changed from a visual perspective over a year.
Behind the scenes: The amount of change you see above is very common. The amount of change is equally dramatic when you peel off the good-looking exterior and look behind the scenes. This site is constantly changing. Every week — possibly every day — the content changes, the number of requests change, and the number of domains change.
As you can see from these tables, the basic composition of this page changes dramatically over the year.
One of the key ways we can make sites faster is to minimize server roundtrips. Many of the techniques to minimize roundtrips involve combining objects together into packages. If the content is always changing, then these packages also have to change. Keeping packages updated can be very time consuming and arduous. And as anyone who’s actually done this can tell you, this work is also fraught with the potential to make mistakes when done by hand.
The key ROI from automation for dynamic sites is the ability to offload all of the packaging time and effort onto a computer. Not only does this save a significant amount of time, but it also reduces error.
A few caveats on ROI calculation: Calculating cost savings is best done on a case-by-case basis. All of the numbers I cite in this post are from my experience working with customers that do this stuff by hand. Your mileage may vary. However, here are a few guidelines from my experience:
In one large organization that performs FEO by hand, I have seen three full-time developers in the resource management side of the house. These people not only deal with packaging but also with caching and versioning.
Formula: By implementing automated FEO for resource reduction in this context, I would calculate 5 person-days of dev/QA time saved for every major content change on the site.
Changing packages as content changes is hard and costly to do by hand. Changing packages to take into consideration the performance nuances within each browser is a nightmare to do by hand.
Browsers don’t all support the same standards. As we’ve observed in the past, browsers can also change at a moment’s notice with disastrous effects if you are not careful. If you’re going to hand code by browser, not only does your matrix of supported scenarios exponentially increase, but your QA surface also exponentially increases. You also need to stay on top of all new browser developments and patches.
For a modern website with a standard user profile, you would need to do separate packages for at least five browser groups.
In any large organization that performs FEO by hand (think Google, Facebook, etc.) they have a team of browser experts.
Formula: Depending on the size of your organization I would factor in at least one browser expert and a 1.5x increase in front-end development time and a 2x increase in QA time if you’re going to undertake a per-browser optimization path.
You probably already get the point, but it’s critical to break out mobile. Mobile makes front-end optimization way more difficult. Resource reduction is done totally differently in the context of mobile, so you need a totally different skill set to conduct mobile FEO.
Standards are changing quickly. Browsers are changing quickly. Handsets have very different capabilities. Testing is a big pain in the butt.
Most of the organizations we know that perform front-end optimization by hand have a totally dedicated mobile team (on the dev and QA side).
Formula: I would calculate at least at 2:1 ratio of front-end desktop to mobile team members and a 2x increase in workload if they are going to perform these tasks by hand. You also need to factor in the costs of all the different devices and related plans.
As I have written about before, resource reduction is hard across pages and, if done improperly, it can actually slow you down. One of the key benefits of a good automated FEO tool is the ability to juggle all of these different packaging combinations and provide the optimal package.
Given the amount of change seen in a modern site, I would suggest that hand coding resource reduction in any sophisticated way across pages is nearly impossible by hand. If you’re just going to put a few common files into a package and reference it everywhere, this problem is much closer to perspective 1, but this is not good enough to gain maximum performance.
Given how hard this is to do, I would suggest the ROI here cannot really be measured in person-power reduction, but simply in the value of site speed. If you are crazy enough to try to do this by hand, be prepared for months of extra development and a whole lot of pain.
Adding expires header is easy. All of the difficulty arises from having to add the right header to the right resource and then to deal with version changes.
Imagine a simple example in which you add an expires header of 24 hours to a resource called logo.gif. Now imagine three hours later that the image changes but the name stays the same.
You now have a problem because everyone who has logo.gif cached will not request it again for 24 hours (or, to be exact, 24 hours minus the time elapsed since they first received it). This is a big problem because now you have people seeing stale content, and stale content is not acceptable.
To deal with this issue, organizations often have poor caching headers or else they embark on the long process of building a sub-system to version objects and control expires headers. They also need a vigilant operations staff to find stale content and they need to keep someone glued to the CDN purge tool. (Purging on a CDN is typically done manually or else you need to burn development hours integrating with their APIs. This becomes more complicated if multiple CDNs are in use, since no two are the same.)
Proper object versioning and header management is time consuming and expensive to code. Without it, you risk stale content or more roundtrips because your expires headers are not optimal.
Building a proper version control and header management sub-system is expensive (think many person-months of effort). You also have to factor in the time you spend manually purging CDN caches, as well as the damage that stale content does to your brand.
This post just scratches the surface in terms of ROI benefits when it comes to automating FEO — and we’ve only covered two rules out of fourteen! I could go on forever on this topic, but for now here are a few other considerations:
As we know, managing third-party content is costly. It also requires a great deal of vigilance to ensure you don’t have SPOFs or poor performance because of the third-party content you are forced to add to your site. Automating performance offers significant dev ROI because good FEO tools will help manage your third-party tags and place them in the right order. Your FEO vendor should save you countless hours in the trial and error process of moving tags around.
Images are one of the biggest performance challenges and many organizations spend a great deal of time optimizing their images by hand. One of the big time savers from automated FEO is the ability to have all of this work happen automagically.
Automatically renaming files so they can be served from a CDN can be time consuming. One of the benefits of automated FEO is the ability to quickly get onto and change CDNs, thus saving devs a lot of time.
One of the reasons I’ve avoided writing a post on the person-savings from automated FEO is because I believe very strongly that automated FEO is a tool that is best used in conjunction with great developers – not as a replacement. I have seen benign ROI calculations like this one used by number crunchers to justify making damaging resource reductions. (Don’t be fooled, CFO guy. Your developers are a key asset.) If you run a successful modern ecommerce site, you need great developers working on hard problems. The purpose of automated FEO is to let them work on different problems — in some cases bigger problems.
Offloading the arduous task of automation frees your team to climb bigger mountains and to continue to innovate.
Ever wondered how much time people in your company spend waiting for internal apps to load? One of our clients did some math on this, and they got some eye-opening results:
They calculated that, in just one small department of 20 people, those people spent a total of 130 hours per month waiting for the pages of a single internal web-based application to load. That’s 130 hours (in other words, almost an entire month’s worth of work hours for one person) that could have been spent making sales, responding to clients, fighting with the photocopier – basically doing anything more productive than staring at a screen.
In 2008, Aberdeen released a benchmark report called Application Performance Management: The Lifecycle Approach Brings IT and Business Together. This report stated that application performance issues — including internal app performance — could hurt overall corporate revenues by up to 9%.
At the same time, Aberdeen also found that the average organization was using six business-critical applications, with plans to rollout four more, bringing the total up to ten applications by 2010.
Now let’s apply Aberdeen’s findings to my customer findings at the top of this post. Imagine that this same department of 20 people isn’t using just one application, but instead is using ten. Imagine that four out of ten of these apps are experiencing significant performance problems.
Crazy? No. A bit facile? Yes. But only a bit. While employees are waiting for apps to load, they could be doing other things, such as making calls and multitasking with other slow apps. But bear in mind usability expert Jakob Nielsen’s findings about response time and human behavior: 10 seconds is about the limit for keeping a user’s attention focused on a non-responsive dialogue. After 10 seconds, even the most efficient worker has to struggle to re-focus on the task at hand. Repeated interruptions are a huge detriment to productivity.
We all know about the impact that faster page load had on revenue for Amazon and Shopzilla. But there are some inspiring lesser-known case studies that demonstrate the relationship between improving internal app performance and employee productivity.
In one case study, Hydro-Québec was experiencing some serious performance pains with a shared CAD app, with people in remote offices suffering 30- to 60-second delays between every mouse click. Not surprisingly: “Response time was ugly,” according to Daniel Brisebois, Hydro-Québec’s IT advisor. After application response time was improved by 10- to 25-fold, Hydro-Québec reported benefits such as fewer errors, a faster engineering cycle, and enhanced data integrity.
In another case study, this one for the Hilton Grand Vacations Club, a division of the company that sells timeshares to international customers, the company was experiencing major latency issues that caused delays of up to 30 minutes for a vital contract-processing app. Hilton’s senior director of technology applications, Rich Jackson, said that “As you can imagine, with the customer is sitting in front of you while you are waiting on a computer process, it’s not an ideal situation. When you’re processing a contract, time is of the essence. You don’t want delays due to technology issues.” After accelerating the app, Jackson said that “The contract process has been reduced from more than 30 minutes to just a minute or two. It has a huge effect on customer and employee satisfaction.”
These are good stories, but we need more.
Off the top of my head:
In the months to come, I’m going to do my part to make productivity as sexy as money. If you have productivity stories to share, I’d love to hear about them.
This week at Strangeloop, we’re missing our VP of Product, Hooman Beheshti. He’s in Santa Clara, chairing the Performance and Monitoring track at Cloud Connect. If you want to follow along at home, they’re streaming the keynotes live, and BitCurrent (one of the forces behind Cloud Connect) has been live-blogging some of the sessions.
In the lead-up to Cloud Connect, BitCurrent released a very readable report called The Era of the End User, which discusses the cloud, user experience, and internal productivity. A few highlights:
Download the report: The Era of the End User