Question: What do you think these four waterfalls have in common? (Hint: Same browser. Same connection speed. Same location.)
Why measuring first view and repeat view is not enough
When most of us run simple performance tests, we usually focus on two perspectives: the first view and the repeat view. Unfortunately, life is not this simple.
On non-landing pages (for most of you, that would be all pages but your home page), first view and repeat view only represent about 4% of the total views. There’s another view — the flow view — which represents approximately 96% of the traffic for most of your web pages.
And yet despite its incredible relevance, flow view gets almost zero attention.
For instance: Let’s look at a Lonely Planet product page
1. First view
What is it?
This is the view of the page for someone who has never been to the Lonely Planet site before (or has cleared their cache since their last visit).
Likelihood your users will see this waterfall: Small (but still important)
Product pages are most likely only a first view page when they are linked directly from ads, searches, or other websites. It is very rare for product page to be typed directly into the browsers address bar. Looking at our analytics warehouse data, we see that product pages are viewed as first view pages only 3% of the time, on average. But still, these are important views because they are often targeted leads coming from search results and referral engines.
How to see first view in a waterfall
This is an easy one. Just go to WebPagetest, enter the URL, and click ‘Start test’. Look at the first view.
2. Repeat view
What is it?
This is the view of a page if a user goes only to the page, closes their browser, and then reopens the browser and goes only to that page again as the first page they hit on the site.
Likelihood your users will see this waterfall: Almost never
Repeat views in the context of most test tools are a good proxy to see how well you are using cache headers, but users very rarely just go to one page on a site, close the browser, open it again and go back to that exact page.
How to see repeat view in a waterfall
This is another simple view to get. As with first view, go to WebPagetest, enter the URL, and click ‘Start test’. Look at the repeat view.
3. Flow view
What is it?
This is the view of the page when a customer has previously visited at least one other Lonely Planet page.
Likelihood your users will see this waterfall: Frequently
Product pages are often viewed as part of a flow by your users. Looking at our own customer analytics here at Strangeloop, I’d estimate that upwards of 96% of product page views are flow views.
How to see flow view in a waterfall: Two methods
Seeing a page in a flow is not as straightforward as first or repeat views. Obviously your users don’t all take the same flow, so you need to ensure that you test multiple permutations of pages based on the key flows through your site, which you see in your analytics tool.
Method #1: Using HTTPWatch
The easiest way to see the flow view of a page is to install HTTPWatch, start recording your browsing session, and click from the home page to the product page.
Method #2: Using WebPagetest
Another way to see the flow view is to use the script feature of WebPagetest. Click on ‘Advanced settings’ and then click on ‘Script’.
Create a script by copying and pasting the example below and changing the URLs. (If your typical path has more or less steps, feel free to cut/copy to accommodate your needs. Remember that the URL below logData 1 will be captured as the waterfall for this page.)
With hundreds of flows to choose from, how do you narrow down the most relevant ones?
Obviously you could have hundreds of permutations of flows. I was interested in tracking down just how many different waterfalls can be generated from one page.
Using our performance analytics database, I tracked down a specific product page and looked at all of the different flows. The specific page I identified had been viewed almost 25,000 times and had been viewed in 975 different combinations — far too many combinations to run individuals waterfalls on.
So instead of running hundreds of tests, I simply decided to group the pages into different templates, then look at the different flows.
Home page -> Destinations -> Africa -> Benin -> West Africa Travel Guide
Was translated to:
Home page -> Destinations -> Continent -> Country -> Book
I found that by using this method, I got down to a much more manageable set of 10 unique, relevant flows.
I then performed WebPagetests, using the script feature in WebPagetest that I described above. I found that looking at the first view, repeat view, and important flow view waterfalls really helped me see how the site was doing and how real users would view a page.
What this all serves to illustrate
Measuring performance is an art as much as it is a science. While there are a growing number of tools on the market, we need to be aware of what exactly it is we need to test and why. And we still have to be ready to perform some hacks to get the data we need.
- Progressive vs. regressive enhancement: Where and how do you draw the line?
- Experimental new Webpagetest feature lets you test sites on Chrome
- Above-the-fold time (AFT): Useful, but not yet a substitute for user-centric analysis