Ilya Grigorik (Google)
24 Jan 2013
As a cornerstone of the Make the Web Fast team at Google, what Ilya Grigorik doesn't know about web performance probably isn't worth knowing. In this week's podcast, he and Joshua talk about why big CDNs need to adopt SPDY, how mobile is the next big frontier for Google, and what it's like sharing an office with Steve Souders.
Joshua Bixby: Hi everyone. It’s me again, Joshua Bixby, president of Strangeloop Networks and blogger at webperformancetoday.com, and I’m here with another edition of the Web Performance Today podcast series. We have had really positive feedback on the podcast series so far and I’m excited about a really interesting guest — Ilya Grigorik who is a cornerstone of the Make the Web Fast team at Google and he’s also, simply, a really, really, smart guy. What he doesn’t know about performance probably isn’t worth knowing as far as I can tell, and we talked about a lot of interesting things from how CDNs need to adopt SPDY, about how mobile is the next big frontier for Google, and what it’s like sharing an office with Steve. I hope you Enjoy.
Joshua Bixby: I am joined by Ilya Grigorik, developer advocate at Google, one of the smartest guys in the web performance industry. I am thrilled to have him on. Also, the office mate of Steve Souders. Is that still the case Ilya?
Ilya Grigorik: It is yes that’s where I get all my smarts.
Joshua Bixby: Well I’ve seen you talk enough times to know that’s not true. Welcome to the podcast, thank you so much for making time. We’ve been struggling to get this scheduled between the two of us so it’s nice to have you here. For the world who doesn’t know and for those you don’t you should be Googling immediately, tell me a bit about your experience at PostRank because that’s where you came into the Google family right? And a Canadian company at that so this is a message from the great north back down to you in the hot south. But tell us about your experience coming into Google and your experience at PostRank.
Ilya Grigorik: Sure sure so I joined Google about a year and a half ago, or our company joined Google so I started PostRank back in 2007 and it was a collection of things but effectively what we were doing was we were collecting the social graphs all the signals around the web without people sharing links, things like people voting on links, commenting on links etcetera and using that as a ranking function for all kinds of things so we ended up building a few products in the analytics space for publishers, marketing agencies and so forth and at one point we found ourselves talking with the Google analytics team about doing as they said a tighter integration with the Google analytics product.
Joshua Bixby: Okay
Ilya Grigorik: And so our team was located in Waterloo Canada on the east coast and about a year and a half ago the company was acquired by Google you know in Google analytics and we ended up building the social analytics within Google analytics. So for the past 4 or 5 years I guess I have been really focused on the analytics space so with a vent towards social but performance is kind of the overarching theme for the past 5 years.
Joshua Bixby: So tie that back for me I mean if you’re checking out peoples who comments on what and how they’re ranked and how many followers they have and all the things that go into the social graph how is performance directly related to that business. Why was that important to you?
Ilya Grigorik: Right, so one of the things we found was that a lot of people were in fact tracking things like how many likes did I get, how many followers do I have and all of those are interesting metrics in a sense that if you don’t have anything else at least that’s a proxy for something useful but at the end of the day it really boils down to dollars and cents and that’s what we spent a lot of our time thinking about just in terms of if you were to run a new social campaign and I think that’s why this is relevant to the web performance community because I find that we’re actually up against the same challenge here as well. We could make pages faster because we could get more followers but does that translate to dollars and cents. Does that impact my bottom line and for every single fight in every different marketing campaign the answer will be different based on what is it you’re doing and what are you trying to optimize. But for one site, a 10% improvement in site speed could translate into quite literally millions of dollars in revenue and for another like my blog perhaps that’s not even relevant.
Joshua Bixby: Yeah
Ilya Grigorik: I would have been better off spending time actually working on content as opposed to optimizing performance. So you know that same experience of whether it is social and we were trying to what kind of things you could do within your social strategy or how you spread your content or who you engage and how those things translate to your bottom line and that’s directly applicable to the web performance community. Of course we have now RUM and I think the beginning of the conversation about how do we actually translate those metrics into dollars and cents.
Joshua Bixby: Yeah, that’s a fascinating connection the noise among RUM data not only the noise it’s making in the market but how noisy the data is and how one can operationalize it and actually start learning. Many of our customers are just at the infancy in that trying to figure out what correlates with what. It feels like some of the regressions I did for my thesis many years ago around are any of these things connected so it certainly is something that’s so early. So, what are you working on? What’s catching your attention right now?
Ilya Grigorik: Oh let’s see. There’s a lot of stuff. Too many things if anything. That’s one of the benefits of working at Google and it’s one of the downsides as well. There’s so much surface area to do all the different projects but I still work quite closely with the Google analytics team exactly for the reasons I just described so trying to figure out how do we link web performance to actual on site metrics. Google analytics is a very popular product for that kind of data. A lot of clients use Google Analytics or a lot of sites use Google Analytics to track their performance or their business metrics and connecting those things to web performance I think is one of the things that really interest me.
Joshua Bixby: So I wrote a blog post about this a little while ago where I was looking at who’s actually using RUM and obviously noted that lots of the sites use Google analytics but when I went and talked to the analytics guys at many of our customers, a very informal and hacked survey I found that most of them weren’t using the RUM features of Google analytics. Is that a bad market segment or is that data incorrect or do you what’s your insight? Is RUM in Google analytics being used extensively and regularly?
Ilya Grigorik: Honestly I think it’s awareness. I don’t think a lot of people are still to this day aware of all that data in there and I’ll be the first one to admit that a tool like Google analytics is overwhelming.
Joshua Bixby: Yeah
Ilya Grigorik: I’ve spent a year literally living in that tool and I finally think I’ve found the edges of the tool in terms of capability.
Joshua Bixby: Yeah
Ilya Grigorik: But the first time you log in, it can be overwhelming. So I think a lot of users do get lost in that process and I know it’s something that the team here has been thinking about how to address but that’s also something that I’m hoping others can fix as well by building better dashboards or just extracting important data out of there so I think there’s a lot of room for improvement in that space.
Joshua Bixby: So that still takes up a chunk of your time. What else I know you have many other projects. What else is going on?
Ilya Grigorik: Yeah so lots interest and work on SPDY and http 2.0 I guess now. In the background 6 months ago or so I kicked off a project with O’Reilly to write about SPDY and of course the http 2.0 aspect as well that has also evolved into something much more ambitious so part of my time is occupied documenting all of that and working on the actual book to talk about HTTP and all the things that are coming.
Joshua Bixby: Does that excite you? I know that this is one of those topics that when we get into any of the specs, 80% of people’s eyes glaze over and for the rest of the 20% that’s still there there’s sort of like 1% that is really pushing hard and I put myself sort of in the 19% that don’t glaze over but at a point I just hope somebody else just carries the torch. Is that something that are you sitting there at night thinking about the spec and the future and its value and does it get you going?
Ilya Grigorik: Yeah it does. It absolutely does. I don’t know maybe that makes me some weird bucket.
Joshua Bixby: You’re in the 1% but that’s fine. It’s good to be an outlier I think. So what is it about that that it seems like a very consensus based process almost like things tend to get watered down in that process. Is that a fair assessment or do you think that it really produces good quality and objective and good quality results.
Ilya Grigorik: Yeah I think what excites me is it definitely takes a lot of time and effort to go through the entire process but what initially attracted me to SPDY was this promise of having a big impact for all users on the web. The original goal was a 40% or more in terms of latency improvements just by running over SPDY. That just means we are not asking the web developer to changes any of their pages we are not asking the user to install a new browser or upgrade their connections. Everything stays the same we just swap in this magic layer in between and everything gets a little bit faster. Not a little bit, I should say 30-40% faster. That was kind of in the original test. The actual results will of course vary site by site so you know in the grand scheme of things it might take years to get them deployed but the impact on the overall web is huge and I think that’s the part that motivates me to continue working on it.
Joshua Bixby: That’s a nice way to look at is because it is true as I battle customer by customer and as developers battle page by page on their own sites or template by template, they’re making incremental changes albeit very important but you know to effect the whole web and almost everything that goes over it there are very few opportunities to that and SPDY is one of them ultimately and how’s it going? I mean as you know we were one of the early SPDY partners with Google and continue to work in that vein. How are you seeing adoption and results? How’s it looking from your perspective?
Ilya Grigorik: I think all the signs are actually very positive. We have of course a lot of the Google service actually all of the Google service is run on SPDY if you have a client that supports it we now have Chrome, Opera, Firefox all support SPDY so that alone gives you over 60% of the market share in terms of browser vendors.
Joshua Bixby: Yeah
Ilya Grigorik: So the client support is there and there is also very good now support on the server so if you’re running apache nginx, you can literally just swap in an extra module without changing any of your code and you could potentially see a fairly significant improvement in performance so I think that’s pretty exciting.
Joshua Bixby: It’s always felt to me that there’s a gating the CDNs and the load balancer and whoevers terminating TCP within that data center have this sort of gating effect on the adoption of a protocol like that because if apache does it that’s great but if you’re sitting behind a load balancer in the modern world they’re probably the ones terminating and on most modern websites, you’re probably only serving the html out of the datacenter anyway most of the objects are being served out of a CDN so how is I mean I see that on a small site having a browser and a server is going to work but on the vast majority of the traffic, it must be the CDNs and the load balancers that are the critical path almost to getting mass adoption.
Ilya Grigorik: Yeah, you’re absolutely right and in fact you know speaking of making big impacts what was great to have per server deployment whether its apache or nginx, it’s probably even more meaningful to have a big CDN adopt SPDY or some other optimization technique so I think that’s the an important thing to focus on, not just for SPDY but for all other kinds of optimizations. For SPDY specifically I guess what you’re pointing out is one of the requirements at least today is SSL and specifically next protocol negotiation which is a new extension to the SSL handshake and if you’re running an older SSL termination end point or some other router which is not supported then you can’t actually upgrade your connection to SPDY so practically speaking that is a bottleneck for deploying SPDY today for many people. For example, I recently had a discussion with a big site that’s actually using Amazon web services and they’re using ELBs or elastic load balancers to terminate their SSL and they’re interested in deploying SPDY but ELB’s can’t do NPN negotiations.
Joshua Bixby: Yep
Ilya Grigorik: So quickly that discussion turned into ok well we can turn our ELBs into PCP proxies but then we need to terminate downstream and then all of a sudden there’s an order of magnitude of complexity that is introduced to your operation that you didn’t have to deal with before.
Joshua Bixby: Yeah and I’m going to have to find a CPU and the overhead and CPU to do that somewhere else and the conversation isn’t as smooth as upgrade to newest version of apache and probably people are using Chrome. I mean the world I live in is much more of that enterprise world where it seems like the people that want to do it there are these little blockers that are hey I know Akamai doesn’t support it I know they said they would but they don’t and you know my old F5 isn’t going to work that way and it’s interesting this idea that it makes a big difference is true and we certainly saw results but it also feels like there’s a lot of bureaucracy and politicking that has to happen to get the ecosystem to adapt.
Ilya Grigorik: Yeah and that has its ups and downright so once again it may be hard to get the industry to move and once you have an Akamai or large provider adopt something like this and illustrate and actually be able to show hey we’re seeing an improvement you just know that others will follow as well. We do have that herd mentality.
Joshua Bixby: You have such an interesting career or breadth of experience because you’re not only head deep in google analytics, you spend your time on SPDY but then you’re also looking at you know walking the dom and fiddling with CSS I mean you’re deep in the weeds on the other side of the equation which is how pages are put together, not just the protocol side not just the analytics side so what are you doing in the space you know you’ve spoken a lot about Chrome dev tools. Is that something you add to the pile? Is that also still on your plate?
Ilya Grigorik: Yeah absolutely. So I actually work very closely with the Chrome team here at Google and actually nowadays focusing quite a bit on mobile. I think google is obviously not the only one that realizes mobile is the next big frontier if you will and I think if we pat ourselves on the back and say we actually have a pretty good story for web performance optimization on desktop and compare that to mobile we don’t know what we’re doing yet. At least I’ll claim that I don’t think we know much about how to build a great mobile web app today. I think we need better instrumentation and I think we need more understanding of how the networks work you know we can certainly up the amount of protocol but also just instrumentation like how does the browser actually work. One of the things I think we have a significant gap in today is I think there are well infact there are over 1 billion devices out there which have webkit on them.
Joshua Bixby: That’s amazing.
Ilya Grigorik: And they’re either Android and eyelit devices. So you have 1 billion devices where you have webkit and webkit is effectively nowadays is like an operating system. Some people will call it an operating system, some will call it a VM it doesn’t really matter. What matters is that this is probably the largest and most consistent platform that you can deploy today. And if you look at the internal of the browser, any interesting computer science problem that exists is at some level pushing, the boundaries of that problem is being pushed within that browser. We have GPU, and compositing, and graphics which are getting much more interesting. The network back of every browser is fascinating, it’s much more than just open a pocket and read some data, there’s layers and layers of learning, predictors, all kinds of optimizations that go on through that work. How we construct a dom tree in all of that. Long story short, I think today, if you’re let’s say studying in computer science or software engineering in university, chances are you are taking a networking course, a graphics course, and a variety of other things like how to build a file system. Well I think we have a gap in the sense that we should have at least a course on how to build a browser, how does a browser actually work. I think most people graduating out of colleges and universities will end up building for the browser, and yet they have no idea how the browser’s actually architected, like what are the bottlenecks, what are the issues, how does the rendering pipeline work, how does it interact with the network stacks, etc. So that’s, I find all of those things fascinating, that’s why I end up looking at almost every layer, which of course creates a lot of work but I find it extremely rewarding as well just because you’re able to uncover the really interesting trends and performance bottlenecks.
Joshua Bixby: So when I get that class at Stanford that I’m intending to teach, you’ll come in and do some sessions on that for me? I’ll just bring you in? Ok good good. Note to Stanford.
Ilya Grigorik: I’ve honestly been pitching a couple of universities, like they should have this course.
Joshua Bixby: Yeah, no I think you’re absolutely right. Cause we hire a lot of grads, and we teach them, because they come in as you say with quite siloed knowledge and we teach them about how it all comes together, you know, I still walk in and teach some of these guys about how a mobile phone works, how a radio access network works, it’s amazing the lack of knowledge about really all of the pieces and tying it all together and as you say the browser is probably is going to stitch together 99% of what these people are going to work on and I find the lack of knowledge, not that I have any depth compared to you, but the lack of knowledge frightening, frankly. If you could innovate in one of those areas, if the browser is sort of the hub of that and the spokes come off it, be it the network layer, the graphics layer, where do you think the most innovation is left to be done from an optimization perspective? Where would you go first?
Ilya Grigorik: Oh interesting. So there are too many. Let’s see. Let me pick something that just literally crosses mind because I just spent all of today looking at this problem. Images. Images on the web. We know that an average page on a website is about 1MB in size, and 60% of that is in images. So we’re talking 600 kilobytes for your average page. And yet, if there’s one thing that we could optimize, just kind of going down that list, then you would think that images would be the thing to work on. And in fact we do know that there are formats, for example Google has webp which can do a better job than jpeg. For example, for jpeg’s webp can do about 30% better compression. For png’s, so one of the most popular use cases for png’s is the fact that they support the alpha channel and the transparency. We know that in the latest release of webp, we can actually get 80% better compression ratio, if you use webp over png with an alpha channel. 80% right, so this is significant. If you added up all the numbers between gif’s, the jpg’s, the png’s, just having webp could save us anywhere from 30 to let’s say 60% of bytes on the wire. So we’re talking hundreds of kilobytes of data. This is huge savings for mobile, this is huge savings for desktop, and all the rest. So, the question is, why don’t we have more image formats? Why do we have this scarcity, this fake scarcity of image formats on the web? We’re basically stuck with three, we have gif, png, and jpg. Now why can’t we have dozens? Right, why can’t we have specialized image formats, for all kinds of use cases? I think that’s a gap you know it’s not intuitive at all why we should have three. And in fact I think we should, today we know in fact that as humans we’re terrible at optimizing images. We could be saving images that are better compressed as jpg’s as png’s and vice versa.
Joshua Bixby: Yah we see it all the time.
Ilya Grigorik: So first of all this problem should be automated, and second, once we automate it, let’s just add a whole bunch of formats. Why not, right? Like there’s huge savings to be had here. So just thinking that, what is wrong with respect to where we are today and what do we need to fix to enable that.
Joshua Bixby: And how do you explain why we have coalesced around, you know inefficient image formats for example and why that’s not changing. What’s your, you know when you look at, cause you’ve worked on both sides of the fence, you’ve worked you spent a Google lifetime, you’ve spent a lifetime working in the real world, too, not to suggest that Google’s not the real world, but you know, it isn’t. Why is that the case? No one’s taken the initiative? It’s too hard to do? All of my content management system just saves them the way they save them and that’s obscured from me. What explains it do you think?
Ilya Grigorik: Well it’s probably a combination of all those things, right. We began with just a couple of formats. If you look at png, png took a while – that’s an understatement – to roll out, due to the various kind of bugs and everything that we had through the multiple versions of IE. I think it’s a combination of maybe we didn’t have the tools to actually optimize, maybe their incentives weren’t there, to be honest. Right so the web has grown very fast in terms of size and size of pages and all the rest so perhaps this is only now that we’re actually recognizing this problem as something worth optimizing. And then we actually have some mechanisms in browsers which almost do what they should do except that they don’t. So as an example, how do we negotiate an alternate image format today? Well we just assume that every browser that exists supports all three, right, and we kind of punt on the whole problem. But we do have content negotiations, this was added in HTTP 1.1 back in 1997, so the mechanism has been around, except that we don’t actually use it. If you look at the accept header that for example Google Chrome sends, today it’s * / *, which basically tells you hey I accept everything. So I have a server and I invent my new magic image format and I throw it at Chrome, it claims that it can understand it except that it doesn’t. So we can’t actually do server negotiations properly if the user agent doesn’t actually report what image formats it supports. So that’s problem number one, right, we need to fix that. And problem number two is that you have all the intermediate caches. So now caches, and this is not a new problem right, this is the same thing as having a g-zipped resource versus a regular resource. Multiple representations of the same object, living at the same URL. So we also have a solution for that called that it’s called the vary header, so we say we’ll vary this constant based on constants in coding.
Joshua Bixby: CDN’s hate that one, but they oblige.
Ilya Grigorik: Right. So what can we do vary accept, so we say well I support webp, the service serves the webp image, and then you can do the proper negotiations. So it turns out that many of the caches don’t respect or don’t work well with vary. So you know those are the problems, but they are solvable problems, we need to fix these things. And I think, in the grand scheme of things, out of all the things that we are doing or could be doing, optimizing or removing hundreds of kilobytes of every page load is definitely something worth fighting for. And because of that, it is the only kind of, in the sequence, we need to fix the browser to actually record useful accept headers, that gives motivation to the server, to actually be able to do the negotiation and actually do the encoding. Alright like as a user, as somebody who’s offering content, I don’t want to think about what are the optimal image format. I want to save it whatever format I happen to prefer, whether that’s a png or a jpg doesn’t matter, and let the servers determine what the best format is. Servers are great at iterating through dozens of formats and finding the optimal one. I’m not.
Joshua Bixby: Agreed. Does this stuff ever get you down? I mean when you think about the waste and the slowness, cause you sound really optimistic about it which is wonderful and you know I’ve had this conversation with Steve where he’s just the eternal optimist. Do you ever get frustrated?
Ilya Grigorik: Well you can look at it from both ways, right. It’s, oh my god it’s so broken that we can’t fix it, or this is a ginormous opportunity. I think I’m in the second camp, I think this is a huge opportunity for us to make impact. So, that’s my biased view.
Joshua Bixby: I love it. I love it man. OK so, and I’ve taken way more of your time than I should but I can’t leave the “you’re Steve’s office mate” alone, so you gotta tell me something that, like does he bring smelly lunches? Does he sleep at his desk? Like you gotta give me something cause he’s just so crispy clean, everything, he’s the nicest guy in the world, he makes time for everyone, you gotta give me something. Gossip! Give me some gossip.
Ilya Grigorik: Well I think I told you at the beginning he’s the one feeding me all the tips.
Joshua Bixby: Yah but it only makes him look more saintly, so now he looks even more saintly. I need something, he’s so high on that pedestal, that I want to humanize him.
Ilya Grigorik: I’ve got no dirt. He’s that good.
Joshua Bixby: Oh man, that’s it, I’m going to set up shop with “Saint Souders” and find something on the man, cause I’ve been trying to dig for years now, and there’s nothing! The man’s basically a saint. I appreciate you taking time to come on the podcast. I could sit and talk to you for days and hours on end because you just have this breadth of knowledge you go from end to end in our world and it’s so rare. Most of the time I get a chance to talk to one guy who knows SPDY, but the minute it comes to something outside or server oriented, they’re out. It’s just, it’s remarkable. I appreciate you taking the time, I’m honoured you came on the podcast, and I will see you soon!
Ilya Grigorik: Great, thanks Josh.
Joshua Bixby: Ok take care.
Ilya Grigorik: Thanks, bye.
Joshua Bixby: Thanks so much for listening, and huge thanks to Ilya for taking time out of his busy day to chat. Always nice to chat to a fellow Canadian and a brilliant man at that. It was indeed a privledge and a pleasure to speak with him. As you can tell we could have gone on for hours. We actually did after the conversation have a few other chats. A few other topics I’ve noted for future podcasts. If you’re new to the podcast, go to webperformancetoday.com/podcast and check out the other really interesting conversations we’ve had. if you have an idea for a topic or you want to be a guest yourself please email me at firstname.lastname@example.org. Have a great day.