14 rules for making your web site faster

Notes:	Hide Notes

Slide 1: High Performance Web Sites 14 rules for faster-loading pages Steve Souders souders@yahoo-inc.com Tenni Theurer tenni@yahoo-inc.com Slide 2: Introduction Slide 3: Exceptional Performance started in 2004 quantify and improve the performance of all Yahoo! products worldwide center of expertise build tools, analyze data gather, research, and evangelize best practices Slide 4: Scope performance breaks into two categories – response time – efficiency current focus is response time of web products Slide 5: Rough Cuts: now Hardcopy: Summer 2007 http://www.oreilly.com/catalog/9780596514211/ Slide 6: The Importance of Front-End Performance Slide 7: Back-end vs. Front-end Empty Cache amazon.com aol.com cnn.com ebay.com google.com msn.com myspace.com wikipedia.org yahoo.com youtube.com 82% 94% 81% 98% 86% 97% 96% 80% 95% 97% Full Cache 86% 86% 92% 92% 64% 95% 86% 88% 88% 95% percentage of time spent on the front-end Slide 8: The Performance Golden Rule 80-90% of the end-user response time is spent on the front-end. Start there. • Greater potential for improvement • Simpler • Proven to work Slide 9: Schedule Performance Research break 14 Rules break Case Studies Live Analysis Slide 10: Performance Research Slide 11: perceived response time enjoyable urgent instant stagnant unexceptional accelerate perception snap yawn unresponsive achievement delay moderate impatient better improve action pleasant drag apathetic blah subdue pace quick promote swift cool prolong slack load sluggish maximum drive unexciting prompt sleepy performance speed slow crawl boring snail reduced lag complex heavy advance fast hurry rush unmemorable obscure satisfying feel exceptional why wait brisk rapid exciting late what is the end user’s experience? Slide 12: User Perception Usability and perception are important for performance. The user’s perception is more relevant than actual unload-to-onload response time. Definition of "user onload" is undefined or varies from one web page to the next. Slide 13: http://yuiblog.com/blog/2006/11/28/performance-research-part-1/ Slide 14: 80/20 Performance Rule Vilfredo Pareto: 80% of consequences come from 20% of causes Focus on the 20% that affects 80% of the end-user response time. Start at the front-end. Slide 15: Empty vs. Full Cache 1 user requests www.yahoo.com 2 user requests other web pages 3 user re-requests www.yahoo.com Slide 16: Empty vs. Full Cache 1 user requests www.yahoo.com user requests other web pages dns lookup html image image dns lookup script image image image image image image image image script image image image image image image image image script dns lookup image image image image image dns lookup script script stylesheet image 2 3 user re-requests www.yahoo.com with an empty cache 0 0.5 1 1.5 2 2.5 3 Slide 17: Empty vs. Full Cache 1 user requests www.yahoo.com 2 user requests other web pages 3 user re-requests www.yahoo.com Slide 18: Empty vs. Full Cache 1 user requests www.yahoo.com 2 user requests other web pages 3 user re-requests www.yahoo.com html image image 0 0.5 1 1.5 2 2.5 3 Expires header with a full cache Slide 19: Empty vs. Full Cache empty cache 2.4 seconds full cache 0.9 seconds 83% fewer bytes 90% fewer HTTP requests Slide 20: How much does this benefit our users? It depends on how many users have components in cache. • What percentage of users view a page with an empty cache? “Empty cache” means the browser has to request the components instead of pulling them from the browser disk cache. • What percentage of page views are done with an empty cache? Slide 21: http://yuiblog.com/blog/2007/01/04/performance-research-part-2/ Slide 22: Browser Cache Experiment Add a new image to your page <img src="image/blank.gif" height="1" width="1"/> }1 px with the following response headers: Expires: Thu, 15 Apr 2004 20:00:00 GMT Last-Modified: Wed, 28 Sep 2006 23:49:57 GMT Slide 23: Browser Cache Experiment Requests from the browser will have one of these response status codes: 200 – The browser does not have the image in its cache. 304 – The browser has the image in its cache, but needs to verify the last modified date. Slide 24: Browser Cache Experiment What percentage of users view with an empty cache? # unique users with at least one 200 response total # unique users What percentage of page views are done with an empty cache? total # of 200 responses # of 200 + # of 304 responses }1 px Slide 25: Surprising Results users with empty cache 100.0% 90.0% page views with empty cache unique users with empty cache 40-60% percentage 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 day of experiment page views with empty cache ~20% Slide 26: Experiment Takeaways Keep in mind the empty cache user experience. It might be more prevalent than you think! Use different techniques to optimize full versus empty cache experience. Slide 27: http://yuiblog.com/blog/2007/03/01/performance-research-part-3 Slide 28: HTTP Quick Review 1 user requests www.yahoo.com HTTP response header sent by the web server: HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 Set-Cookie: C=abcdefghijklmnopqrstuvwxyz; domain=.yahoo.com Slide 29: HTTP Quick Review 1 user requests www.yahoo.com 2 user requests finance.yahoo.com HTTP request header sent by the browser: GET / HTTP/1.1 Host: finance.yahoo.com User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; … Cookie: C=abcdefghijklmnopqrstuvwxyz; Slide 30: HTTP Quick Review 1 user requests www.yahoo.com 3 user requests autos.yahoo.com HTTP request header sent by the browser: GET / HTTP/1.1 Host: autos.yahoo.com User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; … Cookie: C=abcdefghijklmnopqrstuvwxyz; Slide 31: HTTP Quick Review 1 user requests www.yahoo.com 4 user requests mail.yahoo.com HTTP request header sent by the browser: GET / HTTP/1.1 Host: mail.yahoo.com User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; … Cookie: C=abcdefghijklmnopqrstuvwxyz; Slide 32: HTTP Quick Review 1 user requests www.yahoo.com 5 user requests tech.yahoo.com HTTP request header sent by the browser: GET / HTTP/1.1 Host: tech.yahoo.com User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; … Cookie: C=abcdefghijklmnopqrstuvwxyz; Slide 33: Impact of Cookies on Response Time Cookie Size 0 bytes 500 bytes 1000 bytes 1500 bytes 2000 bytes 2500 bytes 3000 bytes Time 78 ms 79 ms 94 ms 109 ms 125 ms 141 ms 156 ms Delta 0 ms +1 ms +16 ms +31 ms +47 ms +63 ms +78 ms keep sizes low 80 ms delay dialup users Slide 34: .yahoo.com cookie sizes 100% 17.79% 1.55% percentage of page views over 1501 bytes 1001-1500 bytes 51.80% 501-1000 bytes 1-500 bytes 28.86% 0% Slide 35: Analysis of Cookie Sizes across the Web Total Cookie Size Amazon Google Yahoo CNN YouTube MSN eBay MySpace 60 bytes 72 bytes 122 bytes 184 bytes 218 bytes 268 bytes 331 bytes 500 bytes Slide 36: Experiment Takeaways eliminate unnecessary cookies keep cookie sizes low set cookies at appropriate domain level set Expires date appropriately – earlier date or none removes cookie sooner Slide 37: http://yuiblog.com/blog/2007/04/11/performance-research-part-4/ Slide 38: Parallel Downloads Two components in parallel per hostname GIF GIF GIF GIF GIF GIF HTTP/1.1 Slide 39: Parallel Downloads Two in parallel Four in parallel Eight in parallel html component component component component component component component component component component 0 0.2 0.4 0.6 0.8 1 1.2 1.4 html component component component component component component component component component component 0 0.2 0.4 0.6 0.8 1 1.2 1.4 html component component component component component component component component component component 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Slide 40: Maximizing Parallel Downloads response time (seconds) aliases Slide 41: Maximizing Parallel Downloads 1.40 36 x 36 px (0.9 Kb) 1.20 116 x 61 px (3.4 Kb) 1.00 response time (seconds) 0.80 0.60 0.40 0.20 0.00 1 2 4 5 10 aliases Slide 42: Maximizing Parallel Downloads 1.40 average 1.20 36 x 36 px (0.9 Kb) 116 x 61 px (3.4 Kb) 1.00 response time (seconds) 0.80 0.60 0.40 0.20 0.00 1 2 4 5 10 aliases Slide 43: Maximizing Parallel Downloads 1.40 average 1.20 36 x 36 px (0.9 Kb) 116 x 61 px (3.4 Kb) 1.00 response time (seconds) 0.80 0.60 0.40 0.20 0.00 1 2 4 5 10 rule of thumb: use at least two but no more than four aliases Slide 44: Experiment Takeaways consider the effects of CPU thrashing DNS lookup times vary across ISPs and geographic locations domain names may not be cached Slide 45: Summary What the 80/20 Rule Tells Us about Reducing HTTP Requests http://yuiblog.com/blog/2007/04/11/performance-research-part-4/ Browser Cache Usage – Exposed! http://yuiblog.com/blog/2007/01/04/performance-research-part-2/ When the Cookie Crumbles http://yuiblog.com/blog/2007/01/04/performance-research-part-2/ Maximizing Parallel Downloads in the Carpool Lane http://yuiblog.com/blog/2007/04/11/performance-research-part-4/ Slide 46: 14 Rules Slide 47: 14 Rules 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. Make fewer HTTP requests Use a CDN Add an Expires header Gzip components Put CSS at the top Move JS to the bottom Avoid CSS expressions Make JS and CSS external Reduce DNS lookups Minify JS Avoid redirects Remove duplicate scripts Turn off ETags Make AJAX cacheable and small Slide 48: Rule 1: Make fewer HTTP requests image maps CSS sprites inline images combined scripts, combined stylesheets Slide 49: Image maps server-side <a href="navbar.cgi"><img ismap src="imagemap.gif"></a> → http://.../navbar.cgi?127,13 client-side – preferred <img usemap="#map1" border=0 src="/images/imagemap.gif"> <map name="map1"> <area shape="rect" coords="0,0,31,31" href="home.html" title="Home"> … </map> drawbacks: – must be contiguous – defining area coordinates – tedious, errors http://www.w3.org/TR/html401/struct/objects.html#h-13.6 Slide 50: CSS Sprites – Preferred <span style=" background-image: url('sprites.gif'); background-position: -260px -90px;"> </span> size of combined image is less not supported in Opera 6 http://alistapart.com/articles/sprites Slide 51: Inline Images data: URL scheme data:[<mediatype>][;base64],<data> <IMG ALT=”Red Star” SRC="data:image/gif;base64,R0lGODlhDAAMALMLAPN8ffBiYvWWlvrKy/FvcPewsO9VVfa jo+w6O/zl5estLv/8/AAAAAAAAAAAAAAAACH5BAEAAAsALAAAAAAMAAwAAAQzcElZyryTEH yTUgknHd9xGV+qKsYirKkwDYiKDBiatt2H1KBLQRFIJAIKywRgmhwAIlEEADs="> not supported in IE avoid increasing size of HTML pages: put inline images in cached stylesheets http://tools.ietf.org/html/rfc2397 Slide 52: Combined Scripts, Combined Stylesheets Scripts amazon.com aol.com cnn.com ebay.com froogle.google.com msn.com myspace.com wikipedia.org yahoo.com youtube.com Average 3 18 11 7 1 9 2 3 4 7 6.5 Stylesheets 1 1 2 2 1 1 2 1 1 3 1.5 Slide 53: Combined Scripts, Combined Stylesheets combining six scripts into one eliminates five HTTP requests challenges: – develop as separate modules – number of possible combinations vs. loading more than needed – maximize browser cache – dynamically combine and cache one solution: Slide 54: Rule 2: Use a CDN amazon.com aol.com cnn.com ebay.com google.com msn.com myspace.com wikipedia.org yahoo.com youtube.com Akamai SAVVIS Akamai, Limelight Akamai, Mirror Image Akamai Akamai distribute your static content before distributing your dynamic content Slide 55: Rule 3: Add an Expires header not just for images Images Stylesheets amazon.com aol.com cnn.com ebay.com froogle.google.com msn.com myspace.com wikipedia.org yahoo.com youtube.com 0/62 23/43 0/138 16/20 1/23 32/35 0/18 6/8 23/23 0/32 0/1 1/1 0/2 0/2 0/1 1/1 0/2 1/1 1/1 0/3 Scripts 0/3 6/18 2/11 0/7 0/1 3/9 0/2 2/3 4/4 0/7 % Median Age 0% 48% 1% 55% 4% 80% 0% 75% 100% 0% 114 days 217 days 227 days 140 days 454 days 34 days 1 day 1 day n/a 26 days Slide 56: Rule 4: Gzip components you can affect users' download times 90%+ of browsers support compression Slide 57: Gzip vs. Deflate Gzip Size Script Script Stylesheet Stylesheet 3.3K 39.7K 1.0K 14.1K Size Savings 1.1K 14.5K 0.4K 3.7K 67% 64% 56% 73% Deflate Size Savings 1.1K 16.6K 0.5K 4.7K 66% 58% 52% 67% Gzip compresses more Gzip supported in more browsers Slide 58: Gzip: not just for HTML HTML amazon.com aol.com cnn.com ebay.com froogle.google.com msn.com myspace.com wikipedia.org yahoo.com youtube.com x x x x x x x x deflate x x x some x deflate x x x some x x some some Scripts Stylesheets gzip scripts, stylesheets, XML, JSON (not images, PDF) Slide 59: Gzip Configuration Apache 2.x: mod_deflate AddOutputFilterByType DEFLATE text/html text/css application/x-javascript HTTP request Accept-Encoding: gzip, deflate HTTP response Content-Encoding: gzip Vary: Accept-Encoding needed for proxies Slide 60: Gzip Edge Cases <1% of browsers have problems with gzip – IE 5.5: – IE 6.0: http://support.microsoft.com/default.aspx?scid=kb;en-us;Q31249 http://support.microsoft.com/default.aspx?scid=kb;en-us;Q313712 – Netscape 3.x, 4.x http://www.schroepl.net/projekte/mod_gzip/browser.htm consider adding Cache-Control: Private remove ETags (Rule 13) hard to diagnose; problem getting smaller Slide 61: Rule 5: Put CSS at the top stylesheets block rendering in IE http://stevesouders.com/examples/css-bottom.php solution: put stylesheets in HEAD (per spec) avoids Flash of Unstyled Content use LINK (not @import) Slide 62: Slowest is Fastest Slide 63: Rule 6: Move scripts to the bottom scripts block parallel downloads across all hostnames scripts block rendering of everything below them in the page IE and FF http://stevesouders.com/examples/js-middle.php Slide 64: Rule 6: Move scripts to the bottom script defer attribute is not a solution – blocks rendering and downloads in FF – slight blocking in IE solution: move them as low in the page as possible Slide 65: Rule 7: Avoid CSS expressions used to set CSS properties dynamically in IE width: expression( document.body.clientWidth < 600 ? “600px” : “auto” ); problem: expressions execute many times – mouse move, key press, resize, scroll, etc. http://stevesouders.com/examples/expression-counter.php Slide 66: One-Time Expressions expression overwrites itself <style> P{ background-color: expression(altBgcolor(this)); } </style> <script> function altBgcolor(elem) { elem.style.backgroundColor = (new Date()).getHours()%2 ? "#F08A00" : "#B8D4FF"; } </script> Slide 67: Event Handlers tie behavior to (fewer) specific events window.onresize = setMinWidth; function setMinWidth() { var aElements = document.getElementsByTagName("p"); for ( var i = 0; i < aElements.length; i++ ) { aElements[i].runtimeStyle.width = ( document.body.clientWidth<600 ? "600px" : "auto" ); } } Slide 68: Rule 8: Make JS and CSS external inline: HTML document is bigger external: more HTTP requests, but cached variables – page views per user (per session) – empty vs. full cache stats – component re-use – home pages may be an exception external is typically better Slide 69: Post-Onload Download inline in front page download external files after onload window.onload = downloadComponents; function downloadComponents() { var elem = document.createElement("script"); elem.src = "http://.../file1.js"; document.body.appendChild(elem); ... } speeds up secondary pages Slide 70: Dynamic Inlining start with post-onload download set cookie after components downloaded server-side: – if cookie, use external – else, do inline with post-onload download cookie expiration date is key speeds up all pages Slide 71: Rule 9: Reduce DNS lookups typically 20-120 ms block parallel downloads OS and browser both have DNS caches Slide 72: TTL (Time To Live) www.amazon.com www.aol.com www.cnn.com www.ebay.com www.google.com www.msn.com www.myspace.com www.wikipedia.org www.yahoo.com www.youtube.com 1 minute 1 minute 10 minutes 1 hour 5 minutes 5 minutes 1 hour 1 hour 1 minute 5 minutes TTL – how long record can be cached browser settings override TTL Slide 73: Browser DNS Cache IE – DnsCacheTimeout: 30 minutes – KeepAliveTimeout: 1 minute – ServerInfoTimeout: 2 minutes – – – – network.dnsCacheExpiration: 1 minute network.dnsCacheEntries: 20 network.http.keep-alive.timeout: 5 minutes Fasterfox: 1 hour, 512 entries, 30 seconds Firefox Slide 74: Reducing DNS Lookups fewer hostnames – 2-4 keep-alive Slide 75: Rule 10: Minify JavaScript Minify External? www.amazon.com www.aol.com www.cnn.com www.ebay.com froogle.google.com www.msn.com www.myspace.com www.wikipedia.org www.yahoo.com www.youtube.com no no no yes yes yes no no yes no Minify Inline? no no no no yes yes no no yes no minify inline scripts, too Slide 76: Minify vs. Obfuscate Original www.amazon.com www.aol.com www.cnn.com www.myspace.com www.wikipedia.org www.youtube.com Average 204K 44K 98K 88K 42K 34K 85K JSMin Savings 31K (15%) 4K (10%) 19K (20%) 23K (27%) 14K (34%) 8K (22%) 17K (21%) Dojo Savings 48K (24%) 4K (10%) 24K (25%) 24K (28%) 16K (38%) 10K (29%) 21K (25%) minify – it's safer http://crockford.com/javascript/jsmin http://dojotoolkit.org/docs/shrinksafe Slide 77: Rule 11: Avoid redirects 3xx status codes – mostly 301 and 302 HTTP/1.1 301 Moved Permanently Location: http://stevesouders.com/newuri add Expires headers to cache redirects worst form of blocking http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html Slide 78: Redirects Redirects www.amazon.com www.aol.com www.cnn.com www.ebay.com froogle.google.com www.msn.com www.myspace.com www.wikipedia.org www.yahoo.com www.youtube.com no yes – secondary page yes – initial page yes – secondary page no yes – initial page yes – secondary page yes – secondary page yes – secondary page no Slide 79: Avoid Redirects missing trailing slash mod_rewrite CNAMEs log referer – track internal links outbound links – harder – beacons – beware of race condition – XHR – bail at readyState 2 – http://astrology.yahoo.com/astrology – use Alias or DirectorySlash Slide 80: Rule 12: Remove duplicate scripts hurts performance – extra HTTP requests (IE only) – extra executions – 2 of 10 top sites contain duplicate scripts atypical? team size, # of scripts Slide 81: Script Insertion Functions <?php function insertScript($jsfile) { if ( alreadyInserted($jsfile) ) { return; } pushInserted($jsfile); if ( hasDependencies($jsfile) ) { $dependencies = getDependencies($jsfile); for ( $i = 0; $i < count($dependencies); $i++ ) { insertScript($dependencies[$i]); } } echo '<script type="text/javascript" src="' . getVersion($jsfile) . '"></script>"; } ?> Slide 82: Rule 13: Turn off ETags unique identifier returned in response ETag: "c8897e-aee-4165acf0" Last-Modified: Thu, 07 Oct 2004 20:54:08 GMT used in conditional GET requests If-None-Match: "c8897e-aee-4165acf0" If-Modified-Since: Thu, 07 Oct 2004 20:54:08 GMT if ETag doesn't match, can't send 304 Slide 83: The Problem with ETags ETag for a single entity is always different across servers ETag format – Apache: inode-size-timestamp – IIS: Filetimestamp:ChangeNumber – (n-1)/n Sites with >1 server return too few 304s Remove them – Apache: FileETag none – IIS: http://support.microsoft.com/kb/922703/ Slide 84: Rule 14: Make AJAX cacheable and small XHR, JSON, iframe, dynamic scripts can still be cached, minified, and gzipped a personalized response should still be cacheable by that person Slide 85: AJAX Example: Yahoo! Mail Beta address book XML request → GET /yab/[...]&r=0.5289571053069156 HTTP/1.1 Host: us.xxx.mail.yahoo.com ← HTTP/1.1 200 OK Date: Thu, 12 Apr 2007 19:39:09 GMT Cache-Control: private,max-age=0 Last-Modified: Sat, 31 Mar 2007 01:17:17 GMT Content-Type: text/xml; charset=utf-8 Content-Encoding: gzip address book changes infrequently – cache it; add last-modified-time in URL Slide 86: Live Analysis Slide 87: IBM Page Detailer packet sniffer Windows only IE, FF, any .exe c:\windows\wd_WS2s.ini Executable=(NETSCAPE.EXE),(NETSCP6.EXE),(firefox.exe) free trial, $300 license http://alphaworks.ibm.com/tech/pagedetailer Slide 88: http://alphaworks.ibm.com/tech/pagedetailer Slide 89: Fasterfox measures load time of pages alters config settings for faster loading Firefox extension free http://fasterfox.mozdev.org/ Slide 90: LiveHTTPHeaders view HTTP headers Firefox extension free http://livehttpheaders.mozdev.org/ Slide 91: Firebug web development evolved inspect and edit HTML tweak and visualize CSS debug and profile JavaScript monitor network activity (caveat) Firefox extension free http://getfirebug.com/ Slide 92: http://getfirebug.com/ Slide 93: YSlow performance lint tool grades web pages for each rule Firefox extension Yahoo! internal tool Slide 95: Conclusion Slide 96: Takeaways focus on the front-end harvest the low-hanging fruit you do control user response times LOFNO – be an advocate for your users Slide 97*: Links book: http://www.oreilly.com/catalog/9780596514211/ examples: http://stevesouders.com/examples/ image maps: http://www.w3.org/TR/html401/struct/objects.html#h-13.6 CSS sprites: http://alistapart.com/articles/sprites inline images: http://tools.ietf.org/html/rfc2397 jsmin: http://crockford.com/javascript/jsmin dojo compressor: http://dojotoolkit.org/docs/shrinksafe HTTP status codes: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html IBM Page Detailer: http://alphaworks.ibm.com/tech/pagedetailer Fasterfox: http://fasterfox.mozdev.org/ LiveHTTPHeaders: http://livehttpheaders.mozdev.org/ Firebug: http://getfirebug.com/ YUIBlog: http://yuiblog.com/blog/2006/11/28/performance-research-part-1/ http://yuiblog.com/blog/2007/01/04/performance-research-part-2/ http://yuiblog.com/blog/2007/03/01/performance-research-part-3/ http://yuiblog.com/blog/2007/04/11/performance-research-part-4/ YDN: http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html http://developer.yahoo.net/blog/archives/2007/04/rule_1_make_few.html