In this post we’re going to talk about HTTP caching. We’ll cover why it’s important to visitors and how to implement it to improve performance and user experiences on sites.
An Example of the Effect of HTTP Caching on Web Performance
Let’s consider a real world example of caching.
Wikipedia.org is a popular website that we often reference for helpful information about general definitions or histories of web performance topics.
Imagine that a visitor has a topic that they want to read about on Wikipedia.org They type wikipedia.org in their browser navigation bar and load the page, but then that visitor gets distracted and navigates away to work on some other task. Then that visitor goes back to Wikipedia.org to search for the topic again. When they visit Wikipedia.org for the second time in the same browser they may see that the page loads much more quickly than it did before.
In the above example we’re looking at two page loads for https://www.wikipedia.org that happened within a single browser session. The first time we hit the page it took 1.2 seconds to render and 1.3 seconds to load. When we hit the page again it rendered in a snappy 965 ms and only took 979 ms to load.
The page loaded more quickly on the second visit because some of the elements from the page had already been downloaded and are stored in the browser’s cache.
All other things equal, we should expect that the cached version of the page should be consistently faster than the non-cached version of the page over time. We can see that in this Page Performance trend report, generated by Rigor Monitoring:
The above example report from Rigor Monitoring shows a trend comparing cached vs. non-cached loads of the same page, where the URL with /? Is the cached version.
We can see that the cached version of the page loads more quickly, but how does this work?
What is HTTP caching?
Every browser has a cache which is a place to store local copies of resources. When the browser downloads a resource and stores it in the cache we call this HTTP caching, or just simply caching. Caching allows the browser to keep copies of files, such as the logo or a style sheet, and reuse those copies the next time we visit the website.
A quick note about the name so you don’t get confused! “Caching” is a general term in computer science for “storing something temporarily to use later because that’s faster than fetching it again.” CPUs, disks, and even Wifi cards all use their own types of caching. For the purposes of this post, when we say “caching” we mean HTTP caching.
Why is caching important?
Caching is important because retrieving a file from a server can be slow and expensive, depending on the size of the file and the geographical distance between the visitor and the server hosting that file.
As visitors, caching lets us use an already downloaded copy of a resource, which helps reduce bandwidth consumption, data consumption, and page load times.
As web developers, caching lets us deliver faster page loads to our visitors while reducing the load on the web server. We should work to decide what elements should be cached and what elements shouldn’t be cached in order to deliver the most efficient and performance experiences to our visitors.
Once we agree that we should implement caching to improve the web performance of a site or web app, we may start to wonder – should I cache everything? How should I implement caching on my resources?
Determining A Caching Policy
As web developers or site owners we may want to develop a Caching Policy, or a set of guidelines for what and how to cache resources on a site. When considering a caching policy it’s important to consider:
- How many pages will an average visitor hit during a single session on the site? Are there common elements between these pages that should be cached?
- How often does a typical visitor come back to the site? Is a typical visitor expected to return to the site several times within a single day or week?
Generally, assets like images, scripts, and stylesheets can be cached heavily, while it might not make sense to cache elements that will be dynamically updated throughout the day.
Specifying A Caching Policy
Once you’ve determined which resources you’d like to cache and for how long you can use the Expires Headers or Cache-Control Headers to update the rules for how visitors can cache resources in their browsers.
An Expires Header is an HTTP Response Header that can tell the browser to cache a resource until a specific date, for example:
Expires: Thu, 1 Sep 2016 20:00:00 GMT
The browser will then have to check this timestamp on the resource against the current date and time and if the resource is expired or stale will make a new requests for a new copy.
When using Expires headers we’ll want to make sure that the date for expiration isn’t coming up too soon in the future, as the visitor will have to get a new copy of the resource at that time.
Fortunately we don’t need to worry too much about whether we set the cache for a resource too far in the future. When implementing far future caching we can also use file versioning to ensure that users will be able to pull the latest version of a file if needed. This post includes some great examples of how to “bust the cache” or tell the browser that it needs to download a more recent version of a file, and it includes some information about using file naming to manage cache.
As you can see, the Expires Header is kind of a blunt instrument. When HTTP/1.1 was standardized, the Cache-Control header was introduced to provide more options and finer granularity. Cache-Control is an HTTP Response Header that can be used to define rules related to the caching policy. Cache-Control includes attributes (called directives) that tell the browser:
- who can cache the response,
- under which conditions caching may occur,
- how long the resource may be cached.
The following table shows some directives you may be interested in using:
|no-cache and no-store||Resources marked as no-cache indicate that it cannot be used on subsequent visits without the browser first checking to see if the response has changed. As such, there will be at least one round trip to validate the cached response, but if the resource hasn’t changed, there will be no download. |
By contrast, no-store forbids any caching of the resource by any entity. Every time a user loads a page requiring this resource, the browser sends a request and downloads the resource.
|public vs. private||Resources marked as public can be cached, regardless of HTTP authentication association or status code. Resources marked as private may also be cached, but they are typically done so only for a single user.|
|max-age||The max-age directive specifies the maximum amount of time (in seconds) that the fetched response may be used. The starting point is from the time of the request, so max-age=30 indicates that the response can be cached and reused for 30 seconds after the request is made.|
For intermediaries, there is a similar directive called s-maxage that contains the same specification.
Note that when using Cache-Control we can set a max-age relative to when the visitor retrieves the file instead of a specific expiration date.
If you are interested in digging deeper into the technical details of caching, we highly recommend Mark Nottingham’s Caching Tutorial for Web Authors and Webmasters.
Identifying Caching Defects
If a resource isn’t cached based on our caching policy, we might consider this a caching defect or an opportunity for optimization.
The above image shows Response Headers and Cache-Control for a public resource that’s set to expire in 4 weeks. We could implement far-future caching to ensure that this resources doesn’t reload unnecessarily for returning visitors.
In these cases we’d want to check our web server and make sure that the resource was cached according to the policy and optimize the caching as needed. In fact, this is exactly what we at Rigor did to optimize our own application.
Caching is a fundamental optimization that can significantly improve the performance of a website on subsequent visits. In this article, we learned how caching works and why it improves performance. Since caching involves setting HTTP response headers, this optimization is accomplished by properly configuring your web server or CDN. You can check your documentation or talk with your team of developers to learn more about implementing caching.
Interested in powerful performance optimizations like caching? Then you will love Rigor’s performance monitoring and optimization platform. Our continuous monitoring provides 24/7 performance visibility of your website’s performance and availability and can identify hundreds of performance defects that are slowing it down. To learn more about Rigor’s web performance products, contact us today!
E-commerce revenue continues to grow,as consumers turn away from shopping in brick-and-mortar stores to shopping online. However, many businesses are not prepared for this growth because they do not fully understand the market and how to invest in...Read More
Because of the multifarious nature of web clients today, it’s important to consider the usage statistics when designing, implementing, and managing your site. However, misconceptions often arise when determining what browsers to design...Read More
Google Webmaster Tools is a web service that allows webmasters to view the status of their sites as seen by Google and the Googlebot crawlers. In addition to indexing your site structure and content, the Googlebot crawlers also record data on perform...Read More
Web designers and developers are always looking for ways to speed up their page load times. Yahoo has an excellent article written on the best practices for speeding up your page. How do you know if implementing one of their suggested practices will...Read More