Modern websites make a lot of requests. And I mean a lot. And many of these requests are to third-party resources. As this trend continues, it is important to routinely analyze the performance cost of your site’s resources to identify areas for optimization.

One approach to such an analysis would be to aggregate requests at the domain-level. Using raw HAR data, the data that underlies the popular waterfall chart, we can calculate the performance cost of each domain that our site uses.

Using this HAR as an example, our domain analysis for the five slowest domains would look like this:

CNN Domain Analysis

This approach makes it obvious which domains contribute the most to our overall load time. But now what? One option is to eliminate requests to a given domain to reduce its cost. For example, let’s remove all requests to z.cdn.turner.com. A quick scan of the page source reveals nine references to this domain:

CNN Page Source

Removing all nine of these should do the trick, right? Unfortunately, no. Looking back at our domain analysis, there are actually 30 requests being made to this domain. So where are the other 11 requests coming from?

Tracking down requests with HTTP Referer

To find the 11 other requests, we can use the HTTP Referer request header to reevaluate our HAR data. This header identifies the resource responsible for making a given request. Here is what the referer analysis looks like for our example HAR:

CNN Referer Analysis

Instead of aggregating requests by domain, we can now see the resources responsible for the majority of the site’s requests. Not surprisingly, the base page (cnn.com, in this case) is often the main referer. But scanning the table reveals other expensive components, one of which is a resource loaded from z.cdn.turner.com. Expanding this referer reveals several requests to z.cdn.turner.com that we weren’t able to find in the page source:

Second Referer

To make this new analysis even more powerful, we can search for all resources referering to or from z.cdn.turner.com. Any resource matching the search is either requesting additional resources from that domain or is hosted on that domain. Here is what our search results would look like using our same example HAR:

Referer Search

Using the power of HTTP Referer, we can now assign costs to each component we add to our site by seeing how many requests it makes. Instead of treating a new JavaScript library as a single resource, for example, we can now include all the dependent resources it requests in our cost analysis, giving us more insight into the cost of a given file.

To simplify this type of analysis, we’ve created a simple tool at insights.rigor.com. Simply upload a HAR file, and the tool will generate domain and referer reports to help you identify costly components. Next time you are adding resources to your website, consider using HTTP Referer to combat bloat and slow load times.

Note: This post is based on a talk I gave at the Atlanta Web Performance Meetup. Here are the slides from that talk.

Suggested Blog Posts

The Perception Gap for Poor Web Performance

E-commerce revenue continues to grow,as consumers turn away from shopping in brick-and-mortar stores to shopping online.  However, many businesses are not prepared for this growth because they do not fully understand the market and how to invest in...

Read More

Using Browser Trends to Maintain a Better Site

Because of the multifarious nature of web clients today, it’s important to consider the usage statistics when designing, implementing, and managing your site. However, misconceptions often arise when determining what browsers to design...

Read More

Finding Causes of Intermittent Errors from Google Webmaster Tools

Google Webmaster Tools is a web service that allows webmasters to view the status of their sites as seen by Google and the Googlebot crawlers. In addition to indexing your site structure and content, the Googlebot crawlers also record data on perform...

Read More

Optimization Options not Always Optimal

Web designers and developers are always looking for ways to speed up their page load times. Yahoo has an excellent article written on the best practices for speeding up your page. How do you know if implementing one of their suggested practices will...

Read More