Holiday shopping week is game time for web performance teams at eCommerce companies. While the holiday shopping season seems to start earlier and end later each year, we still see concentrated web traffic during big promotions throughout the week of Thanksgiving and into the beginning of December. It’s critical that sites perform well under load and in conjunction with high cost advertising and promotions.
Proactively Monitoring Online Retailers
This year Rigor ran proactive monitoring tests to watch the uptime and availability of 65 popular eCommerce companies from the Internet Retailer Top 500. We began tracking performance on November 22nd and ran our tests continuously through November 29th. Our data set includes Black Friday, Cyber Monday, and the days immediately before and after.
To keep things simple and eliminate variables, we ran all of our tests from a location in North Virginia in the United States. We ran tests for availability at 1 minute intervals. Here’s what we found:
Page Load Times
The median page load time for an eCommerce site was just over 10 seconds – 10,180 ms, to be exact. Considering that we know half of web visitors expect a page to load in 2 seconds or less, this was a surprising discovery! That said, many eCommerce sites load in carousels, hero images, product images, and third-party tracking scripts. With that in mind, the 10 second median page load doesn’t seem too outrageous.
In our data set, Dell.com was the fastest loading site on Black Friday and Cyber Monday with median load times of 3627 ms and 3483 ms.
Meanwhile, Macy’s.com was the slowest loading site on our list. The department store’s eCommerce site showed median response times of more than 31 seconds on each day that we tracked.
When setting aside incidents of downtime, we saw that eCommerce sites loaded 5-10% slower on Black Friday and Cyber Monday when traffic was heaviest. This seems to indicate that eCommerce sites planned well for high capacity of visitors.
First, the good news: 38% of the eCommerce sites we monitoring had 0% downtime. This means that we never received an error response code from their site, and we didn’t have any trouble connecting to them across the network. This was an exciting finding. If this was you, great job!
If you did experience downtime on Black Friday, over the weekend, or on Cyber Monday, you were not alone. Let’s look at some of the results for key time periods throughout the peak of the holiday shopping week.
On Black Friday, 30.7% of sites at least one failure. It’s common to have some type of failure or temporary issue of network connectivity. Generally, we wouldn’t treat these an actionable issues unless we saw two failures back-to-back in a row from multiple locations.
9.2% of sites had sustained downtime, meaning that we saw consecutive failures at one minute intervals. We can infer that these consecutive failures likely represented continuous downtime of 2 minutes or more. Although, it’s technically possible that some of these sites could have been ‘back online’ and then ‘down’ again several times within the two minute interval.
We saw some periods of downtime that were quite long and impactful:
Apple.com had a sustained period of downtime lasting 3 hours. The site returned a 503 error with a message saying that they were updating their site with new content. Seems like odd timing for planned maintenance, right?
It was midnight in the EU, but late-night shoppers in North America and some shoppers in Australia and the Asian-Pacific noticed this downtime and took to Twitter to complain.
Macys.com had roughly two hours of sustained downtime. It was an interesting case – the site didn’t return an error response code. The site loaded with a “200 OK” but with much less content than expected. The response times were drastically lower than expected, less than 1 second compared to the 31 seconds that was typical for their site.
Upon close investigation we learned that the site was overloaded and returning a basic error page. The error page was new HTML with only four assets, one of which was a loading indicator GIF. Visitors likely saw an error page with an indicator for how long to wait until the page would refresh and try again to load. This is a wonderful method for attempting to mitigate performance problems. Many visitors likely bounced or were unable to access the site, but it looks like Macy’s was making some effort to prevent the site from crashing entirely.
One key takeaway from watching the Macys.com issue is that you shouldn’t rely on response codes alone to alert your team of performance problems. Set up multi-step tests with success criteria or use thresholds to alert your team if a page size is smaller than expected.
Crate & Barrel
Crate & Barrel had four unique periods of downtime, lasting 3 – 4 minutes each.
During these outages, our data showed that desktop visitors got caught in a redirect loop. Desktop visitors were redirected with a 302 error to a mobile version of the site and then back to the desktop version. We suspect this represents some sort of infrastructure problem, possibly an issue with hardware or a load balancer becoming confused about device types.
Unfortunately for Crate & Barrel, this downtime happened during prime shopping hours for the EU and the US.
Over the Weekend
On the weekend after Black Friday and before Cyber Monday, eCommerce sites continued to see high amounts of traffic.
Over the weekend Walgreens and Neiman Marcus both encountered sustained downtime. Walgreens saw 20 minutes of downtime. Neiman Marcus had 19 minutes of downtime.
The downtime for Neiman Marcus was especially interesting. Their performance throughout the rest of the week was stellar. Outside of this downtime, their site showed 0 errors and consistently loaded faster than the median response time for their peers. This goes to show that even the most solid, well-performing sites must be prepared to encounter unexpected problems. Be sure to have a recovery plan in place.
Compared to Black Friday, it seems that Cyber Monday is just about equally prone for issues. On Cyber Monday, 29.2% of sites had some downtime.
12.3% of the sites we monitored had some sustained downtime with consecutive errors.
Shutterfly had 26 minutes of sustained downtime. As soon as the issue began, their site began redirecting visitors with a 302 error to: https://www.shutterfly.com/priority/
After about 8 minutes of redirecting visitors to the “our site is really busy” page, Shutterfly began failing over completely and returned 503 errors. This is a good approach. When Shutterfly has issues it sends visitors to a landing page and then attempts to redirect them back to the site at set intervals.
What’s unique about Shutterfly’s approach is that their holding tank page is almost too complex in design. The page has a rich design with images, animations, and custom fonts. Also, the page loads in several third-party scripts including content from ad networks.
If you’re going to take this approach, we’d recommend that you make your error page as lean as possible.
On the whole, top eCommerce companies fared well during the high traffic days of this 2016 holiday shopping week. Sites that planned for capacity performed just barely slower than normal and many sites experienced no downtime at all. However, for some popular eCommerce sites that did have downtime, the instances were severe and did not go unnoticed by shoppers on social media.
Now more than ever it’s important to have a plan to respond to downtime, not just on Black Friday or Cyber Monday but throughout the holiday season. We’ll continue to monitor online retailers through the beginning of the New Year.
Interested in gaining insights to how your eCommerce site is performing? Looking for a system to help notify your team when transactions aren’t working as expected? Get ahead of performance issues before they impact visitors and shoppers. Book a kickoff call to start a Rigor trial today.
E-commerce revenue continues to grow,as consumers turn away from shopping in brick-and-mortar stores to shopping online. However, many businesses are not prepared for this growth because they do not fully understand the market and how to invest in...Read More
Because of the multifarious nature of web clients today, it’s important to consider the usage statistics when designing, implementing, and managing your site. However, misconceptions often arise when determining what browsers to design...Read More
Google Webmaster Tools is a web service that allows webmasters to view the status of their sites as seen by Google and the Googlebot crawlers. In addition to indexing your site structure and content, the Googlebot crawlers also record data on perform...Read More
Web designers and developers are always looking for ways to speed up their page load times. Yahoo has an excellent article written on the best practices for speeding up your page. How do you know if implementing one of their suggested practices will...Read More