Testing early and often for performance problems is an essential step in ensuring that you are delivering the best digital experience possible for your users. And, as anyone who has had to address performance problems once they are in production knows: it sucks. Ideally, you want to catch any performance issues at the source. But, for many businesses this isn’t a reality. Why? We’d argue because they are still treating performance as special, rather than addressing it as an ongoing area to focus on.
We advocate for treating performance problems as bugs. This isn’t meant to diminish their significance in any way–they can be both small and big issues–but recasting them in this light does seek to define them as an unintended programing behavior. They simply mean that someone, be it IT/Ops, Designers, or Developers, made a mistake.
By catching these mistakes, teams are able to triage and resolve them before they have a widespread impact. Functional testing is already done under the “early and often” mantra. The same needs to be true of performance testing by integrating it into the development lifecycle.
1.Identify What to Test
To integrate performance testing in pre-production, you need to establish the right metrics to test. Performance-related KPIs that you track and trend might vary depending upon industry and your unique objectives, however, internally you should also create and enforce performance budgets for each of the following categories:
- Milestone Timings
- Quantitative Metrics
- Rules-based Metrics
2. How to Fail Based on Performance
Some performance-related issues will be obvious but easy fixes. For instance, unoptimized images or video are usually easy fixes, but they can result in big performance gains. By examining the Performance Budget metrics mentioned above, it is fairly easy to identify which defects need to be addressed.
Failing on other metrics, however, can require a bit more context. For that reason, it’s important to examine metrics within the bigger performance picture. How does your current performance stack up against your last build? Are your performance metrics better or worse than they were a few dev cycles ago? This additional context can equip you to ensure that you aren’t over or underreacting to the metrics you are focusing on right now.
3. When to Test
Image from Gartner
We’ve outlined the “what” and the thinking behind the “how” of automating performance testing in pre-production, but haven’t delved into the specifics.
Whereas production monitoring is meant to identify unforeseen outages or slowdowns, testing in pre-production seeks to identify the detlas between two code states. So, testing should occur when there is a change in the environment, such as a code push.
Staging is the first key junction and can consist of any common, pre-prod environment that is stable, consistent, and functional. It should be similar to production, and you should be sure to include any third party content. However, testing in staging shouldn’t be seen as a panacea. Testing should also occur in development as a sanity check before commits/PR.
As another important note: Testing pieces of the whole rather than the finished product is akin to checking off the quality of the ingredients in a cookie recipe without ensuring that the finished cookie is good. It’s for this reason that entire user flows should be tested rather than just single pages. Here’s more information on how to identify key user flows.
Testing should also occur once in production. This can help close the loop, verify that no performance defects were deployed, and aid in tracking and trending performance KPIs.
Performance isn’t a finishable feature. It should be an ongoing component of improving your user experience rather than something that is done on an intermittent or ad hoc basis. It’s for this reason that you should automate, test early, and test often. If you’d like more information on how you can prioritize performance within your business, schedule a call with one of our Web Perfectionists using the button below.
As mobile devices continue to play a more prominent role in our 21st century society, their usefulness has spread beyond the casual activities of technological enthusiasts. According to research conducted by FUNDtech, a rapidly growing number of ba...Read More
E-commerce is a growing source of revenue for many companies and businesses, as it continues to capture market-share from brick-and-mortar stores over recent years. However, many businesses are not prepared for this growth of online business becau...Read More
In recent years, client-side browsers have been more involved with processing code before it reaches the user's desktop, shifting away from a reliance on servers handling the bulk of this burden. Websites and applications now rely more on a user's br...Read More
Because of the multifarious nature of web clients today, it’s important to consider the usage statistics when designing, implementing, and managing your site. However, misconceptions often arise when determining what browsers to design for an...Read More