Exterminating Bugs Prior to Deployment

Introduction

True story. As a kid I thought my dad worked as a bug inspector. See, all I understood was that he finds bugs and someone else kills them. Once I got older (maybe too old) and realized that he actually finds software bugs instead of roaches, I knew that I wanted to follow in his footsteps. That's really what quality assurance engineers are, right? Bug inspectors.

It's ultimately our job as QA engineers to be the inspectors of the code, finding as many bugs as possible. And ultimately finding the bugs before deploying code to the production environments. We refer to these types of bugs as "pre-prod bugs", defects inadvertently introduced as part of a code change which are then fixed prior to releasing the final code package.

Our Mission

The software QA engineering team's mission is to deliver the most stable, reliable and bug-free software to our end users. The goal is to find all of the bugs in our testing during the development cycle rather than after. We want to establish customer satisfaction and confidence in the quality of our products. The less bugs in the final product, the better!

Thinking "Outside the Box"

Generally, the first part of QA testing is known as the "happy-path", an essential form of testing which involves providing a normal input from a user's perspective and verifying the expected output is returned. In other words, "Can you successfully perform a basic workflow from an average user's perspective?" We decided that only testing the happy-path is not enough to find every bug. So, we took our testing to the next level by introducing negative and exploratory testing into our processes. This type of testing ensures that we have enough coverage to spot those evil bugs earlier in the development lifecycle.

Our Pre-Prod Bug Process

The pre-prod bug is essentially a sub-task of a ticket (user story, bug, or task). When a software QA engineer is running tests on a ticket and discovers a bug, the following steps are performed:

1. The software QA engineer captures as many details for the identified bug to assist the software engineer in troubleshooting (i.e. screenshots, recordings, steps to reproduce, logs, expected results).
2. The software QA engineer creates the pre-prod bug subtask and adds the details
3. The software engineer is assigned to the pre-prod bug and starts troubleshooting
4. The software engineer fixes the issue in a timely manner and assigns the issue back to the reporting software QA engineer
5. The software QA engineer confirms the issue reported is fixed and tests around the fix to ensure no unintentional side effects
   • If any additional bugs are introduced, the software QA engineer will create separate pre-prod bugs
6. The software QA engineer adds testing evidence to the pre-prod bug and marks it done

Wrap Up

This official process has prevented nasty bugs from escaping into our production instances and hence improving the overall quality of the products. We strive to continuously improve our efforts to catch more bugs early and to truly embrace the art of "doing it right the first time" as part of our testing.