The quality assurance process helps a business ensure whether its products meet the quality standards set by the company or its industry. The problem is not that QA is the bottleneck. The problem is that you don’t know what’s in the bottle.
And that’s a problem that QA addresses.
Coming to the QA in our Organization which was rebranding itself from CredAvenue to YUBI, we wanted to Qualitize (Qualitizing is the process of converting quantitative data into qualitative data) our Quality process and focus towards it.
Everyone in our Org knew what the Bug life cycle is or how to create/report bugs effectively. After a bug is reported, we wanted to focus on a uniform process across the teams to ensure they are getting fixed and closed to the satisfactory level. In this post, we wanted to cover our journey on how we achieved it across 10+ teams in various phases.
Phases of Our Journey
Here are the different phases:
Phase1: Brainstorming
We asked ourselves the below Questions
- We already have a backlog of Production bugs logged in the system. How do we clean them on priority to start on a clean slate?
- How to ensure the engineering teams also prioritize fixing old reported issues along with new features development?
- How to ensure teams who actually are fixing bugs regularly should be credited with a wall of fame (or tortoise board to buck them up) to gamify incentives?
- Who should be the owner of Quality? How do we make the whole Engg Team as an owner instead of just the Quality team??
- How do we ensure that we don’t accumulate/build-up Bugs/Issues once we do the first cleanup?
Solving all the above problems was not a short term goal. We needed a sustainable solution to make it as a culture across the organization. With a lot of brainstorming, we came up with multiple phases to take this up.
Phase 2: Revisiting Bugs
In this phase we revisited all opened bugs and identified the cleaning procedure.
- It was simple to define that if a bug is existing in the system > 1 year and not affecting any customer now then we can definitely de-prioritize it.
- Whether any bugs are impacting any customer or its just an internal issue with lower priority.
With above filtering we made a step closure towards prioritizing bugs backlog which needed utmost attention.
Phase 3: Governance Team and Wall of Fame
We knew that it was a mammoth step to clean the entire prioritized backlog and a single person can’t do it. So we formed a Governance team and called it Men-in-Black team (MIB) the 3 QA Managers.
The primary job of this MIB team was to monitor, identify, acknowledge and appreciate the teams who are closing bugs on priority on a sprint-by-sprint basis. We came up with our own definitions of classifying teams across Tortoise, Rabbit & Fame boards. We started publishing this board in every engineering townhall.
- Tortoise Board ( Lagging teams)
- Rabbit Board ( quick resolution )
- Fame Board ( fully resolved with open bugs)
In due time, we have seen teams getting moved from tortoise board to Rabbit Board and finally moving towards fame board. This was a mission accomplished for our MIB team.
Phase 4: Introducing Bug Jail
Now the MIB Team ensured that the bugs are getting fixed by various engineering teams. We still had a problem of some teams not able to prioritize bug fixes over delivery. To ensure that the team understands that Bug Clean up is also important, we introduced a process called Bug Jail.
What is the bug jail?
Bug jail is not a place where bugs are sent as punishment for their crimes. Rather, it’s a (virtual) place where teams are sent when they have too many open bugs.
Who gets into bug jail?
Our governance (MIB) team established the maximum number of bugs (known as a bug cap) each team is permitted to have and if the bug count exceeds the specified number the team is placed inside the bug jail.
What happens if a team is in bug jail?
Once the team lands in the bug jail, they have to stop new feature development & all efforts need to go towards coming out of bug jail.
How does one come out of bug jail?
The precise trigger for a team to come out of bug jail to get if by getting their bug count back down to 50% of the bug jail trigger or within the limit before you are allowed to exit.
We started publishing the bug trends per team, and whether any team is entering into the bug jail. This prompted teams to proactively close the bugs and not hit the bug jail.
Phase 5: Bug Triage Process
When we successfully cleared backlog of old bugs we needed to ensure all future bugs entering into the system should go through a proper triage process and need to effectively get closed within threshold time, so came the Bug Triage.
What is Bug Triage Meeting?
The word “Triage” is basically used in the medical field. Actually, it used to decide the order in which the patients should be treated. Big hospitals usually witness thousands of patients everyday either for consultation or actual treatment. However, all patients are not admitted or treated immediately.
Similarly, the term Triage is introduced in software testing for Bugs in the application or a project. Bug Triage is a process where the quality team finds out the bug and assigns a degree of risk, re-occurrence, and severity to it. This essentially prioritizes the bugs to be treated first.
Who organises it?
Triage meetings are facilitated by the quality team lead and it is coordinated with Delivery, Development, Testing and Product Manager teams.
What’s the Outcome of Bug Triage Meeting?
In the meeting, the following criterias are identified for every bug.
- Severity: This is a measure of how severe the bug is.
- Priority: The priority could be high, medium, or low.
- Assigned to: This field has the name of the person/team to which the bug has been assigned for resolution.
- Resolution: What steps should be done to resolve the bug.
- Estimated resolution date: Estimated date on which the Bug will be completely resolved.
- Root cause description: This field has information related to why the bug appeared in the first place.
Final Outcome & Results
Finally after all these efforts over a month, we saw a sharp downtrend for the open issues within a month. Kudos to the efforts led by the Men in Black team supported by all engineering teams working towards it.
