Everything you need to know about app bug reproduction

While detecting a bug in an app so that it can be eliminated may seem like the ultimate goal of mobile app testing, it’s only half of the battle.

A clear description of each step leading up to the bug is just as important, if not more so.

Without these bug reproduction details, developers have no choice but to guess what caused the problem—which is often a fruitless and potentially frustrating endeavor.

But when a detailed report is provided about how bugs are triggered and reproduced, it can make all the difference in coming up with effective solutions.

To delve deeper into this topic, we’re going to look at what it means to reproduce a bug, the importance of this process, and how you can use it to your advantage.

Keep reading for the full scoop.

What is bug reproduction

As the name suggests, bug reproduction is a process by which testers attempt to recreate bugs they’ve identified in an application.

This allows them to create reports that explain exactly what steps were taken before encountering a particular bug, as shown in the picture below.

As one of the most critical aspects of testing, bug reproduction is then used to help developers identify where the problem lies, how it can be fixed, and how to prevent similar bugs from cropping up in the future.

The more precise information testers can provide about the steps that led them to discover a bug, the easier it’ll be for the developers to reproduce that bug on their own.

In other words, getting it right and not skipping any steps is paramount for this process.

If the developers don’t receive enough information about the exact steps that resulted in a bug in the right order, they’ll have a hard time trying to recreate and resolve it.

Why is bug reproduction important

When developers are trying to fix a bug, the first thing they need to do is reproduce it.

This means that they need to be able to make the bug happen again on purpose in a controlled environment so that they can investigate what went wrong.

For example, imagine if a developer was aware of a bug in an application but had no idea why it occurred or what steps led to it.

Since the cause is elusive, removing such a bug would likely prove extremely difficult and time-consuming, and may involve a lot of guesswork.

In his article Fixing Bugs—If You Can’t Reproduce a Bug, You Can’t Fix It, Jim Bird puts it this way:

Technically speaking, you can’t be sure you are fixing the problem unless you can run through the same steps, see the problem happen yourself, fix it, and then run through the same steps and make sure that the problem went away. If you can’t reproduce it, then you are only guessing at what’s wrong, and that means you are only guessing that your fix is going to work.

It’s not only developers who benefit from bug reproduction—project managers also reap the rewards.

Generally speaking, when a bug report comes in, project managers are often the first to respond, as you can see in the diagram below.

Since they’re tasked with assigning severity levels to bugs, the steps to reproduce help them understand the bug and its impact.

They can then prioritize fixing the bug, schedule it in the project plan, assign developers to work on it, and communicate with other stakeholders about what’s going on.

As you can see, bug reproduction steps aren’t just for developers.

They give any interested party a chance to see what is going wrong and why.

As such, they’re a great way to make sure that everyone is on the same page and that the process of resolving bugs goes smoothly.

Why do non-reproducible bugs appear

While the majority of bugs are easy to reproduce, there are also some non-reproducible ones that only surface at a particular time or in very specific circumstances.

It’s possible to reproduce non-reproducible bugs, but it can take a long time and a great deal of effort from the developers.

John Kotzian, a test architect at Applause, explains the situation like this:

When [the bug] is not easily reproducible, development teams will need to analyze the bug’s potential impact versus the cost in the time it will take to determine what actually happened.

He goes on to say that, in most cases, such bugs are closed.

However, he also acknowledges that closing them isn’t always the best decision because mitigating issues as they’re discovered can be less expensive than addressing them once users have begun experiencing problems or the brand reputation has been affected.

Therefore, non-reproducible bugs shouldn’t be closed without a proper investigation.

But before we get into that, let’s see a few examples of non-reproducible bugs to better understand what they are and why they’re hard to find.

Vaishali Desarda, a lead QA engineer at Huma, lists several reasons why non-reproducible bugs occur. They include the following:

• A device’s default language may be different from the one that an app supports.
• A third-party library might not support older versions of the operating system.
• A device could be running low on memory, while many apps are open at once.
• An analytics library might crash the app whenever the user switches between apps.

It’s clear that these bugs are often very technical in nature and appear only in specific conditions.

This makes them hard to identify and reproduce because even expert testers usually won’t have this kind of information.

Therefore, developers may need to turn to tools that’ll help them to reproduce bugs of this type— or at least their symptoms.

For that purpose, many bug and crash reporting tools, such as our own Shake, have black box components similar to those found on airplanes.

Shake’s black box records a detailed environment profile 60 seconds before the tester has submitted a bug, including data such as CPU usage, memory usage, network, and device orientation, to name just a few.

Although in many cases this information may not be enough to recreate and fix a non-reproducible bug, it’ll give developers a better chance of success if they have something to work with.

How to reproduce an app bug

If you’re lucky enough to have a bug that’s reproducible, then the next step is to try to get it to occur again.

However, before beginning that process, there are several factors testers should consider for a successful reproduction attempt, one of which is gathering necessary information such as the following:

• Who first detected the bug—was it a tester or an end-user?
• What was the environment in which it was found?
• What actions were performed that triggered the bug?
• Was there an error message?
• Was the testing conducted in production or during staging?
• What was happening around the time the bug occurred?

Taking these factors into account beforehand can help testers to better understand the nature of the issue and, therefore, provide more valuable information to the developers.

Michael Wagstaff, the founder of Spectora, sums it up nicely when he states:

There are infinite combinations of browsers, devices, settings, and versions when it comes to software. This makes it really hard for us to track down bugs! Usually, what you’re experiencing is only affecting 1% or less of our users because of some combination of things we haven’t anticipated or tested.

It’s also worth mentioning that, at this phase, testers should ensure that all cache and cookies are cleared because these can create inconsistencies and make it difficult to reproduce the bug.

Once all this information has been gathered, and the cache has been deleted, testers should create steps to reproduce that can be used to recreate the bug in any environment and by everyone involved in the process.

But more about that in the next section.

How to write the steps to reproduce

To create a list of steps to reproduce, testers must narrate precisely what they did when the bug occurred.

This will allow anyone who handles or sees the bug report to follow these instructions and be able to reproduce the problem without assistance from the original tester.

Or how Claire Bertrand illustrates it perfectly:

The steps to reproduce is the path any other user has to follow to experience the same bug you did.

There are many methods to write these steps, but the most important thing is to write them in a way that can be easily followed by anyone.

The goal here isn’t just to create a bug report that communicates the problem clearly but also one that can be understood by everyone, not just people with technical expertise.

This means avoiding jargon, acronyms, and other forms of shorthand and instead using simple language with clear instructions on how to reproduce the bug.

Another thing to keep in mind is that the steps should be detailed enough to clearly communicate how to reproduce the bug, but not so detailed that they are more than necessary, as the example below shows.

This is important because it’ll help testers avoid including unimportant details that may confuse or distract from the main issue at hand.

The opposite of being too detailed—giving developers too few details—is just as problematic, leaving them confused about what to do next, as you can see in the following picture.

A good rule of thumb here is to also include screenshots and videos (if possible), which can provide additional context or help illustrate the bug.

This’ll help testers avoid leaving out important details and make the process of reproducing the bug much easier for developers to follow.

All things considered, steps to reproduce make the reporting and fixing of bugs as smooth a process as possible, which makes them one of the most important parts of any good bug report.

Conclusion

When a tester reports that an app has a bug, the developer needs more information than just “it doesn’t work.”

A clear explanation of how an error occurred can help pinpoint where and why the bug arose and thus allow for faster fixes.

In this article, we discussed what reproducing a bug means and why it’s so important.

We also covered some tips on how to get the most out of it.

Hopefully, this has given you a basic understanding of the bug reproduction process and made it easier for you to create your own examples.