7 steps for an effective bug bash

A bug bash is an organization-wide effort to try and find as many software bugs as possible.

Commonly held before a major product or feature release, a bug bash is an effective method to ensure a happy user experience.

After all, the fewer bugs in your software, the more satisfied your customers are.

However, organizing the event can be a challenge. It isn’t an activity you should do on a whim.

That’s why we’ve written this article—as a kind of cheat sheet for hosting the event. Continue reading to discover the seven steps necessary for an effective bug bash.

Schedule the bug bash

To start your bug bash, the first step is determining a date, time, and place for it. Although this might sound like a no-brainer, it’s actually fairly challenging.

Bug bashes are typically company-wide events, meaning anyone from your organization is welcome. As a result, it is likely that you will be hosting a large number of people.

Given the number of participants, you’ll need a large enough room and a convenient time slot when most of your colleagues are available. Both tasks require some logistical prowess.

The room where you hold the event should have the following characteristics:

The room should be large enough to accommodate all attendees so that the group doesn’t need to split up.

You want participants to be comfortable, so try to provide various types of chairs—just ensure they’re ergonomic.

A projector is also a must-have, as you’ll be showing attendees testing scenarios and bug examples.

Finally, double-check there are enough power sockets for everyone, and ensure the internet connection is strong enough.

After you’ve acquired the room, you need to set a time that suits everyone. For this, your best bet is a scheduling tool.

For example, with SavvyCal, you can overlay your own calendar with those of your coworkers and view everyone’s availability simultaneously.

Here’s the tool in action:

The red and purple indicate two employees, meaning the calendar above depicts two schedules at once.

Apply this feature to your prospective participants’ calendars, and you’ll easily schedule a time when most can attend your bug bash.

Send out invitations for the bug bash

Once you’ve handled these logistics, you can recruit your bug hunters. In other words, it’s time to send out invitations for the bug bash.

These invitations are best distributed via email, as instant messages are easily missed.

Besides, there are quite a few details to disclose, and emails are helpful for communicating larger quantities of information.

Ideally, your bug bash invitation would contain the following:

The date, time, and location should be clearly stated, so the invitees can immediately verify whether they can attend.

Also significant is the testing scope, as it outlines the focus area of the bug bash. For example, if you’re testing only the mobile app, employees who worked on it are more likely to attend.

It’s worth attaching a list of known defects as well, as this should reduce duplicate bug reports.

Most bug bashes demand prerequisites (i.e., specific devices or applications), so ensure participants know about those ahead of time.

A smooth start is also crucial. To facilitate this, provide access to the systems being tested before the event begins.

Finally, end the invitation by hinting at possible prizes—this incentivizes your coworkers to participate.

Moe Ali also recommends mentioning incentives:

By offering possible rewards, or even free food and drinks, your invitees automatically have an extra reason to attend the event.

Ali is also correct regarding timing—send the invitations one week in advance.

That way, everyone can realistically assess if they can attend and consequently have plenty of time to prepare. Distributing the invitations on time can make all the difference.

Assign roles for the bug bash

After you’ve sent the invitations, you’ll soon know exactly how many of your colleagues will attend the bug bash. With that information, you’re ready to assign roles for the event.

Although the principal actors are your testers—the participants actively looking for bugs—there are also several other roles necessary for a successful bug bash.

The visual below provides a brief overview:

The bug-master (or the moderator) is the chief organizer for the entire event.

They handle logistics such as room, chairs, food, and drinks, as well as preparing testing scenarios for the participants.

This position is also tasked with ensuring the testers focus on bug-finding and providing support whenever necessary.

The testers are your invitees—the individuals you sent that email to. They can be anyone from the company, and all departments, even non-technical ones, are welcome to participate.

Finally, the stager prepares the testing environment. Their job is to provide testing devices, tools, and bug report templates. They’re also the person to talk to for any access issues.

Even though the testers are arguably the most important characters, the bug bash couldn’t be conducted without the other two roles.

When choosing team members for the bug-master and stager positions, it’s best to recruit someone from your QA team.

William Jeffries, an experienced software developer, explains why:

If you have some full-time QA people or people who specialize in QA, involving them in a leadership role and organizing can be really helpful because there’s actually a lot of nuance and subtlety that goes into testing things properly.

Your QA employees have the most experience with bug-hunting, after all, and will be able to support your participants best.

So when assigning roles for the bug bash, keep them in mind for the non-testing positions.

Split participants up into rival teams

With all the roles delegated, you can now divide participants into rival teams. By arranging groups, you’ll encourage some friendly competition, and make the event more dynamic.

If you assign them to distinct competing teams, your testers will be more motivated to find as many bugs as possible. After all, who doesn’t want to win in a contest?

MobilizeOn also took this approach, dividing its bug bash into three teams:

The participants even named their teams: Vikings, Delta, and Tango.

It’s a good idea to suggest the same to your attendees, as it will bring them closer to their team members and strengthen their team spirit.

Here’s team Vikings:

There are both female and male members, which likely worked in their favor.

Diverse teams usually perform better than homogenous ones, as they combine various perspectives and knowledge.

This, in turn, leads to them finding more bugs as each representative brings something else to the table.

Therefore, if possible, arrange your teams so they contain both sexes, different races, various company departments, etc. They’ll likely achieve better results!

However, to really inspire everyone to do their best, try advertising a prize for the best team.

Communicate that whoever has the most impressive result collectively will be rewarded for their efforts.

For example, Kantan designed a congratulatory plaque for their best-performing team:

By offering recognition to the most productive team, you’ll stoke the rivalry between groups and motivate everyone to truly give it their all.

Explain how bugs should be reported

Now that the participants have been divided into teams, it’s a good idea to explain how bugs should be reported.

Since bug bashes are typically organization-wide events, there will likely be some non-technical colleagues who have never logged a bug.

For a bug report to be helpful, it should contain these essential datasets:

The title or bug ID serves for identification and should briefly describe the error.

Next is the environment where the bug appeared—OS, device, browser, app version, etc.

Following this, there should be a description, which is a prolonged version of the title, simply providing more details.

The steps to reproduce are also extremely important, as they direct how to recreate the bug.

Then, the expected vs. actual result explains how the software currently functions as opposed to how it should be functioning.

Near the end comes severity or priority, indicating how severely the software is affected.

Finally, there should always be visual proof of the bug through screenshots and screen recordings.

This is a lot of information to write and can be time-consuming. As such, it’s worth investing in a bug-reporting tool like Shake.

With Shake, simply shake your mobile device, and a detailed bug report will immediately be sent to the developers.

Here’s a sample bug report:

The report contains everything your developers will need: environment details, logs, screenshots and screen recordings, steps to reproduce the error, and more.

Distribute this tool, and your participants will easily deliver comprehensive and thorough bug reports.

Let the bug bash begin

Now that your participants know how to report bugs—it’s time to begin the bug bash!

While attendees can explore the software independently, that approach won’t unearth many errors. For a productive bug bash, it’s best to specify a focus area.

The easiest method to achieve this is by providing testing scenarios. These scenarios offer direction, instructing how exactly to examine the software.

A well-written testing scenario contains the following:

At the top should be a title, briefly yet clearly describing the scenario.

Following that is a description of what to test, usually written as a user story.

So, for example: As a long-time customer, I want to view my accumulated points so that I can trade them in for rewards.

You should also specify testing pre-conditions (e.g., specific devices or browsers) to create the correct environment.

Next are the test steps, which precisely outline which exact actions to perform.

The test goal comes afterward—this describes how the software should react. In our previous example, the customer should see some overview of their points and corresponding rewards.

Finally, testing scenarios should be scalable and re-usable for another bug bash.

This approach is the most structured. However, it also requires the most time and effort.

If you’re short on those resources, you can also devise simpler, more straightforward tasks.

For example, here’s how Rise provides instructions:

Although fairly general, it still provides direction as to where to search for bugs.

Other similar, more straightforward tasks would be Customize your avatar as much as possible or Toggle between all languages.

Whatever approach you choose—general tasks or testing scenarios—a productive bug bash is guaranteed.

End the bug bash

Finally, once your allotted time slot ends, the bug bash comes to a close.

As the event organizer, you should thank everyone for participating in the voluntary event. To truly show your appreciation, it’s a good idea to award small prizes.

Consider devising specific categories, such as Most bugs found, First bug found, etc. Whoever achieves these results should be given a congratulatory award in recognition of a job well done.

Alternatively, you could gamify the event and distribute badges or tokens. Microsoft does this, by consistently offering commemorative badges to all its participants.

Here’s the badge from last year’s event:

The tradition is a long-standing convention, with Microsoft offering badges even in 2017:

Given the system’s popularity, it’s worth considering a similar arrangement yourself.

You don’t need to construct custom badges—even distributing stickers or keeping a leaderboard will do the job.

Once your participants have left the room, your QA team has their final post-bug bash task: triaging the defects.

The bugs found are only helpful once correctly sorted. You must separate major issues from minor ones, identify what needs to be solved immediately, and reject any invalid bugs.

Furthermore, once each bug is categorized, the most pressing problems should be distributed among your developers.

The visual below depicts this well:

Each error is assigned to a developer, who is then responsible for fixing it. However, this is only for major bugs—fixing minor bugs is postponed.

Once you’ve triaged the bugs, your bug bash has well and truly ended.

Conclusion

A bug bash is hugely beneficial as it helps rid your software of pesky bugs. However, there are several essential steps to ensure its effectiveness.

The first steps are scheduling the bug bash and sending out invitations.

Following this, you should assign event roles and split the participants into rival teams.

Then it’s time to explain how bugs should be reported. Once you’ve done that, the bug bash can begin.

Last but not least, the event should be concluded with a defect triage.

Follow these steps, and you’re sure to have a productive bug bash.