The importance of manual app testing

There are millions of mobile apps on Earth today.

You can find one for any purpose you can think of. Therefore, if you want to compete for users, you need to put out a quality product.

Testing your apps is a must, but will you choose the automated or the manual way?

Although automated testing is increasingly popular, there’s a case to be made for the importance of manual app testing.

In this article, we’ll provide you with the reasons why you should consider manual app testing over automated.

Performing tests that can not be automated

Automated testing is becoming increasingly popular in the software industry, which is unsurprising.

Compared to manual testing, automated testing has the advantages of speed, saving time and resources on repeatable tasks, and increased test coverage.

So, you might wonder why we need manual testing at all if automation is so great.

Well, for starters, not all tests can be automated, and if they can, it doesn’t always make sense to automate them.

Let’s see what the data says. Kobiton, a mobile testing platform, released The State of Software Test Automation in 2022 report, in which they surveyed many software teams and testers.

Their data shows that almost all testers use some automation in their work. However, a very small percentage of them use automation for most testing.

In other words, although testing teams embrace automated app testing, they still test apps manually.

That indicates that manual testing is, for one reason or another, in some cases preferable to automated testing—testers wouldn’t use it if it wasn’t.

For instance, developer Tommy Wyher’s piece on DZone describes an example of an iPad app that relies on tap gestures.

Having a human tester test the app by tapping their finger on the app’s interface is much simpler and makes much more sense than creating an automated test that mimics human gestures.

By the time you find a resource to write an automated test, a manual test could be completed with bugs already fixed.

Simply put, it’s more logical to perform certain types of app tests manually, like a smoke test, for example.

A smoke test is like a pre-test. You test basic functions to see if the app is ready for more rigorous types of tests, like functional testing, as you can see in the diagram above.

It can be automated, but most companies prefer to do it manually since it involves testing only a few fundamental elements of the app and doesn’t take much time.

And even if you automate a smoke test or any other type of test, as Dayana Mayfield, SaaS copywriter and SEO strategist, explains, a human still needs to supervise the process.

In other words, achieving 100% automation in mobile app testing is unrealistic.

There will still be some parts of the app that need to be tested manually, whether because it takes less time and money to do it, or because an automated test isn’t available, or simply because someone needs to check whether the computer made any mistakes during the automated testing.

Testing the app from a human perspective

When your mobile app gets into the users’ hands, their experience with it will determine whether they’ll keep using it or abandon it.

That’s why testers should put themselves in the users’ shoes when assessing the app.

Manual app testing allows that perspective for a simple reason—humans are conducting the testing process.

On the other hand, you can’t test an app from a human perspective using automation.

If you want to see the app the way your users will see it, you need to use it yourself and have the complete user experience.

And user experience (UX) is crucial for the app’s success. For example, NRF and IBM conducted a study focused on Generation Z and how they interact with brands.

One of their findings was that 62% of respondents said they wouldn’t use apps that are hard to navigate or load too slowly.

In other words, many of them will abandon an app that doesn’t provide a good user experience.

That’s where human testers can be very beneficial. They can interact with the app like users would, report on their first impressions, notice if something is visually off, etc.

For instance, automated testing might not pick up on the overlapping text, which was an issue TripAdvisor had to deal with.

With manual testing, there’s no doubt that a human will detect that bug within seconds of looking at the screen.

Also, the visual appeal of an app is a significant factor in its success.

Is it easy on the eyes? How do the colors in the interface complement each other?

Of course, aesthetics are partly subjective, but the human eye can assess if something is visually unappealing.

How would you make an automated test that would determine if the app is ugly?

For example, take a look at the VaporCalc interface below.

Now, testers can like some of the visual elements in it or not, although their instincts would most likely tell them that most users won’t like the interface.

However, regardless of the testers’ preferences or instincts, they’ll at least immediately see that some of the text is hard to read due to its color, which can negatively impact the user experience.

That’s why it’s important to test the app manually—you can’t automate instinct, feeling, or excitement when using an app.

And those elements can be vital for the success of an app.

Only manual testing can uncover hidden bugs

Automated app testing can be very efficient in detecting bugs. It’s a fast, reliable, and repeatable process that companies love to use for its many benefits.

But there’s a catch when it comes to automated testing.

To perform an automated app test, you need a test case, a pre-written sequence of actions that aims to validate if the app is functioning as intended.

Below, you can see an example of some test cases.

As you can see, it details what the test is, how to perform it, the expected result, and the outcome based on failing or meeting the expected result.

You also need a test script, a program with individual instructions on how to test a test case.

So, although automated testing can be very efficient and thorough, it’s as efficient and thorough only if a human writes detailed test cases and test scripts.

As Serhii Voloshyn, a QA engineer, points out, there’s no place for the unforeseen in automated testing.

With automated testing, we can make sure that what we expect actually happens. That is, automatic tests can evaluate only what is written in advance.

That ability to uncover a bug no one anticipated and therefore didn’t write an automated test for is a significant advantage of manual app testing.

With manual testing, testers can detect a bug while looking for something else. They can also think out of the box on purpose and explore the software like an automated test just can’t.

That way, testers can detect significant bugs that not even the most experienced developers could predict.

When that happens, testers need a great tool for reporting bugs, which allows them to send a comprehensive bug report to developers quickly.

Shake is a great tool for that.

It automatically creates a bug report with all the crucial data attached to it.

Here is just some of the information Shake attaches to a report.

In addition to that, you can attach custom data to your tickets and get as detailed a bug report as you prefer.

With a tool like Shake, testers can ensure that compiling bug reports doesn’t take up much of their time, which they can then invest in exploring the software and uncovering hidden bugs.

Testers can discover bugs in test scripts

Just like human testers can discover bugs where automated tests can’t, they can also find bugs in programs created for detecting bugs.

Test scripts are programs, and like any other software, they are created with code. And, as you know, code can contain errors.

In other words, code that makes a test script isn’t immune to bugs.

As Sofía Palamarchuk, the co-founder and CEO of Apptim, puts it, the test code is still code and shares the characteristics of any other code.

If the test script contains bugs, that can be a significant problem.

A test script like that can fail to test what it’s intended, generating inaccurate results.

It can also produce false positives and false negatives.

As you can see in the table above, a false positive means that the result appears to be positive, while in actuality is negative.

Simply put, that means the test detects a bug in the app where there isn’t one.

That can cause significant problems for companies and users of software products.

For instance, in 2017, Webroot’s security software identified vital parts of Windows as malicious and put them in quarantine.

The result? Machines that had Webroot installed and used Windows as their operating system were inoperative.

In the context of mobile app testing, false positives can cause a lot of time-wasting while developers are trying to find and fix a bug in the app, while in reality, the bug is in the test script.

On the other hand, a false negative can also have negative consequences because that means that a bug in the software slipped by the test and remained undetected.

Therefore, a company can ship the app to customers and only later learn that an error wasn’t detected because of a bug in the test script that produced a false negative result.

That’s why manual testing is effective.

Sure, a human can make a mistake or two while testing, but if the mistake is in the code of an automated test script, a lot of time can pass before it’s detected and the consequences are fixed.

Manual testing is cheaper for some projects

Some projects can benefit greatly from automated testing and save money in the long run.

On the other hand, there are also projects for which it doesn’t make sense to pay for all the costs of automated testing.

It goes without saying that automated testing has its advantages, mainly the possibility to repeatedly run test after test, which speeds up the process and saves time.

However, that only stands for long-term projects. In smaller projects that don’t take long, the expense of automation outweighs the benefits of saved hours.

Here’s how Michal Kowalkowski, co-founder and CEO of No Spoilers, explains that situation:

In other words, for automated testing, you need specialized software, someone to set it up, write scripts, maintain them and rewrite them as necessary, ensure that they run properly, etc.

On the other hand, with manual testing, a tester can check how the feature works and give feedback.

The point is that automated testing requires more initial investment, so if companies want that investment to pay off, they need bigger projects.

If they don’t have projects on a larger scale, it doesn’t make financial sense to invest in professional software and to pay for a test automation engineer’s salary, which is, according to Glassdoor, more than 100.000 a year on average.

Therefore, manual testing is cheaper for smaller projects.

As UX writer Jane Oriel explains, the resources that automation requires before it can even begin are a significant obstacle for those types of projects.

Because of the necessary investment in time and money before automated testing can begin, there is little to be gained by going through this process for short test runs.

To sum up, you should be careful when it comes to the allure of automated app testing.

Automation can be powerful and beneficial, but if it takes more time and money to set it up than to finish the project you’re working on, it’s better to take the manual route.

Conclusion

While automated testing is an important part of app development, it cannot always replace manual testing.

Manual testing brings some unique benefits to the quality assurance process that you shouldn’t overlook.

We hope that, after reading this article, you know what those are.

Manual testing isn’t going anywhere soon, and you should embrace the benefits it can bring you.