Users are fickle.
Critical bugs such as crashes, freezing issues, slow load times, unintuitive navigation and privacy breaches may trigger the user to uninstall your app instantly.
Mobile apps are now an integral part of our daily micro-moments, with people spending an average of 3-4 hours per day. Mobiles apps play a key role for everyone in both professional and personal life. And remember, it’s a highly competitive market if your users uninstall your app they have plenty of options to choose from.
Obviously, you may consider fixing your problems after the release of the app but it can be damaging to your company’s prestige. It’s good to remember Richard Branson’s words “While a good reputation precedes you, a bad reputation will follow you for a long time.” Hence, it’s essential to have your applications tested in order to provide the best user experience. Mobiles testing plays a major role in building a mobile app to provide a smooth user experience and functionality.
In this blog, you will get to know about mobile testing challenges, types of testing and best practices.
Where does Mobile App Testing fits in Software Development Lifecycle?
The testing phase of the software development lifecycle (SDLC) is where you focus on investigation and discovery. Organizations use different software development approaches also referred as “Software Development Process Models”(e.g. Waterfall model, V-model, iterative model, RAD model, Agile model, etc.). Each process model follows a particular life cycle in order to produce a robust mobile app.
The traditional approach of testing focuses more on “Fix Defect Then Release”. Here testing is carried out after the development cycle is complete. While in the agile approach, the QA team works in collaboration with development team instead of being a separate unit. Thus, test planning is no longer a separate phase. They are involved in all aspects of the project, starting with analysis, requirements and design of every mobile app functionality.
The entire team determines the efforts required to develop and deliver a working component. Since agile methodology demands quick and frequent testing, automation testing plays a crucial role.
The project managers can detect problems very early. Due to the early feedback cycles, the cost of fixing the defects is much less as compared to fixing the same defects in the traditional approach.
The agile methodology can be extended to include continuous integration and continuous deployment (CI/CD). The CI tools such as Jenkins and TeamCity create a workflow to manage code and its regressions and then enabling efficient testing of those builds. With this sort of Agile approach, continuous testing can also happen every time code changes in the repository.
In agile method, it is easy to adopt automation testing. Your organization can focus on improving existing application which includes building new features, product enhancements, and bug fixing, while testing happens automatically and entirely in a managed manner. It provides an efficient way to improve the overall app quality, weed out the bugs instantly and without being dragged in the subsequent builds.
Another important thing is time to market. It can be improved with integrated development and testing flows. Any app’s time-to-market counts not only against competition but also when the revenue starts rolling in.
Challenges in Mobile App Testing
Mobile applications must provide a seamless experience across a wide range of devices and OS versions. QA engineers should be highly responsive to any changes to the hardware and software which may create issues for their users. Teams must account for everything from network carrier settings to battery charge levels. Fragmentation is particularly a challenge when some mobile users continue to use older versions of an OS or device even after new ones are introduced.
Testing every combination of device, OS and network settings creates a large number of test cases. This requires development teams to perform the work of sourcing and maintaining of a growing pool of mobile devices. These challenges represent major roadblocks for mobile app developers.
Multiple data connections
Most of the apps use mobile data connections (Edge, UMTs, 3G, 4G) and wifi (b, g, n). When users move around there’s a possibility that their connection type will change. Unfortunately, some carriers filter the web on their own will, which results in the devices being connected without actually having a connection with a specific service (messaging or calling through apps). The real world environment can’t be replicated and issues may occur even if connection API’s on mobile platforms have been developed. It’s important for QA team to test the bandwidth usage as not many carriers support unlimited data volumes.
Application complexity and third-party integrations
Many mobile applications require third-party integrations for analytics, crash reporting, SMS service. For seamless user experience and functionality these applications rely heavily on third-party integrations or hardware components. This makes test teams switch between multiple personas (or devices) throughout the course of the test scenario. These complex use cases inflate the amount of work required to adequately test the application.
Processing power and battery life
Mobile applications which include gaming or video streaming component can drain battery life very quickly. Users run lots of apps during the day and several processes are running on the background without even noticing us. This all requires CPU cycles which consume power and thus the batteries tend to dry. A dedicated mobile device lab is used to understand the impact of iterations in a mobile application. It matches any new code submitted by engineers and analyses any negative impact on how the app utilizes phone memory, how fast users can scroll through a feed, and battery consumption.
The Mobile Test Pyramid
Anyone who is involved in software testing knows the Mike Cohn’s test automation pyramid. The typical pyramid consists of three layers. At the top, there is the automated end-to-end testing layer (including the user interface tests), in the middle the automated integration testing layer and At the bottom, there is the automated unit-testing layer. Manual testing is not part of the test pyramid, hence it is shown as a cloud for additional testing work. Each layer indicates the number of tests that should be written within each stage and comes with different size.
When it comes to mobile app testing, typical pyramid structure is not applicable for mobile test automation. Unlike, web or desktop applications, mobile apps consists different devices, sensors and network variations which requires different set of test activities.
Test pyramid for mobile applications consists four layers including manual and automated steps. The biggest layer of the pyramid is manual testing and forms the strong foundation for every mobile app project, followed by end-to-end testing, beta testing and a top layer comprising unit testing. Unit tests and end to end tests have same colour and represent automation testing while beta tests and manual tests have some colour which represent manual testing. The beta-testing layer is new to the pyramid but essential to every mobile app project. Keeping the high expectations of the mobile users in mind requires this layer to be part of every mobile project to get early feedback from your mobile customers. You can either use a crowd testing approach for your beta testing or you can ask your colleagues to beta test early versions of your app to provide important feedback.
Unlike web applications, not every unit of mobile apps can be tested in an isolated manner. In some cases, different APIs, layers or systems may need to be faked or mocked in order to get the small unit to work. This is not efficient from a technical or economic point of view. However, it’s no excuse for not writing mobile unit tests at all. The business logic of an app must be tested at unit level.
The biggest change in this pyramid is that manual testing is part of it. Mobile testing requires lots of manual testing, and this can’t be replaced by test automation or any other tools yet. The testing team ought to test the various events which may occur when the application is being executed – Incoming calls, SMSs, low battery, alerts such as emails and roaming.
The end to end and unit tests layers can also be swapped as well as the beta and end to end layer. The amount of end to end tests and unit tests can differ from project to project and from app to app.
The top of the pyramid there are unit tests. Writing unit tests for mobile apps is not as easy as for backend or web applications. There are so many APIs and sensors that can be used by an app and it is really difficult and time consuming to mock all those interfaces to write efficient unit tests.
Types of Mobile App Testing
A unit test takes a small unit of the app, typically a method, isolates it from the remainder of the code, and verifies that it behaves as expected. Its goal is to check that each unit of functionality performs as expected so that errors don’t propagate throughout the app.
Each one small test is dedicated to a small feature. All of them are connected, however, so you get a kind of chain. If any part of this chain fails, the whole chain will fail as well.
Let’s admit that even 100% unit test coverage does not provide bug-free result because there may be some arrangement of data which is not covered in unit testing. Let’s see how to make unit tests more efficient which helps in regression testing and system testing.
If manual testers find bug then it means there’s unit test missing. When it comes to system testing, thousands or millions of the classes, modules, interfaces, and methods hit during single system test because they were untestable during unit testing.
During the system testing, there’s an initial touch point, where the code first gets used for the first time,e.g. webpage calls a web service after clicking the button. Now from that touch point, there will be a constant entry and exit of methods until the test is completed.
That means that any system test is really a collection of possible unit tests. But it would be tedious to trace down the code and then write all the unit tests that represent a system test.
To overcome this problem, you can create them as a part of manual testing so that your long-running regression tests can be reduced to set of unit tests.
If a system test runs and passes, then a batch of unit tests is available. That batch is what you’ll run going forward unless there are significant changes. And that batch of unit tests will run much faster than the system test that produced them.
On the other hand, if the system test fails, then that batch of unit tests becomes a great way to help developers pinpoint where the problem is. Whichever units have the problem won’t pass, and you’ll know which section of code caused the system test to fail.
Functional testing ensures whether the function of code is compliant with requirements or not and checks the overall functionality of an app or module. For example, it tests basic user interactions with the app such as launching the app, logging in, playing a song, checking an account balance and other straightforward user flows. It does not test internal parts of mobile app and focuses only on the output generation.
Functional testing for mobile apps protects you from risks which can undermine the authority and make you lose customer trust.
Functional tests are tests that confirm a given system function. For example, if I have a requirement “the system shall persist users object in the database”, the functional test could verify that requirement by starting the system, saving a user, stopping the system, starting it again, and verifying the user exists and has not changed.
Performance testing is the process of determining how a system responds under a particular workload or task. In general, performance testing tests the speed, stability, and scalability of an application. This is vital to providing not only a good but an exceptional, user experience.
Performance testing checks how a mobile app system responds under a particular workload or task. In general, performance testing tests the stability, speed and scalability of an application.
There are basically four kinds of performance testing:
Load Testing is a type of performance testing conducted to evaluate the behavior of a system at increasing workload.
Stress Testing a type of performance testing conducted to evaluate the behavior of a system at or beyond the limits of its anticipated workload.
Endurance Testing is a type of performance testing conducted to evaluate the behavior of a system when a significant workload is given continuously.
Spike Testing is a type of performance testing conducted to evaluate the behavior of a system when the load is suddenly and substantially increased.
Performance testing doesn’t only mean many users hitting the same thing at the same time.
There are various aspects of performance testing which you can opt to do manually.
It requires following scenarios for complete performance testing.
- Whether it works in the same seamless manner on a phone with low hardware, OS and memory configurations as it does on a phone with high memory and hardware or does it hang or get slow.
- How much battery does it consume? How much memory and CPU does it occupy?
- Does it get slow if used for a long time?
- Does it maintain a local database on the phone? If yes, does it get slow with an increase in size of the database?
- If it requires internet connection how well does it work with slow network connections like 2G or bad WIFI?
- Have few applications open in background and then try to run your application. Does it hang or get slow?
Mobile applications which send and receive sensitive information are tempting targets for man-in-the-middle (MITM) attacks where a correctly positioned attacker can view and manipulate traffic.
Mobile application security testing includes a range of different kinds of tools, including static analysis, dynamic analysis, and penetration testing. Each have a place in a solid mobile application security testing program, and when used correctly, can together find nearly any vulnerability that could be used against you. Using static code analysis throughout the SDLC, penetration testing before release and with each update, and using dynamic analysis to test the application in a runtime environment will help ensure a scalable, repeatable process for mobile application security testing.
In usability testing, real users check the features and functionality of the mobile app. The primary focus of this testing is on easy and quick use of an app, simple on-boarding and user’s satisfaction with the entire experience.
Users are given tasks in a test environment and encouraged to think aloud while trying to accomplish the tasks. This gives you, usability practitioners, information you need on how the user interface matches the natural human way of thinking and acting and highlights the features and processes to be improved.
In usability testing, these five elements require more focus: functionality, context, range of devices, data entry methods, and multimodality.
Context includes different habits of the user such as attitudinal, preferential and situational which provide user’s mobile experience. Context can be defined by analyzing all the habits of the user. There are some constraints you should consider in mobile context like screen sizes, location, network connectivity and users’ fingers.
Usability testing need to be performed collaboratively, keeping in mind the performance, security, localization, functionality, and accessibility aspects of the application.
Due to the diversity of mobile devices and platforms, compatibility testing for mobile apps is indispensable. Compatibility testing helps ensure complete customer satisfaction as it checks whether the application performs or operates as expected for all the intended users across multiple platforms.
The following practices in compatibility testing help covering maximum number devices.
Create the Device Compatibility Library: Take every device or model available in the market and structure the information of platform details, technology features supported by the device (audio/video formats, image, and document formats, etc.), hardware features included in the device, and network and other technology features supported by the device.
To cover maximum users in the region, shortlist the device list based in compliance with region or country’s peculiarities.
Divide all devices into two lists: fully compatible vs. partially compatible devices: Fully compatible devices support all technology features required to make all the application functionalities work seamlessly, while partially compatible devices may not support one or a few features and therefore cause error messages.
Run tests on fully compatible devices: When prioritizing testing, check for 100 percent apps functionality on select devices from this list.
Run tests on partially compatible devices to the extent possible: Try to perform testing on the latest and most widely used set of devices. Place initial focus on the functionality that might be influenced by unsupported features.
As the name indicates, regression testing checks whether such actions as new feature updates, patches or configuration changes hadn’t brought new regressions, or bugs, in both the functional and non-functional areas of a mobile application system. Regression testing confirms that any changes which are making by the development that is walking through to the improvement.
For example, many software as a service (SaaS) providers will regularly update their features or add new functionality to their offerings with each software update. To ensure their core product remains unaffected by new feature additions, these companies will perform regression testing.
End to End to testing
End-to-End Testing is a methodology used to test whether the flow of an application is performing as designed from start to finish. The purpose of carrying out end-to-end tests is to identify system dependencies and to ensure that the right information is passed between various system components and systems. The entire application is tested in a real-world scenario such as communicating with the database, network, hardware and other applications.
There are two different methods of performing end-to-end testing.
Horizontal end-to-end testing: This is the most common way of performing end-to-end testing. This can take place horizontally across multiple applications. For example, a web application of an e-commerce site will have account, inventory, and shipping sections to test.
Vertical end-to-end testing: This kind of end-to-end testing involves evaluating an application from start till end. Here, each individual level is verified and tested. This method is difficult to implement, especially in case of web applications that involve the large chunk of HTML codes, hence requires a proper validation.
Mobile App Testing Best Practices
Mobile Application Testing Tool Selection Criteria
- Cross team collaboration capabilities (Both QA and Dev can easily use the same tools)
- Always perform tool feasibility since mobile technologies and platforms are varied.
- Select tools that support both platform simulators and devices, because you can mix and match devices and simulators to optimize runs on different platforms.
- Look for automation in non functional areas like interruption, hardware scenarios like battery state changes etc.
- Always optimize on the platform support, in some cases, there could be a need for one or more tools to perform automation.
- Look for multiple devices support and versions support.
- Look for utility and reusable functions that would add value to automation.
- Integrated execution with test management tool is going to be important for tool success.
- Look for data-driven automation support as iterations in execution is going to increase coverage and ROI.
With the aim to meet the demands of an agile development process, there are plenty of testing tools which can help the team to test varied parameters of the mobile apps like behavior, performance, security etc. in the completely automated way. Some of these testing tools work competitively on native, hybrid and web applications.
Selendroid is a test automation framework which is used to test native and hybrid Android mobile applications. For scaling and parallel testing, testers can integrate it as a node. Selendroid supports all the android based hardware devices and contains built-in inspector which enables testers to simplify test case development. Also, it is fully compatible with the JSON Wire Protocol/Selenium 3 ready.
Frank iOS is an open source testing tool specifically for iOS which shows combined features of JSON and Cucumber. The main advantage is that there’s no need to change the code to run the tests. It provides “app inspector” tool called Symbiote to inspect the state of your native iOS application.
Calabash is a test automation framework which allows test engineers to write automated tests for acceptance tests. It’s a cross platform, supporting iOS and Android, native or hybrid apps. It is maintained by Xamarin which is a cloud-based testing tool. It also introduces the concept of Behavior Driven Development and works well with Ruby, Java and objectiveC.
Team collaboration is the key in agile testing
When testing process adopted early in the development process it helps developers to write code with minimal bugs. In this test-first approach, developers should know what tests will be run so the tests can be anticipated as part of construction. Team collaboration plays a key role in delivering faster apps with quality. Developers and testing team start working together to discuss the kinds of testing to be performed on the story before they start writing user stories. This will enable testing to help inform construction, even if the developer isn’t following formal, test-driven development practices.
Manage test coverage
Test coverage measures the amount of testing performed by a set of test. Wherever we can count things and can tell whether or not each of those things has been tested by some test, then we can measure coverage and is known as test coverage.
The basic coverage measure is where the ‘coverage item’ is whatever we have been able to count and see whether a test has exercised or used this item.
Test coverage formula:
There is danger in using a coverage measure. But, 100% coverage does not mean 100% tested. Coverage techniques measure only one dimension of a multi-dimensional concept. Two different test cases may achieve exactly the same coverage but the input data of one may find an error that the input data of the other doesn’t.
Benefits of code coverage measurement:
- It creates additional test cases to increase coverage
- It helps in finding areas of a program not exercised by a set of test cases
- It helps in determining a quantitative measure of code coverage, which indirectly measure the quality of the application or product.
Many times test engineers miss out on backend testing
Backend testing is a type of testing that checks the database layer in mobile application architecture. Data entered in the frontend will be stored in the backend and if backend fails it may cause data corruption, data loss and poor performance.
There are so many tests in a frontend which only hit a small area of a backend. Many flaws in database can not be discovered without direct testing. The database testing is no longer a black box to test engineers.
API testing puts all pieces together in your code
In mobile applications, data or logic can be retrieved from bunch of backend web APIs. Notably, overs the last two few years, APIs have turned out to be an important part for applications because they ease the software development. To leverage APIs with full extent, you need to include API testing in your testing plan.
All types of mobile applications whether they are native, hybrid or web will talk to its backend APIs via HTTP and REST principles using XML or JSON as data format. These APIs can be integrated by calling them directly from your application or indirectly via API backend. The APIs backend can be anything from a simple node.js or grails application to a complex SOA based on .NET or J2EE.
To increase the quality of the code, developers perform popular quality efforts like unit testing and code reviews after writing the code. For the better quality, make sure you consider to test the API level also. You can see this API testing as an integration test for your classes/objects/scripts/data-stores used to implement the API.
Restrict permissions and check all log files
Test engineers must test that the app is using only the required permissions, and no more. Most mobile users hesitate to install apps which have unclear permission requirements. For example, when the app requires access to the camera then it makes no sense to require permissions for SMS, contacts or any other personal identification. In this test scenario, test engineers need to check the errors, stack traces, log files, warnings by connecting the mobile device to a computer. Also, test engineers should check the log level before submitting the app to the app store.
Mobile App Testing using Emulators, Simulators and Real Devices
There are different strategies for mobile testing, but when it comes to simulators/ emulators and real devices, it’s not really an either/or approach. Each solution has its benefits and drawbacks, depending on the stage of an app’s lifecycle. Let’s see where and when to use emulators, simulators or real devices.
An emulator works as the real device and duplicates every aspect of the original device’s structure, both software and hardware. It basically simulates all of the hardware the real device uses, allowing the exact same app to run on it unmodified, and all of the software. The main advantages of emulators are that they are open source and hence cost effective.
There are some disadvantages like they are very slow. Also, a mobile device emulator doesn’t take into consideration factors like battery overheating/drainage or conflicts with other (default) apps.
On the other hand, simulator sets up a similar safe environment to the original real-life device’s OS, but it doesn’t attempt to simulate the real device’s hardware. So what you will see is the OS and the interface of the device you want to use, but you won’t experience all the problems the hardware might cause. Some apps may run a little differently and that’s the main reason why simulators aren’t very reliable.
Simulators and automated tests that predict the performance of hardware, but running tests on these simulators is not as reliable as tests conducted on a physical device. A simulator also can’t account for some hardware features – such as specific chip settings, processing power and device memory. It requires real devices for complete hardware and software testing.
Now the question is how to test on real devices with expanding device coverage?
One of the biggest challenges of mobile app testing is the enormous and growing number of devices to test. With mobile phones alone, there are different brands, models, screen sizes, hardware features, and OS versions available out there. With Android, testing is especially challenging because of the large number of devices on the market, as compared to iOS.
The smartphone market is buoyant, with growing number of devices every year. It is one of the biggest challenges of mobile app testing because these devices come with different OS versions, hardware features and screen sizes.
So, how do you approach device testing when testing on all devices can be prohibitively time consuming and expensive?
Understand your target user
It is very important to set boundaries by knowing your target user. With some subsequent research about the installed base of different devices and your target demographics, you might be able to narrow the scope of your testing.
If your app does not have a very targeted and specific audience, you can use more general data to help focus on the most widely used devices. Both Google and Apple provide data on the most commonly used OS versions and hardware. For example, top five OS versions currently in use are Nougat(28.5%), Marshmallow(28.1%), Lollipop(24.6%), Kitkat(12%) and Oreo(1.1%).
Create an app testing device matrix
For wider physical device testing, you need to create “Device coverage matrix” which eliminates testing every possible combination of device, screen resolution (pixel density) and OS.
Generally, pixel density and OS version are two characteristics which cause the vast majority of design bugs. And by covering the most common issues with the device coverage matrix, you can eliminate the most likely causes of user frustration.
Your matrix typically has three prioritized groups of devices: The most important devices we must test, the secondary devices that we will test, and the devices that are lower priority because they have the same OS version or pixel density as the other groups. You can name these groups Tier 1, Tier 2 and Tier 3.
To create an actual matrix, you need to focus on most commonly used Android versions and pixel densities.
Next, You need to combine both tables into a matrix and add the devices that are most common based on our previous research.
Based on above logic you can prioritize test list of Tier 1, Tier 2, and Tier 3 devices.
Above matrix can also help you in post launch. If a user encounters a bug, you can look at the matrix and find related devices to quickly reproduce, isolate, and fix the problem.
Device testing on a cloud platform
For mobile app developers and testers, delivering high-quality apps across all of the different device and OS combinations is a major effort – it’s time consuming, complicated, and expensive. And, as new devices continue to enter the market, developers are looking for an easier way to build and test across them.
Cloud device testing allows developers to upload their apps and test them on “the most commonly used mobile devices across a continually expanding fleet that includes the latest device/OS combination. There several tools available like Kobiton where testers can test on real devices without sacrificing any of the performance and features we need.
Testability in MVC, MVP and MVVM mobile architecture patterns
The user interface often contains a lot of cluttered code primarily because of the complicated logic it needs to handle. The presentation patterns are designed primarily with one objective in mind, reducing the complex code in the presentation layer and making the code in the user interface clean and manageable. Here you can evaluate and benchmark some architectural considerations for the MV(X) patterns for Android and iOS.
Model View Controller(MVC)
The Model View Controller (commonly known as MVC) framework helps you to build applications that are easier to test and maintain. The primary objective of the MVC design pattern is separation of concerns to facilitate testability. The Model View Controller design pattern enables you to isolate the concerns and makes your application’s code easier to test and maintain.
MVC does a great job of separating the model and view. Certainly the model can be easily tested because it’s not tied to anything and the view has nothing much to test at a unit testing level. The Controller has a few problems however.
Testability – The controller is tied so tightly to the Android APIs that it is difficult to unit test.
Modularity & Flexibility – The controllers are tightly coupled to the views. It might as well be an extension of the view. If we change the view, we have to go back and change the controller.
Maintenance – Over time, particularly in applications with anemic models, more and more code starts getting transferred into the controllers, making them bloated and brittle.
In comparison, the testability factors in MVC are considered as standards. Then we will try to compare the factors in MVP and MVVM with these standards to check if they are better or worse than MVC.
Model View Presenter(MVP)
In MVP, presenter refers back to the view due to which mocking of the view becomes easier which ultimately makes unit testing much easier.
In MVP, Size of test cases is the same as in MVC. When two applications have the same functionality, their size of test cases are the same.
Consumed time to run the test case: Some methods put in the presenter class after separating from the activity/fragment. For example, some UI tests were converted to the unit test. It decreases the total testing time because there were more local unit tests and less instrumented tests.
Ease to debug with breakpoints: Overall, it is the same as the MVC. But in MVP, developers need to navigate between two classes continuously because the breakpoints are in two class. As a result, it makes debugging less convenient.
If you would like to have automated unit test on the user interface of your application, the MVP design pattern is well suited and preferred over the traditional MVC design.
Model – View – ViewModel (MVVM)
The MVVM is a refinement of the MVC design and the ViewModel in MVVM is used to facilitation Presentation Separation. In the MVVM, View is completely isolated from the model and the logic is stored in the presenter.
Size of the test case: Local Unit Testing: As we can see from the UML class diagram, methods in presenter are less than in MVP. Therefore, there are less test cases in unit testing, Instrumented Testing: The same as the MVP
Consumed time to run test case: Based on the fact that there are less local unit test cases, the total consumed time is less than MVP.
Ease to debug with breakpoints: The data was in three files, except for the Fragment/View and the Presenter, layout files contain part of the model data. Currently. the layout xml file does not support the breakpoint. It is harder to debug the errors in layout files.
Measuring ROI for Mobile App Testing
The cost of test automation is higher than manual testing and should include any costs related to hardware, software and licenses, time for resources to produce scripts, and cost of the resources themselves.
The benefits of test automation must be calculated over a period of time for the software under test and take into account the reduced time to execute tests and the ability to test as frequently as the organization wishes. These figures should be compared to the costs of manually testing the same software.
Think of the cost of test automation vs. the cost of manual testing as follows:
Cost of test automation = price of hardware needed + price of tool required + time to develop test scripts + (time to maintain test scripts x number of times test scripts are executed) + (time to execute test scripts x number of times)
Cost of manual testing = time to develop test scripts/charters + (time to maintain test scripts/charters x number of times tests are executed) + (time to execute test scripts/charters x number of times)
After figuring the costs of test automation and manual testing, calculate ROI as follows:
ROI = (cost of manual testing – cost of test automation) / cost of test automation
Smaller, less complex applications that won’t take a considerable amount of time to test won’t require test automation because the upfront costs will outweigh the benefits gained. Also, keep in mind the types of testing you are looking to automate. Each of these different types will have a different ROI that needs to be taken into consideration.
We know that mobile app ecosystem is one of the biggest industry now. Due to highly competitive market, it’s obligatory maintain an excellent quality of any kind of application. Just a few clicks and swipes are enough for a user to praise or abandon an app. In this blog, we have provided an overview of every aspect of mobile app testing.