Thursday June 27, 2019
Smoke Testing aka Build Verification Testing is a boon for software development as it can be used as a verification method that can ensure that the product is stable and 100% functional.
In short, it’s the easiest method available to test all the functionalities of an app.
Let’s have a look at the Smoke Testing Process in detail.
Assume you order a book from Amazon. The moment you receive the parcel, the first thing you do is to check that the parcel is addressed to you, and then make sure parcel is intact and not torn.
Next, you open the parcel and see the book is what you ordered and also make sure it is new, not old.
You flip through the pages to make sure everything is good. Right? Well, you just completed your smoke test on the parcel.
So the same lines when you have a software product or mobile app, you do a round of basic checks to make sure that the software or app is ready for testing. This kind of testing which is done to make sure that the build is stable enough to undergo regression and functional testing is called Smoke Testing.
Just imagine a situation where you have a testing team consisting of 10 members.
Now once the build is ready, they all start testing. There can be a situation where, either the expected code changes are not there in the build or even that some major functionality is broken.
Unknown to this fact, all the 10 testers start to test the application and raise the defects for failures they find.
Now, at the end of the day, the development team may come back say, sorry this is not the right build or the QA team may stop the testing saying there are too many issues.
But again 10 people have already wasted their 8 hours for this which means 80 hours of productivity is lost. Also if the issue was found out earlier, the Dev team could have started working on it and resolved it earlier as well.
This is the reason why we need to have a smoke test done, before jumping into a full-fledged testing cycle.
Smoke Testing normally takes a maximum of 60 minutes and should be done for every new build, every new release even if that means doing it on a daily basis.
Once the product is stable, you can even think about automating the smoke tests and running it in a CI pipeline. In a CI/CD pipeline, a smoke test is very critical, because it will prevent an unstable or broken build from being pushed into production.
Now that we have got an idea about Smoke Testing, next would understand what are the scenarios that need to be included as part of Smoke Testing. Read on, to understand the different test cases and the reason why they are part of the Smoke Testing suite.
1) Build Verification: The first and foremost step in a smoke test is to verify the build, the build number, and environment availability. The entire testing effort would be wasted if the build is not right.
2)Account Creation: If your application involves a user creation, then you should try to create a new user and check if the system is successfully allowing you to do that. This is an important point which is missed many times because testers continue to use old login credentials without testing for a new user.
3) Log in Logout: If applicable in your SUT (System Under Test), as part of the smoke test you should try to successfully login with old and newly created credentials. Also, verify that you are able to successfully log out of the system without any errors.
4) Business Critical Features: This is very important. For all the major or business critical features, we should a simple test to ensure that the most commonly used functionalities are not broken.
5) Integration Scenarios: This is the most important part of a smoke test. The effectiveness of this part depends upon the understanding of the system integrations by the tester. For example, if the tester knows that there is some data that flows from system A to system B, then he must make it a point to check that as part of the smoke test (any 1 value). This is also done to ensure that the system does not break in any of these integration points.
6) Add/Edit/Delete: Data is always saved in a database. The three basic operations in a database are added the record, edit record and delete a record. So to ensure proper database connectivity, as part of the smoke test one must try to create, edit and delete an entry as applicable in the system you are testing.
7) Overall Navigations: Last part is overall navigation. That is one should go through the application try to touch upon the frequently used functionality and pages to ensure all the navigations are working as expected.
Usually, Smoke Testing is done manually to verify navigation is happening smoothly without any hindrance to functionality.
However, performing smoke testing can vary from company to company depending upon the requirement.
Once the software build is complete, it goes to the QA where critical functional test cases will be executed.
If it fails the software will be sent to the development team so that the necessary rectification can be done to the software.
After the rectification, the software will be smoke tested again and will be compared to the old build.
When the time is less and the new build is ready for deployment, automation can be used to smoke test.
Pre-recorded smoke test cases can be run against the build.
If the test is a failure, necessary rectification can be done and the software can be deployed within a short span of time.
Prepare – Set the preferred atmosphere for the test such as copying files, setting up a server, installing the license, etc.
Acquire Required Documents – Make sure that all the necessary files that are required to run the test is with you
Script – Make sure that you use a single script to run the tests. Once the script is executed make sure that the report has been saved so that if the build fails, it can be reported to the developers.
Ensure a Clean Environment – Stop the server, delete files or even empty the database tables, etc. Make sure that all the necessary steps have been taken to ensure that the test is run on a clean environment.
Usually QA lead is the one who performs smoke testing.
Once the major build of software has been done, it will be tested to find out it’s working well or not.
The entire QA team sits together and discuss the main features of the software and the smoke test will be done to find out its condition.
In short, a smoke test is done in a development atmosphere to make sure that the build is satisfying the requirement
|ID no:||Description||Steps||Expected Result||Actual Result||Status|
|1||To check login functionality||1. Launch the app
2. Go to the login page
3. Enter credentials
4. Click login
|Successful login||Login Successful||pass|
|2||To check video launch functionality||1. Go to the video page
2. Click the video
|Smooth playback of the video||Video player not popping up||Fail|
Sanity Testing is done to verify functionalities are working perfectly according to the requirements after the fix. Deep testing will not be done while performing sanity testing.
Even though Sanity testing and Smoke testing might sound similar there are differences
|Smoke Testing||Sanity Testing|
|To check critical functionalities||To check new functionalities are working or bugs are fixed|
|Used to the check stability of the system||Used to check rationality in order to move into deeper tests|
|Performed by both developers and testers||Restricted to testers|
|A form of acceptance testing||A form of regression testing|
|Build can be stable and unstable when smoke testing is performed||Relatively stable when sanity testing is performed|
|The entire application is tested||Critical components is tested|
If all the points are covered, then you can be assured that you have a good smoke test suite ready.
One thing we need to always keep in mind is that the smoke test should not take more than 60 minutes.
We need to make sure that we choose the test cases judiciously to cover the most critical functionalities and establish the overall stability of the build.
A tester should enforce a process whereby only Smoke Test passed builds are picked up for further testing and validation.