Build Verification Test is a set of tests run on every new build to verify that build is testable before it is released to test team for further testing. These test cases are core functionality test cases that ensure application is stable and can be tested thoroughly. Typically BVT process is automated. If BVT fails that build is again get assigned to developer for fix. In software testing, a Build Verification Test (BVT), also known as
Build Acceptance Test, is a set of tests run on each new build of a product to verify that the build is testable before the build is released into the hands of the test team. The build acceptance test is generally a short set of tests, which exercises the mainstream functionality of the application software. Any build that fails the build verification test is rejected, and testing continues on the previous build (provided there has been at least one build that has passed the acceptance test).
Here’s the definition we use: “A build acceptance test (sometimes also called build verification test a.k.a. BVT, smoke test, quick check, or the like) is a set of tests run on each new build of a product to verify that the build is testable before the build is released into the hands of the test team. The build acceptance test is generally a short set of tests, which exercises the mainstream functionality of the application. Any build that fails the build verification test is rejected, and testing continues on the previous build (provided there has been at least one build that has passed the acceptance test). So build acceptance tests are a type of regression testing that is done every time a new build is taken. Build acceptance tests are important because they let developers know right away if there is a serious problem with the build, and they save the test team wasted time and frustration.”
SMOKE TESTING:
1) Smoke testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested.
2) A smoke test is scripted–either using a written set of tests or an automated test
3) A Smoke test is designed to touch every part of the application in a cursory way. It’s is shallow and wide.
4) Smoke testing will be conducted to ensure whether the most crucial functions of a program work, but not bothering with finer details. (Such as build verification).
5) Smoke testing is normal health check up to a build of an application before taking it to testing in depth.When a build is received, a smoke test is run to ascertain if the build is stable and it can be considered for further testing.
Smoke testing can be done for testing the stability of any interim build. Smoke testing can be executed for platform qualification tests.
SANITY TESTING:
Once a new build is obtained with minor revisions, instead of doing a through regression, a sanity is performed so as to ascertain the build has indeed rectified the issues and no further issue has been introduced by the fixes. Its generally a subset of regression testing and a group of test cases are executed that are related with the changes made to the app. Generally, when multiple cycles of testing are executed, sanity testing may be done during the later cycles after through regression cycles.
Some BVT basics:
- It is a subset of tests that verify main functionalities.
- The BVT’s are typically run on daily builds and if the BVT fails the build is rejected and a new build is released after the fixes are done.
- The advantage of BVT is it saves the efforts of a test team to setup and test a build when major functionality is broken.
- Design BVTs carefully enough to cover basic functionality.
- Typically BVT should not run more than 30 minutes.
- BVT is a type of regression testing, done on each and every new build.
Tips for BVT success:1) Spend considerable time writing BVT test cases scripts.2) Log as much detailed info as possible to diagnose the BVT pass or fail result. This will help developer team to debug and quickly know the failure cause.3) Select stable test cases to include in BVT. For new features if new critical test case passes consistently on different configuration then promote this test case in your BVT suite. This will reduce the probability of frequent build failure due to new unstable modules and test cases.4) Automate BVT process as much as possible. Right from build release process to BVT result – automate everything.5) Have some penalties for breaking the build 
Some chocolates or team coffee party from developer who breaks the build will do.
Developers migh be doing the unit and intregration testing and not necessarily the BVT. BVT is most of the times done by test engineer. Once the build team deploys the test build on the test environments its the job of test engineer to perform BVT (sniff, sanity smoke and other testing) If you are able to test the applicaiton and are executing the test cases, its natural that the BVT has been passed. Else you wouldn’t have been able to test the applciation… It dosent mattter if the testing is done manually or is automated using any tool. If the build has got a new feature how can you automate it?
Following is the process that you might follow: Developer initiates a mail to build team (also marked to test team with the description of what to be tested in the new build) to make the build >> Build team makes the build and deploys it on test machines and replies all asking the test team to continue testing, else if build fails he replies so in the mail >> If the BVT fails tester replies to mail stating that the BVT failed along with the logs whatever are available, else continues testing.
Conclusion:
In real life investigating all the test case failures requires huge amount of discipline and noticeable amount of intellectual effort. Imagine thousands of test cases running on n different editions of your product. Every failure needs to be properly investigated and if applicable then bugs opened. Every single one of them. BVT is nothing but a set of regression test cases that are executed each time for new build. This is also called as smoke test. Build is not assigned to test team unless and until the BVT passes. BVT can be run by developer or tester and BVT result is communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT process is typically automated by writing scripts for test cases. Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long term builds. This saves significant time, cost, resources and after all no frustration of test team for incomplete build. Different teams have different practices across the Microsoft and even outside the Microsoft you should not try to adjust some specific methodology/process to your needs and blindly follow the process. You should think how every team member can win from BVT in your project. Let me know what testing process and build verification testing you have?
