What are different types of software testing?
Note: Except the Shakeout testing and Unit testing which are respectively done by the CMT and Coder/Developer, all other testing are done by the QA Engineer (Tester).
1) Unit testing: It is a test to check the code whether it is properly working or not as per the requirement. It is done by the developers (Not testers).
2) Shakeout testing: This test is basically carried out to check the networking facility, database connectivity and the integration of modules. (It is done by the Configuration Team)
3) Smoke testing: It is an initial set of test to check whether the major functionalities are working or not and also to check the major breakdowns in the application. It is the preliminary test carried out by the SQA tester.
4) Functional testing: al It is a test to check whether each and every functionality of that application is working as per the requirement. It is major test where 80% of the tests are done. In this test, the Test Cases are ‘executed’.
5) Integration testing: It is a test to check whether all the modules are combined together or not and working successfully as specified in the requirement
6) Regression testing: When a functionality is added to an application, we need to make sure that the newly added functionality does not break the application. In order to make it sure, we perform a repeated testing which is called Regression Testing. We also do regression testing after the developers fix the bugs. See the video below for more understanding. (Courtesy of guru99.com).
7) System testing: Testing which is based on overall requirements specification and it covers all combined parts of a system. It is also a black box type of testing. System testing is black box testing, performed by the Test Team, and at the start of the system testing the complete system is configured in a controlled environment. System testing simulates real life scenarios that occur in a “simulated real life” test environment and test all functions of the system that are required in real life. Upon completion of integration testing, system testing is started. Before system testing, all unit and integration test results are reviewed by Software QA to ensure all problems have been resolved.
8) Load testing: It is a test to check the user’s response time of number of users using any one scenario (single business process) of the same application at the same time.
9) Stress testing: In this type of testing the application is tested against heavy load such as complex numerical values, large number of inputs, large number of queries etc. which checks for the stress/load the applications can withstand.
10) Performance testing: It is a test to check the user’s response time of number of users using multiple scenarios (multiple business process) of the same application at the same time.
11) User acceptance testing: In this type of testing, the software is handed over to the user in order to find out if the software meets the user expectations and works as it is expected to.
12) Black box testing: It is test where a tester performs testing without looking into the code. OR A testing method where the application under test is viewed as a black box and the internal behavior of the program is completely ignored. Testing occurs based upon the external specifications. Also known as behavioral testing, since only the external behavior of the program is evaluated and analyzed.
13) White box testing: It is a test where a tester looks into the code and performs the testing.
14) Alpha testing: In this type of testing, the users are invited at the development center where they use the application and the developers note every particular input or action carried out by the user. Any type of abnormal behavior of the system is noted and rectified by the developers.
15) Beta testing: In this type of testing, the software is distributed as a beta version to the users and users test the application at their sites. As the users explore the software, in case if any exception/defect occurs that is reported to the developers.
16) Acceptance testing: Is black box testing that gives the client/customer/project manager the opportunity to verify the system functionality and usability prior to the system being released to production. The acceptance test is the responsibility of the client/customer or project manager, however, it is conducted with the full support of the project team. The test team also works with the client/customer/project manager to develop the acceptance criteria.
17) Recovery/error testing: Is testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.
18) Security/penetration testing: Is testing how well the system is protected against unauthorized internal or external access, or willful damage. This type of testing usually requires sophisticated testing techniques.
19) Compatibility testing: Is testing how well software performs in a particular hardware, software, operating system, or network environment.
20) Comparison testing: Is testing that compares software weaknesses and strengths to those of competitors’ products.
21) Incremental testing: After unit testing is completed, developer performs integration testing. It is the process of verifying the interfaces and interaction between modules. While integrating, there are lots of techniques used by developers and one of them is the incremental approach. In Incremental integration testing, the developers integrate the modules one by one using stubs or drivers to uncover the defects. This approach is known as incremental integration testing. To the contrary, big bang is one other integration testing technique, where all the modules are integrated in one shot.
22) End-to-end testing: End-to-end testing is a technique used to test whether the flow of an application right from start to finish is behaving as expected. The purpose of performing end-to-end testing is to identify system dependencies and to ensure that the data integrity is maintained between various system components and systems. The entire application is tested for critical functionalities such as communicating with the other systems, interfaces, database, network, and other applications.
23) Sanity testing: Sanity testing, a software testing technique performed by the test team for some basic tests. The aim of basic test is to be conducted whenever a new build is received for testing. The terminologies such as Smoke Test or Build Verification Test or Basic Acceptance Test or Sanity Test are interchangeably used, however, each one of them is used under a slightly different scenario. Sanity test is usually unscripted, helps to identify the dependent missing functionalities. It is used to determine if the section of the application is still working after a minor change. Sanity testing can be narrow and deep. Sanity test is a narrow regression test that focuses on one or a few areas of functionality.
24) Usability testing: Usability testing is a way to see how easy to use something is by testing it with real users. Users are asked to complete tasks, typically while they are being observed by a researcher, to see where they encounter problems and experience confusion.
25) Install/uninstall testing: Installation Testing: It is performed to verify if the software has been installed with all the necessary components and the application is working as expected. This is very important as installation would be the first user interaction with the end users. Companies launch Beta Version just to ensure smoother transition to the actual product. Uninstallation Testing: Uninstallation testing is performed to verify if all the components of the application is removed during the process or NOT. All the files related to the application along with its folder structure have to be removed upon successful uninstallation. Post Uninstallation System should be able to go back to the stable state.
26) Exploratory testing, ad-hoc testing: Exploratory testing is a hands-on approach in which testers are involved in minimum planning and maximum test execution. The planning involves the creation of a test charter, a short declaration of the scope of a short (1 to 2 hour) time-boxed test effort, the objectives and possible approaches to be used.
27) Mutation testing: Mutation Testing is a type of software testing where we mutate (change) certain statements in the source code and check if the test cases are able to find the errors. It is a type of white box testing which is mainly used for unit testing. The changes in mutant program are kept extremely small, so it does not affect the overall objective of the program. The goal of Mutation Testing is to assess the quality of the test cases which should be robust enough to fail mutant code. This method is also called as Fault based testing strategy as it involves creating fault in the program
What is Negative Testing?
Testing the system or application using negative data is called negative testing, for example, testing password entering 6 characters where it should be 8 characters should display a message.
When we test an application by putting negative values (instead of actual values), then the system should not allow the other values rather than the actual value. The system should give an message that the value is not correct. This is called negative testing.
Another example is, if a user tries to type a letter in a numeric field, the correct behavior in this case would be to display the “Incorrect data type, please enter a number” message. The purpose of negative testing is to detect such situations and prevent applications from crashing. Also, negative testing helps you improve the quality of your application and find its weak points. (source: Jerry Ruban)
What is a Test Plan?
A Test Plan is a document describing the scope, approach, resources, and schedule of intended testing activities. It identifies test items, the features to be tested, the testing tasks and who will do each task (roles and responsibilities) and any risks and its solutions.
A Test Plan includes Heading, Revision History, Table of Contents, Introduction, Scope, Approach, Overview, different types of testing that will be carried out, what software and hardware will be required, issues, risks, assumptions and sign off section.
What is a Test Case? What does it include?
A Test Case is a document that describes step by step process how to test the application. A Test Case includes Test Case ID, Steps Description, Expected Output, Actual Output, Pass/Fail, Remarks
What document did you refer to write the Test Cases?
Requirement document. (NOTE: It can also be Use Cases, or Design Document)
Can you tell me what a Use Case is?
A use case is a document that describes the user action and system response for a particular functionality.
What is Software Development Life Cycle?
2. Design phase
3. Coding (programming)
5. Release (Production)
6. Maintenance (Support)
What is Business Requirement Document (BRD)?
It is a document that describes the details of the application functionalities which is required by the user. This document is written by the Business Analysts.
What is Software Testing Life Cycle (STLC)?
The testing of software has its own life cycle. It starts with study and analyzing the requirements. Here is the software testing life cycle:
2. Test Planning
3. Writing Test Cases
4. Review the Test Cases
5. Executing the Test Cases
6. Bug logging and tracking
7. Close or Reopen bugs
What is Business Design Document?
It is the document which describes the application functionalities of the user in detail. This document is the further details of the Business Requirement Document.
What is a Module?
A ‘Module’ is a software component that has a specific task. It can be a ‘link’ which can go inside to its component detail.
What is meant by Walk-thru meeting?
Before start working in a module and/or after accomplishing the testing of a module, the tester calls a meeting to disseminate his findings or to share his queries to other tester or leads of the company working on the same application that is called the Walk-thru meeting.
What is Build?
When each of the different modules of software is prepared, they are put in a single folder by the Configuration Management Team (CMT) and it is called the ‘Build’.
What is meant by the Build Deployment?
When the Build so prepared by the CMT is sent to different Test Environments, it is called the Build Deployment.
What is Test Strategy?
A test strategy is an outline that describes the testing portion of the software development cycle. It is created to inform project managers, testers, and developers about some key issues of the testing process. This includes the testing objective, methods of testing new functions, total time and resources required for the project, and the testing environment.
The test strategy describes how the product risks of the stakeholders are mitigated at the test-level, which types of test are to be performed, and which entry and exit criteria apply. (source: Wikipedia)
The test strategy is created based on development design documents.. It is written by the Test Manager or Lead.
The following are some of the components that the Test Strategy includes:
1 Test Levels.
2 Roles and Responsibilities.
3 Environment Requirements.
4 Testing Tools.
5 Risks and Mitigation.
6 Test Schedule.
7 Regression Test Approach.
8 Test Groups.
9 Test Priorities.
10 Test Status Collections and Reporting.
11 Test Records Maintenance.
12 Requirements traceability matrix.
13 Test Summary
Are Test Plan and Test Strategy same type of document?
No. They are different documents. Test Plan is a document that collects and organizes test cases by functional areas and/or types of testing in a form that can be presented to the other teams and/or customer where as the Test Strategy is the documented approach to testing. Test Plan is prepared by the tester whereas the Test Strategy is prepared by the QA Manager or QA lead.
Both are important pieces of Quality Assurance processes since they help communicate the test approach scope and ensure test coverage while improving the efficiency of the testing effort.
What does the Test Strategy include?
It includes introduction, scope, resource and schedule for test activities, acceptance criteria, test environment, test tools, test priorities, test planning, executing a test pass and types of test to be performed.
What is the difference between Load Testing and Performance Testing?
Basically Load, Stress and Performance Testing are the same. However, Load testing is the test to check the users’ response time of number of users of any one scenario of the application whereas Performance Testing is the test to check the user response time for multiple scenario of the same application.
What is SQL?
SQL stands for Structured Query Language.
What is Database?
It is collection of logically related data designed in a tabular form to meet the information needs of one or more users.
What is Change Control (OR Change Request)?
It is a document that describes the additional functionalities that are added after the Business Requirement Document is signed off. It can be updated in the old business requirement document or it can be a separate document.
What is Backend Testing?
It is a test to check whether the data displayed in the GUI front end report format matches with the particular data in the original database.
What is XML?
-XML stands for EXtensible Markup Language.
-XML is a markup language much like HTML.
-XML was designed to describe data.
-XML tags are not predefined and we must define our own tags.
-XML uses a Document Type Definition (DTD) or an XML Schema to describe data.
-XML with a DTD or XML Schema is designed to be self-descriptive.
-XML is a W3C Recommendation.
How do you make sure that it is quality software?
There is a certain process how the quality of software is guaranteed (ensured). If is defined by the ‘exit criteria’. (What it means is, a QA Manager writes a document called Test Strategy. This Test Strategy defines the ‘exit criteria’.) Normally in practice, for a quality software, there should no critical defects (0 critical), no high defect (0 high), no medium defect (0 medium) and may be 1 low defect
Let us say you have a web application to test. How do you go about testing it? What is the process?
First of all, I will look at the requirement documents (or design document in some companies). The requirement document will tell us what the functionalities in the application (software) are. Once I analyze the requirement documents (one module=one requirement document). After that, I will write test plans for each module (one module =one test plan). Then after the test plan is complete, I will write test cases (One module can have hundreds, even thousands test cases). Once the test cases are ready and the application is ready (or once the build is ready), then I will start testing. Before I start testing, however, I will make sure the test environments, test data and defect logging tools are in place. This is how I will go about testing an application.
What is a “bug?”
A bug is a bug is an error, flaw, mistake, failure, or fault in a computer code (program) that prevents it from behaving as intended (e.g., producing an incorrect result). (You can also add this: When the expected results (accordingly to the requirement documents) don’t match with the actual results (while testing), then it is considered a bug)
How would you ensure that you have covered 100% testing?
The testing coverage is defined by exit criteria (There is exit criteria and entry criteria in the Test Strategy). For example – only 2 low defects are acceptable.
What are all the basic elements in a defect report?
The basic elements in a defect report are: Defect ID, Header, Description, Defect Reported by, Date, Status, Version, Assigned to, Approved by, Module where the defect was found and so on.
What is the difference between verification and validation?
Verification: Verification is a process to ensure that the software that is made, matches the original design. It is to check whether you built the product right as per design.
Validation: Validation is a process to check whether the product design fits the client’s need. It checks whether you built the right thing. It checks whether it is designed properly.
How do you know it is sufficient testing?
Once the exit criteria meet the requirements, then the software is considered to be sufficiently tested.
What are the types of test cases that you write?
We write test cases for smoke testing, integration testing, functional testing, regression testing, load testing, stress testing, system testing and so on.
How to write Integration test cases?
I have never written separate Test Cases Integration Testing. Since Integration Testing is a test to check whether the all the modules are integrated together or not (meaning that when the developers compile all their module and make a build, all modules should be working when they are combined together and those modules when combined, should work as expected). If they are not integrated (combined) in a nice way, then the application breaks. Basically, when we do the functional testing, the integration testing is automatically done. This is my experience.
How to write Regression test cases? What are the criteria?
Regression test cases are also based on the requirement documents. They are written more into detail and with every release (build), the testers need to do regression testing. The criteria for regression testing are; there should be no major defects while we do our smoke test and functional testing.
Is there a format for a test case? Do you follow any methodology for numbering test cases?
Yes. It depends upon the company how the company has followed the numbering of test cases. However, normally, it is just a simple numbering in most of the time (see question 4 of qaquestions.com). But some companies may also relate this numbering to the requirement number. For example, if the requirement for Login is “REQ-LOG-001”, then we can number the test cases like REQ-LOG-001-001 and so on.
What is Test Harness?
In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitor its behavior and outputs. It has two main parts: the test execution engine and the test script repository.
How to write User Acceptance Test plan & test cases?
The way of writing Test Plan and Test Cases is the same in all the test phases. However, specifically for User Acceptance Testing, the testers use data nearly real data (meaning that the data is very much similar to the production data or real data). For the format, please refer to question 3 and 4 in qaquestions.com.
What are the different matrices that you follow?
There are various reports we normally prepare in QA:
· Test summary Report – It is a report that has list of the total test cases, list of executed test cases, remaining test case to be executed, executed date, pass/fail
· Defect Report – In this report we normally prepare a list of defect in spreadsheet e.g. defect # CQ12345 [ if you log a defect in the application called Rational ClearQuest]
· Traceability Matrix
What is parallel/audit testing?
Parallel/audit testing is testing where the user reconciles the output of the new system to the output of the current system to verify the new system performs the operations correctly.
What is software testing methodology?
One software testing methodology is the use a three step process of…
1. Creating a test strategy;
2. Creating a test plan/design; and
3. Executing tests. This methodology can be used and molded to your organization’s needs. Rob Davis believes that using this methodology is important in the development and in ongoing maintenance of his customers’ applications.
What is the general testing process?
The general testing process is the creation of a test strategy (which sometimes includes the creation of test cases), creation of a test plan/design (which usually includes test cases and test procedures) and the execution of tests.
How do you create a test strategy?
The test strategy is a formal description of how a software product will be tested. A test strategy is developed for all levels of testing, as required. The test team analyzes the requirements, writes the test strategy and reviews the plan with the project team. The test plan may include test cases, conditions, the test environment, a list of related tasks, pass/fail criteria and risk assessment. Inputs for this process:
· A description of the required hardware and software components, including test tools. This information comes from the test environment, including test tool data.
· A description of roles and responsibilities of the resources required for the test and schedule constraints. This information comes from man-hours and schedules.
· Testing methodology. This is based on known standards.
· Functional and technical requirements of the application. This information comes from requirements, change request, technical and functional design documents.
· Requirements that the system can not provide, e.g. system limitations. Outputs for this process:
· An approved and signed off test strategy document, test plan, including test cases.
· Testing issues requiring resolution. Usually this requires additional negotiation at the project management level.
How do you create a test plan/design?
Test scenarios and/or cases are prepared by reviewing functional requirements of the release and preparing logical groups of functions that can be further broken into test procedures. Test procedures define test conditions, data to be used for testing and expected results, including database updates, file outputs, report results. Generally speaking…
Test cases and scenarios are designed to represent both typical and unusual situations that may occur in the application.
Test engineers define unit test requirements and unit test cases. Test engineers also execute unit test cases.
It is the test team that, with assistance of developers and clients, develops test cases and scenarios for integration and system testing.
Test scenarios are executed through the use of test procedures or scripts.
Test procedures or scripts define a series of steps necessary to perform one or more test scenarios.
Test procedures or scripts include the specific data that will be used for testing the process or transaction.
Test procedures or scripts may cover multiple test scenarios.
Test scripts are mapped back to the requirements and traceability matrices are used to ensure each test is within scope.
Test data is captured and base lined, prior to testing. This data serves as the foundation for unit and system testing and used to exercise system functionality in a controlled environment.
Some output data is also base-lined for future comparison. Base-lined data is used to support future application maintenance via regression testing.
A pretest meeting is held to assess the readiness of the application and the environment and data to be tested. A test readiness document is created to indicate the status of the entrance criteria of the release.
Inputs for this process:
Approved Test Strategy Document.
Test tools, or automated test tools, if applicable.
Previously developed scripts, if applicable.
Test documentation problems uncovered as a result of testing.
A good understanding of software complexity and module path coverage, derived from general and detailed design documents, e.g. software design document, source code and software complexity data.
Outputs for this process:
Approved documents of test scenarios, test cases, test conditions and test data.
Reports of software design issues, given to software developers for correction.
How do you execute tests?
Execution of tests is completed by following the test documents in a methodical manner. As each test procedure is performed, an entry is recorded in a test execution log to note the execution of the procedure and whether or not the test procedure uncovered any defects. Checkpoint meetings are held throughout the execution phase. Checkpoint meetings are held daily, if required, to address and discuss testing issues, status and activities.The output from the execution of test procedures is known as test results. Test results are evaluated by test engineers to determine whether the expected results have been obtained. All discrepancies/anomalies are logged and discussed with the software team lead, hardware test lead, programmers, software engineers and documented for further investigation and resolution. Every company has a different process for logging and reporting bugs/defects uncovered during testing.A pass/fail criteria is used to determine the severity of a problem, and results are recorded in a test summary report. The severity of a problem, found during system testing, is defined in accordance to the customer’s risk assessment and recorded in their selected tracking tool.Proposed fixes are delivered to the testing environment, based on the severity of the problem. Fixes are regression tested and flawless fixes are migrated to a new baseline. Following completion of the test, members of the test team prepare a summary report. The summary report is reviewed by the Project Manager, Software QA Manager and/or Test Team Lead.
After a particular level of testing has been certified, it is the responsibility of the Configuration Manager to coordinate the migration of the release software components to the next test level, as documented in the Configuration Management Plan. The software is only migrated to the production environment after the Project Manager’s formal acceptance.
How do you divide the application into different sections to create scripts?
First of all, the application is divided in different parts when a business analyst writes the requirement document (or Use Cases or Design Document), he/she writes EACH requirement document for EACH module
What is a ‘Show Stopper’?
A show stopper is a defect or bug that stops the user for further action (testing). It has no work around.
Related ru language articles:
- Тестирование. Книга первая. Тест-кейсы и QA
- Тестирование. Книга первая. Юнит-тестирование, виды тестирования
- Тестирование. Книга вторая. Таблицы и чуть подробнее о QA.
- Вопросы после прочтения книг по тестированию