Verification & Validation Questions
When Do Verification and Validation Start and End?Verification and Validation start as soon as we decide to build a software product, or even before. Even when the project is "complete," the software will continue to evolve and adapt to new conditions, such as a new version of the underlying database, or new requirements. Verification and validation activities continue through each small or large change to the system. [Software Testing and Analysis by Michal Young]
Give examples of Verification & Validation?
Unverifiable (but validatable) specification: if a user presses a request button at floor i, an available elevator must arrive at floor i "SOON"....
elevator response Verifiable specification: if a user presses a request button at floor i, an available elevator must arrive at floor i within "30 SECONDS"...
Verification and Validation Example
Requirement R:
"Reverse thrust shall only be enabled when the aircraft is moving on the runway".
Domain Properties D:
Wheel pulses on, if and only if wheels turning.
Wheels turning, if and only if moving on runway.
Specification S:
Reverse thrust enabled if and only if wheel pulses on.
Verification
Does the flight software, P, running on the aircraft flight computer, C, correctly implement S?
Does S, in the context of assumptions D, satisfy R?
Validation
Are our assumptions, D, about the domain correct? Did we miss any?
Are the requirements, R, what is really needed? Did we miss any?
[Source]
Discuss the difference between Verification and Validation, and explain why validation is a particularly difficult process.
See the difference between verification & validation here and here and diagrammatic difference below:
Verification compares the product with the specifications.
Validation confirms that the End User is satisfied.
Verification is the process of evaluating a system/component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. (IEEE/ANSI)
Validation is the process of evaluating a system/component during or at the end of the development process to determine whether it satisfies specified requirements. (IEEE/ANSI)
Unfortunately, customers are often not sure of exactly what they want and need, so it may be completely possible to build a system that does not do everything the customer expects that it should. In software engineering, validating a software might be harder since client’s expectation may be blurred or unclear. Clients may use adjective for their expectation which may result in confusion or misunderstanding.
What are the objectives of Verification & Validation?
Software verification & validation determines that the software performs its intended functions correctly and also ensures that the software performs no unintended functions. In short the discovery of defects in a system.
Software verification & validation measure and assess the quality and reliability of software. The assessment of whether or not the system is useful and usable in an operational situation.
Explain Verification activities during life cycle.
| The Structured Approach to Testing | |
|---|---|
| Life Cycle Phase | Verification Activities |
| Requirements | • Determine verification approach • Determine adequacy of requirements • Generate functional Test Data • Determine consistency of design with requirements |
| Design | • Determine adequacy of design • Generate structural and functional test data • Determine consistency with design |
| Program (Build/constructi on) | • Determine adequacy of implementation • Generate structural and functional test data for programs |
| Test | • Test application system |
| Installation | • Place Tested system into production |
| Maintenance | • Modify and retest |
[Source]
Where exactly Verification and Validation occurs in SDLC?
Verification & Validation activities during SDLC:
[Source]
Explain Independent Verification & Validation?
Independent verification and validation is utilized when the nature of the application is so critical that extra assurance of functionality and performance is required. IV & V activities include analysis and testing in order to assess the software's quality and to ascertain if the critical, i.e., high risk, requirements are achieved in the software. Although the IV & V activities are similar to those of verification and validation performed in the development process, IV & V is managerially independent from the organization(s) implementing verification, validation, and the development process. [Verification and validation of rule-based expert systems By Suzanne Smith, Abraham Kandel]
What are Independent Verification & Validation activities throughout lifecycle?
IV&V Process provides tools and analysis procedures appropriate to each phase of the software development life cycle:
Formulation Phase:
Is development process sound, repeatable, and managed?
Requirements Phase:
Verify that system and software requirements are correct, complete, traceable and testable.
Analyze system-level requirements: Are test plans and acceptance criteria sufficient to validate system requirements and operational needs?
Are testing methods sufficient to verify and validate software requirements?
Are the correct software development, management, and support processes in place?
Design Phase:
Does the design support the requirements?
Are test plans and test environments sufficient to verify and validate software and operational requirements?
Does the design have any characteristics that will cause it to fail under operational scenarios? What solutions are appropriate?
Coding Phase:
Does the code reflect the design?
Is the code correct?
Verify that test cases trace to and cover software requirements and operational needs.
Verify that software test cases, expected results, and evaluation criteria fully meet testing objectives.
Analyze selected code unit test plans and results to verify full coverage of logic paths, range of input conditions, error handling, etc.
Test Phase:
Analyze correct dispositioning of software test anomalies.
Validate software test results versus acceptance criteria.
Verify tracing and successful completion of all software test objectives.
Operational Phase:
Verify that regression tests are sufficient to identify adverse impacts of changes. [Source]
Briefly justify whether each of the following is used for Validation, Verification, or both:
(a) acceptance testing
Answer: Acceptance testing is used for validation since the tests are written by the customer based on the requirements.
(b) white box testing
Answer: White box testing is used for verification since it is done at the level of the implementation to ensure that the code performs correctly.
(c) specification review
Answer: A specification review is used for validation since a specification is simply a formalization of the requirements.
(d) black box testing
Answer: Black box testing is used for validation since the tests are done at the interface level and can therefore test the requirements. It can also be used for verification in unit testing to check that the implementation satisfies the design.
(e) code review
Answer: A code review is used for verification since it checks the implementation against the design. [Source]
Rewrite these requirement specifications so that they are verifiable, splitting them into multiple requirements if you deem that appropriate.
The goal of this question was to transform these requirements into something that could be objectively tested against an implementation later in the Process. Typically, the answer took the form of replacing the adverbs and adjectives in the supplied requirement with an objective metric. The values are arbitrary.
1. The ATM should authenticate the cardholder quickly
The ATM should authenticate the cardholder in [some reasonably short time period].
2. The word processor's user interface should be intuitive to the new user
After [reasonably short time period] of use with the word processor, a new user should be able to correctly use the system’s features [some high frequency value] of the time.
3. The word processor should be able to run well on consumer computers
The word processor should be able to run on popular operating systems (e.g. Windows 2000) on low-end computers (such as are sold at retail outlets like Radio Shack or Best Buy). When running on such low-end computers, most, i.e., [high-percentage], of the features should be able to run in [short time period].
4. The new bank transaction system should be able to handle our expected traffic for the next five years
The new bank transaction system should be able to handle [high-performance quantity] of transactions, which is our projected load for five year’s hence.
5. The air traffic control system shouldn't crash, and if it does, it should be able to "reboot" immediately
The mean-time to failure of the ATC system should be [very high time period, e.g. a month] under [quantify] standard workloads. If it crashes, it should reboot automatically within [very short time period]. [Source]