AssertIS - Assertion Integration Study
Evaluating the cost-effectiveness of assertions combined with V&V
Introduction
Verification involves checking whether a computer program conforms to its specification, while validation involves checking whether it meets the requirements/expectations of the client; together they are often referred to as V&V. Assertions are logical expressions that can be embedded in software and that can then be checked during a static analysis of the software or when the software is executed. When an assertion violation is detected during such an analysis or execution, this is reported to the V&V personnel, who can then investigate and fix any potential problem(s).
Although assertions have long been recognised as a potentially powerful mechanism that can be used to support a variety of V&V activities, they are not in widespread use in industry. One exception to this is the use of assertions in test code (e.g. in unit-testing frameworks such as JUnit) which has become popular in recent years. However, in this project we focus on the use of assertions in delivered source code, rather than in support code such as test drivers and oracles. Rosenblum [1] suggests several reasons for this: they are not integrated well into existing development tools, it is not fully understood what kind of assertions are the most effective in detecting software faults, and for some programmers, programming with assertions is similar to writing a program twice. Despite the fact that a number of modern programming languages and tools now support the definition, evaluation, and reporting of assertions, their application in industry is still limited. We believe the main reason for this is the lack of research on and practical knowledge about the most cost-effective definition and use of assertions in various V&V activities.
Mission
The main outcome of this project will be a repository of validated empirical knowledge about the cost-effective use of assertions for the verification and validation of distributed and real-time software.
This repository can be used by researchers to guide their research and by practitioners to determine
cost-effective uses of assertions for their particular contexts.
Outcomes and Benefits
This project will:
- develop and evaluate novel ways of using assertions and novel types of assertions in the verification and validation of distrubted and real-time software
- empirically evaluate both the traditional use of assertions and the new techniques developed in this project, through observational studies and industrial case studies
- evaluate a systematic approach to the integration of assertions with other verification and validation technologies by using practical knowledge regarding the cost-effectiveness of various uses of assertions in particular verification and validation contexts
Resources
Test Automation Workshop (TAW2008), Bond University
Detecting Data Races with Assertions and Model Checking (slides)
20th Australian Software Engineering Conference (ASWEC2009), Gold Coast, Australia
Evaluating the cost-effectiveness of assertions combined with V&V (poster)
Evaluating the cost-effectiveness of assertions combined with V&V (handout)
MBT and Assertions
A Case Study on the Role of Assertions in Model-Based Testing (extended version)
Research Project Team
Prof. Paul Strooper
References
[1] D. S. Rosenblum, "A practical approach to programming with assertions," IEEE Transactions in Software Engineering, vol. 21, pp. 19-31, 1995.