Benchmarking Bug Dectection Tools

Speaker: Erica Mealy

When: 10:00, Wednesday 30th January 2008

Venue: 78-420

Tools detecting software bugs are starting to become widely known and available, but with no reliable way to benchmark or compare these tools it is hard to know how effective the tools are. In this talk I will be presenting the Parfait Benchmark Repository and Driver for comparing and benchmarking bug detection tools.

The Parfait Benchmark Repository currently holds approximately 1,200 benchmarks with a range of different software bugs. The repository can also be queried to provide a subset based on a number of different criteria including bug type, size, and architecture. The Parfait Benchmark Driver takes a subset of the benchmarks and runs these against a set of tools providing either text output or HTML graphs showing performance figures, the number of correctly identified bugs, and the number of false positives and false negatives. Using the Parfait Benchmark Repository and Driver, it becomes possible to compare the accuracy of bug detection tools relative to a set of known benchmarks.

Bio: Erica Mealy is a Graduate Intern with Sun Microsystems Down Under working on the Parfait project. Previously, she also interned with Sun Labs in California on the Squawk Java Virtual Machine project. Erica is a PhD student in the SSE group at UQ working on Semi- automated Software Refactoring Support.

Hospitality: Cristina Cifuentes

Contact: Robert Colvin (SSE seminar co-ordinator) (robert@itee.uq.edu.au)

SSE seminar web page: http://www.itee.uq.edu.au/~sse/Seminars.html