Tool Competitions

This year as well we are pleased to announce the thirteenth edition of the testing tool competition. The competition has the goal to experiment with testing tools for a diversified set of traditional and emerging systems and domains. This year we have four competitions, namely the Java Tool Competition, the Self-Driving Car Testing Tool Competition, the Fuzzing Tool Competition and the CPS-UAV Tool Competition.

Java Tool Competition

As for recent years, we invite researchers to participate in the competition with their unit test generation tool for Java. Full details may be found on the tool competition page.

Self-Driving Car Testing Tool Competition

In addition to the traditional Java tool competition, we also organize a Self-Driving Car testing competition. Self-driving cars (SDCs) are equipped with onboard cameras and various sensors, which have already demonstrated the possibility of autonomous driving in real environments, leading to great interest in various application domains. However, despite the necessity of systematically testing such complex and automated systems to ensure their safe operation in real-world environments, there has been relatively limited investment in this direction so far.

The SDC Testing Competition organized jointly by the International Conference on Software Testing, Verification and Validation (ICST) and Search-Based and Fuzz Testing (SBFT) workshop @ ICSE is an initiative designed to inspire and encourage the Software Testing Community to direct their attention toward SDCs as a rapidly emerging and crucial domain. The joint call is meant to help interested authors/participants reduce travel costs by selecting the most convenient and close venue.

The competition focuses on regression testing for test suites executing system tests of SDCs in simulation. Regression testing involves test selection, prioritization, and minimization. For the competition, we focus on the first aspect - the test selection, as detailed in the competition platform repository

.

Fuzzing Tool Competition

We invite you, researchers, practitioners, and hobbyists, to submit your fuzzer to the annual SBFT Fuzzing Competition! This year, we plan to score submissions based on a new scoring system: "novelty coverage", a coverage metric we introduce to score for fuzzers which explore atypical code regions. Full details may be found on the tool competition page.

CPS-UAV Tool Competition

We invite researchers to participate in the competition with their test generation tool for Unmanned Aerial Vehicles (UAVs). The UAV Testing Competition organized by the Search-Based and Fuzz Testing (SBFT) workshop is an initiative designed to inspire and encourage the Software Testing Community to direct their attention toward UAVs as a rapidly emerging and crucial domain. Tools will be evaluated against a benchmark with respect to relevant criteria, as done for other, previous competitions.

The UAV Testing Competition organized jointly by the International Conference on Software Testing, Verification and Validation (ICST) and Search-Based and Fuzz Testing (SBFT) workshop @ ICSE is an initiative designed to inspire and encourage the Software Testing Community to direct their attention toward SDCs as a rapidly emerging and crucial domain. The joint call is meant to help interested authors/participants reduce travel costs by selecting the most convenient and close venue.

The competition is detailed on Github.