Automated Debugging in Use

The process of correcting faults in software is called debugging. Debugging is a time-intensive process because even small programs can consist of several thousand lines of code which have to be manually investigated in order to find those lines which are responsible for an observed error (e.g. wrong computations or program crashes). It is estimated that software developers spend 30 % to 90 % of their time on debugging. Thus the improvement of the debugging process could save an enormous amount of money and time. Many researchers have developed approaches which support software developers when debugging. Unfortunately, these approaches are rarely used in practice. This project therefore aims to close the gap between academic research and debugging in practice and consists of three phases: First, we will examine the reasons why existing academic approaches are rarely used in practice. We will observe how software developers are debugging programs in order to assess the status quo of debugging in practice. Such observational sessions are very time consuming; therefore, they can only be done for a small group of study participants. In order to ensure that our findings are generally valid, we will additionally perform a large-scale online survey. Second, we will use the insights gained from the observational sessions and the online survey to improve existing debugging approaches. Thereby, we will particularly focus on the scalability, the accuracy, and the practicability of the approaches: Scalability: The debugging approaches should be able to handle large programs with more than one million lines of code. Accuracy: The debugging approaches should draw the software developers attention to those lines of code that are responsible for an observed error, but not to other irrelevant lines. Practicability: The debugging approaches should be easy to use. Software developers might struggle to use academic debugging approaches because they do not believe that such an approach can help them. In addition, they do not know which approach is best suited for their debugging problem. Therefore, we are going to answer two particularly interesting research questions in this project phase: (1) Can the combination of debugging approaches help to improve the overall debugging experience? (2) Is it possible to automatically select the best suited debugging method for a given program? Third, we will integrate the debugging approaches into the development environments and development processes. A debugging approach that is integrated into the software engineers development environment and process is more likely to be used than a stand-alone approach. To evaluate the usefulness of our developments, we will conduct extensive experiments.

Correcting errors in software is called debugging. Debugging is a time-consuming process because even small programs can consist of several thousand lines of code that must be examined manually to find the lines of code that are responsible for an observed error (e.g., incorrect calculations or program crashes). It is estimated that software developers spend 30% to 90% of their time debugging. Improvements to the debugging process therefore have the potential to save money and time. As a first step, we reached out to software developers with an online survey to find out what the biggest problems in the debugging process are. This survey showed that most errors are semantic errors that users notice, for example, due to incorrect operation of the program. The debugging process often follows the same pattern: recreating the error (executing a sequence of steps that leads to the error), making observations (What values do certain variables have? What changes to the sequence of steps cause the error to no longer occur? ...) and draw conclusions (e.g., focusing on specific lines of code). While the majority of programmers reported that it is easy to reproduce errors, many find identifying the location of the faulty code difficult. Furthermore, the study shows that errors are often complex and it is not enough to change individual lines of code. It follows that researchers should develop debugging tools that are capable of identifying bugs consisting of multiple lines of code. In a second step, we looked at how we can support software developers with the most difficult part of the debugging process, i.e., fault localization. Here, we focused on improving two existing approaches: The first approach (Information Retrieval Fault Localization) works with textual descriptions of errors, so-called bug reports, and uses artificial intelligence to find code that matches the error description. One of our extensions additionally calculates the error type based on probabilities. This approach is suitable for programs of any size (keyword: scalability). The second approach (Slicing) targets smaller programs: software developers reproduce errors and the developed slicing tool marks all lines of code that have an impact on the calculated error. All data sets, all evaluations carried out and all code written in the project are publicly available (see https://amadeus.ist.tugraz.at/).