bflinks: Reliable Bugfix links via bidirectional references and tuned heuristics

Abstract

Background: Data from software data repositories such as source code version archives and defect databases contains valuable information that can be used for insights (leading to subsequent improvements), in particular defect insertion circumstance analysis and defect prediction. The first step in such analyses is identifying defect-correcting changes in the version archive (bugfix commits) and linking them to corresponding entries in the defect database, thus establishing bugfix links, in order to enrich the content of the defect-correcting change with additional meta-data. Typically, identifying the bugfix commits in a version archive is done via heuristic string matching on the commit message. Research questions: Which filters could be used to obtain a set of bugfix links? How does one set the cutoff parameters of each? What effect (results loss and precision) does each filter then have? Which overall precision, results loss, and recall is achieved? Method: We analyze a comprehensive modular set of seven independent filters, including new ones that make use of reverse links. We describe and evaluate visual heuristics (based on simple diagnostic plots) for setting six filters' cutoff parameter. We apply these to a commercial repository from the Web CMS domain and validate the results with unprecendented precision by making use of a product expert to manually verify over 2500 links. Results: The parameter selection heuristics pick a very good parameter value in five of the six cases and a reasonably good one in the sixth. As a result, the combined filtering, called bflinks, proposes a set of bugfix links that has 93\% precision with only 7\% results loss. Conclusion: The modular filtering approach can provide high-quality results and can be adapted to repositories with different properties.