An alternative confusion matrix implementation for PreCall

Abstract

In this work, we examine literature on creating visualizations for the performance of machine learning classifiers, with our target group being users with limited machine learning experience. The underlying data is taken from Wikipedia, and more specifically ORES - Wikimedia’s service, which employs a machine learning model to score edits and articles. The interface also expands on PreCall’s implementation, and features multiple interactive components allowing the user to dynamically adjust parameters and see the immediate change in the classifier’s performance. After providing a summary of the relevant literature, we go over the ORES API and its relevant endpoints and parameters. Then, we outline the most popular ways to visualize a machine learning classifier’s performance. Following that is a thorough description of our target group, goals, and requirements, as well as the reasoning behind each design decision. Finally, there is an overview of the design and development process and we conduct a feedback session with a machine learning expert with background in ORES, and the feedback we receive is mostly positive, with some suggestions for improvement.