Early and accurate prediction of mastitis is crucial in effective herd management and minimizing economic losses. This study investigated the effects of farm-specific factors on the accuracy of mastitis predictions by applying machine learning (ML) models to an automated milking system (AMS) and farm management data. We analyzed a large dataset consisting of 5.88 million observations over the period of 2019–2024 from four dairy farms in Germany. Six ML algorithms were applied to predict mastitis occurrence, with a focus on understanding how farm-specific factors like herd size, management practices, and farm environment may influence prediction accuracy. For training and testing on combined data, the accuracy, sensitivity and specificity ranged between 83 and 92%, 78 and 93% and 83 and 92%, respectively, with an area under curve (AUC) between 91 and 96%. However, under mixed-to-individual farm effects analysis, results exposed weaknesses in the generalization. Models adapted well to internal patterns when analyzing each individual farm separately, reaching very high AUCs of up to 98%, but the results were significantly different again when analyzed with a leave-one-out approach. The analysis determined that data from each farm carries variable underlying patterns, suggesting that a tailored approach to each farm’s unique characteristics might improve mastitis prediction through ML.
