Soil fungi are a key constituent of global biodiversity and play a pivotal role in agroecosystems. How arable farming affects soil fungal biogeography and whether it has a disproportional impact on rare taxa is poorly understood. Here, we used the high-resolution PacBio Sequel targeting the entire ITS region to investigate the distribution of soil fungi in 217 sites across a 3000 km gradient in Europe. We found a consistently lower diversity of fungi in arable lands than grasslands, with geographic locations significantly impacting fungal community structures. Prevalent fungal groups became even more abundant, whereas rare groups became fewer or absent in arable lands, suggesting a biotic homogenization due to arable farming. The rare fungal groups were narrowly distributed and more common in grasslands. Our findings suggest that rare soil fungi are disproportionally affected by arable farming, and sustainable farming practices should protect rare taxa and the ecosystem services they support.