In late-successional environments, low in available nutrient such as the forest understory, herbaceous plant individuals depend strongly on their mycorrhizal associates for survival. We tested whether in temperate European forests arbuscular mycorrhizal (AM) woody plants might facilitate the establishment of AM herbaceous plants in agreement with the mycorrhizal mediation hypothesis. We used a dataset spanning over 400 vegetation plots in the Weser-Elbe region (northwest Germany). Mycorrhizal status information was obtained from published resources, and Ellenberg indicator values were used to infer environmental data. We carried out tests for both relative richness and relative abundance of herbaceous plants. We found that the subset of herbaceous individuals that associated with AM profited when there was a high cover of AM woody plants. These relationships were retained when we accounted for environmental filtering effects using path analysis. Our findings build on the existing literature highlighting the prominent role of mycorrhiza as a coexistence mechanism in plant communities. From a nature conservation point of view, it may be possible to promote functional diversity in the forest understory through introducing AM woody trees in stands when absent.