We compare the dielectric spectra of aqueous MgSO4 and Na2SO4 solutions calculated from classical molecular dynamics simulations with experimental data, using an optimized thermodynamically consistent sulfate force field. Both the concentration-dependent shift of the static dielectric constant and the spectral shape match the experimental results very well for Na2SO4 solutions. For MgSO4 solutions, the simulations qualitatively reproduce the experimental observation of a slow mode, the origin of which we trace back to the ion-pair relaxation contribution via spectral decomposition. The radial distribution functions show that Mg2+ and SO2−4 ions form extensive water-separated—and thus strongly dipolar—ion pairs, the orientational relaxation of which provides a simple physical explanation for the prominent slow dielectric mode in MgSO4 solutions. Remarkably, the Mg2+–SO2−4 ion-pair relaxation extends all the way into the THz range, which we rationalize by the vibrational relaxation of tightly bound water-separated ion pairs. Thus, the relaxation of divalent ion pairs can give rise to widely separated orientational and vibrational spectroscopic features.