We establish a general connection between ballistic and diffusive transport in systems where the ballistic contribution in the canonical ensemble vanishes. A lower bound on the Green-Kubo diffusion constant is derived in terms of the curvature of the ideal transport coefficient, the Drude weight, with respect to the filling parameter. As an application, we explicitly determine the lower bound on the high-temperature diffusion constant in the anisotropic spin-1/2 Heisenberg chain for anisotropy parameters Δ≥1, thus, settling the question of whether or not the transport is subdiffusive. Additionally, the lower bound is shown to saturate the diffusion constant for a certain classical integrable model.