Defect-enhanced diffusion of magnetic skyrmions