Lower Bounding Diffusion Constant by the Curvature of Drude Weight