Langevin dynamics of a heavy particle and orthogonality effects