Memory effects in colloidal motion under confinement and driving