The transport of individual particles in inhomogeneous environments is complex and exhibits non-Markovian responses. The latter may be quantified by a memory function within the framework of the linear generalised Langevin equation (GLE). Here, we exemplify the implications of steady driving on the memory function of a colloidal model system for Brownian motion in a corrugated potential landscape, specifically, for one-dimensional motion in a sinusoidal potential. To this end, we consider the overdamped limit of the GLE, which is facilitated by separating the memory function into a singular (Markovian) and a regular (non-Markovian) part. Relying on exact solutions for the investigated model, we show that the random force entering the GLE must display a bias far from equilibrium, which corroborates a recent general prediction. Based on data for the mean-square displacement (MSD) obtained from Brownian dynamics simulations, we estimate the memory function for different driving strengths and show that already moderate driving accelerates the decay of the memory function by several orders of magnitude in time. We find that the memory may persist on much longer timescales than expected from the convergence of the MSD to its long-time asymptote. Furthermore, the functional form of the memory function changes from a monotonic decay to a non-monotonic, damped oscillatory behaviour, which can be understood from a competition of confined motion and depinning. Our analysis of the simulation data further reveals a pronounced non-Gaussianity, which questions the Gaussian approximation of the random force entering the GLE.