The energy sources involved in the early stages of the formation of terrestrial bodies can induce partial or even complete melting of the mantle, leading to the emergence of magma oceans. The fractional crystallization of a magma ocean can cause the formation of a compositional layering that can play a fundamental role for the subsequent long-term dynamics of the interior and for the evolution of geochemical reservoirs. In order to assess to what extent primordial compositional heterogeneities generated by magma ocean solidification can be preserved, we investigate the solidification of a whole-mantle Martian magma ocean, and in particular the conditions that allow solid-state convection to start mixing the mantle before solidification is completed. To this end, we performed 2-D numerical simulations in a cylindrical geometry. We treat the liquid magma ocean in a parameterized way while we self-consistently solve the conservation equations of thermochemical convection in the growing solid cumulates accounting for pressure-, temperature-, and, where it applies, melt-dependent viscosity. By testing the effects of different cooling rates and convective vigor, we show that for a lifetime of the liquid magma ocean of 1 Myr or longer, the onset of solid-state convection prior to complete mantle crystallization is likely and that a significant part of the compositional heterogeneities generated by fractionation can be erased by efficient mantle mixing. We discuss the consequences of our findings in relation to the formation and evolution of compositional reservoirs on Mars and on the other terrestrial bodies of the solar system.