In recent decades, post-colonial and anti-racist activists around the world have initiated debates about the representation of colonialism in public space. In Germany, too, the treatment of colonial street names and monuments has become a subject of controversy. What is unique to the debate on the memory of colonialism in Germany, though, is how it is deeply entangled with memories of other political regimes, in particular National Socialism and the German Democratic Republic. To comprehend today's memory–political conflicts, it is therefore important to understand how memories of the colonial past have changed over time and how they have been influenced by memories of those other regimes. This article demonstrates that there are multi-layered memories associated with (post-)colonial monuments in Germany. To this end, it closely examines three memorial sites: the Pogge bust in Rostock, the Bremen elephant and the corner of Wilhelmstraße/An der Kolonnade in Berlin. It illustrates how these sites have served as focal points in public discourse and memory practices, highlighting the various layers of meaning different actors have attached to them throughout the twentieth and twenty-first centuries. Tracing the construction and usage of these sites, the article reveals the (in)visibility of different historical layers and the memory–political negotiations surrounding them. It shows how the complex interplay of these entangled layers of memory is not adequately represented at the memorial sites themselves. In conclusion, the article suggests (re-)designing memorial sites in a way that highlights the struggles related to the past and reveals the relationships between different layers of memory, fostering a deeper understanding of the history of remembrance.