Despite two centuries of fossils collecting, no cetacean remains from the Oligocene marine deposits of the Mainz Basin (western Germany) have ever been reported. Here, we describe a possible mysticete tooth from the sand pit of Eckelsheim, which exposes high energy deposits belonging to the Rupelian Alzey Formation. The latter has yielded a rich assemblage of vertebrates and invertebrates, but so far, only one marine mammal in the form of the sirenian Kaupitherium. The whale tooth in some ways resembles the m2 of Llanocetus from the latest Eocene of Seymour Island, Antarctica. If the find from the Mainz Basin is not a regionally evolved form, Llanocetus, which is known from the South Atlantic, could have migrated through the Atlantic realm during the early Oligocene. It cannot be excluded that the tooth represents a more widely occurring lineage, neither endemic nor necessarily related to llanocetids, that—given the generally poor Rupelian record—has not been well documented yet.