The aim of this Hypothesis and Theory is to question the recently increasing use of the “race” concept in contemporary genetic, psychiatric, neuroscience as well as social studies. We discuss “race” and related terms used to assign individuals to distinct groups and caution that also concepts such as “ethnicity” or “culture” unduly neglect diversity. We suggest that one factor contributing to the dangerous nature of the “race” concept is that it is based on a mixture of traditional stereotypes about “physiognomy”, which are deeply imbued by colonial traditions. Furthermore, the social impact of “race classifications” will be critically reflected. We then examine current ways to apply the term “culture” and caution that while originally derived from a fundamentally different background, “culture” is all too often used as a proxy for “race”, particularly when referring to the population of a certain national state or wider region. When used in such contexts, suggesting that all inhabitants of a geographical or political unit belong to a certain “culture” tends to ignore diversity and to suggest a homogeneity, which consciously or unconsciously appears to extend into the realm of biological similarities and differences. Finally, we discuss alternative approaches and their respective relevance to biological and cultural studies.