Verbal derogation is not only a linguistic but also, and perhaps more importantly, a political phenomenon. In this paper, I argue that to do justice to the political relevance of derogatory terms, we must not neglect the social practices and structures in which the use of these terms is embedded. I aim to show that inferentialist semantics is especially helpful to account for this social embeddedness and, consequently, the political relevance of derogatory terms. I am concerned with specifying the linguistic and political aspects of terms that are at the core of derogatory language use. I explain them with the help of Brandomian inferentialism, which tends to be quickly dismissed or ignored in discussions about the meaning of derogatory terms. My inferentialist account is in line with Lynne Tirrell's broader inferentialist framework for understanding derogatory communicative practices. I provide the semantic details of her broader inferentialist view and thereby aim to show that inferentialist semantics is a promising theory which cannot be neglected in further discussion of meaning and politics in derogatory language use.