This study addresses the heated academic and public debate on the compatibility and comparability of refugees’ and host societies’ democratic values. Comparative values research has long capitalized on global similarities and differences in support for Western democratic values. We argue that such cross-cultural comparisons of culturally diverse groups are challenged by (1) different conceptions of democracy determined by different experiences with democratic systems and (2) bias introduced by linguistic differences and translation processes. In order to analyze whether the conception of democracy is comparable between different nationalities and languages, we test data from the German IAB-BAMF-SOEP Survey of Refugees and the world values survey (WVS) for measurement invariance using multi-group confirmatory factor analysis (MGCFA). Applying strict and conservative criteria for measurement invariance and fit indices, our results suggest that the applied democracy scales are problematic for comparing conceptions of democracy between refugees and Germans and across languages. However if more lenient criteria regarding partial invariance and fit indices are considered acceptable, mean comparisons could be carried out between language groups and between groups of refugees.