Background: Results on parental burden during the COVID-19 pandemic are predominantly available from nonrepresentative samples. Although sample selection can significantly influence results, the effects of sampling strategies have been largely underexplored.
Objective: This study aimed to investigate how sampling strategy may impact study results. Specifically, we aimed to (1) investigate if outcomes on parental health and child maltreatment during the COVID-19 pandemic from a convenience sample differ from those of a specific representative sample and (2) investigate reasons for differences in the results.
Methods: In 2020, we simultaneously conducted 2 studies: (1) a web-based survey using a convenience sample of 4967 parents of underage children, primarily recruited via social media, and (2) a study using a quota sample representative of the German adult population with underage children (N=1024), recruited through a combination of telephone interviews and computer-assisted web interviews. In both studies, the same questionnaire was used. To evaluate the impact of sampling, we compared the results on outcomes (parental stress, subjective health, parental mental health, general stress, pandemic-related stress, and the occurrence of child maltreatment) between the 2 samples. To explain differences in the results between the 2 studies, we controlled for sociodemographic data, parent-related risk factors, and COVID-19–related experiences.
Results: Compared to parents from the quota sample, parents from the convenience sample reported significantly more parental stress (η2=0.024); decreased subjective health (η2=0.016); more anxiety and depression symptoms (η2=0.055); more general stress (η2=0.044); more occurrences of verbal emotional abuse (VEA; φ=0.12), witnessing domestic violence (WDV; φ=0.13), nonverbal emotional abuse (NEA; φ=0.03), physical abuse (φ=0.10), and emotional neglect (φ=0.06); and an increase of child maltreatment (VEA: exp(B)=2.95; WDV: exp(B)=3.19; NEA: exp(B)=1.65). Sociodemographic data, parent-related risk factors, and COVID-19–related experiences explained the differences in parental stress (remaining difference between samples after controlling for covariates: η2=0.002) and subjective health (remaining difference between samples after controlling for covariates: η2=0.004) and partially explained differences in parental mental health (remaining: η2=0.016), general stress (remaining: η2=0.014), and child maltreatment (remaining: VEA: exp(B)=2.05 and WDV: exp(B)=2.02) between the 2 samples. The covariates could not explain the difference in NEA (exp(B)=1.70). We discuss further factors that may explain the unexplained differences.
Conclusions: Results of studies can be heavily impacted by the sampling strategy. Scientists are advised to collect relevant explaining variables (covariates) that are possibly related to sample selection and the outcome under investigation. This approach enables us to identify the individuals to whom the results apply and to combine findings from different studies. Furthermore, if data on the distribution of these explanatory variables in the population are available, it becomes possible to adjust for sample selection bias. J Med Internet Res 2025;27:e52043
doi:10.2196/52043
