Cell-intrinsic responses mounted in PBMCs during mild and severe COVID-19 differ quantitatively and qualitatively. Whether they are triggered by signals emitted by productively infected cells of the respiratory tract or result from physical interaction with virus particles remains unclear. Here, we analyzed susceptibility and expression profiles of PBMCs from healthy donors upon ex vivo exposure to SARS-CoV and SARS-CoV-2. In line with the absence of detectable ACE2 receptor expression, human PBMCs were refractory to productive infection. RT-PCR experiments and single-cell RNA sequencing revealed JAK/STAT-dependent induction of interferon-stimulated genes (ISGs) but not proinflammatory cytokines. This SARS-CoV-2-specific response was most pronounced in monocytes. SARS-CoV-2-RNA-positive monocytes displayed a lower ISG signature as compared to bystander cells of the identical culture. This suggests a preferential invasion of cells with a low ISG baseline profile or delivery of a SARS-CoV-2-specific sensing antagonist upon efficient particle internalization. Together, nonproductive physical interaction of PBMCs with SARS-CoV-2- and, to a much lesser extent, SARS-CoV particles stimulate JAK/STAT-dependent, monocyte-accentuated innate immune responses that resemble those detected in vivo in patients with mild COVID-19.