Epidemiological modeling has a long history and is often used to forecast the course of infectious diseases or pandemics. These models come in different complexities, ranging from systems of simple ordinary differential equations (ODEs) to complex agent-based models (ABMs). The former allow a fast and straightforward optimization, but are limited in accuracy, detail, and parameterization, while the latter can resolve spreading processes in detail, but are extremely expensive to optimize. Epidemiological modeling can also be used to propose and design non-pharmaceutical interventions such as lockdowns. In general, their optimal design often leads to nonlinear optimization problems. We consider policy optimization in a prototypical situation modeled as both ODE and ABM, review numerical optimization approaches, and propose a heterogeneous multilevel approach based on combining a fine-resolution ABM and a coarse ODE model. Numerical experiments, in particular with respect to convergence speed, are given for illustrative examples.