We investigate the dielectric profile of water confined between two planar polar walls using atomistic molecular dynamics simulations. For a water slab thickness below 1 nm the dielectric response is highly asymmetric: while the parallel component slightly increases compared to bulk, the perpendicular one decreases drastically due to anticorrelated polarization of neighboring water molecules. We demonstrate the importance of the dielectric contribution due to flexible polar headgroups and derive an effective dielectric tensorial box model suitable for coarse-grained electrostatic modeling.