Humans and other land vertebrates localize sound by comparing the signals in each ear. Even though these differences are virtually absent underwater, fish are still able to sense the direction of sound. In 1975, Arie Schuijf proposed that this ability could arise from a comparison of the particle motion phase and the pressure phase of sound, a prediction that was recently confirmed experimentally for near-field sounds. However, in natural environments, sounds arrive from variable distances, altering the motion-pressure-phase relationship. Thus, directional hearing and distance hearing are potentially in conflict. There is currently neither a model nor experimental data for how fish deal with this complexity. Here, we systematically introduce phase differences between the particle motion and pressure components of sound pulses to quantify the directional tuning of startle responses in Danionella cerebrum. We find that the fish's directed startle behavior is both frequency and phase dependent, and we introduce a new model that quantitatively predicts the sensorimotor transformation across all observed stimuli. This framework likely extends to other otophysan fishes with evolutionarily conserved hearing apparatus, representing ∼15% of all vertebrate species.