Deep learning has revolutionized computer vision, particularly enabling automated analysis across various fields. However, a significant challenge for deep learning methods, especially in complex applications like medical imaging, lies in their generalization capabilities. This is often hindered by two critical issues: the limited size of annotated datasets and the variability of images across different machines or institutions. This dissertation explores the pivotal role of spatial inductive bias in improving the generalization of deep learning models for medical imaging tasks. To address the data scarcity challenge, this thesis leverages self-supervised learning as a pre-training strategy on large amounts of unlabeled data, significantly reducing the dependency on scarce labeled examples. This approach directly encodes spatial information into the learning process, enhancing the model’s ability to understand spatial dependencies. The combination of self-supervised learning techniques with spatially biased models demonstrates significant performance improvements across various medical imaging modalities. Specialized spatial encoding is crucial for applications dealing with high-dimensional data, such as medical imaging. A central aspect of this research is the integration of positional encodings within deep learning architectures to introduce spatial inductive bias. The experiments conducted highlight the power of incorporating domain-specific biases into deep learning models to facilitate generalization. This research presents a comprehensive analysis of how spatial inductive biases can be designed, implemented, and optimized, providing insights into their impact on model interpretability and diagnostic accuracy. The findings suggest that a well-designed spatial bias not only enhances segmentation precision but also contributes to a model’s ability to generalize from limited medical datasets, paving the way for more reliable and interpretable AI-driven solutions in clinical practice.