The nanoindentation method was applied to determine the elastic modulus and hardness of knee articular cartilage. Cartilage samples from both high weight bearing (HWB) and low weight bearing (LWB) femoral condyles were collected from patients diagnosed with osteoarthritis (OA). The mean elastic modulus of HWB cartilage was 4.46 ± 4.44 MPa in comparison to that of the LWB region (9.81 ± 8.88 MPa, p < 0.001). Similarly, the hardness was significantly lower in HWB tissue (0.317 ± 0.397 MPa) than in LWB cartilage (0.455 ± 0.434 MPa, p < 0.001). When adjusted to patients’ ages, the mean elastic modulus and hardness were both significantly lower in the age group over 70 years (p < 0.001). A statistically significant difference in mechanical parameters was also found in grade 3 and 4 OA. This study provides an insight into the nanomechanical properties of the knee articular cartilage and provides a starting point for personalized cartilage grafts that are compatible with the mechanical properties of the native tissue.