Objective Ayurveda is commonly used in South Asia to treat knee osteoarthritis (OA). We aimed to evaluate the effectiveness of Ayurvedic treatment compared to conventional conservative care in patients with knee OA. Method According to American College of Rheumatology (ACR) criteria knee OA patients were included in a multicenter randomized, controlled, open-label trial and treated in 2 hospital clinics and 2 private outpatient clinics in Germany. Participants received either a multi-modal Ayurvedic treatment or multi-modal conventional care with 15 treatments over 12 weeks respectively. Primary outcome was the change on the Western Ontario and McMaster University Osteoarthritis (WOMAC) Index after 12 weeks. Secondary outcomes included WOMAC subscales; the pain disability index and a pain experience scale, numeric rating scales for pain and sleep quality, quality-of-life and mood, rescue medication use, and safety issues. Results One hundred fifty-one participants (Ayurveda n = 77, conventional care n = 74) were included. Changes of the WOMAC Index from baseline to 12 weeks were more pronounced in the Ayurveda group (mean difference 61.0 [95%CI: 52.4;69.6]) than in the conventional group (32.0 [95%CI: 21.4;42.6]) resulting in a significant between-group difference (p < 0.001) and a clinically relevant effect size (Cohen's d 0.68 [95% CI:0.35;1.01]). Similar trends were observed for all secondary outcomes at week 12. Effects were sustained at follow-ups after 6 and 12 months. Conclusion Results suggest that Ayurvedic treatment is beneficial in reducing knee OA symptoms. Further studies should be conducted to confirm the magnitude of the effect and to clarify the role of different treatment components and non-specific effects. Registration at clinicaltrials.gov (NCT01225133; initial release 10/06/2010).