Background Retention to HIV care is vital for patients’ survival, to prevent onward transmission and emergence of drug resistance. Travelling to receive care might influence adherence. Data on the functioning of and retention to HIV care in the Central African region are limited. Methods This retrospective study reports outcomes and factors associated with retention to HIV care at a primary HIV clinic in Lambaréné, Gabon. Adult patients who presented to this clinic between January 2010 and January 2012 were included. Outcomes were retention in care (defined as documented show-up for clinical visits, regardless of delay) or LTFU (defined as a patient not retained in care; on ART or ART naïve, not returning to care during the study period with a patient delay for scheduled visits of more than 6 months), and mortality. Cox regression analysis was used to assess factors associated with respective outcomes. Qualitative data on reasons for LTFU were obtained from focus-group discussions. Results Of 223 patients included, 67.3% were female. The mean age was 40.5 (standard deviation 11.4) years and the median CD4 count 275 (interquartile range 100.5–449.5) cells/μL. In total, 34.1% were lost to follow up and 8.1% died. Documented tuberculosis was associated with increased risk of being LTFU (adjusted hazard ratio (aHR) 1.80, 95% confidence interval (95% CI) 1.05–3.11, P = 0.03), whereas early starting anti-retroviral therapy (ART) was associated with a decreased risk of LTFU (aHR 0.43, 95%CI 0.24–0.76, P = 0.004), as was confirmed by qualitative data. Conclusions Retention to HIV care in a primary clinic in Gabon is relatively poor and interventions to address this should be prioritized in the HIV program. Early initiation of ART might improve retention in care.