We study the thermal conductivity of the one-dimensional Fermi-Hubbard model at a finite temperature using a density matrix renormalization group approach. The integrability of this model gives rise to ballistic thermal transport. We calculate the temperature dependence of the thermal Drude weight at half filling for various interaction strengths. The finite-frequency contributions originating from the fact that the energy current is not a conserved quantity are investigated as well. We report evidence that breaking the integrability through a nearest-neighbor interaction leads to vanishing Drude weights and diffusive energy transport. Moreover, we demonstrate that energy spreads ballistically in local quenches with initially inhomogeneous energy density profiles in the integrable case. We discuss the relevance of our results for thermalization in ultracold quantum-gas experiments and for transport measurements with quasi-one-dimensional materials.