Systematic bottom-up theory for the viscoelastic response of worm-like chain networks