An effective Hamiltonian for the simulation of open quantum molecular systems