A unified diagrammatic approach to topological fixed point models