We study the entanglement entropy of gapped phases of matter in three spatial dimensions. We focus in particular on size-independent contributions to the entropy across entanglement surfaces of arbitrary topologies. We show that for low energy fixed-point theories, the constant part of the entanglement entropy across any surface can be reduced to a linear combination of the entropies across a sphere and a torus. We first derive our results using strong sub-additivity inequalities along with assumptions about the entanglement entropy of fixed-point models, and identify the topological contribution by considering the renormalization group flow; in this way we give an explicit definition of topological entanglement entropy Stopo in (3+1)D, which sharpens previous results. We illustrate our results using several concrete examples and independent calculations, and show adding “twist” terms to the Lagrangian can change Stopo in (3+1)D. For the generalized Walker-Wang models, we find that the ground state degeneracy on a 3-torus is given by exp(−3Stopo[T2]) in terms of the topological entanglement entropy across a 2-torus. We conjecture that a similar relationship holds for Abelian theories in (d+1) dimensional spacetime, with the ground state degeneracy on the d-torus given by exp(−dStopo[Td−1]).