The development of efficient noble-metal free electrocatalysts is crucial for clean hydrogen production through water splitting. As carbon-based supports are expected to play a major role in low cost electrocatalysis, improved synthetic methods and a deeper understanding of their mechanisms of action are now required. To this end, we synthesized transition metal catalysts for overall water splitting encapsulated into nitrogen-doped carbon nanotubes (M–N-CNTs, M = Ni, Co, Fe) through a direct and convenient pyrolysis of bulk g-C3N4. Furthermore, the addition of reduced graphene oxide (rGO) leads to a significant dispersion of the catalytic N-CNTs. Among the obtained catalyst series, NiFe–N-CNT with rGO (NiFe–N-CNT–rGO) exhibits extremely low overpotential of 270 mV (on glassy carbon) for the oxygen evolution reaction (OER) at a current density of 10 mA cm−2. This performance is superior to most of the previously reported noble metal-free catalysts for OER. Our comprehensive study unravels that the growth of CNTs follows a “reduction–nucleation–growth” process. The thermally reduced metallic nanoparticles (NPs) serve as nucleation sites of carbon species on their surface to further promote N-CNT growth. Density functional theory (DFT) calculations reveal that the CNT walls and N-dopants in the catalysts modify the electronic structure and adjust the free energy toward the adsorption of intermediates. The one-step hydrogen evolution reaction (HER) process is influenced more strongly by N-centers when compared to the four-electron transfer OER process. The scalable and straightforward synthesis together with excellent electrocatalytic performance renders the NiFe–N-CNT–rGO hybrid catalyst quite promising for large-scale water splitting applications.