We present the second-order general relativistic description of the observed galaxy number density in a cosmological framework. The observed galaxy number density is affected by the volume and the source effects, both of which arise due to the mismatch between physical and observationally inferred quantities such as the redshift, the angular position, the volume, and the luminosity of the observed galaxies. These effects are computed to the second order in metric perturbations without choosing a gauge condition or adopting any restrictions on vector and tensor perturbations, extending the previous linear-order calculations. Paying particular attention to the second-order gauge transformation, we explicitly isolate unphysical gauge modes and construct second-order gauge-invariant variables. Moreover, by constructing second-order tetrads in the observer's rest frame, we clarify the relation between the physical and the parametrized photon wave vectors. Our second-order relativistic description will provide an essential tool for going beyond the power spectrum in the era of precision measurements of galaxy clustering. We discuss potential applications and extensions of the second-order relativistic description of galaxy clustering.