Theoretical descriptions of observable quantities in cosmological perturbation theory should be independent of coordinate systems. This statement is often referred to as gauge-invariance of observable quantities, and the sanity of their theoretical description is verified by checking its gauge-invariance. We argue that cosmological observables are invariant scalars under diffeomorphisms and their theoretical description is gauge-invariant, only at linear order in perturbations. Beyond linear order, they are usually not gauge-invariant, and we provide the general law for the gauge-transformation that the perturbation part of an observable does obey. We apply this finding to derive the second-order expression for the observational light-cone average in cosmology and demonstrate that our expression is indeed invariant under diffeomorphisms.