This paper focuses on two aspects of the statistics of cosmological observables that are important for the next stages of precision cosmology. First, we note that the theory of reduced angular N-point spectra has only been developed in detail up to the trispectrum case and in a fashion that makes it difficult to go beyond. To fill this gap, here we present a constructive approach that provides a systematic description of reduced angular N-point spectra and their covariance matrices, for arbitrary N. Second, we focus on the common practice in the literature on cosmological observables, which consists in simply discarding a part of the expression, namely, the terms containing fields evaluated at the observer position. We point out that this is not justified beyond linear order in perturbation theory, as these terms contribute to all the multipoles of the corresponding spectra and with a magnitude that is of the same order as the rest of the nonlinear corrections. We consider the possibility that the reason for neglecting these terms is a conceptual discomfort when using ensemble averages, which originates in an apparent tension between the ergodic hypothesis and the privileged position of the observer on the light-cone. We clarify this subtle issue by performing a careful derivation of the relation between the theoretical statistical predictions and the observational estimators for all N. We conclude that there is no inconsistency whatsoever in ensemble-averaging fields at and near the observer position, thus clearing the way for consistent and robust high-precision calculations.