We show in this paper that after proper scalings, the characteristic polynomial of a random unitary matrix converges to a random analytic function whose zeros, which are on the real line, form a determinantal point process with sine kernel. Our scaling is performed at the so-called “microscopic” level, that is we consider the characteristic polynomial at points whose distance to $1$ has order $1 / n$. We prove that the rescaled characteristic polynomial does not even have a moment of order one, hence making the classical techniques of random matrix theory difficult to apply. In order to deal with this issue, we couple all the dimensions $n$ on a single probability space, in such a way that almost sure convergence occurs when $n$ goes to infinity. The strong convergence results in this setup provide us with a new approach to ratios: we are able to solve open problems about the limiting distribution of ratios of characteristic polynomials evaluated at points of the form exp($2i\pi\alpha/n$) and related objects (such as the logarithmic derivative). We also explicitly describe the dependence relation for the logarithm of the characteristic polynomial evaluated at several points on the microscopic scale. On the number theory side, inspired by the work by Keating and Snaith, we conjecture some new limit theorems for the value distribution of the Riemann zeta function on the critical line at the level of stochastic processes.