In this paper we present a recursive method for the computation of dynamic competitive equilibria in models with heterogeneous agents and market frictions. This method is based on a convergent operator over an expanded set of state variables. The fixed point of this operator defines the set of all Markovian equilibria. We study approximation properties of the operator as well as the convergence of the moments of simulated sample paths. We apply our numerical algorithm to two growth models, an overlapping generations economy with money, and an asset pricing model with financial frictions.