Measurements are presented of single and double-differential dijet cross sections in diffractive photoproduction based on a data sample with an integrated luminosity of 47 pb^-1. The events are of the type ep -> eXY, where the hadronic system X contains at least two jets and is separated by a large rapidity gap from the system Y, which consists of a leading proton or low-mass proton excitation. The dijet cross sections are compared with QCD calculations at next-to-leading order and with a Monte Carlo model based on leading order matrix elements with parton showers. The measured cross sections are smaller than those obtained from the next-to-leading order calculations by a factor of about 0.6. This suppression factor has no significant dependence on the fraction x_gamma of the photon four-momentum entering the hard subprocess. Ratios of the diffractive to the inclusive dijet cross sections are measured for the first time and are compared with Monte Carlo models.