Chandra and XMM–Newton observations of the Cartwheel galaxy show ∼17 bright X-ray sources (≳5 × 1038 erg s−1) , all within the gas-rich outer ring. We explore the hypothesis that these X-ray sources are powered by intermediate-mass black holes (IMBHs) accreting gas or undergoing mass transfer from a stellar companion. To this purpose, we run N-body/smoothed particle hydrodynamics simulations of the galaxy interaction which might have led to the formation of Cartwheel, tracking the dynamical evolution of two different IMBH populations: halo and disc IMBHs. Halo IMBHs cannot account for the observed X-ray sources, as only a few of them cross the outer ring. Instead, more than half of the disc IMBHs are pulled in the outer ring as a consequence of the galaxy collision. However, also in the case of disc IMBHs, accretion from surrounding gas clouds cannot account for the high luminosities of the observed sources. Finally, more than 500 disc IMBHs are required to produce ≲15 X-ray sources via mass transfer from very young stellar companions. Such number of IMBHs is very large and implies extreme assumptions. Thus, the hypothesis that all the observed X-ray sources in Cartwheel are associated with IMBHs is hardly consistent with our simulations, even if it is still possible that IMBHs account for the few (≲1–5) brightest ultraluminous X-ray sources.