Non-collusive corruption, i.e., corruption that imposes an additional burden on business activity, is particularly widespread in low-income countries. We build a macroeconomic model with credit market imperfections and heterogeneous agents to explore the roots and consequences of this type of corruption. We find that credit market imperfections, by generating rents for the incumbent entrepreneurs, create strong incentives for corrupt behavior by state officials. However, non-collusive corruption not only redistributes income from non-officials towards officials but also within the group of potential entrepreneurs. If borrowing is limited, bribes prevent poorer but talented individuals from starting a business. But this is likely to benefit those who may enter anyway; the cost of capital is lower and there is less competition on the goods markets.