Human mobility survey data usually suffer from a lack of resources for validation. Epidemiological survey records, which are released to the public as a containment measure by local authorities, provide place visitation details validated by the authority. This study collected and analyzed the epidemiological survey reports published by local governments in the Chinese mainland, between January 2020 and November 2021. To reveal the mobility patterns during the COVID-19 pandemic across the urban-rural gradient in China’s mainland, we derived key mobility indicators from the epidemiological survey data from rural to megacities. We then applied exploratory factor analysis to identify latent factors that affected people’s mobility. We found that the pandemic poses varying impacts across the urban-rural gradient in the Chinese mainland, and the mobility patterns of middle and small cities are more influenced. Our results also showed that the pandemic did not enlarge gender gap in people’s mobility, as gender was not a significant driving factor for explaining people’s quantity of out-of-home activities as well as extent of life space, while age group and city levels were significant. Overall, we argue that the epidemiological survey data are valuable data sources for daily mobility modeling, especially for relevant studies to understand human mobility patterns during the pandemic.