Infants’ pointing frequency is a predictor of their later language abilities. Yet, predictors of pointing frequency in the first year of life are not well understood. Study 1 explored what factors in infants and caregivers at 10 months would predict the pointing frequency of infants at 12 months (N = 35). Infant‐driven predictors were infants’ fine‐motor skills and point‐following abilities. Caregiver‐mediated predictors were caregivers’ pointing frequency and responsiveness toward infants’ pointing. Relevant caregiver responsiveness at 10 months predicted infants’ pointing frequency at 12 months, controlling for the other factors and infants’ prior pointing frequency. Study 2 explored whether child‐level factors influence caregivers’ responsiveness (N = 49). We examined the hand shape of infants’ pointing (whole‐hand versus index‐finger) and the presence of point‐accompanying vocalizations. Infants’ vocalization‐accompanied points were more likely to elicit relevant responses from caregivers, while hand shapes played a less pronounced role. Together, the findings reveal an early emerging mutual relationship between infant pointing and caregiver behavior such that certain characteristics of infant pointing predict caregivers’ responsiveness, and relevant responsiveness toward infants’ pointing predicts the increase in infants’ pointing frequencies.