What information animals derive from eavesdropping on interactions between conspecifics, and whether they assign value to it, is difficult to assess because overt behavioral reactions are often lacking. An inside perspective of how observers perceive and process such interactions is thus paramount. Here, we investigate what happens in the mind of marmoset monkeys when they hear playbacks of positive or negative third-party vocal interactions, by combining thermography to assess physiological reactions and behavioral preference measures. The physiological reactions show that playbacks were perceived and processed holistically as interactions rather than as the sum of the separate elements. Subsequently, the animals preferred those individuals who had been simulated to engage in positive, cooperative vocal interactions during the playbacks. By using thermography to disentangle the mechanics of marmoset sociality, we thus find that marmosets eavesdrop on and socially evaluate vocal exchanges and use this information to distinguish between cooperative and noncooperative conspecifics.