Introduction Past research suggests a link between post-traumatic stress disorder (PTSD) and an increased risk for sexual problems. However, there is still no clear picture whether these higher rates are related to trauma exposure or to PTSD itself. Aim The aim of the present study was to complement existing knowledge on the relative impact of trauma and PTSD on sexuality in later life, considering different aspects of trauma exposure on both men and women. Methods The study was conducted on a unique population sample of former Swiss indentured child laborers (55 men, Mage 78, age range 60–95 years) who have repeatedly experienced a variety of severe childhood traumas. Main Outcome Measures Sexual outcomes were measured using two scales from the Trauma Symptom Inventory—Dysfunctional Sexual Behavior (DSB) and Sexual Concerns (SC). PTSD symptoms and trauma were assessed with the Short Screening Scale for PTSD and the Composite International Diagnostic Interview, respectively. Results Twenty-two individuals showed PTSD symptoms, and 53 reported having experienced childhood trauma. Significant differences between men and women were reported for DSB and SC. Men reported a significantly higher prevalence of both SC and DSB compared with women. Conclusions This is the very first study investigating DSB and SC in a sample of older adults exposed to similar traumatic experiences and settings. However, some study limitations need to be considered such as the small sample size. Additional studies are needed to further explore the relative role of traumatization and PTSD on sexual behavior and well-being, especially to improve sexual therapy for patients who experience trauma.