A considerable amount of research has been carried out towards making long-standing smart home visions technically feasible. The technologically augmented homes made possible by this work are starting to become reality, but thus far living in and interacting with such homes has introduced significant complexity while offering limited benefit. As these technologies are increasingly adopted, the knowledge we gain from their use suggests a need to revisit the opportunities and challenges they pose. Synthesizing a broad body of research on smart homes with observations of industry and experiences from our own empirical work, we provide a discussion of ongoing and emerging challenges, namely challenges for meaningful technologies, complex domestic spaces, and human-home collaboration. Within each of these three challenges we discuss our visions for future smart homes and identify promising directions for the field.