The ability to flexibly predict others' behaviors has been ascribed to a theory of mind (ToM) system. Most research has focused on formal conceptual definitions of such a system, and the question of whom to credit with a ToM. In this article, I suggest shifting perspective from formal definitions to a usage-based approach. This approach views action within human interaction as central to the emergence and continuous development of the ability to flexibly predict others' behaviors. Addressing the current debate about whether infants have a ToM, I illustrate how infants use flexible action expectations to interact with others appropriately. I also discuss the continuous development of ToM and its natural structure from a usage-based perspective.