How can agents learn hierarchies that are useful across a wide range of tasks? At OpenAI, I set out to answer this question.