We present SOAR, an approach to autonomously improve instruction following policies leveraging foundation models. SOAR breaks down the autonomous improvement problem into components that import ...