Moving Forward by Looking Backwards