Ghosts of Inflation Past