Learning from the Secret Past