Learning from the Chosen One