New College, 1856-1906