Lord Dufferin, Ireland and the British Empire, c. 1820–1900