Retrievers and How to Break Them