From Shortest Paths to Reinforcement Learning