Markov Models & Optimization