Computations with Markov Chains