Summary
Markov process is a stochastic process with no memory
Full specification of process is given by a matrix of transition probabilities P
A distribution of states are generated by repeatedly stepping from one state to another according to P
A desired limiting distribution can be used to construct transition probabilities using detailed balance
- Many different P matrices can be constructed to satisfy detailed balance
- Metropolis algorithm is one such choice, widely used in MC simulation
Markov Monte Carlo is good for evaluating averages, but not absolute integrals
Next up: Monte Carlo simulation