Stochastic Processes

Discrete Time Processes

1 Discrete Time Stochastic Process

2 Discrete Time Markov Chains

3 Transition Matrices

4 Recurrence and Transience

5 Hitting Times

6 Classification of States

7 Balance Equations

8 Reversibility

9 Birth and Death Processes

10 Martingales

Continuous Time Processes

1 Continuous Time Stochastic Process

2 Continuous Time Markov Processes

3 Generating Functions

4 Poisson Processes

5 Queueing Systems