Module manager: Dr Benjamin Lees
Email: B.T.Lees@leeds.ac.uk
Taught: Semester 1 (Sep to Jan) View Timetable
Year running 2025/26
| MATH1000 | Core Mathematics |
| MATH1700 | Probability and Statistics |
| MATH2715 | Statistical Methods |
MATH2750 Introduction to Markov Processes
This module is not approved as a discovery module
This module introduces students to stochastic processes: any quantity which changes randomly in time, such as the capacity of a reservoir, an individual’s level of no claims discount and the size of a population. The linking model for all these examples is the Markov process, which is defined. Appropriate modifications are then introduced to extend the Markov process to model stochastic processes which change over continuous time, not just at regularly spaced time points.
The aim of this module is to study stochastic processes, with a particular emphasis on Markov processes by building a rigorous theory that naturally follows on from first year probability. Applications to areas such as Biology and Physics, as well as financial and actuarial sciences will also be explored.
On successful completion of the module students will have demonstrated the following learning outcomes:
1. Explain key features of stochastic process including the Poisson process;
2. Define the random walk and implications of some classic results on the behaviour of random walks;
3. Solve hitting time and expected duration problems relating to random walks;
4. Define and classify Markov chain models;
5. Calculate long-term probability distributions for simple stochastic process models;
6. Define continuous time Markov jump processes.
Skills Learning Outcomes
On successful completion of the module students will have demonstrated the following skills learning outcomes:
a. Understand important and critical concepts of stochastic processes and some of their important applications and examples.
b. The ability to write in a clear, concise, focused and structured manner that is supported by relevant evidence.
c. The ability to take a logical approach to solving problems; understand and apply relevant theory and interpret results/data appropriately.
d. The ability to prioritise, work efficiently and manage your time well in order to meet deadlines.
e. Use technology appropriately in work and studies, such as access to resources or use of programming languages.
1. Difference between deterministic and stochastic models. The role of models.
2. Definitions of stochastic processes, state space and time, mixed processes, the Markov property.
3. Random walks and their properties, such as transition probabilities, hitting times, recurrence and transience, absorbing and reflecting barriers, gambler's ruin problem. Examples, eg insurance models, financial indexes, examples from physics, biology, or other areas of science.
4. General theory of Markov chains: transition matrix, Chapman-Kolmogorov equations, classification of states, stationary distribution, convergence to equilibrium.
5. Application of Markov chain models.
6. Poisson process and its properties. Counting processes.
7. Markov processes in continuous time with discrete state space: transition rates, forward and backward equations, stationary distribution
8. Simulation of stochastic processes.
Additional topics that build on these may be covered as time allows. Further details of possible topics will be delivered closer to the time that the module runs.
Methods of assessment
The assessment details for this module will be provided at the start of the academic year
| Delivery type | Number | Length hours | Student hours |
|---|---|---|---|
| Lecture | 22 | 1 | 22 |
| Practical | 5 | 1 | 5 |
| Seminar | 5 | 1 | 5 |
| Private study hours | 68 | ||
| Total Contact hours | 32 | ||
| Total hours (100hr per 10 credits) | 100 | ||
The reading list is available from the Library website
Last updated: 28/05/2025
Errors, omissions, failed links etc should be notified to the Catalogue Team