COURSE DESCRIPTION
Discrete stochastic processes are essentially probabilistic systems that evolve in time via random changes occurring at discrete fixed or random intervals. This course aims to help students acquire both the mathematical principles and the intuition necessary to create, analyze, and understand insightful models for a broad range of these processes. The range of areas for which discrete stochastic-process models are useful is constantly expanding, and includes many applications in engineering, physics, biology, operations research and finance.
Syllabus
Course Meeting Times
Lectures: 2 sessions / week, 1.5 hours / session
Course Description
Discrete stochastic processes are essentially probabilistic systems that evolve in time via random changes occurring at discrete fixed or random intervals. This course aims to help students acquire both the mathematical principles and the intuition necessary to create, analyze, and understand insightful models for a broad range of these processes. The range of areas for which discrete stochastic-process models are useful is constantly expanding, and includes many applications in engineering, physics, biology, operations research and finance.
Prerequisites
Thorough understanding of elementary probability at the level of , which uses the following text:
Bertsekas, Dimitri, and John Tsitsiklis. . 2nd ed. Athena Scientific, 2008. ISBN: 9781886529236.
Some patience and affinity for careful mathematical reasoning.
Video Lectures
Lecture 1: Introduction and Probability Review
Description: Probability, as it appears in the real world, is related to axiomatic mathematical models. Events, independence, and random variables are reviewed, stressing both the axioms and intuition.
Instructor: Prof. Robert Gallager
Description: The review of probability is continued with expectation, multiple random variables, and conditioning. We then move on to develop the weak law of large numbers (WLLN) and the Bernoulli process.
Instructor: Prof. Robert Gallager
Lecture 3: Law of Large Numbers, Convergence
Description: This lecture begins with the use of the WLLN in probabilistic modeling. Next the central limit theorem, the strong law of large numbers (SLLN), and convergence are discussed.
Instructor: Prof. Robert Gallager
Lecture 4: Poisson (The Perfect Arrival Process)
Description: This lecture begins with a description of arrival processes, and continues on to describe the Poisson process from three different viewpoints.
Instructor: Prof. Robert Gallager
Lecture 5: Poisson Combining and Splitting
Description: In this lecture, many problem solving techniques are developed using, first, combining and splitting of various Poisson processes, and, second, conditioning on the number of arrivals in an interval.
Instructor: Prof. Robert Gallager
Lecture 6: From Poisson to Markov
Description: This lecture treats joint conditional densities for Poisson processes and then defines finite-state Markov chains. Recurrent and transient states, periodic states, and ergodic chains are discussed.
(Courtesy of Mina Karzand. Used with permission.)
Instructor: Mina Karzand
Lecture 7: Finite-state Markov Chains; The Matrix Approach
Description: The transition matrix approach to finite-state Markov chains is developed in this lecture. The powers of the transition matrix are analyzed to understand steady-state behavior.
(Courtesy of Shan-Yuan Ho. Used with permission.)
Instructor: Shan-Yuan Ho
Lecture 8: Markov Eigenvalues and Eigenvectors
Description: This lecture covers eigenvalues and eigenvectors of the transition matrix and the steady-state vector of Markov chains. It also includes an analysis of a 2-state Markov chain and a discussion of the Jordan form.
Instructor: Prof. Robert Gallager
