Stochastic Processes

Prerequisites:

Probability theory

Course description:

This course is intended for undergraduate students planning to pursue further study in pure or applied mathematics, as well as for those interested in the rigorous mathematical modeling of random phenomena. Its primary objective is to equip students with the fundamental concepts, terminology, and tools of stochastic processes, thereby laying a solid foundation for advanced study in modern probability theory and mathematical physics. The course emphasizes core mathematical structures such as probability spaces, Markov chains, jump processes, and Brownian motion.

Learning Objectives：

By the end of this course, students will be able to:

Understand and use the fundamental language of probability theory, including probability spaces, random variables, distributions, and expectations, with mathematical rigor.

Formulate and analyze stochastic processes using precise definitions and structural viewpoints, with particular emphasis on Markov chains, jump processes, and Brownian motion.

Apply core analytical and probabilistic tools to study the behavior of stochastic processes, including transition mechanisms, hitting probabilities, and long-term behavior.

Develop rigorous mathematical reasoning skills for modeling and analyzing random phenomena, bridging abstract theory with concrete examples.

Read and understand advanced mathematical texts in probability and stochastic processes, preparing for further study in modern probability theory and mathematical physics.

Solve theoretical and computational problems involving stochastic processes, both independently and through collaborative problem-solving.

Detailed Topics Covered:

1. Preliminaries:

(1) Probability Foundations: Sample spaces, sigma-algebras, probability measures, and random variables.

(2) Advanced Concepts: Independence, Borel-Cantelli lemmas, modes of convergence (almost sure, in probability).

(3) Conditioning: Conditional probability and Conditional Expectation (definitions, properties, and computation).

2. Markov Chains:

(1) Definitions: Discrete-time Markov chains, state spaces, and transition probability matrices.

(2) Structural Properties: Chapman-Kolmogorov equations, communicating classes, irreducibility, and periodicity.

(3) Limit Theorems: Classification of states (transience and recurrence), existence and uniqueness of Stationary Distributions, and ergodic theorems.

(4) Special Topics: Reversibility, Random Walks on graphs, and stopping times.

3. Jump Processes:

(1) Poisson Processes: Definition (counting process vs. inter-arrival times), properties of the exponential distribution, superposition, and thinning.

(2) Continuous-Time Markov Chains (CTMC): Q-matrices (infinitesimal generators), forward and backward equations, and the embedded discrete Markov chain.

(3) Birth-and-Death Processes: Stationary measures, explosion criteria, and applications to queueing theory and population dynamics.

4. Brownian Motion:

(1) Fundamentals: Definition of standard Brownian Motion (Wiener Process), finite-dimensional distributions, and Gaussian processes.

(2) Path Properties: Continuity, quadratic variation, and non-differentiability of paths.

(3) Key Results: The Reflection Principle, distribution of the maximum, hitting times, and the invariance principle (Donsker's Theorem).

(4) (Optional Extension): Brief introduction to stochastic integration or geometric Brownian motion as applied to finance.