Stochastic Processes

Welcome to the “Stochastic Processes” (CE-40695) course! This is a graduate level course that aims to provide a fundamental understanding of stochastic processes for computer science students.

Main References of the Course

  • Robert G. Gallager, “Stochastic Processes: Theory for Applications,” Cambridge University Press, 1st edition, Feb., 2014.
  • Athanasios Papoulis and S. Unnikrishna Pillai, “Probability, Random Variables and Stochastic Processes,” McGraw-Hill Europe, 4th edition, Jan., 2002.
  • George Casella and Roger L. Berger, “Statistical Inference,” Wadsworth Press, 2nd edition, Jun., 2001.

Sample Simulation Codes

Having access to computers and programming languages that can produce (though approximately) many random variables, it is a great opportunity that many aspects of stochastic processes can be show by writing simulation programs. In this page, I will gradually add more simulations that help students to have a better understanding of many stochastic processes concepts.

Spring 2021 Lectures

In the following, the lecture videos and notes of spring 2021 semester can be found (the lectures and notes are in Persian).

11399-11-25Introduction to probability:
Axioms of probability theory,
Random variables
Lec 1Note 1
21399-11-27Introduction to probability:
Random variables, Expected value,
Moments of a random variable
Lec 2Note 2
31399-12-02Introduction to probability:
Moment generating function,
Some probabilistic inequalities
Lec 3Note 3
41399-12-04Introduction to probability:
Law of large number,
Central limit theorem,
Convergence of random variables
Lec 4Note 4
51399-12-09Introduction to stochastic processes:
Basic definitions, Statistical
properties of stochastic processes
Lec 5Note 5
61399-12-11Introduction to stochastic processes:
Statistical properties of stochastic
processes, Stationary processes
Lec6Note 6
71399-12-16Introduction to stochastic processes:
Stationary processes,
Periodic processes
Lec 7Note 7
81399-12-18Introduction to stochastic processes:
Lec 8Note 8
91399-12-23Introduction to stochastic processes:
Systems with random input
Lec 9Note 9
101400-01-14Introduction to stochastic processes:
Systems with random input,
Power spectrum
Lec 10Note 10
111400-01-16Poisson processes:
Arrival and renewal processes,
Memoryless property, Definition
and properties of a Poisson process
Lec 11Note 11
121400-01-21Poisson processes:
Properties of a Poisson process
Lec 12Note 12
131400-01-23Poisson processes:
Other definitions of a Poisson process,
Combining and splitting Poisson processes
Lec 13Note 13
141400-01-28Gaussian processes:
Gaussian random variables and
random vectors, Jointly-Gaussian
random vectors
Lec 14Note 14
151400-01-30Gaussian processes:
Generating functions of Gaussian
random vectors, Gaussian processes
Lec 15Note 15
161400-02-04Finite-state Markov chains:
Representations of a Markov chain,
Classification of states
Lec 16Note 16
171400-02-06Finite-state Markov chains:
Classification of states,
Matrix representation, Steady state
Lec 17Note 17
181400-02-11Finite-state Markov chains:
Matrix representation, Steady state,
Steady state assuming P>0
Lec 18Note 18
191400-02-13Finite-state Markov chains:
Steady state assuming P>0,
Steady state for ergodic Markov chains
and unichains
Lec 19Note 19
201400-02-18Finite-state Markov chains:
Steady state for Markov chain with
two states
Lec 20Note 20
211400-02-25Finite-state Markov chains:
Steady state for arbitrary Markov chains

Estimation theory:
Statistics, sufficiency principle, and
sufficient statistic
Lec 21Note 21
221400-02-27Estimation theory:
Sufficient statistic, Factorization theorem
Lec 22Note 22
231400-03-01Estimation theory:
Minimal sufficient statistic
Lec 23Note 23
241400-03-08Estimation theory:
The likelihood principle

Point estimation:
Methods of finding estimators, Method
of moments
Lec 24Note 24
251400-03-10Point estimation:
Maximum likelihood estimators
Lec 25Note 25
261400-03-17Point estimation:
Bayes estimators, Methods of evaluating
estimators, Mean squared error
Lec 26Note 26
271400-03-25Point estimation:
Best unbiased estimators
Lec 27Note 27