Stochastic Processes

Welcome to the “Stochastic Processes” (CE-40695) course! This is a graduate level course that aims to provide a fundamental understanding of stochastic processes for computer science students.

Main References of the Course

  • Robert G. Gallager, “Stochastic Processes: Theory for Applications,” Cambridge University Press, 1st edition, Feb., 2014.
  • Athanasios Papoulis and S. Unnikrishna Pillai, “Probability, Random Variables and Stochastic Processes,” McGraw-Hill Europe, 4th edition, Jan., 2002.
  • George Casella and Roger L. Berger, “Statistical Inference,” Wadsworth Press, 2nd edition, Jun., 2001.

Sample Simulation Codes

Having access to computers and programming languages that can produce (though approximately) many random variables, it is a great opportunity that many aspects of stochastic processes can be show by writing simulation programs. In this page, I will gradually add more simulations that help students to have a better understanding of many stochastic processes concepts.

Spring 2021 Lectures

In the following, the lecture videos and notes of spring 2021 semester can be found (the lectures and notes are in Persian).

#DateSubjectVideoNote
11399-11-25Introduction to probability:
Axioms of probability theory,
Random variables		Lec 1Note 1
21399-11-27Introduction to probability:
Random variables, Expected value,
Moments of a random variable		Lec 2Note 2
31399-12-02Introduction to probability:
Moment generating function,
Some probabilistic inequalities		Lec 3Note 3
41399-12-04Introduction to probability:
Law of large number,
Central limit theorem,
Convergence of random variables		Lec 4Note 4
51399-12-09Introduction to stochastic processes:
Basic definitions, Statistical
properties of stochastic processes		Lec 5Note 5
61399-12-11Introduction to stochastic processes:
Statistical properties of stochastic
processes, Stationary processes		Lec6Note 6
71399-12-16Introduction to stochastic processes:
Stationary processes,
Periodic processes		Lec 7Note 7
81399-12-18Introduction to stochastic processes:
Ergodicity		Lec 8Note 8
91399-12-23Introduction to stochastic processes:
Ergodicity,
Systems with random input		Lec 9Note 9
101400-01-14Introduction to stochastic processes:
Systems with random input,
Power spectrum		Lec 10Note 10
111400-01-16Poisson processes:
Arrival and renewal processes,
Memoryless property, Definition
and properties of a Poisson process		Lec 11Note 11
121400-01-21Poisson processes:
Properties of a Poisson process		Lec 12Note 12
131400-01-23Poisson processes:
Other definitions of a Poisson process,
Combining and splitting Poisson processes		Lec 13Note 13
141400-01-28Gaussian processes:
Gaussian random variables and
random vectors, Jointly-Gaussian
random vectors		Lec 14Note 14
151400-01-30Gaussian processes:
Generating functions of Gaussian
random vectors, Gaussian processes		Lec 15Note 15
161400-02-04Finite-state Markov chains:
Representations of a Markov chain,
Classification of states		Lec 16Note 16
171400-02-06Finite-state Markov chains:
Classification of states,
Matrix representation, Steady state		Lec 17Note 17
181400-02-11Finite-state Markov chains:
Matrix representation, Steady state,
Steady state assuming P>0		Lec 18Note 18
191400-02-13Finite-state Markov chains:
Steady state assuming P>0,
Steady state for ergodic Markov chains
and unichains		Lec 19Note 19
201400-02-18Finite-state Markov chains:
Steady state for Markov chain with
two states		Lec 20Note 20
211400-02-25Finite-state Markov chains:
Steady state for arbitrary Markov chains
—
Estimation theory:
Statistics, sufficiency principle, and
sufficient statistic		Lec 21Note 21
221400-02-27Estimation theory:
Sufficient statistic, Factorization theorem		Lec 22Note 22
231400-03-01Estimation theory:
Minimal sufficient statistic		Lec 23Note 23
241400-03-08Estimation theory:
The likelihood principle
—
Point estimation:
Methods of finding estimators, Method
of moments		Lec 24Note 24
251400-03-10Point estimation:
Maximum likelihood estimators		Lec 25Note 25
261400-03-17Point estimation:
Bayes estimators, Methods of evaluating
estimators, Mean squared error		Lec 26Note 26
271400-03-25Point estimation:
Best unbiased estimators		Lec 27Note 27