Instructor

Chung-Chin Lu (呂忠津) (EECS Building Room 609, ext. 31145 )

Teaching Assistants

Dong-Tsai Huang(黃棟材) (EECS Building Room 608, ext. 34032)

Lecture Hours

T3T4R3R4
Lecture at EECS Building Room 104

TA Office Hours

Wed. 14:00~18:00 at EECS Building Room 609.

Homework

	Homework	Due day	Reference Solution
Homework 1	1.1, 1.5, 1.6, 1.9, 1.16, 1.17, 1.19	09-24-2009	To be distributed in class
Homework 2	1.13, 1.23, 1.24, 1.27, 1.34	10-01-2009	To be distributed in class
Homework 3	1.20, 1.31, 1.37, 1.38, 1.41	10-08-2009	To be distributed in class
Homework 4	2.2, 2.9, 2.17, 2.19, 2.22	10-15-2009	To be distributed in class
Homework 5	part I: 2.29, 2.32 part II: 2.33, 2.39, 2.41	10-27-2009	To be distributed in class
Homework 6	3.2, 3.3, 3.4, 3.8, 3.9	10-29-2009	To be distributed in class
Homework 7	3.6, 3.12,3.15, 3.16, 3.17		To be distributed in class
Homework 8	3.22, 3.24, 3.25, 3.27, 3.31		To be distributed in class
Homework 9	4.1, 4.4, 4.8, 4.9, 4.10		To be distributed in class
Homework 10	4.13, 4.16, 4.17, 4.19, 4.21, 4.26, 4.29, 4.31		To be distributed in class
Homework 11	4.33, 4.42, 4.47, 4.48, 4.50		To be distributed in class
Homework 12	5.2, 5.4, 5.7, 5.13, 5.15		To be distributed in class
Homework 13	5.17, 5.19, 5.23, 5.32, 5.36		To be distributed in class
Homework 14
Homework 15

Course Description

This course discusses mathematical models for physical phenomena which possess random features. Physical phenomena which occur in the study of networking, economics, finance, control, communications, signal processing, statistical physics and biological processes often present random behaviors. Stochastic processes which are useful in the modeling of these random phenomena will be discussed, including point processes, Markov processes, second-order processes and martingales. This course requires knowledge on calculus and elementary probability theory. It is recommended for students to read chapter 1 of the textbook by Ross before class begins.

Text Books

1. S. M. Ross, Stochastic Processes, 2nd edn. John Wiley & Sons, Inc., 1996.
    (chapters 1- 8; you have to buy a copy from a bookstore)
2. C.-C. Lu, Lecturenotes on Stochastic Processes. Department of Electrical
    Engineering, National Tsing Hua University, 2009.
    (chapter 3; this chapter can be downloaded from my website in early November)

Reference

I. Probability Theory
I.1 S. Ross, A First Course in Probability, 7th edn. Prentice Hall, 2005.
(a very good first course on this subject)
I.2 K.-L. Chung, A Course in Probability Theory, 2nd edn.
New York: Academic Press, 1974. (highly recommended)
I.3 A. N. Shiryayev, Probability. New York: Springer-Verlag, 1984.
(chapters 1-4)
I.4 C.-C. Lu, Lecturenotes on Stochastic Processes. Department of Electrical
Engineering, National Tsing Hua University, 2009.
(chapter 1)

II. Stochastic Processes
II.1 E. Cinlar, Introduction to Stochastic Processes. Englewood Cliffs,
NJ: Prentice-Hall Inc., 1975.
II.2 E. Wong and B. Hajek, Stochastic Processes in Engineering Systems.
New York: Springer-Verlag, 1985.
II.3 J. Lamperti, Stochastic Processes. New York: Springer-Verlag,1977.
II.4 A. N. Shiryayev, Probability. New York: Springer-Verlag, 1984.
(chapters 5-8)

Teaching Method

This is an English–teaching course. After a quick review on essentials of probability theory in the first chapter of the textbook by Ross, we will cover Poisson process, renewal theory and discrete-time Markov chains (chapters 2 – 4 of Ross) before the first midterm in early November. Then we will continue to cover continuous-time Markov chain (chapter 5 of Ross) and second-order processes (chapter 3 of Lu) before the second midterm in early December. Finally we will study martingale theory wil applications to random walks, Brownian motion and other Markov processes before the final exam in mid January. Homeworks will be assigned weekly, graded and can be retrieved from my web site. And homework solutions will be distributed in the classroom.

Syllabus

1. Preliminaries: a review of probability theory (3 weeks)
2. The Poisson process (1.5 weeks)
3. Renewal theory (1.5 weeks)
4. Discrete-time Markov chains (1.5 weeks)
5. Continuous-time Markov chains (1.5 weeks)
6. Second-order processes (3 weeks)
7. Martingales (2 weeks)
8. Random walks (1 week)
9. Brownian motion and Markov processes (2 weeks)

Grading

There are weekly homeworks (25%), two midterms (50%) and one final (25%).
The time schedule is as follows:
(1) Homeworks will be assigned every Thursday and should be handed in on the
next Thursday. Homeworks will be posted on my website and homework
solutions will be distributed in the classroom.
(2) Midterm I – 7:00 – 10:00 pm, November 12, 2008
Coverage: Poisson process, renewal theory and discrete-time Markov chains
(3) Midterm II – 7:00 – 10:00 pm, December 17, 2008
Coverage: continuous-time Markov chains and second-order processes
(4) Final – 9:00 am – noon, January 14, 2010
Coverage: martingales, random walks, Brownian motion and Markov processes