Introduction to Probability and Statistic

1. Basic Probability
and Statistic
Editor: Nguyen Duc Minh Khoi
Email: ducminhkhoi@gmail.com
Website: https://nguyenducminhkhoi.blogspot.com

2. Probability: The basics

3. Probability: The basics (cont.)
• Conditional Probability
• Independence

4. Probability: The basics (cont.)
• Rule of total Probability:
• Bayes Rule:
• Chain rule
      ii
BAPBPAp |
 
)(
)|()(
|
yP
xyPxP
yxP 

5. Probability: Random Variable (rv)
– PMF vs. PDF
• Discrete RV
• Continuous RV

6. Probability: Random Variable (rv)
- Cumulative Density Functions (CDF)

7. Probability: Expectations

8. Probability: Expectations (cont.)
• Conditional expectation

9. Probability: Expectations (cont.)
• Other important Values
• Example:

10. Probability: Important discrete rv

11. Probability: Important continuous rv

12. Probability: Multiple variables

13. Probability: Multiple variables
(Covariance & Correlation)

14. Law of large number & Central
Limit Theorem

15. Statistics
• Give data, how to find the model (pattern) of this
data.
• 2 school of thoughts:
• 𝜃in Bayesian is a rv (have prior p(𝜃));
𝜃 in Classical is unknown constant.

16. Classical Inference:
Maximum Likelihood Estimator (MLE)
Step: 1. Log; 2. derive; 3. solve for 𝜃

17. Classical Inference: Other methods
• Linear Regression

18. Bayesian Inference:

19. Bayesian Inference (cont.)

20. References
• http://www.stanford.edu/class/cme308/OldWebsit
e/notes/chap2.pdf
• http://ocw.mit.edu/courses/electrical-engineering-
and-computer-science/6-041sc-probabilistic-
systems-analysis-and-applied-probability-fall-
2013/resource-index/
• Bertsekas, Dimitri P. Introduction to Probability:
Dimitri P. Bertsekas and John N. Tsitsiklis.
Athena scientific, 2002.