Probability & Bayesian Theorem

1. Probability
Dr Azmi Mohd Tamil
Dept of Community Health
Universiti Kebangsaan Malaysia

2. Sample Spaces
4 A sample space is the set of all possible outcomes.
However, some sample spaces are better than others.
4 Consider the experiment of flipping two coins. It is
possible to get 0 heads, 1 head, or 2 heads. Thus, the
sample space could be {0, 1, 2}. Another way to look
at it is flip { HH, HT, TH, TT }. The second way is
better because each event is as equally likely to occur
as any other.
4 When writing the sample space, it is highly desirable
to have events which are equally likely.

3. Sample Spaces
4 Another example is rolling two dice. The
sums are { 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12 }. However, each of these aren't
equally likely. The only way to get a sum
2 is to roll a 1 on both dice, but you can
get a sum of 4 by rolling a 3-1, 2-2, or 3-
1. The following table illustrates a better
sample space for the sum obtain when
rolling two dice.

4. Example
Second Die
First Die 1 2 3 4 5 6
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12

5. Classical Probability
Sum Freq Relative Freq 4 The relative frequency of a
frequency distribution is the
probability of the event
2 1 1/36 occurring. This is only true,
3 2 2/36 however, if the events are
4 3 3/36 equally likely.
5 4 4/36
4 This gives us the formula for
6 5 5/36 classical probability. The
7 6 6/36 probability of an event occurring
8 5 5/36 is the number in the event
9 4 4/36 divided by the number in the
sample space.
10 3 3/36
11 2 2/36 4 P(E) = n(E) / n(S)
12 1 1/36

6. Empirical Probability
4 Empirical probability
is based on
4 P(E) = f / n
observation. The
empirical probability
of an event is the
relative frequency of
a frequency
distribution based
upon observation.

7. Probability Rules
4 All probabilities are between 0 and 1 inclusive
0 <= P(E) <= 1
4 The sum of all the probabilities in the sample
space is 1
4 The probability of an event which cannot
occur is 0.
4 The probability of an event which must occur
is 1.
4 The probability of an event not occurring is
one minus the probability of it occurring.
P(E') = 1 - P(E)

8. Mutually Exclusive Events
4 Two events are mutually exclusive if
they cannot occur at the same time.
4 If two events are mutually exclusive ,
then the probability of them both
occurring at the same time is 0.
Mutually Exclusive : P(A and B) = 0
4 If two events are mutually exclusive,
then the probability of either occurring is
the sum of the probabilities of each
occurring.

9. Specific Addition Rule
4 Only valid when the events are mutually
exclusive.
P(A or B) = P(A) + P(B)

10. Example 1
4 Given:
P(A) = 0.20, P(B) = 0.70, A and B
are mutually exclusive
B B' Total
A 0 0.2 0.2
A' 0.7 0.1 0.8
Total 0.7 0.3 1

11. Non-Mutually Exclusive Events
4 In events which aren't mutually
exclusive, there is some overlap. When
P(A) and P(B) are added, the probability
of the intersection (and) is added twice.
To compensate for that double addition,
the intersection needs to be subtracted.
4 General Addition Rule
P(A or B) = P(A) + P(B) - P(A and B)

12. Example 2
4 Given P(A) = 0.20, P(B) = 0.70, P(A and
B) = 0.15
B B' Total
A 0.15 0.05 0.2
A' 0.55 0.25 0.8
Total 0.7 0.3 1

13. Independent Events
4 Two events are independent if the occurrence
of one does not change the probability of the
other occurring.
4 An example would be rolling a 2 on a die and
flipping a head on a coin. Rolling the 2 does
not affect the probability of flipping the head.
4 If events are independent, then the probability
of them both occurring is the product of the
probabilities of each occurring.

14. Specific Multiplication Rule
4 Only valid for independent events
P(A and B) = P(A) * P(B)

15. Example 3
4 P(A)= 0.20, P(B) = 0.70, A and B are
independent.
B B' Total
A 0.14 0.06 0.2
A' 0.56 0.24 0.8
Total 0.7 0.3 1

16. Dependent Events
4Ifthe occurrence of one event
does affect the probability of the
other occurring, then the events
are dependent.

17. Conditional Probability
4The probability of event B occurring
that event A has already occurred is
read "the probability of B given A"
and is written: P(B|A)
4 GeneralMultiplication Rule
P(A and B) = P(A) * P(B|A)

18. Example 4
4 P(A) = 0.20, P(B) = 0.70, P(B|A) = 0.40
B B' Total
A 0.08 0.12 0.2
A' 0.62 0.18 0.8
Total 0.7 0.3 1

19. Independence Revisited
4 Thefollowing four statements are
equivalent
1.A and B are independent events
2.P(A and B) = P(A) * P(B)
3.P(A|B) = P(A)
4.P(B|A) = P(B)

20. The question, "Do you smoke?" was asked of
100 people. Results are shown in the table.
. Yes No Total
Male 19 41 60
Female 12 28 40
Total 31 69 100
•What is the probability of a randomly selected individual being a male who smokes? This is just a
joint probability. The number of "Male and Smoke" divided by the total = 19/100 = 0.19
•What is the probability of a randomly selected individual being a male? This is the total for male
divided by the total = 60/100 = 0.60. Since no mention is made of smoking or not smoking, it
includes all the cases.
•What is the probability of a randomly selected individual smoking? Again, since no mention is
made of gender, this is a marginal probability, the total who smoke divided by the total = 31/100 =
0.31.
•What is the probability of a randomly selected male smoking? This time, you're told that you have
a male - think of stratified sampling. What is the probability that the male smokes? Well, 19 males
smoke out of 60 males, so 19/60 = 0.31666...
•What is the probability that a randomly selected smoker is male? This time, you're told that you
have a smoker and asked to find the probability that the smoker is also male. There are 19 male
smokers out of 31 total smokers, so 19/31 = 0.6129 (approx)

21. There are three major manufacturing companies that make a product:
Aberations, Brochmailians, and Chompielians. Aberations has a 50% market
share, and Brochmailians has a 30% market share. 5% of Aberations' product is
defective, 7% of Brochmailians' product is defective, and 10% of Chompieliens'
product is defective.
This information can be placed into a joint probability distribution
Company Good Defective Total
Aberations 0.50-0.025 = 0.475 0.05(0.50) = 0.025 0.50
Brochmailians 0.30-0.021 = 0.279 0.07(0.30) = 0.021 0.30
Chompieliens 0.20-0.020 = 0.180 0.10(0.20) = 0.020 0.20
Total 0.934 0.066 1.00
•What is the probability a randomly selected product is defective?
•What is the probability that a defective product came from
Brochmailians?

22. The percent of the market share for Chompieliens wasn't given, but since the marginals must
add to be 1.00, they have a 20% market share.
Notice that the 5%, 7%, and 10% defective rates don't go into the table directly. This is
because they are conditional probabilities and the table is a joint probability table. These
defective probabilities are conditional upon which company was given. That is, the 7% is not
P(Defective), but P(Defective|Brochmailians). The joint probability P(Defective and
Brochmailians) = P(Defective|Brochmailians) * P(Brochmailians).
The "good" probabilities can be found by subtraction as shown above, or by multiplication
using conditional probabilities. If 7% of Brochmailians' product is defective, then 93% is good.
0.93(0.30)=0.279.
•What is the probability a randomly selected product is defective? P(Defective) = 0.066
•What is the probability that a defective product came from Brochmailians?
P(Brochmailian|Defective) = P(Brochmailian and Defective) / P(Defective) = 0.021/0.066 =
7/22 = 0.318 (approx).
•Are these events independent? No. If they were, then P(Brochmailians|Defective)=0.318
would have to equal the P(Brochmailians)=0.30, but it doesn't. Also, the P(Aberations and
Defective)=0.025 would have to be P(Aberations)*P(Defective) = 0.50*0.066=0.033, and it
doesn't.

23. Bayes' Theorem
Let's use the same example, but shorten each event to its one letter initial, ie: A, B,
C, and D instead of Aberations, Brochmailians, Chompieliens, and Defective.
P(D|B) is not a Bayes problem.
This is given in the problem. Bayes' formula finds the reverse conditional probability
P(B|D).
It is based that the Given (D) is made of three parts, the part of D in A, the part of D
in B, and the part of D in C.
P(B and D)
P(B|D) = -----------------------------------------------------
P(A and D) + P(B and D) + P(C and D)
Inserting the multiplication rule for each of these joint probabilities gives
P(D|B)*P(B)
P(B|D) = ------------------------------------------------------------
P(D|A)*P(A) + P(D|B)*P(B) + P(D|C)*P(C)