basic probability concepts

1. Irrigation and Hydraulics Department
Faculty of Engineering – Cairo University
Basic Probability Concepts
by
Dr. Mohamed Attia
Assistant Professor
Academic Year 2011-2012
Probability Experiments
A random experiment is an experiment that
can result in different outcomes, even though
it is repeated in the same manner every time.
1

2. Probability Experiments
An experiment:
Flipping a coin once.
Rolling a die once.
Pulling a card from a deck.
Sample Spaces
The set of all possible outcomes of a random
experiment is called a sample space.
You may think of a sample space as the set of
all values that a variable may assume.
We are going to denote the sample space by S.
2

3. Examples
Experiment: Tossing a coin once.
S = {H, T}
Experiment: Rolling a die once.
S = {1, 2, 3, 4, 5, 6}
Examples
Experiment: Drawing a card.
S = 52 cards in a deck
K, Q, J, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 (or Ace)
13 spades (♠), 13 hearts (♥), 13 diamonds (♦)
and 13 clubs (♣)
3

4. Classification of Sample
Spaces
We distinguish between discrete and
continuous sample spaces.
The outcomes in discrete sample spaces can
be counted. The number of outcomes can be
finite or infinite.
In the case of continuous sample spaces, the
outcomes fill an entire region in the space
(interval on the line).
Events
An event, E, is a set of outcomes of a probability
experiment, i.e. a subset in the sample space.
E ⊆ S.
Simple event
Outcome from a sample space with one
characteristic
e.g.: A red card from a deck of cards
Joint event
Involves two outcomes simultaneously
e.g.: An ace that is also red from a deck of cards
4

5. Visualizing Events
Contingency Tables
Ace Not Ace Total
Black 2 24 26
Red 2 24 26
Total 4 48 52
Tree Diagrams Ace
Red
Full Cards Not an Ace
Deck Black Ace
of Cards Cards Not an Ace
Visualizing Events
Sometimes it is convenient to represent the
sample space by a rectangular region, with
events being circles within the region. Such a
representation is called Venn diagrams.
Venn Diagram
A B
Sample space S
S
Events A and B
5

6. Simple Events
The Event of a Triangle
There are 5 triangles in this collection of 18 objects
Joint Events
The event of a triangle AND red in color
Two triangles that are red
6

7. Special Events
Impossible event
Null Event
e.g.: Club & diamond on one card
draw
♣♣
Complement of event
For event A, all events not in A
Denoted as A’
e.g.: A: queen of diamonds
A’: all cards in a deck that are
not queen of diamonds
Special Events
Complement of an event
Sample space S A’
A
Event A
S
Complement of A A’
7

8. Mutually Exclusive Events
Two events, E and F, are called mutually
exclusive, if
EIF =Φ
That is, it is impossible for an outcome to be
an occurrence of both events.
e.g.: A: queen of diamonds; B: queen of
clubs
Events A and B are mutually exclusive
Collectively exhaustive events
One of the events must occur
The set of events covers the whole sample space
e.g.: -- A: all the aces; B: all the black cards; C: all
the diamonds; D: all the hearts
Events A, B, C and D are collectively
exhaustive
Events B, C and D are also collectively
exhaustive
8

9. Special Events
Collectively Exhaustive Events
A, B, C, D are collectively exhaustive
A and B are NOT mutually exclusive
A and C are NOT mutually exclusive
A and D are NOT mutually exclusive
B, C, and D are also collectively exhaustive
B, C, and D are mutually exclusive A
C D
B
♦ ♥
Special Events
AU B A B
S
AI B A B
S
9

10. Contingency Table
A Deck of 52 Cards
Red Ace
Ace Not an Total
Ace
Red 2 24 26
Black 2 24 26
Total 4 48 52
Sample
Space
Tree Diagram
Event Possibilities
Ace
Red
Cards Not an Ace
Full
Deck of Ace
Cards
Black
Cards Not an Ace
10

11. Probability
Probability is the numerical
measure of the likelihood
1 Certain
that an event will occur
The probability of an event E
is a number, P(E), such that
0 ≤ P( E ) ≤ 1 .5
Sum of the probabilities of
all mutually exclusive and
collective exhaustive events
is 1 0 Impossible
P( S ) = 1
Computing Probabilities
The probability of an event E:
number of event outcomes
P( E ) =
total number of possible outcomes in the sample space
X
=
T e.g. P( ) = 2/36
(There are 2 ways to get one 6 and the other 4)
Each of the outcomes in the sample space is
equally likely to occur
11

12. Computing Joint Probability
The probability of a joint event, A and B:
P (A and B ) = P (A ∩ B )
number of outcomes from both A and B
=
total number of possible outcomes in sample space
E.g. P (Red Card and Ace)
2 Red Aces 1
= =
52 Total Number of Cards 26
Joint Probability Using
Contingency Table
Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)
A2 P(A2 and B1) P(A2 and B2) P(A2)
Total P(B1) P(B2) 1
Joint Probability Marginal (Simple) Probability
12

13. Computing Compound (or
Multiple) Probability
Probability of a compound event, A or B:
P( A or B ) = P( A ∪ B )
number of outcomes from either A or B or both
=
total number of outcomes in sample space
E.g. P (Red Card or Ace)
4 Aces + 26 Red Cards - 2 Red Aces
=
52 total number of cards
28 7
= =
52 13
Compound Probability
(Addition Rule)
P(A1 or B1 ) = P(A1) + P(B1) - P(A1 and B1)
Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)
A2 P(A2 and B1) P(A2 and B2) P(A2)
Total P(B1) P(B2) 1
For Mutually Exclusive Events: P(A or B) = P(A) + P(B)
13

14. Computing Conditional Probability
The probability of event A given that event B has
occurred:
P ( A and B ) A B
P( A | B) =
P( B) A&B
It means the likelihood of realizing a sample point in
A assuming that it belongs to B
E.g.
P(Red Card given that it is an Ace)
2 Red Aces 1/52
= = = 0.5
4 Aces 2 /52
Conditional Probability Using
Contingency Table
Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
Revised Sample Space
P (Ace and Red) 2 / 52 2
P (Ace | Red) = = =
P(Red) 26 / 52 26
14

15. Conditional Probability and
Statistical Independence
Conditional probability:
P ( A and B)
P( A | B) =
P( B)
Multiplication rule:
P ( A and B) = P ( A | B ) P ( B )
= P( B | A) P ( A)
Conditional Probability and
Statistical Independence (continued)
Events A and B are independent if
P( A | B) = P ( A)
or P( B | A) = P ( B )
or P( A and B) = P( A) P( B)
Events A and B are independent when the
probability of one event, A, is not affected by
another event, B
15

16. Bayes’s Theorem
B1, B2, …, Bk are mutually exclusive and
collectively exhaustive
Suppose it is known that event A has
occurred
What is the (conditional) probability that event
Bi has occurred?
B1 B2 Bk
A
Bayes’s Theorem
P ( A | Bi ) P ( Bi )
P ( Bi | A ) =
P ( A | B1 ) P ( B1 ) + • • • + P ( A | Bk ) P ( Bk )
P ( Bi and A )
=
P ( A) Adding up
the parts
Same of A in all
Event the B’s
16

17. Bayes’s Theorem Using Contingency
Table (Example 1)
Fifty percent of graduating engineers in a certain year
worked as water resources (WR) managers. Out of those
who worked as WR managers, 40% had a Masters degree.
Ten percent of those graduating engineers who did not work
as WR managers had a Masters degree. What is the
probability that a randomly selected engineer who has a
Masters degree is a WR manager?
P (WR) = 0.5 P( M / WR ) = 0.4 P( M / W R ) = 0.1
P (WR / M ) = ?
Bayes’s Theorem
Using Contingency Table (continued)
WR WR Total
Masters .2 .05 .25
Masters .3 .45 .75
Total .5 .5 1.0
P( M | WR ) P (WR )
P (WR / M ) =
P( M | WR ) P (WR ) + P( M | W R ) P(W R )
(0.4)(0.5) 0.2
= = = 0.8
(0.4)(0.5) + (0.1)(0.5) 0.25
17

18. Example 2
A water treatment plant may fail for two reasons: inadequacy
of materials (event A) or mechanical failure (event B).
If P(A) = 2 P(B), P(A | B) = 0.8 and the probability of failure of
the treatment plant equals 0.001 , what is the probability that
a mechanical failure occurs? What is the probability of a
failure due to inadequate materials?
The probability of failure P (A U B) may be written as:
P (A U B) = P (A) + P (B) – P (A ∩ B)
= 2 P (B) + P (B) – P (B) P (A | B)
= 3 P (B) – 0.8P (B)
0.001 = 3 P (B) – 0.8P (B)
P (B) = 0.00045
P (A) = 0.0009
Example 3
Each of two pumps Q1 and Q2 may not operate. On inspection
one may observe one of the outcomes of the following set: {(ƒ,
ƒ), (ƒ, o), (o, ƒ), (o, o)}. The notation (ƒ, o) means that pump Q1
fails and pump Q2 operates. The sample space has four
elements. From experience one knows that:
P ((ƒ, ƒ)) = 0.1, P ((ƒ, o )) = 0.2
P ((o, ƒ)) = 0.3, P ((o, o )) = 0.4
Let
A: Q1 operates
B: Q2 operates
C: at least one of the pumps operates
Compute P(A), P(B), P(A ∩ B), P(C)
18

19. Example 3 (Cont.)
P ( A ) = 0.7
P ( B ) = 0.6
P ( A ∩ B ) = P (( o , o )) = 0.4
P(C)=(AUB)
= P ( A ) + P ( B ) – P ( A ∩ B ) = 0.9
Example 4
The demand of a water supply system can be low (event L) ,
moderate (event M) or high (event H) with known probabilities
P(L) = 0.1, P(M) = 0.7 and P(H) = 0.2. Failure of the system (event
F) can only occur if a certain pump fails to function. From past
experience the following conditional probabilities are known:
P(F | L) = 0.05, P(F | M) = 0.10 and P(F | H) = 0.25. What is the
probability of a pump failure P(F).
First we observe that the events L, M and H are clearly disjoint
and that their union is just the sure event S. Hence
P ( F ) = P (( L ∩ F ) U ( M ∩ F ) U ( H ∩ F ))
= P (L ∩ F) + P (M ∩ F) + P (H ∩ F)
Using the definition of conditional probability, we get
P ( F ) = P (L) P (F | L) + P (M) P (F | M) + P (H) P (F | H)
= 0.1 × 0.05 + 0.7 × 0.10 + 0.2 × 0.25
= 0.125
19

20. Example 5
Weather forecast information is sent using one of four
routes. Let Ri denote the event that route i is used for
sending a message (i = 1, 2, 3, 4). The probabilities of using
the four routes are 0.1, 0.2, 0.3 and 0.4, respectively. Further,
it is known that the probabilities of an error being introduced
while transmitting messages are 0.10, 0.15, 0.20 and 0.25 for
routes 1, 2, 3 and 4, respectively. In sending a message an
error occurred. What is the probability that route 2 was used
for transmission?
Let E denote the event of having an erroneous message
P(E | R1) = 0.10, P(E | R2) = 0.15,
P(E | R3) = 0.20, P(E | R4) = 0.25.
Example 5 (Cont.)
Using Bayes' rule one can compute the probability that the
message was sent via route 2 given the event that an error
has occurred. This probability is given by
P( R2 I E ) P( R2 ) P( E | R2 )
P(R2 | E) = = 4
P( E ) ∑ i =1
P( Ri ) P( E | Ri )
0.2 × 0.15
=
0.1× 0.10 + 0.2 × 0.15 + 0.3 × 0.20 + 0.4 × 0.25
0.03
= = 0.15.
0.20
20

21. Example 6
The probability of receiving more than 50 mm of rain in the
months of the year is 0.25, 0.30, 0.35, 0.40, 0.20, 0.10, 0.05,
0.05, 0.05, 0.05, 0.10 and 0.20 for January, February, ...,
December, respectively. A monthly rainfall record selected at
random is found to be more than 50 mm. What is the
probability that this record belongs to month m (m = 1 for
January, ..., m = 12 for December).
Denote the probability of selecting month m as P(Mm).
Denote the conditional probability of receiving more than 50
mm of rain, given that month m is selected as P(E | Mm).
Then P(Mm | E) can be computed using Bayes' rule as
indicated in the following equation and table
P( M j ) P( E | M j ) P(M j ) P( E | M j )
P( M j | E ) = = 12
P( E )
∑ P(M
m =1
m ) P( E | M m )
Example 6 (Cont.)
Application of Bayes’ rule
m P(Mm) P(E | Mm) P(Mm) P(E | Mm) P(Mm | E)
1 1/12 0.25 0.02083 0.1191
2 1/12 0.30 0.02500 0.1429
3 1/12 0.35 0.02917 0.1667
4 1/12 0.40 0.03333 0.1905
5 1/12 0.20 0.01667 0.0952
6 1/12 0.10 0.00833 0.0476
7 1/12 0.05 0.00417 0.0238
8 1/12 0.05 0.00417 0.0238
9 1/12 0.05 0.00417 0.0238
10 1/12 0.05 0.00417 0.0238
11 1/12 0.10 0.00833 0.0476
12 1/12 0.20 0.01667 0.0952
12/12 0.17500 = P(E) 1.0000
21

22. Thank you
22