2. Probability Distribution
A probability distribution is a table or an equation that links each outcome of a
statistical experiment with its probability of occurrence.
A probability distribution is a statistical function that describes all the possible
values and likelihoods that a random variable can take within a given range.
This range will be between the minimum and maximum statistically possible
values, but where the possible value is likely to be plotted on the probability
distribution depends on a number of factors. These factors include
the distribution's mean, standard deviation, skewness and kurtosis.
Consider a simple experiment in which we flip a coin two times. An outcome of
the experiment might be the number of heads that we see in two coin flips.
The table below associates each possible outcome with its probability.
Number of heads Probability
0 0.25
1 0.50
2 0.25
Supposethe random variable X is defined as the number of heads that result
fromtwo coin flips. Then, the abovetable represents the probability
distribution of the random variable X.
Types of Probability Distributions
3. What is a Continuous Distribution?
A continuous distribution describes the probabilities of the possible values of a
continuous random variable. A continuous random variable is a random
variable with a set of possible values (known as the range) that is infinite and
uncountable.
Probabilities of continuous random variables (X) are defined as the area under
the curve of its PDF. Thus, only ranges of values can have a nonzero
probability. The probability that a continuous random variable equals some
value is always zero.
Example of the distribution of weights
The continuous normal distribution can describe the distribution of weight of
adult males. For example, you can calculate the probability that a man weighs
between 160 and 170 pounds.
4. Distribution plot of the weight of adult males
The shaded region under the curve in this example represents the range from
160 and 170 pounds. The area of this range is 0.136; therefore, the probability
that a randomly selected man weighs between 160 and 170 pounds is 13.6%.
The entire area under the curve equals 1.0.
However, the probability that X is exactly equal to some value is always zero
because the area under the curve at a single point, which has no width, is zero.
For example, the probability that a man weighs exactly 190 pounds to infinite
precision is zero. You could calculate a nonzero probability that a man weighs
more than 190 pounds, or less than 190 pounds, or between 189.9 and 190.1
pounds, but the probability that he weighs exactly 190 pounds is zero.
What is a discrete distribution?
A discrete distribution describes the probability of occurrence of each value of
a discrete random variable. A discrete random variable is a random variable
that has countable values, such as a list of non-negative integers.
With a discrete probability distribution, each possible value of the discrete
random variable can be associated with a non-zero probability. Thus, a discrete
probability distribution is often presented in tabular form.
Example of the number of customer complaints
With a discrete distribution, unlike with a continuous distribution, you can
calculate the probability that X is exactly equal to some value. For example,
you can use the discrete Poisson distribution to describe the number of
customer complaints within a day. Suppose the average number of complaints
per day is 10 and you want to know the probability of receiving 5, 10, and 15
customer complaints in a day.
x P (X = x)
5 0.037833
10 0.125110
15 0.034718
You can also view a discrete distribution on a distribution plot to see the
probabilities between ranges
5. .
Distribution plot of the number of customer complaints
The shaded bars in this example represents the number of occurrences when
the daily customer complaints is 15 or more. The height of the bars sums to
0.08346; therefore, the probability that the number of calls per day is 15 or
more is 8.35%.
What is a Binomial Distribution?
A binomial distribution is a probability distribution. It refers to the probabilities
associated with the number of successes in a binomial experiment.
For example, supposewetoss a coin three times and suppose we define Heads
as a success. This binomial experiment has four possible outcomes: 0 Heads, 1
Head, 2 Heads, or 3 Heads. The probabilities associated with each possible
outcome are an example of a binomial distribution, as shown below.
Outcome,
x
Binomial probability,
P(X = x)
Cumulative probability,
P(X < x)
0 Heads 0.125 0.125
1 Head 0.375 0.500
2 Heads 0.375 0.875
3 Heads 0.125 1.000
There is a set of assumptions which, if valid, would lead to a binomial
distribution. These are:
6. • A set of n experiments or trials are conducted.
• Each trial could result in either a success or a failure.
• The probability p of success is the same for all trials.
• The outcomes of different trials are independent.
• We are interested in the total number of successes in these n trials.
Under the above assumptions, let X be the total number of successes. Then, X
is called a binomial random variable, and the probability distribution of X is
called the binomial distribution.
Binomial Probability-Mass Function. . .
Let X be a binomial random variable. Then, its probability mass function is:
The values of n and p are called the parameters of the distribution. To
understand above equation,
note that:
7. Example:
Multiple-Choice Exam
Consider an exam that contains 10 multiple-choice questions with 4 possible
choices for each question, only one of which is correct.
Suppose a student is to select the answer for every question randomly.
Let X be the number of questions the student answers correctly. Then, X has a
binomial distribution with parameters n = 10 and p = 0.25. (Convince yourself
that all assumptions for a binomial distribution are reasonable in this setting.)
8. What is a Normal Distribution?
The normal distribution is the most important distribution in statistics, since it
arises naturally in numerous applications. The key reason is that large sums of
(small) random variables often turn out to be normally distributed;
Normal probability distribution, also called Gaussian distribution refers to a
family of distributions that are bell shaped.
The normal distribution can be characterized by the mean and standard
deviation. The mean determines where the peak occurs, which is at 0 in the
figure for all the curves. The standard deviation is a measure of the spread of
the normal probability distribution, which can be seen as differing widths of
the bell curves in the figure.
Probability and the Normal Curve
The normal distribution is a continuous probability distribution. This has
several implications for probability.
The total area under the normal curve is equal to 1.
The probability that a normal random variable X equals any particular
value is 0.
9. The probability that X is greater than a equals the area under the normal
curve bounded by a and plus infinity (as indicated by the non-
shaded area in the figure below).
The probability that X is less than a equals the area under the normal
curve bounded by a and minus infinity (as indicated by the shaded area
in the figure below).
A random variable X is said to have the normal distribution with parameters µ
and σ if its density function is given by:
Thus, the normal distribution is characterized by a mean µ and a standard
deviation σ .
What is a standard score?
A standard score (aka, a z-score) is the normal random variable of a standard
normal distribution. To transform a normal random variable (x) into an
equivalent standard score (z), use the following formula:
z = (x - μ) / σ
where μ is the mean, and σ is the standard deviation.
10. What is a Student’s t Distribution?
The t distribution (aka, Student’s t-distribution) is a probability distribution
that is used to estimate population parameters when the sample size is small
and/or when the population variance is unknown.
Why Use the t Distribution?
According to the central limit theorem, the sampling distribution of a statistic
(like a sample mean) will follow a normal distribution, as long as the sample
size is sufficiently large. Therefore, when we know the standard deviation of
the population, we can compute a z-score, and use the normal distribution to
evaluate probabilities with the sample mean.
But sample sizes are sometimes small, and often we do not know the standard
deviation of the population. When either of these problems occur, statisticians
rely on the distribution of the t statistic(also known as the t score), whose
values are given by:
t = [ x - μ ] / [ s / sqrt( n ) ]
where x is the sample mean, μ is the population mean, s is the standard
deviation of the sample, and n is the sample size. The distribution of
the t statistic is called the t distribution or the Student t distribution.
The t distribution allows us to conduct statistical analyses on certain data sets
that are not appropriate for analysis, using the normal distribution.
Properties of the t Distribution
The t distribution has the following properties:
The mean of the distribution is equal to 0 .
The variance is equal to v / ( v - 2 ), where v is the degrees of freedom
(see last section) and v >2.
The variance is always greater than 1, although it is close to 1 when
there are many degrees of freedom. With infinite degrees of freedom,
the t distribution is the same as the standard normal distribution.
11. Probability and the Student t Distribution
When a sample of size n is drawn from a population having a normal (or nearly
normal) distribution, the sample mean can be transformed into a t statistic,
using the equation presented at the beginning of this lesson. We repeat that
equation below:
t = [ x - μ ] / [ s / sqrt( n ) ]
where x is the sample mean, μ is the population mean, s is the standard
deviation of the sample, n is the sample size, and degrees of freedom are equal
to n - 1.
The t statistic produced by this transformation can be associated with a
unique cumulative probability. This cumulative probability represents the
likelihood of finding a sample mean less than or equal to x, given a random
sample of size n.
What are degrees of freedom?
Degrees of freedom can be described as the number of scores that are free to
vary.
For example, suppose you tossed three dice. The total score adds up to 12. If
you rolled a 3 on the first die and a 5 on the second, then you know that the
third die must be a 4 (otherwise, the total would not add up to 12). In this
example, 2 die are free to vary while the third is not. Therefore, there are 2
degrees of freedom.
There are actually many different t distributions. The particular form of the t
distribution is determined by its degrees of freedom. The degrees of freedom
refers to the number of independent observations in a set of data.
The t distribution is very similar to the normal distribution when the
estimate of variance is based on many degrees of freedom, but has relatively
more scores in its tails when there are fewer degrees of freedom. Figure 1
shows t distributions with 2, 4, and 10 degrees of freedom and the standard
normal distribution. Notice that the normal distribution has relatively more
scores in the center of the distribution and the t distribution has relatively
more in the tails. The t distribution is therefore leptokurtic. The t distribution
approaches the normal distribution as the degrees of freedom increase.
12. Figure 1. A comparison of t distributions with 2, 4, and 10 df and the standard
normal distribution. The distribution with the lowest peak is the 2 df
distribution, the next lowest is 4 df, the lowest after that is 10 df, and the
highest is the standard normal distribution.
When to Use the t Distribution
The t distribution can be used with any statistic having a bell-shaped
distribution (i.e., approximately normal). The sampling distribution of a statistic
should be bell-shaped if any of the following conditions apply.
The population distribution is normal.
The population distribution is symmetric, unimodal, without outliers,
and the sample size is at least 30.
The population distribution is moderately skewed, unimodal, without
outliers, and the sample size is at least 40.
The sample size is greater than 40, without outliers.
The t distribution should not be used with small samples from populations that
are not approximately normal.