John E. 1:-reund’ sMArJ I
S1A11 1 ICS
\”111 11
PP I I .,,A I I >
f. I I I I :-..:
John E. Freund’s Mathematical Statistics
with Applications
Irwin Miller Marylees Miller
Eighth Edition
Pearson Education Limited
Edinburgh Gate
Harlow
Essex CM20 2JE
England and Associated Companies throughout the world
Visit us on the World Wide Web at: www.pearsoned.co.uk
© Pearson Education Limited 2014
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without either the
prior written permission of the publisher or a licence permitting restricted copying in the United Kingdom
issued by the Copyright Licensing Agency Ltd, Saffron House, 6–10 Kirby Street, London EC1N 8TS.
All trademarks used herein are the property of their respective owners. The use of any trademark
in this text does not vest in the author or publisher any trademark ownership rights in such
trademarks, nor does the use of such trademarks imply any affiliation with or endorsement of this
book by such owners.
ISBN 10: 1-292-02500-X
ISBN 13: 978-1-292-02500-1
British Library Cataloguing-in-Publication Data
A catalogue record for this book is available from the British Library
Printed in the United States of America
P E
A
R
S
O
N
C U
S T O
M
L
I
B
R
A
R Y
Table of Contents
1. Introduction
Irwin Miller/Marylees Miller
1
2. Probability
Irwin Miller/Marylees Miller
21
3. Probability Distributions and Probability Densities
Irwin Miller/Marylees Miller
61
4. Mathematical Expectation
Irwin Miller/Marylees Miller
113
5. Special Probability Distributions
Irwin Miller/Marylees Miller
145
6. Special Probability Densities
Irwin Miller/Marylees Miller
177
7. Functions of Random Variables
Irwin Miller/Marylees Miller
207
8. Sampling Distributions
Irwin Miller/Marylees Miller
233
9. Decision Theory
Irwin Miller/Marylees Miller
261
10. Point Estimation
Irwin Miller/Marylees Miller
283
11. Interval Estimation
Irwin Miller/Marylees Miller
317
12. Hypothesis Testing
Irwin Miller/Marylees Miller
337
13. Tests of Hypothesis Involving Means, Variances, and Proportions
Irwin Miller/Marylees Miller
359
I
14. Regression and Correlation
II
Irwin Miller/Marylees Miller
391
Appendix: Sums and Products
Irwin Miller/Marylees Miller
433
Appendix: Special Probability Distributions
Irwin Miller/Marylees Miller
437
Appendix: Special Probability Densities
Irwin Miller/Marylees Miller
439
Statistical Tables
Irwin Miller/Marylees Miller
443
Index
469
Introduction
1 Introduction
2 Combinatorial Methods
1 Introduction
3 Binomial Coefficients
4 The Theory in Practice
In recent years, the growth of statistics has made itself felt in almost every phase
of human activity. Statistics no longer consists merely of the collection of data and
their presentation in charts and tables; it is now considered to encompass the science
of basing inferences on observed data and the entire problem of making decisions
in the face of uncertainty. This covers considerable ground since uncertainties are
met when we flip a coin, when a dietician experiments with food additives, when an
actuary determines life insurance premiums, when a quality control engineer accepts
or rejects manufactured products, when a teacher compares the abilities of students,
when an economist forecasts trends, when a newspaper predicts an election, and
even when a physicist describes quantum mechanics.
It would be presumptuous to say that statistics, in its present state of development, can handle all situations involving uncertainties, but new techniques are
constantly being developed and modern statistics can, at least, provide the framework for looking at these situations in a logical and systematic fashion. In other
words, statistics provides the models that are needed to study situations involving
uncertainties, in the same way as calculus provides the models that are needed to
describe, say, the concepts of Newtonian physics.
The beginnings of the mathematics of statistics may be found in mid-eighteenthcentury studies in probability motivated by interest in games of chance. The theory
thus developed for “heads or tails” or “red or black” soon found applications in situations where the outcomes were “boy or girl,” “life or death,” or “pass or fail,” and
scholars began to apply probability theory to actuarial problems and some aspects
of the social sciences. Later, probability and statistics were introduced into physics
by L. Boltzmann, J. Gibbs, and J. Maxwell, and by this century they have found
applications in all phases of human endeavor that in some way involve an element
of uncertainty or risk. The names that are connected most prominently with the
growth of mathematical statistics in the first half of the twentieth century are those
of R. A. Fisher, J. Neyman, E. S. Pearson, and A. Wald. More recently, the work of
R. Schlaifer, L. J. Savage, and others has given impetus to statistical theories based
essentially on methods that date back to the eighteenth-century English clergyman
Thomas Bayes.
Mathematical statistics is a recognized branch of mathematics, and it can be
studied for its own sake by students of mathematics. Today, the theory of statistics is
applied to engineering, physics and astronomy, quality assurance and reliability, drug
development, public health and medicine, the design of agricultural or industrial
experiments, experimental psychology, and so forth. Those wishing to participate
From Chapter 1 of John E. Freund’s Mathematical Statistics with Applications,
Eighth Edition. Irwin Miller, Marylees Miller. Copyright 2014 by Pearson Education, Inc.
All rights reserved.
Introduction
in such applications or to develop new applications will do well to understand the
mathematical theory of statistics. For only through such an understanding can applications proceed without the serious mistakes that sometimes occur. The applications
are illustrated by means of examples and a separate set of applied exercises, many
of them involving the use of computers. To this end, we have added at the end of the
chapter a discussion of how the theory of the chapter can be applied in practice.
We begin with a brief review of combinatorial methods and binomial
coefficients.
2 Combinatorial Methods
In many problems of statistics we must list all the alternatives that are possible in a
given situation, or at least determine how many different possibilities there are. In
connection with the latter, we often use the following theorem, sometimes called the
basic principle of counting, the counting rule for compound events, or the rule for
the multiplication of choices.
THEOREM 1. If an operation consists of two steps, of which the first can be
done in n1 ways and for each of these the second can be done in n2 ways,
then the whole operation can be done in n1 · n2 ways.
Here, “operation” stands for any kind of procedure, process, or method of selection.
To justify this theorem, let us define the ordered pair (xi , yj ) to be the outcome
that arises when the first step results in possibility xi and the second step results in
possibility yj . Then, the set of all possible outcomes is composed of the following
n1 · n2 pairs:
(x1 , y1 ), (x1 , y2 ), . . . , (x1 , yn2 )
(x2 , y1 ), (x2 , y2 ), . . . , (x2 , yn2 )
…
…
…
(xn1 , y1 ), (xn1 , y2 ), . . . , (xn1 , yn2 )
EXAMPLE 1
Suppose that someone wants to go by bus, train, or plane on a week’s vacation to one
of the five East North Central States. Find the number of different ways in which this
can be done.
Solution
The particular state can be chosen in n1 = 5 ways and the means of transportation
can be chosen in n2 = 3 ways. Therefore, the trip can be carried out in 5 · 3 = 15
possible ways. If an actual listing of all the possibilities is desirable, a tree diagram
like that in Figure 1 provides a systematic approach. This diagram shows that there
are n1 = 5 branches (possibilities) for the number of states, and for each of these
branches there are n2 = 3 branches (possibilities) for the different means of transportation. It is apparent that the 15 possible ways of taking the vacation are represented by the 15 distinct paths along the branches of the tree.
Introduction
bus
train
plane
Oh
io
bus
train
plane
a
an
di
In
Illinois
bus
train
plane
M
ich
ig
an
sin
on
sc
Wi
bus
train
plane
bus
train
plane
Figure 1. Tree diagram.
EXAMPLE 2
How many possible outcomes are there when we roll a pair of dice, one red and
one green?
Solution
The red die can land in any one of six ways, and for each of these six ways the green
die can also land in six ways. Therefore, the pair of dice can land in 6 · 6 = 36 ways.
Theorem 1 may be extended to cover situations where an operation consists of
two or more steps. In this case, we have the following theorem.
Introduction
THEOREM 2. If an operation consists of k steps, of which the first can be
done in n1 ways, for each of these the second step can be done in n2 ways,
for each of the first two the third step can be done in n3 ways, and so forth,
then the whole operation can be done in n1 · n2 · . . . · nk ways.
EXAMPLE 3
A quality control inspector wishes to select a part for inspection from each of four
different bins containing 4, 3, 5, and 4 parts, respectively. In how many different ways
can she choose the four parts?
Solution
The total number of ways is 4 · 3 · 5 · 4 = 240.
EXAMPLE 4
In how many different ways can one answer all the questions of a true–false test
consisting of 20 questions?
Solution
Altogether there are
2 · 2 · 2 · 2 · . . . · 2 · 2 = 220 = 1,048,576
different ways in which one can answer all the questions; only one of these corresponds to the case where all the questions are correct and only one corresponds to
the case where all the answers are wrong.
Frequently, we are interested in situations where the outcomes are the different
ways in which a group of objects can be ordered or arranged. For instance, we might
want to know in how many different ways the 24 members of a club can elect a president, a vice president, a treasurer, and a secretary, or we might want to know in how
many different ways six persons can be seated around a table. Different arrangements like these are called permutations.
DEFINITION 1. PERMUTATIONS. A permutation is a distinct arrangement of n different elements of a set.
EXAMPLE 5
How many permutations are there of the letters a, b, and c?
Solution
The possible arrangements are abc, acb, bac, bca, cab, and cba, so the number of
distinct permutations is six. Using Theorem 2, we could have arrived at this answer
without actually listing the different permutations. Since there are three choices to
Introduction
select a letter for the first position, then two for the second position, leaving only
one letter for the third position, the total number of permutations is 3 · 2 · 1 = 6.
Generalizing the argument used in the preceding example, we find that n distinct
objects can be arranged in n(n − 1)(n − 2) · . . . · 3 · 2 · 1 different ways. To simplify our
notation, we represent this product by the symbol n!, which is read “n factorial.”
Thus, 1! = 1, 2! = 2 · 1 = 2, 3! = 3 · 2 · 1 = 6, 4! = 4 · 3 · 2 · 1 = 24, 5! = 5 · 4 · 3 · 2 · 1 =
120, and so on. Also, by definition we let 0! = 1.
THEOREM 3. The number of permutations of n distinct objects is n!.
EXAMPLE 6
In how many different ways can the five starting players of a basketball team be
introduced to the public?
Solution
There are 5! = 5 · 4 · 3 · 2 · 1 = 120 ways in which they can be introduced.
EXAMPLE 7
The number of permutations of the four letters a, b, c, and d is 24, but what is the
number of permutations if we take only two of the four letters or, as it is usually put,
if we take the four letters two at a time?
Solution
We have two positions to fill, with four choices for the first and then three choices for
the second. Therefore, by Theorem 1, the number of permutations is 4 · 3 = 12.
Generalizing the argument that we used in the preceding example, we find that n
distinct objects taken r at a time, for r > 0, can be arranged in n(n − 1) · . . . ·
(n − r + 1) ways. We denote this product by n Pr , and we let n P0 = 1 by definition.
Therefore, we can state the following theorem.
THEOREM 4. The number of permutations of n distinct objects taken r at a
time is
n Pr =
for r = 0, 1, 2, . . . , n.
n!
(n − r)!
Proof The formula n Pr = n(n − 1) · . . . · (n − r + 1) cannot be used for
r = 0, but we do have
n!
=1
n P0 =
(n − 0)!
Introduction
For r = 1, 2, . . . , n, we have
n Pr = n(n − 1)(n − 2) · . . . · (n − r − 1)
=
=
n(n − 1)(n − 2) · . . . · (n − r − 1)(n − r)!
(n − r)!
n!
(n − r)!
In problems concerning permutations, it is usually easier to proceed by using
Theorem 2 as in Example 7, but the factorial formula of Theorem 4 is somewhat
easier to remember. Many statistical software packages provide values of n Pr and
other combinatorial quantities upon simple commands. Indeed, these quantities are
also preprogrammed in many hand-held statistical (or scientific) calculators.
EXAMPLE 8
Four names are drawn from among the 24 members of a club for the offices of president, vice president, treasurer, and secretary. In how many different ways can this
be done?
Solution
The number of permutations of 24 distinct objects taken four at a time is
24 P4 =
24!
= 24 · 23 · 22 · 21 = 255,024
20!
EXAMPLE 9
In how many ways can a local chapter of the American Chemical Society schedule
three speakers for three different meetings if they are all available on any of five
possible dates?
Solution
Since we must choose three of the five dates and the order in which they are chosen
(assigned to the three speakers) matters, we get
5 P3 =
120
5!
=
= 60
2!
2
We might also argue that the first speaker can be scheduled in five ways, the second speaker in four ways, and the third speaker in three ways, so that the answer is
5 · 4 · 3 = 60.
Permutations that occur when objects are arranged in a circle are called
circular permutations. Two circular permutations are not considered different (and
are counted only once) if corresponding objects in the two arrangements have the
same objects to their left and to their right. For example, if four persons are playing
bridge, we do not get a different permutation if everyone moves to the chair at his
or her right.
Introduction
EXAMPLE 10
How many circular permutations are there of four persons playing bridge?
Solution
If we arbitrarily consider the position of one of the four players as fixed, we can seat
(arrange) the other three players in 3! = 6 different ways. In other words, there are
six different circular permutations.
Generalizing the argument used in the preceding example, we obtain the following theorem.
THEOREM 5. The number of permutations of n distinct objects arranged in
a circle is (n − 1)!.
We have been assuming until now that the n objects from which we select r
objects and form permutations are all distinct. Thus, the various formulas cannot be
used, for example, to determine the number of ways in which we can arrange the
letters in the word “book,” or the number of ways in which three copies of one novel
and one copy each of four other novels can be arranged on a shelf.
EXAMPLE 11
How many different permutations are there of the letters in the word “book”?
Solution
If we distinguish for the moment between the two o’s by labeling them o1 and o2 ,
there are 4! = 24 different permutations of the symbols b, o1 , o2 , and k. However, if
we drop the subscripts, then bo1 ko2 and bo2 ko1 , for instance, both yield boko, and
since each pair of permutations with subscripts yields but one arrangement without
subscripts, the total number of arrangements of the letters in the word “book” is
24
2 = 12.
EXAMPLE 12
In how many different ways can three copies of one novel and one copy each of four
other novels be arranged on a shelf?
Solution
If we denote the three copies of the first novel by a1 , a2 , and a3 and the other four
novels by b, c, d, and e, we find that with subscripts there are 7! different permutations of a1 , a2 , a3 , b, c, d, and e. However, since there are 3! permutations of a1 , a2 ,
and a3 that lead to the same permutation of a, a, a, b, c, d, and e, we find that there
are only 7!
3! = 7 · 6 · 5 · 4 = 840 ways in which the seven books can be arranged on a
shelf.
Generalizing the argument that we used in the two preceding examples, we
obtain the following theorem.
Introduction
THEOREM 6. The number of permutations of n objects of which n1 are of
one kind, n2 are of a second kind, . . . , nk are of a kth kind, and
n1 + n2 + · · · + nk = n is
n!
n1 ! · n2 ! · . . . · nk !
EXAMPLE 13
In how many ways can two paintings by Monet, three paintings by Renoir, and two
paintings by Degas be hung side by side on a museum wall if we do not distinguish
between the paintings by the same artists?
Solution
Substituting n = 7, n1 = 2, n2 = 3, and n3 = 2 into the formula of Theorem 6, we get
7!
= 210
2! · 3! · 2!
There are many problems in which we are interested in determining the number
of ways in which r objects can be selected from among n distinct objects without
regard to the order in which they are selected.
DEFINITION 2. COMBINATIONS. A combination is a selection of r objects taken from
n distinct objects without regard to the order of selection.
EXAMPLE 14
In how many different ways can a person gathering data for a market research organization select three of the 20 households living in a certain apartment complex?
Solution
If we care about the order in which the households are selected, the answer is
20 P3 = 20 · 19 · 18 = 6,840
but each set of three households would then be counted 3! = 6 times. If we do not
care about the order in which the households are selected, there are only
6,840
= 1,140 ways in which the person gathering the data can do his or her job.
6
Actually, “combination” means the same as “subset,” and when we ask for the
number of combinations of r objects selected from a set of n distinct objects, we are
simply asking for the total number of subsets of r objects that can be selected from
a set of n distinct objects. In general, there are r! permutations of the objects in a
subset of r objects, so that the n Pr permutations of r objects selected from a set of
n distinct objects contain each subset r! times. Dividing n Pr by r! and denoting the
! ”
n
result by the symbol r , we thus have the following theorem.
Introduction
THEOREM 7. The number of combinations of n distinct objects taken r at a
# $
n!
n
=
r
r!(n − r)!
time is
for r = 0, 1, 2, . . . , n.
EXAMPLE 15
In how many different ways can six tosses of a coin yield two heads and four tails?
Solution
This question is the same as asking for the number of ways in which we can select
the two tosses on which heads is to occur. Therefore, applying Theorem 7, we find
that the answer is
# $
6!
6
= 15
=
2
2! · 4!
This result could also have been obtained by the rather tedious process of enumerating the various possibilities, HHTTTT, TTHTHT, HTHTTT, . . . , where H stands
for head and T for tail.
EXAMPLE 16
How many different committees of two chemists and one physicist can be formed
from the four chemists and three physicists on the faculty of a small college?
Solution
% &
4!
4
= 6 ways and one of
Since two of four chemists can be selected in 2 =
2!
· 2!
% &
3!
3
= 3 ways, Theorem 1 shows that the
three physicists can be selected in 1 =
1! · 2!
number of committees is 6 · 3 = 18.
A combination of r objects selected from a set of n distinct objects may be considered a partition of the n objects into two subsets containing, respectively, the r
objects that are selected and the n − r objects that are left. Often, we are concerned
with the more general problem of partitioning a set of n distinct objects into k subsets, which requires that each of the n objects must belong to one and only one of
the subsets.† The order of the objects within a subset is of no importance.
EXAMPLE 17
In how many ways can a set of four objects be partitioned into three subsets containing, respectively, two, one, and one of the objects?
† Symbolically, the subsets A , A2 , . . . , A constitute a partition of set A if A ∪ A2 ∪ · · · ∪ A = A and Ai ∩ Aj =
k
k
1
1
Ø for all i Z j.
Introduction
Solution
Denoting the four objects by a, b, c, and d, we find by enumeration that there are
the following 12 possibilities:
ab|c|d ab|d|c ac|b|d ac|d|b
ad|b|c ad|c|b bc|a|d bc|d|a
bd|a|c bd|c|a cd|a|b cd|b|a
The number of partitions for this example is denoted by the symbol
#
$
4
= 12
2, 1, 1
where the number at the top represents the total number of objects and the numbers
at the bottom represent the number of objects going into each subset.
Had we not wanted to enumerate all the possibilities in the preceding example,
we
could
have argued that the two objects going into the first subset can be chosen
% & in
% &
2
4
2 = 6 ways, the object going into the second subset can then be chosen in 1 = 2
% &
1
ways, and the object going into the third subset can then be chosen in 1 = 1 way.
Thus, by Theorem 2 there are 6 · 2 · 1 = 12 partitions. Generalizing this argument, we
have the following theorem.
THEOREM 8. The number of ways in which a set of n distinct objects can be
partitioned into k subsets with n1 objects in the first subset, n2 objects in
the second subset, . . . , and nk objects in the kth subset is
#
$
n!
n
=
n1 , n2 , . . . , nk
n1 ! · n2 ! · . . . · nk !
Proof Since the n1 objects going into the first subset can be chosen in
! ”
n
n1 ways, the n2 objects going into the second subset can then be chosen
”
!
n − n1
ways, the n3 objects going into the third subset can then be
in
n2
”
!
n − n1 − n2
ways, and so forth, it follows by Theorem 2 that
chosen in
n3
the total number of partitions is
#
n
n1 , n2 , . . . , nk
$
# $ #
$
#
$
n
n − n1
n − n1 − n2 − · · · − nk−1
=
·
·…·
n1
n2
nk
=
n!
(n − n1 )!
·
n1 ! · (n − n1 )! n2 ! · (n − n1 − n2 )!
· … ·
=
(n − n1 − n2 − · · · − nk−1 )!
nk ! · 0!
n!
n1 ! · n2 ! · . . . · nk !
Introduction
EXAMPLE 18
In how many ways can seven businessmen attending a convention be assigned to one
triple and two double hotel rooms?
Solution
Substituting n = 7, n1 = 3, n2 = 2, and n3 = 2 into the formula of Theorem 8, we get
#
$
7!
7
= 210
=
3, 2, 2
3! · 2! · 2!
3 Binomial Coefficients
If n is a positive integer and we multiply out (x + y)n term by term, each term will be
the product of x’s and y’s, with an x or a y coming from each of the n factors x + y.
For instance, the expansion
(x + y)3 = (x + y)(x + y)(x + y)
= x·x·x+x·x·y+x·y·x+x·y·y
+ y·x·x+y·x·y+y·y·x+y·y·y
= x3 + 3×2 y + 3xy2 + y3
yields terms of the form x3 , x2 y, xy2 , and y3 . Their coefficients are 1, 3, 3, and 1, and
! ”
3
the coefficient of xy2 , for example, is 2 = 3, the number of ways in which we can
! ”
3
choose the two factors providing the y’s. Similarly, the coefficient of x2 y is 1 = 3,
the number of ways in which we can choose the one factor providing the y, and the
! ”
! ”
3
3
coefficients of x3 and y3 are 0 = 1 and 3 = 1.
More generally, if n is a positive integer and we multiply out (x + y)n term by
! ”
n
term, the coefficient of xn−r yr is r , the number of ways in which we can choose
! ”
n
the r factors providing the y’s. Accordingly, we refer to r as a binomial coefficient.
Values of the binomial coefficients for n = 0, 1, . . . , 20 and r = 0, 1, . . . , 10 are given
in table Factorials and Binomial Coefficients of “Statistical Tables.” We can now
state the following theorem.
THEOREM 9.
# $
n
‘
n n−r r
(x + y)n =
x y
r
r=0
for any positive integer n
Introduction
DEFINITION 3. BINOMIAL COEFFICIENTS. The coefficient of( x) n−r yr in the binomial
expansion of (x + y)n is called the binomial coefficient nr .
The calculation of binomial coefficients can often be simplified by making use
of the three theorems that follow.
THEOREM 10. For any positive integers n and r = 0, 1, 2, . . . , n,
# $ #
$
n
n
=
r
n−r
Proof We might argue that when we select a subset of r objects from a set
of n distinct objects, we leave a subset of n − r objects; hence, there are as
many ways of selecting r objects as there are ways of leaving (or selecting)
n − r objects. To prove the theorem algebraically, we write
#
$
n!
n!
n
=
=
n−r
(n − r)![n − (n − r)]!
(n − r)!r!
# $
n!
n
=
=
r
r!(n − r)!
Theorem 10 implies that if we calculate the binomial coefficients for
r = 0, 1, . . . , n2 when n is even and for r = 0, 1, . . . , n−1
2 when n is odd, the remaining
binomial coefficients can be obtained by making use of the theorem.
EXAMPLE 19
# $
# $
# $
# $
# $
4
4
4
4
4
Given
= 1,
= 4, and
= 6, find
and
.
0
1
2
3
4
Solution
# $ #
$ # $
# $ #
$ # $
4
4
4
4
4
4
=
=
= 4 and
=
=
=1
3
4−3
1
4
4−4
0
EXAMPLE 20
# $
# $
# $
# $ # $
# $
5
5
5
5
5
5
Given
= 1,
= 5, and
= 10, find
,
, and
.
0
1
2
3
4
5
Solution
# $ #
$ # $
# $ #
$ # $
5
5
5
5
5
5
=
=
= 10,
=
=
= 5, and
3
5−3
2
4
5−4
1
# $ #
$ # $
5
5
5
=
=
=1
5
5−5
0
It is precisely in this fashion that Theorem 10 may have to be used in connection
with table Factorials and Binomial Coefficients of “Statistical Tables.”
Introduction
EXAMPLE 21
! ”
! ”
17
20
Find 12 and 10 .
Solution
! ” ! ”
! ”
20
20
20
Since 12 is not given in the table, we make use of the fact that 12 = 8 , look
! ”
! ”
! ”
17
20
20
up 8 , and get 12 = 125,970. Similarly, to find 10 , we make use of the fact
! ”
! ”
! ” ! ”
17
17
17
17
that 10 = 7 , look up 7 , and get 10 = 19,448.
THEOREM 11. For any positive integer n and r = 1, 2, . . . , n − 1,
# $ #
$ #
$
n
n−1
n−1
=
+
r
r
r−1
Proof Substituting x = 1 into (x + y)n , let us write (1 + y)n = (1 + y)
(1 + y)n−1 = (1 + y)n−1 + y(1 + y)n−1 and equate the coefficient of yr in
(1 + y)n with that in (1 + y)n−1 + y(1 + y)n−1 . Since the coefficient of yr in
! ”
n
(1 + y)n is r and the coefficient of yr in (1 + y)n−1 + y(1 + y)n−1 is the
”
!
n−1
sum of the coefficient of yr in (1 + y)n−1 , that is,
r , and the coeffi”
!
n−1
cient of yr−1 in (1 + y)n−1 , that is, r − 1 , we obtain
# $ #
$ #
$
n
n−1
n−1
=
+
r
r
r−1
which completes the proof.
Alternatively, take any one of the n objects. If it is not to be included among the
”
!
n−1
ways of selecting the r objects; if it is to be included, there
r objects, there are
r
”
”
!
!
n−1
n−1
+
are r − 1 ways of selecting the other r − 1 objects. Therefore, there are
r
”
!
n−1
r − 1 ways of selecting the r objects, that is,
# $ #
$ #
$
n
n−1
n−1
=
+
r
r
r−1
Theorem 11 can also be proved by expressing the binomial coefficients on both
sides of the equation in terms of factorials and then proceeding algebraically, but we
shall leave this to the reader in Exercise 12.
An important application of Theorem 11 is a construct known as Pascal’s
triangle. When no table is available, it is sometimes convenient to determine binomial coefficients by means of a simple construction. Applying Theorem 11, we can
generate Pascal’s triangle as follows:
Introduction
1
1
1
1
1
1
2
3
4
1
3
6
1
4
1
1
5
10
10
5
1
……………………….
In this triangle, the first and last entries of each row are the numeral “1” each other
entry in any given row is obtained by adding the two entries in the preceding row
immediately to its left and to its right.
To state the third theorem about binomial coefficients, let us make the following
! ”
n
definition: r = 0 whenever n is a positive integer and r is a positive integer greater
than n. (Clearly, there is no way in which we can select a subset that contains more
elements than the whole set itself.)
THEOREM 12.
# $#
$ #
$
k
‘
m
n
m+n
=
r
k−r
k
r=0
Proof Using the same technique as in the proof of Theorem 11, let us
prove this theorem by equating the coefficients of yk in the expressions
on both sides of the equation
(1 + y)m+n = (1 + y)m (1 + y)n
!m+n”
, and the coefficient of yk in
k
# $ # $
# $
m
m
m m
+
y+···+
y
(1 + y)m (1 + y)n =
0
1
m
The coefficient of yk in (1+y)m+n is
# $ # $
# $
n
n
n n
*
+
y+···+
y
0
1
n
is the sum of the products that we obtain by multiplying the constant
term of the first factor by the coefficient of yk in the second factor, the
coefficient of y in the first factor by the coefficient of yk−1 in the second
factor, . . . , and the coefficient of yk in the first factor by the constant term
of the second factor. Thus, the coefficient of yk in (1 + y)m (1 + y)n is
# $# $ # $#
$ # $#
$
# $# $
m
n
m
n
m
n
m
n
+
+
+···+
0
k
1
k−1
2
k−2
k
0
# $#
$
k
‘
m
n
=
r
k−r
r=0
and this completes the proof.
Introduction
EXAMPLE 22
Verify Theorem 12 numerically for m = 2, n = 3, and k = 4.
Solution
Substituting these values, we get
# $# $ # $# $ # $# $ # $# $ # $# $ # $
2
3
2
3
2
3
2
3
2
3
5
+
+
+
+
=
0
4
1
3
2
2
3
1
4
0
4
! ”
! ” ! ”
2
2
3
and since 4 , 3 , and 4 equal 0 according to the definition on the previous page,
the equation reduces to
# $# $ # $# $ # $
2
3
2
3
5
+
=
1
3
2
2
4
which checks, since 2 · 1 + 1 · 3 = 5.
Using Theorem 8, we can extend our discussion to multinomial coefficients, that
is, to the coefficients that arise in the expansion of (x1 + x2 + · · · + xk )n . The multir
r
r
nomial coefficient of the term x11 · x22 · . . . · xkk in the expansion of (x1 + x2 + · · · +
xk )n is
#
$
n!
n
=
r1 , r2 , . . . , rk
r 1 ! · r2 ! · . . . · rk !
EXAMPLE 23
What is the coefficient of x31 x2 x23 in the expansion of (x1 + x2 + x3 )6 ?
Solution
Substituting n = 6, r1 = 3, r2 = 1, and r3 = 2 into the preceding formula, we get
6!
= 60
3! · 1! · 2!
Exercises
1. An operation consists of two steps, of which the first
can be made in n1 ways. If the first step is made in the ith
way, the second step can be made in n2i ways.†
(a) Use a tree diagram to find a formula for the total number of ways in which the total operation can be made.
(b) A student can study 0, 1, 2, or 3 hours for a history
test on any given day. Use the formula obtained in part
(a) to verify that there are 13 ways in which the student
can study at most 4 hours for the test on two consecutive
days.
2. With reference to Exercise 1, verify that if n2i equals
the constant n2 , the formula obtained in part (a) reduces
to that of Theorem 1.
3. With reference to Exercise 1, suppose that there is a
third step, and if the first step is made in the ith way and
the second step in the jth way, the third step can be made
in n3ij ways.
(a) Use a tree diagram to verify that the whole operation
can be made in
† The first subscript denotes the row to which a particular element belongs, and the second subscript denotes the column.
Introduction
n1 ‘
n2i
‘
n3ij
i=1 j=1
different ways.
(b) With reference to part (b) of Exercise 1, use the formula of part (a) to verify that there are 32 ways in which
the student can study at most 4 hours for the test on three
consecutive days.
4. Show that if n2i equals the constant n2 and n3ij equals
the constant n3 , the formula of part (a) of Exercise 3
reduces to that of Theorem 2.
5. In a two-team basketball play-off, the winner is the first
team to win m games.
(a) Counting separately the number of play-offs requiring
m, m + 1, . . . , and 2m − 1 games, show that the total number of different outcomes (sequences of wins and losses
by one of the teams) is
.%
& %
&
%
&/
m−1
m
2m − 2
2
+
+···+
m−1
m−1
m−1
(b) How many different outcomes are there in a “2 out
of 3” play-off, a “3 out of 5” play-off, and a “4 out of 7”
play-off?
6. When n is large, n! can be approximated by means of
the expression
% &n
√
n
2πn
e
called Stirling’s formula, where e is the base of natural
logarithms. (A derivation of this formula may be found in
the book by W. Feller cited among the references at the
end of this chapter.)
(a) Use Stirling’s formula to obtain approximations for
10! and 12!, and find the percentage errors of these
approximations by comparing them with the exact values given in table Factorials and Binomial Coefficients of
“Statistical Tables.”
(b) Use Stirling’s formula to obtain an approximation for
the number of 13-card bridge hands that can be dealt with
an ordinary deck of 52 playing cards.
7. Using Stirling’s formula (see Exercise 6) to approximate 2n! and n!, show that
% &
2n √
πn
n
L1
22n
8. In some problems of occupancy theory we are concerned with the number of ways in which certain distinguishable objects can be distributed among individuals,
urns, boxes, or cells. Find an expression for the number of
ways in which r distinguishable objects can be distributed
among n cells, and use it to find the number of ways in
which three different books can be distributed among the
12 students in an English literature class.
9. In some problems of occupancy theory we are concerned with the number of ways in which certain indistinguishable objects can be distributed among individuals,
urns, boxes, or cells. Find an expression for the number
of ways in which r indistinguishable objects can be distributed among n cells, and use it to find the number of
ways in which a baker can sell five (indistinguishable)
loaves of bread to three customers. (Hint: We might argue
that L|LLL|L represents the case where the three customers buy one loaf, three loaves, and one loaf, respectively, and that LLLL||L represents the case where the
three customers buy four loaves, none of the loaves, and
one loaf. Thus, we must look for the number of ways
in which we can arrange the five L’s and the two vertical bars.)
10. In some problems of occupancy theory we are concerned with the number of ways in which certain indistinguishable objects can be distributed among individuals,
urns, boxes, or cells with at least one in each cell. Find
an expression for the number of ways in which r indistinguishable objects can be distributed among n cells with
at least one in each cell, and rework the numerical part
of Exercise 9 with each of the three customers getting at
least one loaf of bread.
11. Construct the seventh and eighth rows of Pascal’s triangle and write the binomial expansions of (x + y)6 and
(x + y)7 .
12. Prove Theorem 11 by expressing all the binomial
coefficients in terms of factorials and then simplifying
algebraically.
13. Expressing the binomial coefficients in terms of factorials and simplifying algebraically, show that
% &
%
&
n−r+1
n
n
(a)
=
·
;
r
r−1
r
% &
%
&
n
n
n−1
(b)
=
·
;
r
r
n−r
%
&
%
&
n−1
n
(c) n
= (r + 1)
.
r
r+1
14. Substituting appropriate values for x and y into the
formula of Theorem 9, show that
n % &
‘
n
= 2n ;
(a)
r
r=0
% &
n
‘
n
(−1)r
= 0;
(b)
r
r=0
n % &
‘
n
(a − 1)r = an .
(c)
r
r=0
Introduction
15. Repeatedly applying Theorem 11, show that
(a)
&
% & ‘
r+1 %
n−i
n
=
r−i+1
r
16. Use Theorem 12 to show that
r=0
r
=
% &
2n
n
% &
n
= n2n−1 by setting x = 1 in Ther
r=0
orem 9, then differentiating the expressions on both sides
with respect to y, and finally substituting y = 1.
17. Show that
n
0
1
2
4
and
%
&
−3
;
3
√
√
(b) 5 writing 5 = 2(1 + 14 )1/2 and using the first four
terms of the binomial expansion of (1 + 14 )1/2 .
i=1
n % &2
‘
n
# $
r
20. With reference to the generalized definition of binomial coefficients in Exercise 19, show that
% &
−1
(a)
= (−1)r ;
r
% &
%
&
−n
n+r−1
(b)
= (−1)r
for n > 0.
r
r
21. Find the coefficient of x2 y3 z3 in the expansion of
(x + y + z)8 .
18. Rework Exercise 17 by making use of part (a) of
Exercise 14 and part (c) of Exercise 13.
22. Find the coefficient of x3 y2 z3 w in the expansion of
(2x + 3y − 4z + w)9 .
19. If n is not a positive integer or zero, the binomial
expansion of (1 + y)n yields, for −1 < y < 1, the infinite series
23. Show that
%
& %
&
n
n−1
=
n1 , n2 , . . . , nk
n1 − 1, n2 , . . . , nk
&
%
n−1
+···
+
n1 , n2 − 1, . . . , nk
&
%
n−1
+
n1 , n2 , . . . , nk − 1
% &
% &
% &
% &
n
n 2
n 3
n r
1+
y+
y +
y +···+
y +···
1
2
3
r
% &
n(n − 1) · . . . · (n − r + 1)
n
=
for r = 1, 2, 3, . . . .
r
r!
Use this generalized definition of binomial coefficients
to evaluate
where
by expressing all these multinomial coefficients in terms
of factorials and simplifying algebraically.
4 The Theory in Practice
Applications of the preceding theory of combinatorial methods and binomial coefficients are quite straightforward, and a variety of them have been given in Sections 2
and 3. The following examples illustrate further applications of this theory.
EXAMPLE 24
An assembler of electronic equipment has 20 integrated-circuit chips on her table,
and she must solder three of them as part of a larger component. In how many ways
can she choose the three chips for assembly?
Solution
Using Theorem 6, we obtain the result
20 P3 = 20!/17! = 20 · 19 · 18 = 6,840
EXAMPLE 25
A lot of manufactured goods, presented for sampling inspection, contains 16 units.
In how many ways can 4 of the 16 units be selected for inspection?
Introduction
Solution
According to Theorem 7,
# $
16
= 16!/4!12! = 16 · 15 · 14 · 13/4 · 3 · 2 · 1 = 1,092 ways
4
Applied Exercises
24. A thermostat will call for heat 0, 1, or 2 times a night.
Construct a tree diagram to show that there are 10 different ways that it can turn on the furnace for a total of 6
times over 4 nights.
25. On August 31 there are five wild-card terms in the
American League that can make it to the play-offs, and
only two will win spots. Draw a tree diagram which shows
the various possible play-off wild-card teams.
26. There are four routes, A, B, C, and D, between a person’s home and the place where he works, but route B
is one-way, so he cannot take it on the way to work, and
route C is one-way, so he cannot take it on the way home.
(a) Draw a tree diagram showing the various ways the
person can go to and from work.
(b) Draw a tree diagram showing the various ways he
can go to and from work without taking the same route
both ways.
27. A person with $2 in her pocket bets $1, even money,
on the flip of a coin, and she continues to bet $1 as long
as she has any money. Draw a tree diagram to show the
various things that can happen during the first four flips
of the coin. After the fourth flip of the coin, in how many
of the cases will she be
(a) exactly even;
(b) exactly $2 ahead?
28. The pro at a golf course stocks two identical sets of
women’s clubs, reordering at the end of each day (for
delivery early the next morning) if and only if he has sold
them both. Construct a tree diagram to show that if he
starts on a Monday with two sets of the clubs, there are
altogether eight different ways in which he can make sales
on the first two days of that week.
29. Suppose that in a baseball World Series (in which the
winner is the first team to win four games) the National
League champion leads the American League champion
three games to two. Construct a tree diagram to show the
number of ways in which these teams may win or lose the
remaining game or games.
30. If the NCAA has applications from six universities
for hosting its intercollegiate tennis championships in two
SECS. 1–4
consecutive years, in how many ways can they select the
hosts for these championships
(a) if they are not both to be held at the same university;
(b) if they may both be held at the same university?
31. Counting the number of outcomes in games of chance
has been a popular pastime for many centuries. This was
of interest not only because of the gambling that was
involved, but also because the outcomes of games of
chance were often interpreted as divine intent. Thus, it
was just about a thousand years ago that a bishop in what
is now Belgium determined that there are 56 different
ways in which three dice can fall provided one is interested only in the overall result and not in which die does
what. He assigned a virtue to each of these possibilities
and each sinner had to concentrate for some time on the
virtue that corresponded to his cast of the dice.
(a) Find the number of ways in which three dice can all
come up with the same number of points.
(b) Find the number of ways in which two of the three
dice can come up with the same number of points, while
the third comes up with a different number of points.
(c) Find the number of ways in which all three of the dice
can come up with a different number of points.
(d) Use the results of parts (a), (b), and (c) to verify
the bishop’s calculations that there are altogether 56
possibilities.
32. In a primary election, there are four candidates for
mayor, five candidates for city treasurer, and two candidates for county attorney.
(a) In how many ways can a voter mark his ballot for all
three of these offices?
(b) In how many ways can a person vote if he exercises
his option of not voting for a candidate for any or all of
these offices?
33. The five finalists in the Miss Universe contest are Miss
Argentina, Miss Belgium, Miss U.S.A., Miss Japan, and
Miss Norway. In how many ways can the judges choose
(a) the winner and the first runner-up;
(b) the winner, the first runner-up, and the second
runner-up?
Introduction
34. A multiple-choice test consists of 15 questions, each
permitting a choice of three alternatives. In how many different ways can a student check off her answers to these
questions?
35. Determine the number of ways in which a distributor
can choose 2 of 15 warehouses to ship a large order.
36. The price of a European tour includes four stopovers
to be selected from among 10 cities. In how many different ways can one plan such a tour
(a) if the order of the stopovers matters;
(b) if the order of the stopovers does not matter?
37. A carton of 15 light bulbs contains one that is defective. In how many ways can an inspector choose 3 of the
bulbs and
(a) get the one that is defective;
(b) not get the one that is defective?
38. In how many ways can a television director schedule a sponsor’s six different commercials during the six
time slots allocated to commercials during a two-hour
program?
39. In how many ways can the television director of Exercise 38 fill the six time slots for commercials if there are
three different sponsors and the commercial for each is to
be shown twice?
40. In how many ways can five persons line up to get on
a bus? In how many ways can they line up if two of the
persons refuse to follow each other?
41. In how many ways can eight persons form a circle for
a folk dance?
42. How many permutations are there of the letters in the
word
(a) “great”;
(b) “greet”?
43. How many distinct permutations are there of the letters in the word “statistics”? How many of these begin
and end with the letter s?
44. A college team plays 10 football games during a season. In how many ways can it end the season with five
wins, four losses, and one tie?
45. If eight persons are having dinner together, in how
many different ways can three order chicken, four order
steak, and one order lobster?
46. In Example 4 we showed that a true–false test consisting of 20 questions can be marked in 1,048,576 different
ways. In how many ways can each question be marked
true or false so that
(a) 7 are right and 13 are wrong;
(b) 10 are right and 10 are wrong;
(c) at least 17 are right?
47. Among the seven nominees for two vacancies on a
city council are three men and four women. In how many
ways can these vacancies be filled
(a) with any two of the seven nominees;
(b) with any two of the four women;
(c) with one of the men and one of the women?
48. A shipment of 10 television sets includes three that
are defective. In how many ways can a hotel purchase
four of these sets and receive at least two of the defective
sets?
49. Ms. Jones has four skirts, seven blouses, and three
sweaters. In how many ways can she choose two of the
skirts, three of the blouses, and one of the sweaters to take
along on a trip?
50. How many different bridge hands are possible containing five spades, three diamonds, three clubs, and two
hearts?
51. Find the number of ways in which one A, three B’s,
two C’s, and one F can be distributed among seven students taking a course in statistics.
52. An art collector, who owns 10 paintings by famous
artists, is preparing her will. In how many different ways
can she leave these paintings to her three heirs?
53. A baseball fan has a pair of tickets for six different
home games of the Chicago Cubs. If he has five friends
who like baseball, in how many different ways can he take
one of them along to each of the six games?
54. At the end of the day, a bakery gives everything that
is unsold to food banks for the needy. If it has 12 apple
pies left at the end of a given day, in how many different
ways can it distribute these pies among six food banks for
the needy?
55. With reference to Exercise 54, in how many different ways can the bakery distribute the 12 apple pies
if each of the six food banks is to receive at least
one pie?
56. On a Friday morning, the pro shop of a tennis club
has 14 identical cans of tennis balls. If they are all sold
by Sunday night and we are interested only in how many
were sold on each day, in how many different ways could
the tennis balls have been sold on Friday, Saturday, and
Sunday?
57. Rework Exercise 56 given that at least two of the cans
of tennis balls were sold on each of the three days.
Introduction
References
Among the books on the history of statistics there are
Walker, H. M., Studies in the History of Statistical
Method. Baltimore: The Williams & Wilkins Company,
1929,
Westergaard, H., Contributions to the History of Statistics. London: P. S. King & Son, 1932,
and the more recent publications
Kendall, M. G., and Plackett, R. L., eds., Studies in the
History of Statistics and Probability, Vol. II. New York:
Macmillan Publishing Co., Inc., 1977,
Pearson, E. S., and Kendall, M. G., eds., Studies in the
History of Statistics and Probability. Darien, Conn.:
Hafner Publishing Co., Inc., 1970,
Porter, T. M., The Rise of Statistical Thinking, 1820–
1900. Princeton, N.J.: Princeton University Press, 1986,
and
Stigler, S. M., The History of Statistics. Cambridge,
Mass.: Harvard University Press, 1986.
A wealth of material on combinatorial methods can be
found in
Cohen, D. A., Basic Techniques of Combinatorial Theory. New York: John Wiley & Sons, Inc., 1978,
Eisen, M., Elementary Combinatorial Analysis. New
York: Gordon and Breach, Science Publishers, Inc.,
1970,
Feller, W., An Introduction to Probability Theory and Its
Applications, Vol. I, 3rd ed. New York: John Wiley &
Sons, Inc., 1968,
Niven, J., Mathematics of Choice. New York: Random
House, Inc., 1965,
Roberts, F. S., Applied Combinatorics. Upper Saddle
River, N.J.: Prentice Hall, 1984,
and
Whitworth, W. A., Choice and Chance, 5th ed. New
York: Hafner Publishing Co., Inc., 1959, which has
become a classic in this field.
More advanced treatments may be found in
Beckenbach, E. F., ed., Applied Combinatorial Mathematics. New York: John Wiley & Sons, Inc., 1964,
David, F. N., and BARTON, D. E., Combinatorial
Chance. New York: Hafner Publishing Co., Inc., 1962,
and
Riordan, J., An Introduction to Combinatorial Analysis.
New York: John Wiley & Sons, Inc., 1958.
Answers to Odd-Numbered Exercises
1 (a)
n
'
n2ni .
i=1
5 (b) 6, 20, and 70.
#
$
r+n−1
9
and 21.
r
11 Seventh row: 1, 6, 15, 20, 15, 6, 1; Eighth row: 1, 7, 21, 35,
35, 27, 7, 1.
19 (a) −15
384 and −10; (b) 2.230.
21 560.
27 (a) 5; (b) 4.
31 (a) 6; (b) 30; (c) 20; (d) 56.
33 (a) 20; (b) 60.
35 (a) 105.
37 (a) 91; (b) 364.
39 90.
41 5040.
43 50,400 and 3360.
45 280.
47 (a) 21; (b) 6; (c) 12.
49 630.
51 420.
53 15,625.
55 462.
57 45.
Probability
1
2
3
4
5
Introduction
Sample Spaces
Events
The Probability of an Event
Some Rules of Probability
1 Introduction
6
7
8
9
Conditional Probability
Independent Events
Bayes’ Theorem
The Theory in Practice
Historically, the oldest way of defining probabilities, the classical probability concept, applies when all possible outcomes are equally likely, as is presumably the case
in most games of chance. We can then say that if there are N equally likely possibilities, of which one must occur and n are regarded as favorable, or as a “success,” then
n
.
the probability of a “success” is given by the ratio N
EXAMPLE 1
What is the probability of drawing an ace from an ordinary deck of 52 playing cards?
Solution
Since there are n = 4 aces among the N = 52 cards, the probability of drawing
4
1
an ace is 52
= 13
. (It is assumed, of course, that each card has the same chance of
being drawn.)
Although equally likely possibilities are found mostly in games of chance, the
classical probability concept applies also in a great variety of situations where gambling devices are used to make random selections—when office space is assigned to
teaching assistants by lot, when some of the families in a township are chosen in such
a way that each one has the same chance of being included in a sample study, when
machine parts are chosen for inspection so that each part produced has the same
chance of being selected, and so forth.
A major shortcoming of the classical probability concept is its limited applicability, for there are many situations in which the possibilities that arise cannot all
be regarded as equally likely. This would be the case, for instance, if we are concerned with the question whether it will rain on a given day, if we are concerned
with the outcome of an election, or if we are concerned with a person’s recovery
from a disease.
Among the various probability concepts, most widely held is the frequency interpretation, according to which the probability of an event (outcome or happening) is
the proportion of the time that events of the same kind will occur in the long run.
If we say that the probability is 0.84 that a jet from Los Angeles to San Francisco
will arrive on time, we mean (in accordance with the frequency interpretation) that
such flights arrive on time 84 percent of the time. Similarly, if the weather bureau
From Chapter 2 of John E. Freund’s Mathematical Statistics with Applications,
Eighth Edition. Irwin Miller, Marylees Miller. Copyright 2014 by Pearson Education, Inc.
All rights reserved.
Probability
predicts that there is a 30 percent chance for rain (that is, a probability of 0.30), this
means that under the same weather conditions it will rain 30 percent of the time.
More generally, we say that an event has a probability of, say, 0.90, in the same sense
in which we might say that our car will start in cold weather 90 percent of the time.
We cannot guarantee what will happen on any particular occasion—the car may start
and then it may not—but if we kept records over a long period of time, we should
find that the proportion of “successes” is very close to 0.90.
The approach to probability that we shall use in this chapter is the axiomatic
approach, in which probabilities are defined as “mathematical objects” that behave
according to certain well-defined rules. Then, any one of the preceding probability
concepts, or interpretations, can be used in applications as long as it is consistent
with these rules.
2 Sample Spaces
Since all probabilities pertain to the occurrence or nonoccurrence of events, let us
explain first what we mean here by event and by the related terms experiment, outcome, and sample space.
It is customary in statistics to refer to any process of observation or measurement as an experiment. In this sense, an experiment may consist of the simple process of checking whether a switch is turned on or off; it may consist of counting the
imperfections in a piece of cloth; or it may consist of the very complicated process
of determining the mass of an electron. The results one obtains from an experiment, whether they are instrument readings, counts, “yes” or “no” answers, or values
obtained through extensive calculations, are called the outcomes of the experiment.
DEFINITION 1. SAMPLE SPACE. The set of all possible outcomes of an experiment is
called the sample space and it is usually denoted by the letter S. Each outcome
in a sample space is called an element of the sample space, or simply a sample
point.
If a sample space has a finite number of elements, we may list the elements in
the usual set notation; for instance, the sample space for the possible outcomes of
one flip of a coin may be written
S = {H, T}
where H and T stand for head and tail. Sample spaces with a large or infinite number
of elements are best described by a statement or rule; for example, if the possible
outcomes of an experiment are the set of automobiles equipped with satellite radios,
the sample space may be written
S = {x|x is an automobile with a satellite radio}
This is read “S is the set of all x such that x is an automobile with a satellite radio.”
Similarly, if S is the set of odd positive integers, we write
S = {2k + 1|k = 0, 1, 2, . . .}
How we formulate the sample space for a given situation will depend on the
problem at hand. If an experiment consists of one roll of a die and we are interested
in which face is turned up, we would use the sample space
Probability
S1 = {1, 2, 3, 4, 5, 6}
However, if we are interested only in whether the face turned up is even or odd, we
would use the sample space
S2 = {even, odd}
This demonstrates that different sample spaces may well be used to describe an
experiment. In general, it is desirable to use sample spaces whose elements cannot
be divided (partitioned or separated) into more primitive or more elementary kinds
of outcomes. In other words, it is preferable that an element of a sample space not
represent two or more outcomes that are distinguishable in some way. Thus, in the
preceding illustration S1 would be preferable to S2 .
EXAMPLE 2
Describe a sample space that might be appropriate for an experiment in which we
roll a pair of dice, one red and one green. (The different colors are used to emphasize
that the dice are distinct from one another.)
Solution
The sample space that provides the most information consists of the 36 points given by
S1 = {(x, y)|x = 1, 2, . . . , 6; y = 1, 2, . . . , 6}
where x represents the number turned up by the red die and y represents the number
turned up by the green die. A second sample space, adequate for most purposes
(though less desirable in general as it provides less information), is given by
S2 = {2, 3, 4, . . . , 12}
where the elements are the totals of the numbers turned up by the two dice.
Sample spaces are usually classified according to the number of elements that
they contain. In the preceding example the sample spaces S1 and S2 contained a
finite number of elements; but if a coin is flipped until a head appears for the first
time, this could happen on the first flip, the second flip, the third flip, the fourth flip,
. . ., and there are infinitely many possibilities. For this experiment we obtain the
sample space
S = {H, TH, TTH, TTTH, TTTTH, . . .}
with an unending sequence of elements. But even here the number of elements can
be matched one-to-one with the whole numbers, and in this sense the sample space
is said to be countable. If a sample space contains a finite number of elements or an
infinite though countable number of elements, it is said to be discrete.
The outcomes of some experiments are neither finite nor countably infinite. Such
is the case, for example, when one conducts an investigation to determine the distance that a certain make of car will travel over a prescribed test course on 5 liters
of gasoline. If we assume that distance is a variable that can be measured to any
desired degree of accuracy, there is an infinity of possibilities (distances) that cannot be matched one-to-one with the whole numbers. Also, if we want to measure
the amount of time it takes for two chemicals to react, the amounts making up the
sample space are infinite in number and not countable. Thus, sample spaces need
Probability
not be discrete. If a sample space consists of a continuum, such as all the points of
a line segment or all the points in a plane, it is said to be continuous. Continuous
sample spaces arise in practice whenever the outcomes of experiments are measurements of physical properties, such as temperature, speed, pressure, length, . . ., that
are measured on continuous scales.
3 Events
In many problems we are interested in results that are not given directly by a specific
element of a sample space.
EXAMPLE 3
With reference to the first sample space S1 on the previous page, describe the event
A that the number of points rolled with the die is divisible by 3.
Solution
Among 1, 2, 3, 4, 5, and 6, only 3 and 6 are divisible by 3. Therefore, A is represented
by the subset {3, 6} of the sample space S1 .
EXAMPLE 4
With reference to the sample space S1 of Example 2, describe the event B that the
total number of points rolled with the pair of dice is 7.
Solution
Among the 36 possibilities, only (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), and (6, 1) yield
a total of 7. So, we write
B = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}
Note that in Figure 1 the event of rolling a total of 7 with the two dice is represented
by the set of points inside the region bounded by the dotted line.
Green
die
6
5
4
3
2
1
1
2
3
4
5
6
Red die
Figure 1. Rolling a total of 7 with a pair of dice.
Probability
In the same way, any event (outcome or result) can be identified with a collection
of points, which constitute a subset of an appropriate sample space. Such a subset
consists of all the elements of the sample space for which the event occurs, and in
probability and statistics we identify the subset with the event.
DEFINITION 2. EVENT. An event is a subset of a sample space.
EXAMPLE 5
If someone takes three shots at a target and we care only whether each shot is a hit
or a miss, describe a suitable sample space, the elements of the sample space that
constitute event M that the person will miss the target three times in a row, and the
elements of event N that the person will hit the target once and miss it twice.
Solution
If we let 0 and 1 represent a miss and a hit, respectively, the eight possibilities (0, 0, 0),
(1, 0, 0), (0, 1, 0), (0, 0, 1), (1, 1, 0), (1, 0, 1), (0, 1, 1), and (1, 1, 1) may be displayed
as in Figure 2. Thus, it can be seen that
M = {(0, 0, 0)}
and
N = {(1, 0, 0), (0, 1, 0), (0, 0, 1)}
Third
shot
(0, 0, 1)
(0, 1, 1)
(1, 0, 1)
(1, 1, 1)
(1, 0, 0)
(0, 0, 0)
(0, 1, 0)
First
shot
(1, 1, 0)
Second
shot
Figure 2. Sample space for Example 5.
EXAMPLE 6
Construct a sample space for the length of the useful life of a certain electronic
component and indicate the subset that represents the event F that the component
fails before the end of the sixth year.
Probability
Solution
If t is the length of the component’s useful life in years, the sample space may be
written S = {t|t G 0}, and the subset F = {t|0 F t < 6} is the event that the component
fails before the end of the sixth year.
According to our definition, any event is a subset of an appropriate sample
space, but it should be observed that the converse is not necessarily true. For discrete sample spaces, all subsets are events, but in the continuous case some rather
abstruse point sets must be excluded for mathematical reasons. This is discussed further in some of the more advanced texts listed among the references at the end of
this chapter.
In many problems of probability we are interested in events that are actually
combinations of two or more events, formed by taking unions, intersections, and
complements. Although the reader must surely be familiar with these terms, let us
review briefly that, if A and B are any two subsets of a sample space S, their union
A ∪ B is the subset of S that contains all the elements that are either in A, in B,
or in both; their intersection A ∩ B is the subset of S that contains all the elements
that are in both A and B; and the complement A# of A is the subset of S that contains all the elements of S that are not in A. Some of the rules that control the
formation of unions, intersections, and complements may be found in Exercises 1
through 4.
Sample spaces and events, particularly relationships among events, are often
depicted by means of Venn diagrams, in which the sample space is represented by
a rectangle, while events are represented by regions within the rectangle, usually by
circles or parts of circles. For instance, the shaded regions of the four Venn diagrams
of Figure 3 represent, respectively, event A, the complement of event A, the union
of events A and B, and the intersection of events A and B. When we are dealing
with three events, we usually draw the circles as in Figure 4. Here, the regions are
numbered 1 through 8 for easy reference.
Figure 3. Venn diagrams.
Probability
Figure 4. Venn diagram.
Figure 5. Diagrams showing special relationships among events.
To indicate special relationships among events, we sometimes draw diagrams
like those of Figure 5. Here, the one on the left serves to indicate that events A and
B are mutually exclusive.
DEFINITION 3. MUTUALLY EXCLUSIVE EVENTS. Two events having no elements in common are said to be mutually exclusive.
When A and B are mutually exclusive, we write A ∩ B = ∅, where ∅ denotes
the empty set, which has no elements at all. The diagram on the right serves to
indicate that A is contained in B, and symbolically we express this by writing
A ( B.
Exercises
1. Use Venn diagrams to verify that
(a) (A ∪ B) ∪ C is the same event as A ∪ (B ∪ C);
(b) A ∩ (B ∪ C) is the same event as (A ∩ B) ∪ (A ∩ C);
(c) A ∪ (B ∩ C) is the same event as (A ∪ B) ∩ (A ∪ C).
2. Use Venn diagrams to verify the two De Morgan laws:
(b) (A ∪ B)# = A# ∩ B# .
(a) (A ∩ B)# = A# ∪ B# ;
3. Use Venn diagrams to verify that
(a) (A ∩ B) ∪ (A ∩ B# ) = A;
(b) (A ∩ B) ∪ (A ∩ B# ) ∪ (A# ∩ B) = A ∪ B;
(c) A ∪ (A# ∩ B) = A ∪ B.
4. Use Venn diagrams to verify that if A is contained in
B, then A ∩ B = A and A ∩ B# = ∅.
Probability
4 The Probability of an Event
To formulate the postulates of probability, we shall follow the practice of denoting
events by means of capital letters, and we shall write the probability of event A as
P(A), the probability of event B as P(B), and so forth. The following postulates of
probability apply only to discrete sample spaces, S.
POSTULATE 1
POSTULATE 2
POSTULATE 3
The probability of an event is a nonnegative real number;
that is, P(A) G 0 for any subset A of S.
P(S) = 1.
If A1 , A2 , A3 , . . ., is a finite or infinite sequence of mutually
exclusive events of S, then
P(A1 ∪ A2 ∪ A3 ∪ · · · ) = P(A1 ) + P(A2 ) + P(A3 ) + · · ·
Postulates per se require no proof, but if the resulting theory is to be applied,
we must show that the postulates are satisfied when we give probabilities a “real”
meaning. Let us illustrate this in connection with the frequency interpretation; the
relationship between the postulates and the classical probability concept will be
discussed below, while the relationship between the postulates and subjective probabilities is left for the reader to examine in Exercises 16 and 82.
Since proportions are always positive or zero, the first postulate is in complete
agreement with the frequency interpretation. The second postulate states indirectly
that certainty is identified with a probability of 1; after all, it is always assumed that
one of the possibilities in S must occur, and it is to this certain event that we assign
a probability of 1. As far as the frequency interpretation is concerned, a probability
of 1 implies that the event in question will occur 100 percent of the time or, in other
words, that it is certain to occur.
Taking the third postulate in the simplest case, that is, for two mutually exclusive
events A1 and A2 , it can easily be seen that it is satisfied by the frequency interpretation. If one event occurs, say, 28 percent of the time, another event occurs 39 percent
of the time, and the two events cannot both occur at the same time (that is, they are
mutually exclusive), then one or the other will occur 28 + 39 = 67 percent of the
time. Thus, the third postulate is satisfied, and the same kind of argument applies
when there are more than two mutually exclusive events.
Before we study some of the immediate consequences of the postulates of probability, let us emphasize the point that the three postulates do not tell us how to
assign probabilities to events; they merely restrict the ways in which it can be done.
EXAMPLE 7
An experiment has four possible outcomes, A, B, C, and D, that are mutually exclusive. Explain why the following assignments of probabilities are not permissible:
(a) P(A) = 0.12, P(B) = 0.63, P(C) = 0.45, P(D) = −0.20;
9
45
27
46
, P(B) = 120
, P(C) = 120
, P(D) = 120
.
(b) P(A) = 120
Solution
(a) P(D) = −0.20 violates Postulate 1;
9
45
27
46
(b) P(S) = P(A ∪ B ∪ C ∪ D) = 120
+ 120
+ 120
+ 120
= 127
120 Z 1, and this violates
Postulate 2.
Probability
Of course, in actual practice probabilities are assigned on the basis of past experience, on the basis of a careful analysis of all underlying conditions, on the basis
of subjective judgments, or on the basis of assumptions—sometimes the assumption
that all possible outcomes are equiprobable.
To assign a probability measure to a sample space, it is not necessary to specify
the probability of each possible subset. This is fortunate, for a sample space with as
few as 20 possible outcomes has already 220 = 1,048,576 subsets, and the number
of subsets grows very rapidly when there are 50 possible outcomes, 100 possible
outcomes, or more. Instead of listing the probabilities of all possible subsets, we
often list the probabilities of the individual outcomes, or sample points of S, and
then make use of the following theorem.
THEOREM 1. If A is an event in a discrete sample space S, then P(A) equals
the sum of the probabilities of the individual outcomes comprising A.
Proof Let O1 , O2 , O3 , . . ., be the finite or infinite sequence of outcomes
that comprise the event A. Thus,
A = O1 ∪ O2 ∪ O3 · · ·
and since the individual outcomes, the O’s, are mutually exclusive, the
third postulate of probability yields
P(A) = P(O1 ) + P(O2 ) + P(O3 ) + · · ·
This completes the proof.
To use this theorem, we must be able to assign probabilities to the individual
outcomes of experiments. How this is done in some special situations is illustrated
by the following examples.
EXAMPLE 8
If we twice flip a balanced coin, what is the probability of getting at least one head?
Solution
The sample space is S = {HH, HT, TH, TT}, where H and T denote head and tail.
Since we assume that the coin is balanced, these outcomes are equally likely and we
assign to each sample point the probability 14 . Letting A denote the event that we
will get at least one head, we get A = {HH, HT, TH} and
P(A) = P(HH) + P(HT) + P(TH)
1 1 1
+ +
4 4 4
3
=
4
=
Probability
EXAMPLE 9
A die is loaded in such a way that each odd number is twice as likely to occur as each
even number. Find P(G), where G is the event that a number greater than 3 occurs
on a single roll of the die.
Solution
The sample space is S = {1, 2, 3, 4, 5, 6}. Hence, if we assign probability w to each
even number and probability 2w to each odd number, we find that 2w + w + 2w +
w + 2w + w = 9w = 1 in accordance with Postulate 2. It follows that w = 19 and
1 2 1
4
P(G) = + + =
9 9 9
9
If a sample space is countably infinite, probabilities will have to be assigned to
the individual outcomes by means of a mathematical rule, preferably by means of a
formula or equation.
EXAMPLE 10
If, for a given experiment, O1 , O2 , O3 , . . ., is an infinite sequence of outcomes, verify that
! "i
1
P(Oi ) =
for i = 1, 2, 3, . . .
2
is, indeed, a probability measure.
Solution
Since the probabilities are all positive, it remains to be shown that P(S) = 1. Getting
P(S) =
1 1 1
1
+ + +
+···
2 4 8 16
and making use of the formula for the sum of the terms of an infinite geometric
progression, we find that
P(S) =
1
2
1 − 12
=1
In connection with the preceding example, the word “sum” in Theorem 1 will
have to be interpreted so that it includes the value of an infinite series.
The probability measure of Example 10 would be appropriate, for example, if
Oi is the event that a person flipping a balanced coin will get a tail for the first time
on the ith flip of the coin. Thus, the probability that the first tail will come on the
third, fourth, or fifth flip of the coin is
! "3 ! "4 ! "5
1
1
7
1
+
+
=
2
2
2
32
and the probability that the first tail will come on an odd-numbered flip of the coin is
! "1 ! "3 ! "5
1
1
1
1
2
+
+
+··· = 2 1 =
2
2
2
3
1−
4
Probability
Here again we made use of the formula for the sum of the terms of an infinite geometric progression.
If an experiment is such that we can assume equal probabilities for all the sample
points, as was the case in Example 8, we can take advantage of the following special
case of Theorem 1.
THEOREM 2. If an experiment can result in any one of N different equally
likely outcomes, and if n of these outcomes together constitute event A,
then the probability of event A is
P(A) =
n
N
Proof Let O1 , O2 , . . . , ON represent the individual outcomes in S, each
1
with probability . If A is the union of n of these mutually exclusive
N
outcomes, and it does not matter which ones, then
P(A) = P(O1 ∪ O2 ∪ · · · ∪ On )
= P(O1 ) + P(O2 ) + · · · + P(On )
=
1
1
1
+ +···+
N&
#N N $%
n
=
N
n terms
n
of Theorem 2 is identical with the one for
N
the classical probability concept (see below). Indeed, what we have shown here is
that the classical probability concept is consistent with the postulates of
probability—it follows from the postulates in the special case where the individual
outcomes are all equiprobable.
Observe that the formula P(A) =
EXAMPLE 11
A five-card poker hand dealt from a deck of 52 playing cards is said to be a full house
if it consists of three of a kind and a pair. If all the five-card hands are equally likely,
what is the probability of being dealt a full house?
Solution
The number of ways in which we can be dealt a particular full house, say three kings
' (' (
4 4
and two aces, is 3 2 . Since there are 13 ways of selecting the face value for the
three of a kind and for each of these there are 12 ways of selecting the face value for
the pair, there are altogether
) *) *
4
4
n = 13 · 12 ·
3
2
different full houses. Also, the total number of equally likely five-card poker
hands is
Probability
) *
52
N=
5
and it follows by Theorem 2 that the probability of getting a full house is
) *) *
4
4
13 · 12
3
2
n
) *
=
P(A) =
= 0.0014
N
52
5
5 Some Rules of Probability
Based on the three postulates of probability, we can derive many other rules that
have important applications. Among them, the next four theorems are immediate
consequences of the postulates.
THEOREM 3. If A and A# are complementary events in a sample space S, then
P(A# ) = 1 − P(A)
Proof In the second and third steps of the proof that follows, we make
use of the definition of a complement, according to which A and A# are
mutually exclusive and A ∪ A# = S. Thus, we write
1 = P(S)
(by Postulate 2)
#
= P(A ∪ A )
= P(A) + P(A# )
(by Postulate 3)
and it follows that P(A# ) = 1 − P(A).
In connection with the frequency interpretation, this result implies that if an
event occurs, say, 37 percent of the time, then it does not occur 63 percent of
the time.
THEOREM 4. P(∅) = 0 for any sample space S.
Proof Since S and ∅ are mutually exclusive and S ∪ ∅ = S in accordance
with the definition of the empty set ∅, it follows that
P(S) = P(S ∪ ∅)
= P(S) + P(∅)
(by Postulate 3)
and, hence, that P(∅) = 0.
It is important to note that it does not necessarily follow from P(A) = 0 that
A = ∅. In practice, we often assign 0 probability to events that, in colloquial terms,
Probability
would not happen in a million years. For instance, there is the classical example that
we assign a probability of 0 to the event that a monkey set loose on a typewriter will
type Plato’s Republic word for word without a mistake. The fact that P(A) = 0 does
not imply that A = ∅ is of relevance, especially, in the continuous case.
THEOREM 5. If A and B are events in a sample space S and A ( B, then
P(A) F P(B).
Proof Since A ( B, we can write
B = A ∪ (A# ∩ B)
as can easily be verified by means of a Venn diagram. Then, since A and
A# ∩ B are mutually exclusive, we get
P(B) = P(A) + P(A# ∩ B)
G P(A)
(by Postulate 3)
(by Postulate 1)
In words, this theorem states that if A is a subset of B, then P(A) cannot be
greater than P(B). For instance, the probability of drawing a heart from an ordinary
deck of 52 playing cards cannot be greater than the probability of drawing a red card.
Indeed, the probability of drawing a heart is 14 , compared with 12 , the probability of
drawing a red card.
THEOREM 6. 0 F P(A) F 1 for any event A.
Proof Using Theorem 5 and the fact that ∅ ( A ( S for any event A in S,
we have
P(∅) F P(A) F P(S)
Then, P(∅) = 0 and P(S) = 1 leads to the result that
0 F P(A) F 1
The third postulate of probability is sometimes referred to as the special addition rule; it is special in the sense that events A1 , A2 , A3 , . . ., must all be mutually
exclusive. For any two events A and B, there exists the general addition rule, or the
inclusion–exclusion principle:
THEOREM 7. If A and B are any two events in a sample space S, then
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Proof Assigning the probabilities a, b, and c to the mutually exclusive
events A ∩ B, A ∩ B# , and A# ∩ B as in the Venn diagram of Figure 6, we
find that
P(A ∪ B) = a + b + c
= (a + b) + (c + a) − a
= P(A) + P(B) − P(A ∩ B)
Probability
A
B
b
a
c
Figure 6. Venn diagram for proof of Theorem 7.
EXAMPLE 12
In a large metropolitan area, the probabilities are 0.86, 0.35, and 0.29, respectively,
that a family (randomly chosen for a sample survey) owns a color television set, a
HDTV set, or both kinds of sets. What is the probability that a family owns either or
both kinds of sets?
Solution
If A is the event that a family in this metropolitan area owns a color television set
and B is the event that it owns a HDTV set, we have P(A) = 0.86, P(B) = 0.35, and
P(A ∩ B) = 0.29; substitution into the formula of Theorem 7 yields
P(A ∪ B) = 0.86 + 0.35 − 0.29
= 0.92
EXAMPLE 13
Near a certain exit of I-17, the probabilities are 0.23 and 0.24, respectively, that
a truck stopped at a roadblock will have faulty brakes or badly worn tires. Also,
the probability is 0.38 that a truck stopped at the roadblock will have faulty brakes
and/or badly worn tires. What is the probability that a truck stopped at this roadblock
will have faulty brakes as well as badly worn tires?
Solution
If B is the event that a truck stopped at the roadblock will have faulty brakes and T
is the event that it will have badly worn tires, we have P(B) = 0.23, P(T) = 0.24, and
P(B ∪ T) = 0.38; substitution into the formula of Theorem 7 yields
0.38 = 0.23 + 0.24 − P(B ∩ T)
Solving for P(B ∩ T), we thus get
P(B ∩ T) = 0.23 + 0.24 − 0.38 = 0.09
Repeatedly using the formula of Theorem 7, we can generalize this addition rule
so that it will apply to any number of events. For instance, for three events we obtain
the following theorem.
Probability
THEOREM 8. If A, B, and C are any three events in a sample space S, then
P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) − P(A ∩ C)
− P(B ∩ C) + P(A ∩ B ∩ C)
Proof Writing A ∪ B ∪ C as A ∪ (B ∪ C) and using the formula of Theorem 7 twice, once for P[A ∪ (B ∪ C)] and once for P(B ∪ C), we get
P(A ∪ B ∪ C) = P[A ∪ (B ∪ C)]
= P(A) + P(B ∪ C) − P[A ∩ (B ∪ C)]
= P(A) + P(B) + P(C) − P(B ∩ C)
− P[A ∩ (B ∪ C)]
Then, using the distributive law that the reader was asked to verify in part
(b) of Exercise 1, we find that
P[A ∩ (B ∪ C)] = P[(A ∩ B) ∪ (A ∩ C)]
= P(A ∩ B) + P(A ∩ C) − P[(A ∩ B) ∩ (A ∩ C)]
= P(A ∩ B) + P(A ∩ C) − P(A ∩ B ∩ C)
and hence that
P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) − P(A ∩ C)
− P(B ∩ C) + P(A ∩ B ∩ C)
(In Exercise 12 the reader will be asked to give an alternative proof of this theorem, based on the method used in the text to prove Theorem 7.)
EXAMPLE 14
If a person visits his dentist, suppose that the probability that he will have his teeth
cleaned is 0.44, the probability that he will have a cavity filled is 0.24, the probability
that he will have a tooth extracted is 0.21, the probability that he will have his teeth
cleaned and a cavity filled is 0.08, the probability that he will have his teeth cleaned
and a tooth extracted is 0.11, the probability that he will have a cavity filled and
a tooth extracted is 0.07, and the probability that he will have his teeth cleaned,
a cavity filled, and a tooth extracted is 0.03. What is the probability that a person
visiting his dentist will have at least one of these things done to him?
Solution
If C is the event that the person will have his teeth cleaned, F is the event that he
will have a cavity filled, and E is the event that he will have a tooth extracted, we
are given P(C) = 0.44, P(F) = 0.24, P(E) = 0.21, P(C ∩ F) = 0.08, P(C ∩ E) = 0.11,
P(F ∩ E) = 0.07, and P(C ∩ F ∩ E) = 0.03, and substitution into the formula of
Theorem 8 yields
P(C ∪ F ∪ E) = 0.44 + 0.24 + 0.21 − 0.08 − 0.11 − 0.07 + 0.03
= 0.66
Probability
Exercises
5. Use parts (a) and (b) of Exercise 3 to show that
(a) P(A) G P(A ∩ B);
(b) P(A) F P(A ∪ B).
6. Referring to Figure 6, verify that
13. Duplicate the method of proof used in Exercise 12 to
show that
P(A ∪ B ∪ C ∪ D) = P(A) + P(B) + P(C) + P(D)
− P(A ∩ B) − P(A ∩ C) − P(A ∩ D)
P(A ∩ B# ) = P(A) − P(A ∩ B)
− P(B ∩ C) − P(B ∩ D) − P(C ∩ D)
7. Referring to Figure 6 and letting P(A# ∩ B# ) = d, verify that
+ P(A ∩ B ∩ C) + P(A ∩ B ∩ D)
P(A# ∩ B# ) = 1 − P(A) − P(B) + P(A ∩ B)
8. The event that “A or B but not both” will occur can be
written as
(A ∩ B# ) ∪ (A# ∩ B)
Express the probability of this event in terms of P(A),
P(B), and P(A ∩ B).
9. Use the formula of Theorem 7 to show that
(a) P(A ∩ B) F P(A) + P(B);
(b) P(A ∩ B) G P(A) + P(B) − 1.
10. Use the Venn diagram of Figure 7 with the probabilities a, b, c, d, e, f , and g assigned to A ∩ B ∩ C,
A ∩ B ∩ C# , . . ., and A ∩ B# ∩ C# to show that if P(A) =
P(B) = P(C) = 1, then P(A ∩ B ∩ C) = 1. [Hint: Start
with the argument that since P(A) = 1, it follows that
e = c = f = 0.]
11. Give an alternative proof of Theorem 7 by making
use of the relationships A ∪ B = A ∪ (A# ∩ B) and B =
(A ∩ B) ∪ (A# ∩ B).
12. Use the Venn diagram of Figure 7 and the method by
which we proved Theorem 7 to prove Theorem 8.
A
B
b
g
e
a
d
c
f
C
+ P(A ∩ C ∩ D) + P(B ∩ C ∩ D)
− P(A ∩ B ∩ C ∩ D)
(Hint: With reference to the Venn diagram of Figure 7,
divide each of the eight regions into two parts, designating one to be inside D and the other outside D and letting
a, b, c, d, e, f , g, h, i, j, k, l, m, n, o, and p be the probabilities associated with the resulting 16 regions.)
14. Prove by induction that
P(E1 ∪ E2 ∪ · · · ∪ En ) F
P(Ei )
i=1
for any finite sequence of events E1 , E2 , . . ., and En .
15. The odds that an event will occur are given by the
ratio of the probability that the event will occur to the
probability that it will not occur, provided neither probability is zero. Odds are usually quoted in terms of positive
integers having no common factor. Show that if the odds
are A to B that an event will occur, its probability is
A
p=
A+B
16. Subjective probabilities may be determined by exposing persons to risk-taking situations and finding the odds
at which they would consider it fair to bet on the outcome.
The odds are then converted into probabilities by means
of the formula of Exercise 15. For instance, if a person
feels that 3 to 2 are fair odds that a business venture will
succeed (or that it would be fair to bet $30 against $20
3
that it will succeed), the probability is
= 0.6 that
3+2
the business venture will succeed. Show that if subjective
probabilities are determined in this way, they satisfy
(a) Postulate 1;
(b) Postulate 2.
See also Exercise 82.
Figure 7. Venn diagram for Exercises 10, 12, and 13.
n
+
Probability
6 Conditional Probability
Difficulties can easily arise when probabilities are quoted without specification of
the sample space. For instance, if we ask for the probability that a lawyer makes
more than $75,000 per year, we may well get several different answers, and they may
all be correct. One of them might apply to all those who are engaged in the private
practice of law, another might apply to lawyers employed by corporations, and so
forth. Since the choice of the sample space (that is, the set of all possibilities under
consideration) is by no means always self-evident, it often helps to use the symbol
P(A|S) to denote the conditional probability of event A relative to the sample space
S or, as we also call it, “the probability of A given S.” The symbol P(A|S) makes it
explicit that we are referring to a particular sample space S, and it is preferable to
the abbreviated notation P(A) unless the tacit choice of S is clearly understood. It is
also preferable when we want to refer to several sample spaces in the same example.
If A is the event that a person makes more than $75,000 per year, G is the event that
a person is a law school graduate, L is the event that a person is licensed to practice
law, and E is the event that a person is actively engaged in the practice of law, then
P(A|G) is the probability that a law school graduate makes more than $75,000 per
year, P(A|L) is the probability that a person licensed to practice law makes more
than $75,000 per year, and P(A|E) is the probability that a person actively engaged
in the practice of law makes more than $75,000 per year.
Some ideas connected with conditional probabilities are illustrated in the following example.
EXAMPLE 15
A consumer research organization has studied the services under warranty provided
by the 50 new-car dealers in a certain city, and its findings are summarized in the
following table.
Good service
under warranty
Poor service
under warranty
In business 10 years or more
16
4
In business less than 10 years
10
20
If a person randomly selects one of these new-car dealers, what is the probability that
he gets one who provides good service under warranty? Also, if a person randomly
selects one of the dealers who has been in business for 10 years or more, what is the
probability that he gets one who provides good service under warranty?
Solution
By “randomly” we mean that, in each case, all possible selections are equally likely,
and we can therefore use the formula of Theorem 2. If we let G denote the selection
of a dealer who provides good service under warranty, and if we let n(G) denote
the number of elements in G and n(S) the number of elements in the whole sample
space, we get
P(G) =
This answers the first question.
16 + 10
n(G)
=
= 0.52
n(S)
50
Probability
For the second question, we limit ourselves to the reduced sample space, which
consists of the first line of the table, that is, the 16 + 4 = 20 dealers who have been
in business 10 years or more. Of these, 16 provide good service under warranty, and
we get
16
P(G|T) =
= 0.80
20
where T denotes the selection of a dealer who has been in business 10 years or
more. This answers the second question and, as should have been expected, P(G|T)
is considerably higher than P(G).
Since the numerator of P(G|T) is n(T ∩ G) = 16 in the preceding example, the
number of dealers who have been in business for 10 years or more and provide good
service under warranty, and the denominator is n(T), the number of dealers who
have been in business 10 years or more, we can write symbolically
P(G|T) =
n(T ∩ G)
n(T)
Then, if we divide the numerator and the denominator by n(S), the total number of
new-car dealers in the given city, we get
n(T∩G)
P(T ∩ G)
n(S)
P(G|T) = n(T) =
P(T)
n(S)
and we have, thus, expressed the conditional probability P(G|T) in terms of two
probabilities defined for the whole sample space S.
Generalizing from the preceding, let us now make the following definition of
conditional probability.
DEFINITION 4. CONDITIONAL PROBABILITY. If A and B are any two events in a sample
space S and P(A) Z 0, the conditional probability of B given A is
P(B|A) =
P(A ∩ B)
P(A)
EXAMPLE 16
With reference to Example 15, what is the probability that one of the dealers who
has been in business less than 10 years will provide good service under warranty?
Solution
10
10 + 20
Since P(T # ∩ G) =
= 0.20 and P(T # ) =
= 0.60, substitution into the
50
50
formula yields
P(G|T # ) =
0.20
1
P(T # ∩ G)
=
=
#
P(T )
0.60
3
Probability
Although we introduced the formula for P(B|A) by means of an example in
which the possibilities were all equally likely, this is not a requirement for its use.
EXAMPLE 17
With reference to the loaded die of Example 9, what is the probability that the number of points rolled is a perfect square? Also, what is the probability that it is a
perfect square given that it is greater than 3?
Solution
If A is the event that the number of points rolled is greater than 3 and B is the event
that it is a perfect square, we have A = {4, 5, 6}, B = {1, 4}, and A ∩ B = {4}. Since
the probabilities of rolling a 1, 2, 3, 4, 5, or 6 with the die are 29 , 19 , 29 , 19 , 29 , and 19 , we
find that the answer to the first question is
P(B) =
2 1
1
+ =
9 9
3
To determine P(B|A), we first calculate
P(A ∩ B) =
1
9
and
P(A) =
1 2 1
4
+ + =
9 9 9
9
Then, substituting into the formula of Definition 4, we get
1
P(A ∩ B)
1
P(B|A) =
= 9 =
4
P(A)
4
9
To verify that the formula of Definition 4 has yielded the “right” answer in the
preceding example, we have only to assign probability v to the two even numbers
in the reduced sample space A and probability 2v to the odd number, such that the
sum of the three probabilities is equal to 1. We thus have v + 2v + v = 1, v = 14 , and,
hence, P(B|A) = 14 as before.
EXAMPLE 18
A manufacturer of airplane parts knows from past experience that the probability
is 0.80 that an order will be ready for shipment on time, and it is 0.72 that an order
will be ready for shipment on time and will also be delivered on time. What is the
probability that such an order will be delivered on time given that it was ready for
shipment on time?
Solution
If we let R stand for the event that an order is ready for shipment on time and D be
the event that it is delivered on time, we have P(R) = 0.80 and P(R ∩ D) = 0.72, and
it follows that
0.72
P(R ∩ D)
=
= 0.90
P(D|R) =
P(R)
0.80
Probability
Thus, 90 percent of the shipments will be delivered on time provided they are shipped
on time. Note that P(R|D), the probability that a shipment that is delivered on time
was also ready for shipment on time, cannot be determined without further information; for this purpose we would also have to know P(D).
If we multiply the expressions on both sides of the formula of Definition 4 by
P(A), we obtain the following multiplication rule.
THEOREM 9. If A and B are any two events in a sample space S and P(A) Z 0,
then
P(A ∩ B) = P(A) · P(B|A)
In words, the probability that A and B will both occur is the product of the probability of A and the conditional probability of B given A. Alternatively, if P(B) Z 0, the
probability that A and B will both occur is the product of the probability of B and
the conditional probability of A given B; symbolically,
P(A ∩ B) = P(B) · P(A|B)
To derive this alternative multiplication rule, we interchange A and B in the formula
of Theorem 9 and make use of the fact that A ∩ B = B ∩ A.
EXAMPLE 19
If we randomly pick two television sets in succession from a shipment of 240 television sets of which 15 are defective, what is the probability that they will both
be defective?
Solution
If we assume equal probabilities for each selection (which is what we mean by “ran15
domly” picking the sets), the probability that the first set will be defective is 240
, and
the probability that the second set will be defective given that the first set is defec14
15
14
7
tive is 239
. Thus, the probability that both sets will be defective is 240
· 239
= 1,912
.
This assumes that we are sampling without replacement; that is, the first set is not
replaced before the second set is selected.
EXAMPLE 20
Find the probabilities of randomly drawing two aces in succession from an ordinary
deck of 52 playing cards if we sample
(a) without replacement;
(b) with replacement.
Solution
(a) If the first card is not replaced before the second card is drawn, the probability
of getting two aces in succession is
1
4 3
·
=
52 51
221
Probability
(b) If the first card is replaced before the second card is drawn, the corresponding
probability is
4 4
1
·
=
52 52
169
In the situations described in the two preceding examples there is a definite
temporal order between the two events A and B. In general, this need not be the
case when we write P(A|B) or P(B|A). For instance, we could ask for the probability that the first card drawn was an ace given that the second card drawn (without
3
.
replacement) is an ace—the answer would also be 51
Theorem 9 can easily be generalized so that it applies to more than two events;
for instance, for three events we have the following theorem.
THEOREM 10. If A, B, and C are any three events in a sample space S such
that P(A ∩ B) Z 0, then
P(A ∩ B ∩ C) = P(A) · P(B|A) · P(C|A ∩ B)
Proof Writing A ∩ B ∩ C as (A ∩ B) ∩ C and using the formula of Theorem 9 twice, we get
P(A ∩ B ∩ C) = P[(A ∩ B) ∩ C]
= P(A ∩ B) · P(C|A ∩ B)
= P(A) · P(B|A) · P(C|A ∩ B)
EXAMPLE 21
A box of fuses contains 20 fuses, of which 5 are defective. If 3 of the fuses are selected
at random and removed from the box in succession without replacement, what is the
probability that all 3 fuses are defective?
Solution
If A is the event that the first fuse is defective, B is the event that the second fuse
5
,
is defective, and C is the event that the third fuse is defective, then P(A) = 20
4
3
P(B|A) = 19 , P(C|A ∩ B) = 18 , and substitution into the formula yields
5 4 3
·
·
20 19 18
1
=
114
P(A ∩ B ∩ C) =
Further generalization of Theorems 9 and 10 to k events is straightforward, and
the resulting formula can be proved by mathematical induction.
Probability
7 Independent Events
Informally speaking, two events A and B are independent if the occurrence or nonoccurrence of either one does not affect the probability of the occurrence of the other.
For instance, in the preceding example the selections would all have been independent had each fuse been replaced before the next one was selected; the probability
5
.
of getting a defective fuse would have remained 20
Symbolically, two events A and B are independent if P(B|A) = P(B) and
P(A|B) = P(A), and it can be shown that either of these equalities implies the other
when both of the conditional probabilities exist, that is, when neither P(A) nor P(B)
equals zero (see Exercise 21).
Now, if we substitute P(B) for P(B|A) into the formula of Theorem 9, we get
P(A ∩ B) = P(A) · P(B|A)
= P(A) · P(B)
and we shall use this as our formal definition of independence.
DEFINITION 5. INDEPENDENCE. Two events A and B are independent if and only if
P(A ∩ B) = P(A) · P(B)
Reversing the steps, we can also show that Definition 5 implies the definition of independence that we gave earlier.
If two events are not independent, they are said to be dependent. In the derivation of the formula of Definition 5, we assume that P(B|A) exists and, hence, that
P(A) Z 0. For mathematical convenience, we shall let the definition apply also when
P(A) = 0 and/or P(B) = 0.
EXAMPLE 22
A coin is tossed three times and the eight possible outcomes, HHH, HHT, HTH,
THH, HTT, THT, TTH, and TTT, are assumed to be equally likely. If A is the event
that a head occurs on each of the first two tosses, B is the event that a tail occurs
on the third toss, and C is the event that exactly two tails occur in the three tosses,
show that
(a) events A and B are independent;
(b) events B and C are dependent.
Solution
Since
A = {HHH, HHT}
B = {HHT, HTT, THT, TTT}
C = {HTT, THT, TTH}
A ∩ B = {HHT}
B ∩ C = {HTT, THT}
Probability
the assumption that the eight possible outcomes are all equiprobable yields
P(A) = 14 , P(B) = 12 , P(C) = 38 , P(A ∩ B) = 18 , and P(B ∩ C) = 14 .
(a) Since P(A) · P(B) = 14 · 12 = 18 = P(A ∩ B), events A and B are independent.
(b) Since P(B) · P(C) =
pendent.
1 3
2·8
=
3
16 Z P(B ∩ C), events B and C are not inde-
In connection with Definition 5, it can be shown that if A and B are independent,
then so are A and B# , A# and B, and A# and B# . For instance, consider the following
theorem.
THEOREM 11. If A and B are independent, then A and B# are also independent.
Proof Since A = (A ∩ B) ∪ (A ∩ B# ), as the reader was asked to show in
part (a) of Exercise 3, A ∩ B and A ∩ B# are mutually exclusive, and A and
B are independent by assumption, we have
P(A) = P[(A ∩ B) ∪ (A ∩ B# )]
= P(A ∩ B) + P(A ∩ B# )
= P(A) · P(B) + P(A ∩ B# )
It follows that
P(A ∩ B# ) = P(A) − P(A) · P(B)
= P(A) · [1 − P(B)]
= P(A) · P(B# )
and hence that A and B# are independent.
In Exercises 22 and 23 the reader will be asked to show that if A and B are
independent, then A# and B are independent and so are A# and B# , and if A and B
are dependent, then A and B# are dependent.
To extend the concept of independence to more than two events, let us make the
following definition.
DEFINITION 6. INDEPENDENCE OF MORE THAN TWO EVENTS. Events A1 , A2 , . . . , and
Ak are independent if and only if the probability of the intersections of any 2, 3,
. . . , or k of these events equals the product of their respective probabilities.
For three events A, B, and C, for example, independence requires that
P(A ∩ B) = P(A) · P(B)
P(A ∩ C) = P(A) · P(C)
P(B ∩ C) = P(B) · P(C)
Probability
and
P(A ∩ B ∩ C) = P(A) · P(B) · P(C)
It is of interest to note that three or more events can be pairwise independent
without being independent.
EXAMPLE 23
Figure 8 shows a Venn diagram with probabilities assigned to its various regions.
Verify that A and B are independent, A and C are independent, and B and C are
independent, but A, B, and C are not independent.
Solution
As can be seen from the diagram, P(A) = P(B) = P(C) =
P(A ∩ C) = P(B ∩ C) = 14 , and P(A ∩ B ∩ C) = 14 . Thus,
1
2 , P(A ∩ B)
=
1
= P(A ∩ B)
4
1
P(A) · P(C) = = P(A ∩ C)
4
1
P(B) · P(C) = = P(B ∩ C)
4
P(A) · P(B) =
but
P(A) · P(B) · P(C) =
1
Z P(A ∩ B ∩ C)
8
A
B
1
4
1
4
1
4
1
4
C
Figure 8. Venn diagram for Example 23.
Incidentally, the preceding example can be given a “real” interpretation by considering a large room that has three separate switches controlling the ceiling lights.
These lights will be on when all three switches are “up” and hence also when one
of the switches is “up” and the other two are “down.” If A is the event that the first
switch is “up,” B is the event that the second switch is “up,” and C is the event that
the third switch is “up,” the Venn diagram of Figure 8 shows a possible set of probabilities associated with the switches being “up” or “down” when the ceiling lights
are on.
Probability
It can also happen that P(A ∩ B ∩ C) = P(A) · P(B) · P(C) without A, B, and C
being pairwise independent—this the reader will be asked to verify in Exercise 24.
Of course, if we are given that certain events are independent, the probability
that they will all occur is simply the product of their respective probabilities.
EXAMPLE 24
Find the probabilities of getting
(a) three heads in three random tosses of a balanced coin;
(b) four sixes and then another number in five random rolls of a balanced die.
Solution
1
(a) The probability of a head on each toss is and the three outcomes are inde2
pendent. Thus we can multiply, obtaining
1 1 1
1
· · =
2 2 2
8
1
(b) The probability of a six on each toss is ; thus the probability of tossing a
6
5
number other than 6 is . Inasmuch as the tosses are independent, we can
6
multiply the respective probabilities to obtain
1 1 1 1 5
5
· · · · =
6 6 6 6 6
7, 776
8 Bayes’ Theorem
In many situations the outcome of an experiment depends on what happens in various intermediate stages. The following is a simple example in which there is one
intermediate stage consisting of two alternatives:
EXAMPLE 25
The completion of a construction job may be delayed because of a strike. The probabilities are 0.60 that there will be a strike, 0.85 that the construction job will be
completed on time if there is no strike, and 0.35 that the construction job will be
completed on time if there is a strike. What is the probability that the construction
job will be completed on time?
Solution
If A is the event that the construction job will be completed on time and B is the
event that there will be a strike, we are given P(B) = 0.60, P(A|B# ) = 0.85, and
P(A|B) = 0.35. Making use of the formula of part (a) of Exercise 3, the fact that A ∩
B and A ∩ B# are mutually exclusive, and the alternative form of the multiplication
rule, we can write
P(A) = P[(A ∩ B) ∪ (A ∩ B# )]
= P(A ∩ B) + P(A ∩ B# )
= P(B) · P(A|B) + P(B# ) · P(A|B# )
Probability
Then, substituting the given numerical values, we get
P(A) = (0.60)(0.35) + (1 − 0.60)(0.85)
= 0.55
An immediate generalization of this kind of situation is the case where the
intermediate stage permits k different alternatives (whose occurrence is denoted by
B1 , B2 , . . . , Bk ). It requires the following theorem, sometimes called the rule of total
probability or the rule of elimination.
THEOREM 12. If the events B1 , B2 , . . . , and Bk constitute a partition of the
sample space S and P(Bi ) Z 0 for i = 1, 2, . . . , k, then for any event A in S
P(A) =
k
+
i=1
P(Bi ) · P(A|Bi )
The B’s constitute a partition of the sample space if they are pairwise mutually exclusive and if their union equals S. A formal proof of Theorem 12 consists, essentially,
of the same steps we used in Example 25, and it is left to the reader in Exercise 32.
EXAMPLE 26
The members of a consulting firm rent cars from three rental agencies: 60 percent
from agency 1, 30 percent from agency 2, and 10 percent from agency 3. If 9 percent
of the cars from agency 1 need an oil change, 20 percent of the cars from agency 2
need an oil change, and 6 percent of the cars from agency 3 need an oil change, what
is the probability that a rental car delivered to the firm will need an oil change?
Solution
If A is the event that the car needs an oil change, and B1 , B2 , and B3 are the events
tha...
Essay Writing Service Features
Our Experience
No matter how complex your assignment is, we can find the right professional for your specific task. Achiever Papers is an essay writing company that hires only the smartest minds to help you with your projects. Our expertise allows us to provide students with high-quality academic writing, editing & proofreading services.
Free Features
Free revision policy
$10Free bibliography & reference
$8Free title page
$8Free formatting
$8How Our Dissertation Writing Service Works
First, you will need to complete an order form. It's not difficult but, if anything is unclear, you may always chat with us so that we can guide you through it. On the order form, you will need to include some basic information concerning your order: subject, topic, number of pages, etc. We also encourage our clients to upload any relevant information or sources that will help.
Complete the order form
Once we have all the information and instructions that we need, we select the most suitable writer for your assignment. While everything seems to be clear, the writer, who has complete knowledge of the subject, may need clarification from you. It is at that point that you would receive a call or email from us.
Writer’s assignment
As soon as the writer has finished, it will be delivered both to the website and to your email address so that you will not miss it. If your deadline is close at hand, we will place a call to you to make sure that you receive the paper on time.
Completing the order and download