# Untitled Page 9

## Chapter 4. Independence of Events

### 4.1. Independence of Events*

Historically, the notion of independence has played a prominent role in probability. If events form an independent class, much less information is required to determine probabilities of Boolean combinations and calculations are correspondingly easier. In this unit, we give a precise formulation of the concept of independence in the probability sense. As in the case of all concepts which attempt to incorporate intuitive notions, the consequences must be evaluated for evidence that these ideas have been captured successfully.

#### Independence as lack of conditioning

There are many situations in which we have an “operational independence.”

• Supose a deck of playing cards is shuffled and a card is selected at random then replaced with reshuffling. A second card picked on a repeated try should not be affected by the first choice.

• If customers come into a well stocked shop at different times, each unaware of the choice made by the others, the the item purchased by one should not be affected by the choice made by the other.

• If two students are taking exams in different courses, the grade one makes should not affect the grade made by the other.

The list of examples could be extended indefinitely. In each case, we should expect to model the events as independent in some way. How should we incorporate the concept in our developing model of probability?

We take our clue from the examples above. Pairs of events are considered. The “operational independence” described indicates that knowledge that one of the events has occured does not affect the likelihood that the other will occur. For a pair of events {A,B}, this is the condition

(4.1) P ( A | B ) = P ( A )

Occurrence of the event A is not “conditioned by” occurrence of the event B. Our basic interpretation is that P(A) indicates of the likelihood of the occurrence of event A. The development of conditional probability in the module Conditional Probability, leads to the interpretation of P(A|B) as the likelihood that A will occur on a trial, given knowledge that B has occurred. If such knowledge of the occurrence of B does not affect the likelihood of the occurrence of A, we should be inclined to think of the events A and B as being independent in a probability sense.

#### Independent pairs

We take our clue from the condition P(A|B)=P(A). Property (CP4) for conditional probability (in the case of equality) yields sixteen equivalent conditions as follows.

 P ( A | B ) = P ( A ) P ( B | A ) = P ( B ) P ( A B ) = P ( A ) P ( B )

These conditions are equivalent in the sense that if any one holds, then all hold. We may chose any one of these as the defining condition and consider the others as equivalents for the defining condition. Because of its simplicity and symmetry with respect to the two events, we adopt the product rule in the upper right hand corner of the table.

Definition. The pair {A,B} of events is said to be (stochastically) independent iff the following product rule holds:

(4.2) P ( A B ) = P ( A ) P ( B )

Remark. Although the product rule is adopted as the basis for definition, in many applications the assumptions leading to independence may be formulated more naturally in terms of one or another of the equivalent expressions. We are free to do this, for the effect of assuming any one condition is to assume them all.

The equivalences in the right-hand column of the upper portion of the table may be expressed as a replacement rule, which we augment and extend below:

If the pair independent, so is any pair obtained by taking the complement of either or both of the events.

We note two relevant facts

• Suppose event N has probability zero (is a null event). Then for any event A, we have 0≤P(AN)≤P(N)=0=P(A)P(N), so that the product rule holds. Thus {N,A} is an independent pair for any event A.

• If event S has probability one (is an almost sure event), then its complement Sc is a null event. By the replacement rule and the fact just established, is independent, so {S,A} is independent.

The replacement rule may thus be extended to:

Replacement Rule

If the pair independent, so is any pair obtained by replacing either or both of the events by their complements or by a null event or by an almost sure event.

CAUTION
1. Unless at least one of the events has probability one or zero, a pair cannot be both independent and mutually exclusive. Intuitively, if the pair is mutually exclusive, then the occurrence of one requires that the other does not occur. Formally: Suppose 0<P(A)<1 and 0<P(B)<1. {A,B} mutually exclusive implies P(AB)=P()=0≠P(A)P(B). {A,B} independent implies P(AB)=P(A)P(B)>0=P()

2. Independence is not a property of events. Two non mutually exclusive events may be independent under one probability measure, but may not be independent for another. This can be seen by considering various probability distributions on a Venn diagram or minterm map.

#### Independent classes

Extension of the concept of independence to an arbitrary class of events utilizes the product rule.

Definition. A class of events is said to be (stochastically) independent iff the product rule holds for every finite subclass of two or more events in the class.

A class is independent iff all four of the following product rules hold

(4.3)

If any one or more of these product expressions fail, the class is not independent. A similar situation holds for a class of four events: the product rule must hold for every pair, for every triple, and for the whole class. Note that we say “not independent” or “nonindependent” rather than dependent. The reason for this becomes clearer in dealing with independent random variables.

We consider some classical exmples of nonindependent classes

Example 4.1. Some nonindependent classes
1. Suppose is a partition, with each . Let

(4.4)A=A1A2B=A1A3C=A1A4

Then the class {A,B,C} has P(A)=P(B)=P(C)=1/2 and is pairwise independent, but not independent, since

(4.5)
(4.6)
2. Consider the class {A,B,C,D} with AD=BD=, C=ABD, P(A)=P(B)=1/4, P(AB)=1/64, and P(D)=15/64. Use of a minterm maps shows these assignments are consistent. Elementary calculations show the product rule applies to the class {A,B,C} but no two of these three events forms an independent pair.

As noted above, the replacement rule holds for any pair of events. It is easy to show, although somewhat cumbersome to write out, that if the rule holds for any finite number k of events in an independent class, it holds for any k+1 of them. By the principle of mathematical induction, the rule must hold for any finite subclass. We may extend the replacement rule as follows.

General Replacement Rule

If a class is independent, we may replace any of the sets by its complement, by a null event, or by an almost sure event, and the resulting class is also independent. Such replacements may be made for any number of the sets in the class. One immediate and important consequence is the following.

Minterm Probabilities

If is an independent class and the the class of individual probabilities is known, then the probability of every minterm may be calculated.

Example 4.2. Minterm probabilities for an independent class

Suppose the class is independent with respective probabilities P(A)=0.3, P(B)=0.6, and P(C)=0.5. Then

is independent and

is independent and

Similarly, the probabilities of the other six minterms, in order, are 0.21, 0.21, 0.06, 0.06, 0.09, and 0.09. With these minterm probabilities, the probability of any Boolean combination of , and C may be calculated

In general, eight appropriate probabilities must be specified to determine the minterm probabilities for a class of three events. In the independent case, three appropriate probabilities are sufficient.

Example 4.3. Three probabilities yield the minterm probabilities

Suppose is independent with P(ABC)=0.51, , and P(A)=0.30. Then and

(4.7)

With each of the basic probabilities determined, we may calculate the minterm probabilities, hence the probability of any Boolean combination of the events.

Example 4.4. MATLAB and the product rule

Frequently we have a large enough independent class that it is desirable to use MATLAB (or some other computational aid) to calculate the probabilities of various “and” combinations (intersections) of the events or their complements. Suppose the independent class has respective probabilities

(4.8)

It is desired to calculate (a) , and (b) .

We may use the MATLAB function prod and the scheme for indexing a matrix.

>> p = 0.01*[13 37 12 56 33 71 22 43 57 31];
>> q = 1-p;
>> % First case
>> e = [1 2 4 7];                  % Uncomplemented positions
>> f = [3 5 6];                    % Complemented positions
>> P = prod(p(e))*prod(q(f))       % p(e) probs of uncomplemented factors
P = 0.0010                         % q(f) probs of complemented factors
>> % Case of uncomplemented in even positions; complemented in odd positions
>> g = find(rem(1:10,2) == 0);     % The even positions
>> h = find(rem(1:10,2) ~= 0);     % The odd positions
>> P = prod(p(g))*prod(q(h))
P = 0.0034

In the unit on MATLAB and Independent Classes, we extend the use of MATLAB in the calculations for such classes.

### 4.2. MATLAB and Independent Classes*

#### MATLAB and Independent Classes

In the unit on Minterms, we show how to use minterm probabilities and minterm vectors to calculate probabilities of Boolean combinations of events. In Independence of Events we show that in the independent case, we may calculate all minterm probabilities from the probabilities of the basic events. While these calculations are straightforward, they may be tedious and subject to errors. Fortunately, in this case we have an m-function minprob which calculates all minterm probabilities from the probabilities of the basic or generating sets. This function uses the m-function mintable to set up the patterns of p's and q's for the various minterms and then takes the products to obtain the set of minterm probabilities.

Example 4.5.
>> pm = minprob(0.1*[4 7 6])
pm = 0.0720  0.1080  0.1680  0.2520  0.0480  0.0720  0.1120  0.1680

It may be desirable to arrange these as on a minterm map. For this we have an m-function minmap which reshapes the row matrix pm, as follows:

>> t = minmap(pm)
t = 0.0720    0.1680    0.0480    0.1120
0.1080    0.2520    0.0720    0.1680

Probability of occurrence of k of n independent events

In Example 2, we show how to use the m-functions mintable and csort to obtain the probability of the occurrence of k of n events, when minterm probabilities are available. In the case of an independent class, the minterm probabilities are calculated easily by minprob, It is only necessary to specify the probabilities for the n basic events and the numbers k of events. The size of the class, hence the mintable, is determined, and the minterm probabilities are calculated by minprob. We have two useful m-functions. If P is a matrix of the n individual event probabilities, and k is a matrix of integers less than or equal to n, then

function y = ikn ( P , k ) calculates individual probabilities that k of n occur

function y = ckn ( P , k ) calculates the probabilities that k or more occur

Example 4.6.
>> p = 0.01*[13 37 12 56 33 71 22 43 57 31];
>> k = [2 5 7];
>> P = ikn(p,k)
P =    0.1401    0.1845    0.0225       % individual probabilities
>> Pc = ckn(p,k)
Pc =   0.9516    0.2921    0.0266       % cumulative probabilities

Reliability of systems with independent components

Suppose a system has n components which fail independently. Let Ei be the event the ith component survives the designated time period. Then is defined to be the reliability of that component. The reliability R of the complete system is a function of the component reliabilities. There are three basic configurations. General systems may be decomposed into subsystems of these types. The subsystems become components in the larger configuration. The three fundamental configurations are:

1. Series. The system operates iff all n components operate:

2. Parallel. The system operates iff not all components fail:

3. k of n. The system operates iff k or more components operate. R may be calculated with the m-function ckn. If the component probabilities are all the same, it is more efficient to use the m-function cbinom (see Bernoulli trials and the binomial distribution, below).

MATLAB solution. Put the component reliabilities in matrix

1. Series Configuration

>> R = prod(RC)     % prod is a built in MATLAB function

2. Parallel Configuration

>> R = parallel(RC) % parallel is a user defined function

3. k of n Configuration

>> R = ckn(RC,k)    % ckn is a user defined function (in file ckn.m).

Example 4.7.

There are eight components, numbered 1 through 8. Component 1 is in series with a parallel combination of components 2 and 3, followed by a 3 of 5 combination of components 4 through 8 (see Figure 1 for a schematic representation). Probabilities of the components in order are

(4.9)

The second and third probabilities are for the parallel pair, and the last five probabilities are for the 3 of 5 combination.

>> RC = 0.01*[95 90 92 80 83 91 85 85];        % Component reliabilities
>> Ra = RC(1)*parallel(RC(2:3))*ckn(RC(4:8),3) % Solution
Ra = 0.9172
Example 4.8.
>> RC = 0.01*[95 90 92 80 83 91 85 85];    % Component reliabilities 1--8
>> Rb = prod(RC(1:2))*parallel([RC(3),ckn(RC(4:8),3)])     % Solution
Rb = 0.8532

A test for independence

It is difficult to look at a list of minterm probabilities and determine whether or not the generating events form an independent class. The m-function imintest has as argument a vector of minterm probabilities. It checks for feasible size, determines the number of variables, and performs a check for independence.

Example 4.9.
>> pm = 0.01*[15 5 2 18 25 5 18 12];   % An arbitrary class
>> disp(imintest(pm))
The class is NOT independent
Minterms for which the product rule fails
1     1     1     0
1     1     1     0

Example 4.10.
>> pm = [0.10 0.15 0.20 0.25 0.30]: %An improper number of probabilities
>> disp(imintest(pm))
The number of minterm probabilities incorrect
Example 4.11.
>> pm = minprob([0.5 0.3 0.7]);
>> disp(imintest(pm))
The class is independent

#### Probabilities of Boolean combinations

As in the nonindependent case, we may utilize the minterm expansion and the minterm probabilities to calculate the probabilities of Boolean combinations of events. However, it is frequently more efficient to manipulate the expressions for the Boolean combination to be a disjoint union of intersections.

Example 4.12. A simple Boolean combination

Suppose the class is independent, with respective probabilities 0.4, 0.6, 0.8. Determine P(ABC). The minterm expansion is

(4.10)

It is not difficult to use the product rule and the replacement theorem to calculate the needed minterm probabilities. Thus . Similarly . The desired probability is the sum of these, 0.6880.

As an alternate approach, we write

(4.11)

Considerbly fewer arithmetic operations are required in this calculation.

In larger problems, or in situations where probabilities of several Boolean combinations are to be determined, it may be desirable to calculate all minterm probabilities then use the minterm vector techniques introduced earlier to calculate probabilities for various Boolean combinations. As a larger example for which computational aid is highly desirable, consider again the class and the probabilities utilized in Example 4.6, above.

Example 4.13.

Consider again the independent class with respective probabilities . We wish to calculate

(4.12)

There are 210=1024 minterm probabilities to be calculated. Each requires the multiplication of ten numbers. The solution with MATLAB is easy, as follows:

>> P = 0.01*[13 37 12 56 33 71 22 43 57 31];
>> minvec10
Vectors are A1 thru A10 and A1c thru A10c
They may be renamed, if desired.
>> F = (A1|(A3&(A4|A7c)))|(A2&(A5c|(A6&A8)))|(A9&A10c);
>> pm = minprob(P);
>> PF = F*pm'
PF =  0.6636

Writing out the expression for F is tedious and error prone. We could simplify as follows:

>> A = A1|(A3&(A4|A7c));
>> B = A2&(A5c|(A6&A8));
>> C = A9&A10c;
>> F = A|B|C;               % This minterm vector is the same as for F above

This decomposition of the problem indicates that it may be solved as a series of smaller problems. First, we need some central facts about independence of Boolean combinations.

#### Independent Boolean combinations

Suppose we have a Boolean combination of the events in the class and a second combination the events in the class . If the combined class is independent, we would expect the combinations of the subclasses to be independent. It is important to see that this is in fact a consequence of the product rule, for it is further evidence that the product rule has captured the essence of the intuitive notion of independence. In the following discussion, we exhibit the essential structure which provides the basis for the following general proposition.

Proposition. Consider n distinct subclasses of an independent class of events. If for each i the event Ai is a Boolean (logical) combination of members of the ith subclass, then the class is an independent class.

Verification of this far reaching result rests on the minterm expansion and two elementary facts about the disjoint subclasses of an independent class. We state these facts and consider in each case an example which exhibits the essential structure. Formulation of the general result, in each case, is simply a matter of careful use of notation.

1. A class each of whose members is a minterm formed by members of a distinct subclass of an independent class is itself an independent class.
Example 4.14.

Consider the independent class , with respective probabilities 0.4, 0.7, 0.3, 0.5, 0.8, 0.3, 0.6. Consider M3, minterm three for the class , and N5, minterm five for the class . Then

(4.13)

Also

(4.14)

The product rule shows the desired independence.

Again, it should be apparent that the result holds for any number of Ai and Bj; and it can be extended to any number of distinct subclasses of an independent class.

2. Suppose each member of a class can be expressed as a disjoint union. If each auxiliary class formed by taking one member from each of the disjoint unions is an independent class, then the original class is independent.
Example 4.15.

Suppose A=A1A2A3 and B=B1B2, with independent for each pair i,j. Suppose

(4.15)

We wish to show that the pair is independent; i.e., the product rule P(AB)=P(A)P(B) holds.

COMPUTATION

(4.16)

Now

(4.17)

By additivity and pairwise independence, we have

(4.18)

The product rule can also be established algebraically from the expression for P(AB), as follows:

(4.19)

It should be clear that the pattern just illustrated can be extended to the general case. If

(4.20)

then the pair is independent. Also, we may extend this rule to the triple

(4.21)

and similarly for any finite number of such combinations, so that the second proposition holds.

3. Begin with an independent class E of n events. Select m distinct subclasses and form Boolean combinations for each of these. Use of the minterm expansion for each of these Boolean combinations and the two propositions just illustrated shows that the class of Boolean combinations is independent

To illustrate, we return to Example 4.13, which involves an independent class of ten events.

Example 4.16. A hybrid approach

Consider again the independent class with respective probabilities . We wish to calculate

(4.22)

In the previous solution, we use minprob to calculate the 210=1024 minterms for all ten of the Ei and determine the minterm vector for F. As we note in the alternate expansion of F,

(4.23)

We may calculate directly P(C)=0.57⋅0.69=0.3933. Now A is a Boolean combination of and B is a combination of . By the result on independence of Boolean combinations, the class is independent. We use the m-procedures to calculate P(A) and P(B). Then we deal with the independent class to obtain the probability of F.

>> p  = 0.01*[13 37 12 56 33 71 22 43 57 31];
>> pa = p([1 3 4 7]);     % Selection of probabilities for A
>> pb = p([2 5 6 8]);     % Selection of probabilities for B
>> pma = minprob(pa);     % Minterm probabilities for calculating P(A)
>> pmb = minprob(pb);     % Minterm probabilities for calculating P(B)
>> minvec4;
>> a = A|(B&(C|Dc));      % A corresponds to E1, B to E3, C to E4, D to E7
>> PA = a*pma'
PA = 0.2243
>> b = A&(Bc|(C&D));      % A corresponds to E2, B to E5, C to E6, D to E8
>> PB = b*pmb'
PB = 0.2852
>> PC = p(9)*(1 - p(10))
PC = 0.3933
>> pm = minprob([PA PB PC]);
>> minvec3                % The problem becomes a three variable problem
>> F = A|B|C;             % with {A,B,C} an independent class
>> PF = F*pm'
PF = 0.6636               % Agrees with the result of Example 4.11

### 4.3. Composite Trials*

#### Composite trials and component events

Often a trial is a composite one. That is, the fundamental trial is completed by performing several steps. In some cases, the steps are carried out sequentially in time. In other situations, the order of performance plays no significant role. Some of the examples in the unit on Conditional Probability involve such multistep trials. We examine more systematically how to model composite trials in terms of events determined by the components of the trials. In the subsequent section, we illustrate this approach in the important special case of Bernoulli trials, in which each outcome results in a success or failure to achieve a specified condition.

We call the individual steps in the composite trial component trials. For example, in the experiment of flipping a coin ten times, we refer the ith toss as the ith component trial. In many cases, the component trials will be performed sequentially in time. But we may have an experiment in which ten coins are flipped simultaneously. For purposes of analysis, we impose an ordering— usually by assigning indices. The question is how to model these repetitions. Should they be considered as ten trials of a single simple experiment? It turns out that this is not a useful formulation. We need to consider the composite trial as a single outcome— i.e., represented by a single point in the basic space Ω.

Some authors give considerable attention the the nature of the basic space, describing it as a Cartesian product space, with each coordinate corresponding to one of the component outcomes. We find that unnecessary, and often confusing, in setting up the basic model. We simply suppose the basic space has enough elements to consider each possible outcome. For the experiment of flipping a coin ten times, there must be at least 210=1024 elements, one for each possible sequence of heads and tails.

Of more importance is describing the various events associated with the experiment. We begin by identifying the appropriate component events. A component event is determined by propositions about the outcomes of the corresponding component trial.

Example 4.17. Component events
• In the coin flipping experiment, consider the event H3 that the third toss results in a head. Each outcome ω of the experiment may be represented by a sequence of H's and T's, representing heads and tails. The event H3 consists of those outcomes represented by sequences with H in the third position. Suppose A is the event of a head on the third toss and a tail on the ninth toss. This consists of those outcomes corresponding to sequences with H in the third position and T in the ninth. Note that this event is the intersection H3H9c.

• A somewhat more complex example is as follows. Suppose there are two boxes, each containing some red and some blue balls. The experiment consists of selecting at random a ball from the first box, placing it in the second box, then making a random selection from the modified contents of the second box. The composite trial is made up of two component selections. We may let R1 be the event of selecting a red ball on the first component trial (from the first box), and R2 be the event of selecting a red ball on the second component trial. Clearly R1 and R2 are component events.

In the first example, it is reasonable to assume that the class is independent, and each component probability is usually taken to be 0.5. In the second case, the assignment of probabilities is somewhat more involved. For one thing, it is necessary to know the numbers of red and blue balls in each box before the composite trial begins. When these are known, the usual assumptions and the properties of conditional probability suffice to assign probabilities. This approach of utilizing component events is used tacitly in some of the examples in the unit on Conditional Probability.

When appropriate component events are determined, various Boolean combinations of these can be expressed as minterm expansions.

Example 4.18.

Four persons take one shot each at a target. Let Ei be the event the ith shooter hits the target center. Let A3 be the event exacty three hit the target. Then A3 is the union of those minterms generated by the Ei which have three places uncomplemented.

(4.24) A3 = E1 E2 E3 E4cE1 E2 E3c E4E1 E2c E3 E4E1c E2 E3 E4

Usually we would be able to assume the Ei form an independent class. If each is known, then all minterm probabilities can be calculated easily.

The following is a somewhat more complicated example of this type.

Example 4.19.

Ten race cars are involved in time trials to determine pole positions for an upcoming race. To qualify, they must post an average speed of 125 mph or more on a trial run. Let Ei be the event the ith car makes qualifying speed. It seems reasonable to suppose the class is independent. If the respective probabilities for success are 0.90, 0.88, 0.93, 0.77, 0.85, 0.96, 0.72, 0.83, 0.91, 0.84, what is the probability that k or more will qualify ()?

SOLUTION

Let Ak be the event exactly k qualify. The class generates 210=1024 minterms. The event Ak is the union of those minterms which have exactly k places uncomplemented. The event Bk that k or more qualify is given by

(4.25)

The task of computing and adding the minterm probabilities by hand would be tedious, to say the least. However, we may use the function ckn, introduced in the unit on MATLAB and Independent Classes and illustrated in Example 4.4.2, to determine the desired probabilities quickly and easily.

>> P = [0.90, 0.88, 0.93, 0.77, 0.85, 0.96,0.72, 0.83, 0.91, 0.84];
>> k = 6:10;
>> PB = ckn(P,k)
PB =   0.9938    0.9628    0.8472    0.5756    0.2114

An alternate approach is considered in the treatment of random variables.

#### Bernoulli trials and the binomial distribution

Many composite trials may be described as a sequence of success-failure trials. For each component trial in the sequence, the outcome is one of two kinds. One we designate a success and the other a failure. Examples abound: heads or tails in a sequence of coin flips, favor or disapprove of a proposition in a survey sample, and items from a production line meet or fail to meet specifications in a sequence of quality control checks. To represent the situation, we let Ei be the event of a success on the ith component trial in the sequence. The event of a failure on the ith component trial is thus Eic.

In many cases, we model the sequence as a Bernoulli sequence, in which the results on the successive component trials are independent and have the same probabilities. Thus, formally, a sequence of success-failure trials is Bernoulli iff

1. The class is independent.

2. The probability , invariant with i.

Simulation of Bernoulli trials

It is frequently desirable to simulate Bernoulli trials. By flipping coins, rolling a die with various numbers of sides (as used in certain games), or using spinners, it is relatively easy to carry this out physically. However, if the number of trials is large—say several hundred—the process may be time consuming. Also, there are limitations on the values of p, the probability of success. We have a convenient two-part m-procedure for simulating Bernoulli sequences. The first part, called btdata, sets the parameters. The second, called bt, uses the random number generator in MATLAB to produce a sequence of zeros and ones (for failures and successes). Repeated calls for bt produce new sequences.

Example 4.20.
>> btdata
Enter n, the number of trials  10
Enter p, the probability of success on each trial  0.37
Call for bt
>> bt
n = 10   p = 0.37     % n is kept small to save printout space
Frequency = 0.4
To view the sequence, call for SEQ
>> disp(SEQ)          % optional call for the sequence
1     1
2     1
3     0
4     0
5     0
6     0
7     0
8     0
9     1
10     1

Repeated calls for bt yield new sequences with the same parameters.

To illustrate the power of the program, it was used to take a run of 100,000 component trials, with probability p of success 0.37, as above. Successive runs gave relative frequencies 0.37001 and 0.36999. Unless the random number generator is “seeded” to make the same starting point each time, successive runs will give different sequences and usually different relative frequencies.

The binomial distribution

A basic problem in Bernoulli sequences is to determine the probability of k successes in n component trials. We let Sn be the number of successes in n trials. This is a special case of a simple random variable, which we study in more detail in the chapter on "Random Variables and Probabilities".

Let us characterize the events . As noted above, the event Akn of exactly k successes is the union of the minterms generated by in which there are k successes (represented by k uncomplemented Ei) and nk failures (represented by nk complemented Eic). Simple combinatorics show there are C(n,k) ways to choose the k places to be uncomplemented. Hence, among the 2n minterms, there are which have k places uncomplemented. Each such minterm has probability pk(1–p)nk. Since the minterms are mutually exclusive, their probabilities add. We conclude that

(4.26)

These probabilities and the corresponding values form the distribution for Sn. This distribution is known as the binomial distribution, with parameters . We shorten this to binomial , and often write Sn binomial . A related set of probabilities is , 0≤kn. If the number n of component trials is small, direct computation of the probabilities is easy with hand calculators.

Example 4.21. A reliability problem

A remote device has five similar components which fail independently, with equal probabilities. The system remains operable if three or more of the components are operative. Suppose each unit remains active for one year with probability 0.8. What is the probability the system will remain operative for that long?

SOLUTION

(4.27)

Because Bernoulli sequences are used in so many practical situations as models for success-failure trials, the probabilities and have been calculated and tabulated for a variety of combinations of the parameters . Such tables are found in most mathematical handbooks. Tables of are usually given a title such as binomial distribution, individual terms. Tables of have a designation such as binomial distribution, cumulative terms. Note, however, some tables for cumulative terms give . Care should be taken to note which convention is used.

Example 4.22. A reliability problem

Consider again the system of Example 5, above. Suppose we attempt to enter a table of Cumulative Terms, Binomial Distribution at n=5, k=3, and p=0.8. Most tables will not have probabilities greater than 0.5. In this case, we may work with failures. We just interchange the role of Ei and Eic. Thus, the number of failures has the binomial distribution. Now there are three or more successes iff there are not three or more failures. We go the the table of cumulative terms at n=5, k=3, and p=0.2. The probability entry is 0.0579. The desired probability is 1 - 0.0579 = 0.9421.

In general, there are k or more successes in n trials iff there are not nk+1 or more failures.

m-functions for binomial probabilities

Although tables are convenient for calculation, they impose serious limitations on the available parameter values, and when the values are found in a table, they must still be entered into the problem. Fortunately, we have convenient m-functions for these distributions. When MATLAB is available, it is much easier to generate the needed probabilities than to look them up in a table, and the numbers are entered directly into the MATLAB workspace. And we have great freedom in selection of parameter values. For example we may use n of a thousand or more, while tables are usually limited to n of 20, or at most 30. The two m-functions for calculating and are

 : is calculated by y = ibinom(n,p,k), where k is a row or column vector of integers between 0 and n. The result y is a row vector of the same size as k. : is calculated by y = cbinom(n,p,k), where k is a row or column vector of integers between 0 and n. The result y is a row vector of the same size as k.
Example 4.23. Use of m-functions ibinom and cbinom

If n=10 and p=0.39, determine and for .

>> p = 0.39;
>> k = [3 5 6 8];
>> Pi = ibinom(10,p,k)  % individual probabilities
Pi = 0.2237    0.1920    0.1023    0.0090
>> Pc = cbinom(10,p,k)  % cumulative probabilities
Pc = 0.8160    0.3420    0.1500    0.0103

Note that we have used probability p=0.39. It is quite unlikely that a table will have this probability. Although we use only n=10, frequently it is desirable to use values of several hundred. The m-functions work well for n up to 1000 (and even higher for small values of p or for values very near to one). Hence, there is great freedom from the limitations of tables. If a table with a specific range of values is desired, an m-procedure called binomial produces such a table. The use of large n raises the question of cumulation of errors in sums or products. The level of precision in MATLAB calculations is sufficient that such roundoff errors are well below pratical concerns.

Example 4.24.
>> binomial                              % call for procedure
Enter n, the number of trials  13
Enter p, the probability of success  0.413
Enter row vector k of success numbers  0:4
n            p
13.0000    0.4130
k      P(X=k)    P(X>=k)
0    0.0010    1.0000
1.0000    0.0090    0.9990
2.0000    0.0379    0.9900
3.0000    0.0979    0.9521
4.0000    0.1721    0.8542

Remark. While the m-procedure binomial is useful for constructing a table, it is usually not as convenient for problems as the m-functions ibinom or cbinom. The latter calculate the desired values and put them directly into the MATLAB workspace.

#### Joint Bernoulli trials

Bernoulli trials may be used to model a variety of practical problems. One such is to compare the results of two sequences of Bernoulli trials carried out independently. The following simple example illustrates the use of MATLAB for this.

Example 4.25. A joint Bernoulli trial

Bill and Mary take ten basketball free throws each. We assume the two seqences of trials are independent of each other, and each is a Bernoulli sequence.

Mary: Has probability 0.80 of success on each trial.

Bill: Has probability 0.85 of success on each trial.

What is the probability Mary makes more free throws than Bill?

SOLUTION

We have two Bernoulli sequences, operating independently.

Mary:

Bill:

Let

M be the event Mary wins

Mk be the event Mary makes k or more freethrows.

Bj be the event Bill makes exactly j freethrows

Then Mary wins if Bill makes none and Mary makes one or more, or Bill makes one and Mary makes two or more, etc. Thus

(4.28)

and

(4.29)

We use cbinom to calculate the cumulative probabilities for Mary and ibinom to obtain the individual probabilities for Bill.

>> pm = cbinom(10,0.8,1:10);     % cumulative probabilities for Mary
>> pb = ibinom(10,0.85,0:9);     % individual probabilities for Bill
>> D = [pm; pb]'                 % display: pm in the first column
D =                           % pb in the second column
1.0000    0.0000
1.0000    0.0000
0.9999    0.0000
0.9991    0.0001
0.9936    0.0012
0.9672    0.0085
0.8791    0.0401
0.6778    0.1298
0.3758    0.2759
0.1074    0.3474

To find the probability P(M) that Mary wins, we need to multiply each of these pairs together, then sum. This is just the dot or scalar product, which MATLAB calculates with the command pm*pb'. We may combine the generation of the probabilities and the multiplication in one command:

>> P = cbinom(10,0.8,1:10)*ibinom(10,0.85,0:9)'
P = 0.2738

The ease and simplicity of calculation with MATLAB make it feasible to consider the effect of different values of n. Is there an optimum number of throws for Mary? Why should there be an optimum?

An alternate treatment of this problem in the unit on Independent Random Variables utilizes techniques for independent simple random variables.

#### Alternate MATLAB implementations

Alternate implementations of the functions for probability calculations are found in the Statistical Package available as a supplementary package. We have utilized our formulation, so that only the basic MATLAB package is needed.

### 4.4. Problems on Independence of Events*

The minterms generated by the class have minterm probabilities

(4.30)

Show that the product rule holds for all three, but the class is not independent.

pm = [0.15 0.05 0.02 0.18 0.25 0.05 0.18 0.12];
y = imintest(pm)
The class is NOT independent
Minterms for which the product rule fails
y =
1     1     1     0
1     1     1     0   % The product rule hold for M7 = ABC

The class is independent, with respective probabilities 0.65, 0.37, 0.48, 0.63. Use the m-function minprob to obtain the minterm probabilities. Use the m-function minmap to put them in a 4 by 4 table corresponding to the minterm map convention we use.

P = [0.65 0.37 0.48 0.63];
p = minmap(minprob(P))
p =
0.0424    0.0249    0.0788    0.0463
0.0722    0.0424    0.1342    0.0788
0.0392    0.0230    0.0727    0.0427
0.0667    0.0392    0.1238    0.0727

The minterm probabilities for the software survey in Example 2 from "Minterms" are

(4.31)

Show whether or not the class {A,B,C} is independent: (1) by hand calculation, and (2) by use of the m-function imintest.

pm = [0 0.05 0.10 0.05 0.20 0.10 0.40 0.10];
y = imintest(pm)
The class is NOT independent
Minterms for which the product rule fails
y =
1     1     1     1    % By hand check product rule for any minterm
1     1     1     1

The minterm probabilities for the computer survey in Example 3 from "Minterms" are

(4.32)

Show whether or not the class {A,B,C} is independent: (1) by hand calculation, and (2) by use of the m-function imintest.

npr04_04
Minterm probabilities for Exercise 4. are in pm
y = imintest(pm)
The class is NOT independent
Minterms for which the product rule fails
y =
1     1     1     1
1     1     1     1

Minterm probabilities p(0) through p(15) for the class are, in order,

Use the m-function imintest to show whether or not the class is independent.

npr04_05
Minterm probabilities for Exercise 5. are in pm
imintest(pm)
The class is NOT independent
Minterms for which the product rule fails
ans =
0     1     0     1
0     0     0     0
0     1     0     1
0     0     0     0

Minterm probabilities p(0) through p(15) for the opinion survey in Example 4 from "Minterms" are

Show whether or not the class is independent.

npr04_06
Minterm probabilities for Exercise 6. are in pm
y = imintest(pm)
The class is NOT independent
Minterms for which the product rule fails
y =
1     1     1     1
1     1     1     1
1     1     1     1
1     1     1     1

The class {A,B,C} is independent, with P(A)=0.30, , and P(AC)=0.12. Determine the minterm probabilities.

P(C)=P(AC)/P(A)=0.40 and .

pm = minprob([0.3 0.2 0.4])
pm =  0.3360  0.2240  0.0840  0.0560  0.1440  0.0960  0.0360  0.0240

The class is independent, with P(AB)=0.6, P(AC)=0.7, and P(C)=0.4. Determine the probability of each minterm.

implies .

implies implies P(B)=0.2

P = [0.5 0.2 0.4];
pm = minprob(P)
pm =  0.2400  0.1600  0.0600  0.0400  0.2400  0.1600  0.0600  0.0400

A pair of dice is rolled five times. What is the probability the first two results are “sevens” and the others are not?

P=(1/6)2(5/6)3=0.0161.

David, Mary, Joan, Hal, Sharon, and Wayne take an exam in their probability course. Their probabilities of making 90 percent or more are

()

respectively. Assume these are independent events. What is the probability three or more, four or more, five or more make grades of at least 90 percent?

P = 0.01*[72 83 75 92 65 79];
y = ckn(P,[3 4 5])
y =   0.9780    0.8756    0.5967

Two independent random numbers between 0 and 1 are selected (say by a random number generator on a calculator). What is the probability the first is no greater than 0.33 and the other is at least 57?

P=0.33•(1–0.57)=0.1419

Helen is wondering how to plan for the weekend. She will get a letter from home (with money) with probability 0.05. There is a probability of 0.85 that she will get a call from Jim at SMU in Dallas. There is also a probability of 0.5 that William will ask for a date. What is the probability she will get money and Jim will not call or that both Jim will call and William will ask for a date?

A letter with money, B call from Jim, C William ask for date

P = 0.01*[5 85 50];
minvec3
Variables are A, B, C, Ac, Bc, Cc
They may be renamed, if desired.
pm = minprob(P);
p = ((A&Bc)|(B&C))*pm'
p =  0.4325

A basketball player takes ten free throws in a contest. On her first shot she is nervous and has probability 0.3 of making the shot. She begins to settle down and probabilities on the next seven shots are 0.5, 0.6 0.7 0.8 0.8, 0.8 and 0.85, respectively. Then she realizes her opponent is doing well, and becomes tense as she takes the last two shots, with probabilities reduced to 0.75, 0.65. Assuming independence between the shots, what is the probability she will make k or more for k=2,3,⋯10?

P = 0.01*[30 50 60 70 80 80 80 85 75 65];
k = 2:10;
p = ckn(P,k)
p =
Columns 1 through 7
0.9999    0.9984    0.9882    0.9441    0.8192    0.5859    0.3043
Columns 8 through 9
0.0966    0.0134

In a group there are M men and W women; m of the men and w of the women are college graduates. An individual is picked at random. Let A be the event the individual is a woman and B be the event he or she is a college graduate. Under what condition is the pair independent?

P(A|B)=w/(m+w)=W/(W+M)=P(A)

Consider the pair of events. Let P(A)=p, , P(B|A)=p1, and . Under what condition is the pair independent?

(see table of equivalent conditions).

Show that if event A is independent of itself, then P(A)=0 or P(A)=1. (This fact is key to an important “zero-one law.”)

P(A)=P(AA)=P(A)P(A). x2=x iff x=0 or x=1.

% No. Consider for example the following minterm probabilities:
pm = [0.2 0.05 0.125 0.125 0.05 0.2 0.125 0.125];
minvec3
Variables are A, B, C, Ac, Bc, Cc
They may be renamed, if desired.
PA = A*pm'
PA =  0.5000
PB = B*pm'
PB =  0.5000
PC = C*pm'
PC =  0.5000
PAB = (A&B)*pm'  % Product rule holds
PAB =  0.2500
PBC = (B&C)*pm' % Product rule holds
PBC =  0.2500
PAC = (A&C)*pm'  % Product rule fails
PAC =  0.3250

Suppose event A implies B (i.e. AB). Show that if the pair is independent, then either P(A)=0 or P(B)=1.

AB implies P(AB)=P(A); independence implies P(AB)=P(A)P(B). P(A)=P(A)P(B) only if P(B)=1 or P(A)=0.

A company has three task forces trying to meet a deadline for a new device. The groups work independently, with respective probabilities 0.8, 0.9, 0.75 of completing on time. What is the probability at least one group completes on time? (Think. Then solve “by hand.”)

At least one completes iff not all fail. P=1–0.2•0.1•0.25=0.9950

Two salesmen work differently. Roland spends more time with his customers than does Betty, hence tends to see fewer customers. On a given day Roland sees five customers and Betty sees six. The customers make decisions independently. If the probabilities for success on Roland's customers are 0.7,0.8,0.8,0.6,0.7 and for Betty's customers are 0.6,0.5,0.4,0.6,0.6,0.4, what is the probability Roland makes more sales than Betty? What is the probability that Roland will make three or more sales? What is the probability that Betty will make three or more sales?

PR = 0.1*[7 8 8 6 7];
PB = 0.1*[6 5 4 6 6 4];
PR3 = ckn(PR,3)
PR3 =  0.8662
PB3 = ckn(PB,3)
PB3 =  0.6906
PRgB = ikn(PB,0:4)*ckn(PR,1:5)'
PRgB = 0.5065

Two teams of students take a probability exam. The entire group performs individually and independently. Team 1 has five members and Team 2 has six members. They have the following indivudal probabilities of making an `”A” on the exam.

Team 1: 0.83 0.87 0.92 0.77 0.86 Team 2: 0.68 0.91 0.74 0.68 0.73 0.83

1. What is the probability team 1 will make at least as many A's as team 2?

2. What is the probability team 1 will make more A's than team 2?

P1 = 0.01*[83 87 92 77 86];
P2 = 0.01*[68 91 74 68 73 83];
P1geq = ikn(P2,0:5)*ckn(P1,0:5)'
P1geq =  0.5527
P1g = ikn(P2,0:4)*ckn(P1,1:5)'
P1g =    0.2561

A system has five components which fail independently. Their respective reliabilities are 0.93, 0.91, 0.78, 0.88, 0.92. Units 1 and 2 operate as a “series” combination. Units 3, 4, 5 operate as a two of three subsytem. The two subsystems operate as a parallel combination to make the complete system. What is reliability of the complete system?

R = 0.01*[93 91 78 88 92];
Ra = prod(R(1:2))
Ra =  0.8463
Rb = ckn(R(3:5),2)
Rb =  0.9506
Rs = parallel([Ra Rb])
Rs =  0.9924

A system has eight components with respective probabilities

(4.33)

Units 1 and 2 form a parallel subsytem in series with unit 3 and a three of five combination of units 4 through 8. What is the reliability of the complete system?

R = 0.01*[96 90 93 82 85 97 88 80];
Ra = parallel(R(1:2))
Ra =  0.9960
Rb = ckn(R(4:8),3)
Rb =  0.9821
Rs = prod([Ra R(3) Rb])
Rs =  0.9097

How would the reliability of the system in Exercise 23. change if units 1, 2, and 3 formed a parallel combination in series with the three of five combination?

Rc = parallel(R(1:3))
Rc =  0.9997

How would the reliability of the system in Exercise 23. change if the reliability of unit 3 were changed from 0.93 to 0.96? What change if the reliability of unit 2 were changed from 0.90 to 0.95 (with unit 3 unchanged)?

R1 = R;
R1(3) =0.96;
Ra = parallel(R1(1:2))
Ra =  0.9960
Rb = ckn(R1(4:8),3)
Rb =  0.9821
Rs3 = prod([Ra R1(3) Rb])
Rs3 = 0.9390
R2 = R;
R2(2) = 0.95;
Ra = parallel(R2(1:2))
Ra =  0.9980
Rb = ckn(R2(4:8),3)
Rb =  0.9821
Rs4 = prod([Ra R2(3) Rb])
Rs4 = 0.9115

Three fair dice are rolled. What is the probability at least one will show a six?

P=1–(5/6)3=0.4213

A hobby shop finds that 35 percent of its customers buy an electronic game. If customers buy independently, what is the probability that at least one of the next five customers will buy an electronic game?

P=1–0.655=0.8840

Under extreme noise conditions, the probability that a certain message will be transmitted correctly is 0.1. Successive messages are acted upon independently by the noise. Suppose the message is transmitted ten times. What is the probability it is transmitted correctly at least once?

P=1–0.910=0.6513

Suppose the class is independent, with , 1≤in. What is the probability that at least one of the events occurs? What is the probability that none occurs?

In one hundred random digits, 0 through 9, with each possible digit equally likely on each choice, what is the probility 8 or more are sevens?

P=cbinom(100,0.1,8)=0.7939

Ten customers come into a store. If the probability is 0.15 that each customer will buy a television set, what is the probability the store will sell three or more?

P=cbinom(10,0.15,3)=0.1798

Seven similar units are put into service at time t=0. The units fail independently. The probability of failure of any unit in the first 400 hours is 0.18. What is the probability that three or more units are still in operation at the end of 400 hours?

P=cbinom(7,0.82,3)=0.9971

A computer system has ten similar modules. The circuit has redundancy which ensures the system operates if any eight or more of the units are operative. Units fail independently, and the probability is 0.93 that any unit will survive between maintenance periods. What is the probability of no system failure due to these units?

P=cbinom(10,0.93,8)=0.9717

Only thirty percent of the items from a production line meet stringent requirements for a special job. Units from the line are tested in succession. Under the usual assumptions for Bernoulli trials, what is the probability that three satisfactory units will be found in eight or fewer trials?

P=cbinom(8,0.3,3)=0.4482

The probability is 0.02 that a virus will survive application of a certain vaccine. What is the probability that in a batch of 500 viruses, fifteen or more will survive treatment?

P=cbinom(500.0.02,15)=0.0814

In a shipment of 20,000 items, 400 are defective. These are scattered randomly throughout the entire lot. Assume the probability of a defective is the same on each choice. What is the probability that

1. Two or more will appear in a random sample of 35?

2. At most five will appear in a random sample of 50?

• P1=cbinom(35,0.02,2)=0.1547.

• P2=1–cbinom(35,0.02,6)=0.9999

A device has probability p of operating successfully on any trial in a sequence. What probability p is necessary to ensure the probability of successes on all of the first four trials is 0.85? With that value of p, what is the probability of four or more successes in five trials?

A survey form is sent to 100 persons. If they decide independently whether or not to reply, and each has probability 1/4 of replying, what is the probability of k or more replies, where k=15,20,25,30,35,40?

P = cbinom(100,1/4,15:5:40)
P =  0.9946    0.9005    0.5383    0.1495    0.0164    0.0007

Ten numbers are produced by a random number generator. What is the probability four or more are less than or equal to 0.63?

P1=cbinom(10,0.63,4)=0.9644

A player rolls a pair of dice five times. She scores a “hit” on any throw if she gets a 6 or 7. She wins iff she scores an odd number of hits in the five throws. What is the probability a player wins on any sequence of five throws? Suppose she plays the game 20 successive times. What is the probability she wins at least 10 times? What is the probability she wins more than half the time?

Each roll yields a hit with probability .

PW = sum(ibinom(5,11/36,[1 3 5]))
PW =  0.4956
P2 = cbinom(20,PW,10)
P2 =  0.5724
P3 = cbinom(20,PW,11)
P3 =  0.3963

Erica and John spin a wheel which turns up the integers 0 through 9 with equal probability. Results on various trials are independent. Each spins the wheel 10 times. What is the probability Erica turns up a seven more times than does John?

P=ibinom(10,0.1,0:9)*cbinom(10,0.1,1:10)'=0.3437

Erica and John play a different game with the wheel, above. Erica scores a point each time she gets an integer 0, 2, 4, 6, or 8. John scores a point each time he turns up a 1, 2, 5, or 7. If Erica spins eight times; John spins 10 times. What is the probability John makes more points than Erica?

P=ibinom(8,0.5,0:8)*cbinom(10,0.4,1:9)'=0.4030

A box contains 100 balls; 30 are red, 40 are blue, and 30 are green. Martha and Alex select at random, with replacement and mixing after each selection. Alex has a success if he selects a red ball; Martha has a success if she selects a blue ball. Alex selects seven times and Martha selects five times. What is the probability Martha has more successes than Alex?

P=ibinom(7,0.3,0:4)*cbinom(5,0.4,1:5)'=0.3613

Two players roll a fair die 30 times each. What is the probability that each rolls the same number of sixes?

P=sum(ibinom(30,1/6,0:30).^2)=0.1386

A warehouse has a stock of n items of a certain kind, r of which are defective. Two of the items are chosen at random, without replacement. What is the probability that at least one is defective? Show that for large n the number is very close to that for selection with replacement, which corresponds to two Bernoulli trials with pobability p=r/n of success on any trial.

(4.34)
(4.35)

A coin is flipped repeatedly, until a head appears. Show that with probability one the game will terminate.

#### Tip

The probability of not terminating in n trials is qn.

Let N= event never terminates and Nk= event does not terminate in k plays. Then NNk for all k implies for all k. We conclude P(N)=0.

Two persons play a game consecutively until one of them is successful or there are ten unsuccesful plays. Let Ei be the event of a success on the ith play of the game. Suppose is an independent class with for i odd and for i even. Let A be the event the first player wins, B be the event the second player wins, and C be the event that neither wins.

1. Express , and C in terms of the Ei.

2. Determine P(A), P(B), and P(C) in terms of p1, p2, q1=1–p1, and q2=1–p2. Obtain numerical values for the case p1=1/4 and p2=1/3.

3. Use appropriate facts about the geometric series to show that P(A)=P(B) iff .

4. Suppose p2=0.5. Use the result of part (c) to find the value of p1 to make P(A)=P(B) and then determine P(A), P(B), and P(C).

1. .

(4.36)A=E1E1cE2cE3E1cE2cE3cE4cE5E1cE2cE3cE4cE5cE6cE7E1cE2cE3cE4cE5cE6cE7cE8cE9
(4.37)B=E1cE2E1cE2cE3cE4E1cE2cE3cE4cE5cE6E1cE2cE3cE4cE5cE6cE7cE8E1cE2cE3cE4cE5cE6cE7cE8cE9cE10
2. (4.38)
(4.39)

For p1=1/4,p2=1/3, we have q1q2=1/2 and q1p2=1/4. In this case

(4.40)

Note that P(A)+P(B)+P(C)=1.

3. P(A)=P(B) iff iff .

4. p1 = 0 . 5 / 1 . 5 = 1 / 3

Three persons play a game consecutively until one achieves his objective. Let Ei be the event of a success on the ith trial, and suppose is an independent class, with for i=1,4,7,⋯, for i=2,5,8,⋯, and for i=3,6,9,⋯. Let A,B,C be the respective events the first, second, and third player wins.

1. Express A,B, and C in terms of the Ei.

2. Determine the probabilities in terms of p1,p2,p3, then obtain numerical values in the case p1=1/4, p2=1/3, and p3=1/2.

• For p1=1/4, p2=1/3, p3=1/2, P(A)=P(B)=P(C)=1/3.

What is the probability of a success on the ith trial in a Bernoulli sequence of n component trials, given there are r successes?

and .

Hence .

A device has N similar components which may fail independently, with probability p of failure of any component. The device fails if one or more of the components fails. In the event of failure of the device, the components are tested sequentially.

1. What is the probability the first defective unit tested is the nth, given one or more components have failed?

2. What is the probability the defective unit is the nth, given that exactly one has failed?

3. What is the probability that more than one unit has failed, given that the first defective unit is the nth?

Let A1= event one failure, B1= event of one or more failures, B2= event of two or more failures, and Fn= the event the first defective unit found is the nth.
1. FnB1 implies

2. ()

(see Exercise 49.)

3. Since probability not all from nth are good is 1–qNn,

()
Solutions