Probability Primer

Counting Principles

Basic Counting Law/First Multiplication Principle: If a procedure can be done in k distinct steps and there are (Subscript[n, 1]) ways the first step can be performed, (Subscript[n, 2]) ways the second step can be performed, ..., and (Subscript[n, k]) distinct ways the last step can be performed, then the number of ways the entire process can be done is n=(Subscript[n, 1])×(Subscript[n, 2])×(Subscript[…n, k]).

Permutations:  Suppose you have a set of n objects from which you must pick k, and this choice is “without replacement” (i.e. no object can be picked more than once) and the order of selection makes a difference.  Then the number of ways to pick the k objects from the n is P(n,k)=(Subscript[, n])(Subscript[P, k])=((n!)/((n-k)!)).

Combinations: Suppose you have a set of n objects from which you must pick k, and this choice is “without replacement” (i.e. no object can be picked more than once) and the order of selection does not make a difference.  Then the number of ways to pick the k objects out of the n is C(n,k)=(Subscript[, n])(Subscript[C, k])=(({{n}, {k}}))=((n!)/((n-k)!k!)).  The number (({{n}, {k}})) (often called a binomial number) appears in the “row n column k” spot in Pascal’s triangle (treating the top as “row 0” and the leftmost column as “column 0”.

Basic Properties of Probability

If E is an event, the probability of E is a number P(E) which measures the likelihood the E will occur.  0≤P(E)≤1, with larger values corresponding to more likely events (in cases where the number of possible outcomes is finite, P(E)=0 means impossible and P(E)=1 means certain; however this is not always the case if there are an infinite number of outcomes).

If X is the set of all possible outcomes and A and B are events (sets of outcomes) from X, then:

P(X)=1

P( ∅)=0

If A⋂B=∅, P(A⋃B)=P(A)+P(B). As a consequence the probability of an event A is equal to the sums of the probabilities of the individual outcomes that make it up.

If (A^C) is the complementary event for A (i.e. the event that the outcome is not in A), then P((A^C))=1-P(A).  This is often useful of the event (A^C) is simpler than/contains fewer outcomes than A.

Events A and B are independent if and only if P(A⋂B)=P(A)×P(B)

The probability that A occurred given that B has occurred (a “conditional” probability) is defined by P(A|B)=((P(A⋂B))/(P(B))).

Bayes' theorem relates the conditional probabilities P(A|B)and P(B|A) via the formula P(A|B)=((P(B|A)×P(A))/(P(B))).

Discrete Probability Distributions

A discrete probability distribution is one in which the outcomes are real numbers and there is a minimum spacing between the outcomes (most commonly the outcomes are subsets of the integers).  The function which assigns to each outcome its probability is the “probability mass function”/p.m.f (sometimes referred to as the “probability density function”/p.d.f.).  The probability mass function essentially defines the experiment/random process in a given problem.  For small problems a probability mass function can be given as a simple list of the outcomes and their probabilities.  Note that a discrete probability distribution does not have have a finite number of outcomes (although it often does) - if there are an infinite number of outcomes then the various sums below have to be done by infinite series.

If the outcomes of an experiment or process are equally likely and there n<∞ outcomes, then for any event E the probability of E is simply ((|E|)/n), where |E| is the number of outcomes in E.

If X is the set of all possible outcomes and f:X->ℝ, the expected value of f is E(f)=(Underscript[∑ , x∈X])P({x})f(x), where the sum is over all possible outcomes x.  If f defines how much you win for each outcome in a random game, the game is said to be fair if the expected value of f is 0.

The mean μ of a discrete probabiity distribution is the expected value is simply the expected value of the function f(x)=x - i.e. it is μ=(Underscript[∑ , x∈X])(x×P({x})).

The standard deviation σ of a discrete distribution is given by σ=(Sqrt[E((((x-μ))^2))])=(Sqrt[E((x^2))-((E(x))^2)]).  The variance is defined to be (σ^2).

Common cases of discrete distributions are:

The discrete uniform distribution from a to b (a,b∈ℤ and a<b) is defined as the choice of an integer between a and b, with all choices equally likely.  For this distribution μ=((a+b)/2)  (the standard deviation σ is messier, too much so for a timed multiple choice test).

The binomial distribution governs the number of successes k which happen from n independent identical trials, each of which has a probability of success p.  The probability of getting exactly k successes out of n if P(x=k)=(({{n}, {k}}))(((p^k)(1-p))^(n-k)) (this is sometimes written as  (({{n}, {k}}))(p^k)(q^(n-k)), where q=1-p  is the probability of a failure).  Common examples of binomial distributions are flipping coins (fair or not) and counting the “heads”, or the score you get when randomly guessing on a multiple choice test.  The mean of the binomial distribution is μ=n p and σ=(Sqrt[n p(1-p)])=(Sqrt[n p q]).

Continuous Probability Distributions

A probability distribution is continuous if its outcomes consists of one or more intervals of real numbers.  Most continuous distributions are defined by a probability density function f(x) for which:

f(x)≥0

f(x) is at least piecewise continuous

the total area under the graph of y=f(x) is 1 (i.e. (∫_(-∞)^∞) f(x)dx=1)

Then for any range of numbers [a,b], the probability that a≤x≤b is defined by

({{({{P(a≤x≤b)=(∫_a^b)f(x)dx}})}})

As these probabilities are done by areas/integrals, on a timed test you will likely be limited to very simple functions (ones were area can be computed geometrically or by quick integrals).  Note that the probability of any given value x=a is automatically 0 (as the area under the curve from a to a is 0) - this does not mean a is impossible, just infinitely unlikely.

The mean of a continuous distribution is given by μ=(∫_(-∞)^∞) x f(x)dx.  Its standard deviation is given by σ=(Sqrt[(∫_(-∞)^∞) (((x-μ))^2)dx]).  The expected value of a function g(x) is given by (∫_(-∞)^∞) f(x)g(x)dx.

Common continuous distributions are:

Uniform distribution on [a,b] - this governs the process of picking a random real number from a to b (as opposed to an integer).  The p.d.f. is given by f(x)=({{{, ({{(1/(b-a)), a≤x≤b}, {0, otherwise}})}}).   The mean is given by μ=((a+b)/2) and the standard deviation by σ=((b-a)/(2(Sqrt[3]))).

Exponential distribution with mean λ:  The exponential function with mean λ has a p.d.f. given by f(x)=({{{, ({{(1/λ)(e^(-λ x)), x≥0}, {0, otherwise}})}}).    The exponential distribution with mean λ  has a mean μ=λ (not surprisingly) and standard deviation σ=λ.  The exponential distribution often shows up in problems which measure the lifetime of simple mechanical or electrical components as it exhibits a “memoryless” property - namely that P(x>a+b|x>a) depends on on b, not a (so the probability of component failure in the next time interval does not depend on how old the component is)

Normal distribution with mean μ and standard deviation σ:  This is the “bell curve”, with the “standard” normal having μ=0 and σ=1.  The p.d.f for this distribution is f(x)=(1/(σ(Sqrt[2π])))(e^(-(((x-μ))^2)/(2(σ^2)))).  In general this is too hard to integrate with referring to a table of values or computers; however this curve is symmetric about the mean (so probabilities for regions below the mean are the same as for similar/symmetric regions above the mean), and estimates for probabilities for intervals that range certain distances below/above the mean are:

({{({{({{interval range, probability of being in that interval}, {μ-σ  to μ+σ, .683}, {μ-2σ  to μ+2σ, .955}, {μ-3σ  to μ+3σ, .997}})}})}})

So for example in a normal distribution with mean 10 and standard deviation 2, the approximate probability a number will be between 6 (2 deviations below the mean) and 14 (2 deviations above the mean) is about .955 or 95.5%.  Because of the symmetry of bell curves intervals that start at μ but only go up (or only go down) a given number of standard deviations would have probabilities one-half that given in the table.  This can be used to quickly approximate certain probabilities in the binomial distribution using the “normal approximation to the binomial”: For large values of n the binomial distribution for n trials with probability of success p (and probability of failure q=1-p) of can be approximated by a normal distribution with the same mean (μ=n p) and the same standard deviation given by (σ=(Sqrt[n p(1-p)])=(Sqrt[n p q])).  Typically “large values for n” mean both n p (the expected number of successes) and n q (the expected number of failures) are both 9 or more.  Note that this technique will underestimate the true probability by a small amount. In practice you correct this by going out an extra 0.5 in each direction instead of exactly σ, 2σ, etc. - but then you couldn’t use the values given above.  This would then require you to convert the scores to the “standard normal distribution” (μ=0, σ=1) and look up the values in a z-table (and the ETS test description doesn’t mention the table).

Geometric Probabilities

Another common probability question is the probability that a randomly selected point in some region (Subscript[R, 1]) will be in a given subregion (Subscript[R, 2]) (this is essentially a continuous probability distribution extended to more dimensions).  If the regions in question are two dimensional, the probability a point (x,y) will be in a given subregion (Subscript[R, 2]) of another region (Subscript[R, 1]) is the ratio ((Area of subregion (Subscript[R, 2]))/(Area of whole area (Subscript[R, 1]))).  If the regions are three-dimensional then the probability would just be the corresponding ratio of volumes.