Probability Distribution Functions

TL;DR

Probability Distribution Functions (PDFs) describe all possible outcomes of a random variable and their likelihoods. They come in two main types: discrete for countable outcomes and continuous for measurable outcomes. Understanding PDFs helps you quantify uncertainty and make predictions about random events.

1. The Mental Model

Imagine you have a magic hat that spits out numbers according to a certain rule. A Probability Distribution Function is like the instruction manual for that hat, telling you exactly what numbers it can spit out and how often you're likely to get each one.

2. The Core Material

A random variable is just a variable whose value is determined by the outcome of a random phenomenon. For instance, the number you get when rolling a die is a random variable. A Probability Distribution Function (PDF), or sometimes just a Probability Distribution, describes how the probabilities are distributed over the possible values of a random variable.

There are two main types of PDFs, depending on whether your random variable is discrete or continuous:

2.1. Discrete Probability Distributions

Photo by DS stories on Pexels

These are for random variables that can only take on a specific, countable number of values. Think of things you can count, like the number of heads in 3 coin flips (0, 1, 2, or 3) or the number of defective items in a batch.

For a discrete variable:
* The sum of all probabilities for all possible outcomes must equal 1.
* Each individual probability must be between 0 and 1 (inclusive).

Here's an example of a simple discrete distribution for rolling a fair six-sided die:

If you wanted to calculate the probability of rolling an even number, you'd add P(2) + P(4) + P(6) = 1/6 + 1/6 + 1/6 = 3/6 = 1/2.

2.2. Continuous Probability Distributions

Photo by DS stories on Pexels

These are for random variables that can take on any value within a given range. Think of things you measure, like a person's height, the temperature, or the time it takes for a lightbulb to burn out. Since there are infinitely many possible values, the probability of any single specific value is essentially zero. Instead, we talk about the probability of a value falling within a range.

For a continuous variable:
* The total area under the probability curve must equal 1.
* The height of the curve (the probability density) at any point must be non-negative.

Some common continuous distributions you'll encounter are the Normal Distribution (the classic bell curve) and the Uniform Distribution.

Here's a diagram to help you see the distinction between discrete and continuous distributions and their common examples:

graph TD
    A["Probability Distribution Functions (PDFs)"] --> B["Discrete PDFs"]
    A --> C["Continuous PDFs"]

    B --> D["Binomial Distribution (fixed trials, two outcomes)"]
    B --> E["Poisson Distribution (events in fixed interval)"]
    B --> F["Geometric Distribution (trials until first success)"]

    C --> G["Normal Distribution (bell curve, common in nature)"]
    C --> H["Uniform Distribution (equal prob. over range)"]
    C --> I["Exponential Distribution (time between events)"]

2.3. Probability Mass Function (PMF) vs. Probability Density Function (PDF)

Photo by DS stories on Pexels

You might hear these terms specifically.
* A Probability Mass Function (PMF) is the formal name for the function describing a discrete probability distribution. It gives you the probability for each specific outcome.
* A Probability Density Function (PDF) is the formal name for the function describing a continuous probability distribution. It doesn't give you the probability of a single point (which is zero) but rather helps you find the probability of a value falling within a range by calculating the area under the curve using calculus (integration). Don't worry too much about the calculus right now; just know that the shape of the curve matters for continuous distributions.

3. Worked Example

Let's say you're a quality control inspector for a factory that produces lightbulbs. Historically, 10% of the lightbulbs produced are defective. You randomly select a batch of 5 lightbulbs to test. What's the probability distribution for the number of defective lightbulbs you find in this batch?

This is a discrete distribution problem because you can only find 0, 1, 2, 3, 4, or 5 defective lightbulbs. This specific scenario follows a Binomial Distribution.

The probability of finding exactly 'k' defective lightbulbs in 'n' trials, with a probability of success (defective) 'p' on each trial, is given by the formula:
P(X=k) = C(n, k) * p^k * (1-p)^(n-k)
where C(n, k) is the number of combinations of 'n' items taken 'k' at a time.

In our case:
* n = 5 (number of lightbulbs inspected)
* p = 0.10 (probability of a single bulb being defective)
* 1-p = 0.90 (probability of a single bulb being non-defective)

Let's calculate the probabilities for each possible number of defective bulbs (X):

• P(X=0): C(5, 0) * (0.10)^0 * (0.90)^5 = 1 * 1 * 0.59049 = 0.5905
• P(X=1): C(5, 1) * (0.10)^1 * (0.90)^4 = 5 * 0.10 * 0.6561 = 0.32805
• P(X=2): C(5, 2) * (0.10)^2 * (0.90)^3 = 10 * 0.01 * 0.729 = 0.0729
• P(X=3): C(5, 3) * (0.10)^3 * (0.90)^2 = 10 * 0.001 * 0.81 = 0.0081
• P(X=4): C(5, 4) * (0.10)^4 * (0.90)^1 = 5 * 0.0001 * 0.90 = 0.00045
• P(X=5): C(5, 5) * (0.10)^5 * (0.90)^0 = 1 * 0.00001 * 1 = 0.00001

Let's check the sum: 0.5905 + 0.32805 + 0.0729 + 0.0081 + 0.00045 + 0.00001 = 1.00001 (very close to 1, small difference due to rounding).

You can also do this easily in Python using scipy.stats.binom:

from scipy.stats import binom

n = 5  # number of trials (lightbulbs)
p = 0.10 # probability of success (defective)

# Calculate probability mass function for each possible outcome
for k in range(n + 1):
    prob = binom.pmf(k, n, p)
    print(f"P(X={k} defective bulbs) = {prob:.5f}")

# You can also get the cumulative probability, e.g., P(X <= 1)
prob_at_most_one = binom.cdf(1, n, p)
print(f"\nP(X <= 1 defective bulb) = {prob_at_most_one:.5f}")

Output:

P(X=0 defective bulbs) = 0.59049
P(X=1 defective bulbs) = 0.32805
P(X=2 defective bulbs) = 0.07290
P(X=3 defective bulbs) = 0.00810
P(X=4 defective bulbs) = 0.00045
P(X=5 defective bulbs) = 0.00001

P(X <= 1 defective bulb) = 0.91854

This output shows you the full probability distribution for the number of defective lightbulbs in your sample. For example, there's about a 59% chance of finding no defective bulbs and about a 33% chance of finding exactly one.

4. Key Takeaways

• PDFs describe all possible outcomes of a random variable and their associated probabilities or likelihoods.
• Discrete PDFs are for countable outcomes, while continuous PDFs are for measurable outcomes.
• For discrete distributions, you can find the probability of exact outcomes using a PMF.
• For continuous distributions, you find probabilities for ranges of outcomes using a PDF (area under the curve).
• The sum of probabilities for all outcomes in a discrete distribution must equal 1.
• The total area under the curve for a continuous distribution must equal 1.
• Specific types of PDFs (like Binomial or Normal) are used for different kinds of random phenomena.

Common mistakes to avoid:
- Confusing discrete with continuous distributions; they require different approaches.
- Assuming a continuous PDF gives you the probability of a single point (it's zero).
- Forgetting that all probabilities in a distribution must sum to 1 (or the area under the curve must be 1).
- Not checking whether your random variable is truly independent and identically distributed if assuming certain distributions like Binomial.

5. Now Try It

You're tracking customer arrivals at a coffee shop. On average, 4 customers arrive every 15 minutes. Using your knowledge of PDFs, describe the probability distribution for the number of customers arriving in a 15-minute period. Calculate the probability of exactly 3 customers arriving in a 15-minute period.

What to do:
1. Identify whether this is a discrete or continuous distribution.
2. Choose an appropriate common probability distribution type for this scenario.
3. State the parameters for that distribution.
4. Calculate the probability of exactly 3 customers arriving.

What success looks like: You'll correctly identify the distribution, state its parameter(s), and provide the probability as a numerical value (e.g., "P(X=3 customers) = 0