Bernoulli and Binomial Distributions

TL;DR

The Bernoulli distribution models a single "yes/no" trial, like a coin flip. The Binomial distribution counts how many "yes" outcomes you get in a fixed number of these independent Bernoulli trials. Together, they're fundamental for understanding success counts in repeated experiments.

1. The Mental Model

Think of these distributions as tools for counting successes. Bernoulli is for a single try, like flipping a coin once. Binomial is for many tries, like flipping that coin ten times and seeing how many heads you get.

2. The Core Material

You're diving into two super common discrete probability distributions: Bernoulli and Binomial. They're related, like a single Lego brick (Bernoulli) and a small structure built from many identical bricks (Binomial).

The Bernoulli Distribution

Photo by Ntate Mohlala Sir on Pexels

The Bernoulli distribution describes a single trial where there are only two possible outcomes: "success" or "failure." Think of it as a single coin flip – it's either heads (success) or tails (failure).

• Parameter: It has one parameter, p, which is the probability of success.
• Outcomes:
  • If the outcome is success, its value is typically 1.
  • If the outcome is failure, its value is typically 0.
• Probability Mass Function (PMF):
  • $P(X=1) = p$ (probability of success)
  • $P(X=0) = 1 - p$ (probability of failure)
• Mean (Expected Value): $E[X] = p$
• Variance: $Var(X) = p * (1-p)$

Example: If you flip a fair coin, $p = 0.5$. The probability of getting a head is 0.5, and the probability of getting a tail is 0.5.

The Binomial Distribution

Photo by Derek Keats on Pexels

The Binomial distribution describes the number of successes in a fixed number of independent Bernoulli trials. Each trial has the same probability of success, p.

• Parameters: It has two parameters:
  • n: The number of trials (e.g., 10 coin flips).
  • p: The probability of success on a single trial (e.g., probability of getting heads).
• Outcomes: The number of successes, $k$, can range from 0 to n (e.g., 0 heads, 1 head, ..., 10 heads).
• Probability Mass Function (PMF):
  $P(X=k) = C(n, k) * p^k * (1-p)^{(n-k)}$
  Where $C(n, k)$ is the binomial coefficient, read as "n choose k," and calculated as $n! / (k! * (n-k)!)$. This just tells you how many different ways you can get k successes in n trials.
• Mean (Expected Value): $E[X] = n * p$
• Variance: $Var(X) = n * p * (1-p)$

Example: You flip a fair coin 10 times (n=10, p=0.5). The Binomial distribution tells you the probability of getting exactly 3 heads, or 7 heads, or any number of heads from 0 to 10.

It's helpful to visualize how these two are connected:

graph TD
    A["Single Trial (Bernoulli)"] --> B{"Outcome: Success (1) or Failure (0)"}
    B -- "Probability of Success = p" --> C["Expected Value = p"]
    B -- "Probability of Failure = 1-p" --> C

    D["Repeat Trial 'n' times"] --> E{"Each Trial is Independent"}
    E --> F{"Each Trial has same 'p'"}
    F --> G["Count total 'k' Successes"]
    G --> H["Multiple Trials (Binomial)"]
    H -- "Parameters: n, p" --> I["Expected Value = n * p"]
    H --> J{"Prob. of 'k' successes (PMF)"}

Using Python for Binomial

Photo by cottonbro studio on Pexels

Python's scipy.stats module makes working with these distributions easy.

from scipy.stats import binom

# Example: Flipping a fair coin 10 times (n=10, p=0.5)

# Probability of getting exactly 3 heads
n_trials = 10
p_success = 0.5
k_successes = 3
prob_3_heads = binom.pmf(k_successes, n_trials, p_success)
print(f"Probability of exactly 3 heads in 10 flips: {prob_3_heads:.4f}")

# Probability of getting 3 or fewer heads (cumulative distribution function - CDF)
prob_3_or_fewer_heads = binom.cdf(k_successes, n_trials, p_success)
print(f"Probability of 3 or fewer heads in 10 flips: {prob_3_or_fewer_heads:.4f}")

# Expected number of heads
expected_heads = n_trials * p_success
print(f"Expected number of heads in 10 flips: {expected_heads}")

# You can also use binom.mean(n, p) and binom.var(n, p)
mean_binom = binom.mean(n_trials, p_success)
variance_binom = binom.var(n_trials, p_success)
print(f"Mean (using binom.mean): {mean_binom}")
print(f"Variance (using binom.var): {variance_binom:.2f}")

3. Worked Example

Let's say you're a quality control inspector. You inspect batches of 20 items. From historical data, you know that 5% of items (p = 0.05) produced by a machine are defective. You want to know:

1. What's the probability that exactly 1 item in a batch of 20 is defective?
2. What's the probability that 2 or fewer items in a batch of 20 are defective?
3. What's the expected number of defective items in a batch of 20?

Solution:

Here, we have a Binomial distribution:
n = 20 (number of trials/items in a batch)
p = 0.05 (probability of success/item being defective)

from scipy.stats import binom

n = 20
p = 0.05

# 1. Probability of exactly 1 defective item (k=1)
prob_exactly_1 = binom.pmf(1, n, p)
print(f"Probability of exactly 1 defective item: {prob_exactly_1:.4f}")

# 2. Probability of 2 or fewer defective items (k <= 2)
# This is P(X=0) + P(X=1) + P(X=2), which is the CDF for k=2
prob_2_or_fewer = binom.cdf(2, n, p)
print(f"Probability of 2 or fewer defective items: {prob_2_or_fewer:.4f}")

# 3. Expected number of defective items
expected_defective = n * p
print(f"Expected number of defective items: {expected_defective}")

# You could also use binom.mean for this
expected_defective_from_func = binom.mean(n, p)
print(f"Expected number of defective items (using binom.mean): {expected_defective_from_func}")

Output:

Probability of exactly 1 defective item: 0.3774
Probability of 2 or fewer defective items: 0.9245
Expected number of defective items: 1.0
Expected number of defective items (using binom.mean): 1.0

4. Key Takeaways

• A Bernoulli trial is a single experiment with only two outcomes: success or failure.
• The probability of success, p, is the single parameter for a Bernoulli distribution.
• A Binomial distribution models the number of successes in a fixed number (n) of independent Bernoulli trials.
• The parameters for a Binomial distribution are n (number of trials) and p (probability of success in each trial).
• You can calculate specific probabilities (PMF) or cumulative probabilities (CDF) for Binomial events.
• The expected value of a Binomial distribution is simply n * p.

Common Mistakes to Avoid:
- Confusing p and 1-p: Always be clear which one represents "success" in your context.
- Not ensuring independence: Binomial trials must be independent; the outcome of one can't affect the next.
- Forgetting n: The Binomial distribution requires a fixed number of trials.
- Incorrectly using PMF vs. CDF: Use PMF for "exactly k successes" and CDF for "k or fewer successes."

5. Now Try It

You're taking a multiple-choice quiz with 12 questions. Each question has 4 options, and only one is correct. You randomly guess every answer.

1. What's the probability you get exactly 5 questions correct?
2. What's the probability you get 3 or fewer questions correct?
3. What's the expected number of questions you'll get correct by guessing?

Write down n and p first, then calculate these three values. Success means you can correctly calculate and interpret these probabilities using the concepts learned.