Advanced Probability and Conditional Probability

TL;DR

Advanced probability builds on basic concepts like unions and intersections using set notation. Conditional probability is about how the likelihood of one event changes given that another event has already occurred. Bayes' Theorem helps you reverse conditional probabilities, finding P(A|B) from P(B|A).

1. The Mental Model

Think of probability as a way to quantify uncertainty. When you know something new (a condition), it often changes how you estimate the chances of other things happening. Conditional probability helps you update your beliefs based on new information.

2. The Core Material

Understanding Set Notation for Probability

Photo by DS stories on Pexels

You've likely seen basic probability before. Now we'll formalize it with set notation, which helps when dealing with more complex scenarios.

• Sample Space (S): All possible outcomes of an experiment.
• Event (A, B): A subset of the sample space; a collection of outcomes.
• Union (A U B): The event that A or B (or both) occurs.
  • P(A U B) = P(A) + P(B) - P(A ∩ B) (This formula prevents double-counting the intersection.)
• Intersection (A ∩ B): The event that A and B both occur.
• Complement (A'): The event that A does not occur.
  • P(A') = 1 - P(A)
• Mutually Exclusive Events: Events that cannot happen at the same time (A ∩ B = Ø). For these, P(A U B) = P(A) + P(B).
• Independent Events: Events where the occurrence of one doesn't affect the probability of the other. If A and B are independent, P(A ∩ B) = P(A) * P(B).

Conditional Probability

Photo by DS stories on Pexels

This is where things get interesting. The probability of event A happening given that event B has already happened is written as P(A|B).

• Formula: P(A|B) = P(A ∩ B) / P(B), where P(B) > 0.
• In plain terms, you're narrowing your sample space to only the outcomes where B occurs, and then seeing what proportion of those outcomes also contain A.

Bayes' Theorem

Photo by Vitaly Gariev on Pexels

Bayes' Theorem is a powerful tool for reversing conditional probabilities. If you know P(B|A), it allows you to find P(A|B).

• Formula: P(A|B) = [P(B|A) * P(A)] / P(B)
• You can also express P(B) in terms of A and A' if needed: P(B) = P(B|A)P(A) + P(B|A')P(A').
• This is incredibly useful in real-world situations like medical testing or spam filtering.

Here's how events flow and interact:

graph TD
    A["Initial Event A (e.g., 'Has Disease')"]
    B["Initial Event B (e.g., 'No Disease')"]
    C["Outcome 1 (e.g., 'Test Positive')"]
    D["Outcome 2 (e.g., 'Test Negative')"]

    A -- P(C|A) --> C
    A -- P(D|A) --> D
    B -- P(C|B) --> C
    B -- P(D|B) --> D

    subgraph Conditional Probabilities
        direction LR
        P_A_given_C["P(A|C) - Probability of A given C"]
        P_C_given_A["P(C|A) - Probability of C given A"]
    end

    C -- "Use Bayes' Theorem" --> P_A_given_C

3. Worked Example

Let's say 5% of a population has a certain disease (D). A test for the disease is 90% accurate (meaning P(Positive|D) = 0.90) and has a 10% false positive rate (meaning P(Positive|Not D) = 0.10). You test positive. What's the probability you actually have the disease, P(D|Positive)?

Let:
* D = Has the disease
* D' = Does not have the disease
* Pos = Tests positive
* Neg = Tests negative

We are given:
* P(D) = 0.05
* P(D') = 1 - P(D) = 0.95
* P(Pos|D) = 0.90
* P(Pos|D') = 0.10

We want to find P(D|Pos). Using Bayes' Theorem:
P(D|Pos) = [P(Pos|D) * P(D)] / P(Pos)

First, we need P(Pos). We can find this using the law of total probability:
P(Pos) = P(Pos|D)P(D) + P(Pos|D')P(D')
P(Pos) = (0.90 * 0.05) + (0.10 * 0.95)
P(Pos) = 0.045 + 0.095
P(Pos) = 0.14

Now, substitute P(Pos) back into Bayes' Theorem:
P(D|Pos) = (0.90 * 0.05) / 0.14
P(D|Pos) = 0.045 / 0.14
P(D|Pos) ≈ 0.3214

So, even with a positive test, there's only about a 32.14% chance you actually have the disease. This is because the disease is rare, and the false positive rate, though seemingly small, affects the outcome significantly for a rare condition.

4. Key Takeaways

• Probability uses set notation (union, intersection, complement) to precisely define events and their relationships.
• Conditional probability P(A|B) tells you the chance of A occurring given that B has already occurred.
• The formula for conditional probability is P(A|B) = P(A ∩ B) / P(B).
• Bayes' Theorem (P(A|B) = [P(B|A) * P(A)] / P(B)) allows you to update prior probabilities based on new evidence.
• Conditional probability is crucial for understanding how information changes the likelihood of events.

Common Mistakes:
- Confusing P(A|B) with P(B|A); they are generally not the same.
- Incorrectly assuming independence between events when they are not.
- Forgetting to subtract the intersection when calculating P(A U B) for non-mutually exclusive events.
- Miscalculating the denominator P(B) in Bayes' Theorem; ensure you consider all ways B can occur.

5. Now Try It

You're in a class where 60% of students study for the test (S) and 40% don't (S'). If a student studies, they have an 80% chance of passing (P(Pass|S) = 0.80). If they don't study, they have a 30% chance of passing (P(Pass|S') = 0.30). A student passes the test. What is the probability that they studied, P(S|Pass)?

What to do: Use the given probabilities and Bayes' Theorem to calculate P(S|Pass).
What success looks like: You should get a probability that indicates how much more likely a student is to have studied given they passed, compared to the overall probability of studying.