Foundations of Probability and Random Variables

TL;DR

Probability helps us quantify uncertainty, describing how likely events are to happen. A random variable assigns a numerical value to outcomes of a random experiment. Understanding these concepts is crucial for building statistical models.

1. The Mental Model

Think of probability as a way to measure the "chanciness" of things. A random variable is just a fancy name for a number that comes out of a chance process.

2. The Core Material

Probability is all about chances. We use it to describe how likely an event is to occur. An experiment is any process where the outcome isn't known beforehand (like flipping a coin or rolling a die). The set of all possible outcomes is called the sample space.

What's an Event?

Photo by RDNE Stock project on Pexels

An event is just a specific outcome or a group of outcomes from an experiment. For example, when rolling a standard six-sided die, rolling an even number (2, 4, or 6) is an event.

How Do We Measure Probability?

Photo by DS stories on Pexels

The probability of an event, usually written as P(Event), is a number between 0 and 1, inclusive:
* P(Event) = 0 means the event is impossible.
* P(Event) = 1 means the event is certain.
* P(Event) = 0.5 means it's equally likely to happen or not happen.

For simple events with equally likely outcomes, you can calculate probability as:
$$P(\text{Event}) = \frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}}$$

Introducing Random Variables

Photo by DS stories on Pexels

A random variable is a function that assigns a numerical value to each outcome in the sample space of a random experiment. It's "random" because the outcome of the experiment is random. It's a "variable" because its value can change depending on the outcome.

Random variables are usually denoted by capital letters like $X$ or $Y$. The actual values they can take are denoted by lowercase letters, like $x$ or $y$.

For example, if you flip a coin twice:
* The sample space is {HH, HT, TH, TT}.
* Let $X$ be the random variable "number of heads".
* $X$ can take on values 0, 1, or 2.

Types of Random Variables

Photo by Arturo Añez. on Pexels

There are two main types of random variables:

1. Discrete Random Variables: These can only take on a finite number of values or a countably infinite number of values. Think whole numbers.

   • Examples: number of heads in coin flips, number of cars passing a point in an hour, shoe size.

2. Continuous Random Variables: These can take on any value within a given range. Think measurements.

   • Examples: height, weight, temperature, time it takes to run a race.

This course focuses on discrete distributions, so we'll mostly be dealing with discrete random variables.

graph TD
    A["Random Experiment (e.g., Coin Flip, Die Roll)"] --> B{"Outcome (e.g., Heads, 3)"}
    B --> C["Sample Space (All Possible Outcomes)"]
    C --> D["Event (Specific Outcome or Group of Outcomes)"]
    D --> E["Probability of Event (0 to 1)"]
    B --> F["Random Variable (Assigns a Number to Each Outcome)"]
    F --> G{"Type of Random Variable"}
    G --> H["Discrete RV (Countable values, e.g., # Heads)"]
    G --> I["Continuous RV (Any value in a range, e.g., Height)"]

3. Worked Example

Let's say you're interested in the number of defective items in a batch of two items. You know that each item has a 10% chance of being defective, independently.

1. Define the Experiment: Inspecting two items.
2. Identify the Sample Space:
   • Let 'D' be defective, 'N' be not defective.
   • The possible outcomes are: {NN, ND, DN, DD}.
3. Define the Random Variable: Let $X$ be the "number of defective items."
4. Assign values to $X$ for each outcome:
   • NN: $X = 0$
   • ND: $X = 1$
   • DN: $X = 1$
   • DD: $X = 2$
5. Identify the possible values of $X$: $X$ can take values {0, 1, 2}. This makes $X$ a discrete random variable.

4. Key Takeaways

• Probability quantifies the likelihood of an event occurring, ranging from 0 (impossible) to 1 (certain).
• An experiment is any process with an uncertain outcome, and its sample space lists all possible outcomes.
• An event is a specific outcome or set of outcomes from an experiment.
• A random variable assigns a numerical value to each outcome of a random experiment.
• Discrete random variables have countable outcomes (like integers), while continuous random variables can take any value within a range.
• We'll focus on discrete random variables and their distributions in this course.

Common Mistakes to Avoid

• Don't confuse an event (e.g., "rolling an even number") with its probability (e.g., P(even) = 0.5).
• Remember that a random variable itself isn't the probability; it's the number representing an outcome.
• Don't assume outcomes are equally likely unless stated; always check the context.
• Forgetting that probabilities must always be between 0 and 1.

5. Now Try It

You're flipping a biased coin twice. This coin lands on heads with a probability of 0.6 and tails with a probability of 0.4.

1. List the sample space for two flips.
2. Define a random variable $Y$ as the "number of tails" in two flips.
3. For each outcome in your sample space, write down the corresponding value of $Y$.
4. State whether $Y$ is a discrete or continuous random variable.

Success looks like clearly listing the outcomes and their assigned numerical values, and correctly classifying the random variable type.