Foundations of Measurement and Sampling

TL;DR

Measurement is about assigning numbers to observations reliably, while sampling helps you understand a larger group by looking at a smaller, representative part of it. Choosing the right measurement scale and sampling method is crucial for getting meaningful and accurate results. Bad measurements or unrepresentative samples lead to wrong conclusions.

1. The Mental Model

Think of measurement as giving things a label using numbers, like saying "this table is 3 feet long." Sampling is like tasting a spoonful of soup to decide if the whole pot needs more salt; you can't eat the whole pot, but that spoonful should tell you enough.

2. The Core Material

When you're trying to understand something about the world, you first need to figure out how to measure it. Then, if the "something" is too big to measure completely, you need to sample it.

What is Measurement?

Photo by William Warby on Pexels

Measurement is the process of assigning numbers or labels to observations or events in a systematic way. It's how you turn qualitative ideas into quantitative data you can analyze. The "systematic way" part is key – it means you're consistent.

Measurement Scales

Not all numbers are created equal. The type of scale you use determines what kind of math you can do with your data.

• Nominal Scale: This is just for naming or categorizing. Think of eye colors (blue, brown, green) or types of cars (sedan, SUV, truck). There's no order or value to these categories. You can count how many are in each category, but you can't say "sedan is greater than SUV."
• Ordinal Scale: This scale allows for ordering or ranking, but the differences between ranks aren't necessarily equal. Think of satisfaction ratings (very unhappy, unhappy, neutral, happy, very happy) or finishing positions in a race (1st, 2nd, 3rd). You know 1st is better than 2nd, but the difference between 1st and 2nd might not be the same as between 2nd and 3rd.
• Interval Scale: Here, the order matters, and the intervals (differences) between values are meaningful and equal. However, there's no true "zero" point, meaning zero doesn't imply the absence of the thing you're measuring. Temperature in Celsius or Fahrenheit is a classic example. The difference between 10°C and 20°C is the same as between 20°C and 30°C. But 0°C doesn't mean "no temperature."
• Ratio Scale: This is the most powerful scale. It has order, equal intervals, and a true zero point. A true zero means the absence of the quantity being measured. Height, weight, income, and time are all ratio scales. If you have 0 height, you have no height. You can say someone who is 6 feet tall is twice as tall as someone who is 3 feet tall.

graph TD
    A["Measurement Scales"] --> B["Nominal (Categories, no order)"]
    A --> C["Ordinal (Categories, order but unequal intervals)"]
    A --> D["Interval (Order, equal intervals, no true zero)"]
    A --> E["Ratio (Order, equal intervals, true zero)"]
    B --> B1["Example: Eye Color"]
    C --> C1["Example: Satisfaction (Low, Med, High)"]
    D --> D1["Example: Temperature (°C/°F)"]
    E --> E1["Example: Height (cm/inches)"]

What is Sampling?

Photo by Pavel Danilyuk on Pexels

Sampling is selecting a subset of individuals or items from a larger group (the population) to gather data from. The goal is that this subset, called the sample, is representative enough that you can make accurate conclusions about the entire population without having to measure everyone or everything.

Key Terms:

• Population: The entire group you're interested in studying. Could be all adults in a country, all products made in a factory, or all bacteria in a culture.
• Sample: The subset of the population you actually collect data from.
• Sampling Frame: A list or source from which the sample is drawn. For example, a phone book for surveying households, or a database of customers.

Common Sampling Methods:

There are two main types:

• Probability (Random) Sampling: Every member of the population has a known, non-zero chance of being selected. This is essential for generalizing your findings to the whole population.

  • Simple Random Sample (SRS): Every individual has an equal chance of being selected. Imagine drawing names out of a hat.
  • Stratified Sample: You divide the population into groups (strata) based on some characteristic (e.g., age, gender, location), then take a random sample from each stratum. This ensures representation from all important subgroups.
  • Systematic Sample: You select every Nth individual from a list. For example, selecting every 10th customer entering a store.
  • Cluster Sample: You divide the population into naturally occurring groups (clusters), then randomly select some clusters and include all individuals within those selected clusters. For example, randomly selecting 5 schools (clusters) and surveying all students in those schools.

• Non-Probability Sampling: Selection is not random, meaning some individuals might have no chance of being selected, or their probability of selection is unknown. These methods are often easier and cheaper but make it difficult to generalize.

  • Convenience Sample: You select individuals who are easiest to reach. Asking your friends or people walking by in a specific location.
  • Quota Sample: Similar to stratified, but instead of random selection within strata, you just keep sampling until you fill a predetermined "quota" for each group.
  • Purposive Sample: You intentionally select individuals based on specific characteristics because you believe they'll provide valuable information for your research question.

The golden rule for sampling: Aim for random sampling whenever possible if you want your results to apply to the wider population. Non-random samples are fine for exploratory research or when you only care about the specific group you're studying.

3. Worked Example

Let's say you're a coffee shop owner and you want to understand customer satisfaction with your new loyalty program. Your population is all customers who've signed up.

You decide to measure satisfaction using a survey question asking, "How satisfied are you with our new loyalty program?" with options: "Very Dissatisfied," "Dissatisfied," "Neutral," "Satisfied," "Very Satisfied." This is an Ordinal Scale measurement.

You can't survey all 10,000 loyalty program members. You need to sample.

Instead of just grabbing the first 50 people who walk in (a convenience sample), you decide to use a Stratified Sample to ensure you hear from different types of customers.

1. Define Strata: You realize you have two main types of loyalty members: those who visit daily, and those who visit occasionally (once a week or less).
   • Daily visitors: 3,000 members
   • Occasional visitors: 7,000 members
2. Determine Sample Size: You decide you want a total sample of 200 customers.
3. Allocate Sample Proportionally: You want the sample to reflect the population proportions.
   • Daily visitors: (3,000 / 10,000) * 200 = 60 daily visitors
   • Occasional visitors: (7,000 / 10,000) * 200 = 140 occasional visitors
4. Randomly Select within Strata: From your list of 3,000 daily visitors, you randomly select 60. From your list of 7,000 occasional visitors, you randomly select 140.

Now you have a sample of 200 customers, representative of your two main visitor types, and you can survey them about their loyalty program satisfaction.

4. Key Takeaways

• Always define what you're measuring and how you'll measure it consistently.
• Understand your measurement scale (nominal, ordinal, interval, ratio) because it dictates what analyses you can perform.
• Clearly define your target population before attempting to sample.
• Random sampling methods are crucial for making reliable generalizations about a population.
• Non-random samples are quicker but limit how broadly your conclusions apply.
• A good sample is representative; it reflects the key characteristics of the population you care about.
• Poor measurement or sampling can invalidate all your subsequent analysis.

5. Now Try It

Imagine you want to find out the average screen time of teenagers in your town.
1. Define your population: Which teenagers? What age range?
2. Choose a measurement: How would you measure "screen time" (daily average, weekly total, specific apps)? What scale would this measurement be on?
3. Propose a sampling strategy: Describe one probability sampling method and one non-probability sampling method you could use, and explain the pros and cons of each for this scenario.

Success looks like: Clearly defining the population and measurement scale, and articulating how two different sampling methods would work, including their strengths and weaknesses in the context of estimating teenage screen time.