The Central Limit Theorem

Notes on the 3Blue1Brown video, explaining the intuition behind one of the most important theorems in statistics.

Introduction

The Central Limit Theorem (CLT) is a fundamental concept in probability theory and statistics. It describes the surprising fact that if you take many samples from any population, the distribution of the means of those samples will be approximately a normal distribution (a bell curve), regardless of the original population's distribution.

Random Variables and Distributions

A random variable is a variable whose value is a numerical outcome of a random phenomenon. For example, the result of a single dice roll is a random variable.

The distribution of a random variable describes the probability of each possible outcome. For a fair six-sided die, the distribution is uniform, as each outcome (1, 2, 3, 4, 5, 6) has an equal probability of 1/6. Other distributions can have different shapes (e.g., skewed, bimodal, etc.).

The Process of Sampling

The core idea of the Central Limit Theorem involves a process of sampling from a population:
1. Take a sample of a certain size, $n$, from a population.
2. Calculate the mean of this sample.
3. Repeat this process many times, collecting a list of sample means.

The Central Limit Theorem is about the distribution of this list of sample means.

The Distribution of Sample Means

As you collect more and more sample means, their distribution starts to take on a familiar shape: the normal distribution (bell curve). This happens even if the original population's distribution was not a bell curve at all.

Imagine rolling a die 10 times and taking the average. This is one sample mean. If you do this thousands of times and plot a histogram of all the averages you found, the histogram will look like a normal distribution.

The Theorem Stated

The Central Limit Theorem states that for a sufficiently large sample size ($n$), the distribution of the sample means ($\bar{x}$) will be approximately normal.

This new distribution has two important properties:
1. Its mean, $\mu_{\bar{x}}$, is the same as the mean of the original population, $\mu$.
$$
\mu_{\bar{x}} = \mu
$$
2. Its standard deviation, $\sigma_{\bar{x}}$ (also known as the standard error), is the standard deviation of the original population, $\sigma$, divided by the square root of the sample size, $n$.
$$
\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}
$$
This second property is very important: it means that as you increase your sample size, the sample means will be more tightly clustered around the true population mean.

The Intuition: Summing Random Variables

The 3Blue1Brown video provides a beautiful intuition for why this happens. The mean of a sample is just the sum of the samples, divided by the sample size. So, understanding the distribution of sums is key.

When you add two random variables together, the distribution of the sum is the convolution of their individual distributions. Visually, this process has a "smoothing" and "centralizing" effect. As you add more and more random variables, the resulting distribution gets smoother and more bell-shaped.

The different ways to get a certain sum pile up in the middle, and there are fewer ways to get extreme values, which naturally leads to a bell curve shape.

Why is the CLT Important?

The Central Limit Theorem is the backbone of many statistical procedures. Because we know that sample means are normally distributed, we can make inferences about a population from a single sample. For example:
* Hypothesis Testing: We can determine how likely it is that our sample mean could have come from a population with a certain hypothetical mean.
* Confidence Intervals: We can create a range of values that we are confident contains the true population mean.

More on

• Central limit theorem: 3Blue1Brown
• Khan Academy on the Central Limit Theorem
• An interactive visualization of the CLT