Correlation

Correlation measures the degree to which two variables move together. The standard measure is Pearson’s r: a single number from −1 to +1 where +1 means perfect positive linear relationship, 0 means no linear relationship, and −1 means perfect negative linear relationship.

Practical interpretation:

• |r| < 0.3 — weak
• 0.3 ≤ |r| < 0.7 — moderate
• |r| ≥ 0.7 — strong

Three things every reader of correlation numbers should know:

1. Pearson’s r only captures linear relationships. Two variables related by a perfect quadratic (y = x²) can have r ≈ 0 if x ranges over both positive and negative values. For non-linear relationships, Spearman’s rho is the more robust alternative.
2. Correlation is not causation. Two variables can correlate strongly because A causes B, B causes A, both are caused by a third variable, or pure coincidence (especially in small samples or comparing many pairs).
3. Outliers distort r dramatically. A single outlier in a small dataset can flip the sign of the correlation. Always plot the data before trusting the number.

For categorical or rank-ordered data, use Spearman’s rank correlation instead of Pearson. For binary outcomes, look up the phi coefficient. For nominal categorical data with more than two levels, Cramér’s V.

Anscombe’s quartet — the famous illustration: in 1973, statistician Francis Anscombe constructed four small datasets that all share the same mean, variance, correlation coefficient (0.816), and linear-regression line — yet look completely different when plotted. One is a clean linear trend; one is a perfect curve; one is a line with a single outlier; one is a vertical line with one rogue point. The quartet is still cited as the canonical case for “always plot the data first.” The Datasaurus Dozen (Matejka & Fitzmaurice, 2017) extends the same idea to twelve datasets sharing summary statistics — including one shaped like a dinosaur. Both make the same point: a single correlation number is necessary but never sufficient. Reference: NIST/SEMATECH e-Handbook — Linear Correlation.

Worked example

Five data points (1,2), (2,4), (3,5), (4,4), (5,5). Means x̄ = 3, ȳ = 4. Deviations x − x̄: −2, −1, 0, 1, 2. Deviations y − ȳ: −2, 0, 1, 0, 1. Sum of cross-products Σ(xᵢ − x̄)(yᵢ − ȳ) = 4 + 0 + 0 + 0 + 2 = 6. Sum of squared x deviations: 10; of y deviations: 6. Pearson r = 6 / √(10 × 6) = 6 / 7.746 ≈ 0.775 — a strong positive linear relationship. A scatter plot would show that interpretation holds; if the third point were (3, 50) instead of (3, 5), r would still appear well-defined but the linear model would be dominated by a single outlier.

When correlation drives decisions

Portfolio diversification: assets with low pairwise correlation reduce overall variance even when their individual volatilities are high. The 2008 financial crisis showed the catastrophic counterexample — equities, corporate bonds, REITs, and even gold all moved together when liquidity dried up, and correlation matrices estimated from calm markets understated tail risk. In ML feature engineering, two features with r > 0.95 are effectively redundant; dropping one rarely degrades model accuracy and speeds training. For experimentation, treating correlated metrics as independent inflates the false-positive rate — apply Bonferroni or Benjamini-Hochberg corrections. Related: regression, variance. Background: Pearson correlation coefficient (Wikipedia).