Correlation and Regression — Predicting from Data

“As the temperature rises, more shaved ice is sold” and “the longer students study, the higher their scores are.” If we can understand the relationship between two quantities, we can predict one from the other. In this lesson, you will learn the correlation coefficient, which measures the strength of a relationship, and the regression line, a tool for prediction. You will also examine common pitfalls in analysis.

Scatter Plots and Correlation

The relationship between two quantitative data sets can be observed with a scatter plot, in which one quantity is placed on the horizontal axis and the other on the vertical axis, and points are plotted. When one quantity tends to increase as the other increases, this is called a positive correlation (the points form an upward pattern). When one increases while the other decreases, it is called a negative correlation (a downward pattern). If neither tendency exists, there is no correlation.

Correlation Coefficient — Quantifying the Strength of a Relationship

The direction and strength of correlation expressed as a single number is the correlation coefficient r. r always takes a value between −1 and 1, inclusive.

・Near r ＝ 1: strong positive correlation (the points lie almost on an upward-sloping straight line)
・Near r ＝ −1: strong negative correlation (the points lie on a downward-sloping straight line)
・Near r ＝ 0: almost no linear correlation

As a general guide, |r| of 0.7 or higher is often read as “strong correlation,” 0.4〜0.7 as “moderate,” and 0.2 or lower as “almost no correlation.” Note that the correlation coefficient can measure only linear relationships (for a U-shaped relationship, r may be close to 0) and that it may change greatly because of a single outlier. Always check it together with a scatter plot.

Regression Line — A Tool for Prediction

The straight line that best fits the arrangement of points in a scatter plot is called a regression line and is expressed in the form y ＝ ax ＋ b. The criterion for “best fit” is often the method of least squares, which minimizes the sum of the squares of the deviations (residuals) between each point and the line. Once a regression line is obtained, the value of y can be predicted from x. Here, x is called the explanatory variable, and y is called the response variable.

Pitfall ① — Correlation and Causation Are Different

Even if the correlation coefficient is large, one variable is not necessarily the cause of the other. There is a strong positive correlation between “ice-cream sales” and “the number of people transported for heatstroke,” but ice cream does not cause heatstroke. A third factor (confounding factor), temperature, increases both (a spurious correlation). We must also consider the possibility that the direction of causation is reversed (does good performance build confidence, or does confidence improve performance?).

Pitfall ② — Sampling Bias

If the collected data (sample) does not properly represent the whole (population), any analysis will be distorted. For example, if a questionnaire is conducted “through a smartphone app,” the opinions of people who do not use smartphones are excluded from the beginning. This is called sampling bias (selection bias). Always check who the subjects were and how the data was collected.

Practice Problems

1. Select all values that are impossible as correlation coefficients. A 0.85　B −0.3　C 1.2　D −1　E −1.5
2. A regression line y ＝ 3x ＋ 10 (x: study time[hours], y: short-test score[points]) has been obtained. Find the predicted value of y when x ＝ 20. Also state what the slope 3 means.
3. Explain, using the term “spurious correlation,” why we cannot immediately conclude from the correlation “students who use the library more have higher grades” that “going to the library raises grades.”