Cook’s Distance is a statistical measure used to identify influential data points or outliers in regression analysis. Specifically, it helps us determine if any particular data point significantly influences the estimated regression model.
What does Cook’s Distance measure?
Cook’s Distance quantifies the influence of each data point on the fitted regression model. It combines both the residuals (how far the point is from the predicted value) and the leverage (how far the point’s predictor values are from the average of all predictor values).
Leverage
This refers to the position of a data point in the predictor space. A point that is far from the average of the predictor values has high leverage, meaning it could disproportionately influence the fitted regression model.
Residual
This is the difference between the observed value and the predicted value (how much the point “deviates” from the model’s prediction).
Cook’s Distance is calculated for each data point i as:
[math]D_i = \frac{(e_i^2)}{p \cdot \text{MSE}} \cdot \frac{h_{ii}}{(1 – h_{ii})^2}[/math]
Where:
- [math]e_i[/math] is the residual for the i-th point.
- [math]h_{ii}[/math] is the leverage of the i-th point (a measure of how much the point affects the predicted values).
- p is the number of predictors in the model.
- MSE is the mean squared error of the model.
