Society of Actuaries (SOA) PA Practice Exam

Which penalty does Ridge Regression use for its coefficients?

1. The sum of the absolute values

2. The sum of the squares of the estimated coefficients

3. Proportional reduction of coefficients

4. No penalty applied

The correct answer is: The sum of the squares of the estimated coefficients

Ridge Regression utilizes a penalty that is applied to the sum of the squares of the estimated coefficients, which serves to shrink the coefficients towards zero. This method is especially beneficial when dealing with multicollinearity in the data or when the model has a high number of predictors relative to the number of observations. By adding this L2 regularization term, Ridge Regression effectively helps to prevent overfitting by penalizing large coefficients, which can lead to a more robust model during prediction. The inclusion of this penalty term not only stabilizes the estimation process but also improves the model's generalization capabilities by encouraging smaller and more evenly distributed coefficient values. Thus, Ridge Regression's distinctive approach relies directly on this squared coefficient penalty, making it a key characteristic of the method.