Society of Actuaries (SOA) PA Practice Exam

What effect does increasing the value of lambda have on the coefficients in regularization?

1. It increases the values of the coefficients

2. It makes the coefficients approach 0 more

3. It reduces the bias of the model

4. It allows the model to capture more variance

The correct answer is: It makes the coefficients approach 0 more

Increasing the value of lambda in regularization, particularly in methods such as Lasso (L1 regularization) or Ridge (L2 regularization), leads to a stronger penalty on the size of the coefficients in the model. This increased penalty means that the model is more heavily discouraged from assigning large values to the coefficients during the optimization process. As lambda increases, the regularization term becomes more significant in the loss function, effectively pushing the coefficients towards zero. This is particularly relevant in scenarios where the model may be overfitting the training data, as smaller coefficients can help simplify the model, enhance generalization to new data, and avoid fitting noise. Therefore, the effect of increasing lambda is to make the coefficients approach zero more closely, contributing to a more parsimonious model.