Society of Actuaries (SOA) PA Practice Exam

How does PCA reduce dimensionality in comparison to LASSO?

1. PCA reduces dimensionality by coercing some coefficients to zero

2. PCA reduces dimensionality without referencing any target variable

3. PCA directly chooses the number of dimensions without any optimization

4. PCA optimizes the penalized function to fit the target variable

The correct answer is: PCA reduces dimensionality without referencing any target variable

PCA (Principal Component Analysis) is a technique primarily used for dimensionality reduction that focuses on capturing the maximum variance in the data while disregarding any specific outcome or target variable. This is achieved through an unsupervised approach, in which PCA identifies the principal components or directions in the feature space that have the highest variance, and these components form a lower-dimensional representation of the original data. The choice regarding how PCA operates highlights that it does not require knowledge of or reference to a dependent variable, unlike methods such as LASSO (Least Absolute Shrinkage and Selection Operator), which is a supervised technique. LASSO involves fitting a model to the target variable and applying a penalty to the size of the coefficients, leading to some coefficients being shrunk to zero to perform variable selection during the optimization process. By focusing solely on the structure of the data rather than on an outcome, PCA effectively transforms the original feature space into a new space defined by these principal components, allowing for reduced dimensionality while preserving as much variance as possible. This characteristic distinguishes it from approaches that utilize optimization based on target variable performance or coefficients manipulation, further solidifying why the selected answer accurately reflects PCA's methodology.