Dimensionality Reduction

transform the data from a high-dimensional space into a low-dimensional space without losing meaningful properties of the original data.

Principal Component Analysis

the process of computing the principal components of the data and using them to perform dimensionality reduction on the data.

Principal Components

principal components are the eigenvectors of covariance matrix of the data matrix and capture most of the variance in the data.

Covariance Matrix

given the data matrix X, the covariance matrix is XX^T where the columns of X are data points.

Eigenvector and Eigenvalue

Ax = ax where x is the eigenvector and a is the eigenvalue of matrix A.

Orthogonal Directions

if two vectors lie on orthogonal directions, their dot product equals to zero.