4 min reading time

Variability explained with each component

6.7.8. Variability explained with each component

We can calculate \(R^2\) values, since PLS explains both the \(\mathbf{X}\)-space and the \(\mathbf{Y}\)-space. We use the \(\mathbf{E}_a\) matrix to calculate the cumulative variance explained for the \(\mathbf{X}\)-space.

\[R^2_{\mathbf{X}, a, \text{cum}} = 1 - \dfrac{\text{Var}(\mathbf{E}_a)}{\text{Var}(\mathbf{X}_{a=1})}\]

Before the first component is extracted we have \(R^2_{\mathbf{X}, a=0} = 0.0\), since \(\mathbf{E}_{a=0} = \mathbf{X}_{a=1}\). After the second component, the residuals, \(\mathbf{E}_{a=1}\), will have decreased, so \(R^2_{\mathbf{X}, a}\) would have increased.

We can construct similar \(R^2\) values for the \(\mathbf{Y}\)-space using the \(\mathbf{Y}_a\) and \(\mathbf{F}_a\) matrices. Furthermore, we construct in an analogous manner the \(R^2\) values for each column of \(\mathbf{X}_a\) and \(\mathbf{Y}_a\), exactly as we did for PCA.

These \(R^2\) values help us understand which components best explain different sources of variation. Bar plots of the \(R^2\) values for each column in \(\mathbf{X}\) and \(\mathbf{Y}\), after a certain number of \(A\) components are one of the best ways to visualize this information.

6.7.8.1. How many components?

For PLS the number of components is chosen by how well the model predicts the \(\mathbf{Y}\)-space on data it has not yet seen, rather than by how much variance it explains in \(\mathbf{X}\). The tool is the same cross-validation described for PCA, but the residuals that matter are those of \(\mathbf{Y}\). Leaving out one group of rows at a time, we predict the held-out \(\mathbf{Y}\) values and gather their prediction error into \(Q^2_Y\), the cross-validated \(R^2\) of the \(\mathbf{Y}\)-space. As with PCA, \(Q^2_Y\) is smaller than \(R^2_Y\), and it stops rising, then falls, once a component no longer improves prediction.

The process_improve package computes this with PLS.select_n_components. It re-derives the centring and scaling inside each cross-validation fold, so the held-out rows do not enter the model that predicts them, and it applies the one-standard-error rule described for PCA: keep the fewest components whose cross-validated error is within one standard error of the best.

Python, 15 lines
import pandas as pd
from process_improve.multivariate import PLS

# Cheddar-cheese data: predict the sensory Taste (Y) from three
# chemical measurements (X). The same data set is used in the PLS
# exercises at the end of this section.
cheese = pd.read_csv("https://openmv.net/file/cheddar-cheese.csv")
Y = cheese[["Taste"]]
X = cheese[["Acetic", "H2S", "Lactic"]]

# Raw X and Y may be passed: the scaling is fit inside every fold.
result = PLS.select_n_components(X, Y, random_state=0)
print(result.n_components)                     # 1
print(result.rmsecv["total"].round(2))         # A=1: 10.88, A=2: 11.02, A=3: 11.08
print(result.r2y_validated["total"].round(3))  # A=1: 0.537, A=2: 0.525, A=3: 0.519

For this small data set cross-validation retains a single component. The cross-validated \(Q^2_Y\) is highest at \(A = 1\) (0.54) and edges down as further components are added, so the second and third components do not improve the prediction of Taste.

There is one column per target, and two that summarise them. With a single target, as here, they agree. With several they need not: total pools the targets on their original scale, so a target with a wide range decides it almost alone, while scaled_total averages the per-target values and gives each the same weight. Which to read follows from the question: scaled_total treats the targets as equally important, and is the one to compare against a fitted \(R^2_Y\) computed on scaled data. The multiblock case study shows a block where the two disagree on which set of predictors is better. Here the one-standard-error rule and the plain minimum of the error curve agree; on data sets where the error curve is flat near its minimum the one-standard-error rule returns the more parsimonious model.

Cross-validation on \(\mathbf{Y}\) stops adding components once the prediction of \(\mathbf{Y}\) stops improving. When the \(\mathbf{X}\)-space itself must also be well described, for example to monitor the squared prediction error of a process, one or two components beyond the cross-validated number are sometimes retained so that \(\mathbf{X}\) is modelled as well.

Contents