Principal Component Analysis

From Latent Variable Methods in Engineering
Jump to navigation Jump to search

Class 2 (16 September)

<pdfreflow> class_date = 16 September 2011 [1.65 Mb] button_label = Create my projector slides! show_page_layout = 1 show_frame_option = 1 pdf_file = lvm-class-2.pdf </pdfreflow> or you may download the lecture slides directly.


Background reading

  • Reading for class 2
  • Linear algebra topics you should be familiar with before class 2:
    • matrix multiplication
    • that matrix multiplication of a vector by a matrix is a transformation from one coordinate system to another (we will review this in class)
    • linear combinations (read the first section of that website: we will review this in class)
    • the dot product of 2 vectors, and that they are related by the cosine of the angle between them (see the geometric interpretation section)

This illustration should help better explain what I trying to get across in class 2B

  • p1 and p2 are the unit vectors for components 1 and 2.
  • xi is a row of data from matrix X.
  • x^i,1=ti,1p1 = the best prediction of xi using only the first component.
  • x^i,2=ti,2p2 = the improvement we add after the first component to better predict xi.
  • x^i=x^i,1+x^i,2 = is the total prediction of xi using 2 components and is the open blue point lying on the plane defined by p1 and p2. Notice that this is just the vector summation of x^i,1 and x^i,2.
  • ei,2 = is the prediction error vector because the prediction x^i is not exact: the data point xi lies above the plane defined by p1 and p2. This ei,2 is the residual distance after using 2 components.
  • xi=x^i+ei,2 is also a vector summation and shows how xi is broken down into two parts: x^i is a vector on the plane, while ei,2 is the vector perpendicular to the plane.

Geometric-interpretation-of-PCA-xhat-residuals.png


Class 3 (23 September)

I would advise printing the slides out no more than 2 per page (leaving space for extra notes in today's class) <pdfreflow> class_date = 23 September 2011 [580 Kb] button_label = Create my projector slides! show_page_layout = 1 show_frame_option = 1 pdf_file = lvm-class-3.pdf </pdfreflow> or you may download the lecture slides directly.


Background reading

  • Least squares:
    • what is the objective function of least squares
    • how to calculate the regression coefficient b for y=bx+e where x and y are centered vectors
    • understand that the residuals in least squares are orthogonal to x
  • Some optimization theory:
    • How an optimization problem is written with equality constraints
    • The Lagrange multiplier principle for solving simple, equality constrained optimization problems.

Class 4 (30 September)

Notes are the Process monitoring section.

Background reading