Difference between revisions of "Principal Component Analysis"

From Latent Variable Methods in Engineering
Jump to navigation Jump to search
m
m
Line 54: Line 54:


* Download these 3 CSV files and bring them on your computer:
* Download these 3 CSV files and bring them on your computer:
** Peas dataset: http://datasets.connectmv.com/info/peas
** Peas dataset: http://openmv.net/info/peas
** Food texture dataset: http://datasets.connectmv.com/info/food-texture
** Food texture dataset: http://openmv.net/info/food-texture
** Food consumption dataset: http://datasets.connectmv.com/info/food-consumption
** Food consumption dataset: http://openmv.net/info/food-consumption


=== Background reading ===
=== Background reading ===
Line 93: Line 93:
===Background reading ===
===Background reading ===


* [http://stats4eng.connectmv.com/wiki/Least_squares_modelling Least squares]:
* Least squares:
** what is the objective function of least squares
** what is the objective function of least squares
** how to calculate the regression coefficient b for y=bx+e where x and y are centered vectors
** how to calculate the regression coefficient b for y=bx+e where x and y are centered vectors

Revision as of 17:36, 25 November 2015

Class date(s): 16, 23, 30 September 2011
Video material (part 1)
Download video: Link (plays in Google Chrome) [290 Mb]


Video material(part 2)
Download video: Link (plays in Google Chrome) [306 Mb]


Video material (part 3)
Download video: Link (plays in Google Chrome) [294 Mb]


Video material (part 4)
Download video: Link (plays in Google Chrome) [152 Mb]


Video material (part 5)
Download video: Link (plays in Google Chrome) [276 Mb]


Video material (part 6)
Download video: Link (plays in Google Chrome) [333 Mb]


Video material (part 7)
Download video: Link (plays in Google Chrome) [198 Mb]


Video material (part 8)
Download video: Link (plays in Google Chrome) [180 Mb]

Class 2

<pdfreflow> class_date = 16 September 2011 [1.65 Mb] button_label = Create my projector slides! show_page_layout = 1 show_frame_option = 1 pdf_file = lvm-class-2.pdf </pdfreflow> or you may download the class slides directly.


Background reading

  • Reading for class 2
  • Linear algebra topics you should be familiar with before class 2:
    • matrix multiplication
    • that matrix multiplication of a vector by a matrix is a transformation from one coordinate system to another (we will review this in class)
    • linear combinations (read the first section of that website: we will review this in class)
    • the dot product of 2 vectors, and that they are related by the cosine of the angle between them (see the geometric interpretation section)

This illustration should help better explain what I trying to get across in class 2B

  • p1 and p2 are the unit vectors for components 1 and 2.
  • xi is a row of data from matrix X.
  • x^i,1=ti,1p1 = the best prediction of xi using only the first component.
  • x^i,2=ti,2p2 = the improvement we add after the first component to better predict xi.
  • x^i=x^i,1+x^i,2 = is the total prediction of xi using 2 components and is the open blue point lying on the plane defined by p1 and p2. Notice that this is just the vector summation of x^i,1 and x^i,2.
  • ei,2 = is the prediction error vector because the prediction x^i is not exact: the data point xi lies above the plane defined by p1 and p2. This ei,2 is the residual distance after using 2 components.
  • xi=x^i+ei,2 is also a vector summation and shows how xi is broken down into two parts: x^i is a vector on the plane, while ei,2 is the vector perpendicular to the plane.

Geometric-interpretation-of-PCA-xhat-residuals.png


Class 3

<pdfreflow> class_date = 23, 30 September 2011 [580 Kb] button_label = Create my projector slides! show_page_layout = 1 show_frame_option = 1 pdf_file = lvm-class-3.pdf </pdfreflow> or you may download the class slides directly.


Background reading

  • Least squares:
    • what is the objective function of least squares
    • how to calculate the regression coefficient b for y=bx+e where x and y are centered vectors
    • understand that the residuals in least squares are orthogonal to x
  • Some optimization theory:
    • How an optimization problem is written with equality constraints
    • The Lagrange multiplier principle for solving simple, equality constrained optimization problems.


Background reading