Difference between revisions of "Principal Component Analysis"

Latest revision as of 14:04, 17 September 2018

Class date(s):

16, 23, 30 September 2011

Video material (part 1)

Download video: Link (plays in Google Chrome) [290 Mb]

Video material(part 2)

Download video: Link (plays in Google Chrome) [306 Mb]

Video material (part 3)

Download video: Link (plays in Google Chrome) [294 Mb]

Video material (part 4)

Download video: Link (plays in Google Chrome) [152 Mb]

Video material (part 5)

Download video: Link (plays in Google Chrome) [276 Mb]

Video material (part 6)

Download video: Link (plays in Google Chrome) [333 Mb]

Video material (part 7)

Download video: Link (plays in Google Chrome) [198 Mb]

Video material (part 8)

Download video: Link (plays in Google Chrome) [180 Mb]

Class 2 (16 September 2011)

Download the class slides (PDF)

Download these 3 CSV files and bring them on your computer:
- Peas dataset: http://openmv.net/info/peas
- Food texture dataset: http://openmv.net/info/food-texture
- Food consumption dataset: http://openmv.net/info/food-consumption

Background reading

Reading for class 2
Linear algebra topics you should be familiar with before class 2:
- matrix multiplication
- that matrix multiplication of a vector by a matrix is a transformation from one coordinate system to another (we will review this in class)
- linear combinations (read the first section of that website: we will review this in class)
- the dot product of 2 vectors, and that they are related by the cosine of the angle between them (see the geometric interpretation section)

This illustration should help better explain what I trying to get across in class 2B

$p_{1}$ and $p_{2}$ are the unit vectors for components 1 and 2.
$x_{i}$ is a row of data from matrix $X$ .
${\hat{x}}_{i, 1} = t_{i, 1} p_{1}$ = the best prediction of $x_{i}$ using only the first component.
${\hat{x}}_{i, 2} = t_{i, 2} p_{2}$ = the improvement we add after the first component to better predict $x_{i}$ .
${\hat{x}}_{i} = {\hat{x}}_{i, 1} + {\hat{x}}_{i, 2}$ = is the total prediction of $x_{i}$ using 2 components and is the open blue point lying on the plane defined by $p_{1}$ and $p_{2}$ . Notice that this is just the vector summation of ${\hat{x}}_{i, 1}$ and ${\hat{x}}_{i, 2}$ .
$e_{i, 2}$ = is the prediction error vector because the prediction ${\hat{x}}_{i}$ is not exact: the data point $x_{i}$ lies above the plane defined by $p_{1}$ and $p_{2}$ . This $e_{i, 2}$ is the residual distance after using 2 components.
$x_{i} = {\hat{x}}_{i} + e_{i, 2}$ is also a vector summation and shows how $x_{i}$ is broken down into two parts: ${\hat{x}}_{i}$ is a vector on the plane, while $e_{i, 2}$ is the vector perpendicular to the plane.

Class 3 (23, 30 September 2011)

Download the class slides (PDF)

Background reading

Least squares:
- what is the objective function of least squares
- how to calculate the regression coefficient $b$ for $y = b x + e$ where $x$ and $y$ are centered vectors
- understand that the residuals in least squares are orthogonal to $x$
Some optimization theory:
- How an optimization problem is written with equality constraints
- The Lagrange multiplier principle for solving simple, equality constrained optimization problems.

Background reading

Reading on cross validation

@@ Line 1: / Line 1: @@
-__NOTOC__
+{{ClassSidebarYouTube
-== Class 2  (16 September)  ==
+| date = 16, 23, 30 September 2011
+| vimeoID1 = 9QzNOz_7i6U
+| vimeoID2 = qDiPZp-FWc4
+| vimeoID3 = y0Alf0VZ-1E
+| vimeoID4 = XfH_p1WAydM
+| vimeoID5 = QLB-UJ1dFiE
+| vimeoID6 = bysqF41Mgc0
+| vimeoID7 = p3i-XsviARM
+| vimeoID8 = Qb28yc3eM0Q
+| vimeoID9 =
+| course_notes_PDF =
+| course_notes_alt = Course notes
+| overheads_PDF =
+| overheads_PDF_alt = Projector notes
+| assignment_instructions =
+| assignment_solutions =
+| video_download_link_MP4 = http://learnche.mcmaster.ca/media/LVM-2011-Class-02A.mp4
+| video_download_link2_MP4 = http://learnche.mcmaster.ca/media/LVM-2011-Class-02B.mp4
+| video_download_link3_MP4 = http://learnche.mcmaster.ca/media/LVM-2011-Class-02C.mp4
+| video_download_link4_MP4 = http://learnche.mcmaster.ca/media/LVM-2011-Class-03A.mp4
+| video_download_link5_MP4 = http://learnche.mcmaster.ca/media/LVM-2011-Class-03B.mp4
+| video_download_link6_MP4 =  http://learnche.mcmaster.ca/media/LVM-2011-Class-03C.mp4
+| video_download_link7_MP4 =  http://learnche.mcmaster.ca/media/LVM-2011-Class-04A.mp4
+| video_download_link8_MP4 =  http://learnche.mcmaster.ca/media/LVM-2011-Class-04B.mp4
+| video_download_link9_MP4 =
+| video_download_link_MP4_size = 290 Mb
+| video_download_link2_MP4_size = 306 Mb
+| video_download_link3_MP4_size = 294 Mb
+| video_download_link4_MP4_size = 152 Mb
+| video_download_link5_MP4_size = 276 Mb
+| video_download_link6_MP4_size = 333 Mb
+| video_download_link7_MP4_size = 198 Mb
+| video_download_link8_MP4_size = 180 Mb
+| video_download_link9_MP4_size =  Mb
+| video_notes1 =
+| video_notes2 =
+| video_notes3 =
+| video_notes4 =
+| video_notes5 =
+| video_notes6 =
+}}__NOTOC__
+== Class 2 (16 September 2011) ==
-<pdfreflow>
-class_date        = 16 September 2011 [1.65 Mb]
+[[Image:Nuvola_mimetypes_pdf.png|20px|link=Media:Lvm-class-2.pdf]] [[Media:Lvm-class-2.pdf|Download the class slides]] (PDF)
-button_label      = Create my projector slides!
-show_page_layout  = 1
-show_frame_option = 1
-pdf_file          = lvm-class-2.pdf
-</pdfreflow>
-or you may [[Media:Lvm-class-2.pdf|download the lecture slides]] directly.
 * Download these 3 CSV files and bring them on your computer:
-** Peas dataset: http://datasets.connectmv.com/info/peas
+** Peas dataset: http://openmv.net/info/peas
-** Food texture dataset: http://datasets.connectmv.com/info/food-texture
+** Food texture dataset: http://openmv.net/info/food-texture
-** Food consumption dataset: http://datasets.connectmv.com/info/food-consumption
+** Food consumption dataset: http://openmv.net/info/food-consumption
 === Background reading ===
@@ Line 23: / Line 58: @@
 ** matrix multiplication
 ** that matrix multiplication of a vector by a matrix is a transformation from one coordinate system to another (we will review this in class)
-** [http://en.wikipedia.org/wiki/Linear_combination linear combinations] (read the first section of that website: we will review this in class)
+** [https://en.wikipedia.org/wiki/Linear_combination linear combinations] (read the first section of that website: we will review this in class)
 ** the dot product of 2 vectors, and that they are related by the cosine of the angle between them (see the [http://en.wikipedia.org/wiki/Dot_product geometric interpretation section])
@@ Line 38: / Line 73: @@
-== Class 3 ==
+== Class 3 (23, 30 September 2011) ==
-<pdfreflow>
+[[Image:Nuvola_mimetypes_pdf.png|20px|link=Media:Lvm-class-3.pdf]] [[Media:Lvm-class-3.pdf|Download the class slides]] (PDF)
-class_date        = 23 September 2011 [580 Kb]
-button_label      = Create my projector slides!
-show_page_layout  = 1
-show_frame_option = 1
-pdf_file          = lvm-class-3.pdf
-</pdfreflow>
-or you may [[Media:Lvm-class-3.pdf|download the lecture slides]] directly.
 ===Background reading ===
-* [http://stats4eng.connectmv.com/wiki/Least_squares_modelling Least squares]:
+* Least squares:
 ** what is the objective function of least squares
 ** how to calculate the regression coefficient  $b$  for  $y = b x + e$  where  $x$  and  $y$  are centered vectors
@@ Line 58: / Line 86: @@
 * Some optimization theory:
 ** How an optimization problem is written with equality constraints
-** The [http://en.wikipedia.org/wiki/Lagrange_multiplier Lagrange multiplier principle] for solving simple, equality constrained optimization problems.
+** The [https://en.wikipedia.org/wiki/Lagrange_multiplier Lagrange multiplier principle] for solving simple, equality constrained optimization problems.
-== Class 4  (30 September) ==
-Notes are the [[Process_monitoring|Process monitoring]] section.
 ===Background reading ===
 * Reading on [http://literature.connectmv.com/item/12/cross-validatory-estimation-of-the-number-of-components-in-factor-and-principal-components-models cross validation]

Difference between revisions of "Principal Component Analysis"

Latest revision as of 14:04, 17 September 2018

Class 2 (16 September 2011)

Background reading

Class 3 (23, 30 September 2011)

Background reading

Background reading

Navigation menu

Search