Difference between revisions of "Non-linear programming"

From Optimization for Chemical Engineering: 4G3
Jump to navigation Jump to search
 
(25 intermediate revisions by the same user not shown)
Line 1: Line 1:
__NOTOC__
__NOTOC__
{{ClassSidebar
{{ClassSidebarYouTube
| date = 04 February 2015
| date = 04 February 2015
| dates_alt_text =  
| dates_alt_text =  
| vimeoID1 = 118739752
| vimeoID1 = w91QogkPMxY
| vimeoID2 = 119151097
| vimeoID2 = lKLCPwPFx4k
| vimeoID3 = 119366104
| vimeoID3 = DXXc77TADGU
| vimeoID4 = 121094463
| vimeoID4 = PqgV9uohNYE
| vimeoID5 = 121375922
| vimeoID5 = 4RK36TnNptE
| vimeoID6 = 121726806
| vimeoID6 = KPjQ5ubm36o
| vimeoID7 =  
| vimeoID7 = 03l748-yjg8
| vimeoID8 =
| vimeoID8 = 8h3PxFKxPKQ
| vimeoID9 =
| vimeoID9 = gbVtbU4WSXM
| vimeoID10 =
| vimeoID10 = Hy3D8qQFVUs
| vimeoID11 =
| vimeoID11 =
| course_notes_PDF =  
| course_notes_PDF =  
Line 19: Line 19:
| assignment_instructions =  
| assignment_instructions =  
| assignment_solutions =  
| assignment_solutions =  
| video_download_link_MP4 = https://www.dropbox.com/s/psk1vl672cyrv8g/2015-4G3-Class-05A.mp4?dl=1
| video_download_link_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-05A.mp4  
| video_download_link_MP4_size = 433 M
| video_download_link_MP4_size = 433 M
| video_notes1 =
| video_notes1 =
| video_download_link2_MP4 = https://www.dropbox.com/s/9r9g72vdz411yrb/2015-4G3-Class-06A.mp4?dl=1
| video_download_link2_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-06A.mp4
| video_download_link2_MP4_size = 797 M
| video_download_link2_MP4_size = 797 M
| video_notes2 =
| video_notes2 =
| video_download_link3_MP4 = https://www.dropbox.com/s/gwfgdhrxs370hqh/2015-4G3-Class-06B.mp4?dl=1
| video_download_link3_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-06B.mp4
| video_download_link3_MP4_size = 820 M
| video_download_link3_MP4_size = 820 M
| video_notes3 =
| video_notes3 =
| video_download_link4_MP4 = https://www.dropbox.com/s/m2v00dfowuzrbo8/2015-4G3-Class-08A.mp4?dl=1
| video_download_link4_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-08A.mp4  
| video_download_link4_MP4_size = 640 M
| video_download_link4_MP4_size = 640 M
| video_notes4 =
| video_notes4 =
| video_download_link5_MP4 = https://www.dropbox.com/s/ieteak2gpshre4t/2015-4G3-Class-08B.mp4?dl=1
| video_download_link5_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-08B.mp4  
| video_download_link5_MP4_size = 923 M
| video_download_link5_MP4_size = 923 M
| video_notes5 =
| video_notes5 =
| video_download_link6_MP4 = https://www.dropbox.com/s/jo8wv7bb4rr0yt3/2015-4G3-Class-09A.mp4?dl=1
| video_download_link6_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-09A.mp4  
| video_download_link6_MP4_size = 943 M
| video_download_link6_MP4_size = 943 M
| video_notes6 =
| video_notes6 =
| video_download_link7_MP4 =  
| video_download_link7_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-09B.mp4   
| video_download_link7_MP4_size = 817 M
| video_download_link7_MP4_size = 667 M
| video_notes7 =
| video_notes7 =
| video_download_link8_MP4 =
| video_download_link8_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-10A.mp4   
| video_download_link8_MP4_size =  540 M
| video_download_link8_MP4_size =  948 M
| video_notes8 =
| video_notes8 =
| video_download_link9_MP4 =  
| video_download_link9_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-10B.mp4   
| video_download_link9_MP4_size = M
| video_download_link9_MP4_size = 935 M
| video_notes9 =
| video_notes9 =
| video_download_link10_MP4 =  
| video_download_link10_MP4 = http://learnche.mcmaster.ca/media/2015-4G3-Class-11A.mp4   
| video_download_link10_MP4_size = M
| video_download_link10_MP4_size = 907 M
| video_notes10 =
| video_notes10 =
| video_download_link11_MP4 =
}}
| video_download_link11_MP4_size = M
| video_notes11 =
| video_download_link12_MP4 =
| video_download_link12_MP4_size = M
| video_notes12 =}}


== Resources ==
== Resources ==
Line 65: Line 60:
! Topic
! Topic
! Slides/handouts for class
! Slides/handouts for class
! Video file
! References and Notes
! References and Notes
|-
|-
Line 75: Line 69:
| align="left" colspan="1"|
| align="left" colspan="1"|
[https://docs.google.com/document/d/1NozRUYfdIw2RLnU0DExOzF-RH-mx5VbjgQU35zxKLCU  Handout from class]
[https://docs.google.com/document/d/1NozRUYfdIw2RLnU0DExOzF-RH-mx5VbjgQU35zxKLCU  Handout from class]
| [https://www.dropbox.com/s/psk1vl672cyrv8g/2015-4G3-Class-05A.mp4?dl=1 Video]
|align="left" colspan="1"|
|align="left" colspan="1"|
|-
|-
Line 86: Line 79:
| align="left" colspan="1"|
| align="left" colspan="1"|
[https://docs.google.com/document/d/10oe_laMhtPl5roJGa3C44YlsSvdbHvUAjt6uRMf9q4k  Handout from class]
[https://docs.google.com/document/d/10oe_laMhtPl5roJGa3C44YlsSvdbHvUAjt6uRMf9q4k  Handout from class]
| [https://www.dropbox.com/s/9r9g72vdz411yrb/2015-4G3-Class-06A.mp4?dl=1 Video]
|align="left" colspan="1"|
|align="left" colspan="1"|
|-
|-
Line 97: Line 89:
| align="left" colspan="1"|
| align="left" colspan="1"|
[https://docs.google.com/document/d/1h9qLQbzeeqtQjf3l9Rmwx-5UKZF-ac2ZqMioupda8cs  Handout from class]
[https://docs.google.com/document/d/1h9qLQbzeeqtQjf3l9Rmwx-5UKZF-ac2ZqMioupda8cs  Handout from class]
| [https://www.dropbox.com/s/gwfgdhrxs370hqh/2015-4G3-Class-06B.mp4?dl=1 Video]
|align="left" colspan="1"|
|align="left" colspan="1"|
|-
|-
Line 113: Line 104:
| align="left" colspan="1"|
| align="left" colspan="1"|
[https://docs.google.com/document/d/1vg7ffZUjSSmh1pL1EgeCGJtQnCSC652UWaBanfmqXr0 Handout from class]
[https://docs.google.com/document/d/1vg7ffZUjSSmh1pL1EgeCGJtQnCSC652UWaBanfmqXr0 Handout from class]
| [https://www.dropbox.com/s/m2v00dfowuzrbo8/2015-4G3-Class-08A.mp4?dl=1 Video]
|align="left" colspan="1"|
|align="left" colspan="1"|
|-
|-
Line 124: Line 114:
| align="left" colspan="1"|
| align="left" colspan="1"|
[https://docs.google.com/document/d/1tISkFj7nYa3RV7G8LF4S_nD7dMzb7_ATwY0z4gslDT0 Handout from class]
[https://docs.google.com/document/d/1tISkFj7nYa3RV7G8LF4S_nD7dMzb7_ATwY0z4gslDT0 Handout from class]
| [https://www.dropbox.com/s/ieteak2gpshre4t/2015-4G3-Class-08B.mp4?dl=1 Video]
|align="left" colspan="1"|
|align="left" colspan="1"|
Code used in class (see below)
Code used in class (see below)
Line 135: Line 124:
| align="left" colspan="1"|
| align="left" colspan="1"|
[https://docs.google.com/document/d/1tHekhHdPWEPhPm_lGVP5cws_2-NsTw_6JpVhhLSlgmo Handout from class]  
[https://docs.google.com/document/d/1tHekhHdPWEPhPm_lGVP5cws_2-NsTw_6JpVhhLSlgmo Handout from class]  
|[https://www.dropbox.com/s/jo8wv7bb4rr0yt3/2015-4G3-Class-09A.mp4?dl=1 Video]  
|align="left" colspan="1"|
|-
| 11 March
| 09B
| align="left" colspan="1"|
Guest lecture
| align="left" colspan="1"|
[[Media:Guest-lecture-4G3-2015.pdf |Handout from class]]
|align="left" colspan="1"|
|-
| 16 March
| 10A
| align="left" colspan="1"|
* Convexity, concavity
* Guarantees on when problems are globally optimal
| align="left" colspan="1"|
[https://docs.google.com/document/d/1tHekhHdPWEPhPm_lGVP5cws_2-NsTw_6JpVhhLSlgmo Handout from class]  (continued with handout 09A)
|align="left" colspan="1"|
|-
| 18 March
| 10B
| align="left" colspan="1"|
* Lagrange multiplier method for constrained optimization
* Interpretation of the Lagrange multiplier constraints
| align="left" colspan="1"|
[https://docs.google.com/document/d/14F-MRq34UNR52Wg2JL9MalYfVZIOHjYCTkRa9tgGQ7g/edit?usp=sharing Handout from class]
|align="left" colspan="1"|
|-
| 23 March
| 11A
| align="left" colspan="1"|
* The Nelder-Mead method (several of you are using it in your projects)
* Practice with using the Nelder Mead method. Optimize this system: http://yint.org/nm
| align="left" colspan="1"|
[https://docs.google.com/document/d/1QdpFX1wgiCfNXgEJz9r6j85k4vXXfwbDPo9weKuCFR0/edit?usp=sharing Handout from class]
|align="left" colspan="1"|
|align="left" colspan="1"|
|}
|}


===Taking full Newton's steps to solve the class example===
===Taking full Newton's steps to solve the class example===

Latest revision as of 11:57, 12 August 2018

Class date(s): 04 February 2015










Resources

Scroll down, if necessary, to see the resources.

Date Class number Topic Slides/handouts for class References and Notes
04 February 05A
  • Why consider unconstrained, single-variable problems
  • Newton's method review to solve these problems

Handout from class

09 February 06A
  • Newton's method reviewed again for unconstrained, single-variable problems
  • Using finite differences instead in Newton's method
  • Multivariate unconstrained optimization

Handout from class

11 February 06B
  • Unconstrained single-variable optimization using gradient search
  • Unconstrained multivariate optimization using gradient search
  • Understanding the line search problem

Handout from class

16 to 27 February 07

Reading week break and midterm

02 March 08A
  • Unconstrained optimization in two variables review
  • Contrasting it back to the single variable case
  • Extending to the multidimensional Newton's method

Handout from class

04 March 08B
  • Examples on the multidimensional Newton's method
  • Quasi Newton method in multiple dimensions
  • Positive and negative definiteness of the Hessian

Handout from class

Code used in class (see below)

09 March 09A
  • Constrained nonlinear optimization introduction
  • Model formulation (convert a problem to mathematics)

Handout from class

11 March 09B

Guest lecture

Handout from class

16 March 10A
  • Convexity, concavity
  • Guarantees on when problems are globally optimal

Handout from class (continued with handout 09A)

18 March 10B
  • Lagrange multiplier method for constrained optimization
  • Interpretation of the Lagrange multiplier constraints

Handout from class

23 March 11A
  • The Nelder-Mead method (several of you are using it in your projects)
  • Practice with using the Nelder Mead method. Optimize this system: http://yint.org/nm

Handout from class



Taking full Newton's steps to solve the class example

clear all;
close all;
clc;
[X1,X2] = meshgrid(-0.5:0.1:6, 0:0.01:9);
Z = func(X1,X2);
contour(X1, X2, Z)
hold on
grid on

x = [1,3]';
plot(x(1), x(2), 'o')
text(x(1)+0.2, x(2), '0')

for k = 1:10
   slope = -first_deriv(x)
   step = hessian(x)\slope;   % Solves the Ax=b problem, as x = A\b
   x = x + step;
   plot(x(1), x(2), '*')
   text(x(1)+0.1, x(2), num2str(k))
end

func.m

function y = func(x1,x2)
  y = 4.*x1.*x2 - 5.*(x1-2).^4 - 3.*(x2-5).^4;

first_deriv.m

function y = first_deriv(x)
  y = [4*x(2) - 20*(x(1)-2)^3; 
       4*x(1) - 12*(x(2)-5)^3];

hessian.m

function y = hessian(x)
  y = [-60*(x(1)-2)^2, 4; 
        4, -36*(x(2)-5)^2];