# Non-linear programming

Class date(s): 04 February 2015

## Resources

Scroll down, if necessary, to see the resources.

Date Class number Topic Slides/handouts for class References and Notes
04 February 05A
• Why consider unconstrained, single-variable problems
• Newton's method review to solve these problems
09 February 06A
• Newton's method reviewed again for unconstrained, single-variable problems
• Using finite differences instead in Newton's method
• Multivariate unconstrained optimization
11 February 06B
• Unconstrained single-variable optimization using gradient search
• Unconstrained multivariate optimization using gradient search
• Understanding the line search problem
16 to 27 February 07

02 March 08A
• Unconstrained optimization in two variables review
• Contrasting it back to the single variable case
• Extending to the multidimensional Newton's method
04 March 08B
• Examples on the multidimensional Newton's method
• Quasi Newton method in multiple dimensions
• Positive and negative definiteness of the Hessian

Code used in class (see below)

09 March 09A
• Constrained nonlinear optimization introduction
• Model formulation (convert a problem to mathematics)
11 March 09B

Guest lecture

16 March 10A
• Convexity, concavity
• Guarantees on when problems are globally optimal

Handout from class (continued with handout 09A)

18 March 10B
• Lagrange multiplier method for constrained optimization
• Interpretation of the Lagrange multiplier constraints
23 March 11A
• The Nelder-Mead method (several of you are using it in your projects)
• Practice with using the Nelder Mead method. Optimize this system: http://yint.org/nm

### Taking full Newton's steps to solve the class example

```clear all;
close all;
clc;
[X1,X2] = meshgrid(-0.5:0.1:6, 0:0.01:9);
Z = func(X1,X2);
contour(X1, X2, Z)
hold on
grid on

x = [1,3]';
plot(x(1), x(2), 'o')
text(x(1)+0.2, x(2), '0')

for k = 1:10
slope = -first_deriv(x)
step = hessian(x)\slope;   % Solves the Ax=b problem, as x = A\b
x = x + step;
plot(x(1), x(2), '*')
text(x(1)+0.1, x(2), num2str(k))
end
```

func.m

```function y = func(x1,x2)
y = 4.*x1.*x2 - 5.*(x1-2).^4 - 3.*(x2-5).^4;
```

first_deriv.m

```function y = first_deriv(x)
y = [4*x(2) - 20*(x(1)-2)^3;
4*x(1) - 12*(x(2)-5)^3];
```

hessian.m

```function y = hessian(x)
y = [-60*(x(1)-2)^2, 4;
4, -36*(x(2)-5)^2];
```