https://datascienceschool.net/view-notebook/968c34fd6b404fcab12cbbbfd36418a2/
================================================================================
* Optimization
* output_data=function_A(input_data)
* Find input_data which minimizes (or maximizes) output_data
================================================================================
* Grid search
input_data_in_interval=cut_to_specific_range(large_input_data)
for one_input_data in input_data_in_interval:
output_data=function_A(one_input_data)
is output_data is mimimum?
break
else:
continue
* Grid search takes much time when input variable has too many values
================================================================================
* Numerical optimization
* It gets output from function, by using small number of calculation
* Steepest gradient method
* Newton method (upgraded from steepest gradient method)
================================================================================
* Optimization has "constraints"
* Equality constraint
- Input data should satisfy specific simultaneous equations (as constraint)
* Inequality constraint
- Input data should satisfy specific simultaneous inequalities (as constraint)
================================================================================
* How to solve optimization problem with equality constraint
- Use Lagrange multiplier
================================================================================
* How to solve optimization problem with inequality constraint
- Use KKT condition
================================================================================
Practical optimization problems
- Linear programming (LP problem)
- Quadratic programming (QP problem)
================================================================================