http://tomwiki.com/api.php?action=feedcontributions&user=Per&feedformat=atomTomWiki - User contributions [en]2024-03-29T14:25:15ZUser contributionsMediaWiki 1.39.1http://tomwiki.com/index.php?title=How_to_set_SNOPT_tolerances_with_ezsolve&diff=3133How to set SNOPT tolerances with ezsolve2015-12-24T14:27:04Z<p>Per: </p>
<hr />
<div>[[SNOPT]] tolerances (and other parameters) can be set in the Prob.SOL.optPar vector. This vector is described in the SNOPT manual, and also in the Matlab help for the function "snoptTL".<br />
<br />
With tomSym/ezsolve, the vector is passed as options.Prob.SOL.optPar. It should first be initialized as -999, to leave all other parameters at their default values.<br />
<br />
Example:<br />
<br />
optPar = repmat(-999,72,1); % -999 means to use default values<br />
optPar(10) = 1e-3; % set major optimality tolerance<br />
options.Prob.SOL.optPar = optPar;<br />
solution = ezsolve(objective, constraints, guess, options)<br />
<br />
[[Category: tomSym]]</div>Perhttp://tomwiki.com/index.php?title=How_to_set_SNOPT_tolerances_with_ezsolve&diff=3132How to set SNOPT tolerances with ezsolve2015-12-24T14:18:00Z<p>Per: </p>
<hr />
<div>[[SNOPT]] tolerances (and other parameters) can be set in the Prob.SOL.optPar vector.<br />
<br />
With tomSym/ezsolve, this vector is set as options.Prob.SOL.optPar. The vector should first be initialized as -999, to leave all other parameters at their default values.<br />
<br />
Example:<br />
<br />
optPar = repmat(-999,72,1); % -999 means to use default values<br />
optPar(10) = 1e-3; % set major optimality tolerance<br />
options.Prob.SOL.optPar = optPar;<br />
solution = ezsolve(objective, constraints, guess, options)<br />
<br />
[[Category: tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_FAQ&diff=3131TomSym FAQ2015-12-15T10:03:35Z<p>Per: </p>
<hr />
<div>This is a list of frequently asked questions about [[tomSym]].<br />
<br />
===Can I be sure that TOMLAB will always return a solution to my optimization problem?===<br />
<br />
-No. Although there are certain subclasses of problems, such as Linear Programming (LP) and Quadratic Programming (QP), where convergence is guaranteed (with upper bounds on the computation time), in the general case there is no guarantee that a given problem can be solved in any reasonable amount of time, nor that the returned solution is a global optimum.<br />
<br />
However, TOMLAB has a large collection of powerful solvers, and is often able to solve problems that are considered extremely hard as long as the optimization parameters are choosen correctly. (If you run into problems, don't hesitate to contact our support team. See our website http://tomopt.comor contact info.)<br />
<br />
===TOMSYM says that "the returned solution may be incorrect". What's wrong?===<br />
<br />
TOMSYM gives this warning message whenever the numeric solver returned any kind of error or warning. Look at the output from the solver for clues.<br />
<br />
If the solver says that the problem was ïnfeasible", then there is likely a problem with the constraints. Try removing or relaxing constraints one at a time to pinpoint the problem. The <tt>feasibilitycheck</tt> function can also be useful in diagnosing the problem and finding a feasible initial point.<br />
<br />
If the solver determines that the problem is ünbounded", then there are either not enough constraints, or the objective function is incorrect. Remember that TOMSYM alwasy assumes a minimization problem, so for maximization problems a minus sign must be put in front of the objective.<br />
<br />
If the solver stoped because the maximum number of iterations was exceeded, then there is likely something wrong with the scaling of the problem. Are some variables/equations thousands of times larger than others? Setting <tt>options.scale = 'auto'</tt> might help. Another likely cause of the problem is that the initial guess is not good enough. Try supplying a better guess.<br />
<br />
If the solver "terminated during ... evaluation" then your objective or equations (or their derivatives) has a numeric problem, such as division by zero. Try to rewrite your code to avoid division by zero, set stricter bounds so that this does not occur, or switch to a more robust solver.<br />
<br />
===How can I know when tomSym is used instead of Matlab's own routines?===<br />
<br />
TomSym routines are ''overloaded'' which means that they will be used whenever a tomSym object is part of a function call. For example, if <tt>x</tt> is a tomSym symbol (or expression), then a function call such as <tt>sin(x)</tt> will use the tomSym version of the <tt>sin</tt> function, and the returned object will be another tomSym expression.<br />
<br />
On the other hand, calling <tt>sin(0.5)</tt> will use Matlab's own <tt>sin</tt> function, even if TOMLAB is installed.<br />
<br />
===The solution contains fields named "symb1", "symb2" etc. Where do these come from?===<br />
The <tt>symb*</tt> fields appear because <tt>ezsolve</tt> calls the function <tt>rewriteV</tt> to rewrite some non-linear functions, like <tt>abs</tt>, as linear equations using extra variables and constraints. This can make convergence much faster. You can safely ignore the <tt>symb*</tt> values in the solution.<br />
<br />
Variables named <tt>symb*</tt> can also appear when a matrix inequality is converted for use with a regular NLP solver. In this case the new variable represents a square root of the positive definite matrix.<br />
<br />
===How do I set solver parameters, when using ezsolve?===<br />
Any options that can be given to tomRun in the Prob stucture can also be used with ezsolve. Just set the corresponding field in the OPTIONS.Prob substructure. See [[How to set SNOPT tolerances with ezsolve]].<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=How_to_set_SNOPT_tolerances_with_ezsolve&diff=3130How to set SNOPT tolerances with ezsolve2015-12-15T10:02:58Z<p>Per: Example of setting SNOPT options through ezsolve</p>
<hr />
<div>[[SNOPT]] tolerances (and other parameters) can be set in the Prob.SOL.optPar vector.<br />
<br />
With tomSym/ezsolve, this vector is set as options.Prob.SOL.optPar. The vector should first be initialized as -999, to leave all other parameters at their default values.<br />
<br />
Example:<br />
<br />
optPar = repmat(-999,72,1); % -999 means to use default values<br />
optPar(10) = 1e-3; % set major optimality tolerance<br />
options.Prob.SOL.optPar = optPar;<br />
solution = ezsolve(objective, constraints, guess, options)</div>Perhttp://tomwiki.com/index.php?title=Getting_started_with_tomSym&diff=3129Getting started with tomSym2015-12-15T09:58:47Z<p>Per: /* Installation */</p>
<hr />
<div>We believe that the best way to learn is to try it out ourself. The easiest way to use TOMSYM is to give commands directly at the MATLAB prompt. Unless you put a semicolon at the end of the line, the results will be displayed as soon as you hit the "enter" key. This guide provides several examples that you can try. The code that you should type into MATLAB is preceeded by the prompt &gt;&gt; , and MATLAB's response appears below it, as in this example:<br />
<br />
<br />
&gt;&gt; toms a b % Create two symbolic objects<br />
&gt;&gt; a + b % Add them together<br />
<br />
ans = tomSym(1x1):<br />
<br />
a+b<br />
<br />
<br />
<br />
Any text after a percent sign (<tt>%</tt>) is a comment, and has no effect on the result.<br />
<br />
===Installation===<br />
<br />
TomSym is part of TOMLAB, which is available for Windows (32bit/64bit), Linux (32bit/64bit) and Mac OS X (64bit).<br />
<br />
TOMLAB can be downloaded from the website http://tomopt.com and the download page also contains a link to the installation manual which explains the rest of the installation process.<br />
<br />
===Examples===<br />
<br />
The tomSym installation contains a directory named <tt>examples</tt>. This contains a number of ready-made examples of optimization problems solved using TOMSYM, and might serve as a source of inspiration.<br />
<br />
===Getting help===<br />
<br />
For help on any specific TOMSYM command, you can always type <tt>help</tt> followed by the command name at the MATLAB prompt. You can also look for help in the FAQ section of this guide.<br />
<br />
Finally, you are always welcome to ask us directly, via the support form on our website: http://tomopt.com<br />
<br />
== Introduction to TOMSYM ==<br />
<br />
TOMSYM is a symbolic package, meaning that it deals with symbols, rather than numbers. A <tt>tomSym</tt> object can be a simple symbol (a name) or it can be a mathematical expression involving many simple symbols.<br />
<br />
<br />
&gt;&gt; a = tom('a') % Create a scalar symbol<br />
<br />
a = tomSym(1x1):<br />
<br />
a<br />
<br />
&gt;&gt; f = cos(a)/a<br />
<br />
f = tomSym(1x1):<br />
<br />
(1/a)*cos(a)<br />
<br />
<br />
<br />
In this example, <tt>a</tt> is a simple symbol, and <tt>f</tt> is an expression.<br />
<br />
TOMSYM expressions can be used in calculations in much the same way as normal Matlab variables, and the result of the calculations will be new TOMSYM expressions. It is, for example, possible to add 2 to the expression we already defined:<br />
<br />
<br />
&gt;&gt; f = f+2<br />
<br />
f = tomSym(1x1):<br />
<br />
(1/a)*cos(a)+2<br />
<br />
<br />
<br />
In optimization, it is often useful to compute the derivative of a function. TOMSYM supports symbolic derivatives via the <tt>derivative</tt> function.<br />
<br />
<br />
&gt;&gt; g = derivative(f,a)<br />
<br />
g = tomSym(1x1):<br />
<br />
-(cos(a)*((1/a)*(1/a)))-(1/a)*sin(a)<br />
<br />
<br />
<br />
TOMSYM objects can be converted into numeric values by substituting values for the symbols it contains.<br />
<br />
<br />
&gt;&gt; subs(g,a==2)<br />
<br />
ans =<br />
<br />
-0.3506<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=Getting_started_with_tomSym&diff=3095Getting started with tomSym2015-01-26T12:05:10Z<p>Per: /* Getting help */</p>
<hr />
<div>We believe that the best way to learn is to try it out ourself. The easiest way to use TOMSYM is to give commands directly at the MATLAB prompt. Unless you put a semicolon at the end of the line, the results will be displayed as soon as you hit the "enter" key. This guide provides several examples that you can try. The code that you should type into MATLAB is preceeded by the prompt &gt;&gt; , and MATLAB's response appears below it, as in this example:<br />
<br />
<br />
&gt;&gt; toms a b % Create two symbolic objects<br />
&gt;&gt; a + b % Add them together<br />
<br />
ans = tomSym(1x1):<br />
<br />
a+b<br />
<br />
<br />
<br />
Any text after a percent sign (<tt>%</tt>) is a comment, and has no effect on the result.<br />
<br />
===Installation===<br />
<br />
TomSym is part of TOMLAB, which is available for Windows (32bit/64bit), Linux (32bit/64bit) and Mac OS X (64bit).<br />
<br />
TOMLAB can be downloaded from the website http://tomopt.comnd the download page also contains a link to the installation manual which explains the rest of the installation process.<br />
<br />
===Examples===<br />
<br />
The tomSym installation contains a directory named <tt>examples</tt>. This contains a number of ready-made examples of optimization problems solved using TOMSYM, and might serve as a source of inspiration.<br />
<br />
===Getting help===<br />
<br />
For help on any specific TOMSYM command, you can always type <tt>help</tt> followed by the command name at the MATLAB prompt. You can also look for help in the FAQ section of this guide.<br />
<br />
Finally, you are always welcome to ask us directly, via the support form on our website: http://tomopt.com<br />
<br />
== Introduction to TOMSYM ==<br />
<br />
TOMSYM is a symbolic package, meaning that it deals with symbols, rather than numbers. A <tt>tomSym</tt> object can be a simple symbol (a name) or it can be a mathematical expression involving many simple symbols.<br />
<br />
<br />
&gt;&gt; a = tom('a') % Create a scalar symbol<br />
<br />
a = tomSym(1x1):<br />
<br />
a<br />
<br />
&gt;&gt; f = cos(a)/a<br />
<br />
f = tomSym(1x1):<br />
<br />
(1/a)*cos(a)<br />
<br />
<br />
<br />
In this example, <tt>a</tt> is a simple symbol, and <tt>f</tt> is an expression.<br />
<br />
TOMSYM expressions can be used in calculations in much the same way as normal Matlab variables, and the result of the calculations will be new TOMSYM expressions. It is, for example, possible to add 2 to the expression we already defined:<br />
<br />
<br />
&gt;&gt; f = f+2<br />
<br />
f = tomSym(1x1):<br />
<br />
(1/a)*cos(a)+2<br />
<br />
<br />
<br />
In optimization, it is often useful to compute the derivative of a function. TOMSYM supports symbolic derivatives via the <tt>derivative</tt> function.<br />
<br />
<br />
&gt;&gt; g = derivative(f,a)<br />
<br />
g = tomSym(1x1):<br />
<br />
-(cos(a)*((1/a)*(1/a)))-(1/a)*sin(a)<br />
<br />
<br />
<br />
TOMSYM objects can be converted into numeric values by substituting values for the symbols it contains.<br />
<br />
<br />
&gt;&gt; subs(g,a==2)<br />
<br />
ans =<br />
<br />
-0.3506<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=PROPT_Oil_Shale_Pyrolysis&diff=3094PROPT Oil Shale Pyrolysis2015-01-21T15:49:30Z<p>Per: /* Problem setup */</p>
<hr />
<div>{{Part Of Manual|title=the PROPT Manual|link=[[PROPT|PROPT Manual]]}}<br />
<br />
Dynamic Optimization of Batch Reactors Using Adaptive Stochastic Algorithms 1997, Eugenio F. Carrasco, Julio R. Banga<br />
<br />
Case Study II: Oil Shale Pyrolysis<br />
<br />
==Problem description==<br />
<br />
A very challenging optimal control problem is the oil shale pyrolysis case study, as considered by Luus (1994). The system consists of a series of five chemical reactions:<br />
<br />
A1 -> A2<br />
<br />
A2 -> A3<br />
<br />
A1+A2 -> A2+A2<br />
<br />
A1+A2 -> A3+A2<br />
<br />
A1+A2 -> A4+A2<br />
<br />
This system is described by the differential equations<br />
<br />
<math> \frac{dx_1}{dt} = -k_1*x_1-(k_3+k_4+k_5)*x_1*x_2 </math><br />
<br />
<math> \frac{dx_2}{dt} = k_1*x_1-k_2*x_2+k_3*x_1*x_2 </math><br />
<br />
<math> \frac{dx_3}{dt} = k_2*x_2+k_4*x_1*x_2 </math><br />
<br />
<math> \frac{dx_4}{dt} = k_5*x_1*x_2 </math><br />
<br />
<br />
where the state variables are the concentrations of species, Ai, i = 1, ..., 4. The initial condition is<br />
<br />
<math> x(t_0) = [1 \ 0 \ 0 \ 0]' </math><br />
<br />
<br />
The rate expressions are given by:<br />
<br />
<math> k_i = k_{i0}*exp(-\frac{Ei}{R*T}), i=1,2,3,4,5 </math><br />
<br />
<br />
where the values of ki0 and Ei are given by Luus (1994). The optimal control problem is to find the residence time t_f and the temperature profile T(t) in the time interval 0 <= t <= t_f so that the production of pyrolytic bitumen, given by x2, is maximized. Therefore, the performance index is<br />
<br />
<math> J = x_2(t_f) </math><br />
<br />
<br />
The constraints on the control variable (temperature) are:<br />
<br />
<math> 698.15 <= T <= 748.15 </math><br />
<br />
<br />
<source lang="matlab"><br />
% Copyright (c) 2007-2008 by Tomlab Optimization Inc.<br />
</source><br />
<br />
==Problem setup==<br />
<br />
<source lang="matlab"><br />
toms t<br />
toms t_f<br />
<br />
ai = [8.86; 24.25; 23.67; 18.75; 20.70];<br />
bi = [20300; 37400; 33800; 28200; 31000]/1.9872;<br />
<br />
for n=[4 10 20 30 35]<br />
p = tomPhase('p', t, 0, t_f, n);<br />
setPhase(p);<br />
<br />
tomStates x1 x2 x3 x4<br />
tomControls T<br />
<br />
% Initial guess<br />
if n == 4<br />
x0 = {t_f == 9.3<br />
collocate(T == 725)};<br />
else<br />
x0 = {t_f == tfopt<br />
icollocate({<br />
x1 == x1opt; x2 == x2opt<br />
x3 == x3opt; x4 == x4opt<br />
})<br />
collocate(T == Topt)};<br />
end<br />
<br />
% Box constraints<br />
cbox = {9.1 <= t_f <= 12<br />
icollocate({0 <= x1 <= 1; 0 <= x2 <= 1<br />
0 <= x3 <= 1; 0 <= x4 <= 1})<br />
698.15 <= collocate(T) <= 748.15};<br />
<br />
% Boundary constraints<br />
cbnd = initial({x1 == 1; x2 == 0; x3 == 0; x4 == 0});<br />
<br />
% ODEs and path constraints<br />
<br />
ki1 = exp(ai(1)-bi(1)./T);<br />
ki2 = exp(ai(2)-bi(2)./T);<br />
ki3 = exp(ai(3)-bi(3)./T);<br />
ki4 = exp(ai(4)-bi(4)./T);<br />
ki5 = exp(ai(5)-bi(5)./T);<br />
<br />
ceq = collocate({<br />
dot(x1) == -ki1.*x1-(ki3+ki4+ki5).*x1.*x2<br />
dot(x2) == ki1.*x1-ki2.*x2+ki3.*x1.*x2<br />
dot(x3) == ki2.*x2+ki4.*x1.*x2<br />
dot(x4) == ki5.*x1.*x2});<br />
<br />
% Objective<br />
objective = -final(x2);<br />
</source><br />
<br />
==Solve the problem==<br />
<br />
<source lang="matlab"><br />
options = struct;<br />
options.name = 'Oil Pyrolysis';<br />
solution = ezsolve(objective, {cbox, cbnd, ceq}, x0, options);<br />
<br />
x1opt = subs(x1, solution);<br />
x2opt = subs(x2, solution);<br />
x3opt = subs(x3, solution);<br />
x4opt = subs(x4, solution);<br />
Topt = subs(T, solution);<br />
tfopt = subs(final(t), solution);<br />
</source><br />
<br />
<pre><br />
Problem type appears to be: lpcon<br />
Time for symbolic processing: 0.42383 seconds<br />
Starting numeric solver<br />
===== * * * =================================================================== * * *<br />
TOMLAB - TOMLAB Development license 999007. Valid to 2011-12-31<br />
=====================================================================================<br />
Problem: --- 1: Oil Pyrolysis f_k -0.357327805323273240<br />
sum(|constr|) 0.000000000957541036<br />
f(x_k) + sum(|constr|) -0.357327804365732190<br />
f(x_0) 0.000000000000000000<br />
<br />
Solver: snopt. EXIT=0. INFORM=1.<br />
SNOPT 7.2-5 NLP code<br />
Optimality conditions satisfied<br />
<br />
FuncEv 1 ConstrEv 93 ConJacEv 93 Iter 50 MinorIter 197<br />
CPU time: 0.062400 sec. Elapsed time: 0.064000 sec. <br />
<br />
</pre><br />
<br />
<pre><br />
Problem type appears to be: lpcon<br />
Time for symbolic processing: 0.42239 seconds<br />
Starting numeric solver<br />
===== * * * =================================================================== * * *<br />
TOMLAB - TOMLAB Development license 999007. Valid to 2011-12-31<br />
=====================================================================================<br />
Problem: --- 1: Oil Pyrolysis f_k -0.354368283541904860<br />
sum(|constr|) 0.000000002719365042<br />
f(x_k) + sum(|constr|) -0.354368280822539790<br />
f(x_0) -0.357327805323273070<br />
<br />
Solver: snopt. EXIT=0. INFORM=1.<br />
SNOPT 7.2-5 NLP code<br />
Optimality conditions satisfied<br />
<br />
FuncEv 1 ConstrEv 208 ConJacEv 208 Iter 112 MinorIter 305<br />
CPU time: 0.156001 sec. Elapsed time: 0.153000 sec. <br />
<br />
</pre><br />
<br />
<pre><br />
Problem type appears to be: lpcon<br />
Time for symbolic processing: 0.43305 seconds<br />
Starting numeric solver<br />
===== * * * =================================================================== * * *<br />
TOMLAB - TOMLAB Development license 999007. Valid to 2011-12-31<br />
=====================================================================================<br />
Problem: --- 1: Oil Pyrolysis f_k -0.351747594492437030<br />
sum(|constr|) 0.000000280199856233<br />
f(x_k) + sum(|constr|) -0.351747314292580830<br />
f(x_0) -0.354368283541905970<br />
<br />
Solver: snopt. EXIT=0. INFORM=1.<br />
SNOPT 7.2-5 NLP code<br />
Optimality conditions satisfied<br />
<br />
FuncEv 1 ConstrEv 124 ConJacEv 124 Iter 71 MinorIter 281<br />
CPU time: 0.140401 sec. Elapsed time: 0.142000 sec. <br />
<br />
</pre><br />
<br />
<pre><br />
Problem type appears to be: lpcon<br />
Time for symbolic processing: 0.42125 seconds<br />
Starting numeric solver<br />
===== * * * =================================================================== * * *<br />
TOMLAB - TOMLAB Development license 999007. Valid to 2011-12-31<br />
=====================================================================================<br />
Problem: --- 1: Oil Pyrolysis f_k -0.352833701465704920<br />
sum(|constr|) 0.000000018165177960<br />
f(x_k) + sum(|constr|) -0.352833683300526950<br />
f(x_0) -0.351747594492436640<br />
<br />
Solver: snopt. EXIT=0. INFORM=1.<br />
SNOPT 7.2-5 NLP code<br />
Optimality conditions satisfied<br />
<br />
FuncEv 1 ConstrEv 409 ConJacEv 409 Iter 194 MinorIter 697<br />
CPU time: 0.639604 sec. Elapsed time: 0.645000 sec. <br />
<br />
</pre><br />
<br />
<pre><br />
Problem type appears to be: lpcon<br />
Time for symbolic processing: 0.42103 seconds<br />
Starting numeric solver<br />
===== * * * =================================================================== * * *<br />
TOMLAB - TOMLAB Development license 999007. Valid to 2011-12-31<br />
=====================================================================================<br />
Problem: --- 1: Oil Pyrolysis f_k -0.352618526247765740<br />
sum(|constr|) 0.000016167955157926<br />
f(x_k) + sum(|constr|) -0.352602358292607830<br />
f(x_0) -0.352833701465704590<br />
<br />
Solver: snopt. EXIT=0. INFORM=1.<br />
SNOPT 7.2-5 NLP code<br />
Optimality conditions satisfied<br />
<br />
FuncEv 1 ConstrEv 64 ConJacEv 64 Iter 46 MinorIter 364<br />
CPU time: 0.187201 sec. Elapsed time: 0.180000 sec. <br />
<br />
</pre><br />
<br />
<br />
<source lang="matlab"><br />
end<br />
<br />
t = subs(collocate(t),solution);<br />
x1 = subs(collocate(x1opt),solution);<br />
x2 = subs(collocate(x2opt),solution);<br />
x3 = subs(collocate(x3opt),solution);<br />
x4 = subs(collocate(x4opt),solution);<br />
T = subs(collocate(Topt),solution);<br />
</source><br />
<br />
==Plot result==<br />
<br />
<source lang="matlab"><br />
subplot(2,1,1)<br />
plot(t,x1,'*-',t,x2,'*-',t,x3,'*-',t,x4,'*-');<br />
legend('x1','x2','x3','x4');<br />
title('Oil Pyrolysis state variables');<br />
<br />
subplot(2,1,2)<br />
plot(t,T,'+-');<br />
legend('T');<br />
title('Oil Pyrolysis control');<br />
</source><br />
<br />
[[File:oilPyrolysis_01.png]]<br />
<br />
[[Category:PROPT Examples]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=3093TomSym compared to the Symbolic Toolbox2015-01-21T12:34:09Z<p>Per: /* tomSym code */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price'*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro'; % also try: 'snopt'<br />
options.Prob.KNITRO.options.ALG=3;<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is constant, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved (until the matrices become so large that the time for numeric computations begins to matter.)<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
symbolic expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=3092TomSym compared to the Symbolic Toolbox2015-01-21T12:31:41Z<p>Per: /* tomSym code */ KNITRO works with ALG=2 or 3.</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price'*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.Prob.KNITRO.options.ALG=3;<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is constant, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved (until the matrices become so large that the time for numeric computations begins to matter.)<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
symbolic expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=3091TomSym compared to the Symbolic Toolbox2015-01-20T16:50:26Z<p>Per: </p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price'*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'snopt';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is constant, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved (until the matrices become so large that the time for numeric computations begins to matter.)<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
symbolic expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=3090TomSym compared to the Symbolic Toolbox2015-01-20T15:48:32Z<p>Per: /* tomSym code */ some versionsn of KNITRO seem to have problems?</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price'*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'snopt';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is constant, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved (until the matrices become so large that the time for numeric computations begins to matter.)<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
symbolic expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=3089TomSym compared to the Symbolic Toolbox2015-01-20T13:31:38Z<p>Per: bugfix (transpose was lost in copying code into wiki)</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price'*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is constant, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved (until the matrices become so large that the time for numeric computations begins to matter.)<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
symbolic expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=Nonconvex_QP&diff=3076Nonconvex QP2014-09-23T12:07:15Z<p>Per: </p>
<hr />
<div>This is an example of how to find the global optimum to a non-convex quadratic programming (QP). The problem set-up is done using [[tomSym]], and a mixed-integer solver, such as [[CPLEX]] or [[GUROBI]] does the numeric work.<br />
<br />
We want to minimize:<br />
<br />
0.5*x'*Q*x'<br />
<br />
Subject to:<br />
<br />
Aeq*x == beq<br />
0 <= x <= 1<br />
<br />
If Q were positive semidefinite, this would be a Quadratic Programming.<br />
However, Q is indefinite, making the problem much harder. We could easily<br />
find a local optimum using a nonlinear solver, but finding a global<br />
optimum is hard.<br />
<br />
To do this, we set up the first-order Karush-Kuhn-Tucker (KKT)<br />
conditions, and obtain a mixed integer programming. This will yield a<br />
solution that is proven to be within tolerances of the global optimum.<br />
<br />
<source lang="matlab"><br />
% Input parameters.<br />
% (Q is a randomly generated 20x20 symmetric matrix.)<br />
Q = setSymmetric([0.99813;-0.9328;-0.41602;0.75425;0.36975;-0.16011;...<br />
0.20505;-0.22754;0.21575;-0.63597;-0.29342;0.27946;-0.91458;...<br />
-0.33758;0.72371;0.91155;-0.95604;-0.58809;0.11226;-0.60359;...<br />
1.3152;-0.45212;-0.0081002;0.73651;-0.89605;1.0736;1.0053;...<br />
-0.089598;-0.86427;-0.83047;-0.35381;-0.10368;-0.20189;-0.5286;...<br />
0.74354;1.142;0.11788;0.57156;0.16935;-1.854;0.80398;1.0658;...<br />
-1.0921;0.4812;0.17866;0.26736;0.68732;-0.04921;0.51998;...<br />
-0.4118;0.20122;0.62257;-0.48735;-1.2293;-0.27855;0.020442;...<br />
-1.1031;-0.85323;0.084205;1.3643;-0.095637;0.12529;0.071885;...<br />
0.74562;-0.36408;-1.0812;0.32137;0.79026;0.4365;1.536;0.73981;...<br />
1.5732;1.0358;0.86736;0.70902;-1.5217;-0.93981;-0.94568;0.77429;...<br />
-0.77916;1.1569;0.26974;-0.00055197;-0.13087;0.25253;-0.83893;...<br />
0.090814;0.025564;0.3809;-0.079573;0.041911;0.12882;1.0671;...<br />
-0.42695;-1.1162;-0.00038358;0.37305;-0.047864;-0.83988;...<br />
-0.074761;0.11005;0.33679;-0.91941;0.13811;1.2319;-0.077739;...<br />
0.40736;-0.70684;0.24293;-0.64002;1.631;-0.36573;0.34084;...<br />
-0.046399;-0.85519;0.065744;-0.99468;-0.47995;0.91457;-0.30563;...<br />
1.2142;-0.9352;-1.1335;0.32376;-0.68342;0.92591;-0.13294;-0.77732;...<br />
0.41786;-0.38176;1.4631;-0.452;-0.54978;0.91733;-0.17072;0.79189;...<br />
0.22303;0.64006;-0.22713;-0.78101;0.17289;1.4549;-1.1471;-2.4195;...<br />
-0.0197;0.11204;0.41687;0.079953;-0.79564;0.55421;0.084912;...<br />
-0.051431;-0.25787;0.29086;0.14115;-1.1786;-0.62126;-0.97309;...<br />
0.33165;-0.59094;-0.1878;-0.20598;-0.48744;0.16878;-1.0183;...<br />
-0.12057;-0.18849;-0.16661;0.20649;-0.53152;0.81186;0.86475;...<br />
0.43589;-0.68085;0.59335;0.49119;-0.75623;0.21621;-1.1849;...<br />
0.29998;-0.5439;0.3483;-0.8631;0.30066;0.092835;0.97066;0.085208;...<br />
-1.0018;-0.36108;-1.0288;0.28215;0.23805;-0.016555;-1.1906;...<br />
-0.23784;-0.19766;1.0435;0.39938;-0.19682;0.33933;-0.67583;...<br />
-0.34775;-0.1395;0.17759;0.082428;-0.43612;1.6515;-1.2706;...<br />
-0.99358;-1.4966]);<br />
Aeq = [0 1 1 1 1 0 0 0 0 0 1 0 0 1 1 1 1 0 0 0; ...<br />
0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0];<br />
beq = [5;2];<br />
<br />
N = size(Q,1);<br />
<br />
% Lower and upper bounds on x, expressed as A*x <= b<br />
% We could also put arbitrary linear inequalities here.<br />
A = [eye(N); -eye(N)];<br />
b = [ones(N,1); zeros(N,1)];<br />
<br />
M = size(A,1);<br />
<br />
% Upper bound on s and y<br />
% If these values are too small, then no solution will be found.<br />
% Too large numbers may make the solver less efficient, or lead to<br />
% incorrect solutions due to rounding.<br />
bnds = 1; <br />
bndy = 100;<br />
<br />
% Symbolic variables<br />
x = tom('x',N,1);<br />
s = tom('s',M,1); % Slack variables<br />
y = tom('y',M,1); % Lagrange multipliers for inequality constraints<br />
r = tom('r',M,1,'integer'); % 0 for inactive constraints, 1 for active.<br />
yeq = tom('yeq',size(Aeq,1)); % Multipliers for equality constraints<br />
<br />
kktconditions = {<br />
A'*y+Aeq'*yeq+Q*x==0<br />
Aeq*x==beq<br />
A*x+s == b<br />
0 <= y<br />
0 <= s<br />
y <= bndy*r<br />
s <= bnds*(1-r)<br />
};<br />
<br />
% This linear objective will be equal to 0.5*x'*C*x at optimum<br />
obj = -0.5*(y'*b+yeq'*beq);<br />
<br />
%% Solve the problem<br />
options = struct;<br />
options.Prob.MIP.cpxControl.EPGAP = 1e-4; % Tolerance relative to global optimum.<br />
solution = ezsolve(obj,kktconditions,[],options);<br />
</source><br />
<br />
[[Category: tomSym]]</div>Perhttp://tomwiki.com/index.php?title=Nonconvex_QP&diff=3075Nonconvex QP2014-09-23T12:05:44Z<p>Per: </p>
<hr />
<div>This is an example of how to find the global optimum to a non-convex quadratic programming (QP). The problem set-up is done using [[tomSym]], and a mixed-integer solver, such as [[CPLEX]] or [[GUROBI]] does the numeric work.<br />
<br />
We want to minimize:<br />
<br />
0.5*x'*Q*x'<br />
<br />
Subject to:<br />
<br />
Aeq*x == beq<br />
0 <= x <= 1<br />
<br />
If Q were positive semidefinite, this would be a Quadratic Programming.<br />
However, Q is indefinite, making the problem much harder. We could easily<br />
find a local optimum using a nonlinear solver, but finding a global<br />
optimum is hard.<br />
<br />
To do this, we set up the first-order Karush-Kuhn-Tucker (KKT)<br />
conditions, and obtain a mixed integer programming. This will yield a<br />
solution that is proven to be within tolerances of the global optimum.<br />
<br />
<source lang="matlab"><br />
% Input parameters.<br />
% (Q is just a randomly generated symmetric matrix.)<br />
Q = setSymmetric([0.99813;-0.9328;-0.41602;0.75425;0.36975;-0.16011;...<br />
0.20505;-0.22754;0.21575;-0.63597;-0.29342;0.27946;-0.91458;...<br />
-0.33758;0.72371;0.91155;-0.95604;-0.58809;0.11226;-0.60359;...<br />
1.3152;-0.45212;-0.0081002;0.73651;-0.89605;1.0736;1.0053;...<br />
-0.089598;-0.86427;-0.83047;-0.35381;-0.10368;-0.20189;-0.5286;...<br />
0.74354;1.142;0.11788;0.57156;0.16935;-1.854;0.80398;1.0658;...<br />
-1.0921;0.4812;0.17866;0.26736;0.68732;-0.04921;0.51998;...<br />
-0.4118;0.20122;0.62257;-0.48735;-1.2293;-0.27855;0.020442;...<br />
-1.1031;-0.85323;0.084205;1.3643;-0.095637;0.12529;0.071885;...<br />
0.74562;-0.36408;-1.0812;0.32137;0.79026;0.4365;1.536;0.73981;...<br />
1.5732;1.0358;0.86736;0.70902;-1.5217;-0.93981;-0.94568;0.77429;...<br />
-0.77916;1.1569;0.26974;-0.00055197;-0.13087;0.25253;-0.83893;...<br />
0.090814;0.025564;0.3809;-0.079573;0.041911;0.12882;1.0671;...<br />
-0.42695;-1.1162;-0.00038358;0.37305;-0.047864;-0.83988;...<br />
-0.074761;0.11005;0.33679;-0.91941;0.13811;1.2319;-0.077739;...<br />
0.40736;-0.70684;0.24293;-0.64002;1.631;-0.36573;0.34084;...<br />
-0.046399;-0.85519;0.065744;-0.99468;-0.47995;0.91457;-0.30563;...<br />
1.2142;-0.9352;-1.1335;0.32376;-0.68342;0.92591;-0.13294;-0.77732;...<br />
0.41786;-0.38176;1.4631;-0.452;-0.54978;0.91733;-0.17072;0.79189;...<br />
0.22303;0.64006;-0.22713;-0.78101;0.17289;1.4549;-1.1471;-2.4195;...<br />
-0.0197;0.11204;0.41687;0.079953;-0.79564;0.55421;0.084912;...<br />
-0.051431;-0.25787;0.29086;0.14115;-1.1786;-0.62126;-0.97309;...<br />
0.33165;-0.59094;-0.1878;-0.20598;-0.48744;0.16878;-1.0183;...<br />
-0.12057;-0.18849;-0.16661;0.20649;-0.53152;0.81186;0.86475;...<br />
0.43589;-0.68085;0.59335;0.49119;-0.75623;0.21621;-1.1849;...<br />
0.29998;-0.5439;0.3483;-0.8631;0.30066;0.092835;0.97066;0.085208;...<br />
-1.0018;-0.36108;-1.0288;0.28215;0.23805;-0.016555;-1.1906;...<br />
-0.23784;-0.19766;1.0435;0.39938;-0.19682;0.33933;-0.67583;...<br />
-0.34775;-0.1395;0.17759;0.082428;-0.43612;1.6515;-1.2706;...<br />
-0.99358;-1.4966]);<br />
Aeq = [0 1 1 1 1 0 0 0 0 0 1 0 0 1 1 1 1 0 0 0; ...<br />
0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0];<br />
beq = [5;2];<br />
<br />
N = size(Q,1);<br />
<br />
% Lower and upper bounds on x, expressed as A*x <= b<br />
% We could also put arbitrary linear inequalities here.<br />
A = [eye(N); -eye(N)];<br />
b = [ones(N,1); zeros(N,1)];<br />
<br />
M = size(A,1);<br />
<br />
% Upper bound on s and y<br />
% If these values are too small, then no solution will be found.<br />
% Too large numbers may make the solver less efficient, or lead to<br />
% incorrect solutions due to rounding.<br />
bnds = 1; <br />
bndy = 100;<br />
<br />
% Symbolic variables<br />
x = tom('x',N,1);<br />
s = tom('s',M,1); % Slack variables<br />
y = tom('y',M,1); % Lagrange multipliers for inequality constraints<br />
r = tom('r',M,1,'integer'); % 0 for inactive constraints, 1 for active.<br />
yeq = tom('yeq',size(Aeq,1)); % Multipliers for equality constraints<br />
<br />
kktconditions = {<br />
A'*y+Aeq'*yeq+Q*x==0<br />
Aeq*x==beq<br />
A*x+s == b<br />
0 <= y<br />
0 <= s<br />
y <= bndy*r<br />
s <= bnds*(1-r)<br />
};<br />
<br />
% This linear objective will be equal to 0.5*x'*C*x at optimum<br />
obj = -0.5*(y'*b+yeq'*beq);<br />
<br />
%% Solve the problem<br />
options = struct;<br />
options.Prob.MIP.cpxControl.EPGAP = 1e-4; % Tolerance relative to global optimum.<br />
solution = ezsolve(obj,kktconditions,[],options);<br />
</source><br />
<br />
[[Category: tomSym]]</div>Perhttp://tomwiki.com/index.php?title=Nonconvex_QP&diff=3074Nonconvex QP2014-09-23T12:02:11Z<p>Per: </p>
<hr />
<div>This is an example of how to find the global optimum to a non-convex quadratic programming (QP). The problem set-up is done using [[tomSym]], and a mixed-integer solver, such as [[CPLEX]] or [[GUROBI]] does the numeric work.<br />
<br />
We want to minimize:<br />
<br />
0.5*x'*Q*x'<br />
<br />
Subject to:<br />
<br />
Aeq*x == beq<br />
0 <= x <= 1<br />
<br />
If Q were positive semidefinite, this would be a Quadratic Programming.<br />
However, Q is indefinite, making the problem much harder. We could easily<br />
find a local optimum using a nonlinear solver, but finding a global<br />
optimum is hard.<br />
<br />
To do this, we set up the first-order Karush-Kuhn-Tucker (KKT)<br />
conditions, and obtain a mixed integer programming. This will yield a<br />
solution that is proven to be within tolerances of the global optimum.<br />
<br />
<source lang="matlab"><br />
% Input parameters.<br />
% (Q is just a randomly generated symmetric matrix.)<br />
Q = setSymmetric([0.99813;-0.9328;-0.41602;0.75425;0.36975;-0.16011;...<br />
0.20505;-0.22754;0.21575;-0.63597;-0.29342;0.27946;-0.91458;...<br />
-0.33758;0.72371;0.91155;-0.95604;-0.58809;0.11226;-0.60359;...<br />
1.3152;-0.45212;-0.0081002;0.73651;-0.89605;1.0736;1.0053;...<br />
-0.089598;-0.86427;-0.83047;-0.35381;-0.10368;-0.20189;-0.5286;...<br />
0.74354;1.142;0.11788;0.57156;0.16935;-1.854;0.80398;1.0658;...<br />
-1.0921;0.4812;0.17866;0.26736;0.68732;-0.04921;0.51998;...<br />
-0.4118;0.20122;0.62257;-0.48735;-1.2293;-0.27855;0.020442;...<br />
-1.1031;-0.85323;0.084205;1.3643;-0.095637;0.12529;0.071885;...<br />
0.74562;-0.36408;-1.0812;0.32137;0.79026;0.4365;1.536;0.73981;...<br />
1.5732;1.0358;0.86736;0.70902;-1.5217;-0.93981;-0.94568;0.77429;...<br />
-0.77916;1.1569;0.26974;-0.00055197;-0.13087;0.25253;-0.83893;...<br />
0.090814;0.025564;0.3809;-0.079573;0.041911;0.12882;1.0671;...<br />
-0.42695;-1.1162;-0.00038358;0.37305;-0.047864;-0.83988;...<br />
-0.074761;0.11005;0.33679;-0.91941;0.13811;1.2319;-0.077739;...<br />
0.40736;-0.70684;0.24293;-0.64002;1.631;-0.36573;0.34084;...<br />
-0.046399;-0.85519;0.065744;-0.99468;-0.47995;0.91457;-0.30563;...<br />
1.2142;-0.9352;-1.1335;0.32376;-0.68342;0.92591;-0.13294;-0.77732;...<br />
0.41786;-0.38176;1.4631;-0.452;-0.54978;0.91733;-0.17072;0.79189;...<br />
0.22303;0.64006;-0.22713;-0.78101;0.17289;1.4549;-1.1471;-2.4195;...<br />
-0.0197;0.11204;0.41687;0.079953;-0.79564;0.55421;0.084912;...<br />
-0.051431;-0.25787;0.29086;0.14115;-1.1786;-0.62126;-0.97309;...<br />
0.33165;-0.59094;-0.1878;-0.20598;-0.48744;0.16878;-1.0183;...<br />
-0.12057;-0.18849;-0.16661;0.20649;-0.53152;0.81186;0.86475;...<br />
0.43589;-0.68085;0.59335;0.49119;-0.75623;0.21621;-1.1849;...<br />
0.29998;-0.5439;0.3483;-0.8631;0.30066;0.092835;0.97066;0.085208;...<br />
-1.0018;-0.36108;-1.0288;0.28215;0.23805;-0.016555;-1.1906;...<br />
-0.23784;-0.19766;1.0435;0.39938;-0.19682;0.33933;-0.67583;...<br />
-0.34775;-0.1395;0.17759;0.082428;-0.43612;1.6515;-1.2706;...<br />
-0.99358;-1.4966]);<br />
Aeq = [0 1 1 1 1 0 0 0 0 0 1 0 0 1 1 1 1 0 0 0; ...<br />
0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0];<br />
beq = [5;2];<br />
<br />
N = size(Q,1);<br />
<br />
% Lower and upper bounds on x, expressed as A*x <= b<br />
% We could also put arbitrary linear inequalities here.<br />
A = [eye(N); -eye(N)];<br />
b = [ones(N,1); zeros(N,1)];<br />
<br />
M = size(A,1);<br />
<br />
% Upper bound on s and y<br />
% If these values are too small, then no solution will be found.<br />
% Too large numbers may make the solver less efficient, or lead to<br />
% incorrect solutions due to rounding.<br />
bnds = 1; <br />
bndy = 100;<br />
<br />
% Symbolic variables<br />
x = tom('x',N,1);<br />
s = tom('s',M,1); % Slack variables<br />
y = tom('y',M,1); % Lagrange multipliers for inequality constraints<br />
r = tom('r',M,1,'integer'); % 0 for inactive constraints, 1 for active.<br />
yeq = tom('yeq',size(Aeq,1)); % Multipliers for equality constraints<br />
<br />
kktconditions = {<br />
A'*y+Aeq'*yeq+Q*x==0<br />
Aeq*x==beq<br />
A*x+s == b<br />
0 <= y<br />
0 <= s<br />
y <= bndy*r<br />
s <= bnds*(1-r)<br />
};<br />
<br />
% This linear objective will be equal to 0.5*x'*C*x at optimum<br />
obj = -0.5*(y'*b+yeq'*beq);<br />
<br />
%% Solve the problem<br />
options = struct;<br />
options.Prob.MIP.cpxControl.EPGAP = 1e-4; % Tolerance relative to global optimum.<br />
solution = ezsolve(obj,kktconditions,[],options);<br />
</source></div>Perhttp://tomwiki.com/index.php?title=Nonconvex_QP&diff=3073Nonconvex QP2014-09-23T11:59:48Z<p>Per: </p>
<hr />
<div>We want to minimize:<br />
<br />
0.5*x'*Q*x'<br />
<br />
Subject to:<br />
<br />
Aeq*x == beq<br />
0 <= x <= 1<br />
<br />
If Q were positive semidefinite, this would be a Quadratic Programming.<br />
However, Q is indefinite, making the problem much harder. We could easily<br />
find a local optimum using a nonlinear solver, but finding a global<br />
optimum is hard.<br />
<br />
To do this, we set up the first-order Karush-Kuhn-Tucker (KKT)<br />
conditions, and obtain a mixed integer programming. This will yield a<br />
solution that is proven to be within tolerances of the global optimum.<br />
<br />
<source lang="matlab"><br />
% Input parameters.<br />
% (Q is just a randomly generated symmetric matrix.)<br />
Q = setSymmetric([0.99813;-0.9328;-0.41602;0.75425;0.36975;-0.16011;...<br />
0.20505;-0.22754;0.21575;-0.63597;-0.29342;0.27946;-0.91458;...<br />
-0.33758;0.72371;0.91155;-0.95604;-0.58809;0.11226;-0.60359;...<br />
1.3152;-0.45212;-0.0081002;0.73651;-0.89605;1.0736;1.0053;...<br />
-0.089598;-0.86427;-0.83047;-0.35381;-0.10368;-0.20189;-0.5286;...<br />
0.74354;1.142;0.11788;0.57156;0.16935;-1.854;0.80398;1.0658;...<br />
-1.0921;0.4812;0.17866;0.26736;0.68732;-0.04921;0.51998;...<br />
-0.4118;0.20122;0.62257;-0.48735;-1.2293;-0.27855;0.020442;...<br />
-1.1031;-0.85323;0.084205;1.3643;-0.095637;0.12529;0.071885;...<br />
0.74562;-0.36408;-1.0812;0.32137;0.79026;0.4365;1.536;0.73981;...<br />
1.5732;1.0358;0.86736;0.70902;-1.5217;-0.93981;-0.94568;0.77429;...<br />
-0.77916;1.1569;0.26974;-0.00055197;-0.13087;0.25253;-0.83893;...<br />
0.090814;0.025564;0.3809;-0.079573;0.041911;0.12882;1.0671;...<br />
-0.42695;-1.1162;-0.00038358;0.37305;-0.047864;-0.83988;...<br />
-0.074761;0.11005;0.33679;-0.91941;0.13811;1.2319;-0.077739;...<br />
0.40736;-0.70684;0.24293;-0.64002;1.631;-0.36573;0.34084;...<br />
-0.046399;-0.85519;0.065744;-0.99468;-0.47995;0.91457;-0.30563;...<br />
1.2142;-0.9352;-1.1335;0.32376;-0.68342;0.92591;-0.13294;-0.77732;...<br />
0.41786;-0.38176;1.4631;-0.452;-0.54978;0.91733;-0.17072;0.79189;...<br />
0.22303;0.64006;-0.22713;-0.78101;0.17289;1.4549;-1.1471;-2.4195;...<br />
-0.0197;0.11204;0.41687;0.079953;-0.79564;0.55421;0.084912;...<br />
-0.051431;-0.25787;0.29086;0.14115;-1.1786;-0.62126;-0.97309;...<br />
0.33165;-0.59094;-0.1878;-0.20598;-0.48744;0.16878;-1.0183;...<br />
-0.12057;-0.18849;-0.16661;0.20649;-0.53152;0.81186;0.86475;...<br />
0.43589;-0.68085;0.59335;0.49119;-0.75623;0.21621;-1.1849;...<br />
0.29998;-0.5439;0.3483;-0.8631;0.30066;0.092835;0.97066;0.085208;...<br />
-1.0018;-0.36108;-1.0288;0.28215;0.23805;-0.016555;-1.1906;...<br />
-0.23784;-0.19766;1.0435;0.39938;-0.19682;0.33933;-0.67583;...<br />
-0.34775;-0.1395;0.17759;0.082428;-0.43612;1.6515;-1.2706;...<br />
-0.99358;-1.4966]);<br />
Aeq = [0 1 1 1 1 0 0 0 0 0 1 0 0 1 1 1 1 0 0 0; ...<br />
0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0];<br />
beq = [5;2];<br />
<br />
N = size(Q,1);<br />
<br />
% Lower and upper bounds on x, expressed as A*x <= b<br />
% We could also put arbitrary linear inequalities here.<br />
A = [eye(N); -eye(N)];<br />
b = [ones(N,1); zeros(N,1)];<br />
<br />
M = size(A,1);<br />
<br />
% Upper bound on s and y<br />
% If these values are too small, then no solution will be found.<br />
% Too large numbers may make the solver less efficient, or lead to<br />
% incorrect solutions due to rounding.<br />
bnds = 1; <br />
bndy = 100;<br />
<br />
% Symbolic variables<br />
x = tom('x',N,1);<br />
s = tom('s',M,1); % Slack variables<br />
y = tom('y',M,1); % Lagrange multipliers for inequality constraints<br />
r = tom('r',M,1,'integer'); % 0 for inactive constraints, 1 for active.<br />
yeq = tom('yeq',size(Aeq,1)); % Multipliers for equality constraints<br />
<br />
kktconditions = {<br />
A'*y+Aeq'*yeq+Q*x==0<br />
Aeq*x==beq<br />
A*x+s == b<br />
0 <= y<br />
0 <= s<br />
y <= bndy*r<br />
s <= bnds*(1-r)<br />
};<br />
<br />
% This linear objective will be equal to 0.5*x'*C*x at optimum<br />
obj = -0.5*(y'*b+yeq'*beq);<br />
<br />
%% Solve the problem<br />
options = struct;<br />
options.Prob.MIP.cpxControl.EPGAP = 1e-4; % Tolerance relative to global optimum.<br />
solution = ezsolve(obj,kktconditions,[],options);<br />
</source></div>Perhttp://tomwiki.com/index.php?title=Nonconvex_QP&diff=3072Nonconvex QP2014-09-23T11:57:52Z<p>Per: Copy code from tomSym examples collection.</p>
<hr />
<div><source lang="matlab"><br />
% We want to minimize:<br />
%<br />
% 0.5*x'*Q*x'<br />
%<br />
% Subject to:<br />
%<br />
% Aeq*x == beq<br />
% 0 <= x <= 1<br />
%<br />
% If Q were positive semidefinite, this would be a Quadratic Programming.<br />
% However, Q is indefinite, making the problem much harder. We could easily<br />
% find a local optimum using a nonlinear solver, but finding a global<br />
% optimum is hard.<br />
%<br />
% To do this, we set up the first-order Karush-Kuhn-Tucker (KKT)<br />
% conditions, and obtain a mixed integer programming. This will yield a<br />
% solution that is proven to be within tolerances of the global optimum.<br />
<br />
% Input parameters.<br />
% (Q is just a randomly generated symmetric matrix.)<br />
Q = setSymmetric([0.99813;-0.9328;-0.41602;0.75425;0.36975;-0.16011;...<br />
0.20505;-0.22754;0.21575;-0.63597;-0.29342;0.27946;-0.91458;...<br />
-0.33758;0.72371;0.91155;-0.95604;-0.58809;0.11226;-0.60359;...<br />
1.3152;-0.45212;-0.0081002;0.73651;-0.89605;1.0736;1.0053;...<br />
-0.089598;-0.86427;-0.83047;-0.35381;-0.10368;-0.20189;-0.5286;...<br />
0.74354;1.142;0.11788;0.57156;0.16935;-1.854;0.80398;1.0658;...<br />
-1.0921;0.4812;0.17866;0.26736;0.68732;-0.04921;0.51998;...<br />
-0.4118;0.20122;0.62257;-0.48735;-1.2293;-0.27855;0.020442;...<br />
-1.1031;-0.85323;0.084205;1.3643;-0.095637;0.12529;0.071885;...<br />
0.74562;-0.36408;-1.0812;0.32137;0.79026;0.4365;1.536;0.73981;...<br />
1.5732;1.0358;0.86736;0.70902;-1.5217;-0.93981;-0.94568;0.77429;...<br />
-0.77916;1.1569;0.26974;-0.00055197;-0.13087;0.25253;-0.83893;...<br />
0.090814;0.025564;0.3809;-0.079573;0.041911;0.12882;1.0671;...<br />
-0.42695;-1.1162;-0.00038358;0.37305;-0.047864;-0.83988;...<br />
-0.074761;0.11005;0.33679;-0.91941;0.13811;1.2319;-0.077739;...<br />
0.40736;-0.70684;0.24293;-0.64002;1.631;-0.36573;0.34084;...<br />
-0.046399;-0.85519;0.065744;-0.99468;-0.47995;0.91457;-0.30563;...<br />
1.2142;-0.9352;-1.1335;0.32376;-0.68342;0.92591;-0.13294;-0.77732;...<br />
0.41786;-0.38176;1.4631;-0.452;-0.54978;0.91733;-0.17072;0.79189;...<br />
0.22303;0.64006;-0.22713;-0.78101;0.17289;1.4549;-1.1471;-2.4195;...<br />
-0.0197;0.11204;0.41687;0.079953;-0.79564;0.55421;0.084912;...<br />
-0.051431;-0.25787;0.29086;0.14115;-1.1786;-0.62126;-0.97309;...<br />
0.33165;-0.59094;-0.1878;-0.20598;-0.48744;0.16878;-1.0183;...<br />
-0.12057;-0.18849;-0.16661;0.20649;-0.53152;0.81186;0.86475;...<br />
0.43589;-0.68085;0.59335;0.49119;-0.75623;0.21621;-1.1849;...<br />
0.29998;-0.5439;0.3483;-0.8631;0.30066;0.092835;0.97066;0.085208;...<br />
-1.0018;-0.36108;-1.0288;0.28215;0.23805;-0.016555;-1.1906;...<br />
-0.23784;-0.19766;1.0435;0.39938;-0.19682;0.33933;-0.67583;...<br />
-0.34775;-0.1395;0.17759;0.082428;-0.43612;1.6515;-1.2706;...<br />
-0.99358;-1.4966]);<br />
Aeq = [0 1 1 1 1 0 0 0 0 0 1 0 0 1 1 1 1 0 0 0; ...<br />
0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 0];<br />
beq = [5;2];<br />
<br />
N = size(Q,1);<br />
<br />
% Lower and upper bounds on x, expressed as A*x <= b<br />
% We could also put arbitrary linear inequalities here.<br />
A = [eye(N); -eye(N)];<br />
b = [ones(N,1); zeros(N,1)];<br />
<br />
M = size(A,1);<br />
<br />
% Upper bound on s and y<br />
% If these values are too small, then no solution will be found.<br />
% Too large numbers may make the solver less efficient, or lead to<br />
% incorrect solutions due to rounding.<br />
bnds = 1; <br />
bndy = 100;<br />
<br />
% Symbolic variables<br />
x = tom('x',N,1);<br />
s = tom('s',M,1); % Slack variables<br />
y = tom('y',M,1); % Lagrange multipliers for inequality constraints<br />
r = tom('r',M,1,'integer'); % 0 for inactive constraints, 1 for active.<br />
yeq = tom('yeq',size(Aeq,1)); % Multipliers for equality constraints<br />
<br />
kktconditions = {<br />
A'*y+Aeq'*yeq+Q*x==0<br />
Aeq*x==beq<br />
A*x+s == b<br />
0 <= y<br />
0 <= s<br />
y <= bndy*r<br />
s <= bnds*(1-r)<br />
};<br />
<br />
% This linear objective will be equal to 0.5*x'*C*x at optimum<br />
obj = -0.5*(y'*b+yeq'*beq);<br />
<br />
%% Solve the problem<br />
options = struct;<br />
options.Prob.MIP.cpxControl.EPGAP = 1e-4; % Tolerance relative to global optimum.<br />
solution = ezsolve(obj,kktconditions,[],options);<br />
</source></div>Perhttp://tomwiki.com/index.php?title=Solving_the_HJB_equation_with_state_constraints_(source_code)&diff=3071Solving the HJB equation with state constraints (source code)2014-09-15T20:52:12Z<p>Per: Add Matlab code.</p>
<hr />
<div>This code is based on collocation using [[PROPT]], and the [[SNOPT]] nonlinear solver.<br />
<br />
For more information see, ''Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints'' by P. Rutquist et al, in Procedings from the 53rd IEEE Conference on Decision and Control, or the technical report with the same name in the Chalmers Publication Library.<br />
<br />
<source lang="matlab"><br />
%% Solving the HJB equation for a stochastic system with state constraints<br />
<br />
% Problem definition<br />
% xdot = A*x + G*(u+w)<br />
% cost = u'*R*u + x'*Q*x<br />
% noise covariance = W<br />
% xmin < x < xmax<br />
<br />
% For more info:<br />
% http://publications.lib.chalmers.se/publication/195410-solving-the-hamilton-jacobi-bellman-equation-for-a-stochastic-system-with-state-constraints<br />
<br />
%% Define the system<br />
<br />
A = [0 0; 1 0];<br />
G = [1 0; 0 1];<br />
R = [1 0; 0 4];<br />
Q = [1 0; 0 1];<br />
W = [1 0; 0 1];<br />
xmin = [-1; -1];<br />
xmax = [1; 1]; <br />
<br />
Nd = size(A,1);<br />
Nu = size(G,2);<br />
<br />
assert(Nd<=9) % Or names might collide, and probably too big to solve anyway.<br />
<br />
% The number of collocation points in each dimension:<br />
N = [6 6];<br />
<br />
%% Set up symbolic variables<br />
<br />
lambda = tom('lambda');<br />
<br />
xc = cell(1,Nd);<br />
pc = cell(1,Nd);<br />
cc = cell(1,Nd);<br />
iic = cell(1,Nd);<br />
for i=1:Nd<br />
xc{i} = tom(['x' num2str(i)]);<br />
pc{i} = tomPhase(['p' num2str(i)],xc{i},xmin(i),xmax(i)-xmin(i),N(i));<br />
cc{i} = collocate(pc{i},xc{i});<br />
iic{i} = [xmin(i); cc{i}; xmax(i)];<br />
end<br />
<br />
x = vertcat(xc{:});<br />
<br />
Zp = tom('Zp',prod(N),1);<br />
Z = Zp;<br />
<br />
Kpc = cell(Nd,Nd);<br />
Kc = cell(Nd,Nd);<br />
<br />
if size(G,2)==1<br />
QW = W*R*eye(size(G,1)); % Works better<br />
else<br />
QW = G*W*R/G;<br />
end<br />
<br />
for i=1:Nd<br />
for j=1:Nd<br />
Kpc{i,j} = tom(['K' num2str(10*i+j) 'p'],prod(N),1);<br />
Kc{i,j} = Kpc{i,j};<br />
end<br />
end<br />
<br />
for k=1:Nd<br />
Z = interp1p(iic{k},[zeros(1,length(Z)/N(k)); reshape(Z, N(k), length(Z)/N(k)); zeros(1,length(Z)/N(k))], xc{k});<br />
for i=1:Nd<br />
for j=1:Nd<br />
Kc{i,j} = interp1p(iic{k},[zeros(1,length(Kc{i,j})/N(k)); reshape(Kc{i,j}, N(k), length(Kc{i,j})/N(k)); zeros(1,length(Kc{i,j})/N(k))], xc{k});<br />
end<br />
end<br />
end<br />
for i=1:Nd<br />
Kc{i,i} = QW(i,i) + Kc{i,i};<br />
end<br />
<br />
K = reshape(vertcat(Kc{:}),Nd,Nd);<br />
<br />
%% Set up equations<br />
<br />
warning off propt:SlowInterp1pJ2<br />
<br />
gradZ = derivative(Z,x);<br />
hessZ = derivative(gradZ,x);<br />
<br />
gradV = -2*(gradZ*K)./Z;<br />
hessV = derivative(gradV,x);<br />
<br />
% The nonlinear term must equal zero:<br />
eq0 = gradZ*K*G*inv(R)*G'*K'*gradZ' - gradZ*G*W*G'*K'*gradZ' == 0;<br />
<br />
% The Hessian of V must be symmetric:<br />
eqH = cell(Nd-1,Nd-1);<br />
for i=1:Nd-1<br />
for j=i+1:Nd<br />
resH = hessV(i,j)-hessV(j,i);<br />
eqH{i,j-1} = ( Z^2*resH == 0 );<br />
end<br />
end<br />
<br />
% The original HJB residual (for testing purpouses):<br />
resHJB = -lambda + (x'*Q*x) - 0.25*gradV*G*inv(R)*G'*gradV' ...<br />
+ gradV*(A*x) + 0.5*sum(vec(hessV'.*(G*W*G')));<br />
<br />
% The transformed HJB equation:<br />
dK = derivative(K,x)*(G*W*G');<br />
trkron = 0;<br />
for i=1:Nd<br />
trkron = trkron + gradZ*dK((i-1)*Nd+(1:Nd),i);<br />
end<br />
eqHJB = lambda*Z == (x'*Q*x)*Z - 2*gradZ*K*(A*x) ...<br />
- trkron - sum(vec(hessZ.*(K*(G*W*G'))'));<br />
<br />
% Collocate PDE:s<br />
PDEs = {eq0, eqH, eqHJB};<br />
for i=Nd:-1:1<br />
PDEs = collocate(pc{i}, PDEs);<br />
end<br />
<br />
% Norming constraint<br />
nrm = Z.^2;<br />
for i=Nd:-1:1<br />
nrm = integrate(pc{i},nrm);<br />
end<br />
nrmEq = ( nrm == 1 );<br />
<br />
% positivity, and positive definiteness constraints<br />
pos = ( Zp >= 1e-6 );<br />
<br />
%% Diffusing minimization objective<br />
<br />
obj = 0;<br />
for i=1:Nd<br />
for j=1:Nd<br />
obj = obj + sum(derivative(Kc{i,j},x).^2);<br />
end<br />
end<br />
for i=Nd:-1:1<br />
obj = integrate(pc{i},obj);<br />
end<br />
<br />
%% Initial guess<br />
<br />
Kgc = cell(Nd,Nd);<br />
for i=1:Nd<br />
for j=1:Nd<br />
Kgc{i,j} = ( Kpc{i,j} == 0 ); % K11p is the offset that is added to QW(1,1)<br />
end<br />
end<br />
evp = subs(PDEs{3}, Kgc);<br />
AA = sym2eig(lambda, evp);<br />
[VV,DD] = eigs(0.5*(AA+AA'),1,'sa');<br />
VV = VV.*sign(VV(ceil(end/2)));<br />
nrms = sqrt(subs(nrm, Zp == VV));<br />
guess = {<br />
Kgc<br />
Zp == VV./nrms<br />
lambda == DD<br />
};<br />
<br />
%% Solve the problem<br />
<br />
options = struct;<br />
options.solver = 'snopt';<br />
options.Prob.SOL.moremem = 1e6;<br />
<br />
solution = ezsolve(obj,{PDEs, nrmEq},guess,options);<br />
<br />
%% We now have a solution. <br />
%<br />
% The remainder of this script is a stochastic simulation <br />
% using the computed optimal control policy.<br />
<br />
%% Compute the control signal<br />
u = (1/Z)*inv(R)*G'*(derivative(Z,x)*K)';<br />
<br />
% Turn the symbolic control into a matlab function, <br />
% since we will call it many times<br />
ufun = 'ufun';<br />
ufile = [ufun '.m'];<br />
udata = mfile(subs(u,solution),ufile);<br />
rehash<br />
<br />
%% Do stochastic simulation<br />
<br />
x_0 = zeros(length(x),1);<br />
<br />
dt = 0.001; % time step<br />
t_0 = 0; % initial time<br />
t_f = 10; % final time<br />
<br />
t = (t_0:dt:t_f);<br />
Nt = length(t);<br />
disp(['Simulating ' num2str(Nt-1) ' timesteps...']);<br />
<br />
sqrtW = W^0.5; % Matrix square root (not element-wise).<br />
<br />
% pre-allocate arrays<br />
xv = zeros(length(x),Nt);<br />
uv = zeros(length(u),Nt);<br />
<br />
xv(:,1) = x_0; % initial state<br />
<br />
xx = cell(1,length(x));<br />
for i=1:Nt-1<br />
for k=1:length(x)<br />
xx{k} = xv(k,i);<br />
end<br />
uv(:,i) = feval(ufun, xx{:}, udata);<br />
xv(:,i+1) = xv(:,i) + dt*(A*xv(:,i) + G*uv(:,i)) + sqrt(dt)*G*sqrtW*randn(size(W,1),1);<br />
% The following assertation may fail due to a combination of bad luck<br />
% and large dt<br />
assert(all(xmin<=xv(:,i+1)) && all(xv(:,i+1)<=xmax));<br />
end<br />
<br />
%% Plot results<br />
<br />
subplot(2,1,1);<br />
plot(t',xv');<br />
xl = cell(1,length(x));<br />
for k=1:length(x)<br />
xl{k} = ['x_' num2str(k)];<br />
end<br />
legend(xl);<br />
<br />
subplot(2,1,2);<br />
plot(t',uv');<br />
ul = cell(1,length(u));<br />
for k=1:length(u)<br />
ul{k} = ['u_' num2str(k)];<br />
end<br />
legend(ul);<br />
<br />
%% Compute cost<br />
% The cost will be different each time, because the noise is generated at<br />
% random, but for small dt and large t_f, the cost should approach lambda.<br />
<br />
cst = zeros(1,Nt-1);<br />
for i=1:Nt-1<br />
cst(i) = xv(:,i)'*Q*xv(:,i) + uv(:,i)'*R*uv(:,i);<br />
end<br />
<br />
disp(['mean cost = ' num2str(mean(cst))]);<br />
disp(['lambda = ' num2str(solution.lambda)]);<br />
<br />
%% Delete temporary m-code file<br />
delete(ufile);<br />
<br />
</source><br />
<br />
[[Category: PROPT]]</div>Perhttp://tomwiki.com/index.php?title=Solving_the_HJB_equation_with_state_constraints_(source_code)&diff=3070Solving the HJB equation with state constraints (source code)2014-09-14T07:59:05Z<p>Per: </p>
<hr />
<div>Source code for solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints will soon appear here. The code is based on collocation using [[PROPT]], and the [[SNOPT]] nonlinear solver.<br />
<br />
For more information see, ''Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints'' by P. Rutquist et al, in Procedings from the 53rd IEEE Conference on Decision and Control, or the technical report with the same name in the Chalmers Publication Library.<br />
[[Category: PROPT]]</div>Perhttp://tomwiki.com/index.php?title=Hjb&diff=3069Hjb2014-09-14T07:58:07Z<p>Per: Redirected page to Solving the HJB equation with state constraints (source code)</p>
<hr />
<div>#REDIRECT [[Solving the HJB equation with state constraints (source code)]]</div>Perhttp://tomwiki.com/index.php?title=Solving_the_HJB_equation_with_state_constraints_(source_code)&diff=3068Solving the HJB equation with state constraints (source code)2014-09-13T20:34:32Z<p>Per: </p>
<hr />
<div>Source code for solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints will soon appear here. The code is based on collocation using [[PROPT]], and the [[SNOPT]] nonlinear solver.<br />
<br />
For more information see, ''Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints'' by P. Rutquist et al, in Procedings from the 53rd IEEE Conference on Decision and Control, or the technical report with the same name at Chalmers University.<br />
<br />
[[Category: PROPT]]</div>Perhttp://tomwiki.com/index.php?title=HJB&diff=3067HJB2014-09-13T20:25:08Z<p>Per: redirect to "Solving the HJB equation with state constraints (source code)"</p>
<hr />
<div>#REDIRECT [[Solving the HJB equation with state constraints (source code)]]</div>Perhttp://tomwiki.com/index.php?title=HJB_with_state_constraints&diff=3066HJB with state constraints2014-09-13T20:24:33Z<p>Per: moved HJB with state constraints to Solving the HJB equation with state constraints (source code): Better title</p>
<hr />
<div>#REDIRECT [[Solving the HJB equation with state constraints (source code)]]</div>Perhttp://tomwiki.com/index.php?title=Solving_the_HJB_equation_with_state_constraints_(source_code)&diff=3065Solving the HJB equation with state constraints (source code)2014-09-13T20:24:33Z<p>Per: moved HJB with state constraints to Solving the HJB equation with state constraints (source code): Better title</p>
<hr />
<div>The source code that is referred to in the paper entitled "Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints" will soon appear here.<br />
<br />
[[Category: PROPT]]</div>Perhttp://tomwiki.com/index.php?title=HBJ_with_state_constraints&diff=3064HBJ with state constraints2014-09-13T10:36:02Z<p>Per: moved HBJ with state constraints to HJB with state constraints</p>
<hr />
<div>#REDIRECT [[HJB with state constraints]]</div>Perhttp://tomwiki.com/index.php?title=Solving_the_HJB_equation_with_state_constraints_(source_code)&diff=3063Solving the HJB equation with state constraints (source code)2014-09-13T10:36:02Z<p>Per: moved HBJ with state constraints to HJB with state constraints</p>
<hr />
<div>The source code that is referred to in the paper entitled "Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints" will soon appear here.<br />
<br />
[[Category: PROPT]]</div>Perhttp://tomwiki.com/index.php?title=Solving_the_HJB_equation_with_state_constraints_(source_code)&diff=3062Solving the HJB equation with state constraints (source code)2014-09-13T10:35:01Z<p>Per: Create placeholder page.</p>
<hr />
<div>The source code that is referred to in the paper entitled "Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints" will soon appear here.<br />
<br />
[[Category: PROPT]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2941TomSym compared to the Symbolic Toolbox2013-11-25T09:28:22Z<p>Per: /* Comparing tomSym to the symbolic toolbox */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved (until the matrices become so large that the time for numeric computations begins to matter.)<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
symbolic expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2940TomSym compared to the Symbolic Toolbox2013-11-25T09:27:47Z<p>Per: /* Comparing tomSym to the symbolic toolbox */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved (until the matrices become so large that the time for numeric computations begins to matter.)<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2939TomSym compared to the Symbolic Toolbox2013-11-25T09:27:35Z<p>Per: /* Comparing tomSym to the symbolic toolbox */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved (until the matrices become so large that the time for numeric computations begin to matter.)<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2938TomSym compared to the Symbolic Toolbox2013-11-25T09:19:24Z<p>Per: /* Timing measurments */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. (This is perhaps not a fair comparison, because quadprog has to do its own pre-scaling, while KNITRO gets a pre-scaled problem from tomSym.) In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2937TomSym compared to the Symbolic Toolbox2013-11-25T09:16:39Z<p>Per: /* Solution */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical (within the tolerances of the solvers).<br />
<br />
Interestingly, the objective value reported by the quadprog solver does not match the value obtained from substituting<br />
the solution back into the objective function. The command<br />
<source lang="matlab"><br />
(subs(Rtot,X,x3)-fval3)./fval3<br />
</source><br />
computes the relative error. This error should be zero, or on the order of machine precision: 1e-16,<br />
but it is on the order of 1e-4. This seems to be due to rounding errors in the gradient and Hessian as computed by the Symbolic Toolbox.<br />
The value obtained by "subs(Rtot,X,x3)" is very close to the optimum value returned by the TOMLAB solvers (within 1e-10 or so).<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2936TomSym compared to the Symbolic Toolbox2013-11-22T13:18:49Z<p>Per: /* Timing measurments */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to slightly better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2935TomSym compared to the Symbolic Toolbox2013-11-22T13:17:47Z<p>Per: /* Code comparison */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to slightly better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions or numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=File:Linesofcode.png&diff=2934File:Linesofcode.png2013-11-22T13:05:15Z<p>Per: uploaded a new version of &quot;File:Linesofcode.png&quot;</p>
<hr />
<div>Graph of the number of lines of code in the hydroelctric dam examle. Symbolic toolbox vs tomsym.</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2933TomSym compared to the Symbolic Toolbox2013-11-22T12:55:15Z<p>Per: </p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running thousands of times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to slightly better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2932TomSym compared to the Symbolic Toolbox2013-11-22T12:54:05Z<p>Per: /* Code comparison */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to slightly better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate inequalities, but the ezsolve function accepts inequalities chained in this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2931TomSym compared to the Symbolic Toolbox2013-11-22T12:44:52Z<p>Per: </p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using [[tomSym]] and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to slightly better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2930TomSym compared to the Symbolic Toolbox2013-11-22T12:42:02Z<p>Per: /* Problem description */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The code and data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to slightly better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2929TomSym compared to the Symbolic Toolbox2013-11-22T12:39:12Z<p>Per: /* Solution */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to slightly better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2928TomSym compared to the Symbolic Toolbox2013-11-22T12:38:18Z<p>Per: /* Footnote */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.4 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2927TomSym compared to the Symbolic Toolbox2013-11-22T12:37:01Z<p>Per: /* Timing measurments */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.3 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2926TomSym compared to the Symbolic Toolbox2013-11-22T12:36:11Z<p>Per: /* Timing measurments */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in ''2.1 seconds'', and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant for problems this small.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.3 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2925TomSym compared to the Symbolic Toolbox2013-11-22T12:34:53Z<p>Per: /* Footnote */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in ''2.1 seconds'', and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200 (corresponding to ca 4.3 months of data), a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2924TomSym compared to the Symbolic Toolbox2013-11-22T12:32:31Z<p>Per: /* Timing measurments */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in ''2.1 seconds'', and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below. (N is the number of time steps. The number of decision variables is 2N.)<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200, a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
This corresponds to ca 4.3 months of data, for which DeLand's code needed half an hour with a manually coded Hessian.<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2923TomSym compared to the Symbolic Toolbox2013-11-22T12:32:01Z<p>Per: /* Timing measurments */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over ''5.1 hours'' to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in ''2.1 seconds'', and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. ''An overall speedup of a factor 3 500.'' <br />
<br />
In to understand the difference between TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using smaller problem sizes. <br />
(In the below talbes, N is the number of time steps. The number of decision variables is 2N.)<br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below.<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200, a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
This corresponds to ca 4.3 months of data, for which DeLand's code needed half an hour with a manually coded Hessian.<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2922TomSym compared to the Symbolic Toolbox2013-11-22T12:27:43Z<p>Per: /* Footnote */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over 5.1 hours to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. A speedup of a factor 3 500. <br />
<br />
In order to compare TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using different problem sizes. <br />
(N is the number of time steps. The number of decision variables is 2N.)<br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below.<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
With tomSym it is very easy to to this kind of modification.<br />
All we need to do is to replace the expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, <br />
but still makes both the symbolic processsing and the numeric solver run much faster.<br />
<br />
For N=3200, a solution was obtained 6.4 seconds. (5.4 seconds symbolic processing and 1 second for the KNITRO numeric solver.)<br />
This corresponds to ca 4.3 months of data, for which DeLand's code needed half an hour with a manually coded Hessian.<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2921TomSym compared to the Symbolic Toolbox2013-11-22T07:52:25Z<p>Per: /* Timing measurments */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over 5.1 hours to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. A speedup of a factor 3 500. <br />
<br />
In order to compare TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using different problem sizes. <br />
(N is the number of time steps. The number of decision variables is 2N.)<br />
Each test was run 5 times, and the mean of the middle 3 is listed in the table below.<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
All we need to do is to replace the previous expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, but still makes the solver run much faster.<br />
For N=480, a solution was obtained in less than a second. - A speedup of a factor 5 compared to the previous result.<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2920TomSym compared to the Symbolic Toolbox2013-11-21T19:59:14Z<p>Per: /* tomSym code */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'knitro';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over 5.1 hours to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. A speedup of a factor 3 500. <br />
<br />
In order to compare TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using different problem sizes. <br />
(N is the number of time steps. The number of decision variables is 2N.)<br />
Each test was run 5 times, and the mean of the middle 3 results was used.<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
All we need to do is to replace the previous expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, but still makes the solver run much faster.<br />
For N=480, a solution was obtained in less than a second. - A speedup of a factor 5 compared to the previous result.<br />
<br />
[[Category:tomSym]]</div>Perhttp://tomwiki.com/index.php?title=TomSym_compared_to_the_Symbolic_Toolbox&diff=2919TomSym compared to the Symbolic Toolbox2013-11-21T19:56:28Z<p>Per: /* Code comparison */</p>
<hr />
<div>[[File:Gordon Dam.jpg|thumb|A hydroelectric dam.]]<br />
<br />
We look at the optimization problem of maximizing the revenue of a hydroelectric dam.<br />
This problem was published by By Seth DeLand of the MathWorks, as an example of how to combine their optimization and symbolic toolboxes. [http://www.mathworks.com/company/newsletters/articles/solving-large-scale-optimization-problems-with-matlab-a-hydroelectric-flow-example.html]<br />
We solve the same probem using tomSym and the TOMLAB solvers, noting a significant speed difference, with tomSym running 9 000 times faster than the Symbolic Toolbox on a problem with 960 decision variables.<br />
<br />
== Problem description ==<br />
<br />
The dynamics of the dam is described by the following equations.<br />
<br />
Electricity(t) = TurbineFlow(t-1)*[½ k1(Storage(t) + Storage(t-1)) + k2]<br />
<br />
Storage(t) = Storage(t-1) + ∆t * [inFlow(t-1) - spillFlow(t-1) - turbineFlow(t-1)]<br />
<br />
The problem is to maximize revenue (electricity times price), subject to the following constraints:<br />
* 0 < Turbine Flow < 25,000 CFS<br />
* 0 < Spill Flow<br />
* Max change in Turbine and Spill Flow < 500 CFS<br />
* Combined Turbine and Spill Flow > 500 CFS<br />
* 50000 < Reservoir Storage < 100000 Acre-Feet<br />
* The final reservoir level must equal the initial level (90000 AF)<br />
<br />
The actual data used by DeLand can be downloaded from Matlab Central.[http://www.mathworks.com/matlabcentral/fileexchange/35856-optimization-in-matlab-an-introduction-to-quadratic-programming]<br />
<br />
== tomSym code ==<br />
<br />
To avoid attaching data files to our example, we will use an approximation to this data, as generated by the following Matlab code:<br />
<source lang="matlab"><br />
N = 480;<br />
t = (0:N-1)';<br />
inFlow = reshape(repmat(10*(107+[0 0 0 -3 3 2 2 -8 9 5 0 1 -6 -4 5 0 -6 -6 -17 -7]),24,1),N,1);<br />
price = 46 + t./57 - 3*cos((t+46)/43) - 4*sin(pi/12*(t+4)) - 4*sin(pi/6*(t+1));<br />
</source><br />
<br />
A few other consants are defined by DeLand as follows:<br />
<source lang="matlab"><br />
stor0 = 90000; % initial vol. of water stored in the reservoir (Acre-Feet)<br />
<br />
k1 = 0.00003; % K-factor coefficient<br />
k2 = 9; % K-factor offset<br />
<br />
MW2kW = 1000; % MW to kW<br />
C2A = 1.98347/24; % Convert from CFS to AF/HR<br />
</source><br />
<br />
One the input data is defined, this is the entire code used to set up and solve the problem with TOMLAB:<br />
<source lang="matlab"><br />
turbineFlow = tom('turbineFlow',N,1);<br />
spillFlow = tom('spillFlow',N,1);<br />
<br />
outFlow = turbineFlow + spillFlow;<br />
<br />
storage = stor0 + C2A*cumsum(inFlow - outFlow);<br />
<br />
electricity = turbineFlow.*(0.5*k1*(storage + [stor0; storage(1:end-1)]) + k2)./MW2kW;<br />
<br />
revenue = price*electricity; <br />
<br />
objective = -revenue; % Minimization objective<br />
<br />
constraints = {<br />
0 <= turbineFlow <= 25000<br />
0 <= spillFlow<br />
-500 <= diff(outFlow) <= 500<br />
outFlow >= 500<br />
50000 <= storage <= 100000<br />
storage(end) == stor0<br />
};<br />
<br />
guess = [];<br />
<br />
options = struct;<br />
options.solver = 'snopt';<br />
options.scale = 'auto'<br />
<br />
solution = ezsolve(objective, constraints, guess, options);<br />
</source><br />
<br />
== Solution ==<br />
<br />
[[File:Hydroelectric plots a.png|thumb|Plots of the result, when the approximate indata is used.]]<br />
<br />
Although the Hessian is numeric, this is <em>not</em> a quadratic programming.<br />
The Hessian is not positive semidefinite, and the objective is therefore not convex. There may exist<br />
local optima that are not as good as the global one. For this reason, the QP solvers in the<br />
TOMLAB suite return error messages, and a nonlinear solver had to be used instead.<br />
<br />
The solution for the approximate input data is very similar to the one presented by DeLand. <br />
With identical input data, the solutions are of course nearly identical. (The TOMLAB solvers<br />
give a slightly worse objective value due to better constraint adherence.)<br />
<br />
== Code comparison ==<br />
[[File:linesofcode.png|thumb|DeLand's example has about twice as much code compared to the tomSym example.]]<br />
<br />
One thing that is immediately noticeable when comparing the above code to DeLand's is that it is much shorter.<br />
After removing comments, blank lines and input data generation, the above code is about half the length<br />
of the part of DeLand's code that is needed for setting up and solving the "quadratic programming".<br />
Shorter code tends to be easier to write and maintain, assuming that readability is the same.<br />
<br />
It is of course difficult to make a fair comparison of code lengths, since it is largely influenced by writing style.<br />
Still, there are a few things about tomSym that makes it inherently more suited for optimization than the symbolic toolbox.<br />
<br />
For example, consider the following lines from the constraint definitions by DeLand:<br />
<source lang="matlab"><br />
% 50000 <= Stor(t) <= 100,000 AF<br />
% Stor(t) = s0+sum(inFlow(1:t))-sum(spillFlow(1:t))-sum(turbineFlow(1:t))<br />
c = stor0 + C2A*cumsum(inFlow); % Convert CFS to AF<br />
b = [b; 100000-c; -50000+c];<br />
s = -C2A*sparse(tril(ones(N)));<br />
s = [s s];<br />
A = [A; s; -s];<br />
</source><br />
These five lines of code would be very hard to understand without the two lines of comments that accompany them.<br />
Even with the comments, it is not obvious how the constraint works.<br />
<br />
With tomSym all of the above is replaced by a single line in the constraints array:<br />
<source lang="matlab"><br />
50000 <= storage <= 100000<br />
</source><br />
(This is an abuse of Matlab syntax. It should be written as two separate expressions, but the ezsolve function accepts inequalities<br />
written this way, and it is a commonly used mathematical notation.)<br />
<br />
This single line of code is clearly easier to read, write and debug than the seven lines above. <br />
Constraints can be written in this way because the ezsolve function automatically parses linear constraints and generates the necessary numeric matrices for the solver. In the same manner it also automatically generates automatic first and second derivatives of both objective and constraints, and passes these to the solver as matlab functions of numeric data depending on the problem type.<br />
<br />
== Timing measurments ==<br />
<br />
The symbolic toolbox needed a little over 5.1 hours to compute the symbolic Hessian for 480 time steps (960 decision variables).<br />
The quadprog solver then solved the numeric problem in about 10 seconds.<br />
<br />
With TOMLAB, the symbolic Hessian was computed in 2.1 seconds, and the KNITRO numeric solver required 3.3 seconds, for a total<br />
of 5.4 seconds. A speedup of a factor 3 500. <br />
<br />
In order to compare TOMLAB to the Mathworks' toolboxes, we ran a series of tests, using different problem sizes. <br />
(N is the number of time steps. The number of decision variables is 2N.)<br />
Each test was run 5 times, and the mean of the middle 3 results was used.<br />
<br />
{| class="wikitable"<br />
|+ Time for symbolic processing<br />
|-<br />
! N || Symbolic TB !! tomSym<br />
|-<br />
| 10 || 0.698 s || 0.502 s<br />
|-<br />
| 50 || 12.86 s || 0.502 s<br />
|-<br />
| 100 || 67.49 s || 0.518 s<br />
|-<br />
| 150 || 237.0 s || 0.564 s<br />
|}<br />
<br />
The time for deriving the Hessian using the symbolic toolbox increased approximately as N^2.7 as the number of variables increased.<br />
With tomSym, on the other hand, the time remained approximately constant.<br />
<br />
{| class="wikitable"<br />
|+ Time for numeric solver<br />
|-<br />
! N || quadprog !! KNITRO<br />
|-<br />
| 10 || 0.0162 s || 0.0165 s<br />
|-<br />
| 50 || 0.0452 s || 0.0318 s<br />
|-<br />
| 100 || 0.2015 s || 0.0829 s<br />
|-<br />
| 150 || 0.4756 s || 0.1847 s<br />
|}<br />
<br />
KNITRO runs at about the same speed as quadprog on the smallest problem sizes, and about 2.5 times faster for larger problems. In every case, the time for the numeric solver is much smaller than the time for symbolic processing for problems this small.<br />
<br />
== Comparing tomSym to the symbolic toolbox ==<br />
<br />
The main reason why tomSym is so much faster than the symbolic toolbox is that it can handle vector and matrix symbols.<br />
This means that symbolic manipulations of matrix expressions are handled at approximately the same speed, regardless of the sizes of the symbolic matrices involved.<br />
<br />
The symbolic toolbox, on the other hand, only handles scalar symbols. This means that both the number of<br />
expressions and their sizes increase as the size of symbolic matrices increase.<br />
The matlab code generated by the symbolic toolbox also becomes quite lengthy because of the use of multiple<br />
scalar operations instead of vector/matrix operations.<br />
<br />
== Footnote ==<br />
<br />
The above code passes the same Hessian to the solver as DeLand's code.<br />
It is possible to get a much sparser Hessian, by making the water level a decision variable instead of a symbolic expression.<br />
All we need to do is to replace the previous expression for "storage" by a symbol definition:<br />
<source lang="matlab"><br />
storage = tom('storage',N,1);<br />
</source><br />
and then add one constraint<br />
<source lang="matlab"><br />
constraints{end+1} = ( storage == [stor0; storage(1:N-1)] + inFlow - outFlow );<br />
</source><br />
<br />
This modification increases the number of decision variables from 2N to 3N, but still makes the solver run much faster.<br />
For N=480, a solution was obtained in less than a second. - A speedup of a factor 5 compared to the previous result.<br />
<br />
[[Category:tomSym]]</div>Per