TOMLAB Solving Least Squares and Parameter Estimation Problems - Revision history

Bjorn: /* Table: Exponential fitting test problems. */

2014-08-05T14:18:56Z

Table: Exponential fitting test problems.

← Older revision		Revision as of 09:18, 5 August 2014
Line 249:		Line 249:
	====Table: Exponential fitting test problems.====		====Table: Exponential fitting test problems.====

	{\| class="wikitable"		{\| class="wikitable" style="background-color:#ffffff;"
	!Function		!Function
	!Description		!Description

Bjorn: /* Table: Exponential models treated in TOMLAB. */

2014-08-05T14:18:41Z

Table: Exponential models treated in TOMLAB.

← Older revision		Revision as of 09:18, 5 August 2014
Line 231:		Line 231:
	====Table: Exponential models treated in TOMLAB.====		====Table: Exponential models treated in TOMLAB.====

	{\| class="wikitable" cellpadding="5"		{\| class="wikitable" cellpadding="5" style="background-color:#ffffff;"
	\|<math>f(t) = \sum\limits_i^p \alpha_i e^{-\beta_i t}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.		\|<math>f(t) = \sum\limits_i^p \alpha_i e^{-\beta_i t}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.
	\|-		\|-

Bjorn: /* Table: Constrained nonlinear least squares (cls) test problems. */

2014-08-05T14:18:18Z

Table: Constrained nonlinear least squares (cls) test problems.

← Older revision		Revision as of 09:18, 5 August 2014
Line 199:		Line 199:
	====Table: Constrained nonlinear least squares (cls) test problems.====		====Table: Constrained nonlinear least squares (cls) test problems.====

	{\| class="wikitable"		{\| class="wikitable" style="background-color:#ffffff;"
	!Function		!Function
	!Description		!Description

Elias at 18:10, 14 January 2012

2012-01-14T18:10:06Z

Show changes

Elias at 11:05, 10 January 2012

2012-01-10T11:05:37Z

← Older revision		Revision as of 06:05, 10 January 2012
Line 103:		Line 103:
	==Nonlinear Least Squares Problems==		==Nonlinear Least Squares Problems==

	This section shows examples how to define and solve nonlinear least squares problems using the TOMLAB format. As a first illustration, the example ''ls1Demo ''in file ''lsDemo ''shows how to fit a nonlinear model of exponential type with three unknown parameters to experimental data. This problem, ''Gisela'', is also defined as problem three in ''ls_prob''. A weighting parameter ''K ''is sent to the residual and Jacobian routine using the ''Prob ''structure. The solver ''clsSolve ''is called directly. Note that the user only defines the routine to compute the residual vector and the Jacobian matrix of derivatives. TOMLAB has special routines ''ls f'', ''ls g ''and ''ls H ''that computes the nonlinear least squares objective function value, given the residuals, as well as the gradient and the approximative Hessian, see ~~<xr id="tab~~:~~clsProb" />~~. The residual routine for this problem is defined in file ''ls1_r ''in the directory ''example'' with the statements		This section shows examples how to define and solve nonlinear least squares problems using the TOMLAB format. As a first illustration, the example ''ls1Demo ''in file ''lsDemo ''shows how to fit a nonlinear model of exponential type with three unknown parameters to experimental data. This problem, ''Gisela'', is also defined as problem three in ''ls_prob''. A weighting parameter ''K ''is sent to the residual and Jacobian routine using the ''Prob ''structure. The solver ''clsSolve ''is called directly. Note that the user only defines the routine to compute the residual vector and the Jacobian matrix of derivatives. TOMLAB has special routines ''ls f'', ''ls g ''and ''ls H ''that computes the nonlinear least squares objective function value, given the residuals, as well as the gradient and the approximative Hessian, see [[#Table: Constrained nonlinear least squares (cls) test problems.]]. The residual routine for this problem is defined in file ''ls1_r ''in the directory ''example'' with the statements

	<syntaxhighlight lang="matlab">		<syntaxhighlight lang="matlab">
Line 195:		Line 195:
	</syntaxhighlight>		</syntaxhighlight>

	~~<xr id="tab~~:~~clsProb" />~~ describes the low level routines and the initialization routines needed for the predefined constrained nonlinear least squares ('''cls''') test problems. Similar routines are needed for the nonlinear least squares ('''ls''') test problems (here no constraint routines are needed).		[[#Table: Constrained nonlinear least squares (cls) test problems.]] describes the low level routines and the initialization routines needed for the predefined constrained nonlinear least squares ('''cls''') test problems. Similar routines are needed for the nonlinear least squares ('''ls''') test problems (here no constraint routines are needed).

			====Table: Constrained nonlinear least squares (cls) test problems.====

	~~<figtable id="tab:clsProb">~~
	{\| class="wikitable"		{\| class="wikitable"
	~~\|+<caption>Constrained nonlinear least squares (cls) test problems.</caption>~~
	\|-
	!Function		!Function
	!Description		!Description
Line 225:		Line 224:
	\|''ls_H''\|\|General routine to compute the Hessian approximation <math>H(x) = J(x)^T * J(x)</math> for nonlinear least squares type of problems.		\|''ls_H''\|\|General routine to compute the Hessian approximation <math>H(x) = J(x)^T * J(x)</math> for nonlinear least squares type of problems.
	\|}		\|}
	~~</figtable>~~

	==Fitting Sums of Exponentials to Empirical Data==		==Fitting Sums of Exponentials to Empirical Data==

			In TOMLAB the problem of fitting sums of positively weighted exponential functions to empirical data may be formulated either as a nonlinear least squares problem or a separable nonlinear least squares problem. Several empirical data series are predefined and artificial data series may also be generated. There are five different types of exponential models with special treatment in TOMLAB, shown in [[#Table: Exponential models treated in TOMLAB.]]. In research in cooperation with Todd Walton, Vicksburg, USA, TOMLAB has been used to estimate parameters using maximum likelihood in simulated Weibull distributions, and Gumbel and Gamma distributions with real data. TOMLAB has also been useful for parameter estimation in stochastic hydrology using real-life data.

	In TOMLAB the problem of fitting sums of positively weighted exponential functions to empirical data may be formulated either as a nonlinear least squares problem or a separable nonlinear least squares problem. Several empirical data series are predefined and artificial data series may also be generated. There are five different types of exponential models with special treatment in TOMLAB, shown in <xr id=~~"tab~~:~~expModels" />. In research~~ in ~~cooperation with Todd Walton, Vicksburg, USA,~~ TOMLAB has been used to estimate parameters using maximum likelihood in simulated Weibull distributions, and Gumbel and Gamma distributions with real data. TOMLAB has also been useful for parameter estimation in stochastic hydrology using real-life data.		====Table: Exponential models treated in TOMLAB.====

	~~<figtable id="tab:expModels">~~
	{\| class="wikitable" cellpadding="5"		{\| class="wikitable" cellpadding="5"
	~~\|+<caption>Exponential models treated in TOMLAB.</caption>~~
	\|-
	\|<math>f(t) = \sum\limits_i^p \alpha_i e^{-\beta_i t}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.		\|<math>f(t) = \sum\limits_i^p \alpha_i e^{-\beta_i t}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.
	\|-		\|-
Line 247:		Line 244:
	\|<math>f(t) = \sum\limits_i^p t \alpha_i e^{-\beta_i (t - \gamma_i)}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.		\|<math>f(t) = \sum\limits_i^p t \alpha_i e^{-\beta_i (t - \gamma_i)}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.
	\|}		\|}
	~~</figtable>~~

	Algorithms to find starting values for different number of exponential terms are implemented. Test results show that these initial value algorithms are very close to the true solution for equidistant problems and fairly good for non-equidistant problems. Good initial values are extremely important when solving real life exponential fitting problems, because they are so ill-conditioned. ~~<xr id="tab~~:~~expFitting" />~~ shows the relevant routines.		Algorithms to find starting values for different number of exponential terms are implemented. Test results show that these initial value algorithms are very close to the true solution for equidistant problems and fairly good for non-equidistant problems. Good initial values are extremely important when solving real life exponential fitting problems, because they are so ill-conditioned. [[#Table: Exponential fitting test problems.]] shows the relevant routines.

			====Table: Exponential fitting test problems.====

	~~<figtable id="tab:expFitting">~~
	{\| class="wikitable"		{\| class="wikitable"
	~~\|+<caption>Exponential fitting test problems.</caption>~~
	\|-
	!Function		!Function
	!Description		!Description
Line 286:		Line 281:
	\|''exp_p''\|\|Find optimal number of exponential terms, ''p''.		\|''exp_p''\|\|Find optimal number of exponential terms, ''p''.
	\|}		\|}
	~~</figtable>~~

	==Large Scale LS problems with Tlsqr==		==Large Scale LS problems with Tlsqr==
Line 298:		Line 292:
	The normal mode of operation of ''Tlsqr ''is that memory for the ''A ''matrix is allocated and deallocated each time the solver is called. In a real-life situation with a very large ''A ''and where the solver is called repeatedly, this may become inefficient and even cause problems getting memory because of memory fragmenting.		The normal mode of operation of ''Tlsqr ''is that memory for the ''A ''matrix is allocated and deallocated each time the solver is called. In a real-life situation with a very large ''A ''and where the solver is called repeatedly, this may become inefficient and even cause problems getting memory because of memory fragmenting.

	The ''Tlsqr ''solver provides a parameter ''Alloc'', given as the second element of the first input parameter to control the memory handling. The possible values of ''Alloc ''and their meanings are given in ~~<xr id~~=~~"tab~~:~~tlsqr" />.~~		The ''Tlsqr ''solver provides a parameter ''Alloc'', given as the second element of the first input parameter to control the memory handling. The possible values of ''Alloc ''and their meanings are given in [[#Table: Alloc values for Tlsqr]].

			====Table: ''Alloc ''values for ''Tlsqr''====

	~~<figtable id="tab:tlsqr">~~
	{\| class="wikitable"		{\| class="wikitable"
	~~\|+<caption>''Alloc ''values for ''Tlsqr''</caption>~~
	\|-
	!Alloc (m(2))		!Alloc (m(2))
	!Meaning		!Meaning
Line 319:		Line 312:
	\|5\|\|Deallocate only, no results returned		\|5\|\|Deallocate only, no results returned
	\|}		\|}
	~~</figtable>~~

	An example of the calling sequence is given below.		An example of the calling sequence is given below.

Elias at 12:53, 6 December 2011

2011-12-06T12:53:56Z

← Older revision		Revision as of 07:53, 6 December 2011
Line 206:		Line 206:
	\|''cls_prob''\|\|Initialization of '''cls '''test problems.		\|''cls_prob''\|\|Initialization of '''cls '''test problems.
	\|-		\|-
	\|''cls_r''\|\|Compute the residual vector <math>r_i(x), i = 1,...,m.~ x \in \~~MATHSET~~{R}^n</math> ''for '''cls ''' test problems.		\|''cls_r''\|\|Compute the residual vector <math>r_i(x), i = 1,...,m.~ x \in \mathbb{R}^n</math> ''for '''cls ''' test problems.
	\|-		\|-
	\|''cls_J''\|\|Compute the Jacobian matrix <math>J_{ij}(x)=\partial r_i / d x_j,		\|''cls_J''\|\|Compute the Jacobian matrix <math>J_{ij}(x)=\partial r_i / d x_j,
Line 235:		Line 235:
	\|+<caption>Exponential models treated in TOMLAB.</caption>		\|+<caption>Exponential models treated in TOMLAB.</caption>
	\|-		\|-
	\|<math>$f(t) = \sum\limits_i^p \alpha_i e^{-\beta_i t}$</math>, \|\| <math>$\alpha_i \geq 0$</math>, \|\| <math>$0\leq\beta_1<\beta_2< ... <\beta_p$</math>.		\|<math>f(t) = \sum\limits_i^p \alpha_i e^{-\beta_i t}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.
	\|-		\|-
	\|<math>$f(t) = \sum\limits_i^p \alpha_i e^{-\beta_i t}$</math>, \|\| <math>$\alpha_i \geq 0$</math>, \|\| <math>$0\leq\beta_1<\beta_2< ... <\beta_p$</math>.		\|<math>f(t) = \sum\limits_i^p \alpha_i e^{-\beta_i t}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.
	\|-		\|-
	\|<math>$f(t) = \sum\limits_i^p \alpha_i(1-e^{-\beta_i t})$</math>, \|\| <math>$\alpha_i \geq 0$</math>, \|\| <math>$0\leq\beta_1<\beta_2< ... <\beta_p$</math>.		\|<math>f(t) = \sum\limits_i^p \alpha_i(1-e^{-\beta_i t})</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.
	\|-		\|-
	\|<math>$f(t) = \sum\limits_i^p t \alpha_i e^{-\beta_i t}$</math>, \|\| <math>$\alpha_i \geq 0$</math>, \|\| <math>$0\leq\beta_1<\beta_2< ... <\beta_p$</math>.		\|<math>f(t) = \sum\limits_i^p t \alpha_i e^{-\beta_i t}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.
	\|-		\|-
	\|<math>$f(t) = \sum\limits_i^p (t \alpha_i-\gamma_i) e^{-\beta_i t}$</math>, \|\| <math>$\alpha_i,\gamma_i \geq 0$</math>, \|\| <math>$0\leq\beta_1<\beta_2< ... >\beta_p$</math>.		\|<math>f(t) = \sum\limits_i^p (t \alpha_i-\gamma_i) e^{-\beta_i t}</math>, \|\| <math>\alpha_i,\gamma_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... >\beta_p</math>.
	\|-		\|-
	\|<math>$f(t) = \sum\limits_i^p t \alpha_i e^{-\beta_i (t - \gamma_i)}$</math>, \|\| <math>$\alpha_i \geq 0$</math>, \|\| <math>$0\leq\beta_1<\beta_2< ... <\beta_p$</math>.		\|<math>f(t) = \sum\limits_i^p t \alpha_i e^{-\beta_i (t - \gamma_i)}</math>, \|\| <math>\alpha_i \geq 0</math>, \|\| <math>0\leq\beta_1<\beta_2< ... <\beta_p</math>.
	\|}		\|}
	</figtable>		</figtable>
Line 270:		Line 270:
	\|''Helax_prob''\|\|Defines 335 medical research problems supplied by Helax AB, Uppsala, Sweden, where an exponential model is fitted to data. The actual data series (''t, y'') are stored on one file each, i.e. 335 data files, 8MB large, and are not distributed. A sample of five similar files are part of ''exp prob''.		\|''Helax_prob''\|\|Defines 335 medical research problems supplied by Helax AB, Uppsala, Sweden, where an exponential model is fitted to data. The actual data series (''t, y'') are stored on one file each, i.e. 335 data files, 8MB large, and are not distributed. A sample of five similar files are part of ''exp prob''.
	\|-		\|-
	\|''exp_r''\|\|Compute the residual vector <math>r_i(x), i = 1,...,m.~ x \in \~~MATHSET~~{R}^n</math>		\|''exp_r''\|\|Compute the residual vector <math>r_i(x), i = 1,...,m.~ x \in \mathbb{R}^n</math>
	\|-		\|-
	\|''exp_J''\|\|Compute the Jacobian matrix <math>\partial r_i / d x_j, i=1,...,m, j=1,...,n</math>		\|''exp_J''\|\|Compute the Jacobian matrix <math>\partial r_i / d x_j, i=1,...,m, j=1,...,n</math>

Elias at 17:04, 17 July 2011

2011-07-17T17:04:18Z

Elias at 16:18, 17 July 2011

2011-07-17T16:18:55Z

Show changes

Elias at 15:24, 17 July 2011

2011-07-17T15:24:24Z

Show changes

Elias at 14:47, 8 July 2011

2011-07-08T14:47:55Z

← Older revision		Revision as of 09:47, 8 July 2011
Line 3:		Line 3:
	{{cleanup\|Clean this article.}}		{{cleanup\|Clean this article.}}

	{{Part Of Manual\|title=the TOMLAB Manual\|link=[[TOMLAB Manual]]}}		{{Part Of Manual\|title=the TOMLAB Manual\|link=[[TOMLAB\|TOMLAB Manual]]}}

	==Solving Least Squares and Parameter Estimation Problems==		==Solving Least Squares and Parameter Estimation Problems==

← Older revision		Revision as of 12:04, 17 July 2011
Line 206:		Line 206:
	\|''cls_prob''\|\|Initialization of '''cls '''test problems.		\|''cls_prob''\|\|Initialization of '''cls '''test problems.
	\|-		\|-
	\|''cls_r''\|\|Compute the residual vector ~~''ri ''~~(''x'')'', i ''= 1'', ..., m. x ~~? ''~~R''n ''for '''cls ''' test problems.		\|''cls_r''\|\|Compute the residual vector <math>r_i(x), i = 1,...,m.~ x \in \MATHSET{R}^n</math> ''for '''cls ''' test problems.
	\|-		\|-
	\|''cls_J''\|\|Compute the Jacobian matrix ~~''Jij ''~~(''x'') = ~~''?ri~~ /~~dxj~~ , i ''= 1'', ..., m, j ''= 1'', ..., n ''for '''cls ''' test problems.		\|''cls_J''\|\|Compute the Jacobian matrix <math>J_{ij}(x)=\partial r_i / d x_j,
			i=1,...,m, j=1,...,n</math> ''for '''cls ''' test problems.
	\|-		\|-
	\|''cls_c''\|\|Compute the vector of constraint functions ''c''(''x'') for '''cls '''test problems.		\|''cls_c''\|\|Compute the vector of constraint functions ''c''(''x'') for '''cls '''test problems.
	\|-		\|-
	\|''cls_dc''\|\|Compute the matrix of constraint normals ~~''?~~c''(''x'')''/dx ''for '''cls '''test problems.		\|''cls_dc''\|\|Compute the matrix of constraint normals <math>\partial
			c(x)/dx</math> ''for '''cls '''test problems.
	\|-		\|-
	\|''cls_d2c''\|\|Compute the second part of the second derivative of the Lagrangian function for '''cls '''test problems.		\|''cls_d2c''\|\|Compute the second part of the second derivative of the Lagrangian function for '''cls '''test problems.
	\|-		\|-
	\|''ls_f''\|\|General routine to compute the objective function value ''f ''(''x'') = 1 ''r''(''x'')''T r''(''x'') for nonlinear least squares type of problems.		\|''ls_f''\|\|General routine to compute the objective function value <math>f(x) =
			{1\over2} r(x)^T r(x)</math> for nonlinear least squares type of problems.
	\|-		\|-
	\|''ls_g''\|\|General routine to compute the gradient ''g''(''x'') = ''J ''(''x'')''T r''(''x'') for nonlinear least squares type of problems.		\|''ls_g''\|\|General routine to compute the gradient <math>g(x) = J(x)^T r(x)</math> for nonlinear least squares type of problems.
	\|-		\|-
	\|''ls_H''\|\|General routine to compute the Hessian approximation ''H ''(''x'') = ''J ''(''x'')''T *J ''(''x'') for nonlinear least squares type of problems.		\|''ls_H''\|\|General routine to compute the Hessian approximation <math>H(x) = J(x)^T * J(x)</math> for nonlinear least squares type of problems.
	\|}		\|}
	</figtable>		</figtable>
Line 259:		Line 262:
	\|''exp_ArtP''\|\|Generate artificial exponential sum problems.		\|''exp_ArtP''\|\|Generate artificial exponential sum problems.
	\|-		\|-
	\|''expInit''\|\|Find starting values for the exponential parameters ~~''?''~~.		\|''expInit''\|\|Find starting values for the exponential parameters <math>\lambda</math>.
	\|-		\|-
	\|''expSolve''\|\|Solve exponential fitting problems.		\|''expSolve''\|\|Solve exponential fitting problems.
Line 267:		Line 270:
	\|''Helax_prob''\|\|Defines 335 medical research problems supplied by Helax AB, Uppsala, Sweden, where an exponential model is fitted to data. The actual data series (''t, y'') are stored on one file each, i.e. 335 data files, 8MB large, and are not distributed. A sample of five similar files are part of ''exp prob''.		\|''Helax_prob''\|\|Defines 335 medical research problems supplied by Helax AB, Uppsala, Sweden, where an exponential model is fitted to data. The actual data series (''t, y'') are stored on one file each, i.e. 335 data files, 8MB large, and are not distributed. A sample of five similar files are part of ''exp prob''.
	\|-		\|-
	\|''exp_r''\|\|Compute the residual vector ~~''ri ''~~(''x'')'', i ''= 1'', ..., m. x ~~? ''~~R''n''		\|''exp_r''\|\|Compute the residual vector <math>r_i(x), i = 1,...,m.~ x \in \MATHSET{R}^n</math>
	\|-		\|-
	\|''exp_J''\|\|Compute the Jacobian matrix ~~''?ri~~ /~~dxj~~ , i ''= 1'', ..., m, j ''= 1'', ..., n~~''.~~		\|''exp_J''\|\|Compute the Jacobian matrix <math>\partial r_i / d x_j, i=1,...,m, j=1,...,n</math>
	\|-		\|-
	\|''exp_d2r''\|\|Compute the 2nd part of the second derivative for the nonlinear least squares exponential fitting problem.		\|''exp_d2r''\|\|Compute the 2nd part of the second derivative for the nonlinear least squares exponential fitting problem.
	\|-		\|-
	\|''exp_c''\|\|Compute the constraints ~~''?''1 ''~~< ~~?''2 ''~~< ... ''on the exponential parameters ~~''?i~~ , i ''= 1'', ..., p''.		\|''exp_c''\|\|Compute the constraints <math>\lambda_1 < \lambda_2 < ...</math> on the exponential parameters <math>\lambda_i, i=1,...,p</math>.
	\|-		\|-
	\|''exp_dc''\|\|Compute matrix of constraint normals for constrained exponential fitting problem.		\|''exp_dc''\|\|Compute matrix of constraint normals for constrained exponential fitting problem.
	\|-		\|-
	\|''exp_d2c''\|\|Compute second part of second derivative matrix of the Lagrangian function for ~~con- strained~~ exponential fitting problem. This is a zero matrix, because the constraints are linear.		\|''exp_d2c''\|\|Compute second part of second derivative matrix of the Lagrangian function for constrained exponential fitting problem. This is a zero matrix, because the constraints are linear.
	\|-		\|-
	\|''exp_q''\|\|Find starting values for exponential parameters ~~''?i~~ , i ''= 1'', ..., p''.		\|''exp_q''\|\|Find starting values for exponential parameters <math>\lambda_i, i=1,...,p</math>.
	\|-		\|-
	\|''exp_p''\|\|Find optimal number of exponential terms, ''p''.		\|''exp_p''\|\|Find optimal number of exponential terms, ''p''.