PENOPT
Contents
Introduction
Overview
Welcome to the TOMLAB /PENOPT User's Guide. TOMLAB /PENOPT includes either PENSDP or PENBMI (depends on the license) and interfaces between The MathWorks' MATLAB and the solver packages from PENOPT GbR. The package includes one of the following solvers:
PENBMI  For large, sparse semidefinite programming problems with linear and bilinear matrix inequality constraints.
PENSDP  For large, sparse linear semidefinite programming problems with linear constraints. It also solves feasibility problems for systems of linear matrix inequalities.
Please visit http://tomopt.com/tomlab/products/pensdp/, http://tomopt.com/tomlab/products/penbmi/ and http://www.penopt.com for more information.
For PENBMI two different input formats may be used for the problem formulation: The PENBMI Structural Format, an extension of the PENSDP format for linear problems, and the TOMLAB format for semidefinite prob lems. Apart from solving the BMI problem, the user can check feasibility of the system of linear and bilinear matrix inequalities.
For PENSDP three different input formats may be used for problem formulation: The standard sparse SDPA format used in SDPLIB, the PENSDP Structural Format, and the TOMLAB format for semidefinite problems.
Problems defined in SeDuMi Matlab format may easily be converted to SDPA format and solved by TOMLAB /PENSDP. The conversion routine, called writesdp, was written by Brian Borcher.
Apart from solving the SDP problem, the user can check feasibility of the system of linear matrix inequalities. The interface between TOMLAB /PENOPT, Matlab and TOMLAB consists of two layers. The first layer gives direct access from Matlab to PENOPT, via calling a Matlab function that calls a precompiled MEX file (DLL under Windows, shared library in UNIX) that defines and solves the problem in PENOPT. The second layer is a Matlab function that takes the input in the TOMLAB format, and calls the first layer function. On return the function creates the output in the TOMLAB format.
Contents of this Manual
 #Using the Matlab Interface gives the basic information needed to run the Matlab interface.
 #PENOPT Solver Reference provides all the solver references for PENBMI and PENSDP.
Prerequisites
In this manual we assume that the user is familiar with semidefinite programming, TOMLAB and the Matlab language.
Using the Matlab Interface
The main routines in the twolayer design of the interface are shown in #Table: The interface routines.. Page and section references are given to detailed descriptions on how to use the routines.
Table: The interface routines.
Function  Description 

penbmiTL  The layer two interface routine called by the TOMLAB driver routine tomRun. This routine then calls penbmi.m, which in turn calls penbmi.dll. 
penfeas_bmi  The layer two TOMLAB interface routine that calls pen.m. Converts the input P rob format before calling penbmiQ.dll and converts back to the output Result structure. 
pensdpTL  The layer two interface routine called by the TOMLAB driver routine tomRun. This routine then calls pensdp.m, which in turn calls pensdp.dll. 
penfeas_sdp  The layer two TOMLAB interface routine that calls pen.m. Converts the input P rob format before calling pensdp.dll and converts back to the output Result structure. 
The PENOPT control parameters are possible to set from Matlab.
They can be set as inputs to the interface routine penbmiTL for example and the others. The user sets fields in a structure called Prob.PENOPT. The following example shows how to set the maximum number of overall iterations.
Prob.PENOPT.ioptions(2) = 200; % Setting maximum number of iterations
PENOPT Solver Reference
The PENOPT solvers are a set of solvers that were developed by the PENOPT GbR. Table 2 lists the solvers included in TOMLAB /PENOPT. The solvers are called using a set of MEXfile interfaces developed as part of TOMLAB. All functionality of the PENOPT solvers are available and changeable in the TOMLAB framework in Matlab.
Detailed descriptions of the TOMLAB /PENOPT solvers are given in the following sections. Also see the Mfile help for each solver.
Additional solvers reference guides for the TOMLAB /PENOPT solvers are available for download from the TOM LAB home page http://tomopt.com. Extensive TOMLAB mfile help is also available, for example help penbmiTL in Matlab will display the features of the PENBMI solver using the TOMLAB format.
TOMLAB /PENOPT solves
The linear semidefinite programming problem with linear matrix inequalities is defined as
where , , and are symmetric matrices of similar dimensions in each constraint . If there are several LMI constraints, each may have it's own dimension.
The linear semidefinite programming problem with linear matrix inequalities (sdp) is defined as are symmetric matrices of similar dimensions in each constraint i. If there are several LMI constraints, each may have it's own dimension.
The linear semidefinite programming problem with bilinear matrix inequalities (bmi)
The MEX solvers pensdp and penbmi treat sdp and bmi problems, respectively. These are available in the TOMLAB /PENSDP and TOMLAB /PENBMI toolboxes.
The MEXfile solver pensdp available in the TOMLAB /PENSDP toolbox implements a penalty method aimed at largescale dense and sparse sdp problems. The interface to the solver allows for data input in the sparse SDPA input format as well as a TOMLAB specific format.
The MEXfile solver penbmi available in the TOMLAB /PENBMI toolbox is similar to pensdp, with added support for the bilinear matrix inequalities.
The addon toolbox TOMLAB /PENOPT solves linear semidefinite programming problems with linear and bilinear matrix inequalities. The solvers are listed in Table 2. They are written in a combination of Matlab and C code.
Function  Description 

penbmi  Sparse and dense linear semidefinite programming using a penalty algorithm. 
penfeas_sdp  Feasibility check of systems of linear matrix inequalities, using pensdp. 
pensdp  Sparse and dense linear semidefinite programming using a penalty algorithm. 
penfeas_sdp  Feasibility check of systems of linear matrix inequalities, using pensdp. 
PENBMI
TOMLAB /PENBMI was developed in cooperation with PENOPT GbR. The following pages describe the solvers and the feasibility checks.
penfeas_bmi
Purpose
penfeas_bmi checks the feasibility of a system of bilinear matrix inequalities (BMI):
with symmetric matrices
Calling Syntax
[ifeas, feas, xfeas] = penfeas bmi(p, x_0, options)
Description of Inputs
Input  Description  

p  PENBMI format structure, e.g. from sdpa2pen.  
x_0  Inital guess of the feasible point. Default = 0.  
options  Optional 3 × 1 vector. May be omitted, in which case the defaults below are used.

Description of Outputs
Output  Description  

ifeas  Feasibility of the system:
 
feas  Value of the minimized maximal eigenvalue of BMI.  
xfeas  Feasible point 
Algorithm
The feasibility of the system of BMI's is checked by solving the following bmi problem:
where x_{bound}is taken from input argument options(2), or if options(2) < 0.
When , the system is strictly feasible; when , the system is feasible but has no interior point. When for any x_{bound}, the system may be infeasible.
As the semidefinite problem solved is nonconvex and penbmiQ is a local method, it cannot be guaranteed that the solution is a global minimum. Consequently, the (local) optimum found by penbmiQ may be a positive number (indicating infeasibility), although the system is feasible (i.e., the global optimum ). In such a case, the user may try various initial points x _0, and also different weighting parameter w (option(3)).
In practice, the conditions on feasibility are as follows: means strict feasibility, means feasibility (not strict) and indicates infeasibility. The user can decide by the actual value of the output parameter feas (including the optimal ).
For the case when the BMI system is unbounded, we have to add artificial bounds on x, otherwise PENBMI might diverge. Of course, it may happen that the feasible point lies outside these bounds. In this case penfeas_bmi claims infeasibility, although the system is actually feasible. When in doubts, the user may try to gradually increase x_{bound} (parameter options(2)).
On the other hand, for very illconditioned and unbounded systems, the default bound 1000 may be too large and PENBMI may not converge. In this case, the user is advised either to increase the weighting parameter w (to 0.01 and bigger) or to decrease the bound (parameter options(2)) to a smaller number (1001).
Mfiles Used
pen.m
MEXfiles Used
penbmiQ
See Also
penfeas sdp.m
penbmiTL
Purpose
Solve (linear) semidefinite programming problems.
penbmiTL solves problems of the form
where and are sparse or dense symmetric matrices. The matrix sizes may vary between different matrix inequalities but must be the same in each particular constraint.
Calling Syntax
Result = tomRun('penbmi',Prob,...)
Description of Inputs
Prob Problem description structure. The following fields are used:
Input  Description  

QP.c  Vector with coefficients for linear objective function.  
A  Linear constraints matrix.  
b_L  Lower bound for linear constraints.  
b_U  Upper bound for linear constraints.  
x_L  Lower bound on variables.  
x_U  Upper bound on variables.  
x_0  Starting point.  
PENOPT  Structure with special fields for SDP parameters. Fields used are:  
LMI  Structure array with matrices for the linear terms of the matrix inequalities.
See Examples on page 17 for a discussion of how to set this correctly.  
ioptions  8 × 1 vector with options, defaults in (). Any element set to a value less than zero will be replaced by a default value, in some cases fetched from standard Tomlab parameters.
 
foptions  1 × 7 vector with options, defaults in ().

Description of Outputs
Result Structure with result from optimization. The following fields are changed:
Output  Description 

x_k  Optimal point. 
f_k  Function value at optimum. 
g_k  Gradient value at optimum, c. 
v_k  Lagrange multipliers. 
x_0  Starting point. 
f_0  Function value at start. 
xState  State of each variable, described in the TOMLAB User's Guide. 
Iter  Number of iterations. 
ExitFlag  0: OK.
1: Maximal number of iterations reached. 2: Unbounded feasible region. 3: Rank problems. Can not find any solution point. 4: Illegal x 0. 5: No feasible point x 0 found. 
Inform  If ExitF lag > 0, I nf orm = ExitF lag. 
QP.B  Optimal active set. See input variable QP.B. 
Solver  Solver used. 
SolverAlgorithm  Solver algorithm used. 
FuncEv  Number of function evaluations. Equal to Iter. ConstrEv Number of constraint evaluations. Equal to Iter. Prob Problem structure used. 
Description
pensdp implements a penalty algorithm based on the PBM method of BenTal and Zibulevsky. It is possible to give input data in three different formats:
 Standard sparse SPDA format
 PENSDP Structural format
 Tomlab Quick format
In all three cases, problem setup is done via sdpAssign.
See Also
sdpAssign.m, sdpa2pen.m, sdpDemo.m, tomlab/docs/penbmi.pdf
Warnings
Currently penbmi does not work well solving problems of sdp type.
Examples
Setting the LMI constraints is best described by an example. Assume 3 variables x = (x_{1}, x_{2} , x_{3}) and 2 linear matrix inequalities of sizes 3 × 3 and 2 × 2 respectively, here given on blockdiagonal form:
The LMI structure could then be initialized with the following commands:
% Constraint 1 >> LMI(1,1).Q0 = []; >> LMI(1,1).Q = [2 1 0 ; ... 0 2 0 ; ... 0 0 2]; >> LMI(1,2).Q = []; >> LMI(1,3).Q = [2 0 1 ; ... 0 2 0 ; ... 0 0 2]; % Constraint 2, diagonal matrices only >> LMI(2,1).Q0 = diag( [0, 1] ); >> LMI(2,1).Q = diag( [1,1] ); >> LMI(2,2).Q = diag( [3,3] ); >> LMI(2,3).Q = [ ]; % Use LMI in call to sdpAssign: >> Prob=sdpAssign(c,LMI,...) % ... or set directly into Prob.PENSDP.LMI field: >> Prob.PENSDP.LMI = LMI;
Some points of interest:
 The LMI structure must be of correct size. This is important if a LMI constraint has zero matrices for the highest numbered variables. If the above example had zero coefficient matrices for x3 , the user would have to set LMI(1,3).Q = [] explicitly, so that the LMI structure array is really 2 × 3. (LMI(2,3).Q would automatically become empty in this case, unless set otherwise by the user).
 MATLAB sparse format is allowed and encouraged.
 Only the upper triangular part of each matrix is used (symmetry is assumed).
Input in Sparse SDPA Format is handled by the conversion routine sdpa2pen. For example, the problem defined in tomlab/examples/arch0.dats can be solved using the following statements:
>> p = sdpa2pen('arch0.dats') p = vars: 174 fobj: [1x174 double] constr: 174 ci: [1x174 double] bi_dim: [1x174 double] bi_idx: [1x174 double] bi_val: [1x174 double] mconstr: 1 ai_dim: 175 ai_row: [1x2874 double] ai_col: [1x2874 double] ai_val: [1x2874 double] msizes: 161 ai_idx: [175x1 double] ai_nzs: [175x1 double] x0: [1x174 double] ioptions: 0 foptions: [] >> Prob = sdpAssign(p); % Can call sdpAssign with only 'p' structure >> Result = tomRun('pensdp',Prob); % Call tomRun to solve problem
PENSDP
TOMLAB /PENSDP was developed in cooperation with PENOPT GbR. The following pages describe the solvers and the feasibility checks.
penfeas_sdp
Purpose
penfeas_sdp checks the feasibility of a system of linear matrix inequalities (LMI):
with symmetric matrices
and
Calling Syntax
[ifeas, feas, xfeas] = penfeas sdp(p, options)
Description of Inputs
Input  Description 

p  PENSDP format structure, e.g. from sdpa2pen. 
options  Optional 2 × 1 vector. May be omitted, in which case the defaults below are used. 
options(1)  Output level of PENSDP solver. Default = 0. 
options(2)  Upper bound on x 8 . Default = 1000.
If < 0, no box bounds are applied. 
Description of Outputs
Output  Description 

ifeas  Feasibility of the system: 
0  System is strictly feasible 
1  System is feasible but not necessarily strictly feasible 
feas  Value of the minimized maximal eigenvalue of LMI. 
xfeas  Feasible point 
Algorithm
The feasibility of the system of LMI's is checked by solving the following sdp problem:
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_0^i + \sum{k=1}{n} A_{k}^{i} x_{k} \preccurlyeq \lambda I_{n \times n}, i=1,\ldots,m }
where xbound is taken from input argument options(2), or Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \infty} if options(2) < 0.
When Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lambda < 0} , the system is strictly feasible; when ? = 0, the system is feasible but has no interior point.
When Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lambda=0} for any Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_{bound}} , the system is infeasible.
In practice, the conditions on feasibility are as follows: Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lambda < 10^{6}} means strict feasibility, Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lambda<10^{6}} means feasibility (not strict) and Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lambda > 10^{6}} indicates infeasibility.
The user can decide by the actual value of the output parameter feas (including the optimal Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \lambda} ).
For the case when the LMI system is unbounded, we have to add artificial bounds on x, otherwise PENSDP might diverge. Of course, it may happen that the feasible point lies outside these bounds. In this case penfeas sdp claims infeasibility, although the system is actually feasible. When in doubts, the user may try to gradually increase xbound (parameter options(2)). On the other hand, for very illconditioned and unbounded systems, the default bound 1000 may be too large and PENSDP may not converge. In this case, the user is advised to decrease parameter options(2) to a smaller number (1001).
Mfiles Used
pen.m
MEXfiles Used
pensdp
See Also
penfeas bmi.m
pensdpTL
Purpose
Solve (linear) semidefinite programming problems.
pensdpTL solves problems of the form
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \begin{array}{ccccccl} \min\limits_{x} & f(x) = c^{T} x & \\ \\ s/t & x_L & \leq & x & \leq & x_U & \\ {} & b_L & \leq & Ax & \leq & b_U & \\ \\ & Q^{(0)_i + \sum_{k=1}^{n} Q^{(i)}_k x_k} & \preceq & 0, & k=1,2,\ldots,m_{LMI} \\ \end{array} }
where Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x,x_L,x_U \in \mathbb{R}^{n},\, A \in \mathbb{R}^{m_l \times n},\, b_L,b_U \in \mathbb{R}^{m_l}} and Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Q^{(i)}_k} are sparse or dense symmetric matrices. The matrix sizes may vary between different linear matrix inequalities (LMI) but must be the same in each particular constraint.
Calling Syntax
Result = tomRun('pensdp',Prob,...)
Description of Inputs
Prob Problem description structure. The following fields are used:
Input  Description  

QP.c  Vector with coefficients for linear objective function.  
A  Linear constraints matrix.  
b_L  Lower bound for linear constraints.  
b_U  Upper bound for linear constraints.  
x_L  Lower bound on variables.  
x_U  Upper bound on variables.  
x_0  Starting point.  
PriLevOpt  Print level in pensdpTL and MEX interface. The print level in the solver is controlled by PENOPT.ioptions(4).  
PENOPT  Structure with special fields for SDP parameters. Fields used are:  
LMI  Structure array with matrices for the linear matrix inequalities. See Examples on page 17 for a discussion of how to set this correctly.  
ioptions  8 × 1 vector with options, defaults in (). Any element set to a value less than zero will be replaced by a default value, in some cases fetched from standard Tomlab parameters.
 
foptions  7 × 1 vector with optimization parameters, defaults in ():

Description of Outputs
Result Structure with result from optimization. The following fields are changed:
Output  Description 

x_k  Optimal point. 
f_k  Function value at optimum. 
g_k  Gradient value at optimum, c. 
v_k  Lagrange multipliers. 
x_0  Starting point. 
f_0  Function value at start. 
xState  State of each variable, described in the TOMLAB User's Guide. 
Iter  Number of iterations. 
ExitFlag  0: OK.
1: Maximal number of iterations reached. 2: Unbounded feasible region. 3: Rank problems. Can not find any solution point. 4: Illegal x 0. 5: No feasible point x 0 found. 
Inform  If ExitF lag > 0, I nf orm = ExitF lag. 
QP.B  Optimal active set. See input variable QP.B. 
Solver  Solver used. 
SolverAlgorithm  Solver algorithm used. 
FuncEv  Number of function evaluations. Equal to Iter. ConstrEv Number of constraint evaluations. Equal to Iter. Prob Problem structure used. 
Description
pensdp implements a penalty algorithm based on the PBM method of BenTal and Zibulevsky. It is possible to give input data in three different formats:
 Standard sparse SPDA format
 PENSDP Structural format
 Tomlab Quick format
In all three cases, problem setup is done via sdpAssign.
See Also
sdpAssign.m, sdpa2pen.m, sdpDemo.m, tomlab/docs/pensdp.pdf, penfeas sdp.m
Examples
Setting the LMI constraints is best described by an example. Assume 3 variables Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x=(x_1,x_2,x_3)} and 2 linear matrix inequalities of sizes 3 × 3 and 2 × 2 respectively, here given on blockdiagonal form:
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left( \begin{array}{cc} \begin{array}{ccc} 0 \\ & 0 \\ && 0 \end{array}\\ \hline & \begin{array}{cc} 0 \\ & 1 \end{array} \end{array} \right) + \left( \begin{array}{cc} \begin{array}{ccc} 2 & 1 & 0 \\ & 2 & 0 \\ && 2\end{array} \\ \hline & \begin{array}{cc} 1 \\ & 1 \end{array} \end{array} \right) x_1 + \left( \begin{array}{cc} \begin{array}{ccc} 0 \\ & 0 \\ && 0 \end{array} \\ \hline & \begin{array}{cc} 3 \\ & 3 \end{array} \end{array} \right) x_2 + \left( \begin{array}{cc} \begin{array}{ccc} 2 & 0 &1 \\ & 2 & 0 \\ && 2\end{array} \\ \hline & \begin{array}{cc} 0 \\ & 0 \end{array} \end{array} \right)x_3 \preceq 0 }
The LMI structure should then be initialized with the following commands:
% Constraint 1 >> LMI(1,1).Q0 = []; >> LMI(1,1).Q = [2 1 0 ; ... 0 2 0 ; ... 0 0 2]; >> LMI(1,2).Q = []; >> LMI(1,3).Q = [2 0 1 ; ... 0 2 0 ; ... 0 0 2]; % Constraint 2, diagonal matrices only >> LMI(2,1).Q0 = diag( [0, 1] ); >> LMI(2,1).Q = diag( [1,1] ); >> LMI(2,2).Q = diag( [3,3] ); >> LMI(2,3).Q = [ ]; % Use LMI in call to sdpAssign: >> Prob=sdpAssign(c,LMI,...) % ... or set directly into Prob.PENSDP.LMI field: >> Prob.PENSDP.LMI = LMI;
Some points of interest:
 The LMI structure must be of correct size. This is important if a LMI constraint has zero matrices for the highest numbered variables. If the above example had zero coefficient matrices for x3 , the user would have to set LMI(1,3).Q = [] explicitly, so that the LMI structure array is really 2 × 3. (LMI(2,3).Q would automatically become empty in this case, unless set otherwise by the user).
 MATLAB sparse format is allowed and encouraged.
 Only the upper triangular part of each matrix is used (symmetry is assumed).
Input in Sparse SDPA Format is handled by the conversion routine sdpa2pen. For example, the problem defined in tomlab/examples/arch0.dats can be solved using the following statements:
>> p = sdpa2pen('arch0.dats') p = vars: 174 fobj: [1x174 double] constr: 174 ci: [1x174 double] bi_dim: [1x174 double] bi_idx: [1x174 double] bi_val: [1x174 double] mconstr: 1 ai_dim: 175 ai_row: [1x2874 double] ai_col: [1x2874 double] ai_val: [1x2874 double] msizes: 161 ai_idx: [175x1 double] ai_nzs: [175x1 double] x0: [1x174 double] ioptions: 0 foptions: [] >> Prob=sdpAssign(p); % Can call sdpAssign with only 'p' structure >> Result=tomRun('pensdp',Prob); % Call tomRun to solve problem