Debugger and Profiler
The Mosel Debugger
In Chapter Correcting errors in Mosel models we have seen how the Mosel Parser helps detect syntax errors during compilation. Other types of errors that are in general more difficult to analyze are mistakes in the data or logical errors in the formulation of Mosel models. The Mosel Debugger may help tracing these.
Using the Mosel Debugger
In this section we shall be working with the model prime2.mos. This is the same model for calculating prime numbers as the example we have seen in Section Working with sets, but with a LIMIT value set to 20,000.
Mosel models that are to be run in the debugger need to be compiled with the option G. After compiling and loading the model, the debugger is started with the command debug:
mosel cl -G prime.mos debugand terminated by typing quit (type quit a second time to terminate Mosel). Just as for the run command the user may specify new settings for the model parameters immediately following the debug command:
mosel cl -G prime2.mos debug 'LIMIT=50'Once the debugger is started, type in the following sequence of commands (Mosel's output is highlighted in bold face):
dbg> break 30
Breakpoint 1 set at prime2.mos:30
dbg> bcond 1 getsize(SNumbers) < 10
dbg> cont
Prime numbers between 2 and 50:
Breakpoint 1.
30 repeat
dbg> print n
13
dbg> display SNumbers
1: SNumbers = [17 19 23 29 31 37 41 43 47]
dbg> display SPrime
2: SPrime = [2 3 5 7 11 13]
dbg> cont
Breakpoint 1.
30 repeat
1: SNumbers = [19 23 29 31 37 41 43 47]
2: SPrime = [2 3 5 7 11 13 17]
dbg> cont
Breakpoint 1.
30 repeat
1: SNumbers = [23 29 31 37 41 43 47]
2: SPrime = [2 3 5 7 11 13 17 19]
dbg> cont
Breakpoint 1.
30 repeat
1: SNumbers = [29 31 37 41 43 47]
2: SPrime = [2 3 5 7 11 13 17 19 23]
dbg> quit
> quit
Exiting.This small example uses many of the standard debugging commands (for a complete list, including commands for navigating in the Mosel stack that are not shown here, please see the Section 'Running Mosel – Command line interpreter: debugger' of the introduction chapter of the Mosel Language Reference Manual):
- break
- Set a breakpoint in the given line. A breakpoint is deleted with delete followed by the breakpoint number. The command breakpoints lists all currently defined breakpoints.
- bcond
- Set a condition on a breakpoint (using the number of the breakpoint returned by the break command). Conditions are logical expressions formed according to the standard rules in Mosel (use of brackets, connectors and and or). They may contain any of the functions listed below.
- cont
- Continue the execution up to the next breakpoint (or to the end of the program). A line-wise evaluation is possible by using next or step (the former jumps over loops and subroutines, the latter steps into them).
- display
- Show the current value of a model object or an expression at every step of the debugger. A display is removed by calling undisplay followed by the number of the display.
- Show (once) the current value of a model object.
The following simple Mosel functions may be used with debugger commands (in conditions or with print / display):
- Arithmetic functions: abs, ceil, floor, round
- Accessing solution values: getsol, getdual, getrcost, getactivity, getslack
- Other: getparam, getsize
Debugger in IVE
With Xpress-IVE the debugger is started by selecting Debug
Start debugger or by clicking on the button
. IVE will automatically recompile the model with the required debugging flag. Navigating in the debugger is possible using the entries of the debug menu or by clicking on the corresponding buttons:
Set/delete breakpoint at the cursor. Start/stop the debugger. Step over an expression. Step into an expression. Run up to the cursor. Show the debugger options dialog. Efficient modeling through the Mosel Profiler
The efficiency of a model may be measured through its execution speed and also its memory consumption. The execution times can be analyzed in detail with the help of the Mosel Profiler. Other commands of the Mosel command line interpreter that are also discussed in this section provide the user with further information, such as memory consumption.
Using the Mosel Profiler
Once a model you are developing is running correctly, you will certainly start testing it with larger data sets. Doing so, you may find that model execution times become increasingly larger. This may be due to the solution algorithms, but a more or less significant part of the time will be spent simply in defining the model itself. The Mosel Profiler lets you analyze the model behavior line-by-line. Solution algorithms, such as LP or MIP optimization with Xpress-Optimizer, may be improved by tuning solver parameters (please refer to the corresponding software manuals). Here we shall be solely concerned with improvements that may be made directly to the Mosel model. Even for large scale models, model execution times can often be reduced to just a few seconds by carefully (re)formulating the model.
Just as for the debugger, Mosel models that are to be run in the profiler need to be compiled with the option G. After compiling and loading the model, the profiler is started with the command profile:
mosel cl -G prime2.mos profile quitor, as a single line:
mosel -c "cl -G prime2.mos; profile"The profiler generates a file filename.prof with the profiling statistics. For the test model prime2.mos this file has the following contents (leaving out the header lines and comments):
model Prime parameters 1 0.00 0.00 LIMIT=20000 end-parameters declarations 1 0.00 0.00 SNumbers: set of integer 1 0.00 0.00 SPrime: set of integer end-declarations 1 0.01 0.00 SNumbers:={2..LIMIT} 1 0.00 0.01 writeln("Prime numbers between 2 and ", LIMIT, ":") 1 0.00 0.01 n:=2 1 0.00 0.01 repeat 2262 0.04 3.44 while (not(n in SNumbers)) n+=1 2262 0.00 3.44 SPrime += {n} 2262 0.00 3.44 i:=n 2262 0.04 3.44 while (i<=LIMIT) do 50126 3.31 3.44 SNumbers-= {i} 50126 0.04 3.44 i+=n end-do 2262 0.00 3.44 until SNumbers={} 1 0.00 3.44 writeln(SPrime) 1 0.00 3.45 writeln(" (", getsize(SPrime), " prime numbers.)") 1 0.00 3.45 end-modelThe first column lists the number of times a statment is executed, the second column the total time spent in a statement, and the third column the time of the last execution; then follows the corresponding model statement. In our example, we see that most of the model execution time is spent in a single line, namely the deletion of elements from the set SNumbers. This line is executed more than 50,000 times, but so is the following statement (i+=n) and it only takes a fraction of a second. Indeed, operations on large (> 1000 entries) sets may be relatively expensive in terms of running time. If our prime number algorithm were to be used in a large, time-critical application we should give preference to a more suitable data structure that is addressed more easily, that is, an array. For instance, by modifying the model as follows the total execution time for this model version becomes 0.19 seconds:
model "Prime (array)" parameters LIMIT=20000 ! Search for prime numbers in 2..LIMIT end-parameters declarations INumbers = 2..LIMIT ! Set of numbers to be checked SNumbers: array(INumbers) of boolean SPrime: set of integer ! Set of prime numbers end-declarations writeln("Prime numbers between 2 and ", LIMIT, ":") n:=2 repeat SPrime += {n} ! n is a prime number i:=n while (i<=LIMIT) do ! Remove n and all its multiples SNumbers(i):= true i+=n end-do while (n <= LIMIT and SNumbers(n)) n+=1 until (n>LIMIT) writeln(SPrime) writeln(" (", getsize(SPrime), " prime numbers.)") end-modelXpress-IVE users may select the menu Debug
Profile or click on the button
to obtain profiling information on screen, in IVE's output window (at the right side of the IVE working area).
Other commands for model analysis
The Mosel command line provides a few other commands that may be helpful with quickly obtaining information about models that have been executed in Mosel (these commands do not require any specific compilation flag).
Consider, for example, the following model flow.mos.
model "Dynamic arrays" declarations Suppliers = 1..150 Customers = 1..10000 COST: dynamic array(Suppliers,Customers) of real flow: dynamic array(Suppliers,Customers) of mpvar end-declarations initializations from "flow.dat" COST end-initializations forall(s in Suppliers, c in Customers | COST(s,c)>0 ) create(flow(s,c)) end-modelNow execute the following sequence of commands at the Mosel command line (as before, Mosel output is printed in bold face).
> exec flow.mos
Returned value: 0
> list
* name: Dynamic arrays number: 1 size: 45264
Sys. com.: `flow.mos'
User com.:
> info COST
`COST' is an array (dim: 2, size: 750) of reals
The command list displays information about all models loaded in Mosel, and in particular their size (= memory usage in bytes). With the command info followed by a symbol name we obtain detailed information about the definition of this symbol (without giving a symbol this command will display release and license information for Mosel). Alternatively, it is also possible to print the complete list of symbols (with type information and sizes) defined by the current model by using the command symbols.
If we now remove the keyword dynamic from the declaration of the two arrays, COST and flow, and re-run the same command sequence as before, we obtain the following output:
> list
* name: Dynamic arrays number: 1 size: 30011536
Sys. com.: `flow.mos'
User com.:
> info COST
`COST' is an array (dim: 2, size: 1500000) of reals
It is easily seen that in this model the use of the keyword dynamic makes a huge difference in terms of memory usage. A model defining several arrays of comparable sizes is likely to run out of memory (or at the least, it may not leave enough memory for an optimization algorithm to be executed).
Note: If COST is defined as a dynamic array, the condition on the forall loop should really be exists(COST(s,c)) for speedier execution of this loop.
Some recommendations for efficient modeling
The following list summarizes some crucial points to be taken into account, especially when writing large-scale models. For more details and examples please see Appendix Good modeling practice with Mosel.
- Use dynamic arrays to
- Don't use dynamic arrays
- General procedure for declaring and initializing data:
- declare all index sets and the minimal collection of data arrays required to initialize the sets,
- initialize the data arrays (which also initializes all index sets),
- finalize the index sets,
- declare and initialize all other arrays.
- Efficient use of dynamic arrays:
- Efficient use of exists:
- use named index sets in the declarations,
- use the same index sets in the loops,
- use the index sets in the same order,
- use the dynamic qualifier if some index sets are constant or finalized,
- make sure exists is the first condition,
- always use exists, even if no condition or an alternative condition is logically correct,
- conditions with or cannot be handled as efficiently as conditions with and.
- Loops (sum, forall, etc.):
- where possible, use conditional loops—loop index set followed by a vertical bar and the condition(s)—instead of a logical test with if within the loop,
- make sure exists is the first condition,
- always use exists, even if no condition or an alternative condition is logically correct,
- enumerate the index sets in the same order as they occur in the arrays within the loop,
- broken up, conditional loops are handled less efficiently.
- Do not use any debugging flag for compiling the final deployment version of your models.
If you have any comments or suggestions about these pages, please send mail to docs@dashoptimization.com.