[Previous chapter] [Next chapter]

Language extensions


It has been said before that the functionality of the Mosel language can be extended by means of modules, that is, dynamic libraries written in C/C++. All through this manual we have used the module mmxprs to access Xpress-Optimizer. Other modules we have used are mmodbc (access to spreadsheets and databases, see Section Reading data from spreadsheets and databases) and mmsystem (Sections Modules and Cut generation).

The full distribution of Mosel includes other functionality (modules and IO drivers) that has not yet been mentioned in this document. In the following sections we give an overview with links where to find additional information.

Generalized file handling

The notion of (data) file encountered repeatedly in this user guide seems to imply a physical file. However, Mosel language statements (such as initializations from / to, fopen, and fclose) and the Mosel library functions (e.g., XPRMcompmod, XPRMloadmod, or XPRMrunmod) actually work with a much more general definition of a file, including (but not limited to)

The type of the file is indicated by adding to its name the name of the IO driver that is to be used to access it. In Section Reading data from spreadsheets and databases we have used mmodbc.odbc:blend.xls to access an MS Excel spreadsheet via the ODBC driver provided by the module mmodbc. If we want to work with a file held in memory we may write, for instance, mem:filename. The default driver (no driver prefix) is the standard Mosel file handling.

The standard distribution of Mosel defines the following IO drivers (tee and null are documented in the 'Mosel Language Reference Manual', all others in the `Mosel Libraries Reference Manual'):

tee
Output into up to 6 files simultaneously (e.g., to display a log on screen and write it to a file at the same time).
Example: adding the line 
fopen("tee:result.txt&", F_OUTPUT)
in a Mosel model will redirect all subsequent model output to the file result.txt and at the same time display the output on the default output (screen), the latter is denoted by the & sign at the end of the filename string. The output to both locations is terminated by the statement 
fclose(F_OUTPUT)
after which output will again only go to default output.
null
Disable a stream (ignore output/read from an empty file).
Example: adding the line 
fopen("null:", F_OUTPUT)
in a Mosel model will disable all subsequent output by this model (until the output stream is closed or a new output stream is opened).
sysfd
Working with operating system file descriptors (for instance, file descriptors returned by the C function open).
Example: in a C program, the line 
XPRMsetdefstream(NULL, XPRM_F_ERROR, "sysfd:1");
will redirect the Mosel error stream to the default output stream.
mem
Use memory instead of physical files for reading or writing data (e.g., for exchanging data between a model and the calling application as shown in Section Exchanging data between an application and a model or for compiling/loading a model to/from memory when working with the Mosel libraries).
Example: the following lines will compile the Mosel model burglar2.mos to memory and then load it from memory (full example in file ugcompmem.c). 
XPRMmodel mod;
char bimfile[2000];              /* Buffer to store BIM file */
char bimfile_name[64];           /* File name of BIM file */

XPRMinit())                      /* Initialize Mosel */

/* Prepare file name for compilation using 'mem' driver: */
/*   "mem:base_address/size[/actual_size_of_pointer]"    */
sprintf(bimfile_name, "mem:%#lx/%u",
        (unsigned long)bimfile, sizeof(bimfile));

                                 /* Compile model file to memory */
XPRMcompmod(NULL, "burglar2.mos", bimfile_name, "Knapsack example")) 

                                 /* Load a BIM file from memory */
mod = XPRMloadmod(bimfile_name, NULL);
cb
Use a (callback) function as a file (e.g., in combination with sysfd to write your own output or error handling functions when working with the Mosel libraries, see Section Redirecting the Mosel output for an example).
raw
Implementation of the initializations block in binary mode, typically used in combination with mem (see Section Exchanging data between an application and a model) or shmem (see Section Exchanging data between models).

Some modules, listed below in alphabetical order, define additional IO drivers. All these drivers are documented with the corresponding module in the `Mosel Language Reference Manual':

The reader is referred to the whitepaper Generalized file handling in Mosel that is provided as a part of the Xpress-MP documentation in the standard distribution and also on the Dash website under `Whitepapers' for further explanation of this topic and a documented set of examples, including some user-written IO drivers (e.g. for file compression).

Multiple models and parallel solving with mmjobs

The module mmjobs makes it possible to exchange information between models running concurrently. Its functionality includes facilities for model management (e.g. compiling, running, or interrupting a model from within a second model), synchronization of concurrent models based on event queues, and a shared memory IO driver for an efficient exchange of data between models that are executed concurrently.

Several complete examples (including examples of Benders decomposition and Dantzig-Wolfe decomposition) of the use of module mmjobs are described in the whitepaper Multiple models and parallel solving with Mosel that is provided as a part of the Xpress-MP documentation and also on the 'Whitepapers' page of the Dash website. We show here how to use the basic functionality for executing a model from a second model.

Running a model from another model

As a test case, we shall once more work with model prime.mos from Section Working with sets. In the first instance, we now show how to compile and run this model from a second model, runprime.mos: 

model "Run model prime"
 uses "mmjobs"

 declarations
  modPrime: Model
  event: Event
 end-declarations
                               ! Compile 'prime.mos'
 if compile("prime.mos")<>0 then exit(1); end-if

 load(modPrime, "prime.bim")   ! Load bim file

 run(modPrime, "LIMIT=50000")  ! Start execution and
 wait(2)                       ! wait 2 seconds for an event

 if isqueueempty then          ! No event has been sent...
  writeln("Model too slow: stopping it!")
  stop(modPrime)               ! ... stop the model, 
  wait                         ! ... and wait for the termination event
 end-if
                               ! An event is available: model finished
 event:=getnextevent
 writeln("Exit status: ", getvalue(event))
 writeln("Exit code  : ", getexitcode(modPrime))

 unload(modPrime)              ! Unload the submodel
end-model

The compile command generates the BIM file for the given submodel; the command load loads the binary file into Mosel; and finally we start the model with the command run. The run command is not used in its basic version (single argument with the model reference): here its second argument sets a new value for the parameter LIMIT of the submodel.

In addition to the standard compile–load–run sequence, the model above shows some basic features of interaction with the submodel: if the submodel has not terminated after 2 seconds (that is, if it has not sent a termination message) it is stopped by the master model. After termination of the submodel (either by finishing its calculations within less than 2 seconds or stopped by the master model) its termination status and the exit value are retrieved (functions getvalue and getexitcode). Unloading a submodel explicitly as shown here is only really necessary in larger applications that continue after the termination of the submodel, so as to free the memory used by it.

Note: our example model shows an important property of submodels—they are running in parallel to the master model and also to any other submodels that may have been started from the master model. It is therefore essential to insert wait at appropriate places to coordinate the execution of the different models.

Compiling to memory

The model shown in the previous section compiles and runs a submodel. The default compilation of a Mosel file filename.mos generates a binary model file filename.bim. To avoid the generation of physical BIM files for submodels we may compile the submodel to memory, making use of the concept of I/O drivers introduced in Section Generalized file handling.

Compiling a submodel to memory is done by replacing the standard compile and load commands by the following lines (model runprime2.mos): 

 if compile("","prime.mos","shmem:bim")<>0 then
  exit(1)
 end-if

 load(modPrime,"shmem:bim")    ! Load bim file from memory...
 fdelete("shmem:bim")          ! ... and release the memory block

The full version of compile takes three arguments: the compilation flags (e.g., use "g" for debugging), the model file name, and the output file name (here a label prefixed by the name of the shared memory driver). Having loaded the model we may free the memory used by the compiled model with a call to fdelete (this subroutine is provided by the module mmsystem).

Exchanging data between models

When working with submodels we are usually not just interested in executing the submodels, we also wish to retrieve their results in the master model. This is done most efficiently by exchanging data in (shared) memory as shown in the model runprimeio.mos below. Besides the retrieval and printout of the solution we have replaced the call to stop by sending the event STOPMOD to the submodel: instead of simply terminating the submodel this event will make it interrupt its calculations and write out the current solution. Once the submodel has terminated (after sending the STOPMOD event we wait for the model's termination message) we may read its solution from memory, using the initializations block with the drivers raw (binary format) and shmem (read from shared memory). 

model "Run model primeio"
 uses "mmjobs"

 declarations
  modPrime: Model
  NumP: integer                ! Number of prime numbers found
  SetP: set of integer         ! Set of prime numbers
  STOPMOD = 2
 end-declarations

                               ! Compile 'prime.mos'
 if compile("primeio.mos")<>0 then exit(1); end-if

 load(modPrime, "primeio.bim") ! Load bim file

                               ! Disable model output
 setdefstream(modPrime,"","null:","null:")
 run(modPrime, "LIMIT=35000")  ! Start execution and
 wait(2)                       ! wait 2 seconds for an event

 if isqueueempty then          ! No event has been sent...
  writeln("Model too slow: stopping it!")
  send(modPrime, STOPMOD, 0)   ! ... stop the model, then wait
  wait
 end-if
 
 initializations from "raw:"
  NumP as "shmem:NumP" SetP as "shmem:SPrime"
 end-initializations
 
 writeln(SetP)                 ! Output the result
 writeln(" (", NumP, " prime numbers.)")

 unload(modPrime)
end-model

We now have to modify the submodel (file primeio.mos) correspondingly: it needs to intercept the `STOPMOD' event interrupting the calculations (via an additional test isqueueempty for the repeat-until loop) and write out the solution to memory in the end: 

model "Prime IO" 
 uses "mmjobs"

 parameters
  LIMIT=100                     ! Search for prime numbers in 2..LIMIT
 end-parameters

 declarations
  SNumbers: set of integer      ! Set of numbers to be checked
  SPrime: set of integer        ! Set of prime numbers
  STOPMOD = 2
 end-declarations

 SNumbers:={2..LIMIT} 
 
 writeln("Prime numbers between 2 and ", LIMIT, ":")

 n:=2
 repeat
   while (not(n in SNumbers)) n+=1
   SPrime += {n}                ! n is a prime number
   i:=n
   while (i<=LIMIT) do          ! Remove n and all its multiples
     SNumbers-= {i}
     i+=n
   end-do
 until (SNumbers={} or not isqueueempty)     
 
 NumP:= getsize(SPrime)
 
 initializations to "raw:"
  NumP as "shmem:NumP" SPrime as "shmem:SPrime"
 end-initializations
 
end-model

Note: since the condition isqueueempty is tested only once per iteration of the repeat-until loop, the termination of the submodel is not immediate for large values of LIMIT. If you wish to run this model with very large values, please see Section Efficient modeling through the Mosel Profiler for an improved implementation of the prime number algorithm that considerably reduces its execution time.

Graphics with mmive and mmxad

Windows users may enrich their Mosel models with graphical output within Xpress-IVE (using the module mmive) or implement complete, interactive graphical applications (module mmxad).

The functionality of module mmive is documented in the Mosel reference manual. The built-in set of models in the Xpress-IVE model wizard (menu Wizards Maths/arrow.png 12. Complete models or button MoselUG/butmod12.png) gives several examples of its use. The Xpress-Application Developer (XAD) comes with its own set of documentation and examples (follow the product link on the Dash website).

Drawing user graphs with mmive

The graphic in Figure User graph in Xpress-IVE is an example of using mmive to produce a graphical representation of the solution to the transport problem from Section A transport example.

MoselUG/transpive.png

Figure 21.1: User graph in Xpress-IVE

It was obtained by calling the following procedure draw_solution at the end of the model file (that is, after the call to minimize). 

 procedure draw_solution
  declarations
   YP: array(PLANT) of integer           ! y-coordinates of plants
   YR: array(REGION) of integer          ! y-coordinates of sales regions
  end-declarations
  
  ! Set the size of the displayed graph
  IVEzoom(0,0,3,getsize(REGION)+1)

  ! Determine y-coordinates for plants and regions  
  ct:= 1+floor((getsize(REGION)-getsize(PLANT))/2)
  forall(p in PLANT) do
   YP(p):= ct
   ct+=1
  end-do 

  ct:=1
  forall(r in REGION) do
   YR(r):= ct
   ct+=1
  end-do 
  
  ! Draw the plants
  PlantGraph:= IVEaddplot("Plants", IVE_BLUE)
  forall(p in PLANT) IVEdrawlabel(PlantGraph, 0.8, YP(p)-0.1, p)
  
  ! Draw the sales regions
  RegGraph:= IVEaddplot("Regions", IVE_CYAN)
  forall(r in REGION) IVEdrawlabel(RegGraph, 2.25, YR(r)-0.1, r)
  
  ! Draw all transport routes
  RouteGraph:= IVEaddplot("Routes", IVE_WHITE)
  forall(p in PLANT, r in REGION | exists(TRANSCAP(p,r)) ) 
   IVEdrawline(RouteGraph, 1, YP(p), 2, YR(r))
   
  ! Draw the routes used by the solution
  SolGraph:= IVEaddplot("Solution", IVE_RED)
  forall(p in PLANT, r in REGION | exists(flow(p,r)) and getsol(flow(p,r)) > 0)
   IVEdrawarrow(SolGraph, 1, YP(p), 2, YR(r))

 end-procedure

Application development with mmxad

As an alternative to the graphical solution output within IVE we might write a complete application with XAD like the example shown in Figure XAD application window that lets the end-user modify the demand data and then resolve the problem.

MoselUG/transpxad.png

Figure 21.2: XAD application window

To be able to replace the demand constraints, we need to name them in the model, for example,

 forall(r in REGION)
  Demand(r):= sum(p in PLANT) flow(p,r) = DEMAND(r)

We also remove the call to optimization and replace it with the start of the application (that will call the optimization when requested by the user): 

 declarations
  ID_WINDOW=1                            ! Identifiers for XAD objects
  ID_BUT_SOL=2
  ID_BUT_EXIT=3
  ID_CANVAS=4
  ID_TEXT=5
  ID_DEMAND=500

  YP: array(PLANT) of integer            ! y-coordinates of plants
  YR: array(REGION) of integer           ! y-coordinates of sales regions
 end-declarations

! Determine y-coordinates for plants and regions  
 ct:= round((getsize(REGION)-getsize(PLANT))/2)
 forall(p in PLANT) do
  YP(p):= ct*25
  ct+=1
 end-do 

 ct:=1
 forall(r in REGION) do
  YR(r):= ct*25
  ct+=1
 end-do 
 						    
 start_xad                               ! Start the XAD application

The procedure start_xad defines the layout of the application: it consists of a single window with some text, an input field for every demand data item, a canvas for the graphical display of the solutions, and two buttons labeled `Solve' and `Exit' respectively. In procedure start_xad we also set an event handler callback, that is, a subroutine that will be called whenever an action (mouse click etc.) occurs in the application window. This callback is implemented by the following procedure process_event. The events captured by our function are `window opened' and `button pressed' (for each of the two buttons in the application window). When the window is opened the possible transport routes are displayed on the canvas. Button `Solve' triggers re-solving of the optimization problem with the demand data currently entered in the input fields of the application, and button `Exit' terminates the application by closing the window. 

 procedure start_xad
  ! Create the application window
  XADcreatewindow(ID_WINDOW, 50, 50, 550, 300, "Transport problem")

  ! Create the demand data input
  XADcreatetext(ID_WINDOW, ID_TEXT, 24, 4, 100, 15, "Demands by regions:")
  forall(r in REGION) do
   XADcreatetext(ID_WINDOW, ID_TEXT+YR(r), 35, YR(r), 45, 15, r)
   XADcreateinput(ID_WINDOW, ID_DEMAND+YR(r), 100, YR(r), 50, 22,
                  string(DEMAND(r)))
  end-do
  
  ! Create the canvas for solution drawing
  XADcreatecanvas(ID_WINDOW, ID_CANVAS, 200,20,300,200)
  
  ! Create 'Solve' and 'Exit' buttons
  XADcreatebutton(ID_WINDOW, ID_BUT_SOL, 50,180,80,24, "Solve")
  XADcreatebutton(ID_WINDOW, ID_BUT_EXIT, 50,220,80,24, "Exit")
 
  ! Set the event handler callback
  XADseteventcallback("process_event")

  ! Display the application window
  XADwindowopen(ID_WINDOW)
 end-procedure

!*********************************************************************** 
 procedure process_event(id:integer, event:integer)
  case id of
   ID_WINDOW:   if event=XAD_EVENT_WINDOW_OPENED then
                 draw_routes               ! Show initial display on canvas
	        end-if
   ID_BUT_SOL:  if event=XAD_EVENT_PRESSED then                 
                 forall(r in REGION) do    ! Update demand data+constraints
		  DEMAND(r):= integer(XADinputgettext(ID_DEMAND+YR(r)))
		  Demand(r):= sum(p in PLANT) flow(p,r) = DEMAND(r)
                 end-do
                 minimize(MinCost)         ! (Re)solve the problem
                 update_solution           ! Show the solution on canvas
                end-if
   ID_BUT_EXIT: if event=XAD_EVENT_PRESSED then
                 XADwindowclose(ID_WINDOW) ! Terminate the application
                end-if
  end-case
 end-procedure

On its turn, the event handler subroutine process_event calls two other procedures (draw_routes and update_solution) for the graphical display of input data and solutions on the canvas. 

 procedure draw_routes
  XADcanvaserase(ID_CANVAS, XAD_WHITE)     ! Empty the canvas
    
  ! Draw the plants
  forall(p in PLANT) XADcanvasdrawtext(ID_CANVAS, 50, YP(p), p, XAD_BLUE)
  
  ! Draw the sales regions
  forall(r in REGION) XADcanvasdrawtext(ID_CANVAS, 250, YR(r), r, XAD_CYAN)
  
  ! Draw all transport routes
  forall(p in PLANT, r in REGION | exists(TRANSCAP(p,r)) ) 
   XADcanvasdrawline(ID_CANVAS, 80, YP(p), 220, YR(r), XAD_BLACK)
 
  XADcanvasrefresh(ID_CANVAS)              ! Display the graphics
 end-procedure

!***********************************************************************
 procedure update_solution
  ! Re-draw all transport routes
  forall(p in PLANT, r in REGION | exists(TRANSCAP(p,r)) ) 
   XADcanvasdrawline(ID_CANVAS, 80, YP(p), 220, YR(r), XAD_BLACK)
   
  ! Draw the routes used by the solution
  forall(p in PLANT, r in REGION | exists(flow(p,r)) and getsol(flow(p,r))>0)
   XADcanvasdrawline(ID_CANVAS, 80, YP(p), 220, YR(r), XAD_RED)
  
  XADcanvasrefresh(ID_CANVAS)              ! Update the canvas
 end-procedure

Solvers

In this user guide we have explained the basics of working with Mosel, focussing on the formulation of Linear and Mixed Integer Programming problems and related solution techniques. However, the Mosel language is not limited to certain types of Mathematical Programming problems.

An extension to standard LP and MIP is the possibility to work with quadratic objective functions (commonly referred to as Quadratic Programming and Mixed Integer Quadratic Programming). This functionality is provided by the module mmquad and documented in the Mosel reference manual.

All other solvers of the Xpress-MP suite (e.g., Xpress-SLP for solving non-linear problems and Xpress-Kalis for Constraint Programming) are provided with separate manuals and their own sets of examples. Please see the Dash website for an overview of the available products.

With Mosel it is possible to combine several solvers to formulate hybrid solution approaches for solving difficult application problems. The whitepaper Hybrid MIP/CP solving with Xpress-Optimizer and Xpress-Kalis, available for download from the Dash website, gives several examples of hybrid solving with LP/MIP and Constraint Programming.


[Next] [Index]
If you have any comments or suggestions about these pages, please send mail to docs@dashoptimization.com.