Chapter B3
Runtime options

This chapter covers all aspects for running your Cactus executable. These include: command-line options, parameter file syntax, understanding screen output, environment variables, and creating thorn documentation.

B3.1 Command-Line Options

Cactus uses the standard GNU style of long-named command-line options; many of these options also have traditional Unix single-letter short forms. The options follow the usual GNU rules:

The Cactus command-line options are specified in Table B3.1, and are as follows:

Short Version Long Version

-O[v] --describe-all-parameters

-o<param> --describe-parameter=<param>

-S --print-schedule

-T --list-thorns


-h,-? --help

-v --version

-L<level> --logging-level=<level>

-W<level> --warning-level=<level>

-E<level> --error-level=<level>

-r[o|e|oe|eo] --redirect=[o|e|oe|eo]


-b[no|line|full] --buffering=[no|line|full]


-i --ignore-next

Table B3.1: This table shows all the Cactus command-line options.

-O or --describe-all-parameters

Prints a full list of all parameters from all thorns which were compiled, along with descriptions and allowed values. This can take an optional extra parameter v (i.e. -Ov to give verbose information about all parameters).

-o<param> or --describe-parameter=<param>

Prints the description and allowed values for a given parameter—takes one argument.

-S or --print-schedule

Print only the schedule tree.

-T or --list-thorns

Prints a list of all the thorns which were compiled in.

-t<arrangement or thorn> or --test-thorn-compiled=<arrangement or thorn>

Checks if a given thorn was compiled in—takes one argument.

-h, -? or --help

Prints a help message.

-v or --version

Prints version information of the code.

-L<level> or --logging-level=<level>

Sets the logging level of the code. All warning messages are given a level—the lower the level the greater the severity. This parameter -L controls the level of messages to be seen, with all warnings of level <level> printed to standard output. The default is a logging level of 0, meaning that only level 0 messages should be printed to standard output.

-W<level> or --warning-level=<level>

Similar to -W, but for standard error instead of standard output. All warnings of level <level> are printed to standard error. The default is a warning level of 1, meaning that level 0 and level 1 messages should be printed to standard error.

-E<level> or --error-level=<level>

Similar to -W, but for fatal errors: Cactus treats all warnings with level <level> as fatal errors, and aborts the Cactus run immediately (after printing the warning message1). The default value is zero, i.e. only level 0 warnings will abort the Cactus run.

-r[o|e|oe|eo] or --redirect=[o|e|oe|eo]

Redirects the standard output (‘o’) and/or standard error (‘e’) of each processor to a file. By default, the standard outputs from processors other than processor 0 are discarded.


Sets the output directory for logfiles created by the -r option. If the directory doesn’t exist yet, it will be created by Cactus.

-b[no|line|full] or --buffering=[no|line|full]

Set the stdout buffering mode. Buffered I/O is a standard feature of C programmes. This delays writing the actual output; instead, the output is collected into an internal buffer, and is then written in large chunks. This improves performance considerably. Line buffering means that output is written whenever a newline character is encountered; full buffering means that output is written, say, once 1000 characters have accumulated. The default setting is line buffering for I/O that goes to a terminal, and full buffering for I/O that goes to a file. For debugging purposes, it is sometimes useful to reduce the amount of buffering. Error messages, i.e. the stderr stream, is always unbuffered (and hence usually slower than stdout).


Sets the level of parameter checking to be used, one of strict (the default), normal, or relaxed. See Section B3.2 for details.

-i or --ignore-next

Causes the next argument on the command line to be ignored.

A dash (“-”) appended at the end of the command line like this:

./cactus_<config> [command-line options] -

lets the user specify parameter values from standard input rather than from a parameter file.

B3.2 Parameter File Syntax

A parameter file (or par file) is used to control the behaviour of a Cactus executable. It specifies initial values for parameters as defined in the various thorns’ param.ccl files (see Chapter C1.4). The name of a parameter file is often given the suffix .par, but this is not mandatory.

A parameter file is a text file whose lines are either blank lines, or parameter statements. Comments may also be included and consist of a ‘#’ and all following characters. A parameter statement is an expression of the form Left-Hand-Side = Right-Hand-Side. The Left-Hand-Side may be a fully qualified parameter name (i.e. a thorn or implementation name, two colons, and a name defined in a param.ccl file), a variable name (the ‘$’ character followed by a c-identifier), or the special variable ActiveThorns. The Right-Hand-Side is a value.

Values can be any of the following (all of which are case insensitive):

Logical operators
+ no-op

&&logical and   ! logical not

||logical or  
Mathematical functions

Relational operators
acos inverse cosine

==tests for equality   asin inverse sine

!=tests for inequality   atan inverse tangent

< tests for less than   ceil round up to nearest integer

> tests for greater than   cos cosine

<=tests for less or equal   cosh hyperbolic cosine

>=tests for greater or equal  exp exponentiation ex

Binary operators
abs absolute value |x|

+ addition   floor round down to nearest integer

- subtraction   log natural logarithm

/ C-like division   bool,int,realconvert to bool, int, or real

% remainder of division   sin sine

* multiplication   sinh hyperbolic sine

**exponentiation xy   sqrt square root

Unary operators
tan tangent

- negate sign   tanh hyperbolic tangent

  trunc integer part of x

Table B3.3: Supported functions inside of expressions, in increasing order of precedence.

Every parameter file should set ActiveThorns, which is a special parameter that tells the program which thorns are to be activated. One may set ActiveThorns on any line or lines of the par file. In the case where multiple specifications of ActiveThorns are supplied, the values will be concatenated.

Only parameters belonging to active thorns can be set (and only those routines scheduled by active thorns are run). By default, all thorns are inactive. For example, the first entry in a parameter file which is using just the two thorns CactusPUGH/PUGH and CactusBase/CartGrid3D should be

ActiveThorns = "PUGH CartGrid3D"

All parameters following the ActiveThorns parameter have names whose syntax depends on the scope (see Section C1.4.2) of the parameter:

Restricted parameters

The name of the implementation which defined the parameter, followed by two colons, then the name of the parameter—e.g. driver::global_nx.

Private parameters

The name of the thorn which defined the parameter, two colons, and the name of the parameter—e.g. wavetoyF77::amplitude.

This notation is not currently strictly enforced in the code. It is sufficient to specify the first part of the parameter name using either the implementation name, or the thorn name. However, we recommend that the above convention be followed.

The Cactus flesh performs checks for consistency and range of parameters. The severity of these checks is controlled by the command-line argument --parameter-level, which can take the following values


Cactus will issue a level 0 warning (that is, the default behaviour will be to terminate) if


This provides the same warnings as the relaxed level, with the addition of a level 0 warning issued for


This is the default, and provides the same warnings as the normal level, with the addition of a level 0 warning issued for


B3.3 Thorn Documentation

The Cactus make system provides a mechanism for generating a Thorn Guide containing separate chapters for each thorn and arrangement in your configuration. The documentation provided for an individual thorn, obviously depends on what the thorn authors added, but the Thorn Guide is a good place to first look for special instructions on how to run and interpret the output from a thorn. Details about parameters, grid variables and scheduling are automatically read from a thorn’s CCL files and included in the Thorn Guide. To construct a Thorn Guide for the configuration <config> use

gmake <config>-ThornGuide

or to make a Thorn Guide for all the thorns in the arrangements directory

gmake <config>.

See Section C1.8.4 for a guide to adding documentation to your own thorns.

1Cactus imposes the constraint, -W level -E level 0, so any fatal-error message will always be printed (first).