Each thorn is configured by three compulsory and one optional files in the top level thorn directory:
These files are written in the Cactus Configuration Language which is case insensitive.
The interface configuration file consists of:
(For a more extensive discussion of Cactus variables, see Chapter C1.3.)
The header block has the form:
The include file section has the form:
If any aliased function is to be used or provided by the thorn, then the prototype must be declared with the form:
If the argument <arg*> is a function pointer, then the argument itself (which will preceded by the return type) should be
If an aliased function is to be required, then the block
If an aliased function is to be (optionally) used, then the block
If a function is provided, then the block
The thorn’s variables are collected into groups. This is not only for convenience, but for collecting like variables together. Storage assignment, communication assignment, and ghostzone synchronization take place for groups only.
The thorn’s variables are defined by:
The process of sharing code among thorns using include files is discussed in Section C1.9.2.
The parameter configuration file consists of a list of parameter object specification items (OSIs) giving the type and range of the parameter separated by optional parameter data scoping items (DSIs), which detail access to the parameter. (For a more extensive discussion of Cactus parameters, see Chapter C1.4.)
The keyword access designates that all parameter object specification items, up to the next parameter data scoping item, are in the same protection or scoping class. access can take the values:
all thorns have access to global parameters
other thorns can have access to these parameters, if they specifically request it in their own param.ccl
only your thorn has access to private parameters
in this case, an implementation name must follow the colon. It declares that all the parameters in the following scoping block are restricted variables from the specified implementation. (Note: only one implementation can be specified on this line.)
The specification of parameter values takes the form of one or more lines, each of the form
The range specification is the same as with integers, except that here, no step implies a continuum of values. Note that numeric constants should be expressed as in C (e.g. 1e-10). Note also that one cannot use the Cactus types such as CCTK_REAL4 to specify the precision of the parameter; parameters always have the default precision.
Each entry in the list of acceptable values for a keyword has the form
Allowed values for strings should be specified using regular expressions. To allow any string, the regular expression "" should be used. (An empty regular expression matches anything.) Regular expressions and string values should be enclosed in double quotes. The double quotes are mandatory if the regular expression or the string value is empty or contains spaces.
No parameter values should be specified for a boolean parameter. The default value for a boolean can be
Boolean values may optionally be enclosed in double quotes.
The value RECOVERY is used in checkpoint/recovery situations, and indicates that the parameter may be altered until the value is read in from a recovery par file, but not after.
(A more extensive discussion of Cactus scheduling is provided in Chapter C1.5.) A schedule configuration file consists of:
Assignment statements, currently only assign storage.
These lines have the form:
If the thorn is active, storage will be allocated, for the given groups, for the duration of program execution (unless storage is explicitly switched off by some call to CCTK_DisableGroupStorage within a thorn).
The storage line includes the number of timelevels to activate storage for, this number can be from 1 up to the maximum number or timelevels for the group, as specified in the defining interface.ccl file. If the maximum number of timelevels is 1 (the default), this number may be omitted. Alternatively timelevels can be the name of a parameter accessible to the thorn. The parameter name is the same as used in C routines of the thorn, fully qualified parameter names of the form thorn::parameter are not allowed. In this case 0 (zero) timelevels can be requested, which is equivalent to the STORAGE statement being absent.
The behaviour of an assignment statement is independent of its position in the schedule file (so long as it is outside a schedule block).
Each schedule block in the file schedule.ccl must have the syntax
Schedule a schedule group with the same options as a schedule function. The schedule group will be created if it doesn’t exist.
The name of a function or a schedule group to be scheduled. Function and schedule group names are case sensitive.
A group of grid variables. Variable groups inherited from other thorns may be used, but they must then be fully qualified with the implementation name.
Functions can be scheduled to run at the Cactus schedule bins, for example, CCTK_EVOL, and CCTK_STARTUP. A complete list and description of these is provided in Appendix D4. The initial letters CCTK_ are optional. Grid variables cannot be used in the CCTK_STARTUP and CCTK_SHUTDOWN timebins.
Schedules a function or schedule group to run in a schedule group, rather than in a Cactus timebin.
Provides an alias for a function or schedule group which should be used for scheduling before, after or in. This can be used to provide thorn independence for other thorns scheduling functions, or schedule groups relative to this one.
Executes a function or schedule group until the given variable (which must be a fully qualified integer grid scalar) has the value zero.
Executes a function or schedule group only if the given variable (which must be a fully qualified integer grid scalar) has a non-zero value.
Takes a function name, a function alias, a schedule group name, or a parentheses-enclosed whitespace-separated list of these. (Any names that are not provided by an active thorn are ignored.) Note that a single schedule block may have multiple BEFORE/AFTER clauses. See Section C1.5 (“Scheduling”) in the Cactus Users’ Guide for more information about BEFORE/AFTER clauses.
The code language for the function (either C or FORTRAN). No language should be specified for a schedule group.
Schedule options are used for mesh refinement and multi-block simulations, and they determine “where” a routine executes. Possible options are:
(Only one of these options may be used.) These options can be combined with the following:
(At most one of the loop_... options may be used.)
Schedule tags. These tags must have the form keyword=value, and must be in a syntax accepted by Util_TableCreateFromString.
List of variable groups which should have storage switched on for the duration of the function or schedule group. Each group must specify how many timelevels to activate storage for, from 1 up to the maximum number for the group as specified in the defining interface.ccl file. If the maximum is 1 (the default) this number may be omitted. Alternatively timelevels can be the name of a parameter accessible to the thorn. The parameter name is the same as used in C routines of the thorn, fully qualified parameter names of the form thorn::parameter are not allowed. In this case 0 (zero) timelevels can be requested, which is equivalent to the STORAGE statement being absent.
READS is used to declare which grid variables are read by the routine. This information is used e.g. to determine which variables need to be copied between host and device for OpenCL or CUDA kernel. This information can also be used to ensure that all variables that are read have previously been written by another routine.
WRITES is used to declare which grid variables are written by the routine. This information is used e.g. to determine which variables need to be copied between host and device for OpenCL or CUDA kernel. This information can also be used to ensure that all variables that are read have previously been written by another routine.
List of grid variables or groups to be used as triggers for causing an ANALYSIS function or group to be executed. Any schedule block for an analysis function or analysis group may contain a TRIGGER line.
List of groups to be synchronised, as soon as the function or schedule group is exited.
List of additional options (see below) for the scheduled function or group of functions
Cactus understands the following options. These options are interpreted by the driver, not by Cactus. The current set of options is useful for Berger-Oliger mesh refinement which has subcycling in time, and for multi-patch simulations in which the domain is split into several distinct patches. Given this, the meanings of the options below is only tentative, and their exact meaning needs to be obtained from the driver documentation. The standard driver PUGH ignores all options.
Option names are case-insensitive. There can be several options given at the same time.
This routine will only be called once, even if several simulations are performed at the same time. This can be used, for example, to initialise external libraries, or to set up data structures that live in global variables.
This option is identical to to META option with the exception that the routine will be called together with the routines on the first subgrid.
This option is identical to to META option with the exception that the routine will be called together with the routines on the last subgrid.
This routine will only be called once on a grid hierarchy, not for all subgrids making up the hierarchy. This can be used, for example, for analysis routines which use global reduction or interpolation routines, rather than the local subgrid passed to them, and hence should only be called once.
This option is identical to to GLOBAL option with the exception that the routine will be called together with the routines on the first subgrid.
This option is identical to to GLOBAL option with the exception that the routine will be called together with the routines on the last subgrid.
This routine will only be called once on any “level” of the grid hierarchy. That is, it will only be called once for any set of sub-grids which have the same cctk_levfac numbers.
This routine will only be called once on any of the “patches” that form a “level” of the grid hierarchy.
This routine will be called on every “component”.
When the above options are used, it is often the case that a certain routine should, e.g. be called at the time for a GLOBAL routine, but should actually loop over all “components”. The following set of options allows this:
Loop over all simulations.
Loop over all “levels”.
Loop over all “patches”.
Loop over all “components”.
For example, the specification
Any schedule block or assignment statements can be optionally surrounded by conditional if-elseif-else constructs using the parameter data base. These can be nested, and have the general form:
¡conditional-expression¿ can be any valid C construct evaluating to a truth value. Such conditionals are evaluated only at program startup, and are used to pick between different static schedule options. For dynamic scheduling, the SCHEDULE WHILE construction should be used.
Conditional constructs cannot be used inside a schedule block.
[NOTE: The configuration.ccl is still relatively new, and not all features listed below may be fully implemented or functional.]
A configuration.ccl file defines capabilities which a thorn either provides or requires, or may use if available. Unlike implementations, only one thorn providing a particular capability may be compiled into a configuration at one time. Thus, this mechanism may be used to, for example: provide access to external libraries; provide access to functions which other thorns must call, but are too complex for function aliasing; or to split a thorn into several thorns, all of which require some common (not aliased) functions.
A configuration options file can contain any number of the following sections:
Informs the CST that this thorn provides a given capability, and that this capability has a given detection script which may be used to configure it (e.g. running an autoconf script or detecting an external library’s location). The script should output configuration information on its standard output—the syntax is described below in Section D2.5.1. The script may also indicate the failure to detect a capability by returning a non-zero exit code; this will stop the build after the CST stage.
All capabilities have an optional version attached to them, so that other thorns can depend on such a specific version. Version strings can only contain letters, numbers, or “.+-:” (dot, plus, minus, colon), and have to start with a number. Specifying a version number is optional. If none is given, “0.0.1” is assumed.
Scripts can be in any language. If an interpreter is needed to run the script, for example Perl, this should be indicated by the LANG option.
The specified options are checked for in the original configuration, and any options passed on the command line (including an ‘options’ file) at compile time when the thorn is added, or if the CST is rerun. These options need be set only once, and will be remembered between builds.
Informs the CST that this thorn requires a certain capability to be present. If no thorn providing the capability is in the ThornList, the build will stop after the CST stage.
Optionally, thorns can depend on a capability version. This has to be enclosed in parentheses, following the capability name. Inside the parenthesis, first a comparison operator determines the kind of dependency on the requested capability version, which has to be attached without spaces. Valid operators and their meaning are:
Version strings are compared from left to right. First the initial part of each string consisting entirely of digits is determined. The integer values of these two parts are compared. If a difference is found it is returned. Then the initial part of the remainder of each string which consists entirely of non-digit characters is determined. The two parts are compared lexically, and any difference found is returned as the result of the comparison. The lexical comparison is a comparison of ASCII values. These two steps (comparing and removing initial non-digit strings and initial digit strings) are repeated until a difference is found or both strings are exhausted. Spaces inside the parentheses are allowed, except inside the operator or version string.
Informs the CST that this thorn may use a certain capability, if a thorn providing it is in the ThornList. If present, the preprocessor macro, macro, will be defined and given the value “1”.
The configuration script may tell the CST to add certain features to the Cactus environment—either to the make system or to header files included by thorns. It does this by outputting lines to its standard output:
Places a set of definitions in a header file which will be included by all thorns using this capability (either through an OPTIONAL or REQUIRES entry in their configuration.ccl files).
Adds a directory to the include path used for compiling files in thorns using this capability.
Adds a makefile definition into the compilation of all thorns using this capability.
Adds makefile dependency information into the compilation of all thorns using this capability.
Adds a library to the final cactus link.
Adds a directory to the list of directories searched for libraries at link time.
No other lines should be output by the script.