Chapter D2
Configuration File Syntax

D2.1 General Concepts

Each thorn is configured by three compulsory and one optional files in the top level thorn directory:

These files are written in the Cactus Configuration Language which is case insensitive.

D2.2 interface.ccl

The interface configuration file consists of:

(For a more extensive discussion of Cactus variables, see Chapter C1.3.)

D2.2.1 Header Block

The header block has the form:

implements: <implementation>
inherits: <implementation>, <implementation>
friend: <implementation>, <implementation>
where

D2.2.2 Include Files

The include file section has the form:

USES INCLUDE [SOURCE|HEADER]: <file_name>
INCLUDE[S] [SOURCE|HEADER]: <file_to_include> in <file_name>
The former is used when a thorn wishes to use an include file from another thorn. The latter indicates that this thorn adds the code in <file_to_include> to the include file <file_name>. If the include file is described as SOURCE, the included code is only executed if the providing thorn is active. Both default to HEADER.

D2.2.3 Function Aliasing

If any aliased function is to be used or provided by the thorn, then the prototype must be declared with the form:

<return_type> FUNCTION <alias>(<arg1_type> <intent1> [ARRAY] <arg1>, ...)
The <return_type> must be either void, CCTK_INT, CCTK_REAL, CCTK_COMPLEX, CCTK_POINTER, or CCTK_POINTER_TO_CONST. The keyword SUBROUTINE is equivalent to void FUNCTION. The name of the aliased function <alias> must contain at least one uppercase and one lowercase letter and follow the C standard for function names. The type of each argument, <arg*_type>, must be either CCTK_INT, CCTK_REAL, CCTK_COMPLEX, CCTK_POINTER, CCTK_POINTER_TO_CONST, or STRING. All string arguments must be the last arguments in the list. The intent of each argument, <intent*>, must be either IN, OUT, or INOUT. An argument may only be modified if it is declared to have intent OUT or INOUT. If the argument is an array then the prefix ARRAY must also be given.

If the argument <arg*> is a function pointer, then the argument itself (which will preceded by the return type) should be

CCTK_FPOINTER <function_arg1>(<arg1_type> <intent1> <arg1>, ...)
Function pointers may not be nested.

If an aliased function is to be required, then the block

REQUIRES FUNCTION <alias>
is required.

If an aliased function is to be (optionally) used, then the block

USES FUNCTION <alias>
is required.

If a function is provided, then the block

PROVIDES FUNCTION <alias> WITH <provider> LANGUAGE <providing_language>
is required. As with the alias name, <provider> must contain at least one uppercase and one lowercase letter, and follow the C standard for function names. Currently, the only supported values of <providing_language> are C and Fortran.

D2.2.4 Variable Blocks

The thorn’s variables are collected into groups. This is not only for convenience, but for collecting like variables together. Storage assignment, communication assignment, and ghostzone synchronization take place for groups only.

The thorn’s variables are defined by:

[<access>:]

<data_type> <group_name>[[<number>]] [TYPE=<group_type>] [DIM=<dim>]
[TIMELEVELS=<num>]
[SIZE=<size in each direction>] [DISTRIB=<distribution_type>]
[GHOSTSIZE=<ghostsize>]
[TAGS=<string>]  ["<group_description>"]
[{
 [ <variable_name>[,]<variable_name>
   <variable_name> ]
} ["<group_description>"] ]
(The options TYPE, DIM, etc., following <group_name> must all appear on one line.) Note that the beginning brace ({) must sit on a line by itself; the ending brace (}) must be preceded by a carriage return.

The process of sharing code among thorns using include files is discussed in Section C1.9.2.

D2.3 param.ccl

The parameter configuration file consists of a list of parameter object specification items (OSIs) giving the type and range of the parameter separated by optional parameter data scoping items (DSIs), which detail access to the parameter. (For a more extensive discussion of Cactus parameters, see Chapter C1.4.)

D2.3.1 Parameter Data Scoping Items

<access>:

The keyword access designates that all parameter object specification items, up to the next parameter data scoping item, are in the same protection or scoping class. access can take the values:

global

all thorns have access to global parameters

restricted

other thorns can have access to these parameters, if they specifically request it in their own param.ccl

private

only your thorn has access to private parameters

shares

in this case, an implementation name must follow the colon. It declares that all the parameters in the following scoping block are restricted variables from the specified implementation. (Note: only one implementation can be specified on this line.)

D2.3.2 Parameter Object Specification Items

[EXTENDS|USES] <parameter type> <parameter name>[[<len>]] "<parameter description>"
[AS <alias>] [STEERABLE=<NEVER|ALWAYS|RECOVER>]
[ACCUMULATOR=<expression>] [ACCUMULATOR-BASE=<parameter name>]
{
  <parameter values>
} <default value>
where the options AS, STEERABLE, etc., following <parameter description>, must all appear in one line. Note that the beginning brace ({) must sit on a line by itself; the ending brace (}) must be at the beginning of a line followed by <default value> on that same line.

D2.4 schedule.ccl

(A more extensive discussion of Cactus scheduling is provided in Chapter C1.5.) A schedule configuration file consists of:

D2.4.1 Assignment Statements

Assignment statements, currently only assign storage.

These lines have the form:

[STORAGE: <group>[timelevels], <group>[timelevels]]

If the thorn is active, storage will be allocated, for the given groups, for the duration of program execution (unless storage is explicitly switched off by some call to CCTK_DisableGroupStorage within a thorn).

The storage line includes the number of timelevels to activate storage for, this number can be from 1 up to the maximum number or timelevels for the group, as specified in the defining interface.ccl file. If the maximum number of timelevels is 1 (the default), this number may be omitted. Alternatively timelevels can be the name of a parameter accessible to the thorn. The parameter name is the same as used in C routines of the thorn, fully qualified parameter names of the form thorn::parameter are not allowed. In this case 0 (zero) timelevels can be requested, which is equivalent to the STORAGE statement being absent.

The behaviour of an assignment statement is independent of its position in the schedule file (so long as it is outside a schedule block).

D2.4.2 Schedule Blocks

Each schedule block in the file schedule.ccl must have the syntax

schedule [GROUP] <function name|group name> AT|IN <time> \
     [AS <alias>] \
     [WHILE <variable>] [IF <variable>] \
     [BEFORE|AFTER <function name>|(<function name> <function name> ...)] \
{
  [LANG: <language>]
  [OPTIONS:       <option>,<option>...]
  [TAGS:          <keyword=value>,<keyword=value>...]
  [STORAGE:       <group>[timelevels],<group>[timelevels]...]
  [READS:         <group>,<group>...]
  [WRITES:        <group>,<group>...]
  [TRIGGER:       <group>,<group>...]
  [SYNCHRONISE:   <group>,<group>...]
  [OPTIONS:       <option>,<option>...]
} "Description of function"

GROUP

Schedule a schedule group with the same options as a schedule function. The schedule group will be created if it doesn’t exist.

<function name|group name>

The name of a function or a schedule group to be scheduled. Function and schedule group names are case sensitive.

<group>

A group of grid variables. Variable groups inherited from other thorns may be used, but they must then be fully qualified with the implementation name.

AT

Functions can be scheduled to run at the Cactus schedule bins, for example, CCTK_EVOL, and CCTK_STARTUP. A complete list and description of these is provided in Appendix D4. The initial letters CCTK_ are optional. Grid variables cannot be used in the CCTK_STARTUP and CCTK_SHUTDOWN timebins.

IN

Schedules a function or schedule group to run in a schedule group, rather than in a Cactus timebin.

AS

Provides an alias for a function or schedule group which should be used for scheduling before, after or in. This can be used to provide thorn independence for other thorns scheduling functions, or schedule groups relative to this one.

WHILE

Executes a function or schedule group until the given variable (which must be a fully qualified integer grid scalar) has the value zero.

IF

Executes a function or schedule group only if the given variable (which must be a fully qualified integer grid scalar) has a non-zero value.

BEFORE/AFTER

Takes a function name, a function alias, a schedule group name, or a parentheses-enclosed whitespace-separated list of these. (Any names that are not provided by an active thorn are ignored.) Note that a single schedule block may have multiple BEFORE/AFTER clauses. See Section C1.5 (“Scheduling”) in the Cactus Users’ Guide for more information about BEFORE/AFTER clauses.

LANG

The code language for the function (either C or FORTRAN). No language should be specified for a schedule group.

OPTIONS

Schedule options are used for mesh refinement and multi-block simulations, and they determine “where” a routine executes. Possible options are:

meta
meta_early
meta_late
global
global_early
global_late
level
singlemap
local
(default, may be omitted)

(Only one of these options may be used.) These options can be combined with the following:

loop_meta
loop_global
loop_level
loop_singlemap
loop_local

(At most one of the loop_... options may be used.)

TAGS

Schedule tags. These tags must have the form keyword=value, and must be in a syntax accepted by Util_TableCreateFromString.

STORAGE

List of variable groups which should have storage switched on for the duration of the function or schedule group. Each group must specify how many timelevels to activate storage for, from 1 up to the maximum number for the group as specified in the defining interface.ccl file. If the maximum is 1 (the default) this number may be omitted. Alternatively timelevels can be the name of a parameter accessible to the thorn. The parameter name is the same as used in C routines of the thorn, fully qualified parameter names of the form thorn::parameter are not allowed. In this case 0 (zero) timelevels can be requested, which is equivalent to the STORAGE statement being absent.

READS

READS is used to declare which grid variables are read by the routine. This information is used e.g. to determine which variables need to be copied between host and device for OpenCL or CUDA kernel. This information can also be used to ensure that all variables that are read have previously been written by another routine.

WRITES

WRITES is used to declare which grid variables are written by the routine. This information is used e.g. to determine which variables need to be copied between host and device for OpenCL or CUDA kernel. This information can also be used to ensure that all variables that are read have previously been written by another routine.

TRIGGER

List of grid variables or groups to be used as triggers for causing an ANALYSIS function or group to be executed. Any schedule block for an analysis function or analysis group may contain a TRIGGER line.

SYNCHRONISE

List of groups to be synchronised, as soon as the function or schedule group is exited.

OPTIONS

List of additional options (see below) for the scheduled function or group of functions

Allowed Options

Cactus understands the following options. These options are interpreted by the driver, not by Cactus. The current set of options is useful for Berger-Oliger mesh refinement which has subcycling in time, and for multi-patch simulations in which the domain is split into several distinct patches. Given this, the meanings of the options below is only tentative, and their exact meaning needs to be obtained from the driver documentation. The standard driver PUGH ignores all options.

Option names are case-insensitive. There can be several options given at the same time.

META

This routine will only be called once, even if several simulations are performed at the same time. This can be used, for example, to initialise external libraries, or to set up data structures that live in global variables.

META-EARLY

This option is identical to to META option with the exception that the routine will be called together with the routines on the first subgrid.

META-LATE

This option is identical to to META option with the exception that the routine will be called together with the routines on the last subgrid.

GLOBAL

This routine will only be called once on a grid hierarchy, not for all subgrids making up the hierarchy. This can be used, for example, for analysis routines which use global reduction or interpolation routines, rather than the local subgrid passed to them, and hence should only be called once.

GLOBAL-EARLY

This option is identical to to GLOBAL option with the exception that the routine will be called together with the routines on the first subgrid.

GLOBAL-LATE

This option is identical to to GLOBAL option with the exception that the routine will be called together with the routines on the last subgrid.

LEVEL

This routine will only be called once on any “level” of the grid hierarchy. That is, it will only be called once for any set of sub-grids which have the same cctk_levfac numbers.

SINGLEMAP

This routine will only be called once on any of the “patches” that form a “level” of the grid hierarchy.

LOCAL (this is the default)

This routine will be called on every “component”.

When the above options are used, it is often the case that a certain routine should, e.g. be called at the time for a GLOBAL routine, but should actually loop over all “components”. The following set of options allows this:

LOOP-META

Loop once.

LOOP-GLOBAL

Loop over all simulations.

LOOP-LEVEL

Loop over all “levels”.

LOOP-SINGLEMAP

Loop over all “patches”.

LOOP-LOCAL

Loop over all “components”.

For example, the specification

  OPTIONS: global loop-local
schedules a routine at the time when a GLOBAL routine is scheduled, and then calls the routine in a loop over all “components”.

D2.4.3 Conditional Statements

Any schedule block or assignment statements can be optionally surrounded by conditional if-elseif-else constructs using the parameter data base. These can be nested, and have the general form:

if (<conditional-expression>)
{
  [<assignments>]
  [<schedule blocks>]
}

conditional-expression can be any valid C construct evaluating to a truth value. Such conditionals are evaluated only at program startup, and are used to pick between different static schedule options. For dynamic scheduling, the SCHEDULE WHILE construction should be used.

Conditional constructs cannot be used inside a schedule block.

D2.5 configuration.ccl

[NOTE: The configuration.ccl is still relatively new, and not all features listed below may be fully implemented or functional.]

A configuration.ccl file defines capabilities which a thorn either provides or requires, or may use if available. Unlike implementations, only one thorn providing a particular capability may be compiled into a configuration at one time. Thus, this mechanism may be used to, for example: provide access to external libraries; provide access to functions which other thorns must call, but are too complex for function aliasing; or to split a thorn into several thorns, all of which require some common (not aliased) functions.

A configuration options file can contain any number of the following sections:

D2.5.1 Configuration Scripts

The configuration script may tell the CST to add certain features to the Cactus environment—either to the make system or to header files included by thorns. It does this by outputting lines to its standard output:

No other lines should be output by the script.