xx Method of Lines

Method of Lines

Ian Hawke

Date

Abstract

The Method of Lines is a way of separating the time integration from the rest of an evolution scheme. This thorn is intended to take care of all the bookwork and provide some basic time integration methods, allowing for easy coupling of different thorns.

1 Purpose

The Method of Lines (MoL) converts a (system of) partial differential equation(s) into an ordinary differential equation containing some spatial differential operator. As an example, consider writing the hyperbolic system of PDE’s in the alternative form which (assuming a given discretization of space) is an ODE.

Given this separation of the time and space discretizations, well known stable ODE integrators such as Runge-Kutta can be used to do the time integration. This is more modular (allowing for simple extensions to higher order methods), more stable (as instabilities can now only arise from the spatial discretization or the equations themselves) and also avoids the problems of retaining high orders of convergence when coupling different physical models.

MoL can be used for hyperbolic, parabolic and even elliptic problems (although I definitely don’t recommend the latter). As it currently stands it is set up for systems of equations in the first order type form of equationx(??). If you want to implement a multilevel scheme such as leapfrog it is not obvious to me that MoL is the thing to use. However if you have lots of thorns that you want to interact, for example ADMxBSSN and a hydro code plus maybe EM or a scalar field, and they can easily be written in this sort of form, then you probably want to use MoL.

This thorn is meant to provide a simple interface that will implement the MoL inside Cactus as transparently as possible. It will initially implement only the optimal Runge-Kutta time integrators (which are TVD up to RK3, so suitable for hydro) up to fourth order and iterated Crank Nicholson. All of the interaction with the MoL thorn should occur directly through the scheduler. For example, all synchronization steps should now be possible at the schedule level. This is essential for interacting cleanly with different drivers, especially to make Mesh Refinement work.

For more information on the Method of Lines the most comprehensive references are the works of Jonathan Thornburgx[1,x2] - see especially section 7.3 of the thesis. From the CFD viewpoint the review of ENO methods by Shu,x[3], has some information. For relativistic fluids the paper of Neilsen and Choptuikx[4] is also quite good.

2 How to use

2.1 Thorn users

For those who used the old version of MoL, this version is unfortunately slightly more effort to use. That is, for most methods you’ll now have to set 4 parameters instead of just one.

If you already have a thorn that uses the method of lines, then there are four main parameters that are relevant to change the integration method. The keyword MoLxODExMethod chooses between the different methods. Currently supported are RK2, RK3, ICN, ICN-Avg and Generic. These are second order Runge-Kutta, third order Runge-Kutta, Iterative Crank Nicholson, Iterative Crank Nicholson with averaging, and the generic Shu-Osher type Runge-Kutta methods. To switch between the different types of generic methods there is also the keyword GenericxType which is currently restricted to RK for the standard TVD Runge-Kutta methods (first to fourth order) and ICN for the implementation of the Iterative Crank Nicholson method in generic form.

Full descriptions of the currently implemented methods are given in sectionx4.

The parameter MoLxIntermediatexSteps controls the number of intermediate steps for the ODE solver. For the generic Runge-Kutta solvers it controls the order of accuracy of the method. For the ICN methods this parameter controls the number of iterations taken, which does not check for stability. This parameter defaults to 3.

The parameter MoLxNumxScratchxLevels controls the amount of scratch space used. If this is insufficient for the method selected there will be an error at parameter checking time. This parameter defaults to 0, as no scratch space is required for the efficient ICN and Runge-Kutta 2 and 3 solvers. For the generic solvers this must be at

least MoLxIntermediatexSteps - 1.

Another parameter is MoLxMemoryxAlwaysxOn which switches on memory for the scratch space always if true and only during evolution if false. This defaults to true for speed reasons; the memory gains are likely to be limited unless you’re doing something very memory intensive at initialization or analysis.

There is also a parameter MoLxNaNxCheck that will check your RHS grid functions for NaNs using the NaNChecker thorn from CactusUtils. This will make certain that you find the exact grid function computing the first NaN; of course, this may not be the real source of your problem.

The parameter disablexprolongation only does anything if you are using mesh refinement, and in particular Carpet. With mesh refinement it may be necessary to disable prolongation in intermediate steps of MoL. This occurs when evolving systems containing second spatial derivatives. This is done by default in MoL. If your system is purely first order in space and time you may wish to set this to

"no".

Ideally, initial data thorns should always set initial data at all time levels. However, sometimes initial data thorns fail to do this. In this case you can do one of three things:

Fix the initial data thorn. This is the best solution.

If you’re using Carpet, it has some facilities to take forward/backward time steps to initialize multiple time levels. See the Carpet parameters init_each_timelevel and init_3_timelevels for details.

Finally, if you set (the MoL parameter) initial_data_is_crap, MoL will copy the current time level of all variables it knows about (more precisely, using the terminology of sectionx2.2, all evolved, save-and-restore, and constrained variables which have multiple time levels) to all the past time levels. Note that this copies the same data to each past time level; this will be wrong if your spacetime is time-dependent!

If enabled, the copy happens in the CCTK_POSTINITIAL schedule bin. By default this happens before the MoL_PostStep schedule group; the parameter copy_ID_after_MoL_PostStep can be used to change this to after MoL_PostStep.

2.2 Thorn writers

To port an existing thorn using the method of lines, or to write a new thorn using it, should hopefully be relatively simple. As an example, within the MoL arrangement is WaveMoL which duplicates the WaveToy thorn given by CactusWave in a form suitable for use by MoL. In this section, “the thorn” will mean the user thorn doing the physics.

We start with some terminology. Grid functions are split into four cateogories.

The first are those that are evolved using a MoL form. That is, a right hand side is calculated and the variable updated using it. These we call evolved variables.

The second category are those variables that are set by a thorn at every intermediate step of the evolution, usually to respect the constraints. Examples of these include the primitive variables in a hydrodynamics code. Another example would be the gauge variables if these were set by constraints at every intermediate step (which is slightly artificial; the usual example would be the use of maximal slicing, which is only applied once every complete steps). These are known as constrained variables.

The third category are those variables that a thorn depends on but does not set or evolve. An example would include the metric terms considered from a thorn evolving matter. Due to the way that MoL deals with these, they are known as Save and Restore variables.

The final category are those variables that do not interact with MoL. These would include temporary variables for analysis or setting up the initial data. These can safely be ignored.

As a generic rule of thumb, variables for which you have a time evolution equation are evolved (obviously), variables which your thorn sets but does not evolve are constrained, and any other variables which your thorn reads during evolution is a Save and Restore variable.

MoL needs to know every GF that falls in one of the first three groups. If a GF is evolved by one thorn but is a constrained variable in another (for example, the metric in full GR Hydro) then each thorn should register the function as they see it. For example, the hydro thorn will register the metric as a Save and Restore variable and the spacetime thorn will register the metric as an evolved variable. The different variable categories are given the priority evolved, constrained, Save and Restore. So if a variable is registered as belonging in two different categories, it is always considered by MoL to belong to the category with the highest priority.

MoL needs to know the total number of GFs in each category at parameter time. To do this, your thorn needs to use some accumulator parameters from MoL. As an example, here are the paramaters from WaveMoL:

shares:xMethodOfLines
USESxCCTK_INTxMoL_Num_Evolved_Vars
USESxCCTK_INTxMoL_Num_Constrained_Vars
USESxCCTK_INTxMoL_Num_SaveAndRestore_Vars
restricted:
CCTK_INTxWaveMoL_MaxNumEvolvedVarsx\
xxxx"ThexmaximumxnumberxofxevolvedxvariablesxusedxbyxWaveMoL"x\
xxxxACCUMULATOR-BASE=MethodofLines::MoL_Num_Evolved_Vars
{
xx5:5xxxxxxxxxxx::x"Justx5:xphixandxthexfourxderivatives"
}x5
CCTK_INTxWaveMoL_MaxNumConstrainedVarsx\
xxxx"ThexmaximumxnumberxofxconstrainedxvariablesxusedxbyxWaveMoL"x\
xxxxACCUMULATOR-BASE=MethodofLines::MoL_Num_Constrained_Vars
{
xx0:1xxxxxxxxxxx::x"Axsmallxrange,xdependingxonxtestingxorxnot"
}x1
CCTK_INTxWaveMoL_MaxNumSandRVarsx\
xxxx"ThexmaximumxnumberxofxsavexandxrestorexvariablesxusedxbyxWaveMoL"x\
xxxxACCUMULATOR-BASE=MethodofLines::MoL_Num_SaveAndRestore_Vars
{
xx0:1xxxxxxxxxxx::x"Axsmallxrange,xdependingxonxtestingxorxnot"
}x1

This should give the maximum number of variables that your thorn will register.

Every thorn should register every grid function that it uses even if you expect it to be registered again by a different thorn. For example, a hydro thorn would register the metric variables as Save and Restore, whilst the spacetime evolution thorn would register them as evolved (in ADM) or constrained (in ADMxBSSN), both of which have precedence. To register your GFs with MoL schedule a routine in the bin MoLxRegister which just contains the relevant function calls. For an evolved variable the GF corresponding to the update term ( in equationx(??)) should be registered at the same time. The appropriate functions are given in sectionx5.

These functions are provided using function aliasing. For details on using function aliasing, see sections B10.5 and F2.2.3 of the UsersGuide. For the case of real GFs, you simply add the following lines to your

interface.ccl:

##########################################
###xPROTOTYPESx-xDELETExASxAPPLICABLE!x###
##########################################
CCTK_INTxFUNCTIONxMoLRegisterEvolved(CCTK_INTxEvolvedIndex,xCCTK_INTxRHSIndex)
CCTK_INTxFUNCTIONxMoLRegisterEvolvedSlow(CCTK_INTxEvolvedIndex,xCCTK_INTxRHSIndex)
CCTK_INTxFUNCTIONxMoLRegisterConstrained(CCTK_INTxConstrainedIndex)
CCTK_INTxFUNCTIONxMoLRegisterSaveAndRestore(CCTK_INTxSandRIndex)
CCTK_INTxFUNCTIONxMoLRegisterEvolvedGroup(CCTK_INTxEvolvedIndex,x\
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)
CCTK_INTxFUNCTIONxMoLRegisterConstrainedGroup(CCTK_INTxConstrainedIndex)
CCTK_INTxFUNCTIONxMoLRegisterSaveAndRestoreGroup(CCTK_INTxSandRIndex)
CCTK_INTxFUNCTIONxMoLChangeToEvolved(CCTK_INTxEvolvedIndex,xCCTK_INTxRHSIndex)
CCTK_INTxFUNCTIONxMoLChangeToEvolvedSlow(CCTK_INTxEvolvedIndex,xCCTK_INTxRHSIndex)
CCTK_INTxFUNCTIONxMoLChangeToConstrained(CCTK_INTxConstrainedIndex)
CCTK_INTxFUNCTIONxMoLChangeToSaveAndRestore(CCTK_INTxSandRIndex)
CCTK_INTxFUNCTIONxMoLChangeToNone(CCTK_INTxRemoveIndex)
#############################################
###xUSExSTATEMENTx-xDELETExASxAPPLICABLE!x###
#############################################
USESxFUNCTIONxMoLRegisterEvolved
USESxFUNCTIONxMoLRegisterEvolvedSlow
USESxFUNCTIONxMoLRegisterConstrained
USESxFUNCTIONxMoLRegisterSaveAndRestore
USESxFUNCTIONxMoLRegisterEvolvedGroup
USESxFUNCTIONxMoLRegisterConstrainedGroup
USESxFUNCTIONxMoLRegisterSaveAndRestoreGroup
USESxFUNCTIONxMoLChangeToEvolved
USESxFUNCTIONxMoLChangeToConstrained
USESxFUNCTIONxMoLChangeToSaveAndRestore
USESxFUNCTIONxMoLChangeToNone

Note that the list of parameters not complete; see the section on parameters for the use of arrays. However, the list of functions is, and is expanded on in sectionx5. MoL will check whether a group or variable is a GF or an array and whether it is real.

Having done that, one routine (or group of routines) which we’ll here call ThornxCalcRHS must be defined. This does all the finite differencing that you’d usually do, applied to , and finds the right hand sides which are stored in . This routine should be scheduled in MoLxCalcRHS. The precise order that these are scheduled should not matter, because no updating of any of the user thorns will be done until after all the RHSs are calculated. Important note: all the finite differencing must be applied to the most recent time level and not to the previous time level as you would normally do. Don’t worry about setting up the data before the calculation, as MoL will do that automatically.

Finally, if you have some things that have to be done after each update to an intermediate level, these should be scheduled in MoLxPostStep. Examples of things that need to go here include the recalculation of primitive variables for hydro codes, the application of boundary

conditions1 1It is possible to alter the calculation of L so that boundary conditions are automatically updated and do not need setting. This is slightly tricksy. For an example of how this would work see the new radiative boundary condition in ADMxBSSN. For more on this see section 7.3.4 ofx[1]. , the solution of elliptic equations (although this would be a very expensive place to solve them, some sets of equations might require the updating of some variables by constraints in this fashion). When applying boundary conditions the cleanest thing to do is to write a routine applying the symmetries to the appropriate GFs and, when calling it from the scheduler, adding the SYNC statement to the appropriate groups. An example is given by the routine WaveToyMoLxBoundaries in thorn WaveMoL.

Points to note. The thorn routine ThornxCalcRHS does not need to know and in fact should definitely not know where precisely in the MoL step it is. It just needs to know that it is receiving some intermediate data stored in the GFs and that it should return the RHS . All the book-keeping to ensure that it is passed the correct intermediate state at that the GFs contain the correct data at the end of the MoL step will be dealt with by the MoL thorn and the flesh.

When using a multirate scheme the thorn routine ThornxCalcRHS must check the MoL grid scalars MoL::MoLxSlowStep and MoL::MoLxSlowPostStep which MoL sets to true (non-zero) if it is time for the slow sector to compute its RHS or to apply its poststep routine. MoL::MoLxSlowPostStep is always true outside of

the RHS loop, eg. in CCTKxPOSTxRESTRICT.

2.3 Evolution method writers

If you want to try adding a new evolution method to MoL the simplest way is to use the generic table option to specify it completely in the parameter file - no coding is required at all.

All the generic methods evolve the equation using the following algorithm for an -step method:

This method is completely specified by (GenericIntermediateSteps) and the (GenericAlphaCoeffs) and (GenericBetaCoeffs) arrays. The names in parentheses give the keys in a table that MoL will use. This table is created from the string parameter GenericxMethodxDescriptor.

As an example, the standard TVD RK2 method that is implemented both in optimized and generic form is written as To implement this using the generic table options, use

methodoflines::MoL_Intermediate_Stepsx=x2
methodoflines::MoL_Num_Scratch_Levelsx=x1
methodoflines::Generic_Method_Descriptorx=x\
xxxxxxxxxxx"GenericIntermediateStepsx=x2x\
xxxxxxxxxxxxGenericAlphaCoeffsx=x{x1.0x0.0x0.5x0.5x}x\
xxxxxxxxxxxxGenericBetaCoeffsx=x{x1.0x0.5x}"

The number of steps specified in the table must be the same as MoLxIntermediatexSteps, and the number of scratch levels should be at least MoLxIntermediatexSteps - 1.

The generic methods are somewhat inefficient for use in production runs, so it is frequently better to write an optimized version once you are happy with the method. To do this you should

write your code into a new file, called from the scheduler under the alias MoLxAdd,

make certain that at each intermediate step the correct values of cctkxtime and cctkxdeltaxtime are set in SetTime.c for mesh refinement, boundary conditions and so on,

make certain that you check for the number of intermediate steps in ParamCheck.c.

3 Example

As a fairly extended example of how to use MoL I’ll outline how ADMxBSSN works in this context. The actual implementation of this is given in the thorn AEIThorns/BSSNxMoL.

As normal the required variables are defined in the interface.ccl file, together with the associated source terms. For example, the conformal factor and source are defined by

realxADM_BSSN_phixtype=GFxtimelevels=2
{
xxADM_BS_phi
}x"ADM_BSSN_phi"
realxADM_BSSN_sourcesxtype=GF
{
...,
xxadm_bs_sphi,
...
}

Also in this file we write the function aliasing prototypes.

Once the sources are defined the registration with MoL is required, for which the essential file is MoLRegister.c. In the ADMxBSSN system the standard metric coefficients are not evolved, and neither are the standard extrinsic curvature components . However these are used by ADMxBSSN in a number of places, and are calculated from evolved quantities at the appropriate points. In the MoL terminology these variables are constrained. As the appropriate storage is defined in thorn ADMBase, the actual calls have the form

xierrx+=xMoLRegisterConstrained(CCTK_VarIndex("ADMBase::kxx"));

The actual evolved variables include things such as the conformal factor. This, and the appropriate source term, is defined in thorn ADMxBSSN, and so the call has the form

xierrx+=xMoLRegisterEvolved(CCTK_VarIndex("adm_bssn::ADM_BS_phi"),
xxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_VarIndex("adm_bssn::adm_bs_sphi"));

As well as the evolved variables, and those constrained variables such as the metric, there are the gauge variables. Precisely what status these have depends on how they are set. If harmonic or 1+log slicing is used then the lapse is evolved:

xierrx+=xMoLRegisterEvolved(CCTK_VarIndex("ADMBase::alp"),
xxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_VarIndex("adm_bssn::adm_bs_salp"));

A matter density might not require such a high order scheme and can be evolved using a multi-rate scheme

xierrx+=xMoLRegisterEvolvedSlow(CCTK_VarIndex("GRHydro::dens"),
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_VarIndex("GRHydro::dendsrhs"));

If maximal or static slicing is used then the lapse is a constrained

variable2 2Note that this is actually a bit of a hack. The rational for Save and Restore variables was to deal with maximal slicing. However it turned out that I hadn’t thought it through correctly and that the treatment for constrained variables was required.:

xierrx+=xMoLRegisterConstrained(CCTK_VarIndex("ADMBase::alp"));

Finally, if none of the above apply we assume that the lapse is evolved in some unknown fashion, and so it must be registered as a Save and Restore variable:

xierrx+=xMoLRegisterSaveAndRestore(CCTK_VarIndex("ADMBase::alp"));

However, it is perfectly possible that we may wish to change how we deal with the gauge during the evolution. This is dealt with in the file PreLoop.F. If the slicing changes then the appropriate routine is called. For example, if we want to use 1+log evolution then we call

xcallxCCTK_VarIndex(lapseindex,"ADMBase::alp")
xcallxCCTK_VarIndex(lapserhsindex,"adm_bssn::adm_bs_salp")
xierrx=xierrx+xMoLChangeToEvolved(lapseindex,xlapserhsindex)

It is not required to tell MoL what the lapse is changing from, or indeed if it is changing at all; MoL will work this out for itself.

Finally there are the routines that we wish to apply after every intermediate step. These are ADMxBSSNxremovetrA which enforces various constraints (such as the tracefree conformal extrinsic curvature remaining trace free), ADMxBSSNxBoundaries which applies symmetry boundary conditions as well as various others (such as some of the radiative boundary conditions). Note all the calls to SYNC at this point. We also convert from the updated BSSN variables back to the standard ADM variables in ADMxBSSNxStandardVariables, and also update the time derivative of the lapse in

ADMxBSSNxLapseChange.

4 Time evolution methods provided by MoL

The default method is Iterative Crank-Nicholson. There are many ways of implementing this. The standard "ICN" and "Generic"/"ICN" methods both implement the following, assuming an iteration method:

The “averaging” ICN method "ICN-avg" instead calculates intermediate steps before averaging:

The Runge-Kutta methods are those typically used in hydrodynamics by, e.g., Shu and others — seex[3] for example. Explicitly the first order method is the Euler method:

The second order method is:

The third order method is:

The fourth order method, which is not strictly TVD, is:

4.1 Multirate methods

A scheme for coupling different parts of a system of equations

representing eg. spacetime and matter variables, respectively, with different RK integrators is given by multirate RK schemes (e.g.x[5,x6]). Here, we pursuit the simple Ansatz of performing one RK2 intermediate RHS evaluation for two RK4 intermediate RHS evaluations. That is, the additional RK4 intermediate RHS evaluations simple use the results from the last intermediate RK2 step.

To be more explicit, given the equation where corresponds to the RHS possibly including spatial derivatives, we write a generic RK scheme according to

The coefficients , , and can be written in the standard Butcher notation.

In our multirate scheme, we use two different sets of coefficients. One set of coefficients determines the RK4 scheme used for integrating the spacetime variables (??), the other set determines the RK2 scheme for the hydro variables (??). The coefficients for the RK2 scheme are arranged such that RHS evaluations coincide with RK4 RHS evaluations. We list the corresponding multirate Butcher tableau in Tablex1.

Tablex1: Butcher tableau for an explicit multirate RK4/RK2 scheme. The right table (separated by the double vertical line) shows the coefficients (bottom line), (first vertical column), and for the classical RK4 scheme. The left table shows the corresponding RK2 coefficients evaluated at timesteps that coincide with RK4 timesteps.

 11011   11 11 11   110 11   11
110 110 11 11 11   111/2111/2
110 110 11011 11   111/2110 111/211   11
111 111 11011011   111 110 110 111/211
11  111/2110110111/211   111/3111/6111/6111/3

5 Functions provided by MoL All the functions listed below return error codes in theory. However at this current point in time they always return 0 (success). Any failure to register or change a GF is assumed fatal and MoL will issue a level 0 warning stopping the code. This may change in future, in which case negative return values will indicate errors. These are all aliased functions. You can get the functions directly through header files, but this feature may be phased out. Using function aliasing is the recommended method.

MoLRegisterEvolved

Tells MoL that the given GF is in the evolved category with the associated update GF. Synopsis

CCTK_INTxierrx=xMoLRegisterEvolved(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)
CCTK_INTxierrx=xMoLRegisterEvolvedSlow(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)

CCTK_INTxierrx=xMoLRegisterEvolved(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)
CCTK_INTxierrx=xMoLRegisterEvolvedSlow(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)

Result

Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters

EvolvedIndex Index of the GF to be evolved. RHSIndex Index of the associated update GF.

Discussion

Should be called in a function scheduled in MoLxRegister. Use the Slow variant to register the slow sector of a multirate scheme.

See Also

CCTKxVarIndex() Get the variable index. MoLRegisterSaveAndRestore() Register Save and Restore variables. MoLRegisterConstrained() Register constrained variables. MoLChangeToEvolved() Change a variable at runtime to be evolved. MoLChangeToEvolvedSlow() Change a variable at runtime to be evolved in the slow sector.

C

ierrx=xMoLRegisterEvolved(CCTK_VarIndex("wavetoymol::phi"),
xxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_VarIndex("wavetoymol::phirhs"));

callxCCTK_VarIndex(EvolvedIndex,x"wavetoymol::phi")
callxCCTK_VarIndex(RHSIndex,x"wavetoymol::phirhs")
ierrx=xMoLRegisterEvolved(EvolvedIndex,xRHSIndex)

MoLRegisterConstrained

Tells MoL that the given GF is in the constrained category.

Synopsis

CCTK_INTxierrx=xMoLRegisterConstrained(CCTK_INTxConstrainedIndex)

Fortran

CCTK_INTxierrx=xMoLRegisterConstrained(CCTK_INTxConstrainedIndex)

Result Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters ConstrainedIndex Index of the constrained GF.

Discussion Should be called in a function scheduled in MoLxRegister.

See Also CCTKxVarIndex() Get the variable index. MoLRegisterEvolved() Register evolved variables. MoLRegisterSaveAndRestore() Register Save and Restore variables. MoLChangeToConstrained() Change a variable at runtime to be constrained.

ierrx=xMoLRegisterConstrained(CCTK_VarIndex("ADMBase::alp"));

Fortran

callxCCTK_VarIndex(ConstrainedIndex,x"ADMBase::alp")
ierrx=xMoLRegisterConstrained(ConstrainedIndex)

MoLRegisterSaveAndRestore

Tells MoL that the given GF is in the Save and Restore category. Synopsis

CCTK_INTxierrx=xMoLRegisterSaveAndRestore(CCTK_INTxSandRIndex)

Fortran

CCTK_INTxierrx=xMoLRegisterSaveAndRestore(CCTK_INTxSandRIndex)

Result Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters SandRIndex Index of the Save and Restore GF.

Discussion Should be called in a function scheduled in MoLxRegister.

See Also CCTKxVarIndex() Get the variable index. MoLRegisterEvolved() Register evolved variables. MoLRegisterConstrained() Register constrained variables. MoLChangeToSaveAndRestore() Change a variable at runtime to be Save and Restore.

ierrx=xMoLRegisterSaveAndRestore(CCTK_VarIndex("ADMBase::alp"));

Fortran

callxCCTK_VarIndex(SandRIndex,x"ADMBase::alp")
ierrx=xMoLRegisterSaveAndRestore(SandRIndex)

MoLRegisterEvolvedGroup

Tells MoL that the given group is in the evolved category with the associated update group. Synopsis

CCTK_INTxierrx=xMoLRegisterEvolvedGroup(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)
CCTK_INTxierrx=xMoLRegisterEvolvedGroupSlow(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)

CCTK_INTxierrx=xMoLRegisterEvolvedGroup(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)
CCTK_INTxierrx=xMoLRegisterEvolvedGroupSlow(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)

Result

Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters

EvolvedIndex Index of the group to be evolved. RHSIndex Index of the associated update group.

Discussion

Should be called in a function scheduled in MoLxRegister. Use the Slow variant to register the slow sector of a multirate scheme.

See Also

CCTKxGroupIndex() Get the group index.
MoLRegisterSaveAndRestoreGroup() Register Save and Restore variables.
MoLRegisterConstrainedGroup() Register constrained variables.

Examples

ierrx=xMoLRegisterEvolvedGroup(CCTK_GroupIndex("wavetoymol::scalarevolvemol"),
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_GroupIndex("wavetoymol::scalarevolvemolrhs"));

callxCCTK_GroupIndex(EvolvedIndex,x"wavetoymol::scalarevolvemol")
callxCCTK_GroupIndex(RHSIndex,x"wavetoymol::scalarevolvemolrhs")
ierrx=xMoLRegisterEvolvedGroup(EvolvedIndex,xRHSIndex)

MoLRegisterConstrainedGroup

Tells MoL that the given group is in the constrained category.

Synopsis

CCTK_INTxierrx=xMoLRegisterConstrainedGroup(CCTK_INTxConstrainedIndex)

Fortran

CCTK_INTxierrx=xMoLRegisterConstrainedGroup(CCTK_INTxConstrainedIndex)

Result Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters ConstrainedIndex Index of the constrained group.

Discussion Should be called in a function scheduled in MoLxRegister.

See Also

CCTKxGroupIndex() Get the group index.
MoLRegisterEvolvedGroup() Register evolved variables.
MoLRegisterSaveAndRestoreGroup() Register Save and Restore variables.

MoLChangeToConstrained() Change a variable at runtime to be constrained.

ierrx=xMoLRegisterConstrainedGroup(CCTK_GroupIndex("ADMBase::lapse"));

Fortran

callxCCTK_GroupIndex(ConstrainedIndex,x"ADMBase::lapse")
ierrx=xMoLRegisterConstrainedGroup(ConstrainedIndex)

MoLRegisterSaveAndRestoreGroup

Tells MoL that the given group is in the Save and Restore category. Synopsis

CCTK_INTxierrx=xMoLRegisterSaveAndRestoreGroup(CCTK_INTxSandRIndex)

Fortran

CCTK_INTxierrx=xMoLRegisterSaveAndRestoreGroup(CCTK_INTxSandRIndex)

Result Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters SandRIndex Index of the save and restore group.

Discussion Should be called in a function scheduled in MoLxRegister.

See Also

CCTKxGroupIndex() Get the group index.
MoLRegisterEvolvedGroup() Register evolved variables.
MoLRegisterConstrainedGroup() Register constrained variables.

ierrx=xMoLRegisterSaveAndRestoreGroup(CCTK_GroupIndex("ADMBase::shift"));

Fortran

callxCCTK_GroupIndex(SandRIndex,x"ADMBase::shift")
ierrx=xMoLRegisterSaveAndRestoreGroup(SandRIndex)

MoLChangeToEvolved

Sets a GF to belong to the evolved category, with the associated update GF. Not used for the initial setting.

Synopsis

CCTK_INTxierrx=xMoLChangeToEvolved(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)
CCTK_INTxierrx=xMoLChangeToEvolvedSlow(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)

CCTK_INTxierrx=xMoLChangeToEvolvedSlow(CCTK_INTxEvolvedIndex,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_INTxRHSIndex)

Result

Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters

EvolvedIndex Index of the evolved GF. RHSIndex Index of the associated update GF.

Discussion

Should be called in a function scheduled in MoLxPreStep. Note that this function was designed to allow mixed slicings for thorn ADMBase. This set of functions is largely untested and should be used with great care.

See Also

CCTKxVarIndex() Get the variable index. MoLRegisterEvolved() Register evolved variables. MoLChangeToSaveAndRestore() Change a variable at runtime to be Save and Restore. MoLChangeToConstrained() Change a variable at runtime to be constrained.

Examples

ierrx=xMoLChangeToEvolved(CCTK_VarIndex("ADMBase::alp"),
xxxxxxxxxxxxxxxxxxxxxxxxxxCCTK_VarIndex("adm_bssn::adm_bs_salp"));

callxCCTK_VarIndex(EvolvedIndex,x"ADMBase::alp")
callxCCTK_VarIndex(RHSIndex,"adm_bssn::adm_bs_salp")
ierrx=xMoLChangeToEvolved(EvolvedIndex,xRHSIndex)

MoLChangeToConstrained

Sets a GF to belong to the constrained category. Not used for the initial setting.

Synopsis

CCTK_INTxierrx=xMoLChangeToConstrained(CCTK_INTxEvolvedIndex)

Fortran

CCTK_INTxierrx=xMoLChangeToConstrained(CCTK_INTxEvolvedIndex)

Result Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters ConstrainedIndex Index of the constrained GF.

Discussion Should be called in a function scheduled in MoLxPreStep. Note that this function was designed to allow mixed slicings for thorn ADMBase. This set of functions is largely untested and should be used with great care.

See Also CCTKxVarIndex() Get the variable index. MoLRegisterConstrained() Register constrained variables. MoLChangeToSaveAndRestore() Change a variable at runtime to be Save and Restore. MoLChangeToEvolved() Change a variable at runtime to be evolved.

ierrx=xMoLChangeToConstrained(CCTK_VarIndex("ADMBase::alp"));

Fortran

callxCCTK_VarIndex(EvolvedIndex,x"ADMBase::alp")
ierrx=xMoLChangeToConstrained(EvolvedIndex)

MoLChangeToSaveAndRestore

Sets a GF to belong to the Save and Restore category. Not used for the initial setting. Synopsis

CCTK_INTxierrx=xMoLChangeToSaveAndRestore(CCTK_INTxSandRIndex)

Fortran

CCTK_INTxierrx=xMoLChangeToSaveAndRestore(CCTK_INTxSandRIndex)

Result Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters SandRIndex Index of the Save and Restore GF.

Discussion Should be called in a function scheduled in MoLxPreStep. Note that this function was designed to allow mixed slicings for thorn ADMBase. This set of functions is largely untested and should be used with great care.

See Also CCTKxVarIndex() Get the variable index. MoLRegisterSaveAndRestore() Register Save and Restore variables. MoLChangeToEvolved() Change a variable at runtime to be evolved. MoLChangeToConstrained() Change a variable at runtime to be constrained.

ierrx=xMoLChangeToSaveAndRestore(CCTK_VarIndex("ADMBase::alp"));

Fortran

callxCCTK_VarIndex(SandRIndex,x"ADMBase::alp")
ierrx=xMoLChangeToSaveAndRestore(SandRIndex)

MoLChangeToNone

Sets a GF to belong to the “unknown” category. Not used for the initial setting. Synopsis

CCTK_INTxierrx=xMoLChangeToNone(CCTK_INTxRemoveIndex)

Fortran

CCTK_INTxierrx=xMoLChangeToNone(CCTK_INTxRemoveIndex)

Result Currently if there is an error, MoL will issue a level 0 warning. No sensible return codes exist. 0 success

Parameters RemoveIndex Index of the “unknown” GF.

Discussion Should be called in a function scheduled in MoLxPreStep. Note that this function was designed to allow mixed slicings for thorn ADMBase. This set of functions is largely untested and should be used with great care.

See Also CCTKxVarIndex() Get the variable index. MoLChangeToEvolved() Change a variable at runtime to be evolved. MoLChangeToSaveAndRestore() Change a variable at runtime to be Save and Restore. MoLChangeToConstrained() Change a variable at runtime to be constrained.

ierrx=xMoLChangeToNone(CCTK_VarIndex("ADMBase::alp"));

Fortran

callxCCTK_VarIndex(RemoveIndex,x"ADMBase::alp")
ierrx=xMoLChangeToNone(RemoveIndex)

MoLQueryEvolvedRHS

Queries MoL for the index of the update variable for given GF in the evolved category. Synopsis

CCTK_INTxRHSindexx=xMoLQueryEvolvedRHS(CCTK_INTxEvolvedIndex)

Fortran

CCTK_INTxRHSindexx=xMoLQueryEvolvedRHS(CCTK_INTxEvolvedIndex)

Result If the grid function passed does not exists, MoL will issue a level 0 warning. If the grid function is not of an evolved type (fast or slow sector) will be returned. Otherwise the variable index of the update GF is returned. variable index of update GF

Parameters EvolvedIndex Index of the GF whose update GF is to be returned.

Discussion Both slow and fast evolved variables can be queried.

See Also CCTKxVarIndex() Get the variable index. MoLRegisterEvolved() Register evolved variables. MoLChangeToEvolvedSlow() Change a variable at runtime to be evolved in the slow sector.

rhsindexx=xMoLQueryEvolvedRHS(CCTK_VarIndex("wavetoymol::phi"));

Fortran

callxCCTK_VarIndex(EvolvedIndex,x"wavetoymol::phi")
rhsindexx=xMoLQueryEvolvedRHS(EvolvedIndex)

References

[1]xxxJ. Thornburg. Numerical Relativity in Black Hole Spacetimes. Unpublished thesis, University of

British Columbia. 1993. Available from http://www.aei.mpg.de/~jthorn/phd/html/phd.html.

[2]xxxJ. Thornburg. A 3+1 Computational Scheme for Dynamic Spherically Symmetric Black Hole Spacetimes – II: Time Evolution. Preprint gr-qc/9906022, submitted to Phys. Rev. D.

[3]xxxC. Shu. High Order ENO and WENO Schemes for Computational Fluid Dynamics. In T.xJ. Barth and H. Deconinck, editors High-Order Methods for Computational Physics. Springer, 1999. A related online version can be found under Essentially Non-Oscillatory and Weighted Essentially Non-Oscillatory Schemes for Hyperbolic Conservation Laws at

http://www.icase.edu/library/reports/rdp/97/97-65RDP.tex.refer.html.

[4]xxxD.xW. Neilsen and M.xW. Choptuik. Ultrarelativistic fluid dynamics. Class. Quantum Grav., 17: 733–759, 2000.

[5]xxxM. Schlegel, O. Knoth, M. Arnold, and R. Wolke Journal of Computational and Applied Mathematics, 226, 345, 2009.

[6]xxxE. Constantinescu and A. Sandu SIAM J. Sci.xComput., 33, 239, 2007.

6 Parameters

11abxinitiallyxreducexorder 10Scope: privatex01 BOOLEAN

11Description: Reduce order of accuracy initially so that no past timelevels of initial data are required

11                                10  x01  Default: yes

11

11abxtype  10Scope: privatex01  KEYWORD

11Description: If using the the AB method, which sort

11 Range 10 x01 Default: 1
111  10same as forward Euler
112  10second order
113  10third order
114  10fourth order
115  10fifth order

11

11adaptivexstepsize  10Scope: privatex01  BOOLEAN

11Description: Choose the time step size adaptively

11                                10  x01  Default: no

11

11copyxidxafterxmolxpoststep  10Scope: privatex01  BOOLEAN

11Description: if initialxdataxisxcrap is true, *when* should we copy the current time level to all previous time levels: false ==> copy *before* MoLxPostStep (default, matches old behavior) true ==> copy *after* MoLxPostStep (maybe preferable for new code)

11                                10  x01  Default: no

11

11disablexprolongation  10Scope: privatex01  BOOLEAN

11Description: If Mesh refinement is enabled should we use buffer zones in intermediate steps?

11                                10  x01  Default: yes

11

11genericxmethodxdescriptor  10Scope: privatex01  STRING

11Description: A string used to create a table containing the description of the generic method

11 Range 10 x01Default: GenericIntermediateSteps = 2 GenericAlphaCoeffs = { 1.0 0.0 0.5 0.5 } GenericBetaCoeffs = { 1.0 0.5 }
11.*  10Should contain the Alpha and Beta arrays, and the number of intermediate steps

11genericxtype  10Scope: privatex01  KEYWORD

11Description: If using the generic method, which sort

11 Range 10 x01 Default: RK
11RK  10One of the standard TVD Runge-Kutta methods
11ICN  10Iterative Crank Nicholson as a generic method
11Table  10Given from the generic method descriptor parameter
11Classic RK3  10Efficient RK3 - classical version

11

11icnxavgxswapped  10Scope: privatex01  BOOLEAN

11Description: Use swapped averages in ICN method?

11                                10  x01  Default: no

11

11icnxavgxtheta  10Scope: privatex01  REAL

11Description: theta of averaged ICN method, usually 0.5

11 Range 10 x01 Default: 0.5
110:1  100 <= theta <= 1 11
11initxrhsxzero  10Scope: privatex01  BOOLEAN

11Description: Initialise the RHS to zero

11                                10  x01  Default: yes

11

11initialxdataxisxcrap  10Scope: privatex01  BOOLEAN

11Description: If the initial data routine fails to set up the previous time levels, copy the current backwards

11                                10  x01  Default: no

11

11maximumxabsolutexerror  10Scope: privatex01  REAL

11Description: Maximum allowed absolute error for adaptive stepsize control

11 Range 10 x01 Default: 1.0e-6
110.0:*)  10

11

11maximumxdecrease  10Scope: privatex01  REAL

11Description: Maximum stepsize decrease factor

11 Range 10 x01 Default: 10.0
11(1.0:*)  10should be larger than one

11

11maximumxincrease  10Scope: privatex01  REAL

11Description: Maximum stepsize increase factor

11 Range 10 x01 Default: 5.0
11(1.0:*)  10should be larger than one

11

11maximumxrelativexerror  10Scope: privatex01  REAL

11Description: Maximum allowed relative error for adaptive stepsize control

11 Range 10 x01 Default: 1.0e-6
110.0:*)  10

11

11molxintermediatexsteps  10Scope: privatex01  INT

11Description: Number of intermediate steps taken by the ODE method

11 Range 10 x01 Default: 3
111:*  10Anything greater than 1

11

11molxmemoryxalwaysxon  10Scope: privatex01  BOOLEAN

11Description: Do we keep the scratch arrays allocated all the time?

11                                10  x01  Default: yes

11molxnanxcheck  10Scope: privatex01  BOOLEAN

11Description: Should the RHS GFs be checked for NaNs?

11                                10  x01  Default: no

11

11molxtiny  10Scope: privatex01  REAL

11Description: Effective local machine zero; required by generic solvers

11 Range 10 x01 Default: 1.e-15
110:*  10Defaults to 1.e-15

11

11odexmethod  10Scope: privatex01  KEYWORD

11Description: The ODE method use by MoL to do time integration

11 Range 10 x01 Default: ICN
11Generic  10Generic Shu-Osher Runge-Kutta type
11ICN  10Iterative Crank Nicholson
11ICN-avg  10Iterative Crank Nicholson with averaging
11Euler  10Euler
11RK2  10Efficient RK2
11RK2-central  10Central RK2
11RK3  10Efficient RK3
11RK4  10Efficient RK4
11RK45  10RK45 (Fehlberg) with error estimation
11RK45CK  10RK45CK (Cash-Karp) with error estimation
11RK65  10RK65 with error estimation
11RK87  10RK87 with error estimation
11AB  10Adams-Bashforth
11RK2-MR-2:1  102nd order 2:1 multirate RK scheme based on RK2 due to Schlegel et al 2009. This requires initxRHSxzero=’no’.
11RK4-MR-2:1  103rd order 2:1 multirate RK scheme based on RK43 due to Schlegel et al 2009. This requires initxRHSxzero=’no’.
11RK4-RK2  10RK4 as fast method and RK2 as slow method

11

11rhsxerrorxweight  10Scope: privatex01  REAL

11Description: Weight of the RHS in the relative error calculation

11 Range 10 x01 Default: 1.0
110.0:*  10should be between zero and one

11

11runxmolxpoststepxinxpostxrecoverxvariables 10Scope: privatex01  BOOLEAN

11Description: Schedule the PostStep parts after recovery so that symmetries are automatically done correctly.

11                                10  x01  Default: yes

11

11safetyxfactor  10Scope: privatex01  REAL

11Description: Safety factor for stepsize control

11 Range 10 x01 Default: 0.9
11(0.0:*)  10should be less than one

11

11setxidxboundaries  10Scope: privatex01  BOOLEAN

11Description: Should boundaries be overwritten (via synchronization, prolongation, boundary conditions) by MoL?

11                                10  x01  Default: yes

11

11skipxinitialxcopy  10Scope: privatex01  BOOLEAN

11Description: Skip initial copy from previous to current time level

11                                10  x01  Default: no

11verbose  10Scope: privatex01  KEYWORD

11Description: How verbose should MoL be?

11 Range 10 x01 Default: normal
11none  10No output at all (not implemented)
11normal  10Standard verbosity
11register  10List the variables registered as well
11extreme  10Everything you never wanted to know

11

11molxmaxxevolvedxarrayxsize  10Scope: restrictedx01  INT

11Description: The maximum total size of any grid arrays to be evolved

11 Range 10 x01 Default: (none)
110:*  10Anything non negative. Accumulated by other thorns

11

11molxnumxarrayconstrainedxvars  10Scope: restrictedx01  INT

11Description: The maximum number of array constrained variables with timelevels that MoL needs to know about (DPRECATED)

11  Range  10  x01  Default: (none)
110:*  10Anything non negative. Added to by other thorns.

11

11molxnumxarrayevolvedxvars  10Scope: restrictedx01  INT

11Description: The maximum number of array variables to be evolved by MoL (DPRECATED)

11 Range 10 x01 Default: (none)
110:*  10Anything non negative. Added to by other thorns.

11

11molxnumxarraysaveandrestorexvars 10Scope: restrictedx01  INT

11Description: The maximum number of array variables to be evolved outside of MoL but that MoL needs to know about (DPRECATED)

11  Range  10  x01  Default: (none)
110:*  10Anything non negative. Added to by other thorns.

11

11molxnumxconstrainedxvars  10Scope: restrictedx01  INT

11Description: The maximum number of constrained variables with timelevels that MoL needs to know about (DPRECATED)

11  Range  10  x01  Default: (none)
110:*  10Anything non negative. Added to by other thorns.

11

11molxnumxevolvedxvars  10Scope: restrictedx01  INT

11Description: The maximum number of variables to be evolved by MoL (DPRECATED)

11 Range 10 x01 Default: (none)
110:*  10Anything non negative. Added to by other thorns.

11

11molxnumxevolvedxvarsxslow  10Scope: restrictedx01  INT

11Description: The maximum number of ’slow’ variables to be evolved by MoL (DPRECATED)

11 Range 10 x01 Default: (none)
110:*  10Anything non negative. Added to by other thorns.

11

11molxnumxsaveandrestorexvars  10Scope: restrictedx01  INT

11Description: The maximum number of variables to be evolved outside of MoL but that MoL needs to know about (DPRECATED)

11  Range  10  x01  Default: (none)
110:*  10Anything non negative. Added to by other thorns.

11molxnumxscratchxlevels  10Scope: restrictedx01  INT

11Description: Number of scratch levels required by the ODE method

11 Range 10 x01 Default: (none)
110:*  10Anything non negative

11

11cctkxinitialxtime  10Scope: shared from CACTUSx01REAL

11

11presyncxmode  10Scope: shared from CACTUSx01KEYWORD

11

7 Interfaces

General Implements: methodoflines

Grid Variables

7.0.1 PRIVATE GROUPS

11x Group Names x10 x Variable Names x x01 Details x11x
11 rkalphacoefficients 10 RKAlphaCoefficients x01  compact110
11                 10  x01 dimensions112
11                 10  x01 distribution11CONSTANT
11                 10  x01 group type11ARRAY
11                 10  x01  size11MOLxINTERMEDIATExSTEPS
11                 10  x  x01  size11MOLxNUMxSCRATCHxLEVELS+1
11                 10  x01  tags11Checkpoint=”no”
11                 10  x01 timelevels111
11                 10  x01variable type11REAL
11 rkbetacoefficients 10 RKBetaCoefficients x01  compact110
11                 10  x01 dimensions111
11                 10  x01 distribution11CONSTANT
11                 10  x01 group type11ARRAY
11                 10  x01  size11MOLxINTERMEDIATExSTEPS
11                 10  x01  tags11Checkpoint=”no”
11                 10  x01 timelevels111
11                 10  x01variable type11REAL
11 molxcounters 10 x01 compact110
11                 10MoLxIntermediatexStepx01 description11The counter for the time integration method
11                 10 MoLxStepsizexBad x01 dimensions110
11 10 MoLxSlowStep x01 distribution11CONSTANT
11                 10 MoLxSlowPostStep x01 group type11SCALAR
11 10 x01 tags11Checkpoint=”no”
11                 10  x01 timelevels111
11                 10  x01variable type11INT
11 molxoriginalxtime 10  x01  compact110
11                 10  OriginalxTime  x01 description11The original time and delta time which are reset by MoL during evolution
11                 10 OriginalxDeltaxTime x01 dimensions110
11                 10  x01 distribution11CONSTANT
11                 10  x01 group type11SCALAR
11 10 x01 tags11Checkpoint=”no”
11                 10  x01 timelevels111
11                 10  x01variable type11REAL
11 scratchspaceslow 10 ScratchSpaceSlow x01  compact110
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11Prolongation=”None” Checkpoint=”no”
11                 10  x01 timelevels1199
11                 10  x01vararrayxsize11MoLxNumxScratchxLevels
11                 10  x01variable type11REAL
11 scratchspace 10 ScratchSpace x01 compact110
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11Prolongation=”None” Checkpoint=”no”
11                 10  x01 timelevels1199
11                 10  x01vararrayxsize11MoLxNumxScratchxLevels
11                 10  x01variable type11REAL

11

11x Group Names x10x Variable Names xx01 Details x11x
11 sandrscratchspace 10 SandRScratchSpace x01  compact110
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11Prolongation=”None” Checkpoint=”no”
11                 10  x01 timelevels1199
11                 10  x01variable type11REAL
11 errorestimate 10 ErrorEstimate x01  compact110
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11Prolongation=”None” Checkpoint=”no”
11                 10  x01 timelevels1199
11                 10  x01variable type11REAL
11  errorscalars  10  x01  compact110
11                 10  Error  x01 description11Global error estimate
11                 10  Count  x01 dimensions110
11                 10  EstimatedDt  x01 distribution11CONSTANT
11                 10  x01 group type11SCALAR
11                 10  x01 timelevels111
11                 10  x01variable type11REAL

Provides: MoLRegisterEvolved to MoLRegisterEvolvedSlow to MoLRegisterConstrained to MoLRegisterSaveAndRestore to MoLRegisterEvolvedGroup to MoLRegisterEvolvedGroupSlow to MoLRegisterConstrainedGroup to MoLRegisterSaveAndRestoreGroup to MoLChangeToEvolved to MoLChangeToEvolvedSlow to MoLChangeToConstrained to MoLChangeToSaveAndRestore to MoLChangeToNone to MoLQueryEvolvedRHS to MoLNumIntegratorSubsteps to

8 Schedule This section lists all the variables which are assigned storage by thorn CactusNumerical/MoL. Storage can either last for the duration of the run (Always means that if this thorn is activated storage will be assigned, Conditional means that if this thorn is activated storage will be assigned for the duration of the run if some condition is met), or can be turned on for the duration of a schedule function.

Storage x

11Always:  11Conditional:
11MoLxCounters MoLxOriginalxTime11RKAlphaCoefficients
11x  11RKBetaCoefficients
11x  11ErrorScalars
11x  11ErrorScalars
11x  11ErrorScalars

11x  11x

11

Scheduled Functions

CCTKxSTARTUP (conditional) x molxstartup x startup banner x

11x11Language:11c
11 x 11Type: 11function

11

CCTKxPARAMCHECK (conditional) x molxparamcheck x basic parameter checking x

11x11Language:11c
11 x 11Type: 11function

11

CCTKxINITIAL (conditional) x molxstartloop x initialise the step size control x

11x11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function
11 x 11Writes: 11molxstepsizexbad
11 x 11x  11estimateddt

11

CCTKxEVOL (conditional) x molxevolution x a single cactus evolution step using mol x

11x11Type: 11group
11 x 11While:11mol::molxstepsizexbad

11

MoLxEvolution (conditional) x molxstartstep x mol internal setup for the evolution step x

11x11Type:11group

11

MoLxStartStep (conditional) x molxsetcounter x set the counter for the ode method to loop over

11x11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function
11 x 11Writes: 11molxintermediatexstep
11 x 11x  11molxslowstep
11 x 11x  11molxslowpoststep

11

MoLxStartStep (conditional) x molxsettime x ensure the correct time and timestep are used x

11x11Language:11c
11 x 11Options: 11level
11 x 11Reads: 11rkalphacoefficients
11 x 11x  11rkbetacoefficients
11 x 11x  11molxintermediatexstep
11 x 11Type: 11function
11 x 11Writes: 11molxoriginalxtime

11

MoLxStartStep (conditional) x molxallocatescratchspace x allocate storage for scratch levels x

11x11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function

11

MoLxEvolution (conditional) x molxprestep x physics thorns can schedule preloop setup routines in here x

11x11After: 11molxstartstep
11 x 11Before:11molxstep
11 x 11Type: 11group

11

MoLxEvolution (conditional) x molxallocatescratch x allocate sufficient space for array scratch variables x

11x11After: 11molxprestep
11 x 11Before: 11molxstep
11 x 11Language:11c
11 x 11Type: 11function

11

MoLxEvolution (conditional) x molxinitialcopy x ensure the data is in the correct timelevel

11x11After: 11molxprestep
11 x 11x  11molxallocatescratch
11 x 11Before: 11molxstep
11 x 11Language:11c
11 x 11Type: 11function

11

MoLxEvolution (conditional) x molxupdatevalidforinitialcopy x automatically update valid regions based on mol x

11x11After: 11molxinitialcopy
11 x 11Before: 11molxstep
11 x 11Language:11c
11 x 11Type: 11function

11

CCTKxWRAGH (conditional) x molxsetupindexarrays x set up the mol bookkeeping index arrays x

11x11Language:11c
11 x 11Type: 11function
11 x 11Writes: 11molxoriginalxtime
11 x 11x  11molxslowpoststep
11 x 11x  11molxslowstep

11

MoLxEvolution (conditional) x molxstep x the loop over the intermediate steps for the ode integrator x

11x11After: 11molxprestep
11 x 11Type: 11group
11 x 11While:11mol::molxintermediatexstep

11

MoLxStep (conditional) x molxicnaverage x averages the time levels for the averaging icn method x

11x11Before: 11molxcalcrhs
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11Type: 11function

11

MoLxStep (conditional) x molxinitrhs x initialise the rhs functions x

11x11Before: 11molxcalcrhs
11 x 11Language:11c
11 x 11Type: 11function

MoLxStep (conditional) x molxcalcrhs x physics thorns schedule the calculation of the discrete spatial operator in here x

11x11Type:11group

11

MoLxStep (conditional) x molxpostrhs x modify rhs functions x

11x11After: 11molxcalcrhs
11 x 11Before:11molxnancheck
11 x 11x  11molxadd
11 x 11Type: 11group

11

MoLxStep (conditional) x molxrhsboundaries x any ’final’ modifications to the rhs functions (boundaries etc.) x

11x11After: 11molxpostrhs
11 x 11Before:11molxnancheck
11 x 11x  11molxadd
11 x 11Type: 11group

11

MoLxStep (conditional) x molxnancheck x check the rhs gfs for nans x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxadd
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11Type: 11function

11

MoLxStep (conditional) x molxgenericrkadd x updates calculated with a generic method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11x  11molxpoststepmodify
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11x  11originalxdeltaxtime
11 x 11x  11rkalphacoefficients
11 x 11x  11rkbetacoefficients
11 x 11Type: 11function

11

MoLxStep (conditional) x molxeuleradd x updates calculated with the euler method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11Type: 11function

11

MoLxStep (conditional) x molxrk2add x updates calculated with the efficient runge-kutta 2 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11Type: 11function

11

CCTKxWRAGH (conditional) x molxsetuprkcoefficients x initialize the generic runge-kutta coefficients x

11x11Language:11c
11 x 11Options: 11global
11 x 11Storage: 11rkalphacoefficients
11 x 11x  11rkbetacoefficients
11 x 11Type: 11function
11 x 11Writes: 11rkalphacoefficients
11 x 11x  11rkbetacoefficients

11

MoLxStep (conditional) x molxrk2centraladd x updates calculated with the central runge-kutta 2 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11originalxdeltaxtime
11 x 11x  11molxintermediatexstep
11 x 11Type: 11function

11

MoLxStep (conditional) x molxrk3add x updates calculated with the efficient runge-kutta 3 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11Type: 11function

MoLxStep (conditional) x molxrk4add x updates calculated with the efficient runge-kutta 4 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11x  11originalxdeltaxtime
11 x 11Type: 11function

11

MoLxStep (conditional) x molxrk45add x updates calculated with the runge-kutta 45 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11x  11originalxdeltaxtime
11 x 11Type: 11function

11

MoLxStep (conditional) x molxrk65add x updates calculated with the runge-kutta 65 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11x  11originalxdeltaxtime
11 x 11Type: 11function

11

MoLxStep (conditional) x molxrk87add x updates calculated with the runge-kutta 87 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11x  11originalxdeltaxtime
11 x 11Type: 11function

11

MoLxStep (conditional) x molxicnadd x updates calculated with the efficient icn method

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11Type: 11function

11

MoLxStep (conditional) x molxicnadd x updates calculated with the averaging icn method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11Type: 11function

11

MoLxStep (conditional) x molxabadd x updates calculated with the adams-bashforth x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11Type: 11function

11

MoLxStep (conditional) x molxrk2xmrx2x1xadd x updates calculated with the multirate runge-kutta 2 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11x  11originalxdeltaxtime
11 x 11Type: 11function

11

CCTKxWRAGH (conditional) x molxsetschedulestatus x set the flag so it is ok to register with mol x

11x11After: 11molxsetupindexarrays
11 x 11Language:11c
11 x 11Options: 11global
11 x 11Type: 11function

11

MoLxStep (conditional) x molxrk4xmrx2x1xadd x updates calculated with the multirate runge-kutta 4 method

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11x  11originalxdeltaxtime
11 x 11Type: 11function

11

MoLxStep (conditional) x molxrk4xrk2xadd x updates calculated with the multirate rk4/rk2 method x

11x11After: 11molxcalcrhs
11 x 11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Reads: 11molxintermediatexstep
11 x 11x  11originalxdeltaxtime
11 x 11Type: 11function

11

MoLxStep (conditional) x molxupdatevalidforadd x automatically update valid regions based on mol x

11x11After: 11molxadd
11 x 11Language:11c
11 x 11Type: 11function

11

MoLxStep (conditional) x molxpoststep x the group for physics thorns to schedule boundary calls etc. x

11x11After:11molxadd
11 x 11Type:11group

11

CCTKxPOSTINITIAL (conditional) x molxpoststep x ensure that everything is correct after the initial data have been set up x

11x11Type:11group

11

CCTKxPOSTREGRID (conditional) x molxpoststep x ensure that everything is correct after regridding x

11x11Type:11group

11

CCTKxPOSTRESTRICTINITIAL (conditional) x molxpoststep x ensure that everything is correct after restriction x

11x11Type:11group

11

CCTKxPOSTRESTRICT (conditional) x molxpoststep x ensure that everything is correct after restriction x

11x11Type:11group

11

CCTKxPOSTxRECOVERxVARIABLES (conditional) x molxpoststep x ensure that everything is correct after recovery x

11x11Type:11group

11

MoLxStep x molxpoststepmodify x the group for physics thorns to schedule enforcing constraints x

11x11After: 11molxadd
11 x 11Before:11molxpoststep
11 x 11Type: 11group

11

CCTKxWRAGH (conditional) x molxregister x the group where physics thorns register variables with mol x

11x11After:11molxsetschedulestatus
11 x 11Type:11group

11

CCTKxPOSTINITIAL x molxpoststepmodify x the group for physics thorns to schedule enforcing constraints x

11x11Before:11molxpoststep
11 x 11Type: 11group

11

CCTKxPOSTINITIAL x molxpseudoevolution x calculate pseudo-evolved quantities

11x11After:11molxpoststep
11 x 11Type:11group

11

CCTKxEVOL x molxpseudoevolution x calculate pseudo-evolved quantities x

11x11After:11molxevolution
11 x 11Type:11group

11

CCTKxPOSTREGRIDINITIAL x molxpseudoevolutionboundaries x apply boundary conditions to pseudo-evolved quantities x

11x11After:11molxpoststep
11 x 11Type:11group

11

CCTKxPOSTREGRID x molxpseudoevolutionboundaries x apply boundary conditions to pseudo-evolved quantities x

11x11After:11molxpoststep
11 x 11Type:11group

11

CCTKxPOSTRESTRICTINITIAL x molxpseudoevolutionboundaries x apply boundary conditions to pseudo-evolved quantities x

11x11After:11molxpoststep
11 x 11Type:11group

11

CCTKxPOSTRESTRICT x molxpseudoevolutionboundaries x apply boundary conditions to pseudo-evolved quantities x

11x11After:11molxpoststep
11 x 11Type:11group

11

MoLxStep x molxdecrementcounter x alter the counter number

11x11After: 11molxadd
11 x 11Before: 11molxpoststep
11 x 11x  11molxpoststepmodify
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function
11 x 11Writes: 11molxintermediatexstep
11 x 11x  11molxslowstep
11 x 11x  11molxslowpoststep

11

MoLxStep x molxresettime x if necessary, change the time x

11x11After: 11molxdecrementcounter
11 x 11Before: 11molxpoststep
11 x 11x  11molxpoststepmodify
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Reads: 11molxoriginalxtime
11 x 11x  11rkalphacoefficients
11 x 11x  11rkbetacoefficients
11 x 11x  11molxintermediatexstep
11 x 11Type: 11function

11

MoLxStep x molxresetdeltatime x if necessary, change the timestep x

11x11After: 11molxpoststep
11 x 11x  11molxpoststepmodify
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Reads: 11originalxdeltaxtime
11 x 11x  11rkalphacoefficients
11 x 11x  11rkbetacoefficients
11 x 11x  11molxintermediatexstep
11 x 11Type: 11function

11

CCTKxWRAGH (conditional) x molxreportnumbervariables x report how many of each type of variable there are x

11x11After: 11molxregister
11 x 11Language:11c
11 x 11Options: 11meta
11 x 11Type: 11function

11

MoLxEvolution x molxrestoresandr x restoring the save and restore variables to the original state

11x11After: 11molxreduceadaptiveerror
11 x 11x  11molxfinishloop
11 x 11Language:11c
11 x 11Type: 11function

11

MoLxEvolution x molxfreescratchspace x free storage for scratch levels x

11x11After: 11molxrestoresandr
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function

11

MoLxEvolution (conditional) x molxinitadaptiveerror x control the step size: initialize error check variables x

11x11After: 11molxstep
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function
11 x 11Writes: 11error
11 x 11x  11count

11

MoLxEvolution (conditional) x molxfindadaptiveerror x control the step size: compute error check variables x

11x11After: 11molxinitadaptiveerror
11 x 11Language:11c
11 x 11Reads: 11originalxdeltaxtime
11 x 11Type: 11function
11 x 11Writes: 11error
11 x 11x  11count

11

MoLxEvolution (conditional) x molxreduceadaptiveerror x control the step size: reduce error check variables x

11x11After: 11molxfindadaptiveerror
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Reads: 11originalxdeltaxtime
11 x 11Type: 11function
11 x 11Writes: 11error
11 x 11x  11count
11 x 11x  11molxstepsizexbad
11 x 11x  11estimateddt

11

CCTKxPOSTSTEP (conditional) x molxsetestimateddt x control the step size: set the new timestep

11x11Language:11c
11 x 11Options: 11level
11 x 11Reads: 11estimateddt
11 x 11Type: 11function

11

MoLxEvolution (conditional) x molxfinishloop x control the step size x

11x11After: 11molxstep
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function
11 x 11Writes: 11molxstepsizexbad
11 x 11x  11molxslowpoststep
11 x 11x  11molxslowstep

11

CCTKxTERMINATE x molxfreeindexarrays x free the mol bookkeeping index arrays x

11x11Before: 11driverxterminate
11 x 11Language:11c
11 x 11Type: 11function

11

CCTKxPOSTINITIAL (conditional) x molxfillalllevels x a bad routine. fills all previous timelevels with data copied from the current. x

11x11After: 11molxpoststep
11 x 11Language:11c
11 x 11Type: 11function

11

CCTKxPOSTINITIAL (conditional) x molxfillalllevels x a bad routine. fills all previous timelevels with data copied from the current. x

11x11Before: 11molxpoststep
11 x 11Language:11c
11 x 11Type: 11function

11

CCTKxEVOL (conditional) x molxstartloop x initialise the step size control

11x11Before: 11molxevolution
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function
11 x 11Writes: 11molxstepsizexbad
11 x 11x  11estimateddt

11

Aliased Functions x

11Alias Name: xxxxxxx 11Function Name:
11MoLxABAdd  11MoLxAdd
11MoLxEulerAdd  11MoLxAdd
11MoLxGenericRKAdd 11MoLxAdd
11MoLxICNAdd  11MoLxAdd
11MoLxICNAverage  11MoLxPrepare
11MoLxRK2Add  11MoLxAdd
11MoLxRK2CentralAdd 11MoLxAdd
11MoLxRK2xMRx2x1xAdd11MoLxAdd
11MoLxRK3Add  11MoLxAdd
11MoLxRK45Add  11MoLxAdd
11MoLxRK4Add  11MoLxAdd
11MoLxRK4xMRx2x1xAdd11MoLxAdd
11MoLxRK4xRK2xAdd 11MoLxAdd
11MoLxRK65Add  11MoLxAdd
11MoLxRK87Add  11MoLxAdd