xx Using SpaceMask

Using SpaceMask

Denis Pollney

Date

Abstract

The SpaceMask thorn provides a grid function which can be used to store user-defined states at each grid point. Routines are provided for quickly setting and checking the mask states.

1 The mask bit-field

The mask is a grid function which can be used to assign a state to each point on the grid. It is used as a bit field, with different bits, or combinations of bits, assigned to represent various states. The mask is currently implemented as a CCTKxINT grid function, providing 32 bits per point.1 1The original implementation of these functions used a CCTKxINT8 for 64 bits, however this data type is not supported by some Fortran compilers. Appropriate numbers of bits within the mask can be allocated to represent individual states of a given type.

For instance, a programmer wants to record whether each point of the grid has state “interior”, “excised” or “boundary”. The collection of these states is called a type, and 2-bits of the mask (enough to represent the three possible states of the type) are allocated to hold this information. If an independent classification of points (ie. a new type) is required, bits are allocated to it from the remaining free bits in the mask. (See Figure 1.)

The SpaceMask thorn provides a set of routines for allocating bits within the mask, and for setting and checking the mask state at individual points.

PIC Figurex1: The SpaceMaskxRegisterType() function allocates bits of the mask to a new “type” (eg. “excision” and “colour” in the figure). The number of bits that are allocated is determined by the number of possible states of the given type.

2 Using the mask

A number of functions are provided for registering new types, and checking and setting mask values at a point. The interface to these functions differs slightly for C and Fortran90 code.

2.1 Accessing the mask from C Code

The following description describes the C interface for manipulating the mask. An example of code using these functions can be found in Appendix 5.1.

2.1.1 Registering a new type

Bits of the mask are allocated to a new type using the function:

 int SpaceMaskxRegisterType(const char* typexname,
xx      int nstates, const char* const statexlist[])

xx  typexname The name of the new type to be registered; nstates The number of states in the new type; statexlist A list of names, one for each state, or NULL.

An appropriate number of bits of the mask are allocated to hold a unique value for the states specified by the nstates argument. The registration must take place before the type can be used, so for instance, this could be scheduled at CCTKxSTARTUP. The function returns a if the required states cannot be allocated (for example, if there are not enough free bits remaining in the mask), otherwise it returns .

It is possible to allocate a portion of the mask without assigning names to any of the states by passing NULL as the statexlist. This is useful when some integer number of states is required for a given type, but the number of states and appropriate names are not known at compile time. States which are allocated in this way can be

accessed using the SpaceMaskxGetStateBitsList().

New states can be added to an already allocated type using the function:

 int SpaceMaskxAppendStatesToType(const char* typexname,
xx      int nstates, const char* const statexlist[])

xx  typexname The name of an already registered type; nstates The number of states to be added; statexlist A list of names, one for each of the new states.

A number of new bits will be added to the specified type appropriate to hold the number of existing states plus the extra states. The allocated bits need not be contiguous within the mask. The function returns if successful, otherwise a if the new states could not be allocated.

2.1.2 Setting and checking mask states

The state of the mask at a given point is set using the function:

 void SpaceMaskxSetState(CCTKxINT* mask, int point,
xx      const char* typexname, const char* state)

xx  mask A pointer to the mask grid function; point The index of the point within the grid function array; typexname The name of the type containing the state; statexname The particular state to be set.

The state of the mask at a given point can be checked using:

 int SpaceMaskxCheckState(const CCTKxINT* mask, int point,
xx      const char* typexname, const char* state)

xx  mask A pointer to the mask grid function; point The index of the point within the grid function array; typexname The name of the type containing the state; statexname The particular state to be compared.

The return value is 1 if the given statexname has been set at the point, or 0 otherwise.

2.1.3 Faster bitwise operators

The above mentioned functions for checking and setting the mask are rather inefficient, since they require strings to be compared at each point. A set of faster macros are provided which operate on the mask by performing bit operations directly. In order to use these functions, the bitmasks corresponding to the desired type and it’s states need to be obtained using the following functions:

 CCTKxINT SpaceMaskxGetTypeBits(const char* typexname)

xx  typexname The type for which the bitmask is to be determined.

 CCTKxINT SpaceMaskxGetStateBits(const char* typexname, const char* statexname)

xx  typexname The type containing the state in question; statexname The state whose bitmask is to be determined.

Each of these functions returns a CCTKxINT8 which holds the bitmask corresponding to the given type or state. For example, if a given type uses three bits of the mask, the returned value could be the integer corresponding to the bitfield x, where the location of s indicates the bits which have been allocated to the given type. A return value of (GetTypeBits) or (GetStateBits) indicates that the bitmask could not be determined for some reason (for example, the requested state was not previously registered).

All of the state masks allocated for a given type can be retrieved using the following function:

 CCTKxINT* SpaceMaskxGetStateBitsList(const char* typexname)

xx  typexname The type for which the bitmask is to be determined.

The return value is a list of integers, with a length corresponding to the number of states of the given type. Each of the entries is a bitmask which identifies the state. List entries are returned in the order in which they were originally assigned. This allows for a simple correspondence between states and integers (given by the index of the list entry). A fortran wrapper does not currently exist for this functionality.

The following macros have been defined for fast setting and checking of the mask by direct bitwise operations:

 void SpaceMaskxSetStateBits(CCTKxINT* mask, int point,
xx      CCTKxINT typexbits, CCTKxINT statexbits)

xx  mask A pointer to the mask grid function; point The index of the point within the grid function array; typexbits The bitmask corresponding to the required type; statexbits The bitmask corrsponding to the required state;

 int SpaceMaskxCheckStateBits(const CCTKxINT* mask, int point,
xx      CCTKxINT typexbits, CCTKxINT statexbits)

xx  mask A pointer to the mask grid function; point The index of the point within the grid function array; typexbits The bitmask corresponding to the required type; statexbits The bitmask corrsponding to the required state;

The latter macro returns a 1 if the given state has been set, a 0 otherwise. The typexbits and statexbits can be obtained using the SpaceMaskxGet*Bits functions, described above.

2.2 Accessing the mask from Fortran90 Code

The following Fortran interfaces to the mask utilities apply only to Fortran90 code, since fast bitwise operations are not a part of the Fortran77 standard. An example of the use of the Fortran90 interface can be found in Appendix 5.2.

2.2.1 Registering a new type

Registration functions have not been implemented for the Fortran90 interface, though they can easily be written in C by following the descriptions in Section 2.1.1 and the example in Appendix 5.1.

2.2.2 Setting and checking mask states The mask states can be manipulated using the following functions:

 SpaceMaskxSetState(CCTKxINT mask(,,), int point, character typexname,
xx      character statexname)

xx  mask A pointer to the mask grid function; point The index of the point within the grid function array; typexname The name of the type containing the state; type; statexname The name of the state to be set.

The value of the mask variable is changed to turn on the specified state at the point.

 SpaceMaskxCheckState(CCTKxINT check, CCTKxINT mask(,,), int point,
xx      character typexname, character statexname)

xx  check A variable which is set to 1 if the requested state has been set; mask A pointer to the mask grid function; point The index of the point within the grid function array; typexname The name of the type containing the state; type; statexname The name of the state to be checked.

2.2.3 Faster bitwise operators

As in the C interface, alternative bitwise operators have been defined in order to more quickly operate on the mask at a point. In order to use these functions, the bitmasks corresponding to the type and states in question must first be obtained using the following functions:

 SpaceMaskxGetTypeBits(CCTKxINT typexbits, character typexname)

xx  typexbits A variable which will be set to the bitmask of the requested type; typexname A character string naming the type.

 SpaceMaskxGetStateBits(CCTKxINT statexbits, character typexname,
xx      character statexname)

xx  statexbits A variable which will be set to the bitmask of the requested state; typexname A character string naming the type containing the state; typexname A character string naming the state.

Once the bitmasks for the type and state have been obtained, the following macros can be used directly:

 SpaceMaskxSetStateBitsF90(mask, i, j, k, typexbits, statexbits)

This macro sets the value of the mask at the index to the given state. It doesn’t have a return value, and can be used directly inline (without a Fortran ‘call’).

 SpaceMaskxCheckStateBitsF90(mask, i, j, k, typexbits, statexbits)

A value of ‘.true.’ is returned if the given state has been set, or ‘.false.’ otherwise. The macro can be used directly in Fortran boolean expressions, as in the examples in Appendix 5.2.

3 Notes

A proposal exists to provide a more versatile solution by allowing tables to be associated with each grid point. This could be implemented using the existing standard outlined here through the addition of an additional integer grid function

spacextablexfield

A bit of the existing spacexmask would be allocated to indicate whether or not a table has been associated with the given point. If so, then the corresponding point in the spacextablexfield would hold the table handle.

4 Acknowledgements

Thanks to Jonathan Thornburg, Tom Goodale, and Erik Schnetter for helpful comments on the original specification. Useful implementation of the mask required grid symmetries to work for integer grid functions. This was done by Thomas Radke. Peter Diener helped with the coding of the Fortran90 bitwise operators.

5 Appendix

5.1 Example: C code

#includex"cctk.h"
#includex"cctk_Arguments.h"
#includex"cctk_Parameters.h"
#includex"SpaceMask.h"
/*
x*xRegisterxaxmaskxtypexwithxthreexstates.
x*/
voidxEg_RegisterMaskx(void)
{
xxcharx*state_list_excision[3]x=x{"normal",x"excise",x"boundary"};
xxSpaceMask_RegisterType("excision",x3,xstate_list_excision);
xxreturn;
}
/*
x*xSetxthexmaskxtoxthex``normal''xstatexatxeachxpointxusingxthe
x*xslowerxname-referencexfunction.xSeexbelowxforxanxexamplexusing
x*xthexfasterxbitwisexoperatorxtoxcheckxthexmaskxstates.
x*/
voidxEg_SetMaskStatesx(CCTK_ARGUMENTS)
{
xxDECLARE_CCTK_ARGUMENTS
xxDECLARE_CCTK_PARAMETERS
xxintxi,xj,xk,xijk;
xxforx(k=0;xk<cctk_lsh[2];x++k)
xx{
xxxxforx(j=0;xj<cctk_lsh[1];x++j)
xxxx{
xxxxxxforx(i=0;xi<cctk_lsh[0];x++i)
xxxxxx{
xxxxxxxxijkx=xCCTK_GFINDEX3D(cctkGH,xi,xj,xk);
xxxxxxxxSpaceMask_SetState(space_mask,xijk,x"excision",x"normal");
xxxxxx}
xxxx}
xx}
xxreturn;
}
/*
x*xCheckxwhichxofxthexpointsxofxthexmaskxhavexbeenxsetxtoxthe

x*x``boundary''xstatexusingxthexbitwisexmacro.
x*/
voidxEg_CheckMaskStatesx(CCTK_ARGUMENTS)
{
xxDECLARE_CCTK_ARGUMENTS
xxDECLARE_CCTK_PARAMETERS
xxintxi,xj,xk,xijk;
xxCCTK_INTxexcision_mask;
xxCCTK_INTxboundary_mask;
xxexcision_maskx=xSpaceMask_GetTypeBits("excision");
xxboundary_maskx=xSpaceMask_GetStateBits("boundary");
xxforx(k=0;xk<cctk_lsh[2];x++k)
xx{
xxxxforx(j=0;xj<cctk_lsh[1];x++j)
xxxx{
xxxxxxforx(i=0;xi<cctk_lsh[0];x++i)
xxxxxx{
xxxxxxxxijkx=xCCTK_GFINDEX3D(cctkGH,xi,xj,xk);
xxxxxxxxifx(SpaceMask_CheckStateBits(space_mask,xijk,xexcision_mask,
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxboundary_mask))
xxxxxxxx{
xxxxxxxxxxCCTK_VINFO(CCTK_THORNSTRING,
xxxxxxxxxxxxxxxxxxxxx"Pointx(%d,%d,%d)xisxaxboundaryxpoint.\n",
xxxxxxxxxxxxxxxxxxxxxi,xj,xk);
xxxxxxxx}
xxxxxx}
xxxx}
xx}
xxreturn;
}

5.2 Example: Fortran90 code

#includex"SpaceMask.h"
!
!xSetsxeachxpointxofxthexmaskxtoxthex'normal'xstate.xHere,x'normal'
!xisxonexofxthexstatesxofxthex'excision'xtype,xwhichxshouldxhave
!xbeenxregisteredxpreviouslyxusingxthexCxSpaceMask_RegisterType()
!xfunction.
!
subroutinexEg_SetStateByName(CCTK_ARGUMENTS)
xximplicitxnone
xxintegerxni,xnj,xnk
xxintegerxi,xj,xk,xijk
xxnix=xcctk_lsh(1)
xxnjx=xcctk_lsh(2)
xxnkx=xcctk_lsh(3)
xxdoxk=1,nk
xxxxxdoxj=1,nj
xxxxxxxxdoxi=1,ni
xxxxxxxxxxxijkx=x(i-1)x+xni*((j-1)x+xnj*(k-1))
xxxxxxxxxxxcallxSpaceMask_SetState(space_mask,xijk,x"excision",x"normal")
xxxxxxxxendxdo
xxxxxendxdo
xxendxdo
endxsubroutinexEg_SetStateByName
!
!xSetsxthexmaskxvaluesxwithinxaxradiusxofx1xtoxthex'excise'xstate,
!xthenxdoesxaxcheckxofxthexstatexofxeachxpoint.xxThexbitwisexmacros
!xarexusedxinxthisxcasexforxgreaterxefficiencyxinxsetting/checking
!xthexmask.
!
subroutinexEg_SetExcisedRegionByBitmask(CCTK_ARGUMENTS)
xximplicitxnone
xxintegerxni,xnj,xnk
xxintegerxi,xj,xk,xijk

xxCCTK_INTxtype_bits,xexcised
xxnix=xcctk_lsh(1)
xxnjx=xcctk_lsh(2)
xxnkx=xcctk_lsh(3)
xxcallxSpaceMask_GetTypeBits(type_bits,x"excision")
xxcallxSpaceMask_GetStateBits(excised,x"excision",x"excised")
xxdoxk=1,nk
xxxxxdoxj=1,nj
xxxxxxxxdoxi=1,ni
xxxxxxxxxxxifx(r(i,j,k).lt.1.d0)xthen
xxxxxxxxxxxxxxSpaceMask_SetStateBitsF90(space_mask,xi,xj,xk,xtype_bits,x&
xxxxxxxxxxxxxxxxxxxexcised)
xxxxxxxxxxxendxif
xxxxxxxxendxdo
xxxxxendxdo
xxendxdo
xxdoxk=1,nk
xxxxxdoxj=1,nj
xxxxxxxxdoxi=1,ni
xxxxxxxxxxxifx(SpaceMask_CheckStateBitsF90(space_mask,xi,xj,xk,xtype_bits,x&
xxxxxxxxxxxxxxxxexcised))xthen
xxxxxxxxxxxxxxwrite(*,*)x"Thexpointx(",xi,xj,xk,x")xhasxbeenxexcised."
xxxxxxxxxxxendxif
xxxxxxxxendxdo
xxxxxendxdo
xxendxdo
endxsubroutinexEg_SetExcisedRegionByBitmask

6 Parameters

11usexmask 10Scope: restrictedx01  BOOLEAN

11Description: Turn on storage for mask?

11        10  x01  Default: no

7 Interfaces

General Implements: spacemask

Inherits: grid

Grid Variables

7.0.1 PUBLIC GROUPS

11x Group Names x10x Variable Names xx01 Details x11x
11 spacexmaskxgroup 10  x01  compact110
11                 10  spacexmask  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11tensortypealias=”Scalar” Prolongation=”None”
11                 10  x01 timelevels111
11                 10  x01variable type11INT
11  mask  10  x01  compact110
11                 10  emask  x01 description11Mask
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11tensortypealias=”Scalar” Prolongation=”None”
11                 10  x01 timelevels111
11                 10  x01variable type11REAL

11

Adds header: SpaceMask.h

Uses header: Symmetry.h 8 Schedule This section lists all the variables which are assigned storage by thorn CactusNumerical/SpaceMask. Storage can either last for the duration of the run (Always means that if this thorn is activated storage will be assigned, Conditional means that if this thorn is activated storage will be assigned for the duration of the run if some condition is met), or can be turned on for the duration of a schedule function.

Storage x

11x11Conditional:
11x 11spacexmaskxgroup
11x 11mask

11x 11x

11

Scheduled Functions

CCTKxBASEGRID (conditional) x maskzero x initialise mask to zero x

11x11Language:11c
11 x 11Type: 11function
11 x 11Writes: 11spacemask::spacexmask(everywhere)

11

CCTKxWRAGH (conditional) x masksym x set grid symmetries for mask x

11x11Language:11c
11 x 11Options: 11global
11 x 11Type: 11function

11

CCTKxPOSTREGRIDINITIAL (conditional) x maskzero x initialise mask to zero x

11x11Language:11c
11 x 11Type: 11function
11 x 11Writes: 11spacemask::spacexmask(everywhere)

11

CCTKxPOSTREGRID (conditional) x maskzero x initialise mask to zero x

11x11Language:11c
11 x 11Type: 11function
11 x 11Writes: 11spacemask::spacexmask(everywhere)

11

CCTKxWRAGH (conditional) x masksymxemask x set grid symmetries for emask (compatibility mode) x

11x11Language:11c
11 x 11Options: 11global
11 x 11Type: 11function

11

CCTKxBASEGRID (conditional) x maskone x set old style mask to one x

11x11Language:11c
11 x 11Type: 11function
11 x 11Writes: 11spacemask::emask(everywhere)

11

CCTKxPOSTSTEP (conditional) x checkmask x ensure that all mask values are legal x

11x11Language:11c
11 x 11Reads: 11spacemask::emask(everywhere)
11 x 11Type: 11function

11

CCTKxPOSTREGRIDINITIAL (conditional) x maskone x set mask to one x

11x11Language:11c
11 x 11Type: 11function
11 x 11Writes: 11spacemask::emask(everywhere)

11

CCTKxPOSTREGRID (conditional) x maskone x set mask to one x

11x11Language:11c
11 x 11Type: 11function
11 x 11Writes: 11spacemask::emask(everywhere)