xx ReadInterpolate

ReadInterpolate

Roland Haas <rhaas@illinois.edu>

August 17 2020

Abstract

A FileReader like thorn that uses InterpolateLocalUniform to interpolate the data read in onto the new grid.

1 Introduction

IOUtilx[1] defines a file reading interface via the IO parameters filereaderxIDxdir, filereaderxIDxfiles, filereaderxIDxvars parameters. IOUtil uses the checkpoint recovery routines to read grid variable data from disk. This requires that the grid resolution used to write the files agrees with the resolution using when reading in data and that, for each refinement level, the to be read in region is fully contained in the corresponding refinement level in the source files.

ReadInterpolate relaxes this restriction by allowing arbitrary changes in resolution and grid structure in between source files and simulation. It uses Cactus local interpolation operators CCTKxInterpLocalUniform to interpolate data using the highest resolution source data available for each target grid point.

It provides parameters files and onlyxthesexdatasets that mirror IOUtil’s parameters to select files and

datasets in IO::filereaderxIDxdir.

Auxiliary parameters are provided to fine tune what is read and to control the interpolator.

2 Physical System

ReadInterpolate takes input data on a grid patch described by its origin , grid spacing and shape . Data is interpolated to the coordinates of the simulation grid using Cactus’ interpolation operators: .

3 Numerical Implementation

For a target point multiple datasets in the source files may contain this physical coordinate. When this happens the following selection algorithm applies:

unless the candidate datasets overlaps the outer boundary of the current grid, require that there are at least cctkxnghostzones or interpolatorxhalfxwidth extra points for use by the interpolator. This may involve using an off-centered interpolation stencil for points near the outer boundary. Then

if reading in data from time level 0, pick the largest refinement level dataset that contains the point, otherwise pick the refinement level that matches the refinement level of the target point. This assumes that the files were written at a timestep when all refinement level align in time and that the timestep in the data files and simulation are identical. Setting

Once a source data set has been selected its data and the target coordinate triplet are passed to

CCTKxInterpLocalUniform for interpolation.

4 Using This Thorn

This thorn is intended as an initial data thorn, scheduling itself in the CCTKxINITIAL time bin of

Cactus.

4.1 Obtaining This Thorn

This thorn is part of the Einstein Toolkit in the EinsteinInitialData repository.

4.2 Basic Usage

To use this thorn, set IO::filereaderxIDxdir to the directory containing the data files, set ReadInterpolate’s files to the base names of the file to read and onlyxthesexdatasets to a comma separated list of regular expressions to match the datasets to read. To obtain the list of data sets present in the file the h5ls tool, which is part of HDF5, can be used. The parameter maxxnumberxofxreadxvariables must be set to the expected number of grid variables to be read.

If desired read in data can be moved to a different position on the grid using shiftxreadxdatasetsxby[3] which relocates the origin of the coordinates in the read in data to the given coordinate triplet.

4.3 Special Behavior

ReadInterpolate attempts to avoid one sided interpolation as much as possible, yet tried to allow reading in data at the outer boundaries of the grid. A heuristic is used to allow one sided derivatives there. This requires changing interpolatorxpars to allow off centered interpolation stencils.

A number of parameters are provided to select which datasets to read:

minimumxreflevel

ignore any datasets with refinement level less than this,

maximumxreflevel

ignore any datasets with refinement level higher than this,

Some Carpet version contain a bug and do not set origin correctly for cell centered grids (it is off by half a grid point, i.e. it does not take the staggering into account). This option fixes this issue by offsetting all datasets that should be offset. Setting fixxcellxcenteredxorigins adds an offset to correct for this. This is incorrect for files where Carpet already used the correct origin. This parameter only affects cell centered input data.

Interpolation parameters

ReadInterpolate uses interpolatorxname as the local interpolator to interpolate read in data to simulation grid points. It passes interpolatorxvars to the interpolator. The default values select AEILocalInterp’s Hermite interpolator and forbid off centered interpolation stencils or extrapolation.

ReadInterpolate uses epsilon as a fuzziness parameter when determining if a point is contained in a read in dataset. This may require changes for very large grids.

When deciding if a read in dataset is large enough to allow for interpolation without off centered stencils ReadInterpolate requires knowledge about the stencil width of the chosen interpolation operator. This can be explicitly specified as interpolatorxhalfxwidth or alternatively assumed to be cctkxnghostzones if this parameter is set to -1, which is the default.

This makes sure that the interpolator does not off center the interpolation stencil if there are insufficient points to interpolate, which can happen if there are insufficient ghost-zones for the interpolation method used, and can lead to processor-number dependent results.

ReadInterpolate is no aware of any symmetry conditions used in either the simulation used to produce the data files the current simulation. Thus it will interpolated into symmetry points in the same manner as for outer boundary points. In these situations the parameter enforcexsymmetriesxafterxreading to apply symmetry boundary conditions after having read in all data. This does not enable ReadInterpolate to read in data produced on, for example, an octant grid and provide data on a full grid.

4.4 Interaction With Other Thorns

ReadInterpolate assumes that data files were written by CarpetIOHDF5 and may not work with files written by

PUGH’s IOHDF5.

This thorn is intended as an initial data thorn, however since it is generic it does not extend ADMBase’s or

HydroBase’s initialxXXX parameters.

ReadInterpolate can be used to read data into curvilinear grids set up by Llama but cannot read data from curvilinear grids.

4.5 Examples

The test suite file admhydro.par shows how to use ReadInterpolate to read in metric and fluid variables:

ActiveThornsx=x"ReadInterpolate"

#xReadInterpolatexignorexthesexbutxtheyxarexrequiredxtoxallocatexstorage
ADMBase::initial_dataxxxx=x"CartesianxMinkowski"
ADMBase::initial_lapsexxx=x"one"
ADMBase::initial_shiftxxx=x"zero"
ADMBase::initial_dtlapsex=x"zero"
ADMBase::initial_dtshiftx=x"zero"
HydroBase::initial_hydrox=x"zero"
IO::filereader_ID_dirx=x"tov_write"
ReadInterpolate::filesx=x"alpxrho"
ReadInterpolate::max_number_of_read_variablesx=x2
ReadInterpolate::shift_read_datasets_by[0]x=x+3.14
ReadInterpolate::shift_read_datasets_by[1]x=x-42
ReadInterpolate::shift_read_datasets_by[2]x=x-6

4.6 Support and Feedback

Please use the Einstein Toolkit ticket system to report issues.

5 History

This thorn was originally inspired by SpEC’sx[2] ReadTensorsFromDiskWithMap item and developed to read in single neutron star data produced by THCx[3] into GRHydrox[4]. It has been found useful to in combination with

CTxMultiLevel since then.

5.1 Acknowledgments

David Radice and Philipp Mxsta provided bug reports and comments during the time this thorn was developed.

References

 [1]xxxhttps://bitbucket.org/cactuscode/cactusbase/src/master/IOUtil/

 [2]xxxSpectral Einstein Code, https://www.black-holes.org/code/SpEC.html

[3]xxxWhiskyTHC:  The  General-Relativistic  Templated  Hydrodynamics  Code,

   http://personal.psu.edu/dur566/whiskythc.html

[4]xxxGRHydro is an evolution code for a general-purpose 3D relativistic hydrodynamics,

   https://bitbucket.org/einsteintoolkit/einsteinevolve/src/master/GRHydro/

6 Parameters

11enforcexsymmetriesxafterxreading 10Scope: privatex01  BOOLEAN

11Description: apply symmetry boundary conditions after reading data

11                          10  x01  Default: no

11epsilon  10Scope: privatex01  REAL

11Description: Tolerance when deciding if a grid point lies within an overlap region

11 Range 10 x01 Default: 1e-12
110:*  10any positive number, should be similar to AEILocalInterp’s tolerance

11

11files  10Scope: privatex01  STRING

11Description: List of basenames of files to read in as initial data (e.g. omit the .filexXXX.h5 part)

11 Range 10 x01 Default: (none)
11.+  10Space-separated list of initial data filenames
11ˆ$  10An empty string for not recovering initial data

11

11fixxcellxcenteredxorigins  10Scope: privatex01  BOOLEAN

11Description: offset rl>1 cell centered input data by half a grid step to fix broken HDF5 files

11                          10  x01  Default: no

11

11interpolatorxhalfxwidth  10Scope: privatex01  INT

11Description: maximum number of points to one side of the interpolated location the interpolator uses

11  Range  10  x01  Default: -1
111:*  10this is similar to cctkxnghostzones, 1 for linear and constant interpolation
112:*  102nd for second order and 3rd interpolation
11-1  10use cctkxnghostzones

11

11interpolatorxname  10Scope: privatex01  STRING

11Description: Which interpolator should I use

11 Range 10 x01 Default: Hermite polynomial interpolation
11.+  10Any nonempty string

11

11interpolatorxpars  10Scope: privatex01  STRING

11Description: Parameters for the interpolator

11 Range 10 x01Default: order=3 boundaryxoffxcenteringxtolerance={1e12 1e12 1e12 1e12 1e12 1e12} boundaryxextrapolationxtolerance={1e-12 1e-12 1e-12 1e-12 1e-12 1e-12}
11.*  10”Any string that UtilxTableSetFromStr ing() will take”

11

11maxxnumberxofxreadxvariables 10Scope: privatex01  INT

11Description: how many variables will be read in?

11 Range 10 x01 Default: 1
111:*  10any positive number

11

11maximumxreflevel  10Scope: privatex01  INT

11Description: ignore any datasets with refinement level higher than this

11 Range 10 x01 Default: 1000
110:*  10maximum refinement level to keep

11

11minimumxreflevel  10Scope: privatex01  INT

11Description: ignore any datasets with refinement level less than this

11 Range 10 x01 Default: (none)
110:*  10minimum refinement level to keep

11onlyxthesexdatasets  10Scope: privatex01  STRING

11Description: comma separated list of dataset regular expression patterns to consider for reading

11 Range 10 x01 Default: (none)
11.+  10any valid Cactus regular expression
11see [1] below  10refinement level 6, all metric
11:g[xyz][xyz],:rho  10metric and rho everywhere

11

ADMBASE[:][:]g[xyz][xyz].*rl=6

11shiftxreadxdatasetsxby  10Scope: privatex01  REAL

11Description: add to all read in coordinates

11 Range 10 x01 Default: 0.0
11*:*  10what used to be origin in datasets appear here

11

11test  10Scope: privatex01  KEYWORD

11Description: control how test data is used

11 Range 10 x01 Default: no
11no  10do not use test data
11generate  10generate test data
11compare  10compare read data to test data

11

11verbosity  10Scope: privatex01  INT

11Description: how much diagnostic output to produce

11 Range 10 x01 Default: 1
110:0  10only errors
111:1  10warn about suspicous states
112:*  10more and more detailed debug output (values up to 10 are defined. 10 is very verbose.)

11

11filereaderxidxdir  10Scope: shared from IOx01STRING

11

7 Interfaces

General Implements: readinterpolate

Inherits: grid

Grid Variables 7.0.1 PRIVATE GROUPS

11x Group Names x10x Variable Names xx01 Details x11x
11 reflevelseen 10 reflevelseen x01 compact110
11                 10  x01 description11highest refinement level in source data that overlaps this point so far
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11Checkpoint=”no” Prolongation=”none”
11                 10  x01 timelevels111
11                 10  x01vararrayxsize11maxxnumberxofxreadxvariables
11                 10  x01variable type11INT
11 interpthispoint 10 interpthispoint x01  compact110
11                 10  x01 description11fill this point this time around
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11Checkpoint=”no” Prolongation=”none”
11                 10  x01 timelevels111
11                 10  x01variable type11INT
11 interpxcoords 10  x01  compact110
11                 10  interpxx  x01 description11coordinate arrays for the interpolator
11                 10  interpxy  x01 dimensions113
11                 10  interpxz  x01 distribution11DEFAULT
11                 10  interpxdata  x01 group type11GF
11                 10  x01  tags11Checkpoint=”no” Prolongation=”none”
11                 10  x01 timelevels111
11                 10  x01variable type11REAL
11 testxvalues 10 testxvalues x01 compact110
11                 10  x01 description11store values for test
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11Checkpoint=”yes” ProlongationOperator=”none”
11                 10  x01 timelevels113
11                 10  x01variable type11REAL
11 testxresults 10 testxresults x01 compact110
11                 10  x01 description11difference values for test
11                 10  x01 dimensions113
11                 10  x01 distribution11DEFAULT
11                 10  x01 group type11GF
11 10 x01 tags11Checkpoint=”no” ProlongationOperator=”none”
11                 10  x01 timelevels111
11                 10  x01variable type11REAL

11

Uses header: carpet.hh

8 Schedule This section lists all the variables which are assigned storage by thorn EinsteinInitialData/ReadInterpolate. Storage can either last for the duration of the run (Always means that if this thorn is activated storage will be assigned, Conditional means that if this thorn is activated storage will be assigned for the duration of the run if some condition is met), or can be turned on for the duration of a schedule function.

Storage x

11x11Conditional:
11x 11testxvalues[3] testxresults

11x 11x

11

Scheduled Functions

CCTKxPARAMCHECK (conditional) x readinterpolatexparamcheck x sanity check given parameters x

11x11Language:11c
11 x 11Options: 11global
11 x 11Type: 11function

11

CCTKxINITIAL (conditional) x readinterpolatexreaddata x read in datasets from disk x

11x11After: 11admbasexinitialdata
11 x 11x  11hydrobasexinitial
11 x 11Before:11admbasexpostinitial
11 x 11x  11hydrobasexprim2coninitial
11 x 11Type: 11group

11

ReadInterpolatexReadData (conditional) x readinterpolatexread x read in datasets x

11x11Language:11c
11 x 11Options: 11level
11 x 11Storage: 11reflevelseen
11 x 11x  11interpthispoint
11 x 11x  11interpxcoords
11 x 11Type: 11function

11

CCTKxPOSTPOSTINITIAL (conditional) x readinterpolatexfreecache x free memory used for dataset caches

11x11Language:11c
11 x 11Options: 11global
11 x 11Type: 11function

11

ReadInterpolatexReadData (conditional) x readinterpolatexenforcesymmetry x enforce symmeries if desired x

11x11After: 11readinterpolatexread
11 x 11Language:11c
11 x 11Options: 11level
11 x 11Type: 11function

11

ReadInterpolatexReadData (conditional) x applybcs x apply symmetry conditions to read in variables x

11x11After:11readinterpolatexenforcesymmetry
11 x 11Type:11group

11

CCTKxINITIAL (conditional) x readinterpolatexgeneratetestdata x generate polynomial test data x

11x11Language:11c
11 x 11Type: 11function

11

CCTKxPOSTPOSTINITIAL (conditional) x readinterpolatexcomparetestdata x compare to polynomial test data x

11x11Language:11c
11 x 11Type: 11function

11

Aliased Functions x

11Alias Name: xxxxxxx11Function Name:
11ApplyBCs  11ReadInterpolatexApplyBCs