August 17 2020

Abstract

A FileReader like thorn that uses InterpolateLocalUniform to interpolate the data read in onto the new grid.

### 1 Introduction

IOUtil [1] defines a file reading interface via the IO parameters filereader_ID_dir, filereader_ID_files, filereader_ID_vars parameters. IOUtil uses the checkpoint recovery routines to read grid variable data from disk. This requires that the grid resolution used to write the files agrees with the resolution using when reading in data and that, for each refinement level, the to be read in region is fully contained in the corresponding refinement level in the source files.

ReadInterpolate relaxes this restriction by allowing arbitrary changes in resolution and grid structure in between source files and simulation. It uses Cactus local interpolation operators CCTK_InterpLocalUniform to interpolate data using the highest resolution source data available for each target grid point.

It provides parameters files and only_these_datasets that mirror IOUtil’s parameters to select files and datasets in IO::filereader_ID_dir.

Auxiliary parameters are provided to fine tune what is read and to control the interpolator.

### 2 Physical System

ReadInterpolate takes input data $\psi$ on a grid patch described by its origin $\stackrel{\to }{O}$, grid spacing $\Delta \stackrel{\to }{x}$ and shape $\stackrel{\to }{N}$. Data is interpolated to the coordinates of the simulation grid $\stackrel{\to }{x}$ using Cactus’ interpolation operators: $\psi \left(\stackrel{\to }{x}\right)=L\left(\psi ,\stackrel{\to }{x};\stackrel{\to }{O},\Delta \stackrel{\to }{x},\stackrel{\to }{N}\right)$.

### 3 Numerical Implementation

For a target point $\stackrel{\to }{x}$ multiple datasets in the source files may contain this physical coordinate. When this happens the following selection algorithm applies:

• unless the candidate datasets overlaps the outer boundary of the current grid, require that there are at least cctk_nghostzones or interpolator_half_width extra points for use by the interpolator. This may involve using an off-centered interpolation stencil for points near the outer boundary. Then
• if reading in data from time level 0, pick the largest refinement level dataset that contains the point, otherwise pick the refinement level that matches the refinement level of the target point. This assumes that the files were written at a timestep when all refinement level align in time and that the timestep in the data files and simulation are identical. Setting

Once a source data set has been selected its data and the target coordinate triplet are passed to CCTK_InterpLocalUniform for interpolation.

### 4 Using This Thorn

This thorn is intended as an initial data thorn, scheduling itself in the CCTK_INITIAL time bin of Cactus.

#### 4.1 Obtaining This Thorn

This thorn is part of the Einstein Toolkit in the EinsteinInitialData repository.

#### 4.2 Basic Usage

To use this thorn, set IO::filereader_ID_dir to the directory containing the data files, set ReadInterpolate’s files to the base names of the file to read and only_these_datasets to a comma separated list of regular expressions to match the datasets to read. To obtain the list of data sets present in the file the h5ls tool, which is part of HDF5, can be used. The parameter max_number_of_read_variables must be set to the expected number of grid variables to be read.

If desired read in data can be moved to a different position on the grid using shift_read_datasets_by[3] which relocates the origin of the coordinates in the read in data to the given coordinate triplet.

#### 4.3 Special Behavior

ReadInterpolate attempts to avoid one sided interpolation as much as possible, yet tried to allow reading in data at the outer boundaries of the grid. A heuristic is used to allow one sided derivatives there. This requires changing interpolator_pars to allow off centered interpolation stencils.

A number of parameters are provided to select which datasets to read:

minimum_reflevel
ignore any datasets with refinement level less than this,
maximum_reflevel
ignore any datasets with refinement level higher than this,

Some Carpet version contain a bug and do not set origin correctly for cell centered grids (it is off by half a grid point, i.e. it does not take the staggering into account). This option fixes this issue by offsetting all datasets that should be offset. Setting fix_cell_centered_origins adds an offset to correct for this. This is incorrect for files where Carpet already used the correct origin. This parameter only affects cell centered input data.

Interpolation parameters ReadInterpolate uses interpolator_name as the local interpolator to interpolate read in data to simulation grid points. It passes interpolator_vars to the interpolator. The default values select AEILocalInterp’s Hermite interpolator and forbid off centered interpolation stencils or extrapolation.

ReadInterpolate uses epsilon as a fuzziness parameter when determining if a point is contained in a read in dataset. This may require changes for very large grids.

When deciding if a read in dataset is large enough to allow for interpolation without off centered stencils ReadInterpolate requires knowledge about the stencil width of the chosen interpolation operator. This can be explicitly specified as interpolator_half_width or alternatively assumed to be cctk_nghostzones if this parameter is set to -1, which is the default.

This makes sure that the interpolator does not off center the interpolation stencil if there are insufficient points to interpolate, which can happen if there are insufficient ghost-zones for the interpolation method used, and can lead to processor-number dependent results.

ReadInterpolate is no aware of any symmetry conditions used in either the simulation used to produce the data files the current simulation. Thus it will interpolated into symmetry points in the same manner as for outer boundary points. In these situations the parameter enforce_symmetries_after_reading to apply symmetry boundary conditions after having read in all data. This does not enable ReadInterpolate to read in data produced on, for example, an octant grid and provide data on a full grid.

#### 4.4 Interaction With Other Thorns

ReadInterpolate assumes that data files were written by CarpetIOHDF5 and may not work with files written by PUGH’s IOHDF5.

This thorn is intended as an initial data thorn, however since it is generic it does not extend ADMBase’s or HydroBase’s initial_XXX parameters.

ReadInterpolate can be used to read data into curvilinear grids set up by Llama but cannot read data from curvilinear grids.

#### 4.5 Examples

The test suite file admhydro.par shows how to use ReadInterpolate to read in metric and fluid variables:

# ReadInterpolate ignore these but they are required to allocate storage
HydroBase::initial_hydro = "zero"

#### 4.6 Support and Feedback

Please use the Einstein Toolkit ticket system to report issues.

### 5 History

This thorn was originally inspired by SpEC’s [2] ReadTensorsFromDiskWithMap item and developed to read in single neutron star data produced by THC [3] into GRHydro [4]. It has been found useful to in combination with CT_MultiLevel since then.

#### 5.1 Acknowledgments

David Radice and Philipp Mösta provided bug reports and comments during the time this thorn was developed.

### References

[1]   https://bitbucket.org/cactuscode/cactusbase/src/master/IOUtil/

[2]   Spectral Einstein Code, https://www.black-holes.org/code/SpEC.html

[3]   WhiskyTHC: The General-Relativistic Templated Hydrodynamics Code, http://personal.psu.edu/dur566/whiskythc.html

[4]   GRHydro is an evolution code for a general-purpose 3D relativistic hydrodynamics, https://bitbucket.org/einsteintoolkit/einsteinevolve/src/master/GRHydro/

### 6 Parameters

 enforce_symmetries_after_reading Scope: private BOOLEAN Description: apply symmetry boundary conditions after reading data Default: no

 epsilon Scope: private REAL Description: Tolerance when deciding if a grid point lies within an overlap region Range Default: 1e-12 0:* any positive number, should be similar to AEILocalInterp’s tolerance

 files Scope: private STRING Description: List of basenames of files to read in as initial data (e.g. omit the .file_XXX.h5 part) Range Default: (none) .+ Space-separated list of initial data filenames \$̂ An empty string for not recovering initial data

 fix_cell_centered_origins Scope: private BOOLEAN Description: offset rl>1 cell centered input data by half a grid step to fix broken HDF5 files Default: no

 interpolator_half_width Scope: private INT Description: maximum number of points to one side of the interpolated location the interpolator uses Range Default: -1 1:* this is similar to cctk_nghostzones, 1 for linear and constant interpolation 2:* 2nd for second order and 3rd interpolation -1 use cctk_nghostzones

 interpolator_name Scope: private STRING Description: Which interpolator should I use Range Default: Hermite polynomial interpolation .+ Any nonempty string

 interpolator_pars Scope: private STRING Description: Parameters for the interpolator Range Default: order=3 boundary_off_centering_tolerance={1e12 1e12 1e12 1e12 1e12 1e12} boundary_extrapolation_tolerance={1e-12 1e-12 1e-12 1e-12 1e-12 1e-12} .* ”Any string that Util_TableSetFromStr ing() will take”

 max_number_of_read_variables Scope: private INT Description: how many variables will be read in? Range Default: 1 1:* any positive number

 maximum_reflevel Scope: private INT Description: ignore any datasets with refinement level higher than this Range Default: 1000 0:* maximum refinement level to keep

 minimum_reflevel Scope: private INT Description: ignore any datasets with refinement level less than this Range Default: (none) 0:* minimum refinement level to keep

 only_these_datasets Scope: private STRING Description: comma separated list of dataset regular expression patterns to consider for reading Range Default: (none) .+ any valid Cactus regular expression see [1] below refinement level 6, all metric :g[xyz][xyz],:rho metric and rho everywhere

[1]

 shift_read_datasets_by Scope: private REAL Description: add to all read in coordinates Range Default: 0.0 *:* what used to be origin in datasets appear here

 test Scope: private KEYWORD Description: control how test data is used Range Default: no no do not use test data generate generate test data compare compare read data to test data

 verbosity Scope: private INT Description: how much diagnostic output to produce Range Default: 1 0:0 only errors 1:1 warn about suspicous states 2:* more and more detailed debug output (values up to 10 are defined. 10 is very verbose.)

 filereader_id_dir Scope: shared from IO STRING

### 7 Interfaces

Implements:

Inherits:

grid

#### Grid Variables

##### 7.0.1 PRIVATE GROUPS
 Group Names Variable Names Details reflevelseen reflevelseen compact 0 description highest refinement level in source data that overlaps this point so far dimensions 3 distribution DEFAULT group type GF tags Checkpoint=”no” Prolongation=”none” timelevels 1 vararray_size max_number_of_read_variables variable type INT interpthispoint interpthispoint compact 0 description fill this point this time around dimensions 3 distribution DEFAULT group type GF tags Checkpoint=”no” Prolongation=”none” timelevels 1 variable type INT interp_coords compact 0 interp_x description coordinate arrays for the interpolator interp_y dimensions 3 interp_z distribution DEFAULT interp_data group type GF tags Checkpoint=”no” Prolongation=”none” timelevels 1 variable type REAL test_values test_values compact 0 description store values for test dimensions 3 distribution DEFAULT group type GF tags Checkpoint=”yes” ProlongationOperator=”none” timelevels 3 variable type REAL test_results test_results compact 0 description difference values for test dimensions 3 distribution DEFAULT group type GF tags Checkpoint=”no” ProlongationOperator=”none” timelevels 1 variable type REAL

carpet.hh

### 8 Schedule

This section lists all the variables which are assigned storage by thorn EinsteinInitialData/ReadInterpolate. Storage can either last for the duration of the run (Always means that if this thorn is activated storage will be assigned, Conditional means that if this thorn is activated storage will be assigned for the duration of the run if some condition is met), or can be turned on for the duration of a schedule function.

#### Storage

 Conditional: test_values[3] test_results

#### Scheduled Functions

CCTK_PARAMCHECK (conditional)

sanity check given parameters

 Language: c Options: global Type: function

CCTK_INITIAL (conditional)

 Language: c Options: level Storage: reflevelseen interpthispoint interp_coords Type: function

CCTK_POSTPOSTINITIAL (conditional)

free memory used for dataset caches

 Language: c Options: global Type: function

enforce symmeries if desired

applybcs

apply symmetry conditions to read in variables

CCTK_INITIAL (conditional)

generate polynomial test data

 Language: c Type: function

CCTK_POSTPOSTINITIAL (conditional)