Release Announcement

We are pleased to announce the twelfth release (code name "Somerville") of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. This release includes, among other things, improvements to the Cactus flesh, re-writes of the space-time solver McLachlan, a new GRMHD evolver (IllinoisGRMHD), and a multi-grid elliptic solver working in Carpet (CTThorns). In addition, bug fixes accumulated since the previous release in May 2015 have been included.

The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic magneto-hydrodynamics code GRHydro. The Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic magneto-hydrodynamics.

The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from six different institutions, and host weekly meetings that are open for anyone to join in.

Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers.

For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at <http://einsteintoolkit.org>.

The Einstein Toolkit is primarily supported by NSF 1212401/1212426/1212433/1212460 (Einstein Toolkit), and also by 0905046/0941653 (PetaCactus) and 0710874 (LONI Grid).

The Einstein Toolkit contains about 200 regression test cases. On a large portion of the tested machines, almost all of these test suites pass, using both MPI and OpenMP parallelization.

The changes between this and the previous release include:

  • Larger changes since last release
  • Almost all repositories moved from the Einstein Toolkit subversion server to bitbucket, with the ExternalLibraries being the only exception. Taking this move as opportunity, some already existing thorns were moved to more appropriate arrangements.
  • Cactus Flesh
    • While not a particular user-visible change, the Cactus version number was bumped to 4.3.0, as a new version of Cactus was released at the same time as this Einstein Toolkit release.
    • Thorns can specify versions for capabilities they provide, and thorns using them can depend on specific versions of those.
    • pthread support got moved into a thorn (ExternalLibraries/pthreads)
  • McLachlan
    • re-write of the BSSN thorns: some parameter names changed. Backwards compatible parameter names exist, but the old are deprecated now. Please switch to the new, more consistent, parameter names.
  • New thorns or tools
    • IllinoisGRMHD (and connected thorns): a new GRMHD code within the Einstein Toolkit. This isn't a drop-in replacement for GRHydro (as a lot of the functionality, and parameters are different), but should be seen as alternative. They are placed in the WVUThorns arrangement.
    • CTThorns: Elliptic solver for Cactus/Carpet. This is a multi-grid solver, and so far has mostly been used for cosmology-type simulations.
    • AH2xdmf.py script (new utility within the AHFinderDirect thorn): reads ASCII AHFinderDirect output files and combines data into hdf5 file that can easily be visualized as time-series within tools like VisIt
    • PITTNullCode/SphericalHarmonicReconGen: generic version of SphericalHarmonicRecon
  • How to upgrade from Hilbert (ET_2015_05)

To upgrade from the previous release, use GetComponents with the new thornlist to check out the new version. Switching to the new branch is not possible for most repositories due to repository moves.

See the Download page on the Einstein Toolkit website for download instructions.

  • Remaining issues with this release
  • Certain machines need to be configured specially in Simfactory because the remote directories cannot be determined automatically just from the user name. See the Machine notes below.
  • Recovering with Carpet: Carpet stores metadata (such as the simulation time) only for Carpet::max_timelevels time levels, although it is possible to allocate more time levels. These additional time levels then cannot be recovered; the symptom is an assertion failure during recovery. The solution is to either increase Carpet::max_timelevels, or to decrease the number of active time levels.
  • Machine notes

Supported (tested) machines include:

  • Default Debian, Ubuntu, Fedora and MacOS installations
  • Bethe
  • Bluewaters *
  • Carver
  • Comet
  • Datura
  • Edison
  • Fermi
  • Galileo
  • Gordon
  • Gullimin
  • Hopper
  • Nvidia
  • Philip
  • Queenbee 2
  • Shelob
  • Supermike II
  • Stampede (CPU) *
  • Zwicky

A * means that a small number of tests fail on that machine. Almost all tests pass on the other machines.

  • Stampede: defs.local.ini needs to have sourcebasedir = $WORK and basedir = $SCRATCH/simulations configured for this machine. You need to determine $WORK and $SCRATCH by logging in to the machine.

All repositories participating in this release carry a branch ET_2015_11 marking this release. These release branches will be updated if severe errors are found.

The "Somerville" Release Team on behalf of the Einstein Toolkit Consortium (2015-11-11)

   Steven R. Brandt
   Peter Diener
   Roland Haas
   Ian Hinder
   Frank Löffler
   Bruno C. Mundim
   Erik Schnetter
   Barry Wardell

November 11, 2015