xx Formaline — conserve meta data about runs forever

Formaline — conserve meta data about runs forever

Erik Schnetter <schnetter@aei.mpg.de>

Date

Abstract

This thorn Formaline collects and preserves meta data about the run. It can contact a database server at run time and store the meta data there. Such meta data include e.g.xthe parameter file, date, time, machine, and user id of the run, location of the output data, number of iterations, an efficiency summary, etc. It also writes a copy of the executable’s source code into the output directory.

1 Utility Programs provided by Formaline

Thorn Formaline provides the following utility programs:

formaline.py

A python script for gdb which allows you to recover the source tarballs from the executable even if the executable can no longer be run. It writes the extracted tarballs into the current directory. Use as

xxgdbx-Pxformaline.pyxexe/cactus_sim

All utility programs are located in src/util.

2 Parameters

11 announcextoxportal 10 Scope: private01 BOOLEAN

11 Description: Announce meta information to the portal

11                               10              01  Default: no

11

11 collectxmetadata  10 Scope: private01  BOOLEAN

11 Description: Collect metadata and output it in various ways

11                               10              01  Default: yes

11

11 createxidxfiles  10 Scope: private01  BOOLEAN

11 Description: Create files from the build and job ids

11                               10              01  Default: yes

11

11 maxxwarnxlevel  10 Scope: private01  INT

11 Description: Only output warnings with at most this level of severity

11  Range  10              01  Default: -1

11 -1:*  10

11

11 metadataxlifetime  10 Scope: private01  INT

11 Description: Lifetime of meta information sent to a server (in hours)

11 Range 10 01 Default: 100
11                               10 zero to store the metadata forever
11 1:*  10 a positive value to limit the lifetime to so many hours

11

11 nrxofxparameterxchangesxtoxbexlogged 10 Scope: private01  INT

11 Description: number of parameter changes to be logged

11 Range 10 01 Default: -1
11 -1:  10 log all changes
11 0:  10 no logs please
11 1:*  10 log this many changes for a given parameter, then stop logging

11 outxreductions  10 Scope: private01  STRING

11 Description: Reduction operators that should be applied to the output variables

11 Range 10 01Default: minimum maximum norm1 norm2 normxinf sum
11                               10 space separated list of reduction operators

11

11 outxvars  10 Scope: private01  STRING

11 Description: Cactus variables that should be output

11 Range 10 01 Default: (none)
11                               10 space separated list of groups or variables

11

11 outputxinfo  10 Scope: private01  BOOLEAN

11 Description: Output info messages?

11                               10              01  Default: no

11

11 outputxsource  10 Scope: private01  BOOLEAN

11 Description: Output a tarball with the cactus source tree

11                               10              01  Default: yes

11

11 outputxsourcexsubdirectory  10 Scope: private01  STRING

11 Description: Subdirectory, relative to IO::outxdir, for the Cactus source tarballs

11  Range  10              01  Default: cactus-source

11                               10

11

11 portalxhostname  10 Scope: private01  STRING

11 Description: Portal host name

11  Range  10              01  Default: (none)

11                               10

11

11 portalxport  10 Scope: private01  INT

11 Description: Portal port

11 Range 10 01 Default: 9296
11 1:65535  10

11

11 portalxusername  10 Scope: private01  STRING

11 Description: User name on the portal

11  Range  10              01  Default: (none)

11                               10

11

11 publishxlevel  10 Scope: private01  INT

11 Description: Importance level for meta data to be published via the Publish API

11 Range 10 01 Default: 1
11 0:10  10 the lower the level the more important the meta data to be published

11 rdfxhostname  10 Scope: private01  STRING

11 Description: RDF server host name

11  Range  10              01  Default: (none)

11                               10

11

11 rdfxport  10 Scope: private01  INT

11 Description: RDF server port

11 Range 10 01 Default: 9296
11 1:65535  10

11

11 relayxhost  10 Scope: private01  STRING

11 Description: Relay host for connecting to the portal

11 Range 10 01 Default: (none)
11 .+  10 host name
11                               10 try to determine a relay host automatically

11

11 sendxasxrdf  10 Scope: private01  BOOLEAN

11 Description: Send meta as RDF to a server

11                               10              01  Default: no

11

11 steeredxparametersxlogxexclusionxlist 10 Scope: private01  STRING

11 Description: Parameters for which runtime changes should not be logged

11 Range 10 01 Default: CarpetRegrid::coordinates
11                               10 ”whitespace-separate d list of parameter names”

11

11 storagexfilename  10 Scope: private01  STRING

11 Description: File name for meta information

11  Range  10              01  Default: formaline-jar.txt

11                               10

11

11 storagexjsonxfilename  10 Scope: private01  STRING

11 Description: File name for meta information in JSON format

11  Range  10              01  Default: formaline-jar.json

11                               10

11

11 storexintoxfile  10 Scope: private01  BOOLEAN

11 Description: Store meta information in a file

11                               10              01  Default: yes

11

11 storexintoxjsonxfile  10 Scope: private01  BOOLEAN

11 Description: Store meta information in a file in JSON format

11                               10              01  Default: yes

11 timeout  10 Scope: private01  INT

11 Description: Timeout for sending meta information to a server (in seconds)

11 Range 10 01 Default: 10
11 1:*  10 a positive value

11

11 updatexinterval  10 Scope: private01  REAL

11 Description: Update interval for the meta information (in seconds)

11  Range  10              01  Default: 3600.0

11 0:*  10

11

11 usexrelayxhost  10 Scope: private01  BOOLEAN

11 Description: Use a relay host for connecting to the portal

11                               10              01  Default: yes

11

11 verbose  10 Scope: private01  BOOLEAN

11 Description: Produce screen output

11                               10              01  Default: no

11

11 outxdir  10 Scope: shared from IO01STRING

11

11 outxsavexparameters  10 Scope: shared from IO01KEYWORD

11

3 Interfaces

General Implements: formaline

Uses header: httpxContent.h Publish.h Provides: UniqueConfigID to UniqueBuildID to UniqueSimulationID to UniqueRunID to

4 Schedule This section lists all the variables which are assigned storage by thorn CactusUtils/Formaline. Storage can either last for the duration of the run (Always means that if this thorn is activated storage will be assigned, Conditional means that if this thorn is activated storage will be assigned for the duration of the run if some condition is met), or can be turned on for the duration of a schedule function.

Storage NONE

Scheduled Functions

CCTKxWRAGH (conditional) x formalinexoutputsource x output cactus source tree x

11 x 11 Language: 11 c
11 x 11 Options: 11 meta
11 x 11 Type:  11 function

11

CCTKxSTARTUP (conditional) x formalinexprintids x print the build and simulation ids x

11 x 11 Language: 11 c
11 x 11 Options: 11 meta
11 x 11 Type:  11 function

11

(conditional) x registerpublishcallbacks x tell cactus that this group exists, but is not scheduled from here

11 x 11 Type: 11 group

11

RegisterPublishCallbacks (conditional) x formalinexregisterpublishrdfxcallbacks x register rdf callbacks for the publish api x

11 x 11 Language: 11 c
11 x 11 Options: 11 meta
11 x 11 Type:  11 function

11

CCTKxWRAGH (conditional) x formalinexregisterwarnings x register to receive warnings and info messages from the flesh x

11 x 11 Language: 11 c
11 x 11 Options: 11 meta
11 x 11 Type:  11 function

11

CCTKxWRAGH (conditional) x formalinexannounceinitial x put some meta information about the current run into permanent storage x

11 x 11 Language: 11 c
11 x 11 Options: 11 global
11 x 11 Type:  11 function

11

CCTKxANALYSIS (conditional) x formalinexannounceupdate x put some meta information about the current run into permanent storage x

11 x 11 Language: 11 c
11 x 11 Options: 11 global
11 x 11 Type:  11 function

11

CCTKxTERMINATE (conditional) x formalinexannouncefinal x put some meta information about the current run into permanent storage x

11 x 11 Language: 11 c
11 x 11 Options: 11 global
11 x 11 Type:  11 function

11

(conditional) x unregisterpublishcallbacks x tell cactus that this group exists, but is not scheduled from here x

11 x 11 Type: 11 group

11

UnregisterPublishCallbacks (conditional) x formalinexunregisterpublishrdfxcallbacks x unregister rdf callbacks for the publish api x

11 x 11 Language: 11 c
11 x 11 Options: 11 meta
11 x 11 Type:  11 function