PAPI

Federico Cipolletta, Roland Haas

2020-07-01

Abstract

This thorn provides access to the PAPI library.

1 Using This Thorn

Refer to the Cactus UserGuide, Sec. B2.2, in order to know how this thorn can be used in a compiled configuration and how to possibly linking another specific version, already installed steparately.

Note on possible ExternalLibraries’ location stripping

Each thorn contained in Cactus/arrangements/ExternalLibraries will automatically adopt the library version contained in the Cactus/arrangements/<library>/dist folder. In particular, the tarball in Cactus/arrangements/<library>/dist is only used if either THORN_DIR is set to BUILD or is left empty and no precompiled copy of the library is found. If another location is specified via the THORN_DIR variable in the <machine>.cfg file at compilation, then the Cactus/lib/sbin/strip-incdirs.sh script will automatically strip away (for safety reasons) the locations:

/include

/usr/include

/usr/local/include

from THORN_INC_DIRS which default to THORN_DIR/include. Therefore, if there is any need for using one already installed version of one external library, the aforementioned location should be avoided (e.g. indicating /home as the THORN_DIR will work with no problems if the required library is installed there) or should be carefully checked, in order to avoid unwanted stripping. The same stripping happens to THORN_LIB_DIRS in lib/sbin/strip-libdirs.sh with a larger list of directories:

/lib

/usr/lib

/usr/local/lib

/lib64

/usr/lib64

/usr/local/lib64

2 Parameters




define_statistics_clocks
Scope: private BOOLEAN



Description: Define multiple clocks for each PAPI counter, providing min/max/avg/sdv



Default: no






dgemm_n
Scope: private INT



Description: Matrix size; mem = 32 Nˆ2  bytes, cpu = 2 Nˆ3  flop



Range Default: 1000
1:*
N=2000 requires 128 Mbyte and 16 Gflop; N=5000 requires 800 Mbyte and 250 Gflop






events_dcache
Scope: private STRING



Description: PAPI events for data cache usage



Range Default: PAPI_L1_DCA PAPI_L1_DCM PAPI_L2_DCA PAPI_L2_DCM PAPI_L3_DCA PAPI_L3_DCM






events_flops
Scope: private STRING



Description: PAPI events for flop/sec



Range Default: PAPI_FP_OPS PAPI_DP_OPS PAPI_VEC_DP PAPI_TOT_CYC PAPI_REF_CYC SIMD_FP_256:PACKED_DOUBLE






events_icache
Scope: private STRING



Description: PAPI events for instruction cache usage



Range Default: PAPI_L1_ICA PAPI_L1_ICM PAPI_L2_ICA PAPI_L2_ICM PAPI_L3_ICA PAPI_L3_ICM






events_ipc
Scope: private STRING



Description: PAPI events for instructions/cycle



Range Default: PAPI_TOT_CYC PAPI_TOT_INS PAPI_FP_INS PAPI_INT_INS






events_memory
Scope: private STRING



Description: PAPI evens for memory accesses



Range Default: PAPI_LD_INS PAPI_SR_INS PAPI_MEM_RCY PAPI_MEM_WCY






out_every
Scope: private INT



Description: Output statistics every so many iterations



Range Default: (none)
after initial data and shutdown
1:*
every so many iterations
-1
no statistics output






per_thread_statistics
Scope: private BOOLEAN



Description: Collect statistics per thread (instead of per process)



Default: yes






use_multiplexing
Scope: private BOOLEAN



Description: Use multiplexing to gather more, but less accurate information



Default: yes






verbose
Scope: private BOOLEAN



Description: Output debug messages?



Default: no



3 Interfaces

General

Implements:

papi

4 Schedule

This section lists all the variables which are assigned storage by thorn ExternalLibraries/PAPI. Storage can either last for the duration of the run (Always means that if this thorn is activated storage will be assigned, Conditional means that if this thorn is activated storage will be assigned for the duration of the run if some condition is met), or can be turned on for the duration of a schedule function.

Storage

NONE

Scheduled Functions

CCTK_WRAGH

  papi_init

  initialise papi

 

  Language: c
  Options: meta
  Type: function

CCTK_WRAGH

  papi_register_clock

  set up cactus clock for papi

 

  After: papi_init
  Language: c
  Options: meta
  Type: function

CCTK_BASEGRID

  papi_dgemm

  run a dgemm benchmark to test papi

 

  Language: c
  Options: meta
  Type: function

CCTK_ANALYSIS

  papi_output_stats_analysis

  output papi statistics

 

  Language: c
  Options: meta
  Type: function

CCTK_TERMINATE

  papi_output_stats_terminate

  output papi statistics

 

  Language: c
  Options: meta
  Type: function