The HDF extension to the Pablo Trace Library is a set of programs that can be substituted for standard HDF calls so that I/O performance data can be captured and measured. These programs perform exactly the same routines as their standard HDF counterparts, but they have been augmented by instrumentation software, attached before and after the call itself, to capture internal data. Programmers compile their instrumented source code and link it with the trace library. During program execution, perfomance data captured by the instrumentation is recorded in SDDF files. Analysts use Pablo analysis tools to study these SDDF files and understand how HDF behaves and interacts with other system components. This tutorial includes: instructions for instrumenting a code, instructions for compiling, linking, and executing instrumented source code as well as access to an end-to-end example of performance analysis.I
In profiling the performance of an application's HDF behavior, Pablo tools track, or trace, the HDF events that take place during program execution. An event, in this context, occurs each time a specific HDF function is executed. When the HDF library is compiled with the special Pablo tracing option, calls to Pablo Tracelibrary routines, which are the trace library versions of the corresponding HDF calls or commands, are made on entry to HDF routines and just prior to exiting them. This enables the function of the original HDF routine to be executed and metrics reflecting its performance to be captured and written to an SDDF record.
When an instrumented event occurs, depending on the type of tracing employed, the trace software either generates a record of the occurrence, a trace record, or records the information in a table containing a summary of the HDF activity. Both types of tracing produce records written in SDDF format. Each SDDF file contains both record descriptors and data records. The record descriptors define the structure of data records to be generated, enabling ready interpretation of trace data records by analysis tools.
The Pablo HDF Trace Utilities provide for the production of two types of HDF trace records: Runtime trace records and Summary trace records.
The data records within an SDDF file contain the actual metrics captured for a given occurence of a trace event, ordered according to the corresponding descriptor record. The descriptors are generated at initialization time with a call to HDFInitTrace. Data records are generated when the targeted event is called.
If the Runtime Tracing option is used, an SDDF packet, called an HDF procedure trace record, is produced each time an HDF procedure is entered. The HDF procedure trace record contains data indicating the type of procedure, the processor number, and the time the record was produced.
On return from a traced HDF procedure, another HDF procedure trace record is produced. Each I/O operation produces an SDDF record packet containing the duration of the operation. The packets produced for Read and Write operations also contain the number of bytes transferred by the operation. The packets of Open operations contain the name of the file opened. Similarly, each MPI I/O operation produces an SDDF packet of information. The packets are written to the trace file associated with the processor.
At the end of the run, the trace files contain enough informaiton to produce a thorough analysis of the I/O, MPI I/O, and HDF activity that occurred during execution of the program.
If the Summary Tracing option is used, statistics about the HDF, Unix I/O, and MPI I/O activity are recorded in tables in memory during runtime. Prior to the end of execution, an SDDF record, called an HDF summary record, is produced for each of the HDF procedures that was traced. These records are written to the trace file associated with the processor. They summarize the total number of I/O operations and calls to MPI I/O and other traced HDF procedures that occurred within each of the traced HDF procedures during runtime.
The Runtime Tracing option produces much more information than the Summary Tracing but may also produce trace files that are extremely large. The size of the Summary tracing output files are roughly proportional to the number of HDF procedures being traced and the number of processors used during execution.
At this time, Pablo HDF tracing only supports the C language. The main thing is that you need to include the Pablo header file, ProcTrace.h. This header file can be found in the include directory for the instrumented HDF distribution.
Suppose the Pablo software is installed in the directory <PabloDir>, the instrumented HDF software is installed in the directory <HDFDir>, and the MPI software is installed in the directory <mpiDir>.
To compile the code, use the following command:
For scalar C cc -c hdfExample.c -I<HDFDir>/include [other compile flags as necessary] For MPI C mpicc -c hdfExample.c -I<HDFDir>/include -I<mpiDir> [other compile flags as necessary]
Linking of Pablo-instrumented HDF 4 and HDF 5 applications differs slightly because of the library names. HDF 4 uses the libraries -libdf-inst.a and libmhdf-inst.a; HDF 5 uses the single library, libhdf5-inst.a. In addition, the application must be linked with the libraries libPabloTrace.a and libPabloTraceExt.a, which can be found in the lib subdirectory of the directory containing the Pablo software.
HDF 4: For scalar C cc -o myEXE hdfExample.o -ldf-inst -lmhdf-inst -L<Pablodir>/lib -lPabloTrace -lPabloTraceExt [other libraries as necessary] HDF 5: For scalar C cc -o myEXE hdfExample.o -lhdf5-inst -L<Pablodir>/lib -lPabloTrace -lPabloTraceExt [other libraries as necessary] FDF 4: For MPI C mpicc -o myEXE hdfExample.o -ldf-inst -lmhdf-inst -L<Pablodir>/lib -lPabloTrace -lPabloTraceExt -lmpi [other libraries as necessary] HDF 5 For MPI C mpicc -o myEXE hdfExample.o -lhdf5-inst -L<Pablodir>/lib -lPabloTrace -lPabloTraceExt -lmpi [other libraries as necessary]
Execution is as usual:
Scalar myExe MPI mpirun -np <nprocs> myEXE
This will produce a single file in the scalar case and a tracefile for each processor in the MPI case.
Download the Users Guide to Pablo I/O Instrumentation
Download
the Pablo HDF Library User Guide