When
the balance between computation and communication for a given dataset yields poor
performance, application developers and analysts can use profiling tools to identify
bottlenecks and draw conclusions about how performance is being degraded. Then they can
modify their code to balance the workload and optimize the size of I/O operations and the
rate at which they take place. This is done by minimizing the total number of I/O
requests and by maximizing the efficiency of those made.
I/O profiling tools enable analysts to understand how their system interacts with complex storage management schemes. Characterizing I/O behavior, these tools illuminate when and where unsuitable data distributions, caching, prefetching, and write-back procedures are employed. They reveal the segments of application programs where code modifications will have the highest impact in enhancing performance.
To identify the code segments that are prime candidates for optimization, analysts and developers use instrumentation software that profiles the I/O behavior of their applications. When attached to segments of application source code, instrumentation software acts as an assortment of probes, capturing performance metrics during program execution. These probes feed the performance metrics they catch into a trace file that can be used with analysis tools to generate a comprehensive portrayal of I/O behavior.
The structure of the trace file is dictated by the data format employed. Typically trace files must accommodate an abundance of individual records, each retaining the metrics of a single instance of a distinct I/O activity. Reads, writes, seeks, opens, closes, and other such discrete I/O activities each represent a specific I/O event type. Performance datasets generally consist of many I/O event types with potentially thousands of data records for each type, so trace files are often quite large. To accommodate this, the data format employed must be compact and, in some cases, it should support data compression. And, because so many I/O events are known to influence performance and others can be expected to emerge as influential, the data format employed should also support a broad, diverse, and extensible set of event types. To accommodate traces collected on one machine and analyzed on another, a portable data format makes sense.
The Pablo Self-Defining Data Format (SDDF) was designed to accommodate this set of requirements, unique to performance trace files. Describing the structural layout of event records in such files, SDDF is a flexible data meta-format that specifies both data record instances and data record structures. Describing not only records with defined layouts, but arbitrary data records as well, SDDF simplifies the process of working with a diverse, expanding set of trace event types. It is used by all of the instrumentation and analysis software developed by the Pablo Research Group.
Once Pablo instrumentation software has been inserted before and after an application’s I/O commands, it captures performance metrics as the application runs, generating data describing the time, duration, size, and other parameters of each I/O operation. This information can be directly recorded as a raw event trace file for off-line analysis. Alternately, for use in statistical summaries, performance data can be processed prior to recording by one or more data-analysis tools. For online access to XML representations of data collected in SDDF records, the Pablo Research group has developed a conversion utility that translates data and data summaries such that they can be represented on the Web. Or, for those working in a visualization environment, data from SDDF records can be directly processed and displayed in real-time. Collectively the raw event traces, the statistical summaries, and the visualization tools provide a detailed view of application I/O request patterns and file system responses.
Go on to Tuning for Performance: Analyzation and Optimization
Return to Primer Table of Contents