OPTIMIZED SYSTEMS

Benchmarking HPC Systems for Big Data

Salvatore Salamone | Date: 12-14-11 | Comments

The Graph500 list ranks supercomputer systems based on their ability to handle data-intensive applications.

Savvy IT managers know that at social events you never discuss religion, politics and benchmarks. But perhaps the situation is about to change.

The challenge with any benchmark is that the more specific the test criteria (a particular version of a software application running on a well-defined hardware configuration, for example), the less the benchmark lends itself to use in broad comparisons.

Certainly, benchmarks for transactions against large databases and servers running large database applications have been available for years. In fact, organizations routinely use Transaction Processing Performance Council (TPC) and Standard Performance Evaluation Corporation (SPEC) benchmarks to evaluate, compare and select hardware for their workloads.

And while there has always been a great debate about the value of benchmarks that, for example, test only one aspect of a system, such benchmarks prove useful in spotting key technology trends. That has certainly been the case with the twice-a-year Top500.org benchmarking effort that ranks the world�s most powerful supercomputers.

Over the years, the Top500 list has used the Linpack benchmark to measure a computer�s floating-point rate of execution. It is determined by running a computer program that solves a dense system of linear equations. Such equations are often used in physical simulations or some financial modeling efforts.

A more recent benchmarking initiative, started by the Graph500 organization, tries to evaluate a computer�s performance when running data-intensive applications. The group�s Graph500 list could be used to help guide the design of hardware architectures and software systems intended to support such applications.

The Graph500 list is meant to complement other benchmarking efforts such as the Top500 list.

The Graph500 approach stresses supercomputer performance on big data scaling problems rather than on the purely arithmetic computations.

Specifically, Graph500 machines are tested for their ability to solve complex problems involving random-appearing graphs, rather than simply for their speed in solving complex problems. According to the group behind the Graph500 effort, such problems are found in the medical world, where large numbers of medical entries must be correlated; in the analysis of social networks, with their enormous numbers of electronically related participants; and in international security, where, for example, huge numbers of containers on ships roaming the world�s ports of call must be tracked.

�Companies are interested in doing well on the Graph500 because large-scale data analytics are an increasingly important problem area and could eclipse traditional high-performance computing (HPC) in overall importance to society,� said Sandia National Laboratories researcher Richard Murphy, chair of the Graph500 steering committee.