Abstract—To determine whether a High-Performance Computing (HPC) data center is energy efﬁcient, various aspects have to be taken into account: the data center’s power distribution and cooling infrastructure, the HPC system itself, the inﬂuence of the system management software, and the HPC workloads; all can contribute to the overall energy efﬁciency of the data center. Currently, two well-established metrics are used to determine energy efﬁciency for HPC data centers and systems: Power UsageEffectiveness (PUE) and FLOPS per Watt (as deﬁned by theGreen500 in their ranking list). PUE evaluates the overhead for running a data center and FLOPS per Watt characterizes the energy efﬁciency of a system running the High-Performance Linpack (HPL) benchmark, i.e. ﬂoating point operations per second achieved with 1 watt of electrical power. Unfortunately, under closer examination even the combination of both metrics does not characterize the overall energy efﬁciency of a HPC data center.

First, HPL does not constitute a representative workload for most of today’s HPC applications and the rev 0.9 Green500 run rules for power measurements allows for excluding subsystems (e.g. networking, storage, cooling). Second, even a combination of PUE with FLOPS per Watt metric neglects that the total energy efﬁciency of a system can vary with the characteristics of the data center in which it is operated. This is due to different cooling technologies implemented in HPC systems and the difference in costs incurred by the data center removing the heat using these technologies.

To address these issues, this paper introduces the metrics system PUE (sPUE) and Data center Workload Power Efﬁciency (DWPE). sPUE calculates the overhead for operating a given system in a certain data center. DWPE is then calculated by determining the energy efﬁciency of a speciﬁc workload and dividing it by the sPUE. DWPE can then be used to deﬁne the energy efﬁciency of running a given workload on a speciﬁc HPC system in a speciﬁc data center and is currently the only fully-integrated metric suitable for rating an HPC data center’s energy efﬁciency. In addition, DWPE allows for predicting the energy efﬁciency of different HPC systems in existing HPC data centers, thus making it an ideal approach for guiding HPC system procurement. This paper concludes with a demonstration of the application of DWPE using a set of representative HPC workloads.