The document discusses optimizing HPC applications for energy efficiency. It finds that optimizing for speed often optimizes energy use as well, but frequency scaling can save energy for some memory-bound workloads. Tools like Allinea MAP and Performance Reports can measure energy at different levels and help developers and users optimize applications when it makes sense to prioritize energy over time. Measuring whole system energy is important, and opportunities to improve efficiency must consider the opportunity cost of slowing down science.

1. The impact of software on energy
use – and what can we do about it

2. Energy in HPC
The world’s top 500
supercomputers cost 400M€
annually in energy alone
If software reduces
its energy footprint
… payback could
be enormous
Solution
Enable developers
and users to
improve application
energy
consumption

3. Two Key Questions
• Can developers optimize code for energy?
• Can owners and users tune applications for
energy?

4. What is
energy?
Approximations for Energy
• Floating point, vector operations,
memory access
• L1 or L2 misses vs main memory
orders of magnitude in energy
Heuristics
• Real data from some processor,
memory subsystems, accelerators
• Available in kernel - Intel RAPL
Low level
measurement
• PDU and server level readings
• Real data – real energy
Server level
monitoring

5. How we optimize for time
Capture
performance
• Profiler creates
application profile
• Allinea MAP
records multiple
processes
Find
bottlenecks
• Source code
viewer pinpoints
key consumers
• Timelines find
unusual patterns
Optimize
• Rewrite key loops
• Reorganize
memory access
patterns
• Change algorithms

6. CPU Package and System Power
Whole System
Power Usage
CPU Package
Power Usage
We can also measure power when profiling

7. Coprocessor Metrics
• Coprocessors have power requirements
– NVIDIA CUDA GPU OR INTEL XEON PHI (KNC)
• Devices provide kernel access to power
– HIGH POWER CONSUMPTION WHEN ACTIVE
– LOW POWER CONSUMPTION WHEN IDLE

8. Two Key Questions
• Can developers optimize code for energy? YES
• Can owners and users tune applications for
energy?

9. How we tune time without changing code
No instrumentation needed
No source code needed
No recompilation needed
Less than 5% runtime overhead
Fully scalable
Explicit and usable output

10. We can include energy usage too

11. Key Observation: In a Nutshell
• For many HPC workloads
– THE FASTER AN APPLICATION COMPLETES, THE LOWER ITS
ENERGY CONSUMPTION
– IE. OPTIMIZE FOR SPEED AND YOU ARE PROBABLY ALREADY
OPTIMIZING FOR ENERGY
• But for some HPC and non-HPC cases
– FREQUENCY SCALING SAVES ENERGY

12. Two Key Questions
• Can developers optimize code for energy? YES
• Can owners and users tune applications for
energy? YES
…. But should they?
• Are we counting all energy?
• Are we considering all costs?
.. Let’s rewrite the key question!

13. Two Key Questions
• When should developers optimize code for
energy?
• When should owners and users tune applications
for energy?

14. What is
energy?
Measuring Energy
• Real data from some processor,
memory subsystems
• Available in kernel - Intel RAPL
Low level
measurement
• PDU and server level readings
• Real data – real energy
Server level
monitoring
• Air-con
• Servers, switches, storage….
Full system
monitoring

15. Frequency Scaling
Some
workloads have
low compute
requirement, but
high data
volume
Data crunching vs number
crunching
Processor is
over-powered
for the speed of
memory, disk or
network
CPU frequency can be scaled
down in software
Providing
information to
developer, user
and system
owner
Allinea MAP
Allinea Performance Reports

16. A lot of codes are memory-bound

17. Multiple cores share bandwidth
Core 1
Core 2
Core 3
Core 4
…
Lots of
clever
technology
Main memory

18. Can we tune them for energy efficiency?
Core 1
Core 2
Core 3
Core 4
…
Lots of
clever
technology
Main
memory

19. How can users improve energy efficiency?
Buy a new cluster
Optimize the code
Reduce CPU frequency?
Run on fewer cores per node?

20. The Experiment
One
simple
code
A well-understood
wave equation
solver
One
compute
node
Minimize effect of
MPI
communications
Change
CPU
frequency
and
#cores
Measure the results
with Allinea
Performance
Reports

21. 4 PPN @ 2.1 Ghz, 30 seconds

22. Observed Results
Lower CPU frequency
• 1.7GHz – 6% less energy use
• Same runtime
Fewer cores/more nodes
• 20% increased runtime
• … but according to RAPL 15% energy saving
• Nonsense!! – 2x servers baseline power!
Important take-aways:
• Measure whole system energy
• Control power per application

23. Improving energy efficiency
• Tools show if the application wastes power
• Developers can see when to optimize and change
code
• Users can improve efficiency without changing code
Each application
and system has
different
characteristics
• Slowing down applications costs science and results
• Machines and PhDs have finite lifetime – their cost
dominates
Don’t forget the
opportunity cost
• Optimize for time before optimizing for energy
Time and energy
are not the same

24. Allinea’s Energy Products
• Allinea MAP
– PROFILER - PART OF ALLINEA FORGE DEVELOPMENT SUITE
– ADD-ON FOR SYSTEM AND PACKAGE ENERGY/POWER
• Allinea Performance Reports
– 1-PAGE BENCHMARKING/ANALYSIS
– ADD-ON FOR SYSTEM AND CPU ENERGY/POWER
• Available now
– RAPL SUPPORT FOR CPU ENERGY/POWER
– WHOLE NODE POWER MEASUREMENT FOR CRAY AND SOME
OTHER SYSTEMS
• Visit: http://www.allinea.com/energy-pack