What's New in Moab 8.0
•Download as PPTX, PDF•
1 like•1,679 views
In this video from ISC'14, Trev Harmon from Adaptive Computing presents: HPC, Cloud & Big Workflow: What's New in Moab 8.0. Learn more: http://adaptivecomputing.com Watch the video presentation: http://wp.me/p3RLHQ-cvN
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to What's New in Moab 8.0
Similar to What's New in Moab 8.0 (20)
More from inside-BigData.com
More from inside-BigData.com (20)
Recently uploaded
Recently uploaded (20)
What's New in Moab 8.0
- 1. © 2014 ADAPTIVE COMPUTING, INC. HPC, Cloud & Big Workflow: What’s New in Moab 8.0 Trev Harmon Adaptive Computing ISC'14
- 2. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Adaptive Computing Highlights ▪ Innovating world-class HPC solutions for over 12 years ▪ Pioneers of HPC schedulers, grid, power management, HPC-Cloud, optimization, scale, dynamic provisioning, Big Workflow and more ▪ 50+ patents issued or pending ▪ Backed by top-tier investors ▪ Many customers in the Top 100 and Fortune 500 ▪ Top systems including: #2 Titan, Cascade, Cielo, Hopper, & Bluewaters ▪ Major multi-nationals including: DOW, Exxon, & Boeing ▪ Largest provider of HPC workload management software to HPC sites* ▪ Global partnerships include Intel, HP, IBM, Cray, SGI, & Microsoft Cloud System Management Innovator
- 3. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Broad HPC Customer Base Oil and Gas, Financial, Manufacturing, Research and Government
- 4. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Expanding HPC Value & Use Greater access to technical computing resources Expanding HPC applicability across industries Demand for more simplified access & management (e.g. SLAs)
- 5. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Growth in System Size & Complexity 100 cores --> 1 Million+ cores Diversity of environments and processing needs Growing organizational complexity
- 6. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Greater Need for Alignment with Organization / Business Directives Increased tracking & accountability Increasing global competition Ability to quickly adapt is vital Growing Collaboration
- 7. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Themes for Accelerating Insights ▪ Unify data center resources ▪ As a single, adaptive ecosystem ▪ Technical computing (HPC & Big Data) ▪ Public and private cloud ▪ Bare metal & virtual machines ▪ Optimize the analysis process ▪ Increase throughput and productivity ▪ Ensure SLAs, maximize uptime ▪ Reduce cost, complexity and errors ▪ Guarantee service to the business ▪ Policies that model your organization ▪ Prove services were delivered ▪ Job completion in spite of failures ▪ Verify resources were allocated fairly
- 8. © 2014 ADAPTIVE COMPUTING, INC. What’s New in 8.0 Enhancing Big Workflow
- 9. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. What’s new in 8.0 - Unify ▪ OpenStack ▪ Breaks down siloed environments ▪ Offers virtual and physical resource provisioning for IaaS and PaaS ▪ Select Beta Customers ▪ Intersect360 ▪ Moab and TORQUE - top two job management packages ▪ Received 40% of the mentions “Adaptive’s Big Workflow…is to provide a way for big data, HPC, and cloud environments to interoperate, and do so dynamically based on what applications are running. With the added benefits of a unified platform, OpenStack is a promising platform to interoperate multiple environments.” -Addison Snell, CEO Intersect360
- 10. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. What’s New in 8.0 – Optimize ▪ Moab Performance Boost ▪ 2-3x overall performance improvements ▪ 100K Job Submission ▪ High Throughput Computing with Nitro ▪ Advanced Data Staging ▪ Multi-job workflow ▪ Staging job runtime prediction ▪ Improved cluster utilization ▪ Multiple transfer methods ▪ Advanced Power Management ▪ New power states options ▪ Suspend ▪ Hybernate ▪ Shutdown ▪ Clock Frequency Control N O D E S Input Output Compute
- 11. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. What’s New in 8.0 – Guarantee ▪ Next Generation Viewpoint ▪ Enhanced Web-based UI ▪ Next Generation dashboard ▪ Today monitors and reports workload and resource utilization ▪ Cray 3D Torus topology awareness
- 12. © 2014 ADAPTIVE COMPUTING, INC. Ascent Project
- 13. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Performance Boost with Operation Ascent ▪ 3x the Performance Boost ▪ Reduce Command Latency ▪ Decrease Scheduling Cycle Time ▪ Improve Multi-Threading ▪ Faster Moab/TORQUE Communication ▪ Advanced High-throughput Computing with Nitro
- 14. Progress Example
- 15. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Nitro – High Throughput Computing ▪ Removes launch speed bottlenecks ▪ Achieves exascale computing ▪ Localizes decision making ▪ Up to 100x faster throughput on short jobs ▪ Launches 10 jobs per node per second ▪ Reduces latency ▪ Runs on Moab/TORQUE environments
- 16. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. How Does Nitro Work? ▪ Ultra high-speed message queue ▪ Different approach to scheduling ▪ Combines small, alike jobs ▪ Creates policies for the entire batch job ▪ Schedules the batch as one job ▪ Incur scheduling overhead only once ▪ Not once per individual small job ▪ Limitations ▪ Speed of your processor & job size ▪ Nitro sacrifices some granularity in management ▪ i.e. individual tasks in a large batch cannot be cancelled or pre-empted in isolation ▪ The batch is the unit of management and reporting
- 17. © 2014 ADAPTIVE COMPUTING, INC. Topology-aware Node Allocation
- 18. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Topology-aware Node Allocation ▪ Cray Gemini 3D Torus ▪ Network characteristics-aware ▪ 3D torus ▪ Y-dimension bandwidth ▪ Dateline zones ▪ Shape-fitting ▪ Six shapes ▪ Built-in Moab node allocation policy
- 19. © 2014 ADAPTIVE COMPUTING, INC. Data-staging
- 20. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Data-staging Refactor ▪ Data-staging using Moab “system” jobs ▪ Input and output data-staging system jobs ▪ System jobs separately scheduled by Moab ▪ Dependencies between system jobs and user job ▪ Calculate system jobs’ data-staging wall time estimates ▪ Support additional file transfer utilities ▪ Linux rsync in addition to scp utility ▪ Commercial data-transfer products (e.g. Aspera)
- 21. © 2014 ADAPTIVE COMPUTING, INC. ▪ Node Allocation Timing Exception ▪ If data staged to local file system, compute nodes allocated during data-staging system jobs ▪ Why? Preserve job execution time consistency! ▪ Grid data-staging ▪ Part of data-staging initiative ▪ Grid Moab chooses cluster ▪ Grid Moab stages data ▪ Can run data-staging system jobs on dedicated data transfer servers Data-staging Refactor
- 22. © 2014 ADAPTIVE COMPUTING, INC. Power Management / Green Computing
- 23. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Power/Performance Profiles ▪ Minimizing energy consumption requires application- specific optimal clock frequency
- 24. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. CPU Clock Frequency Control ▪ New cpuclock= job submission option ▪ Absolute Clock Frequency Number ▪ Example: cpuclock=2200 or cpuclock=1800mhz ▪ Linux Power Governor Policy ▪ Example: cpuclock=conservative ▪ Relative P-state Number ▪ Values 0-15 ▪ 0=“turbo” frequency ▪ 15=slowest frequency ▪ Example: cpuclock=0 or cpuclock=P2 ▪ Can set in job templates
- 25. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. CPU Clock Frequency Control (continued) ▪ TORQUE pbs_mom sets clock frequency ▪ Logs clock frequency changes in pbs_mom log ▪ Moab records ▪ Job’s requested clock frequency in job record ▪ Nodes’ clock frequency in node statistics ▪ Uses ▪ Energy conservation for lower operational costs ▪ Power/performance profile generation ▪ Diagnostics
- 26. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Green Policy Configuration ▪ New Moab Web Services RM Plug-in ▪ Contains power management logic ▪ Specifies power state Moab should place a compute node in when applying “green” policy ▪ Standby ▪ Suspend ▪ Hibernate ▪ Shutdown ▪ Off ▪ Multi-threaded ▪ New power management “reference” scripts
- 27. © 2014 ADAPTIVE COMPUTING, INC. Administrator Portal
- 28. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Admin Portal (8.0 - 2014) ▪ Exciting Features ▪ Dashboards: Workload and Resource Views ▪ Simplified management of credentials ▪ Easy to use Policies ▪ Priority ▪ Fairshare ▪ Backfill ▪ Node Policies ▪ Cluster Management ▪ Historical Database ▪ HPC Web Services
- 29. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Persistent Database ▪ Relational database for historical data! ▪ Published View Schema ▪ Easily extract reports using standard reporting tools / frameworks ▪ Prepopulated Views API
- 30. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Dashboard
- 31. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Credential Management - Details
- 32. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Graphical Policy Management
- 33. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Resource Management – Zoom levels
- 34. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Resource Management – Zoom levels
- 35. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Resource Management – Historic Utilization
- 36. © 2014 ADAPTIVE COMPUTING, INC.© 2014 ADAPTIVE COMPUTING, INC. Submitting a job…
- 37. © 2014 ADAPTIVE COMPUTING, INC. Questions?
Editor's Notes
- Other top 100 notables, Kracken. Jaguar, Cielo, Beeken, GAEA, HPCC, Red Sky, Big Red Long history of running the most powerful systems in the world. See our website for links to our awards. The maker of the world’s most advanced workload and resource management system Adaptive Computing has proven leadership in accelerating IT to speed and improve business based on our 10 + years in delivering workload management and IT decision engine software. This leadership has been recognized in many ways with: 50+patents filed or approved 51% revenue growth in 2010 Acceleration centric investments in 2010 from leaders like Intel Capital, Epic ventures and Tudor Ventures who recognize our HPC market strength and cloud solution leadership and opportunity Global partnerships with organizations like HP, IBM, Cray, Microsoft and many others who utilize our innovative and leading decision engine to make their solutions more competitive and innovative for their customers And most importantly by customers who us to accelerate and manage their IT environments, the most dynamic and scale-intensive on the planet, a few of which are called out here, with heterogeneous, extreme-scale, multiple data centers with complex workloads, resources and priorities and decisions that we help accelerate in self-optimizing environment. In fact our software is used on well over $2 billion of hardware
- Moab needs to improve its data-staging speed and reliability using several means for both grid and normal cluster operations. Moab will stage input and output data using “system jobs”. System jobs execute on the Moab head node, are tracked separately from user jobs, and <click> can have their own walltime limit which Moab would dynamically calculate using the same information used in grid job scheduling. T <click> This allows Moab to stage data with dependencies between the user job and the system jobs. <click> Moab needs to support other file transfer utilities for both grid and cluster operations; e.g., open-source products such as “rsync” and “gsiscp” and commercially available products, such as Aspera, that have additional security, reliability, and possible parallel operation.
- Different applications often require different clock frequencies in order to achieve maximum power savings, as evidenced by the dashed green line in the two graphs. The application on the left will consume the least power at the 1400 MHz clock frequency while the application on the right consumes the least power at the 1600 MHz frequency. Administrators or users must benchmark applications usage at different clock frequencies to obtain the runtimes and power consumption measurements for all frequencies in order to determine the optimal frequency for consuming the least amount of power. Of course, the benefits of power savings - lower operational costs - must be balanced against the benefits of faster execution times - better ROI on capital costs - which really means balancing both benefits to obtain the minimum total cost of ownership (TCO). For example, looking at the right graph the power consumption at 1.6 GHz is ~16950 joules and at 1.8 GHz is ~17125, which is about 1% more power, but the job execution time drops from ~73 seconds to ~66 seconds, which is 7 seconds or about 10% less time. A user may choose to run the application at 1.8 GHz because it can run in less time at very little additional cost. To illustrate, the user can run the application 11 times at 1.8 GHz in the time it takes to run the application 10 times at 1.6 GHz, with a power cost of 11.1 for 1.8 GHz compared to a power cost of 11.0 running the application 11 times at 1.6 GHz, which the user may consider a great deal!
- Kilby’s new CPU clock job submission option indicates the user’s desire to force the compute nodes allocated to the job to run with the specified clock frequency or power governor policy. This new option can be specified on both Moab’s msub and TORQUE’s qsub job submission commands. In addition, an administrator can assign the CPU clock job submission option to a Moab job template, which means all job submissions matching the job template will acquire inherit its CPU clock option value. When a CPU clock job submission option sets the clock frequency of the job’s allocated compute nodes and the job executes, then after the job completes TORQUE sets the clock frequency back to the value the node had before the job executed on the node. <click 1> The cpuclock= option can specify the absolute clock frequency, a Linux power governor policy, or a relative P-state or “performance-state” value. <click 2> An absolute clock frequency is specified in units of megahertz and must have a value of 16 or higher. For example, a value of 2200 means the processor(s) on the job’s compute nodes will run at 2200 MHz or 2.2 GHz. If a processor does not support the exact clock frequency value, TORQUE will select the available clock frequency closest to the specified value. <click 3> A Linux power governor policy is specified using one of the keywords identified in the slide. All compute nodes run the job with the specified power governor policy. <click 4> A relative P-state number is a value in the range 0 to 15. The value 0 means to run a processor at its “turbo” frequency. The values 1-15 refer to the available fastest non-turbo clock frequency to the available slowest non-turbo clock frequency, respectively. Processors may not support all 15 possible P-states, which means any P-state value larger than the last supported P-state defaults to the last supported P-state, which is the lowest and slowest clock frequency the processor supports.
- Moab’s “green” policy management understands only two so-called “power” states that are really scheduling availability states. “On” means a compute node is available for scheduling “Off” means a compute node is not available for scheduling A new “Power Idle Node Action” parameter permits the Moab administrator to specify what the “off” or “not-schedulable” power state should actually be when Moab applies green policy to idle nodes in order to reduce energy consumption. <click 1> Allowed values are standby, suspend, sleep, hibernate, shutdown, and off (currently the only value available). They are listed in least-power-savings to most-power-savings and least-time-to-awaken to most-time-to-awaken order. Note the green power management feature saves energy by placing a compute node in a non-active state, meaning it is not executing software. <click 2> As part of Moab handling additional low-power/no-power states, the Kilby release will include new power management “reference” scripts.
- Better view into your system utilization and what Moab is doing
- He figures his priority formula is not giving the user enough priority so he adjusts it in the priority policy “sound board” interface… With it, he can quickly understand if changing one or more weights will influence his user’s job priority. Notes for Engineering: The mixing board sliders are logarithmic When a component’s weight is set to 0, the whole control is supposed to be grayed out BUT the slider position for the sub components must be maintained (although their weight changes to 0) The search bar on top needs to go away in the policy management page Notice how the policy tabs are actually on the top – this is the design we must follow!