Zettar: Moving Massive Amounts of Data across Any Distance Efficiently

Building the SD-Branch using uCPE

Application developers are key to the success of an edge compute strategy. They are the backbone for any digital ecosystem and their requirements drive the platform architecture. Edge computing is no different. In this talk, we will focus on some key requirements, challenges and possible solutions for a developer centric architecture for multi-access edge computing including abstraction of the service provider’s network complexity, low footprint cloud native builder models, micro-services, hardware abstractions, intelligence layers and massive monitoring of application instances. About the speaker: Shamik Mishra is currently Assistant Vice President (AVP), Technology and Innovation at Aricent. He is a practice leader for new product architectures. He has extensive experience and contributions in software development in cloud, wireless technologies, edge computing and platform software. His research interests are Network Function Virtualization (NFV), Cloud and edge computing and Machine Learning (ML). He has spoken in several conferences and his work is regularly covered in the media. Shamik has a bachelor’s and a master’s degree from Indian Institute of Technology (IIT) Kharagpur, India.

Accelerating Edge Computing Adoption

The Open Network Automation Platform (ONAP) is a leading Linux Foundation Networking open source project that provides fully automated orchestration and lifecycle management of NFV, SDN, analytics and edge computing services. While ONAP can be used for any network service, it is particularly beneficial for 5G and edge computing use cases. In this talk you will learn: * What is ONAP * What use cases does ONAP support * What are the 5G/edge computing workload automation requirements * How does ONAP support these requirements * How can you get involved

Open Source 5G/Edge Automation via ONAP

Machine learning for optical networking: hype, reality and use cases

Artificial Intelligence in the Network

In this video from the UK HPC Conference, Richard Graham from Mellanox presents: InfiniBand In-Network Computing Technology and Roadmap. "In-Network Computing transforms the data center interconnect to become a "distributed CPU", and "distributed memory", enables to overcome performance barriers and to enable faster and more scalable data analysis. HDR 200G InfiniBand In-Network Computing technology includes several elements - Scalable Hierarchical Aggregation and Reduction Protocol (SHARP), smart Tag Matching and rendezvoused protocol, and more. These technologies are in use at some of the recent large scale supercomputers around the world, including the top TOP500 platforms. The session will discuss the InfiniBand In-Network Computing technology and performance results, as well as view to future roadmap." Watch the video: Learn more: http://mellanox.com and http://hpcadvisorycouncil.com/events/2019/uk-conference/agenda.php Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

InfiniBand In-Network Computing Technology and Roadmap

Networks need to incorporate innovative and high-performance packet processing entities to meet the demands of meteoric rise in data coupled with advances in compute capacity and innovative apps. A fully programmable forwarding plane enables network owners to build the network they want and evolve it as the needs change. P4 is a domain specific language for networking and it empowers network builders to craft the functionality they need in a high-level programming language and execute it at line-rate on a variety of devices including the Barefoot Tofino series of Ethernet switches. This talk will give an overview of P4 and go over a couple of use-cases.

Open Source Possibilities for 5G Edge Computing Deployment

Ignacio Verona

P4/FPGA, Packet Acceleration

Mobile edge computing delivers cloud computing at the edge of the cellular network to drive services quality and innovation. The ability for CSPs and ISVs to effectively develop, deliver, and deploy MEC services on a given platform directly correlates with the availability and maturity of associated tools and test environment. Dronava is a hyper-connected, web-scale network reference design for the 5G mobile network, suitable for use as a test and development socket for cloud applications developed for MEC platforms with tools such as the Intel NEV SDK. With Dronava, developers can drive the application with real traffics from the network edge to the EPC core, and if need be, connect with services in the core network in order to fully characterize the functionalities, latency, and throughput of the platform and application.Teranium is an integrated development environment that simplifies the development, packaging, and deployment/management of cloud applications. Teranium can be utilized to develop and deploy MEC applications on a number of platforms. Together with Dronava, Teranium helps to reduce complexity and improve efficiency in the ability of CSPs and ISVs to adopt and deploy MEC-base services.

Development, test, and characterization of MEC platforms with Teranium and Dr...

Transforming optical networking with AI

The Akraino community was proud to announce the availability of its release 1 on June 6th. The community has experienced extremely rapid growth over the past year, in terms of both membership and community activity. Before Akraino, developers had to download multiple open source software packages and integrate/test on deployable hardware, which prolonged innovation and increased cost. The Akraino community came up with a brilliant way to solve this integration challenge with the Blueprint model. An Akraino Blueprint is not just a diagram; it’s real code that brings everything together so users can download and deploy the edge stack in their own environment to address a specific edge use case. Learn more about the Akraino Edge Stack. In this talk, we will share details about R1 blueprints and their use, R2 goals, and how to engage and contribute to the Akraino Community.

Your Path to Edge Computing - Akraino Edge Stack Update

In this deck, Paul Isaacs from Linaro presents: State of ARM-based HPC. This talk provides an overview of applications and infrastructure services successfully ported to Aarch64 and benefiting from scale. "With its debut on the TOP500, the 125,000-core Astra supercomputer at New Mexico’s Sandia Labs uses Cavium ThunderX2 chips to mark Arm’s entry into the petascale world. In Japan, the Fujitsu A64FX Arm-based CPU in the pending Fugaku supercomputer has been optimized to achieve high-level, real-world application performance, anticipating up to one hundred times the application execution performance of the K computer. K was the first computer to top 10 petaflops in 2011." Watch the video: https://wp.me/p3RLHQ-lIT Learn more: https://www.linaro.org/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

State of ARM-based HPC

Enabling Multi-access Edge Computing (MEC) Platform-as-a-Service for Enterprises

Ever growing need of Intelligent Systems evolves analytics and decision making into AI with Machine Learning as tools for knowledge assimilation. What is essential for ML is a form of data that has inherent information that can be translated to useful information (intelligence) for decision making. IoT is the key for intelligent systems as they collect data at every end point. They are like ends of neuron network in human body. And the data collected has to be refined for decision making as it traverses up to the brain (AI Cloud) – like lymph nodes we have Edge Clouds. We will explore in this short talk two aspects of such IoT infrastructure where you have lossy network for IoTs, gateway options for device data and how it can seamlessly integrate with Edge Cloud Networks. We will review such protocols as Wireless Mesh, programmable gateways and extension of overlays into the Cloud. Speaker: Murali Rangachari, Futurewei Technologies

Convergence of device and data at the Edge Cloud

In October 2012, a group of 13 global network operators published an influential white paper describing their vision for the future of telecommunications networks based on technology which they called “Network Functions Virtualisation” or simply “NFV”. Simultaneously they launched the ETSI NFV Industry Specification Group which is the central body converging requirements for NFV and developing specifications which enable standards and open source communities to use NFV in a common framework. This latest NFV white paper co-authored by 23 network operators - including members of the original founding group, outlines what they see as the priorities for NFV to support 5G. This session will overview this paper and describe the rationale for the main proposals. This presentation is from Open Networking Summit 2017.

SDN Service Provider Use Cases

SDxCentral

Enabling 5G with NFV: New Joint White Paper Outlining Network Operator Perspe...

Open Networking Summit

Mellnox Interconnect presentation in OpenPOWER Brazil workshop

Ganesan Narayanasamy

Proven and Emerging Use Cases of Software Defined Network

Open Networking Summits

SEBA is both a Reference Design and an exemplar implementation based on the reference design. This talk will mainly focus on the Exemplar implementation developed by ONF, AT&T's Atlanta Foundry and the SEBA and VOLTHA community with origins in R-CORD and composed of VOLTHA, ONOS apps etc. We will tall about how they all fit together in a modular way and there will be a quick demo to show the current and futures developments in SEBS.

SEBA: SDN Enabled Broadband Access - Transporting SDN principles to PON Networks

The Internet of Things arrived last decade when the number of devices (that can connect) outnumbered the world population. We have now entered a new age. The evolution from #virtualization to #cloud to #IoT and #BigData a consequence of the Moore Nielsen prediction and the rise of Fog Computing. The role of #OpenSource and #OpenStandards and the importance of the new trend: Open Data as the only way to keep sanity in Big Data. This is my presentation at the IEEE International Conference on Cloud Engineering in Boston on Pi Day 2014

Tendances (20)

Operationalizing SDN

Building the SD-Branch using uCPE

Accelerating Edge Computing Adoption

Open Source 5G/Edge Automation via ONAP

Machine learning for optical networking: hype, reality and use cases

Artificial Intelligence in the Network

InfiniBand In-Network Computing Technology and Roadmap

Open Source Possibilities for 5G Edge Computing Deployment

P4/FPGA, Packet Acceleration

Development, test, and characterization of MEC platforms with Teranium and Dr...

Transforming optical networking with AI

Your Path to Edge Computing - Akraino Edge Stack Update

State of ARM-based HPC

Enabling Multi-access Edge Computing (MEC) Platform-as-a-Service for Enterprises

Convergence of device and data at the Edge Cloud

SDN Service Provider Use Cases

Enabling 5G with NFV: New Joint White Paper Outlining Network Operator Perspe...

Mellnox Interconnect presentation in OpenPOWER Brazil workshop

Proven and Emerging Use Cases of Software Defined Network

SEBA: SDN Enabled Broadband Access - Transporting SDN principles to PON Networks

Similaire à Zettar: Moving Massive Amounts of Data across Any Distance Efficiently

Roberto minerva 20181130

Roberto Minerva

Its detroit2018 ndot

robhemmerich

Cloud beyond the obvious, an approach for innovation

Christian Verstraete

Michael Enescu - Cloud + IoT at IEEE

Michael Enescu

Devcon2上海参加報告

Hiroyasu NOHATA

Model-driven Telemetry: The Foundation of Big Data Analytics

Cisco Canada

雲端運算其實就是一種分散式運算，跟許多人熟知的網格運算（Grid Computing）非常相似，都是科技自然演化下為了突破當時運算架構限制的創新技術，兩者應用層面不同，並不會有相互排斥的問題。中研院網格計畫主持人林誠謙表示：「網格運算著重在硬體平台之間的相互合作，以創造更快、更高的運算能力，而雲端計算則是讓個人電腦與行動裝置能透過網路，隨時隨地都能存取應用伺服器所提供的服務。」在網格運算投入非常多心力的IBM，近年來不但轉型成為資訊服務提供商，也非常積極投入雲端運算的領域，並且在2008年與IDA在愛爾蘭共同成立歐洲第一個雲端運算中心。

Cloud Computing,雲端運算-中研院網格計畫主持人林誠謙

Tracy Chen

Why SD-WAN as it Moves to Mainstream Adoption?

ColloqueRISQ

Currently, we face new challenges in realtime analytics of BigData, such as social monitoring, M2M sensor, online advertising optimization, smart energy management and security monitoring. To analyze these data, scalable machine learning technologies are essential. Jubatus is the open source platform for online distributed machine learning on the data streams of BigData. we explain the inside technologies of Jubatus and show how jubatus can achieve realtime analytics in various problems.

Jubatus: Realtime deep analytics for BIgData@Rakuten Technology Conference 2012

Preferred Networks

Network synchronization has come a long way, from native telecoms sync (TDM, SDH/SONET) through sync as a service (SyncE, PTP; 4G and 5G), to synchronization as an enabler of virtual services in open compute servers like TAP. In his WSTS talk, Ken Hann explored how network timing technology has evolved. The session covered methods for bringing synchronization to applications and explored how traditional telecoms synchronization distribution and open compute framework virtualization can be combined and managed.

Syncing the cloud - from T1 to TAP

Boulder Java User's Group

Big Data and OSS at IBM

Davis Chris

Carl Ford

Big data analytics for transport

UKinItaly

Clouds, Grids and Data

Guy Coates

Transforming to Enable 5G

Lumina Networks

Cisco Connect 2018 Thailand - Secured and visualized data center connectivity...

NetworkCollaborators

SNAIL Project for IoT Connectivity

Daeyoung Kim

IRJET- Cost Effective Scheme for Delay Tolerant Data Transmission

IRJET Journal

Big data caching for networking

Hicham HAMMOUCHI

Big data caching for networking : Moving from cloud to edge

Hicham HAMMOUCHI

Similaire à Zettar: Moving Massive Amounts of Data across Any Distance Efficiently (20)

Roberto minerva 20181130

Its detroit2018 ndot

Cloud beyond the obvious, an approach for innovation

Michael Enescu - Cloud + IoT at IEEE

Devcon2上海参加報告

Model-driven Telemetry: The Foundation of Big Data Analytics

Cloud Computing,雲端運算-中研院網格計畫主持人林誠謙

Why SD-WAN as it Moves to Mainstream Adoption?

Jubatus: Realtime deep analytics for BIgData@Rakuten Technology Conference 2012

Syncing the cloud - from T1 to TAP

Big Data and OSS at IBM

Davis Chris

Big data analytics for transport

Clouds, Grids and Data

Transforming to Enable 5G

Cisco Connect 2018 Thailand - Secured and visualized data center connectivity...

SNAIL Project for IoT Connectivity

IRJET- Cost Effective Scheme for Delay Tolerant Data Transmission

Big data caching for networking

Big data caching for networking : Moving from cloud to edge

Plus de inside-BigData.com

In this deck from IWOCL / SYCLcon 2020, Hal Finkel from Argonne National Laboratory presents: Preparing to program Aurora at Exascale - Early experiences and future directions. "Argonne National Laboratory’s Leadership Computing Facility will be home to Aurora, our first exascale supercomputer. Aurora promises to take scientific computing to a whole new level, and scientists and engineers from many different fields will take advantage of Aurora’s unprecedented computational capabilities to push the boundaries of human knowledge. In addition, Aurora’s support for advanced machine-learning and big-data computations will enable scientific workflows incorporating these techniques along with traditional HPC algorithms. Programming the state-of-the-art hardware in Aurora will be accomplished using state-of-the-art programming models. Some of these models, such as OpenMP, are long-established in the HPC ecosystem. Other models, such as Intel’s oneAPI, based on SYCL, are relatively-new models constructed with the benefit of significant experience. Many applications will not use these models directly, but rather, will use C++ abstraction libraries such as Kokkos or RAJA. Python will also be a common entry point to high-performance capabilities. As we look toward the future, features in the C++ standard itself will become increasingly relevant for accessing the extreme parallelism of exascale platforms. This presentation will summarize the experiences of our team as we prepare for Aurora, exploring how to port applications to Aurora’s architecture and programming models, and distilling the challenges and best practices we’ve developed to date. oneAPI/SYCL and OpenMP are both critical models in these efforts, and while the ecosystem for Aurora has yet to mature, we’ve already had a great deal of success. Importantly, we are not passive recipients of programming models developed by others. Our team works not only with vendor-provided compilers and tools, but also develops improved open-source LLVM-based technologies that feed both open-source and vendor-provided capabilities. In addition, we actively participate in the standardization of OpenMP, SYCL, and C++. To conclude, I’ll share our thoughts on how these models can best develop in the future to support exascale-class systems." Watch the video: https://wp.me/p3RLHQ-lPT Learn more: https://www.iwocl.org/iwocl-2020/conference-program/ and https://www.anl.gov/topic/aurora Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Preparing to program Aurora at Exascale - Early experiences and future direct...

In this deck from the Stanford HPC Conference, Katie Lewis from Lawrence Livermore National Laboratory presents: The Incorporation of Machine Learning into Scientific Simulations at Lawrence Livermore National Laboratory. "Scientific simulations have driven computing at Lawrence Livermore National Laboratory (LLNL) for decades. During that time, we have seen significant changes in hardware, tools, and algorithms. Today, data science, including machine learning, is one of the fastest growing areas of computing, and LLNL is investing in hardware, applications, and algorithms in this space. While the use of simulations to focus and understand experiments is well accepted in our community, machine learning brings new challenges that need to be addressed. I will explore applications for machine learning in scientific simulations that are showing promising results and further investigation that is needed to better understand its usefulness." Watch the video: https://youtu.be/NVwmvCWpZ6Y Learn more: https://computing.llnl.gov/research-area/machine-learning and http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

The Incorporation of Machine Learning into Scientific Simulations at Lawrence...

In this deck from the Stanford HPC Conference, DK Panda from Ohio State University presents: How to Achieve High-Performance, Scalable and Distributed DNN Training on Modern HPC Systems? "This talk will start with an overview of challenges being faced by the AI community to achieve high-performance, scalable and distributed DNN training on Modern HPC systems with both scale-up and scale-out strategies. After that, the talk will focus on a range of solutions being carried out in my group to address these challenges. The solutions will include: 1) MPI-driven Deep Learning, 2) Co-designing Deep Learning Stacks with High-Performance MPI, 3) Out-of- core DNN training, and 4) Hybrid (Data and Model) parallelism. Case studies to accelerate DNN training with popular frameworks like TensorFlow, PyTorch, MXNet and Caffe on modern HPC systems will be presented." Watch the video: https://youtu.be/LeUNoKZVuwQ Learn more: http://web.cse.ohio-state.edu/~panda.2/ and http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...

In this deck from the Stanford HPC Conference, Nicole Xu from Stanford University describes how she transformed a common jellyfish into a bionic creature that is part animal and part machine. "Animal locomotion and bioinspiration have the potential to expand the performance capabilities of robots, but current implementations are limited. Mechanical soft robots leverage engineered materials and are highly controllable, but these biomimetic robots consume more power than corresponding animal counterparts. Biological soft robots from a bottom-up approach offer advantages such as speed and controllability but are limited to survival in cell media. Instead, biohybrid robots that comprise live animals and self- contained microelectronic systems leverage the animals’ own metabolism to reduce power constraints and body as an natural scaffold with damage tolerance. We demonstrate that by integrating onboard microelectronics into live jellyfish, we can enhance propulsion up to threefold, using only 10 mW of external power input to the microelectronics and at only a twofold increase in cost of transport to the animal. This robotic system uses 10 to 1000 times less external power per mass than existing swimming robots in literature and can be used in future applications for ocean monitoring to track environmental changes." Watch the video: https://youtu.be/HrmJFyvInj8 Learn more: https://sanfrancisco.cbslocal.com/2020/02/05/stanford-research-project-common-jellyfish-bionic-sea-creatures/ and http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring

In this deck from the Stanford HPC Conference, Peter Dueben from the European Centre for Medium-Range Weather Forecasts (ECMWF) presents: Machine Learning for Weather Forecasts. "I will present recent studies that use deep learning to learn the equations of motion of the atmosphere, to emulate model components of weather forecast models and to enhance usability of weather forecasts. I will than talk about the main challenges for the application of deep learning in cutting-edge weather forecasts and suggest approaches to improve usability in the future." Peter is contributing to the development and optimization of weather and climate models for modern supercomputers. He is focusing on a better understanding of model error and model uncertainty, on the use of reduced numerical precision that is optimised for a given level of model error, on global cloud- resolving simulations with ECMWF's forecast model, and the use of machine learning, and in particular deep learning, to improve the workflow and predictions. Peter has graduated in Physics and wrote his PhD thesis at the Max Planck Institute for Meteorology in Germany. He worked as Postdoc with Tim Palmer at the University of Oxford and has taken up a position as University Research Fellow of the Royal Society at the European Centre for Medium-Range Weather Forecasts (ECMWF) in 2017. Watch the video: https://youtu.be/ks3fkRj8Iqc Learn more: https://www.ecmwf.int/ and http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Machine Learning for Weather Forecasts

In this deck, Gilad Shainer from the HPC AI Advisory Council describes how this organization fosters innovation in the high performance computing community. "The HPC-AI Advisory Council’s mission is to bridge the gap between high-performance computing (HPC) and Artificial Intelligence (AI) use and its potential, bring the beneficial capabilities of HPC and AI to new users for better research, education, innovation and product manufacturing, bring users the expertise needed to operate HPC and AI systems, provide application designers with the tools needed to enable parallel computing, and to strengthen the qualification and integration of HPC and AI system products." Watch the video: https://wp.me/p3RLHQ-lNz Learn more: http://hpcadvisorycouncil.com Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

HPC AI Advisory Council Update

Today RIKEN in Japan announced that the Fugaku supercomputer will be made available for research projects aimed to combat COVID-19. "Fugaku is currently being installed and is scheduled to be available to the public in 2021. However, faced with the devastating disaster unfolding before our eyes, RIKEN and MEXT decided to make a portion of the computational resources of Fugaku available for COVID-19-related projects ahead of schedule while continuing the installation process. Fugaku is being developed not only for the progress in science, but also to help build the society dubbed as the “Society 5.0” by the Japanese government, where all people will live safe and comfortable lives. The current initiative to fight against the novel coronavirus is driven by the philosophy behind the development of Fugaku." Initial Projects Exploring new drug candidates for COVID-19 by "Fugaku" Yasushi Okuno, RIKEN / Kyoto University Prediction of conformational dynamics of proteins on the surface of SARS-Cov-2 using Fugaku Yuji Sugita, RIKEN Simulation analysis of pandemic phenomena Nobuyasu Ito, RIKEN Fragment molecular orbital calculations for COVID-19 proteins Yuji Mochizuki, Rikkyo University

Fugaku Supercomputer joins fight against COVID-19

In this deck from the Performance Optimisation and Productivity group, Lubomir Riha from IT4Innovations presents: Energy Efficient Computing using Dynamic Tuning. "We now live in a world of power-constrained architectures and systems and power consumption represents a significant cost factor in the overall HPC system economy. For these reasons, in recent years researchers, supercomputing centers and major vendors have developed new tools and methodologies to measure and optimize the energy consumption of large-scale high performance system installations. Due to the link between energy consumption, power consumption and execution time of an application executed by the final user, it is important for these tools and the methodology used to consider all these aspects, empowering the final user and the system administrator with the capability of finding the best configuration given different high level objectives. This webinar focused on tools designed to improve the energy-efficiency of HPC applications using a methodology of dynamic tuning of HPC applications, developed under the H2020 READEX project. The READEX methodology has been designed for exploiting the dynamic behaviour of software. At design time, different runtime situations (RTS) are detected and optimized system configurations are determined. RTSs with the same configuration are grouped into scenarios, forming the tuning model. At runtime, the tuning model is used to switch system configurations dynamically. The MERIC tool, that implements the READEX methodology, is presented. It supports manual or binary instrumentation of the analysed applications to simplify the analysis. This instrumentation is used to identify and annotate the significant regions in the HPC application. Automatic binary instrumentation annotates regions with significant runtime. Manual instrumentation, which can be combined with automatic, allows code developer to annotate regions of particular interest." Watch the video: https://wp.me/p3RLHQ-lJP Learn more: https://pop-coe.eu/blog/14th-pop-webinar-energy-efficient-computing-using-dynamic-tuning and https://code.it4i.cz/vys0053/meric Sign up for our insideHPC Newsletter: http://insidehpc.com/newslett

Energy Efficient Computing using Dynamic Tuning

In this deck from the Rice Oil & Gas Conference, Bradley McCredie from AMD presents: Scaling TCO in a Post Moore's Law Era. "While foundries bravely drive forward to overcome the technical and economic challenges posed by scaling to 5nm and beyond, Moore’s law alone can provide only a fraction of the performance / watt and performance / dollar gains needed to satisfy the demands of today’s high performance computing and artificial intelligence applications. To close the gap, multiple strategies are required. First, new levels of innovation and design efficiency will supplement technology gains to continue to deliver meaningful improvements in SoC performance. Second, heterogenous compute architectures will create x-factor increases of performance efficiency for the most critical applications. Finally, open software frameworks, APIs, and toolsets will enable broad ecosystems of application level innovation." Watch the video: Learn more: http://amd.com and https://rice2020oghpc.rice.edu/program-2/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Scaling TCO in a Post Moore's Era

In this deck from the ECSS Symposium, Abe Stern from NVIDIA presents: CUDA-Python and RAPIDS for blazing fast scientific computing. "We will introduce Numba and RAPIDS for GPU programming in Python. Numba allows us to write just-in-time compiled CUDA code in Python, giving us easy access to the power of GPUs from a powerful high-level language. RAPIDS is a suite of tools with a Python interface for machine learning and dataframe operations. Together, Numba and RAPIDS represent a potent set of tools for rapid prototyping, development, and analysis for scientific computing. We will cover the basics of each library and go over simple examples to get users started. Finally, we will briefly highlight several other relevant libraries for GPU programming." Watch the video: https://wp.me/p3RLHQ-lvu Learn more: https://developer.nvidia.com/rapids and https://www.xsede.org/for-users/ecss/ecss-symposium Sign up for our insideHPC Newsletter: http://insidehp.com/newsletter

CUDA-Python and RAPIDS for blazing fast scientific computing

In this deck from FOSDEM 2020, Colin Sauze from Aberystwyth University describes the development of a RaspberryPi cluster for teaching an introduction to HPC. "The motivation for this was to overcome four key problems faced by new HPC users: * The availability of a real HPC system and the effect running training courses can have on the real system, conversely the availability of spare resources on the real system can cause problems for the training course. * A fear of using a large and expensive HPC system for the first time and worries that doing something wrong might damage the system. * That HPC systems are very abstract systems sitting in data centres that users never see, it is difficult for them to understand exactly what it is they are using. * That new users fail to understand resource limitations, in part because of the vast resources in modern HPC systems a lot of mistakes can be made before running out of resources. A more resource constrained system makes it easier to understand this. The talk will also discuss some of the technical challenges in deploying an HPC environment to a Raspberry Pi and attempts to keep that environment as close to a "real" HPC as possible. The issue to trying to automate the installation process will also be covered." Learn more: https://github.com/colinsauze/pi_cluster and https://fosdem.org/2020/schedule/events/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Introducing HPC with a Raspberry Pi Cluster

In this deck from ATPESC 2019, Ken Raffenetti from Argonne presents an overview of HPC interconnects. "The Argonne Training Program on Extreme-Scale Computing (ATPESC) provides intensive, two-week training on the key skills, approaches, and tools to design, implement, and execute computational science and engineering applications on current high-end computing systems and the leadership-class computing systems of the future." Watch the video: https://wp.me/p3RLHQ-luc Learn more: https://extremecomputingtraining.anl.gov/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Overview of HPC Interconnects

In this deck from FOSDEM 2020, Frank McQuillan from Pivotal presents: Efficient Model Selection for Deep Neural Networks on Massively Parallel Processing Databases. "In this session we will present an efficient way to train many deep learning model configurations at the same time with Greenplum, a free and open source massively parallel database based on PostgreSQL. The implementation involves distributing data to the workers that have GPUs available and hopping model state between those workers, without sacrificing reproducibility or accuracy. Then we apply optimization algorithms to generate and prune the set of model configurations to try. Deep neural networks are revolutionizing many machine learning applications, but hundreds of trials may be needed to generate a good model architecture and associated hyperparameters. This is the challenge of model selection. It is time consuming and expensive, especially if you are only training one model at a time. Massively parallel processing databases can have hundreds of workers, so can you use this parallel compute architecture to address the challenge of model selection for deep nets, in order to make it faster and cheaper? It’s possible! We will demonstrate results from this project using a version of Hyperband, which is a well known hyperparameter optimization algorithm, and the deep learning frameworks Keras and TensorFlow, all running on Greenplum database using Apache MADlib. Other topics will include architecture, scalability results and bright opportunities for the future." Watch the video: https://wp.me/p3RLHQ-lsQ Learn more: https://fosdem.org/2020/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...

In this deck, Huihuo Zheng from Argonne National Laboratory presents: Data Parallel Deep Learning. "The Argonne Training Program on Extreme-Scale Computing (ATPESC) provides intensive, two weeks of training on the key skills, approaches, and tools to design, implement, and execute computational science and engineering applications on current high-end computing systems and the leadership-class computing systems of the future." Watch the video: https://wp.me/p3RLHQ-lsl Learn more: https://extremecomputingtraining.anl.gov/archive/atpesc-2019/agenda-2019/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Data Parallel Deep Learning

In this deck from DOE CSGF 2019, Chelsea Harris from the University of Michigan presents: Making Supernovae with Jets. "Supernovae are the explosions of stars. One reason they are fundamentally important is they create and disperse elements heavier than carbon throughout the universe. Different stars explode in different ways, but the most common supernova type is from massive stars (greater than 10 times the mass of the sun) whose cores collapse to form a neutron star or black hole – "core collapse supernovae" or CC SNe. Even among massive stars, though, there are differences that can affect the outcome of core collapse. I am specifically interested in progenitors whose cores are rotating and magnetic (magnetorotational). Such cores may experience instabilities after collapse that launch a fast jet, which could rescue r-process elements formed near the proto-neutron star from destruction. The instabilities can also add power to the explosion and relieve tension between observations and theory. In addition to running simulations with existing FLASH code modules, I am developing a FLASH hydrodynamics module, SparkJoy, to perform these simulations at high order. These projects are part of a DOE INCITE project to explore progenitor effects on CC SNe and of the DOE SciDAC program "Towards Exascale Astrophysics of Mergers and Supernovae," a nationwide collaboration of supernova theorists unprecedented in its collaborative scale. Research like mine is made much easier at Michigan State University through the Department of Computational Mathematics, Science, and Engineering which, like the DOE CSGF, brings together members from different areas to share knowledge and strengthen each other's research." Watch the video: https://wp.me/p3RLHQ-lr0 Learn more: https://www.krellinst.org/csgf/conf/2019/video/charris Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Making Supernovae with Jets

In this deck from ATPESC 2019, Jack Dongarra from UT Knoxville presents: Adaptive Linear Solvers and Eigensolvers. "Success in large-scale scientific computations often depends on algorithm design. Even the fastest machine may prove to be inadequate if insufficient attention is paid to the way in which the computation is organized. We have used several problems from computational physics to illustrate the importance of good algorithms, and we offer some very general principles for designing algorithms. Two subthemes are, first, the strong connection between the algorithm and the architecture of the target machine; and second, the importance of non-numerical methods in scientific computations." Watch the video: https://wp.me/p3RLHQ-lq3 Learn more: https://extremecomputingtraining.anl.gov/archive/atpesc-2019/agenda-2019/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Adaptive Linear Solvers and Eigensolvers

In this deck from ATPESC 2019, Michela Taufer from UT Knoxville presents: Scientific Applications and Heterogeneous Architectures. "This talk discusses two emerging trends in computing (i.e., the convergence of data generation and analytics, and the emergence of edge computing) and how these trends can impact heterogeneous applications. Next-generation supercomputers, with their extremely heterogeneous resources and dramatically higher performance than current systems, will generate more data than we need or, even, can handle. At the same time, more and more data is generated at the “edge,” requiring computing and storage to move closer and closer to data sources. The coordination of data generation and analysis across the spectrum of heterogonous systems including supercomputers, cloud computing, and edge computing adds additional layers of heterogeneity to applications’ workflows. More importantly, the coordination can neither rely on manual, centralized approaches as it is predominately done today in HPC nor exclusively be delegated to be just a problem for commercial Clouds. This talk presents case studies of heterogenous applications in precision medicine and precision farming that expand scientist workflows beyond the supercomputing center and shed our reliance on large-scale simulations exclusively, for the sake of scientific discovery." Watch the video: https://wp.me/p3RLHQ-lq2 Learn more: https://extremecomputingtraining.anl.gov/archive/atpesc-2019/agenda-2019/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Scientific Applications and Heterogeneous Architectures

In this deck from ATPESC 2019, Yunong Shi from the University of Chicago presents: SW/HW co-design for near-term quantum computing. "The Argonne Training Program on Extreme-Scale Computing (ATPESC) provides intensive, two weeks of training on the key skills, approaches, and tools to design, implement, and execute computational science and engineering applications on current high-end computing systems and the leadership-class computing systems of the future." Watch the video: https://wp.me/p3RLHQ-lpv Learn more: https://extremecomputingtraining.anl.gov/archive/atpesc-2019/agenda-2019/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

SW/HW co-design for near-term quantum computing

In this deck from ATPESC 2019, James Moawad and Greg Nash from Intel present: FPGAs and Machine Learning. "Neural networks are inspired by biological systems, in particular the human brain. Through the combination of powerful computing resources and novel architectures for neurons, neural networks have achieved state-of-the-art results in many domains such as computer vision and machine translation. FPGAs are a natural choice for implementing neural networks as they can handle different algorithms in computing, logic, and memory resources in the same device. Faster performance comparing to competitive implementations as the user can hardcore operations into the hardware. Software developers can use the OpenCL device C level programming standard to target FPGAs as accelerators to standard CPUs without having to deal with hardware level design." Watch the video: https://wp.me/p3RLHQ-lnc Learn more: https://extremecomputingtraining.anl.gov/archive/atpesc-2019/agenda-2019/ and https://www.intel.com/content/www/us/en/products/programmable/fpga.html Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

FPGAs and Machine Learning

In this video from the MIT Deep Learning Series, Lex Fridman presents: Deep Learning State of the Art (2020). "This lecture is on the most recent research and developments in deep learning, and hopes for 2020. This is not intended to be a list of SOTA benchmark results, but rather a set of highlights of machine learning and AI innovations and progress in academia, industry, and society in general. This lecture is part of the MIT Deep Learning Lecture Series." Watch the video: https://wp.me/p3RLHQ-lng Learn more: https://deeplearning.mit.edu/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Deep Learning State of the Art (2020)