Das neue Storage ist geliefert und betriebsbereit, und das erste was der Storage-Admin macht, ist ein dd. Mehr oder weniger zufrieden schaut er dann auf den Durchsatz und freut sich, dass der Einkauf dieses Mal was ordentliches gekauft hat. Oder auch nicht. Der Vortrag erläutert, weshalb es so schwierig ist, aussagekräftige Storage-Benchmarks zu fahren und geht kurz auf verschiedene Benchmarking-Tools ein. Im zweiten Teil des Vortrags gehe ich etwas genauer auf die Internas des SPC-1-Benchmarks des Storage Performance Council ein, und zeige, wie sinnvoll oder weniger sinnvoll es ist, sich bei der Beschaffung auf vermeintlich objektive Performance-Messungen zu verlassen.
Wolfgang Stief ist seit Mitte der 1990er Jahre als Dipl.-Ing. in der IT-Branche tätig. Nach vielen Jahren in Support und Presales bei einem Sun-Partner startete er 2011 in die Selbständigkeit. Als Technologieberater und Erklärbär ist er freiberuflich tätig im technischen Marketing mit einem Fokus auf Enterprise Storage, und arbeitet redaktionell für storage-forum.de. Daneben ist er aktiv im Unternehmensvorstand der sys4 AG und beschäftigt sich mit der Historie längst verglühter aber nicht vergessener IT-Konzerne.
Datenspeicherung 2020 bis 2030 – immer noch auf Festplatten? – data://disrupt...data://disrupted®
Toshiba berichtet über aktuelle Trends und zukünftige Entwicklungen bei den Komponenten zur Datenspeicherung – Mobil, zu Hause, in den Rechenzentren und in der Cloud. Dabei werden Szenarien basierend auf Speicherung mit Tapes, Festplatten und SSD anhand der Anforderungen in Sachen Kapazität, Performance, Leistungsaufnahme, Kosten und Produktionskapazitäten miteinander verglichen. Mit einem Einblick in neue Technologien aus den Festplatten-Entwicklungslaboratorien des Konzerns und unter Berücksichtigung der oben genannten Anforderungen wird es einen Ausblick auf die Datenspeicherlandschaft der nächsten 10 Jahre geben. Ein besonderer Focus im Bereich Festplatten liegt diesmal auf der Bewertung der derzeit kontrovers diskutierten SMR („shingled magnetic recording“) Technologie.
Rainer Kaese ist seit über 25 Jahren bei Toshiba. Zunächst spezialisierte er sich auf anwendungsspezifische ICs, leitete das ASIC Design Center und später das Business Development Team für ASIC- und Foundry-Produkte. Derzeit ist er für die Einführung der Enterprise HDD-Produkte von Toshiba in Rechenzentren, Cloud Computing und Enterprise-Anwendungen verantwortlich.
Ceph Day Tokyo - Ceph on ARM: Scaleable and Efficient Ceph Community
This document discusses the opportunity for Ceph storage solutions using ARM processors. It outlines how the ARM ecosystem enables scalable and efficient storage options through increased performance, lower costs, and greater energy efficiency. The recent Jewel release of Ceph added support for AARCH64 processors. Several companies are developing Ceph clusters using ARM-based platforms that demonstrate benefits like reduced power consumption and total cost of ownership compared to x86 solutions.
Ceph Day Tokyo - Delivering cost effective, high performance Ceph clusterCeph Community
This document discusses an all-NVMe Ceph cluster configuration for MySQL hosting. It describes a 5-node Ceph cluster with Intel Xeon processors, 128GB of RAM, and 20 Intel SSDs providing 80 object storage devices (OSDs) for a total effective capacity of 19TB. Benchmark results show the cluster achieving over 1.4 million IOPS for 4K random reads with an average latency of 1ms, and over 220K IOPS for 4K random writes with 5ms latency. Sysbench tests of MySQL databases on the cluster using 16KB IOs showed response times under 10ms for query depths from 2 to 8.
Ceph Day Seoul - Ceph on Arm Scaleable and Efficient Ceph Community
This document discusses how Ceph storage solutions can benefit from ARM-based platforms. It outlines how the ARM ecosystem provides increased efficiency and scale for Ceph through lower costs, higher energy efficiency, and simplified designs. Examples are given of various companies delivering Ceph clusters using ARM processors, including solutions optimized for microservers, converged infrastructure, and enterprise storage. The recent Jewel release of Ceph added support for the AARCH64 instruction set, opening up additional opportunities for Ceph on ARM platforms.
This document provides information on AMD EPYC processors and their world record performance achievements. It lists numerous world records held by EPYC processors in areas such as single and dual socket configurations, database and analytics workloads, HPC, virtualization, Java applications, and more. A total of 93 world records are claimed as of October 28, 2020. The reader is directed to AMD's website for full details on all world records.
This document contains an agenda for a Ceph conference. It lists the scheduled time, duration, topic, and speakers for each presentation during the day. Talks will cover various topics related to optimizing Ceph performance using Intel technologies, all-flash array design using Ceph, global deduplication solutions for Ceph, and experiences deploying large-scale Ceph clusters at companies like Alibaba and China Mobile. The day includes keynotes, presentations, breaks for networking, and a closing session.
The document discusses AMD EPYC processors and lists several world records achieved using EPYC processors across various workloads and benchmarks. These include records for single and dual socket systems in areas like memory bandwidth, I/O expansion, big data analytics, virtualization, Java applications, and more. It provides details on specific benchmark results and dates from June 2020.
Ceph Day San Jose - Red Hat Storage Acceleration Utlizing Flash TechnologyCeph Community
The document discusses three ways to accelerate application performance with flash storage using Ceph software defined storage: 1) utilizing all flash storage to maximize performance, 2) using a hybrid configuration with flash and HDDs to balance performance and capacity, and 3) using all HDD storage for maximum capacity but lowest performance. It also examines using NVMe SSDs versus SATA SSDs, and how to optimize Linux settings and Ceph configuration to improve flash performance for applications.
Datenspeicherung 2020 bis 2030 – immer noch auf Festplatten? – data://disrupt...data://disrupted®
Toshiba berichtet über aktuelle Trends und zukünftige Entwicklungen bei den Komponenten zur Datenspeicherung – Mobil, zu Hause, in den Rechenzentren und in der Cloud. Dabei werden Szenarien basierend auf Speicherung mit Tapes, Festplatten und SSD anhand der Anforderungen in Sachen Kapazität, Performance, Leistungsaufnahme, Kosten und Produktionskapazitäten miteinander verglichen. Mit einem Einblick in neue Technologien aus den Festplatten-Entwicklungslaboratorien des Konzerns und unter Berücksichtigung der oben genannten Anforderungen wird es einen Ausblick auf die Datenspeicherlandschaft der nächsten 10 Jahre geben. Ein besonderer Focus im Bereich Festplatten liegt diesmal auf der Bewertung der derzeit kontrovers diskutierten SMR („shingled magnetic recording“) Technologie.
Rainer Kaese ist seit über 25 Jahren bei Toshiba. Zunächst spezialisierte er sich auf anwendungsspezifische ICs, leitete das ASIC Design Center und später das Business Development Team für ASIC- und Foundry-Produkte. Derzeit ist er für die Einführung der Enterprise HDD-Produkte von Toshiba in Rechenzentren, Cloud Computing und Enterprise-Anwendungen verantwortlich.
Ceph Day Tokyo - Ceph on ARM: Scaleable and Efficient Ceph Community
This document discusses the opportunity for Ceph storage solutions using ARM processors. It outlines how the ARM ecosystem enables scalable and efficient storage options through increased performance, lower costs, and greater energy efficiency. The recent Jewel release of Ceph added support for AARCH64 processors. Several companies are developing Ceph clusters using ARM-based platforms that demonstrate benefits like reduced power consumption and total cost of ownership compared to x86 solutions.
Ceph Day Tokyo - Delivering cost effective, high performance Ceph clusterCeph Community
This document discusses an all-NVMe Ceph cluster configuration for MySQL hosting. It describes a 5-node Ceph cluster with Intel Xeon processors, 128GB of RAM, and 20 Intel SSDs providing 80 object storage devices (OSDs) for a total effective capacity of 19TB. Benchmark results show the cluster achieving over 1.4 million IOPS for 4K random reads with an average latency of 1ms, and over 220K IOPS for 4K random writes with 5ms latency. Sysbench tests of MySQL databases on the cluster using 16KB IOs showed response times under 10ms for query depths from 2 to 8.
Ceph Day Seoul - Ceph on Arm Scaleable and Efficient Ceph Community
This document discusses how Ceph storage solutions can benefit from ARM-based platforms. It outlines how the ARM ecosystem provides increased efficiency and scale for Ceph through lower costs, higher energy efficiency, and simplified designs. Examples are given of various companies delivering Ceph clusters using ARM processors, including solutions optimized for microservers, converged infrastructure, and enterprise storage. The recent Jewel release of Ceph added support for the AARCH64 instruction set, opening up additional opportunities for Ceph on ARM platforms.
This document provides information on AMD EPYC processors and their world record performance achievements. It lists numerous world records held by EPYC processors in areas such as single and dual socket configurations, database and analytics workloads, HPC, virtualization, Java applications, and more. A total of 93 world records are claimed as of October 28, 2020. The reader is directed to AMD's website for full details on all world records.
This document contains an agenda for a Ceph conference. It lists the scheduled time, duration, topic, and speakers for each presentation during the day. Talks will cover various topics related to optimizing Ceph performance using Intel technologies, all-flash array design using Ceph, global deduplication solutions for Ceph, and experiences deploying large-scale Ceph clusters at companies like Alibaba and China Mobile. The day includes keynotes, presentations, breaks for networking, and a closing session.
The document discusses AMD EPYC processors and lists several world records achieved using EPYC processors across various workloads and benchmarks. These include records for single and dual socket systems in areas like memory bandwidth, I/O expansion, big data analytics, virtualization, Java applications, and more. It provides details on specific benchmark results and dates from June 2020.
Ceph Day San Jose - Red Hat Storage Acceleration Utlizing Flash TechnologyCeph Community
The document discusses three ways to accelerate application performance with flash storage using Ceph software defined storage: 1) utilizing all flash storage to maximize performance, 2) using a hybrid configuration with flash and HDDs to balance performance and capacity, and 3) using all HDD storage for maximum capacity but lowest performance. It also examines using NVMe SSDs versus SATA SSDs, and how to optimize Linux settings and Ceph configuration to improve flash performance for applications.
Ceph Day Beijing - Storage Modernization with Intel and CephDanielle Womboldt
The document discusses trends in data growth and storage technologies that are driving the need for storage modernization. It outlines Intel's role in advancing the storage industry through open source technologies and standards. A significant portion of the document focuses on Intel's work optimizing Ceph for Intel platforms, including profiling and benchmarking Ceph performance on Intel SSDs, 3D XPoint, and Optane drives.
AMD held its Next Horizon event on November 6, 2018 to announce its expanded commitment to datacenter computing innovation. The document provides forward-looking statements and estimates regarding AMD's strategy, products, technologies, and the benefits of its 7nm process. These include increased performance and leadership of the upcoming "Rome" CPU and Radeon Instinct MI60 GPU, as well as significant total performance increases expected from future generations. Endnotes provide sources and details to substantiate various claims made in the presentation.
One of the most important things you can do to improve the performance of your flash/SSDs with Aerospike is to properly prepare them. This Presentation goes through how to select, test, and prepare the drives so that you will get the best performance and lifetime out of them.
Ceph Day Beijing - Optimizing Ceph Performance by Leveraging Intel Optane and...Danielle Womboldt
Optimizing Ceph performance by leveraging Intel Optane and 3D NAND TLC SSDs. The document discusses using Intel Optane SSDs as journal/metadata drives and Intel 3D NAND SSDs as data drives in Ceph clusters. It provides examples of configurations and analysis of a 2.8 million IOPS Ceph cluster using this approach. Tuning recommendations are also provided to optimize performance.
HP 3Par StoreServ Storage: HP All Flash Array SSDUnitiv
Solid state drives are a game changer in the storage market. At this point, the performance benefits are common knowledge. But what might not be common knowledge is that SSDs have surpassed high speed spinning disk in capacity density. In the past 5 years, SSDs have increased in capacity by a magnitude of 38x while high speed spinning disks have only doubled
And HP has been helping to drive this transition by working very closely with our SSD suppliers. For example, we are the only major storage vendor to offer a 1.92 TB drive. Our patented Adaptive Sparing technology, which is unique to HP, let’s us work with the SSD vendors to take stock 1.6 TB drives and extend them to 1.92 based on the technology. That’s an increase in usable capacity per drive of up to 20%.
In June we announced this drive for the All-flash 7450 system and we are now extending this across the entire 3PAR Portfolio.
For customers that are interested in deploying Flash in smaller increments, we are also introducing a 480 cMLC Drive across the entire portfolio.
http://www.unitiv.com/hp-all-flash-array/
Ceph Day Tokyo - Bit-Isle's 3 years footprint with Ceph Ceph Community
Bit-isle has been using Ceph storage with OpenStack for 3 years, starting with a proof of concept in 2013. They have three Ceph environments - a development environment using OpenStack Havana and Ceph Dumpling, a staging environment using OpenStack Juno and Ceph Giant, and a production customer environment using OpenStack Kilo and Ceph Hammer. They chose Ceph because it provides high performance scalable storage without the need for expensive dedicated storage appliances or many storage engineers. Their initial POC was successful and showed Ceph could provide fault tolerance and cooperate well with OpenStack.
This document lists over 80 world records set by AMD EPYC 7002 series processors across various computing workloads and benchmarks. These records include the highest performance and efficiency in big data analytics, cloud computing, virtualization, enterprise applications, high performance computing, and more. All records were verified as of April 14, 2020 and additional details can be found at AMD.com/worldrecords.
Ceph Day Seoul - Delivering Cost Effective, High Performance Ceph cluster Ceph Community
Jack Zhang is a Senior Enterprise Architect at Intel Corp. This document discusses Ceph storage configurations using Intel SSDs and discusses benchmark results. Tuning Ceph for all-flash storage can significantly improve performance, with up to 16x better random read performance and 7.6x better random write performance achieved. Using SSDs instead of HDDs provides much higher performance, needing 58x fewer drives for the same write performance and 175x fewer for the same read performance. The document also outlines several suggested Ceph storage node configurations using different ratios of SSDs and HDDs.
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Lars Marowsky-Brée SUSE Distinguished Engineer, Ceph Advisory Board member
Marc Koderer, SAP OpenStack Evangelist
The document discusses 3PAR storage solutions and their benefits for virtualized environments using VMware. 3PAR offers thin provisioning, large volume sizes, and fine-grained virtualization which help address issues with ESX servers like random I/O stresses, time-consuming management as servers consolidate, and preference for large storage volumes. 3PAR solutions provide benefits like reduced storage administration, increased capacity utilization, and support for high server consolidation ratios.
The document summarizes the HP 3PAR Utility Storage product. It is next-generation tier 1 utility storage built for cloud computing. It is efficient, reducing costs by 50%, autonomic, increasing storage management efficiency 10 times, and multi-tenant and federated, supporting unpredictable applications and workloads across storage systems without boundaries. The HP 3PAR storage family provides the same technology at any scale from the F200 to the P10000, eliminating storage boundaries.
The document discusses using the Storage Performance Development Kit (SPDK) to optimize Ceph performance. SPDK provides userspace libraries and drivers to unlock the full potential of Intel storage technologies. It summarizes current SPDK support in Ceph's BlueStore backend and proposes leveraging SPDK further to accelerate Ceph's block services through optimized SPDK targets and caching. Collaboration is needed between the SPDK and Ceph communities to fully realize these optimizations.
This document provides an overview of the AMD EPYCTM microprocessor architecture. It discusses the key tenets of the EPYC processor design including the "Zen" CPU core, virtualization and security features, high per-socket capability through its multi-chip module (MCM) design, high bandwidth fabric interconnect, large memory capacity and disruptive I/O capabilities. It also details the microarchitecture of the "Zen" core and how it was designed and optimized for data center workloads.
The document discusses accelerating Ceph storage performance using SPDK. SPDK introduces optimizations like asynchronous APIs, userspace I/O stacks, and polling mode drivers to reduce software overhead and better utilize fast storage devices. This allows Ceph to better support high performance networks and storage like NVMe SSDs. The document provides an example where SPDK helped XSKY's BlueStore object store achieve significant performance gains over the standard Ceph implementation.
In this deck from the DDN User Group at SC19, Gael Delbray from CEA presents: Optimizing Flash at Scale. CEA, a major player in research and innovation, has been recognized as an expert in HPC through the momentum of the "Simulation Programme" supported by its Direction des Applications Militaires (CEA / DAM) and implemented by the Department of Simulation Sciences and Information (DSSI).
"The major challenges that the HPC will face in the coming years are manifold, such as the development of hardware and software architectures able to deliver very high computing power, modelling methods combining different scales and physical models and the management of huge volumes of numerical data.
High performance computing for numerical simulation has become an essential tool in scientific and technological research, as well as for industrial applications. Simulation can replace experiments that are too dangerous (accidental situations), beyond reach in terms of time or scale (climate, astrophysics) or banned (nuclear tests). Simulation is also time-saving and leverages productivity in many situations."
Watch the video: https://wp.me/p3RLHQ-li0
Learn more: https://www.ddn.com/company/events/user-group-sc/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
The document discusses Ceph storage performance on all-flash storage systems. It describes how SanDisk optimized Ceph for all-flash environments by tuning the OSD to handle the high performance of flash drives. The optimizations allowed over 200,000 IOPS per OSD using 12 CPU cores. Testing on SanDisk's InfiniFlash storage system showed it achieving over 1.5 million random read IOPS and 200,000 random write IOPS at 64KB block size. Latency was also very low, with 99% of operations under 5ms for reads. The document outlines reference configurations for the InfiniFlash system optimized for small, medium and large workloads.
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...Odinot Stanislas
(FR)
Voici un excellent document qui explique étape après étape comment installer, monitorer et surtout correctement benchmarker ses SSD PCIe/NVMe (pas si simple que ça). Autre élément clé : comment analyser la charge I/O de véritables applications? Combien d'IOPS, en read, en write, quelle bande passante et surtout quel impact sur la durée de vie des SSD? Bref à mettre en toute les mains, et un merci à mon collègue Andrey Kudryavtsev.
(EN)
An excellent content which describe step by step how to install, monitor and benchmark PCIe/NVMe SSD (many trick not so simple). Another key learning: how to measure real I/O activities on a real workload? How many R/W IOPS, block size, throughtput, and finally what's the impact on SSD endurance and (real)life? A must read, and a huge thanks to my colleague Andrey Kudryavtsev.
Auteurs/Authors:
Andrey Kudryavtsev, SSD Solution Architect, Intel Corporation
Zhdan Bybin, Application Engineer, Intel Corporation
Cowboy dating with big data TechDays at Lohika-2020b0ris_1
The story about things that happen if data platforms are developed not by data engineers, what pitfalls and mistakes can be made.
This will help you to understand what data engineering is about.
Leveraging Open Source to Manage SAN Performancebrettallison
Scope - The primary focus of this presentation is how to leverage open source software to help in managing Shared Storage performance. The storage server will be the focus with particular emphasis on ESS. This solution is a small one-off solution.
The document discusses optimizations made to Infinispan to improve performance and consistency for a trading application at the Chicago Board Options Exchange (CBOE). Some key points include:
- Early adoption of Infinispan led to some performance and consistency issues that required troubleshooting logs, cache contents, and test applications.
- Optimizations focused on asynchronous communication, queue flushing, and separating state provider/consumer roles to balance performance and consistency across cache tiers.
- Log analysis was important for detecting issues like out-of-order operations between active and passive nodes. Configuration tweaks like increasing queue size also helped.
Ceph Day Beijing - Storage Modernization with Intel and CephDanielle Womboldt
The document discusses trends in data growth and storage technologies that are driving the need for storage modernization. It outlines Intel's role in advancing the storage industry through open source technologies and standards. A significant portion of the document focuses on Intel's work optimizing Ceph for Intel platforms, including profiling and benchmarking Ceph performance on Intel SSDs, 3D XPoint, and Optane drives.
AMD held its Next Horizon event on November 6, 2018 to announce its expanded commitment to datacenter computing innovation. The document provides forward-looking statements and estimates regarding AMD's strategy, products, technologies, and the benefits of its 7nm process. These include increased performance and leadership of the upcoming "Rome" CPU and Radeon Instinct MI60 GPU, as well as significant total performance increases expected from future generations. Endnotes provide sources and details to substantiate various claims made in the presentation.
One of the most important things you can do to improve the performance of your flash/SSDs with Aerospike is to properly prepare them. This Presentation goes through how to select, test, and prepare the drives so that you will get the best performance and lifetime out of them.
Ceph Day Beijing - Optimizing Ceph Performance by Leveraging Intel Optane and...Danielle Womboldt
Optimizing Ceph performance by leveraging Intel Optane and 3D NAND TLC SSDs. The document discusses using Intel Optane SSDs as journal/metadata drives and Intel 3D NAND SSDs as data drives in Ceph clusters. It provides examples of configurations and analysis of a 2.8 million IOPS Ceph cluster using this approach. Tuning recommendations are also provided to optimize performance.
HP 3Par StoreServ Storage: HP All Flash Array SSDUnitiv
Solid state drives are a game changer in the storage market. At this point, the performance benefits are common knowledge. But what might not be common knowledge is that SSDs have surpassed high speed spinning disk in capacity density. In the past 5 years, SSDs have increased in capacity by a magnitude of 38x while high speed spinning disks have only doubled
And HP has been helping to drive this transition by working very closely with our SSD suppliers. For example, we are the only major storage vendor to offer a 1.92 TB drive. Our patented Adaptive Sparing technology, which is unique to HP, let’s us work with the SSD vendors to take stock 1.6 TB drives and extend them to 1.92 based on the technology. That’s an increase in usable capacity per drive of up to 20%.
In June we announced this drive for the All-flash 7450 system and we are now extending this across the entire 3PAR Portfolio.
For customers that are interested in deploying Flash in smaller increments, we are also introducing a 480 cMLC Drive across the entire portfolio.
http://www.unitiv.com/hp-all-flash-array/
Ceph Day Tokyo - Bit-Isle's 3 years footprint with Ceph Ceph Community
Bit-isle has been using Ceph storage with OpenStack for 3 years, starting with a proof of concept in 2013. They have three Ceph environments - a development environment using OpenStack Havana and Ceph Dumpling, a staging environment using OpenStack Juno and Ceph Giant, and a production customer environment using OpenStack Kilo and Ceph Hammer. They chose Ceph because it provides high performance scalable storage without the need for expensive dedicated storage appliances or many storage engineers. Their initial POC was successful and showed Ceph could provide fault tolerance and cooperate well with OpenStack.
This document lists over 80 world records set by AMD EPYC 7002 series processors across various computing workloads and benchmarks. These records include the highest performance and efficiency in big data analytics, cloud computing, virtualization, enterprise applications, high performance computing, and more. All records were verified as of April 14, 2020 and additional details can be found at AMD.com/worldrecords.
Ceph Day Seoul - Delivering Cost Effective, High Performance Ceph cluster Ceph Community
Jack Zhang is a Senior Enterprise Architect at Intel Corp. This document discusses Ceph storage configurations using Intel SSDs and discusses benchmark results. Tuning Ceph for all-flash storage can significantly improve performance, with up to 16x better random read performance and 7.6x better random write performance achieved. Using SSDs instead of HDDs provides much higher performance, needing 58x fewer drives for the same write performance and 175x fewer for the same read performance. The document also outlines several suggested Ceph storage node configurations using different ratios of SSDs and HDDs.
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Lars Marowsky-Brée SUSE Distinguished Engineer, Ceph Advisory Board member
Marc Koderer, SAP OpenStack Evangelist
The document discusses 3PAR storage solutions and their benefits for virtualized environments using VMware. 3PAR offers thin provisioning, large volume sizes, and fine-grained virtualization which help address issues with ESX servers like random I/O stresses, time-consuming management as servers consolidate, and preference for large storage volumes. 3PAR solutions provide benefits like reduced storage administration, increased capacity utilization, and support for high server consolidation ratios.
The document summarizes the HP 3PAR Utility Storage product. It is next-generation tier 1 utility storage built for cloud computing. It is efficient, reducing costs by 50%, autonomic, increasing storage management efficiency 10 times, and multi-tenant and federated, supporting unpredictable applications and workloads across storage systems without boundaries. The HP 3PAR storage family provides the same technology at any scale from the F200 to the P10000, eliminating storage boundaries.
The document discusses using the Storage Performance Development Kit (SPDK) to optimize Ceph performance. SPDK provides userspace libraries and drivers to unlock the full potential of Intel storage technologies. It summarizes current SPDK support in Ceph's BlueStore backend and proposes leveraging SPDK further to accelerate Ceph's block services through optimized SPDK targets and caching. Collaboration is needed between the SPDK and Ceph communities to fully realize these optimizations.
This document provides an overview of the AMD EPYCTM microprocessor architecture. It discusses the key tenets of the EPYC processor design including the "Zen" CPU core, virtualization and security features, high per-socket capability through its multi-chip module (MCM) design, high bandwidth fabric interconnect, large memory capacity and disruptive I/O capabilities. It also details the microarchitecture of the "Zen" core and how it was designed and optimized for data center workloads.
The document discusses accelerating Ceph storage performance using SPDK. SPDK introduces optimizations like asynchronous APIs, userspace I/O stacks, and polling mode drivers to reduce software overhead and better utilize fast storage devices. This allows Ceph to better support high performance networks and storage like NVMe SSDs. The document provides an example where SPDK helped XSKY's BlueStore object store achieve significant performance gains over the standard Ceph implementation.
In this deck from the DDN User Group at SC19, Gael Delbray from CEA presents: Optimizing Flash at Scale. CEA, a major player in research and innovation, has been recognized as an expert in HPC through the momentum of the "Simulation Programme" supported by its Direction des Applications Militaires (CEA / DAM) and implemented by the Department of Simulation Sciences and Information (DSSI).
"The major challenges that the HPC will face in the coming years are manifold, such as the development of hardware and software architectures able to deliver very high computing power, modelling methods combining different scales and physical models and the management of huge volumes of numerical data.
High performance computing for numerical simulation has become an essential tool in scientific and technological research, as well as for industrial applications. Simulation can replace experiments that are too dangerous (accidental situations), beyond reach in terms of time or scale (climate, astrophysics) or banned (nuclear tests). Simulation is also time-saving and leverages productivity in many situations."
Watch the video: https://wp.me/p3RLHQ-li0
Learn more: https://www.ddn.com/company/events/user-group-sc/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
The document discusses Ceph storage performance on all-flash storage systems. It describes how SanDisk optimized Ceph for all-flash environments by tuning the OSD to handle the high performance of flash drives. The optimizations allowed over 200,000 IOPS per OSD using 12 CPU cores. Testing on SanDisk's InfiniFlash storage system showed it achieving over 1.5 million random read IOPS and 200,000 random write IOPS at 64KB block size. Latency was also very low, with 99% of operations under 5ms for reads. The document outlines reference configurations for the InfiniFlash system optimized for small, medium and large workloads.
Hands-on Lab: How to Unleash Your Storage Performance by Using NVM Express™ B...Odinot Stanislas
(FR)
Voici un excellent document qui explique étape après étape comment installer, monitorer et surtout correctement benchmarker ses SSD PCIe/NVMe (pas si simple que ça). Autre élément clé : comment analyser la charge I/O de véritables applications? Combien d'IOPS, en read, en write, quelle bande passante et surtout quel impact sur la durée de vie des SSD? Bref à mettre en toute les mains, et un merci à mon collègue Andrey Kudryavtsev.
(EN)
An excellent content which describe step by step how to install, monitor and benchmark PCIe/NVMe SSD (many trick not so simple). Another key learning: how to measure real I/O activities on a real workload? How many R/W IOPS, block size, throughtput, and finally what's the impact on SSD endurance and (real)life? A must read, and a huge thanks to my colleague Andrey Kudryavtsev.
Auteurs/Authors:
Andrey Kudryavtsev, SSD Solution Architect, Intel Corporation
Zhdan Bybin, Application Engineer, Intel Corporation
Cowboy dating with big data TechDays at Lohika-2020b0ris_1
The story about things that happen if data platforms are developed not by data engineers, what pitfalls and mistakes can be made.
This will help you to understand what data engineering is about.
Leveraging Open Source to Manage SAN Performancebrettallison
Scope - The primary focus of this presentation is how to leverage open source software to help in managing Shared Storage performance. The storage server will be the focus with particular emphasis on ESS. This solution is a small one-off solution.
The document discusses optimizations made to Infinispan to improve performance and consistency for a trading application at the Chicago Board Options Exchange (CBOE). Some key points include:
- Early adoption of Infinispan led to some performance and consistency issues that required troubleshooting logs, cache contents, and test applications.
- Optimizations focused on asynchronous communication, queue flushing, and separating state provider/consumer roles to balance performance and consistency across cache tiers.
- Log analysis was important for detecting issues like out-of-order operations between active and passive nodes. Configuration tweaks like increasing queue size also helped.
2015-06-25 Red Hat Summit 2015 - Security Compliance Made EasyShawn Wells
The document discusses how Security Content Automation Protocol (SCAP) is making security compliance easier. It summarizes that SCAP allows automated compliance checks of systems through profiles that can remediate configurations with a single command. Live demos show using SCAP for installation, scanning systems, and remediating any issues in real-time.
The story about things that happen if data platforms are developed not by data engineers, what pitfalls and mistakes can be made.
This will help you to understand what data engineering is about.
This resume summarizes the qualifications of Edward Wang. He has over 10 years of experience in performance engineering, testing, and analysis. Some of his responsibilities include performance testing, analysis and tuning of Veritas/Symantec products, virtualization solutions from VMware, and storage arrays. He is proficient with various performance benchmarking and debugging tools. He has delivered performance improvements and resolved issues for numerous projects.
- Veeam provides backup and recovery solutions for virtualized environments without needing agents
- Features include application-aware backups, file-level restores, change block tracking for Hyper-V, and SureBackup for validation
- vSphere Operations Manager provides visibility into virtual infrastructure health with performance and efficiency metrics
- vSphere Flash Read Cache uses server-based flash to accelerate VM performance by caching reads, providing up to 2x improvement
- Virtual SAN software-defined storage that pools server HDDs and flash for a shared datastore with policies, resilience, and integration
Troubleshooting Storage Devices Using vRealize Operations (formerly vC Ops)
In this presentation you will discover:
- The challenges facing today’s storage environment
- How vR Ops solves storage troubleshooting
- When to use vR Ops
- Where to get a Management Pack for Storage Devices (MPSD)
- Partner Solutions
ScyllaDB Open Source 5.0 is the latest evolution of our monstrously fast and scalable NoSQL database – powering instantaneous experiences with massive distributed datasets.
Join us to learn about ScyllaDB Open Source 5.0, which represents the first milestone in ScyllaDB V. ScyllaDB 5.0 introduces a host of functional, performance and stability improvements that resolve longstanding challenges of legacy NoSQL databases.
We’ll cover:
- New capabilities including a new IO model and scheduler, Raft-based schema updates, automated tombstone garbage collection, optimized reverse queries, and support for the latest AWS EC2 instances
- How ScyllaDB 5.0 fits into the evolution of ScyllaDB – and what to expect next
- The first look at benchmarks that quantify the impact of ScyllaDB 5.0's numerous optimizations
This will be an interactive session with ample time for Q & A – bring us your questions and feedback!
This document discusses Huawei's vision for making IT simple and business agile through data center innovation. It outlines Huawei's strategy in five areas: 1) Redesigning modern data center architecture for openness, automation, and efficiency. 2) Accelerating storage solutions with all-flash storage arrays. 3) Developing open platforms for critical business applications. 4) Creating a unified ICT cloud operating system. 5) Delivering converged infrastructure through modular systems. The goal is to help customers simplify IT operations, lower costs, and rapidly deploy new services through software-defined infrastructure.
Storage Is Not Virtualized Enough - part 1Zhipeng Huang
This document discusses storage function virtualization (SFV) and why storage is not yet virtualized enough. It provides an overview of SFV and how it can enhance NFV by virtualizing storage functions. It also discusses OpenStack SFV and related blueprints to improve storage scheduling and functionality. Finally, it encourages collaboration on SFV-related work beyond just OpenStack through various communication channels and open source projects.
This document provides an overview and agenda for a presentation on Dell Storage Management tools. It discusses Enterprise Manager (EM) and how it automates management of Dell Storage Center (SC) systems. EM 2015 includes new functionality like localization and thin import migration from EqualLogic PS series to SC series. It also discusses Dell Storage Manager (DSM), a future unified management platform that will provide a single pane of glass for managing SC, PS series, and other Dell storage products. DSM 2016 R1 will add capabilities like management of EqualLogic PS groups and cross-platform replication between SC and PS series storage systems.
This document describes a summer industrial training project to develop an online SQL forum. It includes sections on functional and system requirements, hardware and software specifications, data modeling diagrams, use cases, screen shots, testing approaches, future enhancements, and references. A group of students developed the online application to allow users to ask and answer SQL questions, with an admin able to moderate content.
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Community
SK Telecom is optimizing Ceph for all-flash storage to improve performance and efficiency. Recent work includes enhancing BlueStore, implementing quality of service controls, and exploring data deduplication techniques. Looking ahead, SKT aims to further leverage NVRAM/SSD technologies and expand use of all-flash Ceph in its cloud infrastructure.
Webinar replay: MySQL Query Tuning Trilogy: Query tuning process and toolsSeveralnines
This is part 1 of a webinar trilogy on MySQL Query Tuning, in which we look at query tuning process and tools to help with that. We’ve covered topics such as SQL tuning, indexing, the optimizer and how to leverage EXPLAIN to gain insight into execution plans. Part 1: Query tuning process and tools.
AGENDA
• Query tuning process
- Build
- Collect
- Analyze
- Tune
- Test
• Tools
- tcpdump
- pt-query-digest
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
This report compares the performance of Apache Hadoop to IBM Platform Symphony, which leverages IBM middleware to accelerate Hadoop. A benchmark using 302 jobs from the Statistical Workload Injector for MapReduce (SWIM), based on production Facebook workloads, found that Symphony accelerated Hadoop by an average of 7.3x. Symphony's advantage declined slowly with increasing shuffle size. In a "sleep" test of scheduling latency, Symphony was 74x faster than Hadoop alone. While these results may depend on configuration settings, the test systems used identical hardware, software, and network configurations. The report provides detailed information on the test methodology, systems tested, and results.
IBM SAN Volume Controller Performance Analysisbrettallison
Introduction
Storage Problems and Limitations with Native Storage
SVC Overview
SVC Physical and Logical Overview
Performance and Scalability Implications
Types of Problems
Performance Analysis Techniques
Performance Analysis Tools for SVC
Performance Analysis Metrics for SVC
Online Banking Example
Avoiding Chaos: Methodology for Managing Performance in a Shared Storage A...brettallison
Scope - The primary focus of this presentation is on the methodology we use for managing performance in a very large shared Storage Area Network environment with a Primary focus on Distributed Systems and IBM Enterprise Storage Server. The focus on this presentation is methodology and NOT measurement. There are numerous excellent presentations already out there on measurement. However, there are several references in the back of the presentation to measurement tools.
Learn how upcoming changes in the persistent memory market will affect deployments of in-memory computing and traditional applications. Using software innovations from SanDisk and the broad portfolio of flash storage hardware options, customers and developers can optimize applications for “flash extended memory”, the intersection of in-memory computing and persistent memory technologies.
Red Hat Storage Day Atlanta - Designing Ceph Clusters Using Intel-Based Hardw...Red_Hat_Storage
This document discusses the need for storage modernization driven by trends like mobile, social media, IoT and big data. It outlines how scale-out architectures using open source Ceph software can help meet this need more cost effectively than traditional scale-up storage. Specific optimizations for IOPS, throughput and capacity are described. Intel is presented as helping advance the industry through open source contributions and optimized platforms, software and SSD technologies. Real-world examples are given showing the wide performance range Ceph can provide.
Similaire à Storage Benchmarks - Voodoo oder Wissenschaft? – data://disrupted® 2020 (20)
Benchmarking your cloud performance with top 4 global public cloudsdata://disrupted®
In this presentation, we will present the performance measurement metrics of leading cloud providers - AWS, Google Cloud, Microsoft Azure, and Digital Ocean. We’ll give you useful tools to measure your own cloud performance and a handy guide on how to calculate cloud TCO (total cost of ownership). In addition, you’ll learn how to estimate correctly your market positioning and perform better than the cloud giants.
Boyan Krosnov is a Co-Founder and Chief Product Officer of StorPool Storage. He has been part of the technical teams building 5 service providers from scratch in 4 countries. In most of these projects, he has designed the architecture, led the technical teams, and managed the implementation of projects in the millions.
Many companies build new-age KVM clouds, only to find out that their applications & workloads do not perform well. In this talk we’ll show you how to get the most out of your KVM cloud and how to optimize it for performance: You’ll understand why performance matters and how to measure it properly. We’ll teach you how to optimize CPU and memory for ultimate performance and how to tune the storage layer for performance. You’ll find out what are the main components of an efficient new-age cloud and which network components work best. In addition, you’ll learn how to select the right hardware to achieve unmatched performance for your new-age cloud and applications.
Venko Moyankov is an experienced system administrator and solutions architect at StorPool storage. He has experience with managing large virtualizations, working in telcos, designing and supporting the infrastructure of large enterprises. In the last year, his focus has been in helping companies globally to build the best storage solution according to their needs and projects.
Wie wir inzwischen ja alle wissen, wird die Bedrohungslage durch Ransomware bestehen bleiben. Aktuelle Versionen gehen gezielt auf Unternehmen des Mittelstandes und greifen Online-Backups an. Die einzige Versicherung gegen wochenlangen Ausfall und hohe Kosten sind offline-fähige Medien, die außerhalb des Online-Systems gelagert werden können (Air Gap). Neben Tape, das vor allem für Petabyte-Archive noch seine Berechtigung hat, kann man aber auch moderne Wechselmedien so gestalten, dass sie Random Access, hohe, eingebaute Sicherheit und dennoch Offline-Fähigkeit bieten.
Christian Peschke ist als COO bei der FAST LTA seit über 10 Jahren in der Entwicklung und Produktion von Sekundärspeichertechnologien tätig. Geprägt durch seine interdisziplinären Verantwortlichkeiten, sowie durch seine Vorliebe für die Restaurierung seltener Oldtimer, verfügt er über ein ausgeprägtes Detaildenken sowie einen Hang für clevere Lösungen.
Tape-basierter Object-Storage als S3 Speicherklasse und Cloud-Absicherungdata://disrupted®
This document discusses using tape-based storage as an S3 object storage class and for cloud backup. It describes how tape storage can provide scalable, high-throughput object storage with native S3 API access. Examples of using tape-based S3 storage include big data analytics, tiering from HDD storage, and backing up data from public cloud services like AWS S3 for added protection against risks like human errors and technology failures.
Rook: Storage for Containers in Containers – data://disrupted® 2020data://disrupted®
In this talk Kim-Norman Sahm and Alexander Trost dive into the challenges of storage for containerized applications on Kubernetes. We'll see how the current state is and how Rook can help with that. We are going to especially look at Ceph run through Rook here, but nonetheless trying not to lose sight of the whole picture. There is a lot to keep in mind storage as is, but everything gets more complex with storage for containers. From what type of storage to how much and how "safe" it should, all questions that should be asked and most of them which should be answered as well. Rook's project site https://rook.io/
Kim-Norman Sahm is CTO of Cloudical. He also works as Executive Cloud Architect at Cloudical. Previously, he was OpenStack Cloud Architect at T-Systems (operational services GmbH) and noris network AG. He is an expert of the technologies OpenStack, Ceph and Kubernetes (CKA).
Alexander Trost works as a DevOps Engineer at Cloudical Deutschlang GmbH and is a Certified Kubernetes Administrator (CKA). He is one of four maintainers of the Rook.io Project and is engaged in several more open source projects, for example a Prometheus exporter for Dell Hardware (Dell OMSA Metrics), k8s-vagrant-multi-node an easy local multi node Kubernetes environment, and others. Besides Containers and Kubernetes he is expert on Software Defined Storage, Golang and Continuous Integration (with GitLab CI). He passionately enjoys working on open source projects, such as Rook, Ancientt and other projects.
Speichermedium Tape – Warum es keine Alternative gibt – data://disrupted® 2020data://disrupted®
Leistungsverbesserung in allen Bereichen, Reduzierung der Kosten und überragender Schutz der Daten: Die fortschreitende Digitalisierung sowie neue Gesetze und Richtlinien zur Archivierung von Daten stellen den Ursprung der Datenarchivierung und dem damit verbundenen spektakulären Wachstum dar. Die Tape-Technologie nimmt mit einem Marktanteil von 60% eine dominante Rolle im Bereich der Datenarchivierung ein. Ein jährliches Wachstum von 24% zeigt den ungebrochenen Wachstumstrend der Technologie. Im Vortrag erfahren Sie, welche Hürden Tape-Hersteller überwinden mussten, um die Technologie zukunftssicher zu gestalten. Dabei zeigen wir, wie sich der Speichermarkt in den letzten Jahren zugunsten von Tape entwickelt hat und was wir für die nächsten Jahre prognostizieren können. Weiterhin möchten wir die neuste Tape-Generation LTO9 vorstellen, die bereits in den Startlöchern steht. Sie beweist, welches Potenzial in der Technologie steckt und erweitert die Parameter weiter, um den Nutzern bestmögliche Leistung für Themen der Datenspeicherung an die Hand zu geben: 18 TB Kapazität und 400 MB/s Transferrate sind nur zwei der überzeugenden Eckdaten. Zudem haben wir uns auch den Bedürfnissen von Object-Storage-Anwendern angenommen. Im Vortrag erfahren Sie, wie Fujifilm es nun ermöglicht, auch Ihre Daten objektbasiert auf Tape zu schreiben und Sie so von großen Kostenersparnissen sowie nicht da gewesener Sicherheit im Object Storage Bereich profitieren können.
Florian Brendel begann im Jahr 2016 bei Fujifilm Recording Media im Bereich Business Development für den deutschen Markt. Seit 2017 ist er für die DACH-Region zuständig und berät und betreut Unternehmen in Bereich Speicherlösungen mit Fokus auf Großkunden, welche Speicherkapazitäten im Petabyte-Bereich vorhalten.
Ransomware: Ohne Air Gap & Tape sind Sie verloren! – data://disrupted® 2020data://disrupted®
Die Bedrohungslage verschärft sich: Die Allianz bewertet Cyber Incidents als größtes Geschäftsrisiko überhaupt. Das BSI spricht von „massenhafter Verbreitung raffinierter Angriffsmethoden durch Organisierte Kriminalität.“ An vorderster Front stehen hier natürlich IT-Sicherheitssysteme. Aber wie die letzten Angriffe – unter anderem mit Ransomware – zeigten, greifen diese nicht immer. Und dann sind oft sehr viele oder sogar alle Daten inklusive der Backupdaten zerstört. Hier hilft dann nur noch die Daten von einem Physical-Air-Gap-Medium zurückzusichern. Sehen Sie, warum Tape das sicherste Medium ist. Erfahren Sie auch, warum nahezu alle große Cloud-Service-Provider Tape einsetzen. Sehen Sie die zukünftige Roadmap, Produktentwicklung und die weiteren Einsatzgebiete von Tape.
Josef Weingand ist Business Development Manager für Tape Storage bei IBM, zuständig für die DACH-Region. Er hat über 23 Jahre Erfahrung im Tape Storage und arbeitet sowohl im Technical Support als auch im Sales Support für Data-Protection-, Data-Retention- und Tape-Lösungen. Er hat an mehreren IBM Redbooks mitgearbeitet und einige Patente im Storage-Bereich erarbeitet.
HCI einfach einfach! IT-Infrastruktur wie ein Smartphone! – data://disrupted®...data://disrupted®
HCI einfach einfach. Lassen Sie sich zeigen, wie Scale HC3 ihre IT-Infrastruktur vereinfacht. Scale HC3 ist das Smartphone der Infrastruktur. Skalierbarkeit, Effizienz, Performance und Kosten im Griff!
Thorsten Schäfer ist ein sehr erfahrener Vertriebsleiter mit mehr als 15 Jahren Erfahrung in der Computer-Software-Branche, der sich mit Hyper Converged (HCI), Software Defined Storage (SDS), Storage Area Network (SAN), VMware, Storage und VDI auskennt. Starker Vertriebsprofi sowie sehr starker technischer Hintergrund: als erstes deutscher DCME – aber auch in den Bereichen Business Development, Consulting, Engineering, Channel Development & Management.
William van Collenburg verfügt über fast 20 Jahre Erfahrung in der IT und arbeitet sowohl im Service- als auch im technischen Bereich. Sein Hauptaugenmerk lag immer auf dem Wissenstransfer in einem verständlichen Weg. Aufgrund der Tatsache, dass er seine Erfahrung vielfältig ist, aus vielen Bereichen der IT und Technologien, kann er Kunden nicht nur helfen, sondern auch bei Problemen zur Hand gehen. William hat großes Wissen und Erfahrung mit vielen Speicher-, Netzwerk- und (Desktop-) Virtualisierungstechnologien. Er hat unter anderem bei EMC gearbeitet, war Trainer bei Dell, Pre-Sales-Berater für Virtualisierung bei VMware und Citrix sowie Pre-Sales Vertriebsingenieur bei Springpath (jetzt Cisco Hyperflex).
Zuverlässiger IT-Betrieb braucht redundante Datenhaltung. Die letzten 30 Jahre nutze man dazu RAID-Systeme, die Technologie ist ausgereift und war bisher völlig ausreichend. Mit zunehmenden Festplattengrößen und dem Einzug von All-Flash-Arrays sowie verteilten Dateisystemen ist eine „einfache“ Redundanz wie sie RAID-Mechanismen bieten nicht mehr ausreichend. Erasure Coding schafft hier Abhilfe. Im Vortrag erklärt Wolfgang Stief, wie Erasure Coding grundsätzlich funktioniert. im ersten Teil erläutert er ein paar Grundideen hinter und Grundbegriffe zu Erasure Coding. Im zweiten Teil beschreibt unser Experte beispielhaft derzeit übliche Implementierungen von Erasure Coding. Er versucht dabei, möglichst ohne Mathematik auszukommen. Außerdem werden im Vortrag Vorteile aber auch Probleme, Grenzen und Fallstricke beim Einsatz von Erasure Coding beleuchtet. Abschließend folgen beispielhaft einige bereits heute übliche Anwendungen für Erasure Coding.
Nextcloud als On-Premises Lösung für hochsicheren Datenaustausch (Frank Karli...data://disrupted®
Nextcloud is an open-source, self-hosted content collaboration platform that provides file sharing and editing, calendars, contacts, communication tools and other features. It allows organizations to have centralized access to all enterprise data within their own infrastructure or hosted wherever they prefer. Some key benefits of Nextcloud include being open source with no vendor lock-in, highly scalable architecture to support large user bases, and additional security, privacy and compliance features for sensitive workloads. Nextcloud has over 200 customers including government agencies and educational institutions seeking a secure alternative to proprietary cloud services.
Operation Unthinkable – Software Defined Storage @ Booking.com (Peter Buschman)data://disrupted®
The story of the plan that was just crazy enough to work! Learn how Booking.com failed its way to success on a multi-year journey away from single-purpose storage-appliances, predatory-licensing, and over-complicated networking to create a unique storage solution for their hyper-scale private-cloud environment.
Die IBM 3592 Speicherlösung: Ein Vorgeschmack auf die Zukunft (Anne Ingenhaag)data://disrupted®
Erfahren Sie, was die Hersteller von Tape Technologien aus technischer Sicht alles beachten müssen, damit Tapes mit einer Kapazität von bis zu 20TB und einer Transferrate von bis zu 400MB/s, bei exzellenter Performance, auf den Markt gebracht werden können. Lernen Sie, was die Zukunft der Technologie bringt und woran wir heute schon forschen. Fokus wird hierbei auf den technologischen Unterschieden zwischen der herkömmlichen LTO Technologie und der Enterprise (Jaguar) Technologie von IBM liegen. Die 3592 Technologie bietet nicht nur 67% mehr Speicherkapazität, eine 10-fach höhere Datenintegrität, eine längere Lebensdauer und weniger Migrationszyklen als die LTO8 Technologie, sondern beinhaltet einige technische Features, die den Zugriff auf die Daten um bis zu 50% beschleunigen.
CANDIDATE EXPERIENCE – Was Bewerber tatsächlich erwarten.data://disrupted®
Um im Recruiting erfolgreich zu sein, müssen Unternehmen lernen, bewerberorientiert zu denken und zu handeln. Die Candidate Experience hat das Ziel, den Bewerbungsprozess Ihrer potenziellen Mitarbeiter in ein positives, motivierendes Erlebnis zu wandeln. Doch wie und wo ansetzen? Kennen Sie die Anforderungen Ihrer Bewerber?
get-a-MINT zeigt Ihnen, wie Sie das Konzept der Candidate Experience praktisch auf Ihre Recruitingprozesse übertragen können und die Candidate Journey Ihrer bewerber optimieren.
Cloud/Object-basierte Datenspeicherung mit HSM/ILM in S3 Speicherklassen (Tho...data://disrupted®
PoINT Software & Systems presents a solution for tape-based object storage compatible with the S3 API. The solution uses erasure coding to store data across multiple tape drives in a library. It can be used for archiving cold data in public clouds, replicating and backing up cloud object storage, and implementing hierarchical storage management policies to tier files from primary to secondary storage on tape. The system provides scalability, high throughput, and protects against data loss through its use of erasure coding and an "air gap" approach enabled by tape.
In einem flotten Ritt durch das Land der aktuellen Storage-Buzzwords erläutere ich kurz ein paar technische Hintergründe zu den Begriffen und versuche, nach Sinn und Unsinn zu unterscheiden. Mitreisende werden am Ende der halben Stunde ein grobes Verständnis für Dinge wie Storage Class Memory, Persistent Memory oder Computational Storage haben, und können ein paar Begriffe mit nach Hause nehmen, die beim selbständigen weiterforschen helfen.
Hochleistungsspeichersysteme für Datenanalyse an der TU Dresden (Michael Kluge)data://disrupted®
Zur Unterstützung von Big Data und Machine Learning Szenarien wurde am Zentrum für Informationsdienste und Hochleistungsrechnen (ZIH) der TU Dresden eine neue Speicherlandschaft mit „NVMe Storage“ (2 PB Kapazität und 2 TB/s Bandbreite, <100us Latenz) und „Warm Archive“ auf Basis des S3-Protokolls (10 PB Kapazität und 50 GB/s Bandbreite) aufgebaut. Dr. Michael Kluge vom ZIH (Abteilungsleiter System- und Dienstentwurf) erläutert die besonderen Anforderungen dieses Projektes und berichtet vom Aufbau und Betrieb der Umgebung.
Während das Gros der Storage-Hersteller in IIoT/IoT noch den nächsten großen Storage-Hype sehen, zeigt uns die Analytics-Welt andere wegweisende Trends in der Verarbeitung der Daten aus Sensor und Sensorkette auf. So bringt Google beispielsweise mit Coral bereits Neuronale Netze als embedded KI in die Chipsets der Sensoren zum Preis weniger Cent. Wie in der Mathematik gelernt, geht der Trend nun auch hier in Richtung Vereinfachung und Reduktion bereits in den ersten Schritten der Aufgabenstellung. Redundanzen und anderes unnützes Beiwerk braucht weder übertragen oder gar gespeichert werden ...
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
OpenID AuthZEN Interop Read Out - AuthorizationDavid Brossard
During Identiverse 2024 and EIC 2024, members of the OpenID AuthZEN WG got together and demoed their authorization endpoints conforming to the AuthZEN API
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
8. Herausforderungen @ Benchmarks (generell)
‣ synthetische Last vs. reale Last
➛ Benchmarks müssen vergleichbar sein
➛ Lastverhalten in RZs individuell stark unterschiedlich
‣ Man kann das Falsche messen oder das falsche messen wollen.
‣ Man kann das falsche Ergebnis schlussfolgern.
‣ Man kann Fehler ignorieren.
‣ Bugs in der Benchmark-Software
➛ selber schreiben i. d. R. keine Option, weil sehr (zeit)aufwendig.
‣ »Active Benchmarking«
Brendan Gregg, http://www.brendangregg.com/activebenchmarking.html
»Wer misst, misst Mist«— überlieferte Ingenieursweisheit
14. Warum kein dd?
‣ dd(1) ➛ disk dump
‣ dd if=<input_file> of=<output_file> bs=<blocksize>
➛ sequential only
➛ genau ein Stream mit genau einer Blocksize
‣ if=/dev/zero ➛ liefert Strom von Nullen
➛ lässt sich hervorragend cachen
➛ lässt sich hervorragend komprimieren
➛ lässt sich hervorragend deduplizieren
‣ if=/dev/random oder /dev/urandom
➛ Bottleneck ist häufig CPU
15. Agenda
‣ Warum überhaupt Benchmarks?
Aussagekraft, Herausforderungen, I/O-Stack, Tools
‣Storage Performance Council
SPC-1, SPC-2, Industriestandard, Terminologie, Reports
‣ Und was heißt das jetzt für’s Tagesgeschäft?
Benchmarks selber programmieren?
Benchmarks für die Beschaffung?
Glaskugel
16. Industriestandard SPC-1 und SPC-2
‣ Storage Performance Council
➛ https://spcresults.org/
➛ Full Member: 12.000U$, Toolkit 4.000U$, je Submission 1.000U$ (139 Mitglieder)
➛ Associate Member: 5.000U$, Toolkit 6.000U$, je Submission 1.500U$ (140 Mitglieder)
➛ Academic Member: 0U$, Toolkit 500U$ (limited license), keine Submissions (141 Mitglieder)
➛ »Sponsors«
‣ formale Definition und Spezifikation
➛ reproduzierbar
➛ vgl. auch SPECint/SPECfp für CPUs
‣ Vergleichbarkeit von Ergebnissen
‣ SPC-1 für zeitkritische Anforderungen (OLTP, Response Time)
‣ SPC-2 für »large scale sequential movement of data«
22. SPC-1 — Executive Summary (3)
EXECUTIVE SUMMARY Page 4 of 6
SPC Benchmark 1™ V3.8 Submission Identifier: A31020
Huawei Technologies Co., Ltd Submitted for Review: December 27, 2018
Huawei OCEANSTOR 5600 V5
Storage Configuration Pricing
Third-Party Reseller: Huawei Technologies Co., Ltd. only sells its products to third-
party resellers who, in turn, sell those products to U.S. customers. The above reflects
the pricing quoted by one of those third-party resellers. See Appendix B of the Full
Disclosure Report for a copy of the third-party reseller s quotation.
Description Qty Unit Price Ext. Price Disc. Disc. Price
02351LWK 56V5-256G-AC2 OceanStor 5600 V5 Engine(3U,Dual
Controller,AC240HVDC,256GB
Cache,SPE63C0300) 2 116,820.00 233,640.00 68% 74,764.80
SMARTIO10ETH 4 port SmartIO I/O module(SFP+,10Gb
Eth/FCoE(VN2VF)/Scale-out) 4 6,288.00 25,152.00 68% 8,048.64
SMARTIO8FC 4 port SmartIO I/O module(SFP+,8Gb FC) 8 3,192.00 25,536.00 68% 8,171.52
LPU4S12V3 4 port 4*12Gb SAS I/O module(MiniSAS HD) 8 4,963.00 39,704.00 68% 12,705.28
HSSD-960G2S-A9 960GB SSD SAS Disk Unit(2.5") 72 10,176.00 732,672.00 70% 219,801.60
DAE52525U2-AC-A2
Disk Enclosure(2U,AC240HVDC,2.5",Expanding
Module,25 Disk Slots,without Disk
Unit,DAE52525U2) 8 10,584.00 84,672.00 68% 27,095.04
N8GHBA000 QLOGIC QLE2562 HBA Card,PCIE,8Gbps
DualPort ,Fiber Channel Multimode LC Optic
Interface,English Manual, No Drive CD 12 1,698.00 20,376.00 0% 20,376.00
SN2F01FCPC Patch Cord,DLC/PC,DLC/PC,Multi-
mode,3m,A1a.2,2mm,42mm DLC,OM3 bending
insensitive 24 14.00 336.00 0% 336.00
LIC-56V5-BS Basic Software License(Including
DeviceManager,SmartThin,SmartMulti-
tenant,SmartMigration,SmartErase,SmartMotion,
SystemReporter,eService,SmartQuota,NFS,CIFS,
NDMP 1 9,852.00 9,852.00 70% 2,955.60
374,254.48
02351LWK-88134ULF-36 OceanStor 5600 V5 Engine(3U,Dual
Controller,AC240HVDC,256GB
Cache,SPE63C0300&4*Disk Enclosure-
2U,AC240HVDC,2.5",DAE52525U2&36*960GB
SSD SAS Disk Unit(2.5"))-Hi-Care Onsite
Premier 24x7x4H Engineer Onsite Service-
36Month(s) 2 29,292.00 58,584.00 0% 58,584.00
88034JNY-88134UHK-36 Basic Software License(Including
DeviceManager,SmartThin,SmartMulti-
tenant,SmartMigration,SmartErase,SmartMotion,
SystemReporter,eService,SmartQuota,NFS,CIFS,
NDMP)-Hi-Care Application Software Upgrade
Support Service-36Month(s) 1 2,919.00 2,919.00 0% 2,919.00
8812153244 OceanStor 5600 V5 Installation Service -
Engineering 1 10,267.00 10,267.00 0% 10,267.00
71,770.00
446,024.48
1,100,252
405.39
26,124
17.08
SPC-1 Total System Price
SPC-1 ASU Capacity (GB)
SPC-1 ASU Price ($/GB)
SPC-1 IOPS
SPC-1 P ce-Pe a ce ($/SPC-1 KIOPS )
Hardware & Software
Hardware & Software Subtotal
Support & Maintenance
Support & Maintenance Subtotal
23. SPC-1 — Full Disclosure Report (1)
CONFIGURATION INFORMATION Page 13 of 42
SPC Benchmark 1 3. FULL DISCLOSURE REPORT Submission Identifier: A31020
Huawei Technologies Co., Ltd Submitted for Review: December 27, 2018
Huawei OCEANSTOR 5600 V5
CONFIGURATION INFORMATION
Benchmark Configuration and Tested Storage Configuration
The following diagram illustrates the Benchmark Configuration (BC), including the
Tested Storage Configuration (TSC) and the Host System(s).
Storage Network Configuration
The Tested Storage Configuration (TSC) involved an external storage subsystem
made of 4 Huawei OCEANSTOR 5600 V5, driven by 6 host systems (Huawei
BENCHMARK EXECUTION RESULTS Page 16 of 42
Overview
SPC Benchmark 1 3. FULL DISCLOSURE REPORT Submission Identifier: A31020
Huawei Technologies Co., Ltd Submitted for Review: December 27, 2018
Huawei OCEANSTOR 5600 V5
BENCHMARK EXECUTION RESULTS
This portion of the Full Disclosure Report documents the results of the various SPC-1 Tests,
Test Phases, and Test Runs.
Benchmark Execution Overview
Workload Generator Input Parameters
The SPC-1 Workload Generator commands and input parameters for the Test Phases
are presented in the Supporting Files (see Appendix A).
Primary Metrics Test Phases
The benchmark execution consists of the Primary Metrics Test Phases, including the
Test Phases SUSTAIN, RAMPD_100 to RAMPD_10, RAMPU_50 to RAMPU_100,
RAMP_0, REPEAT_1 and REPEAT_2.
Each Test Phase starts with a transition period followed by a Measurement Interval.
Measurement Intervals by Test Phase Graph
The following graph presents the average IOPS and the average Response Times
measured over the Measurement Interval (MI) of each Test Phase.
Exception and Waiver
None.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0
200,000
400,000
600,000
800,000
1,000,000
1,200,000
AverageMeasuredResponseTime(ms)
AverageMeasuredIOPS
Measurement Intervals by Test Phase Graph
IOPS Response Time
24. SPC-1 — Full Disclosure Report (2)
BENCHMARK EXECUTION RESULTS Page 24 of 42
Primary Metrics – Response Time Ramp Test
SPC Benchmark 1™ V3.8 FULL DISCLOSURE REPORT Submission Identifier: A31020
Huawei Technologies Co., Ltd Submitted for Review: December 27, 2018
Huawei OCEANSTOR 5600 V5
Response Time Ramp Test – Average Response Time Graph
Response Time Ramp Test – RAMPD_10 Response Time Graph
0.000
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
AverageMeasuredResponseTime(ms)
Average Response Time Graph (Response Time Ramp Test)
MI
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
0 2 4 6 8 10 12 14
ResponseTime(ms)
Relative Run Time (minutes)
Response Time Graph (RAMPD_10 @ 110,020 IOPS)
ASU1 ASU2 ASU3 All ASUs
BENCHMARK EXECUTION RESULTS Page 27 of 42
Repeatability Tests
SPC Benchmark 1 3. FULL DISCLOSURE REPORT Submission Identifier: A31020
Huawei Technologies Co., Ltd Submitted for Review: December 27, 2018
Huawei OCEANSTOR 5600 V5
REPEAT_2_100 – Response Time Graph
Repeatability Test – Intensity Multiplier
The following tables lists the targeted intensity multiplier (Defined), the measured
intensity multiplier (Measured) for each I/O STREAM, its coefficient of variation
(Variation) and the percent of difference (Difference) between Target and Measured.
REPEAT_1_100 Test Phase
ASU1-1 ASU1-2 ASU1-3 ASU1-4 ASU2-1 ASU2-2 ASU2-3 ASU3-1
Defined 0.0350 0.2810 0.0700 0.2100 0.0180 0.0700 0.0350 0.2810
Measured 0.0350 0.2810 0.0700 0.2100 0.0180 0.0700 0.0350 0.2810
Variation 0.0005 0.0002 0.0007 0.0003 0.0008 0.0005 0.0005 0.0001
Difference 0.002% 0.005% 0.010% 0.005% 0.025% 0.005% 0.015% 0.003%
REPEAT_2_100 Test Phase
ASU1-1 ASU1-2 ASU1-3 ASU1-4 ASU2-1 ASU2-2 ASU2-3 ASU3-1
Defined 0.0350 0.2810 0.0700 0.2100 0.0180 0.0700 0.0350 0.2810
Measured 0.0350 0.2810 0.0700 0.2100 0.0180 0.0700 0.0350 0.2810
Variation 0.0004 0.0002 0.0005 0.0002 0.0011 0.0003 0.0008 0.0002
Difference 0.043% 0.010% 0.016% 0.003% 0.045% 0.006% 0.011% 0.005%
MI
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
0 2 4 6 8 10 12 14
ResponseTime(ms)
Relative Run Time (minutes)
Response Time Graph (REPEAT_2_100 @ 1,100,200 IOPS)
ASU1 ASU2 ASU3 All ASUs
27. SPC-2 — Executive Summary
EXECUTIVE SUMMARY Page 4 of 9
C BE CH A K 2 V1.7.0 Executive Summary
Vexata Inc. Submitted: August 29, 2018
VX100-F Scalable NVMe Flash Array Submission ID: B12004
SPC-2 Reported Data
VX100-F Scalable NVMe Flash Array
SPC-2 MB
SPC-2 Price-
Performance
ASU Capacity (GB) Total Price
Data Protection
Level
49,042.39 $5.35 20,615.843 $262,572.59
Protected 1
(RAID 5 (N+1).)
The above SPC-2 MBPS al e e e en he agg ega e da a a e of all h ee SPC-2 workloads: Large File Processing (LFP), Large
Database Query (LDQ), and Video On Demand (VOD).
Currency Used: "Target Country":
U.S. Dollars USA
SPC-2 Large File Processing (LFP) Reported Data
Data Rate (MB/second) Number of Streams
Data Rate per
Stream
Price-Performance
LFP Composite 47,554.98 $5.52
Write Only:
1024 KiB Transfer 35,532.23 40 888.31
256 KiB Transfer 34,763.83 80 434.55
Read-Write:
1024 KiB Transfer 59,486.68 184 323.30
256 KiB Transfer 59,810.01 184 325.05
Read Only:
1024 KiB Transfer 48,190.46 184 261.90
256 KiB Transfer 47,546.68 184 258.41
The above SPC-2 Data Rate value for LFP Composite represents the aggregate performance of all three LFP Test Phases: (Write Only,
Read-Write, and Read Only).
SPC-2 Large Database Query (LDQ) Reported Data
Data Rate (MB/second) Number of Streams
Data Rate per
Stream
Price-Performance
LDQ Composite 49,869.23 $5.27
1024 KiB Transfer Size
4 I/Os Outstanding 50,425.48 32 1,575.80
1 I/O Outstanding 50,390.42 96 524.90
64 KiB Transfer Size
4 I/Os Outstanding 50,609.64 96 527.18
1 I/O Outstanding 48,051.39 320 150.16
The above SPC-2 Data Rate value for LDQ Composite represents the aggregate performance of the two LDQ Test Phases: (1024 KiB
and 64 KiB Transfer Sizes).
SPC-2 Video On Demand (VOD) Reported Data
Data Rate (MB/second) Number of Streams
Data Rate per
Stream
Price-Performance
49,702.97 63,200 0.79 $5.28
28. SPC-2 — Full Disclosure Report (1)
SPC-2 DATA REPOSITORY Page 22 of 61
C BE CH A K 2 V1.7.0 Full Disclosure Report
Vexata Inc. Submitted: August 29, 2018
VX100-F Scalable NVMe Flash Array Submission ID: B12004
Storage Hierarchy Ratios
Addressable
Storage
Capacity
Configured
Storage
Capacity
Physical
Storage
Capacity
Total ASU Capacity 100.00% 32.21% 32.21%
Data Protection (RAID 5 (N+1).) 2.18% 2.18%
Addressable Storage Capacity 32.21% 32.21%
Required Storage 37.45% 37.45%
Configured Storage Capacity 100.00%
Global Storage Overhead 0.00%
Unused Storage:
Addressable 0.00%
Configured 25.57%
Physical 0.00%
Storage Capacity Charts
GlobalStorage
Overhead:
0.000 GB (0.00%)
UnusedPhysical
Capacity:
0.000 GB (0.00%)
Data Capacity:
22,284.902 GB
(34.81%)
Data
Protection
Capacity:
1,392.806 GB
(2.18%)
SparingCapacity:
0.000 GB (0.00%)
Overhead&
Metadata:
23,970.195 GB
(37.45%)
ConfiguredStorage
Capacity:
64,013.113 GB
(100.00%)
Physical Storage Capacity: 64,013.113 GB
SPC-2 DATA REPOSITORY Page 23 of 61
C BE CH A K 2 V1.7.0 Full Disclosure Report
Vexata Inc. Submitted: August 29, 2018
VX100-F Scalable NVMe Flash Array Submission ID: B12004
Data Protection
Capacity:
1,392.806 GB (2.18%)
Spares:
0.000 GB (0.00%)
Overhead&
Metadata:
23,970.195 GB
(37.45%)
Addressable
Storage Capacity:
20,615.843 GB
(32.21%)
UnusedData Capacity:
1,669.059 GB (2.61%)
Data Capacity:
22,284.902 GB
(34.81%)
Configured Storage Capacity: 64,013.113 GB
ASU Capacity:
20,615.843 GB
(100.00%)
AddressableStorageCapacity: 20,615.843 GB
UnusedAddressable
Capacity:
0.000 GB (0.00%)
29. SPC-2 — Full Disclosure Report (2)
SPC-2 BENCHMARK EXECUTION RESULTS Page 29 of 61
Large File Processing Test
C BE CH A K 2 V1.7.0 Full Disclosure Report
Vexata Inc. Submitted: August 29, 2018
VX100-F Scalable NVMe Flash Array Submission ID: B12004
Average Data Rates (MB/s)
The average Data Rate (MB/s) for each Test Run in the three Test Phases of the SPC-2 Large File
Processing Test is listed in the table below as well as illustrated in the following graph.
Test Run Sequence 1 Stream
Variable
Streams
Variable
Streams
Variable
Streams
Variable
Streams
Write 1024KiB 2,326.81 9,508.26 17,833.15 31,292.20 35,532.23
Write 256KiB 1,020.06 8,946.47 16,496.39 27,368.93 34,763.83
Read/Write 1024KiB 1,427.00 24,075.32 40,743.49 55,172.27 59,486.68
Read/Write 256KiB 938.41 20,474.16 36,077.14 54,004.62 59,810.01
Read 1024KiB 1,671.50 23,246.57 35,676.06 45,920.89 48,190.46
Read 256KiB 1,181.65 20,775.14 33,564.49 45,945.52 47,546.68
1 Stream, 1,181.65 MB/s
1 Stream, 1,671.50 MB/s
1 Stream, 938.41 MB/s
1 Stream, 1,427.00 MB/s
1 Stream, 1,020.06 MB/s
1 Stream, 2,326.81 MB/s
23 Streams, 20,775.14 MB/s
23 Streams, 23,246.57 MB/s
23 Streams, 20,474.16 MB/s
23 Streams, 24,075.32 MB/s
10 Streams, 8,946.47 MB/s
5 Streams, 9,508.26 MB/s
46 Streams, 33,564.49 MB/s
46 Streams, 35,676.06 MB/s
46 Streams, 36,077.14 MB/s
46 Streams, 40,743.49 MB/s
20 Streams, 16,496.39 MB/s
10 Streams, 17,833.15 MB/s
92 Streams, 45,945.52 MB/s
92 Streams, 45,920.89 MB/s
92 Streams, 54,004.62 MB/s
92 Streams, 55,172.27 MB/s
40 Streams, 27,368.93 MB/s
20 Streams, 31,292.20 MB/s
184 Streams, 47,546.68 MB/s
184 Streams, 48,190.46 MB/s
184 Streams, 59,810.01 MB/s
184 Streams, 59,486.68 MB/s
80 Streams, 34,763.83 MB/s
40 Streams, 35,532.23 MB/s
0 10,000 20,000 30,000 40,000 50,000 60,000 70,000
256KiB transfers
with only
Read operations
1024KiB transfers
with only
Read operations
256KiB transfers
with
50% Read operations
50% Write operations
1024KiB transfers
with
50% Read operations
50% Write operations
256KiB transfers
with only
Write operations
1024KiB transfers
with only
Write operations
Data Rate, MB/sec
Large File Processing - Data Rate
SPC-2 BENCHMARK EXECUTION RESULTS Page 39 of 61
Large Database Query Test
C BE CH A K 2 V1.7.0 Full Disclosure Report
Vexata Inc. Submitted: August 29, 2018
VX100-F Scalable NVMe Flash Array Submission ID: B12004
Average Response Time
The average Response Time, in milliseconds, for each Test Run in the two Test Phases of the SPC-2 Large
Database Query Test is listed in the table below as well as illustrated in the following graph.
Test Run Sequence 1 Stream
Variable
Streams
Variable
Streams
Variable
Streams
Variable
Streams
1024KiB w/ 4 IOs/Stream 0.50 1.26 1.26 2.52 2.66
1024KiB w/ 1 IO/Stream 0.45 0.61 0.68 1.08 2.00
64KiB w/ 4 IOs/Stream 0.11 0.15 0.17 0.26 0.50
64KiB w/ 1 IO/Stream 0.11 0.12 0.15 0.22 0.44
1 Stream, 0.11 ms
1 Stream, 0.11 ms
1 Stream, 0.45 ms
1 Stream, 0.50 ms
40 Streams, 0.12 ms
12 Streams, 0.15 ms
12 Streams, 0.61 ms
4 Streams, 1.26 ms
80 Streams, 0.15 ms
24 Streams, 0.17 ms
24 Streams, 0.68 ms
8 Streams, 1.26 ms
160 Streams, 0.22 ms
48 Streams, 0.26 ms
48 Streams, 1.08 ms
16 Streams, 2.52 ms
320 Streams, 0.44 ms
96 Streams, 0.50 ms
96 Streams, 2.00 ms
32 Streams, 2.66 ms
0 1 1 2 2 3 3
64KiB transfers
with
1 IO outstanding
per Stream
64KiB transfers
with
4 IOs outstanding
per Stream
1024KiB transfers
with
1 IO outstanding
per Stream
1024KiB transfers
with
4 IOs outstanding
per Stream
Response Time, ms
Large Database Query - Average Response Time
30. SPC-1/2 Energy Extension
‣ kompletter Messzyklus ≥ 3 Tage
‣ Temperaturmessung
➛ am Anfang der Idle-Tests
➛ während letzter Minute Last-Test
‣ RMS ≙ quadratischer Mittelwert
31. SPC — Pricing (im Report)
‣ Hardware, Software, zusätzlich erforderliche Komponenten für
Storage-Funktionalität, 3 Jahre Support, alle anfallenden
Gebühren (Steuern, Zoll u. ä.)
‣ ausgenommen: HW für Benchmark-Setup ohne Storage-Funktion
➛ Server, die Workload erzeugen
➛ evtl. HBAs, FC-Switches, Verkabelung
➛ Fracht/Verpackung
‣ Projektpreise sind nicht erlaubt (»individually negotiated«)
➛ Wie aussagekräftig ist dann noch $/IOPS oder $/MBPS?
‣ Support ≙ 4h Response Time + 4h Vor-Ort-Service
➛ Vor-Ort = Ersatzteil und/oder Techniker
32. Agenda
‣ Warum überhaupt Benchmarks?
Aussagekraft, Herausforderungen, I/O-Stack, Tools
‣ Storage Performance Council
SPC-1, SPC-2, Industriestandard, Terminologie, Reports
‣Und was heißt das jetzt für’s Tagesgeschäft?
Benchmarks selber programmieren?
Benchmarks für die Beschaffung?
Glaskugel
33. SPC-1/2 selber bauen?
‣ ja, geht, und ist prinzipiell auch erlaubt
➛ aufwendig in der Entwicklung
➛ muss für offizielle Benchmarks von einem Auditor abgenommen werden
$ cat spc1-preflight.vdbench
***
*** vdbench Parameterfile to emulate SPC-1 workload
***
** storage definitions
**
sd=asu11,lun=/dev/rdsk/c25t2100000E1E19FB51d0s2
sd=asu12,lun=/dev/rdsk/c25t2100000E1E19FB51d13s2
sd=asu21,lun=/dev/rdsk/c26t2100000E1E19F170d32s2
sd=asu22,lun=/dev/rdsk/c26t2100000E1E19F240d68s2
sd=asu31,lun=/dev/rdsk/c27t2100000E1E19F5A1d29s2
sd=asu32,lun=/dev/rdsk/c27t2100000E1E19FB71d39s2
sd=asu41,lun=/dev/rdsk/c28t2100000E1E19F1B1d21s2
sd=asu42,lun=/dev/rdsk/c28t2100000E1E19F261d8s2
** workload definitions
**
wd=asu111,sd=asu11,rdpct=50,xfersize=4k,skew=1
wd=asu112,sd=asu11,rdpct=50,xfersize=4k,skew=6,range=(15,20)
wd=asu113,sd=asu11,rdpct=100,xfersize=(8k,40,16k,24,32k,20,64k,8,128k,8),skew=2,range=(40,50)
wd=asu114,sd=asu11,rdpct=50,xfersize=4k,skew=5,range=(70,75)
wd=asu121,sd=asu12,rdpct=30,xfersize=4k,skew=1
wd=asu122,sd=asu12,rdpct=30,xfersize=4k,skew=2,range=(47,52)
wd=asu123,sd=asu12,rdpct=100,xfersize=(8k,40,16k,24,32k,20,64k,8,128k,8),skew=1,range=(40,50)
wd=asu131,sd=asu13,rdpct=0,xfersize=(8k,40,16k,24,32k,20,64k,8,128k,8),skew=7,range=(35,65)
…
** run definition (raw I/O)
**
rd=spc1emu,wd=(asu111,asu112,asu113,asu114,asu121,…,),iorate=max,elapsed=300
34. Benchmarks in der Storage-Beschaffung?
‣ Wie genau kennen Sie die im Unternehmen benötigten
➛ IOPS und MBPS,
35. Benchmarks in der Storage-Beschaffung?
‣ Wie genau kennen Sie die im Unternehmen benötigten
➛ IOPS und MBPS,
➛ als Funktion der I/O-Blocksize,
36. Benchmarks in der Storage-Beschaffung?
‣ Wie genau kennen Sie die im Unternehmen benötigten
➛ IOPS und MBPS,
➛ als Funktion der I/O-Blocksize,
➛ mit der Verteilung nach read/write?
37. Benchmarks in der Storage-Beschaffung?
‣ Wie genau kennen Sie die im Unternehmen benötigten
➛ IOPS und MBPS,
➛ als Funktion der I/O-Blocksize,
➛ mit der Verteilung nach read/write?
Wie sind dann SPC-Werte zu bewerten?
38. Benchmarks in der Storage-Beschaffung?
‣ Wie genau kennen Sie die im Unternehmen benötigten
➛ IOPS und MBPS,
➛ als Funktion der I/O-Blocksize,
➛ mit der Verteilung nach read/write?
Wie sind dann SPC-Werte zu bewerten?
‣ keine allgemeingültigen IOPS-Muster
➛ »Fingerprint« des Unternehmens, abhängig von vielen Faktoren/Randbedingungen
‣ Herstellerdarstellung immer »so große Zahl als wie gehen tut«
➛ Anpassen der Benchmark-Optionen
‣ Lösungen?
➛ umfangreicher, lange laufender PoC (aufwendig)
➛ flexibles, in alle Richtungen skalierbares Storage (»Wollmilchsau«)
39. Glaskugel — Was bringt die Zukunft?
‣ Cloud Storage (public und private)
➛ viel Software involviert, mehrere Abstraktionslayer
➛ COSbench
‣ Solid State Memory (NAND Flash, Optane u. a.)
➛ keine »bremsende« Mechanik mehr
➛ Applikations-Debugging kann Thema werden, wenn plötzlich
Bottleneck ≠ Storage wird (z. B. komplexe oder »kaputte« SQL Queries)
Bootlenecks verschieben sich nur durch das System,
verschwinden aber nicht.
40. Quellen und »further learning«
‣ Spezifikationen zu den SPC-1/2 Benchmarks
https://spcresults.org/benchmarks
‣ Avishay Traeger et al.
A Nine Year Study of File System and Storage Benchmarking
https://www.fsl.cs.sunysb.edu/docs/fsbench/fsbench.pdf
‣ Brendan Gregg
Broken Linux Performance Tools
SCALE 14x, 2016
https://www.youtube.com/watch?v=OPio8V-z03c
‣ Raj Jain
The Art of Computer Systems Performance Analysis
John Wiley & Sons, Inc., 1991