Hadoop Meetup Jan 2019 - Hadoop Encryption

•Télécharger en tant que PPTX, PDF•

4 j'aime•490 vues

A presentation by Wei-Chiu Chuang of Cloudera regarding the state of Hadoop encryption, with a particular eye towards the Key Management Service (KMS). This is taken from the Apache Hadoop Contributors Meetup on January 30, hosted by LinkedIn in Mountain View.

Technologie

© Cloudera, Inc. All rights reserved.
Hadoop Encryption
Wei-Chiu Chuang, Cloudera

© Cloudera, Inc. All rights reserved.
Why Encryption
• Information leaks affect 10s to 100s of millions of people
• Personally identifiable information (PII)
• Credit cards, SSNs, account logins
• Encryption would have prevented some of these leaks
• Encryption is a regulatory requirement for many business sectors
• Finance (PCI DSS)
• Government (Data Protection Directive)
• Healthcare (DPD, HIPAA)

© Cloudera, Inc. All rights reserved.
Related Technologies
Data In-Motion Encryption
• SSL/TLS
• Hadoop Data Transfer Encryption
• Hadoop RPC Encryption
Data At-Rest Encryption
• At Linux volume level
• Transparent Encryption at HDFS level
• HBase Column Family level
• Parquet Column level Encryption Xinli

© Cloudera, Inc. All rights reserved.
HDFS Transparent Encryption: In a Nutshell
HDFS
Namespace
/
/data /tmp
/data/1 /data/f2
Encryption
zone
Encryption
zone key Data Encryption
Key (per file)

© Cloudera, Inc. All rights reserved.
HDFS Transparent Encryption: In a Nutshell
Client
KMS
MS
NN
DN
NameNode
DataNode
Key
Management
Server
in EZ?

© Cloudera, Inc. All rights reserved.
Features
• Minor performance impact on HDFS reads and writes
• OpenSSL and AES-NI acceleration
• 7.5% for reads, ~0% for writes
• Key ACLs
• Warm-up/Caching (*)
• Key rollover

© Cloudera, Inc. All rights reserved.
Dev History
• First released in Hadoop 2.6.0/ CDH5.3 in 2014 December
• Many, many bug fixes and enhancements
• Functional bugs, failure handling bugs, scale bugs
• Stable after Hadoop 2.8 / CDH5.11-ish

© Cloudera, Inc. All rights reserved.
Lesson Learned
Scale-out is not easy to deploy
Security
Endurance, scale tests are essential
Too little emphasis on KMS as a performance bottleneck
FileSystem#getDelegationToken() API/integration
High throughput REST API layer is hard

© Cloudera, Inc. All rights reserved.
Status Quo
Among Cloudera’s customers (pre-merger):
• 14% Data Transfer Encryption
• 16% Data at Rest Encryption
• 19% RPC Encryption
• 44% Kerberized
Largest at-rest encryption cluster: ~1,000 nodes, > 50PB

© Cloudera, Inc. All rights reserved.
Troubleshooting
Performance anomaly
• Openssl-devel lib
• Entropy
• rng-tools
• Secure Random
• hadoop.security.secure.random.impl = org.apache.hadoop.crypto.random
.OpensslSecureRandom
Proxy user configuration

© Cloudera, Inc. All rights reserved.
Bad Practices
• No KMS HA
• KMS enabled, RPC encryption not enabled
• KMS enabled, but no Kerberos
• KMS w/o SSL
• Data transfer encryption is enabled, but using an unoptimized crypto algorithm
• 3DES, RC4, AES-NI

© Cloudera, Inc. All rights reserved.
Challenges
KMS Low Throughput
• NN can sustain > 100 thousand RPC ops/second
• namespace ops, block reports
• KMS: at most a few thousand RPC ops/second
• create, append, read, reencrypt
• 3-4 KMS servers not uncommon
• Jetty
• SSL Handshake
• Impala/Parquet with wide tables (> 100 columns)

© Cloudera, Inc. All rights reserved.
Future
Pluggable KMS ACL Framework (HADOOP-14951)
WebHDFS At Rest Encryption Support (HDFS-12355)
NFS Gateway At Rest Encryption Support (HDFS-13521)
Performance Improvements (HADOOP-15743, HADOOP-15811)
KMS Benchmark Tool (HADOOP-15967)
KMS over Hadoop RPC?

© Cloudera, Inc. All rights reserved.
●Current KMS Transport Layer
KMSClient
Jetty
http
client
Name
Node
http
client
REST API/HTTP

© Cloudera, Inc. All rights reserved.
KMS over Hadoop RPC
Benefit of KMS over Hadoop RPC:
• Proven performance
• Code reuse
KMSClient
Name
Node
Hadoop RPCHadoop
RPC
Hadoop
RPC
Hadoop
RPC

Contenu connexe

Tendances

Whilst HBase is the most logical answer for use cases requiring random, realtime read/write access to Big Data, it may not be so trivial to design applications that make most of its use, neither the most simple to operate. As it depends/integrates with other components from Hadoop ecosystem (Zookeeper, HDFS, Spark, Hive, etc) or external systems ( Kerberos, LDAP), and its distributed nature requires a "Swiss clockwork" infrastructure, many variables are to be considered when observing anomalies or even outages. Adding to the equation there's also the fact that HBase is still an evolving product, with different release versions being used currently, some of those can carry genuine software bugs. On this presentation, we'll go through the most common HBase issues faced by different organisations, describing identified cause and resolution action over my last 5 years supporting HBase to our heterogeneous customer base.

HBase Tales From the Trenches - Short stories about most common HBase operati...

DataWorks Summit

Ceph as software define storage

Mahmoud Shiri Varamini

Red Hat Ceph Storage Acceleration Utilizing Flash Technology

Red_Hat_Storage

Red Hat Storage 2014 - Product(s) Overview

Marcel Hergaarden

Redis vs Memcached

Gaurav Agrawal

CEPH DAY BERLIN - UNLIMITED FILESERVER WITH SAMBA CTDB AND CEPHFS

Ceph Community

Troubleshooting redis

DaeMyung Kang

Glusterfs and openstack

openstackindia

Red Hat Storage, based on the upstream GlusterFS project, was developed as a distributed file system in the oil-and-gas, high-performance compute arena. With Red Hat Storage, you can easily set up flexible distributed storage using commodity x86 hardware. Simple, inexpensive internal or JBOD storage can be linked across multiple physical servers and presented as a single storage namespace. This storage can be used for log files, web content, virtual machine, home directory, and other storage use cases. In this session, we’ll demonstrate how to: Install Red Hat Gluster Storage Configure disks Link the storage nodes Define storage bricks Present storage to clients We'll talk about tips and tricks, best practices, backup and recovery, and other storage-related topics.

Red Hat Storage for Mere Mortals

Red_Hat_Storage

Building Scalable, Real Time Applications for Financial Services with DataStax

DataStax

Deploying Ceph can be a hassle. And once it's up and running it can be time consuming to add servers: install an operating system, configure the network, configure Ceph, ...|But that doesn't have to be that hard.|We've build a full Ceph management suite to help you with these tasks.|With croit, you just need to plug in your server and our zero touch provisioning takes care of the rest. croit boots a customized Linux image that is ready to be used directly from memory: no installation necessary!||We'll show a live demo, deploying a cluster from scratch and performing a few typical admin tasks; all without using the command line at all. In addition, we show you how croit can help you save a lot of money.

CEPH DAY BERLIN - CEPH MANAGEMENT THE EASY AND RELIABLE WAY

Ceph Community

Dustin Black - Red Hat Storage Server Administration Deep Dive

Gluster.org

Many individuals/organizations have a desire to utilize NoSQL technology, but often lack an understanding of how the underlying functional bits can be utilized to enable their use case. This situation can result in drastic increases in the desire to put the SQL back in NoSQL. Since the initial commit, Apache Accumulo has provided a number of examples to help jumpstart comprehension of how some of these bits function as well as potentially help tease out an understanding of how they might be applied to a NoSQL friendly use case. One very relatable example demonstrates how Accumulo could be used to emulate a filesystem (dirlist). In this session we will walk through the dirlist implementation. Attendees should come away with an understanding of the supporting table designs, a simple text search supporting a single wildcard (on file/directory names), and how the dirlist elements work together to accomplish its feature set. Attendees should (hopefully) also come away with a justification for sometimes keeping the SQL out of NoSQL.

Practical NoSQL: Accumulo's dirlist Example

DataWorks Summit

Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...

Red_Hat_Storage

Ceph Introduction 2017

Karan Singh

Xiaomi is a Chinese technology company, it sells more than 100 million smartphones worldwide in 2018, and also owns one of the world's largest IoT device platforms. Xiaomi builds dozens of mobile apps and Internet services based on intelligent devices, including Ads, news feeds, finance service, game, music, video, personal cloud service and so on. The rapid growth of business results in exponential growth of the data analytics infrastructure. The amount of data has roared more than 20 times in the past 3 years, which renders us big challenges on the HDFS scalability In this talk, we introduce how we scale HDFS to support hundreds of PB data with thousands nodes: 1. How Xiaomi use Hadoop and the characteristic of our usage 2. We made HDFS federation cluster to be used like a single cluster, most applications don't need to change any code to migrate from a single cluster to a federation cluster. Our works include a wrapper FileSystem compatible with DistributedFileSystem, supporting rename among different name spaces and zookeeper-based mount table renewer. 3. Experience of tuning NameNode to improve scalability 4. How to maintain hundreds of HDFS clusters and the optimization we did on client-side to make user and programs access these clusters easily with high performance

Scaling HDFS at Xiaomi

DataWorks Summit

"In this session for administrators of all skill levels, you’ll get a deep technical dive into Red Hat Storage Server and GlusterFS administration. We’ll start with the basics of what scale-out storage is, and learn about the unique implementation of Red Hat Storage Server and its advantages over legacy and competing technologies. From the basic knowledge and design principles, we’ll move to a live start-to-finish demonstration. Your experience will include: Building a cluster. Allocating resources. Creating and modifying volumes of different types. Accessing data via multiple client protocols. A resiliency demonstration. Expanding and contracting volumes. Implementing directory quotas. Recovering from and preventing split-brain. Asynchronous parallel geo-replication. Behind-the-curtain views of configuration files and logs. Extended attributes used by GlusterFS. Performance tuning basics. New and upcoming feature demonstrations. Those new to the scale-out product will leave this session with the knowledge and confidence to set up their first Red Hat Storage Server environment. Experienced administrators will sharpen their skills and gain insights into the newest features. IT executives and managers will gain a valuable overview to help fuel the drive for next-generation infrastructures."

Red Hat Storage Server Administration Deep Dive

Red_Hat_Storage

Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong

Ceph Community

Experiences building a distributed shared log on RADOS - Noah Watkins

Ceph Community

What's New with Ceph - Ceph Day Silicon Valley

Ceph Community

Tendances (20)

HBase Tales From the Trenches - Short stories about most common HBase operati...

Ceph as software define storage

Red Hat Ceph Storage Acceleration Utilizing Flash Technology

Red Hat Storage 2014 - Product(s) Overview

Redis vs Memcached

CEPH DAY BERLIN - UNLIMITED FILESERVER WITH SAMBA CTDB AND CEPHFS

Troubleshooting redis

Glusterfs and openstack

Red Hat Storage for Mere Mortals

Building Scalable, Real Time Applications for Financial Services with DataStax

CEPH DAY BERLIN - CEPH MANAGEMENT THE EASY AND RELIABLE WAY

Dustin Black - Red Hat Storage Server Administration Deep Dive

Practical NoSQL: Accumulo's dirlist Example

Red Hat Storage Day Dallas - Red Hat Ceph Storage Acceleration Utilizing Flas...

Ceph Introduction 2017

Scaling HDFS at Xiaomi

Red Hat Storage Server Administration Deep Dive

Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong

Experiences building a distributed shared log on RADOS - Noah Watkins

What's New with Ceph - Ceph Day Silicon Valley

Similaire à Hadoop Meetup Jan 2019 - Hadoop Encryption

Project Rhino: Enhancing Data Protection for Hadoop

Cloudera, Inc.

Securing Spark Applications

DataWorks Summit/Hadoop Summit

Big Data is an increasingly powerful enterprise asset and this talk will explore the relationship between big data and cyber security, how we preserve privacy whilst exploiting the advantages of data collection and processing. Big Data technologies provide both governments and corporations powerful tools to offer more efficient and personalized services. The rapid adoption of these technologies has of course created tremendous social benefits. Unfortunately unwanted side effects are the potential rich pickings available to those with malicious intentions. Increasingly, the sophisticated cyber attacker is able to exploit the rich array public data to build detailed profiles on their adversaries to support their malicious intentions

Fighting cyber fraud with hadoop

Niel Dunnage

PCI Compliane With Hadoop

Rommel Garcia

The Future of Data Management - the Enterprise Data Hub

DataWorks Summit

Hadoop deployments are rapidly moving from pilots to production, enabling unprecedented opportunity to build big data applications that deliver faster access to more information to more users than ever before possible. Yet without the ability to address data security and compliance regulations, Hadoop will be limited to another data silo. In this talk, Matt Brandwein and David Tishgart discuss the requirements for securing Hadoop and how Cloudera (now with Gazzang) and Intel are collaborating in the open to deliver comprehensive, transparent, compliance-ready security to unlock the potential of the Hadoop ecosystem and enable innovation without compromise.

The Future of Hadoop Security - Hadoop Summit 2014

Cloudera, Inc.

Hadoop security @ Philly Hadoop Meetup May 2015

Shravan (Sean) Pabba

Cloudera Tech Day Presentation by Eddie Garcia, Chief Security Architect, Cloudera. Protecting enterprise data is an increasingly complex challenge given the diversity and sophistication of threat actors and their cyber-tactics. In this session, participants will hear a comprehensive introduction to Hadoop Security, including the “three A’s” for secure operating environments: Authentication, Authorization, and Audit. In addition, the presenter will cover strategies to orchestrate data security, encryption, and compliance, and will explain the Cloudera Security Maturity Model for Hadoop. Attendees will leave with a greater understanding of how effective INFOSEC relies on an enterprise big data governance and risk management approach.

Risk Management for Data: Secured and Governed

Cloudera, Inc.

This session shows how to secure different Big Data sensitive data items such as log files, metastore databases, control files, config files, data directories or data files for different Big Data technologies. As Hadoop, MongoDB, Cassandra and other massively distributed Big Data stores grow in popularity, so too does the volume of sensitive regulatory data that gets captured for analysis. Cloudera Navigator Encrypt gives peace of mind, knowing the sensitive information used to run massive-scale queries and analytics is secure. Navigator Encrypt works as a last line of defense for protecting data, by providing a transparent layer between the application and file system and securing information as it gets written to disk, ensuring minimal performance lag in the encryption or decryption process. The solution also includes robust key management and process-based access controls, while simultaneously preventing admins or super users like root from accessing data that they don’t need to see allowing users to store their cryptographic keys separate from the encrypted data. Session presented at Big Data Spain 2015 Conference 15th Oct 2015 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Abstract: http://www.bigdataspain.org/program/thu/slot-13.html

Securing Big Data at rest with encryption for Hadoop, Cassandra and MongoDB o...

Big Data Spain

Hadoop Distributed File System (HDFS) Encryption with Cloudera Navigator Key ...

Cloudera, Inc.

Big data security challenges are bit different from traditional client-server applications and are distributed in nature, introducing unique security vulnerabilities. Cloud Security Alliance (CSA) has categorized the different security and privacy challenges into four different aspects of the big data ecosystem. These aspects are infrastructure security, data privacy, data management and, integrity and reactive security. Each of these aspects are further divided into following security challenges: 1. Infrastructure security a. Secure distributed processing of data b. Security best practices for non-relational data stores 2. Data privacy a. Privacy-preserving analytics b. Cryptographic technologies for big data c. Granular access control 3. Data management a. Secure data storage and transaction logs b. Granular audits c. Data provenance 4. Integrity and reactive security a. Endpoint input validation/filtering b. Real-time security/compliance monitoring In this talk, we are going to refer above classification and identify existing security controls, best practices, and guidelines. We will also paint a big picture about how collective usage of all discussed security controls (Kerberos, TDE, LDAP, SSO, SSL/TLS, Apache Knox, Apache Ranger, Apache Atlas, Ambari Infra, etc.) can address fundamental security and privacy challenges that encompass the entire Hadoop ecosystem. We will also discuss briefly recent security incidents involving Hadoop systems. Speakers Krishna Pandey, Staff Software Engineer, Hortonworks Kunal Rajguru, Premier Support Engineer, Hortonworks

Saving the elephant—now, not later

DataWorks Summit

Transparent Encryption in HDFS

DataWorks Summit

BigData Security - A Point of View

Karan Alang

Hadoop security implementationon 20171003

lee tracie

Security implementation on hadoop

Wei-Chiu Chuang

Data is being generated at a feverish pace and many businesses want all of it at their disposal to solve complex strategic problems. As decision making moves to real-time, enterprises need data ready for analysis immediately. Sean Anderson and Amandeep Khurana will discuss common pipeline trends in modern streaming architectures, Hadoop components that enable streaming capabilities, and popular use cases that are enabling the world of IOT and real-time data science.

End to End Streaming Architectures

Cloudera, Inc.

Open Source Security Tools for Big Data

Rommel Garcia

Open Source Security Tools for Big Data

Great Wide Open

Owasp Indy Q2 2012 Cheat Sheet Overview

owaspindy

Big data security

Joey Echeverria

Similaire à Hadoop Meetup Jan 2019 - Hadoop Encryption (20)

Project Rhino: Enhancing Data Protection for Hadoop

Securing Spark Applications

Fighting cyber fraud with hadoop

PCI Compliane With Hadoop

The Future of Data Management - the Enterprise Data Hub

The Future of Hadoop Security - Hadoop Summit 2014

Hadoop security @ Philly Hadoop Meetup May 2015

Risk Management for Data: Secured and Governed

Securing Big Data at rest with encryption for Hadoop, Cassandra and MongoDB o...

Hadoop Distributed File System (HDFS) Encryption with Cloudera Navigator Key ...

Saving the elephant—now, not later

Transparent Encryption in HDFS

BigData Security - A Point of View

Hadoop security implementationon 20171003

Security implementation on hadoop

End to End Streaming Architectures

Open Source Security Tools for Big Data

Owasp Indy Q2 2012 Cheat Sheet Overview

Big data security

Dernier

Sara Mae O’Brien Scott and Tatiana Baquero Cakici, Senior Consultants at Enterprise Knowledge (EK), presented “AI Fast Track to Search-Focused AI Solutions” at the Information Architecture Conference (IAC24) that took place on April 11, 2024 in Seattle, WA. In their presentation, O’Brien-Scott and Cakici focused on what Enterprise AI is, why it is important, and what it takes to empower organizations to get started on a search-based AI journey and stay on track. The presentation explored the complexities of enterprise search challenges and how IA principles can be leveraged to provide AI solutions through the use of a semantic layer. O’Brien-Scott and Cakici showcased a case study where a taxonomy, an ontology, and a knowledge graph were used to structure content at a healthcare workforce solutions organization, providing personalized content recommendations and increasing content findability. In this session, participants gained insights about the following: Most common types of AI categories and use cases; Recommended steps to design and implement taxonomies and ontologies, ensuring they evolve effectively and support the organization’s search objectives; Taxonomy and ontology design considerations and best practices; Real-world AI applications that illustrated the value of taxonomies, ontologies, and knowledge graphs; and Tools, roles, and skills to design and implement AI-powered search solutions.

IAC 2024 - IA Fast Track to Search Focused AI Solutions

Enterprise Knowledge

How to convert PDF to text with Nanonets

naman860154

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Heather Hedden, Senior Consultant at Enterprise Knowledge, presented “The Role of Taxonomy and Ontology in Semantic Layers” at a webinar hosted by Progress Semaphore on April 16, 2024. Taxonomies at their core enable effective tagging and retrieval of content, and combined with ontologies they extend to the management and understanding of related data. There are even greater benefits of taxonomies and ontologies to enhance your enterprise information architecture when applying them to a semantic layer. A survey by DBP-Institute found that enterprises using a semantic layer see their business outcomes improve by four times, while reducing their data and analytics costs. Extending taxonomies to a semantic layer can be a game-changing solution, allowing you to connect information silos, alleviate knowledge gaps, and derive new insights. Hedden, who specializes in taxonomy design and implementation, presented how the value of taxonomies shouldn’t reside in silos but be integrated with ontologies into a semantic layer. Learn about: - The essence and purpose of taxonomies and ontologies in information and knowledge management; - Advantages of semantic layers leveraging organizational taxonomies; and - Components and approaches to creating a semantic layer, including the integration of taxonomies and ontologies

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Delhi Call girls

Scaling API-first – The story of a global engineering organization

Radu Cotescu

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

A Call to Action for Generative AI in 2024

Results

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Delhi Call girls

Dernier (20)

IAC 2024 - IA Fast Track to Search Focused AI Solutions

How to convert PDF to text with Nanonets

Automating Google Workspace (GWS) & more with Apps Script

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Boost PC performance: How more available memory can improve productivity

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

A Year of the Servo Reboot: Where Are We Now?

Powerful Google developer tools for immediate impact! (2023-24 C)

Boost Fertility New Invention Ups Success Rates.pdf

[2024]Digital Global Overview Report 2024 Meltwater.pdf

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Scaling API-first – The story of a global engineering organization

Exploring the Future Potential of AI-Enabled Smartphone Processors

How to Troubleshoot Apps for the Modern Connected Worker

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Data Cloud, More than a CDP by Matt Robison

A Call to Action for Generative AI in 2024

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Hadoop Meetup Jan 2019 - Hadoop Encryption

2. © Cloudera, Inc. All rights reserved. Why Encryption • Information leaks affect 10s to 100s of millions of people • Personally identifiable information (PII) • Credit cards, SSNs, account logins • Encryption would have prevented some of these leaks • Encryption is a regulatory requirement for many business sectors • Finance (PCI DSS) • Government (Data Protection Directive) • Healthcare (DPD, HIPAA)

3. © Cloudera, Inc. All rights reserved. Related Technologies Data In-Motion Encryption • SSL/TLS • Hadoop Data Transfer Encryption • Hadoop RPC Encryption Data At-Rest Encryption • At Linux volume level • Transparent Encryption at HDFS level • HBase Column Family level • Parquet Column level Encryption Xinli

6. © Cloudera, Inc. All rights reserved. Features • Minor performance impact on HDFS reads and writes • OpenSSL and AES-NI acceleration • 7.5% for reads, ~0% for writes • Key ACLs • Warm-up/Caching (*) • Key rollover

7. © Cloudera, Inc. All rights reserved. Dev History • First released in Hadoop 2.6.0/ CDH5.3 in 2014 December • Many, many bug fixes and enhancements • Functional bugs, failure handling bugs, scale bugs • Stable after Hadoop 2.8 / CDH5.11-ish

8. © Cloudera, Inc. All rights reserved. Lesson Learned Scale-out is not easy to deploy Security Endurance, scale tests are essential Too little emphasis on KMS as a performance bottleneck FileSystem#getDelegationToken() API/integration High throughput REST API layer is hard

9. © Cloudera, Inc. All rights reserved. Status Quo Among Cloudera’s customers (pre-merger): • 14% Data Transfer Encryption • 16% Data at Rest Encryption • 19% RPC Encryption • 44% Kerberized Largest at-rest encryption cluster: ~1,000 nodes, > 50PB

10. © Cloudera, Inc. All rights reserved. Troubleshooting Performance anomaly • Openssl-devel lib • Entropy • rng-tools • Secure Random • hadoop.security.secure.random.impl = org.apache.hadoop.crypto.random .OpensslSecureRandom Proxy user configuration

11. © Cloudera, Inc. All rights reserved. Bad Practices • No KMS HA • KMS enabled, RPC encryption not enabled • KMS enabled, but no Kerberos • KMS w/o SSL • Data transfer encryption is enabled, but using an unoptimized crypto algorithm • 3DES, RC4, AES-NI

12. © Cloudera, Inc. All rights reserved. Challenges KMS Low Throughput • NN can sustain > 100 thousand RPC ops/second • namespace ops, block reports • KMS: at most a few thousand RPC ops/second • create, append, read, reencrypt • 3-4 KMS servers not uncommon • Jetty • SSL Handshake • Impala/Parquet with wide tables (> 100 columns)

13. © Cloudera, Inc. All rights reserved. Future Pluggable KMS ACL Framework (HADOOP-14951) WebHDFS At Rest Encryption Support (HDFS-12355) NFS Gateway At Rest Encryption Support (HDFS-13521) Performance Improvements (HADOOP-15743, HADOOP-15811) KMS Benchmark Tool (HADOOP-15967) KMS over Hadoop RPC?

Notes de l'éditeur

Caching improves performance. But some users’ environment prohibit caching due to security concerns.
KMS was designed to be horizontally scalable. However, because Cloudera recommend 2 KMS-HA and 2 Keytrustee Servers for production workload, the cost for HA is high.

Hadoop Meetup Jan 2019 - Hadoop Encryption

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Hadoop Meetup Jan 2019 - Hadoop Encryption

Similaire à Hadoop Meetup Jan 2019 - Hadoop Encryption (20)

Dernier

Dernier (20)

Hadoop Meetup Jan 2019 - Hadoop Encryption

Notes de l'éditeur