High order bits from cassandra & hadoop

•Télécharger en tant que PPTX, PDF•

0 j'aime•932 vues

srisatish ambati

Technologie

Thank You! svccg in first page of search results for “cloud” on google!

points Usecases Why cassandra? Usecase: Hadoop, Brisk FUD:Consistency Why facebook is not using Cassandra? Anti-patterns Community, Code, Tools Q&A

Users. Netflix. Key by Customer, read-heavy Key by Customer:Movie, write-heavy

TimeSeries: (several customers) periodic readings: dev0, dev1…deviceID:metric:timestamp ->value Metrics typically way larger dataset than users.

Replication: Multi-datacenter Multi-region ec2 Multi-availability zones

reads local dc1 dc2 Replication: Multi-datacenter Multi-region ec2, aws Multi-availability zones

4.21.2011, Amazon Web Services outage: “Movie marathons on Netflix awaiting AWS to come back up.” #ec2disabled

4.21.2011, Amazon Web Services outage: Netflix was running on AWS.

Writes Sequential, append-only. ~1-5ms On cloud: ephemeral disks rock!

Reads Local Key & row caches, (also, jna-based 0xffheap) indexes, materialized

Reads Local Key & row caches, (also, jna-based 0xffheap) indexes, materialized ssds: improved read performance!

Distribution between nodes Gossip Anti-entropy Failure-detector L i g h t w e i g h t

Clients: cql, thrift pycassa, phpcassa hector, pelops (scala, ruby, clojure)

Usecase #3: hadoop Hdfs cassandra hive Logs stats analytics

jobtracker, tasktracker hdfs: namenode, datanode

cloudera amazon: elastic map reduce hortonworks mapR brisk

Tools & Analytics Hive, Pig, R Karmasphere Datameer … dozens of stealth startups!

Use column families (tables) inode sblock

near-real time hadoop Low latency: cassandra_dc nodes Batch Analytics: brisk_dc nodes

Consistency: R + W > N ORACLE, 2-node: R=1, W=2, N=2,(T=2) DNS * N is replication factor. Not to be confused with T=total #of nodes

Tune-able, flexibility. For High Consistency: read:quorum, write:quorum For High Availability: high W, low R.

Inbox Search: 600+cores.120+TB (2008) Went from 100-500m users. Average NoSQL deployment size: ~6-12 nodes.

Usecase #5: search Apache Solr + Cassandra = Solandra Other inbox/file Searches: xobni, c3 github.com/tjake/solandra

“Eventual consistency is harder to program.” mostly immutable data. complex systems at scale.

Miscellaneous, Myth: data-loss, partial rows. writes are durable.

Anti-Patterns Transactions Joins Read before write

Anti-Patterns for cloud ebs jvm, virtualized single region

Tools AMIs, OpsCenter, DataStax AppDynamics Netflix just builds AMIs for deployment!

B e a u t i f u l C 0 d e = new code(); //less is more ~90k.java.concurrent.@annotate. bloomfilters, merkletrees. non-blocking, staged-event-driven. bigtable, dynamo.

Current & Future Focus: Distributed Counters, CQL. Simple client. operational smoothening. compaction.

Community Robust. Rapid. # Professional support from DataStax. Filesysteminnovatin from Acunu engineers: independent,startups, large companies, Rackspace, Twitter, Netflix.. Come join the efforts!

Usecase #4: first NoSQL, then scale! simpledb Cassandra mongodb Cassandra

Copyright: plantoys … more than one way to do it!

Summary - high scale peer-to-peer datastore best friend for multi-region, multi-zone availability. Hadoop – HDFS engulfing the DataWorld

Contenu connexe

Tendances

The Automation Factory

Nathan Milford

We built an application based on the principles of CQRS and Event Sourcing using Cassandra and Spark. During the project we encountered a number of challenges and problems with Cassandra and the Spark Connector. In this talk we want to outline a few of those problems and our actions to solve them. While some problems are specific to CQRS and Event Sourcing applications most of them are use case independent. About the Speakers Matthias Niehoff IT-Consultant, codecentric AG works as an IT-Consultant at codecentric AG in Germany. His focus is on big data & streaming applications with Apache Cassandra & Apache Spark. Yet he does not lose track of other tools in the area of big data. Matthias shares his experiences on conferences, meetups and usergroups. Stephan Kepser Senior IT Consultant and Data Architect, codecentric AG Dr. Stephan Kepser is an expert on cloud computing and big data. He wrote a couple of journal articles and blog posts on subjects of both fields. His interests reach from legal questions to questions of architecture and design of cloud computing and big data systems to technical details of NoSQL databases.

Lessons from Cassandra & Spark (Matthias Niehoff & Stephan Kepser, codecentri...

DataStax

Spark application on ec2 cluster

Chao-Hsuan Shen

Cassandra CLuster Management by Japan Cassandra Community

Hiromitsu Komatsu

SSTable Reader Cassandra Day Denver 2014

Ben Vanberg

After a brief technical introduction to Apache Cassandra we'll then go into the exciting world of Apache Spark integration, and learn how you can turn your transactional datastore into an analytics platform. Apache Spark has taken the Hadoop world by storm (no pun intended!), and is widely seen as the replacement to Hadoop Map Reduce. Apache Spark coupled with Cassandra are perfect allies, Cassandra does the distributed data storage, Spark does the distributed computation.

Big Data Day LA 2015 - Sparking up your Cassandra Cluster- Analytics made Awe...

Data Con LA

Spark on Mesos-A Deep Dive-(Dean Wampler and Tim Chen, Typesafe and Mesosphere)

Spark Summit

Traditionally, machines were statically partitioned across the different services at Uber. In an effort to increase the machine utilization, Uber has recently started transitioning most of its services, including the storage services, to run on top of Mesos. This presentation will describe the initial experience building and operating a framework for running Cassandra on top of Mesos running across multiple datacenters at Uber. This framework automates several Cassandra operations such as node repairs, addition of new nodes and backup/restore. It improves efficiency by co-locating CPU-intensive services as well as multiple Cassandra nodes on the same Mesos agent. It handles failure and restart of Mesos agents by using persistent volumes and dynamic reservations. This talk includes statistics about the number of Cassandra clusters in production, time taken to start a new cluster, add a new node, detect a node failure; and the observed Cassandra query throughput and latency. About the Speaker Abhishek Verma Software Engineer, Uber Dr. Abhishek Verma is currently working on running Cassandra on top of Mesos at Uber. Prior to this, he worked on BorgMaster at Google and was the first author of the Borg paper published in Eurosys 2015. He received an MS in 2010 and a PhD in 2012 in Computer Science from the University of Illinois at Urbana-Champaign, during which he authored more than 20 publications in conferences, journals and books and presented tens of talks.

Cassandra on Mesos Across Multiple Datacenters at Uber (Abhishek Verma) | C* ...

DataStax

With the addition of vnodes (Virtual Nodes), Cassandra users were able to gain a few benefits as a result of streaming when it came to bootstrapping and decommissioning nodes. On the flip side, having to route requests on larger clusters became a lot more intensive of a workload for all nodes that were then forced to act coordinator nodes. By setting up a tier of proxy nodes, we were able to have our cluster of 50 nodes perform with a 300% improvement on average in a mixed workload environment. This is an explanation of what we did, how we did it, and why it works. About the Speaker Eric Lubow CTO, SimpleReach Eric Lubow is CTO of SimpleReach, where he builds highly-scalable distributed systems for processing analytics data. Eric is also a DataStax MVP for Cassandra, and co-author of Practical Cassandra. In his spare time, Eric is a skydiver, motorcycle rider, mixed martial artist, and dog dad.

Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...

DataStax

Instaclustr Webinar 50,000 Transactions Per Second with Apache Spark on Apach...

Instaclustr

Learn Cassandra at edureka!

Edureka!

Introduction to Cassandra

Gokhan Atil

Instaclustr webinar 2017 feb 08 japan

Hiromitsu Komatsu

Up and running with pyspark

Krishna Sangeeth KS

Scylla db deck, july 2017

Dor Laor

Introduction to Cassandra: Replication and Consistency

Benjamin Black

Over the past year data at GumGum has quadrupled. Now a days we process 20 TB of new data every day. New data/reporting requirements pour in every week. Usage of the real time data is growing with the daily data. In this talk we are tying to answer the following questions: How do we serve real time data with daily batched data to our consumers together? What is Lambda Architecture and how does it help? What role does Cassandra play in Lambda Architecture at GumGum? How did we solve few bottlenecks in the architecture using Cassandra? How Cassandra can help you avoid microbatching and give you a true realtime data? About the Speaker Vaibhav Puranik VP of Engineering, Big Data & Platform, GumGum Vaibhav has 15 years of experience in Software. He began his career with Johnson Space Center in Houston and has a masters degree in computer science. For past 5 years Vaibhav has been responsible for architecting multiple big data systems at GumGum. He manages Data Science, Data Engineering and DevOps teams at GumGum.

Lambda Architecture with Cassandra (Vaibhav Puranik, GumGum) | C* Summit 2016

DataStax

Time to Science, Time to Results. Accelerating Scientific research in the Cloud

Amazon Web Services

From DataFrames to Tungsten: A Peek into Spark's Future-(Reynold Xin, Databri...

Spark Summit

Apache Cassandra is a popular choice for a wide variety of application persistence needs. There are many design choices that can effect uptime and performance. In this talk we'll look at some of the many things to consider from a single server to multiple data centers. Basic understanding of Cassandra features coupled with client driver features can be a very powerful combination. This talk will be an introduction but will deep dive into the technical details of how Cassandra works.

Introduction to cassandra 2014

Patrick McFadin

Tendances (20)

The Automation Factory

Lessons from Cassandra & Spark (Matthias Niehoff & Stephan Kepser, codecentri...

Spark application on ec2 cluster

Cassandra CLuster Management by Japan Cassandra Community

SSTable Reader Cassandra Day Denver 2014

Big Data Day LA 2015 - Sparking up your Cassandra Cluster- Analytics made Awe...

Spark on Mesos-A Deep Dive-(Dean Wampler and Tim Chen, Typesafe and Mesosphere)

Cassandra on Mesos Across Multiple Datacenters at Uber (Abhishek Verma) | C* ...

Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...

Instaclustr Webinar 50,000 Transactions Per Second with Apache Spark on Apach...

Learn Cassandra at edureka!

Introduction to Cassandra

Instaclustr webinar 2017 feb 08 japan

Up and running with pyspark

Scylla db deck, july 2017

Introduction to Cassandra: Replication and Consistency

Lambda Architecture with Cassandra (Vaibhav Puranik, GumGum) | C* Summit 2016

Time to Science, Time to Results. Accelerating Scientific research in the Cloud

From DataFrames to Tungsten: A Peek into Spark's Future-(Reynold Xin, Databri...

Introduction to cassandra 2014

Similaire à High order bits from cassandra & hadoop

Netflix and Open Source

Adrian Cockcroft

MySQL Cluster Scaling to a Billion Queries

Bernd Ocklin

Spring one2gx2010 spring-nonrelational_data

Roger Xia

In Cassandra Lunch #72, we will discuss how we can use Databricks with Cassandra. Accompanying Blog: https://blog.anant.us/apache-cassandra-lunch-72-databricks-and-cassandra Accompanying YouTube: https://youtu.be/5zCN27KHADo Sign Up For Our Newsletter: http://eepurl.com/grdMkn Join Cassandra Lunch Weekly at 12 PM EST Every Wednesday: https://www.meetup.com/Cassandra-DataStax-DC/events/ Cassandra.Link: https://cassandra.link/ Follow Us and Reach Us At: Anant: https://www.anant.us/ Awesome Cassandra: https://github.com/Anant/awesome-cassandra Cassandra.Lunch: https://github.com/Anant/Cassandra.Lunch Email: solutions@anant.us LinkedIn: https://www.linkedin.com/company/anant/ Twitter: https://twitter.com/anantcorp Eventbrite: https://www.eventbrite.com/o/anant-1072927283 Facebook: https://www.facebook.com/AnantCorp/ Join The Anant Team: https://www.careers.anant.us

Apache Cassandra Lunch #72: Databricks and Cassandra

Anant Corporation

Cassandra & Python - Springfield MO User Group

Adam Hutson

NoSQL is not a buzzword anymore. The array of nonrelational technologies have found wide-scale adoption even in non-Internet scale focus areas. With the advent of the Cloud...the churn has increased even more yet there is no crystal clear guidance on adoption techniques and architectural choices surrounding the plethora of options available. This session initiates you into the whys & wherefores, architectural patterns, caveats and techniques that will augment your decision making process & boost your perception of architecting scalable, fault-tolerant & distributed solutions.

Navigating NoSQL in cloudy skies

shnkr_rmchndrn

On Rails with Apache Cassandra

Stu Hood

Apache cassandra and spark. you got the the lighter, let's start the fire

Patrick McFadin

Netflix has updated and added new tools and benchmarks for Cassandra in the last year. In this talk we will cover the latest additions and recipes for the Astyanax Java client, updates to Priam to support Cassandra 1.2 Vnodes, plus newly released and upcoming tools that are all part of the NetflixOSS platform. Following on from the Cassandra on SSD on AWS benchmark that was run live during the 2012 Summit, we've been benchmarking a large write intensive multi-region cluster to see how far we can push it. Cassandra is the data storage and global replication foundation for the Cloud Native architecture that runs Netflix streaming for 36 Million users. Netflix is also offering a Cloud Prize for open source contributions to NetflixOSS, and there are ten categories including Best Datastore Integration and Best Contribution to Performance Improvements, with $10K cash and $5K of AWS credits for each winner. We'd like to pay you to use our free software!

C* Summit 2013: Netflix Open Source Tools and Benchmarks for Cassandra by Adr...

DataStax Academy

Breakthrough OLAP performance with Cassandra and Spark

Evan Chan

Big data vahidamiri-tabriz-13960226-datastack.ir

datastack

Spinnaker VLDB 2011

sandeep_tata

Developing with Cassandra

Sperasoft

Introduction to parallel iterative deep learning on hadoop’s next generation...

Anh Le

No Sql

Michael Marth

Of all the developers’ delight, none is more attractive than a set of APIs that make developers productive, that are easy to use, and that are intuitive and expressive. Apache Spark offers these APIs across components such as Spark SQL, Streaming, Machine Learning, and Graph Processing to operate on large data sets in languages such as Scala, Java, Python, and R for doing distributed big data processing at scale. In this talk, I will explore the evolution of three sets of APIs-RDDs, DataFrames, and Datasets-available in Apache Spark 2.x. In particular, I will emphasize three takeaways: 1) why and when you should use each set as best practices 2) outline its performance and optimization benefits; and 3) underscore scenarios when to use DataFrames and Datasets instead of RDDs for your big data distributed processing. Through simple notebook demonstrations with API code examples, you’ll learn how to process big data using RDDs, DataFrames, and Datasets and interoperate among them. (this will be vocalization of the blog, along with the latest developments in Apache Spark 2.x Dataframe/Datasets and Spark SQL APIs: https://databricks.com/blog/2016/07/14/a-tale-of-three-apache-spark-apis-rdds-dataframes-and-datasets.html)

A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...

Databricks

Scala Days, Amsterdam, 2015: Lambda Architecture - Batch and Streaming with Spark, Cassandra, Kafka, Akka and Scala; Fault Tolerance, Data Pipelines, Data Flows, Data Locality, Akka Actors, Spark, Spark Cassandra Connector, Big Data, Asynchronous data flows. Time series data, KillrWeather, Scalable Infrastructure, Partition For Scale, Replicate For Resiliency, Parallelism Isolation, Data Locality, Location Transparency

Lambda Architecture with Spark Streaming, Kafka, Cassandra, Akka, Scala

Helena Edelson

NativeX (formerly W3i) recently transitioned a large portion of their backend infrastructure from MS SQL Server to Apache Cassandra. Today, its Cassandra cluster backs its mobile advertising network supporting over 10 million daily active users producing over 10,000 transactions per second with an average database request latency of under 2 milliseconds. Going from relational to noSQL required NativeX's engineers to re-train, re-tool and re-think the way it architects applications and infrastructure. Learn why Cassandra was selected as a replacement, what challenges were encountered along the way, and what architecture and infrastructure were involved in the implementation.

Minnebar 2013 - Scaling with Cassandra

Jeff Bollinger

Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open-source and horizontally scalable. The original intention has been modern web-scale databases. The movement began early 2009 and is growing rapidly. Often more characteristics apply such as: schema-free, easy replication support, simple API, eventually consistent / BASE (not ACID), a huge amount of data and more. So the misleading term "nosql" (the community now translates it mostly with "not only sql") should be seen as an alias to something like the definition above.

NoSql Database

Suresh Parmar

Cassandra

Upaang Saxena

Similaire à High order bits from cassandra & hadoop (20)

Netflix and Open Source

MySQL Cluster Scaling to a Billion Queries

Spring one2gx2010 spring-nonrelational_data

Apache Cassandra Lunch #72: Databricks and Cassandra

Cassandra & Python - Springfield MO User Group

Navigating NoSQL in cloudy skies

On Rails with Apache Cassandra

Apache cassandra and spark. you got the the lighter, let's start the fire

C* Summit 2013: Netflix Open Source Tools and Benchmarks for Cassandra by Adr...

Breakthrough OLAP performance with Cassandra and Spark

Big data vahidamiri-tabriz-13960226-datastack.ir

Spinnaker VLDB 2011

Developing with Cassandra

Introduction to parallel iterative deep learning on hadoop’s next generation...

No Sql

A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...

Lambda Architecture with Spark Streaming, Kafka, Cassandra, Akka, Scala

Minnebar 2013 - Scaling with Cassandra

NoSql Database

Cassandra

Plus de srisatish ambati

H2O Open Dallas 2016 keynote for Business Transformation

srisatish ambati

Digital Transformation with AI and Data - H2O.ai and Open Source

srisatish ambati

Top 10 Data Parallelism and Model Parallelism lessons from scaling H2O. "Math Algorithms have primarily been the domain of desktop data science. With the success of scalable algorithms at Google, Amazon, and Netflix, there is an ever growing demand for sophisticated algorithms over big data. In this talk, we get a ringside view in the making of the world's most scalable and fastest machine learning framework, H2O, and the performance lessons learnt scaling it over EC2 for Netflix and over commodity hardware for other power users. Top 10 Performance Gotchas is about the white hot stories of i/o wars, S3 resets, and muxers, as well as the power of primitive byte arrays, non-blocking structures, and fork/join queues. Of good data distribution & fine-grain decomposition of Algorithms to fine-grain blocks of parallel computation. It's a 10-point story of the rage of a network of machines against the tyranny of Amdahl while keeping the statistical properties of the data and accuracy of the algorithm."

Top 10 Performance Gotchas for scaling in-memory Algorithms.

srisatish ambati

Cacheconcurrencyconsistency cassandra svcc

srisatish ambati

SF Java presentation of jvm goes to big data. “Slowly yet surely the JVM is going to Big Data! In this fun filled presentation we see what pieces of Java & JVM triumph or unravel in the battle for performance at high scale!” Concurrency is the currency of scale on multi-core & the new generation of collections and non-blocking hashmaps are well worth the time taking a deep dive into. We take a quick look at the next gen serialization techniques as well as implementation pitfalls around UUID. The achilles' heel for JVM remains Garbage Collection: a deep dive into the internals of the memory model, common GC algorithms and their tuning knobs is always a big draw. EC2 & cloud present us with a virtualized & unchartered territory for scaling the JVM. We will leave some room for Q&A or fill it up with any asynchronous I/O that might queue up during the talk. A round of applause will be due to the various tools that are essentials for Java performance debugging.

Jvm goes big_data_sfjava

srisatish ambati

jvm goes to big data

srisatish ambati

Svccg nosql 2011_sri-cassandra

srisatish ambati

Cache is King ( Or How To Stop Worrying And Start Caching in Java) at Chicago...

srisatish ambati

Caching in Java - A review of different caching vendors (Oracle Coherence, Apache Cassandra, Infinispan, Ehcache/Terracotta, etc) and limitations presented by the underlying Java Platform. Presented at RedHat Summit 2010, Boston Speakers: SriSatish Ambati, Performance Engg Manik Surtani, InfiniSpan Lead Presentation details from RH Summit: How to Stop Worrying & Start Caching in Java SriSatish Ambati — Performance & Partner Engineer, Azul Systems, Inc. Manik Surtani — Principal Software Engineer, Red Hat Application data caching has come of age as distributed and large cache clusters are now common. The next generation of applications that depend on efficient caching has come into being and data and cache size explosion has set in. In this session, Azul Systems’ SriSatish Ambati and Red Hat’s Manik Surtani will survey performance characteristics of different cache algorithms, their implementations (e.g., implementing a 200Gb data cache size), and how well they work in practical JVM deployments. In each scenario, they will present patterns of architecture that scale, and demonstrate where read and write performance stands in the context of increasing cache sizes and concurrency. Throughout this discussion, they will recognize several villains, including heap fragmentation, long-lived objects, multi-VM communication, socket handlers, and queue managers. SriSatish and Manik will take a fun-filled “whodunit” approach to portray the roles played by each villain in killing cache performance. http://www.redhat.com/promo/summit/2010/sessions/jboss.html

How to Stop Worrying and Start Caching in Java

srisatish ambati

Top 10 Causes for Java Issues in Production and What to Do When Things Go Wrong JavaOne 2010. Abstract: It's Friday evening and you hear the first rumble . . . one java node has become slightly unresponsive. You lookup the process, get a thread dump, and for good measure restart it at 8 p.m. Saturday afternoon is when you realize that other nodes have caught the flu and you get the ugly call from the customer. In a matter of hours, you're on that conference bridge with support groups of different packages and Java vendors and one of your uberarchitects. Yes, production instances are up and down, and restarting like there's no tomorrow. Here's an accumulated compendium of the op 10 things that can cause Java production heartburn and what to do when your Java production is on fire. And yes, please have your tools belt on. Speaker(s): Cliff Click, Azul Systems, Distinguished Engineer SriSatish Ambati, Azul Systems, Performance Engineer

JavaOne 2010: Top 10 Causes for Java Issues in Production and What to Do When...

srisatish ambati

Cache & Concurrency considerations for a high performance Cassandra deployment. SriSatish Ambati Cassandra has hit it's stride as a distributed java NoSQL database! It's fast, it's in-memory, it's scalable, it's seda; It's eventually consistent model makes it practical for the large & growing volumes of unstructured data usecases. It is also time to run it through the filters of performance analysis. For starters it runs on the java virtual machine and inherits the capabilities and culpabilities of the platform. This presentation reviews the runtime architecture, cache behavior & performance of a real-world workload on Cassandra. We blend existing system & jvm tools to get a quick overview & a breakdown of hotspots in the get, put & update operations. We highlight the role played by garbage collection & fragmentation due to long lived objects; We investigate lock contention in the data structures under concurrent usage. Cassandra uses UDP for management & TCP for data: we look at robustness of the communication patterns during high spikes and cluster-wide events. We review Non-Blocking Hashmap modifications to Cassandra that improve concurrency & amplify performance of this frontrunner in the NoSQL space ApacheCon2010 NA Wed, 03 November 2010 15:00 cassandra

ApacheCon2010: Cache & Concurrency Considerations in Cassandra (& limits of JVM)

srisatish ambati

Plus de srisatish ambati (11)

H2O Open Dallas 2016 keynote for Business Transformation

Digital Transformation with AI and Data - H2O.ai and Open Source

Top 10 Performance Gotchas for scaling in-memory Algorithms.

Cacheconcurrencyconsistency cassandra svcc

Jvm goes big_data_sfjava

jvm goes to big data

Svccg nosql 2011_sri-cassandra

Cache is King ( Or How To Stop Worrying And Start Caching in Java) at Chicago...

How to Stop Worrying and Start Caching in Java

JavaOne 2010: Top 10 Causes for Java Issues in Production and What to Do When...

ApacheCon2010: Cache & Concurrency Considerations in Cassandra (& limits of JVM)

Dernier

In this keynote, Asanka Abeysinghe, CTO,WSO2 will explore the shift towards platformless technology ecosystems and their importance in driving digital adaptability and innovation. We will discuss strategies for leveraging decentralized architectures and integrating diverse technologies, with a focus on building resilient, flexible, and future-ready IT infrastructures. We will also highlight WSO2's roadmap, emphasizing our commitment to supporting this transformative journey with our evolving product suite.

Platformless Horizons for Digital Adaptability

WSO2

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

MS Copilot expands with MS Graph connectors

Nanddeep Nachan

Dubai, known for its towering skyscrapers, luxurious lifestyle, and relentless pursuit of innovation, often finds itself in the global spotlight. However, amidst the glitz and glamour, the emirate faces its own set of challenges, including the occasional threat of flooding. In recent years, Dubai has experienced sporadic but significant floods, disrupting normalcy and posing unique challenges to its infrastructure. Among the critical nodes in this bustling metropolis is the Dubai International Airport, a vital hub connecting the world. This article delves into the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Orbitshub

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Passkeys: Developing APIs to enable passwordless authentication Cody Salas, Sr Developer Advocate | Solutions Architect - Yubico Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

apidays

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Discover the innovative features and strategic vision that keep WSO2 an industry leader. Explore the exciting 2024 roadmap of WSO2 API management, showcasing innovations, unified APIM/APK control plane, natural language API interaction, and cloud native agility. Discover how open source solutions, microservices architecture, and cloud native technologies unlock seamless API management in today's dynamic landscapes. Leave with a clear blueprint to revolutionize your API journey and achieve industry success!

WSO2's API Vision: Unifying Control, Empowering Developers

WSO2

Dubai, often portrayed as a shimmering oasis in the desert, faces its own set of challenges, including the occasional threat of flooding. Despite its reputation for opulence and modernity, the emirate is not immune to the forces of nature. In recent years, Dubai has experienced sporadic but significant floods, testing the resilience of its infrastructure and communities. Among the critical lifelines in this bustling metropolis is the Dubai International Airport, a bustling hub that connects the city to the world. This article explores the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Orbitshub

Elevate Developer Efficiency & build GenAI Application with Amazon Q

Bhuvaneswari Subramani

The microservices honeymoon is over. When starting a new project or revamping a legacy monolith, teams started looking for alternatives to microservices. The Modular Monolith, or 'Modulith', is an architecture that reaps the benefits of (vertical) functional decoupling without the high costs associated with separate deployments. This talk will delve into the advantages and challenges of this progressive architecture, beginning with exploring the concept of a 'module', its internal structure, public API, and inter-module communication patterns. Supported by spring-modulith, the talk provides practical guidance on addressing the main challenges of a Modultith Architecture: finding and guarding module boundaries, data decoupling, and integration module-testing. You should not miss this talk if you are a software architect or tech lead seeking practical, scalable solutions. About the author With two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Victor Rentea

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Victor Rentea

Retrieval augmented generation (RAG) is the most popular style of large language model application to emerge from 2023. The most basic style of RAG works by vectorizing your data and injecting it into a vector database like Milvus for retrieval to augment the text output generated by an LLM. This is just the beginning. One of the ways that we can extend RAG, and extend AI, is through multilingual use cases. Typical RAG is done in English using embedding models that are trained in English. In this talk, we’ll explore how RAG could work in languages other than English. We’ll explore French, Chinese, and Polish.

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Zilliz

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Dernier (20)

Platformless Horizons for Digital Adaptability

Strategies for Landing an Oracle DBA Job as a Fresher

MS Copilot expands with MS Graph connectors

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

presentation ICT roal in 21st century education

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Why Teams call analytics are critical to your entire business

WSO2's API Vision: Unifying Control, Empowering Developers

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Elevate Developer Efficiency & build GenAI Application with Amazon Q

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Introduction to Multilingual Retrieval Augmented Generation (RAG)

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

High order bits from cassandra & hadoop

1. High-order bits from Cassandra & Hadoop srisatishambati @srisatish

2. Thank You! svccg in first page of search results for “cloud” on google!

3. NoSQL- Know your queries.

4. points Usecases Why cassandra? Usecase: Hadoop, Brisk FUD:Consistency Why facebook is not using Cassandra? Anti-patterns Community, Code, Tools Q&A

5. Users. Netflix. Key by Customer, read-heavy Key by Customer:Movie, write-heavy

6. TimeSeries: (several customers) periodic readings: dev0, dev1…deviceID:metric:timestamp ->value Metrics typically way larger dataset than users.

7. Why Cassandra?

8. Operational simplicity peer-to-peer

9. Operational simplicity peer-to-peer

10. Replication: Multi-datacenter Multi-region ec2 Multi-availability zones

11. reads local dc1 dc2 Replication: Multi-datacenter Multi-region ec2, aws Multi-availability zones

12. 4.21.2011, Amazon Web Services outage: “Movie marathons on Netflix awaiting AWS to come back up.” #ec2disabled

13. 4.21.2011, Amazon Web Services outage: Netflix was running on AWS.

14. fast durable writes. fast reads.

15. Writes Sequential, append-only. ~1-5ms

16. Writes Sequential, append-only. ~1-5ms On cloud: ephemeral disks rock!

17. Reads Local Key & row caches, (also, jna-based 0xffheap) indexes, materialized

18. Reads Local Key & row caches, (also, jna-based 0xffheap) indexes, materialized ssds: improved read performance!

19. Distribution between nodes Gossip Anti-entropy Failure-detector L i g h t w e i g h t

20. Clients: cql, thrift pycassa, phpcassa hector, pelops (scala, ruby, clojure)

21. Usecase #3: hadoop Hdfs cassandra hive Logs stats analytics

22. Brisk Truly peer-to-peer hadoop.

23. mv computation not data

24.

25. Parallel Execution View

26.

27. jobtracker, tasktracker hdfs: namenode, datanode

28. cloudera amazon: elastic map reduce hortonworks mapR brisk

29. Tools & Analytics Hive, Pig, R Karmasphere Datameer … dozens of stealth startups!

30. Namenode decomposition, explained.

31.

32.

33. Use column families (tables) inode sblock

34. near-real time hadoop Low latency: cassandra_dc nodes Batch Analytics: brisk_dc nodes

35. FUD, acronym: fear, uncertainty, doubt.

36. Consistency: R + W > N ORACLE, 2-node: R=1, W=2, N=2,(T=2) DNS * N is replication factor. Not to be confused with T=total #of nodes

37. Tune-able, flexibility. For High Consistency: read:quorum, write:quorum For High Availability: high W, low R.

38.

39. Inbox Search: 600+cores.120+TB (2008) Went from 100-500m users. Average NoSQL deployment size: ~6-12 nodes.

40. Usecase #5: search Apache Solr + Cassandra = Solandra Other inbox/file Searches: xobni, c3 github.com/tjake/solandra

41. “Eventual consistency is harder to program.” mostly immutable data. complex systems at scale.

42. Miscellaneous, Myth: data-loss, partial rows. writes are durable.

43. Anti-Patterns Transactions Joins Read before write

44. Anti-Patterns for cloud ebs jvm, virtualized single region

45. Three good reasons for Cassandra...

46. Tools AMIs, OpsCenter, DataStax AppDynamics Netflix just builds AMIs for deployment!

47. B e a u t i f u l C 0 d e = new code(); //less is more ~90k.java.concurrent.@annotate. bloomfilters, merkletrees. non-blocking, staged-event-driven. bigtable, dynamo.

48. Current & Future Focus: Distributed Counters, CQL. Simple client. operational smoothening. compaction.

49. Community Robust. Rapid. # Professional support from DataStax. Filesysteminnovatin from Acunu engineers: independent,startups, large companies, Rackspace, Twitter, Netflix.. Come join the efforts!

50.

51. Usecase #4: first NoSQL, then scale! simpledb Cassandra mongodb Cassandra

52.

53.

54.

55. Copyright: xkcd

56. Copyright: plantoys … more than one way to do it!

57. Summary - high scale peer-to-peer datastore best friend for multi-region, multi-zone availability. Hadoop – HDFS engulfing the DataWorld

58. Q&A @srisatish

59. NoSQL- Know your queries.

High order bits from cassandra & hadoop

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à High order bits from cassandra & hadoop

Similaire à High order bits from cassandra & hadoop (20)

Plus de srisatish ambati

Plus de srisatish ambati (11)

Dernier

Dernier (20)

High order bits from cassandra & hadoop