Cascalog

•Télécharger en tant que KEY, PDF•

16 j'aime•2,650 vues

Presentation I gave at Bay Area Clojure Meetup Group on May 6th, 2010. Also demoed examples from introductory tutorial: http://nathanmarz.com/blog/introducing-cascalog/

Technologie

Cascalog
Nathan Marz, BackType

Po wer fu l a n d ea sy-t o- us e data a n a lysi s to ol fo r H adoo p

About Me

Tech Lead at BackType

Have been working on many-terabyte scale
systems for two years

ETL workflows

Data warehouses

What is Hadoop?

Distributed Filesystem

MapReduce Framework

Scales to thousands of machines and petabytes of
data

What is Cascalog?

Clojure-based query language for Hadoop with
Datalog-inspired syntax

Queries compile to one or more MapReduce jobs

The tool I wish I had two years ago

Features

Inner and outer joins

Aggregators

Functions

Subqueries

Sorting

High performance

What sets Cascalog apart?

Super simple

Full power of Clojure always available

Easy to extend with custom operations

Dynamic queries

Arbitrary inputs and outputs

Experiment with Cascalog

Ships with test
dataset that can be
queried locally (the
“playground”)

5 minutes to setup
Hadoop, Clojure, and
Cascalog locally - see
README

News feed generator

Ranks events in
social network
for each person
based on
“importance”
and recency

38 lines of code

News Feed
“Follows” and “Action” data sources

Text files on HDFS
Follows Action

News Feed
Custom Aggregator to produce a
news feed in JSON-like form

News Feed

Custom Function
to score each item
in the feed

News Feed

Subquery to compute
follower count for
each person

News Feed

Tie everything
together in a single
Cascalog query

Questions?

Project page:
http://www.github.com/nathanmarz/cascalog

Tutorial:
http://nathanmarz.com/blog/introducing-cascalog

Follow me on Twitter: @nathanmarz

Contenu connexe

Tendances

Developing high frequency indicators using real time tick data on apache supe...

Zekeriya Besiroglu

Distributed ML in Apache Spark

Databricks

The prevailing issue when working with Operating Room (OR) scheduling within a hospital setting is that it is difficult to schedule and predict available OR block times. This leads to empty and unused operating rooms leading to longer waiting times for patients for their procedures. In this three-part session, Ayad Shammout and Denny will show: 1) How we tried to solve this problem using traditional DW techniques 2) How we took advantage of the DW capabilities in Apache Spark AND easily transition to Spark MLlib so we could more easily predict available OR block times resulting in better OR utilization and shorter wait times for patients. 3) Some of the key learnings we had when migrating from DW to Spark.

Transitioning from Traditional DW to Apache® Spark™ in Operating Room Predict...

Databricks

Open source data ingestion

Treasure Data, Inc.

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2nwSwEh. Marco Bonzanini discusses the process of building data pipelines, e.g. extraction, cleaning, integration, pre-processing of data; in general, all the steps necessary to prepare data for a data-driven product. In particular, he focuses on data plumbing and on the practice of going from prototype to production. Filmed at qconlondon.com. Marco Bonzanini is Data Scientist and co-organizer of PyData London Meetup.

Building Data Pipelines in Python

C4Media

Elasticsearch provides native integration with Apache Spark through ES-Hadoop. However, especially during development, it is at best cumbersome to have Elasticsearch running in a separate machine/instance. Leveraging Spark Cluster with Elasticsearch Inside it is possible to run an embedded instance of Elasticsearch in the driver node of a Spark Cluster. This opens up new opportunities to develop cutting-edge applications. One such application is Dataset Search. Oscar will give a demo of a Dataset Search Engine built on Spark Cluster with Elasticsearch Inside. Motivation is that once Elasticsearch is running on Spark it becomes possible and interesting to have the Elasticsearch in-memory instance join an (existing) Elasticsearch cluster. And this in turn enables indexing of Datasets that are processed as part of Data Pipelines running on Spark. Dataset Search and Data Management are R&D topics that should be of interest to Spark Summit East attendees who are looking for a way to organize their Data Lake and make it searchable.

Building a Dataset Search Engine with Spark and Elasticsearch: Spark Summit E...

Spark Summit

Lessons from Running Large Scale Spark Workloads

Databricks

Spark Summit San Francisco 2016 - Ali Ghodsi Keynote

Databricks

Big data remains a rapidly evolving field with new applications and infrastructure appearing every year. In this talk, I’ll cover new trends in 2016 / 2017 and how Apache Spark is moving to meet them. In particular, I’ll talk about work Databricks is doing to make Apache Spark interact better with native code (e.g. deep learning libraries), support heterogeneous hardware, and simplify production data pipelines in both streaming and batch settings through Structured Streaming.

Trends for Big Data and Apache Spark in 2017 by Matei Zaharia

Spark Summit

Analyzing and comparing your energy consumption with that of other consumers provides healthy peer pressure and useful insight leading to energy conservation and impacting the bottom line. We helped GridPocket (http://www.gridpocket.com/), a smart grid company developing energy management applications for electricity water and gas utilities, implement high scale anonymized energy comparison queries with an order of magnitude lower cost and higher performance than was previously possible. IoT use cases like that of GridPocket are swamping our planet with data, and drive demand for analytics on extremely scalable and low cost storage. Enter Spark SQL over Object Storage: highly scalable and low cost storage which provides RESTful APIs to store and retrieve objects and their metadata. Key performance indicators (KPIs) of query performance and cost are the number of bytes shipped from Object Storage to Spark and the number of incurred REST requests. We propose Pluggable Spark SQL Filters, which extend the existing Spark SQL partitioning mechanism with an ability to dynamically filter irrelevant objects during query execution. Our approach handles any data format supported by Spark SQL (Parquet, JSON, csv etc.), and unlike pushdown compatible formats such as Parquet which require touching each object to determine its relevance, it avoids accessing irrelevant objects altogether. We developed a pluggable interface for developing and deploying Filters, and implemented GridPocket’s filter which screens objects according to their metadata, for example geo-spatial bounding boxes which describe the area covered by an object’s data points. This leads to drastically lower KPIs since there is no need to ship the entire dataset from Object Storage to Spark if you are only comparing yourself with your neighborhood. We demonstrate GridPocket analytics notebooks, report on our implementation and resulting 10-20x speedups, explain how to implement a Pluggable File Filter, and how we applied this to other use cases.

Using Pluggable Apache Spark SQL Filters to Help GridPocket Users Keep Up wit...

Spark Summit

Paris Spark Meetup (Feb2015) ccarbone : SPARK Streaming vs Storm / MLLib / Ne...

Cedric CARBONE

Machine learning has its challenges, and understanding the algorithms is not always easy. In this session, you’ll discover methods to make these challenges less daunting. Intended for software engineers who need to understand the requirements and constraints of data scientists, and data scientists who need to implement or help implement production systems, the session will begin with a quick introduction to data quality and a level-set on common vocabulary. You’ll then explore the formats that are required by Spark ML to run its algorithms, and see how to automate the build through user-defined functions and other techniques. Automation will make reproducibility easy, minimize errors and increase the efficiency of data scientists. Key takeaways will include: – How to build the required tool set in Java – Understanding the formats required by Spark ML (a new vocabulary) – Learning fundamentals about data quality and how to make sure the data is usable

The Key to Machine Learning is Prepping the Right Data with Jean Georges Perrin

Databricks

Spark SQL is one of the most popular components in big data warehouse for SQL queries in batch mode, and it allows user to process data from various data sources in a highly efficient way. However, Spark SQL is a general purpose SQL engine and not well designed for ad hoc queries. Intel invented an Apache Spark data source plugin called Spinach for fulfilling such requirements, by leveraging user-customized indices and fine-grained data cache mechanisms. To be more specific, Spinach defines a new Parquet-like data storage format, offering a fine-grained hierarchical cache mechanism in the unit of “Fiber” in memory. Even existing Parquet or ORC data files can be loaded using corresponding adaptors. Data can be cached in off-heap memory to boost data loading. What’s more, Spinach has extended the Spark SQL DDL, to allow users to define the customized indices based on relation. Currently, B+ tree and bloom filter are the first two types of indices supported. Last but not least, since Spinach resides in the process of Spark executor, there’s no extra effort in deployment. All you need to do is to pick Spinach from Spark packages when launching the Spark SQL. sing corresponding adaptors. Data can be cached in off-heap memory to boost data loading. What’s more, Spinach has extended the Spark SQL DDL, to allow user to define the customized indices based on relation. Currently B+ tree and bloom filter are the first 2 types of index we’ve supported. Last but not least, since Spinach resides in the process of Spark executor, there’s no extra effort in deployment, all we need to do is to pick Spinach from Spark packages when launch the Spark SQL. Spinach has been imported in Baidu’s production environment since Q4 2016. It helps several teams migrating their regular data analysis tasks from Hive or MR jobs to ad-hoc queries. In Baidu search ads system FengChao, data engineers analyze advertising effectiveness based on several TBs data of display and click logs every day. Spinach brings a 5x boost compared to original Spark SQL (version 2.1), especially in the scenario of complex search and large data volume. It optimizes the average search cost from minutes to seconds, while brings only 3% data size increase for adding a single index.

OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...

Databricks

Intro to Python Data Analysis in Wakari

Karissa Rae McKelvey

Configuring elasticsearch for performance and scale

Bharvi Dixit

Solr + Hadoop = Big Data Search

Mark Miller

Search Analytics Component: Presented by Steven Bower, Bloomberg L.P.

Lucidworks

Scaling Self Service Analytics with Databricks and Apache Spark with Amelia C...

Databricks

Text Analytics Summit 2009 - Roddy Lindsay - "Social Media, Happiness, Petaby...

guest5b1607

Benjamin Hopp (Solutions Architect) @ Imply: Druid is an emerging standard in the data infrastructure world, designed for high-performance slice-and-dice analytics (“OLAP”-style) on large data sets. This talk is for you if you’re interested in learning more about pushing Druid’s analytical performance to the limit. Perhaps you’re already running Druid and are looking to speed up your deployment, or perhaps you aren’t familiar with Druid and are interested in learning the basics. Some of the tips in this talk are Druid-specific, but many of them will apply to any operational analytics technology stack. The most important contributor to a fast analytical setup is getting the data model right. The talk will center around various choices you can make to prepare your data to get best possible query performance. We’ll look at some general best practices to model your data before ingestion such as OLAP dimensional modeling (called “roll-up” in Druid), data partitioning, and tips for choosing column types and indexes. We’ll also look at how more can be less: often, storing copies of your data partitioned, sorted, or aggregated in different ways can speed up queries by reducing the amount of computation needed. We’ll also look at Druid-specific optimizations that take advantage of approximations; where you can trade accuracy for performance and reduced storage. You’ll get introduced to Druid’s features for approximate counting, set operations, ranking, quantiles, and more. And we will finish with the latest and greatest Druid news, including details about the latest roadmap and releases.

A Day in the Life of a Druid Implementor and Druid's Roadmap

Itai Yaffe

Tendances (20)

Developing high frequency indicators using real time tick data on apache supe...

Distributed ML in Apache Spark

Transitioning from Traditional DW to Apache® Spark™ in Operating Room Predict...

Open source data ingestion

Building Data Pipelines in Python

Building a Dataset Search Engine with Spark and Elasticsearch: Spark Summit E...

Lessons from Running Large Scale Spark Workloads

Spark Summit San Francisco 2016 - Ali Ghodsi Keynote

Trends for Big Data and Apache Spark in 2017 by Matei Zaharia

Using Pluggable Apache Spark SQL Filters to Help GridPocket Users Keep Up wit...

Paris Spark Meetup (Feb2015) ccarbone : SPARK Streaming vs Storm / MLLib / Ne...

The Key to Machine Learning is Prepping the Right Data with Jean Georges Perrin

OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...

Intro to Python Data Analysis in Wakari

Configuring elasticsearch for performance and scale

Solr + Hadoop = Big Data Search

Search Analytics Component: Presented by Steven Bower, Bloomberg L.P.

Scaling Self Service Analytics with Databricks and Apache Spark with Amelia C...

Text Analytics Summit 2009 - Roddy Lindsay - "Social Media, Happiness, Petaby...

A Day in the Life of a Druid Implementor and Druid's Roadmap

En vedette

Play station 4 camilo q

carolinagonzalezcsj

JourneyCare Discover Your Balance Week 7 Gratitude & Gratefulness

altonbaird

Communities: Build Or Join

Yahoo Developer Network

Operating Systems - Advanced File Systems

Emery Berger

F1041028_George_Chen_Resume_9_with_Publications_Training

Wei-Su Chen

Ailanto 2013 independent living community investment

Hugo Ribadeneira

Hellen e vitoria musicas ....

Vitoria Dos Santos

LinkedIn for Beginners

Ahmed Abdel Rahman

PlayStation 4

carolinagonzalezcsj

Datos importantes para planificar un evento edith giraldo productora de eve...

Edith Giraldo

Cascalog workshop

nathanmarz

Ahead Week 1 Key Slides

altonbaird

不喜歡上班是因為喜歡當個自由工作者？創業是個好途徑！創業可以自己當老闆，自由自在規劃自己的生活與工作，但要如何創造持續性的收入則可能得要多思考了。近來興起的專業工作室或微型創業熱潮，是個不錯的新工作概念，但你知道你適合哪一種的創業類型嗎？微型創業需要注意的事情有哪些呢？在數位時代五月份特刊中有了詳細描述，將實用的知識分享給大家。

幸福創業計畫

funpo Lin

Finding a job using social media

Intranet Future

02 epidemio enf reum

iloaeza_89

Marc Prensky & the digital divides

kavismusings

How to Create Creative Commons Licensing Buttons for Your Website

NET:101

Banja Luka

Dr David Hancock

En vedette (18)

Play station 4 camilo q

JourneyCare Discover Your Balance Week 7 Gratitude & Gratefulness

Communities: Build Or Join

Operating Systems - Advanced File Systems

F1041028_George_Chen_Resume_9_with_Publications_Training

Ailanto 2013 independent living community investment

Hellen e vitoria musicas ....

LinkedIn for Beginners

PlayStation 4

Datos importantes para planificar un evento edith giraldo productora de eve...

Cascalog workshop

Ahead Week 1 Key Slides

幸福創業計畫

Finding a job using social media

02 epidemio enf reum

Marc Prensky & the digital divides

How to Create Creative Commons Licensing Buttons for Your Website

Banja Luka

Similaire à Cascalog

Yahoo! Hadoop User Group - May Meetup - Extraordinarily rapid and robust data...

Hadoop User Group

Data scientists spend too much of their time collecting, cleaning and wrangling data as well as curating and enriching it. Some of this work is inevitable due to the variety of data sources, but there are tools and frameworks that help automate many of these non-creative tasks. A unifying feature of these tools is support for rich metadata for data sets, jobs, and data policies. In this talk, I will introduce state-of-the-art tools for automating data science and I will show how you can use metadata to help automate common tasks in Data Science. I will also introduce a new architecture for extensible, distributed metadata in Hadoop, called Hops (Hadoop Open Platform-as-a-Service), and show how tinker-friendly metadata (for jobs, files, users, and projects) opens up new ways to build smarter applications.

Data Science with the Help of Metadata

Jim Dowling

At ResearchGate, we constantly analyze scientific data to connect the world of science and make research open to all. It can be tricky to set up a process to continuously deliver improved versions of algorithms that tap into more than 100 million publications and corresponding bibliographic metadata. In this talk, we illustrate some (big) data engineering challenges of running data pipelines and incorporating results into the live databases that power our user-facing features every day. We show how Apache Flink helps us to improve performance, robustness, ease of maintenance - and most importantly - have more fun while building big data pipelines.

Tapping into Scientific Data with Hadoop and Flink

Michael Häusler

What’s important about a technology is what you can use it to do. I’ve looked at what a number of groups are doing with Apache Hadoop and NoSQL in production, and I will relay what worked well for them and what did not. Drawing from real world use cases, I show how people who understand these new approaches can employ them well in conjunction with traditional approaches and existing applications. Thread Detection, Datawarehouse optimization, Marketing Efficiency, Biometric Database are some examples exposed during this presentation.

Tugdual Grall - Real World Use Cases: Hadoop and NoSQL in Production

Codemotion

Fast and Simplified Streaming, Ad-Hoc and Batch Analytics with FiloDB and Spa...

Helena Edelson

Apache Kafka has come the modern central point for a fast and scalable streaming platform. Now, thanks to the open source explosion over the last decade, there are now numerous data stores available as sinks for Kafka-brokered data, from search to document stores, columnular DBs, time series DBs and more. While many claim they are the swiss army knife, in reality each is designed for specific types of data and analytics approaches. In this talk, we will cover the taxonomy of various data sinks, delve into each categories pros, cons and ideal use cases, so you can select the right ones and tie them together with Kafka into a well-considered architecture.

Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...

confluent

Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...

Cloudera, Inc.

Enterprise Data Workflows with Cascading and Windows Azure HDInsight

Paco Nathan

Supreet Oberoi's presentation on "Large scale log processing with Cascading & Elastic Search". Elasticsearch is becoming a popular platform for log analysis with its ELK stack: Elasticsearch for search, Logstash for centralized logging, and Kibana for visualization. Complemented with Cascading, the application development platform for building Data applications on Apache Hadoop, developers can correlate at scale multiple log and data streams to perform rich and complex log processing before making it available to the ELK stack.

Elasticsearch + Cascading for Scalable Log Processing

Cascading

The ABC of Big Data

André Faria Gomes

Build Deep Learning Applications for Big Data Platforms (CVPR 2018 tutorial)

Jason Dai

Building Scalable Data Pipelines - 2016 DataPalooza Seattle

Evan Chan

Hadoop Frameworks Panel__HadoopSummit2010

Yahoo Developer Network

XDF 2019 Xilinx Accelerated Database and Data Analytics Ecosystem

Dan Eaton

Elasticsearch quick Intro (English)

Federico Panini

Hive @ Hadoop day seattle_2010

nzhang

Scaling up with Cisco Big Data: Data + Science = Data Science

eRic Choo

Johnny Miller – Cassandra + Spark = Awesome- NoSQL matters Barcelona 2014

NoSQLmatters

Behind the growing interest in Generate AI and LLM-based enterprise applications lies an expanded set of requirements for data integrations and ML orchestration. Enterprises want to use proprietary data to power LLM-based applications that create new business value, but they face challenges in moving beyond experimentation. The pipelines that power these models need to run reliably at scale, bringing together data from many sources and reacting continuously to changing conditions. This talk focuses on the design patterns for using Apache Airflow to support LLM applications created using private enterprise data. We’ll go through a real-world example of what this looks like, as well as a proposal to improve Airflow and to add additional Airflow Providers to make it easier to interact with LLMs such as the ones from OpenAI (such as GPT4) and the ones on HuggingFace, while working with both structured and unstructured data. In short, this shows how these Airflow patterns enable reliable, traceable, and scalable LLM applications within the enterprise. https://airflowsummit.org/sessions/2023/keynote-llm/

Building and deploying LLM applications with Apache Airflow

Kaxil Naik

The millions of people that use Spotify each day generate a lot of data, roughly a few terabytes per day. What does it take to handle datasets of that scale, and what can be done with it? I will briefly cover how Spotify uses data to provide a better music listening experience, and to strengthen their busineess. Most of the talk will be spent on our data processing architecture, and how we leverage state of the art data processing and storage tools, such as Hadoop, Cassandra, Kafka, Storm, Hive, and Crunch. Last, I'll present observations and thoughts on innovation in the data processing aka Big Data field.

Data Infrastructure for a World of Music

Lars Albertsson

Similaire à Cascalog (20)

Yahoo! Hadoop User Group - May Meetup - Extraordinarily rapid and robust data...

Data Science with the Help of Metadata

Tapping into Scientific Data with Hadoop and Flink

Tugdual Grall - Real World Use Cases: Hadoop and NoSQL in Production

Fast and Simplified Streaming, Ad-Hoc and Batch Analytics with FiloDB and Spa...

Five Fabulous Sinks for Your Kafka Data. #3 will surprise you! (Rachel Pedres...

Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...

Enterprise Data Workflows with Cascading and Windows Azure HDInsight

Elasticsearch + Cascading for Scalable Log Processing

The ABC of Big Data

Build Deep Learning Applications for Big Data Platforms (CVPR 2018 tutorial)

Building Scalable Data Pipelines - 2016 DataPalooza Seattle

Hadoop Frameworks Panel__HadoopSummit2010

XDF 2019 Xilinx Accelerated Database and Data Analytics Ecosystem

Elasticsearch quick Intro (English)

Hive @ Hadoop day seattle_2010

Scaling up with Cisco Big Data: Data + Science = Data Science

Johnny Miller – Cassandra + Spark = Awesome- NoSQL matters Barcelona 2014

Building and deploying LLM applications with Apache Airflow

Data Infrastructure for a World of Music

Plus de nathanmarz

Demystifying Data Engineering

nathanmarz

The inherent complexity of stream processing

nathanmarz

Using Simplicity to Make Hard Big Data Problems Easy

nathanmarz

The Epistemology of Software Engineering

nathanmarz

Your Code is Wrong

nathanmarz

Runaway complexity in Big Data... and a plan to stop it

nathanmarz

Storm

nathanmarz

Storm: distributed and fault-tolerant realtime computation

nathanmarz

ElephantDB

nathanmarz

Become Efficient or Die: The Story of BackType

nathanmarz

The Secrets of Building Realtime Big Data Systems

nathanmarz

Clojure at BackType

nathanmarz

Cascalog at Strange Loop

nathanmarz

Cascalog at Hadoop Day

nathanmarz

Cascading

nathanmarz

Plus de nathanmarz (15)

Demystifying Data Engineering

The inherent complexity of stream processing

Using Simplicity to Make Hard Big Data Problems Easy

The Epistemology of Software Engineering

Your Code is Wrong

Runaway complexity in Big Data... and a plan to stop it

Storm

Storm: distributed and fault-tolerant realtime computation

ElephantDB

Become Efficient or Die: The Story of BackType

The Secrets of Building Realtime Big Data Systems

Clojure at BackType

Cascalog at Strange Loop

Cascalog at Hadoop Day

Cascading

Dernier

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Tech Trends Report 2024 Future Today Institute.pdf

hans926745

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Real Time Object Detection Using Open CV

Khem

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

GenCyber Cyber Security Day Presentation

Michael W. Hawkins

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

What is a good lead in your organisation? Which leads are priority? What happens to leads? When sales and marketing give different answers to these questions, or perhaps aren't sure of the answers at all, frustrations build and opportunities are left on the table. Join us for an illuminating session with Cian McLoughlin, HubSpot Principal Customer Success Manager, as we look at that crucial piece of the customer journey in which leads are transferred from marketing to sales.

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

HampshireHUG

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

What are drone anti-jamming systems? The drone anti-jamming systems and anti-spoof technology protect against interference, jamming, and spoofing of the UAVs. To protect their security, countries are beginning to research drone anti-jamming systems, also known as drone strike weapons. The anti-jam and anti-spoof technology protects against interference, jamming and spoofing. A drone strike weapon is a drone attack weapon that can attack and destroy enemy drones. So what is so unique about this amazing system?

What Are The Drone Anti-jamming Systems Technology?

Antenna Manufacturer Coco

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

Dernier (20)

Boost Fertility New Invention Ups Success Rates.pdf

Tech Trends Report 2024 Future Today Institute.pdf

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

Handwritten Text Recognition for manuscripts and early printed texts

[2024]Digital Global Overview Report 2024 Meltwater.pdf

Powerful Google developer tools for immediate impact! (2023-24 C)

🐬 The future of MySQL is Postgres 🐘

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Real Time Object Detection Using Open CV

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

How to Troubleshoot Apps for the Modern Connected Worker

GenCyber Cyber Security Day Presentation

presentation ICT roal in 21st century education

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Automating Google Workspace (GWS) & more with Apps Script

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

A Domino Admins Adventures (Engage 2024)

What Are The Drone Anti-jamming Systems Technology?

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

Cascalog

1. Cascalog Nathan Marz, BackType Po wer fu l a n d ea sy-t o- us e data a n a lysi s to ol fo r H adoo p

2. About Me Tech Lead at BackType Have been working on many-terabyte scale systems for two years ETL workflows Data warehouses

3. What is Hadoop? Distributed Filesystem MapReduce Framework Scales to thousands of machines and petabytes of data

4. What is Cascalog? Clojure-based query language for Hadoop with Datalog-inspired syntax Queries compile to one or more MapReduce jobs The tool I wish I had two years ago

5. Features Inner and outer joins Aggregators Functions Subqueries Sorting High performance

6. What sets Cascalog apart? Super simple Full power of Clojure always available Easy to extend with custom operations Dynamic queries Arbitrary inputs and outputs

7. What sets Cascalog apart? Super simple Full power of Clojure always available Easy to extend with custom operations Dynamic queries Arbitrary inputs and outputs

8. Experiment with Cascalog Ships with test dataset that can be queried locally (the “playground”) 5 minutes to setup Hadoop, Clojure, and Cascalog locally - see README

9. News feed generator Ranks events in social network for each person based on “importance” and recency 38 lines of code

10. Demo time!

11. News Feed “Follows” and “Action” data sources Text files on HDFS Follows Action

12. News Feed

13. News Feed Custom Aggregator to produce a news feed in JSON-like form

14. News Feed Custom Function to score each item in the feed

15. News Feed Data sources

16. News Feed Subquery to compute follower count for each person

17. News Feed Tie everything together in a single Cascalog query

18. Questions? Project page: http://www.github.com/nathanmarz/cascalog Tutorial: http://nathanmarz.com/blog/introducing-cascalog Follow me on Twitter: @nathanmarz

Cascalog

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (18)

Similaire à Cascalog

Similaire à Cascalog (20)

Plus de nathanmarz

Plus de nathanmarz (15)

Dernier

Dernier (20)

Cascalog

Notes de l'éditeur