Hw09 Counting And Clustering And Other Data Tricks

•Télécharger en tant que PPT, PDF•

1 j'aime•1,110 vues

Cloudera, Inc.

Technologie

Cheap Parlor Tricks, Counting, and Clustering Derek Gottfrid The New York Times October 2009

Solution ,[object Object],[object Object],[object Object],[object Object]

Problem Bits ,[object Object],[object Object],[object Object],[object Object],[object Object]

Background What goes into making a PDF of a NYTimes.com article? ,[object Object]

A Few Details ,[object Object],[object Object]

Breakdown ,[object Object],[object Object],[object Object]

TimesMachine http://timesmachine.nytimes.com

Data ,[object Object],[object Object],[object Object]

Counting ,[object Object],[object Object],[object Object],[object Object]

A Few Details ,[object Object],[object Object],[object Object],[object Object]

Usage Data July 2009 ???M Page Views ??M Unique Users

Twitter Click Backs By Age Group July 2009

Usage Data combined with Article Data July 2009 40 Articles

Clustering ,[object Object],[object Object],[object Object]

Questions? [email_address] @derekg http://open.nytimes.com/

Recommandé

Twitter analytics in BluemixWilfried Hoge

Tropos.io - Hadoop in the Cloud - BA4ALL 2016Tropos.io

Is it harder to find a taxi when it is raining? Wilfried Hoge

FMI Open Data on S3Roope Tervo

JanusGraph, Jupyter Meetup NYCJason Plurad

HBaseCon 2015: Warcbase - Scaling 'Out' and 'Down' HBase for Web ArchivingHBaseCon

Deploying Your Data Warehouse on AWSAmazon Web Services

Apache Iceberg - A Table Format for Hige Analytic DatasetsAlluxio, Inc.

Recommandé

Twitter analytics in BluemixWilfried Hoge

Tropos.io - Hadoop in the Cloud - BA4ALL 2016Tropos.io

Is it harder to find a taxi when it is raining? Wilfried Hoge

FMI Open Data on S3Roope Tervo

JanusGraph, Jupyter Meetup NYCJason Plurad

HBaseCon 2015: Warcbase - Scaling 'Out' and 'Down' HBase for Web ArchivingHBaseCon

Deploying Your Data Warehouse on AWSAmazon Web Services

Apache Iceberg - A Table Format for Hige Analytic DatasetsAlluxio, Inc.

Presto Summit 2018 - 08 - FINRAkbajda

Hadoop summit-ams-2014-04-03SDanzanvilliersCriteo

Munging Solo: the Joy of Small Datarobmillr

Performance and Application of GIS and Big Data ETL Processes Using FMESafe Software

Big Data Meets FMESafe Software

Graph Computing with JanusGraphJason Plurad

Serverless Big Data Analytics with Amazon Athena and QuickSightAmazon Web Services

kintoneがAWSで目指すDevOpsQAな開発Teppei Sato

Open Source Tools for Big DataTeemu Heikkilä

From Raw Data to DeploymentKNIMESlides

Building an open data platform with apache icebergAlluxio, Inc.

Big Data – Tap into Cloud Infrastructure with FMESafe Software

Chemistry Data Basics with KNIME Analytics PlatformKNIMESlides

An Intro to Elasticsearch and KibanaObjectRocket

Janus graph lookingbackwardreachingforwardDemai Ni

Building Scalable Big Data PipelinesChristian Gügi

Powers of Ten ReduxJason Plurad

Exploring MongoDB & Elasticsearch: Better TogetherObjectRocket

JanusGraph: Looking Backward, Reaching ForwardJason Plurad

Big Data at Tube: Events to Insights to ActionMurtaza Doctor

Hw09 Map Reduce Over Tahoe A Least Authority Encrypted Distributed Filesy...Cloudera, Inc.

Hw09 Next Steps For HadoopCloudera, Inc.

Contenu connexe

Tendances

Presto Summit 2018 - 08 - FINRAkbajda

Hadoop summit-ams-2014-04-03SDanzanvilliersCriteo

Munging Solo: the Joy of Small Datarobmillr

Performance and Application of GIS and Big Data ETL Processes Using FMESafe Software

Big Data Meets FMESafe Software

Graph Computing with JanusGraphJason Plurad

Serverless Big Data Analytics with Amazon Athena and QuickSightAmazon Web Services

kintoneがAWSで目指すDevOpsQAな開発Teppei Sato

Open Source Tools for Big DataTeemu Heikkilä

From Raw Data to DeploymentKNIMESlides

Building an open data platform with apache icebergAlluxio, Inc.

Big Data – Tap into Cloud Infrastructure with FMESafe Software

Chemistry Data Basics with KNIME Analytics PlatformKNIMESlides

An Intro to Elasticsearch and KibanaObjectRocket

Janus graph lookingbackwardreachingforwardDemai Ni

Building Scalable Big Data PipelinesChristian Gügi

Powers of Ten ReduxJason Plurad

Exploring MongoDB & Elasticsearch: Better TogetherObjectRocket

JanusGraph: Looking Backward, Reaching ForwardJason Plurad

Big Data at Tube: Events to Insights to ActionMurtaza Doctor

Tendances (20)

Presto Summit 2018 - 08 - FINRA

Hadoop summit-ams-2014-04-03

Munging Solo: the Joy of Small Data

Performance and Application of GIS and Big Data ETL Processes Using FME

Big Data Meets FME

Graph Computing with JanusGraph

Serverless Big Data Analytics with Amazon Athena and QuickSight

kintoneがAWSで目指すDevOpsQAな開発

Open Source Tools for Big Data

From Raw Data to Deployment

Building an open data platform with apache iceberg

Big Data – Tap into Cloud Infrastructure with FME

Chemistry Data Basics with KNIME Analytics Platform

An Intro to Elasticsearch and Kibana

Janus graph lookingbackwardreachingforward

Building Scalable Big Data Pipelines

Powers of Ten Redux

Exploring MongoDB & Elasticsearch: Better Together

JanusGraph: Looking Backward, Reaching Forward

Big Data at Tube: Events to Insights to Action

En vedette

Hw09 Map Reduce Over Tahoe A Least Authority Encrypted Distributed Filesy...Cloudera, Inc.

Hw09 Next Steps For HadoopCloudera, Inc.

Hw09 Protein AlignmentCloudera, Inc.

Hadoop Summit 2012 | HDFS High AvailabilityCloudera, Inc.

Strata + Hadoop World 2012: Apache HBase Features for the EnterpriseCloudera, Inc.

Hadoop Lecture for Harvard's CS 264 -- October 19, 2009Cloudera, Inc.

Hw09 Sqoop Database Import For HadoopCloudera, Inc.

Masterclass Webinar - Amazon Elastic MapReduce (EMR)Amazon Web Services

En vedette (8)

Hw09 Map Reduce Over Tahoe A Least Authority Encrypted Distributed Filesy...

Hw09 Next Steps For Hadoop

Hw09 Protein Alignment

Hadoop Summit 2012 | HDFS High Availability

Strata + Hadoop World 2012: Apache HBase Features for the Enterprise

Hadoop Lecture for Harvard's CS 264 -- October 19, 2009

Hw09 Sqoop Database Import For Hadoop

Masterclass Webinar - Amazon Elastic MapReduce (EMR)

Similaire à Hw09 Counting And Clustering And Other Data Tricks

Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...Cloudera, Inc.

HadoopWorkshopJuly2014Dieter De Witte

Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...Precisely

Foundation for Success: How Big Data Fits in an Information ArchitectureInside Analysis

SendGrid Improves Email Delivery with Hybrid Data WarehousingAmazon Web Services

Data & Analytics - Session 1 - Big Data AnalyticsAmazon Web Services

Amazon Elastic Map Reduce - Ian Meyershuguk

Agile data warehousingSneha Challa

Analytics on the Cloud with Tableau on AWSAmazon Web Services

Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Etha...HostedbyConfluent

AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...Amazon Web Services

Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...Amazon Web Services

Big Data Meetup #7Paul Lo

SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)Sascha Dittmann

Graph databases: Tinkerpop and Titan DBMohamed Taher Alrefaie

Hadoop basicsAntonio Silveira

Владимир Слободянюк «DWH & BigData – architecture approaches»Anna Shymchenko

DWH & big data architecture approachesLuxoft

L'architettura di classe enterprise di nuova generazione - Massimo BrignoliData Driven Innovation

Design Choices for Cloud Data PlatformsAshish Mrig

Similaire à Hw09 Counting And Clustering And Other Data Tricks (20)

Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...

HadoopWorkshopJuly2014

Big Data Goes Airborne. Propelling Your Big Data Initiative with Ironcluster ...

Foundation for Success: How Big Data Fits in an Information Architecture

SendGrid Improves Email Delivery with Hybrid Data Warehousing

Data & Analytics - Session 1 - Big Data Analytics

Amazon Elastic Map Reduce - Ian Meyers

Agile data warehousing

Analytics on the Cloud with Tableau on AWS

Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Etha...

AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...

Scaling your Analytics with Amazon Elastic MapReduce (BDT301) | AWS re:Invent...

Big Data Meetup #7

SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)

Graph databases: Tinkerpop and Titan DB

Hadoop basics

Владимир Слободянюк «DWH & BigData – architecture approaches»

DWH & big data architecture approaches

L'architettura di classe enterprise di nuova generazione - Massimo Brignoli

Design Choices for Cloud Data Platforms

Plus de Cloudera, Inc.

Partner Briefing_January 25 (FINAL).pptxCloudera, Inc.

Cloudera Data Impact Awards 2021 - Finalists Cloudera, Inc.

2020 Cloudera Data Impact Awards FinalistsCloudera, Inc.

Edc event vienna presentation 1 oct 2019Cloudera, Inc.

Machine Learning with Limited Labeled Data 4/3/19Cloudera, Inc.

Data Driven With the Cloudera Modern Data Warehouse 3.19.19Cloudera, Inc.

Introducing Cloudera DataFlow (CDF) 2.13.19Cloudera, Inc.

Introducing Cloudera Data Science Workbench for HDP 2.12.19Cloudera, Inc.

Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19Cloudera, Inc.

Leveraging the cloud for analytics and machine learning 1.29.19Cloudera, Inc.

Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19Cloudera, Inc.

Leveraging the Cloud for Big Data Analytics 12.11.18Cloudera, Inc.

Modern Data Warehouse Fundamentals Part 3Cloudera, Inc.

Modern Data Warehouse Fundamentals Part 2Cloudera, Inc.

Modern Data Warehouse Fundamentals Part 1Cloudera, Inc.

Extending Cloudera SDX beyond the PlatformCloudera, Inc.

Federated Learning: ML with Privacy on the Edge 11.15.18Cloudera, Inc.

Analyst Webinar: Doing a 180 on Customer 360Cloudera, Inc.

Build a modern platform for anti-money laundering 9.19.18Cloudera, Inc.

Introducing the data science sandbox as a service 8.30.18Cloudera, Inc.

Plus de Cloudera, Inc. (20)

Partner Briefing_January 25 (FINAL).pptx

Cloudera Data Impact Awards 2021 - Finalists

2020 Cloudera Data Impact Awards Finalists

Edc event vienna presentation 1 oct 2019

Machine Learning with Limited Labeled Data 4/3/19

Data Driven With the Cloudera Modern Data Warehouse 3.19.19

Introducing Cloudera DataFlow (CDF) 2.13.19

Introducing Cloudera Data Science Workbench for HDP 2.12.19

Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19

Leveraging the cloud for analytics and machine learning 1.29.19

Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19

Leveraging the Cloud for Big Data Analytics 12.11.18

Modern Data Warehouse Fundamentals Part 3

Modern Data Warehouse Fundamentals Part 2

Modern Data Warehouse Fundamentals Part 1

Extending Cloudera SDX beyond the Platform

Federated Learning: ML with Privacy on the Edge 11.15.18

Analyst Webinar: Doing a 180 on Customer 360

Build a modern platform for anti-money laundering 9.19.18

Introducing the data science sandbox as a service 8.30.18

Dernier

Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765

Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal

MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar

So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda

Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq

Data governance with Unity Catalog PresentationKnoldus Inc.

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3

Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3

How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe

Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3

The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech

React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech

A Framework for Development in the AI AgeCprime

How to write a Business Continuity PlanDatabarracks

QCon London: Mastering long-running processes in modern architecturesBernd Ruecker

The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney

Scale your database traffic with Read & Write split using MySQL RouterMydbops

Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh

Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3

Dernier (20)

Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration

Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...

MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes

So einfach geht modernes Roaming fuer Notes und Nomad.pdf

Genislab builds better products and faster go-to-market with Lean project man...

Data governance with Unity Catalog Presentation

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx

Glenn Lazarus- Why Your Observability Strategy Needs Security Observability

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx

How AI, OpenAI, and ChatGPT impact business and software.

Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx

The Ultimate Guide to Choosing WordPress Pros and Cons

React Native vs Ionic - The Best Mobile App Framework

A Framework for Development in the AI Age

How to write a Business Continuity Plan

QCon London: Mastering long-running processes in modern architectures

The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...

Scale your database traffic with Read & Write split using MySQL Router

Generative AI - Gitex v1Generative AI - Gitex v1.pptx

Digital Identity is Under Attack: FIDO Paris Seminar.pptx

Hw09 Counting And Clustering And Other Data Tricks

1. Cheap Parlor Tricks, Counting, and Clustering Derek Gottfrid The New York Times October 2009

2. Evolution of Hadoop @ NYTimes.com

10.

11.

12. TimesMachine http://timesmachine.nytimes.com

13.

14.

15.

16.

17. Usage Data July 2009 ???M Page Views ??M Unique Users

18.

19. Twitter Click Backs By Age Group July 2009

20.

21. Usage Data combined with Article Data July 2009 40 Articles

22. Usage Data combined with Article Data July 2009 40 Articles

23.

24.

25. Clustering

26. Clustering

27.

28. Questions? [email_address] @derekg http://open.nytimes.com/

Notes de l'éditeur

The ability to compute across large data is a key to success. Accumulating data is easy. Computing is hard Personify, Omniture, Webtrends - specialized not general computing tools.
Fixed reports are great and we can do that but the really interesting part is asking questions after the fact in an adhoc exploratory manner.
We have always had data. Collecting data is not a issue for us. We are okay at that part.
We have always had data. Collecting data is not a issue for us. We are okay at that part.
We have always had data. Collecting data is not a issue for us. We are okay at that part.
Distribution of page views per users for the July 2009. Most users view 1 Page. Not new but shows we have some mastery over the data.This is based of just user data - 380G of compressed data covering over 700million page views.
We have always had data. Collecting data is not a issue for us. We are okay at that part.
We have always had data. Collecting data is not a issue for us. We are okay at that part.
mapping, collbrative filtering, segementation,
We have always had data. Collecting data is not a issue for us. We are okay at that part.
We have always had data. Collecting data is not a issue for us. We are okay at that part.
We have always had data. Collecting data is not a issue for us. We are okay at that part.