THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

•

1 j'aime•626 vues

Gigaom

Presentation from Robert Lefkowitz, 1010data #dataconf More at http://event.gigaom.com/structuredata/

Technologie Business

THE TRILLION-ROW SPREADSHEET™
SPEAKER: Robert Lefkowitz
Director of Web
Development
1010data

Friday, July 27, 2012

The Trillion Row Spreadsheet
r0ml@1010data.com

2
Friday, July 27, 2012

Trillion Row Spreadsheet

5
Friday, July 27, 2012

Trillion Row Spreadsheet
Trillion Row Data Warehouse

6
Friday, July 27, 2012

Data Warehouse
• Operational -> Analytical
- Design/Model
- Data (cleaning)
- Multiple data sources
- Date (timeseries) analytics
- Reporting (business intelligence)

12
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

14
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

15
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

16
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

17
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

18
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Spreadsheet

19
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Web based
Spreadsheet

20
Friday, July 27, 2012

Spreadsheets lack ... ?
• Content management
• Business process management
• Automated scheduling
• Authentication
• Audit
• Collaboration
• Enterprise level roll-up

21
Friday, July 27, 2012

Spreadsheets lack ... ?
• Scale

22
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Web-based
Spreadsheet

23
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse
Spreadsheet

Web-based
Spreadsheet

24
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse
Spreadsheet

If this is fast enough, you can
recalculate the summaries
from the detail on the fly....

Web-based
Spreadsheet

25
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Web-based
Spreadsheet

26
Friday, July 27, 2012

Spreadsheets lack ... ?
• Professional programming practices
- Requirements analysis
- Formal regression testing
- Model/View separation
- Object-Orientation
- Static typing
- Naming conventions

28
Friday, July 27, 2012

The Trillion Row Spreadsheet
r0ml@1010data.com

30
Friday, July 27, 2012

Recommandé

Adam and tennis and ping pong BentonGeometry

BIG DATA: AN AUGMENTED INTELLIGENCE FOR STRATEGIC DECISION MAKING from Struct...Gigaom

REALIZING REAL-TIME VALUE ON THE REAL-TIME WEB from Structure:Data 2012Gigaom

THE DATA WAREHOUSING DEBATE: TRADITIONAL vs. OPEN SOURCE from Structure:Data ...Gigaom

鱼与熊掌－软件质量和交付速度andyhu1007

CLOUD LOCK-IN AND THE MYTH OF OPEN APIS at Structure 2012Gigaom

Continous Delivery in Actionandyhu1007

Big data - teams not technologyUpside Energy Ltd

Recommandé

Adam and tennis and ping pong BentonGeometry

BIG DATA: AN AUGMENTED INTELLIGENCE FOR STRATEGIC DECISION MAKING from Struct...Gigaom

REALIZING REAL-TIME VALUE ON THE REAL-TIME WEB from Structure:Data 2012Gigaom

THE DATA WAREHOUSING DEBATE: TRADITIONAL vs. OPEN SOURCE from Structure:Data ...Gigaom

鱼与熊掌－软件质量和交付速度andyhu1007

CLOUD LOCK-IN AND THE MYTH OF OPEN APIS at Structure 2012Gigaom

Continous Delivery in Actionandyhu1007

Big data - teams not technologyUpside Energy Ltd

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012Gigaom

Digitale fabriek - I2 - DsquareSirris

Investigative Analytics- What's in a Data Scientists ToolboxData Science London

PhD thesis defense of Ajith RanabahuArtificial Intelligence Institute at UofSC

SPONSORED WORKSHOP by Amplidata from Structure:Data 2012: Gigaom

THE STATE OF GLOBAL INFRASTRUCTURE PERFORMANCE from Structure 2012Gigaom

Data warehouseRishabh Dogra

Jisc RSC Eastern Microsoft Briefing - Ridgian BI Jisc RSC Eastern 19th march ...JISC RSC Eastern

Ms system centar 2012 hp-badurini prezentacijaTomislav Lulic

Evolutionary Development MethodologyDonna Kelly

THE 3V’S OF BIG DATA: VARIETY, VELOCITY, and VOLUMEGigaom

Final Year Project GuidanceVarad Meru

Introduction to Data WarehousingTameem Ahmad

The 5 levels of embedded biMike Boyarski

Campus Party Recife - Sua primeira e próximas aplicações Android: como fazer ...Nelson Glauber Leal

Structure 2014 - The strategic value of the cloud - Joe WeinmanGigaom

Structure 2014 - The right and wrong way to scale - RackspaceGigaom

Structure 2014 - The future of cloud computing survey resultsGigaom

Structure 2014 - Launchpad CompetitionGigaom

Structure 2014 - Disrupting the data center - Intel sponsor workshopGigaom

Structure 2014 - Cloud trends - BatteryGigaom

Contenu connexe

Similaire à THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012Gigaom

Digitale fabriek - I2 - DsquareSirris

Investigative Analytics- What's in a Data Scientists ToolboxData Science London

PhD thesis defense of Ajith RanabahuArtificial Intelligence Institute at UofSC

SPONSORED WORKSHOP by Amplidata from Structure:Data 2012: Gigaom

THE STATE OF GLOBAL INFRASTRUCTURE PERFORMANCE from Structure 2012Gigaom

Data warehouseRishabh Dogra

Jisc RSC Eastern Microsoft Briefing - Ridgian BI Jisc RSC Eastern 19th march ...JISC RSC Eastern

Ms system centar 2012 hp-badurini prezentacijaTomislav Lulic

Evolutionary Development MethodologyDonna Kelly

THE 3V’S OF BIG DATA: VARIETY, VELOCITY, and VOLUMEGigaom

Final Year Project GuidanceVarad Meru

Introduction to Data WarehousingTameem Ahmad

The 5 levels of embedded biMike Boyarski

Campus Party Recife - Sua primeira e próximas aplicações Android: como fazer ...Nelson Glauber Leal

Similaire à THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012 (16)

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012

Digitale fabriek - I2 - Dsquare

Investigative Analytics- What's in a Data Scientists Toolbox

PhD thesis defense of Ajith Ranabahu

SPONSORED WORKSHOP by Amplidata from Structure:Data 2012:

THE STATE OF GLOBAL INFRASTRUCTURE PERFORMANCE from Structure 2012

Data warehouse

Jisc RSC Eastern Microsoft Briefing - Ridgian BI Jisc RSC Eastern 19th march ...

Ms system centar 2012 hp-badurini prezentacija

Evolutionary Development Methodology

THE 3V’S OF BIG DATA: VARIETY, VELOCITY, and VOLUME

Final Year Project Guidance

Introduction to Data Warehousing

The 5 levels of embedded bi

Campus Party Recife - Sua primeira e próximas aplicações Android: como fazer ...

Plus de Gigaom

Structure 2014 - The strategic value of the cloud - Joe WeinmanGigaom

Structure 2014 - The right and wrong way to scale - RackspaceGigaom

Structure 2014 - The future of cloud computing survey resultsGigaom

Structure 2014 - Launchpad CompetitionGigaom

Structure 2014 - Disrupting the data center - Intel sponsor workshopGigaom

Structure 2014 - Cloud trends - BatteryGigaom

Structure Data 2014: HOW MICRODATA CAN SAY A LOT ABOUT MACROECONOMICS, David ...Gigaom

Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...Gigaom

Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit BendovGigaom

Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...Gigaom

Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA, Gigaom

Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari GesherGigaom

Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris HaddadGigaom

Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...Gigaom

Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrathGigaom

Structure Data 2014: IS VIDEO BIG DATA?, Steve RussellGigaom

Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan WaiteGigaom

How Data is Remaking E-commerce - from Roadmap 2013Gigaom

25 Favorite Experiences in Tech - from Roadmap 2013Gigaom

How Moore’s Law is Influencing Design - from Roadmap 2013Gigaom

Plus de Gigaom (20)

Structure 2014 - The strategic value of the cloud - Joe Weinman

Structure 2014 - The right and wrong way to scale - Rackspace

Structure 2014 - The future of cloud computing survey results

Structure 2014 - Launchpad Competition

Structure 2014 - Disrupting the data center - Intel sponsor workshop

Structure 2014 - Cloud trends - Battery

Structure Data 2014: HOW MICRODATA CAN SAY A LOT ABOUT MACROECONOMICS, David ...

Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...

Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov

Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...

Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,

Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher

Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad

Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...

Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath

Structure Data 2014: IS VIDEO BIG DATA?, Steve Russell

Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite

How Data is Remaking E-commerce - from Roadmap 2013

25 Favorite Experiences in Tech - from Roadmap 2013

How Moore’s Law is Influencing Design - from Roadmap 2013

Dernier

Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen

Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA

Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda

How to write a Business Continuity PlanDatabarracks

Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3

Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq

Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll

Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes

Data governance with Unity Catalog PresentationKnoldus Inc.

UiPath Community: Communication Mining from Zero to HeroUiPathCommunity

The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3

Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein

What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina

Connecting the Dots for Information Discovery.pdfNeo4j

A Framework for Development in the AI AgeCprime

Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3

Sample pptx for embedding into website for demoHarshalMandlekar2

Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan

From Family Reminiscence to Scholarly Archive .Alan Dix

Dernier (20)

Testing tools and AI - ideas what to try with some tool examples

Long journey of Ruby standard library at RubyConf AU 2024

Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...

How to write a Business Continuity Plan

Digital Identity is Under Attack: FIDO Paris Seminar.pptx

Genislab builds better products and faster go-to-market with Lean project man...

Emixa Mendix Meetup 11 April 2024 about Mendix Native development

Assure Ecommerce and Retail Operations Uptime with ThousandEyes

Data governance with Unity Catalog Presentation

UiPath Community: Communication Mining from Zero to Hero

The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx

Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24

What is DBT - The Ultimate Data Build Tool.pdf

Connecting the Dots for Information Discovery.pdf

A Framework for Development in the AI Age

Potential of AI (Generative AI) in Business: Learnings and Insights

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx

Sample pptx for embedding into website for demo

Generative AI for Technical Writer or Information Developers

From Family Reminiscence to Scholarly Archive .

THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

1. THE TRILLION-ROW SPREADSHEET™ SPEAKER: Robert Lefkowitz Director of Web Development 1010data Friday, July 27, 2012

2. The Trillion Row Spreadsheet r0ml@1010data.com 2 Friday, July 27, 2012

3. Friday, July 27, 2012

4. 4 Friday, July 27, 2012

5. Trillion Row Spreadsheet 5 Friday, July 27, 2012

6. Trillion Row Spreadsheet Trillion Row Data Warehouse 6 Friday, July 27, 2012

7. 7 Friday, July 27, 2012

8. 8 Friday, July 27, 2012

9. 9 Friday, July 27, 2012

10. 10 Friday, July 27, 2012

11. 11 Friday, July 27, 2012

12. Data Warehouse • Operational -> Analytical - Design/Model - Data (cleaning) - Multiple data sources - Date (timeseries) analytics - Reporting (business intelligence) 12 Friday, July 27, 2012

13. 13 Friday, July 27, 2012

14. Operational Data Operational Data Operational Data Data Warehouse 14 Friday, July 27, 2012

15. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart 15 Friday, July 27, 2012

16. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 16 Friday, July 27, 2012

17. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 17 Friday, July 27, 2012

18. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 18 Friday, July 27, 2012

19. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet 19 Friday, July 27, 2012

20. Operational Data Operational Data Operational Data Data Warehouse Web based Spreadsheet 20 Friday, July 27, 2012

21. Spreadsheets lack ... ? • Content management • Business process management • Automated scheduling • Authentication • Audit • Collaboration • Enterprise level roll-up 21 Friday, July 27, 2012

22. Spreadsheets lack ... ? • Scale 22 Friday, July 27, 2012

23. Operational Data Operational Data Operational Data Data Warehouse Web-based Spreadsheet 23 Friday, July 27, 2012

24. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet Web-based Spreadsheet 24 Friday, July 27, 2012

25. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet If this is fast enough, you can recalculate the summaries from the detail on the fly.... Web-based Spreadsheet 25 Friday, July 27, 2012

26. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Web-based Spreadsheet 26 Friday, July 27, 2012

27. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet If this is fast enough, you can recalculate the summaries from the detail on the fly.... Web-based Spreadsheet 27 Friday, July 27, 2012

28. Spreadsheets lack ... ? • Professional programming practices - Requirements analysis - Formal regression testing - Model/View separation - Object-Orientation - Static typing - Naming conventions 28 Friday, July 27, 2012

29. 29 Friday, July 27, 2012

30. The Trillion Row Spreadsheet r0ml@1010data.com 30 Friday, July 27, 2012

31. Friday, July 27, 2012