THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

•

1 j'aime•626 vues

Gigaom

Presentation from Robert Lefkowitz, 1010data #dataconf More at http://event.gigaom.com/structuredata/

Technologie Business

THE TRILLION-ROW SPREADSHEET™
SPEAKER: Robert Lefkowitz
Director of Web
Development
1010data

Friday, July 27, 2012

The Trillion Row Spreadsheet
r0ml@1010data.com

2
Friday, July 27, 2012

Trillion Row Spreadsheet

5
Friday, July 27, 2012

Trillion Row Spreadsheet
Trillion Row Data Warehouse

6
Friday, July 27, 2012

Data Warehouse
• Operational -> Analytical
- Design/Model
- Data (cleaning)
- Multiple data sources
- Date (timeseries) analytics
- Reporting (business intelligence)

12
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

14
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

15
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

16
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

17
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

18
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Spreadsheet

19
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Web based
Spreadsheet

20
Friday, July 27, 2012

Spreadsheets lack ... ?
• Content management
• Business process management
• Automated scheduling
• Authentication
• Audit
• Collaboration
• Enterprise level roll-up

21
Friday, July 27, 2012

Spreadsheets lack ... ?
• Scale

22
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Web-based
Spreadsheet

23
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse
Spreadsheet

Web-based
Spreadsheet

24
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse
Spreadsheet

If this is fast enough, you can
recalculate the summaries
from the detail on the fly....

Web-based
Spreadsheet

25
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Web-based
Spreadsheet

26
Friday, July 27, 2012

Spreadsheets lack ... ?
• Professional programming practices
- Requirements analysis
- Formal regression testing
- Model/View separation
- Object-Orientation
- Static typing
- Naming conventions

28
Friday, July 27, 2012

The Trillion Row Spreadsheet
r0ml@1010data.com

30
Friday, July 27, 2012

Recommandé

Adam and tennis and ping pong BentonGeometry

BIG DATA: AN AUGMENTED INTELLIGENCE FOR STRATEGIC DECISION MAKING from Struct...Gigaom

REALIZING REAL-TIME VALUE ON THE REAL-TIME WEB from Structure:Data 2012Gigaom

THE DATA WAREHOUSING DEBATE: TRADITIONAL vs. OPEN SOURCE from Structure:Data ...Gigaom

鱼与熊掌－软件质量和交付速度andyhu1007

CLOUD LOCK-IN AND THE MYTH OF OPEN APIS at Structure 2012Gigaom

Continous Delivery in Actionandyhu1007

Big data - teams not technologyUpside Energy Ltd

Recommandé

Adam and tennis and ping pong BentonGeometry

BIG DATA: AN AUGMENTED INTELLIGENCE FOR STRATEGIC DECISION MAKING from Struct...Gigaom

REALIZING REAL-TIME VALUE ON THE REAL-TIME WEB from Structure:Data 2012Gigaom

THE DATA WAREHOUSING DEBATE: TRADITIONAL vs. OPEN SOURCE from Structure:Data ...Gigaom

鱼与熊掌－软件质量和交付速度andyhu1007

CLOUD LOCK-IN AND THE MYTH OF OPEN APIS at Structure 2012Gigaom

Continous Delivery in Actionandyhu1007

Big data - teams not technologyUpside Energy Ltd

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012Gigaom

Digitale fabriek - I2 - DsquareSirris

Investigative Analytics- What's in a Data Scientists ToolboxData Science London

PhD thesis defense of Ajith RanabahuArtificial Intelligence Institute at UofSC

SPONSORED WORKSHOP by Amplidata from Structure:Data 2012: Gigaom

THE STATE OF GLOBAL INFRASTRUCTURE PERFORMANCE from Structure 2012Gigaom

Data warehouseRishabh Dogra

Jisc RSC Eastern Microsoft Briefing - Ridgian BI Jisc RSC Eastern 19th march ...JISC RSC Eastern

Ms system centar 2012 hp-badurini prezentacijaTomislav Lulic

Evolutionary Development MethodologyDonna Kelly

THE 3V’S OF BIG DATA: VARIETY, VELOCITY, and VOLUMEGigaom

Final Year Project GuidanceVarad Meru

Introduction to Data WarehousingTameem Ahmad

The 5 levels of embedded biMike Boyarski

Campus Party Recife - Sua primeira e próximas aplicações Android: como fazer ...Nelson Glauber Leal

Structure 2014 - The strategic value of the cloud - Joe WeinmanGigaom

Structure 2014 - The right and wrong way to scale - RackspaceGigaom

Structure 2014 - The future of cloud computing survey resultsGigaom

Structure 2014 - Launchpad CompetitionGigaom

Structure 2014 - Disrupting the data center - Intel sponsor workshopGigaom

Structure 2014 - Cloud trends - BatteryGigaom

Contenu connexe

Similaire à THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012Gigaom

Digitale fabriek - I2 - DsquareSirris

Investigative Analytics- What's in a Data Scientists ToolboxData Science London

PhD thesis defense of Ajith RanabahuArtificial Intelligence Institute at UofSC

SPONSORED WORKSHOP by Amplidata from Structure:Data 2012: Gigaom

THE STATE OF GLOBAL INFRASTRUCTURE PERFORMANCE from Structure 2012Gigaom

Data warehouseRishabh Dogra

Jisc RSC Eastern Microsoft Briefing - Ridgian BI Jisc RSC Eastern 19th march ...JISC RSC Eastern

Ms system centar 2012 hp-badurini prezentacijaTomislav Lulic

Evolutionary Development MethodologyDonna Kelly

THE 3V’S OF BIG DATA: VARIETY, VELOCITY, and VOLUMEGigaom

Final Year Project GuidanceVarad Meru

Introduction to Data WarehousingTameem Ahmad

The 5 levels of embedded biMike Boyarski

Campus Party Recife - Sua primeira e próximas aplicações Android: como fazer ...Nelson Glauber Leal

Similaire à THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012 (16)

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012

Digitale fabriek - I2 - Dsquare

Investigative Analytics- What's in a Data Scientists Toolbox

PhD thesis defense of Ajith Ranabahu

SPONSORED WORKSHOP by Amplidata from Structure:Data 2012:

THE STATE OF GLOBAL INFRASTRUCTURE PERFORMANCE from Structure 2012

Data warehouse

Jisc RSC Eastern Microsoft Briefing - Ridgian BI Jisc RSC Eastern 19th march ...

Ms system centar 2012 hp-badurini prezentacija

Evolutionary Development Methodology

THE 3V’S OF BIG DATA: VARIETY, VELOCITY, and VOLUME

Final Year Project Guidance

Introduction to Data Warehousing

The 5 levels of embedded bi

Campus Party Recife - Sua primeira e próximas aplicações Android: como fazer ...

Plus de Gigaom

Structure 2014 - The strategic value of the cloud - Joe WeinmanGigaom

Structure 2014 - The right and wrong way to scale - RackspaceGigaom

Structure 2014 - The future of cloud computing survey resultsGigaom

Structure 2014 - Launchpad CompetitionGigaom

Structure 2014 - Disrupting the data center - Intel sponsor workshopGigaom

Structure 2014 - Cloud trends - BatteryGigaom

Structure Data 2014: HOW MICRODATA CAN SAY A LOT ABOUT MACROECONOMICS, David ...Gigaom

Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...Gigaom

Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit BendovGigaom

Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...Gigaom

Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA, Gigaom

Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari GesherGigaom

Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris HaddadGigaom

Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...Gigaom

Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrathGigaom

Structure Data 2014: IS VIDEO BIG DATA?, Steve RussellGigaom

Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan WaiteGigaom

How Data is Remaking E-commerce - from Roadmap 2013Gigaom

25 Favorite Experiences in Tech - from Roadmap 2013Gigaom

How Moore’s Law is Influencing Design - from Roadmap 2013Gigaom

Plus de Gigaom (20)

Structure 2014 - The strategic value of the cloud - Joe Weinman

Structure 2014 - The right and wrong way to scale - Rackspace

Structure 2014 - The future of cloud computing survey results

Structure 2014 - Launchpad Competition

Structure 2014 - Disrupting the data center - Intel sponsor workshop

Structure 2014 - Cloud trends - Battery

Structure Data 2014: HOW MICRODATA CAN SAY A LOT ABOUT MACROECONOMICS, David ...

Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...

Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov

Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...

Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,

Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher

Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad

Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...

Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath

Structure Data 2014: IS VIDEO BIG DATA?, Steve Russell

Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite

How Data is Remaking E-commerce - from Roadmap 2013

25 Favorite Experiences in Tech - from Roadmap 2013

How Moore’s Law is Influencing Design - from Roadmap 2013

Dernier

Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j

08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls

Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix

GenCyber Cyber Security Day PresentationMichael W. Hawkins

Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst

IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge

How to convert PDF to text with Nanonetsnaman860154

Next-generation AAM aircraft unveiled by Supernal, S-A2Hyundai Motor Group

08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls

WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal

Artificial intelligence in the post-deep learning eraDeakin University

AI as an Interface for Commercial BuildingsMemoori

Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard

Key Features Of Token Development (1).pptxLBM Solutions

Understanding the Laravel MVC ArchitecturePixlogix Infotech

Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC

Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106

Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada

E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxnull - The Open Security Community

Vulnerability_Management_GRC_by Sohang Sengupta.pptxnull - The Open Security Community

Dernier (20)

Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...

08448380779 Call Girls In Civil Lines Women Seeking Men

Swan(sea) Song – personal research during my six years at Swansea ... and bey...

GenCyber Cyber Security Day Presentation

Human Factors of XR: Using Human Factors to Design XR Systems

IAC 2024 - IA Fast Track to Search Focused AI Solutions

How to convert PDF to text with Nanonets

Next-generation AAM aircraft unveiled by Supernal, S-A2

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service

Artificial intelligence in the post-deep learning era

AI as an Interface for Commercial Buildings

Maximizing Board Effectiveness 2024 Webinar.pptx

Key Features Of Token Development (1).pptx

Understanding the Laravel MVC Architecture

Breaking the Kubernetes Kill Chain: Host Path Mount

Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics

Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024

E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx

Vulnerability_Management_GRC_by Sohang Sengupta.pptx

THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

1. THE TRILLION-ROW SPREADSHEET™ SPEAKER: Robert Lefkowitz Director of Web Development 1010data Friday, July 27, 2012

2. The Trillion Row Spreadsheet r0ml@1010data.com 2 Friday, July 27, 2012

3. Friday, July 27, 2012

4. 4 Friday, July 27, 2012

5. Trillion Row Spreadsheet 5 Friday, July 27, 2012

6. Trillion Row Spreadsheet Trillion Row Data Warehouse 6 Friday, July 27, 2012

7. 7 Friday, July 27, 2012

8. 8 Friday, July 27, 2012

9. 9 Friday, July 27, 2012

10. 10 Friday, July 27, 2012

11. 11 Friday, July 27, 2012

12. Data Warehouse • Operational -> Analytical - Design/Model - Data (cleaning) - Multiple data sources - Date (timeseries) analytics - Reporting (business intelligence) 12 Friday, July 27, 2012

13. 13 Friday, July 27, 2012

14. Operational Data Operational Data Operational Data Data Warehouse 14 Friday, July 27, 2012

15. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart 15 Friday, July 27, 2012

16. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 16 Friday, July 27, 2012

17. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 17 Friday, July 27, 2012

18. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 18 Friday, July 27, 2012

19. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet 19 Friday, July 27, 2012

20. Operational Data Operational Data Operational Data Data Warehouse Web based Spreadsheet 20 Friday, July 27, 2012

21. Spreadsheets lack ... ? • Content management • Business process management • Automated scheduling • Authentication • Audit • Collaboration • Enterprise level roll-up 21 Friday, July 27, 2012

22. Spreadsheets lack ... ? • Scale 22 Friday, July 27, 2012

23. Operational Data Operational Data Operational Data Data Warehouse Web-based Spreadsheet 23 Friday, July 27, 2012

24. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet Web-based Spreadsheet 24 Friday, July 27, 2012

25. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet If this is fast enough, you can recalculate the summaries from the detail on the fly.... Web-based Spreadsheet 25 Friday, July 27, 2012

26. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Web-based Spreadsheet 26 Friday, July 27, 2012

27. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet If this is fast enough, you can recalculate the summaries from the detail on the fly.... Web-based Spreadsheet 27 Friday, July 27, 2012

28. Spreadsheets lack ... ? • Professional programming practices - Requirements analysis - Formal regression testing - Model/View separation - Object-Orientation - Static typing - Naming conventions 28 Friday, July 27, 2012

29. 29 Friday, July 27, 2012

30. The Trillion Row Spreadsheet r0ml@1010data.com 30 Friday, July 27, 2012

31. Friday, July 27, 2012