Cubes - Lightweight Python OLAP (EuroPython 2012 talk)

software engineer à Facebook
6 Jul 2012
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
Cubes - Lightweight Python OLAP (EuroPython 2012 talk)
1 sur 75

Contenu connexe

Tendances

Apache Spark's Built-in File Sources in DepthApache Spark's Built-in File Sources in Depth
Apache Spark's Built-in File Sources in DepthDatabricks
The openCypher Project - An Open Graph Query LanguageThe openCypher Project - An Open Graph Query Language
The openCypher Project - An Open Graph Query LanguageNeo4j
Data Warehouse Design and Best PracticesData Warehouse Design and Best Practices
Data Warehouse Design and Best PracticesIvo Andreev
Understanding Presto - Presto meetup @ Tokyo #1Understanding Presto - Presto meetup @ Tokyo #1
Understanding Presto - Presto meetup @ Tokyo #1Sadayuki Furuhashi
SQL Analytics Powering Telemetry Analysis at ComcastSQL Analytics Powering Telemetry Analysis at Comcast
SQL Analytics Powering Telemetry Analysis at ComcastDatabricks
Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...
Apache Calcite: A Foundational Framework for Optimized Query Processing Over ...Julian Hyde

Similaire à Cubes - Lightweight Python OLAP (EuroPython 2012 talk)

Python business intelligence (PyData 2012 talk)Python business intelligence (PyData 2012 talk)
Python business intelligence (PyData 2012 talk)Stefan Urbanek
PoEAA by ExamplePoEAA by Example
PoEAA by ExamplePaulo Gandra de Sousa
Patterns of Enterprise Application Architecture (by example)Patterns of Enterprise Application Architecture (by example)
Patterns of Enterprise Application Architecture (by example)Paulo Gandra de Sousa
Dublin Ireland Spark Meetup October 15, 2015Dublin Ireland Spark Meetup October 15, 2015
Dublin Ireland Spark Meetup October 15, 2015eddiebaggott
MongoDB MeetupMongoDB Meetup
MongoDB MeetupMaxime Beugnet
Scalding big ADtaScalding big ADta
Scalding big ADtab0ris_1

Plus de Stefan Urbanek

StepTalk IntroductionStepTalk Introduction
StepTalk IntroductionStefan Urbanek
Forces and Threats in a Data Warehouse (and why metadata and architecture is ...Forces and Threats in a Data Warehouse (and why metadata and architecture is ...
Forces and Threats in a Data Warehouse (and why metadata and architecture is ...Stefan Urbanek
Sepro - introductionSepro - introduction
Sepro - introductionStefan Urbanek
New york data brewery meetup #1 – introductionNew york data brewery meetup #1 – introduction
New york data brewery meetup #1 – introductionStefan Urbanek
Cubes – pluggable model explainedCubes – pluggable model explained
Cubes – pluggable model explainedStefan Urbanek
Cubes – ways of deploymentCubes – ways of deployment
Cubes – ways of deploymentStefan Urbanek

Dernier

Google cloud Study Jam 2023.pptxGoogle cloud Study Jam 2023.pptx
Google cloud Study Jam 2023.pptxGDSCNiT
h2 meet pdf test.pdfh2 meet pdf test.pdf
h2 meet pdf test.pdfJohnLee971654
Scaling out with WordPressScaling out with WordPress
Scaling out with WordPressKonstantin Kovshenin
How resolve Gem dependencies in your code?How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?Hiroshi SHIBATA
How is AI changing journalism? Strategic considerations for publishers and ne...How is AI changing journalism? Strategic considerations for publishers and ne...
How is AI changing journalism? Strategic considerations for publishers and ne...Damian Radcliffe
Google Cloud Study Jams Info SessionGoogle Cloud Study Jams Info Session
Google Cloud Study Jams Info SessionGDSCPCCE

Dernier(20)

Cubes - Lightweight Python OLAP (EuroPython 2012 talk)

Notes de l'éditeur

  1. OLAP and Logical Model, Architecture, Slicing and Dicing, HTTP Server, SQL Backend\n\n
  2. \n
  3. \n
  4. Q: Who is familiar with OLAP?\n
  5. quick setup and reporting\ndoes not cover everything (intentionally)\n
  6. example application - public procurements of slovakia\n
  7. quick setup and reporting\ndoes not cover everything (intentionally)\n
  8. will talk about modelling first, then reporting, then going to mix\n
  9. how it looks like and what it does?\n
  10. FIXME: add slicer tool here\n
  11. not going into details, but just to align terminology and define context\n
  12. not so rare we see creating reports directly from what is available, instead of starting with business needs and tryig to find a way how to derive it from what is available\n
  13. different approach to data use, different needs\nwhile in apps you are focusing on transactions - trans data/oltp, in reporting you are focusing on analysis -> analytical data\nlogically separate (does not have to be physically separate)\n
  14. \n
  15. \n
  16. \n
  17. CONTEXT: where did the sale happened? who signed the contract?\nFILTER: how much was spent for construction work?\nAGGREGATION SCOPE: what was the revenue by country?\n\nused for ordering or sorting\ndefine master-detail relationships\n
  18. \n
  19. \n
  20. provides metadata to easily create apps\n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. what the browser does?\n
  28. aggregating measures\n
  29. \n
  30. aggregation browser has to have concrete backend implementation\n
  31. + bunch of other stuff\n
  32. context\n
  33. before I will talk about aggregation browser, I have to introduce a cell\n
  34. \n
  35. \n
  36. our filter/selection defines the cell\nthis is kind of multidimensional “breadcrumbs”\n
  37. path - taken from file system terminology for easier understanding\nthose are keys\nnote that displayed is level label, not a key\n
  38. ... let’s put it into a picture\n
  39. \n
  40. “aggregation result” was created according to usual report look\n
  41. FIXME: add picture\n
  42. you can specify multiple dimensions and explicit level to be drilled down (for example “month” level of a date dimension)\n
  43. it provides list of records, which are represented as dictionaries \nyou have to find out which one is level attribute or the key\n\n
  44. no need to find the context of dimension of interest\nif not sufficient, one can still fall-back to the manual method\n
  45. \n
  46. facts – get details\nvalues - can be used to create selection boxes, also level can be specified\ncell_details is used for creating the multidimensional breadcrumbs mentioned before - it contains data to humanly describe current context of interest\nordering and pagination is supported\n
  47. what was that “cell” thing?\n
  48. \n
  49. also show hierarchy\n
  50. \n
  51. \n
  52. same drilldown, different cell\n
  53. implicit: raises error if current level is the last one\nexample: you are exploring year 2010 (cell) and would like to see split by year (higher level)\n
  54. \n
  55. \n
  56. \n
  57. \n
  58. \n
  59. \n
  60. \n
  61. \n
  62. just to name a few...\n
  63. \n
  64. \n
  65. \n
  66. \n
  67. \n
  68. \n
  69. \n
  70. \n
  71. \n
  72. powered by sqlalchemy\n
  73. powered by great abstraction framework\nconstruction of SQL statements\n
  74. \n
  75. \n
  76. \n
  77. denormalized\n
  78. thanks to new browser and browsing context it is possible to transparently switch between original snowflake and generated denormalized view (which can be materialized and indexed based on dimension level keys)\n
  79. in which table and which column is the attribute?\n
  80. \n
  81. \n
  82. \n
  83. \n
  84. if someone would like to contribute with his skills, he is more than welcome and I will help\n
  85. so if you have OS app, like Django that more users use, you can publish reporting model for others.\nput your cube in the Wiki\n
  86. \n
  87. MIT license\n
  88. \n
  89. \n
  90. \n