Dark Data: A Data Scientists Exploration of the Unknown by Rob Witoff PyData SV 2014
•
3 j'aime•1,406 vues
Modern Data Science is enabling NASA's engineers uncover actionable information from our "dark" data coffers. From starting small to operating at scale, Rob will discuss applications in telemetry, workforce analytics and liberating data from the Mars Rovers. Tools include iPython, Pandas, Boto and more.
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Dark Data: A Data Scientists Exploration of the Unknown by Rob Witoff PyData SV 2014
Similaire à Dark Data: A Data Scientists Exploration of the Unknown by Rob Witoff PyData SV 2014 (20)
Plus de PyData
Plus de PyData (20)
Dernier
Dernier (20)
Dark Data: A Data Scientists Exploration of the Unknown by Rob Witoff PyData SV 2014
- 1. Dark Data A Data Scientist’s Exploration of the Unknown Rob Witoff ! Data Scientist IT CTO Office @rwitoff Jet Propulsion Laboratory California Institute of Technology
- 4. Exploration brings home data …a lot of it
- 5. This talk is about how we’re enabling _ to uncover and act on our Dark Data
- 11. Goldstone, USA Madrid, Spain Canberra, Australia
- 12. Amazon SWF Decider Data Processing Tasks File Transfer Tasks Decision Tasks Data Processing Workers EC2 EC2 EC2 EC2 S3 JPL Data Center Decider File Transfer Workers Data Processing Workers Polyphony Cloud Computing
- 13. ! ! ! ! !
- 16. 1967
- 17. Today
- 19. Dark Matter ! ! ! 84.5% of the known Universe
- 20. Dark Data ?? % of our universe
- 21. Dark Data is the data all around us, that we can’t use today.can’t use
- 23. can’t imagine
- 24. “If this species is to survive indefinitely we need to become a multi-planet species. We need to go to Mars, and Mars is a stepping stone to other solar systems.” ! NASA Administrator, Charlie Bolden
- 26. Mars 2020
- 28. And Beyond!
- 30. “After exploring the universe for decades, what unfound discoveries lie in our data?”
- 34. http://www.cgtrader.com/3d-models/character-people/man/nasa-astronaut-apollo Tools Software & Libs World Watching Open Source Gloves Interpreted Languages Camera Data Viz Mission Control Active Communities Vehicles Scalable Storage & Compute
- 36. Liberate
- 37. JPL Data Center Decider File Transfer Workers Data Processing Workers Polyphony Amazon SWF Decider Data Processing Tasks File Transfer Tasks Decision Tasks Data Processing Workers EC2 EC2 EC2 EC2 S3 Cloud Computing
- 39. Closed Silos
- 42. How Do you Liberate New Data?
- 43. Greedy Solution: Explore For Data Wise Solution: Explore For Problems Best Solution: Explore For Questions
- 44. Greedy Solution: Explore For Data Find APIs!
- 45. Greedy Solution: Explore For Data Explore Find Get Excited! Find APIs! miss the point
- 46. Greedy Solution: Explore For Data Explore :-( Data is a means, not the end Find
- 47. Wise Solution: Explore For Problems Explore Integrate Increment! Incremental Successes!
- 48. Wise Solution: Explore For Problems Explore Integrate :-/ Incremental Expectations.
- 49. Best Solution: Explore For Questions Explore Rapid Prototype Like it’s 2014 Reset Expectations
- 53. Reddit!
- 56. Best Solution: Explore For Questions
- 61. Data What can we do with this brain data?
- 63. Data Problem How healthy is my lobe?
- 65. Data Problem Question What if we could see the brain evolve?
- 67. Simple Sankey Flow Diagram
- 71. Added more data.
- 73. Data ProblemIdea What about the rest of our brain data?
- 74. what about this? Or this?
- 75. what about this? Or this? ?
- 79. People ?Projects
- 83. ? Investments Helpdesk Finance Sentiment? Resumes ecruiting People Engineering Science Exploration Degrees Confer HR Projects Orgs Calendars
- 84. ? Investments Helpdesk Finance Sentiment? Resumes ecruiting People Engineering Science Exploration Degrees Confer HR Projects Orgs Calendars Connect Your Dots
- 85. Enable
- 87. Expertise Data
- 97. “Human Problems Won’t be Solved by Root Mean Square Error” -Drew Conway
- 98. Engage
- 102. Engage before you Answer
- 110. Outside the Notebook
- 111. 12k Interesting Files 12k Documents Dynamo Results ReST
- 112. Making a Difference
- 113. Amazon SWF Decider Data Processing Tasks File Transfer Tasks Decision Tasks Data Processing Workers EC2 EC2 EC2 EC2 S3 JPL Data Center Decider File Transfer Workers Data Processing Workers Polyphony
- 115. Q What if we could interact with ALL of our Data?
- 120. QWhat if we were even closer to our data?
- 127. Liberate your Dark Data Enable your Engineers Let’s Grow Data Science Together
- 128. Thank you! @rwitoff