ThoughtWorkers David Elliman and Ashok Subramanian present how the big data world is moving quickly with predictions of amazing industry growth. For more information on how the 'Internet of Things' is playing an increasingly larger role, read David's blog post or watch the video from the London-based event. http://www.thoughtworks.com/insights/blog/big-data-and-internet-things
26. Key Takeaways
• This isn’t a new problem
• The problem isn’t going away
• Remember to focus on the
VALUE
https://www.flickr.com/photos/djwtwo/8331524425/
46. Key Takeaways
• Start small
• Start with the ?
• Iteratively follow the value
• Using freely available tooling
• Volume vs Velocity
https://www.flickr.com/photos/djwtwo/8331524425/
50. –attributed to Gene Amdahl 1967
“Amdahl’s law is used to find the maximum
expected improvement to an overall system
when only part of the system is improved.”
Dave
http://www.forbes.com/sites/gilpress/2013/05/09/a-very-short-history-of-big-data/Reference
Ashok
Big data analytics are driving rapid growth for public cloud computing vendors with revenues for the top 50 public cloud providers shooting up 47% in the fourth quarter last year to $6.2 billion
Dave
http://nsa.gov1.info/utah-data-center/
Ashok
Who is that handsome man!
Dave & Ashok
Growth in retail, usage of iBeacons, Precision marketing, some sophistication with web analytics & CRM - greater penetration.
Healthcare - remote monitoring, automated procedures
Ashok
Ashok
Validation or Discovery
picture of fork in the road?
Ashok & Dave
Dave
Ashok
Exploring alternate models
Dave
Ashok
Lambda Architecture - section heading
Ashok - high level description of components
Dave
Batch Hadoop 2.0/MR2
goal: allows you to share a large cluster of machines between different frameworks. Similar to Mesos, both are steps towards distributed data OS.
Dave
Data Lakes
Dave
Ashok
Ashok
Ashok
Fast and Scalable Analytics depends on efficient data structures
Matching the Algorithm to the data structure
Morphing the Raw data into the data structure
Raw data > Data Structure > Algorithm > Insight
Conclusion
Ashok
Balance shifting from Commercial to Open-Source
Innovations coming from the open source world