The session covers how to architect big data solutions in Azure. Azure provides different Hadoop clusters types. The session covers the basic understanding of Lambda Architecture and how to build end to end big data solution using Apache Hadoop HDFS, HBase, Hive, Storm and Spark. The session covers how to integrate by building apps using any programming language such as Java, Python, or Scala with HDInsight clusters using various Hadoop integration frameworks and libraries. The session is a jump start for Architects, engineers and DBAs with RDBMS experience who are looking for starting building big data solutions based on Hadoop. The session is a demo driven and will cover the basics of Hadoop open source products.

2. Session Objectives And Takeaways

3. Lambda Architecture
http://lambda-architecture.net/
1.All data entering the system is dispatched to both the
batch layer and the speed layer for processing.
2.The batch layer has two functions: (i) managing the
master dataset (an immutable, append-only set of raw
data), and (ii) to pre-compute the batch views.
3.The serving layer indexes the batch views so that they
can be queried in low-latency, ad-hoc way.
4.The speed layer compensates for the high latency of
updates to the serving layer and deals with recent data
only.
5.Any incoming query can be answered by merging
results from batch views and real-time views.

4. Lambda Architecture

5. Linux
Windows
What is HDInsight

6. C# Java
.NET
HDInsight clusters on Azure

7. HDInsight clusters on Azure

8. What is HBase

9. Order No Customer Name Customer Phone Company Name Company Address
12012015 Mostafa 101-232-2345 Microsoft Redmond, WA
Customer Company
Order No Customer
Name
Customer
Phone
Company Name Company
Address
12012015 Mostafa 101-232-2345 Microsoft Redmond, WA

10. Create
Select
Update
Select
What is HBase

11. data warehouse system
What is Hive

13. distributed fault-tolerant open-source
analytics
solutions
templates
What is Apache Storm

14. Topologies
topology
Stream
Tuple
Spout
Bolt streams tuples streams
Apache Storm Components

16. 100x 10x
What is Apache Spark

18. complexities of ingesting and storing all of your data
batch streaming interactive analytics
Azure Data Lake (ADL)

19. Azure Data
Lake (ADL)

20. Azure Data Lake Analytics

23. http://mostafa.rocks