This document provides information about big data and cloud computing. It discusses the volume, velocity, and variety of big data, noting that in 2017 there were 2.5 exabytes of data per day and it is expected to grow to 74 zettabytes by 2021. The sources of big data are also summarized, including the internet of things, sensors, and cameras. An example is given of data from the Large Hadron Collider at CERN. Tools and approaches for big data are also listed. The remainder of the document focuses on cloud computing, including cloud services, career paths to becoming a cloud solution architect, skills needed, certification options, and typical salaries.
4. Big data
• Big Data
2017: 2.5 exabytes (quintillion bytes) of data per day,
largely unstructured
2020: 64.2 zettabytes
2021: expected 74 zettabytes
• 90% of the data created in last two years
1 million
TB
1000 billion
TB
6. Source of data
• Big data is transforming science and engineering
Internet of things
Sensor
Camera
• In chemistry
• In medicine
• In smart building/smart city
7. Get a sense of big data
Example: CERN Large Hadron Collider Data
CERN has publicly released over 300TB of data: CERN Open
Data Portal.
This is equivalent to:
• 63,830 DVD ( 1 DVD is about 4.7 GB)
• 1,230 year of music streaming
• 98 years, or 857,142 hours of 4K Video Streaming (at
350MB/Hour)
Still minor in comparison to what National Security Agency
works with.
By 2013, the NSA’s "touch“ 300 TB of data every 15 minutes or
8. Tools and approaches –
ACM Data Science Task Force, January 2021
Big data
computing
architectures
Parallel
computing
frameworks
Distributed
data storage
Parallel
programming
Techniques for
Big Data
applications
Cloud
computing
Complexity
theory
Software
support for Big
Data
applications
16. What is a
Solution
Architect?
Solutions architect is responsible
for:
• Evaluating an organization’s
business needs
• Determining how IT can
support those needs leveraging
software, hardware, or
infrastructure
• Aligning IT strategy with
business goals has become
paramount
• Determine, develop, and
improve technical solutions in
support of business goals.
17. Difference
between
SA(s)
job titles
System Administrator: who is
responsible for administration,
management, and support activities
associated with the IT infrastructure
at a multi-user organization.
Software Architect: mainly in charge
of engineering problems and software
architecture. We have a detailed
article describing the role of a
software architect and the
importance of this position to the
company’s success.
Solution Architect: is a practice of
designing, describing, and managing
the solution engineering in relation to
specific business problems.
18. Prepare to
become a
SA
Solution architect skills:
• Strong leadership and
communication skills
• Technical knowledge
• Business knowledge
• Understand the Software
Development Life Cycle
• Other skills: hypothesis thinking,
problem solving, negotiation,…
19. Prepare to become a SA
AWS/Azure
/GCP
SAP Business
Warehouse
Apache
Kafka
ServiceNow
Big Data
Analytics
Data
Management
Enterprise
Service Bus
(ESB)
Understanding
of IT security,
infrastructure,
and
governance
Experience
with
enterprise
architecture
Software
Development
LifeCycle
Informatica
Experience
working with IT
architecture
frameworks
…
Technical Stack
20. Prepare to
become a
SA
Certification
▪AWS Certified: Solution
Architect (Associate,
Professional)
▪Google Professional Cloud
Architect
▪Microsoft Certified: Azure
Solutions Architect Expert
24. Experience sharing
How I became a Solution Architect?
Java
developer
System
enginee
r
Cloud
Enginee
r
Project
Manage
r
Cloud
Solution
Architect
1 year
1 year
1
year
4 years
Start to learn
cloud
computing
25. Your
chance!
Solutions architect salary
Salary Range
◦ America: $76,000 – $160,000
/year
◦ Vietnam: $40k ~ $80k /year
Difference between SA and other
job position
◦ Longer career age
◦ Become rare (~3 - 5%
employee in software company
is working as Solution Architect)
◦ Only do what is in your strength
◦ Change to go to global
environment
◦ Faster salary up! Easier to nego
with your boss ☺
