SlideShare une entreprise Scribd logo
1  sur  17
Télécharger pour lire hors ligne
Loading Data into Azure SQL DW
(Synapse Analytics)
Global Azure Greece 2020
Apr 25, 2020
Presenter: Antonios Chatzipavlis
Please mute
your mic
Antonios Chatzipavlis
Data Solutions Consultant & Trainer
Since 1999
30+Years in a Field
20+
Experience with
60+
Certifications
SQLschool.gr
Founder
A community for Greek professionals who use the
Microsoft Data Platform
Connect / Explore / Learn
@antoniosch - @sqlschool
./sqlschoolgr - ./groups/sqlschool
./c/SqlschoolGr
SQLschool.gr Group
help@sqlschool.gr
Join us
Articles
SQL Saturday Nights
SQL Server in Greek
Webcasts
News
Resources
Connect / Explore / Learn
V.102010-2020
Azure Synapse Analytics
Limitless analytics service with unmatched time to insight
Platform
Azure
Data Lake Storage
Common Data Model
Enterprise Security
Optimized for Analytics
METASTORE
SECURITY
MANAGEMENT
MONITORING
DATA INTEGRATION
Analytics Runtimes
PROVISIONED ON-DEMAND
Form Factors
SQL
Languages
Python .NET Java Scala R
Experience Synapse Analytics Studio
Artificial Intelligence / Machine Learning / Internet of Things
Intelligent Apps / Business Intelligence
METASTORE
SECURITY
MANAGEMENT
MONITORING
Synapse SQL pool
• Extract the source data into text files.
• Land the data into Azure Data Lake Store Gen2.
• Prepare the data for loading.
• Load the data into staging tables with PolyBase or the COPY command.
• Transform the data.
• Insert the data into production tables.
Data loading strategy for Synapse SQL pool
The fastest and most scalable way to load data
ETL vs. ELT
ETL ELT
SSIS vs. ADF
SSIS ADF
Demonstration
Loading data in Synapse SQL pool
Any
questions
Thank you!
@antoniosch - @sqlschool
./sqlschoolgr - ./groups/sqlschool
./c/SqlschoolGr
SQLschool.gr Group
Antonios Chatzipavlis
Data Solutions Consultant & Trainer
A community for Greek professionals who use the Microsoft Data Platform
Copyright © SQLschool.gr. All right reserved. PRESENTER MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION

Contenu connexe

Tendances

Tendances (20)

Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Azure Databricks - An Introduction (by Kris Bock)
Azure Databricks - An Introduction (by Kris Bock)Azure Databricks - An Introduction (by Kris Bock)
Azure Databricks - An Introduction (by Kris Bock)
 
Big Data Day LA 2016/ NoSQL track - Architecting Real Life IoT Architecture, ...
Big Data Day LA 2016/ NoSQL track - Architecting Real Life IoT Architecture, ...Big Data Day LA 2016/ NoSQL track - Architecting Real Life IoT Architecture, ...
Big Data Day LA 2016/ NoSQL track - Architecting Real Life IoT Architecture, ...
 
About Pragmatic Works
About Pragmatic WorksAbout Pragmatic Works
About Pragmatic Works
 
Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
Azure Data Lake Store and Analytics
Azure Data Lake Store and AnalyticsAzure Data Lake Store and Analytics
Azure Data Lake Store and Analytics
 
Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...
Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...
Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...
 
Perth Microsoft Data & Analytics User Group - Building Solutions with Azure D...
Perth Microsoft Data & Analytics User Group - Building Solutions with Azure D...Perth Microsoft Data & Analytics User Group - Building Solutions with Azure D...
Perth Microsoft Data & Analytics User Group - Building Solutions with Azure D...
 
Overview on Azure Machine Learning
Overview on Azure Machine LearningOverview on Azure Machine Learning
Overview on Azure Machine Learning
 
Scaling Privacy in a Spark Ecosystem
Scaling Privacy in a Spark EcosystemScaling Privacy in a Spark Ecosystem
Scaling Privacy in a Spark Ecosystem
 
Azure Days 2019: Grösser und Komplexer ist nicht immer besser (Meinrad Weiss)
Azure Days 2019: Grösser und Komplexer ist nicht immer besser (Meinrad Weiss)Azure Days 2019: Grösser und Komplexer ist nicht immer besser (Meinrad Weiss)
Azure Days 2019: Grösser und Komplexer ist nicht immer besser (Meinrad Weiss)
 
Unleash the Power of Azure Data Factory - SQL User Group
Unleash the Power of Azure Data Factory - SQL User GroupUnleash the Power of Azure Data Factory - SQL User Group
Unleash the Power of Azure Data Factory - SQL User Group
 
Azure cafe marketplace with looker data analytics
Azure cafe marketplace with looker data analyticsAzure cafe marketplace with looker data analytics
Azure cafe marketplace with looker data analytics
 
DataOps for the Modern Data Warehouse on Microsoft Azure @ NDCOslo 2020 - Lac...
DataOps for the Modern Data Warehouse on Microsoft Azure @ NDCOslo 2020 - Lac...DataOps for the Modern Data Warehouse on Microsoft Azure @ NDCOslo 2020 - Lac...
DataOps for the Modern Data Warehouse on Microsoft Azure @ NDCOslo 2020 - Lac...
 
Azure Con Cortana Analytics Suite
Azure Con Cortana Analytics Suite Azure Con Cortana Analytics Suite
Azure Con Cortana Analytics Suite
 
Azure Days 2019: Keynote Azure Switzerland – Status Quo und Ausblick (Primo A...
Azure Days 2019: Keynote Azure Switzerland – Status Quo und Ausblick (Primo A...Azure Days 2019: Keynote Azure Switzerland – Status Quo und Ausblick (Primo A...
Azure Days 2019: Keynote Azure Switzerland – Status Quo und Ausblick (Primo A...
 
Azure Data Factory v2
Azure Data Factory v2Azure Data Factory v2
Azure Data Factory v2
 
Unleash the power of Azure Data Factory
Unleash the power of Azure Data Factory Unleash the power of Azure Data Factory
Unleash the power of Azure Data Factory
 
Machine Learning and AI
Machine Learning and AIMachine Learning and AI
Machine Learning and AI
 
Leveraging Azure Analysis Services Tabular Data Models with Power BI by Tim M...
Leveraging Azure Analysis Services Tabular Data Models with Power BI by Tim M...Leveraging Azure Analysis Services Tabular Data Models with Power BI by Tim M...
Leveraging Azure Analysis Services Tabular Data Models with Power BI by Tim M...
 

Similaire à Loading Data into Azure SQL DW (Synapse Analytics)

Session6-SharePoint and Azure- steve fox-windows-and_azure_spfs
Session6-SharePoint and Azure- steve fox-windows-and_azure_spfsSession6-SharePoint and Azure- steve fox-windows-and_azure_spfs
Session6-SharePoint and Azure- steve fox-windows-and_azure_spfs
Mithun T. Dhar
 
SBA Security Meetup - Deploying and managing azure sentinel as code by Bojan ...
SBA Security Meetup - Deploying and managing azure sentinel as code by Bojan ...SBA Security Meetup - Deploying and managing azure sentinel as code by Bojan ...
SBA Security Meetup - Deploying and managing azure sentinel as code by Bojan ...
SBA Research
 

Similaire à Loading Data into Azure SQL DW (Synapse Analytics) (20)

Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...
Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...
Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...
 
Cortana Intelligence Solutions
Cortana Intelligence SolutionsCortana Intelligence Solutions
Cortana Intelligence Solutions
 
Analytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual WorkshopAnalytics in a Day Ft. Synapse Virtual Workshop
Analytics in a Day Ft. Synapse Virtual Workshop
 
Azure Synapse 101 Webinar Presentation
Azure Synapse 101 Webinar PresentationAzure Synapse 101 Webinar Presentation
Azure Synapse 101 Webinar Presentation
 
Session6-SharePoint and Azure- steve fox-windows-and_azure_spfs
Session6-SharePoint and Azure- steve fox-windows-and_azure_spfsSession6-SharePoint and Azure- steve fox-windows-and_azure_spfs
Session6-SharePoint and Azure- steve fox-windows-and_azure_spfs
 
SQLArgentina - Data Platform Summit BA - Keynote
SQLArgentina - Data Platform Summit BA - KeynoteSQLArgentina - Data Platform Summit BA - Keynote
SQLArgentina - Data Platform Summit BA - Keynote
 
Azure SQL Database Part 1 Setup and Monitoring
Azure SQL Database Part 1 Setup and MonitoringAzure SQL Database Part 1 Setup and Monitoring
Azure SQL Database Part 1 Setup and Monitoring
 
Remediate and secure your organization with azure sentinel
Remediate and secure your organization with azure sentinelRemediate and secure your organization with azure sentinel
Remediate and secure your organization with azure sentinel
 
DevIntersection2015-Hybrid2015Final2.pptx
DevIntersection2015-Hybrid2015Final2.pptxDevIntersection2015-Hybrid2015Final2.pptx
DevIntersection2015-Hybrid2015Final2.pptx
 
SBA Security Meetup - Deploying and managing azure sentinel as code by Bojan ...
SBA Security Meetup - Deploying and managing azure sentinel as code by Bojan ...SBA Security Meetup - Deploying and managing azure sentinel as code by Bojan ...
SBA Security Meetup - Deploying and managing azure sentinel as code by Bojan ...
 
Hadoop et bases de données relationnelles ultra performantes : le meilleur de...
Hadoop et bases de données relationnelles ultra performantes : le meilleur de...Hadoop et bases de données relationnelles ultra performantes : le meilleur de...
Hadoop et bases de données relationnelles ultra performantes : le meilleur de...
 
Hadoop et bases de données relationnelles ultra performantes : le meilleur de...
Hadoop et bases de données relationnelles ultra performantes : le meilleur de...Hadoop et bases de données relationnelles ultra performantes : le meilleur de...
Hadoop et bases de données relationnelles ultra performantes : le meilleur de...
 
Microsoft Azure Overview
Microsoft Azure OverviewMicrosoft Azure Overview
Microsoft Azure Overview
 
Microsoft Azure News - Sept 2016
Microsoft Azure News - Sept 2016Microsoft Azure News - Sept 2016
Microsoft Azure News - Sept 2016
 
Cloud Scale Analytics Pitch Deck
Cloud Scale Analytics Pitch DeckCloud Scale Analytics Pitch Deck
Cloud Scale Analytics Pitch Deck
 
.Net development with Azure Machine Learning (AzureML) Nov 2014
.Net development with Azure Machine Learning (AzureML) Nov 2014.Net development with Azure Machine Learning (AzureML) Nov 2014
.Net development with Azure Machine Learning (AzureML) Nov 2014
 
Azure for gaming
Azure for gamingAzure for gaming
Azure for gaming
 
Advanced Analytics and Artificial Intelligence - Transforming Your Business T...
Advanced Analytics and Artificial Intelligence - Transforming Your Business T...Advanced Analytics and Artificial Intelligence - Transforming Your Business T...
Advanced Analytics and Artificial Intelligence - Transforming Your Business T...
 
Microsoft Azure and Microsoft 365 - How Will They Help You
Microsoft Azure and Microsoft 365 - How Will They Help YouMicrosoft Azure and Microsoft 365 - How Will They Help You
Microsoft Azure and Microsoft 365 - How Will They Help You
 
Microsoft Azure News - Nov 2020
Microsoft Azure News - Nov 2020Microsoft Azure News - Nov 2020
Microsoft Azure News - Nov 2020
 

Plus de Antonios Chatzipavlis

Plus de Antonios Chatzipavlis (20)

Data virtualization using polybase
Data virtualization using polybaseData virtualization using polybase
Data virtualization using polybase
 
SQL server Backup Restore Revealed
SQL server Backup Restore RevealedSQL server Backup Restore Revealed
SQL server Backup Restore Revealed
 
Migrate SQL Workloads to Azure
Migrate SQL Workloads to AzureMigrate SQL Workloads to Azure
Migrate SQL Workloads to Azure
 
Machine Learning in SQL Server 2019
Machine Learning in SQL Server 2019Machine Learning in SQL Server 2019
Machine Learning in SQL Server 2019
 
Workload Management in SQL Server 2019
Workload Management in SQL Server 2019Workload Management in SQL Server 2019
Workload Management in SQL Server 2019
 
Introduction to DAX Language
Introduction to DAX LanguageIntroduction to DAX Language
Introduction to DAX Language
 
Building diagnostic queries using DMVs and DMFs
Building diagnostic queries using DMVs and DMFs Building diagnostic queries using DMVs and DMFs
Building diagnostic queries using DMVs and DMFs
 
Exploring T-SQL Anti-Patterns
Exploring T-SQL Anti-Patterns Exploring T-SQL Anti-Patterns
Exploring T-SQL Anti-Patterns
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
 
Modernizing your database with SQL Server 2019
Modernizing your database with SQL Server 2019Modernizing your database with SQL Server 2019
Modernizing your database with SQL Server 2019
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
 
SQLServer Database Structures
SQLServer Database Structures SQLServer Database Structures
SQLServer Database Structures
 
Sqlschool 2017 recap - 2018 plans
Sqlschool 2017 recap - 2018 plansSqlschool 2017 recap - 2018 plans
Sqlschool 2017 recap - 2018 plans
 
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018 Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
 
Microsoft SQL Family and GDPR
Microsoft SQL Family and GDPRMicrosoft SQL Family and GDPR
Microsoft SQL Family and GDPR
 
Statistics and Indexes Internals
Statistics and Indexes InternalsStatistics and Indexes Internals
Statistics and Indexes Internals
 
Introduction to Azure Data Lake
Introduction to Azure Data LakeIntroduction to Azure Data Lake
Introduction to Azure Data Lake
 
Azure SQL Data Warehouse
Azure SQL Data Warehouse Azure SQL Data Warehouse
Azure SQL Data Warehouse
 
Introduction to azure document db
Introduction to azure document dbIntroduction to azure document db
Introduction to azure document db
 
Introduction to Machine Learning on Azure
Introduction to Machine Learning on AzureIntroduction to Machine Learning on Azure
Introduction to Machine Learning on Azure
 

Dernier

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Dernier (20)

The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 

Loading Data into Azure SQL DW (Synapse Analytics)

  • 1.
  • 2. Loading Data into Azure SQL DW (Synapse Analytics) Global Azure Greece 2020 Apr 25, 2020 Presenter: Antonios Chatzipavlis
  • 3.
  • 5. Antonios Chatzipavlis Data Solutions Consultant & Trainer Since 1999 30+Years in a Field 20+ Experience with 60+ Certifications SQLschool.gr Founder
  • 6. A community for Greek professionals who use the Microsoft Data Platform Connect / Explore / Learn @antoniosch - @sqlschool ./sqlschoolgr - ./groups/sqlschool ./c/SqlschoolGr SQLschool.gr Group help@sqlschool.gr Join us Articles SQL Saturday Nights SQL Server in Greek Webcasts News Resources
  • 7. Connect / Explore / Learn V.102010-2020
  • 8.
  • 9. Azure Synapse Analytics Limitless analytics service with unmatched time to insight Platform Azure Data Lake Storage Common Data Model Enterprise Security Optimized for Analytics METASTORE SECURITY MANAGEMENT MONITORING DATA INTEGRATION Analytics Runtimes PROVISIONED ON-DEMAND Form Factors SQL Languages Python .NET Java Scala R Experience Synapse Analytics Studio Artificial Intelligence / Machine Learning / Internet of Things Intelligent Apps / Business Intelligence METASTORE SECURITY MANAGEMENT MONITORING
  • 11. • Extract the source data into text files. • Land the data into Azure Data Lake Store Gen2. • Prepare the data for loading. • Load the data into staging tables with PolyBase or the COPY command. • Transform the data. • Insert the data into production tables. Data loading strategy for Synapse SQL pool The fastest and most scalable way to load data
  • 14. Demonstration Loading data in Synapse SQL pool
  • 16. Thank you! @antoniosch - @sqlschool ./sqlschoolgr - ./groups/sqlschool ./c/SqlschoolGr SQLschool.gr Group Antonios Chatzipavlis Data Solutions Consultant & Trainer
  • 17. A community for Greek professionals who use the Microsoft Data Platform Copyright © SQLschool.gr. All right reserved. PRESENTER MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION