Call Us: +32 2 466 00 16
Email: info@u2u.be
Follow Us:

Modern Data Warehouse with Azure SQL

2 days
uadatawh
2 days

Upcoming Sessions

Date: currently not scheduled

Format: Classroom

Price: 0€

Subscribe to waiting list

Date: currently not scheduled

Format: Classroom

Price: 0€

Subscribe to waiting list

Interested in a private company training? Request it here.

Part 1 - Modern Data Warehouse

The Modern Data Warehouse

The cloud requires to reconsider some of the choices made for on-premisses data handling. This module introduces the different services in Azure that can be used for data processing, and compares them to the traditional on-premisses data stack.

  • From traditional to modern data warehouse
  • Lambda architecture
  • Overview of Big Data related Azure services
  • Getting started with Azure

Storing your data in Azure Storage

Azure Storage is like a sort of file share that can be used by many of the Azure services. Often the output of one Azure service is stored in Azure Storage before being consumed by another component. In this module you will learn about the different types of storage available in Azure Storage. Also will you become familiar with some of the tools to load and manage files in Azure storage.

  • Microsoft Azure Storage Concepts: Storage accounts and Containers
  • Azure blob storage
  • Tools for storing data in Azure Storage

Azure Data Warehouse

Azure SQL Databases have their limitations in compute power since they run on a single machine, and their size is limited to the Terabyte range. Azure Data Warehouse is a service aiming at an analytical workload on data volumes hundreds of times larger than what Azure SQL databases can handle. Yet at the same time we can keep on using the familiar T-SQL query language, or we can connect traditional applications such as Excel and Management Studio to interact with this service. Both storage and compute can be scaled independently.

  • Architecture of Azure Data Warehouse
  • Loading data via PolyBase
  • CTAS and CETAS
  • Setting up table distributions
  • Indexing
  • Partitioning
  • Performance monitoring and tuning

Using Azure Data Factory for ETL

When the data is stored and analysed on on-premisses we typically use ETL tools such as SQL Server Integration Services for this. But what if the data is stored in the cloud? Then we need Azure Data Factory, the cloud-based ETL service. First we need to get used to the terminology, then we can start creating the proper objects in the portal.

  • Data Factory V2 terminology
  • The Data Factory wizard
  • Developing Data Factory pipelines in the browser
  • Creating Data Factory Data flows
  • Setup of Integration Runtimes
  • Debugging, scheduling and monitoring DF pipelines

Advanced data processing with Databricks

Azure Databricks allows us to use the power of Spark without the configuration hassle of Hadoop clusters. Using popular languages such as Python, SQL and R data can be loaded, visualized, transformed and analyzed via interactive notebooks.

  • Introduction Azure Databricks
  • Cluster setup
  • Databricks Notebooks
  • Connecting to Azure Storage and Data Warehouse
  • Processing Spark Dataframes in Python
  • Using Spark SQL
  • Scheduling Databricks jobs

Part 2 - Workshop: Migrate EDW to Azure SQL Data Warehouse

In this workshop, you will look at the process of migrating an on-premises data warehouse to Azure SQL Data Warehouse. Throughout the whiteboard design session and hands-on lab, you will look at the planning process for data warehouse migration, identifying schema and data incompatibilities, efficiently migrating data from on-premises databases to the cloud, data distribution in Azure SQL Data Warehouse, migrating ETL jobs to Azure Data Factory, and supporting ad-hoc workloads in an Azure SQL Data Warehouse through Azure Analysis Services.

At the end of this workshop, you will be better able to plan and implement a migration of your existing on-premises enterprise data warehouse to Azure SQL Data Warehouse and integrating it with both cloud-based and on-premises services and data sources.

 

In this workshop, you will look at the process of migrating an on-premises data warehouse to Azure SQL Data Warehouse. Throughout the whiteboard design session and hands-on lab, you will look at the planning process for data warehouse migration, identifying schema and data incompatibilities, efficiently migrating data from on-premises databases to the cloud, data distribution in Azure SQL Data Warehouse, migrating ETL jobs to Azure Data Factory, and supporting ad-hoc workloads in an Azure SQL Data Warehouse through Azure Analysis Services.

At the end of this workshop, you will be better able to plan and implement a migration of your existing on-premises enterprise data warehouse to Azure SQL Data Warehouse and integrating it with both cloud-based and on-premises services and data sources.

Target Audience:

  • Database Administrators
  • Database Developers
  • Data Architects
© 2020 U2U All rights reserved.