AI, Data, and Machine Learning, Microsoft Sessions

T06 Building Data Pipelines for Modern Data Warehouse with Spark and .NET in Azure

03/31/2020

9:30am - 10:45am

Level: Intermediate

Michael Rys

Principal Program Manager

Microsoft

Democratizing data empowers customers by enabling more and more users to gain value from data through self-service analytics. Building the data pipelines that process raw data to gain deeper insights is one of the critical tasks. However, the existing open source platforms such as Spark have not provided good .NET support until fairly recently. In this session we will show you how to build data pipelines with Spark and your favorite .NET programming language (C#, F#) using the Azure Spark offerings such as Azure HDInsight and Azure Databricks.

You will learn:

  • What .NET Support is available for Spark
  • How to build some simple .NET Spark applications
  • Know how to use the .NET Spark support in different Spark Engines