AI, Data, and Machine Learning, Microsoft Sessions

W06 Building Data Pipelines for Modern Data Warehouse with Spark and .NET in Azure


9:30am - 10:45am

Level: Intermediate

Brigit Murtaugh

.NET team Program Manager


Democratizing data empowers customers to gain value from data through self-service analytics. Building the data pipelines that process raw data to gain deeper insights is one of the critical tasks. However, the existing open source platforms such as Spark have not provided good .NET support until fairly recently. In this session we will show you how to build data pipelines with Spark and your favorite .NET programming language (C#, F#) using the Azure Spark offerings such as Azure Synapse, Azure HDInsight and Azure Databricks.

You will learn:

  • What .NET Support is available for Spark
  • How to build some simple .NET Spark applications
  • Know how to use the .NET Spark support in different Spark Engines