AI, Data, and Machine Learning, Microsoft Sessions

T14 Building Data Pipelines for Modern Data Warehouse with Spark and .NET in Azure


3:00pm - 4:15pm

Level: Intermediate to Advanced

Michael Rys

Principal Program Manager


Democratizing data empowers customers to gain value from data through self-service analytics. Building the data pipelines that process raw data to gain deeper insights is one of the critical tasks. However, the existing open source platforms such as Spark have not provided good .NET support until fairly recently. In this session we will show you how to build data pipelines with Spark and your favorite .NET programming language (C#, F#) using the Azure Spark offerings such as Azure Synapse, Azure HDInsight and Azure Databricks.

You will learn:

  • What .NET Support is available for Spark
  • How to build some simple .NET Spark applications
  • Know how to use the .NET Spark support in different Spark Engines