Level: Introductory to Intermediate
Big Data Processing not only requires easy ways to scale and optimize your queries, but also requires scaling custom code written by domain experts who aren't necessarily experts in distributed computing. This session will introduce U-SQL, the new big data processing language introduced with the Azure Data Lake Analytics, and show you how it deeply integrates the ability to extend SQL with custom code for large scale processing. You'll discuss why and how we chose to design U-SQL and cover some of its lineage from its roots in Scope, the Microsoft-internal Big Data Processing Language.
You'll cover some of the use cases and show how U-SQL makes it easy to work with a variety of data, add custom code and perform federated queries against remote data sources. You'll also learn about the execution and debugging model for the language using Visual Studio.
You will learn:
- About the syntax and common patterns in U-SQL for Big Data processing
- How to use custom code seamlessly to transform and process complex unstructured and semi-structured data
- How to use VS to debug complex big data jobs with many parallel components