Azure Databricks: What Is It and What Can You Do with It?

"I will talk about two sets of things. One is how productivity and collaboration are reinventing the nature of work, and how this will be very important for the global economy. And two, data. In other words, the profound impact of digital technology that stems from data and the data feedback loop." ~ Microsoft CEO Satya Nadella

Collaboration, productivity and data are what Azure Databricks is all about.

Data, of course, is everything. How many F150 trucks did Ford build this year? What is the patient’s heart rate and blood pressure? Where can we find sushi at this time of night?

The iPhone in your back pocket is filled with data and searching for more.

How does that data get organized so you can find it when you need it?

Machines do a lot of that work. But people working in collaboration with other people and other machines make the data driven world go around.

Azure Databricks is a collaboration between Microsoft and the creators of Apache Spark, which is described on its homepage as an "analytics engine for big data processing, with built-in modules for streaming, SQL, machine learning and graph processing."

Databricks, the company created to commercialize Spark, "provides a Unified Analytics Platform for data science teams to collaborate with data engineering and lines of business to build data products," the company states. "Users achieve faster time-to-value with Databricks by creating analytic workflows that go from ETL and interactive exploration to production. The company also makes it easier for its users to focus on their data by providing a fully managed, scalable, and secure cloud infrastructure that reduces operational complexity and total cost of ownership."

One Click Setup and Management
In announcing the collaboration with Databricks in 2017, Microsoft touted Azure Databricks as "a fast, easy and collaborative Apache Spark-based analytics platform that delivers one-click setup, streamlined workflows and an interactive workspace. Native integration with Azure SQL Data Warehouse, Azure Storage, Azure Cosmos DB, Azure Active Directory and Power BI simplifies the creation of modern data warehouses that enable organizations to provide self-service analytics and machine learning over all data with enterprise-grade performance and governance."

In a Microsoft overview of Azure Databricks, the company explains its value-add: "Azure Databricks features optimized connectors to Azure storage platforms (e.g. Data Lake and Blob Storage) for the fastest possible data access, and one-click management directly from the Azure console. This is the first time that an Apache Spark platform provider has partnered closely with a cloud provider to optimize data analytics workloads from the ground up."

What Does All This Mean?
So here’s this extensive set of data tools, what can you build with them? In January, Databricks provided answers from data industry thought leaders, who focused on the need for solutions to issues organizations face with AI, Big Data and Analytics.

Kamelia Aryafar, chief algorithm officer at Overstock, sees deep learning, which is a class of machine learning algorithms facilitated by the Spark technology, paying dividends for organizations. "Deep learning innovations will create a lot of new AI applications, some of which are already in production and making massive changes in the industry," she is quoted as saying. She noted that Overstock is currently using deep learning to improve marketing projects such as email campaigns.

Other thought leaders quoted by Databricks see the need for the latest data tools to be used to improve long-standing issues including data processing and providing trusted data with "Explainable AI."

Because of the social, economic and commercial implications of the data being generated, "it is critical to develop AI that is explainable, provable and transparent," said Mainak Mazumdar, chief research officer at Nielsen.

Databricks CEO and co-founder Ali Ghodsi finds data processing to still be a challenge in the AI era. "As an industry we tend to believe that data scientists are spending the majority of their time developing models, shares. Truth be told, data processing remains the hardest and most time consuming part of any AI initiative. The highly iterative nature of AI forces data teams to switch between data processing tools and machine learning tools. For organizations to succeed at AI in 2019, they have to leverage a platform that unifies these disparate tools."

Reading between the lines, Databricks provides the platform companies need to leverage.

Databricks concluded the survey on the near future of AI and Big Data, stating: “Solving the world’s toughest data problems starts with bringing all of the data teams together within an organization. Data science and engineering teams’ ability to innovate faster has historically been hindered by poor data quality, complex machine learning tool environments, and limited talent pools. Additionally, organizational separation creates friction and slows projects down, becoming an impediment to the highly iterative nature of AI projects. Much like in 2018, organizations that leverage Unified Analytics will have a competitive advantage with the ability to build data pipelines across various siloed data storage systems and to prepare labelled datasets for model building, which allows organizations to do AI on their existing data and iteratively do AI on massive data sets.”

Practical Use Cases
Healthcare is an area where AI can be used to parse patient data to provide diagnostic and other assistance to medical professionals.

Last June, Databricks announced that it has been working with pharmaceutical and healthcare providers "to improve their drug discovery processes."

"One such customer, the Regeneron Genetics Center (a wholly-owned subsidiary of Regeneron, a leading biotechnology company), has sequenced over 300,000 consented volunteers and paired their de-identified genetic data with de-identified electronic health records to uncover actionable insights for drug discovery and development,” Databricks said in the announcement.

Jeffrey Reid, PhD, Head of Genome Informatics at Regeneron, was quoted as saying: "As this dataset has grown rapidly, we encountered significant barriers in simple tasks, like gathering all of the data for a given analysis, and querying the 10s of billions of results from our studies. Not only has the Databricks Unified Analytics Platform solved these big data problems, but it is enabling everyone in our integrated drug development process – from physician-scientists to computational biologists – to easily access, analyze, and extract insights from all of our data. Drug development is still a long and difficult process rife with failure, but we have already significantly reduced the amount of time it takes to generate important early insights."

Databricks cited the following areas where its platform enables medical researchers to:

  • Accelerate discovery with simplified genomic pipelines: Simplify workflows with prebuilt genomic pipelines hosted in the cloud to process large datasets up to 100x faster than existing solutions.
  • Innovate faster with interactive, tertiary analytics and AI at scale: Quickly and simply run tertiary analytics and machine learning algorithms on massive genomic datasets with prepackaged frameworks designed to run in parallel.
  • Improve productivity across data, analytics and research teams: Create a collaborative environment and shared workspaces for bioinformaticians, computational biologists and researchers to work together across the research lifecycle with shared workspaces, saving teams precious time and resources.

Training for AI, Data, and Machine Learning
Working with AI, Big Data and Machine Learning is the future of application development. If you want to build Azure Databricks skills, Visual Studio Live! New Orleans this April offers a session on AI and Analytics with Apache Spark on Azure Databricks where you will learn:

  • About the fundamentals of Apache Spark, Spark SQL and Spark MLlib
  • How to use Databricks notebooks
  • How to manage clusters and jobs
  • How to integrate Azure Databricks with blob storage and Azure Data Lake Store (ADLS)
  • How to write Python code for both analytics and machine learning

Find out more here.

Posted by Richard Seeley on 02/20/20190 comments


Xamarin Is a Hot Topic

Xamarin is one of the biggest tech success stories of the decade.

In 2011 few if any developers had even heard of Xamarin. That was the year its namesake San Francisco-based company was founded. Six years later, “1.4 million developers were using Xamarin's products in 120 countries around the world,” according to Wikipedia. Is it any wonder that Microsoft acquired it in 2016 and made it a free tool for Visual Studio developers?

So now Xamarin is in the big time, touting itself in Wikipedia as “the only IDE that allows for native Android, iOS and Windows app development within Microsoft Visual Studio.” Plus, it’s become its own standalone IDE, Xamarin Studio for Windows and Mac mobile app development. There also:

  • Xamarin.Forms
  • Xamarin Test Cloud
  • Xamarin.Mac
  • Xamarin .NET Mobility Scanner
  • Xamarin RoboVM for Java

Then Came Flutter
As 2019 dawned, Visual Studio Magazine popped the question: Will Flutter Become a Xamarin Option?

“Flutter for Xamarin” is something of a work in progress, explains David Ramel, editor of Visual Studio Magazine. It is an under-construction project championed by some developers on GitHub. And there is some debate as to whether Flutter and Xamarin is a match made in app dev heaven. Reading the article the Flutter project is somewhere between a potential breakthrough for developers and a whimsical hobby for coders with time on their hands. The latter view comes from Adam Pedly, the leader of the Flutter for Xamarin project, who describes it on GitHub this way:

"This project is never expected to be commercially viable, unless it is picked up or supported by a larger company. As it currently stands, this is just a fun side project, done by a bunch of developers in their spare time. We offer no support for solutions ever built with this framework, or any guarantee of completion."

There are developers who question why this project even exists. However, if we are talking about support from “a larger company,” it’s important to remember that Flutter is Google's mobile UI framework and you don’t get much larger than Google.

In the article, Ramel found some serious interest in the potential value of Xamarin + Flutter. One developer offered some reasons why this match might work:

  • Because you still want to work in C# but want the ease of use and performance of Flutter?
  • Because you have existing .NET logic code shared with other projects that you want to re-use?
  • Because you still want to take advantage of the vast array of .NET assemblies and nuggets at your disposal?
  • Because the maintainer wanted a hobby project to see what was possible?

There is no telling where this will go but in the comments section for the Visual Studio Magazine article, it did spark debate with comments ranging from: “I honestly don't see the point of this. Xamarin is already an alternative to Flutter. Why would one combine React and Angular?” to “Yes I know that [it’s an] unofficial project, it is very interesting & promising …”

So, stay tuned.

Enter Xamarin.Forms 4.0
Another recent Xamarin news flash is about the big changes Microsoft is unveiling in Xamarin.Forms 4.0.

Having just released Xamarin.Forms 3.4.0, Microsoft took the unusual step of simultaneously releasing a preview of Xamarin.Forms 4.0. You can read all about it in this Visual Studio Magazine article.

Microsoft’s here’s-a-new-release-and-here’s-a-preview-of-the-next-release strategy seems to be about keeping up with developer demands.

In a blog announcing the early preview, Microsoft's David Ortinau wrote: "Through countless interviews, conversations, and surveys, we have heard your voice loud and clear. You want Xamarin.Forms to be easier to use 'out of the box', navigation to be ever present and easier to control, to have a more consistent design across iOS and Android, and to have a faster, more flexible list control."

You can read the entire blog here.

Xamarin Does Las Vegas
If you are interested in getting up close and personal with Xamarin, the place to be is Bally's Hotel & Casino in Las Vegas March 3 ‐ 8 for Visual Studio Live!

There is a “Full Day Hands‐On Lab: Xamarin and Azure: Build the Mobile Apps of Tomorrow” featuring Brandon Minnick, who works “helping fellow Xamarin developers build 5-star apps,” along with Laurent Bugnion, Microsoft Most Valuable Professional (Client Dev) and a Microsoft Regional Director.

If you want to learn more about working with Xamarin.Forms, there’s “Xamarin.Forms Takes You Places!” lead by Microsoft MVP Sam Basu.

Beyond Xamarin, there’s 6 days of in-depth training on the Microsoft platform, including hot topics .NET, Visual Studio, DevOps, SQL Server and more.

Find out more about Visual Studio Live! in Las Vegas.

Posted by Richard Seeley on 01/17/20190 comments


Angular

Finding Your Way Around Angular

Whether you’ve dabbled in AngularJS or even worked extensively with it, you want to upgrade your skills to move to the latest versions of Angular because that’s where the jobs are.

As this blog noted back in April, the JavaScript (hence the JS) version of Angular from 2010 was pretty much eclipsed by the TypeScript rewrite in 2016, which was dubbed Angular2 although it was a whole new platform for building web apps.

In the past two years, the Angular team at Google released new versions so we have Angular 2/4/5, and in May of this year, Angular version 6.0.0 appeared. (In case you’re wondering, Angular 3 got lost in the development shuffle and like Windows 9 it basically doesn’t exist.) These TypeScript versions are sometimes lumped together as “Angular 2+” according to a Wikipedia article.

But you can think of “Angular v2 and above” as “plain ol' Angular,” says Ted Neward, who is director of Developer Relations at Smartsheet.com and well-known as a presenter at Visual Studio Live! His interview, Angular Q&A: Components, Getting-Started Tips (and that 'Total Rewrite' Thing) in Visual Studio Magazine is a great place to start learning about all things Angular.

Beyond that if you want to get up to speed on plain ol’ Angular, Neward recommended the resources available on the official Angular website.

Angular IDEs and Tools
The Angular website provides links to a number of Integrated Development Environments (IDEs) designed to make coding easier with features like drag and drop.

First of the list is Amexio API v5.2.1 with Angular 6 support, and D3 Charts for visualization. The product from MetaMAGIC Global Inc., based in India and New Jersey, provides an Angular UI automation platform including:

  • Angular 6
  • 130+ UI Widgets
  • Drag & Drop Widget
  • Responsive Web Design
  • 57 Material Design Themes
  • D3 Charts / Maps / Dashboard / Layouts

Texas-based Genuitec, a founding member of the Eclipse Foundation, is the developer of a commercial Eclipse tool, Angular IDE by Webclipse, which the company markets as “Simple for beginners; powerful for experts” providing:

  • TypeScript 3.x validation and debugging
  • Advanced coding of HTML templates with validation and auto-complete
  • Integrated usage of the angular-cli for Angular best practices

Among the tools listed on the Angular site are:

  • Angular CLI, a command line interface for Angular
  • Angular Universal, a server-side rendering for Angular applications, which is on GitHub
  • Angular Augury, a “developer tool extension for debugging and profiling Angular applications inside the Google Chrome and Mozilla Firefox browsers”
  • Celerio Angular Quickstart, which GitHub lists as a simple way to generate an Angular 5 create, read, update, and delete (CRUD) application from an existing database schema
  • Codelyzer, with a GitHub link to code for static analysis for Angular projects
  • Compodoc for documenting your Angular project touts its capability to “Generate your Angular project documentation in seconds.”

If you are creating on-line storefronts with Angular, AngularCommerce provides a framework for building e-commerce applications with Google Firebase. The Angular site says AngularCommerce provides a set of components that is design agnostic and allows developers to easily extend functionality.

Quick Start for TypeScript
If you are new to TypeScript, you may want to start at the Typescript open source group’s website. It offers Quick Start with a five-minute tutorial as well as other documentation, downloads and a “Playground” with code samples.

Whether you are a TypeScript veteran or have just learned to work with it, the aforementioned Angular site has lots of helpful stuff starting with an Angular CheatSheet that Neward recommends bookmarking. It has code samples, starting with how to bootstrap the Angular platform and then moves on to:

  • Ngmodules
  • Template syntax
  • Built-in directives
  • Forms
  • Class decorators
  • Directive configuration
  • Component configuration
  • Class field decorators for directives and components
  • Directive and component change detection and lifecycle hooks
  • Dependency injection configuration
  • Routing and navigation

There are Angular how-to books listed on Amazon that you might want to browse. However, you could spend your time rather than your money on the Angular site and probably learn a lot.

Beyond Angular with Microsoft Blazor
Beyond developing in Angular, you might want to check out Microsoft’s Blazor-based .NET technology, which stole the show at a recent VSLive! keynote by Scott Hunter, Partner Director Program Management, .NET, at Microsoft.

"Blazor is something that we're working on as an internal project at Microsoft, on thinking about next-generation .NET applications," Hunter told the VSLive attendees earlier this year. "The primary premise in Blazor is that you can build client-side UI in your browser in .NET. What it means is, you don't have to know Angular or React or Vue, you don't have to know JavaScript because you're actually going to run C# in the browser.”

Find out more about the futuristic Blazor technology in this article based on the keynote by David Ramel, editor of Visual Studio Magazine.

Posted by Richard Seeley on 10/16/20180 comments


When JavaScript Met TypeScript

Recent news involving JavaScript is being overshadowed by its smarter brother TypeScript.

This summer Microsoft shipped the final TypeScript 3.0 release after a brief preview. “The open source programming language is primarily developed and maintained by Microsoft, providing a strict syntactical superset of JavaScript with the addition of optional static typing,” as Visual Studio Magazine explained in announcing TypeScript.

More than brothers, the two scripting languages are almost Siamese twins as a Wikipedia article explains: “TypeScript is a strict superset of ECMAScript 2015, which is itself a superset of ECMAScript 5, commonly referred to as JavaScript. As such, a JavaScript program is also a valid TypeScript program, and a TypeScript program can seamlessly consume JavaScript.”

Now you know.

The Typescript open source group’s website explains the relationship this way: “TypeScript starts from the same syntax and semantics that millions of JavaScript developers know today. Use existing JavaScript code, incorporate popular JavaScript libraries, and call TypeScript code from JavaScript. TypeScript compiles to clean, simple JavaScript code which runs on any browser, in Node.js, or in any JavaScript engine that supports ECMAScript 3 (or newer).”

Okay, glad we got that cleared up.

As with any relationship between brothers, there is sibling rivalry.

“There's a dialogue that goes on between TypeScript and JavaScript because, essentially, JavaScript isn't typed and TypeScript is,” Peter Vogel explained in a Visual Studio Magazine article. He goes on to add: “As Douglas Crockford points out in JavaScript: The Good Parts, the lack of data typing in JavaScript shouldn't be regarded as a mistake but, instead, as one of the language's features. With that view, by adding data typing, TypeScript takes some of the power of JavaScript away.”

For Vogel, the debate isn’t either/or but both/and as he writes: “I'm not complaining about that loss. My background is in data-typed languages and I value the IntelliSense support and compile-time checking that TypeScript gives me. However, I've done enough JavaScript programming to also value the advantages that developing without types gives me.”

However, TypeScript’s super power is the ability to essentially transform itself into its brother. As another Wikipedia article explains: “TypeScript provides static typing through type annotations to enable type checking at compile time. This is optional and can be ignored to use the regular dynamic typing of JavaScript.”

Imitation is the sincerest form of flattery.

Just be prepared for a lot of JavaScript meets TypeScript news coming out of Microsoft and the open source community.

In other JavaScript news...

Babel 7 JavaScript Compiler Adds TypeScript Support – Earlier this month, Visual Studio Magazine reported: “The popular open source Babel compiler that makes modern JavaScript compatible with older environments has shipped in version 7 and, with help from Microsoft, now supports TypeScript. Babel, a compiler toolchain, is used by millions of JavaScript developers to convert ECMAScript 2015 and later code into backwards-compatible JavaScript versions to make it work in older Web browsers or other environments.”

This summer, Microsoft released documentation for ASP.NET Core SignalR JavaScript client library, which “enables developers to call server-side hub code.” For Visual Studio developers, the page explains: “The SignalR JavaScript client library is delivered as an npm package. If you're using Visual Studio, run npm install from the Package Manager Console while in the root folder. For Visual Studio Code, run the command from the Integrated Terminal.”

In June, up-and-coming TypeScript broke into the Top 100 in programming language rankings, according to this Visual Studio Magazine article, which quotes the TIOBE Index popularity report: "This month TypeScript debuts at position 93 in the TIOBE index top 100 … The Microsoft language has been tracked for a couple of years now, but although its popularity in industry seems high, it never made it to the top 100. So finally it has got sufficient traction to be noticed.”

Little brothers are so competitive.

Posted by Richard Seeley on 09/13/20180 comments


Making Use of Dependency Injection in ASP.NET Core

One of the cool features in ASP.NET Core is support for the Dependency Injection software design pattern.

For developers working in Object-oriented programming and web services, dependency injection provides what Microsoft defines in documentation as valuable best practices to help:

  • Design services to use dependency injection to obtain their dependencies.
  • Avoid stateful, static method calls (a practice known as static cling).
  • Avoid direct instantiation of dependent classes within services. Direct instantiation couples the code to a particular implementation.

If you are not familiar with the concept Dependency Injection, Wikipedia provides this basic definition:

“… dependency injection is a technique whereby one object (or static method) supplies the dependencies of another object. A dependency is an object that can be used [as] (a service). An injection is the passing of a dependency to a dependent object (a client) that would use it. The service is made part of the client's state. Passing the service to the client, rather than allowing a client to build or find the service, is the fundamental requirement of the pattern.

In Object-oriented programming, web services and Software Oriented Architecture (SOA), the goal is to use objects as building blocks of an application in a way that allows you to make changes by adding and removing objects rather than writing extensive code. As Wikipedia explains: “The intent behind dependency injection is to decouple objects to the extent that no client code has to be changed simply because an object it depends on needs to be changed to a different one.”

In Microsoft pages outlining the fundamentals of dependency injection, it offers a simple definition: “A dependency is any object that another object requires.”

Diving into the Documentation

For developers who specifically want to start using dependency injection with ASP.NET Core, Microsoft offers step-by-step documentation with code samples and simple demonstration apps.

On Microsoft’s ASP.NET Blog, Jeffrey T. Fritz, Microsoft .NET Program Manager explains the basics of documentationDependency Injection in ASP.NET Core, noting: “With ASP.NET Core, dependency injection is a fundamental tenet of the framework. All classes instantiated by the framework are done so through the container service that is maintained by the framework in a container and configured by default in the Startup/ConfigureServices method.”

Fritz reassures developers that the framework makes adopting dependency injection very straightforward: “ASP.NET Core makes it easy to get started with this design pattern by shipping a container that you can use with your application. Configure your application’s controllers, views, and other classes that are instantiated by the framework with parameters on the constructor method to have those types automatically created and passed in to your class.”

The blog contains code samples to illustrate how the design pattern works and points developers to Microsoft ASP.NET Dependency Injection documentation. The authors provide loads of code samples and offer the following recommendations to developers working with dependency injection:

  • Avoid storing data and configuration directly in the service container. For example, a user's shopping cart shouldn't typically be added to the service container. Configuration should use the options pattern. Similarly, avoid "data holder" objects that only exist to allow access to some other object. It's better to request the actual item via dependency injection, if possible.
  • Avoid static access to services
  • Avoid using the service locator pattern (for example, IServiceProvider.GetService).
  • Avoid static access to HttpContext (for example, IHttpContextAccessor.HttpContext).

That is just a sample of the detailed information available on that site.

Beyond the Basics

Once you have the basics down, Steve Smith, Microsoft MVP since 2002 and a founding member of the ASPInsiders, an external advisory group for the ASP.NET product team, provides insight on the MSDN site for Writing Clean Code in ASP.NET Core with Dependency Injection.

He notes that dependency injection is “an increasingly common technique in .NET development, because of the decoupling it affords to applications that employ it.”

“ASP.NET Core not only supports DI,” Smith writes, “it also includes a DI container—also referred to as an Inversion of Control (IoC) container or a services container. Every ASP.NET Core app configures its dependencies using this container in the Startup class’s ConfigureServices method. This container provides the basic support required, but it can be replaced with a custom implementation if desired. What’s more, EF Core also has built-in support for DI, so configuring it within an ASP.NET Core application is as simple as calling an extension method.”

More about ASP.NET Core

If you are looking for an enthusiastic endorsement and overview of ASP.NET Core, don’t miss Philip Japikse's Q&A on Hands-On with ASP.NET Core and EF Core in Visual Studio Magazine.

“I think ASP.NET Core is the biggest game changer in the history of Web development using the Microsoft stack,” asserts Japikse, Developer, Coach, Author, Teacher, Microsoft MVP and Visual Studio Live! presenter.

“Microsoft developers are no longer restricted to running their applications on Windows Server, but can essentially run anywhere,” he explains. “This brings in a myriad of options for deployment targets, including popular containers (like Docker) and lower cost (than Windows) Linux distros. This also opens up .NET as a viable option in those organizations that require development tools to run cross platform. In the past, .NET developers were shut out from those opportunities since Java was the only large-scale enterprise toolset that could ‘check the box’ regarding running cross platform.”

Posted by Richard Seeley on 08/09/20180 comments


Getting Up to Speed with Everything .NET

The .NET bandwagon is rolling and this may be the time to jump onboard and get up to speed on all things .NET.

There's so much to learn about .NET Standard, .NET Core, .NET Framework, Mono, and Xamarin. Plus you need to look to the future of C# and Roslyn, so you know how everything will come together with Visual Studio and Visual Studio Code.

As an article in Visual Studio Magazine noted recently .Net Core, the open source, cross-platform alternative to .NET Framework, is increasingly becoming the runtime of choice for C# coders, according to a survey of developers. Released two years ago, usage has increased since the advent of .NET Core. 2.0 one year ago.

"Microsoft advises that .NET Core be used for certain specific projects -- including highly scalable Web apps, Web apps on Linux or self-contained deployments -- while .NET Framework remains the best option for Windows-only projects," writes David Ramel, editor of Visual Studio Magazine.

For C# programmers using Visual Studio, Microsoft provides a simple tutorial, Build a C# Hello World application with .NET Core in Visual Studio 2017. Following the online instructions and screenshots, it looks like a pretty straightforward way to get started even though "Hello World" is not the equivalent of a mobile banking app.

For .NET overall, the developer community is growing by leaps and bounds since it was open sourced in 2015.

.NET Everywhere for Everyone

Another Visual Studio Magazine article, James Montemagno on .NET 'Everywhere for Everyone' backed up what seems at first to be an extravagant claim.

Montemagno, who serves as Principal Program Manager in Microsoft's Mobile Developer Tools group, gave a keynote speech at the Boston edition of the Visual Studio Live! Conference this past month where he called .NET "a vast, open, constantly growing, and ever-evolving ecosystem." He noted that millions of developers are leveraging .NET to build applications for virtually any platform.

"This is the best time to be a .NET developer," he told the audience.

The .NET ecosystems needs new developers, Montemagno told Visual Studio Magazine, and Microsoft wants to nurture a growing community with easily accessible educational resources.

"It has never been easier to get started with .NET and C# or any of the other languages," Montemagno said.

He suggested going to Microsoft's .NET Web site where "interactive online learning experiences" are available for free to anyone interested. "Those are the types of experiences that are going to enable the next generation of developers," he said, "right there in the browser, ready to go."

While C# is getting all the coverage, .NET developers can work in almost any known programming language. On a Microsoft Channel 9 YouTube video, The Future of .NET, a panel of .NET aficionados noted that Fortran and COBAL code is being ported to .NET as banks and other businesses seek to move from mainframes to more modern metal.

ASP.NET Core Security

Security being top of mind these days, security expert Brock Allen will present a session titled "Modern Security Architecture for ASP.NET Core," at the September 17-20 Visual Studio Live! conference in Chicago. It’s designed to help developers get up to speed on the main components in ASP.NET Core for securing Web applications and Web APIs when using the open-source and cross-platform framework for building cloud based internet connected applications. In Top Tips for Securing ASP.NET Core, an interview this month in Visual Studio Magazine, Allen gives a preview of his upcoming session explaining the security architecture that is important for those working with the framework.

Visual Studio Live! Chicago will feature tracks with topics including:

  • What's new in C#
  • Sharing code between full .NET, .NET Core, and Mono with .NET Standard
  • Roslyn

Additional .NET related tools

Roslyn is a code name for the .NET Compiler Platform, and according to a Wikipedia article, may be the namesake of Roslyn, WA, or Roslyn, the Alaska town where the Northern Exposure TV series was set. Whatever the inspiration for the code name, it is "a set of open-source compilers and code analysis APIs for C# and Visual Basic .NET languages." It is designed to end the frustration developers have faced with “black box” compilers by exposing the APIs.

Xamarin tools share the C# codebase and are used "to write native Android, iOS, and Windows apps with native user interfaces and share code across multiple platforms, including Windows and macOS," according to a Wikipedia article.

Mono is an open-source project piloted by Xamarin, a subsidiary of Microsoft, and the .NET Foundation. It is designed for running .NET applications cross-platform including "Android, most Linux distributions, BSD, macOS, Windows, Solaris, and even some game consoles such as PlayStation 3, Wii, and Xbox 360," according to Wikipedia.

Posted by Richard Seeley on 07/13/20180 comments


DevOps for Beginners

“In the beginner’s mind there are many possibilities, but in the expert’s there are few.” ― Shunryu Suzuki, Zen Mind, Beginner's Mind

If you’re a beginner in DevOps and you can’t figure out how to learn it on your own, don’t worry. DevOps is not about doing things on your own. It’s a team sport. Thinking of doing DevOps on your own is like thinking you could play football on your own. What are you going to do? Throw the football up in the air, run under it and catch it?

One of the keys to learning DevOps is to think about application development in terms of teamwork.

Here’s how Abel Wang, in a keynote at a recent VSLive! conference, explained how Microsoft's DevOps journey evolved when the company made the decision to give up its traditional waterfall approach to development.

He began with Microsoft’s definition of DevOps as "the union of people, process, and products to enable the continuous delivery of value to our end users," according to a report on VisualStudiomagazine.com.

A Cultural Change

It was a cultural change as much if not more than a technology change. My colleague John K. Waters, explained it this way in his report on Wang’s talk.

The new team structure now recognizes only two roles: program manager and engineer. The program manager is roughly the equivalent of a product owner in the Scrum process. Everyone else is an engineer, with no distinctions between developers and testers. Also, restructured: the teams themselves, which had operated in segregated environments: UI developers worked on the UI layer, for example, while database people worked on the database layer. The restructured teams now own the entire feature set from beginning to end, including the UI layer, the data layer, and the database itself, as well as installation, deployment, and quality. Even the workspace was reconfigured: individual offices were replaced by team rooms, where everyone works together, including the program managers.

So if you’re going to begin a career in DevOps, you have to have programming and testing skills, but mostly you have to give up on the idea of the lone coder sitting in a dark concrete block room working on an app. That is not what DevOps is about.

If you are a tester, you’re going to have to learn programming skills you may not have and vice versa if you’re a programmer. No one, least of all Wang, says this is easy.

"Developers traditionally make incredibly bad testers," Wang pointed out in his keynote. "And testers traditionally make very bad coders. So how did we do this? We trained our people and we required them to adapt...."

Considering the talent and brainpower of the average Microsoft employee, it is a gauge of how difficult this was that the attrition rate on Wang’s team was 20 percent. But on the positive side 80 percent adapted to being engineers who could go with the DevOps flow.

"It was incredibly painful," Wang recalled. "We suffered a lot of attrition from all sides -- management, developers, testers -- because the new way of looking at things and doing things was very different from the way we did things before. And we all know no one really likes change. But if there's one constant in our industry it's change."

The IBM Way

IBM has its own take on DevOps and even offers a free eBook, DevOps for Dummies, written by Sanjeev Sharma and Bernie Coyne. Sharma also has a series of video blogs including, DevOps: Where to Start on YouTube. This video is designed to provide an overview for folks who basically know what DevOps is but want to actually implement it in their organizations.

The IBM approach to the team concept in DevOps is different from how Wang and his colleagues at Microsoft do it. As Sharma explains Big Blue’s approach, the team members do have defined roles:

  • Developer
  • Build Engineer
  • QA Team
  • Integration Tester
  • User Tester
  • Operations

He considers the team members with different roles as stakeholders in the overall project who need to collaborate as software moves, for instance, from developers to tester to operations, which is basically the DevOps lifecycle. He acknowledges that there are going to be pain points, for example, testing becoming a bottleneck.

“You look at how you can address those,” Sharma explains. “How you can improve communications and build trust between these teams.”

As can be seen in the different approaches of two giants of the computing world, DevOps, which has only been around as a concept since 2007, is not a paradigm with hard and fast rules that can be employed by everybody everywhere.

For the individual going to work on a DevOps team, where to start may depend on whether your organization is taking an egalitarian approach where you may have to do programming, testing and perhaps go get takeout pizza. So individual skills required will vary. In a Microsoft type system, you may need to become a generalist with coding and testing skills. Following the IBM path, you might specialize in an area such as Integration Testing.

If no one in your organization has DevOps background or skills there are lots of vendors who are willing to help you. But since there are different approaches to DevOps it will be important to make sure the selected vendor’s practices jive with your organization’s culture.

Of course, one place to acquire skills is at VSLive! That’s where you can tap the brainpower of experts like Microsoft’s Abel Wang.

Posted by Richard Seeley on 06/07/20180 comments


Why Get Agile Certification?

This blog started to answer the question: How do you get Agile certified?

But that turns out to be the easy question. A Google search turns up plenty of universities and training organizations willing to teach you so you can get certified for a price.

The Project Management Institute offers an Agile Certified Practitioner (ACP) certification program.

Villanova University offers education and certification programs including Certificate in Agile Management, which it states “is the only comprehensive Agile program offered 100% online by a top-tier university.” It also offers a Professional Certification in Agile & Scrum (PCAS). So finding ways to get Agile credentials is a Google search away, but there are other questions including why go after these certificates?


Why Agile training?

Of course, most developers want to keep their skills current so they are in demand for top projects and top organizations. Perhaps the best bottom line answer comes from Mark J. Balbes, Ph.D., the Agile Architect at ADTmag.com. In Agile and CMMI: Tips from the Trenches, he explained why his company sought Capability Maturity Model Integration (CMMI) certification: “Frankly, it's for the money.” He explains that his “project shop” works on government contracts that increasingly required that credential, so it was a practical business decision.

Developers working in organizations or freelancing in the gig economy will probably understand that level of pragmatism.

Beyond the Business 101 reason for getting Agile certifications, Balbes is an advocate of embracing Agile and getting the training you need.


Thinking differently

In The Survivor's Guide to Agile, he suggests: “Learn what it means to truly be agile and start thinking about your work in those terms.” To learn to think in Agile terms, you’ll need training. Balbes advocates being aggressive in getting the training you need.

“Agile is not immediately intuitive to most people,” The Agile Architect explains. “It takes training and experience to be able to interpret the Agile Manifesto, the 12 Principles and the various practices. Change management is an important part of every successful agile transformation. While your company should be providing the appropriate training and mentoring, take charge of your own education. Read books. Read blogs. Practice techniques like Test Driven Development on your own.”

A key point that Balbes makes is that while learning about new Agile practices is important, unlearning old practices is equally important.

“This is perhaps the hardest thing to do,” he writes. “You already know techniques to address specific needs and situations. When these situations present themselves, it’s only natural to turn to the techniques you have used successfully in the past. However, many traditional techniques aren't aligned with core agile values.”


Suggested reading

Much of what needs to be unlearned is suggested on the first page of The Manifesto for Agile Software Development:
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more. To make sure you have the philosophy supporting these points, it is good to read the brief but deep 12 Principles behind the Agile Manifesto.

ADT’s Agile Architect also suggests that developers start their Agile journey by learning and practicing Test-Driven Development (TDD), which a detailed Wikipedia article defines as “a software development process that relies on the repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only. This is opposed to software development that allows software to be added that is not proven to meet requirements.”


Visual Studio resources

For Visual Studio developers, Microsoft provides Agile resources:

What is Agile? by Aaron Bjork, principal group program manager at Microsoft, who stresses that Agile is more about how you think about development than any specific set of processes:

“It’s important to understand that Agile is not a thing … you don’t do Agile. Rather, Agile is a mindset. A mindset that drives an approach to software development. There’s not one approach here that works for all situations, rather the term Agile has come to represent a variety of methods and practices that align with the value statements in the manifesto.”

Microsoft is advocating the team concept for its Agile Tools with tooling information for Visual Studio Team Services, which “provides you the tools you need to run your agile team.”

Posted by Richard Seeley on 05/11/20180 comments


Beyond AngularJS: Angular 2 is a Whole New World

Transitioning to Angular 2 from Angular 1 also known as AngularJS, the Google supported open-source web application framework, is not so much about upgrading as moving to something different, according to experts.

The differences between the two frameworks were explained in a Visual Studio Magazine Q&A with Ted Neward, who is director of Developer Relations at Smartsheet.com and well-known as a presenter at Visual Studio Live!.

When asked about the challenges developers faced when moving from Angular 1 to Angular 2, Neward pointed first to the naming conventions that started with AngularJS, the JS reference to its JavaScript frontend for client side web apps.

“The Angular team chose to rename the framework from AngularJS to Angular, meaning that Angular should actually be AngularJS, and Angular 2 should actually be just plain ol' Angular, Neward said. “That raises no end of confusion when speaking about the two frameworks, to be sure.”

Basically, Angular 2 is a complete rewrite of Angular 1, the framework formerly known as AngularJS.

It seems AngularJS had issues.

Ashish Bakshi explained in his What is the difference between AngularJs and Angular 2?, Quora blog that “… around year 2012–14 frameworks like ember.js and react.js (developed by Facebook) popped in with better benchmark results and performance, highlighting the AngularJS drawbacks to the developer community.”

As Bakshi relates this history, “… the angular team decided to create a new framework instead of upgrading AngularJS by incorporating all their hard learned lessons from AngularJS. Hence, Angular 2 was released in Sept. 2016 which is a complete re-write of AngularJS.”

Since the release of Angular 2, there have also been more traditional upgraded versions called Angular 4 and Angular 5. (Angular 3 didn’t make the release cut at some point in its development.) Basically Angular 2/4/5 are all best thought of as what Neward referred to as “plain ol' Angular.”

He said “… the naming change was appropriate, because in many ways Angular was a near-total rewrite of AngularJS, meaning that any AngularJS code will not be silently upgradable to Angular. They kept many of the same concepts, but sought to strengthen those concepts and make them more apparent and clear.”

While there has been resistance among AngularJS/Angular1 coders to adopting the new improved plain ol’ Angular, upgrading seems to be the wisest career choice.

In a blog titled Angular 2: Should You Upgrade? Dave Ceddia offered his reasons including:

  • Leaving your software using the old version of a library is Just Not Done.
  • Because the features are better
  • Because Components are the way of the future and the future is awesome.
  • I don’t want to fall behind.
  • I don’t want to be stuck holding the bag (and 100k lines of code) when they deprecate the old one.
  • If I don’t know the newest thing then no one will hire me.

From a practical I-need-a-job standpoint, the last bullet may be the strongest reason to move into the new Angular world.

Also the component approach in plain ol’ Angular is a big deal.

Neward cited it as his favorite part of the re-written framework: “Component-based design is like object design, but with thicker skin, meaning we treat components in a more opaque fashion, making them more accessible and usable for reuse purposes, among other things. Components were what enabled the Golden Age of GUI Builders (the days of VisualBasic, Delphi, PowerBuilder and the like), and there's solid reasons to imagine that something similar will emerge out of this approach for the web -- which in turn means that developers can deliver useful and powerful applications for the web so much faster than before.”

When Neward was asked for tips that would help developers transition to the new version of Angular, he listed:

  • Bookmark the Angular CheatSheet on the Angular Web site.
  • Learn and master the TypeScript language.

He also pointed to Visual Studio Live! sessions on Angular.

At the upcoming VSLive! in Austin, Texas, April 30 - May 4, sessions include:

  • “Angular 101” with Deborah Kurata, Microsoft MVP and Google Developer Expert
  • “Fast Focus: Living Happily Together: Visual Studio and Angular” with Brian Noyes, Solliance CTO, Microsoft Regional Director, MVP
  • “Securing Angular Apps” with Brian Noyes

Posted by Richard Seeley on 04/24/20180 comments


ASP.NET Core Logging

Whenever an app is running, you need to know what’s happening behind the scenes. And whenever an app crashes or otherwise has trouble, you definitely want to know what’s going on. Logging is the simple, yet critical, way to ensure your apps are doing what they’re supposed to be doing. Logging helps you detect and identify any issues.

Now that ASP.Net Core is open source, and cross-platform, building in and linking logging functions is more straightforward than ever. The logging API in ASP.Net Core supports for a whole bunch of logging providers. You can send log details to one location or several locations. You can also connect to a third-party logger. And it’s straightforward. Notice I didn’t say easy, but it’s certainly straightforward. Thankfully, as is often the case, there are several blogs ready to help out.

Logging in ASP.NET Core

Once again, Microsoft’s extensive knowledge base comes to the rescue. While the Microsoft knowledge base, tutorials, and other blog posts are rarely the only source of helpful tips, it’s almost always the best place to start. This guide to logging in ASP.NET Core begins with a brief definition:

“ASP.NET Core supports a logging API that works with a variety of logging providers. Built-in providers let you send logs to one or more destinations, and you can plug in a third-party logging framework. This article shows how to use the built-in logging API and providers in your code.

How to create logs
To create logs, get an ILogger object from the dependency injection container. This example creates logs with the TodoController class as the category. Categories are explained later in this article.

ASP.NET Core doesn't provide async logger methods because logging should be so fast that it isn't worth the cost of using async. If you're in a situation where that's not true, consider changing the way you log. If your data store is slow, write the log messages to a fast store first, then move them to a slow store later. For example, log to a message queue that's read and persisted to slow storage by another process.

How to add providers
A logging provider takes the messages that you create with an ILogger object and displays or stores them. For example, the Console provider displays messages on the console, and the Azure App Service provider can store them in Azure blob storage. To use a provider, call the provider's Add extension method in Program.cs.”

This knowledge base entry also illustrates each instructional narrative with code snippets. After showing you the basics of how to create a log and add a provider, this tutorial post goes on to cover:

  • Sample logging output
  • NuGet packages
  • Log category
  • Log level
  • Log event ID
  • Log message template
  • Logging exceptions
  • Log filtering
  • Log scopes
  • Built-in logging providers
  • Third-party logging providers
  • Azure log streaming

ASP.NET Core - Logging

This tutorial takes you through the whole process of setting up logging for all different versions of ASP.NET Core. It also provides code snippets and listings pf third party loggers you can connect with ASP.NET Core.

“ASP.NET Core framework provides built-in supports for logging. However, we can also use third party logging provider easily in ASP.NET Core application. Before we see how to implement logging in ASP.NET Core application, let's understand the important logging interfaces and classes available in ASP.NET Core framework. The following are built-in interfaces and classes available for logging under Microsoft.Extensions.Logging namespace:

  • ILoggingFactory
  • ILoggingProvider
  • ILogger
  • LoggingFactory

ASP.NET Core framework includes built-in LoggerFactory class that implements ILoggerFactory interface. We can use it to add an instance of type ILoggerProvider and to retrieve ILogger instance for the specified category. ASP.NET Core runtime creates an instance of LoggerFactory class and registers it for ILoggerFactory with the built-in IoC container when the application starts. Thus, we can use ILoggerFactory interface anywhere in your application. The IoC container will pass an instance of LoggerFactory to your application wherever it encounters ILoggerFactory type.”

You’ll find this tutorial requires some familiarity with ASP.NET and ASP.NET Core, but does a comprehensive job of walking you through the process.

Leaner, meaner ASP.NET Core 2 logging

Blogger Nicolas Blumhardt provides a clean, well-organized tutorial on incorporating Serilog with ASP.NET Core. He also supports his statements with code snippets. “You don’t need anything special to use Serilog with .NET Core: the Serilog package works the same way across the .NET Framework and .NET Core on Windows, macOS and Linux. If you are writing .NET Core web apps with ASP.NET, you’ll want to plug into the Microsoft.Extensions.Logging subsystem to receive events from the framework: unhandled errors, diagnostic info from the request processing pipeline, events from EF, and so on.

In ASP.NET Core 2.0, the default logging provider has gained some features, like its own level control and filtering. This is good thing for the out-of-the-box experience, but if you use Serilog, you won’t want two sets of configuration to keep in sync or two logging pipelines running with subtle differences between them.”

Posted by Lafe Low on 04/13/20180 comments


Keep Up-to-Date with Visual Studio Live!

Email address*Country*
Upcoming Events