Spark + AI Summit 2019

Last week at Spark + AI Summit 2019, Microsoft announced joining the open source MLflow project as an active contributor. Developers can use the standard MLflow tracking API to track runs and deploy models directly into Azure Machine Learning service. We also announced that managed MLflow is generally available on Azure Databricks and will use Azure Machine Learning to track the full ML lifecycle. The combination of Azure Databricks and Azure Machine Learning makes Azure the best cloud for machine learning. Databricks open sourced Databricks Delta, which Azure Databricks customers get greater reliability, improved performance, and the ability to simplify their data pipelines. Lastly, .NET for Apache Spark is available in preview, which is a free, open-source, .NET Standard compliant, and cross-platform big data analytics framework.

With only a few lines of code you can start integrating the power of Azure Cognitive Services into your big data workflows on Apache Spark™. The Spark bindings offer high throughput and run anywhere you run Spark. The Cognitive Services on Spark fully integrate with containers for high performance, on premises, or low connectivity scenarios. Finally, we have provided a general framework for working with any web service on Spark. You can start leveraging the Cognitive Services for your project with our open source initiative MMLSpark on Azure Databricks.

Now in preview

Announcing the second preview release of the Azure Services App Authentication library, version 1.2.0, which release enables simple and seamless authentication to Azure SQL Database for existing .NET applications with no code changes – only configuration changes. Try out the new functionality in existing SQL-backed solutions and gain the security benefits that the App Authentication library and managed identities afford.

Now generally available

Announcing general availability support of the move functionality for recovery services vaults, which is an Azure Resource Manager resource to manage your backup and disaster recovery needs natively in the cloud. Migrate a vault between subscriptions and resource groups with a few steps, in minimal downtime and without any data-loss of old backups. Move a Recovery Services vault and retain recovery points of protected virtual machines (VMs) to restore to any point in time later.

Announcing the general availability of Azure SQL Data Warehouse reserved capacity and software plans for RedHat Enterprise Linux and SUSE. Purchase Reserved Capacity for Azure SQL Data Warehouse and get up to a 65 percent discount over pay-as-you-go rates. Select from 1-year or 3-year pre-commit options. Purchase plans for RedHat Enterprise Linux and save up to 18 percent. Plans are only available for Red Hat Enterprise Linux virtual machines and the discount does not apply to RedHat Enterprise Linux SAP HANA VMs or RedHat Enterprise Linux SAP Business Apps VMs. Save up to 64 percent on your SUSE software costs. SUSE plans get the auto-fit benefit, so you can scale up or down your SUSE VM sizes and the reservations will continue to apply. In addition, there is a new experience to purchase reservations and software plans, including REST APIs to purchase azure reservation and software plans.

Announcing the general availability of Azure Cost Management features for all Pay-As-You-Go and Azure Government customers, which will greatly enhance your ability to analyze and proactively manage your cloud costs. These features enable you to analyze your cost data, configure budgets to drive accountability for cloud costs, and export pre-configured reports on a schedule to support deeper data analysis within your own systems. This release for Pay-As-You-Go customers also provides invoice reconciliation support in the Azure portal via a usage csv download of all charges applicable to your invoices.

News and updates

Docker recently announced Docker Hub had a brief security exposure that enabled unauthorized access to a Docker Hub database, exposing 190k Hub accounts and their associated GitHub tokens for automated builds. While initial information led people to believe the hashes of the accounts could lead to image:tags being updated with vulnerabilities, including official and microsoft/ org images, this was not the case. Microsoft has confirmed that the official Microsoft images hosted in Docker Hub have not been compromised. Regardless of which cloud you use, or if you are working on-premises, importing production images to a private registry is a best practice that puts you in control of the authentication, availability, reliability and performance of image pulls.

Do you have an idea that could improve and empower the lives of everyone in a more accessible way? Or perhaps you have an idea that would help create a sustainable balance between modern society and the environment? Even if it’s just the kernel of an idea, it’s a concept worth exploring with the AI for Good Idea Challenge. If you’re a developer, a data scientist, a student of AI, or even just passionate about AI and machine learning, we encourage you to take part in the AI for Good: Developer challenge and improve the world by sharing your ideas.

When Google announced its migration from Google Cloud Messaging (GCM) to Firebase Cloud Messaging (FCM), push services like Azure Notification Hubs had to adjust how we send notifications to Android devices to accommodate the change. If your app uses the GCM library, follow Google’s instructions to upgrade to the FCM library in your app. Our SDK is compatible with either. As long as you’re up to date with our SDK version, you won’t have to update anything in your app on our side.

Part 2 covers a common scenario where SAP customers can experience the speed and agility of the Azure platform is the ability to migrate from a SAP Business Suite running on-premises to SAP S/4HANA in the cloud.

When customers are impressed or dissatisfied about a product, they come back to where it was purchased looking for a way to leave feedback. See how to use artificial intelligence (Cognitive Service) to suggest star ratings based on sentiment - detected as customers write positive or negative words in their product reviews. Learn about CogS, Sentiment Analysis, and Azure Functions through a full tutorial - as well as where to go to learn more and setup a database to store and manage submissions.

See how Tim Heuer refactored a solution to a common problem NYC residents face—trying to figure out street-side parking rules— using Azure Logic Apps and provided connectors for Twitter and Twilio to accomplish the same thing.

Dave Glover demonstrates how one can use Azure Custom Vision and Azure IoT Edge to build a self-service checkout experience for visually impaired people—all without not needing to be a data scientist. The solution is extended with Python Azure Function, SignalR and Static Website Single Page App.

For those who prefer the command line, it's possible to interact with Azure DevOps using the Azure CLI. Neil Peterson takes a quick look at the configuration and basic functionality of the CLI extension as related to Azure Pipelines.

The options for pushing your .NET Core application to the cloud are not lacking depending on what IDE or editor you have in front of you. But what if you just wanted to deploy your application to Azure with a single command? Shayne Boyer shows you how to do just that with the dotnet-azure global tool.

More and more services are moving to the cloud and they bring their security challenges with them. In this blog post, we will focus on the security concerns of container environments. This post goes over several security concerns in containerized environments, from the Docker level to the Kubernetes cluster level, and shows how Azure Security Center can help you detect and mitigate threats in the environment as they’re occurring in real time.

Cloud optimization is critical to ensuring you get the most out of your Azure investment, especially in complex environments with many Azure subscriptions and resource groups. Learn how Azure Advisor helps you optimize your Azure resources for high availability, security, performance, and cost by providing free, personalized recommendations based on your Azure usage and configurations.

Azure Stream Analytics is an on-demand real-time analytics service to power intelligent action. Azure Stream Analytics tools for Visual Studio make it easier for you to develop, manage, and test Stream Analytics jobs. This post introduces capabilities and features to help you improve productivity that were included in two major updates from earlier this year: test partial scripts locally; share inputs, outputs, and functions across multiple scripts; duplicate a job to other regions; local input schema auto-completion; and testing queries against SQL database as reference data.

Since inception in 2017, the Azure Tips & Tricks collection has grown to over 200+ tips as well as videos, conference talks, and several eBooks spanning the entire breadth of the Azure platform. Featuring a new weekly tip and video it is designed to help you boost your productivity with Azure, and all tips are based off of practical real-world scenarios. This post re-introduces a web resource called Azure Tips and Tricks that helps existing developers using Azure learn something new within a couple of minutes.

You no longer have to be a database expert to optimize your database. Make your job easier and start taking advantage of Azure Database for PostgreSQL Recommendation for Microsoft Azure Database for PostgreSQL today. By analyzing the workloads on your server, the recommendations feature gives you daily insights about the Azure Database for PostgreSQL resources that you can optimize for performance. These recommendations are tightly integrated with Azure Advisor to provide you with best practices directly within the Azure portal.

Azure shows

Derek Martin, a Technology Solutions Principal (TSP) at Microsoft talks about his approach to ensuring that customers get the foundational elements of Azure in place first before deploying anything else. He discusses why Microsoft is getting more opinionated, as a company, when advocating for best practices.

Gaurav Malhotra joins Scott Hanselman to show how to build a modern data warehouse solution from ingress of structured, unstructured, semi-structured data to code-free data transformation at scale and finally to extracting business insights into your Azure SQL Data Warehouse.

Eamon O'Reilly joins Scott Hanselman to show how PowerShell in Azure Functions makes it possible for you to automate operational tasks and take advantage of the native Azure integration to deliver and maintain services.

Mukund Ghangurde is part of the Industry x.0 practice at Accenture focused on driving digital transformation and digital reinvention with industry customers. Mukund joined us on the IoT Show to discuss the scenarios he is seeing in the industry where IoT is really transforming businesses and how (and why) Accenture is partnering with Azure IoT to accelerate and scale their IoT solutions.

Integration with smart contracts is a common topic with developers. Whether with apps, data, messaging or services there is a desire to connect the functions and events of smart contracts in an end to end scenario. In this episode, we look at the different types of scenarios and look at the most common use case – how to quickly expose your smart contract functions as microservices with the Ethereum Blockchain Connector for Logic Apps or Flow.

Learn how to configure load balancers and how to add virtual machines to them in the Azure Portal.

Rockford Lhotka on Software Architecture | The Azure DevOps Podcast

This week, Jeffrey Palermo and Rocky Lhotka are discussing software architecture. They discuss what Rocky is seeing transformation-wise on both the client side and server side, compare and visit the spectrum of Containers vs. virtual machines vs. PaaS vs. Azure Functions, and take a look at microservice architecture. Rocky also gives his tips and recommendations for companies who identify as .NET shops, and whether you should go with Containers or PaaS.

Microsoft's vast partner-ecosystem is a big part of the Azure value proposition. Listen in as Microsoft Enterprise Channel Manager Christine Schanne and Louis Berman delve into the partner experience with Neudesic; a top Gold Microsoft Partner.

HTML5 audio not supported

Events

Global Azure Bootcamp is a free, one-day, local event that takes place globally. This annual event, which is run by the Azure community, took place this past Saturday, April 27, 2019. Each year, thousands attend these free events to expand their knowledge about Azure using a variety of formats as chosen by each location. Did you attend?

We added a little “cosmic touch” to the Global Azure Bootcamp this past weekend by enabling attendees to greet each other with a chat app powered by Azure Cosmos DB. For a chat app, this means low latency in the ingestion and delivery of messages. To achieve that, we deployed our web chat over several Azure regions worldwide and let Azure Traffic Manager route users’ requests to the nearest region where our Cosmos database was deployed to bring data close to the compute and the users being served. That was enough to yield near real-time message delivery performance as we let Azure Cosmos DB replicate new messages to each covered region.

Customers, partners, and industries

Extracting insights from multiple data sources is a new goal for manufacturers. Industrial IoT (IIoT) data is the starting point for new solutions, with the potential for giving manufacturers a competitive edge. These systems contain vast and vital kinds of information, but they run in silos. This data is rarely correlated and exchanged. To help solve this problem, Altizon created the Datonis Suite, which is a complete industrial IoT solution for manufacturers to leverage their existing data sources.

Just over 25 years ago, Bill Gates and Hasso Plattner met to form an alliance between Microsoft and SAP that has become one of our industry’s longest lasting alliances. At the time their conversation was focused on how Windows could be the leading operating system for SAP’s SAPGUI desktop client and when released a few years later, how Windows NT could be a server operating system of choice for running SAP R/3. Ninety percent of today’s Fortune 500 customers use Microsoft Azure and an estimated 80 percent of Fortune customers run SAP solutions, so it makes sense why SAP running on Azure is a key joint initiative between Microsoft and SAP. Over the next three weeks leading up to this year’s SAPPHIRENOW conference in Orlando, we’re publishing an SAP on Azure technical blog series (see Parts 1 & 2 in Technical content above).

The Azure Marketplace is the premier destination for all your software needs – certified and optimized to run on Azure. Find, try, purchase, and provision applications & services from hundreds of leading software providers. You can also connect with Gold and Silver Microsoft Cloud Competency partners to help your adoption of Azure. In the first half of March, we published 68 new offers.