HDInsight tool for IntelliJ is generally available

We are excited to announce the general availability of the HDInsight Tool for IntelliJ. The HDInsight Tool for IntelliJ serves the open source community and will be of interest to HDInsight Spark developers. It has seamless integration with IntelliJ and enables users to enjoy the native support for the Scala and Java Spark application development experience.

For users who develop Spark applications on HDInsight, it can sometimes be hard to get started with building their first application, and the development cycle of Spark application from IDE could be long and tedious. With this tool, users can get started with HDInsight Spark in just a few minutes. Experienced Spark developers can iterate their development cycle faster and easier.

Tool overview

The IntelliJ HDInsight tool extends IntelliJ to support Spark job life cycle from create, author, debug, to submit job to an Azure HDInsight Spark cluster. It integrates well with Azure to allow users to navigate HDInsight Spark clusters and to view associated Azure storage account. To further boost productivity, the IntelliJ HDInsight tool also offers the capability to view Spark job history, display detailed job logs, and show the job output. A few usability improvements have been implemented upon user preview feedback, including auto create artifact, locate SDK, cache Spark logs, etc..

Create, author, submit and stop a Spark application

The HDInsight Tool for IntelliJ supports Spark application life cycle management from creation to submission. It allows users to view job history, information and statistics for Spark applications running in the cluster with tight integration with IntelliJ and seamless integration with Azure.

Create a new Spark application

View Spark cluster and navigate the associated storage accounts

Submit Spark application

Stop Spark applications from IntelliJ UI

After submitting Spark jobs to HDInsight, the tool allows users to stop a running Spark application in IntelliJ before its completion.

View job history, job output and job log

Installation

User can get the latest bits by going to IntelliJ repository, and searching “HDInsight.” IntelliJ will also prompt users for latest update if user has already installed the plugin.