Discover input/output paths of U-SQL jobs using Azure PowerShell

Do you use Azure Data Lake Analytics? Have you ever wanted to programmatically fetch the input and output paths of one of your U-SQL jobs? Now you can!

This is especially useful for developers who want to quickly debug an input file, fetch the results of an analysis, or check which files a particular job touched. With this new script, we’re helping those users by providing a simple Azure PowerShell script that reduces several steps into a one-line command.

How it works

Each time you submit a U-SQL job, a job folder is created in your Azure Data Lake Store account. This folder contains useful debugging information about the job, including a file called the U-SQL algebra file. This is an XML file containing information about your job graph, the list of input and output files, and other key U-SQL job metadata.

We’ve just published a sample script that reads the U-SQL algebra file for a specified job and returns the input or output files. Give it a try!

Running the sample

First, find the Job ID of the job for which you’d like to get the input or output paths. You can easily find this in the Azure Portal by opening your ADLA account, navigating to the desired job, and copying the Job ID.
Alternatively, you can get the job ID of a job that you just submitted via Azure PowerShell.