Persistent Data Monitoring

While accomplishing use cases through the EmcienScan user interface can save a large amount of time and effort over traditional methods of data discovery, the true value in using EmcienScan lies in automating the use cases through the RESTful API's and running scans continuously. This allows for constant, up-to-date knowledge of your data and its qualities. Although you may use a variety of techniques to automate these steps, below we have listed the steps to take to automate these use cases through cURL commands executed on the command line.

Although the examples below list directions for collections of tables, running on individual tables within a database is easy. Just change the API endpoints from "scan_collections" to "scans". For the exact API destinations, see the "API Reference" section within EmcienScan, located here:

When building a profile of your data, the relevant data points will be within the "data_data" structure, most notably the "overall_scan_outlierness". For an exhaustive guide of what each parameter means, scroll down to the bottom of this page. However, if you would like to retrieve the columns for a specific scan within a collection, run:

The relevant information for understanding the data and generating use cases lies within the individual column JSON files. However, you must include column relations in the API call to retrieve this information. In the UI, type "column_relations" in the "include" parameter. The API endpoint and cURL request for this request would be: