Using the Shopify REST Source Component

The REST Source component is an SSIS source component that can be used to read/retrieve data from Shopify REST services and produce column data which can then be consumed by a downstream SSIS pipeline component.

The component includes the following two pages to configure how you want to read data.

General

Columns

General Page

The General page of the REST Source Component allows you to specify the general settings of the component.

Connection Manager

The source component requires a REST connection to Shopify. The Connection Manager drop-down will show a list of all connection managers that are available to your current SSIS package.

Search

Click the search button (or press Ctrl + F on your keyboard) to launch the Search Objects & Endpoints dialog. This dialog will open to the side of the source editor and allows you to search for an object or endpoint. Results are ordered by relevance. Clicking on a result will select that object or endpoint in the source editor.

Category

The Category drop-down is used to divide up objects and endpoints into categories. Selecting a category here will filter the list of available objects in the Source Object drop-down.

Source Object

The Source Object drop-down lists all available objects for the selected Category that you can read from.

Endpoint

The Endpoint drop-down lists all available rest endpoints for the selected object.

Batch Size

The Batch Size lets you specify how many records to retrieve per service call to Shopify.

Note: Some objects do not support batch size, so this option will become disabled.

Output Time Zone

The Output Timezone option lets you specify how all datetime fields should be retrieved. Available options are:

UTC

Local

LinkId Mode

The LinkId Mode option allows you to specify how _LinkIId field should be populated. The _LinkId field is a special column which is used to identify records in the parent output. During the runtime, the value of this field will go into the _{parentObjectName}._LinkId field of all this output's children. This is important for linking outputs with each other.

Implicit (RowIndex): a special field that contains the current count of this output node. This is useful as many times a JSON object will have no key field because relationships are defined hierarchically.

Explicit (Id): the _LinkId column will be populated with the Id of the parent record.

Filter Parameters

You can specify Filter Parameters so that the source component only returns those records that satisfy such filter parameters. If a filter parameter has a yellow asterisk beside it, it means the parameter is required and a value must be provided.

Note: Filter fields may vary from object to object depending on endpoints.

Refresh Component Button

Clicking the Refresh Component button causes the component to retrieve the latest metadata and update each field to its most recent metadata.

Expression fx Icon

Click the blue fx icon to launch SSIS Expression Editor to enable dynamic update of the property at run time.

Generate Documentation Icon

Click the Generate Documentation icon to generate a Word document which describes the component's metadata including relevant mapping, and so on.

Columns page

The Columns page of the REST Source Component shows you all available attributes from the endpoint that you specified on the General page. In this page, you can select the output whose columns you wish to configure in the top left drop-down.

On the top left of the grid, you can see a checkbox, which can be used to toggle the selection of all available fields. This is a productive way to check or uncheck all available fields.

The Columns Page grid consists of:

REST Field: Column that will be retrieved from Shopify REST service.

Additional Field Details: An information icon will appear here if the column is special. Hover over the icon to see what makes the column special.

Data Type: The data type of this field.

Note: As a general best practice, you should only select the fields that are needed for the downstream pipeline components.