Updates to database source data catalog

From time to time, your database source data definition changes. The change could be as small as the data type is modified or as big as new tables added, other tables dropped and columns added or dropped. All of these changes can be seamlessly synched to DvSum using Refresh Catalog.

STEPS

Step1. Make changes to your source data object(s). If you are using Staging configuration for any of those, remember to also make those changes on the staging table.

Step3. Open Manage Sources under Administration folder from left menu. (Note: Only Admin or Owner roles have access to this folder)

Step4. Search for and select the source(s) that you want to update

Step5. Click on Refresh Catalog. DvSum application will read your latest database definition from your source and synchronize it with the catalog in DvSum.

Step6. The synchronization will result in following possible updates.

Tables or Views

- if new tables or views are added, they will automatically be imported if they are part of the filter condition in catalog definition

- if tables or views that previously existed but were dropped, will be marked for deletion in DvSum. Under the catalog list in Profiling page, they will show up with a prefix (deleted). The administrator can after review, go and delete these from Profiling screen

b. Columns or Fields

- if new fields are added, they are automatically imported and become visible

- if fields that previously existed but are not dropped, will be automatically removed from DvSum. Note that if there were rules that used those fields, those rules will become invalid and may have to be re-configured

- if fields were renamed, those would be done as a combination of drop of existing field and adding of a new field. So those would have to be re-profiled to generate statistics

Steps 7-12 are required only if you are using the staging configuration and if you added new columns or changed names of any columns.

When you use the staging configuration, the new field is not automatically included in the staging configuration as its domain data type and pre-processing and post-processing logic has to be set. Go to Staging Configuration

Step 7. Go to Staging configuration of your data source by navigating from Profiling Main Page and clicking on Show Details.

Step8. Select the Staging Configuration tab and click on Edit. Then press Save. This action will re-initialize the configuration. Edit one more time.

Step9. Scroll to the bottom of the screen. The new fields will show up in the list, but are not included in the mapping

Step10. Check the box to include the field in staging workflow. Click on the pencil against the field to also set its match data type and any pre-process validation or standardization settings

Step11. Scroll to the top and press Save. The system will warn you that if there are any rules associated with the configuration, those will be reset due to change in structure. Go ahead and click OK.

Step12. Object maintenance is complete and you can continue using the application.