In this update, we also continued working on Server features—adding support for Remote File Listeners (the ability to watch
FTP, SFTP, Samba, and S3 folders) and introducing Task Failure Listeners, an additional automation option for monitoring and recovery
of Event listeners.

Don’t forget to pay attention to the Compatibility section. We’re changing some defaults that may potentially
affect many core components.
However, you shouldn’t see any negative impact, as the changes we’re making are aimed towards more predictable, and
thus error-proof, behavior.

With these components you also save your API limits—each query or write takes only a single API call (per 2,000 records) and returns results faster
compared to SalesforceBulkReader/Writer, which work best for large batches.

However, these new components are limited to low data volumes, typically up to thousands of records. That makes them
great for lookups or ad hoc queries, but for large data sets, SalesforceBulkReader/Writer would be a much better option.

File Listeners—Remote Folders and "File Finished" Check

Corporate Server and Cluster only

You can now watch "hot" remote folders—FTP, SFTP, Amazon S3 and Samba shares. Previously, only local folders were supported.

Also, you can now avoid jobs triggering on unfinished files (large files might appear in a folder before they finish copying/uploading).
Do so by setting a polling interval that blocks the trigger until the file size has stabilized.

Task Failure Listeners

Corporate Server and Cluster only

Task Failure listeners allow you to configure reactions to failures of other listeners
(e.g. a File Event Listener watching a "hot" folder starts failing because the folder becomes unavailable).

The reaction can be anything a normal listener can do, from sending emails (default) to running cleanup jobs, scripts, etc.

ParallelReader Supports S3

Filter shortcut in Data Inspector

Right-click a value to quickly turn it into a filter in Data Inspector.

Omit XML declaration in XMLWriter to easily generate fragments

You can instruct XMLWriter to omit <?xml ... ?> declaration to write out fragments which you can later easily append or merge together.

By default the declaration is generated (set to "false"), maintaining backwards compatibility.

Compatibility

Please read carefully:

We've changed the default encoding for many components to UTF-8 (from ISO-8859-1).

This might affect your existing transformations (e.g. failing or records that are were okay before but are now invalid in UTF-8 definition).

We recommend reviewing the Compatibility section for this release. We're introducing some changes to defaults
that could affect your production data (only if you are relying on some strangely behaved defaults).

Installation/Upgrade Instructions for CloverETL 4.4

To help you install or upgrade to this version, we've prepared a simple checklist:

Before You Upgrade

Be sure to check the "Compatibility" notes for ALL intermediary releases.
We mark all changes that can potentially alter the function of your existing transformations with a "Compatibility" label to make it easier for you.
You can safely ignore most of them, as we try hard to keep as much backwards compatibility as we can.
There's a comprehensive list of all releases that will help you get the information quickly.

Upgrade Designer and Server together. We always release Designer and Server together under a single version. It's highly recommended
to upgrade Server and Designer at the same time. Although using different versions of Designer to connect to Server might work, it is not generally supported.

There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over older version,
will automatically update whatever is necessary in your workspaces, sandboxes, and Server databases as needed.

Don't forget to backup.
Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations,
jobflows, and configurations safe. However, as a good word of advice, it never hurts to have a backup.

Designer Upgrade

Install the new version of Designer.
You can install Designer over your existing installation. The process will automatically clean up the old version.
Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins
to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that.
When you start the application, point it to your existing workspace directory. With some
major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able
to use the workspace with previous versions. Thus, be sure to upgrade all Designers if you're sharing the workspace.

Activate the product on first start. You will need a new key as we issue new license keys for every new major version
(i.e. from 4.2 to 4.3). However, if you're on our maintenance program, we automatically renew the keys for you.
Just go to the download area again and copy/paste the license keys from there.
If you can't find the latest keys, please contact us to renew your product maintenance.

Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your checks there first.

Follow step-by-step Server Upgrade Guide.
We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.

Activate the product on the login screen of Server Console. The license key changes with every major version
(i.e. from 4.2 to 4.3) and we automatically renew the keys for you. Just go to the download area again and copy/paste the license key from there.

SalesforceReader can be set to read deleted records. Salesforce keeps deleted/archived records temporarily (~2 weeks) available for reading. This is quite useful for CDC, comparing two snapshots and finding deleted records.

Task Failure Listeners

Introducing "Task Failure" listeners. A new way to trigger tasks (emails, job execution, script) when a task fails to execute (e.g. a file listener is unable to check a folder because the folder is no longer available)

Remote File Event Listeners

File Event Listeners

Improved URL entry in File Listeners (driven by the need for more complex URLs for the new Remote File Listeners). A new Paste URL option will automatically parse and populate the form fields (host, port, …). Validate Accessibility button can be used to verify that CloverETL can connect to the specified remote location.

Please note this applies to IMPLICIT mappings ONLY. If you've defined any explicit mapping ($out.decimalField = $in.0.sourceDecimal) or star mapping ($out.0.* = $in.0.*), the new behavior was already there—you would get a warning in the mapping dialog and you would get runtime errors for values that wouldn't fit.