Connecting To Magento API with CloverETL

Do you use Magento in your company? Have you been thinking of making Magento play together with your other applications such as a POS system? Or are you planning to automate your supply chain? That’s where extensive Magento API comes in handy and CloverETL is the perfect tool for handling the data integration part of it. Read further to learn how to retrieve data from Magento API using CloverETL.

Today we take a look at how CloverETL can help you transfer data back and forth between your Magento platform and other systems.

What is Magento?

Magento is an open-source e-commerce platform and content management system that you can use to build online stores. Magento provides all common e-commerce features such as shopping carts and inventory management. The platform is highly customizable and brings to you all the features to build an enterprise shopping website that would meet all you company specific goals.

Magento provides both REST and SOAP APIs. Both of them allow us to apply more or less the same operations to your Magento data. In this article I will be using Magento’s REST API. The SOAP version was not yet documented at the time of writing this text.

Magento API innovation

With the recent Magento 2 release the API was significantly extended with new functions making it now even easier to operate with third party solutions.

Moreover, Magento API has significantly enhanced its authentication options. While in older versions we were limited to using OAuth only, in Magento 2 we can also use token- and session-based authentication schemes.

CloverETL provides two generic components for working with web services. WebServiceClient for SOAP APIs and HTTPConnector for REST based interfaces.

With this generic approach you can easily build our own connectors and connect to any available web service no matter what the application specifics are.

When building connectors to modern web services, you often need to use multiple HTTPConnector components in sequence to retrieve authentication tokens, issue search query, retrieve resulting data, loop through response results and more. And this will also be the case of Magento.

Using Subgraphs to Create Magento Writer and Search Components

In the following example I am going to utilize CloverETL’s ability to wrap a piece of ETL process into a subgraph hiding the underlying logic from the user and showing only necessary configuration options. I will build two new components – MagentoCustomersWriter and MagentoCustomersSearch.

The idea is that the components can be later reused by anyone with just a few mouse clicks, no coding. This is what the resulting usage might look like:

MagentoCustomersWriter and MagentoCustomersSearch are two new components that are easily configured with only few parameters. Both components require credentials for the Magento account.

The MagentoCustomersSearch component expects an email address that is used to query the customers’ repository.

MagentoCustomersWriter takes data records from its input port and writes them to Magento as customers.

Let’s dive into how these two components are built:

MagentoCustomersSearch Subgraph

Inside the MagentoCustomersSearch subgraph, we need to make two calls to Magento API. First, we need to authenticate with Magento. We are using Magento’s token-based authentication in this example, so we need to request an authentication token (using our user name and password) for later use on the Magento administration page.

The authentication request itself is encapsulated in a shared subgraph called GetAuthenticationToken (we use the same piece of logic for both Magento components). From there we receive an authentication token which we will later pass to Magento. In the Call Magento API step, which is a HTTPConnector, we put together all required parameters and the authentication token and send out a customer search request. Lastly, we parse the JSON content of HTTP response containing the search results into a delimited records.

MagentoCustomersWriter Subgraph

MagentoCustomersWriter receives data through its only input edge. First, we take input records and build a JSON request from them. At the same tame we ask Magento for the authentication token. Then we combine the request and the authentication token, build all request parameters and finally call the Magento API.

If the call succeeds, we pass the newly created customers to the output. In case of an error, we can read the error message from the output port.

Try It Yourself

You can download the example and customize it to your needs. This is how to get it up and running:

If you forget to set these properly, the graph WILL finish OK, but won’t produce any data. You must be granted to manage at least Customers interface under Roles Resources in your Magento instance as displayed on below picture.

Set Email Search to any email you want to search for. The component is configured to search for an exact match of the email.