Posted on Nov 11, 2011

We’ve struck the need to debug client’s calls to the Magento API during product import and update processes on a couple of sites. Magento’s API isn’t the fastest, but various factors could be contributing to performance issues and errors being returned. In a couple of cases recently, we’ve needed to check:

What does the raw request look like, are similar calls similarly slow?

If we can establish some of the above we are better equipped to pinpoint the problem and fix, so on the suggestion of a good mate I decided to write a proxy script to run all requests through, log things and see what was going on.

API Versions

First you need to get your versions right. Magento has two versions of its API, with different URLs. For this to work we’ll need to make sure our proxy is inline with the right version, as we discovered when a client was calling V2 and the proxy calling V1. Not surprisingly that didn’t work, so we’ll configure the proxy script to an api on a per-site basis.

The Test Script

Here’s a basic script, which will call the Magento API first to create a session then to request a customer record or other API calls. Note that we point the SoapClient at our proxy script, not the Magento API URl. There are calls to both V1 and V2 APIs in here. There are also calls to a logger class which is bespoke for this purpose (see below).

All we need to do is call this via a browser and it’ll run some SOAP requests via the API. Then we check logs to see what happened, or monitor using tail -F to view in realtime.

The Logger Class

This is pretty simple but capable of producing some nice easy to read logs. It’s instantiated as shown above and takes some optional parameters of sessionID and requestID so we can easily trace each session, and each call:

See the display_errors Off at the top? We need to ensure nothing spurious gets written out as it’ll mess up the SOAP response and cause bigger errors on the client. I might examine the SOAP error response Magento uses and wrap exceptions in that, but ultimately a failure is a failure…

The proxy script – proxy.php

While it’s a bit big to post in full here, it’s available as part of the download

Still, to give some explanation, the first 4 variables in the class are configurable. The last (api_url) is not required as we can derive the (actual Magento) URL from environmental variables providing this is a standard installation:

On initialisation, the proxy will create two log files (if not already created). Depending on debug level above it’ll create proxylog_YYYYMMDD.log and proxylog_raw_YYYYMMDD.log. It’ll then set about constructing the API URL we want to hit, it’s own URL, and an md5 request ID.

Next it checks for raw POST data from the client. Note a SOAP request won’t turn up using $_POST as it’s not assigned to a variable in that array. We use:

$this->PostData = file_get_contents('php://input');

Initialisation done, we run the request.

If there’s no POST data, but there is a ?wsdl in the URL we simply need to deliver the WSDL so that PHP can create a SoapClient object. We do this with a simple file_get_contents, replace the actual API url with our own proxy URL and return to the client:

If we do have some POST data, we assume this is a SOAP request and process it. If we can parse out a node of sessionId from the XML we store it for logging. If it’s not there we assume it’s a new session request and log that. We’ll then extract the sessionID from the response once we’ve received it back.

If we have a sessionID within the call then this is a main API request to do something more serious. Depending on the API version we extract the call handle from the XML and log that the request has arrived in. We log this to both logs and add the raw post to the raw log. These are each tied together by the request ID.

Next we start a millisecond timer, send the raw POST data on to the actual API, and get the response back:

We don’t need to worry too much about what comes back, other than to log it, including the duration of the response, and send it back to the client. This is just a case of setting headers and echoing the result out:

My good friend Steve prompted me to write this over a pint one night when discussing these problems. He also (rather profoundly) stated “Log files give you power” and he’s dead right. So far this script has given us the ability to prove what’s actually going on inside the hidden world of web service requests, and I can see us applying it far beyond the Magento API.

Feedback / suggestions / bug reports as always appreciated!

Download the files here
Please test in a dev environment and examine the code before you do. I’m not responsible for anything that may go wrong from their use.

Magento is such a pig. Why doesn’t it show the error message? WTF? What the f*** is the point in having an error message when there’s no way to get it without a hack? I f***ing hate Magento, it’s API is the dog worst thing in the universe. I’d rather work with Hitler than Magento.
I spent all yesterday getting an auto-update with the magento API working, guess what! Magento’s API doesn’t maintain referential integrity! Deleting a parent category while it has children throws “Internal Error” and then corrupts the database so the backend wont run! What a heap of junk

Yep – not sure why this would be different calling it from your ERP system to the test script, but that error is related to the version of SoapClient differing from the version the Soap Server (in this case, the proxy script itself).

Can take another parameter to specify the soap_version, 1.1 or 1.2 (it should default to 1.1, which is what the proxy should output) as follows:
$soap = new SoapClient( $mage_proxy_url, array(‘trace’ => 1, ‘cache_wsdl’=>0, ‘soap_version’ => SOAP_1_2) );

It looks like version 1.2 uses application/soap+xml as the content type where 1.1 uses text/xml. I’m using “application/xml” which is probably the problem in the SendPost method.

Thanks for the help so far. I’m back on this after a break in Dec. Unfortunately I’m not much of a programmer – I can see whats going on here to some degree, but just don’t quite know how to tackle it.