PHP: Perform HTTP requests in parallel

Ever had to request multiple HTTP-resources in your web application? Often, you need data from one request to be able to request the second – in this case there is little you can do but wait for the first to return. However, if the requests are not dependent on each other, you can use a pretty cool trick: curl_multi_*.

Let’s say you wanted to fetch the public data for VG.no and tech.vg.no from Facebook’s Graph API. One might use file_get_contents, or a standard curl-call:

The problem with this approach is obviously that it waits for each request to return before proceeding to the next. If each request takes 5 seconds to perform, we’d have to wait 10 seconds before we’d be able to process all our data and return it to the user.

So, how do we make these request perform in parallel? curl_multi_*. This feature seems to have been introduced in PHP5, but I’m sure many (like me) have not come across it before. Let’s take a look at how we could optimize our code using these functions:

Using this technique, we’ve reduced the time it takes to perform the requests down to the slowest request in the batch. Nice!

However, I’m sure many (like me) are looking at this code and thinking: “Wow. That is a lot of code, and far from readable”. I agree. Thanks to a wonderful PHP HTTP client called Guzzle, we can achieve the same result with much more readable code:

I would argue that the CURL solution (especially when using Guzzle or similar) is a simpler approach. Performance-wise it shouldn't really matter much - your bottleneck will be in network, not local resources.

I have used this technique before. We were doing a custom social button implementation and in order to get all the counts of likes, comments, tweets, +1's, and a couple other miscellaneous counts I had to make a total of up to 24 requests to hit all the API's we needed. Thankfully facebook has a batch API or it would have been 35 requests. Dropped the total time for that ajax call down to just 2.5 seconds down from closer to 15-20 per page load.