Ayrıca Bakınız

User Contributed Notes 134 notes

Please everyone, stop setting CURLOPT_SSL_VERIFYPEER to false or 0. If your PHP installation doesn't have an up-to-date CA root certificate bundle, download the one at the curl website and save it on your server:

If you use cURL to fetch user-supplied URLs (for instance, in a web-based RSS aggregator), be aware of the risk of server-side request forgery (SSRF). This is an attack where the user takes advantage of the fact that cURL requests are sent from the web server itself, to reach network locations they wouldn't be able to reach from outside the network.

For instance, they could enter a "http://localhost" URL, and access things on the web server via "localhost". Or, "ftp://localhost". cURL supports a lot of protocols!

If you are using CURLOPT_FOLLOWLOCATION, the malicious URL could be in a redirect from the original request. cURL also will follow redirect headers to other protocols! (303 See Other; Location: ftp://localhost).

So if you're using cURL with user-supplied URLs, at the very least use CURLOPT_PROTOCOLS (which also sets CURLOPT_REDIR_PROTOCOLS), and either disable CURLOPT_FOLLOWLOCATION or use the "SafeCurl" library to safely follow redirects.

It is important that anyone working with cURL and PHP keep in mind that not all of the CURLOPT and CURLINFO constants are documented. I always recommend reading the cURL documentation directly as it sometimes contains better information. The cURL API in tends to be fubar as well so do not expect things to be where you would normally logically look for them.

curl is especially difficult to work with when it comes to cookies. So I will talk about what I found with PHP 5.6 and curl 7.26.

If you want to manage cookies in memory without using files including reading, writing and clearing custom cookies then continue reading.

To start with, the way to enable in memory only cookies associated with a cURL handle you should use:

curl_setopt($curl, CURLOPT_COOKIEFILE, "");

cURL likes to use magic strings in options as special commands. Rather than having an option to enable the cookie engine in memory it uses a magic string to do that. Although vaguely the documentation here mentions this however most people like me wouldn't even read that because a COOKIEFILE is the complete opposite of what we want.

To get the cookies for a curl handle you can use:

curl_getinfo($curl, CURLINFO_COOKIELIST);

This will give an array containing a string for each cookie. It is tab delimited and unfortunately you will have to parse it yourself if you want to do anything beyond copying the cookies.

To clear the in memory cookies for a cURL handle you can use:

curl_setopt($curl, CURLOPT_COOKIELIST, "ALL");

This is a magic string. There are others in the cURL documentation. If a magic string isn't used, this field should take a cookie in the same string format as in getinfo for the cookielist constant. This can be used to delete individual cookies although it's not the most elegant API for doing so.

An inelegant way to delete a cookie would be to skip the one you don't want.

I only recommend using COOKIELIST with magic strings because the cookie format is not secure or stable. You can inject tabs into at least path and name so it becomes impossible to parse reliably. If you must parse this then to keep it secure I recommend prohibiting more than 6 tabs in the content which probably isn't a big loss to most people.

1440 is the the default number of bytes curl will call the write function (BUFFERSIZE does not affect this, i actually think you can not change this value), so it means the headers are going to be set only one time.

write_function must return the exact number of bytes of the string, so you can return a value with mb_strlen.

- CURLOPT_HEADERFUNCTION is for handling header lines received *in the response*,- CURLOPT_WRITEFUNCTION is for handling data received *from the response*,- CURLOPT_READFUNCTION is for handling data passed along *in the request*.

The callback "string" can be any callable function, that includes the array(&$obj, 'someMethodName') format.

If you are trying to use CURLOPT_FOLLOWLOCATION and you get this warning:Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when in safe_mode or an open_basedir is set...

then you will want to read http://www.php.net/ChangeLog-4.php which says "Disabled CURLOPT_FOLLOWLOCATION in curl when open_basedir or safe_mode are enabled." as of PHP 4.4.4/5.1.5. This is due to the fact that curl is not part of PHP and doesn't know the values of open_basedir or safe_mode, so you could comprimise your webserver operating in safe_mode by redirecting (using header('Location: ...')) to "file://" urls, which curl would have gladly retrieved.

Until the curl extension is changed in PHP or curl (if it ever will) to deal with "Location:" headers, here is a far from perfect remake of the curl_exec function that I am using.

Since there's no curl_getopt function equivalent, you'll have to tweak the function to make it work for your specific use. As it is here, it returns the body of the response and not the header. It also doesn't deal with redirection urls with username and passwords in them.

in particular this is NECESSARY if you are using PEAR_SOAP libraries to build a webservice client over https and the remote server need to establish a session cookie. in fact each soap message is sent using a different curl session!!

Many hosters use PHP safe_mode or/and open_basedir, so you can't use CURLOPT_FOLLOWLOCATION. If you try, you see message like this:CURLOPT_FOLLOWLOCATION cannot be activated when safe_mode is enabled or an open_basedir is set in [you script name & path] on line XXX

It can be use instead of curl_exec. If server HTTP response codes is 30x, function will forward the request as long as the response is not different from 30x (for example, 200 Ok). Also you can use POST.

Sometimes you can't use CURLOPT_COOKIEJAR and CURLOPT_COOKIEFILE becoz of the server php-settings(They say u may grab any files from server using these options). Here is the solution1)Don't use CURLOPT_FOLLOWLOCATION2)Use curl_setopt($ch, CURLOPT_HEADER, 1)3)Grab from the header cookies like this:preg_match_all('|Set-Cookie: (.*);|U', $content, $results); $cookies = implode(';', $results[1]);4)Set them using curl_setopt($ch, CURLOPT_COOKIE, $cookies);

Please note that if you want to handle progress using CURLOPT_PROGRESSFUNCTION option, you need to take into consideration what version of PHP are you using. Since version 5.5.0, compatibility-breaking change was introduced in number/order of the arguments passed to the callback function, and cURL resource is now passed as first argument.

I spent a couple of days trying to POST a multi-dimensional array of form fields, including a file upload, to a remote server to update a product. Here are the breakthroughs that FINALLY allowed the script to run as desired.

Firstly, the HTML form used input names like these:<input type="text" name="product[name]" /><input type="text" name="product[cost]" /><input type="file" name="product[thumbnail]" />in conjunction with two other form inputs not part of the product array<input type="text" name="method" value="put" /><input type="text" name="mode" />

I used several cURL options, but the only two (other than URL) that mattered were:curl_setopt($handle, CURLOPT_POST, true);curl_setopt($handle, CURLOPT_POSTFIELDS, $postfields);Pretty standard so far.Note: headers didn't need to be set, cURL automatically sets headers (like content-type: multipart/form-data; content-length...) when you pass an array into CURLOPT_POSTFIELDS.Note: even though this is supposed to be a PUT command through an HTTP POST form, no special PUT options needed to be passed natively through cURL. Options such ascurl_setopt($handle, CURLOPT_HTTPHEADER, array('X-HTTP-Method-Override: PUT', 'Content-Length: ' . strlen($fields)));orcurl_setopt($handle, CURLOPT_PUT, true);orcurl_setopt($handle, CURLOPT_CUSTOMREQUEST, "PUT);were not needed to make the code work.

-Notice how the @ precedes the temporary filename, this creates a link so PHP will upload/transfer an actual file instead of just the file name, which would happen if the @ isn't included.-Notice how I forcefully set the mime-type of the file to upload. I was having issues where images filetypes were defaulting to octet-stream instead of image/png or image/jpeg or whatever the type of the selected image.

I then tried passing $postfields straight into curl_setopt($this->handle, CURLOPT_POSTFIELDS, $postfields); but it didn't work.I tried using http_build_query($postfields); but that didn't work properly either.In both cases either the file wouldn't be treated as an actual file and the form data wasn't being sent properly. The problem was HTTP's methods of transmitting arrays. While PHP and other languages can figure out how to handle arrays passed via forms, HTTP isn't quite as sofisticated. I had to rewrite the $postfields array like so:$postfields = array("method" => $_POST["method"], "mode" => $_POST["mode"], "product[name]" => $_POST["product"], "product[cost]" => $_POST["product"]["cost"], "product[thumbnail]" => "@{$_FILES["thumbnail"]["tmp_name"]}");curl_setopt($handle, CURLOPT_POSTFIELDS, $postfields);

This, without the use of http_build_query, solved all of my problems. Now the receiving host outputs both $_POST and $_FILES vars correctly.

When CURLOPT_FOLLOWLOCATION and CURLOPT_HEADER are both true and redirect/s have happened then the header returned by curl_exec() will contain all the headers in the redirect chain in the order they were encountered.

If you want cURL to timeout in less than one second, you can use CURLOPT_TIMEOUT_MS, although there is a bug/"feature" on "Unix-like systems" that causes libcurl to timeout immediately if the value is < 1000 ms with the error "cURL Error (28): Timeout was reached". The explanation for this behavior is:

"If libcurl is built to use the standard system name resolver, that portion of the transfer will still use full-second resolution for timeouts with a minimum timeout allowed of one second."

What this means to PHP developers is "You can use this function without testing it first, because you can't tell if libcurl is using the standard system name resolver (but you can be pretty sure it is)"

The problem is that on (Li|U)nix, when libcurl uses the standard name resolver, a SIGALRM is raised during name resolution which libcurl thinks is the timeout alarm.

The solution is to disable signals using CURLOPT_NOSIGNAL. Here's an example script that requests itself causing a 10-second delay so you can test timeouts:

Please notice that CURLINFO_HEADER_OUT and CURLOPT_VERBOSE option does not work together:"When CURLINFO_HEADER_OUT is set to TRUE than CURLOPT_VERBOSE does not work."(from https://bugs.php.net/bug.php?id=65348). This took me an hour or two to figure it out.

If you have a mixture of strings starting with @ (at character) and files in CURLOPT_POSTFIELDS you have a problem (such as posting a tweet with attached media) because curl tries to interpret anything starting with @ as a file.

CURLOPT_POST must be left unset if you want the Content-Type header set to "multipart/form-data" (e.g., when CURLOPT_POSTFIELDS is an array). If you set CURLOPT_POSTFIELDS to an array and have CURLOPT_POST set to TRUE, Content-Length will be -1 and most sane servers will reject the request. If you set CURLOPT_POSTFIELDS to an array and have CURLOPT_POST set to FALSE, cURL will send a GET request.

If you specify a CAINFO, note that the file must be in PEM format! (If not, it won't work).Using Openssl you can use: openssl x509 -in <cert> -inform d -outform PEM -out cert.pem To create a pem formatted certificate from a binary certificate (the one you get if you download the ca somewhere).

Resetting CURLOPT_FILE to STDOUT won't work by calling curl_setopt() with the STDOUT constant or a php://output stream handle (at least I get error messages when trying the code from phpnet at andywaite dot com). Instead, one can simply reset it as a side effect of CURLOPT_RETURNTRANSFER. Just say

# Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
# not to print out the results of its query.
# Instead, it will return the results as a string return value
# from curl_exec() instead of the usual true/false.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);

Handling redirections with curl if safe_mode or open_basedir is enabled. The function working transparent, no problem with header and returntransfer options. You can handle the max redirection with the optional second argument (the function is set the variable to zero if max redirection exceeded).
Second parameter values:
- maxredirect is null or not set: redirect maximum five time, after raise PHP warning
- maxredirect is greather then zero: no raiser error, but parameter variable set to zero
- maxredirect is less or equal zero: no follow redirections

Main issue in existing functions was lack of information, how many redirects was done. This one will count it. First parameter as usual. Second should be already initialized integer, it will be incremented by number of done redirects. You can set CURLOPT_HEADER if You need it.

<?php
/*
* Author: Ojas Ojasvi
* Released: September 25, 2007
* Description: An example of the disguise_curl() function in order to grab contents from a website while remaining fully camouflaged by using a fake user agent and fake headers.
*/

Some additional notes for curlopt_writefunction. I struggled with this at first because it really isn't documented very well.

When you write a callback function and use it with curlopt_writefunction it will be called MULTIPLE times. Your function MUST return the ammount of data written to it each time. It is very picky about this. Here is a snippet from my code that may help you

Now I did this for a class. If you aren't doing OOP then you will obviously need to modify this for your own use.

CURL calls your script MULTIPLE times because the data will not always be sent all at once. Were talking internet here so its broken up into packets. You need to take your data and concatenate it all together until it is all written. I was about to pull my damn hair out because I would get broken chunks of XML back from the server and at random lengths. I finally figured out what was going on. Hope this helps

About the CURLOPT_HTTPHEADER option, it took me some time to figure out how to format the so-called 'Array'. It fact, it is a list of strings. If Curl was already defining a header item, yours will replace it. Here is an example to change the Content Type in a POST:

In my case I wanted to prevent curl from talking to any HTTPS server except my own using a self signed certificate. To do this, you'll need openssl installed and access to the HTTPS Server Certificate (server.crt by default on apache)

You can then use a command simiar to this to translate your apache certificate into one that curl likes.

$ openssl x509 -in server.crt -out outcert.pem -text

Then set CURLOPT_CAINFO equal to the the full path to outcert.pem and turn on CURLOPT_SSL_VERIFYPEER.

If you want to use the CURLOPT_CAPATH option, you should create a directory for all the valid certificates you have created, then use the c_rehash script that is included with openssl to "prepare" the directory.

If you dont use the c_rehash utility, curl will ignore any file in the directory you set.

One note of importance when you open several cURL handles simultaneously: If you want to share cookies via cookie-jar file among all your handles - be sure to curl_close() one before using the cookie-jar file from the other.

It appears that during cURL handler execution the cookies are kept in some sort of handler specific internal session storage and only upon explicit curl_close() call or interpreter exit garbage collection these cookies are actually flushed to the file on the hard disk ( I guess for performance reasons ).

If your POST data seems to be disappearing (POST data empty, request is being handled by the server as a GET), try rearranging the order of CURLOPT_POSTFIELDS setting with CURLOPT_NOBODY. CURLOPT_POSTFIELDS has to come AFTER CURLOPT_NOBODY setting because if it comes after it wipes out the Content-Type header that tells your URL target that the request is a POST not a GET.

Not sure if this is expected behavior but it certainly isn't documented (except on Stackoverflow.com, which is supremely unhelpful - BTW, guys over on stack overflow... once you've figured out a PHP problem, posting the solution here would save everyone extra search time).

As of at least PHP 5.3.9, if you are continuing to use a cURL session handle after downloading a file and closing the file handle, you will need to change CURLOPT_FILE back to stdout, and cannot count simply on a side effect of CURLOPT_RETURNTRANSFER to do so, even if you are setting it. For example:

When you are using CURLOPT_FILE to download directly into a file you must close the file handler after the curl_close() otherwise the file will be incomplete and you will not be able to use it until the end of the execution of the php process.

You can use also use object methods as callback functions. This is usefull if your curl ressource is part of an object handling transfers.
Instead of curl_setopt($curl, CURLOPT_WRITEFUNCTION, "curl_handler_recv") you can use array($object, "method") as value for callback options.

Just a small detail I too easily overlooked.<?php/* If you set: */curl_setopt ($ch, CURLOPT_POST, 1);/* then you must have the data: */curl_setopt ($ch, CURLOPT_POSTFIELDS, $PostData);?>I found with only the CURLOPT_POST set (from copy, paste editing of course) cookies were not getting sent with CURLOPT_COOKIE. Just something subtle to watch out for.

Problems can occur if you mix CURLOPT_URL with a 'Host:' header in CURLOPT_HEADERS on redirects because cURL will combine the host you explicitly stated in the 'Host:' header with the host from the Location: header of the redirect response.

If you want to connect to a server which requires that you identify yourself with a certificate, use following code. Your certificate and servers certificate are signed by an authority whose certificate is in ca.ctr.

if you want to do a GET request with additional body data it will become tricky not to implicitly change the request to a POST, like many notes below correctly state.So to do the analogy of command line's

I managed to use curl to retrieve information from severs on ports other than 80 or 443 (for https) on some installations but not on all.
If you get an "CURLE_COULDNT_CONNECT /* 7 */" error, try adding the port : (for example)

If you are using curl to do a soap request and consistently get the following error back: The server cannot service the request because the media type is unsupported.You are sending the Content-type of soap 1.2 to a 1.1 server.Soap 1.1 needs Content-Type: text/xml;Soap 1.2 should have Content-Type: application/soap+xml;

- CURLOPT_WRITEFUNCTION- CURLOPT_HEADERFUNCTION Pass a function which will be called to write data or headers respectively. The callback function prototype:

long write_callback (resource ch, string data)

The ch argument is CURL session handle. The data argument is data received. Note that its size is variable. When writing data, as much data as possible will be returned in all invokes. When writing headers, exactly one complete header line is returned for better parsing.The function must return number of bytes actually taken care of. If that amount differs from the amount passed to this function, an error will occur.

- CURLOPT_READFUNCTION Pass a function which will be called to read data. The callback function prototype:

string read_callback (resource ch, resource fd, long length)

The ch argument is CURL session handle. The fd argument is file descriptor passed to CURL by CURLOPT_INFILE option. The length argument is maximum length which can be returned.The function must return string containing the data which were read. If length of the data is more than maximum length, it will be truncated to maximum length. Returning anything else than a string means an EOF.

[Note: there is more callbacks implemented in current cURL library but they aren't unfortunately implemented in php curl interface yet.]

If you want to connect to a secure server for posting info/reading info, you need to make cURL with the openSSL options. Then the sequence is nearly identical to the previous example (except http_S_://, and possibly add the useragent):

means that you will tunnel THROUGH the proxy, as in "your communications will go as if the proxy is NOT THERE".

Why do you care? - Well, if you are trying to use, say, Paros, to debug HTTP between your cURL and the server, with CURLOPT_HTTPPROXYTUNNEL set to TRUE Paros will not see or log your traffic thus defeating the purpose and driving you nuts.

There are other cases, of course, where this option is extremely useful...

- 'Server-side' cookies exists as information even before they were set on browser agent(HTTP COOKIE HEADER),- javascript cookies does NOT exists as information before they were set on browser agent,

so, if you're trying to save cookies using CURLOPT_COOKIEJAR to a local file, that cookie must be server - side cookie, otherwise you are wasting time, javascript-produced cookies only exists when client browser's JS interpreter set them.

If you only want to enable cookie handling and you don't need to save the cookies for a separate session, just set CURLOPT_COOKIEFILE to an empty string. I was given the advice to use php://memory but that did not seem to have the same effect.

Although this is stated in the documentation I thought it was worth reiterating since it cause me so much trouble.

Sorry, I made a mistake. For validating cookie entries it is best to use at least:

/^([^\t]+\t){6}[^\t]+$/

There was not enough space for me to put in the rationale for not using persistent storage with cookies but it should be obvious. It's YAGNI for most scenarios. In this case at best it complicate things, at the worst you perform an operation using the wrong cookie session. It can also increase the chance of failure, waste resources, reduce performance and create mess in the file system.

The plus of persistent is that In some cases it may be used to accelerate across processes but not many people actually need that and when they do there tend to be better options such as using memcached.

If you need to do DELETE request, use CURLOPT_CUSTOMREQUEST with "DELETE" and use CURLOPT_POSTFIELDS for parameters. Do not put request parameters into the URL (GET-like) or bad things will happen (at least Apache+mod_php does not like such requests).

With the legacy file upload feature, Curl sends the file name of the actual file and there isn't a documented way to change that behaviour. If you aren't able to use the CURLFile class there's a workaround that apparently works: append "; filename=" after the value (and make sure it comes after "type=").

Using CURLOPT_NOPROXY to avoid using the proxy for some urls is very convenient.For example when the page is trying to look for itself.The parameter can be found at least in version 5.5.7, (probably earlier)Unfortunately it's not present on debian wheezy (5.4.4) but it will be on jessie (it's already there)

to complement shiplu's comment on the neccessary option sequence of CURLOPT_POST before CURLOPT_POSTFIELDS:

The crux is not some error on nginx, but that nothing at all will be send over the line by curl. Parameters set by a "CURLOPT_POSTFIELDS" option setting will be completely ignored, as long as no "CURLOPT_POST" has been encountered beforehand: Neigther the Content-Type header will be set/generated accordingly nor Content-Length nor any data will be send in the body.

When using curl_setopt_array, the sequence in the array matters as well.

If you need to read page contents in between file downloads, while still using the same curl handle, you'll probably need this code:<?php curl_setopt($handle, CURLOPT_FILE, fopen('php://stdout','w')); // 'php://output' didn't work for mecurl_setopt($handle, CURLOPT_RETURNTRANSFER, true); // using CURLOPT_FILE sets this to false automatically?>

Be careful when setting the CURLOPT_POSTFIELDS setting using an array. The array used to set the POST fields must only contain scalar values. Multidimentional arrays or objects lacking a __toString implementation will cause Curl to error.

If there is a need to send non-scalar values using a POST request, consider serializing them before transmission.

The problem I ran into was the filename had an '@' in the middle of it. It turned out that at least on my system if I encoded the file path using the quoted_printable_encode() function the upload works.

I'm posting this in the hopes that it will help someone else, and for my own future reference.

$post must be an array
$page is the page where POST datas will be send.
$n must be true to continue if they are php redirection (Location: )
$session must be define true if you want to use cookies
$referer must be a link to get a wrong referer or only to have a referer.

I noticed that if you want to get current cookie file after curl_exec() - you need to close current curl handle (like it said in manual), but if you want cookies to be dumped to file after any curl_exec (without curl_close) you can:

When POSTing with cURL, my POSTs were magically being converted to GETs and I debugged it until finding the issue. I was setting the CURLOPT_MUTE option. Not sure why this conflicts, since the documentation doesn't specify as such. Anyways, if your $_POST is empty, make sure you aren't setting CURLOPT_MUTE.

Sending a post file upload across a squid proxy, the request was rejected by the proxy. In the error page returned it provided among other possible causes:"Expect:" feature is being asked from a HTTP/one.zero.Solution: Add the option <?php curl_setopt($cl,CURLOPT_HTTPHEADER,array("Expect:")); ?>. This will remove the expect http header.

When passing CURLOPT_POSTFIELDS a url-encoded string in order to use Content-Type: application/x-www-form-urlencoded, you can pass a string directly:<?phpcurl_setopt(CURLOPT_POSTFIELDS, 'field1=value&field2=value2');?>

rather than passing the string in an array, as in fred at themancan dot com's example.

When using CURLOPT_FILE, pass it the file handle that is open for write only (eg fopen('blahblah', 'w+')). If you also open the file for reading (eg fopen('blahblah', 'rw')), curl will fail with error 23.

Note : Having based my snipet on Chemo demonstration (oscommerce user know who he is), XML_POST_URL and XML_PAYLOAD where defined as constant with define().

The point is : at the opposite of .xml , SOAP must send the header 'SOAPAction: ""' that can be a valid URI, an empty string (that is here) or nothing ('SOAPAction: '). The later case baing not accepted by all server, the second one indicating the target is the URI used to post the SOAP.
http://www.w3.org/TR/2000/NOTE-SOAP-20000508/#_Toc478383528

curl_setopt($curl, CURLOPT_URL, "url2"); # this is where you are requesting POST-method form results (working with secure connection using cookies after auth)
curl_setopt($curl, CURLOPT_POSTFIELDS, "var1=value&var2=value&var3=value&"); # form params that'll be used to get form results
$xxx = curl_exec($curl);

Hi,
Anyone who is interested in submitting their information by post to HTTPS site (e.g. payment gateway) where https page needs basic authentication before submitting the information. below code will be helpful.

beware that not all cURLlib constants are supported under php :
e.g. CURLOPT_PROGRESSFUNCTION or CURLOPT_WRITEDATA are not supported.

CURLOPT_WRITEFUNCTION, although undocumented is supported. It takes the name of a user_defined function.
the function should take two arguments (the curl handle, and the inputdata) and return the length of the written data
e.g.

Also be aware that CURLOPT_WRITEFUNCTION does NOT take the CURLOPT_FILE as a parameter!
in curl lib it would take CURLOPT_WRITEDATA but this is not supported by php; that's why I use "global $fd;" in my exemple function.

CURLOPT_HEADERFUNCTION works the same, and is guaranteed to receive complete header lines as input!

The description of the use of the CURLOPT_POSTFIELDS option should be emphasize, that using POST with HTTP/1.1 with cURL implies the use of a "Expect: 100-continue" header. Some web servers will not understand the handling of chunked transfer of post data.

To disable this behavior one must disable the use of the "Expect:" header with

Anyone trying to connect to .NET with CURL to send a simple XML post, pay attention to the following. This will save you hours! There is a previous note that I saw either on this page, or somewhere else on this site that explains the correct way to specify the header option is to create an array, then reference the array from the CURLOPT.

I used to download www pages to my script and one of the pages was different in MS explorer and different, when I downloaded it. Namely, information, I was really interested in was missing. That was because the server on the other bank of the river was looking at who is downloading the page. Everything got fixed when I pretended I was MSIE. It is done with curl. Here is a function, that you may use in similar situation

When you set ($ch, curlopt_post, 1) , after you have posted your data with curl_exec , you need to set ($ch, curlopt_post, 0), Otherwise all your subsequent requests seems as a post with no postdata and some reverse proxy servers send 500 or 403 error for these case ( access denied or forbidden )!

CURLOPT_HTTPHEADER is NOT like the -H command line switch. The command line switch adds or replaces headers (much like the header() line in PHP, but for HTTP clients instead of servers), but the curl extension will eliminate the headers cURL sends by default.

For instance, your Authorization, Host, Referer, Pragma, and Accept headers which are normally written by default or by other CURLOPT_*'s.

Also, it might seem intuitive that this should accept an array hash of header->values, but this is not the case. It accepts an array of strings of the format "Header: Value", much like the -H command-line switch.

Another note addressing the issues with servers that have open_basedir and safe mode turned on. Such an issue spawns the following E_WARNING:

Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when safe_mode is enabled or an open_basedir is set

After looking through the notes, most of the proposed manual implementations were kind of clunky and in some cases just didn't work at all. Most importantly (in my case), was the behaviour of the 302 Header. Anyway, here's the code I ended up using which has worked well for me in all cases so far, it even addresses the issue that caused FOLLOWLOCATION to be turned off in some cases :)

EDIT: Unfortunately the code itself is deemed "too long" for PHP's note system. I've uploaded it to a few paste sites below so hopefully the links will live for a while at least.