Enough of this simple stuff! The LWP object oriented interface gives you more control over the request sent to the server. Using this interface you have full control over headers sent and how you want to handle the response returned.

The lwp-request program (alias GET) that is distributed with the library can also be used to fetch documents from WWW servers.

HEAD

If you just want to check if a document is present (i.e. the URL is valid) try to run code that looks like this:

use LWP::Simple;
if (head($url)) {
# ok document exists
}

The head() function really returns a list of meta-information about the document. The first three values of the list returned are the document type, the size of the document, and the age of the document.

More control over the request or access to all header values returned require that you use the object oriented interface described for GET above. Just s/GET/HEAD/g.

POST

There is no simple procedural interface for posting data to a WWW server. You must use the object oriented interface for this. The most common POST operation is to access a WWW form application:

The lwp-request program (alias POST) that is distributed with the library can also be used for posting data.

PROXIES

Some sites use proxies to go through fire wall machines, or just as cache in order to improve performance. Proxies can also be used for accessing resources through protocols not supported directly (or supported badly :-) by the libwww-perl library.

You should initialize your proxy setting before you start sending requests:

The other alternative is to provide a subclass of LWP::UserAgent that overrides the get_basic_credentials() method. Study the lwp-request program for an example of this.

COOKIES

Some sites like to play games with cookies. By default LWP ignores cookies provided by the servers it visits. LWP will collect cookies and respond to cookie requests if you set up a cookie jar.

use LWP::UserAgent;
use HTTP::Cookies;
$ua = LWP::UserAgent->new;
$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
autosave => 1));
# and then send requests just as you used to do
$res = $ua->request(HTTP::Request->new(GET => "http://www.yahoo.no"));
print $res->status_line, "\n";

As you visit sites that send you cookies to keep, then the file lwpcookies.txt" will grow.

HTTPS

URLs with https scheme are accessed in exactly the same way as with http scheme, provided that an SSL interface module for LWP has been properly installed (see the README.SSL file found in the libwww-perl distribution for more details). If no SSL interface is installed for LWP to use, then you will get "501 Protocol scheme 'https' is not supported" errors when accessing such URLs.

LARGE DOCUMENTS

If the document you want to fetch is too large to be kept in memory, then you have two alternatives. You can instruct the library to write the document content to a file (second $ua->request() argument is a file name):