Installation

Project Tracking

Usage

Introduction

The primary interface for Typhoeus is comprised of three classes: Request, Response, and Hydra. Request represents an HTTP request object, response represents an HTTP response, and Hydra manages making parallel HTTP connections.

request=Typhoeus::Request.new("www.example.com",method::post,body:"this is a request body",params:{field1:"a field"},headers:{Accept:"text/html"})

We can see from this that the first argument is the url. The second is a set of options.
The options are all optional. The default for :method is :get.

When you want to send URL parameters, you can use :params hash to do so. Please note that in case of you should send a request via x-www-form-urlencoded parameters, you need to use :body hash instead. params are for URL parameters and :body is for the request body.

Sending params in the body with PUT

When using POST the content-type is set automatically to 'application/x-www-form-urlencoded'. That's not the case for any other method like PUT, PATCH, HEAD and so on, irrespective of whether you are using body or not. To get the same result as POST, i.e. a hash in the body coming through as params in the receiver, you need to set the content-type as shown below:

Handling HTTP errors

You can query the response object to figure out if you had a successful
request or not. Here’s some example code that you might use to handle errors.
The callbacks are executed right after the request is finished, make sure to define
them before running the request.

request=Typhoeus::Request.new("www.example.com",followlocation:true)request.on_completedo|response|ifresponse.success?# hell yeah
elsifresponse.timed_out?# aw hell no
log("got a time out")elsifresponse.code==0# Could not get an http response, something's wrong.
log(response.return_message)else# Received a non-successful http response.
log("HTTP request failed: "+response.code.to_s)endendrequest.run

This also works with serial (blocking) requests in the same fashion. Both
serial and parallel requests return a Response object.

Handling file uploads

A File object can be passed as a param for a POST request to handle uploading
files to the server. Typhoeus will upload the file as the original file name
and use Mime::Types to set the content type.

Streaming the response body

Typhoeus can stream responses. When you're expecting a large response,
set the on_body callback on a request. Typhoeus will yield to the callback
with chunks of the response, as they're read. When you set an on_body callback,
Typhoeus will not store the complete response.

If you need to interrupt the stream halfway,
you can return the :abort symbol from the on_body block, example:

request.on_bodydo|chunk|buffer<<chunk:abortifbuffer.size>1024*1024end

This will properly stop the stream internally and avoid any memory leak which
may happen if you interrupt with something like a return, throw or raise.

Making Parallel Requests

Generally, you should be running requests through hydra. Here is how that looks:

hydra=Typhoeus::Hydra.hydrafirst_request=Typhoeus::Request.new("http://example.com/posts/1")first_request.on_completedo|response|third_url=response.bodythird_request=Typhoeus::Request.new(third_url)hydra.queuethird_requestendsecond_request=Typhoeus::Request.new("http://example.com/posts/2")hydra.queuefirst_requesthydra.queuesecond_requesthydra.run# this is a blocking call that returns once all requests are complete

The execution of that code goes something like this. The first and second requests are built and queued. When hydra is run the first and second requests run in parallel. When the first request completes, the third request is then built and queued, in this example based on the result of the first request. The moment it is queued Hydra starts executing it. Meanwhile the second request would continue to run (or it could have completed before the first). Once the third request is done, hydra.run returns.

Making Parallel Requests with Faraday + Typhoeus

require'faraday'conn=Faraday.new(:url=>'http://httppage.com')do|builder|builder.request:url_encodedbuilder.response:loggerbuilder.adapter:typhoeusendconn.in_paralleldoresponse1=conn.get('/first')response2=conn.get('/second')# these will return nil here since the
# requests have not been completed
response1.bodyresponse2.bodyend# after it has been completed the response information is fully available
# response1.status, etc
response1.bodyresponse2.body

Specifying Max Concurrency

Hydra will also handle how many requests you can make in parallel. Things will get flakey if you try to make too many requests at the same time. The built in limit is 200. When more requests than that are queued up, hydra will save them for later and start the requests as others are finished. You can raise or lower the concurrency limit through the Hydra constructor.

There are two different timeouts available: timeout
and connecttimeout.
timeout is the time limit for the entire request in seconds.
connecttimeout is the time limit for just the connection phase, again in seconds.

There are two additional more fine grained options timeout_ms and
connecttimeout_ms. These options offer millisecond precision but are not always available (for instance on linux if nosignal is not set to true).

When you pass a floating point timeout (or connecttimeout) Typhoeus will set timeout_ms for you if it has not been defined. The actual timeout values passed to curl will always be rounded up.

DNS timeouts of less than one second are not supported unless curl is compiled with an asynchronous resolver.

The default timeout is 0 (zero) which means curl never times out during transfer. The default connecttimeout is 300 seconds. A connecttimeout of 0 will also result in the default connecttimeout of 300 seconds.

Setting the header hash directly will not include the --compressed flag in the libcurl command and therefore libcurl will not decompress the response. If you want the --compressed flag to be added automatically, set :accept_encoding Typhoeus option.

Other CURL options

SSL

SSL comes built in to libcurl so it’s in Typhoeus as well. If you pass in a
url with "https" it should just work assuming that you have your cert
bundle in order and the server is
verifiable. You must also have libcurl built with SSL support enabled. You can
check that by doing this:

curl--version

Now, even if you have libcurl built with OpenSSL you may still have a messed
up cert bundle or if you’re hitting a non-verifiable SSL server then you’ll
have to disable peer verification to make SSL work. Like this:

If you are getting "SSL: certificate subject name does not match target host
name" from curl (ex:- you are trying to access to b.c.host.com when the
certificate subject is *.host.com). You can disable host verification. Like
this:

Verbose debug output

Just remember that libcurl prints it’s debug output to the console (to
STDERR), so you’ll need to run your scripts from the console to see it.

Default User Agent Header

In many cases, all HTTP requests made by an application require the same User-Agent header set. Instead of supplying it on a per-request basis by supplying a custom header, it is possible to override it for all requests using:

Semantic Versioning

LICENSE

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without
limitation the rights to use, copy, modify, merge, publish, distribute,
sublicense, and/or sell copies of the Software, and to permit persons
to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.