vignettes/src/toxiproxyr.R

## ---## title: "toxiproxyr"## author: "Rich FitzJohn"## date: "`r Sys.Date()`"## output: rmarkdown::html_vignette## vignette: >## %\VignetteIndexEntry{toxiproxyr}## %\VignetteEngine{knitr::rmarkdown}## %\VignetteEncoding{UTF-8}## ---##+ echo = FALSE, results = "hide"loadNamespace("toxiproxy")loadNamespace("RMySQL")loadNamespace("httr")Sys.setenv(TOXIPROXY_PORT =8474)
toxiproxyr::toxiproxy_clear()## `toxiproxyr` is an R client for the## [`toxiproxy`](https://toxiproxy.io) chaotic proxy. It is designed## for use within test suites of R packages that use network## connections and want to test how the package will respond to things## like network outages, slow connections, etc. Toxiproxy can## simulate a number of problems with a network connection and can## apply these problems all the time or stochastically.## The `toxiproxyr` package is a simple wrapper around the `toxiproxy`## API. It is not necessary for use with `toxiproxy`, but is designed## to make it easy to use from R tests.## The general approach for using `toxiproxyr` is to:#### 1. Create a "proxy" for your service## 2. Add "toxics" to that proxy## 3. Connect your client to the proxy and see how it responds## On this computer, I have toxiproxy running on port 8474 (its## default). We also need a *second* service to create a proxy for;## this is the service that we care about testing. Anything that uses## a TCP connection would be good here (a webserver, database like## postgres, mysql, redis, mongodb). For this vignette, to keep## things simple I'm going to proxy traffic to the toxiproxy## webserver.
test_client <-function(port){function(){
r <- httr::GET(sprintf("http://127.0.0.1:%d/version", port))
httr::status_code(r)<300}}## (Note: I am using `127.0.0.1` rather than `localhost` as at least## on the windows machine I have, `localhost` suffers a noticable## delay - possibly due to a DNS lookup.)## With this function I can test if traffic to a port is working,## having sent a round trip. So this works (because I have a## webserver that is listening on 8474; it happens to be the toxiproxy## server but this could be anything really)
cl_direct <- test_client(8474)
cl_direct()## But this does not work because I'm not running a webserver on port## 222222 and will throw an error##+ error = TRUE
test_client(22222)()## Next, we need a function for timing how long a roundtrip takes
time_roundtrip <-function(client){system.time(client(), gcFirst =FALSE)[["elapsed"]]}## So we can establish a benchmark for how long a roundtrip should take:
time_roundtrip(cl_direct)replicate(20, time_roundtrip(cl_direct))## ## Setting up a proxy## To route this through toxiproxy we create proxy that points at the## webserver port. I'm calling this "unreliable_webserver". We need## to specify the upstream port here, 8474. If not given then it will## set up a proxy on an available ephemeral port
tox <- toxiproxyr::toxiproxy_create("unreliable_webserver",8474)## The proxy object is an [`R6`](http://cran.r-project.org/package=R6)## object, which has a number of fields and methods that can be used.## The printed representation of the object gives details about what## it can do:
tox
## The read-only fields `listen`, `listen_port` and possibly## `listen_host` can be used to connect your client to the proxy.## Here, we create a test client that runs through `tox`, using the## randomly assigned port `r tox$listen_port`.
cl_proxy <- test_client(tox$listen_port)
cl_proxy()## With the proxy in place, but nothing on it, communication should be## approximately as fast as without the proxy:replicate(20, time_roundtrip(cl_direct))replicate(20, time_roundtrip(cl_proxy))## This is because there are no *toxics* on the proxy
tox$list()## ## Adding toxics## Next we need to attach some bad behaviour to this proxy; let's add## a bunch of latency (the argument here is the latency time in## milliseconds).
tox$add(toxiproxyr::latency(50))## The active toxics are listed here:
tox$list()## Now things are much slower, with elapsed times just a little over## 0.1s (which is what the latency was set at)replicate(20, time_roundtrip(cl_direct))replicate(20, time_roundtrip(cl_proxy))## Toxics can be removed from the proxy using `$clear()`;
tox$clear()## Multiple toxics can be added together; here bandwidth reducing## toxics have been added to both "upstream" and "downstream" data## movement, throttling the connection to 10 kilobytes per second,## enough to slow down even the simple data exchange here
tox$add(toxiproxyr::bandwidth(10,"upstream"),
toxiproxyr::bandwidth(10,"downstream"))replicate(20, time_roundtrip(cl_proxy))
tox$clear()## ## Temporarily enable toxics while running code## This pattern of add toxics/run code/remove toxics requires you to## keep track of more than necessary, so there is a `with`## method that simplifies it:
tox$with(toxiproxyr::latency(10),replicate(20, time_roundtrip(cl_proxy)))## If you want to pass multiple toxics into this you must group them## together into a `toxic_set`
slow_connection <-
toxiproxyr::toxic_set(toxiproxyr::bandwidth(1,"upstream"),
toxiproxyr::bandwidth(1,"downstream"))
slow_connection
## And then use this:
tox$with(slow_connection,replicate(20, time_roundtrip(cl_proxy)))## ## Simulating connection loss## Connection loss is simulated somewhat differently to toxics in## toxiproxy; there is no `drop_connection` toxic. Instead, you## disable the proxy which prevents all traffic over it (note that## this does not change any active toxics on the connection - it just## stops the connection).## The state of the connection is available with
tox$is_enabled()## it can be disabled with `$disable()` and re-enabled with## `$disable()`
tox$disable()
tox$is_enabled()## Now that the connection is down we cannot contact the server over## the proxied connection:##+ error = TRUE
cl_proxy()## though of course the webserver itself is still up:
cl_direct()## Bringing the connection back up:
tox$enable()
tox$is_enabled()## And we can use the client over the proxied connection again
cl_proxy()## There is an interface for running this temporarily, too, using## `with_down`:##+ error = TRUE
tox$is_enabled()
tox$with_down(cl_proxy())## After running this, the proxy has come back up automatically:
tox$is_enabled()## ## Proxying services on remote machines## The above examples used toxiproxy to simulate poor connections on a## service that was running on the same machine as both toxiproxy and## the client. However, toxiproxy will happily work with services## that are running on other machines (and toxiproxy can itself run on## a different machine).## To demonstrate this, here we make a MySQL connection to the## [ensembl](http://ensembl.org) database. They have a public MySQL## server running serving data from the project (see [this## page](http://www.ensembl.org/info/data/mysql.html) for more## details).## To make a plain (unproxied) connection, following the instructions## this is what we would do:
con <- DBI::dbConnect(RMySQL::MySQL(),
host ="ensembldb.ensembl.org",
port =3306,
password ="",
dbname ="homo_sapiens_core_83_38",
username ="anonymous")## and we can test the connection like so:head(DBI::dbListTables(con))## (there are 72 tables in this database). This connection created## here is not needed for anything below; it's just to show how one## would connect to this database.## To create a toxic proxy for this service, we can do
tox <- toxiproxyr::toxiproxy_create("slow_ensembl",
upstream ="ensembldb.ensembl.org:3306")
tox
## This creates a new proxy called `slow_ensembl` (the name being## there purely for labelling purposes) with the upstream location of## `ensembldb.ensembl.org:3306` which is the same as what was used to## create the direct connection. We can connect to the toxic proxy on## port `tox$listen_port`. Note that no authentication or SQL## specific things go in here; this is just going to shuttle traffic## between port `tox$listen_port` and `r tox$upstream`.## To create a MySQL connection over this toxic proxy, replace the## host and port details with those from the toxic proxy:
con <- DBI::dbConnect(RMySQL::MySQL(),
host = tox$listen_host,
port = tox$listen_port,
password ="",
dbname ="homo_sapiens_core_83_38",
username ="anonymous")## Other than the `host` and `port` arguments this is identical to the## direct connection created above.## With the connection created we can use it in any tests as if it was## a direct connection.head(DBI::dbListTables(con))## Using the toxic proxy then proceeds as above; we can add different## latency to the connection:system.time(DBI::dbListTables(con))[["elapsed"]]
tox$with(latency(1000),system.time(DBI::dbListTables(con))[["elapsed"]])## And we can see how the SQL driver behaves when the connection has## been lost:##+ error = TRUE
tox$with_down(DBI::dbListTables(con))## In this case, once the network connection has been lost, further## use of the connection object will not work:##+ error = TRUE
DBI::dbListTables(con)## even though the connection over the toxic proxy is re-enabled.
tox$is_enabled()## ## Using with continuous integration## toxiproxy/toxiproxyr are designed for testing, and so working well## with continuous integration ([travis](https://travis-ci.org/),## [appveyor](https://ci.appveyor.com/), etc is important).## This is fully documented in an example package## [toxiproxyr.test](https://github.com/richfitz/toxiproxyr.test#toxiproxyr.test),## but mostly consists of#### 1. Adding `toxiproxyr` to the `Suggests` field of your package## `DESCRIPTION`file.## 2. Adding a line to the `.Rbuildignore` to ignore some cached files## 3. Starting toxiproxy server on the CI by adding `Rscript -e 'toxiproxyr:::toxiproxy_start_ci(".toxiproxy")'` in the approproate place for your CI system## 4. Use toxiproxy in your tests## ## Other toxics## In addition to `latency` and `bandwidth`, used above:#### * `slow_close`; delay the TCP socket from closing until delay has## elapsed#### * `timeout`; Stops all data from getting through, and close the## connection after timeout#### * `slicer`; Slices TCP data up into small bits#### * `limit_data`; Closes connection when transmitted data exceeded## limit (only available with toxiproxy server 2.1.0 or later)