services

Review: In the last blog entry I went through more than a few examples of using cURL to issue GET requests against various end points using Node.js & Restify. I also covered the basics on where to go to find cURL in case it isn’t installed. The last part I covered was a little bit of WebStorm info to boot. In this part of the series I’m now going to dive into the HTTP verbs beyond GET.

POST

The practice around issuing a command via http verb to save data is via a post. When you issue a post via cURL use the -X followed by POST to designate a post verb, then -H to assign the content type parameter. In this particular example I’ve set it to application/json since my payload of data will be JSON format. Then add the final data with a -d option, followed by the actual data.

Other data types can be sent, which the content type can be appropriately set for including; html, json, script, text or html. One example of this same command, issued with jQuery on the client side would actually look like this.

When building post end points via express one of the things you may run into is the following message being displayed in the console.

/usr/local/bin/node app.js
connect.multipart() will be removed in connect 3.0
visit https://github.com/senchalabs/connect/wiki/Connect-3.0 for alternatives
connect.limit() will be removed in connect 3.0

The immediate fix for this, until the changes are made (which may or may not mean to just alwasy is to replace this line

So often I end up putting together some RESTful services (or the intent is to at least build them with that premise, but we all know how that ends up). The API URIs routing gets put together and one wants to take a crack at the service as soon as possible. Here’s a quick guide for using cURL to take some basic actions against the services and understand what you’re getting back.

The first thing to do is make sure you can run JavaScript, which means you have a computer. The second thing is to get cURL, which means you’re running some variant of Linux or UNIX. In most scenarios one would be running OS-X. The easiest way to determine if it is installed on your computer just open up a terminal and type ‘curl –help’. You should get a result with all the switches, which is almost always a bit of overload.

Don’t get intimidated! It goes on and on and on, but just know it’s installed if you see all these goodies. If you don’t get the results above, then installing cURL is the next step. I’ll leave that to you. Here’s some links to download and get started however.

Next you’ll of course need Node.js and Restify installed. I’ll assume you have Node.js installed. Create a directory and in that directory just run the following command.

npm install restify

Next create a file called server.js in that directory you’ve just installed restify in. Here’s the initial JavaScript code for that file that I’ve used to put together for the first few examples of using cURL.

Ok, now to run this with node.js just issue the command to launch node.js with this file that was just created.

node server.js
restify listening at http://0.0.0.0:8080

Getting Get

Now the service is running on port 8080 against 0.0.0.0. To check out what a standard GET verb will do in a browser, open up a browser and navigate to http://0.0.0.0:8080.

Browsing the GET response via Chrome.

You’ll see this in the browser window. Just straight plain text too. If you look at source, this is all you get back. Now open up a terminal and run the following cURL command to execute a GET against the URI & port. This is the most basic cURL command one can make. It is simply issuing a GET request against the URI and will display the body of the response.

curl 0.0.0.0:8080

The response will be similar to this for the particular request.

{"code":"ResourceNotFound","message":"/ does not exist"}

Your terminal will probably stick the subsequent prompt at the end of the result too, because the result doesn’t end in a newline. Beware of that, your prompt hasn’t disappeared. ;)

To get a little more information you can get the header of the response dumped into the terminal with a -i. The -i option stands for –include, to include the header. Issue the command as either line shown below.

curl -i http://0.0.0.0:8080
curl --include http://0.0.0.0:8080

The response will be provide a little bit more about what is going on.

With this response the actual response error code number is shown. In this case we have a 404, which points us to the problem with this curl request. The server isn’t returning anything to our curl request. If we look at the code, we can see that the ‘get’ route is setup as ‘/hello/:name’ which means that the domain root is only looking at http://url_root/hello/someName for a request to be made in order to return a response.

Here the content is returned as “hello Adron” and the header returns a 200. The content type is application/json format with the length returned as 13. Note also the connection is set to keep-alive. Let’s dive into that.

If we change the connection type, which is important for many scenarios, we have to send extra header information to ask for the response to be returned accordingly. In order to do that we can pass the -H or –header option in with the curl request. If the command is issued with an -i and -H as shown below the result will be as follows.

If we take away the -i we’ll just get the response, which is “hello Adron” and wouldn’t get the header, which now returns Connection: close in the response. By default, curl sets the connection as keep-alive, but in order to make the request return right away the connection needs to be issued a request for it to close. By setting the -H or –header value of connection to close, we get the response immediately. With restify, it is also important to note that it checks if the user agent is curl.

If it is curl the connection header to close and removes the content-length header. However I’ve experienced that restify is not doing this in all circumstances or that the use of curl is being changed in some of my usage. So don’t always assume that this will be the case. The safest bet is to set the connection closed when done. Thus, adding -H or –header and setting connection to close with a “Connection: close”.

Beyond Basic Get

Ok, so that’s a pretty solid use of GET with cURL. Let’s dive into some puts and deletes with a get or two thrown in for comparison. Change the executing code to the code shown in the server.js file below.

This function is setup to take req, res, and then handle next. The req is the request, the res is the response and the next is for issuing to return and continue with the result. The next bit of code starts the server with the restify.createServer();. Just below that there are several handlers that are setup.

Now at this point I got a little sidetracked writing this blog entry. But I thought to myself, “hell, I’m just figuring out some parts of Webstorm, I ought to blog a little about it!” So, here’s…

A Little Webstorm Love

Webstorm and cURL. Click the image for a full size image.

Before continuing on I wanted to cover a few tidbits of the Jetbrains Webstorm IDE. I often switch back and forth between the Sublime/Terminal combo and the Webstorm IDE. The really cool thing about this IDE is that it actually has a Terminal built in, color coding and autocomplete of the code, refactoring, and file and folder viewer and a whole slew of other features. In the image above that I’ve included there are four neon pointers that are displaying some of the key functionality that I’m using to work through this blog entry with cURL and Restify.

The arrows, from left to right are pointing to the following IDE elements. The first is pointing to the javascript files storgie.js and starter.js which I added specifically to show the git status colors. Each color reflect if the file is new (green), has changes (light blue) or is committed with no changes (white). The second arrow is just pointing to the general folder structure. Here you can see the hidden .* files like the .gitignore and .npmignore and also easy to dig through the node_modules directory. Webstorm also uses the node_modules directory to provide extra information and autocomplete to the code as you work through your coding session. The next arrow is pointing out the terminal in the editor, which is where I’m working up the curl examples in this blog entry. Then of course the color coded starter.js file that is one of the working examples. Webstorm, simply, is pretty sweet. I’m looking to do some more walk throughs and work sessions with the editor in the near future. So if interested, be sure to keep reading and subscribe, I’ll be sure to post any links to wherever the material ends up right here.

Now, back to the cURLing. ;)

After I toyed around with Webstorm and bit to get it work in a way that was efficient for me to use it for developing these APIs I stumbled into an idea. I’d provide a page for the APIs that could be located at the root of the API service such as http://api.blagh.com. The APIs would still be a restful type schema like http://api.blagh.com/thing/create or http://api.blagh.com/thing/destroy but at the very root would be a kind of docs. Maybe this could just be a status page even. Whatever the case, there needs to be something at http://api.blagh.com so I decided right then and there I’d switch to express.js to build the rest of the API services. Restify is fine and all but for this, it seemed like express would have all of the pieces I need for this.

Just to boot, I then read a few articles about express being faster such as this one. But then I read this issue on github and almost thought, “maybe I should keep using restify” but then I thought, “dammit, just get it done the way you want it built” so it was back to express. It’s easy enough to change this later so I just got back to coding, albeit with express now. So keep reading and in the next day or two I’ll have part two of this series on using cURL to hack at your APIs.

Back in December Uhuru Software and Tier 3 released two different forks of Cloud Foundry that enabled .NET Support. I wasn’t sure which I wanted to use, since I had some serious Cloud Foundry work I was about to dive into, so I’ve picked them apart to determine how each works. This is what I’ve found so far.

That covers the basic links to the downloads, community, and other points of presence, now it is time to dig into some of the differences I’ve found. First though, I got a good environment setup to test each of the forks, from within the same Cloud Foundry Environment! So this is how I’ve set this up… Setting up the Virtual Machines w/ VMware Fusion I suspect, you could tangibly do this with some other virtualization software, but VMware is probably the easiest to use and setup on OS-X & Windows. I haven’t tried this on Linux so there’s another space I’d have to give it a go. Using ESX I also suspect this would also be extremely easy to setup. It’s up to you, but I’m doing all of this with VMware Fusion. The environment I’m using for this comparison consists of the following virtual images:

Iron Foundry Instances

Uhuru Instances

Setting up Some Controllers

So the first thing I did was dive into setting up a controller, or actually two, because I wanted to have an Iron Foundry Environment and a Uhuru Software Environment. After that I’d then try to mix and match them and figure out differences or conflicts. The instructions listed under the “Uhuru Instances” has information regarding setup of a controller for the Uhuru Software Environment, which is what I followed. It is also a good idea to get setup with Putty or ready with SSH for usage of Cloud Foundry, Uhuru Software, and Iron Foundry.

I’ve been working through an architecture scenario recently. This is what I have so far. Multiple external web services, some SOAP and some REST, and some data sources in a SQL Server Database, Azure Table Storage, and flat files of some sort. All of these sources need to be accessed by a web site for read-only display. In the diagram below I’ve drawn out the primary three points of reference.

The services that are external; Contract, Table Store, Document, Search, and Help Desk Services.

The Website Web Services Facade, which would be an aggregated layer that then provides the various services via an internally controlled services layer.

On top of that will be the web site, accessing the services from the aggregated layer with jQuery.

Basic Three Tiers

After creating this to get some basic idea of how these things should fit together, I moved on to elaborate on the web services aggregation layer. What I’ve sketched in this diagram is the correlation to architectural elements and the physical environments they would prospectively be deployed to. Again, broken out by the three tiers as shown above.

The cache is intended to use SQL Server, thus the red call out to the physical SQL Server cluster.

The last tier, which isn’t being developed, but just providing data is the External Services, primarily shown to provide a full picture of all the layers.

Aggregate Web Services

I primarily drew up these diagrams for discussion of the architecture, poke holes in it, or otherwise. Which speaking of, if any readers have input, question, or are curious please type up a comment and I’ll answer it ASAP.

As the effort continues there are some other great how-to write ups I will be putting together. Everything from unit testing, mocking (with moq), how to setup test services, test services, and other elements of the project. I’ll have all this coming, so keep reading & let me know what you think of the design so far, subscribe via e-mail (look to the metadata section below), or grab the RSS for the blog (see below also).