Introduction

So this is a part 2 of a series of posts I am planning to write to create a very basic distributed system in Go. I am adding new features as I keep exploring the topic and as I read new papers on the subject. The posts are sometimes slow just because I am not putting a lot of dedicated time on it yet but this may improve over time 🙂

What are we building

In the part 1 of the series we built a simple server that can communicate with other nodes in the cluster over TCP. The first node starts as a master and other nodes connect to the existing node and communicate with each other. This part we will expose

a minimal HTTP endpoint that can be used to query the cluster and serve json responses

a minimal client that can send/receive json messages to the http endpoint

How are we planning to use this in future

Well the simple plan I have in mind now is that all the server nodes communicate to other nodes over the TCP and any client/web-interface communication would happen over the HTTP. Note: we could had used either of them to handle all the communications as well, I am just breaking this down to explore both the communication mechanisms.

Getting to the code

Messages

Messages are the useful structs we will be using to exchange all out future json messages. We will keep on improvising these as we keep using them.

Http Server

This is the main server code that exposes multiple endpoints and receives requests and serves appropriate response back to the client. We currently create two endpoints here.

Client

This is a minimal client that listens continously for messages on the console and sends the messages to the server every time we hit a newline. The messages are send as Json and prints the received message back on console.

Testing the Http server

We can use chrome extension Postman which is really popular for testing web-api’s. We will query our 2 http endpoints and check the response back.

The top part of the screenshots show the request made and the bottom part shows the response received back-

1. localhost:9001 : This is a hello world endpoint that is supposed to get the details of the node connected or, can be used to fetch the health status of our cluster. For now it just displays nothing in the node info.

2. localhost:9001/query : This endpoint expects a json message with a user query and sends back a message back to the client.

Note: in the above is the body of the request where we changed type to JSON and added the message in body.

Hope this really small post was helpful. Will keep on adding more posts as the client matures. You can find all the code I have come up with here.

Nice initiative and kudos on the posts! Thanks for demonstrating what you’ve done thus far. It’s always nice to see more and more people venture down Go as a distributed systems language; it has enormous potential.

I’m just wondering on how much more work you’ve done on this since last year? The reason I ask is because I’m super interested in attempting to create my own Go-lang clustering framework for a distributed system I’m working with, however I wanted to see what was available in the Open Source community before I went down this path. It would be really interesting to find out what your design thoughts are on how to make this scalable, open and easy to use for generic software development in Go.

Hi Barry, This post was part of fun learning task. I am not a Go expert but I do see a lot of frameworks on Github written in Go. Contributing to some matured framework would be a great idea instead of creating one from scratch.
Cheers.