Stuff I came across…

Main menu

Post navigation

This weekend, I came across an interesting article regarding building a static Bitbucket add-on comprising just HTML, CSS and JS. Generally, add-ons for Atlassian products are built using the connect framework which is based on REST APIs and web hooks. However, it’s also possible to build some add-ons that do not have any server component.

From the README file, it looks like the author hosted the example code on firebase. However, another way to host such add-ons is on Github. Github has had static file hosting support for some time. I used this approach for hosting my version of this example. The biggest convenience is that there is no separate deployment step required. Just pushing the code to Github updates the add-on!

Like this:

I spent the past weekend learning Go. A Tour of Go is a great resource to quickly come up to speed with Go. It’s a very small language (the spec is only around 50 pages!) and is really easy to pick up if you are already familiar with languages like C, JavaScript, Java or C#. I really liked the exercises that were scattered around in the tour. The ones that I found interesting were:

Share this:

Like this:

The WordPress.com stats helper monkeys prepared a 2015 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 10,000 times in 2015. If it were a concert at Sydney Opera House, it would take about 4 sold-out performances for that many people to see it.

Share this:

Like this:

Although “Big data” buzzword has been around for a couple of years, there’s still a lot of confusion regarding what exactly it is, how it can help businesses and what do people working on big-data really do. From Wikipedia:

Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate

Today, you can use freely available technologies to process and analyse really large sets of data and gather information from them. For example, some of the applications I worked on involved analysing network packets from all the routers of a large telco. Although the data is sampled, it is still huge – in the order of terabytes per day.

Given that you can process such amounts of data (fairly cheaply), as a business, the hardest question you need to answer is:

What do I want to know from my data?

Once you figure out what information you need, the next questions to answer are:

What algorithms & techniques can I use to get the information I need from my raw data? The people who can help you here are data scientists and statisticians.

What technologies can I use to implement and run my algorithms? Some of the popular technologies here are Hadoop, Hive, Spark, Elasticsearch, etc. You would expect the software engineers on your big data team to be comfortable with one or more of these.

Over the last couple of weeks, I spent some of my free time trying to run a basic Haskell+Docker web service. As it is with every new language or platform, I had to set up and understand a lot of things to get my app up and running. I’m sharing my experience here, hoping that it will help others who might be inclined to get into the Haskell world.

We’ll go through the following 4 steps:

Set up the development environment

Build and package a simple app using cabal

Run a basic Haskell web app using Scotty

Dockerize your scotty-webapp

As of this writing, I have less than 100 hours of Haskell development experience. Therefore, this is a pretty basic guide and focuses on the development tools rather than the language. It was possible for a newbie like me to build and run a basic Haskell web app using docker without much difficulty. Hopefully it’ll encourage more people to experiment with Haskell for their side projects!

1. Set up the development environment

You may skip this step if you already have ghc and cabal installed on your system.

Install the Haskell Platform (GHC)

GHC is the Glasgow Haskell Compiler package which provides you the compiler (ghc) as well as a REPL (ghci). Cabal is Haskell’s package manager and works similar to npm (NodeJS), sbt (Scala) and maven (Java).

% brew update
% brew install ghc cabal-install

Haskell REPL (ghci)

Next, we’ll start the Haskell REPL which is called ghci and get comfortable with a bit of Haskell syntax.

% ghci

Let’s define two simple functions that convert temperature values between celsius and fahrenheit:

Build an executable

runhaskell makes the development cycle easier when you are writing a Haskell application. But you can also create executable files using the ghc compiler. Let’s convert our script to a binary and then run it:

% ghc c2f.hs
[1 of 1] Compiling Main ( c2f.hs, c2f.o )
Linking c2f …

% ls
c2f c2f.hi c2f.hs c2f.o

% ./c2f
98.6

Load your source files into ghci

You can load external source files into ghci by using the :load or :l command. Let’s start a ghci session and load ourMain.hs file so that we can use the c2f function inside the REPL:

Now that our development environment is set up, we’ll move on to building a simple app using cabal which is Haskell’s package manager.

2. Build and package a simple app using cabal

Introducing Cabal

Cabal (Common Architecture for Building Applications and Libraries) is a system for building and packaging Haskell libraries and programs. Think of it as Haskell’s equivalent of Python’s pip, Node’s npm and Scala’s sbt.

Before we get started, make sure that you can cabal installed and on the system path:

% cabal — version
cabal-install version 1.22.6.0
using version 1.22.4.0 of the Cabal library

Create a cabal app

Cabal makes it easy to build, package and test your application. It’s also easy to add dependencies on other libraries that cabal will fetch for you.

It is possible that different Haskell apps have dependencies on different versions of the same library. To prevent conflicts in your global Haskell environment, it’s a good idea to create a sandbox for your app. Cabal will then install the dependent libraries inside the sandbox.

Now, we can initialise our app. The easiest way to create a cabal app is using the cabal init command. Follow the prompts and cabal will generate the config file for you. Here’s an example interaction on my system:

You will now have a cabal-example.cabal file in your source directory. Remember that we selected Main.hs as our main module during the cabal init process. Add a Main.hs file now to the root of your source folder with the following content:

Next…

3. Run a basic Haskell web app using Scotty

There are many web frameworks available for Haskell: Yesod, Snap, Scotty, etc. I chose Scotty over others as it seemed easier to get started with.

We’ll write a simple web service that responds to various HTTP request types (GET, PUT, POST, DELETE). We’ll see how to get request headers, path parameters and form fields and how to respond with plain-text, html or JSON response.

Handling more complex requests

Let’s add a few more handlers to handle different kinds of requests. Add the following to server/Main.hs:

Encode/Decode JSON

Most web services these days interact via JSON. Haskell provides a type safe way to encode/decode JSON strings using theAeson library.

Defining the model

Let’s create an Article data type that could represent a news article for example. An article consists of 3 fields: anInteger id, a Text title and a Text bodyText. By making Article an instance of FromJSON andToJSON typeclasses, we can use Aeson library for converting between JSON strings and Article objects. Add the following code to the file server/Article.hs:

We’ll need to add a couple of routes to our Scotty router function to handle encoding and decoding Article types:

We’ll also need to add a couple of dependencies to scotty-webapp-example.cabal file:

Like this:

My experience with Intellij for downloading sources for the dependencies hasn’t been good. It frequently fails to download the sources. I’ve tried a few (googled) suggestions but they haven’t helped. However, there’s an alternative way using the command line. The downside to this is that it will download the sources for all dependencies. But, I don’t really mind that till I start running out of space on my laptop!