Recent Posts

Meta

Let’s dip our toes in to computational English by exploring the classic algorithm, Soundex. Soundex was created in 1918 by Robert Russel and Margaret Odel King, but most people know it from work by Donald Knuth.

Soundex indexes words based on their ’sound’. It is one way to checking spelling based on similar sounds, so we should get similar scores for ‘been’ and ‘bean’ and, of course, ‘their’, ‘there’, and ‘they’re’.

Is it useful?

Probably not. There are newer, better phonetic algorithms like Metaphone and modern spell checkers are interested in keyboard distances between words. So it’s just for funsies. And that seems like a good enough reason for me!

Understanding the Algorithm

There is an interesting part of the wikipedia article where the editors re-order the steps to simplify some of the duplication detection. Great! Let’s use THAT version, and not necessarily the National Archives and Records Administration’s General Information Leaflet 55, “Using the Census Soundex”.

Save the first letter, remove all occurrences of ‘h’ and ‘w’ except the first.

Replace all consonants (including the first letter) with digits

bfpv 1

cgjkqsxz 2

dt 3

l 4

mn 5

r 6

Replace all adjacent same digits with one digit

Remove all vowels (including y), except the first letter

If the first symbol is a digit, replace with the letter

Pad with zeros or trim so it only has a letter and three digits

Some Intuition and Test Cases

Grate and great should be the same. Same and sim. In fact most homophones, like homafoen, should be the same.

The wikipedia article gives us some example values.

Robert ‘R163’

Rupert ‘R150’

Ashcroft ‘A261’

Tymczak ’T522’

Pfister ‘P236’

We can include a couple of hand picked cases, like ‘the’ and ‘a’ which are all zeros.

But the Implementation!?

The first function pretty much describes the algorithm using the steps I listed above. I use types to help guide what comes next and focus on each piece as I can. I made the initial implementations of each step as dumb as possible, mostly passing from one stage to the next. As I filled out each step, I could see what was happening to my examples. I could see the error cases coalesce toward correctness. The output words show missing letters. Then numeric substitutions. And duplicates dropped. Then vowels go missing. Until it finally worked!

Reductio ad absurdum

Soundex is for matching things that ‘sound like’ other things. Like when my five year old mishears a Taylor Swift lyric to hilarious results. Or when the well-intentioned hard worker on the other side of the counter has to listen over the bustle of other customers at busy shop while standing next to HVAC and loud brewing equipment to hear you speak in your best indoor voice and has to make a best guess as to what your name might be.

Give my Starbucks Name Generator a try, but you need to keep a few things in mind. It takes a while (10 – 20 seconds) to load all the words in to a Soundex lookup before you see anything. I could do this preloading in a web worker, but that would take me longer because the Elm doesn’t yet have a robust backgrounding story and the easy path would be to rewrite the algorithm in JavaScript, and that would defeat the point of using Elm. That said, after the loading, it’s pretty fast! Also, feel free to check out the elm-soundex code on GitHub.

It’s all about time. SEP, like most organizations, lives and dies by time sheets. Our time sheets are two weeks long. Each entry has a start time, a project code, a description, and a duration. Easy enough, but what does that have to do with EmberJS?

To Quote Miss Jackson, “I Want to be the One in Control”.

Each timesheet is an array of such entries. If we use an array controller to wrap the array model, we can leverage a lot of assumptions about controlling a number of items.

OK, Computed Properties

Controllers can compute properties, such as the total duration of time logged on a time sheet. In addition, we can validate the timesheet, checking for overlaps and showing the billable time.

If you want to show anything on your page, you’re going to have to write a template. A template is the html to display on each page view. And if you use resources, they nest! Ember, by default, uses the Handlebars template engine.

Lunchtime!

It’s a Friday and I want ribs for lunch. But I don’t want to go alone. I’ll just write an intranet app that lets me invite a few coworkers to lunch. Because email is hard.

Begin at the beginning.

The default template is the application template. you can tell it’s the application because it doesn’t have a name. The application template is used to lay out the main page structure and all the site branding. all other templates are nested inside and show up where the {{outlet}} tag is used.

Okay, lets start off by managing some expectations. I’ll cover Controllers next week, but I’ll be mentioning them often. About 1/2 of the cool things about templates have some counterpart in the controller supporting them. That said, I’m getting hungry. Let’s gather up the team and head out for Barbecue!

I Decide who Leaves and who Dines!

I need to display a list of my coworkers so I can pick a couple of them.

A few things to note in the above example. I’m using an {{input}} block to make a text field that binds to the controller property lunchType. That will be used to let everyone know what kind of food we’re going to eat! Next, I’m looping over the a list of employees using an {{#each}} block. The collection comes from another controller property. Each element is then scoped as ‘employee’ inside the each block. I can conditionally skip displaying an employee if they are not available. more correctly, I can use an {{#if}} block to show only the available employees. I display a checkbox using a built in helper, {{input}}. It binds the checked state of the input control to the isSelected property of the employee. The last three are simple data bindings to text. They just forward properties from the model to html.

The Web Without Links isn’t the True Web.

Let’s tweak the employee listing. I want to make the name link to their info page. I can use the {{link-to}} block.

But we need to DO something!

Wonderful! Things are moving right along. Now, how do I submit my selections? With an action! An action is a function in the controller. I promise, we will get to that later. Just trust me when I say our bindings put all the information in the model, so we don’t need to pass anything to the function. We just need to bind the action to a button, right after the end of our {{#each}} block.

We can pass parameters in to the function, but we don’t need to at this time. In addition, we don’t have to worry about controlling event propagation or target a different controller. All of these are possible, just not necessary for us.

Your Bill Comes with Mints

There are a few things we can do to help us clean up our code from here.

Helper isn’t just for Hamburgers

A helper is a function that gets called like a block. It returns text to be displayed in the Html. If you need to format dates, for example, or want to display an employee’s spark-line for their time on the tread mill in the past two weeks.

Partials

Partials are templates that can be reused. They bind to what ever model or controller is in the context, just incase your data doesn’t look like you expected because you named something slightly wrong.

Views

A view can pick it’s own template. It has a bit more logic, but I’ wouldn’t go crazy building complex controls with them. In a couple weeks, we’ll cover Components and then we’ll go nuts.

That’s probably the wrong choice, but I made it for Pedagogical reasons. I regret nothing, but we are going to play with the routes of an example application to think through some design decisions and pick up a small detail of the router I overlooked. Resources.

A Picture is worth 1000 words

I’m a Father, Husband, etc. That means we are generating Gigabytes of pictures. Some are even in focus. She’s two, and fast. So, we are going to build a photo sharing site! Let’s give it a classy name. I’m thinking Phlickr. Um, legal informs me that’s a bit too similar some other upstart.

Introducing PhotoAlbum

PhotoAlbum is the latest in picture sharing technology. Some of our neat features are:

What’s the Difference?

Routes are for individual pages. Routes cannot be nested. If we want nested pages, we use resources. And resources give us a few new defaults.

Defaults

By default, the router gives us the route named ‘index’. And that in turn builds the controller, route, and loads the template named ‘index’. A resource is like a mini router in that way. If we name a resource ‘posters’, then all the child routes are prefixed with the name, including a new default ‘index’. The tetrad is listed out below. When deeply nested, only the resource and route matter for naming as opposed to the parent resources. Example, the template for the new albums is ‘albumsNew’ despite the albums resource being a child of the user resource.

RouteName – postersIndex

Controller – PostersIndexController

Route – PostersIndexRoute

template – postersIndex

Chains

The defaults are not the only things. They nest, just like applications. The posters resource looks for a template named ‘posters’ and then the child template. While I won’t dwell too much templates now, remember when the application template had an {{outlet}}? Well, the ‘posters’ template can have an {{outlet}} as well. this allow the templates to nest.

But it’s not just the templates, the controllers nest as well. So the model and properties of a parent controller can be accessed by a child. And the routes nest. Each parent finishing before the child runs.

A Time to Choose

Of course we can use routes to describe everything here. I’m fairly sure. But we will have all kinds of issues with reuse. the nesting of controllers and routes gives us a sensible place to hang some code reuse. The really interesting thing is I chose my routes and resources strictly based on what Urls made sense, what parts belong to other pieces of information. For example, there is only one dashboard, yours. The login is also independent of other data. However, each photo album belongs to a user. Matt’s album Dinners I Made is totally different from my album Dinners I Made. And while a picture has a tag, the collection of tags is only useful when it belongs to the system as a whole, allowing me to find all the pictures tagged sunrise regardless of who posted it. Once again, it all comes down to following the Urls and letting them tell you all you need to know.

Where do we go from here? We begin with the Url. The history of the web is based around the URL and our ability to transition from one url to another. Ember is built around the Url.

You are a Web Developer if you build apps with Urls

– Tom Dale

Every view we want to represent in our application will be represented as a different Url. When we want to show a specific piece of data, we use a different Url. Want to modify how we are showing the screen, then we add data to the Ulr. This will allow us to Deep Link and share specific views and app state with colleagues and coworkers. This is how the Web has worked since the web began. And Below, we see how the Url communicates with Ember to make choices on what to show the browser.

Ember’s Url Structure taken from the presentation mentioned above.

The Router

The router dissects the Url and chooses which Route, Controller, Model, and Template to compose, execute, and use the results for displaying to the screen. By default, you get an ‘index’ route for free. As if I went to http://example.com/, that would be the default index.

Baby-steps to Customization

I can specify just a route named ‘about’ and that will show an ‘about’ template that will use the path http://example.com/about without specifying that path.

App.Router.map(function(){
this.route("about");
});

A Route With a Different Name, Would It Navigate as Sweet?

Lets say I have a dumb legacy Url (http://example.com/meet-the-team) I need to keep working. Because Urls are important. But I want my internal code to use clear names (about). I can specify the path used in the route mapping.

My Pokémans, let me show you them.

One more mapping to cover for now, before we start talking about the magic a route actually does. What if we want to show the same page, but vary the data based on part of the path? I have a super awesome collection of Pokémans, let me show you my favorite Squirtle. I keep him at http://www.example.com/pokedex/poke_1. I can specify part of the path to be a dynamic segment. The name of the dynamic segment will show up as a param in the Route code.

What’s a lifecycle?

There are so many hooks in the route that I can override. I can override the controller creation. I can specify a non-standard template. Oh, that might be fun! This is the BEST pokemans, so he needs a custom template.

Odds and Ends

If your code is slow, because for example not tied to anything real and not to name names or anything you query exchange, you can create a template called ‘loading’. Or you an overload the loading behavior.

The same thing with handling errors.

Customizing the Base

You may notice that the Url ember gives you has a hash (#). You can pick the location strategy as one of the following three. If you don’t like the hash, you can use the ‘location’. If you’re running an app out of a page, you can use ‘none’. The ‘hash’ strategy is the default.

App.Router.reopen({
location: 'history'
});

Transition Overrides

When you edit the Pokedex, you don’t want to update the data until the Pokemaster is finished with her entry and presses the submit button. But what if she clicks on a link to another page? We can intercept the transition and quit!

You Are Here

The router is all about defining the transitions between your views. In all honesty, the next time I write a desktop application, I may have to write something like this. It’s super easy to manage and override just about any transition. It’s useful, flexible, and the heart and soul of your application.

Rise of the Tetrad

Ember uses naming conventions to tie together the four moving parts of an ember application; the Route Name, the Route, the Controller, and the Template.

The Route Name

When the route is defined like I did below, it is given a name. By default, the path mapped to the route is the same as the route. For example, the route below defines a route named “level1” and maps the url “/level1” to the route of the same name.

Route.map(function(){
this.route("level1");
...
});

The Route

By default we are generating a route named App.Level1Route. If we want to change the behaviors of the route, which we will cover in more depth in a different post, we need to define our Route and provide it with some overrides. A common override is to provide some model data, like below

The Controller

Again by default, we have a controller called App.Level1Controller created for us. But if we want to override its behavior, we just make a new one with the correct name! For example, we can add an action to save the current state back to the server.

Numbers are tricky. You said there were six, told us about four, but it looks more like only three.

Right. The Route, Controller, and Template are all based off of the Route’s Name. So, four for the price of three. The Other Fourth is the model. I’m just going to skip it for now. The three Ember apps I have written have not yet needed the complexity of an Ember Data model.

The other pieces you will come across are the Components, the views, and Router.

The Router

The router is the part that wires all the components together. We used it to set up our route mapping.

Views

Views let you add behavior to an html tag. You can add behaviors and event handlers to the tag.

Components

Components are an implementation of the W3C Custom Element Spec. They are far more functional than overriding an html tag, they are custom tags backed by a template and code that allow you to build custom controls. If you need a calendar control, start here. Need a tabbed page control? Check out Ember-Components.com for some examples.

Next Steps

Now that we know our way around the major concepts of the Ember framework, we can start to understand them one at a time. Maybe next time we will start with the routes.

There was a recent blog post by Rob Conery about learning EmberJs by just flinging yourself at it. That was something that kind of resonated with me. I’ve been trying to understand EmberJS for a while, but I wasn’t getting it. I wasn’t feeling confident enough to actually start a project and get it right. You can never learn without actually doing and Rob reminded me of this.

So I tried it. I flung myself in to the void of two projects.

Define the Problems

I chose two projects.

Project one is a rewrite of a comic strip site I wrote using JavaScript and HTML in about 2002. Considering I’ve not written anything for the comic itself in almost 10 years, It’s not important, but it is a healthy amount of data to display.

Project two is much smaller and uses an ASP.NET web service to wrap the Microsoft Exchange APIs. It simply asks what is currently happening with the different conference rooms at SEP.

Layout the path

I wrote both of the sites using mostly the same path. I wrote them at about the same time, so this is only logical. I accepted as fact that I would be rewriting functionality and moving code between responsibilities as I got more familiar with Ember’s various pieces.

Wonder Twins Routes and Templates (via URLs)

With routes and templates, a large amount of the work can be done. I can lay out a number of screens and how they transition. I can specify the Urls for each page.

Wait, there’s data?

If we want to add a touch of data to a page, we can change the route. Let’s look at the cast page. We don’t alter how we define the route mapping.

this.route("cast", {path: "/cast"});

Then we can override the default router by adding a touch of code. Here, we have a static data structure defining our cast members. I then override the default router for the Cast route and define a model attribute.

But in all honestly, that takes forEVER because exchange is SLOW. So I just define a ‘loading’ template and ember makes it all better because of it’s internal use of promises to handle the async call to my server. Ember automatically shows this while waiting for my data, then shows the page with actual data.

Is That It?

Nope. That’s not it. Granted, I’ve gotten a bunch of functionality out of those two parts of Ember; Routes and Templates. I’m now picked up on my bootstraps and I can pick my next battles based on my needs. I’ll sum up the learning path I have left for me using these two apps, and more importantly why I need to learn these things.

Components – I’ve used these to reduce the duplication in my templates. They can be passed data so I use them to define how I render an Exchange meeting or for the navigation bar for my comics site.

Controllers – I’m currently using them to handle some button triggered actions on the Exchange site. they can also be used to compute properties based on the model values or add two way binging to things like text input fields.

Models – I don’t know if my current projects have a need for Ember.Data, but I’m going to use it to represent the comic site’s data using fixtures. This will, hopefully, give me the experience to start with a larger data driven application.

Testing – When learning a framework, NEVER worry about automated programatic testing until you have a decent understanding of how things are working. THEN add tests. THEN start considering adding new features test-first. Ember has a fairly decent testing story that I want to start exploring. The more work I do in JavaScript, the more I’m going to need to test. While not needed for sites I have here, thin wrappers around data or APIs, I will be writing some sites with far more complex logic.

There is more EmberJs to learn, but I don’t need it yet. But when I do, I’m confident the documentation will be clear. And so far, the code has bee easy to use. This is the first time I’ve enjoyed web development in several years. I think I’m ready to take back the web.

I’ve been working with our corporate website for the past few days. I’m trying to get analytics for some A/B split testing. As a part of that, I got to play with some really neat FSharp features. Type providers have allowed me to use a snippet of json to build a type hierarchy without doing the heavy lifting myself. And recursive sequences allow me to consume a paginated webservice as though it was a single data set.

Bringing My Customers Together

The data I’m currently interested in is, as far as I can discover, only available by asking each Hubspot contact for their list of form submissions so I can count the number of submissions to each form. But I can only ask for 100 customers at a time. The response will tell me if I can fetch more and the vid-offset I can use to get to the right spot in the pagination.

What’s happening? The seq structure creates a seq computation (think IEnumerable) and allows us to write code that generates a seq. The yield! keywork takes a collection and doles each element out in turn. If we find the collection has more, we yield the results of a call with the offset. This means our first call will use the offset “”.

Types are your friends

FSharp type providers are wonderful. They can, at compile time, parse out some code and generate a type tree. In particular, using the FSharp.Data library I can give it a snippet of JSON and get back a strongly typed parser.

I’ve been playing with Clojure recently and loving it. However, most of the stuff I’ve been doing with it recently has been Exercism.io, and not actual application code. To rectify that, I pounded out (slammed my head against a few problems) a simple application (sure, now that it’s written) to get my secret feed from NSScreencast and download the videos.

Two Great Houses

Reading a feed and downloading movie files is very simple on the surface. There are two things complicating the issue, but they are two very important parts. First, feeds are stored as RSS, and XML is best treated as a tree data structure, not as text to Regex over. Second, NSScreencast uses a CDN, but that means the video URLs return a redirect.

The Forest of Xml

The Clojure community has taken a shining to using Zippers for traversing large trees of data. Xml is just a tree of data. Once you parse the damn thing. Fortunately, Clojure has a few libraries for handling this; Clojure.xml, Clojure.zip, and Clojure.data.zip.xml.

Once I have the zipper for the feed, I need to traverse the feed’s tree. What I want eventually is a list of file names and urls for that file. the file name will come from the feed, but I need to find the url from traversing all entry tags, their child link tags, but only the link tags with a rel="enclosure". But honestly, I only care about the href attribute.

Once we have this, we need to find the file name. Anything between the last / and the only ? is the file name. The way clojure returns Regex matches, however, is a match followed by the group. So, if we spread out the regex to match the whole Uri, we have the two peices we need for further processing. Well, we need to map over first before doing anything because of the way re-seq returns data.

Looking in the wrong place

While Clojure’s input-stream is capable of handling a uri, it has some trickyness with SSL urls and more importantly, the redirect to the CDN is handed back as a document I have to parse by hand. Fortunately, clj-http.client handles all this for us. We’ll require the lib as client. To copy a file from a uri to the disk, we use our copy-file routine.

the function client/get returns a map of data, and we only care about the :body. We ask for the body to be returned as a byte-array with {:as :byte-array}. with-open is used to close the output-stream once we’re done. And since we are using this to map over a vector of uri and file, we will destructure the vector in the argument list.

Fleshing out the rest

The main algorithm of the downloader is to get a list of link-file-pairs, filter the videos I’ve already downloaded, and copy all the files. I tag a doall to ensure any lazy seqs actually execute.

Determining if a file is downloaded is fairly simple. Figure out the full path of the expected file, make it a file object, and ask if it exists. But I want the inverse of exists, so we’ll tag a not on the end.

What’s in a name?

I’ve already downloaded some files, so I wanted to make sure they had the same naming scheme. Also, the fullname function knows where on my machine the files are supposed to go. The details arent interesting as the prefix makes life more difficult than it needs to be.

About Brian Ball

I'm a Software Engineer in Indianapolis. I'm interested in programming languages. I love dynamic languages like Ruby and Perl. I'm exploring functional languages like Haskell, F#, Clojure, and Erlang. I also use the .NET framework and occasionally deal with System Testing.

If you're not careful, you might also see some sketches and other art come up every once in a while.