Erlang TDD hands on project – WorkerNet part 4

Continuing the job layer which had a whole lot of stories – we will now device and flesh out the test & implementation sets. A word of warning first, the following test looks simple by design, but it only a tip of the iceberg that begs for implementation. First, a small change to the create_file_at/1 function in wn_job_layer_tests.erl

Pretty tidy, small and easily understandable. Now off to implement the code needed for this design. First of all we need to have a clear picture of how to implement the internals of this. For me at least, a conceptual image always goes a long way (the previous image seen in part 3 was a rougher outline)

Seems good enough for now, the rough corners are there, what else, how would we want to iron out? Yeah, one thing that needs consideration as we broke the seal on it already (circled in green on step (3)). How should the directory structure be designed properly? We want something scalable and isolated for each isolated piece of data, this should do it

The green-circled text is the path on an example File stored in the file-layer on node n2@zen. Phew – that was a lot of “conceptual imaging” before we get to coding. First of, the new meat of the resource_process depicted in the first image (let’s call it Fig-1) ,

Previously the wn_resource_process.erl held the implementation of a minimal server, but the need for more logic made it easier to implement with standard OTP nuts and bolts. Next up the new meat of the wn_job_keeper.erl, previously just the minimal possible code, but now with more requirements on it.

The rationale behind this module is to keep the code clean, tidy and maintainable. Hiding the implementation details of the structure. Running the ‘full’ make target with this code in hand should now produce the following heartwarming output

We can see the output from the stream/2 in the console as well. Nice!
The seen implementation was the heaviest part and we have now fulfilled the requirements for

“The output generated by a job should be possible to see in realtime, through any node and stored to logfiles.”

This is possible with the stream/2 function as it will put the result to whatever iodevice we choose. File or screen. However, there is not yet total proof for

“Several jobs should be queued on the same resource type(s), being processed one and one in order as they where queued“

We need a method for retrieving the timestamps of each job, proving the order of execution. A test for this will be devised next time. Usually I always supply code listings at the end of each post, and this one would get really long if I did so, as you have already seen the code above.

Therefore, no such listing this time, if it is requested – I will post it immediately for the requested modules. There are som unseen changes to wn_file_layer.erl since it now uses the wn_util.erl of course, these changes also change wn_file_layer_tests.erl. I leave it as an exercise to figure it out.

3 Comments

Curtis Carter

Diagrams: OmniGraffle – a wickedly nice sketching and graphing tool for OS X
Highlighting: I am using Emacs and generally use the arjen color theme, on top of that, but for the blog posts I just take whatever is there and use htmlize to generate the html for the posts.

Thanks – I try to keep this biweekly, started out weekly – but the long days make the churning slower.

Curtis Carter

I’m working on a distributed/concurrent Erlang/Ruby app and the prototype is about done. I’ve had some challenges that I could not find documented anywhere and want to get them on a blog to save someone the frustration I had. Should get the blog started up in a couple of weeks. Will be at beardedcoder.com or blog.beardedcoder.com.

Keep up the good work. I believe the more Erlang oriented blogs the better for attracting new developers to the language.