I was recently chatting with some coworkers about podcasts I listen to, so I thought I should document that list for easy sharing and to find some gems I am missing.

I have taken advantage of my commute time and turned my commute into Automobile University as talked about by Zig Ziglar. I heard this idea via some fitness blogs I was reading where the trainers were talking about ways to continuously improve, and decided I would apply that idea to my commute, walks, or even running errands.

The other thing I have started taking advantage of is the ability of podcast players to play at double speed. Most podcasts out there do well at one-and-a-half or double speed, and have heard that some players even support three-times speed. This allows you to dramatically increase your consumption rate if you can follow along at those speeds. You may not understand everything that is said, but you can always go back and re-listen to sections if needed, let it broaden your known unknowns, and at the least it should help to remove some of your unknown unknowns.

I did a listing of Software Development Podcasts previously, and am going to try and make this a yearly or bi-yearly update based off how frequently this list of podcasts change in my rotation.

Ruby Podcasts

Clojure Podcasts

Mostly λazy – Infrequent updates, but enjoyed the episodes that have been released

JavaScript Podcasts

JavaScript Jabber – Panel discussion on JavaScript topics, started by the host who started Ruby Rogues. The first episodes were hard to listen to due to some negativity, but have picked up listening to it again in the 50’s episode numbers, and working my way back as I get a chance.

Erlang Podcasts

Mostly Erlang – Panel discussion mostly about Erlang, but touches on related topics and other functional programming languages and how they relate to Erlang.

General

The Wide Teams Podcast – Hosted by one of the panelists of Ruby Rogues, with a focus on distributed software development, with the goal to find out the good and the bad experiences and help share information on how distributed teams work.

Software Engineering Radio – Recently I have only been finding a few shows on topics that seem interesting, but have a large backlog of shows with interesting topics.

On my list to check out

The Freelancers Show – Started by the same host of JavaScript Jabber and Ruby Rogues about freelancing. I would think the information would be relevant to full time employees even for working to build ones career.

If you have any other podcasts that are good please feel free to add your list of podcasts that I have left out to the comments.

**Updated 2013-10-24 7:54 CDT to include link to previous list of Software Development Podcasts**Updated 2013-10-24 22:13 CDT to include The Changelog, a “podcast covering what’s new and interesting in open source”**Updated 2013-10-24 22:28 CDT to include GitMinutes

At this point, this is sufficient for what I am needing, but have created an issue on the Github project to address large log files, and the ability to lazily read in the lines so the whole log file does not have to reside in memory.

The function parse-line, holds a regex, and does a match of each line against the pattern. It takes each part of the match and associates to a hash using the different parts of the log entry as a vector of the keywords that represent each part of the regex. This is done by reducing against an empty hash and taking the index of the part into match, the result of re-find.

Looking at this again a few days later, I went and created and issue to pull out the definition of pattern into a different definition, outside of the let, and even the parse-line function. I also want to go back and clean up the parsed-line from the let statement as it does not need to be declared inside the let, but can just pass the empty hash to the reduce. This was setup there before I refactored to a reduce, and was just associating keys one at a time to the index of matched as I was adding parts of the log entry.

Any comments on this are welcome, and I will be posting details on the other files soon as well.

I have just pushed a 0.1.0 version of a new project called Lumberjack. The goal is to be a library of functions to help parse and analyze log files in Clojure.

At work I have to occasionally pull down log files and do some visualization of log files from our Nginx webservers. I decided that this could be a useful project to play with to help me on my journey with Clojure and Open Source Software.

This library will read in a set of Nginx log files from a sequence, and parse them to a structure to be able to analyze them. It currently also provides functionality to be able to visualize the data as a set of time series graphs using Incanter, as that is currently the only graphing library I have seen so far.

A short future list of things I would like to be able to support that come to mind very quickly, and not at all comprehensive:

Update to support use of BufferedReader for very long log files so the whole file does not have to reside in memory before parsing, and take advantage of lazyness.

The ability to only construct records with a subset of the parsed data, such as request type, and timestamp.

The ability to parse log lines of different types, e.g. Apache, IIS or other formats

Additional graphs other than time series, e.g. bar graphs to show number of hits based off of IP Address.

Possibility of using futures, or another concurrency mechanism, to do some of the parsing and transformation of log lines into the data structures when working on large log files.

The above are just some of my thoughts on things that might fit well as updates to this as I start to use this more and flush out more use cases.

I would love comments on my code, and any other feedback that you may have. This is still early but I wanted to put something out there that might be of some use to others as well.

As the follow up to my post Running Clojure shell scripts in *nix enviornments, here is how I implemented an example using futures to parse lines read in from standard in as if the input was piped from a tail and writing out the result of parsing the line to standard out.

First due to wanting to run this a script from the command line I add this a the first line of the script:

!/usr/bin/env lein exec

As well, I will also be wanting to use the join function from the clojure.string namespace.

(use '[clojure.string :only (join)])

When dealing with futures I knew I would need an agent to adapt standard out.

(def out (agent *out*))

I also wanted to separate each line by a new line so I created a function writeln. The function takes a Java Writer and calls write and flush on each line passed in to the function:

The analyze-line function is just some sample code to return a string of the line and the frequencies of each character in the line passed in. The process-line function takes a line and calls send-off to the agent out for the function writeln with the results of calling the function analyze-line.

With all of these functions defined I now need to just loop continuously and process lines that are not empty, and call process-line for each line as a future.

I was recently trying to create a basic piece of Clojure code to play with “real-time” log file parsing by playing with futures. The longer term goal of the experiment is to be able to tail -f a log file pipe that into my Clojure log parser as input.

As I wasn’t sure exactly what I would need to be doing, I wanted an easy way to run some code quickly without having to rebuild the jars through Leiningen every time I wanted to try something, in a manner similar to the way I am thinking I will be using it if the experiment succeeds.

I created a file test_input with the following lines:

1 hello
2 test
3 abacus
4 qwerty
5 what
6 dvorak

With this in place, my goal was to be able to run something like cat test_file | parser_concept. After a bit of searching I found the lein-exec plugin for Leiningen, and after very minor setup I was able to start iterating with input piped in from elsewhere.

The first step was to open my profiles.clj file in my ~/.lein directory. I made sure lein-exec was specified in my user plugins as so:

{:user {:plugins [[lein-exec "0.2.1"]
;other plugins for lein
]}}

With this in place I just put the following line at the top of my script.clj file:

#!/usr/bin/env lein exec

I then changed the permissions of script.clj file to make it executable, I was able to run the following and have my code run against the input.

cat test_input | ./script.clj

I will be posting a follow up entry outlining my next step of experimenting with “processing” each line read in as a future.

The company I have been working for had to just go through a round of layoffs, due to not getting one of their main contracts renewed do to lack of funding by the state, and I have been included in that round.

My primary experience has been in C# on the Microsoft .Net framework for the past 10 years. Of those 10 years, 9 of them were working against the same product, and product suite allowing me to learn how some inconsequential decisions are not so inconsequential long term, and the value of care and commitment to a code base can be.

I am thinking that this would be a good opportunity to open myself up in trying to venture into a new language and toolset such as Ruby or Clojure. This is my open announcement for anybody who would like to poach a .Net developer if you are having problems finding developers fluent in your programming language.

Software Developer in the Dallas/Fort Worth Metroplex. Developed software using Java, Microsoft .NET stack, Ruby, Clojure and Erlang; but enjoy playing with and learning about different technologies to find different ways of doing things.
Host of the podcast Functional Geekery (http://www.functionalgeekery.com/) and founder of DFW Erlang user group (http://www.meetup.com/DFW-Erlang-User-Group/).