Content

Jessica & I teach a lot of beginner programmers the basic of python programming during the 2-day Boston Python Workshop, and often we bump into past students at python meetups or other community events. These new programmers often have the same question: Now that I know the basics, how do I start a project? What is the first thing I should type into my text editor?

What do I do now?

If you’re reading this and you’re an experienced programmer, you may not remember the feeling of being unsure about what the right way to start a project is. You, the experienced developer, have developed an instinct for how to start a project. You’re already decomposing the problem into smaller chunks before you even begin writing any code. You know the components you need, and you’re sure that if you’re wrong about what you needed you can find the right component or approach. You’re good at expressing your problem in terms Google can understand, and even better at co-opting someone’s Stack Overflow answer for use in your code.

In this blog post I’m going to provide commentary on the coding of EnPassant, a musical chessboard that I wrote for Music Hack Day Boston 2013 over the span of a weekend. I’ll discuss how I picked this project, what I thought I might need to code such a project, and every commit I made to github and why I made it. I’ll provide screenshot or examples of the project at each commit. Let’s get started!

Defining the Problem

Music Hack Day Boston is a weekend long event where programmers get together and hack together cool musical projects. The projects can be software (like mine), hardware, or anything else, provided they’re started, finished and demo’d in the span of the weekend.

As an avid chess player I was interested in ways of turning chess games into songs, and the square grid of a chess board mapped well to an instrument I had played with once called a Tenori-on, so I decided to make a Tenori-on out of a chess board.

A Tenori-on is an instrument that is a grid of buttons, with the y axis determining the pitch of the button and the x axis determining the beat at which the note is played. The instrument loops over and over again and plays all the buttons that have been pressed, creating a unique and easily controllable musical instrument. The pitches of the buttons are typically set to something that works well musically such that no matter how many or which buttons are pressed you’re probably getting something that sounds good. It’s easier to understand this by seeing one in action, so take a moment to watch the video below.

What I imagined for my project was that the chess board would be the grid, and for any chess piece that had moved I’d play a note for that piece, and maybe if I had time I could make some special features like different instruments for each piece or color, or a noise when captured.

Now I know my problem statement:

Make a chessboard that plays a note for every piece on the board that has moved at least once, with a different note played depending on the location on the y axis of the chess board. Play notes for each piece on every column on the x axis in order, and loop back to the first column.

I can call my project done when I’ve got a chessboard (where?) that can play music (how?) as I move the pieces (with what?). At this point I know I need to pick some technology that will help answer the where/how/what questions.

Picking The Technology

I think that this is the point where my experience plays the biggest part in making this project achievable in a weekend. A beginner programmer doesn’t yet have an internal library of technology they know with which to pick from. Here are some questions you can ask yourself to help you pick your technology:

What’s your audience for this project?

How will your users use this project?

What technology will help me reach my audience and help my users use my project?

I decided that my audience would be everyone on the web, and I wanted my users to experience the project on the web. Given this, I knew that I’d not be able to use python for most if not all of my project, because the languages of the web are Javascript, HTML and CSS. I also wouldn’t be making a mobile application or a command line tool – I don’t know enough about mobile to think I could do my project in a weekend and I didn’t think that a command-line chess board would be all that much fun (and, it’s not on the web).

When you are thinking about your project it may be that you come up with your problem statement and technology choices right away. I picked Javascript, HTML and CSS immediately after deciding I wanted to make a web page. You also may only have one answer for the technology question as you may only know one technology. That’s okay!

If you’re starting out, don’t over think what you are doing – perfection is the enemy of done. I easily could think of how to over engineer my little chess board instrument (“What I need is a RESTful chess music API, with a nosql database of chess games, instantly searchable by music phrase, exposed to a suite of client applications on web, iOS and Android, integrated with a user’s social network, that can also generate new, valid chess games based on the user’s Spotify playlists, played back on my chessboard Tenori-On.”). Furthermore, on small projects you can always switch technologies, and I’ve only got a weekend to deliver on this project.

Okay, I know my problem statement, and I know what technology I’m going to try to use to solve my problem. I’ve really only spent about 20 minutes refining my problem statement and picking my technology. How do I decide what to (try) to code first?

Let’s think about our problem statement and figure out the most important things we need to create to solve our problem; the things that are directly required to solve the problem but don’t need anything else. For example, I know I need a chessboard that can be displayed on the web. I know I need something to make music in the browser. I also know I need to know when a chess piece is moved on my board, but I’m going to ignore that for now as it is not directly involved in solving our problem. I need to be able to map a square on a chessboard to a note, but again that depends on the chessboard and music player, so let’s ignore that for now. I guess the most important things for building my project are a chessboard and a something that can play notes, so let’s pick one and get started.

Both seem equally important, so I’ll start out with the chessboard since that seems easier. Personally, when working on a project with some visual component (like a website or a game), I like to get the visual component working first because I find such feedback very motivational.

Writing the First Line of Code

Now I can finally write some code. I create a github project and clone it to my machine… and then I google for “open source javascript chess board” since I figure someone has probably written a nice chessboard for the web and made it open source. This isn’t always the case, and if you are writing code for work you may not be able use open source libraries, but for a game like chess (very popular among programmers) I figured someone had done 80% of my work.

Drawing A Chess Board

Indeed, http://chessboardjs.com/ has an MIT licensed embeddable Javascript chessboard, and look at that it can move pieces around and everything! Sweet! Let’s setup a test web page on my machine and embed a chessboard in my test web page. This will be the very first lines of code I write for my project.

The code is basically the same as the code from the example on chessboardjs.com. That’s fine, I’m only interested in getting started (and secretly interested in building my own confidence that I can do this project). I’ve got one of my important items done: display a chessboard. This actually took a bit longer than the code shows, because I also had to figure out how to display a web page on my local machine (I ended up using Google App Engine as a simple local web server), download the chessboard.js and jQuery files, and make sure all my <script> and <link> tags point to the correct locations for my Javascript and CSS files.

I also want to be able to make sure all the moves work, and doing so by hand every time I make a change would be no fun, so I grab some famous chess games from the internet in the standard format for chess games, Portable Game Notation. However, when I look at chessboard.js it doesn’t appear to be able to load PGN files, but instead recommends I use https://github.com/jhlywa/chess.js to load PGN files and replay the moves. Being a lazy programmer, I download this library and follow the directions on chessboard.js’ and chess.js’ documentation to get PGN loading working.

Some interesting things occur in the code at this point. One is that I’ve started adding some functions, which I know I should put in a different file but haven’t felt the need to yet. Another is that I’ve had to write some code because the castling code in the chess libraries needed help, another is that I loop on the movePiece() call forever… and it looks ugly, doesn’t play music, and only replays one game. Again, this is all okay right now – I’ve made good progress on one of my main goals: getting a chess board displayed and working.

Making Some Music

Let’s try to get some music playing, really any noise at all. I didn’t actually know how to make my web browser play music, so this was a learning experience. I’d heard that HTML5 had a way to play music in the browser, so I did some reading on HTML5 web audio and found some tutorials that showed how to get your web browser to beep. In a test javascript file I got the simple beeping working, and then started adding that to my code.

A note on learning a piece of new tech: trying out some new technique or technology in a separate file by creating the very simplest program that uses that technique or technology is a something I highly recommend for any programmer, but especially for beginners. You want your mind to focus only on the whatever it is your are learning about; all your other code will just be distracting and make it hard to understand the new stuff you’re learning about. As with bugs, you really want to learn with a separate, self-contained example.

My simple test file setup an web page (warning: annoying noise) that generated a short beep. After I got that beep working, I opened up my project and worked on getting beeping on move working, which proved challenging. The first issues I ran into were that I need to shut off my beeps after some time, or I’d just add beep upon beep as my moves happened. The second, which at this revision remains unfixed, is that my code throws various HTML5 web audio exceptions that at this revision I don’t understand.

Despite this bug I’m pretty happy. It’s a been a few hours of development but I’ve already got my chessboard replaying a game and have some admittedly horrible beeps coming out of the speakers. I know that my bugs will be hard to track down, but despite having not worked with HTML5 audio before I’m also pretty sure I’m not the first person to run into this problem and some googling will help me find the answer. I now take some time to figure out what to work on next – the bugs? The horrible noise? The fact that I can’t actually move my pieces and can only replay chess games?

Making the Noise Less Horrible

I choose to work on the horrible noises, because I figure that the chessboard related issues are easily fixable and if I run out of time my project at least can replay a game, which technically solves one of the important items from my problem statement. As for the bugs, I think they’ll work out as I play with the beeping code, which I know I’m going to make lots of changes too, and besides, the beeps are not in any way music.

Looking at the HTML5 web audio API I can see that it plays the beeps at a specific frequency and note a musical note. Real Tenori-Ons are typically setup to play in a specific musical scale, so what I’d really like to have is some way to say “make each row in in the chessboard a note from the E minor pentatonic scale, with the lower pitch notes being at rank 0 (the first row of the chessboard for the white pieces) and increasing in pitch as the ranks go up.” This becomes my new problem statement.

Since the web audio API only cares about frequencies and I want notes, the first thing I decide to do is find a mapping of notes and octaves (“E3”) to frequencies (164.814 hertz). I do some googling and start to write my own mapping of notes to frequencies… but quickly stop, as just like with the chessboard, I figure someone has done this before me. One google query gets me to music.js, which is a library for working with music in javascript. This library includes the ability to map note and octave to frequency, and as a bonus also includes the ability to generate all the notes in a specific scale. Laziness is rewarded once again!

Commit 5583777acc0e2326ca2823c4b5c0be05d2703d1d has the addition of music.js and start using notes instead of one frequency per row and per piece. I also add a pieceEffect object in hopes of being able to do interesting based on the piece (for example, make the queen louder than the pawns). This is all great but the reality is that the project still sounds terrible. More musically terrible, but terrible none-the-less. Looking at the javascript console I think a good bit of the terrible is from the bugs in my code, as the notes keep playing whenever the bugs are triggered. I guess it is time to fix the bugs.

Googling around shows me that I’m trying to re-fire a synth that is already being fired, and that I need to be sure to stop a synth before I re-use it. Commit 0ee2f06b41adce09723c5042a266f8f7713e59ef has this change. There’s still some popping noise I’ll have to fix later, but at least now my software accomplishes my two main goals: it plays music, and it moves chess pieces around. Nice!

No exceptions!

Make it Awesome

I’d accomplished my primary goal – my chessboard makes music – but had some time before I had to demo my application. I didn’t like the look of my application, and I didn’t like the sound of my application. My new problems:

Make the music more like music

Make the application visually pleasing & easy to use

I’d like my code to be “better”, whatever that means

To make the application sound more like music I decided to speed up the game, and hence the music, as the game progresses. This would add tension and make the game more exciting to watch and listen too, and isn’t a lot of code.

Game with decreasing inter-note duration.

Commit 8ff043d5c4016083a4f5ed64efa67783722b6e23 shows this version, and this change does indeed add some nice tension. There’s probably a lot more to do here, but I’d like to make progress on the visual aspect of the application.

Visually, it’d be nice to center the board, and have a few ‘songs’ to play. While horizontally and vertically centering a <div> with CSS is the hardest problem in computer science, I struggle through and get the board centered. I also change the background color, add some games from famous players to the game database, and pick a few nice pictures of the chess players of those games to line the top of the screen. I make those pictures clickable and give a nice flash of color on each picture to hint to the user that the pictures are the means to start the game.

Finally, I take a look over the code and make sure it isn’t too terrible and doesn’t obviously crash. That done, I go up on stage and demo the chess Tenori-On!

Final Thoughts

I hope this review of my thinking during this quick software project was useful for those beginning their programming journey. Were this a more serious project we’d also go into how we decide what tests to write, how we want to maintain the project and its dependencies, how to collaborate with others, and the many other issues that make up a software engineering project.

There’s a lack of good books to help the beginning programmer move beyond tutorials on a languages syntax and semantics; practice fills this gap. When are you learning a new skill you need to spend time practicing. For the new programmer events like Music Hack Day are a great way to find and work on small projects alone or in a group. Most such hack days start with group brainstorming to help you find a fun project, so don’t worry if you don’t know what you want to work on when you get there.

There’s a few lessons from this project that a new programmer should take to heart:

If you’re reading this and just starting out in software engineering, what are you doing to improve and learn? If you’re an experienced programmer, what did you do to get better? What are you doing now? Leave a comment with your thoughts!

After the disclosure of the source behind the iOS/OS X TLS/SSL bug being a caused by missing curly braces following an if statement, I decided to see if such a problem might happen to openssl. I ran N’SIQ Cpp Style on openssl-1.0.1f‘s crypto directory. I trimmed the nsiqcppstyle ruleset down to one rule which checks for lack of braces on an if, while, for, or do statement, then further modified that rule to only check if statements.

The report is presented below. Note that this is just openssl-1.0.1f/crypto, and not the full OpenSSL source tree.

There were a number of valid criticisms of my curriculum. First, a fewpeople thought that I should move Probabilistic Graphical Models to the 4th year and move Machine Learning to the second year. I had picked PGM as one of the available math courses and underestimated the difficulty of this course after reading the syllabus, leading me to put PGM in the 2nd year, and for ML I overestimated the difficulty. Swapping these two courses makes sense.

Commenters on reddit and on the blog wrote that there wasn’t much math in the curriculum. I didn’t see other fitting math courses on Cousera, but I’m sure Coursera will pick up more math courses which could then be added to the curriculum, or could replace a few of the later CS courses that would be considered optional in most university curriculums (some of the image processing courses, perhaps).

Others pointed out that MIT’s OpenCourseWare has an excellent selection of material available for the online student – which is true, there are 2100 courses and counting (although, some of those 2100 courses are different instances of the same course) – but the OpenCourseWare courses largely don’t have recorded lecture, and none appear to have online quizzing, online homework grading, forums, or peer grading, all of which are aspects of the other massive online open courses such as Udacity, Coursera, or EdX. That’s not to diminish OpenCourseWare, which is an amazing effort and in many ways could be seen as the mother of all the MOOCs that exist today. Redditor bstpierre777 used OCW to create a matching set of electives for my curriculum.

Saylor’s computer science curriculum was brought up in a few comments. Honestly, I’d never seen Saylor before. They have (as of October 2nd, 2012) 267 courses available in a variety of subjects and they offer curricula for various subject matter, including Computer Science. I think that’s awesome! A quick look at a few of the Saylor courses makes me think that the Saylor approach is very self-directed; there are no video lectures by a professor, but instead suggested readings, homeworks and tests that the student completes at their own pace. If you’ve taken a Saylor course I’d love to hear about it in the comments.

I think the classes are only part of a complete education. The people you meet in the course of your education (however you get that education) are just as important as the content of the classes. Top tier schools will not lose students to MOOCs because their selection processes create a cohort of smart, talented students that will interact with each other for the rest of their lives. When I attended RIT I lived in Computer Science House, and that community was as valuable as the classes at RIT.

That’s not at all to say you couldn’t get the same experience and value from online learning. The forums for the classes I’ve taken on Coursera have been active with a very high signal to noise ratio (although, some of that signal is answers to homeworks and quizzing, which facilitates both checking your answers and cheating). It’s easy to think that a MOOC like Coursera could have a forum for students in the same “year” of their curriculum, a forum that exists beyond a single class. The forum members would form a graduating class, and while they may never meet in person, it’s not clear that they need to.

The most popular day on my blog, ever.

I’d like to thank everyone who read the post (and thanks to @jessicamckellar for suggestions, edits and proofreading). I had over 30,000 unique page views from the reddit link and my posted netted 2111 upvotes on Reddit. Online education is clearly a hot topic and I’m excited to see where it goes next and happy to have contributed to the conversation.

With the recent announcement of 17 new schools participating in the massive open online course (MOOC) site Coursera.org, I thought it would be an interesting exercise to see if it was possible to design a reasonable computer science curriculum using just Coursera courses, where “reasonable” is a curriculum that roughly mirrors the coursework required for a four-year university computer science degree.

I’ve looked over all of the available Coursera courses as of September 21st, 2012, and created a four-year curriculum. I’ve tried to follow the curricula suggested by real world colleges; in particular I’ve loosely based the approach on MIT’s course 6 curriculum (specifically, 6-3).

“Semester” is a loose term in this case, as the courses vary in length from 6 to 14 weeks. I’ve assumed it would be possible to take 4 core curriculum courses in the same semester, and that the student would take an additional course that is not computer science related.

My aim is a computer science education that gives the student a background in:

Algorithms

Control (robotics, other mechanical interactions)

Data structures

Debugging

Electrical engineering

Hardware

Programming language theory

Software architecture & design

Networking

Systems programming

Mathematics, including:

Statistics

Probability

Logic

Calculus

Little time is spent on specific programming languages; once the student is exposed to Python and some other languages in the first semester they are expected to be able to pick up other languages. That will take some work outside of the core course work but the Internet has so many good resources for learning the specifics of programming languages that I’ve chosen not to include classes on specific languages.

Below is the curriculum. I’ll discuss some of the reasons behind the choices later, but first take some time to click through to the course descriptions and see what you think.

The curriculum above is purely focused on computer science and mathematics. I’ve left off the basic science and liberal arts classes, but Coursera offers many options in this regard and the student can pick or choose the classes that best fit the student’s interests.

It’d be awesome if Coursera had a course in discrete math. I think that would complement the other math courses in my curriculum nicely. Additionally, it’d be cool if Coursera had a course on operating systems development, or possibly another low-level programming topic like file systems or databases. Frankly, I’m amazed at the depths and breadth of the computer science courses available, and it’s a credit to Coursera that they’ve built such an impressive catalog. In 2012, a student can take the majority of a university computer science curriculum for free thanks to Coursera. That was not possible in 2011.

Learning Online

I’ve taken a few courses on Coursera and other MOOCs and for the classes I’ve taken the medium works well. Like in-person classes, the Coursera courses have lecture notes, slides, homework assignments, tests and quizzes, and online forums for the students to discuss the class. However, with a MOOC, the student can fast forward and rewind the lecturer. Online, quizzes and tests are primarily multiple choice, but Coursera randomizes the questions and the order of the answer (and often picks a set of multiple choices answers from a large pool of answers for a single question). Essay questions are peer graded, which is a process that I think is still evolving – in the class I took with peer graded papers, two of the papers I graded (out of 3) were verbatim copies of Wikipedia articles-which made grading those easy-but the 3rd paper was original content, and I’m sure I evaluated that content with less skill than a professor would.

In my Compilers class, the professor had a clever online tool that could be used to create proofs, and using this tool was required to complete the homework assignments. I think that more and more tools of this kind will need to be created to deal with assignments that aren’t multiple choice quizzes or peer graded essays. For the Compilers class our programming assignments were graded by a script that ran a series of unit tests against our compilers. Grading programming assignments this way felt natural to me – after all, at work my code has to pass a gauntlet of unit tests before the code can go into production.

On the other hand, MOOCs have a few challenges due to their scale (challenges which I’m sure people are working on solutions for). One of these is group projects. As a software engineer I work in groups all the time, and as much as I hated group work in college it is an important skill for the professional programmer. Public speaking is another important skill that is easy to teach in a classroom setting but hard at scale.

There’s also a set of classes where students can be given the same assignment but produce correct but radically different work, or classes where the deliverables are too complex to be judged by software or other students. For example, MIT’s 6.033 class has several design projects whose deliverables are complex system designs. What is the best way to grade such assignments?

Coursera and its kin are evolving rapidly, and there’s been a lot of research into online education. I’m excited to see what learning online looks like next year.

Final Thoughts

I don’t think it will be long before Coursera, Udacity, EdX, or other MOOCs start grouping courses into curricula. Khan Academy already has a knowledge map of exercises that build on another, only letting the student progress once they have mastered a required skill for the next level. I hope we’ll see curricula in MOOCs that is created and curated by professors or other education professionals.

What do you think of my computer science curriculum? What would you add or remove? Have you tried a class with Coursera or another MOOC?

During the Watson-related madness earlier this year I wrote a script to put all of the Jeopardy! game data from J! Archive into a SQL database (they have no API so I parsed the very-broken HTML for each game). I never ended up doing anything with the data, but there’s been renewed interest around the house in this data so I took some time to make a word cloud of the Jeopardy! categories.

At Boston DevOpsDays 2011 last week I hosted an open spaces discussion during which we prognosticated on what the everyday sysadmin would look like in 10 years time.

A lively discussion followed and out of it we came up with a few key predictions that we all loosely agreed on; the future sysadmin will:

Write code.

We all agreed that there’s little place for a future sysadmin that can’t (at the minimum) write scripts, and nominally write and understand code in a non-shell programming language.

Do a lot of data analytics.

We thought that any future sysadmin will be much more of a data-driven engineer; they’ll build systems based on engineering not gut feelings or “because it worked last time”. The future sysadmin can do math because the future sysadmin does more science.

Work on a higher level of abstraction.

The future sysadmin needs to build complex systems by treating what we now think of as systems as building blocks. They’ll not think as much about network ports, IP addresses and machines but instead think about the interactions of applications and instances of those applications.

Focus on service delivery.

The future sysadmin has monitors that says if the service is providing the business function, not if the host is pingable. The future sysadmin understands what the service provides and how to make sure that service is being delivered reliably.

Be on call (with her developer friends) and own the problem.

She’s the crisis manager and may have the best understanding of the data as they relate to the system as a whole. She’s got the full stack view that gives her the credibility to the be first line of defense in a tough situation.

Lead the root cause analysis.

The future sysadmin, if anything, has a more complex job because they need to bring together not just computing resources but data and human resources. The future systems administrator is more about the system and less about the administration.

What do you predict?

With the rise of PaaS and IaaS, we see movement away from the traditional role of the sysadmin into a more app-focused, development-aware admin; someone who understands the business needs and the full technology stack, and I feel like the leading edge of administration is already at this point, as the people you see speaking at all the conferences and writing all the books do these “future sysadmin” tasks already. What do you think about these predictions? What do you think the future sysadmin will look like?

Don’t tell this blog, but I’ve recently published two articles are part of SysAdvent, a collaborative advent calendar style blog put together by Jordan Sissel (@jordansissel). SysAdvent released a new sysadmin related article every day of December though Christmas Day.

Garner’s Modern American Usage - Okay, I’ve not read the whole thing, but I keep it on my desk at work to figure out if my word choice is correct, or if it makes me sound like an asshole (and, all of the usage examples are from recent media, including great references to Superbad when needed).

Web Operations – Allspaw is listed as the author but this book is written by many people in the DevOps community (and hey, I reviewed Patrick DeBois’ chapter on monitoring). This book offers excellent practical advice to people doing web operations.

Rework – The 37signals guys wrote this book about the lessons they’ve learned running a successful startup, and while the book is pretentious, it does have solid advice (for example: don’t hire fast; don’t worry about being “professional”, build software you want to use).

Every year ITA Software sends us all out on a boat to eat, drink and make merry, and this year I thought it would be fun to host a Nerd Pub Trivia. The idea came when, while playing regular pub trivia, the picture round category was “Famous Nerds”. My friends and I thought we were a lock for a perfect picture round score, but instead of pictures of Turing Award winners we got pictures of movie and TV nerds. We joked about how awesome it would be if there was real nerd pub trivia, so that’s what I made happen on ITA’s booze cruise.

The format was a shortened pub trivia, with songs played after reading each question:

1 round of 4 questions with each question being worth either 1, 3, 5 or 7 points (you decide, you can only use each value once)

1 special picture round (2 points for each item correctly identified)

1 round of 4 questions with each question being worth 2, 4, 6 or 8 points (akin to the first round)

2 final questions on which you wager up to 10 points each (win or lose what you wager)

Without further ado, here are the questions:

Famous Nerds: Who wrote in a famous RFC, “be conservative in what you do, be liberal in what you accept from others”?

Databases: What does ACID stand for?

Networking: How many bytes are there in an Ethernet address?

Systems Programming: What is the only UNIX syscall that returns twice?

Data Structures: This colorful binary tree is self-balancing and contains no data in the tree’s leaf nodes – what is it?

Operating Systems: Which process typically has the PID of 1 on a UNIX system?

Computing History: Which of these technologies was not invented at XEROX PARC: Ethernet, The Mouse, The Windowing GUI, laser printer?

SciFi/Fantasy: Who is the Kwistaz Haderach?

Video Games: What item must you retrieve from the Dungeons Of Doom in Nethack?

I won’t post the answers here but feel free to post your answers in the comments (and be aware there’s a few correct answers to some of these questions).

Also, if you’re a nerd looking for a job at a cool place to work, check out ITA’s current job offerings. Not to spam my readers but it really is fun to work at a place where you can host a nerd pub trivia and 10 teams join the fun.