Ph.D Researcher in Software Engineering and Human Factors of Computing

Tag Archives: programming

When I program, I often try to split up my code into modules. This requires that I use different files. However, I also want to run my code from a “main” or a “master” file as well. It’s pretty annoying in Sublime Text, or any text editor, to have to switch to your main file to run your project.

That’s it! While you’re editing your project settings, you can make other customizations as well as you see fit.

Obviously, this isn’t limited to just setting a file to be run no matter what file you’re focused on. You could use this to add command-line arguments to a single project, to run project-specific scripts for building and testing, or to otherwise customize the way the “Control-B” (Command-B on Mac OS X) works when you are running the project. You can essentially define your own build system on a per-project basis in a project file’s “build_systems” section and use any information in Sublime Text’s build system documentation to do so. Unfortunately, the build system documentation doesn’t really describe this because it’s focused more toward Package development.

For more information about how project settings work, look at Sublime Text’s documentation for build systems and for projects!

When I program, I use a text editor. Now, even though I recently bought Sublime Text (and I’ll probably do a post on that later), I still use vi a lot – specifically, vim. I also occasionally used GNU screen (mostly for managing remote IRC sessions with the irssi client).

So why use the console to program? Well, the main reason is that typing in the keyboard and not using the mouse is faster if your keyboard skills are good. Given the quality of most laptop trackpads, getting away from the need to use the mouse is actually something of a relief sometimes. The second reason to be familiar with the console is if you’re logging in to a remote session, you often don’t have a mouse available to you at all.

The learning curve is quite steep for console applications, but once you get familiar with them you can get your editing tasks done very quickly because everything’s quite literally right at your fingertips.

Screen: Access Terminal Applications All The Time, Anywhere You Want

GNU Screen is a terminal emulator that’s available on most flavours of Unix-like operating systems, including Linux, BSD, and Mac OS X. Screen runs inside a shell. I think the “killer feature” of screen is the fact that screen is persistent, so you can essentially make your terminal applications run all the time, from anywhere you want.

A typical screen session

What this means is that if you log into a remote machine, and use “screen”, you can start commands that you expect to run for a long time, close the SSH window, and then log back in and resume what you were doing. Everything will be chugging along just as you had it. It’s especially useful for applications that need to run in the background for a long time, like IRC windows or really, really long compilations or analysis.

Another feature of screen is the fact that it has “tabs”, or multiple windows within screen. You can then run multiple terminal programs in the same screen session, resuming them when you want.

A Basic Screen Workflow

To use screen, open a terminal window and then type screen. You will get a splash screen, but then you will be kicked out into what is essentially a shell. From here, you can basically do everything you normally would in a console.

Let’s say you want a new console to work in – a new “tab”, so to speak. In screen, press Control-a, let go of control, then c (c for create). This will create a new console for you to work in. You now have two consoles!

In screen, every command is preceded by a “Control-a”. It’s kind of the secret shortcut in the program, it’s often abbreviated, so Control-a and a (which means to go to the beginning of the current line) is often written as C-a a.

Let’s say you’re doing a lot of work in screen and now you need to log out, but you want to keep all of your work active. This is a “detach”. If you want to “detach” a screen and do other things in your original shell, you can do that with C-a d. If you’re working in a windowed environment, if you close the program that screen is running in, it’ll also detach for you.

Later that day, you want to log back in to the machine and resume your work. To “attach” a screen, type screen -R -D. That finds a screen for you, detaches it from other consoles if necessary, and resumes it.

The workflow for screen is pretty simple overall: you basically attach a screen when you start, then you create new windows, and detach when you’re done (or if you’re like me, you’ll just lock your computer and reattach it whenever you get around to it from whereever).

I find that using screen in conjunction with vi is extremely useful: you edit it one screen session, then compile/run/test in another.

Making Screen Look Pretty and Useful

You’ll notice that screen on the surface is pretty minimal. However, there are a lot of features built in that make it quite handy as a terminal emulator.

It’s possible to list the windows in your screen at the bottom of the terminal. You can see which one you’re currently working on. You can list the hostname, the current time, and so forth. With a little bit of work, you can make a very simple statusbar that will increase the usability of screen significantly.

To do this, first edit the .screenrc file in your home directory. Then, add the following lines:

I won’t step through the syntax in its entirety (it’s very cryptic and confusing), but you’ll end up with a handy listing of the current screen windows in the session with the current one visible in red.

I followed the syntax on the GNU Screen manual to customize the string to my liking. %{.kw} colours the string with a black background and white text…. %-w lists all of the windows in screen up to, but not including your current session, %{= kR} makes the next section black and red, ( is a parentheses, %n is the number of the current window, and so forth. It takes a bit of starting at to understand, but it’s not too difficult to customize to your linking.

Vim: Navigate your Documents in an Instant

Vi is a text editor that originated in the mid 70s. One of the most popular modern incarnations is Vi Improved (vim). One of the most powerful features of Vim is the speed at which you can navigate through your documents. You can immediately jump to lines, delete a range of lines, move forward four spaces (or four words), search for words easily, and more, without your hands leaving the keyboard. I use it for much of my small-scale programming work even though I like my GUI-based text editors.

To launch it, open a text editor and type vi.

Configuring vim

I’m really not that much of a vim power user; my .vimrc file is really simple compared to some people who live in vim full-time. I like navigating code in vim and I use it for most of my console-based editing but if I am working on large, multi-project files I end up using a text editor like Sublime Text instead.

Regardless, it’s nice to have a decently usable set of vim defaults. Here’s mine:

syntax on
set number
set smartindent
set tabstop=4
set shiftwidth=4
set expandtab

In this setup, I turn on syntax highlighting, print numbers on the side (considering how dependent vim is on navigating by line numbers, this is essential), I turn on auto-indentation, set the tab size and indentation sizes to 4, and expand tabs to spaces.

A swift introduction to using vi

vim is pretty intimidating to use, because it uses multiple modes. The default mode is a “navigation” mode where you can move the cursor around, and then there’s a “line editor” mode where you actually modify the text on the screen. There’s also a “command” mode where you enter custom commands, too.

The most important command in vi

If you don’t know anything else about vi, learn this:

ESC :q!

Press the escape key, press colon, press q, then press !. This exits the program with no changes! If you have absolutely no idea how to do anything, you can at least get back out to the console.

Related but important as well:

ESC :wq

Press the escape key, press colon, press w, then press q.

This writes the current file, then quits the program. Basically, it’s a “save and quit”.

Navigating

When you start vim, you’re always in navigation mode. You can navigate using the h, j, k, and l keys. These correspond to left, down, up, and right respectively (this was so ingrained into me that I had to actually start vim to check the directions). One reason why this is great is because the controls are on the home row, so navigating in vi is pretty fast.

If you want to jump to a line, you enter the line number and press G. So going to the first line of a file is 1 G whereas going to line 23 is 23 G.

You can move multiple lines and characters up, left, down, and right as well. 3 h moves three characters left. 5 j moves five lines down.

Editing

i to enter “insert” mode. You can then type to enter characters at the current cursor position. You’ll see -- INSERT-- at the bottom of your screen. To stop editing, press Escape.

There are many variations of insert in vi to help people insert things really, really quickly. I (capital I, vi is case-sensitive) inserts at the beginning of the current line, A inserts at the end of the current line, o inserts a new line before the current line, O inserts a new line after the current line. You can even combine these with numbers and the directional hjkl keys to insert multiple lines. That’s probably too much information.

You can also delete quickly vi as well. x is the basic “delete one character”. dd deletes a line.

Finally, u undoes the last command, so if you find yourself deleting the wrong lines, hit u repeatedly.

Conclusion

I hope this serves as a brief introduction to vim and screen for most people. Anyone with a Unix-like computer and the need to write text or markup should consider looking into these kinds of programs because they increase the flexibility of what you can do.

As I mentioned in my previous posts, I participated in Ludum Dare, a game development “compo”, where you build a video game by yourself, from scratch, in 48 hours (a variation gives 72 hours and a team, but I did the 48-hour version).

A “compo” is a “composition competition”, but I’ve yet to participate in a compo where the competition aspect is what actually gets people energized. If anything, the compo is more community-oriented than it is competition-oriented. I participate weekly at ThaSauce.net One Hour Compo, which is a music compo in which you create a song from scratch in one hour. The competition aspect supposedly is because people vote on their favourites at the end, but in the end I don’t think the votes are what anyone’s really fighting for.

In any case, I feel that compo has probably been one of the top ways for me to improve my music making skills and that doing the Ludum Dare compo was an excellent way for me to simply program for the sheer joy of it.

But, I think one of the greatest benefits of doing these compos is simply being able to succeed, and to feel happy and proud doing it. It’s a real self-esteem booster and it also helps you beef up your skills and the ability for you to work under huge time pressure.

Below I’m going to present the post that I wrote for Ludum Dare 26. Most people there tend to write technical post-mortems, but I thought that the emotional barrier was actually a bigger barrier to cross than the technical ones!

I’m actually happy I participated and would like to thank the handful of people who encouraged me (some strangers, some friends) to enter anyway. It was a great experience and I want to be around next time, if I have time for it.

But, what led me to go down a road of, “I don’t think I can do it?”

I’ve never designed a game before

Well, technically I did. In my first ever C programming class when I was 15, the final project was a video game. Mine compiled but didn’t work – we had Macs at school, and when I realised that my game wasn’t getting close to finished, I brought my game home, wrote almost all of my code on my PC, and hand-checked it to see if it would work, in theory, when I brought into school the next day.

With some work I made it compile, but it didn’t really work as I wanted it to.

In some respect though, it doesn’t take much to be a game. I’ve seen a lot of things that people recognize as games. Top-down shooters, side-scrollers, role-playing games, adventure games, text-based games, board games, and so forth. But I’ve also seen a lot of creative work as well. A game where the main idea was to walk in a city. A game where you woke up, experienced a main character’s morning, and looked at all of the objects in his or her room. A game where you planted seeds in a garden and watched them grow. They’re barely games by the standard definition – but they are all welcome in Ludum Dare.

Even if you don’t have a strong idea of what you want to do, build a game archetype anyway and then see what comes out. In some respect, one of the thrills of doing a game in 48 hours is that the first few ideas you get, you have to stick with because there’s no time to really make it better. So you get all of these raw, unrefined concepts that are the pure essence of creativity. And it’s great to see so many of these concepts work.

My Programming Chops

I don’t program a lot. I have a computer science degree so I know how to program, but my work is primarily focused on research activities that don’t require any development. I find a lot of programming boring – especially mathematics riddles. “Compute the least-common denominator of two numbers?” Snooze. “Write an algorithm that will identify is a string is a palindrome?” Ugh.

But then I start building a game and suddenly, everything is fun, even when I groan at thinking about the geometry and trigonometry. It’s because those things suddenly aren’t just mathematics. They’re situated in my game as a core concept now. I don’t need to understand them for the sake of knowing them – I’m understanding them because I know that they’re useful to me, now.

I learned a lot on the fly, and suddenly I realized that programming isn’t just about what I know – but about how fast I can learn what I need to know. I didn’t know PyGame existed until the morning I decided to do Ludum Dare. I didn’t know how to blit a sprite to a screen before reading about it on my lunch break. I had never thought about sprite rectangles, mouse movements and controls, or drawing tiled backgrounds until the hour the competition started – so I learned those things with a lot of help from the Internet. I can’t say that I know all of those things well, even now… but I’ve done them before now and I can only improve from here.

So even if you’re not a hot-shot programmer, it’s not just about how well you program – it’s about how well you can get what you need done by learning what you need to learn.

Ludum Dare and self-esteem

I think a lot of people who post on this site have a lot of confidence – you have to to enter something that is billed (albiet weakly) as a competition. The games that get all of the press are the ones that have the shiniest graphics, the best lighting engines, the cutest artwork, the most thrilling sound. In the end, history remembers the winners, and all of the winners kind of blob up together into this gigantic “super-winner” amoeba where it feels like one guy participated in 30 Ludum Dares and came up with a hundred amazing games along with a procedural level generator and a memory-management allocation system in the span of a week. The secret though is that this mythical superhuman game programmer doesn’t exist. That programmer is really hundreds of individuals

I think one big lesson to learn from this is that very few of us are superhuman, and more importantly, the majority of people who participate in a Ludum Dare are normal people. They’re not all rock-star programmers, hotshot artists, or amazing musicians. They’re regular people and normal people. Just like you, just like me.

The second point to remember is that Ludum Dare isn’t really a competition. Sure, it’s about getting votes and comments and people get into the top list at the end, but in the end there’s no reward and there’s very few bragging rights. This really isn’t a competition.

I think a big outcome of these two points is that Ludum Dare is a showcase. It’s not just “how good your game is”. It’s the fact that you’ve managed to produce a game at all. No where else would I have been able to build almost any program in 48 hours and then, in the span of under a week (so far!) convince 50 people to play and download my game and leave constructive comments.

In that sense, everyone’s a winner. And even though I said above that almost everyone in Ludum Dare is a “normal person”, the fact that we’ve programmed something from absolutely nothing to a working, deliverable product in 48 hours (or in a team in 72 hours) makes every one of us extraordinary.

I hadn’t been as excited about something before as I had right after Ludum Dare – and I think that’s because that, as soon as I had finished, I realized that I had done something extraordinary. A few thousand of us, together, had each accomplished something to be proud of.

How to Create a Single Windows Executable from a Python and PyGame Project (Summary)

On Windows, you might also need pywin32 (and possibly MinGW if you don’t have Visual Studio).

On Mac OS X, you will need XCode’s command line tools. To install the Command Line tools, first install XCode from the App Store, then go to Preferences – Downloads and there is an option to download them there.

Modify your code so that whenever you refer to your data directory, you wrap it using the following function:

This is mostly for convenience – it allows you to access your resources while developing, but then it’ll add the right prefix when it’s in the deployment environment.

Specify exactly where your fonts are (and include them in the data directory). In other words, don’t use font = Font(None, 26). Instead, use something like font = Font(resource_path(os.path.join('data', 'freesansbold.ttf')), 14).

So why was I trying to package a Python executable file anyway? Read on…

Ludum Dare 26: 48-hour Game Design Compo

This weekend, I decided to participate in a 48-hour game design “competition”. Ludum Dare is a compo that asks you to create a video game from scratch in a 48-hour time period – you have to write your code and create all of your assets in that time period.

This means no reusing graphics, pictures, music, or sound from other projects, for example. You’re also not supposed to reuse code either. I decided to participate on the Thursday the day before. Most people use the previous weekend as a “warmup weekend” to test their tools, get some practice, and so forth. (My entry is located here, by the way).

I’ll do a more detailed compo writeup later, but I just want to concentrate on one thing that kept me up for hours after the competition: getting a Windows executable created from a Python project that uses PyGame and a data directory.

Python, Distribution, and You

I rather enjoy Python as a programming language. The syntax is reasonably concise, the language does a lot of things for you, and it’s well-laid out. There’s also a lot of good support in the form of third-party libraries. I’ve been using Python for various things for the past few years (usually small scripts for data extraction and analysis in research).

One thing I had never thought about before was distributing a Python project as an executable package, and while it was on my mind throughout the entire compo, I didn’t actually learn the process of creating the package until the last hour of the comp before submission. After you submit your primary platform, Ludum Dare allows you around 48 hours to compile for Windows, since the majority of reviewers use Windows.

The ideal submission is a single binary file (an .exe file for Windows) that doesn’t have to extract a lot of data, so that it’s easy for people to download and run your game.

PyInstaller vs. Py2exe vs. Py2app

I went on a wild goose chase trying to find out how to make a single executable file out of a Python project that would include all of my data assets. I first tried py2exe and py2app. py2app mostly worked all right, but py2exe was a pretty big mess.

The end story is that PyInstaller is newer and shinier than py2exe, and that you need to secret sauce code that someone out there on the Internet found before I did. PyInstaller basically runs EXE files by extracting the assets into a temporary data file that has a path _MEIPASS in it ((technical details here). Be sure that you check that every file is loaded in through that wrapper. The Tree() TOC syntax was also confusing, but basically, it’s the relative path of your data files and it will automatically load all of the files in that directory. Make sure it exists in the EXE portion (Windows) or the APP portion (Mac).

There’s a Make/Build cycle in PyInstaller to generate the spec file and build it in a single step as well – I find it easier to do that to generate the spec file and do an initial binary run, then to modify the spec and run PyInstaller again with the spec file as the argument. PyInstaller is pretty smart about rebuilding, and you save a lot of time.

I think in the long run, if you compare py2exe, py2app, and PyInstaller, PyInstaller is the program worth learning. It did have a pretty sharp curve for me – it didn’t help that I was trying to do this late at night after a challenging weekend!

If you do wish to use py2app to build your Mac OS X application bundle, then do keep in mind that you need to have a import pygame._view because of some kind of obscure issue.

Information seeking is one of the most important activities in human-computer interaction! One of the most influential theories in understanding, modelling, and predicting information seeking is information foraging theory. In our research, we want to understand what kinds of diets – that is, the types of information goals programmers seek while debugging. By investigating the information diets of professional programmers from an information foraging theory perspective, our work aims to help bridge the gap between results from software engineering research and Information Foraging Theory foundations as well as results from human-computer interaction research.

Is this tasty?

Is this tasty?

My co-author, David Piorkowski, is travelling soon to Paris to present our latest work: “The Whats and Hows of Programmers’ Foraging Diets”. It’s a great time to expand on this paper. Here’s the PDF Preprint!

Our Method

We had two coders examine video of nine professional programmers to identify what exactly they were looking for when trying to fix a bug in an unfamiliar open-source program. We tried to identify their overall diet by identifying if they asked questions (and received answers) belonging to one of four categories: (1) finding a place to start in code, (2) expanding on that initial starting point, (3) understanding a group of code, or (4) understanding groups of groups of code.

What is a programmer’s diet while debugging?

Overall, we found that programmers spend 50% of their debugging time foraging for information.

Surprisingly, even though all participants were pursuing the same overall goal (the bug), they sought highly diverse diets. For example, Participant 2 asked mostly about groups of groups, Participant 3 asked about finding a place to start, Participant 5 didn’t really ask about anything at all, and Participant 6 also looked for a place to start. This suggests a need for debugging tools to support “long tail” demand curves of programmer information.

How did a programmer consume these diets?

How exactly did programmers go about finding what they wanted to consume?

Again, participants used a diverse mix of strategies. Participants spent only 24% of their time following between-patch foraging strategies (such as code inspection or simply reading the package explorer straight up-and-down), but between-patch foraging (such as doing data flow or control flow) has received most of the research attention.

Surprisingly, search was not a very popular strategy, accounting for less than 15% of participants’ information foraging – and not used at all by 4 of our 9 participants—suggesting that tool support is still critical for non-search strategies in debugging!

Whats Meets Hows

Participants stubbornly pursued particular information in the face of high costs and meager returns. Some participants followed a single pattern over and over again, using the same strategy. For example, in the cases that involved a programmer looking for Type 1-initial goals, participants used code search and spatial strategies extensively but not particularly fruitfully. This emphasizes a key difference between software development and other foraging domains: the highly selective nature of programmers’ dietary needs!

Takeaways

Thus, we considered what programmers want in their diets and how they forage to fulfill each of their dietary needs. Our results suggest that the diet perspective can help reveal when programming tools help to reduce this net demand—and when they do not—during the 50% of debugging time programmers spend foraging.

I know many Ph.D candidates and professors who can program and do program on a regular basis, but I didn’t really consider how often most of these students spend their time programming.

During an academic job interview, I was asked if I programmed a lot. Yes, I program. Do I do it a lot? Well, not exactly. None of the projects I currently work on rely on my programming skills, but I use programmatic thinking frequently. Like most computer science students, I write short scripts for frequently-used tasks. I build my CS361 web site using a shell script, Mustache and JQuery. I write 50-line Python programs to generate level templates for a research project I’m working on. I fix Javascript bugs here and there. I write R scripts to make my data analysis repeatable. But I don’t program like a programmer working in industry would program. I’m very much an end-user programmer, now. Not a novice programmer – an end user.

These end user programmers are the very people we usually assume have no formal computer science background but need to engage in programmatic thinking. Still – with all of this exploration and discovery, I would be hard-pressed to say that I program 50% of the time at work, even.

Most of my time these days is spent writing and designing materials, as well as assisting other students with analysis of their data; I also spend time preparing materials for the Software Engineering I class that I teach this term. Where in that do I find time to program? Generally, I don’t, so most of my programming is relegated to my free time. Perhaps I am not efficient with my free time. I often try to spend time learning frameworks and toolkits that I know about but haven’t worked with extensively or I try to find tools that may help me immediately or in the future. Lately, I’ve also found that I’ve been programming for the pure fun of it – doing projects in Processing or trying to learn live coding in Clojure.

Thus I come out of this post with two questions. How many of you out there have a programming background, but program now as an “end user”, that is, the software products that you build are not the deliverable, but instead they help you get other deliverables out the door? Second, how many people in academia program “a lot”, perhaps, let’s say, program for more than 40% of their work time and 40% of their free time?