How I ran OkCupid with a text editor: aka Vim-shell

A lot of great things happen by accident. But so do a lot of weird things (and this is one of those.) I was joking around to some friends about using Vim for all the things, and someone mentioned using it as a shell (likely as a stab). One thing lead to another and I thought "why not?" This is my story on why not! 🙂

If anything, Vim is crazy flexible. It also happens to be my favourite text editor. At OkCupid, we run traditional headless Debian Linux boxes. Most of my day-to-day is realistically spent in either shell or Vim anyway, so combining them both makes sense, right?

The naive way to set this up would be editing your default shell in /etc/passwd to run Vim. This will get Vim up and running, sure, but there are several subtleties along the way.

How is this going to work?

My first priority was figuring out what I need to actually do. That can be generalized fairly easily:

Edit files

List directory contents

Change working directories

Run commands

SSH to various machines

With my list enumerated, I could start to realize how each bulleted item is going to behave in this model.

Edit files

Editing files is easy. That's literally what Vim is made for, so we get to cross that off the list for free.

List directory contents

It is also simple to tackle directory listings. A lesser known feature of Vim is Netrw, which actually happens to be a neat file-explorer. Most installs of Vim should have it. At it's most basic level, you can open a directory with Netrw and view the listing of its contents.

Keeping with the theme of simplicity, you can use two techniques to work with Netrw. First, just specify a directory as Vim's target. For example:

sh$ Vim /path/

Alternately, you can use the :Explore command to open Netrw from within Vim. Either way will give you a curses-based interface for browsing your file system. You can use your standard Vim navigation to highlight different listing items, and use return to select them. This pretty much continues until you select a file (not a directory) on the filesystem.

Conveniently, Netrw will let you delete and rename files, create new directories, mark and tag files, and a whole bunch of other things. There's a fairly comprehensive help built-in, so just open Netrw and hit F1 if you want to learn more.

Change working directories

Changing working directories is going to feel pretty familiar. You can change directories by using the :cd command in Vim, and print the working directory by using the :pwd command. If you want to get a little advanced, you can even use :cd - to return to the previous directory.

Run commands

Here's where things get a little tricky. Executing external commands is provided by Vim through the bang command (:!). For example, if we wanted to run whoami:

:!whoami

The caveat here is, unlike a more traditional shell (such as bash) Vim does not execute the process directly. Instead, it's actually calling the binary specified by your $SHELL variable, and passing your command as as parameter to the -c flag. So in reality, when you run :!whoami it's really executing /bin/bash -c whoami (assuming your shell is Bash.)

Of course, the problem now is that we have just set our shell to Vim so it's trying to execute Vim -c whoami. This fails spectacularly.

I decided to cheat a little bit here (at least it feels that way to me.) In .Vimrc, I set SHELL to be Bash (this way executing commands will actually behave somewhat like they are supposed to.)

Perhaps I can improve the process later, but for now I decided to move on. With command execution working, I wanted to know how much of shell basics were going to be available.

Globbing works as expected when shelling out commands, but also when referencing files. So for example, :e Make* opened up Makefile in my current directory.

Output redirection is actually kind of fun. You can use the :read command to append the output of your shelled out commands to the current buffer. For example:

:read !date; uname -a

You can also use the contents of the current buffer as the file target of a command. For example, let's say you had the following in your current buffer:

moo
moose
cow

You could grep against the contents, and overwrite the current buffer with the output:

:%! grep moo

Now you would be left with the following:

moo
moose

Of course, you can always just use redirection when shelling out:

:! echo moose | grep moo

Since I still had SHELL set to Bash, I feel like that was cheating more than I wanted, so I avoided this approach.

Looping was kept pretty simple. Of course, you can shell out loops to Bash in the normal way, but I again tried to avoid this. Instead, I just used command repetition in Vim. [email protected]: Where x is an integer (or implied 1 if missing.)

SSH to various machines

Finally, we come to moving between machines. With SSH keys and agent forwarding enabled, it's easy to move between SSH jump-box and second tier infrastructure just executing SSH as we would expect.

:!ssh webserver.domain

With Vim now as my default shell, I will get a new instance of Vim on the remote machine. Quitting Vim is sufficient to return to the previous machine in our chain (the jump-box, in my environment.)

And then the grind...

So all of this is fine in theory, but how does it hold up to real-life work? I'll be honest, I used this for about three weeks to really give it a try. It's definitely doable but I wouldn't necessarily call it usable. I generally use Bash as my primary shell because I enjoy the many built-in workflow improvements it has to offer. From simple things like command history to more interesting features like syntax and command expansion, all of these great features are more-or-less lost using Vim. And again, when you spend that much time inside a shell, not being able to do things like this can often hurt:

A major consideration when attempting to use Vim as a shell is SSH remote command execution breaks because it relies on the user's shell which (when set to Vim) is now trying to open filenames rather than execute commands. At OkCupid, we have a critical utility called colodo which assists in executing statements across multiple nodes throughout the infrastructure. This utility relies on SSH remote command execution to work, and outright fails in this model. That probably became my biggest pain point. Running commands individually across 100 or more nodes is bad enough, but try doing that through a text editor. 😝

I love Vim, and this little experiment hasn't changed that at all (though I do have a stronger appreciation for Bourne shells now.) In the end, I reverted my shell back to Bash so I could get real work done.

To make this a bit more practical, I think I would have to have a third mode mode in Vim for "process execution". Preferably, it would not shell out process execution, It would need command history, and it would have at least some basic syntax expansion. I think I would also need to find a better means of executing remote commands in bulk. Maybe someone wants to put in a PR for Vim 9. 😉