Systems, Tools, and Terminal Science

Main menu

Post navigation

Unix as IDE: Introduction

This series has been independently translated into Chinese and
Russian. Thank you very much to the authors of these translations!

Newbies and experienced professional programmers alike appreciate the concept
of the IDE, or integrated development environment. Having the primary
tools necessary for organising, writing, maintaining, testing, and debugging
code in an integrated application with common interfaces for all the different
tools is certainly a very valuable asset. Additionally, an environment
expressly designed for programming in various languages affords advantages such
as autocompletion, and syntax checking and highlighting.

With such tools available to developers on all major desktop operating systems
including Linux and BSD, and with many of the best free of charge, there’s not
really a good reason to write your code in Windows Notepad, or with nano or
cat.

However, there’s a minor meme among devotees of Unix and its modern-day
derivatives that “Unix is an IDE”, meaning that the tools available to
developers on the terminal cover the major features in cutting-edge desktop
IDEs with some ease. Opinion is quite divided on this, but whether or not you
feel it’s fair to call Unix an IDE in the same sense as Eclipse or Microsoft
Visual Studio, it may surprise you just how comprehensive a development
environment the humble Bash shell can be.

How is UNIX an IDE?

The primary rationale for using an IDE is that it gathers all your tools in the
same place, and you can use them in concert with roughly the same user
interface paradigm, and without having to exert too much effort to make
separate applications cooperate. The reason this becomes especially desirable
with GUI applications is because it’s very difficult to make windowed
applications speak a common language or work well with each other; aside from
cutting and pasting text, they don’t share a common interface.

The interesting thing about this problem for shell users is that well-designed
and enduring Unix tools already share a common user interface in streams of
text and files as persistent objects, otherwise expressed in the axiom
“everything’s a file”. Pretty much everything in Unix is built around these two
concepts, and it’s this common user interface, coupled with a forty-year
history of high-powered tools whose users and developers have especially prized
interoperability, that goes a long way to making Unix as powerful as a
full-blown IDE.

The right idea

This attitude isn’t the preserve of battle-hardened Unix greybeards; you can
see it in another form in the way the modern incarnations of the two grand old
text editors Emacs and Vi (GNU Emacs and Vim) have such active communities
developing plugins to make them support pretty much any kind of editing task.
There are plugins to do pretty much anything you could really want to do in
programming in both editors, and like any Vim junkie I could spout off at least
six or seven that I feel are “essential”.

However, it often becomes apparent to me when reading about these efforts that
the developers concerned are trying to make these text editors into IDEs in
their own right. There are posts about never needing to leave Vim, or
never needing to leave Emacs. But I think that trying to shoehorn Vim or
Emacs into becoming something that it’s not isn’t quite thinking about the
problem in the right way. Bram Moolenaar, the author of Vim, appears to agree
to some extent, as you can see by reading :help design-not. The shell is
only ever a Ctrl+Z away, and its mature, highly composable toolset will afford
you more power than either editor ever could.

About this series

In this series of posts, I will be going through six major features of an IDE,
and giving examples showing how common tools available in Linux allow you to
use them together with ease. This will by no means be a comprehensive survey,
nor are the tools I will demonstrate the only options.

File and project management — ls, find, grep/ack, bash

Text editor and editing tools — vim, awk, sort, column

Compiler and/or interpreter — gcc, perl

Build tools — make

Debugger — gdb, valgrind, ltrace, lsof, pmap

Version control — diff, patch, svn, git

What I’m not trying to say

I don’t think IDEs are bad; I think they’re brilliant, which is why I’m trying
to convince you that Unix can be used as one, or at least thought of as one.
I’m also not going to say that Unix is always the best tool for any programming
task; it is arguably much better suited for C, C++, Python, Perl, or Shell
development than it is for more “industry” languages like Java or C#,
especially if writing GUI-heavy applications. In particular, I’m not going to
try to convince you to scrap your hard-won Eclipse or Microsoft Visual Studio
knowledge for the sometimes esoteric world of the command line. All I want to
do is show you what we’re doing on the other side of the fence.

18 thoughts on “Unix as IDE: Introduction”

Unix was an IDE before IDEs were invented. An early version of UNIX out of Bell Labs was called PWB/UNIX for Programmer’s Work Bench. The editors and text processing tools along with job control were state of the art back then.

I wouldn’t be surprised to find out that PWB/UNIX was written with fewer lines of code than Eclipse, and started up a lot faster.

The book “The UNIX Programming Environment” by Kernighan and Pike is a good read, and most of it still applies in modern flavors of UNIX such as Linux and MacOS.

As Greg points out, Unix was initially invented to be a development environment, not a general-purpose operating system. For a long time, you never saw a Unix (or Unix-like) OS that didn’t come with a full complement of development tools. It wasn’t “Unix” if $CC didn’t point to the system’s C compiler.

So it’s perfectly fair to call Unix a “DE”. Perhaps even an SDK. But calling it an “IDE” is wrong. The “integrated” part implies that all of the disparate tools have been gathered together under one interface, like Eclipse or KDevelop.

Agreed; the reason I’m using “as IDE” here is to give a concept that non-Unix people might better relate to. The orthogonality and interoperability of Unix’s tools with streams of text goes a long way to make it cohesive, but you are quite right when you say it is not, strictly speaking, “integrated”. I briefly considered naming the series “Unix as DDE” for “Distributed Development Environment”, but I don’t want to make terms up in post/series titles.

OTOH, you could argue that the tight integration between different components, without a clear delimitation and definition of generic interfaces, is just a bad architectural trait of modern IDEs.

I don’t really care if my plain text editor allows me to configure a regexp for file/line/column pointers to use in parsing tool output, to plug in a generic debugger via a standard & configurable command interface or to add a source code indexer of my choice.

Integration done this way is still very strong, only, the architecture of the system is cleaner, sitting on top of narrower, more specialized interfaces, respecting the Unix philosophy of doing one thing and doing it well, and giving me the freedom to choose whichever debugger/compiler/source code indexer I want to use with lower cost and little variation or need to change in the frame which is the editor.

Why bash? It’s not as portable as sh; not as resilient in the fact of platform catastrophe as Bourne shell (sh) and its clones, csh, and tcsh; not as well-suited to heavy-duty scripting as Perl, Python, Ruby, mksh, or zsh; not served by as friendly a documentation set as almost any major interactive shell for Unix; and not as well thought-out in its feature set for interactive use as mksh or zsh. It’s kinda the worst of all worlds when it comes to the popular Unix shells. There are other reasons to prefer any of the above shells (dismissing the mostly scripting-only options) as well, but they’re starting to get into very controversial territory, and I’d prefer to avoid the controversial aspects of the decision for the moment.

Thanks for the well-informed feedback; I chose to write about Bash largely because of its ubiquity on Linux systems, as opposed to its technical or design virtues. The theme in general for the explanations of common Unix tools is stuff you can do with a stock-standard Linux setup on Debian or Ubuntu, like most of my readers will have. I would also not be qualified to write about Csh or Zsh as I know comparatively little about both.

I do agree however that long/complex shell scripts are generally bad news when Perl is everywhere, and Python and Ruby generally just a package install away. I haven’t written a shell script longer than five lines or so in years.

I guess I can get the defaulting to the most common default user shell in Linux systems, though I noticed you mentioned “BSD”, and none of the core BSD Unix systems (FreeBSD, OpenBSD, NetBSD, Dragonfly BSD . . .) use bash as the default user shell. FreeBSD uses csh or tcsh, for instance, by default. I honestly might not have thought to comment on the selection of bash there if you had not said “BSD” in there.

Almost everyone I know who has a specific preference other than just defaulting to bash chooses either zsh or mksh (the latter being a Korn shell descendant that originated with another BSD Unix system, MirOS). When writing about shell usage, if I do not have a specific need to focus on a particular shell, I tend to refer to shells in the generic. From what I see here, there’s little need to specify a given Unix shell, so I probably would have just referred to Unix shells in the generic. The first case where it would become an issue, I think, is if you need to redirect both STDOUT and STDERR, in different directions, which could cause problems for csh (for instance), but that doesn’t seem particularly relevant to this.

Anyway . . . in general, I quite like what you’ve done here with the discussion of using the Unix shell environment as an IDE. I don’t even have any objections to the “integrated” part of that, in that the Unix environment as a whole is an “integrated” interface by way of being Unix (or a Unix-like OS, as in the case of your favorite Linux distribution, naturally). I’ve expressed a similar sentiment to yours here — that the Unix environment is my IDE — quite a few times in the past, and you did a great job of articulating that notion. I hope you took my comments about bash vs. the alternatives in the spirit in which I meant it, trying to help enhance the message for the best results of those who follow in your footsteps, rather than a hypercritical denunciation or something like that (which I see, in retrospect, is one potential interpretation of my question and commentary).

I did indeed take your comments in that way, and they’re most welcome. Though I did explicitly choose Bash as the shell with which to demonstrate, I tried to avoid features that wouldn’t work in other shells, or had no functional/syntactical equivalents. The emphasis is on composing orthogonal tools consistent with Unix’s spirit of text streams as interfaces, rather than on specific choice of tools. In this series, I just chose tools that will almost certainly at least be familiar to most intermediate to advanced Unix users, and accessible for beginners.

With regard to choice of operating system, the article does mention the demonstration takes place on Linux several times, which is what informs my choice of Bash; I do have some BSD experience but not as much and felt the article would be better informed if I demonstrated using a system more familiar to me. The point is that Linux is a Unix-like OS, and using its command line in the manner I’ve been describing here is something you ought to be able to in BSD as well, or even something like Solaris if someone so chose.

I think unix is also best suit for industrial language like java. I have been using eclipse to develop android project and i have almost knowing every details of eclipse and remembered every shortcut key. But after i switching to vim, my work efficiency improved quite a lot. I don’t have to wait the ide to draw the gui for a long time.

Nice series for those not familiar with Unix’s original goals. One you missed, however, if that it was to serve as a documentation platform. Today’s fancy IDEs don’t have them, but Unix & friends do. You might want to consider addressing this and discussing the various documentation and formatting utilities that are available on any base distribution.

Note that while unix is indeed an integrated development environment, in the best sense of the word, it’s not without warts and has significant room for improvement.

For example, find (which you mention) is most useful when used with xargs. But xargs has a specification which is rooted in some technicalities of shell design and the most commonly useful case (treating text as newline delimited) is not implemented. Instead we have to choose between the default (whitespace delimited) and the special case (null delimited — at and one point, someone that had ownership of an xargs code base actually tried to deprecate this mechanism…).

There are other issues, also, having to do with compiler toolchains, or whatever else. And I think most of them are rooted in the viewpoint that a specification defines correctness, without any concern as to how well that specification relates to the kind of integration we have that makes the unix environment valuable.

Put differently, the free software community which implements these tools does not do much integration between tools — instead we seem to rely on inheriting the integration resulting from historical efforts.

Sorry for being late to the party: this post just popped on reddit and I read it. I use Linux as a development environment exclusively since Turbo C on DOS, which was my last time to touch an IDE. I never looked back.

That said, in my mind there is exactly one point that distinguishes an Integrated development environment from Other/Generic development environment: it is the integration between the text editor and the debugger. All other things: support for code management, build processing, etc. are secondary, an IDE will still be an IDE without them. But if you cannot set a breakpoint to the source line with the cursor, and when the breakpoint is reached the relevant area of the source does not appear on screen automatically, it does not qualify as an IDE.

Otherwise, a good read for the newcomers to Unix development, thanks for writing it!

If you’re a ruby/rails dev using unix as your IDE you have that with the pry library. You throw the line binding.pry into any line of your ruby code and you get a REPL right on that line as soon as you run the code. Bam! Instant terminal-based debugger.