Posted
by
timothy
on Wednesday February 21, 2001 @02:45PM
from the he-ain't-heavy-he's-my-code-gnome! dept.

Not long ago, chromatic wrote about one aspect of the quest to create software that doesn't suck. This time he's back with
a proposal that's simultaneously practical and idealistic. Namely: If you as a whiz at a certain language or other aspect of
programming want the world of programming to be better, you can help other
people become programmers, or better programmers, with well-considered instruction. I hope someone in the Computer Science department at a
high school near chromatic gives him a call, because he outlines here something more important than a "learn to program" curriculum.

Making Software Suck Less, Pt. II

Coders Code Whether You Like It Or Not

The insidious thing about Free software, the really subversive part, is that
it takes so little to start writing it. Anyone with a few tools, spare time,
and the wherewithall to start hacking can. Witness the long listings of
low-version-number IM clients and MP3 players on software announcement sites.
People like to code.

Every new project represents a coder who wants to write free software, but
chooses not to work with an existing project for whatever reason. How many more
people would like to contribute but don't know where to start? Hundreds, even
thousands of free software projects could use another coder, some testers, and
someone -- anyone -- offering suggestions and attaboys. For each veteran
programmer, battle-hardened and wizened by experience, a dozen novices spend
evenings honing their skills.

Mentoring Beginning Programmers

The obvious solution is to match availability with opportunity and
enthusiasm with experience. The free software community can produce better
programmers by giving new recruits mentors to emulate. It offers the
possibility for programmers to learn by improving existing projects, instead of
reinventing wheels. This doesn't require expert programmers. It takes people
with practical experience, patience, and the willingness to invest time in
another person's education.

Though this article draws from experience with free software, there's no
reason similar procedures could not succeed in commercial settings. Computer
science sophmores (and higher) would benefit from internship programs organized
similarly.

The Usual Suspects

Candidates for mentoring exhibit a combination of at least three different
characteristics. These are expressable as three different archetypes: the
self-taught hacker, the computer science student, and the new programmer.
Individual personalities and experiences also come into play. These are gross
generalizations, but serve the purpose of categorizing the types of information
to present.

Self taught hackers learn by experimentation. They dissect existing code,
copying and modifying implementations. Their technique gradually coalesces from
dominant influences. Hackers often amass a library to improve their education.
One danger for members of this class is that they may pick up questionable
practices from questionable code ("cargo cult programming").

Computer science students learn theory and are expected to teach themselves
various languages and implementations. (This category does not cover degrees
like 'information services.') The education covers ideal solutions, often emphasising
aesthetics and mathematical perfection. Students may not be exposed to practical issues in specific
languages and techniques, focusing on working code.

New learners enter the world of coding with a strong sense of need.
Management might pick a hapless victim for a project of undefined scope. A
home user may just wish to do more with her computer. People in this category
often don't know where to turn. They may not all grow into dedicated coders,
but can fend for themselves with some guidance and direction.

Besides these categories, recruits may include artists, musicians, writers,
and testers. Though direct mentoring from programmers may not awaken nascent
coding abilities, these folks are certainly welcome. Even experienced
programmers new to the idea of free software or to a class of software can
benefit from directed guidance. (Many people in this position just need to see
how to contribute.)

Course Requirements

Mentoring must provide examples of secure, clean, and idiomatic
programming. It should also include issues related to the practice of
programming -- time management, software design, tools and classes of tools, and
project leadership. Additionally, mentors should introduce their students to a
network of peers, whether on IRC, through mailing lists, or in user groups.

Benefits of Mentoring

Besides the warm glow of humanitarianism, being a mentor bestows personal
benefits. Your project immediately gains an extra set of hands and eyes. Each
new person brings a fresh point of view, with different expectations and
stories for the code and unique experiences. At first, the additional overhead
of explaining will slow progress; wise mentors invest slightly more time to
turn these lessons into improved documentation and tutorials. After a short
period of training, the project gains another person familiar with internals.

The discipline of explaining your personal technique, especially the more
reflexive elements of your coding style, will also improve your skills. ("Why do I do it that way?") Describing system
architecture and complex codes in words forces you to organize your thoughts.
It can even clarify future design decisions. Your student must be free to ask
questions and to challenge your assumptions. Either strengthen your arguments
or discover a better alternative.

Leaving the Nest

If you've mentored correctly, at some point you'll have taught every lesson
you can. The student will have met the right people and will have tasted the
fruit of hard work. Your example will still teach him, but he will make his
own way. You will be peers.

Prepare for this from the beginning. Allow the student to lead subprojects
with the freedom to make mistakes but the supervision to produce working
results. Provide opportunities for personal growth. Your job is to remove
obstacles from the path of enlightenment. Writing good software is hard, but
teaching people to contribute is very rewarding.

I think nobody teaches new programmers how to write their code so that other people can read it. This includes writing comments for everything you do, indenting correctly and creating variable names that actually make sense. Effective coding style makes it easier to debug and release code faster.
I believe that you can easily judge how long a person has been coding (or how well he knows programming) by looking at a snippet of his code.
MT

I've seen the database structure for slashcode. I know you need some help. I'm willing to offer it.

Our first lesson will be on denormalization for better performance. I'll try to be a patient and dilligent instructor, but the fact that you fetch the sig from the users table for every comment rather than storing it in the comments table may render that impossible. It's quite possible that I may have to beat you. Particularly when I look at some of the 3 and 4 tabe joins you do for really common operations. I know Hope College isn't much of a school, but surely at some point they explained that joins were slower than single table selects, right?

Anyway, I'm more than willing to help you with your database design and implementation problems.
Drop me an email.

Why not mention IE? IE seems to be stable and functional under Windows2k. I run Linux on one box and Win2k on another. They both have relatively similar uptimes. IE is by far the best browser available currently. Konqueror is nice, but I do not see too many plugins available for the linux platform. I use my linux box as a server and firewall. My Win2k box is my documentation/development box. Both work fine. I fail to see why everyone is so damn rabid about one or the other. I just use what ever works and gets my intended job done. I have no politcal ambitions, no hidden intents when I decide to pick up a piece of commercial software. I also have no "stick it to the man" ideology when I decide on using a piece of free software.

I really fail to see where and why all this politcal b.s. got started. It is not "us" against "them." There is no us and there is no them.

Another thing that big companies do, that open source developers often don't (flame gates are now opened, commencing in 5...4...3...2) is contributing resources completely and totally to bug fixing. Think about it this way, if a big company wants to use a program and they've commited themselves to it, they will lose a lot of productive time if their developers/users are held up by bugs. It is in their economic interest to hire people to squash these bugs so that their resources will be able to work w/o having to implement work arounds and things of that nature.

To your average user, the program crashing every once in awhile isn't the biggest itch in the world (its a bit annoying, but we can live with it usually because what we are working on is more of a liesure quality than a life and death quality). A big company, on the other hand, is in more of a life and death situation, thus this is not just an "itch" perse, its more of an inflamed rash that would be rather nice to get rid of.

Now, as far as contributing to mozilla, I guess that's a personal thing for each person... I could see it being a major world of suck if you had to download the entire source tree over a 56k connection (its not all that wonderful over DSL as it is), but that's the unfortunate fact about a product that is that big.

PostgresSQL has actually improved a lot, but the pricey overhead for new connections makes it unsuitable for client-server environments.

Why aren't you pooling your DB handles? The 90's are over, Shoeboy...
And speaking of Postgres, are you running it on MS or a un*x? Many open-source projects make unix assumptions, ie, that fork is a lightweight operation -- that could be the case here.

And if you're doing something besides an index search on a large table, stop.

Maybe. Are you using a rule- or cost-based query planner? Does the DB have sufficient table analysis for accurate guesses? You need to answer both those questions if you hope to get excellent performance -- blindly indexing everything is just a bad, bad call.

Actually, having a record company id is redundant data. Using the name as the primary key is the proper method. ID's are actually non-relational.

I'm heartily agreeing with you on this count. Particularly with Oracle8, varchar keys can be near as fast as integer keys. And if you can't stomach that (lots of people can't), the next best solution would be mapping tables.

In general, you shouldn't have to worry about DB stability *AT ALL*. If you want to worry about deadlocks, sure, be my guest, use MSSQL/Sybase, but no matter your choice, stability should be primary. The DB needs to stay up for indefinite periods of time; serve data to all comers, and keep it consistent. Your userinfo page testifies to your skill as a DBA; I don't see why you would settle for 'quite'.

I will take my current company as an example. We, the developers, are given ample time to seriously think about our problems and develop robust solutions. This method of development actually came about because they did it the other way the first time around.

My company went through the stage of "making it work" and ended up with software products that had to be constantly supported, patched, re-coded, etc., and it's costing them. So they decided to do it better the 2nd, 3rd,...,nth time around the track. Maybe companies do need to go through this stage to learn these things, a "trial by fire" sort of thing, but why waste the time and money on inadequate solutions when there's a better method just a few more steps down the road? My company realized it needed to spend more time developing better solutions, rather than spending less time developing any solution.

I'm just saying that there are companies out there who know that the time spent in development is worth the cost, in the long run. Of course, if they only need to get something out there today and don't care if they exist 2 years down the road, that's fine also... I just won't work there.

I started reading your comment, and when you said "document your program" I thought you meant to write an overview of how all the pieces of code interwork to make the whole work. i.e. line comments and block comments are not enough, there should be some structural documentation.

Then I realized that this is not what you were saying, so can I add this in as well?;)

In the mainframe-based transaction environment that I work in (modified TIP/1100 environment on a Unisys 2200), for example, all we can do is put octal and character snaps in the code and generate a trace file when executing the transaction.--
-Rich (OS/2 Warp 4 and Linux user in Eden Prairie MN)

I'm an embedded/DSP guy and I'd kill for a debugger. The only thing I have for debugging are outputs and inputs. Everything else is a black-box, with no way to see interactions within or between processing units.

I completely agree with the importance of "problem solving". I'm a "new learner" becoming "comp. sci. student" simply because, like chromatic says, "People in this [new learner] category often don't know where to turn." But one thing I can do -- I can algorithmicly solve just about every problem I've had to deal with. I've done some work with one of my friends, who is a Flash "programmer". I now know ActionScripting fairly well, but originally I would write the "psuedo code" and he'd implement it.

All of that to say...it's not easy, or at least it wasn't for me, to find a mentor. I guess I still don't have a mentor, I just finess projects untill they're *relatively* bullet-proof. For example, I just did an infix-to-postfix lab, in which we had to use stack implementation. After finishing the program, I fed it it's own source code, and it didn't crash. Now, that's not *elite* or anything, but for those of us that "drift", it's the best we can do.

It would be nice if there was a way to get in touch with people who want to mentor hungry students. If there is anyone who is interested, my e-mail is joecool12321@yahoo.com.

At my high school, last year I took part in a Cisco networking course, all about how to design, implement, and maintain networks (of course, using Cisco technology every step of the way), how TCP/IP works, structure of a packet, and other stuff.

While I lost interest right about where it said "ISDN is the prefered method of adding high speed internet connections to homes, remote sites, etc" and "Some users who are willing to buy expensive modems have 33.3 connections, but not everyone will...." but a lot of other students found it extremely useful, and it got them interested these fields.

(the other half of the class who decided it was just too hard for them and spent the rest of the year playing TetriNet)

That Cisco class would have worked out quite well, had the information been more current, but as it stands, I still learned a lot, and I hear things are better this year.

In short, it does work, and a lot of students are interested in taking advantage of Cisco's offer of $30,000/year right out of high school, or continuing to take Cisco sponsored coursework through college and earning more later on.

I think one major point which has been missed in the previous discussion was the aspect of debugging code.

I know when I started out, I rarely used a debugger when coding. I'd print out hard copies of my code, go over it with a pen, show it to others, but never touch a debugger. I know many professional programmers who also don't know the value of a good debugger.

My point is simple: teach aspiring programmers that no one can code perfectly, and show them how to debug their code appropriately. gdb is fine. ddd is better for beginners. There are other debuggers for other languages. I have no clue what can be used under windows or on a macintosh, but I'm sure there are good debuggers for them too.

One of the tenets of being a good programmer is writing solid code and knowing it's limitiations. Stress testing extreme conditions, and not being afraid of sanity checks. I've seen professional apps die from OBOBs (off by one bugs), come on guys, that should never happen, it's so easy to test for.

I know I've been babbling, so this is my last point. Teach people that just because a program compiles and runs without crashing, doesn't mean it always will.

This is because most programmers hate testing. Programmers see testers the way sysadmins view the first level tech support staff. As a direct result testing rarely gets a lot of attention. One of the basic rules of testing is that the extreme odd cases are the most important to check (and often the simplest). One of the signs of a really good programmer is how well they debug and test their code. The very best have lots of neat debugging techniques (like this one).

btw: Thanks for the breakpoint thingy. I usually check every branch but (as you discovered) the breakpoints will confirm test coverage. I expect similar results.

...and if you don't have a debugger put a unique print statement in each and check em off on paper until you hit em all.

Good point. MySQL is closer to access than it is to a real database. It'd be hard to do proper database development on MySQL just because of all the limitations. I wasn't aware that it didn't have the equivalent of page level or row level locking. That's awful.
Why do people insist that it's a real database again?
--Shoeboy

Anyone go into a Pep Boys about 4 years ago? I sure hope they're not using it anymore, but their cash registers ran a win 3.1 VB application... complete with 150x150 pixel "CHANGE" and "RECEIPT" command buttons. BARF!

Do you think that if we got enough of a mentoring organization together, we could perhaps turn the programming field into a guild or a union?

Now, rub your knees for a minute from jerking and hitting your desk bottom, don't think of union as a janitor's or autoworkers union, where most of the members work 2 hours a day, and pay a small fortune to the union bosses, but rather think of a group of like minded craftsmen, who only want to see their field prosper, who only want to pass along the best techniques to their journeymen and apprentices.

Perhaps a Guild is a better image. We senior programmers could pass along good programming tips and technics, as well as a good sense of programming ethos to younger, more impressionable programmers. We could instill a much greater sense of professionalism in the field.

Eventually, we could assert our power for the good of the field, perhaps modeling ourselves on the AMA. We could certify certain schools, and not allow programmers to be board certified if they didn't graduate from our chosen schools, and pass our chosen board tests. Really, you wouldn't let any quack operate on you, or any hack defend you in court, wouldn't you feel better knowing a professional, certified programmer was writing your web page?

I agree that there are so many projects out there that start because someone scratches their itch and then gets bogged down in finishing it and documenting it. Mentoring is a wonderful idea.

I am a fairly good coder and have a couple of projects going on. I think I would certainly welcome any help on those projects and would easily be flattered and appreciative of beginners trying to contribute. I think the problem is that people think that they must be 1337 coders in order to contribute to projects. So I say to beginners out there, come to my projects (see sig)! Help out! I'm willing to train with the fairly good skills I have.

Management has to be completely on board. Management often evaluates individual rather than team performance, and monthly rather than decade-long performance. Currently, potential mentors often get in trouble for mentoring.

With job changes about every year or two, it becomes more difficult for employers to measure and reward the effectiveness of mentoring (unless some royalty or residual renumeration were set up).

Finally, three women can't make a baby in three months. Saying "mentoring programming" is like saying "mentoring algebra". It takes time and practice on the part of the student to grok. Yes, it goes several times faster with a mentor, but there is an "aha!" that takes a while to acquire.

Without addressing these obstacles, the suggestion is merely just a start of an idea (and one that frequently comes up anyway in researched software engineering "process" and "culture" literature).

It seems like most people have missed the point of this article. It rightly asserts that the schooling the computer/software engineer/scientist has a big influence on the suckiness of software. As a software engineer who has been involved in recruiting at universities lately it is pretty tough to find good people.

One problem with Computer Science is students get little education in physics or technology. The more often receive all their computer knowledge from university and have little breadth of experience. Engineers on the other hand generally get only 2 or 3 computer courses, but have broader experience and interesy both inside and outside of school which more than compensates. I would advocate the transfer of CS departments from Math faculties to engineering as a result (this seems to be a growing trend). Note that certification programs like MSCE, etc. carry zero weight with us and are almost a negative.

The teaching of Java in school is particularly troubling. Students who mainly have background in Java often really don't have a depth of understanding on how the computer works. This results in bloated kludges in whatever language they work in.

Example - anybody can put together an HTML web page. But, it requires a real depth of knowledge and breadth of experience to architect a commercial web application which doesn't suck. This 'big picture' view is what I try to hire for.

Of course with most free software, since there is no one promised a product that withdraw their money if it doesn't happen there aren't many deadlines.

Except that tons of software is written as a kind of modern day duel. Why else would someone write a message board when ten already exist? To scratch an itch? Partially, but that only explains why they write it, not why they gift it. They gift it in order to win the acclaim of their peers.

In order to win said acclaim you must be current. The deadlines are therefore sometimes much more onerous because they are less tangible. More like a nightmare where you are terrified and running, but you don't necessarily know from what instead of a deadling where you are running, but it's only because your boss said so and he has no spine, no brain and no power. The worst he can do is fire you and then you'll just go somewhere else for more money.

First of all, that wasn't a spelling error, it was a typo. Secondly, I never claimed I was "above" anyone or that I anyone is inferior. What I stated was that there are not enough competent people in the field, which is a fact. If this were not true, then there would not be such a demand for IT people. I respect people and I expect the same from others. My point however, was that software sucks because the competent people get stuck with ALL the work (which is an enormous amount), while the incompetent get paid the same and do nothing 90% of the time.

Don't test me on this, I'm knee deep in it. And in the future, don't try to warp my words into me sounding like an asshole.

This is why every programmer should read a book like Code Complete. It teaches *agressive* defect catching in your code. Most programmers are too afraid of their code. They want it to run so badly that when it runs correctly once they are happy to sign off on it.

That makes no sense. That's a horrible mentality and it's far too prevalent in the field. Anwyays, any new programmers should read the book, it explains things far better than I could.

btw: Ironically, when I started programming all I ever did was use a debugger (Turbo Pascal 7 was excellent)... nowadays I prefer to read code. This avoids a problem I grew up with... hack and slash debugging. Often I would get an off by one error and do a quick fix and then run it through and it would pass. Often other bugs would slip by me or I'd forget to change affected variables, etc. By reading I am forced to take more time to verify the code integrity. Oh, and most studies show that code reading reveals about 3 times more bugs than does stress testing.

What the hell platform are you writing code on that doesn't have a debugger???

In some environments, the only debugger you have is printf() or the equivalent. How do you run GDB on an embedded system such as a cellphone or a game console [parodius.com]? How do you debug a fullscreen application (game, media player, etc.) if you can't see GDB's window?

Most professional programmers that I know of have never taken a single computer class...In addition, many professional programmers have no time to write 'good' code.

Gee, maybe if your friends took the time to learn about their chosen field, they'd come to realize that writing "good" code ultimately saves time.

That's why you don't see too many comp sci people making the big bucks as professionals.

Interesting...I've worked at companies like TRW, IBM, and Raytheon, and I think 95% of the professional programmers I worked with had CS degrees. (Yes, one can be a good programmer without a degree; however, without at least some classes, I'd say it's very very unlikely.)

Where are all these "professional" programmers who've never taken a class working? I'd like to know so I can be sure to avoid their products...would you go to a dentist who proudly claimed he'd never taken a class? Or even a mechanic who bragged about never taking even auto shop?

It takes people with practical experience, patience, and the willingness to invest time in another person's education.

To this you have to add that the person also has little else to do.

So, this is the flaw with the above. In any environment (other than an academic envoronment), the people who an actually do the work best spend their time doing the work. This is why we have schools, colleges, and universities. That is where the people go to get the "practical experience, patience, and the willingness", and they get it because they pay.

Mentoring is inefficient in a business environment because it reduces the productivity of two people to the average of the collective.(you have a 100% productive person and a 50% productive person producing at 75%. It is better to not have mentoring because your productivity doubles (150%))

Thus, in a business perspective, if mentoring happens, it is quickly realised that the solution is to put the mentor back to work, and in extreme circumstances, to replace the 'trainee' with a person who does not need mentoring.

Mozilla is A good web browser, not spectacular but decent. It is a spectacular architecture. the ability to write code for many different platforms, to expose that code to scripting languages so relative neophytes can code applications is extremely powerful.

A brief explanation of XUL/XPCOM/JavaScript:

Most of mozilla's core class's are built in XPCOM objects. Certain methods of these objects can be exported as scriptable, meaning they can be run from javascript code.

XUL is a layout language, similiar to HTML, that uses CSS to define it's appearence. Javascript can be embeded in XUL.

The result is an architecture that allows anyone to creat an application (chrome) as long as they can learn javascript and XUL. Heavy lifting can still be done in C++ behind the scenes and then exposed to Javascript. Dev time (once you get past the sizeable learning curve) is small. The resultant apps are cross platform and fully skinnable (because they're appearance is defined in stylesheets).

Everyone should check out www.mozilla.org and learn about this. It's a much better rapid development enviornment than VB, and it's cross platform, and open source.

... most of the time. Because there is not one person alone who can build a whole system, you need to do team work and dispatch responsibilities. You need some managers who usually are quite distant with technical realities (deadlines, budgets). You need techies who are quite distant with the real world (what do you need a GUI for ?). You need a nice working environment and what's more you need to communicate. Eventually, you need a customer who knows what he wants and have precise ways to describe what he wants. In addition, get me some heavy competition and so the problems start!

There are already different methodologies to enhance the quality of the work like Rational Unified Process or Extreme Programming which correspond to different project scales. Mentoring is very good if good techies were no so contemptful with junior programmers. None of these techniques will be enough to counterweigh the human factor though. This is why I believe that writing good software all the time is impossible.

Also, it is a complete misconception that good software is only a question of technique! A good software is a software which is widely used and that people are happy with. Customers, programmers, managers and marketing people are people before anything. And this is why I strongly believe that good software is much more a question of people and not of methodologies.

Look, for the biggest majority of the users, Microsoft Office is a very good product. For the geeks community, Linux is a good software. The notion of "good" is sooo variable...:)

I'd have to say my experiences have been different. We've had repeated problems with IE not handling standard stuff in anything like a normal method and being forced to do frequent upgrades to fix these problems as quickly as possible as well as writing our code around the bugs. Stability has been somewhat of an issue in IE and Win2k also. Last week for example we had to completely reinstall Win2k Pro and all apps on a machine just to keep IE from crashing. These machines are brand new (from a good company) and have very clean installs of their software and run almost nothing besides Office 2000 and IE so they shouldn't have much to conflict with. Under any version of Windows I've tried IE has been fairly sucky.

We also run MacOS 9 which has an entirely different set of IE problems but usually does somewhat better.

Netscape is just as bad as IE and is full of bugs on Windows, MacOS, and on Linux. While it usually handles http/html stuff better than IE it has less conformance with newer standards and crashes more often.

Konqueror is okay for a very light yet somewhat usable browser and may turn into something nice but so far I don't see it even counting.

Opera is okay but doesn't seem worth the cost to me and I'm not going to look at ads.

Mozilla is a little tricky as you still have to find a good build but there is plenty of community support for making this easier and usually it isn't that hard to find a good one that is recent. It is fairly full featured, standard compliant, fast, and stable on Windows, MacOS, and Linux. Even components such as Mail/News and the HTML Composser are shaping up really well. The fact that it is so cross-platform is also really good IMO. On the average day I work with Mozilla as my browser and mail program for around 10 hours and really beat the crap out of it and it rarely has any problem.

I don't care about 'sticking it' to anyone but I like software that doesn't suck and my experience has been that all software sucks and with commercial software you are stuck up a creek without a paddle. With open source I can either fix it myself or get online and get a fix often within minutes or hours. It isn't often closed source software has that good of turn around. I just wish I could find an open sourced bandwidth provider (whatever that would mean) cus companies are a pain to work with. "Ughh well we can have it done sometime a month after we promised." Of course some companies are better than others and so is some software.:)

In other words/. should have all the comments posted to another database and implement a once-per-minute table swap with the table used to display comments.

Agreed about splitting read and write tables, but why a swap? You'd have to synchronize the two tables in order to not lose comments, at which point there's not a lot of point in swapping them. Am I missing something?--

Might there not be other reasons for not trying to pick-up and fix someone else's work? Have you not considered that trying to fix someone else's code is a lot more difficult that implementing a feature from scratch?

I code professionally, and the hardest part of the job is trying to understand the mindset of the programmer who wrote this code originally. Most commercial code is worked on by a lot of hands, and it shows. You see the same problem solved in different ways in different places. Some of this may be because programmer B didn't know that programmer A had already implemented a function to handle the problem. It may be that programmer B did not like programmer A's implementation. How many times have you heard a programmer proclaim 'this code' sucks, only to replace it with something that is marginalling 'different'.

Regardless of the reason, being programmer N, I pick up this morass of code, and think that the whole thing needs to be thrown out and re-implemented.

With OS code, seeing as the programmers are volunteers, they get to work on what ever makes them feel good. What feels better, implementing a feature and actually making something work, or digging through someone else's obfuscated idea of code in order to fix an obscure bug?

In my opinion, new programmers would be willing to work on bug fixing if UNIX programmers would use real names for variables.

Don't bother with manual memory management unless you're writing OS kernels or device drivers. A good gc *will* do it faster than you, especially in a multithreaded program. If you *are* a memory management guru who knows how to optimize it better than any gc, you've probably written your own gc anyway and don't need this link. Otherwise, just get it, use it, and #define free(x) from now on.--

Apparantly you are talking about the default distribution on mozilla.org?

when it comes down to it, Mozilla is absolutely horrible as a web browser.

I disagree completely, it's good and getting better as they progress.

Everyone is seeking their own selfish gratification in being recognized for adding a new feature to a program, but no one is willing to try and make the existing features work properly.

That's a load of shit. Yes, there are some people that only want to add new features, features they want or need. That's the nature of an OSS project.

However, the VAST number of real engineers working on Mozilla are fixing bugs submitted by users and other developers. Take a look at Bugzilla if you don't believe me.

I'd like to see this issue addressed more often in programming circles, because something desperately needs to be done.

Herein lies the beauty. If you don't like the way Mozilla is being run, grab the source tree and branch off into your own group of coders that concentrate entirely on the current feature set, no bloat, and bug fixes only.

I think schooling can help a lot, if you are really serious about programming, then I think a well-schooled person generally grasps new concepts faster and adjusts more easily to different paradigms. Not to knock engineers, but I knew quite a few electronic engineers at university that were self-taught, and thought they could program really well, that they could program as well as the best comp sci students. After examining some of their code and their coding practices though, it becomes quite clear that their knowledge is not that broad (e.g. they'll say they know all about classes, when they don't really understand even virtual functions). The lack of schooling shows too; they will learn techniques slooowly, through trial and error, that are learned quickly when taught. I guess it depends on the individual though. I think some of the people I'm talking about were fairly arrogant, and would say stuff like "pah, computer science is such a joke, I don't need books and lessons to learn the stuff". Then their code shows that they haven't read books or taken lessons! But I guess if those same individuals were a bit more humble, and actually read some books first, then schooling wouldn't be so useful. So I guess if you *aren't* going to do a comp sci course, then at least adopt the right attitude:) Myself, the most useful things I learned at Univ. was NOT programming, but all the other things that I would not have had the initiative to pick up a book and teach myself, e.g. all the math, phsyics, electronics, AI etc. That stuff is pretty useful in my job.

This increases the load on the competent while the incompetent weazle by managing the build and doing other such simple tasks.

Do you really want the incompetent to be writing code?!? Seriously, though, not everyone can code. It's just a fact of life: I can't draw, some people can't write coherent sentences, some people can't spell (it's "weasel"), some people can't code. That's why we have job specializations like build management, release management, etc.

Just because you see someone doing a job that you feel is "beneath you", ask yourself where you would be if that person wasn't in there in that support role. Thank those people! I, personally, am glad that we employ a janitor/maintenance/facilities person. That means that I don't have to change light bulbs, vacuum the floor, etc. And I don't give him any crap because he can't code!

Don't teach them to program at all -- teach them to solve problems. After you have a sensible solution strategy, expressing it in the syntax of a given programming language is trivial (boring) work.

The problem that I see over and over and over again, among both students and "professionals", is that they sit down and start throwing code at a problem without knowing a solution strategy for that problem. And for many of these people, if you try using the Socratic method to bring out their solution strategy you'll find that they not only don't know what strategy they're trying to encode, but that they also don't even want to talk about strategies -- they want to talk about the details of the incorrect code they've already written.

I'm not against hacking, and I certainly don't think every program anyone writes needs to be supported by an engineering discipline, but I do feel, very strongly, that people need to think of programming languages as a tool for expressing a solution to a problem, not as being the solution itself.

I've noticed that not only do some people have a problem with documenting their code, but documenting their program, which in some ways is even more important. If you are writing your program just for you and maybe friends/coworkers, then you don't necessiarly need docs, since you can explain everything as needed. However, if you intend to release your program to the world, good documentation is a must. Prentend like your users are complete idiots (sometimes a valid assumption) and write your documentation accordingly. Throughly explain all the options, even if they seem a little obvious to you. If you have good documentation (as well as a generally intuitive interface) you'll find a lot more people using your program and a lot less requests for help with it.

On top of such suckiness is the competence of the coder. Not everyone in the industry is competent. In fact, I'd say that the vast majority are incompetent and that there is a select few who actually get the work done. This increases the load on the competent while the incompetent weazle by managing the build and doing other such simple tasks.

A travelling programmer came to the cubicle.
"I seek Elegance" he said.
"Why have you come here?" asked the Master.
"Because I have read much of your code, and it is elegant. I have come to
find the source of it, so that I may write good code also."
"Then you have come to the wrong place." said the master.
"But are you not the writer of the code?" said the programmer.
"How did you know the code was elegant?" asked the master.
The programmer was enlightened.

but surely at some point they explained that joins were slower than single table selects, right?

I haven't looked really hard at the slashcode, but I have to take exception to this - in some cases. A select is slowed down by joins, true, but its also slowed down by the size of your tables. The whole purpose of normalization is to reduce the size of your database by eliminating redundant data. Reducing the size of the data searched will then speed up the selects.

For example, if you are an online music store and you have 250,000 music titles in your database, but the vast majority of those titles comes from 7 different record companies, it doesn't make a whole lot of sense to repeat the record company name in each title row. I would join the little table to the big table and index the record company id in the title table itself. I do agree with your comments on the sig - that should just be written to the comment and then be done with.

I think slashdots problems are more related to mySQL's locking mechanisms. I have to deal with this sort of thing myself. IMHO, if you have an online database at all, you should completely separate the "read" database from the "update" database. In other words/. should have all the comments posted to another database and implement a once-per-minute table swap with the table used to display comments.

While I was out for a while I thought of a few more things to post that should have been included in the above.

While I don't think either of them were really overtly trying to mentor me, I owe a lot of credit for what I know and what I can do to a couple of brilliant programmers that I've had the privilege to work with. Both of these fellows are very kind, pleasant people and went out of their way to help me. They also both go out of their way to write correct code, as opposed to, say, just screwing around with it until it sort of works.

I met Haim Zamir [geboing.com] at Live Picture (now MGI Software [mgisoft.com]) in 1997 where I really began my C++ effort in a serious way (I tried it in 1990 to write test tools at Apple but didn't really enjoy the experience). Have a look at Haim's Resume [geboing.com], particularly under "Skills" where he lists:

Well grounded in disciplines of software engineering for correctness, robustness, performance, and longevity

Haim can write the most difficult code, and it doesn't just work right, it is unquestionable.

Another brilliant programmer is my friend Andrew Green [em.net]. Andy spares no amount of effort to get his code just right - he devoted nine years to developing the ZooLib [sourceforge.net] cross-platform application framework before releasing under the
MIT License [sourceforge.net]. (Not five years as I say on the page.)

If you think being correct, as opposed to merely working ok isn't important, imagine trying to get platform-independent reference counted smart pointers to work in a multithreaded application framework. Andy did.

For an archive of anecdotes of interesting, funny and sometimes tragic technology quality problems, please read:

Reading Risks for many years is what has made me such a zealot for software quality. It has also made me tend to avoid using software for anything of real importance in my life, and to feel uneasy while flying aboard modern aircraft - even though, with a B.A. in Physics, I am very comfortable with the aerodynamic principles that hold airplanes up and used to enjoy flying as a kid.

If you write software, another good investment (more important than your hardware investment), is buying and reading good books. As a software consultant I keep the canceled checks and receipts for my technical book purchases; in 1999 I deducted about $750 worth of technical books from my taxes and about $250 in 1998.

But there are a lot of bad software books out there; much as there was a gold rush due to the Internet, there was a smaller-scale gold rush for technical book authors over the last couple years. A really good source of straight-talking book reviews by people who have good reason to know what they're talking about is maintainted by the
Association of C and C++ Users [accu.org] at:

The ACCU is interested in more than just C and C++ these days, if you program in those languages, Java or (dare I say it) C-sharp you should join. The mailing lists is pretty low traffic and has some of the best signal-to-noise ratio of any list I've seen (except Risks). The ACCU's technical journals, with articles written by the members, are a valuable source of information on such things as how to write exception-safe code.

(Note to CowboyNeal - writing C-sharp with the pound sign set off the lameness filter, driving me damn near out of my skull. How about adding something to the preview to let us know which characters are lame, exactly?).

And good news for those of you across the pond (but bad news for me), it's a British organization and holds regular technical conferences. I believe they also send observers to the ISO standards bodies.

If you program in C++ you should read these two books by Scott Meyers and put them to practice in your code. Read each item one at a time and then go through your code from beginning to end to see how you can apply it:

After reading Effective C++ and More Effective C++, if you're programming with g++ (or using gcc to compile ".cpp" files) then you should use the -Weffc++ option to g++ to warn you about style problems. From the
GCC Online Manual [gnu.org]:

-Weffc++ (C++ only)

Warn about violations of various style guidelines from Scott Meyers'

Effective C++ books. If you use this option, you should be aware that the standard library headers do not obey all of these guidelines; you can use `grep -v' to filter out those warnings.

Importantly, in any language, make sure your code compiles cleanly without warnings with all the warnings enabled in the compiler - use the -pedantic option in gcc.

C++ is not the problem language it's often said to be if you follow Meyers' advice, but if you prefer C you certainly can have problems there too - and note that the preferred language for Gnome is C (while KDE is an extended C++), for C programmers you should read:

Back to the topic of compiler warnings, remember reading about lint in Kernighan and Ritchey's The C Programming Language? When I started out in my first real programming job, doing Sun system administration and writing image processing software back in the late '80's, I learned to write "lint" targets in my Makefiles, and I'd type "make lint" after editing but before compiling to actual machine code. This made my code much easier to debug and quicker to develop.

Much of lint's function is now available in the warnings of GCC (but I don't think all of it), but there are some proprietary products that will do extremely rigorous statis analysis of your source code. I haven't yet used either (although I plan to) but the two I know about are:

And speaking of web programming, many Slashdot readers write web applications (Linux being a "server OS" as they say). How many of you validate the HTML that's generated by the web applications you write?

Your HTML should work well in any browser [anybrowser.com] and it should be well designed for easy usability. I don't mean attractive graphics. I mean it shouldn't suck. Two links on design:

I think the main reason that programmers start new projects instead of contributing to existing ones is simply that programmers naturally just want to rewrite everything. Its more fun. They would rather rewrite something than use somebody else's component. I think when one is trying to use someone else's stuff, one tends to often think "this is not exactly how I would have designed this" or "I would have named this function differently" or "I can write a better class than this" etc. It often also *seems* easier to rewrite something than to learn how somebody else's code works. This problem can be alleviated by teaching programmers to be disciplined about documenting their code nicely. But quite frankly it is usually more fun to write something yourself (the fun part is solving the problems) than wading through documentation learning how to use somebody else's existing solution.

I believe the OSS world could benefit a lot from an attitude change, more people working together. Take a look at just how much effort is being duplicated just between KDE and gnome. So rather than having one good 3d modelling program that rivals 3DSMAX, we have 5 or 6 crummy ones. Rather than having one good development environment that rivals Visual Studio, we have about a dozen (ranging from crummy to mediocre). Rather than having one good DirectX-equivalent, we have about half a dozen (ranging from crummy to fair.) And I think it makes it that much harder for newcomers to join a project. For example, I'd like to contribute to building a Visual Studio equivalent, but quite frankly, I'm stuck in indecision as to which of the several contenders might succeed. It seems impossible to predict. Perhaps this also causes people to start new projects - they see that there are 5 or 6 *crummy* ones, so they think "we need a *good* one", and off they go starting yet another mediocre project. Gosh, I'm rambling now.

It doesn't always turn out like this though - some projects are clearly "the" projects in their class, e.g. Apache, GIMP. I wonder why.. ?

For quite some time before that I have been writing GoingWare's Bag of Programming Tricks [goingware.com], a collection of articles on the business and practice of programming. Of most interest in learning how to program well would be:

I worked on an article on C++ programming style that is not done yet, and has some errors in it. But I'll post the URL here with the understanding that you're to take is as a preview and not gospel truth:

Purify [rational.com] for Windows and Unix (but I think it doesn't support Linux)

If you use Java, you don't have to worry about dangling pointers, but you do have to worry about memory leaks (quiz: why do you in a garbage collected language?), deadlocks and so on. For that we have:

Don't let another day pass without availing yourself of one of these tools. I know most aren't Free Software and some are very expensive. I want to tell you that the money I spent on Spotlight is some of the best money I have spent on anything in my life. I wouldn't dream of shipping a Mac application to my clients unless it tested completely cleanly under Spotlight; on the other hand, the first time I'm given a client's code to work on, Spotlight usually reveals a multitude of sins.

Presumably, people who write Open Source code are doing it for their own enjoyment, which means they will tend to concentrate on the things they enjoy (adding new features) not the things they dislike (debugging someone else's code, documentation). This is one area where I feel the Open Source methods can be weak... there's no real imperative for quality control, since everyone's doing it for the enjoyment. A prestigous project may be able to enforce some quality, but for smaller projects, I can't think of any way to induce people to concentrate on the necessary but boring tasks as opposed to the fun (and maybe not necessary) tasks.

Most of you will read this suggestion and snicker at it. I know I did when I first heard it. I've since learned how blind I was.

There is one simple thing you can do to make your code a lot better. It takes a bit of discipline, but once you do it for a while, it becomes second nature. It is this: whenever you write new code, make sure that you walk through every new line with a debugger. That's it.

At first blush, this seems kind of like a "duh" sort of step. After all, most people run their code through a debugger to check it. The key to it, though is the phrase "every line". Put breakpoints in every branch of the code you write. When you get into that branch, remove the breakpoint. When you think you are done, check for any breakpoints that are left. I was extremely surprised the first time I did this. I thought I had been thourough, but I hadn't checked three different branches. And there were bugs in two of them.

What I have learned after doing this for a number of years is that programmers pay more attention to the important part of a loop or branch and ignore the fringe cases. What ends up happening, then, is that the fringe cases contain a proportinally large number of the bugs. Worse, because they are fringe cases, they don't happen often, leading to bugs that are hard to reproduce.

I agree with the concept and began doing this last fall, however I have only been able to find one student. I've approached teachers at a local university about sending me people who might want some free tutoring, but I was met complete silence. Does anyone have any suggestions for good ways to connect mentors and students?

If anyone out there in/. land is in the NYC Metro area and wants some help with Java, let me know.

I'd suggest all of you who teach at any level to encourage your students to contribute to an on-going open source application as their class project. I do this for my web enabled database development class for UMASS and I think if nothing else it's exposed a lot students to a cooperative method of developing software.

If you take this route you'll be rewarded and they will also as their project will live on instead of being part of the academic waste heap.

I also had my eyes opened- most of my students work in the "industry" but maybe 10% of them are aware of open source or what Linux, BSD et al are all about. If nothing else, you can use your classroom to spread the good word.

I think we should be teaching kids how to be better independent testers as well as better programmers...it will cost everyone less time and money in the long run if there is a good plan for independent integration test, requirement verification test and system test. Just think of all the X.Y.Z releases we could avoid by having robust testing during the entire sdlc...Unit test is fine, but you can't think of everything yourself.

Consider the quote from Vanders (found in Pt 1 of Making Software Suck Less)

"...why is it that so little OSS software goes through any sort of structured testing phase?

Sorry, that isn't a proper structured testing scheme. Beta & Alpha releases are no good other than for limited releases. If you don't know what uses your users are putting to software too, some features may never get tested until it is too late.

Developers seem to view testing as a neccesary evil, but are happy to release fix, after fix, after fix for eternity, rather than writing & following a test plan. Honestly people, it saves time in the long run! "

Over and over, we see people talk about programming and how to improve the coding that's being done. One thing that the current system emphasizes is that code that just works is OK. i.e. given ok input, we get good output

One problem I see is that we don't penalize people when they write something that takes bad output and produces unexpected results. i.e. how many of your programs can handle random input without crashing or worse? Got fixed length buffers? Ever think about what happens if the file your code depends on is suddenly changed by someone else?
This is why systems crash. This is why security is so poor in software.

The teaching system rewards code that works but doesn't try to penalize code that has extra unintended features.

For a troll, I have to agree with you. Open Source software very rarely seems to go through a proper build, test, fix stage. They just throw the software out to the users and let them find, and in a lot of cases fix, the bugs. This isn't good.

I'm a software tester by trade, and I code in my spare time. There are some cardinal sins that any peice of software can commit. They tend to be:

Not having a clear set of features that you want in your code for at least version 1.0

Not writing proper specs before any code is written

Not having testers, or having testers and not listening to them

Not having a defined roadmap or plan. At a minimum, this should cover the planing, and upto the second build of your software

Now on smaller projects, it's usually fine to sit down and start coding. With larger projects, you're going to be royally screwed about 50% ofthe way through your project, when you find you'll have to re-design part of your project to add functionality you didn't know you needed.

If you don't have testers (And no, outside of unit testing, the coder cannot teste their own code properly!) your software will be full of bugs you just don't know are there, until your users start using the software under normal conditions. You'll then find you need to recode large parts of your software to fix even small bugs.

Release early, release often, is good provided you have proper controls in place before your first release!

The problem that you speak of is not new at all, and is definitly not confined to the realms of computer programming. When it comes right down to it, nobody wants to do the dirty work that is required to get a project done (i.e. bug squashing, documentation, general project managements) , they would rather go out and do the new and fun stuff (which in this case is adding new features).

This is just a fundamental flaw in human nature. People want to get all the glory and credit and nobody wants to do the background work that is required to make something actually work. Look at politics... everybody wants to be president, nobody wants to be his aides that actually run around and get the shit done (well, some people do, but most don't).

This was also a problem in ancient Rome, when you actually look at it. Everything used to be paid for by donations from the rich folk. However, everyone wanted their money to pay for something big and wonderful that everybody loved (such as theathers and circuses, etc), but nobody wanted to pay for the smaller bits of infrastructure that held everything together, as they wouldn't be remembered eternally if they gave money for the roads to be resurfaced or for general maintenance of the aquaducts.

Its the same thing with programming. Nobody thinks they'll get any credit for the bug fixes that they work out that makes the product actually work, they want to be known for the feature which they implemented (even if nobody actually needs it or wants it in any way).

The first lesson I believe you should offer is the concept of "right tool for the job".

For instance, when you have a database that's supposed to be receiving many inserts (as say, one that accepts lots of comments for posts or changes to user profiles) it's a very bad idea to use database software which does full table locks.

My first real job out of college was with a small startup doing a turnkey system for medical labs on PDP-11s. I was an Astronomy BA with a couple of coding classes (FORTRAN, APL and PL/I) under my belt and a couple of summers working in the DP shop (COBOL, RPG II, and PL/I) of a Fortune 500 company as an operator and programmer. I'd read a few classic books on programming and software engineering. In other words, I knew enough to be dangerous.

I joined a team of 4 other programmers that were under tight deadlines to rewrite a medium-sized database and reporting system. Despite the time pressure, the lead programmer/architect made me do a several month long apprenticeship under him, where all my code was reviewed by him before released. All the programmers that joined the company, no matter their experience level, had to do one also. It made me a better programmer, and it had the side effect of forcing a common style on the entire programming staff, easing maintenance. As programming is essentially a craft, an apprenticeship is a good way to learn from a master. Just make sure you apprentice from a master and not a poseur.

I think you're forgetting something. Who are the testers of open source projects? The _users_. That's why they let untested software out to the users, because they _are_ the testers. If you don't want untested software, either (a) use well-commended versions of software, or (b), test it yourself. You might say "Oh, a commercial organization shouldn't have to test software itself!" But then you would be living in a dream world. Every medium-to-large organization has to test its software. If you pay millions of dollars to Oracle to get the latest version of Oracle Applications, then you, yes you, will still have to put in a LARGE amount of work testing. In fact, some releases of Oracle Applications are completely non-functional. AND THIS IS WHAT THEY ACTUALLY SELL TO END-USERS! So, as the user, you have to test free software. And you have to test proprietary software, too. Which would you choose?

The idea is that the would-be bard spends some period of time under the apprenticeship of one learned in the art. After acquiring some basic skills (melody, harmony, scales, etc.), and some examples of how to combine those skills (a repertoire of songs), the bard is then sent off to earn a living in the world. While doing so, the bard experiments with new ideas, merging them with the body of knowledge acquired from the master. After some time, the bard takes on apprentices, and shows them what they've learned from their master, and from their own experimentation.

All too often, what's lacking in the free software world of half finished IMs and mp3 front ends, is the "combinations" of those skills. A lot of these programmers barely understand pointers and event driven programming, yet they're building full GUI programs as a method of learning more about how to program. Frequently, they bite off more than they can chew...

There are too kinds of software that you'll find on download sites. Software that is intended to get the job done and software that is intended to help the programmer learn wtf he is doing. You'll very often find more of the latter than of the former, but each plays its own role in the programming community. If you see a piece of software that sucks, dont just delete it, email the author and tell him that it sucks. Let him know what you think is good and bad, so he can learn. Who knows, you may be helping out a future alan cox.

That's OK if you can see at a glance what the application does. The current system I'm working on has the logic so obfuscated by programmers trying to protect their jobs that it's quicker to just put in hacks and workarounds than to rewrite it - even though it offends my sensibilities as a programmer. The reason code sucks in the commercial world is almost completely due to poor project managers and butt-kissing incompetents. Remove these from the industry and good code will flow from good coders. Easier said than done, but chromatic should at least focus on project manager education as well as this is a large part of the problem.

A select is slowed down by joins, true, but its also slowed down by the size of your tables.That's more true for the length of your table then the width. As long as you're doing index scans rather than table scans, width isn't all that important. In fact, on systems that support page level locking but not row level locking, it's sometimes in your interest to widen columns.

The whole purpose of normalization is to reduce the size of your database by eliminating redundant data. Reducing the size of the data searched will then speed up the selects.

But (assuming that your database system uses b-trees to store and index the data) it won't speed index searches by much. And if you're doing something besides an index search on a large table, stop.

For example, if you are an online music store and you have 250,000 music titles in your database, but the vast majority of those titles comes from 7 different record companies, it doesn't make a whole lot of sense to repeat the record company name in each title row. I would join the little table to the big table and index the record company id in the title table itself.Actually, having a record company id is redundant data. Using the name as the primary key is the proper method. ID's are actually non-relational. It all depends on how you query your database though. Data warehousing schemas and oltp schemas are very different beasts.

I think slashdots problems are more related to mySQL's locking mechanisms. I have to deal with this sort of thing myself. IMHO, if you have an online database at all, you should completely separate the "read" database from the "update" database. In other words/. should have all the comments posted to another database and implement a once-per-minute table swap with the table used to display comments.

Yeah, I've been made aware of this. I took a look through the MySQL manual, and I'm shocked that people describe it as a real database. It's barely more advanced than Access. I personally recommend Microsoft SQL server for this sort of application. It's simple, fast, full featured, highly tunable, quite stable (versions 7.0 and above, 6.5 and below is a stability nightmare) and scales very well up to 8 processors. On commodity intel hardware, it's the fastest database there is. (numa machines don't count) Once you outgrow the capacity of a 8-way proliant, it's time for DB2 or Oracle.

It's a pity the free databases all suck so much. PostgresSQL has actually improved a lot, but the pricey overhead for new connections makes it unsuitable for client-server environments.

There are different criteria for sucking. Most of what the article goes into (which is what most of the Linux world is focused on) is suckiness that falls along the lines of security holes, crashing, bloated code, inefficient algorithms, etc. But there are other types of suckiness that exist

For example, to people who simply want to get their work done in an easy and efficent manner, software with an ill-designed, illogical, and confusing graphical interface sucks. To people who don't want to spend hours on end mainting their computer and updating and reinstalling things, software that easily breaks other software or easily gets broken by other software sucks. To someone like a cashier, software that was written by a programmer who has never been a cashier and has never processed data the way a cashier does sucks (take my word for it. I've been a cashier who's had to deal with sucky software. It sucks).

I think part of the thing that is really dogging open source on the desktop is that many of the people working on open source software have a very limited definition of "sucks". As long as software doesn't suck according to their personal criteria, the software doesn't suck. Maybe it's time that the open source world took a good long look at The People On The Other End Of The Software(tm) and incorporate some of their ideas of suckiness.

I've got a suggestion that I believe will help improve the quality of software, especially open source software. It seems that whenever an open source project is out there, coders seem to be overly focused on adding new features rather than improving/fixing existing ones.

Have you tried to use Mozilla lately. It's loaded with bloat and crappy features I'd never want to use, but when it comes down to it, Mozilla is absolutely horrible as a web browser. When the Mozilla project was begun, I thought that it was an noble idea, and was eager to start contributing code, but I soon realized that human nature goes against the open source model by seeking out self gratification and promotion, often against the benefit of the group as a whole. This is an inherent flaw in the open-source philosophy. Everyone is seeking their own selfish gratification in being recognized for adding a new feature to a program, but no one is willing to try and make the existing features work properly.

I'd like to see this issue addressed more often in programming circles, because something desperately needs to be done.

Interesting...I've worked at companies like TRW, IBM, and Raytheon, and I think 95% of the professional programmers I worked with had CS degrees.

In my experience, engineering and math graduates make better programmers than computer science graduates. The problem is that CS types know a lot about "using a computer to solve a problem", but not so much about "solving a problem, then implementing that solution with a computer". It's the latter that's valuable in the industrial/commercial world.

In my opinion, if you want people to be able to program, then you should think of your brother/sister or neighbour participating in the programming world. First the apprentice sees is the Presentation of the application, then it's the data they'll see, and lastly you could bother them with the algorithms used for the data-flow-processes. If they can handle this, bother them with quicksorts and hashtables. Can they handle that? Let them enter the world of Lex and Yacc.

What you can see is that levels are going from up to down. Not everybody is able to program at deep levels, but everybody should be able to make themselves useful. In order to do this, I think certain programming model (i.e. based on n-tier systems) should be made. People can focus into the level they want, and hop upwards and downwards at their own will. Perhaps a new programming model should be designed for something like this, I'm not sure it's possible...

I disagree on several accounts. First my company does consulting,mentoring, and education. The people who teach the classes also program and mentor. And the teaching pays better, so those who can teach at my company. Also the loss of productivity for mentoring may be like you described for a short period of time, but the person being mentored improves his effieiency, so there is an upfront hit in productivity but over the long run you gain as the student learns and no longer needs mentored.

We have had a great deal of success with our mentoring program. We have gone in to many companies, helped design architecture, mentor the team on OO methods and Java, and help with the first iteration. Those companies usually have us back when the have new products because the projects we started were successful

I wish that schools would teach how to properly document your code. All too often I download something from sourceforge, only to find it be a horrible mess of code that has perhaps 3 or 4 lines of documentation in the entire file.

Without proper documentation, an idea or program is next to USELESS to future viewers.

All the greatest ideas have been well documented..take the RFC documents as an example. They are very rigoursly reviewed and follow a very strict code of documentation. And they form the basis for all that the internet is today.

...they just write code that works, but don't worry about "elegant," algorithimically well thought out code.

Hmm. There's can be a vast difference between "elegant" code and "code that works". I've seen beautifully written, well thought out code - written by incredibly knowledgeable folks with CS backgrounds - that did just plain damn stupid things when you considered the hardware or OS. Things that resulted in programs running about 10% as fast as required, or using about 10 times more memory then available... but boy, it was sweet code to review:-/

OTOH, working code - code that does the job to spec (speed/memory), runs on the target hardware - can be just plain damn ugly. Not unused-variables-scattered-all-over-the-place and no-error-checking ugly, but convoluted, non-obvious, "we did it this way to squeeze a 2% performance increase out of the system" or "we did it this way to avoid an obscure error condition" type of ugly.

There's a time and a place for both types of solution. Knowing when you have to do something ugly is as important as knowing when you need to put the thought in and come up with something elegant.

Excluding multi-process/multi-thread timing conditions, politics of design, promises made by salespeople, and ill-formed specifications, this is a reason why systems crash.

My current project at work has a very fluid specification. I cannot tell you what is garbage and what is good for a particular module until the module is written and given for inspection to those who are designing the overall process to use the software. That means I have to make guesses about something I know nothing about so as to make a design that will work with something. All I have are guidelines. Should the guildelines change a week before delivery, in goes the fix--sans testing and sans rational design philosophy. In a word, I knowingly write shoddy code to fit political design decisions to meet deadlines set by ill-advised individuals who know not the first thing about how a computer operates.

Quite possibly, the single biggest problem with coding is the concept of a deadline. The second, almost because of the deadline, is the "I'll figure that out later" philosophy. Without a full blueprint of an application or system, (hell, I'd be happy with an accurate spec once in a while) it is impossible to make sound decisions. Ergo, bad software.

What you speak of are luxuries to your average code-for-a-living individual.

I tried to avoid the term 'apprenticeship' because it might bring to mind unpleasant connotations, and because it assumes that all people being mentored are less capable programmers than their mentors.

Interestingly, the Perl 6 community has had a similar proposal [mail-archive.com] since I wrote this article. The objective there is to get capable programmers familiar with Perl internals. It's possible to help out beginning programmers, too, but someone who already knows how to code fairly well could get into the guts of the regex engine or optimize pp_hot.c better with guidance from someone who's already been there.

(OT Sig comment: If I could pick a guitar mentor, it would be Michael Roe.)

Is partly due to the inexperience/naiveness of the coders. But another factor IMHO that i've noticed in my experience has been big is simply human laziness. So many coders I know don't document, "can't be bothered to", they just write code that works, but don't worry about "elegant," algorithimically well thought out code. You can never just sit down and begin to code. And please don't tell me you can, I can just sit down and crank out a buncha programs too, but I mean well thought out plans. ( I dont' mean the BS stuff like Requirement's Engineering, Software Architecture, etc. although that stuff is sometimes useful), but just to sit down and think about what you wanna do and what's the most efficient way to do it.

I'm not trying to be troll at all, so please don't think that. I'm just saying something that I've noticed. Plus I can't emphasize the importance of documentation. Currently I'm expanding upon a project that has a few thousand lines of code, and it's hardly documented. Hacking the code and trying to figure out what it's doing is tedious. Please document, it doesn't mean you are a bad programmer, actually means you are a good one.

I'm on several mailing lists, including ones
regarding oldskool music ripping. One particular
person had written a ripper in a high level language, and wanted to know more about assembler language so he could make it rip much faster. I quite happily offered to help teach him assembler.

He understood most of the assembler instructions, by my comparing them to the high level constructs, and showing how they are built. However, the point he got really stuck on was optimisation (which is
really the point of asm coding nowadays), particularly memory access. He didn't see why accessing unaligned data was bad. So I showed him how memory is logically addressed versus the physical bus requests the CPU has to put out. I think this was the straw that broke the camel's back. He said 'thanks, but I think that's too hard for me', and he's now a Visual Basic coder. (No, really. This isn't a joke.)

"Skinnable" sounds a lovely idea. Then you find that it gives you no consistent way of using an application. Borrow a friend's PC, and the button which prints a page on your version may close his window, or bring up his favourite pr0n page.

"Cross-platform" sounds a lovely idea. Then you find that every program you write runs like treacle. Cross-platform at _compile_ time is reasonable; at _run_ time is just plain ridiculous except for some ultra-specialised niches (eg. Web applets). Computer scientists have gone to the "cycles are cheap" model - ESR's coding guidelines explicitly say this. Back in the real world, performance is important! A 0.1s delay for a menu to open is acceptable, a 0.5s delay is not. A search taking 10s is acceptable, a search taking a minute or so isn't.

And why these features? Bcos it's a "beautiful thing" for a _coder_. No mention ANYWHERE of the user. Most users WON'T change their app skins, WON'T run the application on 3 different platforms at once. Most users just want something that works. If they have to download it, they want it to be a fairly small code size. If they have to use it all the time, they want it to work at a reasonable speed. A typical computer scientist works at a university (student/postgrad/lecturer) and runs his programs on a mainframe, or works in business and runs his programs on the latest hardware; they're likely to have a hot PC at home too. A typical home user buys a low-end PC (maybe 300 MHz) with a small amount of RAM. It's not acceptable for the computer scientist to say "Well it works fine on mine". You end up with exactly the same problems of platform-dependency in that a percentage of users just can't run it; this time though, the dependency isn't on architecture/manufacturer but on how much you can afford to spend on your PC.

This gives the odd situation of an "upper-class" program. Only those who are rich enough can run it; if you don't have the money to buy a high-spec machine, you get a definitely second-class performance out of the program. And it keeps the hardware manufacturers rubbing their hands with glee.

Mozilla was intended to be a replacement to a buggy, bloated and over-featured browser. What's emerged is a buggy, bloated and over-featured browser. Tell me again why anyone bothered?

The haves and have-nots will become even more sharply divided in the years to come. Technology is quickly becoming a requirement for work experience. But how many lower income area schools can afford technology, let a lone instructors skilled in teaching that technology?

I have had this idea for a while, but it is taking me longer to implement it.

Enter a non-profit/profit company combo. It teaches, instructs, and mentors young under-privileged children in IT skills (specifically programming). It then turns around and seeks business support from the community: i.e. a local business has the company build their website for a cut rate, and the company uses the local kids exclusively for the project. Kids then get work experience and knowledge. The key focus being training the kids and getting them work experience--not necessarily profit. It wouldn't be the latest greatest way to make millions.

Perhaps I am idealistic AND stupid, but this seems like an effective way to reverse the tide a little bit. Local programmers could volunteer time for instruction, whereas someone would have to work full-time for the company to take care of project management.

Rule 0: Don't break a rule unless you have a very good reason for doing so.

Rec: Make sure a project manager is dedicated to your project.
Rec: Have regular project meetings, typically once a week, with all participants, where all kinds of project issues are discussed.
Rec: Manage risks (by using the likelyhood * severity formula)
Rule: Involve the user in the process. (This is the single most important reason why projects are successful).
Rec: Select a set of tools that works well, document its use, and make it the default tools for the project.
Rule: Use a version management tool, such as CVS, to handle your code and documentation.
Rec: When checking in code, write meaningful descriptions for the changes.
Rule: Before developing a completely new product, make sure you can't buy a similar product, or use an open source product.
Rule: Use short release cycles.
Rec: Create Use Cases (or stories) for how features are used by the customer.
Rec: Let developers estimate their own tasks so that they are realistic.
Rec: Use the Stories as the basis for writing test cases.
Rec: Write test cases for new features before you implement the feature itself.
Rec: Make sure test cases are easy to configure, run and automate.
Rec: Automatically run the test suite after each build.
Rec: Update the list of desired features after each release cycle.
Rec: Write a Release Note after each release, where all known issues are listed.
Rec: Use a programming standard.
Rec: Use a code style standard.
Rec: Review eachothers code.
Rule: Don't leave commented out code in the files, unless you explain why it is there.
Rule: Check in files in the correct order, so that the risk of corrupt intermediate checkouts by your coworkers are minimized.
Rule: Make a diff on your changes before you check them in.
Rec: A function should do one thing and one thing only.
Rec: You should never make. You should always "make clean;make".
Rec: If you've changed a file, don't ever close it until you've checked it in, since open files are a good way to remember which files you've edited and needs to be checked in.

Everyone is seeking their own selfish gratification in being recognized for adding a new feature to a program, but no one is willing to try and make the existing features work properly. I'd like to see this issue addressed more often in programming circles, because something desperately needs to be done.

I agree with you about the problem but there's no reason why debugging, usability improvements and documentation have to be considered less important contributions than adding new, marginally useful features. The celebrity-worshipping pecking order that's built up around free software makes that judgement. Slashdot raves about shaded transparent pull-down menus and the guy who added them - not about the person who translated the app into Japanese, wrote the documentation or made the interface more intuitive. If that situation were reversed, you'd see a lot more people doing translation and documentation and fewer stupid features like text preview in file manager icons.

Everyone knows who Miguel is - how many people can name the heads of the KDE and Gnome documentation projects?

I think your first paragraph pretty much summed it up. I've been asked many times to modify some existing in-house application to perform some new function. After barfing on the horrid spagetti code of "programmers" who came before me, I usually decide that it would take more time to figure out the mess than to start from scratch.

We usually end up with a much tighter application that does a better job than the original did and with the added functionality they wanted. I even spend time doing ---documentation! Each incarnation improves on the previous.

Of course, I wouldn't suggest throwing out a project consisting of a million lines of code (unless absolutely necessary;) But for the size and scope of the apps that you mentioned, sometimes it's for the best.

I mostly work on a DSP-style CISC processor that requires all pipelining to be handled in the firmware. Due to our whopping 1K of code space and 1K of data space, a compiled language is simply out of the question, everything is hand-tuned assembly. Now, I could make this code elegant and readable, but it would be absurd: at least 30% of the code would consist of no-ops doing it the way my CS professors told me was The Right Way (tm). So my code ends up being a horrible convuluted, out-of-order mess with weird dependencies, self-modifying parts, and comments that attempt to describe the indescribable (mostly we end up saying fsck it, and add "Do not change this chunk of code! Ever! Please!":). But it fits and it works. Making it work is key here, the constraints are physical, not arbitrary.

On the other hand, if I need to add a previously unused instruction to our assembler, which runs on a big-ass Sun with 4GB of RAM, I forcibly re-wire my brain and put it in there with intuitiveness, elegance, and comments that point out exactly what the code is doing so that someone else can maintain it. Again, making it work is key, but there are no physical constraints here, and while I'd like my code to assemble 10% faster, its just not worth losing proper functionality and ease of maintenance over.

However, regardless of what I said about the former instance, the vast majority of free software doesn't have physical constraints to deal with. Any constraints the authors place on themselves are arbitrary, and when it comes to writing code, the arbitrary constraints should be ignored until it can be demonstrated to run properly (correct functionality, no memory leaks, etc). Then, after it works, go back to look at what the performance goals were, and decide which are worth pursuing.

You seem to have focused on the 'scratch the itch part'. If it isn't acclaim (and, of course, I didn't invent this idea. ESR popularized it in our culture) that drives people the release as I claim, then what is it in your explaination?

Hi,
Your comment on denormalization demonstrates a lack of real world experience. Denormalization on databases where reads massively outnumber writes (which is slashdot) is a very useful thing.
As far as making it difficult to make future modifications goes, this is complete B.S. As long as you encapsulate your data retrieval and manipulation through stored procedures (which slashcode doesn't do) all changes to the database schema are easy to make and very few modifications to front end or middleware code are ever required.
The simple fact is that no perfomance critical system should ever run in 3rd normal form. Period.
--Shoeboy

You're right, the users are the testers. But my point was, that there is no structured testing done.

Actually in almost every single Open Source project that I can think of there is the equivalent of the "unstable" and the "stable" branch. The unstable branch generally has some nifty new (but untested features). The users that actually need those features excercise them (often in real world conditions). When it looks like it works as advertised it gets rolled into the "stable" branch.

There is no need for a more formal set of tests for the simple reason that the people testing the software are generally in close touch with the people writing the software (ie, there isn't a focus group in the middle). In other words the features generally get specified due to specific needs, and testing against these needs is very straightforward. If you have features that no one is willing to test, then you probably should consider removing those "features" for the good of the project. Clearly they aren't useful.

Now, the best projects do have a set of regression tests. That way you can excercise the various components of your software individually and as a whole. This is quite helpful to make sure you aren't introducing any new bugs in your software.

Why else would someone write a message board when ten already exist? To scratch an itch? Partially, but that only explains why they write it, not why they gift it. They gift it in order to win the acclaim of their peers.

Sure, words of praise are nice, but that's hardly the only benefit from giving.

If you've written mainly to "scratch an itch", there's little reason not to give it away. You can gift it, try to sell it, or keep it for yourself. Keeping gains you nothing (unless you've written The Program That Will Enable You To Take Over The World); and as for selling, the population of people who have the same itch and are willing to pay to have it scratched could be very small.

When you've put time into something, you want it to be seen and used. I've spent years working on projects that never amounted to anything except a dusty tape on someone's shelf - that sucks.

If you gift it, they will come - maybe only to snag parts for other program they're writing, maybe to take what you've written and keep it alive and growing. A program that only I ever use is like a poem that only I ever read...I don't share my poetry primarily for acclaim, I share it because I'm wired to delight in infecting others with my memes.

ALmost every programming language and application out there has a community of some sort.

Being involved in the community and being able to hands on talk with people who have been doing it a long time and have these gurus respond to you the little guy and see how they would do something can give a new programmer more insight in one email post than four chapters from a book.

Learn about the thought process from a brilliant mind and how to analyze a situation rather than learn about a specific way a book says to do something and again you have just made huge leaps and bounds over where you previously were.

Enthusiasm can take you a very long way, never giving up persistence and above all studying others works.

Its just like at chess, if you always play people worse than you your not learning much, whereas if you play someone who beats you in three or four moves the first time you play them you must work that much harder and you will become a much better player as a result. Just like having a mini Linus Torvalds with you as you write a new kernel module the benefits are quite obvious..

you will find that langauges and platforms with the etter communities can some of the time attract developers much easier than other language.