You are heading towards exactly the same style of
commenting that lead to one of the absolute worst
monstrosities I have ever had the displeasure of working
with. You doubt that anyone has ever complained about
having too many comments. Well I have, and many excellent
programmers that I know have, and your commenting style is
exactly the kind of thing that makes me (based on painful
experience) cringe.

Please do yourself and the next programmer a favour and
pick up a classic like Code Complete or The Pragmatic
Programmer and learn about the real maintainance problems
that synchronizing code and comments will introduce. They
are real and severe. Instead of having 2 documents (and
creating a tremendous amount of work tidying up comments
that experienced programmers know not to trust) attempt to
have your code stand as a comment in and of itself. Take
some time to break it up, structure it, use meaningful
variable names. Use meaningful subroutines. Don't use
globals.

I am sorry that I am coming across like a ton of bricks,
but rather than trying to (in your words) shoot down
arguments against overcommenting, you should try to
understand them. What is the job of code? It is to
instruct the computer in a way that a human can understand,
and that humans later can figure out, refactor, and reuse
with a minimum of work. Now when you run the program you
can verify what the computer has been told. If you change
it and run it, you can test whether the computer did the
right thing. However if you change the human instructions,
you do not find mistakes until after a human gets confused.
And if you need to change what the instructions are, then
you need to change both. In reality this does not happen.
And trying to synchronize the two documents will be a
constant real source of problems.

The solution is to, as much as is feasible, have the code
itself stand as a comment. As merlyn pointed out, one of
your (bad) examples of how to comment uses a variable $x
and then in the comment explains what $x is and what you are
doing. I cannot say strongly enough that I do not want
to ever work with code that is written like this!
If instead you took half the energy that you are currently
wasting in writing and then formatting comments that
experienced programmers know not to trust and instead spent
it in structuring your code and giving your variables proper
names, then your code will have a much better chance of
being understandable to maintainance programmers. I should
know what they want - after all I spend a lot of my time
doing maintainance on things that other people wrote. :-)

For an example of what I mean take a look at Text::xSV.
I wrote that last night in a couple of hours. Were it
only meant for internal use, I would probably be missing the
pod section, and the API comments that currently appear in
the pod would be in front of the various functions. What
I find interesting is that though many people have disagreed
with me about commenting, they usually also say after seeing
my code that they would have no problem working with it.
Given that, I think that my opinions on how to make code
easy to work with are worth something...

Also note that I don't actually write uncommented code.
But my comments are about different aspects of the
code than the mere mechanics of what each line is doing...

In my own defense, I would like to make it clear that I would never actually use $x as a variable name in a program. That was just for a quickie example. Any good programmer would do as you say with variable/subroutine names, no globals, and good structure. I did not make that sufficiently clear in my post. In no way can commenting, however good, make up for bad code. Good code always comes first, and good comments come second.

However, I still contend that the good comments must, in fact, come. I, too, have been stuck doing a lot of maintenance work, and never have I come across any code so clear that it negates the need for comments. Your assertion that code can stand as its own comment seems unrealistic. Perhaps you just write much, much better code than anyone I've ever encountered :)

Yet in my experience, the code is never sufficient. Just a few days ago, I came across some code which confused me. I even wrote to perlmonks about it. As it turns out, my understanding of the code was flawless, but I still didn't know what it did, because the author had given no indication of why he did it.

I also took the time to peruse your example, and I must say that it seems a far better proof of my point than yours. When you read it, do you suppose that if you were not the author that you would instantly say, "Oh, of course! This code reads text data separated by an arbitrary character!"? No. But by simply reading the abstract which you so thoughtfully included, the reasoning behind the code is immediately revealed. Now, I don't know for sure if that code works as written, but I can guarantee that if I had to test or debug it, it would take at least twice as long to do so without those comments.

Also note that it was those style of comments that were what I was attempting to advocate earlier, as opposed to play-by-play commentary on the mechanics. I also would like to apologize if there was any hostility creeping into my tone. I've simply wasted way too much time shoveling through the crap of otherwise good programmers who proclaimed their code self-explanatory.

If you peruse my example and walked away from it thinking
that it stands as an example of why to comment every line,
use pretty formatting of comments, put comments at the
end of lines where they have to be adjusted by anyone
making changes to the code, etc, etc, etc - then you
missed the point.

My last paragraph of what you were responding to said:

Also note that I don't actually write uncommented code.
But my comments are about different aspects of the code
than the mere mechanics of what each line is doing...

Stare at that carefully.

The sample commenting examples that you gave resulted in
comments whose formatting would be a maintainance issue
that so closely duplicate what the code says that there is
little added value. Secondly the very verbosity of the
comments led you to use a useless variable and a confusing
comment. You claim you wouldn't use such a variable name
in practice, but I only have what you wrote to judge by.
And in what you wrote you used a useless variable name and
a useless - confusing - comment.

The sample code I pointed you at is commented. But it is
commented in a completely different way and style. Plus I
can tell you how that style would change in a slightly
different circumstance.

I comment. Yes. I comment intent. I highlight key points.
I do not explain my algorithms. I do not
explain my variable names. I do not produce a
separate document for the human and the computer. And
above all I have a commenting style that is not
going to be a lot of work to maintain.

So if you need to change the code and make it do something
else, or if you need to understand the mechanics so you can
do something similar, then you will need to read the code.
Humans should be able to figure out how the code does
what it says it does.

Now you claim I misunderstood you and you never meant to
advocate play by play commentary on the mechanics. But
the examples that you gave were play by play commentary
on the mechanics. The statement you made (that I cannot
disagree strongly enough with) is that there is no such
thing as overcommenting. The formatting of comments that
you advocated poses a serious maintainance problem. You
claim that overcommenting only accounts for perhaps
0.1% of comment related problems out there.

I am sorry, but I cannot reconcile these two points of
view. What you are saying now does not fit what you
showed as an example of what you think people should do.

I have dealt with problems due to overcommenting, poorly
maintained comments, comments that only make sense if you
are in the original author's point of view, comments that
cover for poorly thought out code, comments whose formatting
makes code hard to tinker with...and most of these problems
would have either not been there or been lessened if the
original programmers understood that not all comments are
good.

In fact for every one of those items I either saw them
in your examples of how to comment, or I saw you doing
things that have lead to these errors in code that I have
worked with.

Now you claim that the style of commenting that I
demonstrated is what you were attempting to advocate.
Well it certainly isn't what you describe. So let
me explicitly explain the style that I follow:

Every function that is part of the public API gets a
brief description of input, output, and action. If the
function is not documented, it is not public. If the
function name starts with an _, it is probably not even
(usefully) callable.

The overall package gets a brief description of
what it does.

If the package is somewhat private (ie for me and
people I work closely with), documentation goes inline
in front of the described function.

If it is to be more generally used, documentation
goes into pod.

Key conceptual points may be highlighted with comments.

Function names describe what they do. If that means
long function names, so be it.

Variable names describe what they do. For instance
flags should be questions which the current value is
the answer to. If the meaning of a variable is unclear,
then it needs a better name, not a comment.

Variables are not documented unless they are
part of the public API. Documentation of variables is
to inform others what may be used, not to explain what
the variable does.

The formatting and naming scheme is consistent.

The formatting of comments is chosen to not cause any
issues upon maintainance of code. This generally means
being on separate lines from code, and means a minimum
of fancy formatting.

The mechanics of the code are not documented
unless it uses a non-obvious API. In that case the
other API is commented about inline.

Whenever reasonable, large functions are broken into a
series of smaller functions so that the body of the
function reads as a series of hopefully self-descriptive
function calls.

Wherever I spot a significant opportunity for error,
I place an error check with an informative error message
in case things go wrong. The error message is intended
to serve as a useful comment both upon reading the
source and during any failure condition.

Functions are grouped into modules. Within a module
functions appear in alphabetical order. (This is a
personal style settled on after finding that initial
attempts to divide by what the functions did inevitably
broke down when at different points in the project I
thought differently about the code.)

I describe this as being, as much as is feasible, the code
standing as its own comment. What cannot be done by
making the code clear is commented. Whatever can be done
by making the code clear, is.

A better way to put this might be that I do not comment on
the code. I comment on the usage of the code and I comment
on other code. I comment on desired changes, I comment
about what is critical, and what the external
dependencies are. But the code at hand should be
readable and should stand as its own comment.

Now with all that in mind, please go back and read my
example again and compare to this detailed summary. Does
my code not fit my description? Does my code need detailed
comments about how it works? Do I, despite that, comment
in a useful way while requiring a minimum of work to
maintain the comments as the code changes?

Also note that I don't actually write uncommented code. But my comments are about different aspects of the code than the mere mechanics of what each line is doing...

Stare at that carefully.

Then please do me the same favor. In my first post, I wrote quite clearly:

The "why not what" model of commenting is a really good idea

I had already expressed my agreement with the previously stated idea that comments should focus on areas other than what most people comment on. The reason I wrote what I did was that some people's advice "don't comment badly" was being expressed as "don't comment." I merely wanted to reassert the importance of comments for future developers.

Secondly the very verbosity of the comments led you to use a useless variable and a confusing comment. You claim you wouldn't use such a variable name in practice, but I only have what you wrote to judge by. And in what you wrote you used a useless variable name and a useless - confusing - comment.

Really? I also wrote this:

###########################################################
#
# Here are some comments on the following code.
# It does stuff.
# Its cool.
#
###########################################################

Do you suppose that is an accurate reflection of the type of comments I would put in real code? If you're going to use what I write to judge by, use all of it. I wasn't attempting to give examples of "real" comments and their content, because like I said that had already been expressed, and I agreed with it. I was trying to point out that A. comments are necessary and B. you can find ways to stick them in clearly and readably wherever they may be necessary.

Now you claim that the style of commenting that I demonstrated is what you were attempting to advocate

After reading your subsequent list of style points, I see that we are farther apart than I had thought from viewing one small piece of code. I would like to say that I still agree with the content that you chose to include in your comments. However, there were a few areas that I took issue with. Out of politeness and benefit of the doubt, I had assumed that these were merely oversights, but after reading your style-guide I can see that these were conscious decisions, that I would like to comment on (no pun intended).

If the function is not documented, it is not public

Are you saying that you never document a function that is not part of a public interface? If so, that is severely negligent. The purpose of comments is not solely, or even primarily, to describe the interface. Comments exist for the benefit of people who might need to maintain or modify the code. There is no reason to assume that because a function is not part of the public interface that it is magically immune to the need for update.

If the function name starts with an _, it is probably not even (usefully) callable

OOC, I've never seen this before, is it commonly used? If not, then how is that useful for other people?

Function names describe what they do

Really, so get_row and _get_row do what, the same thing?

Variable names describe what they do

Hmm, let's see. In one short example I find the following variables: $lookup, @data, @allowed, $q_sep, $match_sep, $expected, $piece, $req, $opt, $default, @res, $bad, and $sep. So in some 200 lines you managed to declare 13 variables that give no indication of what they do or why they were declared.

Now, I have to say that I more or less agree with the other points you mentioned. And I really would like to reiterate that I'm not trying to be a jerk, or to attack you personally (I say this because I know I have a very aggressive argument style, which doesn't reflect my emotional opinions of the person I'm arguing with--really!). What I'm trying to point out is that every programmer's own code always makes more sense to him than it will to anyone else. The variable and function names that make oh so perfect sense to you now mean nothing to the next guy who comes to read your code six months later.

And all of the points that you cautioned against are valid and accurate. It is important to make sure that comments are not so specific that they need to be updated continually with every minor code change. It is important that every comment should be written to add something useful to the reader's understanding. "There is no such thing as too many comments" was obvious (or so I thought) hyperbole, attempting to make the point that every programmer since first we did away with punch-cards has always thought he needed less comments than he did.

You are obviously a good programmer, and obviously a better one than me. However, that is part of my point. SE4's write the programs that SE1's get stuck trying to maintain. The threshold of what qualifies as "non-obvious" varies greatly. You seem to code under the impression that whatever makes perfect sense to you will make just as much sense to everyone else.