October 1999 Archives

Short guide to DBI (The Perl Database Interface Module)

Relational databases started to get to be a big deal in the 1970's, andthey're still a big deal today, which is a little peculiar, because they're
a 1960's technology.

A relational database is a bunch of rectangular tables. Each row of a table
is a record about one person or thing; the record contains several pieces
of information called fields. Here is an example table:

The names of the fields are LASTNAME, FIRSTNAME, ID,
POSTAL_CODE, AGE, and SEX. Each line in the table is a
record, or sometimes a row or tuple. For example, the first row of the table represents a 30-year-old male
whose name is Karl Gauss, who lives at postal code 19107, and whose ID
number is 119.

Sometimes this is a very silly way to store information. When the
information naturally has a tabular structure it's fine. When it doesn't,
you have to squeeze it into a table, and some of the techniques for doing
that are more successful than others. Nevertheless, tables are simple and
are easy to understand, and most of the high-performance database systems
you can buy today operate under this 1960's model.

SQL stands for Structured Query Language. It was invented at
IBM in the 1970's. It's a language for describing searches and
modifications to a relational database.

SQL was a huge success, probably because it's incredibly simple and anyone
can pick it up in ten minutes. As a result, all the important database
systems support it in some fashion or another. This includes the big
players, like Oracle and Sybase, high-quality free or inexpensive database
systems like MySQL, and funny hacks like Perl's
DBD::CSV module, which we'll see later.

Those are the four most important SQL commands, also called
queries. Suppose that the example table above is named people. Here are examples of each of the four important kinds of queries:

SELECT firstname FROM people WHERE lastname = 'Smith'

(Locate the first names of all the Smiths.)

DELETE FROM people WHERE id = 3

(Delete Mark Smith from the table)

UPDATE people SET age = age+1 WHERE id = 247

(William Hamilton just had a birthday.)

INSERT INTO people VALUES ('Euler', 'Leonhard', 248, NULL, 58, 'M')

(Add Leonhard Euler to the table.)

There are a bunch of other SQL commands for creating and discarding tables,
for granting and revoking access permissions, for committing and abandoning
transactions, and so forth. But these four are the important ones.
Congratulations; you are now a SQL programmer. For the details, go to any
reasonable bookstore and pick up a SQL quick reference.

Every database system is a little different. You talk to some databases over
the network and make requests of the database engine; other databases you
talk to through files or something else.

Typically when you buy a commercial database, you get a library with it. The vendor has written some functions for talking to the database
in some language like C, compiled the functions, and the compiled code is
the library. You can write a C program that calls the functions in the
library when it wants to talk to the database.

Every vendor's library is different. The names of the functions vary, and
the order in which you call them varies, and the details of passing queries
to the functions and getting the data back out will vary. Some libraries,
like Oracle's, are very thinthey just send the query over to the network
to the real database and let the giant expensive real database engine deal
with it directly. Other libraries will do more predigestion of the query,
and more work afterwards to turn the data into a data structure. Some
databases will want you to spin around three times and bark like a chicken;
others want you to stand on your head and drink out of your sneaker.

There's a saying that any software problem can be solved by adding a layer
of indirection. That's what Perl's DBI (`Database Interface') module is all about. It was written by Tim Bunce.

DBI is designed to protect you from the details of the vendor libraries. It has
a very simple interface for saying what SQL queries you want to make, and
for getting the results back. DBI doesn't know how to talk to any particular database, but it does know how
to locate and load in DBD (`Database Driver') modules. The DBD modules have the vendor libraries in them and know how to talk to the real
databases; there is one DBD module for every different database.

When you ask DBI to make a query for you, it sends the query to the appropriate DBD module, which spins around three times or drinks out of its sneaker or
whatever is necessary to communicate with the real database. When it gets the
results back, it passes them to DBI. Then DBI gives you the results. Since your program only has to deal with DBI, and not with the real database, you don't have to worry about barking
like a chicken.

Here's your program talking to the DBI library. You are using two databases at once. One is an Oracle database
server on some other machine, and another is a DBD::CSV database that stores the data in a bunch of plain text files on the local
disk.

Your program sends a query to DBI, which forwards it to the appropriate DBD module; let's say it's DBD::Oracle. DBD::Oracle knows how to translate what it gets from DBI into the format demanded by the Oracle library, which is built into it. The
library forwards the request across the network, gets the results back, and
returns them to
DBD::Oracle. DBD::Oracle returns the results to DBI as a Perl data structure. Finally, your program can get the results from
DBI.

On the other hand, suppose that your program was querying the text files.
It would prepare the same sort of query in exactly the same way, and send
it to DBI in exactly the same way. DBI would see that you were trying to talk to the DBD::CSV database and forward the request to the DBD::CSV module. The DBD::CSV module has Perl functions in it that tell it how to parse SQL and how to
hunt around in the text files to find the information you asked for. It
then returns the results to DBI as a Perl data structure. Finally, your program gets the results from DBI in exactly the same way that it would have if you were talking to Oracle
instead.

There are two big wins that result from this organization. First, you don't
have to worry about the details of hunting around in text files or talking
on the network to the Oracle server or dealing with Oracle's library. You
just have to know how to talk to DBI.

Second, if you build your program to use Oracle, and then the following
week upper management signs a new Strategic Partnership with Sybase, it's
easy to convert your code to use Sybase instead of Oracle. You change
exactly one line in your program, the line that tells DBI to talk to DBD::Oracle, and have it use DBD::Sybase instead. Or you might build your program to talk to a cheap, crappy
database like MS Access, and then next year when the application is doing
well and getting more use than you expected, you can upgrade to a better
database next year without changing any of your code.

There are DBD modules for talking to every important kind of SQL database. DBD::Oracle will talk to Oracle, and DBD::Sybase will talk to Sybase. DBD::ODBC will talk to any ODBC database including Microsoft Acesss. (ODBC is a
Microsoft invention that is analogous to
DBI itself. There is no DBD module for talking to Access directly.) DBD::CSV allows SQL queries on plain text files.
DBD::mysql talks to the excellent MySQL database from TCX DataKonsultAB in Sweden.
(MySQL is a tremendous bargain: It's $200 for commercial use,
and free for noncommerical use.)

Here's a typical program. When you run it, it waits for you to type a last
name. Then it searches the database for people with that last name and
prints out the full name and ID number for each person it finds. For
example:

The connect call tries to connect to a database. The first argument, DBI:Oracle:payroll, tells DBI what kind of database it is connecting to. The Oracle part tells it to load DBD::Oracle and to use that to communicate with the database. If we had to switch to
Sybase next week, this is the one line of the program that we would change.
We would have to change Oracle to Sybase.

payroll is the name of the database we will be searching. If we were going to
supply a username and password to the database, we would do it in the connect call:

If DBI connects to the database, it returns a database handle object, which we store into $dbh. This object represents the database connection. We can be connected to
many databases at once and have many such database connection objects.

If DBI can't connect, it returns an undefined value. In this case, we use die to abort the program with an error message.
DBI->errstr returns the reason why we couldn't connect``Bad password'' for example.

The prepare call prepares a query to be executed by the database. The argument is any
SQL at all. On high-end databases, prepare will send the SQL to the database server, which will compile it. If
prepare is successful, it returns a statement handle object which represents the statement; otherwise it returns an undefined
value and we abort the program. $dbh->errstr will return the reason for failure, which might be ``Syntax error in SQL''.
It gets this reason from the actual database, if possible.

The ? in the SQL will be filled in later. Most databases can handle this. For
some databases that don't understand the ?, the DBD module will emulate it for you and will pretend that the database
understands how to fill values in later, even though it doesn't.

print "Enter name> ";

Here we just print a prompt for the user.

while ($lastname = <>) { # Read input from the user
...
}

This loop will repeat over and over again as long as the user enters a last
name. If they type a blank line, it will exit. The Perl
<> symbol means to read from the terminal or from files named on the command
line if there were any.

my @data;

This declares a variable to hold the data that we will get back from the
database.

execute executes the statement that we prepared before. The argument $lastname is substituted into the SQL in place of the
? that we saw earlier. execute returns a true value if it succeeds and a false value otherwise, so we
abort if for some reason the execution fails.

while (@data = $sth->fetchrow_array()) {
...
}

fetchrow_array returns one of the selected rows from the database. You get back an array
whose elements contain the data from the selected row. In this case, the
array you get back has six elements. The first element is the person's
last name; the second element is the first name; the third element is the
ID, and then the other elements are the postal code, age, and sex.

Each time we call fetchrow_array, we get back a different record from the database. When there are no more
matching records,
fetchrow_array returns the empty list and the while loop exits.

my $firstname = $data[1];
my $id = $data[2];

These lines extract the first name and the ID number from the record data.

print "\t$id: $firstname $lastname\n";

This prints out the result.

if ($sth->rows == 0) {
print "No names matched `$lastname'.\n\n";
}

The rows method returns the number of rows of the
database that were selected. If no rows were selected, then there is
nobody in the database with the last name that the user is looking
for. In that case, we print out a message. We have to do this
after the while loop that fetches whatever rows
were available, because with some databases you don't know how many
rows there were until after you've gotten them all.

$sth->finish;
print "\n";
print "Enter name> ";

Once we're done reporting about the result of the query, we print another
prompt so that the user can enter another name.
finish tells the database that we have finished retrieving all the data for this query and allows it to reinitialize the handle so that we can execute it again for the next query.

$dbh->disconnect;

When the user has finished querying the database, they type a blank line
and the main while loop exits. disconnect closes the connection to the database.

There's a problem here though. Even though the function works correctly,
it's inefficient. Every time it's called, it prepares a new query.
Typically, preparing a query is a relatively expensive operation. For
example, the database engine may parse and understand the SQL and translate
it into an internal format. Since the query is the same every time, it's
wasteful to throw away this work when the function returns.

There are two big changes to this function from the previous version.
First, the $sth variable has moved outside of the function; this tells Perl that its value
should persist even after the function returns. Next time the function is
called, $sth will have the same value as before.

Second, the prepare code is in a conditional block. It's only executed if $sth does not yet have a value. The first time the function is called, the prepare code is executed and the statement handle is stored into $sth. This value persists after the function returns, and the next time the
function is called, $sth still contains the statement handle and the prepare code is skipped.

Here the only change to to replace prepare with prepare_cached. The prepare_cached call is just like prepare, except that it looks to see if the query is the same as last time. If so,
it gives you the statement handle that it gave you before.

Many databases support transactions. This means that you can make a whole bunch of queries which would modify
the databases, but none of the changes are actually made. Then at the end
you issue the special SQL query COMMIT, and all the changes are made simultaneously. Alternatively, you can issue
the query ROLLBACK, in which case all the queries are thrown away.

As an example of this, consider a function to add a new employee to a
database. The database has a table called employees that looks like this:

We create two handles, one for an insert query that will insert the new employee's name and department number into
the employees table, and an update query that will increment the number of members in the new employee's
department in the department table. Then we execute the two queries with the appropriate arguments.

There's a big problem here: Suppose, for some reason, the second query
fails. Our function returns a failure code, but it's too late, it has
already added the employee to the employees table, and that means that the count in the departments table is wrong. The database now has corrupted data in it.

The solution is to make both updates part of the same transaction. Most
databases will do this automatically, but without an explicit instruction
about whether or not to commit the changes, some databases will commit the
changes when we disconnect from the database, and others will roll them
back. We should specify the behavior explicitly.

Typically, no changes will actually be made to the database until we issue
a commit. The version of our program with commit looks like this:

We perform both queries, and record in $success whether they both succeeded. $success will be true if both queries succeeded, false otherwise. If the queries
succeded, we commit the transaction; otherwise, we roll it back, cancelling
all our changes.

The problem of concurrent
database access is also solved by transactions. Suppose that queries were executed immediately, and that
some other program came along and examined the database after our insert
but before our update. It would see inconsistent data in the database, even
if our update would eventually have succeeded. But with transactions, all
the changes happen simultaneously when we do the commit, and the changes are committed automatically, which means that any other
program looking at the database either sees all of them or none.

If you're doing an UPDATE, INSERT, or DELETE there is no data that comes back from the database, so there is a short
cut. You can say

$dbh->do('DELETE FROM people WHERE age > 65');

for example, and DBI will prepare the statement, execute it, and finish it. do returns a true value if it succeeded, and a false value if it failed.
Actually, if it succeeds it returns the number of affected rows. In the
example it would return the number of rows that were actually deleted. (DBI plays a magic trick so that the value it turns is true even when it is 0.
This is bizarre, because 0 is usually false in Perl. But it's convenient
because you can use it either as a number or as a true-or-false success
code, and it works both ways.)

If your transactions are simple, you can save yourself the trouble of
having to issue a lot of commits. When you make the connect call, you can specify an AutoCommit option which will perform an automatic commit operation after every successful query. Here's what it looks like:

When you make the connect call, you can specify a RaiseErrors option that handles errors for you automatically. When an error occurs, DBI will abort your program instead of returning a failure code. If all you
want is to abort the program on an error, this can be convenient:

while ($lastname = <>) {
my $sth = $dbh->prepare("SELECT * FROM people
WHERE lastname = '$lastname'");
$sth->execute();
# and so on ...
}

Here we interpolated the value of $lastname directly into the SQL in the prepare call.

This is a bad thing to do for three reasons.

First, prepare calls can take a long time. The database server has to compile the SQL and
figure out how it is going to run the query. If you have many similar
queries, that is a waste of time.

Second, it will not work if $lastname contains a name like O'Malley or D'Amico or some other name with an '. The ' has a special meaning in SQL, and the database will not understand when you
ask it to prepare a statement that looks like

SELECT * FROM people WHERE lastname = 'O'Malley'

It will see that you have three 's and complain that you
don't have a fourth matching ' somewhere else.

Finally, if you're going to be constructing your query based on a user
input, as we did in the example program, it's unsafe to simply
interpolate the input directly into the query, because the user can
construct a strange input in an attempt to trick your program into
doing something it didn't expect. For example, suppose the user
enters the following bizarre value for $input:

x' or lastname = lastname or lastname = 'y

Now our query has become something very surprising:

SELECT * FROM people WHERE lastname = 'x'
or lastname = lastname or lastname = 'y'

The part of this query that our sneaky user is interested in is the
second or clause. This clause selects all the records
for which lastname is equal to lastname;
that is, all of them. We thought that the user was only going to be
able to see a few records at a time, and now they've found a way to
get them all at once. This probably wasn't what we wanted.

References

 A complete list of DBD modules are available here  You can download these modules here  DBI modules are available here  You can get MySQL from www.tcx.se

People go to all sorts of trouble to get around these problems with
interpolation. They write a function that puts the last name in quotes
and then backslashes any apostrophes that appear in it. Then it breaks
because they forgot to backslash backslashes. Then they make their
escape function better. Then their code is a big message because they
are calling the backslashing function every other line. They put a lot
of work into it the backslashing function, and it was all for nothing,
because the whole problem is solved by just putting a ? into the query, like this

SELECT * FROM people WHERE lastname = ?

All my examples look like this. It is safer and more convenient and more efficient to do it this way.

Notes

It is hard to keep track of everything that happens. As before,
please let me know if you have any corrections or additions. Send
them to mjd-perl-thisweek-YYYYMM@plover.com where
YYYYMM is the current year and month.

There was a gigantic discussion of $^O and related
matters. This was brought on by Tom, who wants to write a program that
cross-checks the SEE ALSO sections of the man pages. The
problem: Every version of Linux has a man command that is
slightly incompatible with every other. In particular, each system has
a different idea of where the pages are and how they are organized.
Tom wants his program to find out what sort of Linux it is on, `Red
Hat' or `Debian' or whatever, but $^O (and also the
uname command) only says linux, which is not
enough.

Various discussion ensued. Suggestion 1: Make $^O look like
linux-redhat or something. Objections: Changing $^O will break stupid programs that have $^O eq 'linux' instead of $^O =~
/linux/. Putting redhat into $^O will not actually solve Tom's problem, at least not in general, since the
semantics of redhat
changes from release to release.

Suggestion 2: Add a Config.pm field for the distribution vendor. Objections: Config.pm only reflects the state of the system at the time Perl was built, not at
the time your program runs. Possible solution to this: Have Config determine the OS at run time at the moment the information is requested.
Second objection: If Config
can do this, why can't Tom's program do it the same way, but without
Config? Well, OK, the nastiness could be encapsulated in a module. But Sarathy
didn't like the idea of putting this dynamic information into Config. He suggested:

Suggestion 3: A new module, OS, to provide functions for
looking up this sort of thing dynamically. There were other similar
suggestions. Dan Sugalski suggested adding a new magical
%^O variable that would behave similarly. Nick
Ing-Simmons suggested an OS_Info module. This
multiplicity suggests that I was the only one following the whole
tedious discussion. (And, if so, that everyone else had good sense.)

Gosh. When I took this job, I knew there would be occasional weeks where
there was some gigantic but trivial discussion. But I wasn't expecting one
so soon.

If there was a conclusion to this discussion, I was not able to find
it. Maybe there will be an update next week, or maybe everyone will
just get tired of the whole thing and forget about it. Tom eventually
punted on the problem, and his program now assumes that it is running
under Red Hat.

In this midst of this, there were some sidetracks I found interesting.
There was discussion of Sarathy's hack to create fork()
on forkless Microsoft OSes (more about this below.) Tom Horsley had a
really delightful rant about Configure, which unfortunately is too long to
reproduce here:

[Configure] acts, in fact, as though it were a compressed archive chock full of
config.h files for all kinds of different systems, and pressing the button
merely unpacks one of the files.

The problem comes when you attempt to extract a file that was never put
into the archive in the first place. ...

One of the changes in perl 5.005_62 was that END blocks would no longer be run under -c mode. Nick Ing-Simmons wanted to know how the compiler would work; it had
formerly worked by enabling -c
mode, and walking the op tree and dumping out the compiled code in an
END block, which was executed after the program file was parsed and compiled.
(This may be an incorrect description; I would be grateful for corrections
here.) Disabling END blocks under -c mode, while correct, would break the compiler.

When he made the change, Sarathy planned a workaround, which you can
find in perldelta if you are interested. But the
workaround is annoying for the compiler, and Sarathy suggested that
the best solution would be STOP blocks. These would be
run after the compilation phase, but before the run phase; they are in
contrast to INIT blocks, which are run at the start of
the run phase. Normally, these two things happen at almost the same
time, with STOP blocks immediately before
INIT blocks. But if you think of a compiler
module, which pauses after the compilation phase, writes out the
compiled code and exits, the usefulness of STOP becomes clear.

Vishal Bhatia pointed out that this would solve an existing compiler bug:
END blocks are presently not executed at all by compiled scripts. If the B:: modules did their work in STOP blocks instead of
END blocks, they would not have to usurp the END blocks.

Larry Virden submitted a minor doc patch: There was a line which looked
empty, but which contained white space. This prevented the POD parser from
recognizing a =head directive on the following line, because directives are only recognized
when they begin `paragraphs', and a line is not deemed to end a paragraph
unless it is entirely empty.

Ed Peschko wanted a new PERL_HEADER environment variable, somewhat analogous to PERLLIB or PERL5OPT, which would contain code that would be prepended to the source file
before it was executed. He wanted this so that he could make an environment
setting to tell Perl to always load up some standard, locally defined
modules before compiling the rest of any program.

Many people found persuasive reasons why this would be a bad thing to do,
and many other people suggested ways that it could be accomplished. For
example, you could set PERL5OPT to
-MFoo -MBar.

Michael Schwern pointed out that there are several modules being
distributed with Perl for which more recent versions exist on CPAN.

It turns out that many of these cases are for good reasons. For example,
Ilya keeps the version number of the Devel::Peek on CPAN higher than the
version in Perl so that if you ask CPAN.pm to
install Devel::Peek, it does not go and try to install the latest version of Perl for you.
(Why does it do that, anyway?)

However, some modules really are out of date in the distribution. Sarathy
asked that authors of modules in the Perl distribution send him a note when
they update their modules.

Mark Mielke suggested enhancing isa so that you could
give it and object and several class names and it would return true if
the object belonged to any of the classes. At present, only one class
is allowed. No conclusion was reached. My guess is that this is not
going in because it is easy to write such a function if you want it.

I don't fully understand this yet, but it looks interesting. It
appears that Peter Haworth wants to have Perl notice when a sort
comparator function is prototyped with ($$), and to
optimize the argument passing to such a function to get the speed of
the $a-$b hack, but without actually using
$a and $b. Then you could use any
two-argument function as a sort comparator but it would be as fast as
if it were using the special $a-$b method. I
have asked Peter to confirm this, and I will report back next week.

Note added 26 October: Peter cofirms that I have it mostly right, but adds:

The gains aren't so much for
performance, as getting rid of package annoyances. If I manage to get this
patch working properly, you can use a comparator function from a different
package, and it can just get its arguments from @_, rather than
${caller.'::a'} and ${caller.'::b'}. Also, Ilya says this will allow XSUBs to
be used as comparators, but I don't know the history of this well enough to
know why they can't be used now.

Perl 5.005_62 optionally has a new built-in implementation of the glob function; it does not need to call the shell to do a glob. Paul Moore
pointed out that the new internal globber is case-sensitive, even on his
Win32 system with the case-insensitive filesystem; formerly, glob had been case-insensitive.

Some discussion ensued about what to do. Sarathy seemed inclined to let the
new globber continue to be insensitive on case-insensitive filesystems, and
vice versa; on Windows systems there is an API for finding this out. He
asked Paul for a patch for this. He said that people could use the File::Glob or File::DosGlob modules if they needed a specific semantics.

Jeff Pinyan posted a complaint about the behavior of a function prototyped
with (;$). He wants print f arg1,
arg2 to be parsed as if he had written
print f(arg1),
arg2. At present, Perl aborts, complaining that f
got two arguments and expected at most one. Jeff encountered this
behavior while he was writing a function to determine what kind
of reference (array, hash, whatever) its argument is.

(This is more difficult than it seems. You cannot use only ref, because if you have an object blessed into a class named
ARRAY, ref will return ARRAY even if the object is a hash, and you run into similar problems with
classes named 0 and so forth.)

Nobody addressed the (;$) issue, but there was discussion of how to
build such a function. Spider Boardman revealed that he had such a function
named attributes::reftype already in the standard Perl
distribution. It is written in C as an XS, which is clearly the Right Way to Do It. Sarathy
said he thought that attribute.pm was a good place for
the function to be.

Sarathy has been working for some time on making fork work on forkless Win32 systems. The idea: fork will create a new thread, running a separate copy of the Perl interpreter,
which will run the fictional child process. The child process will somehow
have its own current working directory, environment, open file table,
and so forth. exec in the `child' thread will terminate the thread and its associated
interpreter, rather than the entire process.

Dan Sugalski: I see there's
going to be something interesting to implement for VMS before 5.6 gets
released. Cool. :)

This continued from last week. Michael King split up his module
functionality into Import::ShortName for module aliasing, and
Import::JavaPkg, to load a whole bunch of modules in a single namespace all at once, with
aliasing.

At the tail end of this discussion, several people complained that although
they thought that they'd followed the documented procedure for reserving
namespaces in the CPAN module list, nothing ever seemed to come of it, and
their names never appeared in the list. Andreas König took
responsibility for this problem. He is rewriting the PAUSE software to
handle the bookkeeping, because the module list owners are too overworked
to do it all manually.

On some systems, the cron daemon has this bug. (It is a bug in cron, because cron should know to restore the signal handling to the default case when running
a job; otherwise the job will inherit this unusual signal environment and
might get unexpected results.)

Tom Phoenix added a patch to the linux hints file to try to detect this,
and print out a warning at Perl build time if so. Sarathy said it was bad
to put this in the hints, because it does not actually
affect the build process, and that it should be documented more prominently.

Mike Guy asked: ``Wouldn't it be better for Perl just to set
$SIG{CHLD} = 'DEFAULT' automatically at startup in this case? Would it do any harm to do it in all cases?'' Sarathy agreed, and put in a patch to do that, and also to issue a
warning if so.

If you ask timelocal to convert a date where the day of the month is larger than 31, it aborts
with a warning like

Day '32' out of range 1..31

John L. Allen complained that this was stupid for two reasons: First, it
doesn't abort when you ask for February 30, and second, it prevents you
from asking for January 280 to find out the date of the 280th day of the
year. He submitted a patch that eliminated the check.

A patch like that had been in before, but Sarathy took it out because it
caused a test failure in libwww; Sarathy wants it to be
conditionalized on a nocroak variable or something, for
backward compatibility. In the ensuing discussion, Jonathan Scott Duff made
a list of new features he'd like to see in Time::Local---features like `fast' and `correct'.

Mike Guy said that he had worked on such a thing, but run into some
annoying backward compatibility issues. For example, the current
timelocal returns -1 on an error. But because -1 also
indicates
a valid time before 1970, timelocal
cannot work for dates before 1970 and be
backward-compatible with the current version at the same time. Also, the
existing timelocal has a very nasty interpretation of the
year: 2070, 170, and 70 all
mean the year 2070, contrary to good sense and the documentation.

Sarathy said he would accept the timelocal replacement if there were a command to enable the improved behaviors that
were not backward compatible with the old behavior.

Kragen Sitaker asked, on comp.lang.perl.misc, whether it wouldn't be nice for Perl to recognize additional kinds of
parentheses once Unicode support is really in. For example, U+3010
and U+3011
are left and right `black lenticular brackets'. The q operator understands q{...} and q(....)q[...] and the like; why not the black lenticular brackets also?

Kragen also suggested that, the Japanese `corner quote' characters U+300C
and U+300D
(for example) could be used to imply the qr operator, in the same way that ordinary double quotes presently imply the qq operator and ordinary backquotes imply the qx operator.

It is presently lexically scoped. There was discussion some weeks ago about
whether to make it dynamically scoped; then the caller of a function could
set the utf8 behavior of the library functions it called. I did not
understand the issues at the time, so I cannot rehash them here.

Ed Peschko asked if it would be possible to include the full path of the
current directory in @INC, rather than just a dot. The usual objections: 1. There is already an easy
way to put the full path in, if that is what you want: you use
the FindBin module. 2. It would be expensive for the large population that did not need
it.

I am writing to report a bug in the localtime function. It returns the wrong month.

Person L

WORK FROM HOME!!!!

A lot of the stuff I omitted is on topic and has real value, but
isn't particularly interesting. For example I have omitted a bunch of
cases where someone submitted a minor patch that was accepted with no
discsusion. I omitted some discussions that did not seem to be of
general interest. For example, this week, Brad Appleton and Ilya
Zakharevich had an exchange about the pod-formatting features of
cperl-mode.

It is hard to keep track of everything, and I may occasionally omit
something you think is important, or I might misunderstand some
important issue. Your additions and emendations are welcome.
Please send any corrections, suggestions, additions, or
embellishments to mjd-perl-thisweek-YYYYMM@plover.com where YYYYMM
is the current year and month. For a more complete view of
perl5-porters, either subscribe to the mailing list or check
the archive.

I wanted to include hot links to the relevant messages in the
archive, but the archive was down and I could not get the URLs.
This will be corrected in a future issue, even if we have to start our
own archive.

Jarkko realized that since Perl was moving into beta mode for 5.6,
this was his last chance to propose a new feature. He wants Perl
regexes to support the `equivalence class' feature of POSIX 1003.2.
What this means is that certain characters in the character set may be
deemed `equivalent', and the notation [=c=]
denotes a character class containing all the characters equivalent to
c.

How do you decide which characters are considered `equivalent'?
Unicode provides a definition that allows you to understand a
character like &eacute; as an e with an acute
accent; we could have Perl understand this to be equivalent to a plain
e. Then the notation [=e=] would match any of
e, é, è, ë, or
ê. This might be useful.

Jarkko also wants people to be able to define their own
equivalence relations, and he wants the m// and s///
operators to support a new option, analogous to
/i, which would say to ignore all diacritical marks, again
using the Unicode tables to decide what a diacritical mark is.

People brought up a number of potential problems; for example, in
Danish, the character å (U00E5) is considered to be an
entirely different letter from a and not equivalent to it at
all. But according to Unicode, å is indeed an
a with a diacritical mark. (Jarkko: ``Ha! I am just expecting some Danes . . . to jump up
here and wave frantically their hands. . . '' )

Tim Bunce forwarded a message that Michael King had sent to the
modules@perl.com mailing list. Michael had written a new
module, which he named import. The idea of import is
this: Suppose you have a bunch of modules that are for internal use at
your company only. You are worried about namespace collisions with
CPAN modules. You can name your internal-use modules with names that
all begin with com::yourcompany and then use

use import 'com::yourcompany';

This does two things. First, it locates all modules in
the com::yourcompany space on the local machine and imports
them all. But second, it imports each com::yourcompany::Foo
module with the com::yourcompany part stripped off. This
means that if, for example, you have a
com::yourcompany::Template module, you can now call
Template->new() instead of
com::yourcompany::Template->new().

Michael says that this is like the import keyword in
Java.

This touched off a number of interesting discussions:

Andreas König suggested that Perl should support a

package "www.foo.org";

directive which would be equivalent to package
org::foo::www;. But Chip Salzenberg said that this was
usually understood to have been a bad idea, because
organizations often change URLs for various reasons. For
example, foo.com and bar.com might merge to
become foobar.com. Says Chip: One of the best
things about CPAN is that it is the de facto root namespace for
all shared Perl modules. Let's not throw that
away!

This led to a general discussion about how entire module
namespaces might be reserved.

Damian Conway pointed out that we could
simply establish the convention that if you have a PAUSE id,
that module space is reserved for you. For example, Damian's
PAUSE id is DCONWAY; under this convention, all modules
beginning with DCONWAY::* would belong to him.

Problems with this scheme: Uri Guttman would now own the
URI::* space, including the URI::URL module.
Nick Ing-Simmons's PAUSE id is NI-S, which is not a
valid package identifier.

John Redford suggested a module that lets you
bundle many modules into one. He says:

Then you could
write a bundling module, like this:

package MyCompany::CGIBundle;
use MyCompany::CGI::BobsCode::Foo;
use MyCompany::CGI::Test::Bar;
use MyCompany::CGI::BobsBetterCode::Foo2;
....
use NameSpace::Transitive;

And then people could just write:

use MyCompany::CGIBundle;

to get all the symbols that were exported into
MyCompany::CGIBundle re-exported into their own namespace.

I had written a module something like this back in
February, so I decided that put it on CPAN. It is now available at ModuleBundle-0.01d.
Nick Ing-Simmons also pointed out that his Tk::widgets
module does something similar: use TK::widgets qw(Text
Entry Canvas) is equivalent to:

use Tk::Text ();
use Tk::Entry ();
use Tk::Canvas();

A couple of people wanted the two functions of this module to
be separated. They liked the idea of being able to alias namespaces,
so that the objects in com::yourcompany::Template could be
referred to as if they were in Template, but they were
worried about the other function, which is to locate and import a
whole lot of stuff indiscriminately. There was some discussion of a
namespace aliasing pragma, or of adding this functionality to the
existing Alias module.

There were also a number of uninteresting discussions: Someone
wanted to know what would happen if you said use import
'CGI'. Michael's answer to that was that that was not what
import was for and that whoever did that would get the
bizarre behavior that they deserved. That did not stop a lot of
people from making a big fuss about it, however. One person even said
``If you want that functionality, why don't you write a module to do
that?'' apparently having forgotten that the way the discussion
started was that Michael had written a module to do that, notified the
modules list, and then Tim Bunce forwarded his note to P5P.

It appears that Michael is now pursuing a name in the
Import namespace, and may make some changes to the module's
calling interface to better seprate the two functions of his
module.

Tuomas Lukka discovered a gotcha in field.pm. The gotcha
is this: Suppose you have a base and a derived class, and both contain
a field with the same name, say f. Suppose the method
m is defined in the base class and inherited by the derived
class. Now create an object of the derived class, and call m
on the object. Suppose m contains code to modify field
f. There are two fields named f. Which one will be
modified?

You would expect that, because m is defined in the base
class, it should modify the base class's f. And so it does,
if m is written correctly:

Tuomas points out that the declaration and use of $self
might be very far apart, and that a mistake in a far-away declaration
could introduce a bug in the program that was difficult to find.
Worse, suppose the object was stored inside some other object, so that
instead of $self->{f} you had
$object->{subobject}->{f}; in this case no declaration at all
applies and you get the same problem.

Tuomas' solution is to simply forbid conflicting field names.
fields.pm and base.pm will detect this and throw a
fatal exception if they detect that a derived class is using a member
with the same name as one of its parent classes.

Tuomas's rationale: It is very dangerous at present, and is `action
at a distance', which means that two apparently unrelated declarations
far away, even in entirely different files, might drastially alter the
behavior of a subroutine in a third location. The prohibition can be
lifted later if a way is found to make it safer, and nobody appears to
be using it now. (Someone thought they were, but realized they were
mistaken.)

Sarathy appeared to be persuaded that the situation required at
least a warning. Tuomas is pushing for a fatal compile-time error.

Ilya had submitted a patch which would have made my Class
$foo; behave as if you had also written
Class->PREPARE($foo), if there was such a method. If you
had my Class $foo = 'bar' instead, the assignment would occur
after the PREPARE call.

Sarathy did not like the implementation, for reasons I did not
completely understand. Sarathy did not like that an AUTOLOAD
call would be made at compile time if PREPARE was not found
where it was supposed to be. Ilya did this by analogy with
DESTROY, but the compile-time call to AUTOLOAD is
peculiar. (Sarathy: ``Yikes!'') Sarathy also objected to the way
that the check for PREPARE was done at compile time, rather
than at run time; Ilya said he did it that way for efficiency so that
the check would not have to be done every time the declaration was
executed.

Sarathy also complained that even though most uses of my Class
$foo would not involve PREPARE, the compiler would have
to make an AUTOLOAD call for each one of them. Ilya appeared
to agree that this was a problem. Sarathy suggested a pragma that
declares the PREPARE method for a class. Ilya pointed out
that if the autloading part was removed from his patch, then the
definition of the PREPARE subroutine itself would serve as
exactly such a pragma.

An interesting sidetrack developed: Chip said it would be simpler
in Topaz if the argument to PREPARE were \$foo rather
than just $foo. Ilya said he did not want to do that because
constructing a reference costs as much as 21 `simple' operations.
What this means is: Perl takes a certain amount of time to dispatch
each operation. For some `simple' operations, such as performing a
scalar assignment, the time to actually perform the operation is
dominated by the opcode dispatch time, so they all take about the same
amount of time. Constructing a scalar reference, according to Ilya,
takes 21 times as long as one of these `simple' operations, and
constructing an array reference takes 50 times as long. Chip was
surprised, and so am I.

Damien Neil reported a bug in 5.005_03 that is triggered when you
use goto out of a conditional block. He also supplied a
patch. Watch out for Damiennewcomers who provide core patches are
people to pay attention to.

Sarathy said that this was already fixed in the development
version. His solution is a little different. Damien's patch wraps up
each branch of the if as a separate block; Sarathy's wraps up
the entire if as one block. Sarathy wants to keep his patch
because it makes the op tree smaller and so the code is faster at run
time. But he notes that Damien's approach might be better if the
peephole optimizer could be instructed to remove the extra
instructions for blocks that do not contain goto.

Faisal Nasim had trouble with a regex that included [\w-]
and [\w-.]. Different versions of Perl behaved differently
for this, depending on whether the - was seen as indicating a
range or not. 5.005_03 interpreted it as if you had written
[\w\-.]; 5.005_62 generated a syntax error. (``Invalid
range''.) This was Jarkko's doing. Opinions varied about what
behavior was best, especially since the documentation seemed to
support the latter viewbut the changed behavior broke old code, as
Faisal pointed out.

Larry suggested making it a warning. Jarkko thought this was
peachy and put in the patch. While on the subject, he put in a
warning for use of \A etc. in character classes.

Tod Irwin wants use lib 'foo' to append foo to
the front of @INC (which it does already) and to also remove
any other appearances of foo that happen to be in
@INC already. His motivation: mod_perl scripts
that have use lib have @INC lists that get longer
and longer and longer.

There were some objections, but the change is in.

Nick Ing-Simmons: Modules which inject things into @INC are highly suspect
beasts - its like lacing the fruit juice with vodka.

This almost slipped by me, but I wrote to ask Ilya what it was
about. Here's what he said:

Build Perl without Configure support. First you build a
bastardized version (nanoperl), use it to build a correctly working
version with some functionality missing (microperl), then use this to
run Configure.PL which will make an analogue of
config.sh, then you continue with miniperl and perl as
before.

The supplied target crazyperl is very close to become nanoperl of
the above classification. Minor changes to supplied micro0/config.sh
should (when bugs are ironed out) produce a microperl.

Currently crazyperl passes a lot of tests. This should be improved
yet more (apparently the code is there to support the situation when
no non-portable services are found, but it has some bugs).

This presentation was created for a meeting of AZSage (see http://www.azsage.org). "I had been attending monthly
AZSage meetings for over a year," said Brett Berry, "and finally I had something worth
presenting. The presentation was aimed at the Perl/Tk novice, and there
was a whole lot of ad-lib comedy to keep the mood light, and the people
awake!"

On Perl.com, we are presenting this as part of what we hope will be an ongoing series of articles, titled
"Source Illustrated." The presentation by Lee and Brett is a wonderfully concise example of showing
annotated code and its result. There's no running narrative; just a code-walkthrough that is a very
interesting way to learn how these examples work.

Lee Minniear and Brett Berry are in business together, offering the world Business
Card CDs and such. Visit them at
SculptedCD.com