Perl's Persistant Library: DBD Creator Tim Bunce at OSCON 2008

James Turner: This is James Turner for O'Reilly News live at OSCON 2008. I'm talking with Tim Bunce, who is currently a Consultant working with Shop-Zilla and is probably better known as the creator and maintainer of the Perl DBI Library. So thank you for coming.

Tim Bunce: Thank you for having me.

JT: So DBI was probably one of the earliest libraries that got into Perl. When did it first come into being?

TB: When did it start? Well actually it started--the idea started in 1992; back then it was Perl 3 and there were a bunch of extensions to Perl 4, different databases. Back then we didn't have dynamlic linking, so you had to embed the API for a particular database into Perl and it was a tricky process. So we had INGPerl, SyPerl, OraPerl for each of the different databases and they all had different API(s) and they couldn't--you couldn't port programs between them. And Ted Lemon sent an email in September '92 to the authors of these things saying, hey guys; wouldn't it be a good idea if you came up with a common interface? And about the same time, Larry released Perl 4.0 and we started talking and we talked and we talked and two years later we finally sorted out a specification. And Larry released the first Alpha(s) of Perl 5 and we want; ugh, now we've got to redo all this now with objects and references. And about that time, the guy who was kind of leading the editing of the spec, he left and one of the other contributors left. So I said oh well I'll take over the editing for now until somebody else comes along. So I took on that responsibility and I needed an Oracle driver for my work. And so I ended up doing that and the rest is history; so that's the actual first release of the DBI, I think was in '94 and it almost became a standard module in the distribution. But at the last minute I said no; you know but I think it would be a bad idea.

JT: Why was that?

TB: There's a lot of problems in dual life modules that if you're in the core it creates a whole set of extra problems over maintenance and life cycles and release cycles and so forth. It just quite--

JT: So you want to kind of be your own creature that you could release on your own cycles?

TB: Yes.

JT: DBI probably is one of the more challenging modules to maintain because you're trying to track how many databases now?

TB: Well if you take a bold definition of database and just in terms of the number of different database drivers available for DBI then I think we're over 30. There are drivers for Google, drivers for Amazon, drivers for your iPod, you know anything that you could remotely regard as a--like a store, yeah or even--I mean is Google relational? You know it's--people have taken very inventive views of sort of squeezing things through the DBI API which is great.

JT: I imagine there's been kind of a dynamic tension between people who are saying can't you extend it so that this cool feature of my database is part of the native library versus people saying no we had to keep it in kind of the lowest common denominator approach?

TB: Right; there is a tension there and we saw that back in the early days. So it was understood that the best way to balance those was to have a lowest common denominator focused on ease of use and the most common operations. But then to provide every database driver with an escape hatch, some way that they could add their own functionality and the important bit is that adding functionality to one driver should be done in such a way that an application using it can still port to a different driver and not clash. So for example, some drivers need you to specify a data type and we use a specific attribute for that. But the attribute for a particular driver has a different name to the attribute for a different purpose, so all the drivers are given a unique prefix. So for Oracle it's ORA--O-R-A and Underscore and for Informix it's IX. So all of the private attributes and all of the private methods have that prefix; so it means that when you're porting an application any existing private attributes for one driver won't clash with those of another. And that's--and it's giving the driver authors a lot of freedom; they can buy into the DBI way of doing things, get 80-percent of what--of the functionality and if they need a little bit extra they got to find a way of doing it that doesn't cause problems.

JT: Do you ever get pressure from the vendors in the sense that it's almost in--not in their best interest for a code to be easily portable between databases because then people can move from one database to another?

TB: No; I've never had feedback like that and it's--it works both ways, you know. If it--if code is so easily portable, then it makes it easy for applications to port to your database.

JT: Right.

TB: So it evens out.

JT: I know that MySQL you can get the drivers right from their site; you don't have to go through CPAN. How is--how do you keep all that in sync?

TB: I don't; you know it's up to the driver authors to deal with that. You know it would be impractical for me to try and manage that sort of closely and it would be inappropriate as well I think.

JT: How much of a control do you have, I mean you know Larry obviously has admitted he takes the benevolent dictator role for Perl in general; do you see a similar kind of role for you being kind of the Uber Database person for Perl?

TB: No, no; for starters--Larry has stepped back from working on the Perl 5 code base, so there are other people doing that. So the day-to-day maintenance is no longer Larry's. Whereas for me the DBI is still kind of my code and I freely accept patches and the source code repository is available and some people have commit rights and I'm happy to hand out more commit rights. But you know it's important for me not to be in the way of anybody's creativity, so this is why the kind of escape mechanism in the DBI are so important and it just lets driver authors get on with it without having to come back to me. I do get requests for you know let's define this cool feature and so for example calling stored procedures. Wouldn't it be nice if the DBI had a stardard way of calling stored procedures? And my usual response now is to say well do it in your driver as private methods and when two or three drivers support it then we'll look and see if there's a commonality to define something about it. So I did something similar with array execution, click-on bulk execution, but there I took the view that I didn't want to add that to the DBI until there was a pure Perl implementation of it. So if the driver didn't support it, applications would still work, so if you ported an application from a driver that supported bulk uploads, bulk inserts, to a driver that didn't the code should still work. It shouldn't break. So although some drivers had implemented it I didn't put it in the DBI until--I didn't define a standard interface until there was a pure Perl way of doing it so it would be safely portable.

JT: So you don't have any real control over if someone does a new DBI:: "fill in the blank" database, you don't have any real control over whether or not that ends up for instance in CPAN?

TB: I wouldn't want to--

JT: Has there ever been anything that came in under your package where you just like held your head and said why, why, oh why?

TB: No; I mean just let me clarify that. Are you talking about the DBI main space?

JT: Right.

TB: There's kind of a convention there that the DBI main space sort of belongs to modules that ship with the DBI and there's a DBIX main space where people are encouraged to put extensions. So that--and that's--

JT: That's kind of like the sandbox?

TB: You can kind of call it more like a branding exercise, sort of a way of demarcating what is the DBI and what isn't? So but it's kind of--

JT: There are a lot of database drivers that are in DBI::?

TB: DBD::.

JT: Or DBD; right.

TB: Yeah; so the DBD(s)--people can upload drivers in DBD-whatever main space. That isn't an issue and even the DBI main space you know I'm kind of trying to encourage people not to put random things in that. If you look on CPAN there are a few modules in that main space that aren't part of the DBI.