What Will Be in Perl 6? - TranscriptTranscript

At O'Reilly's Open Source Convention and Perl Conference 4.0 in Monterey in July, Larry Wall announced that work would begin on Perl 6, and it could be a complete rewrite of the language. Joining the O'Reilly Network Roundtable to talk about the launch of Perl 6 was brian d foy, director of Perl Mongers and spokesperson for Perl 6, and Mark-Jason Dominus, managing editor of Perl.com.

O'Reilly Network publisher Dale Dougherty began the Roundtable by asking how the announcement is being received by the core Perl community, whether with excitement or anxiety.

brian d foy: Well, there's actually a little bit of both. Some people are really excited at the opportunity they have to get things that they have been asking for in Perl 5 to actually show up in Perl 6. Things like Unicode support for instance. But also there is some concern -- I wouldn't really say it's anxiety, but some concern -- that people are going to be able to be included in the process, and we've been trying to get information out there that says this is going to be a community effort. And that's what Larry Wall said at his keynote address at the Perl Conference. He made Perl 5, but he wants Perl 6 to be a product of the community.

Mark-Jason Dominus: The only concern I've seen -- it seemed like there was some initial worry on Simon Cozens' Web site, which is sourcetalk.perlhacker.org. Simon got a really incomplete, very early report about what was going on, and then SlashDot was able to grab it and blow it up into a big tempest. But it seems to me that the overwhelming reaction has been positive, and lots and lots of people seem to be interested and want to get involved.

Dale Dougherty: Can we talk a little bit about some of the technical agenda for Perl 6? Looking at Adam Turoff's notes, one of the more interesting comments to me was to learn from the Mozilla project and to kind of reevaluate the code, perhaps making it smaller, simpler. A lot of it has to do with the internals and how to interface into Perl. Mark, do you want to talk about problems developers have had with Perl 5?

Dominus: Yeah, the interface -- the API to the Perl internals is very complicated. It's partly C, and then built on top of the C there's this fairly complicated macro-language called XS that deals with the Perl internal data types. So you not only have to know C, which is -- you know, most people manage that. But then there are these XS macros on top of that, which make it look not even like C. And I understand from Larry's speech that he wants XS to go away and be replaced by something more simple and more straightforward. And then on the other side, there's a lot of implementation hacks in Perl's internals that make it difficult to add anything without breaking other things. I think Larry must have decided that a clean rewrite was necessary to get some more moving room.

Dougherty: Brian, Larry also made mention of the fact that one can learn a lot from languages that have come up since Perl was born; he can go back and look at those lessons and implement them in Perl 6.

foy: Yeah. I mean, Perl 5 was made several years ago and so between that time and now there's been a lot of developments, and technology has moved further, and people have decided that they want to use certain things. And Perl 5 wasn't made to use that stuff. It wasn't even on the horizon yet. And as Mark was saying, if we go back and rewrite Perl 6 now and take those things into account, we can put those things right in without having to have these unwieldy add-ons to Perl 5. Larry hasn't really decided what he wants to add to all this yet. I mean he's sequestered himself, I guess, and he's thinking about what Perl 6 is going to be, but there's really no word on what new features or things he's going to steal from other languages. He did say that other languages have been stealing from Perl for a while now, and maybe it's time that Perl stole back.

Dougherty: Now, Chip Salzenburg was working on Topaz, which was a C++ reimplementation of Perl. What is the connection of that to Perl 6? Brian?

foy: Well, Chip originally started that project to explore various things that he thought might be interesting in the implementation of Perl. Although I can't speak for Chip, as I understand it, some other things came up and he was being pulled in different directions, and he didn't quite make the progress that he wanted to, although along the way he learned several things about reimplementing Perl -- everything is going to be rolled into how we redesign Perl 6. So at the Perl Conference, Chip actually sat down with the developers and the community at large, or I guess the portion that was at the conference, and said, "This is what I learned, and this is how we can save some time on the next implementation." If it's C++ or if it's something else like Chip has done, we don't know yet.

Dougherty: Right, so he was -- at the conference he wasn't really clear whether the future involved C++ or not.

foy: That's right. It's still up for grabs. Although, as Mark was saying earlier, we might be able to attract more developers if we use something that's more common like C. But still, there's no promises yet, and that's still being hashed out with the Perl 6 developers.

Dougherty: So for me the most interesting aspect of the Perl 6 announcement was really not technical, but more about governance, if you will, or how the Perl community will create its own process to produce Perl 6. It was some admission on one hand that the P5P process -- the Perl Porters' list, its loose structure -- really is inadequate for getting work done in a timely fashion. Mark, you write a digest of that list and have noted some changes in the last couple of months in that. Do you want to talk a bit about how things work on the Perl 5 Porter's list?

Dominus: It was one big list on which absolutely everything was discussed, and as Perl got bigger, traffic got larger and larger, and it became less and less feasible for the entire list to take a week off to discuss some trivial problem that came up. For example, one thing that came up more than once is an extension to Perl's OR operator. People were very divided about that, and it came up more than once, and then there were these huge discussions about it, which really sidetracked the other business in the list. And I think that may be part of what's behind Larry's notion of having more than one list. He specifically talked about having working groups, each of which would be responsible for one aspect of Perl 6. And maybe one of the things he was thinking of is, that way it would contain this kind of large discussion about a small issue.

foy: Going back to learning from the other sort of communities that have developed: There's all sorts of big open source projects now, and Perl 5 was really just Larry's product, although it afterwards was given out to the community, and all sorts of things have been added on to the community in sort of an ad hoc or -- I don't really want to say disorganized, but not tightly controlled fashion. As Mark said, it was P5P and everything was discussed on that list. So if Perl 6 is going to take things from other groups, I think what they've learned (is), in large projects it's sometimes good to segment the work so that people who are interested in just doing the internals, for instance, can sit there and read stuff and post stuff about internals, while people who are interested in something else, perhaps the language or the syntax, can focus on that without having to pay attention to hundreds and hundreds of messages in areas they're not interested in. And I think that the working group stuff can probably do that, although they are still working out how they are going to divide up that work and how it's actually going to happen.

Dougherty: Okay, it almost seems like it's evolving into a kind of standards body, if you look at how they work in terms of breaking things up and having some accountability among different working groups to produce a product in a timely way.

foy: Yeah, Nat Torkington is so far the interim project manager who should be overseeing all the working groups. And we're not exactly sure how that hierarchy, or if it's going to be a hierarchy at all, is going to work, but it's just something to work with at the moment. So we'll have, as I like to call them, lieutenants -- various people who are empowered to work on some certain area and make decisions in some certain area -- control the discussion, or, as we have been saying, facilitate the discussion in that area, and then report back to one person who can then tie all the different areas together.

Dougherty: Well, Brian, how do you see Larry's role changing -- or will it change in this process?

foy: Well, Larry has said that he doesn't really want to be the executive power in Perl, meaning, I think, he wants to focus on what his strengths are, going back to language design and taking his many skills and experiences and putting that into the language, but not necessarily being the person who puts the rubber stamp on every decision. He said he wants to be more like the Supreme Court, where if it absolutely has to come to a moderated decision, that he would finally do that, but only in extreme circumstances. He has gone back and decided that he'll be in charge of the language rewrite. And he doesn't have any timetable for when he's going to announce that stuff or what he's going to put in. But I think this will get him back to the things that he's interested in doing rather than dealing with sort of the mundane things that more other, or excuse me, that people more interested in the low-level stuff might get a chance to move up.

Dominus: Yeah, I've thought for a long time that Larry really needed to devolve some of his moral authority onto other people, and I'm really glad to see that he's decided that he's going to do that because, you know, often he was unavailable. He was out for months while he was working for the Camel, and it's hard for the group to get along without somebody who's officially able to make a decision.

Dougherty: Right. Mark, do you think there is -- in absence of a "dictator," to use a word that sometimes Linus Torvald is described as -- you know, in absence of that, is there a consensus process? Do you think the Perl community can work productively towards consensus on development issues?

Dominus: Oh yeah, absolutely. But one thing that I've noticed often happens on technical mailing lists is that somebody will ask a question -- say, okay, should we do it this way or that way? -- and there will be a general agreement that it should be done that way, but because there's nobody who's been recognized as being able to end that process and say, "Okay it looks like the consensus is that way, so why don't you go ahead and do it that way like you said," the person who made the original proposal is never sure whether or not they should go ahead. Well, has it been decided or hasn't it? Just having somebody there who is able to say, "Well, we've heard the arguments and it looks like we should go that way," to announce what the consensus was after the fact and give permission to go ahead, is a really valuable thing, and I think it's something that has been missing in the past.

Dougherty: How about the Perl community itself? To some degree it's one of the older communities in open source. Is there any effort to try to bring in new people, new blood I suppose, both in terms of this development and in terms of even Perl developers?

foy: Well, actually, I don't know if I would agree with saying it's one of the oldest communities. I mean, it has certainly been around for a while, but I think that the community of Perl has changed so much in the past couple years that most of the users of Perl are actually fairly new to the community. And that's one of the reasons that Perl has to change, to keep meeting the needs of that community.

Dougherty: Let me give you an example. I think one of the -- obviously the largest impact Perl has had is in the Web development space, yet I don't get a real good sense that -- and I've actually heard this articulated -- that the core Perl language developers are that connected to that kind of application of Perl.

Dougherty: Well, I'm just talking about -- you know, well, I'm certainly talking about the people that are contributing to the development of the language in this Perl 6 thing, but also the developers who use Perl, that sometimes -- I perceive sometimes a disconnect between the application of Perl in, say, the Web space, and the language developers of Perl, who sometimes don't really recognize that, I think the way they should, as an important application of Perl.

foy: Actually, I think there's a pretty good diversity of end user viewpoints in the Perl developer community. I've always been impressed with what some of the actual Perl developers are doing, and although I don't think I'm at liberty to say what they're all doing, people are doing some really amazing things out there with Perl, including stuff in the Web community and stuff with new technology.

Dougherty: Well, I have no disagreement there. I mean, they are doing incredible things. I think sometimes -- you know, for instance, I was speaking to one of the authors of the programming Perl book, and there's relatively little mention of the Web in that book, and his comment was, "Well, this is about the language, and the Web is just an application space." But if you take something like Java, they see the role of the network and its connection to the language much more deeply than at least is expressed sometimes in the ideas about Perl.

foy: Well, Java was actually made just for that niche, at least initially. And so they were aiming their language at a much narrower focus, I think, than Perl is. As I like to say when I go out and teach Perl, and maybe Mark is able to do this as well, to say, "Here's the language and everything else is just a simple matter of programming." It doesn't matter if you're doing it on the Web; it doesn't matter if you're doing it inside a hardware testing facility. All it is is a simple matter of programming. You adapt your program to fit whatever little niche you want to. Perl is so flexible that you don't need to have all sorts of weird special features like some other languages do, to fit into any particular place. It's a great glue language; it can go almost anywhere. And so I don't think Perl really needs that niche marketing sort of angle. It's been fabulous for all sorts of things so far without having to be specialized.

Dominus: Yeah, I would agree one hundred percent with Brian. I think that what he said is absolutely true. And I want to embellish, which is that a few weeks ago there was a post on the comp.lang.perl.misc newsgroup from someone who said that he had gone to conferences and talked to many gurus there and they all said they had nothing to do with the Web. And he was trying to make the point that Perl wasn't just a Web language, but when he said that, a huge number of gurus immediately followed up and said, "Well, you must not have talked to me because I've been doing --" and then they would describe the extent to which they've been involved with Web programming, for years and years, sometimes doing it for a living. So I think the expertise and the concern is definitely there in the developer community, that a lot of people are very closely concerned with this.

Dougherty: Okay. Well, getting back to just Perl 6 per se, it didn't seem like there was any timetable in place here other than a year from now they thought there might be -- Larry thought there might an alpha ready. Is that an aggressive enough timetable?

foy: Ummm, we're not really sure yet. We definitely need to get something out there if we're going to have people using Perl 6. We can't wait two or three years for the next thing or we're going to miss out on things like Unicode support and the way that the world's going. So we need to get something out there for people to start playing with right away. The thing that you might see by the next Perl conference is not something that people are going to be able to take home and start doing useful programming with. It might be something, however, that engineers in companies that ship Perl, original equipment manufacturers and such, can look at and start adding features that support their platforms into Perl, and then maybe six months to a year after that, there can be a stable release of Perl 6 that people can actually start playing with. One of the big concerns, however, in this process is that Larry is very committed to having some way for people who have a significant code base in Perl 5 to be able to convert that stuff to Perl 6 with very little work. So until we get that process down, I don't think they're going to put Perl 6 out there and say, "Everyone change to Perl 6 now because we have this." We have to have the process that continues to support people with Perl 5, at the same time giving them a clear migration path to Perl 6, if they want to go that way. And we've also committed to supporting Perl 5 as long as it has to be supported.

Other Open Source Roundtables:

The Jabber Jihad: Universal Instant Messaging
AOL says it supports a universal protocol for open IM, but the company appears to be dragging its feet -- possibly to its proprietary advantage. But with workarounds such as Jabber, is that a problem? [10/27/2000]

Free Riding on Gnutella
Xerox PARC researchers found that nearly half the files shared on the service come from one percent of the hosts. Could this unequal distribution of the burden jeopardize the file-sharing service? [10/09/2000]

Foil the Filters Contest
Does House Majority Leader Dick Armey, who advocates the use of filtering software, know that it blocks access to his web site? We talk to the Digital Freedom Network about the pros and cons
of censorware on the Open Source Roundtable. [09/29/2000]

When Will We See Linux PDAs?
Tim Wilkinson, CEO of Transvirtual Technologies, talks about PocketLinux and the future of Linux in the palmtop. [09/01/2000]

Developers Explain - Why RSS 1.0?
O'Reilly Network publisher Dale Dougherty talks with some of the core developers behind the new spec proposal for RDF Site Summary (RSS 1.0) about the background behind RDF, the need for a standard, and what RSS enables [08/25/2000]

Developers Explain - Why RSS 1.0?
O'Reilly Network publisher Dale Dougherty talks with some of the core developers behind the new spec proposal for RDF Site Summary (RSS 1.0) about the background behind RDF, the need for a standard, and what RSS enables [08/25/2000]

Where's That Lizard?!
The Web Standards Project has taken Mozilla to task for ceding the browser market to Microsoft. Is there still hope for the project? An audio roundtable with WSP's Tim Bray and Alphanumerica's David Boswell. [08/04/2000]

What will be in Perl 6?
At O'Reilly's Open Source Conference in Monterey earlier this month, Perl creator Larry Wall announced the start of work on Perl 6. This week, O'Reilly Network Publisher Dale Dougherty talks with Perl.com managing editor Mark-Jason Dominus and Perl Mongers director brian d foy about the announcement, its meaning, and the reaction of the Perl community [07/28/2000]

Is Carnivore Eating You?
The FBI wants to install black boxes at ISPs to monitor email traffic of suspects. What are civil libertarians doing to try to stop it? [07/18/2000]

Napster Rides Again
Is the song-swapping service a model for a new breed of Net application?
A roundtable discussion with News.com's John Borland and Steve Pizzo of
Forbes ASAP. [07/14/2000]

Napster Rides Again
Is the song-swapping service a model for a new breed of Net application?
A roundtable discussion with News.com's John Borland and Steve Pizzo of
Forbes ASAP. [07/14/2000]