Saturday, June 14, 2008

Rhinos and Tigers

I will once again plagiarize myself by transcribing a talk I gave.

First: be warned! I offer this gesture of respect to you — yes, you! — when I say that this is at least 20 minutes of reading. This is long even for me. If you're surfing reddit, gobbling up little information snacks, then it's best to think of this entry as being more like a big downer cow. Unless you're really hungry, you should wait for it to be sliced into little bite-sized prion patties before consuming it.

If you do read it, you'll see the CJD analogy is surprisingly apt. I ramble even more than usual, and lose my train of thought, and the slides might as well be scenes from a David Lynch movie for all the relation they have to my actual talk.

But once again I find myself astonished at how much I agree with myself, by and large. Funny how that works. And I made a few decent jokes here and there. So I'm transcribing it.

If you're impatient, and I wouldn't blame you a bit, the best part is probably "Static Typing's Paper Tigers". That might be worth reading. As for the rest... *shrug* If you're really starved for content, you might find some of it entertaining.

The Setting

I gave this talk at the Google I/O Conference in San Francisco a few weeks ago. My talk was boringly titled "Server-Side JavaScript on the Java Virtual Machine", and there were initially only about 40 or 50 people in the room (out of a 2500-person conference) when I started the talk.

Even though I personally thought the talk was pretty boring, people kept trickling in, and I estimate there were about 400 people stuffed in the room by the end. It was standing-room only, and people were spilling out into the hall. The conclusion? The other talks must have been really boring.

After my talk it became pretty clear to me that it should have been titled "Rhinos and Tigers", so that's its new name. I've tried to make it flow well by splitting it into arbitrary sub-sections, whose titles aren't really part of the talk. But otherwise it's pretty much a word-for-word transcription, except with most of the umms and aaaahs elided. I've included the subset of the slides that seemed relevant; you can find the rest at the Google I/O site.

So enjoy! Or not. I've given you plenty of warnings. You'll see!

Rhinos and Tigers

Hello! I'm Steve Yegge. I work at Google. I've been there about three years, and it's real nice.

I'm going to be talking about server-side scripting in general, and talking a lot about Mozilla Rhino and the technology behind it. I'm going to try to get through it in maybe 20-25 minutes, maybe 30 if I start ranting, at which point I'll open it up for questions. I kind of want you guys to help drive how this goes.

Make sense? (Ed: Well, it made sense at the time. Sigh.)

All right, cool. Let's get started.

Sooo... I'm going to be talking about Server-Side JavaScript on the Java Virtual Machine.

Yes. We've got this big animal. Rhino.

So let's see... who here has used a JVM language before? Oooh! My god, lots of you, almost all of you. Great!

Well I'm going to be talking about Rhino in particular. I'll be making reference to other JVM languages. I want to kind of help you see how this fits into the landscape, why you might use it, why you might not use it.

But for those of you who haven't used a JVM language, the Java Virtual Machine is sort of like .NET: you can run multiple languages on it. You can write an interpreter in Java, or you can compile your language down to Java bytecode. Or you can compile it down to your own bytecode; there are different ways to do it.

But typically these languages are sort of drop-in replacements for Java. Which means you can implement classes, you can implement interfaces, you can subclass things. It gives you an alternate syntax and semantic layer on top of the libraries, and on top of the virtual machine.

I'll assume that this makes sense... well, actually, I won't!

FOO Chaos

There's this dude named Walter Bright, who wrote the D programming language, among many other things. (Raise hand) Has anyone heard of Walter? He's this really smart dude. He wrote Symantec Cafe, and the game Empire [and Zortech C++].

He told me the other day, [talking about] one of my blog rants, that he didn't agree with the point that I'd made that virtual machines are "obvious". You know? I mean, of course you use a virtual machine!

But he's a compiler dude, and he says they're a sham, they're a farce, "I don't get it!" And so I explained it [my viewpoint] to him, and he went: Ohhhhhhh.

Virtual machines are great for language interoperability. If everybody in the world used his language, then yeah, you probably wouldn't need a virtual machine. You'd probably still want one eventually, because of the just-in-time compilers, and all the runtime information they can get.

But by and large, we don't all use D. In fact, we probably don't all use the same five languages in this room. And so the VM, whether it's the CLR, or the Java VM, or Parrot, or whatever... it provides a way for us to interoperate.

Now I'll tell ya — I was at Foo Camp last summer. I've been wanting to tell this story... I'm telling you guys; it's the coolest story. And it's relevant here. Heh. Very relevant.

So I was in this tent... you know what Foo Camp is? It's O'Reilly's, you know, Friends Of O'Reilly invite thing that they do each summer. It's coming up in a couple of weeks. And people give presentations; people show up and just wander into your [presentation] tent, and wander back out if they don't like it.

So I was in this discussion at the very end of the last day, where the Apple LLVM guy Chris [Lattner] was hosting a talk on dynamic languages running on different VMs. And there was the Smalltalk Squeak guy there, and there was Charles Nutter for JRuby and representing the JVM. John Lam was there for IronRuby and CLR, and there were the Parrot people. I can't even remember them all, but the whole room was packed with the VM implementors of the VMs today, and people who are implementing languages on top of them.

This was a smart group of people, and well-informed. And you know, I was like a fly on the wall, thinking man, look at all [these brains].

And Chris, well, he let everybody go around the room and talk about why their VM was the best. And they were all right! That's the weird thing: every single one of them was right. Their VM was the best for what they wanted their VM to do.

Like, Smalltalk [Squeak]'s VM was the best in the sense of being the purest, and it was the cleanest. Whereas the Java one was the best because, you know, it has Java! Everybody's was the best. Parrot's was the best because it was vaporware. Ha! Ha, ha ha. Sorry guys.

So! He [Chris] asked this really innocent question. He goes, "You know, I don't really know much about this stuff..."

Which is bad, you know. When somebody says that to you at Foo Camp, it means they're settin' you up.

He says, "So how do these languages talk to each other?"

And the room just erupted! It was chaos. All these people are like, "Oh, it's easy!" And the rest of them are like "No, it's hard!" And they're arguing, and arguing, and arguing. They argued for an hour.

And then they stood up, still arguing, and they kept talking about it, heading into the dinner tent. And they sat down, going at it for like three hours.

It was chaos.

Because some people were saying, "Well, you know, if Ruby's gonna call Python, well, uh, you just call, right? You just share the same stack, right?"

And the others are like, "Well, what about different calling conventions? What if they support optional vs. non-optional arguments? What if they support default arguments? What about the threading model? What about the semantics of, you know, the this pointer? What about all this stuff?"

And they're like (waving hands) "Ooooh, we'll gloss over it, gloss over it, smooth it over." And the reply is: "You can't. This is fundamental. These languages work differently!"

And oh my god, it was really interesting. And it was also very clear that it's ten years of research and implementation practice before they get this right. Before you'll be able to have a stack of calls, where you're calling from library to function, library to function in different languages.

So today, VMs are good for interoperability, but you've gotta use a bridge. Whether it's JRuby, or Jython, or Rhino, they provide a set of APIs — you know about javax.script, right? It's this new thing introduced to the JDK, where they try to formalize just a couple of APIs around how you call the scripting language from your language... you know, it's a sort of "reverse foreign-function interface". And then how they call each other, maybe.

But it's all done through... kind of like serialization. You marshal up your parameters as an array of arguments, and it's a heavyweight call that goes over [to the script runtime] and comes back, and it's like... it's a pain! You don't want that. But today, that's kind of what we're stuck with.

At least we have that, though, right? I mean, try having Ruby call Python today, and they have different FFIs. You can do it, but you definitely want the VM's help.

So, Walter, that's why you need VMs.

Power to your users

So! Yeah, there's a lot of stuff I could talk about. I gave a practice of this talk to Mike Loukides, an editor at O'Reilly, and it completely changed what I wanted to talk about.

I do want to talk about Rhino's technology; I want you to come away understanding it. But more importantly, I want you guys to understand where this fits in this Google conference. And where it fits in your plans, going forward.

See, it's really interesting. We all know, or at least most of us I think agree, that server-side computing is finally starting to make inroads onto the desktop. Fat clients aren't so much the norm anymore. You've got applications like Google Maps, GMail, Google Docs, those kinds of apps, that are doing "desktop quality" applications in the browser, with the server handling your storage.

That's kind of one of the messages of this conference. Everybody's doing it, right? It's not just Google. And it makes a certain amount of sense, so I'm not going to go into the reasons why you'd do that. I'm assuming it's sort of a given. (Editor's Note: you'd be amazed at how many emails I get from people who maintain it's a fad of some sort, one that's going away, which is why I bother to make this disclaimer.)

The interesting thing is this: all applications... who was it who said "All apps will eventually grow to the point where they can read mail, and if they don't, they'll be replaced by ones that can"? (audience: "JWZ") JWZ? Jamie Zawinski. Yeah. It's a variant of somebody else's quote [Greg Kuperberg's], but...

So it's true, right? Apps grow. If you like an app, you're gonna want to start doing more and more stuff in it. If you like it a lot, like I like Emacs, heh... you know, you like your editor. Everybody here is a programmer, right? You all use development environments? Do you ever find it kind of annoying that you have to switch from your IDE to your browser? Why isn't the IDE the browser too? Why aren't these unified?

I mean, let's face it: I only run two apps. Unless I need to run, like, OmniGraffle or the Gimp, or something to do a document, or Keynote here to do the presentation — I just switched to Macs, so I'm learning all these names, but, this PowerPoint thing — most of the time, when I'm developing, I'm running shells, and I'm running Emacs, and I'm running a browser. That's it! So you kind of wish they'd be the same.

Well, once they get big enough, your IDE and Emacs and the browser have this thing in common, which is that they are scriptable!

That's the magic point at which your application becomes sort of alive. Right? Because people can change it, if it doesn't work the way they like it.

GreaseMonkey! Perfect example. You don't like our web page that we give you? Write a GreaseMonkey script and change it all around, right? That's cool! Scripting is really important.

I mean, Emacs, it stands for "Editor Macros", and it started off as a really thin engine, and the whole editor was written in scripts. And now it's huge. It has a million lines or so of Emacs-Lisp code floating around.

So it's weird... you go through this transformation, where your scripting languages are originally for, well, scripting. And it eventually grows into application level/scale development. OK?

Now we all see this happening in clients. Excel, for instance, is scriptable. And the reason that Excel is so powerful, I mean the reason that you can go to the bookstore and get a book that's this thick on Excel, and scientific computing people use it, whatever, is that it has a very very powerful scripting engine.

In fact, all of Microsoft Office has it. Right? You can fire up Ruby or Python or Perl, and you can actually control, though the COM interface, you can actually tell IE to open a document and scroll to a certain point in it. Or you can open up Microsoft Word and actually... I mean, if you want to do the work, you could actually get to where you're typing into your Perl console and it's showing up over in Word.

Server-side computing has to get there. It's gonna get there.

But how many server-side apps are user scriptable today? Precious few. Google has a couple, you know, like our JotSpot acquisition, which is [scriptable] in Rhino...

So we're talking about something that's kind of new. I mean, we can all see it coming! But it's still kind of new, the idea, right? Why?

Because this is scary hard, from a security perspective. Heh. You're going to run code on my servers? Uh... OK...

I wrote a game that was really cool. Scriptable! I mean, high school kids were teaching themselves to program so they could write areas and monsters and spells and quests for this game that I wrote, which was written in Java and scriptable in Jython.

It's a big deal! I mean, people want to be able to write these apps.

However, I had to live with the fact that I didn't personally have enough bandwidth to come up with a decent security model to keep them from... it's a trust-based security model in my game. They write a script, they could erase my hard disk, right? So I've got to be very careful, and recognize that I can only let certain people that I trust do it. And that I've got to be prepared for really big disasters.

Because also there's denial-of-service. It's inadvertent: oh, their script is taking up all the bandwidth [or CPU or memory] on my server, and everybody else in the game is paralyzed. Right? I mean, how do you deal with it?

You've got to deal with user [i.e., programmer] throttling: memory usage, the database or datastore usage, like Amazon's computing cloud, you know, they have a lot of this stuff in place. But usually it's pretty coarse-grained when it gets started, right? You get a box, and a certain amount of disk storage, and you get the whole CPU, because how are you gonna allocate CPUs out to people when the languages themselves that are being used for scripting don't support that? (Editor's Note: obviously you can just use process scheduling, but I'm talking more about multithreaded processes like my game, or Second Life, where many users may be scripting within the same processes. It makes things harder.)

We're getting there; it's happening. But it's new. And it's hard. Because you don't want people to be able to go and get access to your company's proprietary code or resources and wreak havoc. You just want to host their computing.

So when you decide you're going to take your server-side application, with its beautiful Ajax app talking to the server, and now you want open it up: to add extensibility for your end users — they're not just scripting the client; there's scripting happening on the server that's theirs — you have to make a decision!

Namely, what language are you going to give them?

We have... see, unfortunately it's hard for me to talk about Google products, because all I know are their internal code names, and not their launch names. I can never remember them. But we have... something like that. Heh. Called... Prometheus, I think? Uh, wha... what is it?

And I think it's... Python. Right now. But of course they want to open it up, because, it's like, you don't really want to force people to switch editors, unless you want a real fight on your hands. You kinda don't really want to force people to switch languages either, right? People want to use whatever they're comfortable with.

So again, you wind up with this hosted environment, where you're supporting multiple languages; which one do you pick [first]? They picked Python. You can pick [anything], but you've got these problems to deal with. And I'm going to argue today that Rhino is actually a really good choice for this particular problem space.

OK, we've got people pooling up in the back here. Is it time to invite them in? Come on in, sit down, there's space! All right, cool. Yeah. Welcome!

So yeah. That's what I'm talking about today. Do you guys understand the perspective now, the context? I'm talking about server-side scripting, that either you do yourself inside your company, because you feel like you've got some logic that needs to be kind of "glue" logic — "scripting" — or, more importantly, you're opening it up to your users. Which means you need to sandbox it, and you need to meter it and throttle it.

Advantages of scripting on the JVM

All right. Yeah. So this is a JVM language. A JVM language can share the Java libraries, and the Java Virtual Machine. It's really cool, right? And really powerful.

Right off the bat, these JVM implementations of other languages, like JRuby vs. Ruby, Jython vs. Python, right? They get all these free benefits, that may not necessarily exist in the C runtimes for these languages.

Example? Java has a really good garbage collector these days. A generational garbage collector that's becoming an incremental [and/or concurrent] generational garbage collector... I mean, it's good! Whereas for a lot of these [C-based] languages, they use mark-and-sweep, reference-counting...

Another one is native threads. It's veeery nice to have native threads, and also have well-defined semantics for the threads and how they interact with the memory model. I mean, a lot of these [non-JVM] languages are like, "Well, we have threads, but you probably... don't want to use them." Because you're kind of on your own.

So what happens is people use process-switching; it's the share-nothing model. And that's great too, for certain situations. Provided you've got good engineering library support around it, like the java.util concurrency libraries. They can help you design something without having to do a formal proof around it to get it to work.

That helps a lot in multicore. It helps! JavaScript has no [language-level] threads, because Brendan Eich says "over his dead body". I totally understand where he's coming from on this, right? There's certainly the "promise" of better concurrency out there. Erlang, you know, and maybe STM...

But hey man, today? I mean, right now? You want to write something with high throughput, and you've got a lot of I/O to do, and it's parallelizable? And you want to get a lot of throughput on one box, because it's got multiple cores?

(shrugging) Well, threads get the job done. So if you've got it in your so-called "scripting language", it's a big win.

We've got garbage collection, threads... and asynchronous I/O, right? When Java first came out there was the whole "one thread per socket" model [actually, two], which meant that you couldn't write a webserver that could handle ten thousand concurrent requests. It didn't matter how much memory or CPU your box had. Anyone here ever tried to fire up 10,000 threads on one box?

Yeah... yeah. What happens is, the scheduler and task-switching resources for managing the threads swamp your machine. So eventually Java wrote a wrapper around the Unix or Windows or whatever native interfaces so you could get super-high throughput.

So all of the sudden, by sticking something on the JVM... Sure, you initially get a bit of a hit in performance. When these people first port a language to the Java Virtual Machine, it's usually about twice as slow, right? BUT, it's got async I/O, and it's got [native] threads, and it's got better non-pausing (by and large) garbage collection. And from there, they can make it smarter.

But they've also got the JIT. I don't know, I mean, did anybody here... I gave a talk on dynamic languages recently at Stanford, but I don't want to rehash that if you guys already know about that.

Basically I argued in that talk — successfully at Stanford, so I think that was... something — that for just-in-time compilers, it's becoming pretty clear, they have a lot better access, a lot better data at runtime about how you're actually using your program right now than the compiler ever had.

So the JITs can do all kinds of inlining and optimizations that you just can't do in a compiler. And what that means is that everybody running on this VM gets to benefit from the JIT.

So there are lots of advantages to these JVM languages. Or .NET, if you happen to be using Microsoft stuff. It's a good model.

But why Rhino?

So why Rhiiiiino? Why JavaScript?

(loudly) Who here thinks JavaScript is kind of icky? Come on, come on, be honest. Yeah, there we go, a couple of people. Yeah.

Yeahhhh... and you know what? It is! Right? Because, well, because of vendor implementation issues. That's one [reason]. Also, Brendan was kind of forced to rush it out the door. You guys know... back at Netscape, when they built JavaScript, it was called, um, LiveScript?

OK, well, as it happens, you guys are not "representative". (laughter)

And so, Netscape kinda looked at it, and said: "Yeah, well, we did say Scheme, but, uh, there's this Java thing, this giant marketing thing, and we want to be involved with it." And some back-door deals happened, and then they came to Brendan and said: "Make it look like Java."

So now it's Scheme with Java syntax, right? So he had to pull a lot of all-nighters for a couple of weeks, and he came up with JavaScript.

So, you know, it's got some flaws. Some of which make you want to go scrape your teeth on the sidewalk because it's less painful. So it's true, but what language doesn't have some flaws?

The interesting thing about Rhino, which is an implementation of JavaScript in Java, is that there's only one language. You don't have to worry about vendor-implementation or cross-platform problems because... it's just Rhino. So right out of the starting gate, that's a win.

Plus, Rhino gives you the ability to work around some of the problems. A classic one is the problem in JavaScript where you can't define non-enumerable properties. Right? You know how you can go for (i in foo) ..., and it'll enumerate the keys of your object as if it were a hashtable.

Nice feature, right? And you can add properties to objects; you can go to Object.prototype, which is the root object of the whole system, and add your own function(s) there. But what happens is, you've added a function that's now enumerable in everybody's arrays, and everybody's data structures, so their for..in loops break.

Which means that fundamentally you can't install a library that's completely seamless and emulates Ruby or Python or some other really expressive set of library functions. Like, you want your map and collect, and your String.reverse, and... you know what I mean?

You can't do it in browser JavaScript, so people wind up going different routes. They either do what Prototype does, and just install it, and you're screwed if you use for..in, but you don't use for..in, right?

Or they use functions. They don't use object-oriented programming. And you know, functional programming is great and everything, but OOP, as we've discovered in programming in the large, is a nice organizational tool. Putting the function or method together with the thing that it's operating on is a nice way of organizing things.

So it's kind of unfortunate when you have to use functions, because if you have to say, you know, HTMLElement.getChildren.whatever, it gets inverted with functions: whatever(getChildren(HTMLElement)). You have to call from the innermost one to the outermost... it's "backwards", right?

Rhino winds up getting around that problem completely. We did, anyway, internally. Because it's written in Java. So you can call the Rhino interface. You can call Parser, or the interpreter, or the runtime; you can do whatever you want.

JavaScript has non-enumerable properties; it just doesn't let you add your own. It's just a language flaw, right?

That [defineBuiltin function] enabled us, in the project I'm going to be talking about a little bit later here, to implement all of Ruby and Python's runtime — all the functions we liked — in [server-side] JavaScript, in a non-intrusive way. We were also able to implement a class system, and all this other stuff.

So Rhino is not browser JavaScript.

Man, we've got more people pooling up at the entrances. You guys are welcome to come in, squeeze in and sit down... come on in... welcome. There's still space. Especially up here kinda in the front, in the middle, where nobody wants to sit. But trust me, it's better there.

So yeah. Rhino's history: it's like ten years old. Or more? More than ten years, maybe. It started inside Netscape, side by side with SpiderMonkey. A lot of people have been hacking on it. Rhino's pretty robust.

Rhino at the shootout

I have a question for ya. I did this "JVM shootout" like three and a half years ago. I was kind of tired of using Java for scripting, and I wanted to look at all the JVM languages. So I did this game. You know about the game Sokoban? I would have done Sudoku if the craze had hit then. It's a little dude who pushes these blocks around these mazes?

Well, I reimplemented this thing, which is about, you know, six or seven hundred lines of Java code. It had a [user] interface, and a little search algorithm that had him chase your mouse. It was just big enough of an application that I could reimplement it in like 10 different languages, and actually compare how it was speed-wise, how to use them [the languages], how well they interoperated with Java... it was an actual apples-to-apples comparison.

Most of them really, really, REALLY stank. It was baaaad. I mean, there are like 250 JVM languages out there, but most of them are just complete toys. But there were ten or so that were actually pretty good. You could do anything in them, and they had decent performance, and they were good, right?

And it [the shootout] kind of petered out, because it started looking like Rhino-JavaScript was going to win. I had this sort of solution selection matrix of criteria where... it was kind of a heuristic function where I weighted all these terms, right? Just to kind of get a feel for which one [was best].

And I wanted JRuby to win. You should never go into these comparisons wanting one of them to win, because, you know, you're either going to bias it or you're gonna be disappointed. JRuby at the time was really slow. It's much faster now, and everything, but at the time, it was so new.

Jython was good, but it wasn't fast enough, and the author of Jython had gone off to greener pastures.

Rhino! Rhino had good tools, and it had good performance, and it was... JavaScript! Eeeeww!

So I never even really... I published it, but I didn't leave it up. I'm actually going to bring it back up again soon; I'm going to update it and do a couple of new languages. Because I find this an eternally fascinating question, which is: what is the next big language, especially on the JVM, going to be?

Domain-specific languages

Java will always have a place. But I think there are domains, like Java Swing, you know? The Java GUI stuff? Java's really not very good for that. We've kind of figured out that Object-Oriented Programming doesn't work that well for UIs. You want it to be declarative. HTML showed that you want a dialog, with a title bar, and a body, and it nests, to match the [UI] tree.

That works really well. It's succinct. Even in HTML it's succinct, right? Whereas with a [Java-style] object-oriented language, you've got to say, you know, createContainer(), addChild(), addChild(), addChild(), 'til the cows come home. And it doesn't look anything like... you can't pattern-match it and say "ah yes! this looks just like my UI!"

So people write these wrappers around Swing. Like there's Apache Jelly, which wound up with this XML framework to do Swing programming, that was 30% less verbose than Java.

What are the odds that XML's going to wind up being less verbose than anything?(loud laughter) Really! I mean, I was shocked. 30% less verbose. And I looked at it, too. They weren't cheating. I mean, they did the best Swing implementation in Java that they could, but Jelly was better.

So there are domains for which Java is just not appropriate. But you still maybe want to use a VM for all the reasons that I outlined earlier.

So yeah! There's room for these other languages. But which one? All of them? Are they going to solve the problem I brought up from Foo Camp earlier? To where it doesn't matter which language you're using; they can call each other, and [mix however you like?]

I mean, how's your editor going to feel about that? How's your team member going to feel about it? A lot of people don't like learning new languages.

Who here doesn't like learning new languages? Come on, be honest... (A few people raise hands) Yeah! New languages. No fun!

It's actually kind of... you should try it. (laughter) You know? It is. It's a good idea.

Dan Friedman, the guy who [was] one of the collaborators on those books — I was reading an article he wrote. Early in his career he realized that languages are fundamental to computer science and to engineering; they're really important. And he wanted to be able to learn a new language every quarter.

And after he did that for a while, he said, you know what? I want to learn a new language every week. OK? And you can actually get to the point where you can do this. Now it probably takes 2-3 months before you're actually as comfortable with the new language as you were with your favorite old one. This happened to me with JavaScript; I was freaking out for the first couple of months, thinking "this is never gonna work".

But eventually you get over the hump, and you're like (relieved sigh) "aaaah, yes." Right? You learn how to work around the problems. It's just like learning Java or whatever your first language happened to be. You've got to learn your way around all these problems, and you've gotta learn how things work, and how the libraries are laid out. And all of the sudden it becomes like breathing.

So Dan Friedman, after he said he was learning a language a week, I thought, "wow, that's pretty macho." But he said, nah, that wasn't good enough: he wanted to be able to implement a language a week. And he got pretty serious about it. Spent years researching how to do this effectively. (Well, now I'm just speculating – Ed.)

This is where I'd love to see engineers today. Knowing languages will make you a better programmer. It will! It will even if you're not using them. Just write a little application in it, and it opens your mind. [Each new one] opens your mind. And now suddenly you know the superset of all the languages out there. So it's not scary to learn new ones.

And you can also recognize situations where a language is actually the right tool for the job. Not a library, not a framework, not some new object or some new interface. A language!

Who likes to do lots and lots of manual DOM manipulations, as opposed to, say, XPath? XPath is a language, right? DOM manipulations, you know... it depends, but usually, no: not if you can get away with using a language for it.

I could talk for hours about that, so I'm not going to. But, you know... it's good to learn new languages. So I'm gonna teach you JavaScript today. I'm gonna dive in. So let's go!

The right way to do unit testing

Oh yeah. So unit testing. I mean, like, all the other stuff on this slide is like "blah blah blah", but then Unit Testing [in Rhino] — this was a real surprise to me.

I write a lot of Java code day to day, [out of the] probably five different languages I code in regularly. And unit testing was always a chore. Unit testing is a chore.

Well, not really, since you have to write unit tests for them [the static languages] too. (more laughter)

You need to write unit tests, and unfortunately in Java it's very painful. I'm speaking into the mic now, so that everybody can hear. Unit testing in Java is painful!

It's so painful that people, the Java... community, the Java world, has evolved around it. OK? They've said: "Don't use constructors. Constructors are baaaaad."

(pointed pause) I mean... what!? (laughter)

I mean, like, if you program in Ruby, say, you know that you can actually change the way the metaclass produces objects. So if you want it to be a singleton, or you want it to meet certain... you want it to be a Mock object, you just replace the new function on the fly, and you've got yourself a Mock constructor, right?

But Java's set in stone! So they use factories, which leads to this proliferation of classes. And you have to use a "Dependency Injection Framework" to decouple things, right?

And it's just, like, (panting)... We were doing business logic early on in Java. When Java came out, it was like: "Ooooh, it's a lot easier to write C code", basically, or C++. Rather than focusing on your memory allocation strategy, or which of your company's six conflicting string classes you're gonna use, you get to focus on "business logic".

But unfortunately that only took you so far. Now you're focusing on Mock object frameworks. It [Java] only took you a little farther.

Now I swear, man, doing Mock objects in Rhino is so easy! It's easier, even, than in JRuby or in Jython, because JavaScript has JSON. I didn't even know, like, the name of it when I started doing this, right? But JSON... I've gotta show you guys this. This is so cool. (flipping through slides)

OK. Down on the bottom we've got some code here. Actually on the top, too. So I do a new Thread with a new Runnable, and, uh... it sure looks a lot like Java code, huh? This is one advantage of JavaScript, actually. Java...Script, right? Ten years later it's finally becoming the scripting language for Java?

So that syntax (with an obj literal following new Runnable()) is a little weird, but there's another one here that says:

js> obj = {run: function() { print('hi') }}

So I've declared an object literal, using "JSON style". Now JSON doesn't let you do — does JSON let you do functions? Probably not, right? But I mean, fundamentally you're doing this declarative property-value list, right?

And so what I've got is this anonymous thing that has a named "run" property whose value is a function! That prints "hi". And now I can create a new Thread, with a new Runnable that wraps it, and what effectively I've done is I've used that thing as the Runnable interface [implementation], which expects a function called "run" that takes no arguments and does whatever the thread's supposed to do.

This is how you do mock objects!

I have this huuuge legacy system, right? With hundreds of static methods. Static methods are also bad these days, right? Noooo static methods. 'Cuz they're not mockable. Right? Java has changed Java. Because Java's not unit-testable. So now you can't just go to the store and [buy a book and] learn Java. You have to learn all these... fashions. You have to learn what's in vogue.

Subclassing! Not in vogue right now. You talk about subclassing, and people are like "NNnnnnnooooo, you should use manual delegation even though it's really hard and cumbersome and awkward."

And you're like, "but I just want to change this one method, and plus it's built into the language, and there's syntax for it, and it's kind of well-understood..." And they just say "NO!"

It's out of favor. For similar reasons. Oh my god...

And I'm telling ya: the reason unit testing is easy, is, fundamentally, the way you develop in a dynamic language is different from the way you develop in a static language: C++, Java... OCaml, Scala, whatever. Your favorite static language.

To a large extent, especially in C++ and Java, the way you develop is:

In a dynamic language — and this is clearest when you're writing in Emacs Lisp [because of the *scratch* buffer] — but it's somewhat clear when you're developing in a console, in Python or Ruby, Perl, Lua, whatever, you write an expression, and you give it some mock data.

You're writing a function, you're building it on the fly. And when it works [for that mock data], you're like, "Oh yeah, it works!" You don't run it through a compiler. You copy it and paste it into your unit test suite. That's one unit test, right? And you copy it into your code, ok, this is your function.

So you're actually proving to yourself that the thing works by construction. Proooof by construction.

Obviously you still need your unit tests (er, I meant integration tests – Ed.), because there's going to be higher-order semantics, you know, calling conventions between classes and methods...

Whew! This room is really filling up. Um, is there anything we can do to help here, guys in the back? (Tech guy says something inaudible in the video) Yeah, please! There're more seats here. I just want to... I don't want to get to where people can't even make it into the room.

Yeah, so unit testing. I know you guys all hate unit testing. So did I. Or you say, "I looooove unit testing," but then, you know, your test coverage is still sitting at like 80%.

I'm telling you, man, this is a huge, huge thing. It changes the way you do your development.

Rhino's not Ruby

And oh, yeah... I'm going to be talking shortly here about Rhino on Rails, which is this thing that I did... it's not Rhino on Rails, actually. It's actually, I called it "Rhino's not Ruby". Because I got kinda burned at Google for using Ruby. Yeah. Uh, for good reasons, good reasons. But they were like: "No."

And so of course I called it "Rhino's not Ruby": RnR. Because people know JavaScript; they're kinda comfortable with JavaScript, so they were OK with it. So I had to port Rails; it was kind of a lot of work, but, you know, well it works! We're using it here internally; it's nice. I mean, I actually know it's nice, because six months went by and I didn't look at it for those six months. And for this recent project, I picked it up, and I was, like, is this gonna be gross?

But actually, it's really pretty nice. Because you've got all the Java libraries, and all the integration with Google stuff. It's cool. I'll try to open-source it this year, if I forget to say that later on.

I've diverged from the slides. I'll come back to RnR shortly. Basically, I got unit-testing religion. That's the end of that sort of stack.

If you can do it easily, and you don't have to rewrite your application to be unit-testable? Man. That's a big difference.

So why would you not use Rhino for server-side scripting?

Well, it's not super-duper fast right now. It's on the order of about twice as slow as Java, depending on what you're doing. So if it really has to be super, super fast — use C++! Right? Naaaah, you can use Java. Like I was saying the other day [at Stanford], it's widely admitted inside of Google — there's this whole discussion, is Java as fast as C++, right? And Google Java developers definitely admit that it's as fast as C++. The C++ people, well... yeah. (sigh)

Let's see... if you're writing a library, then Rhino's actually not so good right now. There is no standard library interface for scripting languages. We haven't got there yet. It's all, it's all related to what I told you about before, you know the calling interop. A lot of these [languages] have their own package systems: their own import, their own require, right? So if you're gonna write a library, you should probably still write it in Java. Maybe.

If you're doing a framework, where you're defining how things are called: whether we're calling you, or you're calling us, then it's OK.

And if you really hate JavaScript, then that's, you know, that's fine... But keep in mind, again, that you may be providing something for your end-users. If you go out to a high school and you survey people, and you ask, "So what language are you learning? How are you teaching yourself programming?" It's a sure bet that a lot of them are doing Ajax stuff. Right? A lot of them are doing JavaScript.

If you want to make your end-users happy, and you want to immediately capture a very big user base, then no matter how you detest JavaScript (and again, Rhino-JavaScript's really not as bad as browser JavaScript, it's much better), your users probably will prefer it. It's something to keep in mind.

Scala is a very strongly typed language for the JVM. It's from researchers in Switzerland; they're professors. It's from sort of the same school of thought that static typing has evolved with over the last fifteen years in academia: Haskell, SML, Caml, these sorts of H-M functional languages.

And Scala's interesting because it actually takes a functional static type system and it layers... it merges it with Java's object-oriented type system, to produce.... Frankenstein's Monster.

I've got the language spec here in my backpack. Oh, my god... I mean, like, because it's getting a little bit of momentum, right? So I figure I've got to speak from a position of sort of knowledge, not ignorance, when I'm dissing it. (Heh heh.)

And so before, I was like: "Oh yeah, Scala! Strongly typed. Could be very cool, very expressive!"

The... the the the... the language spec... oh, my god. I've gotta blog about this. It's, like, ninety percent [about the type system]. It's the biggest type system you've ever seen in your life, by 5x. Not by an order of magnitude, but man! There are type types, and type type types; there's complexity...

They have this concept called complexity complexity<T> Meaning it's not just complexity; it's not just complexity-complexity: it's parameterized complexity-complexity. (mild laughter) OK? Whoo! I mean, this thing has types on its types on its types. It's gnarly.

But the funny thing about Scala, the really interesting thing — you guys are the first to hear my amazing insight on this, OK? — is: it's put the Java people in a dilemma. There's a reeeeeeeal problem.

The problem is, the Java people say, "Well, dynamic languages, you know, suck, because they don't have static types." Which is kind of circular, right? But what they mean, is they say: No good tools, no good performance. But even if you say, look, the tools and performance can get as good, they say, "Well, static types can help you write safer code!"

It's... you guys know about those talismans? The ones, where, "What's it for?" "To keep tigers away"? (some chuckling) Yeah? And you know, people are like, "How do you know it keeps tigers away?" And your reply is: (sneering) "Do you see any tigers around here!?" (minor laughter)

So this is what... OK, so for a long time, for many years... and you know, I've written more Java code than most Java programmers ever will. (Editor's note: nearly 1M lines in production. Ouch.) So trust me. I tried. OK? I'm not just coming in and saying "I don't want to learn Java." No. I know Java as well as the next person.

But I come to them and say, let's do proof by – say, argument by example! You know, an existence proof. IMDB is written in Perl, right? Yahoo! – many of their properties are written in PHP. A lot of Microsoft stuff's written in VB, right? ASP .NET? Amazon.com's portal site is Perl/Mason.

A lot of companies out there are building big, scalable systems – and I mean scalable in the sense of throughput and transactions, stuff like that, but also scalable in terms of human engineering — in these dynamic languages with no static types. [Using nothing more than good engineering principles.]

So... isn't that a demonstration that you don't need the static types to keep those tigers away?

And they're like: "Well! But! What if... what if a tiger came?" (laughter) Right? "People need shotguns in their house in case a bear comes through the door, right?" The Simpsons made fun of that. (laughing continues)

Yeah. So, you know, for a long time, I was like: "Yeah, yeah, yeah. OK. So tigers could come. Fine."

Scala, now, is the tiger that's going to kill Java. Because their [type-talisman] argument now has become a paradox, similar to the Paul Graham Blub Paradox thing, right? Because they're like, "Well, we need static typing in order to engineer good systems. It's simply not possible otherwise."

The Scala people come in and they go: "Your type system suuuuuucks. It's not sound. It's not safe. It's not complete. You're casting your way around it. It doesn't actually prevent this large class of bugs. How many times have you written catch (NullPointerException x) ... in Java? Our type system doesn't allow [things like] that."

Our type system does what you said your type system was doing.

So, therefore, you should be using it! ∴

And the Java people look at it and go: "Wehellll... (cough cough)... I mean, yeah, I mean... (*ahem*)" (running finger under collar, as if sweating profusely) They say, "Welllll... you know... it's awfully... cummmmmbersome... I..."

"We can actually get around the problems in practice that you guys say your type system is solving through Good Engineering Practices."

(laughter begins to grow)

HA!!! (I point accusingly at the audience, and there's more laughter)

Yeah.

So Scala is creating a real problem for [Java's] static typing camp now. Because their last little bastion of why they're using it, the whole tigers argument, they're like, "Ah, well... we... we keep shotguns in our house." [This is what they've been reduced to.]

OK? Yeeeeahhhh...

So back to dynamic languages!

But my point was — from a previous slide actually — it's very interesting. See, I wrote this Rails port, and it wasn't... I never got it to where it was quite as succinct as Rails, because JavaScript has curly braces and a little bit of extra syntactic cruft. But it was close!

And then we used this framework to build this web app internally. It was for external consumption. It's kind of a long story that I won't go into. But we had like 20 engineers on this thing, for close to a year. And we had a huge application. I mean in terms of user functionality: Ajax-enabled pages, server-side persistence stuff... it was a big app.

And it was, like 40,000 lines of code, including the templates and the client-side JavaScript. The whole thing! OK? I mean, you add in unit tests, you know, you add in everything, including build files and stuff, and this thing was up to like, maybe 55,000 lines of code.

So unfortunately we have thirteen minutes left. I'm sorry. So let's really quickly go through some of the really cool things about Rhino, the technology here.

You can JavaScript from Java, and Java from JavaScript. Guess which one's easier?

Obviously calling Java from JavaScript is easier, because Java's really cumbersome. It doesn't have anything to help you, so you have to do basically what I was talking about with Swing earlier. JavaScriptObject j = new JavaScriptObject() You know. JavaScriptObject, Context.enter! You've got all this stuff on the Java side. 'cuz it's Java.

But, uh... but it works! And you can do both directions. Here's an example of a Java program to bootstrap... actually I believe this is completely standalone; it works out of the box. It's a Rhino Demo:

This is what you need to do to create a JavaScript object called foo that has a function called a. A property called "a", sorry, whose value is "hello".

So what you do is you call Context.initStandardObjects(), which sets up the JavaScript runtime. You only have to do it once. And then you call newObject to create a new JavaScript object. And then you call evaluateString to evaluate it in the context of this object.

It's one example of how you do it, but it's not too hard. You can call back and forth.

So that means that anything that was written in JavaScript that you feel, oh Gosh this really needs to be componentized, you need to stick it down in a Java library for whatever reason: you can do it! You can migrate code back and forth between the JavaScript layer and the Java layer. This is true for all JVM languages, I think.

Uh... this is the actual code that I was referring to earlier, where you can define non-enumerable properties. I called it defineBuiltin. There's some closure stuff going on here... I don't want to bore you guys. (Editor's note: Function.bind() based on Douglas Crockford's original)

Runtime delegation: this is one of the reasons unit testing is really easy. You guys know about Smalltalk, uh, method-missing? [doesNotUnderstand actually] It's method_missing in Ruby. It's the... if you call an object — I think Objective C probably has something like this too.

You call an object, and you say: "I'm calling foo", and it says: "I don't have a foo". Right?

Normally, what happens when you do this? In Java it goes *BARF*. As it should, probably... unless what you really wanted to do was delegate to some other object. Right? "Design Patterns". Say you want to write a Bridge that says: "Oh! You're calling foo, but you don't want to call it on me. You want to go to the game server, call it there, marshal it, send it back. We'll pretend it's a remote method call."

Right? There's a lot of stuff you've got to go through in Java to do stuff like this. In JavaScript — as you all know, if you're using dynamic languages...

Man, we've got a huge pool of people in the back. It's getting pretty rough. But we're almost out of time! Fortunately. Heh.

OK, so let me tell you a little bit about embedded XML. It's kind of interesting, kinda neat. This is supported in Firefox, in some browsers. It's a spec that Adobe and some other people, BEA, put together.

When I was a kid — when I was a kid, jeez... When I was twenty, it feels like when I was a kid — I used to have tropical fish. And my brothers and I noticed two things about tropical fish.

One is that they die, because we're not in the tropics. (some laughter) Sad.

And the other is that if you put a bunch of different species of tropical fish in a tank together, they ignore each other... except for the ones that are the same [or nearly the same] species. They bite each other. That's what they do. Fish bite each other. They have a pecking order, right?

JSON and XML are muscling in on each others' space, and there are bristles, OK, and it's so silly! It's silly. The whole thing, right? I mean, XML is better if you have more text and fewer tags. And JSON is better if you have more tags and less text. Argh! I mean, come on, it's that easy. But you know, there's a big debate about it.

Nevertheless, sometimes XML is appropriate [in JavaScript], especially if you're loading it from a file or whatever. These literals are interesting. And so it provides new operators and new syntax for actually going in and... it's kind of like XPath. Except it's JavaScript-y.

And I tell you: it is the worst documented thing on the planet! It's horrible, man, working with E4X initially. But... eventually you can figure it out. And I have a document that hopefully I'll be ready to release pretty soon, that actually covers it pretty well. And Adobe has some good documentation for it.

And then eventually it clicks, like learning any other language. This is a minilanguage. And you go: "Ha, I get it! I get it. It's not as crazy and dumb as I thought. It actually works."

It's kind of a neat feature. You guys know other languages that embed XML? Scala does. I don't see all of you using that, but C# 3, I think, does XML? Coming [soon]? (Editor's note: it's apparently been deprioritized by the C# team, although VB 9 has them.)

Anyway, it's kind of an interesting approach.

Inside the Rhino

All right. So this is Rhino. Now you know. After I explain this diagram, you'll know what you need to know about Rhino to talk to other people about it.

You start with some JavaScript code. It goes to a parser. That turns it into a tree. A syntax tree.

Rhino's parser today, currently, immediately begins the next step of rewriting it as code generation. Right away, as it's parsing. Now this is a problem. Because if it takes an if-statement or a switch-statement or a for-loop, and it generates sort of assembly-language like jumps to targets? And generates labels, you know, converts it into sort of three-address code, that's eventually going to actually become three-address code: assembly or bytecode.

Then it kind of sucks if you're trying to use the parse tree for your IDE. To, like, syntax highlight, or pretty-print, or show errors and warnings, or whatever. Unfortunately a lot of languages — most languages — do this because they're written by compiler guys and compiler gals. And they don't see the value. But unfortunately we're all doing more and more processing of code. Language tools, right? Frontend stuff.

So I rewrote Rhino's parser recently, and I'm currently fixing the code generator. And I'm gonna get it out into the Rhino mainstream in a couple of weeks here. Because my project at Google is doing a lot of code processing. And it's a faithful representation. So if your big beef about Rhino is that there's no Visitor over the AST: I'm fixing that.

And then there are two paths here: you see one on the left that goes code generator to bytecode. And there's the bytecode, or pseudo-bytecode, for the JavaScript code up there. And then it goes to an interpreter. The interpreter is a big loop. Bytecode is this [roughly] postorder traversal of the tree, that flattens it in a way that allows you to push onto a stack to evaluate the operands.

It's all actually very important; you should go read up on how it works if you're not familiar with it, or if you've forgotten since you first learned it.

And the interpreter is actually pretty fast, because it's a loop. There's not a lot of calling out. I mean, there are some calls out into the runtime, but mostly it's this loop: push, pop, push, pop. So the JIT picks it up and can optimize it pretty well.

The reason that there's two code paths here, the reason that they wrote the interpreter — they originally had just a classfile compiler — was that compiling to a classfile is this batch/static operation, where you want it [the resulting bytecode] to be fast. You want to do the standard, classic compiler optimizations. You want to generate the control-flow graph, you want to eliminate dead code, you want to do good register allocation.

In JavaScript's case, it's often possible not to generate a closure. You can actually use Java instance variables and Java local variables instead of these heavier-weight JavaScript [activation objects]. Because at the logical level, JavaScript doesn't really even have a stack. It has object allocations on the heap; those are your Function invocations. Sloooow. Right? Because [in comparison] the Java stack translates to the C stack.

So the fact that the compiler can go through and optimize away a lot of this JavaScript dynamic stuff that's provable you're not gonna need, well, that's nice! But it takes time. The interpreter is a path that allows you to dynamically develop: load code in and see how it's gonna work right now, at the unfortunate expense of the Rhino people having to maintain these two code paths. But, you know, there's a lot shared between them.

And that's it! The script runtime implements all the JavaScript, you know, Ecma spec stuff: Array, String, Boolean, Date, the Math functions. And a lot of it just delegates down to Java where possible.

Pretty clean! Pretty standard. It's a pretty standard compiler, interpreter and runtime. You're gonna see this if you dig into your favorite JVM language. You'll probably see something similar to this. This is actually more mature than a lot of them. A lot of them start off by interpreting the parse tree, and it's slow from the method calls.

So this is why Rhino's fast. Now it could be a lot faster, and we're working on it. Hopefully, you know, these things can be as fast as Java, in the same way that Java made the claim that it can be "as fast as C++". And for long-running applications, that's usually true. Especially with parallelism, right? Threads. And especially if the JIT has a chance to look at the actual code paths being used and compile them down into machine code specific for that code path, as a fall-through.

Obviously for benchmarks, where they fire something up and run a loop a thousand times, or whatever? C++ is faster because the JIT hasn't had any time to kick in and evaluate what's going on. But for long-running services — which is what we're all writing, yeah? At this Ajax conference — the JIT will kick in. And your Rhino code now will get very close to where Java's performance is. (Provided you're not doing number-crunching - Ed.) So don't worry about that so much.

So RnR, I already talked about it. It doesn't have a database backend yet, because we're using Google's internal store, like Bigtables. Which is why I haven't open-sourced the thing yet.

It's weird: somebody told me the other day, they sent me mail and said: "I think you're today's Paul Graham". And he meant this in the most negative connotation possible. "You're today's Paul Graham, and RnR is the next Arc."

I was like, "What!?" And he said, well, he's a server-side JavaScript guy. I mean, there aren't that many, right? Most of us are thinking client-side. But he's a server-side JavaScript guy. And he goes to people and says, why aren't you using server-side JavaScript? And they say: "We're waiting for Steve Yegge to release RnR."

And I'm... this is news to me! I'm working on... stuff, you know. Work. And this [open-sourcing RnR] is part-time and everything.

This year, now that we know people are interested in it, we will release it. (At least we'll try for this year - Ed.)

It's just a little weird, right? Because Sun hired the JRuby guys, and they're doing JRuby on Rails, and it's eventually going to be part of the Java Development Kit. It's gonna be, you know: it's Sun's lightweight answer to EJB and all those giant frameworks. You want to build something quickly and use the Rails model, well, run Rails on the JVM!

So I thought: if JRuby on Rails had been (a) ready when we started using it [i.e. writing RnR], and (b) Google would let me use Ruby, then I would have used that! So RnR was like a transitional thing.

But... again, you know, I think that there are other people in situations where you really prefer to use JavaScript. So yeah, I guess I'll... open-source it. We're working on it.

This is the last slide, by the way; I know you guys are tired. We're doing a lot of work on it. I'm working on it personally, I mean working on Rhino. Because I think it's all right. I'm used to JavaScript now, and I think it's a good implementation. It's a good compiler, so it's good practice for me, to learn how compilers work.

We've got a debugger, but we're making the debugger better. We've got a sandboxing model, but that could definitely be wrapped up and made available to you folks.

We'd like to open-source our JSCompiler: the thing that compresses JavaScript for Google Maps and GMail and stuff. I know there are some open-source ones out there. We don't think that it's competitively in our best interest to keep the thing internal. It'd be better to get it out there so you all benefit from it, and so you can all hack on it, right? We're working on open-sourcing our JSCompiler and other stuff.

So that's it! I wanted to cover Rhino, but I also wanted to leave time for questions. And I've left you (looking at big LED clock in the back) one minute and sixteen seconds for questions. Sorry about that.

So really quickly, if there are any burning questions, I'll repeat the question and try to answer it. Otherwise feel free to come up afterwards and chat.

The short answer is: it imposes a tax on the systems people, which means that it's not completely self-contained within the team that's using it. And for that reason, primarily, it's really not a good idea right now.

Any other burning questions?

Q: Do threads suck?

Well, you know... they're... you know... Yeah. But I mean, what other options do you have? I mean, you have multiprocessing/share-nothing, which is heavyweight and it requires more work.

So I use threads. I'd prefer something better, but they're what we've got today.

Q: There are some guys that I work with, and one of their comments on JavaScript lately, since I've been wanting to use Rhino because I love JavaScript... what they brought up is that JavaScript is becoming a lot like Python, and that may or may not be such a great thing. I wanted to know what you have to say about that.

Ah. OK. Well, yeah, it already has borrowed some stuff from Python in JavaScript 1.7: yield, Array comprehensions, destructuring assignment. And these are good features. They're good. They're not going to change it to be syntactically whitespace sensitive, right?

I don't know. The guys working on it have really taken off in completely different directions from Python. They're looking at an optional static type system, so in that sense it's maybe more like Groovy. And they're looking at maybe fixing some of the bugs.

But I don't know how that's going to evolve yet. Because there's obviously a lot of people who have skin in the game, a lot of people interested in affecting the way the spec evolves. So it's all kind of up in the air right now.

All right, so we're really out of time. I'd like to thank you for coming. And please come up afterwards. Thanks!

94 Comments:

Nice talk. I'm currently taking a masters on languages engineering just wished some (any?) of my teachers would go anywhere near all this juice you always show in your presentations.

Kind of wonder why you never referred to yacc or antlr in your blog :) Really agree with you on learning new languages. I love it. Ruby, java, python, erlang, haskell, ruby again, prolog, perl. I'm not a guru at haskell but i like it a lot and I'm really thinking in trying scala.

Scala is like ruby and erlang and haskell all together. Sounds delicious.

Hello 2008, 1996 is calling, they want their appserver back. Welcome to BroadVision 1998Been there done that.... Except in 1998 everyone wanted Java and we already had an entire application system that could be scripted in JavaScript

Also, ASP has had JavaScript on the server-side since the beginning of ASP. Which is like 15 years or so now. But of course there is no embedded XML, or plans for it either. But through COM you can optimize with C++. Several smart companies do this still today even if this kind of ASP is considered "classic".

Aren't scheme and smalltalk classic? Is it a refusal to die, or withstanding the test of time?

page_failed: actually Scala is a pretty cool language. It's amazingly clever, and as far as I can tell it has all the hallmarks of being written for "real" programming use. I love the implicits and the case classes. But the language is frighteningly huge, so I'll stick with my assertion that it's going to have trouble taking off.

ANTLR's cool - I've been doing stuff with it for years, and I may even try writing an elisp backend for it at some point.

trevorfsmith: sorry, I should have mentioned that lots of folks have been doing it for years.

ajai: check out "LiveConnect" - I think IE may be using something like that, and yes, it's been around a while.

Introducing the Google Web Server Toolkit. Now you can write your server code in Java and have it compiled down to Javascript so you can run it in Rhino on the JVM. Need access to raw platform APIs? Just use JJSJSJNI!

I wanted to ask if you have actually used Scala for anything semi-serious (a couple thousand kloc)? I have, and while the description of the type system is scary, it is pretty intuitive to use and seldom seems to get in your way. I mean, I like dynamic languages as much as the next guy, but I gotta say Scala's type system works pretty well if you're into that kinda stuff.

Personally, my biggest beef with the language is actually that so much is implicit, it makes it hard to read sometimes.

This shit rant almost triggered me to respond to the overwhelming amount of under-qualified nonsense... almost.

sigh I guess you'll leave it up to people like us (e.g. the Scala IRC channel, Haskell community, etc.) to respond to all the misconceptions that the learners come to us with sourced from 'guess who'.

The straw man was a delightful classic.

Here's a quiz for you Steve. I have a terminating function that has this signature:

forall A. A -> A

Now write some unit tests for it. You can't, can you? That's because this type signature is called 'once type inhabited'. There is only one implementation for this function. This is called a proof of correctness Steve. Yes, unit tests are a poor man's type system Steve. However, sadly, there is a well known problem which I expect you do not understand called the 'Turing Halting Problem'. This means we cannot obtain a proof for the correctness of the *general* program Steve. So, for some programs we can, but not for all.

That's why we unit test Steve - to get a failed proof of the contraposition of a property of a function; the next best thing after proof of correctness of a function - they serve the same purpose. Now you might dilute your unit tests with some impoverished method (assert this and assert that perhaps?), but this does not take away from the fact that you are indeed seeking out properties of your function.

Have you ever looked at automated testing Steve? You know, QuickCheck or Reductio (No, JUnit is not automated Steve, despite the hijacking of the adjective). There is a lot to learn there Steve. You do embrace learning don't you? Are you interested in learning about software correctness verification?

Have you ever seen a type-safe sort Steve? You know, it fails to compile if the result of the function is not a sorted list. Think about it Steve. This is a proof of correctness as opposed to a failed proof of incorrectness (unit tests). Would you like me to elaborate Steve? I'm more than happy to, but it is going to challenge most of what you believe, because it is wrong; vastly wrong and this is fine Steve; I'm not out to correct every idiot in the world, but please think of the noobs; PLEASE, they don't deserve to be misled by an amateur posing to the contrary. It's just not fair to them.

My patience is running thin; I wish you'd get a clue before opening your trap Steve. Please stop it. No really, I am begging you to learn "some stuff" before making wildly erroneous claims about it. It's only fair don't you think?

Steve: I think actually trying using Scala a bit more might colour your opinion a bit more favourably. In essence the language specification is complex because of the way that the language works - largely the static typing issue - but also because the language hides so many of those complexities from the coder.

Generally speaking I'd say that Scala's language complexity stems mostly from its focus on being practically useful - the language spec is complex because it lets you use a static language in more-or-less the same way as you use a dynamic one. I don't think the spec document is a good introduction to the language - the best I've seen is the book that is coming out soon (available on one of those pre-release PDF schemes) being written by Odersky and others on the language. It comes much more from the perspective of how you use the language, rather than how its guts work under the hood.

"We can actually get around the problems in practice that you guys say your type system is solving through Good Engineering Practices."That one made me laugh.Oh, sure, I'm sure I will not forget to use the fuzzy "Good Engineering Pratices" thing because I'm such a good programmer !Seriously !But here's the thing : as a programmer you CAN'T trust yourself ! You can't trust others neither. That's the n. 1 lesson of good engineering !You need a way of eliminating the basic errors of your code *automaticaly*. Basic error means : things like adding an integer and a string.That's why you need a type system. Simple.

Most of the arguments for a JVM are arguments for libraries, maybe like the DLR. Generational garbage collection and threads can mostly be in libraries. Yeah, the generated code has to be written to be compatible with the GC and threads, but that's doable.

The exception is JIT. But if you like JIT, why not just compile to machine code in the first place? Eval will be slower, of course, but for long-running services, machine code is smaller and faster than an app + JVM, often enough.

You might like to take a look at Clojure, as well, if you haven't already. It's a weird Lisp-1 for the JVM, and has both an STM system and something vaguely similar to Erlang message passing. It has also recently become rather fast, and it someone has written a backend for the (Common Lisp) SLIME Emacs thing for it.

Some of the domain-specific language bits you talked about remind me of a PDF on Steps Toward the Reinvention of Programming (First Year Progress Report)I read recently from the Viewpoints Research Institute (headed by Alan Kay). The kind of stuff they talk about suggests that programmers should create their own languages to express the problem domain at hand, and then go solve that problem in the domain itself. For a first year, they have some very intersting, concrete results (e.g. TCP/IP in 200 lines, and Cairo in 500 lines).

(BTW: There's more to Internet Computing than finding The Next Great Language. Visually, thins thing blows chunks. It's difficult enough to encourage people to read the erst of the comments, without the "Leave your comment" box being in the upper left-hand side. But I digress.)

You kinda miss out "beauty."

Now, I'll admit that C++ isn't beautiful. Hell no. Given two years or so (and it's still relevant in the real world, so two years is quite possibly what you'll get), it has an ethereal almost beauty. Which ECMA/Javascript derivatives don't have. They're just plain butt-ugly. We're left with a choice of the most beautiful language that anybody out there will let us program in -- which I would suggest is Python, with all its faults -- or dealing with crap.

Java is full of crap.

Javascript (in all its flavours) is full of crap.

I think you're missing at least two points about Scala.

(1) Like C++ in 1985 or so, you don't have to suck the whole lot up in one go. Nobody but you cares whether the language reference manual could be used as a door-stop for the palace at Nineveh.

(2) There's a huge number of Java prorammers out there who are looking for something slightly less brain-dead. Scala seems to me to be the first realistic JVM language that can give them the option. Think Perl, think Python, thin Ruby if you will. I predict that Scala will do this for the JVM inside the next eighteen months.

I also predict that your version of Rhino will die inside the same period. (Nothing personal -- I just don't see the point of it. Well-crafted, JIT, optimised, server-side glue for morons? Huff it up...)

And you really, really, should stop posting links to things like http://en.wikipedia.org/wiki/Greasemonkey.

That's an exceptionally bad idea for somebody working at Google.

On further prdiction: inside two or thrree years, Google's business model will be dead, precisely because people will block out advertisements.

I am not insecure Michael. I am annoyed when an innocent beginner comes in to the Scala IRC channel with a question that makes no sense whatsoever, because he read some rubbish claims on Steve's blog. I also wish Steve would clue up a bit on the topics he talks about since I expect he is also misleading others. I don't think Steve realises how out-of-his-depth he is, so strong language is necessary to somehow ram the point home.

I hope he will think about my questions and perhaps have a couple of "oh crap, I'm so wrong" moments and subsequently shift position.

All of his arguments have been debunked over and over (mostly because they are so easy to do due to their lack of sophistication), though he continues to ignore them - assumingly because he doesn't understand them - he seems capable of mocking the fool's arguments. I think a bit of brutishness is in order.

If it means I am condescending because I claim that Steve is full of shit and his under-qualified words are detrimental to the programming community, then I accept the charge. I assume the best of Steve and that he will address the truth value of my propositions (though they are very weak at the moment - he ignores the evidence, so I'd I'll only be preaching to the choir) rather than worry about whether his feelings have been hurt by hitherto "condescension".

Homer: Not a bear in sight. The Bear Patrol must be working like a charm. Lisa: That's specious reasoning, Dad.Homer: Thank you, dear. Lisa: By your logic I could claim that this rock keeps tigers away.Homer: Oh, how does it work? Lisa: It doesn't work.Homer: Uh-huh. Lisa: It's just a stupid rock.Homer: Uh-huh. Lisa: But I don't see any tigers around, do you? [Homer thinks of this, then pulls out some money]Homer: Lisa, I want to buy your rock. [Lisa refuses at first, then takes the exchange]

Alright, you've convinced my to give rhino a whirl. Javascript on the server sounds like a lot of fun.

So I have rhino up and running, and I've played around with the shell for a good 10 minutes. It seems pretty rough compared to say irb or python's shell. No history or tab completion is pretty weak. Also there doesn't seem to be any good builtin introspection tool.

Do you have any suggestions on how to make this better? Are people working to improve the shell, or could it use some more attention?

By the way, thank you for js2-mode. It has significantly improved the javascript development experience at my company (all the developers live emacs).

- what the user on the IRC channel asked, and exactly how their question was my fault.

- which of my claims are "erroneous" and "wildly false".

I'm assuming you know the difference between an opinion and something that's formally provable. So I ask you: which of my claims is formally provably false? We're all ears.

Right now it sounds to everyone reading, and I do mean everyone, like you're throwing a tantrum because of a perceived slight, and it isn't helping your credibility.

As it happens, I like Scala, so far at least. But I actually got a note verging on a death threat from a colleague for considering using it at work. This was a smart person at a smart company. In other places I suspect I'd have encountered even harsher resistance.

There's far more at stake here than you realize. For every person trying to advance the cause of Scala, there are a thousand who will fight it to the death. People won't listen to academic arguments over something they feel threatens their livelihood, and most folks feel that new languages (especially complicated ones) do just that.

I've been beaten up and down by coworkers at enough companies over trying to use languages like OCaml and Haskell that I'm not about to stick my neck out for Scala. I'm done with being the flag-bearer for academic static languages. If everyone starts using it, sign me up. But Scala enthusiasts have a much longer road ahead than they could possibly imagine, in trying to make this thing successful.

My primary advice to you, and I'll assume you deserve it, is this: learn marketing. Learn everything there is to know about it. Then learn sales. A good salesman doesn't freak out. A good salesman is calm, funny, friendly, inviting, and persistent as all hell.

You're not off to a good start, but it's not too late to make a change.

We are using helma (helma.org) since 2000, which provides a server side javascript framework (using rhino). I also started to grow the idea of porting the rails framework to helma/rhino, and i was excited about reading about your efforts at google. Would be nice to learn from your project, which is pretty hard as long as it's just "something you would like to open source". maybe there are other ways than open sourcing the whole thing. maybe you could start to share just some parts of it. Would be realy interesting to learn from your work. If you are interested you can take a look at my efforts at: dev.helma.org which relies on helma-ng

I'm sorry, but the Scala argument in Paper Tigers is incorrect. If Java-style static typing catches 90% of the bugs that can be caught with 5x more complex type system, I would say some trade-off for simplicity is acceptable and it is reasonable to just use the simpler type system.

The whole argument of static vs dynamic typing seems wrongheaded. It should be all about minimal code size. Normally you want the shortest program possible. But you also want some confidence that your program works, hence you write unit tests. If tigers are too scary, than you can use the optional static typing, write more unit test, etc - all at the cost of increasing code size.

We've been trying to write testable javascript at http://guardian.co.uk/ for the last six months, and have come up with some good results (see pluck.js, included on most pages).

I was wondering how you write your tests? I ended up writing a unit testing framework which can be found at [shameless plug]

http://code.google.com/p/rhinounit

If you had a chance I'd appreciate any feedback on that project (marketing or code), or on how you do your stuff.

I also wondered what your approach was to object definition/construction in JS. We ended up going with a quite Java like construct, mainly so the devs didn't have to venture too far from their comfort zone, mixed in with a bunch of object literals.

Finally, what would you do to fix the language? I'd love to get rid of 'this' defaulting to global, and things like that.

I find, as both a Scala library author (lift) and a Scala library consumer (a bunch of apps), there's a material difference in how much time and effort the type system takes.

As a library author, the type system requires a lot of work and I often spend a whole day writing very little code, but define complex types (this is especially the case when I'm designing DSLs.)

On the other hand, I find that when I'm consuming libraries, I spend very little time knowing or caring about the type system... it's a lot like coding in Ruby or JavaScript.

I also see lines of code numbers for apps similar to those I saw when I was doing hard-core Rails coding. I see tests that are actually shorter because I'm testing for logic, not for types, method names, etc.

Like Ruby's meta-programming, Scala's type system is complex and powerful. It's used by people who spend time and effort designing things.

Thanks for the blog, which is very entertaining, as always. Let me just take the opportunity to correct a couple of inaccuracies:

You write: Scala is a very strongly typed language for the JVM. It's from researchers in Switzerland; they're professors.

No. Scala is from one EPFL professor and his team. Besides being a professor that person has also done a lot of Java stuff. Among other things he wrote the javac compiler that almost everybody in Java-land uses to this day.

You write: It's from sort of the same school of thought that static typing has evolved with over the last fifteen years in academia: Haskell, SML, Caml, these sorts of H-M functional languages.

Not really. Scala does not have H-M type inference. It adopts closures, pattern matching, and a lot of the other constructs of functional programming, but very little of their types. The bit that tends to scare people (like your Prog Lang doctorate friend) is that it has a very advanced *object-oriented* type system. That's something rather new, so I'm not surprised some people are scared.

You write: The language spec is, like 90 percent about the type system. That's true, but it does not mean what you imply. First, the spec is intended for compiler writers, not for programmers wanting to learn the language. It therefore trades readability for precision. If you want to learn Scala, there are some books coming out soon, including one I co-wrote.

Second, Scala's spec is built around types, because types are the mechanism we chose to speak precisely about programs. Since a spec should do that (speak precisely about programs) types are the natural vehicle for that. Other language specs cover the same ground, but in different terms. I give you an example: Other languages talk about methods and their signatures. The Scala spec talks instead about method types. That's just a notational vehicle, because that way we could get more uniformity in the spec. There are no method types that you can write down in a Scala program; they are purely internal to the spec and the compiler.

You write that there are many types in Scala. Again that's true, but I believe it's conceived to be scary only because people are not used to them. It's a superficial reaction, that usually disappears pretty quickly when people get to know the language a bit better.

People usually are not scared of many different forms of statements or expressions in a language. There's no reason to be scared of many different forms of types either, in particular, because most of them are just abbreviations. For instance, you need not be scared of a ``pair type'' such as (Int, String) because that's just an abbreviation for the parameterized type Tuple2[Int, String]. Now, if you'd rather not learn new notation you might prefer the parameterized Tuple2 type over the shorthand, but then your programs would look more bloated and ugly. It's a tradeoff between specialized notation / short programs and minimal notation / long programs. Because Scala takes types seriously, it offers specialized notation to make their notation convenient.

Even if you omit all shorthands, the type system of Scala is still pretty powerful. It's certainly nontrivial to specify and to implement. Dynamic languages are much, much simpler to define and to implement (at least as long as performance is secondary). But is that a reason to shun static languages? In the end, it comes down to the question whether one should refuse a powerful tool because one finds it hard to understand all the inner workings of the tool. Other industries have made the step from craftsman to engineer, where more formalized learning lead to higher precision and productivity. It remains to be seen whether the software industry will do the same transition.

Martin: I totally agree with your corrections, and Steve maybe exaggerated a bit in his presentation (remember that it's actually a transcription of a talk. I know that I for one have a tendency to be more inaccurate while talking, then when writing).

The one part I take offense to is your implicit assumption that dynamic languages equals craftsmanship and Scala equals engineering. I dislike it even more when you polarize the question on a one dimensional value scale.

Yes. Scala is an extremely powerful tool. It has definitely got the potential of being useful in many places . But it's not the end all of computer languages, and it's not a replacement for many of the existing paradigms. Languages without compile time type verification fulfill a very different niche, and I don't see much gain in trying to put them in competition with languages like Scala.

You pose the question if it's valid to shun a powerful tool if you don't understand all the inner workings of it, and yes, in many cases I think you should.

When you need to understand how the exposed type system of Scala works to understand a line of Scala code, and that line utilizes lots of Scala features, it might sometimes be too much. In contrast, dynamic languages generally are very easy to reason about and understand intuitively how they will act, meaning it's easier to be productive both while reading and writing it.

I'm not really arguing about absolutes here, but there are definitely circumstances where Scala should not be used, because of the complexity of the language. The main problem is that a language is never a complete black box. You can't completely separate the inner workings from the outwards behavior, and if you could, then what's the point of having the complicated inner behavior?

Ola, I did not mean to imply that dynamic language programming equals craftsmanship (besides, I have a lot of respect for craftmanship!). What I was aiming at was the all-too-common sentiment that all of programming needs to be dumbed down to be understandable without any intellectual effort. That's not at all the same as programming in a dynamic language. In fact, some aspects of these languages (such as Ruby's meta-programming) look pretty intricate by themselves to be.

You write: You pose the question if it's valid to shun a powerful tool if you don't understand all the inner workings of it, and yes, in many cases I think you should.

I would also refuse such a tool if it meant I could not understand how my program behaves. But for a static type system, I essentially have a theorem prover that tries to find inconsistencies in my program. I need not be able to understand the intricacies of the prover to profit from the error diagnostics it provides.

Let me compare to bridge building: Even if I am not an expert in numerical analysis, I might still use a finite elements package to prove that my bridge will not fall down. I need not understand the intricacies of that package, it's sufficient to trust the people who wrote it.

Likewise, an application programmer can simply trust the type checker to find trouble spots in his or her code, without needing to understand the precise reasoning that led to the diagnostics.

However, as David Pollak has pointed out a couple of messages earlier, things change when you progress from application programmer to library designer. Then you might need to spend more time to get the types right, and you might save time using a dynamic language instead. However, your statically typed library tends to provide more to your clients than the dynamically typed version: Not only the basic functionality but also a set of proven properties (embodied in types) which your clients can use to guide their coding and to find problems in it.

dibblego: Show me a terminating subset of Haskell, and I'll show you a language in which 95% of programmers can get no useful (read: billable) work done. Or better yet, try to explain the difference between structural recursion on data and guarded recursion on codata (and the monads which wrap it all up in a pretty surface syntax) to your average web developer. Then, watch them either ignore you, or die painfully as their head explodes.

Regardless, my sense was that Steve was not attacking Scala so much as he was pointing out how the Scala type system's formality provides the reductio ad absurdum counterproof to the Java's community's long-held disdain for dynamic languages. Arguing for even further adherence to mathematical purity disregards the basic thesis Steve puts forward in almost all of his writing (at least as I read it): namely, that simplicity of mechanism wins over almost everything.

Exposed type systems, aspects, compiler annotations, pragmas, etc., all increase the mental overhead involved in programming. There may be cases where the trade-offs are worthwhile, but every bit of attention taken away from implementing programs is precious, and should be carefully and actively defended.

Martin: OK, now I'm feeling a bit like mr negative here. You might not have done it intentionally, but in your last paragraph you wrote "Other industries have made the step from craftsman to engineer, where more formalized learning lead to higher precision and productivity. It remains to be seen whether the software industry will do the same transition." just after doing a comparison between dynamic languages and more advanced statically typed systems. It came off as the correspondence was intentional, and the part I quote also makes it include a value judgment.

"But for a static type system, I essentially have a theorem prover that tries to find inconsistencies in my program. I need not be able to understand the intricacies of the prover to profit from the error diagnostics it provides." Now, this statement is totally true, as long as you have no failures. But as soon as you have a failure, you need to understand the theorem checker sufficiently to understand why it gives you the failure message and how to correct it. This can be partially rectified with good error messages, but not completely. This brings to mind the lovely examples from H-M style systems where you can get things like "expected signature Int*Int->Int but got Int*Int->Int".

Having a theorem checker that you don't understand why it fails on you means that the benefit is quite small.

The same thing is totally true with your bridge example. Yeah, if you use your finite elements package and it responds negatively, that is just an indication that your bridge probably will fall down at some point. If you don't understand the mathematics behind it you have no idea how to actual fix the problem and strengthen the bridge where needed, or change the angle, or whatever.

Finally, I find that making a distinction between applications and libraries are getting to be more and more of a disservice. If you design an application and don't think like a library designer, you will almost guaranteed write worse code, that someone else will have to maintain. I think that fundamentally there shouldn't be a difference. An application is just a number of libraries glued together, and that means that when writing the components of the application you should approach it as any library development.

Finally, I agree with the sentiment that programming should not be dumbed down. That doesn't mean that you need to complicate the tools. The interactions between seemingly simple components give rise to extreme complexity and possibility for expression - just look at the Lisp family as an extremely simple model.

(And incidentally, the main difference between something with internal complexity, and without it: Ruby's metaprogramming capabilities are extremely simple to understand based on understanding the internals of Ruby (which is quite simple in this area), while understanding the type system of Scala is the inverse: easy on the surface, but gets more complicated if you look at the all the corner cases and the implementation of these.)

I was never not calm. That you think I was is worrying, because it means you have sidestepped what I have said in favour of how you'd like to perceive my current emotional state.

> what the user on the IRC channel asked, and exactly how their question was my fault.

Let's not concern ourselves with this unless you *really* are interested. It was one example of many. To be honest, this shit rant is far less offensive to the scientific community than many of your others. I'd rather address false claims than attribute blame. I'm merely bringing it to your attention that it is happening (as are other things that you seem unaware of - see below).

> which of my claims are "erroneous" and "wildly false".

Well this is the key point; where shall I begin? I'd like to start right at the beginning; try answering a couple of my questions above. You need to realise that you are significantly undermining a huge body of scientific knowledge here mate. I'm more than happy to introduce it to you. I think Chris Smith said it most succintly in his document "What to Know Before Debating Type Systems". Have you read that? I strongly suggest you do and adopt its recommendations.

> I'm assuming you know the difference between an opinion and something that's formally provable. So I ask you: which of my claims is formally provably false? We're all ears.

This would be fun. Before I get started, could you please find a tutorial of your choosing on formal logic, Goedel's Incompleteness Theorem and the Curry-Howard Isomorphism (off the top of my head - just to get started). I am not being sarcastic here Steve; I love teaching and learning and I hope you do too. A good dose on the topic of the type lambda calculi (of which the untyped lambda calculus is one instance) would also be helpful.

> Right now it sounds to everyone reading, and I do mean everyone, ...

No you don't; you mean "everyone in your little circle". There are many counter-examples to your claim because I am talking to them right now, on IRC, in my office, etc. All of us agree on my position except they believe that I am foolish to bother with pursuing the issue as I am where I am being more optimistic. They seem happy to shrug it off as "just another fool making silly claims on his blog" and I am taking the position of "a smart person making a silly mistake".

Steve, I want to make it clear, I am not referring specifically to this blog post (though it is one that has misled at least one learner), but all of your claims collectively - especially about static type systems and specific programming languages (which brings up a misunderstanding that you have - see below).

That people think I am throwing a tantrum is what I hypothesise to be a symptom of the ages in which we live. I reject this culture completely as do many people who I consider to be of intellect higher than my own. They are just words; use their English meaning instead of imposing the idea of "if I were using those words, I'd be throwing a tantrum, therefore, this person must be too". This narrow-mindedness fills me with sadness.

> As it happens, I like Scala, so far at least. But I actually got a note verging on a death threat from a colleague for considering using it at work. This was a smart person at a smart company. In other places I suspect I'd have encountered even harsher resistance.

Yes Steve, I have worked in large corporations too. They are full of stupid people who are afraid of exercising their brain. You're not one of them Steve - clearly - but you're making a grave mistake and perpetuating this form of anti-intellectualism by failing to acquire the required level of understanding to make the claims that you do. Your intentions are good; your execution is incredibly poor.

> There's far more at stake here than you realize. For every person trying to advance the cause of Scala, there are a thousand who will fight it to the death. People won't listen to academic arguments over something they feel threatens their livelihood, and most folks feel that new languages (especially complicated ones) do just that.

No Steve, I am perfectly aware of "what is at stake". I simply have no ambition whatsoever to have the world use Scala or Haskell or whatever you decide is the so-called "NBL".

Here is how I see it. This industry is full of people who have very sophisticated neurological mechanisms that they employ when faced with "something they don't understand". This is because they have a heavy investment in their self-worth so that the proposition "I don't understand much at all" is threatening to their well being. This is when the aforementioned mechanism is deployed in a variety of forms. Psychologists have spent a great deal of effort studying this behaviour. I concede that this will always be the case and have no ambition to change it.

However, the industry also has some inquisitive people who turn to people like you and I in order to attain a higher understanding. I embrace these people. My most favourite job ever was as a university lecturer; not during class but after class when the truly curious students would probe me for knowledge. They loved it and a couple of them today can even teach me a thing or two; how wonderful!

This is my objective Steve. I don't care about the various programming language memes; "oh, I know how to use Scala or Haskell or O'Caml; they are the best!". I couldn't give two hoots about this behaviour. You might, but I do not. I care about knowledge sharing and I am bothering here because your silly rants prevent this. People internalise and repeat what you say like little myth spreaders. They are not capable of thinking for themselves; again, a symptom of the ages in which we live.

> I've been beaten up and down by coworkers at enough companies over trying to use languages like OCaml and Haskell that I'm not about to stick my neck out for Scala. I'm done with being the flag-bearer for academic static languages. If everyone starts using it, sign me up. But Scala enthusiasts have a much longer road ahead than they could possibly imagine, in trying to make this thing successful.

You are doomed so long as you stay at Google. It is an inherent attribute of these large corporations to perpetuate a culture of anti-think. (btw, please don't give me this "yeah, but Google is different" nonsense). Just wtf is a "Scala enthusiast"? Why are some people so ready to assign a memetic membership? Am I a Scala enthusiast? I use the language quite a lot as I do many other languages, including Java even! I wish to be relieved of any subscription to these silly little clans as do many others that I know for that matter. It's simply a misnomer perpetuated by an overhwhelming majority of amateurs.

> My primary advice to you, and I'll assume you deserve it, is this: learn marketing. Learn everything there is to know about it. Then learn sales. A good salesman doesn't freak out. A good salesman is calm, funny, friendly, inviting, and persistent as all hell.

Thank you for the advice Steve, but please be aware that it would violate my intention. Indeed, I spend a great deal of time in my armchair reading about various topics that I expect are most appropriate to marketing, but to employ them in a way that you intend, would not assist me at all.

> You're not off to a good start, but it's not too late to make a change.

I'm off to a great start; I've asked that you stop making false claims and you have responded in such a way to indicate that you are open to the possibility. I'm wondering if you're ready to take the next step?

Secondly, I had a presentation at Google I/O as well that provides a counter point. Javascript may well be great for the server, but ironically, if you want the smallest possible download code, that starts up the quickest in the browser with the least latency (that is, best user experience, as opposed to developer experience), than GWT shows how Java does a much better job than Javascript. See http://www.youtube.com/watch?v=2ScPbu8ga1Q, where I demonstrate the reduction of a jQuery-like API from 100+kb down to just 712bytes of compiled Javascript.

dibblego, you may have noticed that Steve asked you if you, dibblego, have any specific points to refute. But dibblego, you evaded and gave Steve a reading list instead. How are we supposed to take you seriously, dibblego, if you can't answer a direct question like that? Well, dibblego?

Honestly, based purely on the overriding tone of condescension I'm inclined to believe dibblego's a troll. Especially the bit about the terminating subset of Haskell. Har! You had me going for a moment there... dibblego.

Yitz,Steve did ask that and I answered; the answer is yes. I'd like Steve to understand that refutation.

I also asked Steve some questions which are yet to be answered. I hope he will try.

I'm not sure about your comment regarding a terminating subset of Haskell. Do I need to change the example for you? The point is, to provide an answer greater than "one" requires someone to draw on something that does not exist (i.e. _|_). The language is inconsequential. Programming language theorists talk about terminating subsets of turing complete languages all the time.

Ola: Again, I'm sorry to have given the impression that I equated dynamically typed languages with craftsmanship and statically typed languages like Scala with engineering. Believe me, that's not what I wanted to say. The phrases before this comparison said that dynamic languages are simpler to define and implement that statically typed ones. Simplicity is good, and we should go for it whereever feasible. But sometimes we need to invest in a more complex solution, to achieve a goal. For me, a sign of engineering is that one can ponder and evaluate such solutions without dismissing them out of hand as being overly complex.

Dibblego: Thank you for the slightly less incensed, albeit no less condescending response.

You and I live on different planets, that much is clear. I do know far more about your pretty formalisms than you realize; I've made a detailed study of them for many years. They're just lovely. But I consider their current type-system reifications to be little better than Security Theater in the real world.

Your attitude betrays you as the programming equivalent of a neocon. Your basic mode of operation is fear tactics: trying to scare people into thinking they can't live without your straitjackets, without curtailing their "civil liberties" as programmers; for instance, the liberty to run a program containing code the compiler doesn't like or doesn't understand. Real-world systems are large, and messy, and very much alive. They are living things. Rigorous type-checking is almost always an impediment at some point in the evolution of any system. Yet you advocate refusal to compile!

This approach is not "human-scalable": in systems with hundreds of people contributing code, the daily build will be broken perpetually. This is already crippling many C++ and Java-based organizations; your neoconservative approach would kill them outright. This, I believe, is why your camp has never produced any meaningful real-world software. Oh, you have? Show us! Name *any* ubiquitous piece of software written in one of your languages. The glove is off; the gauntlet is down; the ball is in your court.

We all know what your answer will be.

It's certainly not the case that all static typing aficionados share your fundamentalist bigotry, but there are enough of you out there to poison the well pretty thoroughly.

You have tried to give the impression that you represent all academics, and that they represent all the smart people. As it happens, and I say this for the benefit of our readers, there are many academics who loathe "the static typers", as they're called. The practical utility of the "huge body of scientific knowledge" you refer to is a subject of some controversy even in academia. Your lack of successful software outside ICFP contests doesn't help your cause.

Though for what it's worth, you haven't articulated your concern clearly enough for us to identify a cause. I'm inferring from your outrage that you believe strongly statically-typed languages are much better than their freewheeling peers. But you've also complained indirectly that I'm somehow responsible for decreased student interest in type theory classes. If that's your main complaint, well, we have no disagreement: I think they should take the classes.

But I think your objection is more to my general claim that your languages will never be widely adopted. That's the heart of it, n'est-ce pas?

I stand by my assertion.

I live and work in the real world. It's ugly, but I'm happy there. And I will continue to present real-world viewpoints. I'm not educating people; I'm echoing them! It's a genuine opportunity for you to hear what real people think, what they're saying. But you want me to stop talking, as if somehow I'm the source. Once again, this is a pretty clear neocon-style sentiment: silence those with whom you disagree.

The picture you've painted of "industry's" neurological mechanisms that trigger when one's self-worth is threatened is quite accurate, and, ironically, is also an equally accurate portrayal of your own reactions. Funny how that works, isn't it?

I will continue to treat static type systems as an interesting experiment until you folks have done something interesting and compelling with them. I'm not talking about a shorter way to write QuickSort. Lord knows we have enough of those. When you guys write the world's most popular web browser, or the next iTunes, or a great video game, or a web service my friends and I simply cannot live without -- then I'll pay more attention to you.

The problem of many programmers out there is that they did not do enough maths. They lack rigorous thinking. They're the kind of people that say : let's just call the method, i'm sure it's here !What we need is reliability. The real reliability is the mathematical one. Functional programming brings us closer to it.Here's a real world example : the electric flight control of the Airbus A380 series is a C program, but the whole code is checked by ASTREE, an ocaml static analyser, to PROVE the absence of Run Time Errors.You may argue that such a strong reliability is not needed in the "Real World", but I think that the fact that personal computer software is crippled with bugs today is due the industry's inertia.The thing is, Steve : you are the conservative one. As a no-type-system advocate, you are to static typing programmers what alchemists are to scientists.

anocka: very nice fear mongering. Good to bring airplane software in when you want to make it extra scary, even though those systems account for an infinitesimally negligible percentage of all software.

I can't help but look around at all my applications and notice they're not crippled with bugs. In fact, I'll bet you used a computer to enter your comment. And I'll bet even more than NONE of the software between your keyboard and my screen (a long, long path) is written in a strong validating language.

You people are making the whole static-typing camp look bad. You need to keep your trap shut and let Martin do the talking for you; it's the best thing you can do for your cause.

If I get any more of this nonsense from the far-right wing of static typing, I'm closing the comments. I don't want you people hurting yourselves anymore.

I imagine this comment is belated enough that it will be completely missed. And on top of that, it's responding to a (very long) aside and not the main topic.

But still. You weren't very fair to scala, Stevey.

The big one is that you judged scala based on its specification document, which is indeed very long and complex. However, compare to Ruby, which shares w/ scala a huge syntax in the name of useful & readable shorthand. (Insert crickets chirping here.) Ah, that's right. Ruby never had a formal (or informal) specification! How about Python? Ah, hmmm. I'm not as familiar with the Python community, but I believe no spec exists there either. Javascript -does- have a spec, and a reasonably simple one... but does anyone learn/judge javascript based on its spec? No, they write some code in it.

Since I have, unlike you, written code in scala, here's my assessment: - The language is a superset of Java. You can restrict yourself to the java-like portions of it and produce the exact same code, but with many, many, fewer keystrokes. If you do this, the big scary type system stays completely out of your way. - If you wish to go beyond, it has some stupendously useful tools for doing so. E.g.: The (unfortunately-named) "pimp my library" pattern, which gives you the benefits (and then some) of ruby-style open classes over any/all java APIs. Which means you can, with a small amount of work, wrap those hugely verbose enterprisey libraries into a succinct and to-the-point form. - If you go too far beyond "basic java", you will have to wrestle with understanding some somewhat tricky type issues that have a quite steep learning curve. (In particular: covariance & contravariance.) - The existence of strong types enables a fantastically powerful style of programming that I will hand-wavingly call 'type-directed metaprogramming.' The most useful and practical example of this, ironically given your opinion that static typing impedes it, comes into play with unit testing: scalacheck (a haskell quickcheck clone) uses static type information to automatically and reasonably-exhaustively test correctness properties. 1000% awesome and way, way better than handwriting unit tests when it's applicable. - The language is still young, probably too young to be promoting widely. They are still tweaking syntax & semantics from time to time. - There is an unfortunately huge gap between writing 'novice' business logic code (essentially java w/ a nicer syntax & type inference) and writing 'wizard' library code. - The community is being settled mainly by static typing Haskell wizard immigrants, because the language allows that style of programming. I think this is unfortunate as their heavy leaning towards monads, category theory, and the like artifically steepens the learning curve. - Scala's really, really, sweet spot: writing complicated algorithms (particularly functional ones) for the java ecosystem. - Of course, since it's a kitchen sink language, it has many 'sweet spots'; another is wrapping APIs up into a vastly simplified embedded-dsl-ish form.

Enough rambling from me, although to match your output I'd have to go for another several pages.... ;)

Steve,I have explicitly asked to be admonished from your agenda and do not accept any assignment to your artifcial cliques. I am not a "static typer" or a "Scala programmer". I use the less well-typed languages too. I also know the purpose of a type system. It is becoming quite apparent that you do not. I implore you to read the aforementioned Chris Smith's take on the "debate" that you think is raging. It is most accurate.

I am beginning to give up; my colleagues can have their laugh at my expense. You throw straw man arguments like it is perfectly acceptable and reasonable. Keep your head in the sand; I'll take care of your victims.

Excellent talk; it's a shame one has to get assaulted by overzealous, closed-minded types when sharing information on the web, though.

It still amazes me that there's so much religion and so little pragmatism in selecting tools (languages, frameworks, etc.) for software projects. A strict, proven-formally-correct type system might be ideal for a mission-critical flight system. but it's probably too cumbersome for a twitter client. How hard was that?

It's like telling everyone to wear a 50lb. bulletproof vest every day because we should have that kind of protection. But the day you build that protection into my t-shirt, sure I'll wear it every day.

Anyone discounting you is likely doing so because your writing style recommends it, not because they're "anti-intellectual". Ad hominem attacks--among which I include your quite silly claim that Steve, an advanced software engineer with a strong computer science background, doesn't know the Halting Problem--don't really bolster your point, to him or anyone else.

It reminds me of a thread over on Lambda the Ultimate, where a poster was lamenting that he had given a talk and someone had asked him what covariance and contravariance were. This was clearly a guy with an advanced CS degree or two, and I got the distinct feeling that his idea of "properly educated" was that someone should know what he knew, and be able to discuss it at his level. And for CS graduate students, I'm sure he's right; but not for the other 987238974 people who write code.

Unfortunately, that's not really how the world is, nor does it need to be; and it's a lot more helpful to respect what people *do* know and the level they *can* engage on. I think Steve knows quite a lot, even if he lacks your expertise and probably wouldn't always agree with you if he did. So maybe entering the discussion with calmness and courtesy, with a starting premise other than that you are inescapably correct, might generate a more productive conversation.

I'm one of those Java programmers who clings to the static typing even though you rail against it in every post (roughly: "a small codebase is more important than static typing").

Three thoughts that I'd enjoy a response to:

1. A new principle: PAT. "Protect Against Typos." At least 10 times a day I type "nmLevels" instead of "numLevels". I want a language that tells me immediately that I have a typo. I want a wavy red line under the variable name that tells me "TYPO". Eclipse has that. Perl in strict mode at least makes it possible, though I've never seen the red wavy line (although types allow another level of red wavy: type mismatch, which catches more of my typos). What do dynamic languages do about PAT and red wavy lines?

2. Unit tests guard against mistakes at runtime. Programs have a large number of possible runtime paths .. combinatorially many. So, to produce enough unit tests to protect myself against typos (PAT) I potentially have to fight the "curse of dimensionality" (i.e. write a lot of damn tests). However, a compiler has a damn fast way (linear, small-polynomial .. whatever, fast) of protecting me against all possible typos of a certain class. Type checking seems faster and more complete than unit tests.

3. Cross-cutting changes in code. Several times I've had to change a fundamental behavior in a reasonably large system. When it's in a dynamic language (e.g., Python), I grep like a madman and hope that I've stumbled on enough of the runtime paths to be reasonably safe, but there can often be painful misses. In a dynamic language, I can change a signature or make something private, and all of a sudden there's a helpful nag forcing me to at least examine every one of the 500 places that change. How does one gain comfort in cross-cutting changes?

I think Steve's point, that dynamically typed languages can be made to run fast was significantly diluted by his "bash static languages" tangent. Had he stuck to the subject at hand, I don't think people would disagree with his points. I think the entire line of Scala bashing simply was not needed and did not serve to enforce his JS/Rhino on server-side Java VM subject matter.

I also think that there is too much myopia in these static vs dynamic wars. Steve wants to beat back FUD from the static camp that dynamic languages are slow, by saying "look, write a sufficiently advanced JIT and most of the issues go away", but then he adopts a similar myopic attitude that static typing requires batch compilation. The sort of interactive incremental development, the equivalent of evaluating lisp in emacs, he advocates is not outside the realm of typed languages. You can use scala-mode.el today in Emacs and hit ctlr-c/ctrl-b to eval Scala in a running interpreter for example. It's just not available with Java, because no one has bothered to engineer a sufficiently advanced IDE/VM, but the problem is tractable.

Even the issue of "dynamic with optional static typing" and "static typing with optional dynamic" isn't an either-or situation. I could certainly imagine a language, where the type-check policy is library or module specific.

Too often people want to category languages into polar opposite categories, when it's more of a continuum, and it doesn't help to cherry pick anecdotal examples and claim that X is impossible in a static language, or a dynamic language, simply because it hasn't been done, or is hard.

Scala may turn out to be too complex for the Joe Sixpack programmer, but if so, it isn't an indictment of static typing, and claims that "static typing has failed" just seem ludicrous.

"Welcome to BroadVision 1998... we already had an entire application system that could be scripted in JavaScript"

I'm curious. Did anyone get to use this? My employer circa 1999-2000 announced they'd be migrating to it, and I'd just recently learned enough Javascript to be excited about the idea ("Hey! First-class functions! And this prototyping stuff is cool! And objects can be hashes!").

The actual adoption process was slow and pointy-hairedness was becoming endemic, so I left, but I always wondered.

I found Rhino about 18 months ago. Wish I'd found it earlier. I'm excited about its prospects.

Welcome to 1997 :), I was using Netscape Enterprise Server and Server-Side JS ("LiveScript") back then. We were even using it in a 3-tier architecture, with presentation rendered by JS libraries (View/Controller) and making RMI/CORBA calls from JS to Java for the middle tier. This was partially done to work around the memory limits of NES (max-heap of the NES JVM was fixed at something ridiculous like 16mb and unchangeable) We had a Swing-like set of widgets written in JS that would swallow up Swing-like models returned by the RMI calls, e.g. a RMI DAO call would return a TableModel, which would be consumed by a paginated JS table widget.

I do remember being frustrated that LiveScript, IIRC, had a compile step, so you had to recompile JS, and restart the server everytime you made a change. :(

Chris,I admit that I may have had a jerky knee. After all, I am usually one of those people who just sighs and thinks "oh no, Yegge is talking nonsense again". It is the fact that another newbie came looking for help because he had been misguided that provoked my response.

However, I did not use ad hominem fallacies. I've had to point out what an ad hominem fallacy is before and I'll happily do it again. It should be noted that a statement of the form, "you are clueless" is not an ad hominem. However, a statement of the form "you are clueless, therefore, q" (for some q) is ad hominem. However, it is not to be confused with the converse "p, therefore, you are clueless", which is *not* ad hominem. I may have made statements of this latter form. I'll expand if required

Second, you are using a logical fallacy where you state that (paraphrased) "I should expect Steve to have knowledge of the halting problem because he is advanced/experienced/etc.". It may be true that Steve possesses the properties, however, it has no bearing on the truth value of the proposition at hand.

Steve's gross misunderstandings of proof/correctness verification systems have a direct implication that convinces me that Steve has no idea about the fundamental implications of the halting problem. Not that I wish to pick on Steve at this point; I merely wanted to give him a term that is related to his misunderstanding so he can explore it further if desired.

Of course, it may be the case that Steve has a thorough understanding of the topic at hand and exists in a state of "double-think" i.e. holding two contradicting positions at once, however, although double-think is often a good default candidate on many issues, I have assigned it to "improbable" it out in this case (though, I think it is highly probable that there is double-think in some other context here on this blog).

In any case, I want to put it to rest. Steve has his head comfortably in the sand and I have spent more effort here than I have dealing with Steve's victims. There is nothing more to be gained.

Interesting talk; I hadn't considered the interoperability benefits of the JVM before.

It'd be nice to see your compiler/interpreter work open-sourced; for one thing, more eyes and brains on it might bring it closer to Java speeds, which would promote adoption, since lots of people reject possibly-superior languages due to relative slowness even when it doesn't actually matter in the code they end up writing.

p.s. dibblego:"It is the fact that another newbie came looking for help because he had been misguided that provoked my response."

I haven't seen the 'misguided' position held by this newbie, in any of your comments (= vitriolic diatribes).So it's not really fair to say with no qualification at all that "some newbie was misguided and specifically because of you".

Post the misconception he came to you with, as well as a reasonable proof that it is Steve's fault (for apparently promoting incorrect ideas/assumptions).Otherwise, you'll just continue to sound like a rude, arrogant twat.

Static typing advocates may be the neocons of the programming world ... but I could swear I just saw you "swift-boat" Scala, no?

Programming language pundit numero uno, Steve Yegge, has a big readership and he knows it. And if JavaScript is going to be the NBL, then something had to be done about this young upstart, Scala.

But unlike John Kerry in the 2004 US presidential election who failed to respond to the Swift-Boat ads, Martin Odersky was on the case, responding in measured tones in a dignified and scholarly manner, striking just the right contrast with the semi-serious rant-style of pundit numero uno.

One can almost hear a slight quaver in the normally cocksure voice of a slightly chastenend super-pundit:

"Martin: thank you for the corrections and overview. I am in the process of learning Scala and look forward to using it more."

What Scala really needs right now is name identification and buzz. All publicity is good publicity, at this point -- even a little swift-boating helps the Scala cause.

Although you say that it will take 10 years of work to get language interoperability, I'd just like to point out that the i OS (formerly the AS/400, or System i) from IBM has what is called an Integrated Language Environment (ILE) that provides a framework within which you can make all kind of cross-language calls, share data, etc. Here is a link to the public documentation.

Wonder if Flow-based Programming will help with the language communication issue - Justin Bozonier pointed me at your talk (which he must have attended). I have posted a conversation about this with him at http://www.jpaulmorrison.com/cgi-bin/wiki.pl?JustinTalk . I'd be interested in your reaction to FBP.

This is a nice talk. Steve does highlight some of the best features provided by Rhino.He also tries to convince his audience that Javascript is a great language (having himself being convinced). It's a pity that he fails to mention another opensource (built on top of rhino)called seppia. (http://www.seppia.org)

In my opinion the topics should not always revert around javascript on the serverside vs.javascript on a web-browser but it should just be emphasized how javascript can well combine its strengths with java.For example, Seppia uses their relationship to provide an environment to build up applications. But unfortunately not many people know about it...

Steve- I think I've read everything you've written, both here and on the old rants site. I'm a CS major interested in languages and systems programming, and I find that nothing inspires my interest in the field more than your posts.

I imagine that the negativity and rudeness from commenters can be very discouraging, so I'd just like to humbly thank you for your work. I eagerly await your next post.

Maybe this has already been addressed, but I've been feeling the need to ask.

You've made reference to developing Rhino on Rails as the framework for a a huge web project. Was that project Google Wave? If not, what the heck kind of project are you doing that it needs a bigger, more powerful framework than Wave does?

The problem of many programmers out there is that they did not do enough maths. They lack rigorous thinking. They're the kind of people that say : let's just call the method, i'm sure it's here !What we need is reliability. The real reliability is the mathematical one. Functional programming brings us closer to it.Here's a real world example : the electric flight control of the Airbus A380 series is a C program, but the whole code is checked by ASTREE, an ocaml static analyser, to PROVE the absence of Run Time Errors.You may argue that such a strong reliability is not needed in the "Real World", but I think that the fact that personal computer software is crippled with bugs today is due the industry's inertia.The thing is, Steve : you are the conservative one. As a no-type-system advocate, you are to static typing programmers what alchemists are to scientists.seslisohbet

You and Tony Morris(are you the same person? You both have the same condending and boring approach to beating down heathens) are a big reason why FP hasn't taken off. The community flat out sucks. It is full of arrogant, self-important, psuedo-intellectuals.

Funny how you can't name a single FP program that is successful. Even two years later, there is nothing noteworthy from the Haskell and Scala camps.

Actually, it is not funny it is sad. You guess are so busy circle jerking to your self-importance to actually do anything of substance.

Yeah. Java as a language sucks, not as bad as PHP, but still quite sucky. But at least there are more than a few significant and successful projects written in it. Scala and Haskell programmers can't say that.