Sunday, July 11, 2010

Converting Smalltalk to Newspeak

One of the things that has surprised me working with Newspeak is how easy it is to convert Smaltalk code to Newspeak. We have converted most of the core libraries of Smalltalk (aka ‘The Blue Book Libraries’) into Newspeak. Most of the conversion was done by people who had never coded in Newspeak before. In some cases, they had never coded in Smalltalk before.

The syntactic differences are fairly trivial (though they will grow in time) and easily handled - either automatically by tools or by very simple transformations in a text editor. Semantically moving from Smalltalk to Newspeak is easy (see this document for a detailed discussion of the mechanics), but the opposite is harder.

The main issues one has are API differences between libraries; but this is no different than converting from one Smalltalk implementation to another.

The converted code may not be the most idiomatic Newspeak. The easiest conversion path sticks a bunch of related Smalltalk classes inside a Newspeak module class, without creating any interesting substructure. Nevertheless, the converted code is invariably better than the original. It becomes very clear what the code’s external dependencies are, what the module boundaries should be, who is responsible for initialization etc. There is no longer any static state. It’s much easier to tie libraries together (or tear them apart), test them independently and so forth. Once we start enforcing access control, it should be much clearer what the public API really is. All this without losing the flexibility and power of the original.

This is very encouraging, because it means Newspeakers can take advantage of the large body of Smalltalk code that already exists. Any useful Smalltalk library that is publicly available can be converted to Newspeak in pretty short order. It also means that Smalltalkers can take the plunge and migrate to Newspeak relatively easily.

Of course, I don’t expect all the world’s Smalltalkers to instantly convert to Newspeak. Even if they did, Newspeak would still be a niche language. For Newspeak to be a success, we need to reach out to programmers in a variety of communities. This post, however, is aimed squarely at the Smalltalk community.

There are no doubt many Smalltalk projects that are tied to a specific Smalltalk. There are also Smalltalkers who are too conservative, and don’t want to deal with any language changes.

Still, if you are (or were, in some happier time) a Smalltalker and want to move into the future rather than dwelling on the glorious past, I assert that Newspeak is for you. If you are using an open source Smalltalk, it is likely you could do better using Newspeak. Newspeak explicitly addresses Smalltalk’s weaknesses: modularity, security, interoperability. Of course, some people aren’t bothered by these weaknesses.

Tangent: Arguably, the Smalltalk community is, by natural selection, composed largely of such people; if they were bothered by Smalltalk’s deficiencies, they wouldn’t use it.

Newspeak should interest those who appreciate the power of Smalltalk but want to move forward.

Of course, you have to be an early adopter by nature. Things will evolve and change under your feet. The syntax will become less Smalltalk-ish over time. Most importantly, you’ll have to learn something new. This is good for your neurons. You may have to port some libraries you rely on. You may have to make changes so that you do not rely on libraries you wouldn’t want to port (like Morphic). However, the result will likely be a product that is easier to deploy, more visually appealing and better integrated with its surrounding environment. Your code will be much more maintainable and better structured.

15 comments:

As long as you don't remove named parameters, which always has been a strong selling point of Smalltalk and Objective-C for me because it makes code so much more readable, you can change as much as you want :)

Newspeak looks really incredible. I like that you both know what you are doing from a technical standpoint and from a business standpoint. Not many people in the industry can do both. Some people make really cool stuff that would never work outside of education/research. With Newspeak, you seem to really nail a lot of the big issues to make it usable for producing commercial and/or mission critical applications (native GUI, easy deployment, no brainer security through class based sandboxing, etc.).

The only downside of Newspeak I can see at the moment is that it's not beginner friendly at all, especially writing GUI apps. It's not something I'd recommend to someone as a first language. But I'm sure that will improve over time. That there is already a really big and helpful PDF (105 pages) that comes with Newspeak tells me that the documentation will probably be really good as well (and we all know how important good documentation is, even if we also always know that we will never spend as much time on documentation as we should ^^).

Thanks for the kind words! It is really great to get that kind of feedback! It's hard to make the right thing happen, but we're getting there. We need more volunteers to move things forward. So please consider becoming a contributor.

@HaiColon: Oh, and not to worry: keyword messages stay, because they really work so well. However, I expect to use standard braces, get rid of ^ etc., even though it all works for me. We need to reduce barriers to entry.

Replacing it by what? `return` in so many letters? Adding that as a keyword?

in fact, this touches to one of the issues I have with Smalltalk: methods returning self by default, which I see as pointless considering Smalltalk has message cascading (did self keep it? I hope so), Couldn't self's methods just return the last expression by default (as in most functional languages, or in Ruby or Smalltalk) and keep ^ for other kinds of returns?

@Gilad:I do plan on contributing to Newspeak, the only question is if I will be of any help :) I've watched all the talks you gave on Newspeak that are available online and am trying to read through all the papers at the moment while trying out Newspeak. I've also spent the last month catching up on a handful of Smalltalks and took a look at Self which I had never heard about before.

One thing I am planning to do is an interactive Newspeak tutorial that starts as a web application (like TryRuby, the Factor Playground, Clamato, etc.) to catch people's interest in Newspeak and then continues more in-depth in Newspeak itself after they downloaded it, giving you some tasks and helping you along the way instead of just making you copy code verbatim from the tutorial.

This will hopefully help spread the word about Newspeak, lower the barrier to get started and teach me enough to help out with more advanced stuff.

With some regret, I'd replace ^ with return:. Conceptually, it is a send to the activation. Is it more verbose? Of course. BUt for every person who understands and appreciates ^, there are a hundred (or a thousand even) who will be able t read code that uses 'return:'. Sad but true.

Masklinn: return: would work exactly like ^ does today. I'm a big fan of NLRs. I don't see a need for return (without a :).

As for returning self by default, I don't wish to tamper with it. While returning the last expression makes sense from a logical viewpoint and is more uniform, I can see that in a lot of methods it will return something useless, or worse, leak implementation detail.

Also, returning self helps sends compose. Overall, the Smalltalk expression language design has stood the test of time very well. It is very finely tuned and won't mess with it.

Not to mention how disruptive such a semantic change (as oppose dto the syntactic ones I'm contemplating) would be.

just found the news about NEWSPEAK in Planet Squeak, looks interesting for me as an old Smalltalker - just 2 questions / remarks for now: you wrote "You may have to make changes so that you do not rely on libraries you wouldn’t want to port (like Morphic)" - that sounds like a lot of work. if I understand you right. Two years ago I moved from Visual Works to Squeak for some good reasons. The biggest effort was to change from MVC to Morphs. You want me to change that again, if I want to use NEWSPEAK?

Second point is: I didn't see anything like audio and video - please comment.

I will have a closer look to find out potential benefit for my work - you may have a look: http://myNeat.net

For video/audio, not sure what you mean. At first I thought you were looking for presentations. In that case, seehttp://newspeaklanguage.org/newspeak-videos/

and in general checkout newspeaklanguage.org.

However, perhaps you meant libraries for video/audio. We don't have any right now. We do have a real FFI, and the preferred tack would be to call some external library rather than re-invent the wheel. All this works best on Windows at the moment.

Nice wrappers for such libs would be interesting. One can of course port Squeak code (or use it as it stands, with some loss of modularity).

Regarding the UI: I absolutely expect people to get off of morphic if they are interested in production quality software. I realize this is an issue for some people. However, if you want UIs that look professional and can (eventually) be deployed unchanged to a variety of native platforms, you'll have to do that anyway.

If you want to learn more, contact me diretcly, or join the forum (links at newspeaklanguage.org)