Posted
by
kdawson
on Tuesday August 18, 2009 @01:00PM
from the means-scaleable dept.

inkslinger77 notes a Computerworld interview with Martin Odersky on the Scala language, which is getting a lot of attention from its use on high-profile sites such as Twitter and LinkedIn. The strongly typed language is intended to be a usable melding of functional and object-oriented programming techniques. "My co-workers and I spend a lot of time writing code so we wanted to have something that was a joy to program in. That was a very definite goal. We wanted to remove as many of the incantations of traditional high-protocol languages as possible and give Scala great expressiveness so that developers can model things in the ways they want to. ... You can express Scala programs in several ways. You can make them look very much like Java programs which is nice for programmers who start out coming from Java. ... But you can also express Scala programs in a purely functional way and those programs can end up looking quite different from typical Java programs. Often they are much more concise. ... Twitter has been able to sustain phenomenal growth, and it seems with more stability than what they had before the switch, so I think that's a good testament to Scala. ... [W]e are looking at new ways to program multicore processors and other parallel systems. We already have a head start here because Scala has a popular actor system which gives you a high-level way to express concurrency. ... The interesting thing is that actors in Scala are not a language feature, they have been done purely as a Scala library. So they are a good witness to Scala's flexibility..."

To be fair, FailWhale has not been seen by me of late, and the Twitter devs attribute their salvation to Scala. However, there are some pretty good arguments that the Twitter developers Mother of All Fail was initially trying to write their own hand-rolled message queue [unlimitednovelty.com] instead of simply using one off-the-shelf.

Recently I decided to move from contracting to full time work as the job market is balls here in the Bay currently for Contracts. Twitter was one of the companies which I applied and I had the pleasure of having a "phone screen" with them for a senior unix position. Here's what this screen was, a basic unix question, that any lunix user could get. A more intermediate type question that could trick some people. And finally their *BIG SCREEN* a tricky question that was based on esoteric knowledge that had absolutely nothing to do with one's ability to perform the job.

The person calling me was just reading these off a list, she didn't know why they were picked and was only able to write down the answers. Here's the hilarious part, I informed her that the question was silly and there's no reason anyone should really care about this sort of information except in extreme situations. That this was the question that lead me to believe they had a culture of primadonnas. She diligently wrote all this down, in case they still wanted to talk to me.

But here's the REAL kicker, their stupid asinine esoteric question? Was wrong. They had the phrasing wrong... what they were asking and looking for in an answer were not the same things. Being a pedantic asshole, in my followup to tell them what I thought of their process I pointed this out. Never heard anything back;) Wonder if they have fixed their question yet?

I was completely pleasant with the woman, we joked about the questions in fact. The simple fact was they had a stupid call screen process straight out the egotistical dot.com days, which showed much about the types of "engineers" they like to bring in.

You said yourself that you were a... let me scroll it up here... a "pedantic asshole" who told the interviewer, quite presumptuously, that a question was "silly." Furthermore, completely on the basis of the phrasing of one question, you had concluded that the entire company had a "prima donna" culture.

they must have some of the old Amazon.com folks there. Amazon was like that. Same type of questions, same type of mistakes in what and how they asked the question except they did it in person. I had the same reaction as you did, told them they were wrong in the specification which led to their answer being an incorrect solution. They got pissed and ended the interview. I caught an early flight back:) Plus people were on call 24x7 and worked hellacious hours for low pay and lots of worthless stock options.

She diligently wrote all this down, in case they still wanted to talk to me.

Point of case; they never DID want to talk to you again, did they?Why employ somebody that considers difficult questions "esoteric knowledge that had absolutely nothing to do with one's ability to perform the job".Perhaps that little bit of information is exactly the kind of thing you need to know if you're working on Twitter's systems.Or more simply put; "There are no stupid questions, only stupid people".

Scala is great, but one really annoying thing about it is that it inherits type erasure [safalra.com] implementation of generics from Java. This means that you cannot overload methods on argument with the same generic class with different type parameters, cannot implement the same generic interface with different type parameters on the same class, cannot check whether a class implements a particular generic interface for a given type parameter, etc. They did fix some issues [lamp.epfl.ch] - for example, you can instantiate arrays - but it's still far from perfect.

I understand the need to match Java's broken model for the sake of interoperability, but surely a better way can be devised for pure Scala code? It's pretty much the only area where Scala noticeably lags behind advanced.NET-hosted languages (such as Nemerle or F#).

Ah, my comment just down the page didn't take into account that you want to use the full Java standard libraries. However, it still seems to me that you're projecting deficiences of the Java language onto the Java VM to some extent. You write:

Since interface is actually just one, on JVM level we're stuck with a single compareTo(Object)

It's more complicated than that. If Foo implements Comparable then Foo.class will contain methods

The third of your listed limitations is semi-bogus: you can't check generic type information with instanceof, but java.lang.Class.getGenericInterfaces gives you the information you need if you're prepared to write a method to process it.

I can't help feeling that the first two can be worked around too if you're prepared to be more creative with your Miranda methods than javac.

The limitations are all bogus; the point is that it should be Scala compiler that's doing all the jumping through hoops, and without telling me, too. I should just be able to write code in a straightforward way, same as I can on.NET (where it's the VM itself that's doing all the dirty work).

Am I the only one not terribly enamored with Scala? It's a massive language (have you seen the book?), but a lot of the syntax is somewhat redundant and doesn't seem to add that much. The type system is downright byzantine, and the Java interface is, let's say, somewhat inelegant. And the whole object-functional thing seems like a paradigm in search of an audience (maybe I'm just not getting it).

And you get to pay for all this with a huge performance hit.

I guess their "more is more" approach is mostly making Clojure look more attractive.

Am I the only one not terribly enamored with Scala? It's a massive language (have you seen the book?), but a lot of the syntax is somewhat redundant and doesn't seem to add that much. The type system is downright byzantine, and the Java interface is, let's say, somewhat inelegant.

I've read the book. Yes, it's fairly complicated, but then all languages tend to become that as they move from academia or design-by-committee stage into real world - witness Java. In terms of power, however, it's one of the most advanced languages today which can be used in production... and yet it's statically typed, which is a plus in my book.

And the whole object-functional thing seems like a paradigm in search of an audience (maybe I'm just not getting it).

You probably aren't. It's been a big thing in.NET land ever since C# 3.0 appeared, and grows even bigger now with F#. It's a pragmatic approach - it gives you both OO and FP tools, and lets you mix and match freely to get the optimal balance for the task at hand.

And you get to pay for all this with a huge performance hit.

I guess their "more is more" approach is mostly making Clojure look more attractive.

Funny how you speak about performance hit, and then immediately mention Clojure...

Anyway, where did you see the "huge" perf hit there? Examples? It's still JVM bytecode, remember, and it's statically typed, so in the end it's mostly normal Java method invocations all around. It has to fall back on reflection for some things that JVM simply doesn't support otherwise, but those are corner cases, not normal operation.

I have not only seen the book but also read it (assuming you mean Programming in Scala [artima.com]). It is very well written and gets you started with Scala easily.

I do not agree that the syntax is redundant. To the contrary: an important part of the language design is that Scala enables the programmer to extend the language using libraries. A good example is the way collections and iteration work in Scala vs. the way the foreach loop has been added in Java 5.

Whether static type systems help or hinder depends on whether you work with or agaist it, and on the kind of programs you write. The Scala type system is rather advanced (in the sense of complete) such that it allows you to express more in the type system than other languages do.

For Java style programming Scala is just much simpler and shorter to write. When you have written some code in both you can appreciate the way Scala works.

I just finished reading Programming In Scala. Yes, it's a big language. There is a lot going on for sure. Extractors, Case Classes, Pattern Matching, Implicit Functions, just to name a few. Reading about scala can be a bit overwhelming. I had to really get my hands dirty with a small project before I really was able to understand and appreciate all the features and how they work together. The fun thing about Scala though is that one can start out programming in a Java style and slowly incorporate its

I could use an alternative programming language for the JVM that is more expressive than Java. Both Scala and Groovy integrate well with Java at the language level, albeit with different type systems. While I do use Groovy from time to time, what kept me from Scala was that it is not well supported by IDEs (Eclipse/Netbeans - I hear things are somewhat better with IntelliJ). The problem is the nature of Java libraries. They tend to be deeply nested and often expressed in lower level abstractions and are difficult to use without strong IDE support. I don't need an IDE for Python (flat module systems, high level libraries), but certainly do for Java. With solid IDE support however, I am nearly as productive in Eclipse + Java as with dynamic languages, even for medium apps. Scala and Groovy come with their own standard libraries and I don't need IDE support as long as I stay within them. But sooner or later, I will need to step into plain Java land and I no longer feel productive. I would rather use straight Java for them.

The development experience is language + tools, not just the language. While Scala can piggy back on JVM and undercut the rest of the process that languages need to go through to mature and be accepted, Scala plugin (or someone else) has to provide a JDT equivalent first to have popular appeal.

I just wish there was a well supported superset of Java with productivity considerations that maintains 1:1 byte-code compatibility on compilation. Java purists can keep their language clean. But the rest of us can be happy too. Scala has that potential.... almost. So far, Groovy has been filling that role for me. Groovy will never have the kind of edit-time IDE intelligence simply due to it's dynamic nature. But for now, it stands ahead (after all, it has been around longer).

I like Groovy, it's java-like and scratches some of my biggest itches about java.

However I like Clojure better. It's hard for me to resist the power of a lisp, with a full macro system, *and* full java interop. To me it's the only JVM language that stands out, Scala and Groovy are nice, but I don't see why they are any better or different than JRuby or Jython.

Groovy is different because it's easy for Java programmers to learn. In fact, most Java devs can understand Groovy code with little or no explanation. That's certainly not the case for JRuby or Jython.

In any event, I agree Clojure is pretty sweet. However, being a Lisp, it's future is questionable. A lot of devs won't be willing to deal with the brain melting process necessary to grock it.

Groovy is different because it's easy for Java programmers to learn. In fact, most Java devs can understand Groovy code with little or no explanation. That's certainly not the case for JRuby or Jython.
In any event, I agree Clojure is pretty sweet. However, being a Lisp, it's future is questionable. A lot of devs won't be willing to deal with the brain melting process necessary to grock it.

Yeah I agree with you there. I do use groovy as my "java and then some" language. It looks an awful lot like ruby to me, but yes it's more java-like.

Its unfortunate about lisp(s) and their popularily, because honestly I don't see what's so difficult about them. Macros are hard, but lisps don't force you to use them. Other languages don't even give you an option, you can't. Paren matching is done by any modern editor. Prefix notation is a bit unintuitive I guess, but that slowed me down for maybe a couple of weeks, about the same as new syntax for almost any language. I am starting to think that "a lot of devs" just don't want to understand it. Or maybe a lot of devs just don't get programming in general, they just learn their one language, and can maybe pick up a few similar ones.

Scala is better and different from Groovy and JRuby and Jython in that it is statically typed (but does a fantastic job at hiding all the type nonsense as long as you are doing something sensible). Those of us who make type errors as one of their most common errors really benefit from this. It is also better and different in that it is quite a lot faster than the interpreted languages. (It is worse and different in that a few fancy tricks are impossible, such as method-missing stuff, or generic operation

I should have mentioned that I did try Clojure. While it is certainly the epitome of dynamic typing and syntactic flexibility for the JVM, I rejected it for the same reasons (no code completion and other validations). Most of the software I choose to write in Java is so because of the specific libraries. Most of code ends up manipulating these libraries, not as my custom logic. If I had a lot of custom logic, I could see value in Clojure (or in Scala as I mentioned in my previous post) where I could use it

I've found NetBeans + Java + Scala to not detract at all from NetBeans + Java. So to me, it's a win to add Scala to the mix. It would also be a win to add Groovy or Clojure or Jython or whatever; I just find adding Scala to be the biggest win because the interoperability with Java is easiest (at least in the Java->Scala direction) and because performance is by far the best.

According to common usage, Python and Scheme are both "strongly typed" as well, since they guarantee that all type errors in programs are detected. This is in contrast to "weakly typed" languages like Perl or K&R C, in which many type errors are silently ignored. That is, all four combinations of strong/weak and static/dynamic typing are possible.

Some people are using the term "strong typing" as a synonym for "static typing"; I wouldn't really care, except that there is no good other term to describe what "strong typing" means.

From a practical point of view, it seems pretty clear that "strong typing" (in the first sense) is important, but I have seen little evidence that static typing is all that useful in a general purpose programming language.

... but I have seen little evidence that static typing is all that useful in a general purpose programming language.

That is a matter of the way how the mind of a particular developer works.I have troubel with dynamic typing, and I hate it to get me my errors shown by runtime when a compiler could have detected that error. Some people don't have this trouble, so they don't need static type checking. Other people develop even the simples script in a test driven development approach and don't need static type

But if all you're doing is reinventing Perl with C-like syntax, it's not really a step forward.

Perl already has C-like syntax. It doesn't, however, have static type checking, let alone robust static type checking (with nice features like variance annotation etc.); nor does it have particularly robust functional programming support -- you can certainly do functional programming in Perl, but it isn't as clean and syntactically sugared as it could be; nor does perl have particularly clean OO if we're being honest -- yes it works and can be made to work quite well, but elegant isn't a word that comes to

IIRC it was originally intended to be able to use the JVM or Microsoft's CLR; they just finished the JVM version first and that's what's being promoted.

It's certainly a terse language. That's both good and bad.

Terseness is largely irrelevant. The right question is not "does this language let me express something in the minimal number of keypresses", but rather "does this language force me to repeat myself endlessly".

Why not use the C instead then, or even better C++. After all many top websites run C++ (Google,eBay,Yahoo) as it is the fastest, well memory utilizing, best threading performance and green/CPU saving solution. With the native processor exceptions used properly it is also the most robust solution.

... Twitter has been able to sustain phenomenal growth, and it seems with more stability than what they had before the switch, so I think that's a good testament to Scala...Hmmmm, maybe I should switch my company's programming language to Scalia. After all, it cause market share to increase, computer systems to become more stable, and maybe it will make my hair grow back. I'm in!

No, this is Scala, a language that is a blend of functional and object oriented programming. Scalia [wikipedia.org] is mix of textualism and originalism with a very conservative framework. Some consider its inability to recuse itself to be its greatest asset.

No, this is Scala, a language that is a blend of functional and object oriented programming. Scalia [wikipedia.org] is mix of textualism and originalism with a very conservative framework. Some consider its inability to recuse itself to be its greatest asset.

Explicit type conversions disallow a value of type T to be treated as a value of type S without invoking a function that takes a value of type T and returns a corresponding value of type S. For example, a conversion from an integer type to a floating point type requires the invocation of a function that performs the conversion. Contrast this with implicit type conversions where a value can be treated as almost any type depending on how it is to be used.

Type restrictions only allow certain operations to be done to certain types. For example, numerical addition mïay only be performed on numerical types. A lack of type restrictions allow for numerical addition to apply for, say, booleans, for example.

One of the biggest problems with "strongly typed" that I've seen is that sometimes it becomes synonymous with with "statically typed" and they just aren't the same thing. Some people seem to think that dynamically typed languages are automatically "weakly typed" which is just flat out wrong.

I actually really like dynamic/strong languages (like, for example, Python). It allows you the security and readability of a strongly typed language, without all the whiny syntax of a statically typed language.

Some languages don't care much about variable types and do conversions on the fly. So you could write code to doIntNum2 = (FloatNum1 + IntNum1) and nobody would complain. No errors. Nothing. It would compile and run just fine. At least, assuming you didn'

Funny that you picked this particular example, because even strongly typed languages more often than not allow implicit floatint conversions. Take C/C++ for example, which would be a prime example of a strongly typed language but still it allows implicit int->float conversions, and many compilers don't mind going from int->float conversions either, even though it's a narrowing type conversion.

A better example would be something like string s = '0nice' and int i = 1. A typical weakly typed language all

It basically means that variable types are rigidly enforced and conversions are explicit.

Actually, no, it means that value types are strongly enforced. A language may not have typed variables at all, and still be strongly typed - Python and Ruby are both examples of that. Static/dynamic typing, which is what you describe, is orthogonal to weak/strong typing.

Perl is weakly typed on a lot of things, but it's static for data structures and subroutines.

So you can have a hash of hashes, or an array of arrays, but you can't get arrays in your hashes or vice versa.

Yep. I mainly referred to Perl's and PHP's ability to treat strings as numbers and vice versa, depending on the context - Python wouldn't let you get away with that, for example. In that sense VB6 was also weakly typed, even though it had strongly typed objects.

Assuming you're not being merely rhetorical (because the definition is kind of loose), strongly typed just means that the language makes some restrictions on how operations operating on different value types can be intermixed.
Assuming that you're not a programmer, I'll give an example. Letâ(TM)s say you have a BMW Z4 roadster. It's a car. Understanding the nature of cars, you know that it can be classified as a vehicle, a sports car, a BMW sports car, and a BMW Z4 roadster. Strongly type languages ma

Most languages are strongly typed so that's not a particularly useful metric.

Hardly. The One True Language for client-side web development, aka Javascript, is weakly typed, as is PHP, that ubiquitous server-side programming language. Perl is weakly typed. Good ol' shell script is weakly typed. Heck, even C is considered weakly typed (unless you enable -Werror, of course).

I don't mean to sound pedantic, or borish, but C is actually "yeah baby yeah" typed, to enable pointer arithmetic, stack space exhaustion, buffer overflows, and system level development. It's not incorrect to say that it is weakly typed, per se. It's just awkward having to try to explain the direct parallels between the C type system and a 70s style love-in (where anything goes) -- to your manager.

I don't mean to sound pedantic, or borish, but C is actually "yeah baby yeah" typed, to enable pointer arithmetic, stack space exhaustion, buffer overflows, and system level development. It's not incorrect to say that it is weakly typed, per se. It's just awkward having to try to explain the direct parallels between the C type system and a 70s style love-in (where anything goes) -- to your manager.

I hear where you're coming from, brother/.er.

I'm rather amazed at what work has been done in the Linux kernel regarding moving the code towards the static typing model with innovations in C and GCC extensions. Much of the Linux kernel *can* be statically type checked.

I am *so* hoping that Linus writes the Mythical Penguin Month as Fred Brooks did for O/S 360 the 3nd most important O/S, but most important at the time. For all his sometimes abrasive writing style, he has done something I didn't think could

And when the going got tough, as I recall, Twitter fell over, as did most of the other darlings of the current social networking hypewave.

It's kind of amusing that people have started to think of simple web front ends to simple CRUD applications as if they're demanding. Indexing the entire WWW is demanding. Running the financial systems behind a bank is demanding. Modelling weather systems is demanding. The CAD models for a 60' yacht are demanding.

Learn some other language that isn't C#, Java, C++, VB.NET, or in fact particularly related.

Specifically, learn something with functional programming, or a different sort of object orientation (Common Lisp comes to mind, or Smalltalk or Objective-C), or declarative programming (like Prolog)...anything that's considerably different from any of the standard compiled OO languages.

Learn and improve, or don't learn and become unable to earn a living in th

Yes, but understand that, when you're learning that language, you're doing it to expand your mind and development toolkit, *not* to add said language to the list you put on your resume, as the very fact it's niche means no one will care.

And by "development toolkit", I mean things like design patterns and approaches to software construction. Take Haskell, for example. Compared to a typical imperative language, it requires a *very* different mindset when constructing applications. But the lessons you learn

Not much for professionalism, are you? I can say with about 99% certainty that the fact that you know Haskell will have absolutely no effect on your career. The fact that you learned it will have a large effect on how well you can turn problems into programs.

The only stable asset you have is you. If you don't improve yourself, you'll get left behind, and in ten years you'll be working on legacy C# systems and not getting paid as much as the people working on the interesting stuff in whatever language

No question that anyone who wants to make a living programming probably needs to learn a business safe language like the ones you mention.

Still, some of us really enjoy programming, even when we aren't making a buck off it. Working with a new language can make you think about programming in a new way, and can teach you lessons that you can bring back to your day to day business work.

Those supposedly cute features are there for a reason--they make abnormally clean and maintainable code possible.

The best Scala code I've seen is clearer (to an outside observer) than the best Java code could possibly be, since the language features allow you to focus more on what is going on rather than necessary theatrics.

2) Any developer who says he 'remembers the types of our variables' is either lying, or inexperienced. You may remember it today, a week from today, or perhaps even a month. You will not in a year. Nobody who looks at your code will 'instantly know' unless you name your variables 'thisIsAnUnsigne

As a programmer with 25 years of experience, I could hardly disagree more. When developing any large project, static typing is hugely beneficial. It lets you catch far, far more errors at compile time. In a dynamic language, those would all be runtime errors that could easily slip through your testing and make it out to a customer. It makes automated refactoring possible. (Ever try doing a major restructuring of a 500k line code base written in a dynamic language? Good luck!) It enables all sorts of static analyses that let you be a more productive programmer. Try using a really good Java IDE like Intellij IDEA, then try the best Python or Ruby editor available, and you'll feel like you've gone back to the stone age. No "Find Usages"? No autocompletion? No "Go to Definition"? You simply can't do these things in a dynamic language, because the editor has no idea what the type of any variable is.

This isn't meant to criticize dynamic languages. They definitely have their place. But 90% of the symbols in most programs should be statically typed. I don't see why more languages don't simply offer both options. In those that do (e.g. Groovy), type declarations are simply optional. If you declare a type, it gets checked at compile time. If you don't, that symbol becomes dynamically typed. It works very well.

That "why i hate" is a terrible editorial. What he is saying is that he doesn't want to think carefully about what he's doing. His objection isn't to languages really it is to clarity of thought. Tinkering and right brain is great for art, it isn't the right mode for engineering.

So who do you think will be writing the "components" for your magic system, and the infrastructure to make the system executable, designable, testable and usable? We have been progressing to increasingly high level code (and in some cases, as you say, component architectures), but the amount of code being written and maintained continues to grow, not shrink.

So the amount of code is growing, and it does more per unit mass, and software engineering skills are increasingly valuable and applicable, and you think this is a trend towards the elimination of code and software engineers? You may want to check your water and food sources for contamination.

I'm not too worried. The idea behind COBOL was that once you got rid of that mathy way of writing code anyone could do it. The fact is that taking a process and breaking it into a series of steps and dealing with all the cases takes practice. The problem is not syntax or language structure the problem is thinking through a problem that way.

Until computers have common sense all languages are going have to look like