Tuesday, 29 November 2011

Real life Scala feedback from Yammer

Following up on my recent Scalaposts, some commented that because I haven't used Scala seriously, my opinion is of little value. I responded by noting that writing the FCM closures proposal for Java, altering the javac compiler, and talking at conferences about language design might qualify me to have an opinion.

Having said all that, I do believe that the most valuable feedback comes from those that have tried Scala, and found it wanting.

Today there was a good example of this, the most important kind of feedback. Yammer, the Enterprise Social Network, announced provided an indication that they are moving away from Scala, and back to Java. And whats more, they provided a very detailed rationale. The original gist disappeared, so for certainty I've copied the current version inline here (my bold highlights):

Update: Twitter user coda says "@jodastephen I'm saying you're misrepresenting it. It was a personal email which did not represent the views of my employer. In the same sense your Twitter account isn't an official publication of OpenGamma, neither is my personal email.". The main point appears to be that this isn't official Yammer policy. Personally, I think the contents are more interesting than whether its an official announcement or not, but YMMV... For the record, here is the original Hacker News thread.

Update 2011-11-30: Coda has explained how this document (originally a private email) came to be public. For the record, I got the story off Twitter, then Hacker News, and did not receive anything privately, but wanted to ensure the quality of this feedback reached a wider audience (especially given the original gist was deleted). The text of the explanation is also in a comment below.

Update 2011-12-01: A formal response has now arrived from Yammer. If you read the text below, you should also read the response linked above to get the fuller picture. Very little in tech is black and white, and, despite what some readers may think, I understand that very well.

Originally:
https://gist.github.com/7565976a89d5da1511ce

Hi Donald (and Martin),

Thanks for pinging me; it's nice to know Typesafe is keeping tabs on this, and I appreciate the tone. This is a Yegge-long response, but given that you and Martin are the two people best-situated to do anything about this, I'd rather err on the side of giving you too much to think about. I realize I'm being very critical of something in which you've invested a great deal (both financially and professionally) and I want to be explicit about my intentions: I think the world could benefit from a better Scala, and I'd like to see that work out even if it doesn't change what we're doing here.

Right now at Yammer we're moving our basic infrastructure stack over to Java, and keeping Scala support around in the form of façades and legacy libraries. It's not a hurried process and we're just starting out on it, but it's been a long time coming. The essence of it is that the friction and complexity that comes with using Scala instead of Java isn't offset by enough productivity benefit or reduction of maintenance burden for it to make sense as our default language. We'll still have Scala in production, probably in perpetuity, but going forward our main development target will be Java.

So.

Scala, as a language, has some profoundly interesting ideas in it. That's one of the things which attracted me to it in the first place. But it's also a very complex language. The number of concepts I had to explain to new members of our team for even the simplest usage of a collection was surprising: implicit parameters, builder typeclasses, "operator overloading", return type inference, etc. etc. Then the particulars: what's a Traversable vs. a TraversableOnce? GenTraversable? Iterable? IterableLike? Should they be choosing the most general type for parameters, and if so what was that? What was a =:= and where could they get one from?

A lot of this has been waved away as something only library authors really need to know about, but when an library's API bubbles all of this up to the top (and since most of these features resolve specifics at the call site, they do), engineers need to have an accurate mental model of how these libraries work or they shift into cargo-culting snippets of code as magic talismans of functionality.

In addition to the concepts and specific implementations that Scala introduces, there is also a cultural layer of what it means to write idiomatic Scala. The most vocal — and thus most visible — members of the Scala community at large seem to tend either towards the comic buffoonery of attempting to compile their Haskell using scalac or towards vigorously and enthusiastically reinventing the wheel as a way of exercising concepts they'd been struggling with or curious about. As my team navigated these waters, they would occasionally ask things like: "So this one guy says the only way to do this is with a bijective map on a semi-algebra, whatever the hell that is, and this other guy says to use a library which doesn't have docs and didn't exist until last week and that he wrote. The first guy and the second guy seem to hate each other. What's the Scala way of sending an HTTP request to a server?" We had some patchwork code where idioms which had been heartily recommended and then hotly criticized on Stack Overflow threads were tried out, but at some point a best practice emerged: ignore the community entirely.

Not being able to rely on a strong community presence meant we had to fend for ourselves in figuring out what "good" Scala was. In hindsight, I definitely underestimated both the difficulty and importance of learning (and teaching) Scala. Because it's effectively impossible to hire people with prior Scala experience (of the hundreds of people we've interviewed perhaps three had Scala experience, of those three we hired one), this matters much more than it might otherwise. If we take even the strongest of JVM engineers and rush them into writing Scala, we increase our maintenance burden with their funky code; if we invest heavily in teaching new hires Scala they won't be writing production code for a while, increasing our time-to-market. Contrast this with the default for the JVM ecosystem: if new hires write Java, they're productive as soon as we can get them a keyboard.

Even once our team members got up to speed on Scala, the development story was never as easy as I'd thought it would be. Because one never writes pure Scala in an industrial setting, we found ourselves having to superimpose four different levels of mental model — the Scala we wrote, the Java we didn't write, the bytecode it all compiles into, and the actual problem we were writing code to solve. It wasn't until I wrote some pure Java that I realized how much extra burden that had been, and I've heard similar comments from other team members. Even with services that only used Scala libraries, the choice was never between Java and Scala; it was between Java and Scala-and-Java.

Adding to the unease in development were issues with the build toolchain. We started with SBT 0.7, which offered a pleasant interface to some rather dubious internals, but by the time SBT 0.10 came out, we'd had endless issues trying to debug or extend SBT. We looked at using 0.10, but we found it to have the exact same problems managing dependencies (read: Ivy), two new, different flavors of inpenetrable, undocumented, symbol-heavy API, and an implementation which can only be described as an idioglossia. The fact that SBT plugin authors had to discover what "best practices" are in order to avoid making two plugins accidentally incompatible should have been a red flag for any tool which includes typesafety as a selling point. (The fact that I tried to write a plugin to replace SBT's usage of Ivy with Maven's Aether library should have been a red flag for me.)

We ended up moving to Maven, which isn't pretty but works. We jettisoned all of the SBT plugins I wrote to duplicate Maven functionality, our IDE integration worked properly, and the rest of our release toolchain (CI, deployment, etc.) no longer needed custom shims to work. But using Maven really highlighted the second-class status assigned to it in the Scala ecosystem. In addition to the "enterprisey" cat-calls and disbelief from the community, we found out that pointing out scalac's incremental compilation bugs had gotten that feature removed outright. Even the deprecation warning for -make: suggests using SBT or an IDE. This emphasis on SBT being the one true way has meant the marginalization of Maven and Ant -- the two main build tools in the Java ecosystem.

Cross-building is also crazy-making. I don't have any good solutions for backwards compatibility, but each major Scala release being incompatible with the previous one biases Scala developers towards newer libraries and promotes wheel-reinventing in the general ecosystem. Most Scala releases contain improvements in day-to-day programming (including compilation speed), but an application developer has to wait until all their dependencies are upgraded before they themselves can upgrade. If they can't wait, they have to take on the maintenance burden of that library indefinitely. In order to reduce their maintenance overhead, they naturally look for another, roughly equivalent library with a more responsive author. Even if the older library is better-tested, better-documented, and better-featured it will still lose out over time as developers jump ship for something that works with Scala 2.next sooner. (It's also worth noting that most companies using Scala at scale or in mission-critical capacities will not immediately upgrade; the library authors they employ will likely be similarly conservative, and the benefit their experience brings to their code will benefit the community less and less over time. As far as I've found, we're the only big startup in SF using 2.9.)

Once in production, Scala's runtime characteristics were the least subtle problem. At one point, half the team was working on a distributed database, and given the write fanout for our large networks some parts of the code could be called 10-20M times per write. Via profiling and examining the bytecode we managed to get a 100x improvement by adopting some simple rules:

1. Don't ever use a for-loop. Creating a new object for the loop closure, passing it to the iterable, etc., ends up being a forest of invokevirtual calls, even for the simple case of iterating over an array. Writing the same code as a while-loop or tail recursive call brings it back to simple field access and gotos. While I'm sure Scala will be have better optimizations in the future, we had to mutilate a fair portion of our code in order to actually ship it. (In another service, we got away with just using the ScalaCL compiler plugin and copying things to and from arrays instead of using immutable collections.)

2. Don't ever use scala.collection.mutable. Replacing a scala.collection.mutable.HashMap with a java.util.HashMap in a wrapper produced an order-of-magnitude performance benefit for one of these loops. Again, this led to some heinous code as any of its methods which took a Builder or CanBuildFrom would immediately land us with a mutable.HashMap. (We ended up using explicit external iterators and a while-loop, too.)

3. Don't ever use scala.collection.immutable. Replacing a scala.collection.immutable.HashMap with a java.util.concurrent.ConcurrentHashMap in a wrapper also produced a large performance benefit for a strictly read-only workload. Replacing a small Set with an array for lookups was another big win, performance-wise.

4. Always use private[this]. Doing so avoids turning simple field access into an invokevirtual on generated getters and setters. Generally HotSpot would end up inlining these, but inside our inner serialization loop this made a huge difference.

5. Avoid closures. Ditching Specs2 for my little JUnit wrapper meant that the main test class for one of our projects (~600-700 lines) no longer took three minutes to compile or produced 6MB of .class files. It did this by not capturing everything as closures. At some point, we stopped seeing lambdas as free and started seeing them as syntactic sugar on top of anonymous classes and thus acquired the same distaste for them as we did anonymous classes.

Now, every language has its performance issues, and the best a standard library can hope to do is to hit 80% of use cases. But what we found were pervasive issues — we could replace all of our own usages of s.c.i.HashMap, but it's a class which is extensively used throughout the standard library. It being slower than j.u.HashMap means groupBy is slower, as is a lot of other collections functionality I like.

At some point, I wondered if the positive aspects of our development experience owed less to Scala and more to the set of libraries we use, so I spent a few days and roughly ported a medium-sized service to pure Java. I broached this issue with the team, demo'd the two codebases, and was actually surprised by the rather immediate consensus on switching. There's definitely aspects of Scala we'll miss, but it's not enough to keep us around.

Already I've moved our base web service stack to Java, with Scala support as a separate module. New services are already being written on it, and given the results from our Hack Day at the beginning of this week it hasn't slowed our ability to quickly ship complex code. I'm keeping a close eye on the effects of this change, but I'm optimistic, and the team seems excited. We'll see.

So.

I've tried hard here not to offer you advice. Some of these problems could easily be specific to our team and our workload; some of them won't make a difference in how your company does; some of them aren't even your problems to solve, really. But they're still the problems we've encountered over the past two years, and they compose the bulk of what's motivating this change.

Despite the fact that we're moving away from Scala, I still think it's one of the most interesting, innovative, and exciting languages I've used, and I hope this giant wall of opinion helps you in some way to see it succeed. If there's anything here I can clarify for you, please let me know.

Its not for me to add any more to this, Yammer's opinion. There is a gold mine of information about Scala in there. And in my opinion, everyone thinking of adopting Scala should read it - in detail.

Update 2011-12-01: Just a quick reminder to read the responses from Coda/Yammer: immediate and formal.

79 comments:

By the responses from the devs at Yammer this is complete and utter bullshit. I'll make sure to no longer read anything from you as you're obviously on a track to simply tear down a language you, for some reason, are unable to learn or use. Good luck with that and feed well troll.

I find the criticism is warranted, and the response to it I hope is not indicative of the attitude of the Scala community at large.

If there is one criticism the Scala community needs to hear (again and again, apparently), it's the backwards binary incompatibility. Dependency management is hard as it is in an enterprise environment, and I hate difficult upgrades more than anything else in my job.

The above reason alone has lead me away from Scala to favor Clojure, Erlang, and Elixir instead (and even Haskell when the benefits of running on a VM are not applicable to the project).

I'm sure the reply to this statement will receive a similar response to the one above.

Yammer's performance and production needs are no doubt unique so I'm not sure how relevant this is to all. But some of the points regarding loop and collection performance are things I'd definitely like to see some further discussion of. My hunch though is that these performance concerns would exist for any non-Java language running on the JVM. The point about Maven certainly rang true. Typesafe, IMO, shouldn't be picking winners and losers with 3rd party tools. They should let the marketplace of engineers/companies do that.

@Josh Stone: what's left to elaborate on? The code and build process was a rat's nest of pain and people should avoid using scala at all costs on any commercial project where these things can become a big issue.

I've used and learned lots of different languages but can't recall anything more painful than scala. Cross compiling and maintaining c codebases with native ui on different Linux/winblows envs was easier and less traumatizing than this. It's fine for academic pursuits and personal projects but you're smoking some serious crack to try and justify it commercially.

Are you suggesting that the people at Twitter, Novell and Intel are on crack? The original email seems an authentic description of one leading developers experience. It would certainly seem to suggest that back in November 2011, Scala wouldn't be the best choice for every project. What is positively shocking is how few of these return to Java stories there are. it seems the overwhelming majority of companies that switch to Scala in a major way are very happy with their choice.

Stephen Colebourne is a Fantom fan. And that's a perfectly legitimate preference. Fantom virtually eliminates generics. Again a perfectly legitimate preference. However I strongly suspect that he wants to go in the opposite direction to which the large majority of those who are considering moving to Scala wish to go. If you consider Java to be an overly powerful language then no Scala is probably not the right choice for you.

@Jesse - "where these things can become an issue" is exactly the point. Where can they become an issue? Scala IS used commercially, successfully. Assuming that issues effecting one deployment will exist everywhere or that Scala is unsuitable for commercial development everywhere is an overly simplistic conclusion.

Such conclusions aside, I am interested to learn more about these issues, particularly how they might effect _ANY_ non-Java JVM language.

"I responded by noting that writing the FCM closures proposal for Java, altering the javac compiler, and talking at conferences about language design might qualify me to have an opinion."

That more than qualifies you to *have* an opinion. However, if your FCM "proposal" had actually been a specification, or if your javac modifications had actually implemented your proposal, you might have learned something that would make your opinion more valuable. As for the talking at conferences, you have it backwards: having an opinion qualifies you to talk at conferences, not the other way around.

@Josh Stone My response was simplistic because it's true. I can't say I've worked on -all- types of commercial projects but I feel confident enough in my experience that anything that starts to form a constant time-suck of essentially "nothing" other than maintaining code is a pretty big fuckin problem. One that needs to be removed if you are being a responsible leader/developer.

Still, it ends with "I still think it's one of the most interesting, innovative, and exciting languages I've used". The comments are pretty valid, but they're meant for improvement and not as an excuse...

@Neal You are my language hero. I quickly shut out the annoyed thoughts stirred up by mention of FCM and conference talks (I have done that too, but don't view it as a high point of anything ever done with code) and couldn't help but remember you p0wning everyone regarding that. Ah well, good times. Good times..

@josh what do you mean by not relevant to any specific scala project? It was extremely relevant on a day by day basis for a good 1.5 years of my life living with it. What exactly qualifies your experience with scala in all of this? What kind and how large and how long of a scala codebase have you been involved in?

@Anonymous Convenient of you to elect to remain nameless while offering nothing constructive in your contradiction. No doubt Yammer's performance requirements are out of the ordinary but as a Clojure user I feel grateful for not having to negotiate a type system of baroque complexity.

There are a lot of great points in that email, although I don't think any of it should be a surprise to someone choosing a language to build their product on.

Yes, Scala is a big, complex language. However over the course of the past year I've monitored the progress of 4 (frankly mediocre) developers picking up Scala 2.8 from scratch, and their Scala code is no worse than the other languages they use on a daily basis: Python, Ruby, PHP.

Half of the mail discusses performance issues that really shouldn't surprise anyone. To get the maximum performance out of a computer you must write C-like code, minimize allocations, and avoid cache misses. Clearly allocating closures in a tight loop, invoking accessors and mutators, using HashMaps instead of arrays and so-on doesn't meet that criteria. Even Java is terrible for implementing some types of code! Compare the performance of cryptography using a pure Java library, vs OpenSSL's C and ASM implementations. Turns out ASM is by far the fastest, yet completely unportable and a real pain to write.

Higher levels of abstraction nearly always come with a performance cost, but 90% of the time they're worth it for the productivity increase.

We really enjoy building our apps in Scala. It's a concise and expressive language, with complex features that are mostly optional and very handy when needed.

About a week ago, I mentioned in passing to an old co-worker on Twitter that Yammer was moving some of our stack from Scala to Java. A few days later, Donald Fischer (the CEO of Typesafe) emailed me at my personal account, asking for more details. He CC'ed Martin Odersky, the lead designer of Scala and Typesafe’s Chief Architect. Given that the two people best-situated to improve Scala had just asked me about my experience over the past two years of using Scala, I wrote a long, considered, brutally honest response.

In the process of composing that email, I asked a few personal friends to review a draft of it. One of those friends shared that draft with someone else, who shared it with someone else, etc. etc. and today I woke up to find my email to Donald and Martin splashed across Hacker News and Twitter. I deleted the gist, but the cat was out of the bag, and now I find myself having to publicly explain the context of a private conversation.

I wrote that email for a very specific reason: Donald asked me for my opinion. If someone asks me for an honest opinion of them or their work, in private, I feel morally compelled to be as honest as I can with them. The only way we can ever know how we appear to other people is through other people; the only way we can know what others think of what we build is to ask and listen to them. The fundamental asymmetry of consciousness means triangulation is the only path to understanding. Anyone running a business understands this, at some level: what customers say does not by itself constitute objective reality, but does offer critical insight into it that one is essentially incapable of discovering by oneself.

But it’s simplistic and naive to assume that I wrote what I did in an unguarded moment and that somehow this represents a more truthful account of what I’d say in public. The most that you can possibly know about this is the text of my email to Donald and Martin, not the context.

Yes, that email is not what I would say in public. The Scala community needs another giant blog post about ways in which someone doesn’t like Scala like I need a hole in my head, and I’d rather suck a dog’s nose dry than lend a hand to the nerd slapfights on Hacker News. The world has yet to take me aside and ask me for my opinion of it, and in the past few years I’ve found that it’s far more profitable to build things rather than tilt at windmills.

Quite frankly, and awful lot of the Scala sucks discussions that I have read here and elsewhere seem to be written by Java developers, who had high expectations for Scala, and feel let down. It's too bad that this happened to them but part of the blame surely has to fall on the high expectations.

Personally, my main tool has been Python, the CPython variant, and I have no interest in becoming a Java developer. That said, I am familiar with basic Java, wrote some servlets back in '98, and currently find myself maintaining some Java code from time to time. But Scala brings enough to the party, that I find it well worth my while to learn it, and to use it as my main tool for building stuff to run on the JVM. I suppose I could have just used Jython, but it is not as compelling as Scala. In particular, the whole actor model, especially as implemented by Akka, is something that attracts me to Scala. I've actually been writing some Python recently that has an actor-like architecture using AMQP messaging, and given the choice of learning Scala or Erlang, I think that I will get a lot more benefit from Scala.

We have one RESTful web app that we are currently writing in Scala and it is working out OK. We don't do anything outrageously complex in it. In fact the farthest we get from the world of conventional languages is using the Dispatch library to make HTTP calls as a client. This particular library takes advantage of Scala's lack of operators, and ability to use operator-like method names to write some interesting things that would be.a.chain.of.dot.invocations.in mostother OO languages. 98% of shops do not need to do extreme scaling. It seems Yammer is in the top 2% where it matters, and they will do what they need to do. As the original writer pointed out, this may eventually be solved by better optimization in the compiler. Some of us remember Java before the JIT. The naysayers all pointed to the low performance inherent in the design and then, lo, magic happened, and the Hotspot VM and others, showed that there are workable solutions to performance problems that seem inherently unsolvable at first glance.

I won't begrude Coda his opinion, and I fully recognize that this is one man's position after using Scala in a particular context. That said, I could have written this email myself just by listing off my concerns about Scala as a language, without ever having been a serious user. Coda's issues with Scala, enumerated in this email, mirror my own thoughts on the language to a tee...it was like reading all the negative impressions I'd gotten as a cursory Scala user expanded and correlated to real-world experiences.

For the record, I think Scala is an amazing language. It's incredibly powerful and flexible...moreso than any other mainstream language today. Unfortunately, that's the problem.

Did Scala run over your wife and sleep with your dog? Seriously, I can't understand the degree of personal animus on display here. The post and the responses alike speak volumes about the character of the communities they're coming from -- and little of it is good.

@Michael: Everyone needs scaling, every company is better of with paying for 1 server instead of 10 servers for the same number of users (when the language has approx. the same productivity, Scala/Java)

I think it would be better if people could be self critical about the specific issues raised, rather than ad hominen attacks on the author. Stephen may have a bias against Scala (which he has himself acknowledged) but whether or not his opinions co-incide with yours, you can at least look into the details of the thread.

I have written up a more detailed analysis at InfoQ, in which I investigate the claims of the sub-par performance of the Scala collections, and found that in trivial micro-benchmarks the complaints are valid.

http://www.infoq.com/news/2011/11/yammer-scala

Of course, Coda's mail was intended to highlight areas for improvement for the Scala team to focus on, and it's important to realise that these improvements, if made, will help everyone who is interested in Scala.

However, the big problem remains Scala's lack of backward compatibility with prior (or future compatibility with future) releases. This isn't apparent when you start working on your first project with Scala, and is entirely orthogonal to the language or its syntax (or the power of its type system). The problem remains that this causes brittle dependencies and has resulted in tools like SBT which can cross-compile against different releases.

Most of the cost of software development is not in the initial writing, nor of the number of characters it takes to create a file. IDEs help reduce the character typing problem in any case. The real cost comes in maintenance of a system, and until the backwards compatibility problems are solved, Scala will continue to cost more from a maintenance perspective than a similar system - even with double the number of lines of code - of an equivalent Java system.

That doesn't mean Java is better (or worse) than Scala at the language, typing or syntax level. It just means there is work to do to make Scala a sensible enterprise choice.

I don't get why people need to add personal insults against Stephen, or try to divine his ulterior motives instead of just realizing, hey he doesn't like this language and he wants to express that opinion.

He got a lot of criticism, many on his comment section that were not technical arguments or even semi respectful, so he found an email that kind of outlines some of the issues he has (or brings up some new ones). Why do you have to make up this "he's threatened" by Scala argument to diminish the honesty of his opinion?

Take at face value, and if he's saying things that are incorrect, point them out. But I don't really see a need for some of the insults and the overreaction! Some of the comments I've seen in response to this are too emotional and make no sense in what should be a technical discussion.

Thanks for this post, I believe there's real value in such an informed opinion. Is theresomewhere any other analysis such as this one that tries to highlight Sala benefits with such detailed analysis? It would be beneficial to try and compare results.

On a side note, it's a shame technologists can't reason here about technology without looking like fervent, religious zealots with PMS. If you try to argue on right and wrong, please read before posting http://www.nizkor.org/features/fallacies/

Geez, Talking about Scala with this nice analysis I dont need to write an opinion in any blog or article about Scala anymore, Here is the answer to all the Scala freaks that said Scala is the next thing. Here is the prove that they are wrong, What else can said about?. Scala is for a niche and that's all.

@Jbaruch I shouldn't continue, but I'm an idiot and I will (shame on me).

When you are a product advocate (whatever it may be, IT related or not), I expect you to show me why your product is good. You may compare with others, but your focus should be your product.

Throwing crap on another product (even if your arguments where 100% true, which is not the case in this discussion), it's not good for your case nor your image as product or advocate. Talking without hands-on experience is the worst example you can set in technical scenarios. And doing all this not once, nor twice, but 3 times? When the last argument is using a leaked letter of 1 failure case as "told you so"? Let's ask Foursquare?

You are right, it may not be fear of change. The options which are left for this attitude aren't too good either.

In the end this is not about him bashing Scala. Hey, that happens, it's fine, it's good if the bashing is reasoned and comes from people who knows the stuff. The problem is how he has done it. Honestly, if you are somebody with certain reputation, you should know better than to do it like this.

To me, as a mediocre engineer, being able to spend less time worrying on using something like Joda or Guava because my language doesn't need them, means I have less dependencies to 3rd party programs, less configuration and more productivity. If that's so horrible, even at the cost of complex types, hey, I'm a crappy developer, what can I do.

In the end no laguange is perfect, people complains about all of them, but only a few are really used in production. And that's the metric that should matter the most.

M. Colebourne: I would be curious to see Joda Time written in Scala, it would be much shorter and incredibly easier to use. Being shorter it would be easier to maintain ?

The argument about binary compatibility is somewhat valid: except it is when you inhirit traits (which is rare), no problem when using them.

For the collections, does he have anything faster then parallel collections ?

Many of those issue seems to be that in that particular case it might have been better not to use Scala. But it mainly look like they did not do their homework before starting the project. All the Scala issue mentioned aren't new.

While Scala need to improve, I think their management needs it much more !

Java supposed to be the complex, verbose and old language and we need a new fresh air of language that will relieve us of it and here enters Scala but wait a moment instead of relieve us it is making more complex and difficult the situation. So for what is Scala?. Scala, It has the speed of C++? Nah, Easier than Java? Nah, Dynamic as Ruby? Nah, So Scala where match in all this?, Me NO CLUE. Im not sure what the Scala people are advocating.

Functional Paradigm?, Right now almost all the programming languages on earth use in one way on another functional concepts so that is not the point, What is Scala point on all this?, Scala advocates please speak it up?!.

Groovy 2.0 will feature static type checking, which may help overcoming its performance problems. Groovy always had the right attitude: "Embrace existing knowledge, but streamline the experience". Compare that to what is written about real world experiences with Scala. The mainstream developer resistance towards Scala is just too high at this point. I foresee a mixture of Java8 + Groovy2 will pave the way for the future of JVM usage.

While every language has things that need improving, certainly including Scala, it seems to me to be an incredible leap to then say it's unsuited to enterprise development.

Let's just see what we have so far.

- Many companies using Scala and liking it very much. - Many many positive reports and blogs of people learning and adopting Scala. - Scala's growth looking pretty phenomenal right now. - A few blog posts saying how much they don't like Scala, usually backed up by little or no practical examples, and often by a person who hasn't actually done any development with it. - One company's private feedback to Scala's Typesafe company, mainly about some performance concerns using Scala (used in what appears to be pretty extreme circumstances), and problems integrating with their existing toolchain. - Blog posters and commenters then automatically assuming that all their criticism is therefore valid.

While this is a very natural and human reaction, I think it grossly mischaracterises the situation. What we have here is one company moving back to Java. Fair enough, that seems to make the most sense for them.

Does that then mean that Scala is not useful? That it is not being used by plenty of other companies quite happily? Of course not.

There are some things to be aware of, and it's not like the Scala community is unaware of those problems. That is true for any language. Look at Effective Java to see these things documented in detail - we just know all that stuff and have got used to it.

Scala is extremely useful, and when you are talking about readability of code, it provides huge advantages on Java. I admit that this may not be easy if you only see all the very functional and type-wizzy looking code out there, for sure. You don't have to write code like that. I don't write code like that. I write code to be consumed, not for blog posts. See my comments on the previous posts for examples.

Scala has several features that C++, Java, and Ruby lack. Sure, some functional extensions are slowly being added (like closures). But scala is way ahead of them. First, it's got a very nice persistent, immutable collections. I believe Phil Bagwell put in much of the effort on these collections. I also believe that Clojure persistent collections also derive many of their collections from Phil's work at EPFL. Correct me if I'm wrong. Second, it's got pattern matching. It's much, much, much more capable than Java and C# switch statement. Third, it has things like partial function application and also currying. Fourth, it has lazy vals and streams. Fifth, it has built-in support for parser combinators and DSLs.

Is it the fastest? No. But it's performance is comparable enough to Java -- Am I wrong, or are there not quite a large number of applications that are written in java, and therefore don't need to be of the ultimate _fastest_ performance?

Easier than Java? If by easy, you mean easy to learn? No. If by easy, you mean easier to code solutions once you've learned the langauge? Yes. There are many, many problems that are more easily solved with Scala than with Java. (For a fun example, see http://www.scala-blogs.org/2007/12/project-euler-fun-in-scala.html -- try implementing those in Java and call it "easier".)

Dynamic as Ruby? Not in terms of dynamic typing. In fact, it strives not to be. I find that a good thing. With scala's type inference, the code can be more concise than ruby AND you still get the benefit of strong typing and great IDE support. It's sure a heck of a lot faster than Ruby too. Scala is smoking fast even compared to JRuby. So for those that like the concise Ruby syntax but not Ruby's performance, Scala is a great alternative. See the great ruby vs scala twitter debate.

SteveJ, Hehe yeah all day I have been flaming some of the forums and blogs about Scala because is Hot right now and really I want to know more what is all this noise that Scala is doing right now but really Im not a hater, I have been doing Scala little bit here and there, not big deal, I like some stuff, even I recommend it to a friend and Lift framework I like it, Play framework 2 with Scala looks ok but still verbose not for my like it.

Im agree the collections and pattern matching is awesome, Actually I love pattern matching in Haskell and Scala yes bring that. About Fastest I would like to be more fast than Java so I can kick Java out. Type inference nice feature. The problem of scala is the type system is complex.

But anyway lets resume, Scala brings this on your opinion:

-Pettern Matching (I can live without it, Its cool, awesome but not required)+Collections(This are important and save time, are cool, can be a good point)+type inference(Java only generics, Another point)-Easier(I want to code right away, Not learning complex concepts, The email of the author it said is even complex knowing it so they went back to java)-Performance(Still not match Java performance because the abstractions)

So this let us with Collections and Type Inference.

Im not sure if I need to learn a new programing language for this. Change mentality, way of coding and many things. And thats the 99% of Java developers. So my point is Scala is doing noise but do you think all this is just now and Scala will continue to be for a niche as Haskell and Erlang are? or it could get more mind share and become a key player on the JVM as it is Java right now?.

@Pere In your opinion, how well do I, or Stephen -- or anyone for that matter --need to know Scala to be allowed to criticize it? I am asking this, because I have the impression that, on the one hand, there is a consensus in the Scala community that extensive learning is needed in order to use Scala correctly. On the other hand Coda's letter clearly stated (whether true or not I don't really know) that within the Scala community there is *not* a consensus on best practice approaches to certain problems.

To put it in a more provocative way: If I drive a car and it breaks down, am I allowed to come the conclusion that it is crap without knowing how the engine works? OK. I admit that I would not ask this question, if the car were wrapped around a tree... ;)

In general, the JVM is only going to run as fast as it can run byte code. I guess the challenge here is to see which compiler can make the most efficient byte code. Java and Scala are very neck-and-neck in performance, I believe. Scala does not have primitives in the language, but I believe the compiler converts all the primitive-equivalent objects (Int, Float, Double, etc) to primitives where possible. I don't believe Java does this. In java, if you use primitive, you get a primitive. If you use the object equivalent, you have object creation overhead and deal with autoboxing, etc.

As far as tools go, the scala IDE for eclipse 2.0 RC is very nice. Your project can mix and match java and scala source files very well. I've been watching it, and the tool is continually improving, and it's moving at a rapid pace. Features include built-in REPL within eclipse, error and syntax checking as you type, auto completion for variables, methods, members, and the the other usual eclipse functions like references, and jumping to definitions, finding callers, etc. Others swear by IDEA IntelliJ IDE and it's scala support.

Another benefit you left off was conciseness. Every line of code you write has the potential for a bug. Scala programs tend to be much more concise than the java equivalent.

"To put it in a more provocative way: If I drive a car and it breaks down, am I allowed to come the conclusion that it is crap without knowing how the engine works? OK. I admit that I would not ask this question, if the car were wrapped around a tree... ;)"

I'm sorry, but this is quite a terrible analogy. It would be a good analogy if the scala runtime broke or crashed regularly, and then you came to the conclusion it was crap. I think a more appropriate analogy to what Pere is saying that you are saying the car drives like crap without ever having taken it for a drive.

@SteveJ Yeah, the analogy is more provocative than precise, I admit. :) But the question remains: What amount of Scala knowledge and experience do you require for criticm that you'd (and possibly Pere) accept as valid? Is Coda's criticism valid?

I think after having worked on a project, say porting an open source library like "joda time" from java to scala , is enough knowledge and experience before someone can offer criticism.

Ok, I kid, I kid.... Honestly, I think it's fair to criticize at any point. I even agree that some of Stephen's criticisms are valid. I think what grates on many is that he has such disdain for the language, and is talking lots of smack, spreading FUD, and is generally so biased against something, after perhaps too superficial of an inspection.

I'm starting to believe that the one of biggest Scala issues is the overzealous members of our community who are not able to respect and reflect different opinions.

I fully understand Coda's opinion since it reflects a lot of my latest thoughts on Scala. I can live with the performance issues for now, however the build toolchain is huge pain. Plus "I definitely underestimated both the difficulty and importance of learning (and teaching) Scala." - been there, done that.

Nevertheless, my new projects will probably still use Scala. At least to some degree. From the business point of view, it's really tough to find Scala-able developers. Hell, it's difficult to find decent Java developers.

Scala is a priestly language (just like C++) and that is the fundamental problem with Scala.

It is also a tour de force in language design, without question, and well deserving of praise for its merit. But as an economical proposition for getting business done -- what a "Java replacement" need to do -- it is clearly, and obviously, a dead-end.

@neal: you came across a bit arrogant there, Neal. We further disagree on what is "valuable" in this context: actual programmers who must cut code for a living in contrast to elite programmers who are paid to develop ideas.

Sad to see so many personal insults, just because someone does not share the authors opinion. Unfortunately that's the kind of tone that's been emanating from Scala advocates for a few years now and frankly it makes me want to have even less to do with the language. Hmm perhaps I should revise my christmas wish-list and ask for an F# book instead of a Scala book.

There's a bunch of interesting and useful performance hints in this article. Also, sbt, I believe, went in a wrong direction; we use maven, and it's okay, although not that kosher.

But the main point... gosh, it is San Francisco! There are tons of young software developers eager to learn and apply new stuff. Could yammer find and hire them, instead of the usual herds of boring bored mediocre Java coders?

See what I mean. In our company, we have a bunch of interns from Waterloo; they never heard of Scala before coming here; they did not have this issue of "Scala is complicated" because for them it was all the same, Java, Scala... they picked it up, I guess, in a week, and now they are writing pretty decent production code, in Scala/Lift. Well, there's a lot of space for improvement, but basically they do not have problems. They use it, and it's fine with them.

I was recently looking around for a new language to learn after many years of Python. In general I'd like to expand my horizons but I admit I'm not willing to type (or have my ide type) 3x-4x as much code to get equivalent results (my attempts at engaging with Java consistently fail). Type safety, for it's own sake, isn't yet high on my list of things to explore.

Scala seemed initially appealing (I didn't realize how important the type bits were to the vibe). The more source code I looked at, the less appealing it was. Some looked so much like Java that I couldn't understand why I would bother learning it. Some looked nice, concise and intriguing. Some looked really truly arcane and dreadful.

I came to the general conclusion given the broad divergences that I saw in how people used Scala that, while that complexity and possibility would be very attractive to some people, it would be a very unpleasant environment to learn and share code in (for me).

I settled on Go. Once I found the very clear and easy to think about concurrency primitives in it, I knew this would be the best experience for me. It's also most complementary to the systems work I do. The Go team seems to have really found a perfect spot to create a higher level C, I think it will be a very significant language over time.

Here is a good article on concurrency primitives Go vs Erlang.

http://www.informit.com/articles/printerfriendly.aspx?p=1768317

Clojure was very tempting, I'll likely try it out in the longer term. The Android support in it wasn't enough (no treeshaker) to make it as worthwhile to fiddle about in.

Mirah seemed like the most promising of the JVM languages. Ruby's syntax seemingly without the speed issues of Jruby or Jython. Some minimal changes to support type hinting but it doesn't look burdensome at all. My success with Coffeescript lead me to believe Mirah will have similar benefits. I'll likely try a small Android app in the medium term with it. Looking forward to exploring it.

Regardless of whether Scala is a good choice for me, it's really great to see all the languages available to us these days. :D

Finally some official words from Yammer. So Stephen would you show some integrity by prominently featuring this article on your blog as you did with that leaked email so people can see both side of the story?

@All, Just a reminder that my key goal (as stated in the last blog) is "to provide reassurance to others who feel as I do that Scala just isn't right". That means that I personally am taking some heat with the express purpose of allowing others to feel more confident to speak out in their views of Scala. Given the level of vitriol expressed here (and on related threads around the web) I'm certainly taking the heat. But I've also noticed more people willing to speak out and say they have negative points about Scala. Having views from all sides in the debate is a Good Thing, as it forces those looking at the language to think more deeply about whether it is good for them, and more importantly for their team and company.

Right, this "discussion" (flame war is a more apt nomer for the string of reactions on a blog post which I consider well-written, well-argued and seems to raise valid points even if I can't currently validate them personally, at the moment) has quenched any ambitions on my part of picking up Scala, period. I'll stick to http://xtend-lang.org/: very close to Java, with (almost) everything you miss and without (almost) everything you hate about Java. It's quite a new language though, so stand-alone compilation (outside of Eclipse IDE) is still being worked on.

On a more serious note, I'd like to thank Colebourne for what he said at: "1 December 2011 08:28" -- specifically, "... I personally am taking some heat with the express purpose of allowing others to feel more confident to speak out in their views of Scala."

Frankly, when I evaluated Scala I had some concerns about the complexity / productivity trade-off, but after looking around at the wicked-nasty behavior of the Scala community when someone doubted it publicly, I sort of kept my opinion to myself after that.

If you go on the Internet and say something like, "I found the complexity of Scala to be intimidating" you often get back a response like "Actually, Scala is LESS complex; it has a smaller footprint than Java does." You might also complain that "Scala is too hard to read" or that "Scala has too many features". The responses might be "Actually, Scala is a joy to read because so much of the punctuation is optional" or "You do not have to use all of them". These are just examples of course.

One of the features I most liked about Scala initially was "infix notation". For example, "string1.startsWith(string2)" could instead be written as "string1 startsWith string2". This is brilliant and it is one reason why Scala is so great for DSLs. It also blows the "smaller footprint" argument out of the water.

Much of the Scala standard library is written in such a way that it really extends the language. If you add all this up, the effective language footprint is quite large. It is also an example of why you cannot just ignore features you do not like or understand.

Scala enthusiasts, and certainly the language authors, use every powerful feature of the language whenever they can--and why not after all. The problem is that the only way to avoid a feature and the interaction between features is to write much more code than you read (unlikely). These are the kinds of things that make many developers see Scala as "complex".

Another thing that infix notation is an example of is how some Scala features work to obscure what is actually going on (like implicits do). Punctuation is meant to convey meaning. Removing punctuation may be esthitically pleasing but it also hinders communication.

This preference for implication and "magic" is also common in languages like Ruby, Perl, or even C. It is nothing like Java though and a complete mismatch IMHO for the expectations of Java programmers.

"The problem is that the only way to avoid a feature and the interaction between features is to write much more code than you read (unlikely)."

@Justin: This is so true. One of the best ways to learn a language and become a better programmer in general is to read other people's code. However trying to learn by reading code in Scala is like an English speaker trying to learn Japanese by reading a book that alternates between Japanese, German, and Latin.

> I responded by noting that writing the FCM closures proposal for Java, altering the javac compiler,

Isn't this the same javac compiler that Odersky originally wrote himself. If altering the compiler entitles you to an opinion, it entitles authority status to Odersky on all things Java, and this is by your own metric :-) So when Odersky speaks, nobody will pay attention to authors of time libraries!