Simple or Complicated?

Created by odersky on 2010-08-26.
Updated: 2010-08-26, 11:29

Recently we have seen a heated debate on whether Scala is too complicated for normal programmers or whether it’s in fact a rather simple language to program in. Here are two representativeblogposts of the debate. The comments on the posts are also worth reading. I have written up some thoughts here.

In my very humble opinion, I think that at least part of the problem here is that the average programmer hasn't seen a discrete math textbook for several years and if like me their memory is fuzzy w.r.t. the mathematical ideas of 'function' and (the more general) 'relation'. A quick review of en.wikipedia.org/wiki/Function_(mathematics) may (or may not) help, I find myself returning to these wiki's time and time again to keep the ideas fresh in my head when I need to employ FP constructs. Whether I'm using them in Scala, Ruby (with ActiveSupport) or javascript (with a lib like wu.js or underscore.js).

I think that as more and more people join the FP bandwagon, they will find they can't live without 'maps' and more specialized constructs like 'reductions' and 'filters', just like we can't live without the 'foreach' construct.

(Scala is a very very attractive language and I'm hoping a project comes along where I can push it to heavier use in the near future!)

It is obvious and we keep talking about it: it's almost like there are two flavors of scala: the basic client version and the framework/library builder version.

In an "average" project, you don't want everyone to start using 15 paranthesis and <% %%> /:, of course all via implicits...in a nightmarish version of the "worse C++ code" contests...you would have to deal with many former Java developers and I'd really like them to stick with the basic scala.

How do we do this? How do we define what "basic scala" is? Can we have scala "profiles" where the use of certain constructs generates warnings? Do we have guidelines as to what people can use? Is that even feasible?

That may be a valuable discussion as well...like you said, in fact it's both simpler and more complicated :)

I have thought about doing something like profiles within Kojo (let's say a 'beginner' profile and an 'advanced' profile to begin with) - with the intent of generating friendlier error messages for the 'beginner' profile.

I'll hopefully get to try this out in a month or two. Any assistance with this would be very welcome ;)

PS. It was great to see the reference to Kojo (and Himjyoti school) in your post!

I don't think it's a matter of a beginners/advanced distinction, or profiles, or friendly warnings.

I think the core of the problem is a philosophical one and lies in a lot of scala developers not believing that clear maintainable code being is an important goal. Lots of the community libraries favour terseness over all else, abusing operator overloading, and trying to write everything in as few operations as possible. "Cleverness" for it's own sake is seen as a plus.

Separating the language into profiles won't really solve the problem, as all of Scala's constructs are simple enough when used "appropriately" (whatever that means). Getting compiler warnings would just result in switching over the profile to advanced to shut the warnings up.

While I dislike a lot of the Ruby mindset, the language's expressiveness combined with an obsession for clean code leads to elegant and understandable code (even if in Ruby's case it's not maintainable due to dynamic typing + MP).

There's nothing holding Scala back from being at least as simple to understand as Ruby, it just comes down to the goals and beliefs of the developer. To be honest, Scala is simple until abused.

In my case (teaching programming to kids with Kojo) - the 'friendliness' of the error messages matters a great deal. A lot of the time, the kids are unable to understand the error messages that they see when they make a mistake - so they tend to just ignore the messages.

The Fortress Language Specification is in 2 main sections, 'Fortress for application programmers' and 'Fortress for library writers'. The latter's for things like _defining_ operators (as opposed to using them).

I've often thought it would help if Scala did this systematically with (say) an @advanced annotation. Maybe if you define an implicit you have to mark it @advanced (to avoid a warning), but not if you use an implicit?

It would define a smaller core language which programmers can learn easily, and show them tricky areas in other people's code.

I think it should be enough to tell people not to "over-hype" features like operator overloading, implicit imports, abstract types and complex generic scenarios. In general, nearly all client code should omit these 90% of the time, and all programmers will have a happy time with scala.

Also, the talking about "average programmers" should be dropped. I think this is just hurtful to make scala appealing for the larger audience. Scala is a language which nearly everyone can use, as long as all people in a project use the same style and do not try to be functional at all costs. It IS a strict and imperative language after all, and can be used that way: Even then the code is much nicer and shorter than java code.

Just try to use real names for your methods and avoid too much inheritance (both single and mixin), abstract types and complicated generics and you can create code which is pretty much free of side-effects, yet easily to understand for any programmer who does not dwell in the functional realms. Closures and immutable values / collections are concepts which are so useful that is worth to teach them to any programmer. Most, I think, will even be willing to embrace the change of their style if the reasons for them are explained well.

That should be the route out of the "scala is too complicated" arguments. It is in our hands, after all, to swallow our arrogance for "shorter and better code" and write readable, yet concise, code for the masses. :-)

As someone very new to Scala (just started playing with it last week), but with a long background in modern strongly-typed functional languages, I can say that I'm not convinced that Scala is too complex as a language. However, I'm also convinced that it needs much better documentation. Not tutorials or articles, but reference documentation.

I'm convinced that it's the serious lack in this area makes things feel much more complicated than they actually are.

Some of this is in the basics--although that part seems to be because of some documentation regression from the 2.7 scala docs to the 2.8 docs. (See below.) But the lack of good reference docs for some of the more complicated pieces of the standard library can make it quite challenging to work out how things work. Most importantly, the apparent complexity of the API is off-putting, even though the actual use of the API is not so bad. If someone who's completely comfortable with monadic parser combinators is needing a few days to work out how things go together, a programmer without any PL theory background is going to be going cross-eyed.

Here are some samples of things I've had trouble with, just starting out:

1) Double. What methods are on Double? If you go to the current API documentation and search for "Double", you find scala.runtime.Double (an object), and scala.runtime.RichDouble (a proxy class that it's still not clear to me what it's for). What you don't find is scala.Double. In fact, when scala.Double does appear (for example, as an argument to RichDouble), it's not a link. Does RichDouble's interface mirror Double's? As a new user, I can't tell. Eventually, I found the 2.7.7 scala API documentation, which does in fact have an entry for scala.Double.

2) While exploring, I came across Numeric, and thought I'd give things a shot. This isn't so bad, since the whole concept is fairly advanced, but I was also disheartened here by the lack of any documentation in the API docs about what this is for or how to use it. I found an example and explanation at stackoverflow, but again: this isn't the sort of thing you should have to go to an external site to understand--the standard library API docs should explain how to use it.

Why? Because a programmer who's advancing in skill is going to occasionally be browsing around through the docs looking at things, and will come across the whole Numeric hierarchy of traits... if there's good documentation, then a new world of possibilities will open up to them. If there's not, they'll go "Hunh. Wonder what that is for... *shrug*" Not only will they not be making use of the capability in the standard library, but their thinking won't be stretched by the exercise of trying it out... which means that the techniques used will still be opaque to them when they encounter their next advanced piece of the API.

3) The 2.8 collection types. Wow. This stuff is seriously powerful, and I'm currently working my way through "Scala 2.8 Collections" in order to understand how what I write should fit into it. What's my complaint here? Primarily that there's a real lack of clear documentation (that I've found so far) on how to write a container that fits into the hierarchy. Specifically, what's required of it, a skeleton for doing it, and so on. I'm writing some classes to express grid systems, and my first thought was "Oh, this should be the sort of thing you can iterate over, map over, etc. It should be easy."

And as far as I can tell, it *is* easy... but my first thought upon looking at the documentation was "Holy sh*t! What is this?" I even started to doubt that I understood enough about it to *use* the system, much less implement something to fit into it. Now I'm starting to get things, but again, a new programmer is going to have trouble. And more, the documentation that does exist (linked above) isn't written as a reference work for someone who says "Okay, bring up the API and let's see about Maps and Sets and such." It's written as an explanation for people who already know Scala well, as an explanation about "what things are going to be like" (or, now: what things *are* like).

The big missing thing with the collection types is a clear explanation for new implementors, in terms of "this is the minimum you need to do to make your new class act like a proper collection, this is what you need to do to add a bit of functionality, and why you want to do it", and so on.

Of all of these problems, the difficulty of finding the APIs for basic types like scala.Double was the most troubling to me as a new Scala user who is an experienced programmer. Longer term, things like the new collection system worry me because with something that conceptually complicated you absolutely should have documentation that makes it clear to the API consumer "this is really easy to use", and to the API coder "it's really pretty easy to write to this API, as well".

There's tons of "here's a book about programming that is written using Scala" and "here's an introduction to the core features of Scala for an experienced programmer" or "here's a cookbook of the basic things" out there. But the reference material is *awful*.

Maybe I just came in at the worst time, and my experience is extra bad because of the fresh transition to 2.8. But it's still frustrating.

On the plus side, the language looks great, and I'm really looking forward to digging in and coding on my current project... once I understand the core library enough to stop having to play with it all the time to figure out how it works.

(P.S. I still haven't found a web page that talks about the invocation options of "scalac".)

(P.P.S. I would offer to try to improve the documentation, but as someone who's been using Scala for less than a week, I really don't think I'm the right person for the job.)

I agree there's still a lot to be done on documentation. In particular publicly available one. Right now by far the best way to get into Scala is to buy one of the numerous books on the language and work through that.

That would help you with Double, but would not be of much help for some of the newer, advanced subjects such as Numeric. We are working on it. In particular we will hopefully soon release a collaborative documentation tool that would give the community the power to improve documentations where they are lacking. Furthermore, there will be more articles in the "what's new in 2.8 series". One which is already planned is an article how to add new collections to the framework.

I think many people try to put too much emphasis on the "everyone should be able to program" theme.

Scala is a very nice multiparadigm language, and I have doubts that anyone with a proper CS background would had any major issue picking it up. The same goes for C++, F#, Haskell and other languages touted as complex.

Life is complex, many problems are also complex and we need tools that help solve complex problems, while making easy to do so. Like Larry Wall says, when he speaks about Perl.

From my consulting experience the reason many managers like average programmers is that they are easily replaceable.

I don't expect a car to be built by average people, but by people with proper engineering background.

Thank you all for your comments, they are the comments I'd like to have made. I'm also in the "wanna learn it" camp. I have two books to read to see I can get over the hump.... I'm hoping for one of them to encourage using some of the coding style practices that encouarge less tricks, more clarity in your code.

I really think someone needs to emulate what Perl eventually had. There was the easy, introductory book on the simple parts of Perl. Then there was the Perl book that took you beyond that and into some of the bigger concepts of Perl.

One final word along the Perl lines. I think it's funny that the Scala people are having the same arguments that the Perl people always did. ie It's too hard for beginner's / just read the docs, it's really easy... or you can get it down to 2 lines just by doing XYZ...

I find this comparison between Perl and Scala worrisome, as the reasons for the respective complexities of both languages are entirely different. Perl is complicated because as a language it is poorly designed. There is no real expressiveness advantage between Perl, Python, or Ruby. In the hands of a talented programmer, these languages end up being about equal. However, Ruby and Python are far less complex without sacrificing much of anything in the way of expressiveness. The Perl designers simply piled on language features in an ad-hoc matter and made very poor design choices early on that had to be corrected later. There are in fact a host of CPAN modules like Moose and Coro, and even a module perl5i that bundles a lot of these individual language-fixing modules together to fix the core language (See http://yapc2010.com/yn2010/talk/2708).

Scala on the other hand is complex because it is ambitiously trying to accomplish what appear to be inherently complex things. I know of no other language that has so successfully merged the functional and object-oriented programming styles. I also know of no other language that managed to implement delimited continuations on a hostile VM with a compiler plugin. As a final example, Scala made it a priority to make the syntax of the language flexible in order to facilitate the creation of internal DSL's, and this flexible syntax takes some time to learn. Unfortunately, there is inherent complexity in understanding and using these powerful features, and a few accidental quirks have even been noticed over time, caused by conflicting edge cases in the language that were simply not foreseen.

While on the surface Perl and Scala may share some common strengths and weaknesses, I believe that they are entirely different beasts. More importantly, twenty years from now there will be no "Scala5i" module with two dozen compiler plugins to fix the design mistakes made early on because Scala as a language is much better designed and organized than Perl ever was. And, most importantly of all, while there are alternatives to Perl that can accomplish the exact same things Perl can but with much less complexity, there are no such alternatives to replace Scala in existence. Hopefully people can look past the superficial comparisons being made between the two languages and see Scala for what it really is.

I intended to compare it to Perl :) But mostly it's about the community. I see the people who are comfortable with functional programming saying how easy it is, and not understanding that everyone doesn't come from that background.

I just read the first 5 chapters of the "Scala In Action", and you can see the author falling into that mindset. Rather than giving good examples about using the functional aspects of the language, the author starts describing how things are implemented. I was very quickly lost trying to read it. He jumped into Generics(even the + syntax), functional, using the shortened syntax all within the 1st 4 chapters.

I'm just looking for a book that is somewhere between over-simplified and over-complex :( Start by showing how to use the basic features (You can assume I'm already a Java person) before getting into the esoteric. At least clearly explain a syntax before you start using it.

I know this topic is a bit stale, but wanted to add my 2 cents. I believe Scala is complicated. I spent a couple days reading books, doing tuturials and I even started (but gave up) converting my own code. Here are some reasons why it is complicated:

Scala looked very promising, but then I tried to convert what I thought was very simple Java to generic Scala:

I thought, I can make this generic now and it will be efficient, using primitives not boxed objects ... awesome! However I ran into my first issue:

1.) Scala claims to support imperative programming, but is strongly biased towards functional. For example, there is no simple for loop. In the books I read I saw no examples of how to do a simple for loop. I had to to use a clumsy while loop or dig into the library and learn about Range objects and how to create them, etc. This consumed a lot of time.

I got the simple 'equivalent' of this to work after far too much effort (after diggin into documentation to find Arrays have an indices Range object.

for(i<-a.indices) { ...

That works for this case, but what about when I want to iterate by 2 ... etc. Then I ran into problem #2

2.) Scala typing is extremely complex. Scala tries to do typing 'right', which is admirable, but this also comes at the expense of simplicity. Unlike C++ where templating the above would be trivial as templates there are basically just sophisticated C macros, Typing in Scala demands exreme precision. To overcome this there are 'hacks' (and they definitely appear like hacks to me) like 'implicit'. Scala takes typing so far that it makes dynamic languages look good (and I'm a strong defender of typed langauges).

I tried to convert this, I gave it a real effort. I failed, I gave up and went back to Java.

This forum is full of strong supporters and so I'm pretty sure my comments will be dismissed with little thought, but if not, who knows. Maybe Scala 3.0 will be easier to use. Seriously I love so many other things about the language, but it is simply too burdensome at the moment.

Yes, once you learn about the to method on integers (and that type of OO has aways baffled me, but that's another topic). Instead of showing me a for loop, why not show me the answer to the full problem (i.e. a generic version of the Java code).

I'm not exactly sure what you mean by "generic", but if you mean that it will work for containers other than arrays, you're going to have to give up on indexing anyway. Dunno if these offhand attempts are what you were looking for, but they seem less complicated than the java to me, with less explicit type declaration.

// somewhat imperative like your original, will fail for similar cases (im is too short)

Generic means it takes a type <T> which is substituted in later. Instead of having to define this function for all possible types (in addition to type combinations), you define the function once using a placeholder <T> etc.

This is one of the big promises of Scala. That you can do 'real' generics. In Java generics just use 'Object' behind the scenes and in the case of primitives this is memory and computationally wasteful. 'Real generics allows efficient primitive substitution.

Also. In the case of the 'for' loops, I'm not able to do a 'for' loop without having to learn the API. Methods introduced so far have been things like Integer.to, Array.toInteger, zip, map, ... so on. So yes this handles this very specific case. Change the circumstances just a little though and now you have to search for new methods, new 'ways' to do the problem. For example, how would you do this for loop in Scala and notice how you will now have to take a completely different approach.

I understand what generics are, I just wasn't clear what they had to do with your original example. What type are you wanting to parameterize, Double or Array? You're doing sqrt operations, so it doesn't make sense to parameterize Double. If you want to parameterize Array, you're probably dealing with a collection, so you might as well just use Iterable like my example.

As for the for loops, if your complaint boils down to "i'd rather write 3-argument for loops instead of an equivalent while loop" . . . no, scala doesn't have 3-argument for loops, but it does have call by name arguments and very concise closures, which is roughly enough power to define your own syntax. Write your own 3-argument for loop:

Why this focus on for loops? I merely used it as an example to show the bias of Scala towards functional. There are problaby 25 ways to do for loops equivalent. Many of they rely on knowledge of the API. Great, except this doesn't make reading and understanding someone else's code easy, and this for such a simple construct.

Why has no one tackled the generic example? Show me the generic example of the Java code in my original post that handles double, int, long, BigInteger ... etc.

I don't hate Scala. It seems cool. Your 'write your own for loop example' shows it's power. Cool. But the focus was on 'is it complex? And yes. The type system is extremely complicated. Prove me wrong and show me the generic example of my simple for loop (this is trivial in C++, which Scala claims it's template system is superior to). The fact that there are 20+ ways to do a simple for loop, most of whch involve learning specific API methods and such shows that it is complex.

I'm writing this not to disparage Scala, but to point out some of it's real complexity and hope that there could be some acknoledgement and maybe with some thought, some way to improve it (i.e. maybe typing could be looser and simpler for example).

Note you don't really need to define your own method to do all this as you would in Java. It's essentially given to you with the language and collection constructs.

I'm no Scala wiz, so my example may be simplified further or approached more elegantly; but I really don't see how this is complex - it's just different (and quite simple if you approach problems using the tools Scala offers).

I don't know why you were focusing on for loops - you're the one who brought up wanting to do a for loop iterating by 2 and then by the square root of 2.

Now that you've made it clear that you wanted to abstract over the numeric type in your original answer, this would be one way to change my example to support bigdecimal, etc (make the similar change to the iterator version if you don't like zip):

toDouble is because Java doesn't have an arbitrary precision square root operation in standard library, which is why I originally said it didn't make sense to abstract away from double. I don't believe c++ has one in standard library either, but feel free to show your trivial example.

Thank you Cody. I have asked this question to several Scala enthusiasts. You are the first one to come up with your excellent answer.

The question is not 'can Scala do it'. Every language will have some way of solving this, even if an unpleasant copy/paste non-generic solution, or an inefficient boxing solution. I believe Dr. Odersky's analogy of building blocks (meccano versus lego, etc) is flawed in that all complete programming languages are equivalent in what they can do, so the question is not how capable is a language (because they are fundamentally equivalent), but which languages allows the problem to be solved more easily.

In your example you use API features Iterable, Numeric, zip, map, and toDouble. You use implicit (how many Scala users know what that does?) and you use currying. Does that seem simple? In contrast C++ (which people complain is complex) requires non-of-the-above.

Anyways I thank you very much for your time and I may give my conversion another shot now that I have an answer I was not able to find myself.

I don't think it's fair to compare Scala to C++ in this way. This particular example shows off one of the JVM's weaknesses: it has no facilities for abstracting over primitive types effectively. Scala's Numeric type class with implicits is one way to overcome this. Other languages just use what amounts to Object instances and runtime method dispatch, but that isn't type safe. Also, if you wanted, you could write a Scala code example that is identical to the C++ example, but you would use iterators (not iterables!) instead of pointers.

I really think you are making Scala out to be more complicated than it needs to be. Writing procedural code in Scala is almost as easy as it is in Java, and in this case scala is enabling you to do something that you can't even do in Java at the cost of some added complexity(implicits and generics).

Other languages allow me to do more things than Java as well. My impression is that the focus of Scala was to "do typing properly", especially for generics and such and to "write code as compactly as possible". Part of the assumption of the second part was that compact code is more easily written, understood and maintained. The assumption basically was "compact equals simple". I think this assumption has proved to be incorrect (and not just with Scala). Compactness is an admirable goal, but it doesn't necessarily translate into simple and easily understood.

I think you entirely misssed the point of what I tried to say. I'm saying your example highlights a problem with ALL JVM languages because it highlights a problem with the JVM itself. Scala's approach to handling the issue of abstracting over primitive types is new to the JVM language arena and comes with its own strengths and weaknesses compared to previous approaches, such as type safety and possibly performance gains (not sure about the latter) versus having to learn a couple of unfamiliar language features.

I understand you don't like Scala, and that's fine, but I disagree with the conclusions that you are trying to draw from this simple example. If anything, Scala is ahead of the game relative to other JVM languages in what it is trying to accomplish in this particular area. If you want to write relatively performant generic numeric code in a high-level language as simply as possible, I suggest something more along the lines of Haskell or OCaml. Otherwise, for those of us on the JVM, these new 2.8 features look interesting and promising.

I don't think there is anything fundamental about the JVM that prevents a langauge to use something like C++ templates. Imagine a macro-like facility that substitues in the real types in each spot at compile time (I believe this is basically what C++ does). C# also has primitive 'generics' and I don't think there is anything significantly different about the JVM that prevents it, but in CLI that allows it. I could be wrong.

I don't dislike Scala. In fact i invested a significant amount of time reading books and learning it. Some people complain it is too complicated. This thread was about "are those complaints valid?" and also "if so, is the complexity in Scala an unavoidable side-effect of it's power?"

I merely sought to put in my 2 cents. Obviously there is a passionate community and that's great. Good luck to you all.

This example is definitely not type-safe as compiler doesn't know if class T has operators "*" and "+" and doesn't know what to expect (eg. int.*(int) may return long). This is the reason that it's not possible in Scala - JVM unfortunately treats primitives in the same way as C++.

First off. Type safety isn't the focus. It's "is Scala Simple or Complicated?". Scala puts an extremely strong emphasis on typing, but as a result it is fairly complex (at least that was my argument). Some languages don't even have typing, so 'typing' isn't necessary and the level of type checking is a language design choice.

Second, I think you are incorrect that the C++ example isn't type safe. As soon as you try to use the template the compiler will let you know if it will work or not (because class T doesn't have "*" or "+"). You can try to apply the template to anything, but the compiler 'checks' to make sure that it will work; that all the required methods exist, and the correct types are returned. So it is perfectly 'type' safe.

Your template approach requires that we must analyze the templates to find the required methods ("*" or "+" is a normal method in Scala). Instead of implementing interfaces we would have to resolve dependencies manually. That would be a lot of additional, unnecessary work.

I like Scala very much and I don't want it to have too much simplifications at the cost of non-uniformity. I think it doesn't take more than two months of training to be fluent in Scala (provided you're fluent in Java).

Scala is designed to be easy to reuse and maintain, not to be quick to learn. In order to save time in future, you must spend some time to improve your skills. I know no language which is extremely easy to learn, type safe, efficient and with functional features.