Hi, I'm Tony Arcieri. You may remember me from such software projects as Celluloid, Reia, and Cool.io...

Tuesday, October 4, 2011

Node.js has jumped the shark

I've been trying to understand this whole Node.js thing that's been going on for the past few days. It's taught me a lot about how the Node.js community tries to reason about problems, and perhaps what's the real problem with Node.

It began with a relatively short rant by Ryan Dahl. Ryan talk's about how he's mad at complex software, which is at least an arguable position. I liked his rant, but Ted Dziuba did not.

Ted's argument centered around the problem of doing non-trivial computation within the event loop of a system like Node, and how that would affect the performance of the system. To illustrate this point, he gave an intentionally deoptimized Fibonacci function, along with the boldface caveat "Yes, I know there's a closed form solution. Shouldn't you be in front of a mirror somewhere, figuring out how to introduce yourself to her?"

Reading through the lines of Ted's trolling, his point is "I'm not specifically talking about Fibonacci. I'm talking about non-trivial computation as a general problem in Node." He specifically chose a slow algorithm to illustrate his point. However, what was the Node community's reaction?

Redditor "headzoo" suggests that we provide a "non-blocking solution" by unrolling the Fibonacci loop and execute one iteration of the Node.js event loop, calculating one iteration for each I/O multiplexing system call Node is making. When Ted Dziuba asked "Have you taken epoll into your heart?," I think he had no idea of the depths Node people actually have.

I don't really get what's going on in Node people's heads here. I think they seriously believe that throwing computation into the middle of an event loop is a good idea, because Node's event loop is FAST!!#$#! funroll-loops + nextTick = LUDICROUS SPEED!!

This crap is absolutely insane. For each iteration of the loop, Node is making an expensive system call. It's also defeating all of the mechanisms that V8's impressive JIT would use to optimize this problem, as substantiated by the previously mentioned Nodian's roflbenchmark. Clearly this is a very bad idea. But wait, it gets worse!

It wouldn't take long before another Node.js fan put together a Github project for this technique, which also added an "optimization": memoization! In addition to unrolling each iteration of the Fibonacci function to run within the event loop, it now caches the nth result. Now let's throw ApacheBench at it and see how fast Node can serve the precomputed result where n = 100.

Okay, so mistakenly implementing an algorithm this pathologically bad when you're trying to prove a point to someone who's just trolling you is pretty bad, right? But it gets worse.

After pointing this out, a member of the Node.js community (post now deleted) suggested I might have an obsolete version of Node with a 1GB heap limit (because Node uses a VM targeted at client-side JavaScript where 1GB heaps are an uncommon use case, but hey, let's throw it on the server!) and that I recompile without the 1GB restriction so that this retarded algorithm can continue eating up all my system RAM. He says this as if this is a good idea instead of, I dunno, using a better algorithm.

45 comments:

good post :)I'm guessing the moderate users stayed silent on that one. Clearly the sane answer was: don't use node for cpu intensive tasks - though can perform well for IO-intensive ones. But I guess when you have a nice gilded hammer ...

So dude, I'm like, a lumberjack and I swing this axe probably better than anyone else in the world because I'm so fucking cool and everyone else is a total idiot they probably use saws to get the job done (perrish the thought) without realizing they don't get this wicked sweet gouge in the tree trunk that helps it fall in the proper direction when you hit it with an axe.

Some guy came in here talking about how cool chainsaws are one day. Well, I picked it up and started smashing the blade of the chainsaw into the tree trunk. This thing totally sucks, no matter how hard I bash the thing into the wood the tree barely took a scratch, meanwhile, the chainsaw is totally obliterated.

I can only come to one conclusion: chainsaws are for idiot because it's terrible at cutting down a tree.

Just letting you know, any fib libraries for Node were made as a JOKE against Ted's post. we had a field day with it in the Node.js IRC channel. AKA that was a troll library. The recursion of trolling has been fierce.

Hey, Event Machine isn't the promise, the promise is Twisted, and I explained here (http://corbinsimpson.com/entry/posion-was-the-cure) how to do it in Twisted. The solution? Just put your CPU-intensive blocking stuff in a thread. Ugly, but effective. If only Node could do it...

@Reinh - take it from a maintainer, "ugh" is right, you've clearly never actually read the code. As far as "the most promising solution to the 10k problem", well lol, either you're trolling or you really drank waaaaay too much kool aid on the way through.

How embarrassing that the King of Node made such stupid decrees. I wish that communities like this were formed of many, many individuals, so that I could safely disregard the stupid prattlings and focus on the good parts. Unfortunately, much like with the gays and black people, there are official representatives that speak for all of us and negate our individuality. Too bad! I guess we're toast. This article is the final nail in the coffin (until the next 20-something from 4chan posts his own babbling counter-counter-counter troll, of course)

what people who are making these "node.js is not concurrent" rants dont understand that there is huge latency involved between a html server and client. And then they use stupid number crunching loops in single threaded code as a benchmark which itself is not doing anything concurrent nor factors in out of order or very late response times.

To realistically think heavy-weight preemptively-scheduled processes are the solution for parallelism leads me to believe you're an idiot.Get with the program, using native thread-level parallelism is even too granular for current taste. Most abstract the parallelism behind a thread pool, and schedule short-lived tasks. For something like calculating a term in the Fibonacci sequence, you could implement it explicitly in a parallel divide-and-conquer (or "fork-join") way.

NodeJS is good at very few things and there is a huge misconception in the community to fit it everywhere. I once had debate on the same topic (URL: http://tumblr.com/xrm4g1ijnx) and results were pretty much same. Node.js community needs some education we should help them, I would call it an effort to help humanity.

I know a lot of the stupid people who responded with ways to optimize the terrible algorithm were being genuine, but has it occurred to you that some of them might be trolling the troll?

Node provides mechanisms that let you get complex calculations out of the event loop.

In previous rants by Ryan, he clearly says that we should NEVER block the event loop. The whole point of node is to make everything that blocks an asynchronous operation so we can get on with doing something useful with the system resources.

So when you say that "Node" has jumped the shark, it's not really a stab at node itself, but at stupid people using a good tool in a very bad way.

Or Ted Dziuba - who gets full credit for the idea of intentionally blocking the event loop. =)

I wonder about coordination challenges in event loop systems. Especially with mix of real time issues:race condition (several IO calls overlapping on some shared state), exceptions (timeout, error) and important ones like cancellation of execution.

Ted is a harsh sanity check for fanboys (aka developers who don't think past the marketing) but he makes a valid point to keep your programming light when using Node.js. However, I think this should be the case when doing any web development tasks. Leave it to batch processing and web service calls to do the heavy lifting. Node.js is great for IO which cannot be denied and executes JavaScript which everyone already knows. Node.js is only going to get more powerful. Just use it for its power though, not as a panacea.

The problem I noted with node.js is that though it is true that for trivial high IO problems node.js is okay, node.js is not generally being adopted for that reason. Rather some designer who knows HTML and learned some JavaScript from Dreamweaver wants to get in on the real business, and tries to sneak node.js in because it promises to allow JavaScript programmers the same pay and status of PHP. .NET and C++ programmers.

Programming properly in JS is actually a lot harder than in PHP or Java. There's a lot of magic and creativity as there used to be in Perl (actually more) and for that reason, a lot of people used to standard programming concepts can't get their mind to understand it and thus hate it. So when Nodejs start to invade the server, all the haters are eager to claim that Nodejs opens the door for less competent programmers to enter the realm of server programming. Entirely not true! Most nodejs programmers made the transition from either PHP, Ruby or Java, because it opens some new perspectives. It seems to me that this entire Nodejs hate has more to do with ignorance, fear, and contempt than anything else.