Also: You need to kill the standards committees that keep coming up with stuff like WSDL and SOAP. Pretty much anything based on XML and you should get out the pitchforks and torches.

Stop hurting yourselves.

I am speaking as an OS guy who also does embedded systems. We have our own pieces of hell to deal with (over-designed and under-implemented protocols in USB and networking, natch; also hardware engineers who like to take away components to save money).

The last time I looked at web development it was a steaming heap of vile, corrupted technologies that barely talked to each other. Maybe alien unicorns have descended and made everything fluffy and nice since I looked last. There /is/ hope. I mean, you deprecated BLINK tags.

Honestly, do the database people and the client people ever talk to each other? Do the people doing browsers ever talk to the people doing IDEs? Or is this like different tribes, and they are trying to screw each other over in some kind of zero-sum game?

Y'all are fooling yourselves if you think there's not a problem. Look at the number of frameworks; these things wouldn't exist unless something needed fixing. Last time I was in the bookstore it seemed like there a dozen linear feet of dead trees describing frameworks. God knows how many there are online.

Web development has long since moved on past WSDL and SOAP; they've now been relegated to the domain of Enterprise software.

As for the claim that the proliferation of frameworks indicates a problem, well I completely disagree. They are so many frameworks because progress is being made improving the technology all the time. This is a good thing; it is how progress is made. By your logic there is a problem with the automotive industry (people keep making new cars), the computer hardware industry (new chips), in fact pretty much every industry that exists today.

It's human (hacker?) nature to think there must be a better way, and do something about it.

If you find the first statement controversial then I suggest stepping outside of the world of HN once in a while.

As to the second claim, SOAP is very popular with Java and .Net developers, especially when they have taken up the latest fad, SOA. There are also entire industries, such as healthcare, that are heavily pushing SOAP-based standards for data exchange.

Yes it can. In fact if the article was about how enterprise web development using SOAP and XML was broken I'd be fully supportive of it, given how web development has matured past that to leaner, more suitable technologies (REST/JSON)

I empathize with this point (I even up voted) but I don't entirely agree.

I want to say that we should have some grand unified abstraction for describing web applications but that doesn't represent reality outside of web development. We're still living in a world with multiple OSs (never mind the browser war).

The technology world isn't isn't unified.

if it truly were, would we be able to express solutions to arbitrary problems?

Such a unified solution as you hope for would implicitly prevent the flexibility that allows for technological advancement. Ruby on Rails "golden path" approach is actually a good example of this. Development is easier following the path. Deviate and it gets tricky quickly. But at least it's possible to deviate.

I believe the solution you posit would result in a black box approach to development with only one right way if even one at all. Deviation from the intended use of such a tool would likely be impossible.

It should be titled "web development for the platforms and scenarios I target has too many moving parts."

"I don’t care about browser wars - my standard toolkit must work on all browsers, end of the story".

ALL browsers is very broad, and "work" is pretty vague. "Behave the same way"? Or just "not cause an error"? You want one codebase to work the same way on links, IE9, Safari and Blackberry? All of those browsers are part of the community on "the web".

"I don’t want no glue - the web is one platform, time to stop forcing us to treat it as a collection of heterogeneous components".

No, the web is a collection of heterogenous components - stop trying to force everyone to treat is as one platform. The portability and interop is what's allowed people to keep migrating bits and pieces around to get something that works best for them. I can change my logic from PHP to Node.js without having to relearn a new markup language for a different browser ecosystem - the CSS, JS and images I used earlier will still work from a new server.

The broad scope of what 'the web' is, and all the various technologies involved and required are what make web development so interesting. "Web developers" is an extremely broad range of people, constantly evolving and churning and growing and changing. If you want "one platform", you're free to focus solely on writing only Rails apps that talk to a Silverlight front-end, for example, or use only a PHP/IIS/Flex stack.

If you focus only on that one stack, some of your issues and concerns will be addressed, and you can become an expert in that one set of technologies. Beware that the larger web world will eventually evolve past you, and much of your skill won't be moveable to the next stage of web evolution.

"Web development", almost by its nature, requires a "jack of all trades, master of none" type of person, who can understand a broad range of problems and solutions, but also needs to call in experts for specific areas when required.

I disagree with parts of what you say. Your answer largely sounds like: "I like it how it is, I'm an expert, don't you even dare to think of a better way".

> I can change my logic from PHP to Node.js without having to relearn a new markup language for a different browser ecosystem - the CSS, JS and images I used earlier will still work from a new server.
Sure, you won't have to relearn a new markup language and that's clearly good. But, look, your database will still stop working and your client-server I/O will suddenly show a completely different behavior. The current generation of web development requires a modularity that generally doesn't fit the application, and it shows.

There is an element of that, certainly. I am an expert now, at certain aspects of web development. In others I'm not.

But that said, the beauty of things is the interop. By all means, think of something better. I dare you to. I just don't think anything 'better' will retain any degree of cross-platformness, either for developers or end users. We had HyperCard and Director and VB and many other platforms which took care of a lot of things for us, but also locked you down to a relatively tiny audience of people. "The Web" doesn't do that (at least not to the extent earlier platforms did).

Thing is, nothing like this has ever been done before - app access distributed around the world with a massive ecosystem of development tools and services which by and large can work together, even if they weren't designed to work explicitly with each other.

re: "database will stop working". No it won't. I will need to use similar APIs to access it - the database engine itself will keep chugging away just fine. And... my client/server I/O will change? How so? A browser sending in a POST will be accessed in a request.post (or whatever the exact syntax on node is) and $_POST in PHP. So what? Presumably I'm switching because there's some other benefit (concurrency, team skills, etc) that I'd get beyond relearning the specific syntax of general concepts in a new platform.

Again - I wholeheartedly welcome people to envision a new paradigm. I'm old, cynical and skeptical enough to demand that I "see it before I believe it", and given the network effects at play, it'll have to be magnitudes better for all parties involved to catch on.

> Again - I wholeheartedly welcome people to envision a new paradigm. I'm old, cynical and skeptical enough to demand that I "see it before I believe it", and given the network effects at play, it'll have to be magnitudes better for all parties involved to catch on.

Fair enough. Let's meet again in a few months/years and compare notes :)

Web development is not broken, it is just non-trivial; you need to think at different levels of abstraction: server-side, client-side, database, etc., and there are usually different technologies for each.

But web-based software is taking over the world; empirically it is not broken, it is very very successful.

And if you want to imagine some magical technology that solves these problems, you are probably falling for the silver bullet fallacy - anything that is advanced enough to solve complex problems (which web software now does) needs to be sufficiently complex to solve those problems. And any abstraction layer is likely to suffer from leaky abstractions (http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...)

That's not to say that improvements can't be made (Rails is one example that aims to make the web development process more coherent and consistent), but fundamentally (and empirically) it works.

I've written web apps professionally for years now. I recently went back and wrote a GUI app.

It was horrifying.

First, I had to pick a language. Then a GUI framework. Finding a combination of those that was cross-platform was trivial... As long as the answer was 'Java' and 'Swing'. -sigh-

I really wanted to write it in Ruby, but Shoes was not a good solution (too many features missing... Ones that I desperately needed) and all the other GUIs required special installation, or weren't up-to-date in the Ubuntu repos.

In the end, I chose Mirah. Java, but with some Ruby-ish syntax. That was it's own special torture because it's such a new language that there isn't much out there in the way of documentation or examples. (One time that I complained about that, someone suggested I contribute some. https://github.com/wccrawford/TV-Director is sample that is the answer to that.)

> all the other GUIs required special installation, or weren't up-to-date in the Ubuntu repos.

If you can get past the snide comments by "in the know" people who used it in 1998 and didn't like it, Tcl and Tk are actually still a good way of doing cross platform GUI development, depending on what your needs are.

Is it perfect? No. It has improved a lot though, and if you want cross platform, you're probably never going to get "Apple beautiful". You're going to get a jeep, that gets you where you need even if it's not as nice to look at as the Ferrari.

Tk also screwed something up that PHP later copied in a certain sense: it made it really easy for people to do something. Many of those people were not "qualified" to create UI's and thus went on to make some really ugly stuff. Moral of the story: if you create tech that makes things significantly easier, lowering barriers to entry, try and foster a culture that helps create "beautiful" results.

The very idea of cross-platform GUI frameworks is a fallacy. That's because in GUIs, the important stuff is in the details; but platforms are all different, in different ways, so on each platform, care must be taken to offer the best possible experience.

Most toolkits (Swing, Fox, Tk, Qt etc.) work around this problem by ignoring it and merely instating their idea of what a GUI should be, or offering the lowest common denominator. So you end up with non-native widgets that don't "feel" right, bad performance (often because you can't take advantage of the platform's native graphics stuff) and all sorts of impedace-mismatches in the integration with things like file dialogs.

The only way to do good cross-platform GUI development is to go full MVC and isolate the non-visual parts from the UI to the point where the UI bits are just controllers and views that interact with a generic backend. This way, the UI is small enough that it's relatively trivial to custom-build a UI for each platform, making sure that each UI gives the optimal user experience for that platform.

It's more work, but it's considerable less work than writing a cross-platform GUI toolkit. With some careful planning, a lot of GUI-technology-independent code may be factored out. For example, a toolbar can be abstracted out in a generic implementation that the native implementation can invoke to make decisions. In a 2D-drawing app, the entire graphical buffer system can be platform-independent. And so on.

I know just one app that is implemented as a cross-platform GUI yet is equally "slick" on all platforms: Spotify. The app is written in C++ and draws its own widgets, but uses native widgets for things like input fields. However, Spotify's UI is very simple, and it camouflages its non-nativeness by using a dark colour scheme with some pretty neutral/browser-like design choices.

Interestingly enough, one of the solutions I looked for was a way to design my interface in HTML, since I was a lot more used to it. I didn't find anything that was far enough along to bother with it for a GUI app, and I couldn't really make it work right as a web-page that communicated with a local process on the server to launch certain other GUI apps.

This is largely a solved problem, at least if you choose the right framework.

For example, Rails + Heroku gives you sensible defaults for your programming language (Ruby), you web server (Heroku), your database (PostgreSQL), your server side web framework (Rails), and your client side web framework (Prototype—perhaps not my favorite, but certainly a default). You can have this all set up from scratch in well under half an hour, your first time through.

If you don't like Rails, you can find similar pre-configured environments elsewhere.

The real web development headache is just how many of these layers you need to understand: SQL, a server-side language, HTML, CSS, JavaScript, and multiple libraries designed to make server-side and client-side coding sane.

According to other comments in this thread, the author has a commercial product designed to address the issues he raises. But if so, he does not seem to realize that he's working on a problem with many existing solutions.

"This is largely a solved problem, at least if you choose the right framework."

I've never used Rails or Heroku, so I have a question - how many companies can depend on deploying to Heroku? Is it really flexible enough?

I have only used Python + Django. A few solutions are starting to crop up for Heroku-like easy deployment. But I have some special needs that I don't think will be covered well by these services (for example, I have to do Comet, which means setting up an extra server to do Comet, etc.)

My gut has always told me that Heroku is fine for deploying your application initially, but that most companies will quickly outgrow it and have to resort to manual deployment solutions.

By the way, in my case, I started off with deploying on DreamHost, which is almost brain-dead simple. Then I moved to WebFaction, which gave me more flexibility but I had to do some extra stuff for deployment. Then at some point, when I started making more serious apps, I moved to using VPSs and encountered exactly the nightmare the OP is talking about. I suddenly had to learn how to deploy from scratch, which involved learning to install and configure various programs, and get them working together. It was... not fun.

I've never used Rails or Heroku, so I have a question - how many companies can depend on deploying to Heroku? Is it really flexible enough?

As long as you don't write to the file system, and you don't need non-web, non-worker, non-cron server processes, you're probably OK on Heroku. You also have to be willing to trust Amazon and Salesforce, which isn't too big a stretch in many industries.

On the other hand, if you need to respond to ZeroMQ messages, or if you need to poll S3 every 5 seconds, Heroku's just not going to work for you. In that case, you can either toss everything onto a server running Phusion Passenger, or set up a real deployment infrastructure with Chef (or Puppet). This can take up to 2 days for a sufficiently messy project, assuming you want to fully automate everything, and I agree that's it's not fun.

(Edit: Chef provides pre-packaged deployment instructions for Apache, Unicorn, Rails, ha_proxy, a database, etc. This takes a couple of days to work through the tutorials and set everything up. The setup is a bit of a drag, but it's sweet once everything is working.)

"This can take up to 2 days for a sufficiently messy project, assuming you want to fully automate everything, and I agree that's it's not fun."

It can take much longer for someone who has never built a server at all. Learning to install and configure Apache + mod_wsgi (in the proper modes) etc. is the hard part. Doing it the second time is much easier.

> The real web development headache is just how many of these layers you need to understand: SQL, a server-side language, HTML, CSS, JavaScript, and multiple libraries designed to make server-side and client-side coding sane.

So I take it that you agree with me on the brokenness of web development, even if you consider that setting things up is not all that bad?

You mention the layers themselves as a headache, and I could certainly have added them to my rant - for many, they are a nightmare by themselves.

But let's just assume that we're all seasoned developers and that no language or sum of languages is too scary for us, as long as they interoperate correctly. We still have a nightmare, here, because these languages simply do not cooperate. Just getting them to work together takes considerable amounts of glue.

And please tell me if I missed something, but last time I checked, RoR + Heroku really didn't solve much of the glue nightmare.

So I take it that you agree with me on the brokenness of web development, even if you consider that setting things up is not all that bad?

I completely agree that web development is a mess. I just don't think that the first major problem described in your rant (choosing a programming language, web server, database, server-side framework, and client-side framework) is actually a major problem these days: Just pick any modern stack, and use the defaults.

And please tell me if I missed something, but last time I checked, RoR + Heroku really didn't solve much of the glue nightmare.

Rails provides excellent glue between the database and server-side code, at least if you like the sort of APIs provided by ActiveModel. (I do, some people don't.)

Rails provides less glue between the server-side code and the client-side code, at least out of the box.
Basically, it pre-configures Prototype, and makes it easy to do basic AJAX stuff via view helpers and unobtrusive JavaScript. If you prefer jQuery, you can switch over in about 30 seconds using jquery-rails.

Rails does not provide a default client-side toolkit for highly interactive JavaScript applications, such as those typically built using Backbone.js or SproutCore.

> I completely agree that web development is a mess. I just don't think that the first major problem described in your rant (choosing a programming language, web server, database, server-side framework, and client-side framework) is actually a major problem these days: Just pick any modern stack, and use the defaults.

Actually, the first problem mentioned was not choosing, it was _dependencies_ and then _documentation_.

That's the _dependency nightmare_ you're mentioning. I agree that RoR solves a non-negligible chunk of it. But I have never seen a "rails new foobar" configure my SELinux or introduce any sane security policy in the database, for instance. So, the nightmare is still here.

The _glue nightmare_ appears when you're trying to pass non-trivial data from the client to the server, or worse, from the server to the client. Or when you're trying to validate the heck out of input data to ensure that there is no CouchDB injection or SQL injection or anything such.

If he is as successful as Rails, one of the biggest impacts he'll have is the creation of a hundred other frameworks in other languages.

Also, the "I don't want no glue" line is a lie. Any such framework to coerce Web Development into a single language is all glue, all the time.

Further, he has a marketing problem. Most seasoned developers probably will not flock to a framework that prevents them from writing their own HTML/CSS/JS/SQL, so he'll have to grow his user base on people who haven't been programmers for long.

When I said you could set everything up in under 30 minutes, I meant starting with a bare Macintosh (or Linux box), with no development tools or Heroku account, and ending with fully deployed and scalable production application.

Gems are super simple once you get the hang of it. Look into installing RVM[0]. It will let you manage multiple versions of ruby at once. Also get familiar with Bundler[1] and watch the Rails 3 screencast[2] that explains it.

Take bricestacey's advice, use RVM. I was in the same boat as you when getting started with Ruby - it seemed like I was always fighting with gems and other dependencies - "there has to be a better way!", I thought. And there was...

At the moment, web development is the equivalent of using Linux in the early 90s - you need to enjoy the process of getting it running more than you care about producing results. We're hoping to bring the Apple experience to web app development by providing a consistent set of core libraries, ready made API wrappers, a browser based IDE and one click deployment.

Currently the service targets fairly advanced developers who are well versed in JavaScript, understand relational databases (yet don't want to write SQL) and know Git, but have better things to do than administer servers.

With time, we are looking to add one click GitHub push/pull/fork+clone integration, web applications & API mashup templates and finally tutorials for getting started with development in JavaScript for the very beginners.

Sure, if you're planning to do something special, you can pick and choose all the bits the author talks about. But if you're doing that, you (hopefully by definition) know what you're doing and are able to make intelligent choices about all of them.

If you just want to build a web app, all you need do is pick one of the big 3 stacks and get going. The Microsoft Stack gives you everything you need with no mandatory decisions on your part. Same with LAMP (read the acronym and you've got your list of pieces). Even Rails has a standardized stack at this point.

From TFA: "You have reached that stage, because you have the time and skills to do this, but what about Joe beginner? Do they deserve this?"

Joe Beginner will be fine provided he has the tenacity to learn, learn, and continue to learn. What he "deserves" isn't relevant. The reality is we're all Joe Beginner at something. For example, I have been writing code for almost 30 years (started real young) and have been doing web-dev for 10 years, but I'm Joe Beginner dealing with Ruby on Rails, MongoDB, and several other technologies I'm trying to wrap my head around.

Part of...well, actually one of the key aspects of being a programmer is maintaining a complex and diverse skill set. This was true back in the 80's when I started with AppleBasic, it was true in the 90's when we were writing Win32 in C++, it's true now, and it's not going to change.

Call me crazy, but I think using three or four different languages is of the things that makes web development really fun, more fun than trying to wade through pointer pointers to figure out where the hell I forgot to toss memory back on the heap. And isn't that why most of us hack anyway, because it's freaking fun?

Interesting point of view. I started coding 7 years after you (started real young, too) and I'm pretty sure the reason I picked up coding was because it was so easy to just get "hello world" running in GWBasic (and then Turbo Pascal).

Good 'ol Turbo Pascal. I wrote an encrypt/decrypt program in TP and a friend of mine and I used to pass encoded messages back and forth on 3.5" floppy disks.

To the subject at hand...I think it's a question of motivation. "Easy" was never part of the seduction; I wanted to make games, freak my folks out by making the screen flash and beep when they turned the computer on, hack the computers at school to lock out Walt the dickhead computer teacher guy, etc. "Fun" was always the motivation; "easy" never part of it because easy is...sorta boring.

"Easy" has value, to be certain: IMO it's a signal that it's time to automate that task. For example, I used to hack Apache vhosts files by hand. Now it's easy, and we've automated it (and the rest of the LAMP stack setup, for that matter).

I agree that "Fun" is just as important as "Easy", if not more. But the nightmares I'm mentioning? In my experience, they're just waste of days of my life. Not Easy. Not Fun. Not even remotely interesting.

Web development is complicated because it grew organically, isn't at the mercy of a single vendor and its security model supports both untrusted clients and servers. If you want simple and straightforward, go back to writing Windows Forms applications (or a similar equivalent)

If you try applying this POV to any field, it'll sound like "developing anything on any platform is broken"

If you want to develop something significant, you'll have to dive deep and spend some time boosting yourself on the learning curve.

Speaking of web development, most good MVC frameworks today solve this problem for you.

Ex: when you use rails, it makes your dev server a command away. It automatically creates an sqlite database by default. If you are building trivial stuff, you do not have to understand SQL and directly deal with the database. All you have to understand is some ruby and a bit of high-school HTML to create simple sites. Whether those bare minimal skills (i.e, minus CSS, minus Javascript) help you build a startup or get profitable is a topic for another day. But it does get you started!

Deploying... whether you use Heroku or any other platform or your own rock solid VPS, is upto you.

Joe Beginner huh? Is this just complaining that web development is hard? Seriously, why don't I start a rant saying: "hey building a house is hard, how the heck are us beginners supposed to do it?".

If you want a web-site made easy use Wordpress or something else that provides what is needed for basic scenarios out of the box. If you are expecting standard interfaces for web-servers, programming languages and web standards that are not going to change regularly as software development matures as a profession you are insane.

Just take an example of something that is already relatively standard like SQL, now write some SQL for MSSQL and then switch to Oracle...

Lets face it beginners should not be building non-trivial web applications where things like scaling matter. At least not on their own, and if they do they should expect it to be a challenge and a difficult learning process they will need to dedicate themselves to. .NET tried this, they wanted to make web development as easy as drag-and-drop. In the end .NET ended up with two things:

I absolutely agree with the author! Remembering programming GUI applications in 90th it feels that there is a huge regression. Just to add, it is not enough to know 1 single programming language - you need 4: HTML (not really programming but still syntacs and semantics to learn), JavaScript, server side language (php, java, ...), SQL. Not talking about all the different configuration languages that might be needed (XML, ...). Not yet speaking of the difficulties for session handling. Where are all those nice innovations from the past MVC, OODBs, ... It would be great to have ONE single language that is enough to build a reasonably complex web site without having to cope with all the other syntaxes. I think that because of this something like Node.js might have success, even if I don't like JavaScript. But I still don' t get it why there are no persistent, performant, scalable data stores available inside programming languages itself (HappS seems promising for this, but then you still have to use JavaScript). I hope that after almost 2 decades something more simple will appear.

It seems that nowadays you could in theory write web apps pretty much entirely in JavaScript. You mentioned Node.js for server-side logic. Then there are data stores that support JavaScript as a native query language (MongoDB, CouchDB...). For the client side, you can send an almost empty document to the browser and build the UI entirely in JS. Projects like CouchApps seem to head into that direction.

However I'm not convinced that it's really a good thing for separation of concerns. You made a comparison with GUI development, but actually it's not uncommon to use a separate language to specify layouts when creating GUIs (Qt, Android...) The reason for that is that the declarative style of markup languages can often be a better fit than describing layouts programmatically. As for databases, it's great to have a (more or less) standard interface to query relational database that can easily be used from virtually any programming languages and from the command line. A lot of web apps are not limited to interact with a db and the filesystem but also need to interact with legacy systems. Less technical people might also want to be able to use off-the-shelf graphical tools to query the database. That might be a reason why ZODB, a Python-specific data store, hasn't been tremendously successful, despite meeting your requirements (notwithstanding scalability perhaps).

On the bright-side, these things provide a nice barrier-for-entry that makes it easier to find jobs. But I agree it does feel quite 'broken' a lot of the time. I've been trying out jQuery-ahm, which lets you reduce ajax calls to one line, and send everything else back from the server - the data and the logic/functions/instructions. I'm not sure how well this scales (debugging could be a problem), but I'm using it for simple things like tooltips. I feel that writing all the normal ajax code for a simple tooltip (for instance the example given in the Rhino book) is preposterous.

Yeah I see your point, but when I said 'one line' I meant more like 'one word.' jQuery-ahm just uses a 'ahm' class on the html link to the Ajax endpoint. like <a href='/tooltip3' class='ahm'> and that's it. The line of jQuery you wrote is still very dense.

I mean, it's not as if things are the way they are just to make your life miserable. It's a matter of the incentives involved. And anyway, if you want to 'just get started', PHP is pretty good at that. You can have your hello world page up on the internet in a few minutes.

A "Hello World" PHP site is nowhere near the complexity of a web web application. More importantly, when you start cutting corners by avoiding those steps (As so many PHP developers seem to do) the complexity gets worse.

I'm not really a fan of PHP, but I think it's fair to say that it does let you get started doing really basic stuff quickly and easily. Especially when it came out, that was a pretty big point in its favor compared to the other things out there at the time.

The author is right, if by broken he means insanely difficult. The number of things that a person has to be aware of to launch even a simple web application is simply daunting. Even after launch, there are even more pitfalls and gotchas awaiting. We've come to accept this insanity as normal, and perhaps the time has come to demand better.

Has anyone ever thought about how nice it would be to have one really nice strong, static, inferred type system that ran from the javascript in the gui all the way to the db and even spanned any rpc stuff? Then imagine if there was a single IDE that could catch type errors anywhere in the stack. In my opinion this is why there are no trivial webapps. Even something that wants to add up a list of numbers and store in the db has to deal with so many type systems (or lack thereof), even though all your data becomes strings as it is posted over HTML.

Of course Web development is broken. Because Web was never meant to be developed for.

Originally, Web was intended as a collection of resources -- actually, a filesystem of sorts. But it grew out of proportions as it became popular since it allowed users to see color and pictures and animations on the Internet, which was up to that point either limited to plain text, or required some heavy-weight, non-standard applications to be installed on the client. Actually, the name used for the application used to access the Web -- a browser -- tells a lot about how it was intended to use: to "browse" the resources, not to execute them. Could you imagine what desktop development would look like if you were limited to using just some sort of file viewer to program for it?

Each Web application is, actually, two completely unrelated Web applications. One is executed on the host and is preparing the data for the Web server to serve; but there is another, which is running in each of the users' browsers, only connected to the former one by asymmetric pairs of requests and responses. Even if it consists only of HTML (and CSS), it still has code being interpreted and evaluated on the client; ajax apps only emphasize this.

So it's not Web development that is broken; in fact, it is a miracle how what was created by the developers to work around the fundamental limitations of the platform, which was never meant to be one.

If this post was written 3 or 4 years ago I might be inclined to agree, but with elastic compute, REST, HTML5, CSS3, jQuery, JSON, decent modern browsers and NoSQL databases we really are in (or are very close to) a pretty good place IMO. If the OP is complaining that he needs to know a lot of technologies to construct a website... well, I think that is the nature of the beast and falls out of the de-coupled nature of web-apps.

err.. other than perhaps jQuery, I don't see how any of your examples refute what the OP is stating. To stay current with modern web development (such as the technologies you listed), developers need to continually improve the breadth and depth of their knowledge.

The OP states that web dev is overly complicated and broken. I counter that modern tools are far from being broken, and are infact lightyears ahead of what was available only 3 or 4 years ago. Not a refutation per se, but a counterpoint.

Isn't this just the problem of choice? If you really want an end to end cohesive development stack here you go: use google app engine with Java and gwt, all java, all eclipse, all the way to the client side javascript / packaging etc.

But I'm happy to have the choice to use python, and play with different client side javascript libraries to suit my various needs (I do use app engine). And maybe next week I'll use clojure on the server side next week and coffee script on the client; that's the beauty of choice.

Overall if there were a single SDK for web development ala IOS development of android development, I would enjoy it less. I'll take the rough edges along with the additional choices and flexibility.

Some of those don't look like problems from the mobile side of things. For mobile, replace "you’ll need to choose" with "you're forced to use". While the OS and security stuff might be simpler, it's not exactly solved on the handsets. Depending on the platform, you'll having to deal with potentially dozens of device models and several different form factors (BlackBerry, I'm talking about you). And if you find a serious bug in the current live app? Have fun waiting several days to a week to get it updated in the app store/market/world.

Web development isn't broken - its just difficult to do. Which is why you have to know all this stuff to do it well.

Modern Property development is broken. First I need to have a registered architect design a building, then I need to purchase the land, then get government approval, then deal with contractors and bank financing. I just want to choose a spot in a city and then start building my dream building. This is 2011 and this is the best we can do?

The further away you move from low level control for abstractness (read: easy) you give up just that. Control. One of the requirements for this abstractness is everyone has their 'own' ideas of what 'easy' is, thus you always end up seeing framework N1,N2,N3... and so on.

If you take a look at Rails, there is much argument about "well I don't like Ruby, much prefer lang xyz", or "I don't like the magic, I like abc" (just giving an example). This exists for everything out there today, because we are all different.

It feels like everyone expects to just load up an IDE and start coding, but this will probably never be the case again. The Internet has brought us the ability to not only fork code but fork ideas. Having to rely on a main branch of well water (read: visual studio, xcode) leaves you powerless (see my main point about giving up control) in the end, (eventually as iterations increase).

Perhaps when something is not in physical form, the concept easily escapes us. There are many ways to 'hammer a nail', or 'code hello world'. The basic principal remains the same, blunt force 'could' be applied or you could just use a screw, or... (you see my point).

It is again probably the Internet which has increased the iteration of improvement, and obviously people do find it confusing to have to 'shop' amongst hundreds of options. If you look at it, it's no different than what we do at a grocery or hardware store today.

So, maybe it is really one's lack of understanding with the fundamental problem at hand (that your trying to solve). Knowing that, you could choose the right tool, level of abstraction, faults, etc... for the job at hand.

Frankly, I believe it's that we're working at the wrong level of abstraction. High-level of abstraction is good – and it's good that there are several competing high-level abstractions. That's exactly what happens in desktop programming, e.g. you typically don't use the same languages for UI development, games, scientific processing, etc.

But one of my points is that we definitely haven't reached this level of maturity on the web.

Web development complexity mentioned here is just a reflection of a complexity of the outer world. Activities you perform throughout the day require you to make choises which in it's turn requires knowledge gathered during one's life. Yes, you can't be an 'average Joe' to build a web application, but you can't be the same Joe to build a motorcycle or go rafting or found a glass factory without a (impressive amount of) domain specific knowledge.

World is complex. Technologies are. Deal with it.

Another thing that seriously bugs me is the approach to mask completely different goals and processes by imaginary simplicity of overlaying technology stack. Writing a web app and making it scale are totally different goals, choose what are you going to do first. Textual documents storage will differ from video hosting by a couple of orders of magnitude. The more generalization you achieve, the less freedom you'll have.

Google Appengine (and presumably Azure) takes you some of the way there - it removes all the sysadmin side of things, and solves the scaling problem at the same time. You still need to choose a framework etc, but it is a compelling choice for a developer-centric startup.

Sorry, this might be too pedantic, but I can't listen to someone saying that something "solves the scaling problem" when it doesn't. Both of those examples solve the "having more machines when you need them" problem, but scaling is a) domain specific, and b) a logical problem not a hardware one in general. A million machines and an insoluble deadlock is not a scaling solution.

I'm sure that you know this, but in case anyone out there less well informed takes that at face value I felt compelled to add to it!

I think the larger problem is that the "web" is maturing and as such, clear delineations of roles are forming just as they have in other areas of developers. Lets take for example game development. You have your physics developers, you engine developers, your level developers, level designers, and artists (I am skipping a bunch of other diciplins but for brevity this will do).

Anyways, back it the old days it used to be a 2 man show, usually an artist and a developer and they would knock out a game. As the industry matured and the technology involved became more complex specialization occurred. The same thing is now taking place in the web and we are seeing the technology start to converge around the idea of specialization.

SOA and REST has compartmentalized the server side into addressable business resources for performing business computation and access to data. Used correctly a business developers can provide information and processing to the internet without having to touch a single line of HTML or JavaScript. They can write services in pure Java, C#, Python or whatever. This allows a Java developers the ability to provide deep specialization in Java and provide the products of the specialization to other developers in the work flow without encumbering that Java developers with ancillary technologies.

Further many app servers externalize the management of environmental resources such as file access, which becomes the domain of an infrastructure admin. Server administration and configuration can now be performed by technical personnel that specialize in infrastructure related concerns rather than developmental concerns. This is a good thing because it allows more capacity from developers by freeing them up to develop.

Finally, the client side frameworks have finally liberated the UI from the Hodge-podge of technology soup that we created in ASP, PHP, JSP, whateverP. This in my opinion is where the web went wrong. We created the idea of the documents that just souped all of the technologies together and everyone from all distinct roles had to work in this mix. While it was great for the one man show developer the whateverP where always trading of ease of use for the developer for ease of use for the designer or vice-verse. One only needs to look at the contrast between PHP and JSP to see the two distinct philosophies at work.

So as the web has matured you are now seeing more delineated roles than the old web developer or web master or web guy. Now you see delineations like, SOA/REST developers, Content Manager, SEO Developer, Web Designer, UI Developer.

There are companies like Wavemaker that are trying to make it easy for the one man show developers to write modern web application and I believe that products such as it will grow in popularity as the solo developers looks for tools to keep up, but the reality is that the web has matured and with maturity comes complexity. Complexity that is more and more managed by groups of specialist as opposed to the generalist.

We took a document fetching/presentation system and beat on it until we had an application platform, and we're feeling the pain. We have built workarounds on top of workarounds to try to make applications look better, work better, and better to develop, but we're still stuck with the browser and its limitations. Reading some of the posts here, it seems like some people have developed Stockholm syndrome for the browser.

What we have turned the web into is a resurrection of mainframe computing. Our terminals are just a little bit better this time. It was the only vector for exceptional ease for the user so we flocked there, the trouble is, talking to the terminals makes up have to do a lot of less than ideal things on the serverside. Sure we may have prettied up some parts and made more and more tech for talking to the terminals, but its not great. We and bitched and moaned about the state of the terminals and over the last decade they've even became a smidge better.

Writing an application that can support 1,000s of concurrent users (some malicious) connecting from any operating system and client program, that provides a rich user experience, and one that is read/write and backed by a database is complicated.