Web Development Principles: Replaceable Instances

In my last post I arrived at the conclusion that I would like to dabble in developing for JavaScript on the server, although I would not like to write the JavaScript by hand.
Regardless of the backend technology choice, the past two years of deployment in the cloud have taught me several considerations which I believe impact the design of our library stack.
The first of which involves server state.

A modern web application needs replaceable instances.
A library stack with a default happy path to statelessness strikes me as a nice way to guarantee this.
I want my application servers to be stateless adapters from http to the backend services.
The first reason for this is failure.
Failure happens and we should build our systems to expect it.

Deploying to the cloud makes failures a guaranteed event when we consider auto scaling.
Once traffic gets low, we purposefully decommission instances of the application.
If the instances are stateful in any respect, we have to port the state of any active sessions to the other instances.
However, if they are stateless in the first place the job is already done.

The cost you pay is the server application instance has some state associated with each rendered page.
There is a GUID created for that page render such that any AJAX call from that exact client arrives back at the same server-side function instance.
This means that the rendered page is only serviceable from that Lift application instance.
It’s not just that the server has to create and maintain a GUID in memory (it could store it in a database, for instance).
The GUID is associated with an arbitrary Scala function in memory.
This function could have closed over in-scope data which is difficult or expensive to serialize/deserialize to another application instance.

This is not some lost cause for Lift by any means.
Once I find time, I plan to make it possible to save off a mapping from GUID to server side function in a clustered ContainerVar
(read here for the gory details and discussion).
Even with that in place, there will be restrictions on what one can do in the application so that the state can be restored in new instances.

I would like to work with a stack in which the happy path is stateless from the start, rather than adding it on later.
The stack should be clear about what parts need to be saved off outside of the application.
We should aim to ease the integration into backend persistence, caching, etc to handle this.
To make life easy for the new-comers and development in general, there should be a configuration-free in-memory default for this layer.

At this point in time I’m imagining a uni-directional flow much like React and Halogen.
I’d like to try out this concept where session state is analogous to the page state in those libraries.
Part of that flow would be serialization/deserialization to the backend technology of your choice.

Deployment and failure considerations aside, functional programming in general has pushed me towards this desire for my applications.
It seems to me that our application servers should just be stateless http-to-backend-whatever adapters whether it be a database or some microservice.
Those backend things can fail too, of course.
However, I certainly expect stuff I write to be far more likely to fail than a database backend which has received a lot of work to be good at that.

Assuming I strike out on my own, I’ll be hard-pressed to achieve Lift’s straight-forwardness and security in AJAX and server pushes while maintaining the stateless server story.
My guess is that re-establishing server pushes whether by comet or web sockets will be a tricky to do securely.
I love the challenge this poses, and Lift will serve as my model for the developer experience that I hope to achieve.