http://ahmedsoliman.com/Ghost 0.5Sun, 02 Aug 2015 22:23:53 GMT60With the rise of the NoSQL hype, developers rushed to the schema-free databases ditching years of SQL in search for a true scalability promised by the vendors for the new web-scale era.

Schema-enforced database model

CREATE TABLE users(userid VARCHAR, planet VARCHAR, age INTEGER);

Schema-free database model

// Schema-free
{ /* just put any stuff here */ }

It's kinda frustrating when I see developers advertise schema-less databases as a feature to be celebrated and that it's the future of a web-scale applications!

After a few years of using document-based schema-less databases I came to the conclusion that the schema-less is just a myth. IMHO, it promotes lazier development and less thoughtful designs of software applications, resulting in lots of crappy code that some possibly innocent developer will have to decipher and maintain.

It reminds me with the dynamically-typed language movement that many web developers embraced including me for a few years and still many developers are still fond of dynamically-typed languages because they don't have to deal with compilers, type-checking, and etc.

So, Why do I think that schema-free is a myth?

When you store data in a database, you know what you are storing, when you fetch it back, you expect certain things to be there defined in certain types. Even if you are using a dynamically-typed language you still cannot add an integer with a string and you expect the result to be an integer. That's why you probably will be doing certain type-checks after you retrieve the data or at least you expect that an exception can be thrown trying to do those kinds of operations in runtime, which is even scarier!

So, in fact you are basically moving your compile-time type-enforcement to your runtime. This pattern gives you flexibility that you can have multiple types of documents in the same collection in your database, however, you still need to identify those different documents so you might want to add some kind of identifier there, for example.

{
"_objType": "User"
"userid": "AhmedSoliman",
"planet": "Earth"
}

The field _objType was added by you to ensure that you can classify the documents upon retrieval. Another thing is evolutions, an advertised feature by database vendors is that you don't have to make offline data migrations anymore if you decided to tinkle with your now-imaginary data model. Now what you need to do is to check on every document upon manipulation or retrieval to see if it's the latest updated version or not and based on that you will update the document to the latest data model. So, imagine that we want to change the key userid to username. We will be doing probably something like this on all of our CRUD operations

// check if the document is old-style and migrate on-demand
if (doc.hasKey("userid")) {
doc.set("username", doc.get("userid"));
doc.delete("userid");
doc.save();
}
//let's do the normal work here

The validation you've added is fragile and expensive, if you later decided to bring back the userid you will be in-trouble, plus you are actually repeating your migration-validation-thing almost everywhere, it's easy to forget! And expensive because you will have to do this validation on every document even if it's actually migrated already!

Model Versioning, does it help?

One common solution to manage changing your imaginary data schema is to put a version number along every document. Once you upgrade your imaginary schema you add one to the model version so every new document will get that and for old documents you again run a version-check code on almost every CRUD operation.

Where upgradeDocument takes the current document version and the latest model version and does its magic. What about querying your data? what happens if you query documents that still holds userid instead of your query key username? Oops, no match!

You will then have to think about running background job to upgrade all your old documents to ensure that your query indexes are updated for your queries.

See? there is no silver-bullet, it's just your problem now to handle all of that, this makes building the NoSQL databases relatively simpler but makes your enterprise-ready-code much harder to get right.

How did we arrive here in the first place?

We arrived here because SQL databases didn't perform very well in a web-scale era, poor clustering options, no out-of-the-box data partitioning support, and most commonly what's known as the CAP theorem.
The answer was to give up one of the C.A.P letters and along the way many database vendors dropped the schema-support too! We didn't have to do that, did we? no.

Solving the database scalability problems is absolutely irrelevant here, as long as your schema and data validation doesn't depend on strong consistency of your entire dataset. So, it's technically possible to have a data backend that it's truly horizontally scalable and schema-enforced at the same time.

Conclusion

I really like the idea of document-oriented databases, it offers a unique data-model which most of the time is close to your native JSON data representation. It's particularly awesome when it fits the rest of your data structures, but this doesn't mean that we are giving away the need of having a schema-level checking on the data, right?

Basically, database vendors are implicitly saying that validation and data migration is your problem now, you deal with it. I'm not saying to stop using schema-free databases but your decision be thoughtful of the consequences.

Most developers I saw, think in types and schemas, this is basically how their brain work, you need types and schemas even if they are implicit but then you need to be sharper and more focused, and definitely you need a better battery of tests and much more luck!

Side Note

If you think that schema-enforcement is awesome, it might be a good idea and you know your way through Scala, check out really.io a realtime open source schema-enforced backend-as-a-service solution to help developers build near-real-time applications for the Web and Mobile with a truly scalable modern backend under their palm. If you are interested about the development of this, check out our github repo

]]>http://ahmedsoliman.com/2014/12/10/database-schema-enforcement-for-the-great-good/64a1fe6d-d935-4e53-84d4-9e1eb4904f49Wed, 10 Dec 2014 12:19:00 GMT
Back in the nineties when Yahoo was the search giant and all websites tried to bake all the possible features in the same place we felt that something is wrong and that more focused websites are better, we learned a lot during those years and people started building applications on the internet that focus on offering its core value through a single focused idea.

Take Yahoo as an example, I usually call the nineties by the Yahoo decade, the single website that you can check your email, news, weather, stocks, and almost anything. Yahoo was trying to keep its users by adding more scopes to their websites, it was like their ultimate goal is to replace the entire internet with Yahoo.

It's quite visible to everybody that this plan didn't play very well, it was also very hard to compete with companies that were entirely focused on building a great news services for example. Nowadays, reading news is even a different app than publishing news, service fragmentation across apps is the norm and everybody is trying to build focus apps (Dropbox, timely, etc.)

While on the other face of the universe social networks – in the year 2014 – are trying again to replace the internet for you, it's promoting for a totalitarian internet for me, you, and everybody!

Late in the first decade of the Y2K, we learned that APIs is a great idea, you build micro services, single-focus websites, and apps and offer APIs for developers to integrate those services into more sophisticated services and applications. Although we didn't have the awesomeness of having standard APIs for services, so we couldn't replace services offering the same core value with others. For example, take Dropbox as a storage service, if it was offering an agreed-upon API that is considered a sort of standard, then you would have been able to replace dropbox with Amazon S3 or Copy seamlessly. The user eventually may decide which service he wants to put his files to and you as a service provider integrate with whatever implements the API, you basically give the power of choice back to the user. Modern applications should be designed in layers like an onion but with the ability to replace the inner layers if you want.

I always thought about this thing specially for photo storage, I don't want to have several copies of my photos in different services, I want my photos to be stored in one service that I trust – say Dropbox – then I want to give Facebook, Google+, and etc. access to those photos and if I want to change the storage service, I can. If I want to revoke access to Facebook, I can.

I loved YouTube back when it was outside Google+, it was built as a great free video sharing and hosting service, you can link your YouTube videos anywhere (it's a kinda of API, the embedded player I mean). Now it's still great but I feel that it's baked more into Google+. Same for Facebook, it tries to replace the internet for me, I find lots of videos that exist in Facebook and YouTube as duplicates. That's really bad.

I understand that commercialism is about maximising profit and companies think that locking users to their service is the right way to go, but I still believe that giving users the choice shall be a greater benefit for everybody.

I would love to see the internet as a Lego, where I chose to use different pieces with standard programming interface and eventually everything fit together. I want freedom through choice, not just open source software but also open and standard application programming interfaces, as of today, this is going to the opposite direction as far as I see it.

]]>http://ahmedsoliman.com/2014/05/12/social-networks-promote-to-totalitarian-internet-again/bd422d9d-ef9e-48cc-97c7-af7e4ebe4e3aMon, 12 May 2014 15:24:24 GMTThe most anticipated version of Java (Java 8) was released just a few days ago (18 March 2014) with a set of exciting features that is considered to be the largest to Java since the release of Java 5.

I'm not planning to go into the details of the new features of Java 8 (probably in another post), I want to discuss the question that many people are asking. Is Java 8 going to kill Scala?

Java Folks, Welcome to Planet Lambda!

So, let's start with the new baby coming to the Java world, Lambda Expressions. Lambda functions/expressions are already in a large number of programming languages already and the Java folks are exploring what they can do with that little handy thing.

Lambdas are basically functions that can be passed as function argument, you can think of it as a short hand version of what you was doing as this example

You couldn't send functions as function arguments so you created an anonymous class of the interface ClickListener which contain a single method onClick that you implement, man, that's a lot of boilerplate.

It's the thing which you can do with the new Java 8 like this

someObject.addClickListener(e -> //implementation);

Whoa! That's awesome, right? Exactly, that's why programmer using languages that support functions as first class citizens (like Scala) were screaming for similar support in Java, specially for Android development as an example.

Lambda expressions come in different names, don't get confused with naming coming from lambda calculus or functional programming, it's a world full of new names in general. The concept is really simple

(int x) -> x * x

I will not go into too much details here but I would like to note that lambda functions in Java is not as powerful as in Scala, in fact it's just scratching the surface, for example, you cannot pass any function as first-class to other functions, the function must be explicitly defined as lambda or using Functional Interfaces which is an excellent idea for older java libraries that you can leverage the new syntax to use.

Functional Interfaces

Functional interfaces are interfaces that contain only a single method, like the one I hypothetically used in my previous example ClickListener Similarly you can use the Runnable in the same way since it only contain one method run(). You can pass lambda functions wherever you used to pass anonymous implementations of Functional Interfaces, example:

Runnable r = () -> System.out.println("I am Runnable!");

The right hand side of this line is a lambda function definition that takes (Unit/void) as input and returns Unit/void and magically you are assigning this to Runnable interface! Got it?

Now you can write this

new Thread(
() -> System.out.println("I am Runnable!")
).start();

Now that's really cool, you see how concise your code has become?

Capturing Variables

Just like anonymous classes you cannot capture variables inside a closure/lambda unless it's effectively final which means that it's either local variable that nobody will ever re-assign to, or an explicitly marked final variable.

Scala-like collections

You now can write functional style operations on some of Java collections using the new java.util.stream package. See this:

Still Scala is more concise and more accurate about the types, the Java version returns Optional<Integer> while Scala version explicitly return Int. By the way Optional<Integer> is similar to Scala's Option[Int]

Edit: It turned out as a few people noted in the comments that the Java case here is actually more type-safe and accurate, since Scala will throw runtime exception on Nil input cases. Thanks to Ricky Clarkson.

Edit 2: You can also achieve the same result as java in scala by calling reduceOption instead of reduce which returns Option[Int]

Why people moved/moving to Scala - What's the point?

Scala is not only about lambda functions, although admittedly it's one of its best selling points in comparison to Java. However, Scala offers a comprehensive functional programming syntax tools that makes writing software (specially reactive, concurrent, real-time) a pleasurable experience with great productivity and excellent running performance.

I still see a huge gap between Java and Scala and I see it clearly that Java is not turning into a functional language, it's still Java and people love Java for what it is already, the Enterpris-y stuff and solid backward compatibility. Scala on the other hand is stretching what can be a future programming language that target multi-core software development. It's multi-paradigm but is mainly winning for its unique combination of functional and object oriented features with the world's most advanced type system that no language can compete with so far.

Why Java 8 is actually good for Scala?

I'm genuinely excited about the Java 8 release because I see that java now suck less and at least I can spend my time writing Java code with less pain/cursing. I also truly believe that the move to more functional style is intellectually beneficial to Scala because it closes the gap between the two languages specially with the similar syntactic hacks. So, it's going to be easier to introduce Scala to Java developers and you don't have to explain closures, Unit, lambdas, etc. to them.

It's also an opportunity for Scala compiler developers to learn how Java actually implement the functional programming constructs to match the JVM and how the JVM will be adapted (like the recently added invokedynamic in Java 7) to support the move. For example the generated bytecode from the Java lambda functions is more efficient than Scala's version.

I think that it's a validation that the switch to more functional style offers you more productivity and with type safety you get higher quality code eventually.

This also can add cleaner support between the two languages, for example in Play Framework you can now see that the Java API of the framework allows you to write more idiomatic code like this

Conclusion

Java 8 is a great step towards developer productivity and is something everybody is excited about even Scala developers (like me). I see that the goal is not to kill Scala and in fact I see no risk at all. I see two languages that can play even nicer together now and Typesafe (the company pushing Scala) is actually excited about Java 8 as some of us are.

]]>http://ahmedsoliman.com/2014/03/26/will-java-8-kill-scala/da03f962-d4dd-4c3d-ac61-2979c2f06a42Wed, 26 Mar 2014 22:06:57 GMTKnown for their extensive range of pragmatic IT ebooks, Packt Publishing are celebrating their 2000th book title Learning Dart – they want their customers to celebrate too.

To mark this milestone Packt Publishing will launch a ‘Buy One Get One Free’ offer across all eBooks on March 18th – for a limited period only.

Since 2004, Packt Publishing has been providing practical IT-related information that enables everyone to learn and develop their IT knowledge, from novice to expert.

Packt is one of the most prolific and fast-growing tech book publishers in the world. Originally focused on open source software, Packt contributes back into the community paying a royalty on relevant books directly to open source projects. These projects have received over $400,000 as part of Packt’s Open Source Royalty Scheme to date.

My book Getting Started With Memcached is available in this campaign, Checkout the campaign page

]]>http://ahmedsoliman.com/2014/03/26/buy-my-book-from-packt-in-this-amazing-buy-one-get-one-free-offer/d96b10ea-efac-4127-b2b1-d4af20ae40e1Wed, 26 Mar 2014 08:33:31 GMTIt's been quite sometime since I posted on this blog, but don't get sad, this announcement is kinda special.

Table Of Contents

The book delivers a set of hands-on recipes for the impatient about integrating memcached into your product stack

Getting Started with Memcached

Basic installation of memcached on Ubuntu (Simple)

Basic installation of memcached on Mac (Simple)

Compiling memcached from a source on Ubuntu (Simple)

Talking with memcached (Advanced)

Setting up memcached to start on boot in Ubuntu (Simple)

Setting up distributed memcached (Intermediate)

Using memcached with PHP (Intermediate)

Using memcached with Python (Intermediate)

Using memcached with Ruby (Intermediate)

Using memcached with Java (Intermediate)

Setting up memcached support in Rails (Simple)

Setting up memcached support in Django (Intermediate)

Setting up memcached to support in Play (Intermediate)

I'm super excited about that mini book which you can grab a copy for you and your friends :) from PacktPub

]]>http://ahmedsoliman.com/2013/12/14/my-book-is-out/4176aa34-1d1e-436c-8c18-b73ecb43b6e7Sat, 14 Dec 2013 10:00:00 GMTA few days ago I downloaded this brand new mobile app from Google Play! and when I started that freshly installed app, it asked me to sign up. I didn't think twice, I just switched to another app and never came back!.

Obviously, Signups/ins in mobile apps are much less convenient than in web apps, people normally hate to feed their emails and password on this touchy keyboard, it's just awkward!

Then I thought. "Why do we need signups/ins in mobile apps?!"

A very simple question then came up, 99.99999% of mobile users (especially smart phones) are single users, this means that it's only one person who is using the phone all the time, it's very weird to find a phone that's used by two guys!

Why can't I enter my information once, and all my applications simply access those info to know who I am? Think of single-sign-on for mobiles, I unlock my phone once and all apps magically get access to my name, email, photo. No authentication is needed per application, no password is shared between applications, the mobile phone authenticates on my behalf.

Wouldn't that be so cool?

]]>http://ahmedsoliman.com/2012/07/10/about-mobile-application-sign-ups-and-sign-ins/3b97f654-5b34-4148-a48e-c16798fcd252Tue, 10 Jul 2012 09:00:00 GMTNormally, when you get motivated about contributing to an open-source project, or even get the courage to roll your own, the reasons traditionally announced are that open-source is fun and contributing back to the community is part of what the good guys should do.

While this is true most of the times, it's not the sole reason to roll your own open-source project, there is a very important shift happened in the past few years regarding how employers seek talented engineers, around 15 years ago, almost everything was about the "Certification", you get your "MCP, or RHCE" and you have secured a job, probably after passing a few silly interviews. Today's world is totally different. I -as many others- see certification as a total crap! Technology certificates doesn't mean anything to me anymore, except in rare cases. It's like that showing your driving license doesn't prove that you are a Schumacher. Show me a test-drive and I'll judge if you are or not.

In today's world, this is the code you wrote, you are what you code. Writing good code makes you a good software engineer, how can I see your code unless you are hired?, you got it, Open Source stuff!

I can see your open source code, test cases, documentation, thinking style, even the people who influence you, everything! I can see if your code attracted attention or not, if you know software design or you are just piling your code off.

You know what I want you to do right now, don't you? Go fork an open-source project or let me see an idea of yours (even if extremely simple) on github.com or bitbucket.org

]]>http://ahmedsoliman.com/2012/05/15/you-are-what-you-code/37b3e999-a1ac-4cb4-b738-7dd358de86f6Tue, 15 May 2012 09:00:00 GMTLet's assume that you are getting a list of book titles out of a database and you want to print them in a nice <ul>. Every book's title and number of reviews it has received. You will be doing something roughly similar to the following:

Let's see what happens when you run this code and you get a list of books out of your database that contain some arabic book titles. You get a generated HTML that looks like the following, note that it's generated "correctly" even if you see the arabic title is swapped with the number in a funky way.

Try to copy that to a text file, name it to "t1.html" and open in your browser. Funky?, This is exactly what is going to happen in production and that is/was the typical pain of a developer trying to mix Arabic words with English words in specific/most cases.

What actually happened is that the Bidi (Bidirectional) text-direction detection algorithm decided that the ("- 2") part written after the arabic book title are in the same direction, this means that it's written from right to left, but you actually mean the opposite, you mean that the book title is the only (right-to-left) element. So you basically need a way to isolate the arabic title in an isolated bidi context.

Thanks to the new HTML tag which isolates the bidi context for certain words. Try surrounding the book title with that...ex:

Neat!. Try that generated HTML in your browser and see the effect for yourself. I've been somehow involved in the incorporation of this tag into Firefox, but happily, it's also supported now on Google Chrome too.

]]>http://ahmedsoliman.com/2012/04/09/solving-your-arabic-problems-using-the-new-bdi-element/4134c674-9937-41ca-897b-2bde5b1cb780Mon, 09 Apr 2012 09:00:00 GMTI've been working on a very simple project that allow processes or computers to communicate seamlessly using Redis as a backend. I love Redis because it's very powerful, yet very simple to configure and use. Let's first understand the motivation behind redique

The motivation is the need for a small, cool, json-based, super-fast RPC system between processes, that led me to build a tiny project on top of Redis database and I called it Redique

Redique is an implementation of a high-performance Async RPC/Task Queue system built on top of Redis datastructure store and JSON marshalling protocol.

You normally want to use redique when you need to publish tasks to a set of workers to process asynchronously and retrieve the result using a task_id, or when you want to distribute workload over multiple workers easily without going through the hassle of understanding how message buses work.

Getting Started

You need to install the package first using pip:

pip install redique

Then you need to create a backend class that contains the actual logic you want implement over the transport

]]>http://ahmedsoliman.com/2011/10/04/introducing-redique/93c06cd9-1bf5-4ee9-a0b5-a0e215f662b7Tue, 04 Oct 2011 09:00:00 GMTIf you are a Python hacker, most likely you will be using a virtualenv for your development environment, you are probably an iPython user. Unluckily, IPython doesn't pick the virtualenv paths by default and you will have to tweak it a little bit to make it work. Basically, this a slightly tweaked version of this article but tailored for the recent version of IPython 0.11.

You simply need to drop this snippet of code into the iPython configuration directory as illustrated below so that IPython can add the virtualenv paths.

"An extraordinary experience with the top-notch geeks in the city and nearby cities" - That was my definition to this event "Startup Weekend" after participating a few months ago in SWCairo, I was pleased to launch Fakkarny and very excited to meet the top-notch geeks. That experience changed a lot in the way I think about startups, teamwork, rapid development, pitching ideas in front of investors, and managing geeks :)

This weekend the same extraordinary event is taking place in Alexandria, and guess what, I'm a participant and/or mentor in that event. I've been helping out 4 teams as a mentor and/or technical guy and I'm very proud of this experience, I feel it's a life-changing experience to be able to push and support multiple teams and to witness the progress of every minute. Let me tell you more about those teams.

Khasesny.com: خسسني is a collaboratively-generated idea between me and Mahmoud Metwally. An idea about building a social network for those who need to lose weight, physicians, gyms, and etc. The idea to build a network of supporters and helpers around you if you are into that mission-impossible plan, it let's you get the support from your friends emotionally and the support from physicians technically in the same place. With a plethora of tools to create goals, track them, and to make sure you are eating healthy food.

Weladlbalad.com: ولاد البلد is an awesome idea of building a social network that brings locals and travelers into a place where they share questions, new places for outing or sightseeing. Just as any social network it is driven by the power of people, locals are the best to answer your questions about how to drive from a place to another, or what's the best meal you could have under 5$ in a city. It's an awesome tool for someone like me who travel a lot between cities and I'm sure you will love it!

1001 Inventions: An iPhone/Android game about the global educational initiative that promotes awareness of a thousand years of scientific and cultural achievements from Muslim civilization from the 7th century onwards, and how those contributions helped build the foundations of our modern world. A glimpse on the initiative is

Raga3holy: رجعهولي A team from Mansoura decided to write his first Android application to detect stolen smartphones, the phone reports his location, call log , and recent locations to the phone owner via SMS and/or 3g/wireless connection. Smart algorithms were designed to ensure that 90% of the times the device location can be detected and transferred to the web service.

Conclusion

The effect of every participation in this event will blow your mind away, super excitement is everywhere, energy is countless and the flow of ideas between team will make you happy being around such positive energy. I love Startup Weekends!

]]>http://ahmedsoliman.com/2011/09/24/my-startup-weekend-alex-2011-experience/a88fd7a8-92f1-4337-aae0-d9faf95e65f1Sat, 24 Sep 2011 09:00:00 GMTThis is a cool sample on how to write an automatic self-restarting python application after a timeout, of course you can modify this application to be much more sophisticated so it can close all open files before restarting or restarting based on a signal (SIGHUP maybe?) or whatever you like

]]>http://ahmedsoliman.com/2011/09/21/auto-restarting-python-application/f6008e76-71f7-4aef-a456-04081c2b8f55Wed, 21 Sep 2011 09:00:00 GMTAfter building VirtualBox with VNC support in my previous post I discovered two problems with that installation (VirtualBox 4.1.2 on Mac OS X Lion 10.7):

SDK was not installed by default, import vboxapi from python failed.

VBox Python bindings were built for python 2.6 and segfaults on python 2.7

I contacted some guys on #vbox-dev on IRC and it what happened is that VirtualBox 4.1.2 was not built for Lion and all build scripts are configured for 10.6. So, I decided to hack my way around for the second time to fix the two issues.

To cut the very long story short, you need to create your own port local repository like how I described in that post and remove the virtualbox directory inside emulators/ and replace it with the patched port (Edit: You don't need this file anymore since the patch is already applied to macports now)

Update: The ticket was fixed and you do not need to apply any patches anymore. This post is here for historical reasons only!

]]>http://ahmedsoliman.com/2011/09/19/virtualbox-on-mac-os-x-lion-with-python/22bc5ff5-c960-4952-b68d-7b28024f1431Mon, 19 Sep 2011 09:00:00 GMTI've been using VirtualBox for months for several reasons and recently I've been forced to use only the free version of vbox (without the Extensions Pack), which doesn't include the sweet bits of VRDP server among other things. I wanted to boot my virtual machines headless and access it using VNC which doesn't come by default in Oracle's builds of VirtualBox, so I decided to roll my own build.

First steps

I searched for a VirtualBox port and I was surprised to find one, but I wasn't quite happy with it, because it doesn't build VirtualBox with VNC support, so I had to hack my way around. I wanted to implement that cleanly by adding a new port variant +vnc to that port.

The following steps will help you do the same; I've tested that on Mac OS X Lion 10.7 64bit but should work exactly the same on Snow Leopard 10.6 too.

]]>http://ahmedsoliman.com/2011/09/18/building-virtualbox-on-mac-os-x-with-vnc-support/be9252cf-1c76-4bbe-85fc-6f986252fa4eSun, 18 Sep 2011 09:00:00 GMTI've been working on a big project for more than a year now and during that year I've played with many web development frameworks with different approaches to tackle modern developer problems. Honestly, I've been impressed with some of the frameworks on how simple it's designed and the elegance in their usability. I even wrote somearticles about that. But that didn't last long!

In any web development project, things start out cleanly organized and well-designed, and by adding more features, functionality, code, templates, and javascript, the application becomes very painful to maintain. Applying modularity concepts become harder, following good design rules become impossible, every piece of the web page is conditional. It's simply a mess!

I'm still trying to figure out the best way to split a project into smaller pieces to maintain its tidiness, cleanness, and maintainability. Out of my experience, I can list some of the issues I've seen inherent in all of the frameworks I've used so far.

Template language is not HTML/XML, so it basically breaks most of HTML/CSS/JS code highlighters, scrambles your clean HTML with lots of includes, if conditions, loops, etc. Think of editing the design and forgetting to change the error state, branched code that you missed to see, Sigh.

Common bases are not really common! You start out by writing a base template for your whole application UI, which potentially contains header, footer, main container, and etc. and after a while you discover that you need to customize that base based on some conditions, you need to parametrize its widgets to look differently in different pages. Then you discover that you actually need more than one base, so you create a base for those bases :) You are actually doing OO inheritance but in HTML templates, OO inheritance suck as in HTML inheritance.

Where/when to load a specific Javascript library? In the base? Can we load that on demand? Is there scripts dependency? Is there a case where javascript libraries are loaded twice? Same for CSS. Trust me, this happens all the time.

Maintaining versions of javascript libraries inside your templates is also a mess, with the best code organisation schemes you are still in charge of managing versions, ensuring conflicts doesn't happen, making sure you are loading the correct version every time you need to load a library.

No debugging in template code, most of the frameworks doesn't allow you to debug template's code, you are on your own. Use traditional techniques like printing, logging, generating non-sense output to understand what's actually going wrong in the template. This wastes tons of time.

95% of the time, the template language is just a new language, you need to learn it, understand its tricks, you are limited by its capabilities/stupidity, and guess what? Every framework is inventing its own language and conventions!

Conclusion

As it appears, most of the framework developers were focusing on giving you cool backend features, MVC, and neglecting those issues affecting the UI. I deeply think that a revolutionary solution to those inherent problems must rise soon.

That's why some frameworks started looking at those problems and introduced some interesting solutions. One of those frameworks is Lift.