Using object trees in data stores

Hi All,

I have been using Ext JS on-and-off for quite a while now (the on and off depends on whether I am doing front-end work at the start-up I work at or not). Occasionally I face a situation where I want to make use of actual objects in my data stores instead of just flat objects (or maps). I can't imagine this is an isolated case as I assume other people are doing exactly what I am - trying to get data from my server side model into my Ext JS application which lives in the view of my application. The most common case for applications such as these is to store the data for the application's model in a Relation DB which have one-to-many and many-to-many relationships as we all know.

Server side MVC frameworks provide an object orientated solution to this through the use of Object Relation Mapping. This means we can navigate an object tree that directly represents the underlying data present in the RDB. I am beginning to wonder why Ext JS doesn't do something similar. Ext JS is really pushing the boundaries of what we consider part of the traditional view to the point where it can almost be considered as a MVC framework in itself. As such, I think improvement needs to be made in the movement of data between client and server.

What I would like to see in future versions of Ext JS (and I mention this because I see a major overhaul of Ext.data is in the 4.0 roadmap) is the ability to pull entire segments of our model object tree from the server into the client. That means doing away with the current design of Records with a flat list of fields belonging to a Store. I realise this is a major change from how things are done presently but I think it could have a real big impact on how we design our Ext JS based view components of our web applications going forward.

Let me give you a quick example. Let's say I have the following simple model - a Dog which can have many Toys. Lets say I want to present a Grid to my users that displays a list of Dog records. Currently I would probably create a GridPanel with a backing store to parse some JSON/XML from the server. If I wanted to show a list of toys in that grid (maybe truncated with a QuikTip to show the full list) I would need to pre-process my data server side to return some custom JSON/XML to return to the store. If I then wanted to edit that Dog row in the grid, including its toys, I would then need to pop a window (or something similar) which presented the user with an edit form while also hitting the server again to get the data for my dog record (the list of toys in particular because all the grid has is a flat string representation of toys - I needed to return a flat list of properties not a rich object which would maintain the relationships of my model). Take simple relationships like this and repeat them through-out your model and you have to repeat this process over and over.

If we could import entire portions of the of the model object tree directly into the client side data store, with the example above, we would only have to hit the server once. Not only this, but we might be able to create a generic layer in our server side ORM code that will be able to fulfil requests for data by our view without us needing to code every instance of it (no more generating 1 dimensional JSON/XML representations of our models over and over). With all the advances with Ext JS Grids and UI controls, you might even be able to imagine a scenario where I edit the Dog (including toys) directly in the grid itself. You could even imagine a full version Ext JS client application (IE a border layout full window app) that transparently fetches data from the server when needed in the background using the built-in server side ORM - all that is needed is a definition of the data much like it is server side.

There are some obvious criticisms I can think of at the top of my head:

The view should not be directly accessing data from the model - I think this is a little old school. The traditional server-side MVC was conceived when our views were basic and AJAX was not very wide spread at all. Librarys like Ext JS have redefined what a view is and what it can do. We need to move forward with it and I think version 4.0 sounds like a great place.

Giving the client side direct access to an ORM style data fetching API is a security flaw - I agree - giving blind access is definitely not a good idea. We need to find a happy medium. I really believe that the web is moving towards a RESTful model where all the server does is provide us with the data and operations or processing of that data. Rich clients like what is possible with Ext JS take care of the rest. Even if I am incorrect, REST gives a great example of how to implement a client accessible data API which very much depends on user permissions.

I might be right, maybe some applications are moving this way, but not all are. Most make use of Ext JS to enrich their applications but don't base write their entire client using it - Also true. Ext JS should still be able to support the old way (or maybe an updated version of the old way), but even applications like these would be able to take advantage of the new way.

A version of Ext JS that really embraced data like a server side MVC framework does would be particularly useful for someone like me who is working on an application that uses a full window Ext JS border layout and communicates with it's server via REST. As things currently stand, we have to almost completely re-write the current Ext.data solution and plug in our own.

Did you already see this blog post on some of the Model goodness available in Sencha Touch and (hopefully) coming to Ext4?

Some of what you describe is even possible today, I think. Records in a Store created with JsonReader have a property called json, which contains the original JSON source object sent by the server. So your server can return JSON objects with nested objects or arrays in them and the record.json propery will give you access to it (for example in custom renderers). So you are not really limited to flat records when using a store.

Thanks for your reply Luc and thanks for pointing me to that blog post. I have been keeping up with the sencha touch news but not been following it closely as I haven't quite made it to the mobile space yet.

The blog post looks like it is a very promising start to a solution of the problem I was talking about. It is a little releaving too see this blog post as it means I'm not the only person thinking along this line. I still see a couple of areas that I think could be improved:

The configuration of our client side model should have an option to be automatically configured by the server. Anyone that uses a modern server side MVC framework will already have all the information relating to their model stored on there server. In my case, it is in the form of objects as I use Django mostly. A one off call to the server early on would mean we can automatically configure our client side model using a pre-defined JSON or XML response format.

The implementation documented in the blog post, as far as I can tell, still uses old style Stores at its heart which means flat objects like we currently have. I would really just prefer access to the model object I can navigate instead of a it being wrapped in a store. This issue is easy to solve on server side MVC frameworks because they aren't asynchronous, but will require a little more thought to implement properly in Ext JS.

If these things were implemented, I think this new data model would probably address all issues from my first post. It would definitely simplify writing Ext JS code for myself greatly.