Big step forward in Modular Database Applications with DataObjects

Published 2010-09-01 00:00:00

Being a Software Developer is all about developing applications faster, and delivering quicker. At the same time ensuring that quality is not lost and readibility is kept. DB_DataObjects is one of the key tools in my productivity toolkit. It was originally designed as a way of ensuring that shared database related code ends up in the right place, In the Model layer of the application, rather than the View or Controller (or worse mixed in to some hybrid PHP HTML garbage..), along with doing cool stuff like Query building etc.

Over the years, of using DataObjects, I've built up quite a library of reusable DataObjects, which to some degree can be plugged in to any project. A Person object that handles login/authentication and authorization (working with the Permissions and Group objects). A Image object that handles storage of Images, and Files that can provide File type conversion for rendering (using things like unoconv, etc.)

More recently, I've been using the Roo Javascript library as the UI component, the Pman Components ontop of the very lightweight HTML_FlexyFramework. The result is a very modular set of Application development parts. That can quickly be thrown together to build applications. Here's how they all fit together and how it just got a whole lot more modular and flexible...

HTML_FlexyFramework Provides the basic 'Controller', It's one of those very lightweight, get out of your way, 2 small classes bit of code. In some sense's it's not really a Framework (well 2 classes could hardly classify as that). More a generic Controller, and a Base class for a Page, just implementing the bare bones stub features, like get/post,getAuth() and implementing the output method to run HTML_Template_Flexy.

What I've built ontop of that is the Pman Components, originaly based on a project codenamed Pman (for project management). Which ended up being the most re-usable code I've developed to date. It's the foundation for around 6 completely different projects now, sharing a large amount of modular code. The basic code of the Pman Project is Pman.Base and Pman.Core, in themselves they do not provide much more than the basic Interface outline.

The directory structure for projects

/index.php (the bootstrapper and installed configuration- the link should show a rather messy example)

There's quite a bit more code and data in those Modules, but what changed yesterday and has a significant benefit to the long term re-usabilty of all this code is the autogeneration of cached data for DB_DataObjects schema files, automatic merging of DB_DataObject links files, and the generation of code for Roo Javascript's Reader data.

A Changing Data Model

The background here is that, as I re-used more and more of these componets, there would frequently be an issue where a particular project needed the modification of a core database component. In our example it was the adding of 'dispatch port' to the Company Table. This was a specific requirement for one project, which could have been impemented using a Companys_Extra table or some kind of 'properties' hack (which is what Midgard used to do back in early 2005). However the best solution is really to modify the base Table, so that it can be searched/ordered etc, and in the long run I'm looking at having a 'Designer Module' similar to SugarCRM / netsuite, that allows the end users to actually modify the database structure, adding colums as they need.

The Knock-on effect however for this small change is, unfortunatly it often breaks other code depending on the compoents, due to the process that is involved.

a) Modify the database changes file. normally in the DataObjects directory of the Component

b) Run it against the Database, updating the Data Definition for tables in that module.

b) Run the generator on the module, so that the schema.ini file is regenerated.

This works fine on the development server, and usually fine on the Live server for that project, as you remember to run the database changes from the relivant Module when you do the next install, Where this falls apart is usually a month later...

You go back to working on a project that was on hold for a while, and start by updating the modules in the project. Suddenly Authentication fails, or no data is available in the interface... - All this is due to the fact that the Database Queries being built expect certain columns to be there (based on the schema.ini file) and since you have forgotten to update the database all the queries break.. Not usually a big issue as the builder module has some code in it to apply all the database changes code, by runing the bootstrapper in CLI mode. - But still it's annoying.

The other issue is that you really do not need the Company Table to contain a 'dispatch port' when no other project is going to use it.

The Solution..

I have been thinking about the solution to this for some time, however, just before I jumped into refreshing some code in a project, I thought this would be the ideal time to solve this problem, and speed up my development process in the long run.

- when you load the main page (not when you do JSON calls) it calls generateDataObjectCache(), which will generate the schema files if they have expired. - if you are running with Pman[isDev] then it will run them every time (eg. force generation)..

which adds to the data return value the JsonReader meta arguments which include schema type information, so you do not need the schema information in the UI (reducing the size of the UI considerably). note schema data is only returned if rows of data where also returned..