Archive for February, 2009

Like every similar framework, language, or environment, Zeetix needs LOTS of names. Lots of them are names for new concepts, and lots of them are names for things that are like other things but different in significant ways.

Zeetix draws on Terry Pratchett’s “Discworld” series in the same way that Python draws in Monty Python and Ruby draws on jewels and jewelry. If you haven’t read any Terry Pratchett, if you like British humor (especially if you like “Hitchhiker’s Guide”), fantasy (sort of), and satire, then you might like Terry Pratchett. Discworld is to fantasy what Hitchhiker’s Guide is to science fiction.

In addition to the novels themselves (as far as I know, all are in print and readily available), a multitude of online resources exist to inform your exploration of Discworld. I particularly like lspace.

Some immediate names that I’ll be using in Zeetix are “Glooper”, “Clacks”, and probably “Hex”. I encourage you to see “Glooper“, “Clacks“, and probably “Hex” for context.

Glooper

I (Tom) did some early Zeetix development in Smalltalk (specifically, IBM Smalltalk). One early motivator was my desire to have a tool with which to build families of websites, where each website was represented as a directed graph of objects, instead of as a file hierarchy. I called this tool, for want of a better name, “SiteBuilder”. This was before RubyOnRails existed, and was mostly because I found myself frustrated with the limitations and learning curve of contemporary versions of Dreamweaver, Zope, Django, and so on.

I’m still using the Smalltalk “SiteBuilder” (on my own development PC), and I am acutely aware of the need to import it into the Zeetix framework. When I do so, it will be called “Glooper”.

I’ll write more about Glooper as I get time.

Clacks

Zeetix will need to model and wrap the semantics of the internet and web. Zeetix will use “Clacks” (and variants of this theme) to name that stuff.

Hex

Underneath Zeetix is a universal virtual system. Really universal. Zeetix will use variants of “Hex” and to name related resources and assets.

I know that everybody in the world but me already knew this, but I finally sorted out what appears to be a workable cross-browser way to do tooltips.

As it turns out, the major browsers display the contents of the “title” attribute as the tool-tip that appears on, for example, anchor tags. It turns out that IE also, and in apparent violation of the standard, uses the value of the “alt” attribute if the title attribute is missing.

For now, until I am convinced to the contrary, I’ll use the “alt” attribute of each img tag to describe the image for text-only browsers. I’ll use the “title” attribute to contain the tooltip contents (limiting it to 75 characters in deference to Firefox behavior).

A common and powerful idiom is to supply a two-argument block that, when evaluated with two elements from the collection, returns a value indicating the desired (sorted) order of the two elements. This “sortBlock” is applied to each of the elements in the collection (in some order), and the result is returned as the now-sorted collection.

So far, so good. The problem is in the semantics of how the sort block is applied, and in which order the elements of a given collection are passed to the sort block.

Problem

Smalltalk and Python use different semantics for the sort block, and apply the block in different sequences for the same initial collection.

In Smalltalk, a sortBlock is called with two elements (I’ll call them “arg1” and “arg2”), and is expected to return “true” if arg1 is to be inserted before arg2 and “false” otherwise. In Python, a sortBlock is called again called with two elements, and is expected to return -1 if arg1 is less than arg2, 0 if they are the same, and “1” if arg1 is greater than arg2.

Leaving aside, for the moment, the debate about which behavior is correct, the point is that they are different. The implication of this is that the developer is forced to think about and code for the specific language (in this case Smalltalk and Python) of a particular fragment of code.

Solution

The Zeetix solution is to encapsulate this semantic difference in the SortedCollection class itself, so that the programmer doesn’t have to think about it. The platform can, if desired, support the ability to choose whatever semantics are desired — and that choice is then automatically applied across the board, regardless of the current language binding.

Example Problem
Behavior, the (abstract) superclazz of Metaclazz and Clazz, provides a “subclazzes” method that answers a sorted collection containing the immediate subclazzes of the receiver, ordered according to their “symbol” (the name by which they are referenced in the ecology that contains them).

The Behavior.subclazzes method should answer a SortedCollection with two elements — Clazz and Metaclazz, in that order. Behavior clazz also has two subclazzes, Clazz clazz and Metaclazz clazz, in the same order.

The KernelTest.CoreTest.KernelCoreLoadsTest test case was failing testBehaviorLoads, because the SortedCollection instance returned by Behavior and BehaviorClazz was ordered incorrectly.

My inspection (in the debugger) of the failing test showed that the result was, in fact, an instance of SortedCollection and contained the correct objects. They were, however, in the wrong order. Clearly, SortedCollection was sorting incorrectly.(more…)

I’ve discovered (or rediscovered) an exquisite hack — a way to add or (in this case) replace methods of an existing but out-of-scope module, and then put them back again.

Motivation

The minimal ZeeUnit is intended to discover the minimal Zeetix kernel. It therefore should not depend on any clazzes that are outside the Kernel assembly.

In order to reliably restore the underlying database (from which the Kernel clazzes are loaded), it uses code within the existing kernel that (incorrectly) requires clazzes from two non-kernel assemblies — Admin and Command.

I needed a way to use the existing kernel code, while not adding these two big warts to ZeeUnit. I therefore cloned the required clazzes into KernelTest.CoreTest, and made minor edits so that they work.

Unfortunately, two kernel clazzes — ItClazz and ItRecordClazz — contain methods that expect to find these missing clazzes in their original location.

I therefore needed a way to temporarily replace the offending methods so that the new test code works properly.

Solution

I borrowed a paradigm from Smalltalk — Smalltalk-style class extensions. A class extension, in Smalltalk, is a group of methods that can be atomically added or removed from an existing Smalltalk class. This behavior is simulated in Ruby by the Ruby “module” mechanism.

It turns out that a Python class uses a dictionary-style mechanism, similar to Smalltalk, for method dispatch. Python supports adding and removing methods from classes, at run time, by assignment operations on the class. The trickiest part is determining the specific Python class that is associated with a given Zeetix clazz (or Metaclazz).

I used the setUp and tearDown hooks in TestCase to install and then remove the special methods, leaving the Kernel unaffected except while running the test(s) in question.

Result

The result is that the special behavior added by the new test case is available to every existing instance of every Zeetix object, without modification. This proved extraordinarily valuable in this case, because the desired behavior is used across the clazz and metaclazz hierarchy. The resulting code works like a charm.(more…)

I’ve added a slew of new tests that validate the persistence clazzes (ItRecord descendants).

They exposed an insidious bug, that I’m now repairing: SetRecord and SortedCollectionRecord have never been correctly loaded. Worse, the record_id of their behavior entry is wrong — it is “0”, which should never happen.

This, in turn, means that I now have to adjust the database cleanup and setup routines so that they can be called from regular code, instead of the browser-based admin tools. I suppose I’ll have to be disciplined and write a new test case that confirms THIS change.

The database cleanup and setup routines are invoked from “prepareForDebug”, which is located someplace in the ZeeLife root. I suppose now is as good a time as any to go find and capture it.

In addition, all this means that I’ve started to populate the sql branch of the git tree. I need to move the Kernel.ZeeStore records around. The sql branch is organized like the python and ruby branches; the sql for each assembly/subassembly is located in a correspondingly-named subdirectory of the sql branch.

Create a sql file that adds suitable records to it, behavior, clazz, and metaclazz, and any superclazzes. If Foo is to have persistent instances, this file should also create a new table named “foo”. By convention, the sql file is named “create_foo.sql”, and is kept in the “sql” subdirectory of the “Bar” directory. NOTE: Each new object requires a new version_id/object_id, and these must be kept in synch with the ZeeStore.

Run, from either the command line or MySqlQueryBrowser, the sql created in the previous step. Validate that the new records exist in the ZeeStore and are well-formed.

Create a python file containing class definitions for the new clazz and its metaclazz. The new file should be named “Foo.py”, and should be located in the “Bar” subdirectory of the zeetix root. Inside “Foo.py”, there should be an import statement that imports the superclazz, and class definitions for “Foo” and “FooClazz”.

Create another python file containing class definitions for the record clazzes of the new clazz. The new file should be named “FooRecord.py”, and should be located in the “Bar” subdirectory of the zeetix root (alongside “Foo.py”). Inside “FooRecord.py”, there should be an import statement that imports the superclazz (another descendant of ItRecord), and class definitions for “FooRecord” and “FooRecordClazz”.

Add implementations of hook methods required by the various framework interfaces, such as “doTransientAttributeNames”, “doTableName”, and so on.

Add an import line and installation code to “BarAssembly.py” that imports and installs Foo and FooRecord.

With proper tweaking of the existing Zeetix naming conventions, it should be possible to generate the required column names (in the DB), record attribute names (in the record clazz), and instance attributes (in the new clazz) from the specification provided in the original subclazz creation method. I do some of this now (I use ZeeString), and could do MUCH MUCH more.

We need to incorporate persistence into the tests that document and validate the kernel. Each clazz that can have persistent instances must have a corresponding record clazz. The kernel depends on this behavior (each kernel clazz is loaded from the ZeeStore), and the resulting functionality is a primary aspect of the very reason for Zeetix to exist.

I think (although I’m not sure) that every clazz, even those that do not have persistent instances, should have a corresponding record clazz that describes the clazz itself. There is a potential loop here, which I remember thrashing at for extended periods, about the relationship between instances of MetaclazzRecord, ClazzRecord, and so on.

I have since gotten into a bad habit of not adding transient clazzes (those that I know will never have persistent instances) into the ZeeStore. This will break as soon as we attempt to automatically generate non-python source code for those clazzes, because the ZeeStore will not contain the records needed to construct them.

Meanwhile, I need to document the recipe for creating a new clazz, so that we can add behavior to automate it. My bad habit is a result of the tedium of editing and updating the SQL and record clazzes when I didn’t absolutely need it.

The new tests should validate at least the following existing clazzes:

The new ZeeSuiteTestCase demonstrates the aboutToRun and finishedRun bugs that I found in the original SUnit, and therefore ZeeUnit, test suite.

Interestingly, it demonstrates the utility of clazz instance variables. In this case, a message (aboutToRun and finishedRun) was being sent to all descendants of TestCase instead of just the TestCase descendants whose instances were contained in whatever TestSuite instance was being run.

I created a descendant — CountingTestCase — that contains two clazz instance variables (“ranAboutToRun” and “ranFinishedRun”). It hooks the aboutToRun and finishedRun clazz methods and, within each, sets the corresponding clazz instance variable before delegating to its superclazz (TestCase). It also contains a trivial instance method, “dummy”, that TestSuiteTestCase uses to instantiate test instances. I then created two descendants of CountingTestCase — “ShouldNotRunTestCase” and “ShouldRunTestCase”.

The tests within TestSuiteTestCase (there are currently four of them, two each for aboutToRun and finishedRun) create a new TestSuite and add an instance of ShouldRunTestCase that invokes the “dummy” test method. It then asserts that the ShouldRunTestCase clazz method was run and denies that the ShouldNotRunTestCase clazz method was run.

It is too bad that “lambda”, in Python, is so misleadingly named. It is even more regrettable that the scoping semantics of Python’s attempted closure mechanism are equally broken. Python’s “lambda” is a very thin syntactic veneer on top of Python’s local function behavior.

The specific problem is that the scope of a local function does not nest inside its containing scope(s). Here is an example:(more…)

The finishedRun message follows the same pattern as TestSuite>>aboutToRun, is broken in the same way, and requires the same fix.

This bug also demonstrates that a test case is missing from ZeeTestTest (and, for that matter, from the original Smalltalk unit test test). The test case needs to be created and the bug repaired.

At a meta-level, it seems clear enough that these two bugs are really ONE bug with two symptoms. This, in turn, suggests additional required functionality in the Zeetix environment. It should be possible to express the relationship between these “two” bugs, so that it can be fixed “once and only once”.

At a deeper level, TestSuite>>aboutToRun and TestSuite>>finishedRun are two applications of the same underlying code pattern. That pattern originally delegated each to the clazz. Instead, it should have delegated each to a (missing) enumeration of the contents of the receiver.

This deeper relationship between these two apparently (from the source code) unrelated methods cannot be expressed in a notation-driven environment. This inability demonstrates why the Zeetix approach is needed.

Problem Code

The broken code is the implementation of the instance method (finishedRun) of TestSuite. The current Python implementation (transliterated from Smalltalk) is: