I wonder how many curious Prolog programmers wrote a similar solution in the past. A cut can be added to the first clause of zipper/5 to eliminate a choice point. The zipper constructed is not the same (minus the index start) as in Daniel’s solution, however, as we can no longer simply concatenate Before+Element+After to get the original list. On the other hand, my solution follows the list example on the Wikipedia page:

– New character escape 0'\q unexpectedly quits Prolog for those embarrassing programming moments.

– Due to all controversy regarding char_conversion/2, from now on all standards will be published in Esperanto.

– Attempting printing of cyclic terms will endlessly print the lyrics of “99 bottles of beer on the wall”.

– New operator class yfy for decomposing operands.

– Singleton variable report by read_term/3 is deprecated as it’s not politically correct to complain about a variable choice of life-style. In addition, for privacy reasons, the variable_names/1 option is also deprecated.

– New value for the unknown flag, true, provides a sharp decrease on the number of “no“s reported by Prolog interpreters. Specially relevant in educational environments, it will hopefully contribute to make the language more attractive to students.

One of the oldest and more useful feature found in most Prolog compilers is the term-expansion mechanism. Acting as a souped-up pre-processor, defining term- and goal expansion clauses allows us to rewrite terms and goals as their are read from a source file and apply all kinds of transformations. This made term-expansion the preferred mechanism for implementing all sorts of extensions to Prolog.

You may be wondering if the Logtalk compiler is also implemented using the term-expansion mechanism. The answer is no. One of the main reasons is that the term-expansion mechanism is not implemented by all Prolog compilers. Among those that do implement term-expansion, there are also some minor (and not so minor) syntax and semantics differences. The real trouble with term-expansion results, however, from its usual implementation and usage patterns, combined with module features, not from those portability issues.

The usual pattern when implementing some Prolog extension using the term-expansion mechanism is to define a module, relying on term- and goal-expansion clauses to compile to standard Prolog code the syntactic elements defined by the extension. As the recommended way of using modules is to simply use a use_module/1 directive to make the module resources available in your current context (how convenient and easy is to shoot yourself in the foot… but I digress), the term_expansion/2 and goal_expansion/2 predicates are declared as multifile predicates, with the clauses ending up either on the user module or on a special system module. Either way is fine until you try to use more than one module that defines term_expansion/2 clauses. The Prolog compiler will try term_expansion/2 clauses until it finds one that succeeds for the term being expanded or until there are no more clauses left to try (the goal_expansion/2 predicate, however, is called recursively until no more goal-expansion clauses apply). Assuming that no two Prolog modules want to expand the same term (one can always hope), there should no problem. Sure, there might be a performance issue due to trying and failing term_expansion/2 clauses until the correct one is selected. But what happens if two modules compete for expanding the same term? Say, the end_of_file term. Well, if the module developers are not aware of the problem, one of them will likely lose. Each one? That will depend on the modules loading order. The usual workaround is to define term_expansion/2 clauses that perform their magic using side-effects to record the results and fail when done. The failure leads to backtracking, giving other term_expansion/2 clauses a chance to be applied. Ugliness of using side-effects apart, there is a more sane way of implementing and using the term-expansion mechanism.

One of the advantages of not being first (to design and try a new mechanism) is that you can learn from both the successes and mistakes of others before you. You also have a significant advantage by not having to care about breaking backward compatibility. The implementation of term-expansion in Logtalk differs from the implementation found on most Prolog compilers in two crucial aspects.

First, Logtalk doesn’t support using the term- and goal expansion clauses defined in an object (or source file) to expand the object (or source file) itself. It’s already bad enough that some predicate directives must precede the compilation of calls to those predicates. But having the result of the compilation of an object (or source file) depend on the order of the predicate definitions in the same object (or source file) feels plain wrong. Specially as it’s an exception to the way both Logtalk and Prolog compilers work. Just imagine being forced to define all predicates called by a predicate before it could be defined. It will be painful. It would also be impossible in some cases. Yet, well know Prolog extensions do just that, throwing portability down the drain (not all Prolog compilers make predicate definitions visible and usable, as soon as they are compiled, without waiting for the rest of the source file to be compiled…).

Second, term- and goal-expansion clauses are defined local to the object where they are defined. It’s possible to add those clauses to the pseudo-object user (Logtalk supports multifile object predicates). But that is neither necessary or advised. Do we really want to put all our eggs… err… term-expansion hooks in the same basket?

So, what’s the magic incantation to use term-expansion in Logtalk? Easy. Objects containing clauses for the term- and goal expansion predicates are known as hook objects. When compiling a source file, you use a compiler option to state which hook object should be used to term-expand the source file. For example, assuming a hook object named my_expansion we could write:

| ?- logtalk_load(source_file, [hook(my_expansion)]).

This compiler option also provides a nice solution for combining different hook objects in order to expand the same source file. You simply define another hook object that calls the term- and goal-expansion predicates defined in the base hook objects. This allows fine control on how the different expansions will work together. For example, we can define a pipeline of goal-expansions. Or apply all term-expansions in parallel and combine the resulting terms. For some sample code, check the expanding example in the current Logtalk distribution. Or browse the example source code on-line:

Even when a source file is to be expanded using a single hook object, we can be ensured that the compilation time will not be adversely affected by other hook objects that might be loaded at the time of the compilation. The code also becomes easier to debug as only the relevant term-expansion clauses will be in focus.

P.S. It’s worth noting that the Logtalk term-expansion implementation and recommended usage patterns can be easily adopted by Prolog module systems. In that regard, it’s also worth noting that some Prolog compilers, e.g. SICStus Prolog and Ciao Prolog, already include improved implementations of term-expansion mechanisms.

These coding guidelines are also mandatory for contributions to the Logtalk distribution. Most guidelines in the above document are pretty consensual but one of them, source code layout and indentation, always stirs a lot of discussion:

Format your source file using tabs, not spaces.

Search the web and you can easily find passionate texts defending the use of either tabs or spaces for source code layout and indentation. Recently, I was commenting on a paper on Prolog coding guidelines that stated:

2.1 Indent with spaces instead of tabs.

You cannot normally predict how tabs will be set in the editors and browsers others will use on your code. Moreover, mixing tabs and spaces makes searches and substitutions more problematic. All in all, it is better not to use tabs altogether: almost any editor can be set to substitute spaces for tabs automatically when saving your file.

Take the first sentence. It completely misses the point of using tabs instead of spaces. Try this. Open any of the many examples distributed with Logtalk in your favorite text editor. The example source code is formatted using tabs. Next go to your text editor preferences and adjust the tab size to your liking. I prefer a tab size equivalent to 4 spaces. You may prefer a tab size equivalent to 2 spaces. Or 3 spaces. Or 6 spaces. No matter your choice, the code will remain perfectly indented. That means that I’m not imposing on you my indentation preferences. If someone gives you a file indented using spaces, say, where an indentation level is 2 spaces, and you prefer to use 4 spaces, your preference will be ignored. You will need to edit the file, i.e. change it, even if only for you to feel comfortable reading/studying the source code.

If you think that my argument is only meaningful for text editors but not for browsers, well, think again. I have written (and tested!) support for syntax highlight of Logtalk source code for Pygments, GeSHi, Highlight, Source-highlight, and SyntaxHighlighter. If you’re publishing source code on a web page, a blog, or a wiki, chances are that you will be using one of these syntax highlighters. As you already guessed, they allow you to set your tab preferences. Never found a problem due to the use of tabs instead of spaces.

I have also written (and tested!) Logtalk syntax highlight support for several text editors, including TextMate, SubEthaEdit, Emacs, Vim, Kate, Gedit (i.e. GtkSourceView), NEdit, and jEdit. Again, never found a problem due to the use of tabs instead of spaces. This, I concede, may not have always been true in the past but, nowadays, using spaces for indentation is just a big waste of space. And obnoxious to all people you may share your code with that don’t necessarily share with you the same preferences for the size of an indentation level.

As this post is about a holy war, it’s mandatory and good netiquette that I finnish it with a suitable insult or inflammatory remark to the other side. So, death to the tab infidels! May your spacebar burn in hell!

Try the following experiment in a Prolog compiler with a module system. Define a simple module containing a simple meta-predicate that tests its meta-argument, writing an error message if it’s uninstantiated:

Back to the module example. Is the standard var/1 built-in predicate broken? Or maybe var/1 isn’t module-aware? Should we blame the meta_predicate/1 directive? But we want it in order to be able to call the meta-predicate without qualifying both the call and the meta-arguments(!) when importing the module. At this point, Prolog module pundits are likely loosing their patience and screaming that the solution is to “correct” the meta-predicate definition to:

mp(_:Goal) :-
( var(Goal) ->write(error)
;write(passed)
).

mp(_:Goal) :-
( var(Goal) ->
write(error)
; write(passed)
).

With this “corrected” definition you now get:

| ?- m:mp(_).
error
yes

| ?- m:mp(_).
error
yes

Some will go as far as arguing that, because a simple workaround is available, everything is fine. But one of the main goals of the meta_predicate/1 directive is to avoid using explicit qualification (as we are now doing in the meta-predicate clause head) in the first place. Wouldn’t it be more elegant to make var/1 (and other built-in predicates!) module-aware? That would fit nicely with the common argument that modules are a natural solution for encapsulation Prolog code. And would avoid the trouble of adding not only a meta_predicate/1 directive but also modifying the meta-predicate definition when moving a meta-predicate to another module. Not to mention that the original, broken module meta-predicate definition is more elegant, straightforward, easier to explain and understand by novice programmers when compared with the “corrected” definition, which requires understanding the lower-level details of Prolog modules compilation.

Update: You may wonder what results you get with other Prolog compilers with modules systems. The problem doesn’t exist in ECLiPSe as this compiler uses an alternative solution for declaring and defining meta-predicates. The problem also doesn’t exist in the beta versions of Ciao 1.13 where our first attempt at defining the meta-predicate is also the correct one, working similar to Logtalk in this respect.

The development of the current generation of Logtalk, 2.x, began on January of 1998. At that time, the ISO Prolog Standard (Part 1: General Core) was only three years old. Thus, accepting and dealing with the lack of compliance of most Prolog compilers with the standard was a sensible choice.

Back to the present. The ISO Prolog Standard is now 15 years old. Compliance with the standard greatly improved for some but not all Prolog compilers. Trying to keep minimal Logtalk compatibility with some of the existing Prolog compilers is no longer feasible. In fact, keeping compatibility with the most problematic Prolog compilers (as far as standard-compliance goes) is working as an anchor, slowing Logtalk development and preventing improvements and implementation of new features.

Some of the most problematic Prolog compilers (again, from the point-of-view of standards compliance) are (to the best of my knowledge) no longer being developed. Other Prolog compilers, actively maintained today, decided to ignore the current official and de facto standards. A few, such as IF/Prolog, provided good standard compliance but have been apparently discontinued by their developers.

I plan to ditch Logtalk compatibility with the following Prolog compilers in the upcoming 2.39.0 release: ALS Prolog, Amzi! Prolog, BinProlog, GNU Prolog, IF/Prolog, JIProlog, K-Prolog, LPA MacProlog, LPA WinProlog, Open Prolog, MasterProlog, PrologII+, Quintus Prolog. This may seem like a long list but I suspect this decision will have no consequence for most (if not all) Logtalk users. If If you think it is still worth to support some compiler in this list please contact me as soon as possible.

UPDATE: added GNU Prolog to the list of no longer supported compilers. Support for this compiler will be restored as soon as it implements the ISO Prolog standard directive multifile/1.

Logtalk 2.38.0, released earlier this month, adds support for lambda expressions. A simple example of a lambda expression is:

| ?- meta::map([X,Y]>>(Yis2*X), [1,2,3], Ys).
Ys= [2,4,6]
yes

| ?- meta::map([X,Y]>>(Y is 2*X), [1,2,3], Ys).
Ys = [2,4,6]
yes

In this example, a lambda expression, [X,Y]>>(Y is 2*X), is used as an argument to the map/3 list mapping predicate, defined in the library object meta, in order to double the elements of a list of integers. Using a lambda expression avoids writing an auxiliary predicate for the sole purpose of doubling the list elements. The lambda parameters are represented by the list [X,Y], which is connected to the lambda goal, (Y is 2*X), by the (>>)/2 operator.

Currying is supported. I.e. it is possible to write a lambda expression whose goal is another lambda expression. The above example can be rewritten as:

| ?- meta::map([X]>>([Y]>>(Yis2*X)), [1,2,3], Ys).
Ys= [2,4,6]
yes

| ?- meta::map([X]>>([Y]>>(Y is 2*X)), [1,2,3], Ys).
Ys = [2,4,6]
yes

Lambda expressions may also contain lambda free variables. I.e. variables that are global to the lambda expression. For example, using GNU Prolog as the back-end compiler, we can write:

Logtalk uses the ISO Prolog construct {}/1 for representing the lambda free variables as this representation is often associated with set representation. Note that the order of the free variables is of no consequence (on the other hand, a list is used for the lambda parameters as their order does matter).

Both lambda free variables and lambda parameters can be any Prolog term. Consider the following example by Markus Triska:

As illustrated by the above examples, Logtalk lambda expression syntax reuses the ISO Prolog construct {}/1 and the standard operators (/)/2 and (>>)/2, thus avoiding defining new operators, which is always tricky for a portable system such as Logtalk. The operator (>>)/2 was chosen as it suggests an arrow, similar to the syntax used in other languages such as OCaml and Haskell to connect lambda parameters with lambda functions. This syntax was also chosen in order to simplify parsing, error checking, and, eventually, compilation of lambda expressions. The specification of the Logtalk lambda expression syntax can be found in the Logtalk reference manual. The current Logtalk version also includes an example, lambdas, of using lambda expressions with a fair number of sample queries.

Although the first, experimental implementation of lambda expressions in Logtalk followed Ulrich Neumerkel’s proposal for lambda expression syntax in Prolog, that representation was dropped and replaced by the current one in order to avoid some of the questions raised by the Ulrich’s proposed syntax. In his proposal, lambda expressions start with the (\)/1 prefix operator, which can be seen as an approximation to the greek letter lambda (λ). However, the backslash character is usually associated in Prolog with negation as found e.g. in the standard operators (\+)/1, (\==)/2, (\=)/2, and (=\=)/2. Another issue with this operator is that users must be careful when the first lambda parameter is enclosed in parentheses. Consider the following example by Markus (using SWI-Prolog with Ulrich’s lambda library):

| ?- maplist(\(A-B)^(B-A)^true, [1-a,2-b,3-c], Zs).
false.

| ?- maplist(\(A-B)^(B-A)^true, [1-a,2-b,3-c], Zs).
false.

The goal fails because there is a missing space between the (\)/1 prefix operator and the opening parenthesis that follows. A likely trap for beginners. Ulrich’s syntax for lambda free variables requires adding a new infix operator, (+\)/1, to the base language, something that I prefer to avoid. Not to mention that this operator is too similar to the Prolog negation operator, (\+)/1. Parsing lambda parameters also needs to be careful to avoid calling a non-existing (^)/2 predicate when the lambda expression is misformed. Parsing lambda parameters is arguably simpler in Logtalk due to the use of a list plus using the (>>)/2 operator to connect the parameters with the lambda goal.

The Logtalk implementation of lambda expressions is still evolving. The current development version features improved error-checking and adds support for using a (>>)/2 lambda expression as a goal (besides as a meta-predicate closure). No optimizations are yet in-place. Thus, be aware of possible performance issues if you plan to use lambda expressions heavily in your applications. But don’t let that stop you from having fun playing with Lambda expressions in Logtalk. As always, your feedback is appreciated. Thanks to Ulrich Neumerkel, Richard O’Keefe, and Markus Triska for their lambda expression examples.

A recurring question on the comp.lang.prolog newsgroup is how to work with different data sets, usually loading them from different files, without mixing the data in the plain Prolog database. Unfortunately, these questions often lack enough details for making an informed choice between several potential programming solutions. Two possible solutions are (1) load the data into suitable data structures instead of using the database and (2) use clauses to represent the data but encapsulate each data set in its own Prolog module or Logtalk object. Some combination of both solutions may also be possible. In this post, however, we’re going to sketch the second solution using Logtalk objects. For an alternative but also Logtalk-based solution please see this previous post.

Assuming all data sets are described using the same predicates, the first step is to declare these predicates. The predicate declarations can be encapsulated either in an object or in a protocol (interface). Using a protocol we could write:

Assuming we have the required memory, we can load some of all of our data sets without mixing their data. But that’s not all. We can also encapsulated our data set processing code in its own object (or set of objects, or hierarchy of objects, or whatever is suitable to the complexity of our application). This object, let’s name it processor, will perform its magic by sending messages to the specific data set that we want to process. For example:

If the computations we wish to perform make sense as questions sent to the data sets themselves, an alternative is to move the data set predicate declarations from the data_set protocol to the processor object and make the data set objects extend the resulting object, below renamed as data_set. For example:

An advantage of this solution is that the object data_set can contain default values for the data set description predicates. The ::/1 operator used above is the Logtalk operator for sending a message to self, i.e. to the data set object that received the message compute/1. If the information requested is not found in the data set object, then it will be looked for in its ancestor, where the default values are defined.

The best and most elegant solution will, of course, depend on the details on the data set processing application. For example, above we could have defined the object data_set as a class and the individual data sets as instances of this class (technically, the solution above uses prototypes).

Note that all code above is static. Individual data set description predicates may be declared dynamic (using the predicate directive dynamic/1) if we need to update them during processing of the data sets. If our application requires being able to delete data sets from memory, is simply a question of declaring the data set objects dynamic using the Logtalk object directive dynamic/0 and to use the Logtalk built-in predicate abolish_object/1 when a data set object is no longer needed.

We have only scratched the surface of the Logtalk features that we could make use in our implementation but, hopefully, it’s enough as a starting guide. Feel free to stop by the Logtalk discussion forums to further discuss this programming pattern.

Currently, there are two approved ISO Prolog standards: ISO/IEC 13211-1: General core (first edition published in 1995-06-01; an errata was published recently) and ISO/IEC 13211-1: Modules (first edition published in 2000-06-01). There are also five standardization proposals being discussed: Core Revision, Definite Clause Grammars (DCGs), Globals, Threads, and Portable Operating-System Interface (POSI).

While I was a member of the WG17 standardization group (see my previous post), I always stood for a mandatory core standard, making all other standards optional when talking about ISO compliance of a specific Prolog compiler. This means that a Prolog implementer would only need to comply with the core standard in order to claim conformance to the ISO Prolog specification. The Prolog implementer could also freely chose to implement e.g. DCGs and POSI, disregarding Module, Globals, and Threads.

This view of mandatory and optional Prolog standards was shared by some but not all members of the WG17 standardization group. Some of them want to make the Module standard mandatory and pushed for making some of the other standardization proposals dependent (or at least making reference to) the approved Module standard. I find this a recipe for disaster and for ISO Prolog standards irrelevance. A standard should stand on its own merits. Despite the hard work done on the Module standard, the proposed module system is (rightfully) ignored by most Prolog implementers. Other standard proposals should not be used as a leverage for forcing implementers to implemented a flawed standard.

Why is the current Module standard flawed?

First, it specifies a new module system instead of trying to standardize current practice. Instead of helping existing module implementations to converge, the standard choses to specify a new, and therefore incompatible, module system. For example, the standard introduces a new concept of module interface (which can only be implemented by a single module!) that is not found elsewhere, even today.

Second, it specifies two different and incompatible ways of dealing with meta-predicates (the infamous colon_sets_calling_context flag). This means that two Prolog compilers can comply with this standard and still be incompatible!

Third, specifies a meta-predicate directive that prevents the specification of the number of missing arguments when working with closures. One of the consequences is that only the Prolog implementer knows how to parse and make use of meta-predicate directive for built-in predicates! But check what the inventor of the meta-predicate directive have to say about the flaws on the meta-predicate directive as specified in the Module standard.

Fourth, it makes some poor choices regarding built-in predicates and built-in directives. For example, the specification of the predicate_property/2 predicate defines the properties public and private as stating if clause/2 can be used on the predicate clauses. Thus, the properties public and private cannot be used to infer about predicate scope, which would be a much better match for most programmer expectations. Another example is the meta-predicate directive, which is ugly named metapredicate/1 (while existing module systems use, of course, a meta_predicate/1 directive!).

Fifth, the standard fails to specify a solution for renaming predicates when importing. The consequence is that library developers need to be aware of the predicate names used by other library developers in order to avoid conflicts. So much for the idea that modules provide an encapsulation mechanism. Of course, the standard also states that any module predicate (including not exported ones) can be called using explicit qualification and leaves as “an allowable extension to provide a mechanism that hides certain procedures (…)”.

Other problems with the current module standard could be described here but the ones stated above are hopefully enough to convince you that the standard needs to be thoroughly revised.

You may think that my critics of the current module standard (and WG17 policies) are mostly motivated by my work in Logtalk, which provides an alternative to the use of modules. You are wrong. True, Logtalk objects subsume module functionality and the Logtalk compiler is able to compile most modules as objects. But the Logtalk compiler also goes to great lengths to allow programmers to use both modules and objects in the same applications. Case in point: the fact that Logtalk can compile most modules as objects clearly show that there is enough common core functionality in today’s module systems to warrant a new module standard focused in current practice. But any new or revised module standard should also pay due attention to the advanced modules found on some Prolog systems such as ECLiPSe and Ciao.

In its current state, making the current module standard mandatory or required for implementing other standard proposals will either delay standardization efforts or will tie implementations to a limited and flawed module system for years to come.

I’m stepping down as editor of ISO Prolog standardization proposals. In recent years, I found myself responsible for four different draft proposals: Core Revision, DCGs, Threads, and POSI (Portable Operating-System Interface). My fault really. With the exception of the DCGs proposal, all the other proposals are born from my initiative. Recently I have been unable to fulfill my duties as editor of the DCGs proposal, failing to meet the deadline for its next revision. This resulted both from the proverbial lack of time and from being weary of the ISO standardization process. This process is mostly broken, unable to meet the needs of the Prolog community. I tried to fix it from within. I failed.

The last straw that resulted in my decision to end my standardization work was the lame events at the WG17 meeting at Pasadena. Tired of the lack of sensible priorities in the discussion of the standardization proposals, I succeeded I changing the meeting’s agenda, convincing the others participants to discuss the Core Revision proposal instead of spending another annual meeting discussing DCGs and Globals. Nothing wrong, of course, with DCGs and Globals. Both are worthy subjects for standardization. But fixing and improving the Core Prolog standard is the most important and urgent task. We discussed the Core Revision proposal in the morning, going from A to Z, making decisions and identifying contention aspects that would merit further discussion for the next revision of the proposal. At the end of the morning, I was pretty satisfied with the results and leaved to catch my flight back home, thus unable to attend the WG17 meeting in the afternoon. The remaining participants decided, without me as the editor of the Core Revision proposal being present, to go back and change the decisions made in the morning. I found this behavior regrettable and disrespectful. I also found some of the afternoon decisions will informed and resulting from an apparent lack of knowledge of the current, published standards. Moreover when most of participants aren’t aware that a Core Revision proposal even existed before this meeting and never participated in previous discussions about this proposal.

I still believe that standardization is vital for the future of Prolog as a programming language. But the current ISO process is the wrong way to do it. Case in point. Standardization proposals are voted by countries, instead of being voted by implementers and users groups. Implementers always decided if a proposal is worthy, by implementing it, or if is worthless, by ignoring it (think ISO Prolog Part 2: Modules). Users are the ones using and claiming for a better language.

A saddening aspect of the ISO standardization process is the lack of perception from outsiders that people working on proposals and participating in meetings are volunteers and not necessarily experts on the matters being discussed. I can understand that outsiders find some aspects of the proposals poorly formulated or completely wrong. I cannot understand that, instead of criticizing the proposals and suggesting alternatives, outsiders choose to insult the volunteers that are doing their best to improve the current standards.

Visibility and openness of the standardization work is also a problem. There are standardization discussion forums and a mailing list. Both are mostly ignored. Neither is listed in the official WG17 web site. It gets tiresome quickly to keep explaining and repeating arguments because people either want to remain anonymous when giving feedback to the current proposals or have no knowledge of the reasoning and previous discussions behind the proposals.

Improving the current, published Prolog standards, requires the courage to recognize past errors and fix them, even if that results in revoking and replacing them with hopefully better specifications. Some people refuse this path and desperately cling to the past, painting the whole standardization process in a corner. I have no patience left for this nonsense.

More could be said about the problems in the current ISO standardization process but I hope that the few ones described above are enough for you to understand my decision. Many thanks to all the people that contributed to get this far in my standardization efforts. Hopefully others with more time and energy will continue from here. My only advice, if I may give one, is: throw away the current ISO standardization process and start a new, grassroots movement that brings together implementers and user groups. Some good examples can be found in the recent web standardization processes and in other programming language communities.