"QNEWS"
=======
* 4.5 14 October 2003
This release sports a new ggi module (general graphics interface, see
http://www.ggi-project.org), and some improvements in the debugger (see the
ChangeLog for details).
------------------------------------------------------------------------------
* 4.4 16 September 2003
The new ODBC module allows you to connect to ODBC-compatible databases (MySQL,
PostgreSQL, ...). There are also a few minor changes and bug fixes, see the
ChangeLog for details.
------------------------------------------------------------------------------
* 4.3.2 15 August 2003
Additional changes for BSD compatibility, bug fixes.
------------------------------------------------------------------------------
* 4.3.1 28 July 2003
Fixes for latest autotools and FreeBSD compatibility, bug fixes.
------------------------------------------------------------------------------
* 4.3 13 June 2003
Free variables can now be declared `const', meaning that they are read-only,
i.e., they can only be assigned a value once. A couple of bugs were fixed,
too, see the ChangeLog for details.
------------------------------------------------------------------------------
* 4.2.3 6 May 2003
Added some missing stuff to clib, and a new gdbm module which provides an
interface to the GNU dbm library.
------------------------------------------------------------------------------
* 4.2.2 3 May 2003
Some additional improvements in the standard library. In particular, the new
Ref type in the clib module now allows you to create mutable data structures
in a direct manner.
------------------------------------------------------------------------------
* 4.2.1 25 April 2003
Improvements in the clib module (automatic int/float coercion in the printf
operations, new openpty and forkpty operations) and some minor fixes in the
documentation.
------------------------------------------------------------------------------
* 4.2 26 March 2003
Good news for Macintosh afficionados: Q has finally been ported to OS X!
Moreover, the interpreter now provides signal handling via the new `trap'
builtin, and the clib module has had a major overhaul and now provides well
over 200 functions. Most essential UNIX system operations have been
implemented, including the POSIX termios and BSD socket interfaces.
Furthermore, some bugs have been fixed, and some quirks with realtime thread
scheduling under Linux and Windows have been resolved. NOTE: Take a look at
modules/clib/givertcap if you want to enable realtime scheduling in a non-root
application under Linux.
------------------------------------------------------------------------------
* 4.1.3 18 February 2003
Minor code cleanup and bug fixes; see ChangeLog for details.
------------------------------------------------------------------------------
* 4.1.2 24 November 2002
Some minor bug fixes in the interpreter and the clib and dxl modules.
------------------------------------------------------------------------------
* 4.1.1 19 November 2002
Bugfix in the builtin catch function.
------------------------------------------------------------------------------
* 4.1 11 November 2002
This is a minor update with some cleanup in the standard library, various
improvements in the debugger and q-mode.el, updated documentation, and fixes
for Tcl/Tk 8.4 compatibility. Moreover, a new builtin function `which' has
been added which facilitates package configuration. Please refer to the
ChangeLog for details.
------------------------------------------------------------------------------
* 4.0 3 November 2002
Ok, enough features have been added since release 3.1 now, so I decided it's
time for a major release again. This release includes all the cumulative
changes of the 3.x series, some more bug fixes (see the ChangeLog for
details), and the following additional changes:
- Equations may now include an arbitrary list of conditions and where clauses,
in any order. See below for details.
- The `=' operator must now be parenthesized on *both* the left- and right-
hand sides of equations and definitions.
This means that some definitions in existing scripts might have to be
changed, but this is a small price for enabling the compiler to find obscure
errors where a definition is mistaken for an equality check (which can
happen much too easily, e.g., when forgetting the semicolon at the end of an
equation, or the comma at the end of a definition).
- The tk module now includes additional operations for setting and retrieving
variable values in the Tcl/Tk interpreter.
- The sample exception scripts (except.q, clib_except.q, etc.) have been moved
back to the examples directory, which is where they really belong.
The most notable change in this release is that an equation may now include an
arbitrary list of conditions and where clauses, in any order. Thus the
arbitrary restriction to "where-if-where" constructs has been removed. As
before, the conditions and where clauses are processed from right to left,
whereas the individual definitions in a where clause are considered from left
to right. As soon as a condition or variable definition fails, the entire rule
is aborted. This allows you to have definitions like, for example,
foo X = bar Y if qux Y where Y = baz Z if quux Z where Z = ...;
which interleaves conditions with the definitions of required variables, or
foo X = bar Y where Y = baz Z where Z = qux U where U = ...;
which incrementally builds a value through a series of definitions. (Note that
the latter is actually equivalent to `foo X = bar Y where U = ..., Z = qux U,
Y = baz Z;' but for many examples the "backward" order is just more natural.)
The debugger was modified as well, to acommodate for the above change. It now
stops on the current rule every time a condition or definition has been
processed, and always displays the current qualifier while it is being
processed. This makes it much easier to keep track of a rule with many
different conditions and local definitions.
------------------------------------------------------------------------------
* 3.5.5 28 October 2002
This release fixes segfaults in the builtin hash function and the tk module.
------------------------------------------------------------------------------
* 3.5.4 10 October 2002
This release fixes a compiler bug which caused wrong bytecode to be generated
when parsing multiple right-hand sides involving where clauses for the same
left-hand side. It also works around a bug in the MSVC sscanf function which
affected the clib sscanf function (Windows only).
------------------------------------------------------------------------------
* 3.5.3 8 October 2002
This release fixes some obscure segfaults in the debugger and the builtin
tuple concatenation function.
------------------------------------------------------------------------------
* 3.5.2 6 October 2002
This is a maintenance release which fixes some compilation quirks on the
latest Linux distros which come with Tcl/Tk 8.4 and gcc 3.2.
Other changes:
- The variable file is now named .q_vars by default, so it is hidden on Unix
systems.
- I also started working on the floating point precision of the `save'
command. The precision used by `save' is now larger to reduce rounding
errors when reconstructing these values with `load'. Moreover, I added
experimental support for saving exact floating point values represented in
hexadecimal textual format. Currently this only works on systems with a
C99-compatible printf function which has the %a conversion flag. (Add
-DHAVE_ISO99_PRINTF to the list of compilation flags when compiling to
enable this feature.)
------------------------------------------------------------------------------
* 3.5.1 19 September 2002
New in this release:
- Bug fixes in the standard library (cond.q, list.q, stream.q) and in the
interpreter's garbage collector (--gc option).
- Larger default stack size.
- Clib: New `gets' and `fgets' operations to read a line with the trailing
newline. The former `gets' function (which reads a whole file at once) has
been renamed to `fget'.
------------------------------------------------------------------------------
* 3.5 September 2002
This release features some more bug fixes and cosmetic changes in the
interpreter and Emacs Q mode; see the ChangeLog for details. The most
important user-visible changes are the following:
- Local definitions (`where') are now permitted both before and after a
qualifying condition. A `where' clause before the condition will only be
processed after the condition has been checked.
- The standard library now also includes a hashed dictionary type which can be
used for arbitrary (not necessarily ordered) key values. See hdict.q.
------------------------------------------------------------------------------
* 3.4.3 August 2002
More bug fixes: The builtin syntactic equality predicate is now implemented
non-recursively, to prevent C stack overflows on large expressions. Special
interpreter commands like `echo' and `cd' now accept string arguments enclosed
in single quotes.
------------------------------------------------------------------------------
* 3.4.2 July 2002
This release fixes .qinitrc/.qexitrc execution, and parse stack overflows
which occurred when sourcing long lists and tuples in the interpreter.
Moreover, the import command now also works correctly if the main module is
the empty script.
------------------------------------------------------------------------------
* 3.4.1 18 June 2002
Bug fixes in the tk module. The interpreter doesn't lock up on tk and tk_reads
any more, and the Tcl/Tk interpreter is now thread-local.
(NB: Starting with this release, I'm using 3-level version numbers when
appropriate to make it easier to handle frequent bugfix-only releases.)
------------------------------------------------------------------------------
* 3.4 14 June 2002
Again a bug fix release. Adds some functions to the C interface which help to
alleviate some weird segfaults in the Windows version.
------------------------------------------------------------------------------
* 3.3 June 2002
Bug fix release. Fixes an annoying bug in the compiler which caused wrong
bytecode to be generated if a variable definition followed an equation with a
where clause.
------------------------------------------------------------------------------
* 3.2 May 2002
Maintenance release. Most notably, the size of the expression data structure
has been reduced considerably (24 bytes per expression cell vs. 36-40 bytes in
earlier releases) without any performance losses. Also adds support for
bounded semaphores (see clib.q).
------------------------------------------------------------------------------
* 3.1 April 2002
This is a major new release featuring a lot of smaller bug fixes and
optimizations, as well as some important new features, most notably: improved
modularization facilities, pattern-matching and rule-local variable
definitions, and multithreading support. See the more detailed list of
user-visible changes below.
(Note that 3.0 was never officially released, hence what follows is the list
of cumulative changes since release 2.3.)
- MODULES: This release cures one of the most apparent shortcomings in earlier
Q versions: the lack of decent support for "programming in the large". The
new module system is both simple and effective. Different scripts are now
considered as separate modules, each with their own namespace, and you must
*always* explicitly include a module to gain access to the public symbols of
that module (with the exception of the prelude operations which are still
available without explicit inclusion). The new syntax for including modules
is:
import id1, id2, ...;
include id1, id2, ...;
Modules may either be specified using their name (which must be a valid
identifier) or the full pathname in double quotes (the .q suffix is supplied
automatically if necessary). The import and include directives differ in
that the latter causes all public symbols of the included module to be
reexported, whereas the former only makes the symbols available in the
importing module.
To resolve name clashes between modules, you can import a module under an
alias, e.g.:
import "my/prelude" as myprelude;
To resolve name clashes between public symbols in different modules, you can
use qualified identifiers of the form modname::id. You can also redeclare an
imported symbol under a new name:
private mylib::foo X Y as bar;
If you redeclare an imported symbol as a public symbol, it will be
reexported.
IMPORTANT: The first declaration of an unqualified symbol in a given module
will now always introduce a *new* symbol; this has to be considered when
porting pre-3.0 scripts. Moreover, new symbols in a module are now declared
"private" by default, so you have to explicitly declare symbols with scope
"public" if they are to be used outside a module.
The namespace available in the interpreter is now that of the main module,
and the interpreter also allows you to dynamically import (and unimport)
additional modules in the global scope. As before, it is possible to gain
access to *all* public and private symbols of the program (also in modules
not directly imported in the main module) using qualified identifiers.
The downside of all this good new stuff is that it really breaks backward
compatibility in some places. But most scripts should work after a minor
touch-up, adding some public keywords and module qualifiers here and there.
Also note that cyclic inclusion chains are not allowed any more, as they
cannot be handled in a single-pass compilation.
- PATTERN BINDINGS: Variable definitions can now involve a pattern on the
left-hand-side which is to be matched against the supplied value. E.g.,
def (X,Y) = foo Z;
evaluates foo Z, matches it against (X,Y) and binds the free variables X and
Y accordingly. Non-linear patterns like (X,X) also work as expected. An
error is reported if the match fails.
- ANONYMOUS VARIABLE: Q now supports the use of _ as an anonymous variable:
hd [X|_] = X;
tl [_|Xs] = Xs;
This is handy if you don't care about a component value on the left-hand
side. The anonymous variable can only be used on the left-hand side of a
rule or definition. Multiple instances of the anonymous variable are matched
independently from each other, therefore
foo _ _ = 0;
will be matched for *any* combination of arguments.
The interpreter now also employs the anonymous variable symbol for denoting
the result of the last expression, instead of the $ symbol used in previous
releases.
Note that, for backward compatibility, the underscore normally still counts
as a lowercase letter, and thus any other identifier starting with _ is
considered as a *function* symbol; the anonymous variable is the only
exception from this rule.
- LOCAL VARIABLE DEFINITIONS: Local variables in a rule can now be introduced
by means of "where" clauses:
foo X = bar Y Z where Y = f X, Z = g Y;
Where clauses always come behind the right-hand side *and* qualifier of a
rule, and the defined variables are available in *both* the right-hand side
and the qualifier:
foo X = bar Y Z if check Z where Y = f X, Z = g Y;
Pattern-matching definitions are permitted as well. Here, a failing match
causes the entire rule to fail. For instance,
foo Z = bar X Z where [X|_] = Z;
works like
foo [X|Xs] = bar X [X|Xs];
but avoids repeating the list value on the right.
Variable bindings in where clauses are performed in the given order, and
each definition may refer to all left-hand side variables of the rule, as
well as all variables introduced in earlier definitions. Each use of a
variable refers to its most recent definition. Note that, in difference to
languages like Haskell, "where" clauses cannot be nested, and they can only
be used to introduce local *variable* definitions, i.e., "local equations"
are not supported.
- EXCEPTION HANDLING: The builtin functionality has not changed, except that
now hard error conditions generate an exception of the form syserr N, where
N is the error number. The syserr symbol is actually a constructor symbol of
the new builtin type SysException, which in turn is a subtype of the
predefined (abstract) type Exception. It is recommended that you use this
scheme for your exception generating rules as well, deriving suitable
subtypes of the Exception type for your exceptions. This discipline helps to
make exception handling much more transparent, and allows you to
discriminate over different exception types without having to resort to
matching individual exception patterns.
The new except.q standard library module implements exceptions along this
line for all builtin and standard library operations; the additional
graphics_except module provides exceptions for the optional graphics
module. Note that these modules are not loaded by default, thus you
have to include them explicitly if you want exceptions to be generated.
- MULTITHREADING SUPPORT: This release provides POSIX-based multithreading.
So, for instance, you can now perform a lengthy calculation as a background
task simply as follows:
==> def TASK = thread (sum (nums 1 1000000))
==> // do some other work ...
==> result TASK // get the result
500000500000
==> stats // stats lists secondary threads once they're finished
0 secs, 1 reduction, 0 cells
thread #1: 8.42 secs, 2000003 reductions, 2000007 cells
The multithreading operations are actually implemented in the clib module.
All the good stuff from the POSIX threads library is there: thread
creation, termination, cancellation, mutexes, conditions, semaphores, ...
See the clib documentation for more details. Note that for all this to
work, you must have the pthread library on your system, and the interpreter
must have been built with thread support (which is the default when the
library is available). Also note that the interpreter with thread support is
a little slower and also needs more memory, compared to the no-thread
version. Moreover, the current implementation effectively serializes
bytecode interpretation on the reduction level and thus user-level
threads cannot really take advantage of multi-processor machines.
- STANDARD LIBRARY: The standard library has gone through a major overhaul, to
accommodate for the new language features. For the most part, the library
should be backward-compatible, but note the following changes:
- clib is now "officially" part of the standard library and is thus loaded
in the default prelude
- examples for the external modules are now installed in corresponding
subdirectories of the examples directory
- for convenience, the `in' symbol used in list and stream comprehensions is
now a predefined relational operator, hence parentheses around the pattern
and list expression are not needed anymore
- removed obsolete std module
- removed obsolete `let' operation from lambda.q (use where clauses instead)
- moved the stuff in the type module into stdlib
- new multithreading primitives in the clib module (see above)
- renamed the simple conditional `cond' in cond.q (which collides with the
new condition constructor in clib.q) to `ifelse'
- new except and graphics_except modules (see above)
- new dxl_file function in the dxl module (see dxl_sample.q for an example)
- COMPILER: Bytecode files now have the executable bit set, and the bytecode
format includes an additional #! header line which allows bytecode files to
be executed directly as shell scripts. Note that this will only work with
UNIX shells supporting the "#!" feature.
------------------------------------------------------------------------------
* 2.3 September 2001
This is a maintenance release featuring several bugfixes for Solaris
compatibility (no functional changes). See the ChangeLog for details.
------------------------------------------------------------------------------
* 2.2 August 2001
Well, as you know, Q 2001 was supposed to be the last and final word, but I
didn't get it quite right at the first attempt. :( This release is supposed
to cure the remaining bugs and warts of release 2.1, and also introduces some
new features which have been left out until now due to lack of time.
The most important user-visible changes are:
- Reorganized the sources and integrated all available modules into the main
package. While I was at it, I also automakified and libtoolized the sources,
which should improve portability. In particular, the C interface should now
work on any platform supported by libtool, and it is now possible to
"preload" modules into the interpreter, which is necessary when debugging a
module and to make modules work on systems lacking shared library support.
To facilitate this, I have added a new utility, `qld' a.k.a. the "Q module
linker", to the package.
With all the changes in the module system, the libq library is incompatible
with earlier releases, so you will have to recompile your own modules.
Please note that there are some minor changes in the libq interface (see the
ChangeLog for details), so you'll probably have to modify your module
sources accordingly. Sorry for breaking backward compatibility once again,
but these changes were absolutely necessary to make the new module system
work.
- Compiler: The meaning of the -w compiler option has changed (it now turns
those annoying warnings *on*, not off); also, the warnings about overlapping
rules have been improved.
- Interpreter: Now includes readline 4.2. Various improvements are: useable
--gc option, bug fixes in the run command, and improved rule printing in the
debugger. See the ChangeLog and the manual for details.
- Library: The lambda.q script has been overhauled again, and graphics.q is no
longer part of std.q, so you have to include this script explicitly.
The currently available modules have all been integrated into the main
package and are now installed along with the standard library. As a bonus,
there is a new interface module for IBM's OpenDX data visualization software
(UNIX/X11-based systems only). Since OpenDX is freely available under an
open source license, this should be interesting news for researchers
developing scientific applications using a Q/Octave combo.
New features of compiler and interpreter:
- Improved tail recursion optimization: Toplevel sequences (||) in rule rhs's
and qualifiers as well as variable definitions are now compiled to bytecode
employing direct manipulation of the evaluation stack. This has the
important consequence that basic imperative-style looping constructs like
loop X = foo X || loop X if bar X;
= () otherwise;
are now executed in constant stack space, as one might reasonably expect.
- Exception handling: The new `catch' and `throw' builtins implement the
usual catch/throw exception handling paradigm. This also allows you to
handle runtime errors like memory or stack overflow. Furthermore, the `fail'
function allows you to cause a rule to fail while it is already being
executed.
- Rule priorities: You can now explicitly specify rule priorities with the @N
directive.
------------------------------------------------------------------------------
* 2.1 Dec 2000 "Q 2001" ***10th Q ANNIVERSARY***
If you look further down in this file, you'll notice that it is Q's 10th
anniversary (well, almost), so you can reasonably expect something
special. ;-) So here goes ...
I decided that it was about time to release a polished version of the Q
programming systems, which brings the language and the standard library out of
its notorious beta status. With this "Q 2001" release, the core system,
including the language definition itself, has been dubbed "stable." This means
that future releases will now enforce backward compatibility, and concentrate
on additional scripts and modules as well as improvements and new developments
in the interpreter (such as multi-threading, which is planned for release 3.0,
but don't hold your breath yet ;-).
Please note that the directory layout has changed in the present release, so
make sure that you ***completely remove any existing Q installation*** before
installing the new release.
Q 2001 features a bunch of small changes and additions, too many to list them
here, so please check the documentation for details. The most important
user-visible changes are pointed out below.
*** BUG FIXES ***
Some more (obscure) bugs were fixed:
- C interface (wrong reference counting; missing argument checks; the sym
macro now creates new symbols if they do not exist).
- Rare segfaults in the lexical analyzers and the interpreter's main loop
reader.
- Several compiler bugs, which allowed mismatching declarations of variable
symbols, and equations with a non-symbol constant as the head of an
application on the left-hand side of a rule. Moreover, free variable symbols
on the left-hand side of an equation now produce an error message.
- Invoking the interpreter from Qpad now also works under Windows
2000/NT. Furthermore, the builtin `run' command has been rewritten to avoid
the overhead of an execvp() call, which has the nice side-effect that it works
now in Qpad too.
*** LANGUAGE ***
A lot of smaller fixes here, some of which also affect the library, rendering
some old stuff obsolete.
- The new Char type is the subtype of String which denotes the single-
character strings.
- Builtin relational ops now also apply to "enumeration types" (types solely
consisting of nullary constant symbols), and logical operators also work on
integers (doing bitwise logical operations).
- New syntax for 1-tuples, operator sections, and quote/force/splice
operators.
- Character escapes in strings can now also use the \0xhh and \0ooo syntax to
denote character codes in hexadecimal and octal.
- New builtin functions (fclose, pred, succ, time, sleep) and some minor
changes in existing builtins (ord, isconst, isfun). In particular, the
`random' function now returns 32 bit random _integers_ instead of floating
point values, and uses a new algorithm ("Mersenne Twister" by Makoto Matsumoto
and Takuji Nishimura, see random.c).
*** INTERPRETER ***
You will find some new command line options, as well as new and improved
commands and an overhauled command syntax (more "shell-like" for the "special"
commands). In particular, I'd like to point you to the new `stats' command,
which is quite useful for profiling purposes. Furthermore, the `break' command
now allows you to control the interpreter's behaviour in response to
exceptions (Ctl-C, break function, invalid qualifier). Again, please refer to
the language manual (Appendix "Using Q") for details.
A lot of small improvements have been made in the interpreter code, which will
mostly be invisible to the user. The installation procedure has been
"modernized" to separate architecture-dependent and -independent files. You
will now find the library scripts and some other useful items in
/share/q, whereas the architecture-dependent stuff like external
modules goes to /lib/q.
*** C INTERFACE ***
You'll also find some new features in the C interface (global module
initialization and finalization, access to GMP mpz values). The new `qcc'
utility should make it easier to translate external modules in a
system-independent manner. I tested this with Linux and Windows, other systems
may require some work. If you have any patches to make qcc (and the C
interface/shared library support in general) work on your system please let me
know. (Future versions should probably use libtool, in order to provide dlopen
support for a greater variety of systems.)
*** LIBRARY ***
Some old stuff which is now provided as builtins has been removed, other parts
have been polished, and a few new functions have been added. The special.q
script is now named cond.q, and contains only the conditional expression
constructs (the other stuff which used to be in special.q, namely
defer/force/splice, has been superseded by the builtin quote/force/splice
operators). Some experimental changes in the 2.0 release have been sorted out;
in particular, cons, push, pop, top, hd and tl have reverted to their previous
(<2.0) behaviour -- cons/push/pop/top still apply to both lists and tuples,
though. Some other trivial little changes: `zip' now takes two arguments
instead of one (to improve compatibility with that other well-known functional
language ;-); the operations in sort.q now take an order predicate as their
first argument (very useful); and, for cosmetic purposes, `isempty' and
`ismember' have been renamed to `null' and `member' throughout, and the
`genlist' and `genstream' operations are now called `while' and `iterate'.
Last not least, substantial progress has been made with the lambda.q script
which now provides pattern-matching lambda abstractions (GOOD STUFF!). Thus
expressions like `lambda (foo a (b,c)) (bar a c)' can now be used to match the
function argument against a pattern and bind variables accordingly. Since list
and stream comprehensions are implemented in terms of lambda, you get pattern
bindings there as well. The bad news: since `(x,y,z)' is now a pattern and
*not* a variable "list," multi-argument lambdas must now be denoted using
nested lambda expressions like `lambda x (lambda y ...)', so you probably have
to rewrite existing scripts to make them work with the new lambda
construct. To compensate for the porting hassle, you get some additional
goodies, namely lambdas and comprehensions with special arguments, a new
pattern-matching conditional, and a `let' special form which lets you bind
local variables in an expression.
*** MISC ***
Q emacs mode (q-mode.el) now supports auto-indent and filling. As far as I'm
concerned, this one is finished, except for bug fixes (no more elisp hacking,
please ;-). If anyone wants to improve it, just go ahead and send me the
results.
Some useful external modules are finally available from the Q homepage. At the
time of this writing, there are interfaces to Tcl/Tk (John Ousterhout's GUI
toolkit), GNU Octave (J.W. Eaton's GPL'ed MATLAB clone), and "clib", my first
attempt at a comprehensive system interface. Clib will also speed up -- by
orders of magnitude -- the most important list and string processing
operations from the standard library, so make sure you take a look at this
one.
Last not least, the docs have been updated, so the distribution now includes
the 6th edition of "The Q Programming Language."
--
Given the time I'll need to do the final touches and make all this stuff
available officially, it'll probably be the end of December before "Q 2001"
actually becomes available. So Merry Xmas and a Good New Year to everybody!
------------------------------------------------------------------------------
* 2.0 May 2000
This is a MAJOR release with some important new features as well as many bug
fixes. The most relevant changes are:
- improved tuple construct (modified syntax; C vector implementation)
- explicit short-circut logical connectives (no --complete anymore)
- separate concatenation operator ++
- integers of arbitrary sizes (using the GNU MP package; *GOOD STUFF!*)
- much improved interpreter (beginnings of a real command language)
- interface to the C language (via shared libraries)
I must admit that the first three items necessarily break backward
compatibility. But I think that these changes in the Q language were really
necessary to eliminate some (final?) notorious misfeatures, and that they are
worth the porting effort anyhow. A summary of all changes can be found below
(please refer to man page and info file for more details).
- COMMAND LINE INTERFACE: The main interface to the Q programming system now
is the "q" program, which incorporates the interpreter (formerly separate "qi"
program). You can still run compiler and interpreter separately, but the
interpreter can now automagically invoke the compiler when a source script is
given. Both compiler and interpreter can at last be invoked without giving a
source script, meaning of course that an empty script will be used. By
default, however, the compiler will always include the definitions in the
prelude.q script, which enables you to set up an environment which always
includes certain non-builtin functions and variables. You can disable this
with the --no-prelude option (meaning that only the built-in stuff is
available), and you can specify an alternative prelude name with the --prelude
option.
You can now also run a script directly from the interpreter, using the new
"run" command. A number of new command line options is provided and some
options have been renamed (see man page or info file). The environment strings
QPS and QFS have been removed; these options can now be set using command line
options as well as corresponding interpreter commands. It is now possible to
specify commands and command files on the command line using the new -c and -s
options of the interpreter. Finally, the filename parsing/construction code of
the compiler and interpreter now properly handles volume prefixes and ~
(homedir) notation (this requires that the HOME environment variable is set).
- INTERPRETER: Some nasty bugs in the Q machine (segfaults when the expression
stack was resized during an evaluation; built-in constants being considered as
"reducible") have been fixed. A memory leak in the garbage collector has been
plugged as well. Error handling in the interpreter has been improved (it now
prints the offending line and indicates the position of the error).
The interpreter's automatic garbage collection, which previously was invoked
any time an evaluation was finished, has been disabled, as it was a serious
memory hog (to implement it, the interpreter had to allocate a free expression
block for each one actually in use!). Note that this doesn't affect operation
of the interpreter, it only prevents unused expression memory from being
returned to the system pool until the interpreter is restarted. If you really
liked it the way it was and you have plenty of memory to waste, you can still
build the interpreter with the "automatic gc" option by configuring with
--enable-gc. This also enables an (undocumented) --no-gc interpreter option
for disabling this feature at runtime.
The interpreter now supports GNU readline, which provides command line editing
and history as well as symbol and filename completion when the interpreter is
run interactively. Furthermore, a new built-in "$" variable can be used to
refer to the result of the last expression evaluation, and the interpreter now
understands a number of commands which allow you to inspect and adjust various
parameters, edit and run scripts and command source files, read online info,
load and save variables, etc.
Multi-line input mode is not supported any more. Instead, the line end can now
be escaped using the `\' character, as it is in most other script
languages. Furthermore, you can now put multiple expressions/commands on a
single line, like so:
def X = 99; foo X; bar X
Multiple defs/undefs can be put into a single command:
def X = 99, Y = 2*X; undef A, B, C
Both compiler and interpreter now support C-like hexadecimal and octal
notations for integers, and the interpreter also lets you specify an integer
output format, s.t. integers can also be printed in octal or hexadecimal if
this is desired. Three different formats ("standard", "scientific" and
"fixed", each with a given precision) are available for printing floating
point numbers.
The startup and termination procedure of the interpreter has gone through a
major overhaul. The "environment file" stuff is gone. Instead you can now set
up your environment in a flexible manner by putting arbitrary interpreter
commands in the ~/.qinitrc startup and the ~/.qexitrc termination files which
are sourced when the interpreter starts up resp. exits in interactive mode
(see the sample qinitrc/qexitrc files in the lib directory).
The debugger has also been polished a bit, and some bugs have been fixed. It
supports some new and improved commands, and now also shows the qualifier part
of an equation. You can now walk around on the reduction stack and apply the
step-over command to an arbitrary stacked rule, which facilitates debugging
quite a bit. Moreover, you can either print only the filenames or full
pathnames of scripts.
Built-in variables ($, INPUT, ARGS etc.) are now readonly and hence cannot be
re- or undefined.
Six new built-in functions are provided: list and tuple, which convert between
lists and tuples; sub, which extracts subsequences from strings, lists and
tuples; version and sysinfo, which return as strings the interpreter's version
number and the configure-generated host system description, respectively; and
atan2, a two-argument version of the arcus tangent function. The built-in
exponentiation operator now also allows the first operand to be negative, but
only if the second operand is an integer.
Interpreter and compiler now implement integers of arbitrary sizes, using the
GNU multiprecision (GMP) package. Thus things like `fac 100' and the "stream
of powers" (see the `fac.q' and `powers.q' examples) will now finally work as
expected. :-) Multiprecision floating point numbers with IEEE-style rounding
will probably follow in a future release, as soon as they are fully supported
by GMP.
- Q MODE: The emacs Q mode has been rewritten almost completely to support the
new interpreter features. It also provides for better synchronization with the
inferior interpreter process (via gnuserv/gnuclient), and incorporates a lot
of other small improvements. See the Q info file for details.
The Qpad Windows GUI frontend to the interpreter has been updated to the new
version as well; as usual, you can find it on the Q homepage.
- LANGUAGE: The Q language now supports \ to escape line ends in a string,
integers of arbitrary sizes, hexadecimal and octal notations for integers, and
multiple def's/undef's in a single statement (as in the interpreter).
The new `extern' keyword allows you to declare both functions and (abstract)
types as "external," providing an interface to functions written in the C
language (see "C INTERFACE" below).
Cross-checking of symbol declarations is now more strict. In particular, the
number of arguments must also match in a non-special declaration.
It is now possible to specify which initial arguments of a special form should
be evaluated. E.g., you can use a declaration like
special foo ~X Y;
to indicate that foo has two special args, the first of which should be
evaluated. Thus it is no longer necessary to fiddle with `apply' and
"auxiliary" function symbols to achieve the same effect.
Tuples now work in the same fashion as lists, using the binary constructor
(|). That is,
(X1,...,Xn) = (X1|(X2|...(Xn|()))).
This also implies that the 1-tuple (X|()) is now different from the singleton
expression X. I know that this breaks backward compatibility in some cases (it
also affects some parts of the standard library), and that the (X|()) syntax
for singleton tuples is ugly. But these changes are necessary to avoid some of
the quirks with the pre-2.0 tuple construct (in particular, you can now have
nested tuples as the last element of a tuple). To compensate for the porting
hassle, the interpreter now implements tuples as C vectors (whenever
possible), thus providing constant-time indexing and a much more
space-efficient representation of sequences than can be achieved with lists
(at the cost of slower matching and construction with large tuples, however;
nothing is perfect ;-). Furthermore, concatenation, subscripting and the size
operator now all work on tuples as expected, and the new built-in "Tuple" type
can be used to match tuples of arbitrary sizes. There also is a new built-in
sub function for extracting subsequences of strings, lists and tuples.
Concatenation is now has its own operator symbol (++). I felt that this was
necessary, in order to allow overloading of the (+) operator, e.g., for
componentwise addition of lists and tuples. (This is already used in the new
complex.q module, and will also turn out useful when implementing vector and
matrix operations which might be added to the library in the future.)
The --complete compiler option is gone, and the logical connectives (and) and
(or) now evaluate their arguments completely (and require that both arguments
evaluate to truth values). The short-circuit connectives are now called (and
then) and (or else). So finally it is now possible to use both the "standard"
and the short-circuit connectives in a single script, and the standard
connectives now work in a manner consistent with the arithmetic
operators.
As already mentioned, the last three items above unfortunately break backward
compatibility, and hence existing code will have to be rewritten, as has been
done for the standard library.
- C INTERFACE: A C interface via dll's resp. shared libs is now provided both
for Windows and Unix systems supporting SUN's dlopen interface (SUN, Linux,
maybe others?). The interpreter can load C functions from shared libraries at
runtime, and provides an interface which lets such functions access and create
Q expressions. External data types are supported as well. Please refer to the
info file for details.
- LIBRARY: Unfortunately, some of the language changes discussed above have
had quite some impact on the standard library; I took the opportunity to
straighten a few other parts of the library as well. Some functions related to
pairs are gone, and the quoted app/list/tuple constructors have been moved as
private functions to special.q, which is the only place where they are
actually needed. The value function has been moved to special.q as well, and
the cons function has been removed from stdlib.q (use push
instead). Lexicographic list comparison is now part of stdlib.q, and stdlib.q
now also provides comparison of truth values; the compare.q script has been
removed. The lambda.q script has been overhauled once more, to prevent the
function argument of a saturated _B or _S combinator in a special argument to
be evaluated more than once, and to take care of embedded list and tuple
constructors; I hope it finally works reliably now.
Some changes have been made to the list and stream operations in stdlib.q and
stream.q:
+ there is a new tuplecat operation in stdlib.q which concatenates a list of
tuples
+ for compatibility with well-known languages like Haskell and MATLAB, each
and exists have been renamed to all and any
+ the complete set of list functions (including push, reverse, etc.) is now
available for streams as well
+ push now takes the element to be prepended as its first argument, and it now
also works with tuples; pop, top, hd and tl work with tuples as well; a
prepend function has been added which prepends an element given as second
argument to a list given as first argument
+ new isempty predicates for lists, tuples and streams
+ the genlist function in stdlib.q now takes the predicate as the first
argument; the range function is now called nums and can generate lists of both
integers and floating point numbers; moreover, a numsby function has been
added which allows you to specify the step size; similarly, the stream.q ints
function has been replaced by numstream/numstreamby
+ the streamadd function has been renamed to streamcat2
+ a list/stream transposition function has been added (works only with
rectangular lists/streams, i.e., if all component lists/streams have the same
size)
The other user-visible changes in the library are:
+ a general purpose conditional function "switch" has been added to special.q
+ the constructor symbols of the stdtypes family (array.q, heap.q, set.q,
bag.q, dict.q) are now public (looks less bewildering when printing such
objects)
+ the istuple function has been added to type.q, and the pseudo type
predicates isapp/iscons/ispair have been removed from that module
+ the inverse hyperbolic functions have been added to math.q, and the fmod
function has been removed; the constants e and pi have been removed as well,
as they can now easily be defined in the qinitrc file
+ a complex.q module has been added, which implements complex numbers as pairs
of integer and floating point numbers; this still needs some testing, but I
hope you'll like it ;-)
+ the std.q script now also includes graphics.q
+ the FLUSHPAGE variable has been removed from graphics.q; moreover, the
GRAPHICS variable is not assigned a value any more, so that you *always* have
to define this variable before you can use any of the output operations
+ plain.q is gone, instead there is a new script named prelude.q, which is by
default included whenever a script is compiled or run (see also "COMMAND LINE
INTERFACE" above); as distributed, this script now includes std.q so that all
standard library functions are normally available in the interpreter, without
having to include them explicitly
- DOCUMENTATION: 5th edition of "The Q Programming Language", also updated man
page and README file
------------------------------------------------------------------------------
* 1.9 Jan 1999
This is mainly a maintenance release and should be 100% backward-compatible to
Version 1.8 (except for the discontinued Symantec C++ support, see below).
Bug fixes:
- corrected a =/== typo in path searching code (qbase.c) [thanks to Winfried
Szukalski ]
- qualified identifiers are now recognized correctly in the interpreter
(qmlex.l)
- interpreter in interactive mode now flushes its output and the integrated
debugger runs correctly even if stdin is not a tty (qm.c, qmfuns.c)
- SIGTERM is now handled to shut down compiler and interpreter in a clean
manner (q.c, qc.y, qi.c)
Changes:
- Additional built-in flush and fflush functions.
- Additional "b" (binary flag) for the file mode in fopen and popen. This flag
suppresses the LF/CR-LF conversion on MS-DOS/Windows system, which is
essential if the file is to be used as a binary file. On UNIX systems, this
flag is ignored.
- Added device settings in graphics.q for Windows; uncomment these (and
comment out the corresponding UNIX definitions) when running under Windows.
- Cleaned things up a bit, revived support for DJGPP, and added support for MS
Visual C++. The support for Symantec C++ has been dropped, since I am no
longer able to maintain it.
- Added support for synchronizing with the parent process under MS Visual
C++. The system interface (sys.{h,c}) now contains some MSC-specific code to
implement a safe simulation of SIGINT and SIGTERM on Windows95/98/NT, which is
used by the Qpad application (see below).
- Updated documentation (fourth edition of "The Q Programming Language").
BIG NEWS:
A new GUI frontend to the Q compiler and interpreter is now available for
Windows95/98/NT -- check out the qpad application on the Q homepage.
(Actually, I cannot test whether this program runs under NT, so any feedback
of NT users is appreciated.)
------------------------------------------------------------------------------
* 1.8 Apr 1997
Yes, Q is still alive. ;-) This version fixes some things (mostly cosmetic
changes).
- Changes to the distribution:
The Makefile now also includes a `html' target for creating documentation in
HTML format. (Requires that you have the texi2html script.) The `dist' target
now requires that you have gzip to build a tar.gz file.
The Emacs Q mode (`q-mode.el') has been improved considerably. It now supports
syntactical fontification (using the font-lock mode) and commands for locating
the source lines referred to in the interpreter window. The latter feature
also depends on the improvements in the 1.8 debugger (see below).
- Changes to the programming tools:
The notation `#!

' can now be used to include compiler and interpreter
options at the beginning of the main script. Such lines will be processed by
the driver program and the compiler (but not by the interpreter, as it never
sees the source file; you must use the driver program to pass on such options
to the interpreter).
The Q interpreter now allows to define function symbols (in addition to
variable symbols), provided that the symbol does not denote a built-in
operation or is in use by the loaded script. Such symbols are effectively
treated as variable symbols. Thus it is now possible to write something like:
==> def foo = lambda X (2*X+1)
==> foo 7
15
which is convenient if the variable is used to denote a function. Note that
the compiler still enforces that all symbols introduced with `var' or `def'
are ``real'' variable symbols.
The debugger now also prints, for each rule, the source file name and line
number. The bytecode format has been extended to support this. The debugger
also displays the total number of stacked rules (including the current
rule). The stack indices (`NNN>') now range from 0 (current rule) to stack
size minus 1 (first rule). Thus the indices in the stack printout will always
read 0>, 1>, 2>, ..., from bottom to top. The `p' command now supports an
optional integer argument which specifies the maximum number of stacked rules
to be printed (the default is 6, which used to be the hard-coded number in
previous releases). The limit imposed on the size of printed expressions
(default: 66) can now be controlled with the new `.' command.
- Language changes:
Added a `float' builtin (previously in the standard library). Although this
operation is simple to implement using an equation, I felt that it should be
part of the built-in conversion functions.
The built-in support for operator sections, which has always been rather
experimental, has been removed from the language (but see stdlib.q for a
non-built-in replacement). I also removed the ugly `\' notation for the binary
minus operator which I disliked from the beginning. Binary minus is now simply
denoted as `(-)', as one would expect, and the unary minus operation by the
built-in function symbol `minus'.
Line-oriented comments in the format `// ...' are now also supported, in
addition to the PROLOG-style `% ...' format. This will make all C++
programmers happy.
The `const' modifier can now be used for arbitrary function symbols (not just
for parameterless function symbols). The compiler will enforce that no such
symbol is used as the leftmost term on the left-hand side of an equation. Thus
it is now possible to explicitly declare ``pure'' constructor symbols, e.g.,
as follows:
type BinTree = const niltree, bintree X T1 T2;
bintree X T1 T2 = ...; // forbidden, compiler issues error message
This also works with special symbols, e.g.:
special const binstream X Xs;
Also, the built-in special constructor `quote' is now declared as const.
- Changes to the library:
Wherever possible, constructors are now declared as const. Also, there is a
little change to the Dict type: to retrieve the value associated with a key,
you must now use the subscript operator ! (used to be application before, but
this prohibits the bindict constructor to be declared as const).
The `std.q' script now includes *all* standard library scripts except
`graphics.q'; this makes the `stdm.q' script obsolete, and thus this script
has been removed from the library. Furthermore, the `lex.q' script has been
renamed to `compare.q'.
The `streamp' function (`stream.q') has been renamed to `streamadd' and a
function `eval' has been added to `special.q' which provides a convenient
shortcut for the combination of `value' and `defer'.
The `take' and `takewhile' operations in `stream.q' now return streams (not
lists, which was problematic since it compromised the laziness of the stream
data structure). Use the `list' function to explicitly convert the results of
these operations to lists.
The `lambda.q' script has been overhauled. Some terribly-looking kludges have
been rewritten without changing the interface. I believe that it is much more
readable now.
The graphics device (`GRAPHICS' variable in `graphics.q') is now initialized
to the null device (used to always start up ghostscript, which was annoying if
the user or calling script immediately redefined the device). Therefore you
should initialize this variable as appropriate for your application. Some
standard kinds of devices are now provided by `graphics.q' (`nulldev',
`filedev', `lpdev', `gsdev', `gvdev').
- Documentation:
Third edition of "The Q Programming Language."
------------------------------------------------------------------------------
* 1.7 Feb 1994
New built-ins: isconst, isfun, isvar and isdef. With the popen function
you can now pipe data into a shell command and read the output of such a
command. The ERROR variable has been added as a means to access the
standard error device.
Language changes: explicit declaration of free variable symbols (new
`var' keyword). In particular, this allows you to introduce private
variables. Variables can now also be initialized with def from within a
Q script.
Added graphics.q, a simple PostScript graphics interface, to the standard
library.
The Q interpreter now checks whether it is connected to a terminal in
order to prevent garbled output when input or output is redirected.
All programming tools now also support long option names following the GNU
coding standards.
Added the elisp program q-mode.el which implements a simple mode for editing
and running Q scripts in GNU emacs.
Second edition of "The Q Programming Language."
------------------------------------------------------------------------------
* 1.6 Nov 1993
Allow multiple declarations of overloaded symbols.
Removed the synonyms `nil' and `void' for the empty list and tuple (one
of Klaus Barthelmann's favourite changes).
Eliminated another notorious misfeature, the predefined ordering of
function symbols.
Fixed library, examples and documentation.
------------------------------------------------------------------------------
* 1.5 Oct 1993
Added type guards which allow to verify that a variable is matched to a
value of the proper type. Removed some obsolete built-ins.
Added stdtypes.q, a collection of container data structures (arrays,
heaps, sets, bags and dictionaries), to the standard library.
Major overhaul of docs, standard library and examples.
------------------------------------------------------------------------------
* 1.4 Jan 1993
Straightened out language syntax and semantics, in particular the
treatment of special forms.
------------------------------------------------------------------------------
* 1.3 Sep 1992
Major language revision (curried function applications, tuples, quoted
expressions, lazy evaluation, include files, private symbols). First
edition of "The Q Programming Language."
------------------------------------------------------------------------------
* 1.2 Mar 1992
First UNIX (and C) version.
------------------------------------------------------------------------------
* 1.1 Mar 1991
First prototype of the Q programming system running under MS-DOS.
(Implemented in Turbo Pascal. ;-)