News Feed

New features in darcs elephant-1.0 head

February 7th, 2009

Alex fixed bugs in the postmodern backend and added new
documentation.

Ian promoted a new CLP backend which is an all-lisp solution based
on cl-prevalence and cl-containers that is currently at Alpha quality.
It requires the latest patches to cl-containers and a Mercurial fork
of cl-prevalence which can be retrieved using 'hg clone
http://bitbucket.org/skypher/cl-prevalence/'

We are deprecating the CLSQL backend by moving it into contrib; it
will not be a part of the 1.0 release. You can still use it by
creating a symlink in the elephant root directory to the appropriate
ele-xx files in src/contrib/rread/db-clsql

Elephant 1.0 Alpha 2

Elephant 1.0 Alpha 1

January 3rd, 2009

Significant development work has been taking place on the darcs
elephant-unstable repository over the past year. The last stable
release (0.9.1) is quite stale, lacking many bug fixes and features
that are part of the upcoming 1.0 release.

We've decided to release a stable snapshot of the development tree
and recommend that all users engaged in active development switch to
this version of Elephant. This includes changing your target darcs
repository from elephant-unstable to elephant-1.0.

While the Alpha 1 snapshot has not undergone rigorous testing, it
is in daily use in a number of live applications and our test suites
are currently green for all 0.9.1 and many 1.0 features on all data
stores on SBCL/Linux. We invite users of other platforms to help
expand the scope of testing of the 1.0 tree to ensure that any
platform specific bugs are resolved prior to a final release.

The following new features have been introduced since our last
release, 0.9.1, in November 2007.

New slot types

Hierarchical indexed slots

Indexed slots can now index a base class slot as well as all subclasses inheriting that slot. Index inheritance can be inhibited by redefining the slot in a base class.

Virtual derived index slots

A better way to manage derived indexes (the old derived index model is deprecated). You can define a virtual slot that is used to create the index. A slot argument defines the function of slot values that populates the index. Hints are available that inhibit updates to only occur when certain slots are written.

Set-valued slots

A simple API to an unordered aggregate slot using the pset api for access

Associations

Many-to-1 and many-to-many associations between slots in two different classes. They also uses a variation of the pset api.

'Checkout' slot caching

A simple means for runtime switching between persistent and transient slot behavior with the ability to sync transient values back to the store. Can buy up to 100x in read/write throughput at the cost of automatic thread safety

The code base contains a working prototype of an online garbage
collector using a mark and sweep strategy. Testing and documentation
are not complete but you can experiment with it. Details are available
on the mailing list archive from Jan 5th, 2009. Complete functionality
will be available in the final release.

Query Language and Interpreter (prototype)

We also have a working sketch of a simple query language for
elephant (query.lisp) that uses indices and a preliminary sketch of a
query planner and interpreter (query2.lisp) that does something akin
to joins. A modestly optimized form of this will be available as part
of the final 1.0 release. Comments on the simple constraint language
will be appreciated.

There is still much to do to iron down the final release:

Associations still contain a few bugs, so we don't recommend depending on them yet.

Derived index slots haven't been extensively used or tested

Cached slots have not been extensively tested for corner cases, but do have some modest regression tests to validate their basic functionality and should be in good shape for single-threaded use

Schema evolution There are also some minor problems that can crop up occasionally with schema evolution, but there are almost always workarounds so if you run into trouble e-mail elephant-devel

Garbage collection has not been fully validated and should not be considered reliable

Support a full constraint language specification in the query interface

We need better test coverage (particularly new features)

Documentation of new features

Ian Eslick has primary responsibility for the features in this
release. Alex Mizrahi, Henrik Hjelte and Robert Read have put in
significant work upgrading and improving the SQL backends. Special
thanks to Leslie Polzer for various contributions and picking up some
of the support slack.

Elephant 0.9.1 now available

This release provides official support for using the postmodern
interface to Postgres as a back-end store. This back-end is about
three times faster than the generic SQL interface on top of Postgres.

Note that existing Postgres repositories are not compatible with a
Postmodern repository. However, Elephant supports multi-repository
migration. If you have an existing Postgres repository and want to
migrate to Postmodern, you should create a completely new Postmodern
repository and migrate all of your data into it. The migration can be
accomplished with a single command.

If you have been using the (unofficially released) postmodern backend,
you will have to migrate your data into a CL-SQL based postgres backend,
or into a BDB backend, and then migrate into your new postmodern
repository. This release is not compatible with previous (development)
versions of the postmodern functionality.

This release has been tested on a 64-bit Linux with SBCL 1.0.10, and
other platforms as well. If anybody downloads it and tests it, please
run the (large) set of automated tests and report your results to
elphant-devel at common-lisp.net. We will update the platforms page as we receive your updates.

Elephant 0.9 now available

Elephant 0.9 is a significant improvement in stability, platform
support and feature set over Elephant 0.6.0, our last major release.
This release should be considered a stable release, and any major bugs
that are uncovered will be accommodated by a patch release.

Major Features of Elephant 0.9

Expanded and improved platform support

SBCL 1.0, Allegro 8.0/7.0, Lispworks 5.0 and OpenMCL 1.0

Elephant has been verified on every major OS (Linux, Mac OS X or Windows) for which these lisps are available

Proper upgrades from 0.6, via the ability to open 0.6 databases in 0.9

The Berkeley DB backend is up-to-date with the latest library version and requires Berkeley DB 4.5

A host of performance improvements, bug fixes and minor feature enhancements (see the development list or prior announcements for some of the details)

Roadmap

We are planning a string of interim releases between 0.9 and 1.0, but
we recommend that people using Elephant for active web sites and
important applications use 0.9 until 1.0 is available. We will
not guarantee clean upgrades between 0.9.x releases if the database
format changes and cannot guarantee the same level of stability
outside these major release milestones.
The features planned for the 1.0 release of Elephant are recorded as tickets at
the Trac project
site.

Elephant 0.9 RC1 announced

April 30th, 2007

With the marked improvement in stability and functionality of the code
base, the maintainers have decided to abandon the 0.6.1 release label
and change the upcoming release to 0.9. This change puts us on a new
roadmap
leading to a 1.0 release later in 2007.

New features and improvements in release 0.9 RC1

Additional features integrated since the Beta 0.6.1 announcement:

Full platform support

SBCL 1.0, Allegro 8.0/7.0, Lispworks 5.0 and OpenMCL 1.0

All these lisps should work on every major OS (Linux, Mac OS X, Windows) they are released on

64-bit support has been validated, but not recently and not on all platforms (testers welcome!)

Mapping operators

BTrees, BTree indicies, class index and slot/derived indices all have a map operation

Enable mapping a function over values and ranges

Map in both ascending and descending order

Collect function results into a list

Efficient stack allocation where appropriate

Persistent sets

A lighter-weight API for persistent collections

Provides insert, remove, find, map and list operations

With appropriate methods, makes one to many slots easy to construct,
see tutorial and user-guide for details.

Complete update and rewrite of the users manual

All new and up-to-date tutorial

New section containing more detailed implementation information for advanced users

New sections for prospective data store developers

Detailed installation instructions (supercedes INSTALL file)

A list of use case scenarios that should help developers understand Elephant better

Much work remains to clean up the manual for 1.0, but the bulk of the rewrite is done
and we invite comment on what is missing or unclear.

Feature tweaks

Drop commands for persistent collections to reclaim storage for reuse.

Expanded support for nested aggregates in serializer and
migrate (i.e. an array of persistent objects will be
properly migrated, before only btree collections of
persistent objects were properly copied).

Download and Install

This will put the elephant release into the directory 'elephant'. Read the INSTALL, UPGRADE and UPGRADE-BDB files for further instructions.

Purpose of the beta release

The tree is effectively a release candidate. We only need to verify OpenMCL, beef up the documentation and examples and make sure we have good test coverage. Otherwise this code base appears stable enough to release.

New features in 0.6.1

The beta release is a more stable version of the feature set in the alpha release. A list of the changes between alpha and beta can be found in the doc/oldfiles/TODO file.

Alpha 0.6.1 Release available via CVS

February 20th, 2007

Supported platforms:

SBCL, Allegro, CMU, OpenMCL, Lispworks (build system is not automated for Windows)

CMU, OpenMCL and Lispworks have not been fully tested and may require minor bug fixes

Download and Install

This will put the elephant release into the directory: elephant. Read the INSTALL, UPGRADE and UPGRADE-BDB files for further instructions.

Purpose of the alpha release

The primary developers do not have ready access to all supported platforms and would like to ask the community to help validate the current implementation on other platforms. We also are using the alpha to improve up our test suite. For more details please see the TODO file. If you have a favorite feature such as multi-threading, 64-bit, etc., please free to submit a test for the test suite.

New features in 0.6.1

Simplified build and site configuration support:

See config.sexp in root directory for site customization (no more editing code files)

Linux and Mac systems should automatically build libraries when asdf is called

64-bit lisps are now supported

64-bit and 32-bit lisps can read and run off the same database files (on the same machine)

Multithreading

Elephant should now be thread safe, including sharing a single store-controller across threads

Read comments in src/elephant/transactions.lisp and BDB users should read src/db-bdb/bdb-transactions.lisp

Improved support for mixing transactions and store-controllers

Upgrading

Elephant 0.6.1 can open and directly manipulate 0.6.0 databases

Existing 0.6.0 databases can be upgraded via the 'upgrade' function which takes

*auto-commit* is no longer required. All data access methods auto-commit if there is no active transaction

store-controller accepts :deadlock-detect keyword (t or nil) which will run db_deadlock as a background process to abort deadlocked threads

optimize-storage is a new store-controller method currently supported by BDB backend. It compacts a whole database or only a specific BTree and returns free pages to the free list or to the file system.

Release 0.6.0 Available

May 27th, 2006

The latest version is 0.6.0, is offically released as of May 27, 2006. It has been tested on a variety of lisp implementations and with a variety of storage repositories. This release is mainly the work of Ian Eslick, and offers a significant refactoring as well as the very convenient feature of automatic slot indexing. Julian Stecklina submitted a patch for FreeBSD which is in the CVS repository now but not in the 0.6.0 release.