Berkeley DB XML 2.3.10 Change Log

Changes since Release 2.3.8 are described here. The change log
of release 2.3.8 is also included, detailing the changes
from 2.2.13.

Release 2.3.10 is a patch release for 2.3.8. For detailed information
about the 2.3 release itself, see the 2.3.8 Change Log.

It recommended that all users move to this release. Also see the
documentation included in your download package or on our
website.

2.3 Release Overview

The major focus areas for release 2.3 include:

Conformance to W3C Recommendations -- XQuery 1.0 and
XPath 2.0.

Improved application integration by way of adding additional
interfaces for input and output of XML content, allowing
application-driven parsing and both push and pull content access
without serialization of XML.

Changes in BDB XML 2.3.10

Upgrade Requirements

Upgrade is required for containers created using release 2.2.13.
See the 2.3.8 Change Log. Containers creating
using 2.3.8 do not require upgrade.

General Functionality Changes

Fixed Windows-mostly problem where dynamic_cast (RTTI)
was being used by SWIG-generated code, and RTTI was not enabled in the
build. The fix was to use static casting, as it's safe in these cases, rather
than enabling costly RTTI. The symptom was a mysterious failure
(exception thrown from the C++ runtime library) in Java when
instantiating a class that extends either XmlInputStream or XmlResolver
[#15280].

Fixed a problem where some code that should have been conditionalized
based on the Berkeley DB version was not. The result is failed compilation of
CompactCommand.cpp [#15283].

Fixed a bug where namespaces and global variables defined in the XmlQueryContext were not carried forward to the context of a module [#15277].

Fixed a bug in XmlModify where an assertion failure (debug mode)
was triggered when removing element nodes in a certain sequence [#15291].

Fixed python build problems in 2.3 which caused the setup.py script to be
a no-op on Unix, and even when fixed, there was a compilation problem, again Unix-specific [#15295].

Fix metadata indexing so that a default index on metadata will work [#15300].

Fixed dbxml shell to not reset the base uri in its default context.
This interfered with arguments to fn:collection() [#15299].

Fixed a race condition in initializing function signatures. This only manifested when queries were parsed in parallel [#15298].

Fixed an optimisation bug that meant that using an XQuery module would sometimes cause an assert to be triggered [#15290].

Fixed a bug where namespaces and global variables defined in the XmlQueryContext were not carried forward to the context of a module [#15277].

Fixed a bug in XmlModify where some incorrect logic could lead
to an incorrect query, assertion failure or SEGV. This bug would only occur
after an XmlModify step was performed. It never occurs on the first
modification step [#15321].

Added code to detect, and throw an exception for document/container
mismatches, where a document obtained from one container is used for an
update or delete operation on another container [#15320].

Fixed a bug that caused incorrect query plans to be generated for predicates that used the "or" operator in conjunction with indices [#15328].

FIxed a problem where writing an attribute with an empty value
resulted in a null pointer reference (SEGV). This could happen via
XmlEventWriter, which was used by the 2.3 upgrade code [#15335].

Fixed an XmlModify problem in 2.3.8 where inserting a new element
with content would result in an exception thrown indicating that an
operation was attempted on an invalid context [#15333].

Added -fno-strict-aliasing flag to Java library build if using gcc to avoid
over-optimization of some constructs that SWIG uses for casting pointers to/from jlong. The symptom was an exception thrown indicating that there is a null or empty value [#15307].

Fixed a bug that resulted in the starts-with() function always using a substring index, without considering using a prefix operation on equality index [#15337].

Fixed an optimisation problem that caused document-uri() to fail
under certain conditions [#15336].

getFlags() -- return the flags field used to create the manager [#14403].

truncateContainer() and compactContainer() [#14163].

XmlManager::get/setImplicitTimezone() -- set/get the implicit
timezone for queries where it is not explicit. It is necessary to set this before
doing anything with indices related to data or time types.

XmlContainer:

XmlValue getNode() -- enables loading of a previously-dumped
opaque (string) handle pointing directly to a node within
a document in the container, dumped by XmlValue::getNodeHandle().

getFlags() -- return the flags field used to create/open the container [#14403].

Added XmlContainer::putDocument() method that takes
an XmlEventReader object as content.

Added XmlContainer::putDocumentAsEventWriter() method that
allows the application to write content via an XmlEventWriter.

Changed XmlModify to allow a variant that does not require an
initial context of XmlValue or XmlResult. If there is only a single step, and
the XmlValue object passed to XmlModify::execute() is empty (not
initialized to any value), the query in that step will serve as the context for
the modification [#14774].

Added variants of the XmlModify steps that allow specification of
content as XmlResults object. This change also enables new
content to comprise a sequence of elements, rather than be a single
element [#14533].

getContentAsEventWriter(XmlEventWriter &writer) --
causes the system to write the content as events to
the provided XmlEventWriter object.

Added XmlDocument::equals method to non-C++ APIs [#15212].

XmlException:

Added XmlException::getQueryFile, XmlException::getQueryLine() and
XmlExceptions::getQueryColumn() to provide location information
for XQuery errors, improving error reporting and making debugging queries
easier. The reporting of this information is language binding-dependent,
and may vary among the support APIs [#13465].

Added DBXML_WELL_FORMED_ONLY flag to use to force the
use of the well-formed scanner for documents. This scanner will make no
attempt to retrieve schema or DTDs, even if referenced. The Java
equivalent for this is XmlDocumentConfig.setWellFormedOnly() [#14055].

Changes That May Require Application Modification:

Fixed a bug in the choice of index used for comparisons. Users with decimal indices may find that they need to change them to double indices before they will work correctly [#15093].

Changed the default base URI to be "dbxml:/" (from "dbxml:"), so that is
compliant with RFC 2396 (URI specification). Containers that have been
opened using an absolute unix path may now have to use aliases to
reference them. See the URI rule changes below [#13881].

Changed how BDB XML URIs are resolved against a base URI to be conformant
with the URI specification. This affects naming of containers in
the arguments to fn:collection() and fn:doc(). Here are the current
rules. Notes: by default, there is a base URI of "dbxml:/",
and "container_alias" is either a relative path to a container, or
an alias for that container, set using XmlContainer::addAlias().

Replaced the Pathan and XQuery libraries with a single library
called XQilla, which implements both XPath 2.0 and XQuery 1.0.
This simplifies the build and removes redundant code [#13880].

General Functionality Changes:

Added several new XQuery extension functions that expose access to
BDB XML functionality, as well as documentation for BDB XML
XQuery extensions. New extensions are:

dbxml:lookup-index()

dbxml:lookup-attribute-index()

dbxml:lookup-metadata-index()

dbxml:handle-to-node()

dbxml:node-to-handle()

Fixed a bug where the default namespace for an element in no
namespace was not redeclared, even though the element was in the middle of
a document that already had a default namespace declared. [#13872]

Fixed a bug where a variable without a
prefix was using the default namespace from the statically known
namespaces [#13868]

Fixed bug where an attempt to open a transactional container that
fails due to a version mismatch would result in errors from
Berkeley DB regarding
"Locker not found" and eventually an environment PANIC, requiring recovery.
The same issue could arise if the open were to fail for other reasons as well [#13962].

Fixed a bug where using XmlContainer::updateDocument() on a document
returned from a query that used lazy evaluation could result in an
assertion failure [#13933].

Fixed a bug where index entries were incorrectly deleted when
updating a document either via XmlModify or updateDocument.
The problem only occurred for containers without
node indices and when there were multiple, identical index entries being
added/deleted during the update [#14173].

Fixed a problem where the Java XmlInputStreamWrap class, which implements
java.io.InputStream did not honor the InputStream contract to return -1 at EOF.
Instead it returned 0 [#14216].

Fixed a number of places where database errors, such as DB_ERR_DEADLOCK,
were getting lost and not properly passed to the caller [#14311][#14212]
[#14743][#15087].

Fixed a bug where errors from dbxml_dump were masked, silently resulting
in partial dump files if the container had any corruption [#14388].

Changed dbxml_load so that it will not attempt to load into an existing
container, as doing so may corrupt the container [#14381].

Fixed various problems in example code [#14392].

Fixed double-delete crash that could occur when using XmlModify
to remove and add attributes on the same node, when the node originally has
more than one attribute [#14503].

Fixed a problem where leading whitespace in content passed to
XmlModify steps could result in exceptions or bad updates [#14629].

Fix a situation where memory could leak if an application
were to use another library that includes the Xerces-C library in conjunction
with BDB XML [#14451].

Fixed bug where node storage containers could mangle DOCYTPE
declarations using PUBLIC [#14725].

Fixed a bug in date/time comparisons [#14949].

Changed index syntax names to match the documentation, as well as be
case-insensitive. The old (incorrect) names (for example, year vs gYear)
still work [#14961].

Modified index storage to be more efficient for non-string types, and
fixed bugs in the ordering of index entries with certain types (for
example, boolean). Also removed index types of QName and NOTATION,
because namespace bindings are not available during indexing, and
removed index type of anyURI, because all comparisons on anyURI values
are done after casting to the string type. Upgrade will automatically
replace these types with an equivalent string index [#14371].

Changed document ids to be 64-bit (were 32) [#15069].

Fixed a bug in fn:lowercase() and fn:uppercase() where they did not handle
Unicode characters well [#14382].

Used support in Berkeley DB 4.5 to fully support CDS in BDB XML (deadlock-free,
single-writer, multiple-reader concurrency, not transactional). [#14568].

Fixed XmlResults::next/previous iteration to be consistent with
the semantics of the underlying object. The previous() call would not move
the cursor if it was positioned on the last value in the result set [#15145].

Fixed a problem where adding indices to an encrypted container would not
encrypt the new index databases [#15253].

Added code to attempt to identify mismatched versions of Berkeley DB and BDB XML,
especially in non-C++ APIs such as Java, Python, and PHP.

Utility Changes:

Modified dbxml shell to use NULL as the default path for the DbEnv so
that the DB_HOME environment variable will be used [#15139].

Added commands to dbxml shell:

setIgnore -- causes it to ignore errors
while not in interactive mode, allowing scripts to continue running. It has no
effect on interactive sessions [#15150].

sync -- flushes the cache to the physical databases.

setQueryTimeout -- sets a query timeout in seconds for queries run in
the shell.

echo -- allows echoing of a string to stdout.

improved handling of ^C, which will now interrupt an in-progress query

Java-specific Functionality Changes:

Fixed a Java-specific problem where using
XmlDocument.setContentAsXmlInputStream()
could lead to a double-delete of the underlying native memory for the
XmlInputStream during finalization [#15126].

Fixed Java GettingStarted examples to use XmlContainer.delete() rather
than XmlContainer.close(), and call delete() methods on all objects
that need it to release container resources [#14668].

XmlContainer.close() is now mapped to XmlContainer.delete(), so it is no longer
possible to aggressively close the underlying databases without releasing
container references by explicitly deleting BDB XML objects.

Python-specific Functionality Changes:

Added an XmlException class and exception hierarchy.
It is described in the file, dbxml/src/python/README.exceptions [#13959].

Modified code generation to enable threads and correctly handle
acquisition and release of the Python Global Interpreter Lock (GIL).
If threads are not desired, it is possible to compile without them by
modifying the source of src/python/dbxml_python_wrap.cpp to disable
SWIG_PYTHON_THREADS. This can be done by defining SWIG_PYTHON_NO_THREADS
[#14077].

PHP-specific Functionality Changes:

Fixed PHP XmlManager->createContainer() and openContainer() methods to
use flags arguments passed in [#14617].

Brought PHP interface up to date with respect to the current API, adding
missing interfaces [#13889].

Added XmlException to PHP interface when compiled under PHP5, which
supports exceptions. This allows PHP5 scripts to catch XmlException [#15245].

Tcl-specific Functionality Changes:

None.

Configuration, Documentation, Portability and Build Changes:

Bundled version 4.5.20 of Berkeley DB, which includes Multi-Version
Concurrency Control (MVCC), allowing snapshot semantics for read-write
concurrency. This will be quite useful for concurrent BDB XML
applications (read/write concurrency).

XQuery and Pathan libraries have been replaced with a single library,
XQilla. This library is not currently available from any other location.

Added dependencies to example Windows project files [#13863]. This
showed up as a problem in parallel builds using Visual Studio .NET 2005.

Modified Unix configuration and build to allow the
BDB XML library (libdbxml*) to built in a directory other than
dbxml/build_unix. The buildall.sh script uses the default
build locations [#14772].

Consolidated all projects into the single, BDBXML_all.{dsw,sln} files
for Windows build, and included all Berkeley DB utilities [#14098].