Oracle Berkeley DB Java Edition 11G R2 Change Log

Library Version 11.2.4.1, Release 4.1.27

Changes in 4.1.27

Made DbPreUpgrade_4_1 and DbRepPreUpgrade_4_1 more easily scriptable by exiting
with an appropriate status code. These now exit with status 1 if the utility
fails, and exit with status 0 if the utility succeeds or the environment has
already been upgraded to JE 5. [#21405] (4.1.22)

Fixed a bug that caused data corruption (for example, resulting in an
EnvironmentFailureException with a LOG_FILE_NOT_FOUND message) under certain
circumstances, when many Databases are created and not all Databases are kept
open. In fact two bugs were fixed that contributed to the problem. The first
bug was a memory size calculation error and caused heavy/costly cache eviction,
even when the cache would normally be large enough to avoid such eviction.
This calculation error occurred during log cleaning, and only when closed
databases had been previously evicted from the cache. The second bug caused
Btree corruption when cache eviction was heavy and many closed databases were
evicted. [#21686] (4.1.23)

Fixed a bug where the following exception could be thrown during heavy
concurrent write and read operations. This did not cause data loss and did not
invalidate the Environment instance.

com.sleepycat.je.EnvironmentFailureException:
(JE 5.0.34) com.sleepycat.je.utilint.RelatchRequiredException
UNEXPECTED_EXCEPTION: Unexpected internal Exception, may have side effects.
at com.sleepycat.je.EnvironmentFailureException.unexpectedException(EnvironmentFailureException.java:286)
at com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1268)
at com.sleepycat.je.Cursor.checkForInsertion(Cursor.java:3006)
at com.sleepycat.je.Cursor.retrieveNextAllowPhantoms(Cursor.java:2926)
at com.sleepycat.je.Cursor.retrieveNextNoDups(Cursor.java:2789)
at com.sleepycat.je.Cursor.retrieveNext(Cursor.java:2763)
at com.sleepycat.je.Cursor.getNext(Cursor.java:1116)
...

Thanks to OTN user user591209 for
reporting
this and testing the fix. [#21121] (4.1.24)

Removed an overzealous assertion that could fire during periods of heavy
eviction with a stack trace such as the following:

java.lang.AssertionError
at com.sleepycat.je.tree.INTargetRep$Sparse.updateCacheStats(INTargetRep.java:296)
at com.sleepycat.je.tree.INArrayRep.updateCacheStats(INArrayRep.java:133)
at com.sleepycat.je.tree.INArrayRep.noteRepChange(INArrayRep.java:158)
at com.sleepycat.je.tree.INArrayRep.noteRepChange(INArrayRep.java:163)
at com.sleepycat.je.tree.INTargetRep$None.set(INTargetRep.java:331)
at com.sleepycat.je.tree.INTargetRep$None.set(INTargetRep.java:303)
at com.sleepycat.je.tree.IN.updateNode(IN.java:1498)
at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1332)

This (like any assertion) would invalidate the Environment, but otherwise did
not cause any persistent damage.
[#21990] (4.1.24)

Made several improvements to log cleaning and eviction.

Reduced the CPU usage of the log cleaner, especially when large log
files are configured. Expensive calls to the internal
TrackedFileSummary.containsObsoleteOffset method in the cleaner thread have
been removed. As a result, the cleaner is better able to keep up or reduce
a backlog.

Reduced blocking during eviction. Unnecessary blocking on the internal
UtilizationProfile.flushLocalTracker method during eviction (both in
evictor threads and other threads participating in eviction such as cleaner
threads) has been removed. As a result, the evictor and cleaner are better
able to keep up or reduce a backlog.

Added a new log cleaner statistic to help diagnose problems when record
locks are held by the application and the locks prevent log file deletion
after a log file has been cleaned: EnvironmentStats.getPendingLNQueueSize.

[#22100] (4.1.25)

Fixed a bug that caused two EnvironmentStats -- getNCachedBINs and
getNCachedUpperINs -- to be incorrect when calling Environment.removeDatabase
or truncateDatabase. When a database was removed/truncated, the number of
BINs/INs belonging to that database were not subtracted from these two stats,
even though they were evicted from the cache correctly. So in an application
where databases are removed/truncated, these two stats were larger than the
actual number of BINs/INs in cache and would increase over time.

Also added a new cache statistic to help diagnose potential memory leaks:
EnvironmentStats.getDataAdminBytes.

[#22100] (4.1.26)

Fixed a bug that caused a NullPointerException during log cleaning when a
database is truncated or removed within a small processing window. An example
of the stack trace follows.

DbVerify has been improved to add Btree level information for internal
databases, and internal database names may now be specified with the -s option.
Note that DbVerify is used primarily for analysis and debugging.
[#22209] (4.1.27)

Fixed a bug in EnvironmentConfig.setConfigParam where a boolean "true" property
specified with leading or trailing whitespace was treated as if it were
"false", and no exception was thrown. The same bug impacted boolean properties
specified in the je.properties file with trailing whitespace.

WARNING: This fix could change the behavior of JE if a boolean
EnvironmentConfig property is specified as "true" but with leading or trailing
whitespace. Previously such a property would be set to false, and now it will
be set to true.

[#22212] (4.1.27)

Changes in 4.1.21

Fixed a bug that could cause the contents of a database to be deleted after a
crash, when the database was renamed just prior to the crash. A database can
be renamed explicitly with Environment.renameDatabase, or
implicitly when a DPL Renamer mutation is used to rename an entity
class or a secondary key field. This bug was introduced in JE 4.1.20. [#21537]

Changes in 4.1.20

Fixed a bug that caused an EnvironmentFailureException with LOG_FILE_NOT_FOUND,
for example:

The bug occurred after deleting a range of keys in a deferred-write database
(DatabaseConfig.setDeferredWrite(true) or StoreConfig.setDeferredWrite(true)),
closing the Environment, and then opening the Environment. It is most likely
to occur when deleting large numbers of consecutive keys, and when opening and
closing the Environment often, with the log cleaner enabled.

Because a log file is incorrectly deleted as a result of this bug, restoring
from a backup is necessary, i.e., there is no way to repair the environment.
Many thanks to Vishal Vishnoi and Vladimir Egorov for reproducing the problem
and helping us to diagnose it.

[#21348]

Fixed a bug in the DbPreUpgrade_4_1 and DbRepPreUpgrade_4_1 utilities that
prevented them from working properly due to concurrent log cleaning activity
while the utility is running. The symptom of the bug is that, after running
the utility without errors, when then opening the Environment with JE 5, the
following exception would occur:

com.sleepycat.je.EnvironmentFailureException: (JE 5.0.XX) Before upgrading to
JE 5.0, the following utility must be run using JE 4.1: DbPreUpgrade_4_1
using the JE old version. See the release notes.

WARNING: Due to this bug, JE 4.1.20 or later should be used to run the
DbPreUpgrade_4_1 and DbRepPreUpgrade_4_1 utilities. See the JE 5.0 change log
for information on running these utilities.

Thanks to Vinoth for reporting
the problem and helping us with the diagnosis and testing. [#21304]

Fixed a bug that could cause Environment.removeDatabase or
truncateDatabase to loop in a "livelock" pattern along with active
log cleaner threads, where the removeDatabase or
truncateDatabase thread is not making forward progress. Also
fixed a bug that could cause this state to eventually cause an
EnvironmentFailureException during a subsequent recovery, and
prevent opening the environment. [#20816]

Fixed a bug where the wrong stack trace was sometimes output for an owner or
waiter thread, when using
EnvironmentConfig.TXN_DEADLOCK_STACK_TRACE for debugging.

Added additional fixes to prevent looping and long operations times for
Environment.removeDatabase and truncateDatabase (see
[#20816] above). The additional fixes address long operation times due to log
cleaning with large file sizes in combination with eviction. Also fixed a bug
that slows down log cleaning (contributing to the long operation times as well)
when large numbers of databases are present, and not all databases are kept
open by the application. [#21015]

Fixed a bug that sometimes caused a ConcurrentModificationException when
calling Environment.setMutableConfig, when exception listeners were previously
configured. [#21177]

Changes in 4.1.17

General Changes

Fixed a bug introduced in JE 4.1.10 which could occasionally result in this exception:

Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception:
(JE 4.1.10) ... unexpected get for VLSN: 37,054,546 already waiting for VLSN: 37,054,545
UNEXPECTED_STATE_FATAL: Unexpected internal state, unable to continue. Environment is invalid and must be closed.
at com.sleepycat.je.EnvironmentFailureException.unexpectedState(EnvironmentFailureException.java:391)
at com.sleepycat.je.rep.vlsn.VLSNIndex.setupWait(VLSNIndex.java:550)
at com.sleepycat.je.rep.vlsn.VLSNIndex.waitForVLSN(VLSNIndex.java:502)
at com.sleepycat.je.rep.vlsn.VLSNIndex.awaitConsistency(VLSNIndex.java:1747)
at com.sleepycat.je.rep.impl.RepImpl.awaitVLSNConsistency(RepImpl.java:1378)
at com.sleepycat.je.recovery.Checkpointer.doCheckpoint(Checkpointer.java:712)
at com.sleepycat.je.recovery.Checkpointer.onWakeup(Checkpointer.java:507)

This problem had been introduced in JE 4.1.10 and can only be seen in that release.
The problem is transient and there is no change to the persistent data. After
closing and reopening the environment, the environment will be fully consistent
and reusable.

The master node in a replication group now rebroadcasts election
results on a periodic basis, to help restore normal functioning of a
replication group after a network partition has been resolved. The
default period is a minute and can be modified by the new
ReplicationConfig parameter
ReplicationConfig.ELECTIONS_REBROADCAST_PERIOD. Please review its
javadoc for further details. [#20220]

Fixed a bug introduced in JE 4.1.7 which could cause an incomplete (not
committed) transaction to be applied, as if it were committed, under certain
circumstances. The circumstances that cause the bug are:

Using an explicit Transaction, the first record in the Transaction is
written in a particular timing window at the beginning of a
checkpoint.

The checkpoint finishes and then the application crashes before
Transaction.commit or abort can be called by the application, or if NoSync
durability is used, before the commit or abort can be flushed to the
log.

When the application opens the Environment after the crash, the state
of the database is as if the transaction were committed.

[#20270]

Fixed a problem arising from a network partition event, that would
result in an unnecessary rollback of committed transactions. This
problem typically manifests itself in the application receiving a
RollbackProhibitedException if the number of transactions exceeds
je.rep.txnRollbackLimit. The following describes a scenario leading to
the problem:

Consider a three node replication group: A, B, C with A as the
master and nodes: B, C serving as replicas.

A network partition isolates A from B and C, resulting in
the partitions: (A) and (B,C).

B and C hold and election and B becomes the master.

There are now two masters: the pre-existing A which is isolated
and cannot perform durable writes, and a newly elected B.

The majority side (B,C) is accessible and continues to make
progress performing durable writes.

The master on the majority side B goes down. There is now no
master on the (B,C) side since there is no quorum.

Some time later, the partition is healed.

C now sees A as an established master and syncs with it
potentially rolling back committed transactions as a result.

The fix changes the final step above so that an election is held
before the rollback is allowed to proceed. The election results in C
being elected the new master. Node A encounters a
MasterReplicaTransitionException which it must handle by
closing and re-opening its environment handle so it can resume
operations as a replica. [#20258] [#20572]

Fixed a bug in the processing of internal BDBJE metadata which would
result in the following stack trace when a replicated environment was
re-opened:

java.lang.IndexOutOfBoundsException: Index: 110, Size: 110
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at com.sleepycat.je.rep.vlsn.VLSNBucket.findPopulatedIndex(VLSNBucket.java:441)
at com.sleepycat.je.rep.vlsn.VLSNBucket.removeFromTail(VLSNBucket.java:730)
at com.sleepycat.je.rep.vlsn.VLSNIndex.pruneDatabaseTail(VLSNIndex.java:1087)
at com.sleepycat.je.rep.vlsn.VLSNIndex.merge(VLSNIndex.java:1274)
at com.sleepycat.je.rep.vlsn.VLSNIndex.init(VLSNIndex.java:1161)
at com.sleepycat.je.rep.vlsn.VLSNIndex.(VLSNIndex.java:382)
at com.sleepycat.je.rep.impl.RepImpl.preRecoveryCheckpointInit(RepImpl.java:374)
at com.sleepycat.je.recovery.RecoveryManager.recover(RecoveryManager.java:238)
at com.sleepycat.je.dbi.EnvironmentImpl.finishInit(EnvironmentImpl.java:549)
at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:237)
at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:229)
at com.sleepycat.je.Environment.(Environment.java:211)
at com.sleepycat.je.Environment.(Environment.java:176)
at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:443)
at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:318)
at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:379)

There was no corruption to the persistent data, but without this fix,
the environment would repeatedly fail to open. [#20796]

Fix a bug that could cause a EnvironmentFailureException of type
LOG_FILE_NOT_FOUND during recovery (while opening an environment). The bug has
not been reported in the field and was discovered by a long running, internal
stress test. The signature of the bug is a log file -- one of the last files
in the environment -- with the lastEntryInPrevFileOffset FileHeader field set
to zero, as shown by DbPrintLog:

Added the com.sleepycat.je.rep.util.DbResetRepGroup utility, which
resets the members of a replication group, replacing the group with
a new group consisting of a single new member.

This utility is useful when the user wishes to create a new replication
group, starting with a copy of an existing replicated environment. The
utility can also be used to change the group name associated with the
environment.

Changes in 4.1.10

Fixed a bug that caused a hard deadlock between a thread performing an
Environment.sync (or a checkpoint when deferred-write databases are used) and
another thread (which could be an application thread) that is performing
eviction. The problem occurs when evicting cached information for a database
that is no longer in use, and is most likely to occur after closing a database
and/or when the JE cache is small enough to cause heavy eviction activity. An
example thread dump is below.

Thread performing eviction (in this case the JEEvictor thread):
com.sleepycat.je.recovery.DirtyINMap.coordinateEvictionWithCheckpoint(DirtyINMap.java:92)
waiting to lock <0xaeae7118> (a com.sleepycat.je.recovery.DirtyINMap)
at com.sleepycat.je.recovery.Checkpointer.coordinateEvictionWithCheckpoint(Checkpointer.java:203)
at com.sleepycat.je.evictor.Evictor.coordinateWithCheckpoint(Evictor.java:1073)
at com.sleepycat.je.evictor.Evictor.access$000(Evictor.java:124)
at com.sleepycat.je.evictor.Evictor$1RootEvictor.doWork(Evictor.java:707)
at com.sleepycat.je.tree.Tree.withRootLatchedExclusive(Tree.java:334)
at com.sleepycat.je.evictor.Evictor.evictRoot(Evictor.java:741)
at com.sleepycat.je.evictor.Evictor.evictBatch(Evictor.java:630)
at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:536)
at com.sleepycat.je.evictor.Evictor$BackgroundEvictTask.run(Evictor.java:1215)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Thread calling Environment.sync:
sun.misc.Unsafe.park(Native Method) parking to wait for
<0xaeafb698> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:941)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1261)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:594)
at com.sleepycat.je.latch.SharedLatch.acquireShared(SharedLatch.java:149)
at com.sleepycat.je.tree.Tree.withRootLatchedShared(Tree.java:344)
at com.sleepycat.je.dbi.DbTree.getHighestLevel(DbTree.java:1615)
at com.sleepycat.je.recovery.DirtyINMap.selectDirtyINsForCheckpoint(DirtyINMap.java:236)
locked <0xaeae7118> (a com.sleepycat.je.recovery.DirtyINMap)
at com.sleepycat.je.recovery.Checkpointer.doCheckpoint(Checkpointer.java:495)
locked <0xaeae7058> (a com.sleepycat.je.recovery.Checkpointer)
at com.sleepycat.je.dbi.EnvironmentImpl.invokeCheckpoint(EnvironmentImpl.java:1816)
at com.sleepycat.je.Environment.sync(Environment.java:1472) ...

[#19567]

Fixed a bug that caused an infinite loop when calling the Transaction.commit or
Transaction.abort method, when the transaction is used to both truncate and
remove a single database, using Environment.truncateDatabase and
removeDatabase. Now an exception will be thrown by
Environment.removeDatabase in this situation, and the application must call
Transaction.abort operation to end the txn. Although this is not a final
solution (ideally it should be possible to both truncate and remove a database
in a single transaction), this fix prevents the infinite loop in the commit and
abort methods. Thanks to Christoph for
reporting
this on OTN.
[#19636]

Fixed a race condition that sometimes caused Environments opened with
EnvironmentConfig.setSharedCache(true) to not use the shared
cache and instead use a private cache. This happened only when opening
multiple Environments concurrently in more than one thread. Thanks to Tim
Underwood for
reporting
this on OTN and identifying the specific problem in the JE source code.
[#19680]

Fixed a bug that causes an EnvironmentFailureException when a JE HA environment
is restarted. The stacktrace looks like this:

... recoveryTracker should overlap or follow on disk last VLSN of 52,678
recoveryFirst= 52,680 UNEXPECTED_STATE_FATAL: Unexpected internal state,
unable to continue. Environment is invalid and must be closed.
at com.sleepycat.je.EnvironmentFailureException.unexpectedState
(EnvironmentFailureException.java:391)
at com.sleepycat.je.rep.vlsn.VLSNIndex.merge(VLSNIndex.java:1230)
at com.sleepycat.je.rep.vlsn.VLSNIndex.init(VLSNIndex.java:1141)
at com.sleepycat.je.rep.vlsn.VLSNIndex.(VLSNIndex.java:367)
at com.sleepycat.je.rep.impl.RepImpl.preRecoveryCheckpointInit(RepImpl.java:373)
at com.sleepycat.je.recovery.RecoveryManager.recover(RecoveryManager.java:238)
at com.sleepycat.je.dbi.EnvironmentImpl.finishInit(EnvironmentImpl.java:534)
...

Thanks to Jan for reporting this problem on OTN and working with us to fix it. [#19754]

Changes in 4.1.7

General Changes

Fix a bug that could cause an EnvironmentFailureException with
LOG_FILE_NOT_FOUND during recovery, meaning that the JE
environment cannot be opened. We strongly recommend that all applications
using JE 4.1.6 upgrade immediately. The bug was introduced in JE 4.1.6 and is
not present in earlier releases. [#19346]

Fixed a bug that prevents using a DPL class converter mutation for a proxy
class. Previously, an exception such as the following was thrown:

Exception in thread "main" java.lang.ClassCastException:
com.sleepycat.persist.raw.RawObject cannot be cast to
com.sleepycat.persist.model.PersistentProxy
at com.sleepycat.persist.impl.ProxiedFormat.newInstance(ProxiedFormat.java:91)
at com.sleepycat.persist.impl.RecordInput.readObject(RecordInput.java:174)
at com.sleepycat.persist.impl.ReflectionAccessor$ObjectAccess.read(ReflectionAccessor.java:406)
at com.sleepycat.persist.impl.ReflectionAccessor.readNonKeyFields(ReflectionAccessor.java:285)
...

Thanks to James Li for
reporting
this on OTN and helping us to identify the problem in the source code.
[#19312]

Fix a bug that caused a hard deadlock when attempting to abort a transaction in
one thread, while performing operations using the transaction in another
thread. Now, rather than a hard deadlock, an IllegalStateException will be
thrown in this circumstance. Thanks to Jerven on OTN for
reporting
this.
[#19321]

Changes in 4.1.6

Performance Enhancements

When a BDB JE application has a data set that does not fit entirely in
cache, AND there is no particular working set of data that fits in
cache, the application will see the best performance when as much of the
internal btree nodes, or index, are kept in cache. This release
includes improvements to make JE's cache management more efficient,
and provides statistics to help determine cache efficiency.

Added concurrent eviction, which can be a benefit to any BDB JE
application whose data set does not fit in cache. In past releases,
cache eviction has been carried out by JE daemon threads, application
threads which call JE operations, and an optional single evictor
thread. The actual eviction operation was serialized, and could create
a bottleneck where many threads could be seen waiting upon the method
com.sleepycat.je.evictor.Evictor.doEvict().

In this release, cache eviction is no longer serialized, and can be
executed concurrently. In addition, JE now has a dedicated,
configurable thread pool which will do cache eviction when memory
limits are reached. Eviction is done by this dedicated pool, by other
JE daemon threads, and by application threads. JE attempts to skew the
eviction workload toward the pool and daemon threads, in order to
offload application threads.

The eviction pool is enabled by default, but can be disabled by the
je.env.runEvictor property. The properties je.evictor.coreThreads,
je.evictor.maxThreads and je.evictor.keepAlive are used to configure
the core, max and keepalive attribute for the pool. EnvironmentStats
has new statistics that try to give an indication of whether eviction
is executed by the eviction pool, by operations with specific
com.sleepycat.je.CacheMode settings, by explicit calls to
Environment.evictMemory, by operations that will cause the cache to go
over budget, or by JE daemon threads. [#18626]

This release introduces several new kinds of in-memory btree node
compression, targeted toward reducing the memory needed to hold
internal btree nodes in cache. This is particularly helpful for
applications which have data sets which do not fit in cache.

One of the optimizations reduces the in-memory footprint of an internal
btree node when only a small portion of it has been referenced, as
might be the case when data records are accessed in a random order, or
when a subset of data is accessed. It does not help if the
application is doing a database-wide cursor traversal. [#18623]

Another optimization takes effect when the key less than or equal to
16 bytes in size, which can be true either when the key is naturally
small, or when key prefixing is enabled through
Database.setKeyPrefix(). [#18624]

Optimized Database.sync and Database.close
for deferred-write databases, so that no file write or fsync is
performed if nothing in the database is dirty. Calling these methods
will now have very low cost when no changes to the database have been
made. [#18402]

The javadoc
for com.sleepycat.je.Durability
states that COMMIT_NO_SYNC is the default durablity for replicated
transactions.The documentation is correct, but the actual default used
was incorrect, and was COMMIT_SYNC. This has been fixed. [#18900]

General Changes

The following exception could be seen in rare circumstances from a
replicated environment. It is not an actual error, and was the result
of an over zealous assertion. The assertion has been corrected.

Fixed a bug that causes a EnvironmentFailureException LOG_FILE_NOT_FOUND when
using a particular sequence of operations with a Database configured for
duplicates (DatabaseConfig.setSortedDuplicates(true)) or a DPL secondary key
with a MANY_TO_ONE or MANY_TO_MANY relationship. The sequence of operations
that causes the bug is:

A single record exists (no duplicates) for key A

New transaction T1 starts

Key A is deleted in T1

Two or more duplicates are inserted for key A in T1

The T1 transaction aborts

Later, the log file containing the original version of the record is cleaned
and deleted, causing an exception when that record is subsequently accessed.
An example stack trace is below.

Fixed a bug which could cause PreloadStatus.EXCEEDED_TIME to be
incorrectly returned from the Database.preload()
method. [#18577]
(fixed in 4.0.114)

Improved the exception thrown in the following case: JE does not
support opening an environment with first a non replicated and then a
replicated environment handle. For example, if the user executes these
statements in the following order:

she would see a java.lang.ClassCastException, instead of the more
informative java.lang.UnsupportedOperationException. This has been
fixed. Note that it is permissible to open an environment as a
ReplicatedEnvironment, and then follow by opening it as a read only
Environment. This was initially reported on
the OTN forum. [#18649]

Added the property EnvironmentConfig.CLEANER_LAZY_MIGRATION to
provide finer control over log cleaner, checkpointing and eviction
behavior. See the javadoc for details. [#18650]
(fixed in 4.0.114)

Fixed a bug that causes a fatal exception during recovery (opening of a JE
Environment) under certain conditions. An example of the exception stack trace
is below.

Exception in thread "main" com.sleepycat.je.EnvironmentFailureException:
(JE 4.0.92) last LSN=0x424/0xe4fc1 LOG_INTEGRITY: Log information is incorrect,
problem is likely persistent. Environment is invalid and must be closed.
at com.sleepycat.je.recovery.RecoveryManager.traceAndThrowException(RecoveryManager.java:3052)
at com.sleepycat.je.recovery.RecoveryManager.readINs(RecoveryManager.java:867)
at com.sleepycat.je.recovery.RecoveryManager.buildINs(RecoveryManager.java:621)
at com.sleepycat.je.recovery.RecoveryManager.buildTree(RecoveryManager.java:513)
at com.sleepycat.je.recovery.RecoveryManager.recover(RecoveryManager.java:175)
at com.sleepycat.je.dbi.EnvironmentImpl.finishInit(EnvironmentImpl.java:529)
at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:204)
at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:230)
at com.sleepycat.je.Environment.(Environment.java:212)
at com.sleepycat.je.Environment.(Environment.java:166)
...
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 4.0.92)
fetchTarget of 0x89/0x4c2e29 parent IN=4883580 IN
class=com.sleepycat.je.tree.DIN lastFullVersion=0x424/0xdcec4
parent.getDirty()=false state=0 LOG_FILE_NOT_FOUND: Log file missing, log is
likely invalid. Environment is invalid and must be closed.
at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1241)
at com.sleepycat.je.tree.DIN.fetchTarget(DIN.java:520)
at com.sleepycat.je.tree.IN.findParent(IN.java:2704)
at com.sleepycat.je.tree.Tree.getParentINForChildIN(Tree.java:879)
at com.sleepycat.je.recovery.RecoveryManager.replayINDelete(RecoveryManager.java:1770)
at com.sleepycat.je.recovery.RecoveryManager.replayOneIN(RecoveryManager.java:945)
at com.sleepycat.je.recovery.RecoveryManager.readINs(RecoveryManager.java:846)
... 11 more
Caused by: java.io.FileNotFoundException: 00000089.jdb (The system cannot find
the file specified)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(Unknown Source)
at java.io.RandomAccessFile.(Unknown Source)
at com.sleepycat.je.log.FileManager$1.(FileManager.java:993)
at com.sleepycat.je.log.FileManager.openFileHandle(FileManager.java:992)
at com.sleepycat.je.log.FileManager.getFileHandle(FileManager.java:888)
at com.sleepycat.je.log.LogManager.getLogSource(LogManager.java:1073)
at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:779)
at com.sleepycat.je.log.LogManager.getLogEntryAllowInvisibleAtRecovery(LogManager.java:743)
at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1225)
... 17 more

The bug only occurred under the following conditions:

Databases with duplicates are used, including secondary indices.

Many duplicate records are deleted in the same key range.

The problem is most likely to occur when deletions occur immediately
prior to closing the environment, and the environment is closed and
re-opened frequently.

No data loss occurs as a result of this bug. By using a version of JE with
the bug fix, such environments can be opened and used normally. Thanks to OTN
user justindthomas for reporting the problem and working with us to diagnose
and fix it. [#18663] (fixed in 4.0.114)

Fixed a bug that causes an IllegalStateException when a DPL
Converter mutation is used for class evolution, and Replicas are
upgraded first (as prescribed) in a replication group. [#18690]
(fixed in 4.0.114)

Fixed a bug which can make log utilization inaccurate and prevent log cleaning,
when Environment.removeDatabase or
Environment.truncateDatabase is called, and the program crashes or
exits before any other information is written to disk. For example, if the
Environment is closed normally after the removal/truncation, or a scheduled
checkpoint occurs, then the bug will not occur. [#18696]
(fixed in 4.0.114)

All public JE exception and statistics classes should be
serializable. Some were not, and have been fixed. The following
classes are now certified to be serializable. [#18738]

com.sleepycat.je.BtreeStats

com.sleepycat.je.CommitToken

com.sleepycat.je.DatabaseEntry

com.sleepycat.je.DatabaseException

com.sleepycat.je.DatabaseExistsException

com.sleepycat.je.DatabaseNotFoundException

com.sleepycat.je.DatabaseStats

com.sleepycat.je.DeadlockException

com.sleepycat.je.DeleteConstraintException

com.sleepycat.je.DuplicateDataException

com.sleepycat.je.EnvironmentFailureException

com.sleepycat.je.EnvironmentLockedException

com.sleepycat.je.EnvironmentNotFoundException

com.sleepycat.je.EnvironmentStats

com.sleepycat.je.ForeignConstraintException

com.sleepycat.je.LockConflictException

com.sleepycat.je.LockNotAvailableException

com.sleepycat.je.LockNotGrantedException

com.sleepycat.je.LockStats

com.sleepycat.je.LockTimeoutException

com.sleepycat.je.LogWriteException

com.sleepycat.je.OperationFailureException

com.sleepycat.je.PreloadStats

com.sleepycat.je.PreloadStatus

com.sleepycat.je.RunRecoveryException

com.sleepycat.je.SecondaryConstraintException

com.sleepycat.je.SecondaryIntegrityException

com.sleepycat.je.SecondaryReferenceException

com.sleepycat.je.SequenceExistsException

com.sleepycat.je.SequenceIntegrityException

com.sleepycat.je.SequenceNotFoundException

com.sleepycat.je.SequenceOverflowException

com.sleepycat.je.SequenceStats

com.sleepycat.je.ThreadInterruptedException

com.sleepycat.je.TransactionStats

com.sleepycat.je.TransactionStats.Active

com.sleepycat.je.TransactionTimeoutException

com.sleepycat.je.UniqueConstraintException

com.sleepycat.je.VersionMismatchException

com.sleepycat.je.XAFailureException

com.sleepycat.je.jca.ra.JEException

com.sleepycat.je.rep.DatabasePreemptedException

com.sleepycat.je.rep.GroupShutdownException

com.sleepycat.je.rep.InsufficientAcksException

com.sleepycat.je.rep.InsufficientLogException

com.sleepycat.je.rep.InsufficientReplicasException

com.sleepycat.je.rep.LockPreemptedException

com.sleepycat.je.rep.LogOverwriteException

com.sleepycat.je.rep.MasterStateException

com.sleepycat.je.rep.MemberNotFoundException

com.sleepycat.je.rep.ReplicaConsistencyException

com.sleepycat.je.rep.ReplicaWriteException

com.sleepycat.je.rep.ReplicatedEnvironmentStats

com.sleepycat.je.rep.RestartRequiredException

com.sleepycat.je.rep.RollbackException

com.sleepycat.je.rep.RollbackProhibitedException

com.sleepycat.je.rep.StateChangeException

com.sleepycat.je.rep.UnknownMasterException

com.sleepycat.je.util.LogVerificationException

com.sleepycat.persist.IndexNotAvailableException

com.sleepycat.persist.StoreExistsException

com.sleepycat.persist.StoreNotFoundException

com.sleepycat.persist.evolve.DeletedClassException

com.sleepycat.persist.evolve.IncompatibleClassException

(fixed in 4.0.114)

Fixed a bug which caused a NullPointerException under certain
circumstances when key prefixing is enabled. This was originally reported on the
OTN Forum. [#18773]
(fixed in 4.0.114)

Fixed a bug that caused log cleaner threads to hang with the following
stack trace. This could occur infrequently in applications which use
EnvironmentMutableConfig.setExceptionListener() and have a high level
of concurrent log cleaning.
[#18791]
(fixed in 4.0.114)

java.lang.Thread.State: RUNNABLE
at java.util.HashMap.put(HashMap.java:374)
at java.util.HashSet.add(HashSet.java:200)
at com.sleepycat.je.dbi.EnvironmentImpl.registerExceptionListenerUser(EnvironmentImpl.java:730)
at com.sleepycat.je.utilint.DaemonThread.(DaemonThread.java:60)
at com.sleepycat.je.cleaner.FileProcessor.(FileProcessor.java:110)
at com.sleepycat.je.cleaner.Cleaner.doClean(Cleaner.java:461)
at com.sleepycat.je.dbi.EnvironmentImpl.invokeCleaner(EnvironmentImpl.java:1879)
at com.sleepycat.je.Environment.cleanLog(Environment.java:1559)

Fixed a problem where Database.count() could return the wrong value
for replicated databases with duplicates, after the environment had
experienced a replication rollback. [#18816]
(fixed in 4.0.114)

Fixed a problem which was intermittent and seen only on the master
node of a replication group which using synchronous transaction
commits. The application would see the following stack trace. The
problem was transient, no data was lost, and the system could re-open
successfully. [#18882]
(fixed in 4.0.114)

java.io.FileNotFoundException: ... (The system cannot find the file specified)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(Unknown Source)
at java.io.RandomAccessFile.(Unknown Source)
at com.sleepycat.je.log.FileManager$1.(FileManager.java:998)
at com.sleepycat.je.log.FileManager.openFileHandle(FileManager.java:998)
at com.sleepycat.je.log.FileManager.getFileHandle(FileManager.java:893)
at com.sleepycat.je.rep.stream.FeederReader$SwitchWindow.fillNext(FeederReader.java:523)
at com.sleepycat.je.log.FileReader.readData(FileReader.java:758)
at com.sleepycat.je.log.FileReader.readNextEntryAllowExceptions(FileReader.java:259)
at com.sleepycat.je.log.FileReader.readNextEntry(FileReader.java:230)
at com.sleepycat.je.rep.stream.FeederReader.scanForwards(FeederReader.java:284)
at com.sleepycat.je.rep.stream.MasterFeederSource.getWireRecord(MasterFeederSource.java:62)
at com.sleepycat.je.rep.impl.node.Feeder$OutputThread.run(Feeder.java:659)

When a disk corruption occurs that overwrites a log entry with non-zero data, a
checksum error is now properly reported via an EnvironmentFailureException, and
tools such as DbScavenger and DbVerifyLog work properly. Earlier, if a
corruption overwrote specific parts of the log entry, another exception was
mistakenly throw, such as:

java.lang.ArrayIndexOutOfBoundsException at
java.util.zip.Adler32.update(Adler32.java:47) at
com.sleepycat.je.log.ChecksumValidator.update(ChecksumValidator.java:59) at
com.sleepycat.je.log.ChecksumValidator.update(ChecksumValidator.java:55) at
com.sleepycat.je.log.LogManager.getLogEntryFromLogSource(LogManager.java:928)
...

Thanks to Jean-Christophe on OTN for reporting this bug.
(fixed in 4.0.114)

Opening a com.sleepycat.bind.serial.StoredClassCatalog on a replica
node in a JE HA replication group incorrectly resulted in a
ReplicaWriteException. This has been fixed.
Thanks to user13067358 on OTN for
reporting this problem. [#18938]
(fixed in 4.0.114)

A DPL class evolution bug caused an exception or incorrect behavior when
renaming a field, when the name change caused the alphabetical order of fields
in the class to change. An example of such an exception follows. However,
other types of exceptions could also occur.

java.lang.IllegalArgumentException: Can not set java.lang.Integer field
com.sleepycat.persist.test.EvolveClasses$RenameSecField.new_secKey2 to java.lang.String
at java.lang.reflect.Field.set(Field.java:657)
at com.sleepycat.persist.impl.ReflectionAccessor$ObjectAccess.read(ReflectionAccessor.java:422)
at com.sleepycat.persist.impl.ReflectionAccessor.readSecKeyFields(ReflectionAccessor.java:253)
at com.sleepycat.persist.impl.ComplexFormat$EvolveReader.readObject(ComplexFormat.java:2127)
at com.sleepycat.persist.impl.PersistEntityBinding.readEntity(PersistEntityBinding.java:115)
at com.sleepycat.persist.impl.PersistEntityBinding.entryToObjectInternal(PersistEntityBinding.java:83)
at com.sleepycat.persist.impl.PersistEntityBinding.entryToObject(PersistEntityBinding.java:64)
at com.sleepycat.persist.PrimaryIndex.get(PrimaryIndex.java:597)
at com.sleepycat.persist.PrimaryIndex.get(PrimaryIndex.java:584)
...

This has been fixed. [#18961]
(fixed in 4.0.114)

Fixed a bug that causes Cursor.getNextDup or
EntityCursor.nextDup to advance to a following key under certain
rare conditions. This also has a side effect of causing
Database.delete to delete duplicate records for the following key
under the same conditions. The conditions that lead to the bug are:

A Database is configured for duplicates or a DPL secondary key with a
MANY_TO_XXX relationship is used.

The environment or transaction is explicitly configured for
Serializable Isolation (this is not the default isolation mode).

One or more records for key A exist.

The key following key A, key B, has two or more duplicates, and the
first duplicate for key B is deleted.

Timing and conditions are such that the deleted duplicate for key B has
not been internally compressed out of the Btree.

Cursor.getNextDup or EntityCursor.nextDup is
called when positioned on the last record for key A, or
Database.delete is called for key A. In this case, the
record(s) with key B will mistakenly be returned or deleted.

[#19026]
(fixed in 4.0.114)

Fixed a DPL bug that prevented use of a non-public persistent class.
Previously, an exception such as the following was thrown when the
constructor was public but the class was non-public.

...
Caused by: java.lang.IllegalAccessException: Class
com.sleepycat.persist.impl.ReflectionAccessor can not access a member of class
XXX with modifiers "public"
at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)
at java.lang.reflect.Constructor.newInstance(Constructor.java:505)
at com.sleepycat.persist.impl.ReflectionAccessor.newInstance(ReflectionAccessor.java:151)
...

Thanks to Sheila on OTN for
reporting
this problem and working with us. [#19100]

After changing stats collection settings on the JE JConsole plugin,
the user may see a delay before the jconsole tab is repainted and
changes take effect. This has been improved to provide a more
immediate response. [#19101]

Fixed a rare problem that could possibly cause a LogFileNotFoundException
if the following actions occurred in a very particular ordering:

The environment has been a replica in a JE replication group.

There has been a master failover, resulting in a syncup in this
replica.

During syncup, there was a partial rollback of an in progress transaction.

The transaction deleted and then inserted the same key value.

Log cleaning occurs later.

[#19160]

The javadoc for com.sleepycat.je.rep.ReplicatedEnvironment referred to a
je.rep.group.joinGroupTimeout property. That property doesn't exist,
and the documentation has been corrected to refer to je.rep.envSetupTimeout.

In a JE replication system, it was possible in some cases for a master
node to transfer to replica state and fail to properly reinitialize
internal state. The problem is that some data record locks were not
properly released, and future access to those data records could cause
deadlocks.

To resolve this, a new exception,
com.sleepycat.je.rep.MasterReplicaTransitionException is thrown in these
cases. MasterReplicaTransitionException is a RestartRequiredException, and the
application must close and reopen its environment handle, thereby
properly reinitializing the node. [#19177]

Fixed a bug that replicated parameters in je.properties can't be recognized if
opening a read only standalone Environment on a replicated Environment home.
[#19080]

Fixed a bug that if hard recovery tries to truncate a transaction that spread
multiple log files will cause an infinite recovery loop. [#19463]