100 check-ins occurring around 9fa3ae584ae49366.

Disable extension loading by default. At some point we should change
the configure script to detect the presence of dlopen() automatically
and add the appropriate library to the link. But that requires a
working version of autoconf, which I do not have. Ticket #2124. (CVS 3545)
check-in: 87a9ee07 user: drh tags: trunk

The query optimizer does a better job of optimizing out ORDER BY clauses
that contain the rowid or which use indices that contain the rowid.
Ticket #2116. (CVS 3536)
check-in: f245f5c2 user: drh tags: trunk

Query optimizer enhancement: In "FROM a,b,c left join d" allow the C table
to be reordered with A and B. This used to be the case but the capability
was removed by (3203) and (3052) in response to ticket #1652. This change
restores the capability. (CVS 3529)
check-in: 7393c81b user: drh tags: trunk

http://www.sqlite.org/cvstrac/tktview?tn=2046 The virtual table interface allows for a cursor to field multiple
xFilter() calls. For instance, if a join is done with a virtual
table, there could be a call for each row which potentially matches.
Unfortunately, fulltextFilter() assumes that it has a fresh cursor,
and overwrites a prepared statement and a malloc'ed pointer, resulting
in unfinalized statements and a memory leak. This change hacks the code to manually clean up offending items in
fulltextFilter(), emphasis on "hacks", since it's a fragile fix
insofar as future additions to fulltext_cursor could continue to have
the problem. (CVS 3521)
check-in: 18142fdb user: shess tags: trunk

Delta-encode terms in interior nodes. While experiments have shown
that this is of marginal utility when encoding terms resulting from
regular English text, it turns out to be very useful when encoding
inputs with very large terms. (CVS 3520)
check-in: c8151a99 user: shess tags: trunk

Store minimal terms in interior nodes. Whenever there's a break
between leaf nodes, instead of storing the entire leftmost term of the
rightmost child, store only that portion of the leftmost term
necessary to distinguish it from the rightmost term of the leftmost
child. (CVS 3513)
check-in: f6e0b080 user: shess tags: trunk

Require a minimum fanout for interior nodes. This prevents cases
where excessively large terms keep the tree from finding a single
root. A downside is that this could result in large interior nodes in
the presence of large terms, which may be prone to fragmentation,
though if the nodes were smaller that would translate into more levels
in the tree, which would also have that problem. (CVS 3510)
check-in: 64b7e340 user: shess tags: trunk

The uninitialized file descriptor from the unixFile structure is passed to sqlite3DetectLockingStyle in allocateUnixFile rather than the file descriptor passed in. This was causing the locking detection on NFS file systems to behave somewhat randomly and the result was locks were not respected and data loss could occur. (CVS 3508)
check-in: b9dd16ef user: aswift tags: trunk

Use the difference between the SQLITE_IOERR_SHORT_READ and SQLITE_IOERR_READ
returns from sqlite3OsRead() to make decisions about what to do with the
error. (CVS 3503)
check-in: 6324ea81 user: drh tags: trunk

Change the default prefix for temporary files so that it no longer
contains the text "sqlite". In this way, perhaps we will not get so
many false bug reports such as ticket #2049, #1989, and #1841. (CVS 3498)
check-in: 7ce48000 user: drh tags: trunk

Make memset() uses less error-prone.
http://www.sqlite.org/cvstrac/tktview?tn=2036,35 describes some cases
where we were passing memset() a length which was the sizeof a
pointer, rather than the structure pointed to. Instead, wrap this
idiom up in CLEAR() and SCRAMBLE() macros. (CVS 3488)
check-in: 5878add0 user: shess tags: trunk

Replace the DocList and DocListReader structures. The new structures
distinguish reading from a static buffer from writing to a dynamic
buffer. This allows n-way doclist merging, and in-place merging of
segment leaf nodes, which together cut segment merge times in half. (CVS 3486)
check-in: af5bfb98 user: shess tags: trunk

Don't store empty segments. When inserting empty strings, the code
was writing out a segment made up of a single leaf node containing the
\0 header. LeafReader assumed that leaf nodes always contained at
least one term, so assertions would fail. While it would be possible to support reading and merging empty
segments, there's no reason to do so. While this change could have
been done in writeZeroSegment(), I put it in leafWriterFlush() so that
it would work right if segmentMerge() created an empty segment, which
could happen with future changes to how deleted documents are handled. (CVS 3484)
check-in: fed79bee user: shess tags: trunk

Convert fts2 to store data in a way which allows for much faster
updates. Groups of documents form segments which are encoded in a
btree layered over a table of blocks, with various tricks to make
merges fast. This performs 20x-25x faster than fts1 when loading the
Enron corpus, and is only slightly slower for queries. (CVS 3474)
check-in: 85272b2f user: shess tags: trunk

Fix incorrect doclist initialization in term_select_all().
docListRestrictColumn() generates a DL_POSITIONS doclist, which means
that after the first doclist is processed, the second doclist is
initialized as DL_POSITIONS, but with DL_POSITIONS_OFFSETS data.
(Note that DL_DEFAULT is now DL_POSITIONS, which masks this bug.) (CVS 3467)
check-in: 144e3f11 user: shess tags: trunk

Report the error SQLITE_CORRUPT instead of SQLITE_IOERR if unable
to rollback a hot journal that was damaged (for example) by filesystem
corruption following a power failure. (CVS 3460)
check-in: 70501e4e user: drh tags: trunk