[2cbaf886]
part of check-in
[44ee5383]
Update the PRAGMA data_version command so that it reponse to changes made
by a shared-cache database connection, and also to changes made by the same
database connection. Add test cases to verify the new behavior.
(user:
drh
branch: data_version_pragma)

[8d97b363]
part of check-in
[ecc3544e]
Ensure that the "Any prior cache entry associated with newKey is guaranteed not to be pinned" guarantee made to xRekey implementations is not violated.
(user:
dan
branch: defrag-opt)

[00ef5573]
part of check-in
[67484807]
Call fsync() right after ftruncate() when in journal_mode=TRUNCATE and
when synchronous=FULL in order to ensure that transactions are durable
across a power loss that happens moments after the commit. Proposed
fix for https://bugzilla.mozilla.org/show_bug.cgi?id=1072773. This
is a cherry-pick of [3e922208b68563]
(user:
drh
branch: branch-3.8.6)

[a98547ad]
part of check-in
[3e922208]
Call fsync() right after ftruncate() when in journal_mode=TRUNCATE and
when synchronous=FULL in order to ensure that transactions are durable
across a power loss that happens moments after the commit. Proposed
fix for https://bugzilla.mozilla.org/show_bug.cgi?id=1072773.
(user:
drh
branch: trunk)

[ed122b13]
part of check-in
[209f672e]
Split part of "PRAGMA ota_mode" off into "PRAGMA pager_ota_mode". This allows some specialized custom VFS implementations to intercept and implement the expected pager-related effects of this pragma.
(user:
dan
branch: ota-update)

[3e732d2b]
part of check-in
[bdb6e497]
Refactor the sqlite3PcacheFetch() routine into three separate routines,
which are significantly faster overall and about 100 bytes smaller in
size as well.
(user:
drh
branch: trunk)

[27fb89e6]
part of check-in
[f1f94a97]
Change the page cache so that a new sqlite3_pcache object is allocated as
soon as the page cache is opened, not delayed until the first fetch request.
This give a noticable performance boost. The interface between pager and
the page cache has changed slightly, which might break ZIPVFS.
(user:
drh
branch: trunk)

[e4855453]
part of check-in
[17349a49]
Changes to help ensure that a multiplication does not overflow when computing
the number of bytes needed for a memory allocation, and cause a malfunction.
No problems existing problems were discovered. However, these changes should
help to ensure that no problems arise in the future.
(user:
drh
branch: safer-malloc)

[d878c40e]
part of check-in
[eafd8aa1]
Fix compiler warnings. When forcing the delete of a WAL file, do not
fail if the WAL files does not exist. All "veryquick.tcl" tests are now
passing on Linux under the standard compile-time options.
(user:
drh
branch: apple-osx)

[0f883505]
part of check-in
[9411d7dc]
Initial attempt to merge in all trunk changes over the previous 1.5 years.
This check-in compiles but there are compiler warnings and "make test"
segfaults after only running a few test modules.
(user:
drh
branch: apple-osx)

[6ed54e16]
part of check-in
[725c1c14]
Take out the special handling of nolock=true in os_win.c and add it to
pager.c, so that it works for all VFSes. Add the pPager->noLock boolean
for clarity.
(user:
drh
branch: win32-none)

[5c296b58]
part of check-in
[71f152c8]
Add the immutable=TRUE query parameter and SQLITE_IOCAP_IMMUTABLE, either
of which prevents locking of the database and omits tests for existance
of journal files.
(user:
drh
branch: win32-none)

[ddf6e995]
part of check-in
[34155c40]
Add sqlite3_log() diagnostic messages for a specific type of corruption
where the file size is reported to be too small relative to the size in
the header. This branch is intended to help debug a specific problem
reported from the wild and is not for general use.
(user:
drh
branch: filesize-debug)

[f5805e0e]
part of check-in
[fdc651e2]
Avoid calling sqlite3OsDelete() on a file that is open, since this causes Windows to run *very* slowly. Comes up on error recovery in journal_mode=PERSIST.
(user:
mistachkin
branch: trunk)

[4757194b]
part of check-in
[e00f37e2]
Create and use a new pager interface sqlite3PagerUnrefNotNull() that works
just like sqlite3PagerUnref() but guarantees that its argument is not a
NULL pointer.
(user:
drh
branch: trunk)

[994b3ee3]
part of check-in
[46231af9]
Use SQLITE_FCNTL_SYNC instead of SQLITE_FCNTL_SYNC_OMITTED. Add documentation in for FCNTL_SYNC and FCNTL_COMMIT_PHASETWO.
(user:
dan
branch: zipvfs-multifile-commit)

[b28711af]
part of check-in
[32fb1784]
Modify the way some internal file-controls are invoked. In order to support multi-file transactions in the zipvfs extension.
(user:
dan
branch: zipvfs-multifile-commit)

[d27dc8aa]
part of check-in
[9c59f5af]
Back out the new device capability. The determination of whether or not a
file has moved is now done strictly using a file-control.
(user:
drh
branch: detect-moved-db)

[93887430]
part of check-in
[28348f2a]
Only error out on a database file move when attempting to start a write
transaction. Assume read transactions are still safe. And make the error
SQLITE_READONLY_DBMOVED instead of SQLITE_IOERR_NODB.
(user:
drh
branch: detect-moved-db)

[5d2f7475]
part of check-in
[0ae7e75b]
Drop any existing mapping of the database file when exiting the pager "error state", as it may at this point be too large for the database file. Do not invoke file-control MMAP_SIZE if the database file handle does not support xFetch and xUnfetch (on the grounds that xUnfetch(0) calls to invalidate the mapping cannot be made).
(user:
dan
branch: trunk)

[79df56da]
part of check-in
[d5b084e9]
Updates to the Codec interface to support codecs on attached databases and
to allow rekeying in the middle of a transaction. These changes are only
applicable if SQLite is compiled with SQLITE_HAS_CODEC.
(user:
drh
branch: trunk)

[3709c5c9]
part of check-in
[40cfde8b]
Avoid unnecessary calls to FCNTL_SIZE_HINT. Return an error to the user if the file-control invoked by "PRAGMA mmap_size" returns a value other than SQLITE_OK or SQLITE_NOTFOUND.
(user:
dan
branch: trunk)

[a55adacb]
part of check-in
[ea1404a1]
Refactoring the mmap interface. The controlling pragma is now "mmap_size"
instead of "mmap_limit". Also change SQLITE_CONFIG_MMAP_LIMIT and
SQLITE_FCNTL_MMAP_LIMIT to SQLITE_CONFIG_MMAP_SIZE and
SQLITE_FCNTL_MMAP_SIZE, respecctively.
The default mmap_size is now always 0, meaning that
memory mapped I/O is off by default. There is a new compile-time option
SQLITE_MAX_MMAP_SIZE that determines a hard upper bound on the mmap_size.
Setting SQLITE_MAX_MMAP_SIZE to zero disables the memory-mapped I/O logic
and causes it to be omitted from the build. An extra argument is added
to SQLITE_CONFIG_MMAP_SIZE that can optionally lower the SQLITE_MAX_MMAP_SIZE
at start-time. The SQLITE_MAX_MMAP_SIZE is zero for platforms where we
know that it does not work, meaning that it cannot be turned on by mistake
on those platforms.
(user:
drh
branch: trunk)

[28f45e60]
part of check-in
[be7d2c54]
Add new primary error codes SQLITE_NOTICE and SQLITE_WARNING for use with
sqlite3_log(). Add new extended error codes SQLITE_NOTICE_RECOVER_WAL and
SQLITE_NOTICE_RECOVER_ROLLBACK to use with sqlite3_log() messages that occur
on each recovery.
(user:
drh
branch: trunk)

[eb6d272f]
part of check-in
[8a4314a3]
Disable the use of memory-mapped I/O if the SQLITE_DISABLE_MMAP macro is
defined. Automatically define this macro for OpenBSD and QNX. Other
systems are likely to be added to the disabled list over time.
(user:
drh
branch: trunk)

[2e68df46]
part of check-in
[2d9f1327]
Change the mmap_limit pragma to report the new limit, or to report the
existing limit if called with no arguments. Report the default mmap_limit
as part of PRAGMA compile_options. Set the default mmmap_limit to 0 for
all systems other than linux, mac, windows, and solaris.
(user:
drh
branch: experimental-mmap)

[95e7a64b]
part of check-in
[01ffdabb]
Always send the SQLITE_FCNTL_MMAP_LIMIT pragma to the VFS, even if the limit
is zero and even if the VFS does not support xFetch().
(user:
drh
branch: experimental-mmap)

[072512d0]
part of check-in
[78141d0a]
Add the SQLITE_CONFIG_MMAP_LIMIT configuration option for overriding the
SQLITE_DEFAULT_MMAP_LIMIT compile-time setting. Enhance "PRAGMA mmap_limit"
so that without a specific database name, it sets the limit on all database
files and changes the default for any future databases that might be added
using ATTACH.
(user:
drh
branch: experimental-mmap)

[c532d51c]
part of check-in
[1b37c4ef]
Memory-mapped I/O is now on by default. The "PRAGMA mmap_limit(N)" can be
used to issue a hint to the VFS to limit mmap space to N bytes. The VFS
is free to ignore that hint if desired. However, if "PRAGMA mmap_limit(0)"
is used, xFetch is never called.
(user:
drh
branch: experimental-mmap)

[2dd59f36]
part of check-in
[bbcaab3e]
Do not return SQLITE_IOERR when the user attempts to open a small file that is not a database with mmap enabled. Instead return SQLITE_NOTADB.
(user:
dan
branch: experimental-mmap)

[582f8da5]
part of check-in
[39247b14]
Add extended error code SQLITE_READONLY_ROLLBACK. Returned if a read-only connection cannot read the database because doing so would require it to roll back a hot-journal.
(user:
dan
branch: experimental)

[0dbf5ff5]
part of check-in
[b73847f1]
If a rollback mode transaction reduces the size of the database file, avoid actually truncating the file until after the transaction has been committed (but before the db has been unlocked). This means pages that are removed from the database by truncating the file need not be journalled.
(user:
dan
branch: incr-vacuum-opt)

[4092c907]
part of check-in
[bdceca27]
Since errors are ignored anyway, use sqlite3OsFileControlHint() instead of OsFileControl() to invoke the SQLITE_FCNTL_BUSY_HANDLER file-control.
(user:
dan
branch: trunk)

[521251dd]
part of check-in
[11aa47b0]
If the atomic-write property is enabled, a transaction may be committed in journal_mode=DELETE mode without ever actually creating a journal file on disk. In this case, do not attempt to unlink() the journal file when committing the transaction.
(user:
dan
branch: trunk)

[ed53fe75]
part of check-in
[bed9c172]
Change os_unix.c to propagate ENOENT errors back to sqlite as SQLITE_IOERR_DELETE_NOENT. Have SQLite ignore these where they are benign and propagate them back to the caller where they may indicate a file-system malfunction of some description.
(user:
dan
branch: trunk)

[9f5f2823]
part of check-in
[6b4ff83b]
Ensure that the value returned by xSectorSize() is reasonable (currently defined as between 2^5 and 2^16 bytes) before using it to calculate the amount of padding to add to a wal file.
(user:
dan
branch: trunk)

[9d4d6406]
part of check-in
[595dfdbf]
Have user interfaces report out the filename of in-memory databases as an
empty string, as it always has. This simplifies the changes.
(user:
drh
branch: shared-cache-memdb)

[5a86e85e]
part of check-in
[e3ad61e0]
Only allow :memory: databases to share cache if there are created using a
URI filename. This minimizes the risk of breakages in legacy applications
that have shared-cache enabled but also use :memory: databases which they
expect to keep separate.
(user:
drh
branch: shared-cache-memdb)

[5b14be81]
part of check-in
[4590e433]
Enable the use of shared cache for an in-memory database, so that separate
database connections can share the same in-memory database.
(user:
drh
branch: shared-cache-memdb)

[c296b303]
part of check-in
[65479294]
Added support for SQLITE_ENABLE_PERSIST_WAL compile time macro, retrieving lastErrno from WAL file and setting last errno when writes fail due to space constraints
(user:
adam
branch: apple-osx)

[99ee9e52]
part of check-in
[cb774b26]
Only require double-zero terminators on database filenames, not any every
files supplied to the xOpen method. This backs out [2544f233f1]. Also
refactor the fillInUnixFile() routine in os_unix.c to reduce the number
of parameters.
(user:
drh
branch: trunk)

[f2a3a87d]
part of check-in
[2544f233]
Avoid passing strings with a single nul-terminator (two are required) to the VFS xOpen() method from within the code that checks to see if a master-journal file may be safely deleted.
(user:
dan
branch: trunk)

[e7c940ef]
part of check-in
[af59b182]
Add function sqlite3OsFileControlNoFail(), which is the same as sqlite3OsFileControl() except that it does not simulate OOM errors. This saves adding calls to the BenignMalloc() functions around each of the invocations of sqliteOsFileControl() that ignore the return code.
(user:
dan
branch: trunk)

[818ae6c4]
part of check-in
[38bc4236]
Remove the BenignMalloc() from around the call to FCNTL_OVERWRITE. Add
new ones around sqlite3OsDelete(). Fix an assert that causes problems for
the nx-devkit.
(user:
drh
branch: trunk)

[d03fb1de]
part of check-in
[6191c5e4]
Change the name ZERO_DAMAGE to the more descriptive POWERSAFE_OVERWRITE.
The query parameter used to control this device characteristic is now "psow".
(user:
drh
branch: statvfs)

[6277bebe]
part of check-in
[42f31f19]
Cherry-pick the SQLITE_DIRECT_OVERFLOW_READ fix for ticket
[ac0ff496b7e] of changes [c5256b59ad] and [c723e3e18a]
into the nx-devkit branch.
(user:
drh
branch: nx-devkit)

[523c64f6]
part of check-in
[c723e3e1]
Fix other problems where 32-bit integer overflow may cause a problem. Two that require an improbably large sub-journal and two in test harness code.
Ticket [ac0ff496b7e2]
(user:
dan
branch: trunk)

[2fb503c7]
part of check-in
[a0be6ea4]
Add SQLITE_IOCAP_ZERO_DAMAGE and enable it for both unix and windows. Use
this device characteristic to reduce the required work in journaling.
A side effect is that this changes the default page exists back to 1024
even with the use of statvfs().
(user:
drh
branch: statvfs)

[6b3a7765]
part of check-in
[09be42d5]
Begin making experimental changes to use mmap() for reading content from
a database. The code compiles, but crashes on the test suite.
(user:
drh
branch: mmap-experimental)

[ad62daa0]
part of check-in
[07159e84]
If an error occurs while writing to the database file during a VACUUM, discard the contents of the in-memory cache. This is required as if the database is a zipvfs database, the contents of the cache may be inconsistent with respect to the database as stored on disk.
(user:
dan
branch: trunk)

[60c80db5]
part of check-in
[60fee957]
Add assert() statements and eliminate needless variable assignments in order
to get the clang scan-build utility to report zero problems against the
SQLite core. Clang's static analysis did find one real problem - but it was
in the command-line shell, not in the SQLite core.
(user:
drh
branch: trunk)

[5cafaf38]
part of check-in
[6cb43f6c]
Forward port the Apple-specific changes from [db5b7b778c] in the
apple-osx-377 branch. Fix this up so that it will compile and run on Linux.
(user:
drh
branch: apple-osx)

[eb09ad62]
part of check-in
[89272357]
Add SQLITE_STMTSTATUS_CACHE_HIT/MISS and SQLITE_DB_STATUS_CACHE_HIT/MISS. For querying the number of pager cache hits and misses on a statement or connection basis.
(user:
dan
branch: cache-stats)

[817f7f71]
part of check-in
[c71d7320]
Create a new pager type, PAGER_SORTER, for use in the external merge sort.
Such pagers are always held in memory but do report when they are under
memory pressure by calling pagerStress.
(user:
drh
branch: trunk)

[f582ebb4]
part of check-in
[2e5a912a]
Toward version 3.7.5.1. This check-in updates the version number and also
patches the pager so that it never spills cache while in WAL mode. The
latter patch fixes ticket [2d1a5c67dfc23] and also prevents excessive WAL
file sizes if the pager cache starts thrashing. A few test cases fail.
(user:
drh
branch: branch-3.7.5)

[928d50d3]
part of check-in
[57912327]
Bring the apple-wal-readonly branch up to date with the latest changes
in apple-osx, and especially the fix for disappearing WAL transactions.
(user:
drh
branch: apple-wal-readonly)

[f501684d]
part of check-in
[e4d0f7ac]
Backport the [67bf1c9a88] fix for vanishing WAL transactions due to small
cache spillage. Ticket [2d1a5c67dfc236]. Also bump the version number
to 3.7.6.3 in preparation for patch release.
(user:
drh
branch: branch-3.7.6)

[4b235855]
part of check-in
[67bf1c9a]
When committing a WAL transaction, make sure at least one page is
written to the WAL file so that the WAL subsystem will have a page
on which to set the commit flag.
Ticket [2d1a5c67dfc236].
(user:
drh
branch: trunk)

[b7fe4b8e]
part of check-in
[96d60985]
Add the ability to limit filenames to 8+3 using the
SQLITE_ENABLE_8_3_NAMES compile-time option together with a URI
parameter of "8_3_names=1".
(user:
drh
branch: trunk)

[4056376f]
part of check-in
[671ba5fc]
Only open a read-only connection to shared-memory if the "readonly_shm=1" option is specified as part of the database file URI (and if a read-write connection fails).
(user:
dan
branch: wal-readonly)

[ff40d2a9]
part of check-in
[7155e6f3]
Backport check-ins [0900e35348f4b9bf3] and [4fead8e714c7e50] to the 3.7.2
branch. These check-ins provide hints to the btree layer for when it is
possible to use a hash table rather than a btree to implement an index.
The SQLite BTree layer does not use these hints, but alternative btree layers
might.
(user:
drh
branch: branch-3.7.2)

[055239dc]
part of check-in
[311d0b61]
When committing a WAL transaction, do not write any pages to the WAL file with page numbers greater than the size of the database image in pages.
(user:
dan
branch: trunk)

[5b2210b4]
part of check-in
[442be135]
Make sure the change-counter and SQLite-version fields of the header are
set correctly even after vacuuming. This is a backport of changes
[0be92a7576] and [04fa1e1690] to address ticket [5d863f876ee].
(user:
drh
branch: branch-3.7.4)

[d62dfc1d]
part of check-in
[bac7342c]
Merge in the blocking-checkpoint enhancement, including the new
sqlite3_wal_checkpoint_v2() interface and the
PRAGMA wal_checkpoint(full) statement.
(user:
drh
branch: trunk)

[3327c903]
part of check-in
[58577135]
When extending a database file, do so by writing one or more page-size chunks of data to the file, instead of just a single byte to the end.
(user:
dan
branch: trunk)

[37696d87]
part of check-in
[f12b5d76]
Make sure the return code from doing an xSync is correctly initialized
so that a valid return code is produced regardless of the path taken through
the logic.
(user:
drh
branch: trunk)

[c8975ac9]
part of check-in
[64914980]
Change the name of SQLITE_FCNTL_SYNC to SQLITE_FCNTL_SYNC_OMITTED and only
send it when PRAGMA synchronous=OFF. Add better documentation to explain
what that file-control opcode is used for.
(user:
drh
branch: trunk)

[8b6eb723]
part of check-in
[04fa1e16]
Make sure the change-counter and SQLite-version fields of the header are
set correctly even when doing a VACUUM with locking_mode=EXCLUSIVE.
Ticket [5d863f876ee9561b].
(user:
drh
branch: trunk)

[737025a2]
part of check-in
[c1e0d09c]
Increment the change counter and update the SQLite version number whenever
page 1 is added to the WAL. Ticket [5d863f876ee9561b9]
(user:
drh
branch: bug-5d863f87)

[a3d1b55d]
part of check-in
[ece7efce]
If a rollback is attempted in journal_mode=off mode, force SQLite to discard the contents of the pager cache before processing any subsequent queries.
(user:
dan
branch: trunk)

[7f7587c2]
part of check-in
[7e3fc2c8]
Add experimental command "PRAGMA wal_blocking_checkpoint", which uses the busy-handler to block until all readers have finished in order to ensure the next writer will be able to wrap around to the start of the log file.
(user:
dan
branch: blocking-checkpoint)

[b9809c31]
part of check-in
[756589ad]
Add "PRAGMA checkpoint_fullfsync". Similar to "PRAGMA fullfsync", but enables full fsyncs only during checkpoint operations.
Update: This change was added to the trunk by check-in [a069867301de3ca2e17]
(user:
dan
branch: apple-osx-exp)

[c19b0e8d]
part of check-in
[8dd5c691]
If a database file with the WAL flag set is opened in exclusive-locking mode, use heap memory to store the wal-index instead of shared-memory.
(user:
dan
branch: trunk)

[6c6dde87]
part of check-in
[471a4efb]
Have the pager change to at least WRITER_CACHEMOD state before marking any pages as dirty (instead of immediately after). Otherwise, if an error occurs, the pager may be left in WRITER_LOCKED state with dirty pages in the cache.
(user:
dan
branch: trunk)

[47600422]
part of check-in
[e14ef0e8]
Make sure the pager cache is cleared if there is any difficulty starting
a new read transaction in WAL mode. Ticket [313723c356483eff2a4c4bdd2c].
(user:
drh
branch: trunk)

[10cc2672]
part of check-in
[8ac18523]
If an SQLITE_FULL error occurs during rollback or journal finalization, treat it in the same way as SQLITE_IOERR (i.e. require that the pager internals be completely reset before it is next read from or written to).
(user:
dan
branch: trunk)

[9621456e]
part of check-in
[531abc80]
Fix a problem wherein changing the journal-mode immediately after leaving exclusive-locking mode could lead to the database being unlocked without clearing the changeCountDone flag.
(user:
dan
branch: experimental)

[3f237379]
part of check-in
[ad78ccac]
Changes to the comments describing the Pager.setMaster variable in pager.c. Add an assert() statement to verify that two master journal pointers are not written to a single journal file.
(user:
dan
branch: trunk)

[237bee4c]
part of check-in
[50c0f220]
Make sure a connection has an exclusive lock on all database files involved in a multi-file transaction before writing the master-journal pointer into any journal files. Fix for [f3e5abed55].
(user:
dan
branch: experimental)

[78ca1e1f]
part of check-in
[f4780bde]
Changes so that the xShmOpen VFS method is no longer required. Its job can be done by the first call to xShmMap. Rename xShmClose to xShmUnmap.
(user:
dan
branch: experimental)

[d6454d37]
part of check-in
[3053a4ad]
Allocate a buffer containing the full path-name to the associated WAL file when a pager is created. This saves having to construct a new buffer each time a new read-transaction is opened and SQLite checks for the existance of a WAL file.
(user:
dan
branch: trunk)

[3808cf9c]
part of check-in
[6ae76172]
Add a NEVER to sqlite3PagerCommitPhaseTwo() because it is now no longer
possible to invoke that function without holding a RESERVED lock.
(user:
drh
branch: trunk)

[2ce3612a]
part of check-in
[95c69dfb]
If you delete a database file but not its *-wal file, then open the database
(thus creating a new empty database file) the *-wal file is automatically
deleted.
(user:
drh
branch: trunk)

[813d09dd]
part of check-in
[cdf2c5c2]
Do not call pager_open_journal() from within PagerBegin() if the connection is in exclusive-access mode. It will be called from within PagerWrite() just as it is for non-exclusive mode anyway.
(user:
dan
branch: trunk)

[db0de9e0]
part of check-in
[2eaf5ee0]
When synchronous=NORMAL, use the same journal file format as with synchronous=FULL (i.e. multiple journal headers within the one journal). Fix for [d11f09d36e].
(user:
dan
branch: trunk)

[edf1f8b0]
part of check-in
[7d83fbae]
Revert to allowing a cache spill during writes of multiple pages within a
single sector as long as the spill does not require a journal sync and a
new journal header.
(user:
drh
branch: trunk)

[6ebb4323]
part of check-in
[5a5ff4e3]
Change the name of IOCAP_SAFE_DELETE to IOCAP_UNDELETABLE_WHEN_OPEN. Have the xDeviceCharacteristics() method of the win32 VFS return this flag.
(user:
dan
branch: experimental)

[5968e0d7]
part of check-in
[b9b11855]
A different fix to [fc62af4523]. When changing from journal_mode=PERSIST or TRINCATE to some other rollback mode, delete the journal file only if a RESERVED lock can be obtained on the database file first.
(user:
dan
branch: experimental)

[7f0ebd5d]
part of check-in
[f80c3f92]
Bug fix: Only trust the database size number at offset 28 if the change
counter at offset 24 matches the version number counter at offset 92.
This prevents corruption in the case of two applications writing to the
database where one is an older version of SQLite and the other is a newer
version.
(user:
drh
branch: trunk)

[133cc49d]
part of check-in
[bede8c8a]
Experimental change: On systems where it is not possible to unlink a file while one or more processes has it open (i.e. not unix), avoid closing the journal file each time the database is unlocked and reopening it at the start of each transaction.
(user:
dan
branch: experimental)

[fa5ce633]
part of check-in
[cb571c1b]
If an error occurs while writing frames to the log to commit a transaction, do not mark the pages as clean in the cache. Otherwise, the subsequent rollback does not roll them back (leaving the client to continue with a cache that makes it appear that the transaction was committed - inconsistent with the database on disk).
(user:
dan
branch: trunk)

[f7128f02]
part of check-in
[ef3ba7a1]
Initial code for incremental checkpoint in WAL mode. This check-in compiles
on unix and runs as long as you do not engage WAL mode. WAL mode crashes and
burns. Consider this check-in a baseline implementation for getting the new
capability up and running.
(user:
drh
branch: wal-incr-ckpt)

[0fbfe2cc]
part of check-in
[8a53f12c]
Change the checksum used in WAL files so that each frames checksum depends on the content of the WAL header and all frame headers and content up to and including the frame to which the checksum is attached.
(user:
dan
branch: trunk)

[d3284a6b]
part of check-in
[2b7e3b4a]
Add the SQLITE_FCNTL_SIZE_HINT operator to sqlite3_file_control() and use it
to give the VFS hints about the ultimate size of a database file when the
file is growing.
(user:
drh
branch: trunk)

[c78c9558]
part of check-in
[97d88a86]
Keep additional state information in the pager in an effort to reduce the
number of SQLITE_FCNTL_SIZE_HINTs.
(Note: This change was found to make no performance difference and so
has been moved onto a side branch. Use the parent check-in instead of
this one.)
(user:
drh
branch: branch-3.6.1-exp)

[a7470e87]
part of check-in
[9a083711]
Invoke the SQLITE_FCNTL_SIZE_HINT opcode on the sqlite3_file_control()
interface for database files before extending the size of the file. The
VFS can use this hint to preallocate space.
(user:
drh
branch: branch-3.6.1)

[1e163a82]
part of check-in
[2b00152c]
Refactoring the VFS-SHM methods used by WAL. This version compiles and
runs non-WAL test cases but crashes and burns on wal.test.
(user:
drh
branch: wal-refactor)

[4ff50f4e]
part of check-in
[33cabf27]
If, after obtaining a SHARED lock, there exists a *-wal file in the file-system, use WAL mode. This is necessary to recover from a crash that damages the first page of the database file.
(user:
dan
branch: wal)

[344eddf3]
part of check-in
[0ae91b00]
Do not sync any files in wal mode if "PRAGMA synchronous=off" is set. If files are synced, pass either SQLITE_SYNC_FULL or SQLITE_SYNC_NORMAL to the xSync() callback as configured by "PRAGMA fullfsync".
(user:
dan
branch: wal)

[751ada65]
part of check-in
[9bc9b684]
In synchronous=normal mode, do not sync the log after every transaction. In synchronous=full mode, sync the log and add any extra frames required to avoid blast-radius related problems after each transaction.
(user:
dan
branch: wal)

[da5ed17b]
part of check-in
[51a61395]
Fix a problem where a process in exclusive mode could delete a hot-journal file without rolling it back from within sqlite3_close() or DETACH. This problem was introduced by commits from the past few days; it is not present in any release.
(user:
dan
branch: trunk)

[7ea0549f]
part of check-in
[b1e7a1a6]
Do not journal the content of freelist leaf pages. This restores an
optimization that was accidently undone by check-in [12f7346c13] on
2009-01-20.
(user:
drh
branch: trunk)

[af1c58b7]
part of check-in
[b844ac6f]
Experimental changes that cause SQLite to use bytes 28..31 of the database
header to determine the database size, rather than using the actual database
size. This allows database space to be preallocated.
(user:
drh
branch: experimental)

[1915e3ec]
part of check-in
[ca0bc2a2]
After any rollback that modifies the database file, sync the database
before deleting the rollback journal. This is a cherry-pick merge
of check-ins [b21b911f23] and [f2326dad4e]
(user:
drh
branch: branch-3.6.23)

[2a374f72]
part of check-in
[b21b911f]
After a rollback that writes or truncates the database file, sync the database file before invalidating the journal contents. Fix for [015d3820f2].
(user:
dan
branch: trunk)

[f3d8a68d]
part of check-in
[69d749d9]
Make sure the in-journal boolean vector is cleared and released when ending
a transaction. This fixes an obscure problem with the previous check-in.
(user:
drh
branch: trunk)

[8e5cefa3]
part of check-in
[b78e58ae]
Do not create, open, or initialize the rollback journal until something actually
needs to be written into the journal. That way, expensive filesystem
operations are avoided if the transaction ends up being a no-op.
(user:
drh
branch: trunk)

[db7a3988]
part of check-in
[59f75bba]
Store the database size in pages in bytes 28..31 of the header. Currently
this is for forensic use only, but it might be used in the future.
(user:
drh
branch: trunk)

[aafc314d]
part of check-in
[81ff698f]
Make sure the dbFileVers field in the Pager object is properly initialized
even if there is an I/O error while reading its content off of disk.
(user:
drh
branch: trunk)

[71d2a707]
part of check-in
[46f406b2]
Merge in all of the logging enhancements. This is a cherrypick merge of
the following check-ins: [103321e37a], [a8076aede3], [6d910245ad],
[7c4cca6d1a], [edea3bb740], [1a6d4bb130], [a8c984c1d6], [69a493182f], and
[1168763d2c].
(user:
drh
branch: branch-3.6.22)

[d0e53cd3]
part of check-in
[86d50ce5]
Fix an assertion-fault/segfault problem that comes up when trying to
VACUUM an auto-vacuumed database with a large schema.
Ticket [da1151f97df244].
(user:
drh
branch: trunk)

[5dee83c3]
part of check-in
[f72f8a87]
Allow the secure-delete setting to be changed at run-time using a pragma.
The SQLITE_SECURE_DELETE compile-time option determines the default setting.
(user:
drh
branch: trunk)

[a5c0fa9b]
part of check-in
[f3398fc4]
Fix a bug in transaction rollback during backup when the CODEC is enabled
that can result in a corrupt backup file. This is only a problem when
SQLITE_HAS_CODEC is defined.
(user:
drh
branch: trunk)

[3c408c64]
part of check-in
[2f42f91f]
When moving pages as part of autovacuum on an in-memory database, make sure
that the source location is journalled so that a ROLLBACK can occur.
Part of the fix for ticket [564d412f15a00]
(user:
drh
branch: trunk)

[4adc8baf]
part of check-in
[9a429349]
Avoid unnecessary page cache allocations when move a page while autovacuuming
an in-memory database, since the allocation might fail making it impossible to
rollback the transaction.
(user:
drh
branch: trunk)

[5b5a980a]
part of check-in
[ddb71cd9]
Fix the backup API so that a backup from an empty database to a non-empty
database works. Ticket [0bf974bdf9]. The only changes are in assert()
statements.
(user:
drh
branch: trunk)

[ebd0a8f2]
part of check-in
[5e2f5496]
More thorough initialization of a buffer when writing the rollback journal
header, in order to prevent a harmless valgrind warning.
CVSTrac ticket #4039
(user:
drh
branch: trunk)

[4aa16cc8]
part of check-in
[224bc69a]
If there is a hot-journal in the file-system, a connection with journal_mode=memory set may have to open it to effect rollback. Account for this in pager_end_transaction(). This prevents an assert from failing in the in-memory journal permutation test. (CVS 6947)
(user:
danielk1977
branch: trunk)

[36a14ae2]
part of check-in
[886e665f]
The noReadlock property on a pager object implies the readOnly property.
Use this fact to simplify the logic in sqlite3PagerSharedLock(). (CVS 6944)
(user:
drh
branch: trunk)

[f050740b]
part of check-in
[1d931f77]
Change an unreachable condition in PagerSharedLock to a NEVER(...). Add an assert to pager_error() to show that it is never called to put an in-memory pager to the error-state. (CVS 6941)
(user:
danielk1977
branch: trunk)

[23c9823d]
part of check-in
[cc9430e3]
Add a NEVER() on an always-false conditional in pager.c.
Make sure the text of the master journal file is zero-terminated before
trying to process it - to prevent a buffer overrun in strlen(). (CVS 6937)
(user:
drh
branch: trunk)

[e26d9c5d]
part of check-in
[3b6d370e]
Add assert() statements to more tightly constrain the state of pager.c.
Remove the three pager*.test scripts since they violate the constraints
asserted above by modifying the state of the system in ways that it cannot
be modified in a live system. (CVS 6933)
(user:
drh
branch: trunk)

[283d4e9c]
part of check-in
[1c2bfc43]
Remove the SQLITE_CommitBusy flag. This was an attempt to block recursion
on the sqlite3_commit_hook() interface. But such recursion is explicitly
disallowed, so the flag is pointless. (CVS 6889)
(user:
drh
branch: trunk)

[3bad5d2b]
part of check-in
[b9be365d]
In sqlite3PagerWrite(), do not set the PGHDR_NEED_SYNC flag on a page if an IO error occured while attempting to journal it. (CVS 6887)
(user:
danielk1977
branch: trunk)

[e442d1ca]
part of check-in
[d0964b93]
When an OOM fault occurs while moving a page for autovacuum in an in-memory
database, be sure to undo the page move prior to returning the error, to
avoid corrupting the in-memory database file. (CVS 6883)
(user:
drh
branch: trunk)

[18b9bfe8]
part of check-in
[31a5e819]
Fix the pager so that it remembers the name of the journal for an in-memory
database as an empty string. This is important when committing a multi-database
transaction that includes one or more attached memory databases. (CVS 6852)
(user:
drh
branch: trunk)

[aff52317]
part of check-in
[a5ecffcf]
Avoid writing the 8-byte journal-header magic until the journal-header is synced. In persistent journal-mode, this prevents any old content that follows an unsynced journal-header from being interpreted as part of the rollback journal. (CVS 6817)
(user:
danielk1977
branch: trunk)

[2fc3693e]
part of check-in
[4b46805c]
When recovering from the error-state in pagerSharedLock(), do not search for a hot-journal in the file-system if the pager is configured to use an anonymous temp file as the journal. (CVS 6793)
(user:
danielk1977
branch: trunk)

[f6d5aa97]
part of check-in
[9a066600]
If after obtaining an exclusive lock to rollback a hot-journal file it is found that the journal file has been deleted, do not return an SQLITE_BUSY error. Just downgrade the lock and continue with the current operation. This eliminates a spurious SQLITE_BUSY error caused by a race condition. (CVS 6792)
(user:
danielk1977
branch: trunk)

[50765577]
part of check-in
[62db08bc]
If a call is made to sqlite3PagerAcquire when there are no outstanding references to any pages and the pager is in the error-state, try to exit the error-state at this point. Previously this was only attempted if the pager was configured to use exclusive mode. (CVS 6788)
(user:
danielk1977
branch: trunk)

[efdd56a4]
part of check-in
[73ba841a]
Honor the SQLITE_OPEN_READONLY flag even on a :memory: or temporary database,
even though such a database is pointless. Ticket #3908. (CVS 6748)
(user:
drh
branch: trunk)

[6c0fc1c0]
part of check-in
[d6b5d8e1]
Modify the hasHotJournal() routine to return a false-positive if it is
unable to open the journal file to check its header due to a race
condition. Processing downstream of hasHotJournal() already knows how to
deal with false-positives. Ticket #3883. (CVS 6687)
(user:
drh
branch: trunk)

[8bf62fef]
part of check-in
[90309008]
When locking a database file, check if any backup objects need to be restarted even if the page cache is empty. Ticket #3858. (CVS 6632)
(user:
danielk1977
branch: trunk)

[4c9c51dd]
part of check-in
[38f8c5a1]
When SQLite is configured to use in-memory temp files (either by setting SQLITE_TEMP_STORE during compilation or using the temp_store pragma at run time), store statement/savepoint journals in memory also. Ticket #3825. (CVS 6575)
(user:
danielk1977
branch: trunk)

[1c41ddee]
part of check-in
[ad8aba3e]
Avoid changing the pager journal-mode once a write-transaction is open, even if no database pages have been modified. (CVS 6554)
(user:
danielk1977
branch: trunk)

[adfdf98f]
part of check-in
[419e320a]
Change the journal_mode pragma so that it always returns the current
journal mode, even on a failed attempt to change the journal mode.
Allow the journal mode to be changed as long as there is not a pending
transaction. Ticket #3811. (CVS 6526)
(user:
drh
branch: trunk)

[9775c0db]
part of check-in
[4447c94f]
In sqlite3PagerMovepage(), avoid holding a reference to page number X when calling sqlite3PcacheRekey() to move another page to page number X. (CVS 6492)
(user:
danielk1977
branch: trunk)

[6a932669]
part of check-in
[80241a05]
Check that the first byte of a potentially hot journal file is non-zero before beginning hot-journal rollback. Fix for #3751 and #3745. (CVS 6390)
(user:
danielk1977
branch: trunk)

[da7ffd06]
part of check-in
[db1d4d2f]
Use the ROUND8() macro to round an integer up to the nearest multiple of 8 and ROUNDDOWN8() macro to round down to the nearest multiple of 8. This is a cosmetic change. (CVS 6372)
(user:
danielk1977
branch: trunk)

[eede73b7]
part of check-in
[59a47551]
Make sure result variables are initialized or unused following a failure
of the xFullPathname VFS interface. Problem detected by valgrind. (CVS 6295)
(user:
drh
branch: trunk)

[57665d91]
part of check-in
[315a6692]
Fix a case where during a rollback triggered by an IO or malloc error an unjournalled region of the database could be written to (with it's original data). This was causing an assert in test_journal.c to fail. Add a test case in ioerr2.test to trigger this case. (CVS 6286)
(user:
danielk1977
branch: trunk)

[8c946cca]
part of check-in
[d4af60e5]
If the truncate operation fails in journalmode=TRUNCATE, do not fall back
to overwriting the journal header with zeros. Simply fail the operation. (CVS 6257)
(user:
drh
branch: trunk)

[ce4a0a0f]
part of check-in
[20bd7605]
Do not attempt to use the sub-journal file descriptor if it is not opened (as in journal_mode=off mode). Ticket #3636. (CVS 6252)
(user:
danielk1977
branch: trunk)

[b8f4ae64]
part of check-in
[7f827ba9]
Correction to check-ins (6246) and (6247): The backup object might not
hold a valid destination connection pointer. Also, do not reset the
page cache when establishing a read-lock while there is a persistent
or truncated journal, only if there is a journal that really needs to
rollback. Otherwise backups always reset whenever the source database
file is read. (CVS 6248)
(user:
drh
branch: trunk)

[e8e2ae1e]
part of check-in
[9f07d2d9]
Fix a change-counter bug similar to #3584. This one is much more obscure though, requiring a transient IO or malloc error to occur while running in exclusive mode. (CVS 6189)
(user:
danielk1977
branch: trunk)

[46ab8721]
part of check-in
[b676ccfd]
Add asserts to detect if a transaction commits without first incrementing
the transaction counter. These asserts are intended to prevent future
problems similar to ticket #3584. (CVS 6179)
(user:
drh
branch: trunk)

[add3461c]
part of check-in
[b7d2a477]
Fix a problem with committing a transaction while there are other active statements. Sometimes, the database change counter was not being updated. (CVS 6176)
(user:
danielk1977
branch: trunk)

[5a95868f]
part of check-in
[36fe9a7a]
If the journal file is open when unlocking the database, close it just before unlocking the database file instead of just after. This may fix #3572. (CVS 6171)
(user:
danielk1977
branch: trunk)

[2fd6d5a8]
part of check-in
[15b5b5f9]
Conjecture: a journal header with nRec==0 must be the last header in the
journal. Add asserts to make this conjecture explicit. (CVS 6132)
(user:
drh
branch: trunk)

[151cc073]
part of check-in
[3e9efb76]
Reinitialize page 1 of the database file following a ROLLBACK TO of a
transactional SAVEPOINT on an initially empty database. (CVS 6103)
(user:
drh
branch: trunk)

[59b87841]
part of check-in
[ee0e6eae]
Changes to test scripts so that veryquick.test runs with SQLITE_TEMP_STORE=3 defined. Also a fix to stop the same switch causing a crash in the savepoint code. (CVS 6053)
(user:
danielk1977
branch: trunk)

[4fc24d29]
part of check-in
[bb177e30]
Add a case to permutations.test to run tests with the test_journal.c backend installed. Also many fixes to test_journal.c and one quite obscure fix to pager.c. (CVS 6052)
(user:
danielk1977
branch: trunk)

[5c2b49c3]
part of check-in
[68833626]
Fix the crash-followed-by-corruption bug revealed by savepoint4.test. This is actually the same bug as was fixed by (6043). The fix was not entirely correct. (CVS 6047)
(user:
danielk1977
branch: trunk)

[9c1663c9]
part of check-in
[98a53d91]
Add savepoint2.test, a file containing savepoint tests similar to tests in trans.test and avtrans.test. And a few savepoint bug fixes. (CVS 6039)
(user:
danielk1977
branch: trunk)

[64269ef1]
part of check-in
[c872d554]
Never use strlen(). Use our own internal sqlite3Strlen30() which is
guaranteed to never overflow an integer. Additional explicit casts to
avoid nuisance warning messages. (CVS 6007)
(user:
drh
branch: trunk)

[a193da9e]
part of check-in
[2822cbb9]
Fully initialize the unused bytes of the buffer that will become the journal
file header, in order to silence a complaint from valgrind. (CVS 5968)
(user:
drh
branch: trunk)

[8e2ebfd2]
part of check-in
[faded96f]
On a ROLLBACK, if there page cache entries which are dirty but not in the
rollback journal, make sure they get reinitialized in the btree layer. (CVS 5936)
(user:
drh
branch: trunk)

[847de90e]
part of check-in
[dc5308c7]
If a hot-journal file is detected but the application does not have the required read/write permissions, return SQLITE_CANTOPEN. Prior to this change, SQLITE_BUSY was returned. Ticket #3457. (CVS 5849)
(user:
danielk1977
branch: trunk)

[fd11076a]
part of check-in
[39ebf01a]
Add "pragma journal_mode=memory". Change the way rollback works for in-memory databases so that it reuses the journal_mode=memory code. (CVS 5830)
(user:
danielk1977
branch: trunk)

[d98f5612]
part of check-in
[22d1feb9]
Fix a problem with hot-journal rollback. SQLITE_CANTOPEN was being returned if the hot-journal file contained a pointer to a master journal file that did not exist. (CVS 5775)
(user:
danielk1977
branch: trunk)

[44eba010]
part of check-in
[40425e93]
Instead of storing a pointer to the parent page in the MemPage structure, have each B-Tree cursor keep track of the ancestry of the current page. (CVS 5747)
(user:
danielk1977
branch: trunk)

[34318b85]
part of check-in
[a7fd9e62]
Speed improvements for in-memory databases by omitting flag clearing on pages
where it is impossible for the flag to be set and by avoiding assert()s on
non-debugging builds. Ticket #3384. (CVS 5715)
(user:
drh
branch: trunk)

[f21654d1]
part of check-in
[aa5c9455]
Do not attempt to change the page size after a pager has entered the error state (Pager.errCode!=SQLITE_OK). This prevents an assertion failure in vacuum3.test. (CVS 5703)
(user:
danielk1977
branch: trunk)

[66836244]
part of check-in
[226a9056]
All the page_size pragma to change the page size on a new :memory: database,
but not a vacuumed :memory: database. Ticket #3335 (CVS 5617)
(user:
drh
branch: trunk)

[67d689ef]
part of check-in
[4ad096bd]
Fix bug in the premutation testing that was causing many permutations from
begin skipped. There are now 16 errors reported by the permutation test. (CVS 5610)
(user:
drh
branch: trunk)

[a175ce0a]
part of check-in
[16f612d6]
If the sector size is greater than the database page size, SQLite journals all pages that lie within a sector before writing to any of them. This change ensure that a journal sync does not occur halfway through journalling the set of pages that belong to a single sector. (CVS 5605)
(user:
danielk1977
branch: trunk)

[3b6625d3]
part of check-in
[93dbc542]
When recycling a page, try to find one that does not require a call to xSync() on the journal file. Also simplify some of the mutex related things in pcache. (CVS 5597)
(user:
danielk1977
branch: trunk)

[3a4358c7]
part of check-in
[d025866b]
Fix the functionality associated with sqlite3_release_memory() and sqlite3_soft_heap_limit(). It is automatically disabled if the SQLITE_CONFIG_PAGECACHE option is used. (CVS 5576)
(user:
danielk1977
branch: trunk)

[9f813a8f]
part of check-in
[cb494e10]
Add the pcache module from the experimental branch. Also change things so that most of the built-in SQL functions are kept in single static hash-table, rather than creating and populating a separate hash table for each open database connection. (CVS 5566)
(user:
danielk1977
branch: trunk)

[6ad4d2b9]
part of check-in
[70f20425]
If xAccess() fails while attempting to detect a hot-journal file, do not assume that the error was an out-of-memory condition. (CVS 5515)
(user:
danielk1977
branch: trunk)

[bb286b2f]
part of check-in
[30825f74]
Make the btree layer robust when faced with a corrupt database that
contains duplicate entries on the freelist. Ticket #3209. (CVS 5392)
(user:
drh
branch: trunk)

[4c4d0529]
part of check-in
[71d699de]
Make the Pager.pAll linked-list doubly linked when ENABLE_MEMORY_MANAGEMENT is defined. This is used to speed up sqlite3_release_memory(). (CVS 5346)
(user:
danielk1977
branch: trunk)

[e2a49872]
part of check-in
[56c8af14]
Add a mode to the sqlite3_test_control() interface to register hooks called at the beginning and end of "benign malloc failure" blocks. This allows malloc() failure testing to be done using public APIs only. (CVS 5254)
(user:
danielk1977
branch: trunk)

[5599fb3e]
part of check-in
[65fe7b62]
Continuing work on the new memory allocation subsystem.
Added routines for temporary memory allocation. Right the btree
balance mechanism to only do one temporary allocation at a time. (CVS 5220)
(user:
drh
branch: trunk)

[1f3e20af]
part of check-in
[de8b87d6]
Avoid attempting to delete the journal file of a temporary pager when closing the pager. It will be deleted automatically by the OS layer. (CVS 5193)
(user:
danielk1977
branch: trunk)

[b07ae2a2]
part of check-in
[5173b3e8]
Remove the xGetTempname() method from the vfs structure. Temp files are now opened by passing a NULL pointer as the filename to xOpen(). (CVS 5190)
(user:
danielk1977
branch: trunk)

[d55885a3]
part of check-in
[5c59f469]
Add "pragma journal_size_limit", used to limit the space consumed by persistent journal files left in the file-system after a transaction has concluded in exclusive (or journal_mode=persist) mode. (CVS 5185)
(user:
danielk1977
branch: trunk)

[baf3c74d]
part of check-in
[ffd4e50a]
Do not attempt to write a master journal name to the (non-existent) journal file in "no-journal" mode. Fix for #3127. (CVS 5145)
(user:
danielk1977
branch: trunk)

[2607309c]
part of check-in
[a9d1d931]
Make the benign-fault setting recursive. Make all malloc failures
during a rollback benign since there is nothing we can do about them. (CVS 5128)
(user:
drh
branch: trunk)

[22a5a810]
part of check-in
[4a1ae9d0]
Fix a bug whereby the database file was not always being extended to its original size when rolling back an incremental-vacuum operation. (CVS 5089)
(user:
danielk1977
branch: trunk)

[90df64b9]
part of check-in
[709d17b1]
Fix an assert() failure that can occur if "journal_mode=off" is used in a build with the atomic-write optimization enabled. (CVS 5043)
(user:
danielk1977
branch: trunk)

[c768b505]
part of check-in
[277e4099]
Continuing work on journal_mode. Journal_mode=persist now appears to be
working, though additional testing would be welcomed. (CVS 5033)
(user:
drh
branch: trunk)

[13f1aca7]
part of check-in
[8efb7f4f]
In exclusive locking mode, commit by zeroing the first 28 bytes of the
journal file, not by truncating the journal. Overwriting is much faster
than truncating. (CVS 5023)
(user:
drh
branch: trunk)

[7b302d90]
part of check-in
[fd97f876]
Change the allocator in mem2.c (used when SQLITE_MEMDEBUG is defined) so that allocations are not rounded up to the nearest 4 byte boundary. Fix a couple of errors in malloc.test related to sqlite3OsAccess() returning -1. (CVS 4956)
(user:
danielk1977
branch: trunk)

[f8f61c34]
part of check-in
[17ea0c97]
Change the pager's sector size algorithm to the maximum of the size
reported by xSectorSize() from the VFS and 512. It was using the
maximum of xSectorSize() and the current page size, but that was adding
an extra 512 bytes to the size of the journal file in the common case. (CVS 4929)
(user:
drh
branch: trunk)

[22241b59]
part of check-in
[871df0e7]
Allow a VACUUM operation to change the page-size in the same way as it can be used to change a database between auto-vacuum and normal mode. (CVS 4896)
(user:
danielk1977
branch: trunk)

[2ebd8957]
part of check-in
[72c40726]
Put the statement journal in the temp-file directory since that
directory is often on optimized storage such as RAM disk and because
unlike the main journal, the statement journal does not need to
be colocated with the database file. (CVS 4868)
(user:
drh
branch: trunk)

[803d361f]
part of check-in
[b4c1258e]
Correctly handle I/O errors that occur during OsUnlock(). Before this
fix, an I/O error during OsUnlock() could lead to database corruption.
That is not a serious problem, though, since errors during OsUnlock()
are not possible on most systems. (CVS 4838)
(user:
drh
branch: trunk)

[d9aeb0a1]
part of check-in
[7c57bdbc]
Add the Bitvec object for tracking which pages have been journalled.
This reduces memory consumption and runs faster than the bitmap approach
it replaced. (CVS 4794)
(user:
drh
branch: trunk)

[0cb6ccea]
part of check-in
[4e91a267]
Change all instances of "it's" in comments to either "its" or "it is",
as appropriate, in case the comments are ever again read by a pedantic
grammarian. Ticket #2840. (CVS 4629)
(user:
drh
branch: trunk)

[2f420d46]
part of check-in
[c9eb6591]
In shared-cache mode, make sure the busy hander invoked is the
busy handler associated with the database connection that caused
the lock contention in the first place. (CVS 4598)
(user:
drh
branch: trunk)

[bb524fe4]
part of check-in
[98960132]
Change the BTree so that it uses the Pagers temporary page space when
reorganizing the rows on a page, rather than mallocing for space of
its own. In this way, we avoid having to deal with a malloc failure
deep down inside the page reorganizer. Ticket #2806 (CVS 4577)
(user:
drh
branch: trunk)

[d246d888]
part of check-in
[8b29f5fb]
Add a parameter to specify the size of the output buffer passed to xGetTempname() and xFullPathname(). This, and the previous commit, are changes to the public vfs API introduced in 3.5.0. (CVS 4433)
(user:
danielk1977
branch: trunk)

[f38ad4de]
part of check-in
[ad3687b1]
Change the names of xGetTempName and sqlite3OsGetTempName to xGetTempname. To be consistent with xFullPathname and sqlite3OsFullPathname. (CVS 4432)
(user:
danielk1977
branch: trunk)

[70b4b9b6]
part of check-in
[44d8d1e9]
Base the name of the statement journal on the original database filename.
Remember the statement journal name for the lifetime of the Pager so
that the name pointer passed to xOpen persists as long as the file
exists. (CVS 4410)
(user:
drh
branch: trunk)

[10a95d93]
part of check-in
[2b755def]
Allocate page cache headers and page cache data buffers separately.
The data buffer will be a power of two in size and this gives some
malloc implementation additional optimization opportunitites. (CVS 4409)
(user:
drh
branch: trunk)

[ac52bb1e]
part of check-in
[fb27692a]
Remove the syncOk argument to pager_recycle. Now that sqlite3_memory_release uses a global lru list of page, it is no longer required. (CVS 4364)
(user:
danielk1977
branch: trunk)

[9b898267]
part of check-in
[306586c4]
Remove the xLockState method for sqlite3_io_methods. Replace it with
a defined call to xFileControl(). This simplifies the interface and
also gives us coverage testing of sqlite3_file_control(). (CVS 4355)
(user:
drh
branch: trunk)

[0c5fd990]
part of check-in
[d3ab3e39]
Initial implementation of the sqlite3_file_control() interface.
Compiles and passes all historical tests but the new method is itself
untested. (CVS 4353)
(user:
drh
branch: trunk)

[847674a7]
part of check-in
[481fd3a8]
Add tests for malloc() failure when the atomic-write optimization is being used. Also fix things so that rollback can occur without a malloc(), even if there is a master journal file. (CVS 4330)
(user:
danielk1977
branch: trunk)

[f5c2a4e8]
part of check-in
[b8451da3]
Remove nRef and vfsMutex from the sqlite3_vfs structure. Omit the
sqlite3_vfs_release() interface. Less memory allocated for a pager
in the common case where the size of the pathname is less than MAX_PATH. (CVS 4287)
(user:
drh
branch: trunk)

[ddb0279c]
part of check-in
[deb7ecd6]
Half-way through a major refactoring of the memory allocation.
I have not even attempted to compile so I am certain there are
countless errors. (CVS 4231)
(user:
drh
branch: trunk)

[05ea9dcb]
part of check-in
[af9503da]
Modify the crash-recovery test code in test6.c for 3.5. Also change some other code to use the new sqlite3_io_methods interface. Lots of things are broken now. (CVS 4228)
(user:
danielk1977
branch: trunk)

[e4fec09c]
part of check-in
[7ed2f59e]
Fix the corruption problem of ticket #2565 as demonstrated by the test added
in (4204). There may yet be other instances of similar problems lurking in
the code. (CVS 4206)
(user:
drh
branch: trunk)

[79a4587c]
part of check-in
[31dba1e9]
Cleanup unused function in pager.c. This is the conclusion of the fix
to ticket #2518. We believe the fix is complete and correct. (CVS 4165)
(user:
drh
branch: trunk)

[d7e2b57f]
part of check-in
[57bf8204]
A minor logic correction in the previous check-in. Also added a lengthy
comment describing the meanings of various flags in the
{quote: PgHdr} structure. (CVS 4080)
(user:
drh
branch: trunk)

[a8e5c2fd]
part of check-in
[c9dcf2b9]
Fix a database corruption problem that can occur in auto-vacuum mode when
a malloc() failure causes a statement rollback, additional statements
are run in the same transaction, then the total transaction rolls back. (CVS 4079)
(user:
drh
branch: trunk)

[42cffb2e]
part of check-in
[ce2c9925]
Fix for #2409. Return SQLITE_IOERR_BLOCKED instead of SQLITE_BUSY in cases where failure to obtain a database lock leaves the cache in an inconsistent state.
See additional information at CorruptionFollowingBusyError. (CVS 4060)
(user:
danielk1977
branch: trunk)

[f01e0e57]
part of check-in
[ba4845b3]
Eliminate all uses of sprintf() and strcpy(). These were not being
misused. But getting rid of them removes a library dependency. And
it avoids warnings from the OpenBSD compiler. Ticket #2336. (CVS 3916)
(user:
drh
branch: trunk)

[cd2770b0]
part of check-in
[d0745a43]
In the pager, load the content of pages which were initialized with
noContent==1 if they are subsequently requested with noContent==0. (CVS 3875)
(user:
drh
branch: trunk)

[642f804b]
part of check-in
[9dc4100e]
Always truncate the pager cache when truncating the database file. Also reorganize the code to check the change-counter after first obtaining a shared lock. (CVS 3814)
(user:
danielk1977
branch: trunk)

[30e4a54e]
part of check-in
[22e10cc2]
If an IO error is encountered on a commit, close the journal so that it
persists and can (hopefully) rollback the failed transaction later. (CVS 3792)
(user:
drh
branch: trunk)

[15118c2b]
part of check-in
[f5fad52a]
Remove asserts on the existing of journal files in pager - asserts that
are not valid for certain kinds of simulated I/O errors or for async I/O. (CVS 3769)
(user:
drh
branch: trunk)

[f9131543]
part of check-in
[66b3ad09]
Refactoring the btree and pager routines into distinct two-phase commit
routines. We've always done a two-phase commit - this change is just
making that more apparent in the code. (CVS 3762)
(user:
drh
branch: trunk)

[b89ea0f5]
part of check-in
[0fb9af1d]
Change the name of PAGER_SECTOR_SIZE to SQLITE_DEFAULT_SECTOR_SIZE. Make
the new OS-layer interface routine for finding sector size optional. (CVS 3750)
(user:
drh
branch: trunk)

[2cc40af0]
part of check-in
[b24a6e7b]
Test handling of IO errors that occur in OsDelete() or OsTruncate() operations. Also use an anonymous file for temporary storage during a VACUUM. (CVS 3729)
(user:
danielk1977
branch: trunk)

[dec2b258]
part of check-in
[7dc76588]
Modify the interface to the pager sub-system in preparation for performing IO in blocks based on sector-size, not database page-size. (CVS 3705)
(user:
danielk1977
branch: trunk)

[3688e882]
part of check-in
[9f62ef1e]
Fix some problems with test cases in shared_err.test. Also a real bug causing a segfault after an IO error in pager.c. (CVS 3703)
(user:
danielk1977
branch: trunk)

[ccca82d0]
part of check-in
[ea8507a7]
Fix the pager so that it handles persistent I/O errors correctly.
Update the testing infrastructure so that it tests for persistent
I/O errors instead of just transient errors. (CVS 3690)
(user:
drh
branch: trunk)

[999194d2]
part of check-in
[5af61402]
Enhanced temp-file security under unix. There are no known attacks against
prior versions - this check-in is just an added precaution. (CVS 3687)
(user:
drh
branch: trunk)

[c78d1cc1]
part of check-in
[2ba5be31]
Changes to the btree and pager that reduce the amount of I/O when dealing
with the freelist. (1) Avoid journaling pages of a table that is being
deleted. (2) Do not read the original content of pages being pulled off
of the freelist. (CVS 3671)
(user:
drh
branch: trunk)

[e1ef884e]
part of check-in
[007ca283]
Add the undocumented and experimental I/O tracing interface. This
interface is likely to change and may be completely abandoned in the
near future. (CVS 3665)
(user:
drh
branch: trunk)

[d6ad66eb]
part of check-in
[36a2db96]
In the btree.c, when releasing the page1 lock, first make sure the
pager has not reset and released the lock already. This fixes a bug
introduced by (3549). (CVS 3563)
(user:
drh
branch: trunk)

[4dbd6a12]
part of check-in
[6806b9ec]
Make sure that the database size cache in the pager is invalidated whenever
the database is unlocked. A stale value in the database size cache can
result in database corruption on a heavily loaded system running
autovacuum. (CVS 3548)
(user:
drh
branch: trunk)

[5a153277]
part of check-in
[bf1afd01]
Update the vtab_err test to check for -DSQLITE_MEMDEBUG=1 and skip the tests
if missing. Pager is more careful to clear its file size cache. Remove
an assert() in VDBE that might fail on a corrupt database file. (CVS 3547)
(user:
drh
branch: trunk)

[3114c819]
part of check-in
[6324ea81]
Use the difference between the SQLITE_IOERR_SHORT_READ and SQLITE_IOERR_READ
returns from sqlite3OsRead() to make decisions about what to do with the
error. (CVS 3503)
(user:
drh
branch: trunk)

[a1293bb4]
part of check-in
[70501e4e]
Report the error SQLITE_CORRUPT instead of SQLITE_IOERR if unable
to rollback a hot journal that was damaged (for example) by filesystem
corruption following a power failure. (CVS 3460)
(user:
drh
branch: trunk)

[65bb3314]
part of check-in
[ba579ddc]
Add support for extended result codes - additional result information
carried in the higher bits of the integer return codes. This must be
enabled using the sqlite3_extended_result_code() API. Only a few extra
result codes are currently defined. (CVS 3422)
(user:
drh
branch: trunk)

[e51c079b]
part of check-in
[76e4d418]
Remove most static and global variables from a pure build. There
are still a number of static and global variables used for testing. (CVS 3342)
(user:
drh
branch: trunk)

[7ef4c509]
part of check-in
[fe3e70a7]
Write database pages in sorted order to reduce file fragmentation.
This is an experimental change. It is not clear if it will help any.
It may well be removed in the future. (CVS 3255)
(user:
drh
branch: trunk)

[7b1975e2]
part of check-in
[e510e6dd]
Code changes resulting from Coverity analysis.
http://scan.coverity.com/ Found 1 potential segfault in sqlite3_mprintf().
Also 2 failures to fclose() following a malloc() failure. And lots of
cases where unnecessary conditionals could be removed from the code. (CVS 3126)
(user:
drh
branch: trunk)

[c9fdc3f4]
part of check-in
[c30705a0]
Return code was being dropped because of overridden variable in OP_IsUnique. Fix this and the test logic problem that hid it. (CVS 3025)
(user:
danielk1977
branch: trunk)

[b101634a]
part of check-in
[77969c67]
Add "#ifndef NDEBUG" around sqlite3pager_iswriteable() (only used in assert() expressions). Also set the internal page number of a page to zero if it fails to initialise. (CVS 3005)
(user:
danielk1977
branch: trunk)

[651fda17]
part of check-in
[e1ad9df1]
Modify the sqlite3OsInMutex os-layer interface to take a single parameter
to distinguish between mutexes held by the current thread and mutexes held
by any thread. Part of the fix for ticket #1630. (CVS 2973)
(user:
drh
branch: trunk)

[a96b9c43]
part of check-in
[5d9c6aa9]
Automatically deallocate thread-specific data when it is no longer
being used. Ticket #1601. Also implemented the suggestion of
ticket #1603. Memory management is now off by default at compile-time.
The sqlite3_enable_memory_management() API has been removed. (CVS 2919)
(user:
drh
branch: trunk)

[07509ddb]
part of check-in
[6fdbb8b7]
Disable automatic invocation of sqlite3_release_memory() when a malloc() fails
in those cases where the global mutex is held. (CVS 2860)
(user:
danielk1977
branch: trunk)

[f1969539]
part of check-in
[e6106cc1]
Tentative fix for ticket #1567: disable the sqlite3pager_dont_write()
optimization when a statement transaction is active. We continue to look
for a better fix. (CVS 2827)
(user:
drh
branch: trunk)

[893cb210]
part of check-in
[e1606658]
Some elements of the new malloc() failure handling. Not all cases work properly yet. Also, library is not threadsafe if malloc() fails right now. (CVS 2800)
(user:
danielk1977
branch: trunk)

[bcb54653]
part of check-in
[bd8740d1]
Restructure the OS interface yet again. This time make the OsFile object
a virtual base class which is subclassed for unix, windows, and the crash
test simulator. Add the new file "os.c" for common os layer code. Move
all OS-specific routines into the sqlite3Os structure. (CVS 2795)
(user:
drh
branch: trunk)

[18cc5b27]
part of check-in
[058f3175]
Make the OsFile structure opaque with its internal structure known only to
the appropriate os_*.c implementation. Omit the os_unix.h and os_win.h
include files. The crash tests are broken by this patch. (CVS 2791)
(user:
drh
branch: trunk)

[561657a2]
part of check-in
[1f69aec2]
Modify the OS layer so that I/O routines are called through a virtual function
table. The os_test.c module has been replaced by test6.c. The crash tests
are busted now and need to be fixed. (CVS 2785)
(user:
drh
branch: trunk)

[68b05954]
part of check-in
[5b6dc12b]
Do not journal the locking page which doing an autovacuum. Similar to the
problem of #1432 except that this one occurs on autovacuum instead of vacuum.
An assert() has been added to catch any future incidents of this type. (CVS 2704)
(user:
drh
branch: trunk)

[26a642c1]
part of check-in
[856e2ec9]
When creating a new database, delete any preexisting journal that might be
left over from a prior database with the same name. Ticket #1152. (CVS 2387)
(user:
drh
branch: trunk)

[d0c1f41b]
part of check-in
[2155448d]
Add the (highly experimental) omit_readlock pragma that disables the use
of readlocks on read-only databases that are connected using ATTACH. (CVS 2317)
(user:
drh
branch: trunk)

[0523c8a1]
part of check-in
[081676e4]
Auto-vacuum: Ensure pages to be removed by database truncation are in the journal file. Also fix an sqlite3pager_movepage() bug. (CVS 2074)
(user:
danielk1977
branch: trunk)

[6b00c0d5]
part of check-in
[d12481f0]
Code to auto-vacuum the database if all root pages happen to be in the right place. Not active by default and largely untested. (CVS 2037)
(user:
danielk1977
branch: trunk)

[ae06c85d]
part of check-in
[d1b29156]
Transactions commit on the last sqlite3_step(), not on sqlite3_finalize().
This allows the sqlite3_step() to return SQLITE_BUSY if the commit is
blocked by a lock. Ticket #885. (CVS 1928)
(user:
drh
branch: trunk)

[88a7160c]
part of check-in
[b8aaa3a2]
Change the collation sequence interface to allow collation sequences that
use UTF-16 in non-native byte order to be registered. (CVS 1559)
(user:
danielk1977
branch: trunk)

[3fddd1e5]
part of check-in
[0c2d169c]
More work on windows locking. Fix some problems with unix locking. There
is still an assertion failure on windows locking in attach2.test. (CVS 1539)
(user:
drh
branch: trunk)

[97a675f1]
part of check-in
[8af6474c]
Change lots of internal symbols from sqliteXXX to sqlite3XXX so that the
library links again. It doesn't work yet, due to changes in the btree layer
calling convention. (CVS 1324)
(user:
danielk1977
branch: trunk)

[f2be6a1f]
part of check-in
[48832d35]
Fix inaccuracies and add details to comments in the pager. Change the name
of one function to make its purpose clearer. Ticket #599. (CVS 1209)
(user:
drh
branch: trunk)

[a4fd3a61]
part of check-in
[09c10fe3]
When creating a new journal file, open a (read-only) file descriptor on the
directory containing the journal and sync that directory once to make sure
that the journal filename entry gets into the directory. Ticket #410. (CVS 1066)
(user:
drh
branch: trunk)

[53abd9df]
part of check-in
[185d8dc8]
Simplify the BTree interface by shortening names. Added two new methods
for accessing the current filename and for changing the name of the
database file. (CVS 900)
(user:
drh
branch: trunk)

[bffccf85]
part of check-in
[3ef0ad8a]
Fix the pager so that correct rollbacks occur when synchronous is turned off.
This check-in also included unrelated documentation updates. (CVS 866)
(user:
drh
branch: trunk)

[4adf3cc6]
part of check-in
[8ec56325]
Modify the journal format to be more robust against garbage that might appear
in the file after a power failure. The changes are mostly working but more
testing is still required. This check-in is to checkpoint the changes so far. (CVS 861)
(user:
drh
branch: trunk)

[95f5c5c7]
part of check-in
[b6ea0da7]
Fix a bug in the rollback validation logic (code that only runs during
debugging). The problem was introduced by check-in (835). (CVS 849)
(user:
drh
branch: trunk)

[f35799e6]
part of check-in
[00f08fc0]
In the pager, cache a pointer to the first page on the freelist that does not
need to be synced. This makes a fetch of a page that is not in cache go a
lot faster when the cache is full. This check-in also adds some performance
instrumentation to the OS layer. (CVS 842)
(user:
drh
branch: trunk)

[08115562]
part of check-in
[717523d3]
Improvements to the pager to help large updates against a large database run
faster. Also improved the testing of the pager rollback algorithms. (CVS 835)
(user:
drh
branch: trunk)

[20ea9300]
part of check-in
[2f89e9e6]
Avoid calling truncate() and stat() on the checkpoint journal to improve the
speed of a large number of UPDATEs within a transaction. (CVS 819)
(user:
drh
branch: trunk)

[0cbbde8b]
part of check-in
[1c58b4fc]
Save the full pathname of the database file so that journalling still works
even if the user changes working directories after opening the databae.
Ticket #200. (CVS 798)
(user:
drh
branch: trunk)

[691571c4]
part of check-in
[745d6639]
Back out the changes in the pager that sorted pages prior to writing them
to the database. Additional measurements showed no performance gains. (CVS 785)
(user:
drh
branch: trunk)

[27d9b946]
part of check-in
[a6ef6657]
Two optimizations to the pager: (1) Write dirty pages back to the database
file in order and (2) Keep a separate list of in-memory pages that are in
the checkpoint journal in order to speed a checkpoint commit. (CVS 783)
(user:
drh
branch: trunk)

[292853d0]
part of check-in
[a29d60ec]
Try to better detect when the library is compiled for large file support (LFS)
but the support is not available in the host OS kernel. (CVS 782)
(user:
drh
branch: trunk)

[7d581b20]
part of check-in
[2008b56f]
Add large file support to Windows. Change large file support for Unix so
that it compiles automatically - without requiring special options on the
compiler command line. (CVS 781)
(user:
drh
branch: trunk)

[baf50d83]
part of check-in
[a3f67fe9]
Add support for databases larger than 2GB under Unix. Must be compiled
with -D_FILE_OFFSET_BITS=64 and -D_LARGEFILE_SOURCE in order to work with
larger databases. (CVS 778)
(user:
drh
branch: trunk)

[b8415b02]
part of check-in
[f6e24d5c]
More rollback problems: Fix two more errors introduced by checking (410) that
can cause database corruption after a ROLLBACK. Also add new tests to make
sure everything is working this time. (CVS 663)
(user:
drh
branch: trunk)

[10334160]
part of check-in
[db178646]
Fix for bug #94: Be sure to journal pages that are added to the freelist then
removed from the freelist and reused during the same transaction. (CVS 660)
(user:
drh
branch: trunk)

[58ae9f56]
part of check-in
[36fc0add]
Bug fix: When the database file grows in size during a transaction, make sure
the last page of the file gets written to disk even if that page is on the
free list and contains no data. Otherwise the disk file will be too small
which can lead to database corruption in subsequent transactions. (CVS 643)
(user:
drh
branch: trunk)

[1e41053c]
part of check-in
[7a24336d]
Once it is opened, leave the checkpoint journal file open for the
duration of a transaction, rather than closing it and reopening it
for each statement. (Ticket #53) (CVS 599)
(user:
drh
branch: trunk)

[e1419353]
part of check-in
[480eef1a]
Change the pager locking mechanism so that we don't have to write page 1
to the journal and to the database unless it actually changes. (CVS 419)
(user:
drh
branch: trunk)

[41af8790]
part of check-in
[414da4af]
Add the ability to turn of calls to fsync() using the "synchronous" pragma.
Increased the default cache size from 100 to 2000 and made the "cache_size"
pragma persistent. (CVS 418)
(user:
drh
branch: trunk)

[0aa358a3]
part of check-in
[cf1ebcfb]
Pager optimization: do not write or journal free pages. This results in
a 2x performance gain for large INSERTs and a 5x performance gain for
large DELETEs. (CVS 410)
(user:
drh
branch: trunk)

[f39d99e9]
part of check-in
[b0d21887]
Fix a race condition in the locking code that would sometimes cause
SQLITE_PROTOCOL or SQLITE_CORRUPT to be returned when SQLITE_BUSY should
have been returned. (CVS 326)
(user:
drh
branch: trunk)

[16173a7b]
part of check-in
[585ed5eb]
Bug fix: sqlite_exec() would sometimes return SQLITE_PROTOCOL when it
should have returned SQLITE_BUSY. There was also a deadlock that the
previous bug was masking. (CVS 322)
(user:
drh
branch: trunk)

[4f9e2b39]
part of check-in
[e7b65e37]
All tests now pass. But there are still issues. For example, inserts
are way too slow. And additional tests are needed for new features. (CVS 243)
(user:
drh
branch: trunk)

[1928e68b]
part of check-in
[6ecc8b20]
The code is in place to replace GDBM with BTree. But I have not yet
attempted to compile it. I am sure the code contains bugs. (CVS 238)
(user:
drh
branch: trunk)