100 check-ins occurring around 26f2da24b080bf89.

In the MSVC makefile, allow symbols to be enabled without NDEBUG defined. Also, for the win32lock test, make sure the database is closed prior to shutdown.
Closed-Leaf
check-in: 5ed7633d user: mistachkin tags: winNativeHeap

Create a new pager type, PAGER_SORTER, for use in the external merge sort.
Such pagers are always held in memory but do report when they are under
memory pressure by calling pagerStress.
check-in: c71d7320 user: drh tags: trunk

In the MSVC makefile, support several levels of debugging, each one building on the previous. Also, add comment about the SQLITE_WIN32_MALLOC_VALIDATE macro.
check-in: 4257e9b7 user: mistachkin tags: winNativeHeap

Have the "crash" VFS used by the tcl tests (test6.c) handle SQLITE_FCNTL_SIZE_HINT internally, instead of passing it directly through to the underlying VFS. This is important if the crash VFS is simulating non-default device characteristics such as SQLITE_DEVCAP_SEQUENTIAL or ATOMIC.
check-in: fac8bc8f user: dan tags: trunk

When copying a rollback mode database over a WAL database using the backup API, leave the destination database in WAL mode (instead of switching it to rollback mode). Fix for [af95b8c609].
check-in: 35e6ac18 user: dan tags: trunk

Experimental change: If SQLITE_PAGECACHE_BLOCKALLOC is defined, instead of allocating pages one at a time, allocate blocks of between 15 and 63 pages in a single allocation.
check-in: 63597097 user: dan tags: pager-blockalloc

Use OP_Sort rather than OP_Rewind in order to sort the content of a new
index on the CREATE INDEX statement. Add an ALWAYS() to the merge sort
logic to document an unreachable branch.
check-in: f01766f4 user: drh tags: trunk

When opening a temporary file to use in a merge-sort, pass SQLITE_OPEN_TEMP_JOURNAL instead of SQLITE_OPEN_TEMP_DB. This is a better fit, as merge-sort files are written and read sequentially like journal files.
check-in: bd04756b user: dan tags: trunk

Further testing and bug fixing for sqlite_stat3. Added the Index.avgEq
field to index statistics. Fixed several problems in the query planner
associated with stat3.
check-in: 89b2f708 user: drh tags: stat3-enhancement

In os_unix.c, do not open the directory containing the database file when
the database file is opened. Instead, wait until time to fsync() the
directory. And do not report an error if the open fails, since some
systems (Ex: AIX and a chromium sandbox) are unable to open and fsync
directories.
check-in: 713b1b7d user: drh tags: trunk

Add code to actually use the sqlite_stat2.cnt field in the query planner.
This changes some plans resulting in a few failures in analyze5.test.
Closed-Leaf
check-in: d1248165 user: drh tags: query-planner-tweaks

The sqlite_stat2.cnt field is parsed if it is present. But it is not yet
used. A large comment added to analyze.c to explain the format of the
ANALYZE system tables.
check-in: 6d1e2372 user: drh tags: query-planner-tweaks

In temp files used for merge sorting, store the size of each packed-memory-array at the start of the array itself. This is to avoid having to store the offsets of all arrays in the (potentially very large) file in main-memory.
check-in: 8051c176 user: dan tags: experimental

Add a column to the sqlite_stat2 table that contains the number of entries
with exactly the same key as the sample. We do not yet do anything with this
extra value. Some tests in analyze2.test are failing.
check-in: eb434228 user: drh tags: query-planner-tweaks

Change to using packed-memory-arrays instead of b-trees when performing an offline merge-sort for CREATE INDEX. This makes it easier to control the number of disc seeks required when merging.
check-in: a4770d07 user: dan tags: experimental

Merge the winopen-retry-logic branch into trunk. The biggest change here
is to test scripts, which should now use such as copy_file and
delete_file from tester.tcl rather than the raw file commands of TCL.
check-in: b90c28be user: drh tags: trunk