This is a prototype for compacting memory to reduce external fragmentationso that free memory exists as fewer, but larger contiguous blocks. Ratherthan being a full defragmentation solution, this focuses exclusively onpages that are movable via the page migration mechanism.

The compaction mechanism operates within a zone and moves movable pagestowards the end of. Grouping pages by mobility already biases the locationof unmovable pages is biased towards the lower addresses, so these strategieswork in conjunction.

A single compaction run involves two scanners operating within a zone - amigration and a free scanner. The migration scanner starts at the beginningof a zone and finds all movable pages within one pageblock_nr_pages-sizedarea and isolates them on a migratepages list. The free scanner begins atthe end of the zone and searches on a per-area basis for enough free pages tomigrate all the pages on the migratepages list. As each area is respecivelymigrated or exhaused of free pages, the scanners are advanced one area.A compaction run completes within a zone when the two scanners meet.

This is what /proc/buddyinfo looks like before and after a compaction run.

In this patchset, memory is never compacted automatically and is only triggeredby writing a node number to /proc/sys/vm/compact_node. This version of thepatchset is mainly concerned with getting the compaction mechanism correct.

The first patch is a roll-up patch of changes to grouping pages by mobilityposted to the linux-mm list but not merged into -mm yet. The second patchis from the memory hot-remove patchset which memory compaction can use.

The two patches after that are changes to page migration. The third patchallows CONFIG_MIGRATION to be set without CONFIG_NUMA. The fourth patchallows LRU pages to be isolated in batch instead of acquiring and releasingthe LRU lock a lot.

The fifth patch exports some metrics on external fragmentation which arerelevant to memory compaction. The sixth patch is what implements memorycompaction for a single zone. The final patch enables a node to be compactedexplicitly by writing to a special file in /proc.

This patchset has been tested based on 2.6.22-rc2-mm1 with the following;

o x86 with one CPU, 512MB RAM, FLATMEMo x86 with four CPUs, 2GB RAM, FLATMEMo x86_64 with four CPUs, 1GB of RAM, FLATMEMo x86_64 with four CPUs, 8GB of RAM, DISCONTIG NUMA with 4 nodeso ppc64 with two CPUs, 2GB of RAM, SPARSEMEMo IA64 with four CPUs, 1GB of RAM, FLATMEM + VIRTUAL_MEM_MAP

The x86 with one CPU is the only machine that has been tested understress. The others was a minimal boot-test followed by compaction underno load.

This patchset is incomplete. Here some outstanding items on a TODO list inno particular order.

o Have pageblock_suitable_migration() check the number of free pages properlyo Do not call lru_add_drain_all() on every updateo Add trigger to directly compact before reclaiming for high orderso Make the fragmentation statistics independent of CONFIG_MIGRATIONo Obey watermarks in split_pagebuddy_pageo Handle free pages intelligently when they are larger than pageblock_ordero Implement compaction_debug boot-time option like slub_debugo Implement compaction_disable boot-time option just in caseo Investigate using debugfs as the manual compaction trigger instead of proco Deal with MIGRATE_RESERVE during compaction properlyo Build test to verify correctness and behaviour under load