2011.12.21

Any comp-sci grad worth his or her student loan debt can tell you about virtual memory. And many can tell you the intricacies of dealing with unix-style VM. But for the developers working with Mac OS X (and iOS), there’s a deeper layer hidden underneath that is seldom expressed (and often for good reason) – Mach. Mach, not Unix, is the true undercarriage of OS X and its ilk. And since it’s basically the only wide-spread Mach-based OS on the planet, there’s a relatively large void when it comes to dealing with this layer. There aren’t many reasons to drop down to this layer, but sometimes, for fun or for profit, the need may arise.

For this first post, let’s explore allocating, deallocating, and protecting memory pages.

The functions we’re interested in here are vm_allocate, vm_deallocate, and vm_protect. You can find them in /usr/include/mach/vm_map.h.

vm_allocate, unsurprisingly, allocates memory pages into a task (process). It takes a target task (usually mach_task_self()), an address pointer, a size, and some flags.

The address pointer is often best left unspecified. This allows the kernel to map in pages wherever there’s room. on 64 bit systems this isn’t particularly interesting, but for 32 bit processes address space can be fairly constrained.

Size is often a multiple of a page size (4k, or 4096 bytes). It defines how large of an area you want to map.

Flags is where things get interesting. There are several to choose from, tersely documented in vm_statistics.h. A couple notes on them:

VM_FLAGS_FIXED – This controls whether or not it should try to map the pages at the address you specified. It’s actually implied by the absense of VM_FLAGS_ANYWHERE. VM_FLAGS_ANYWHERE – This indicates that it’s ok to ignore the input address, and simply allocate the pages wherever they’d fit (and return the base address in address). VM_FLAGS_PURGABLE – This indicates that it’s ok for the pages to get purged (that is, discarded without first being written to disk). This can be useful for caches and other data that can be recomputed. I’ll discuss purging pages in a future update. VM_FLAGS_NO_CACHE – This controls how the pager deals with prioritizing pages in low-memory situations.

vm_deallocate is used to undo an allocation. Or in other words, free the memory vm_allocate makes use of. It takes a task (again, often mach_task_self()), a base pointer, and a size. It behaves largely how you might expect.

vm_protect is interesting (and potentially useful). With it you can mark pages are readable (VM_PROT_READ), writable (VM_PROT_WRITE), and executable (VM_PROT_EXECUTE). These are defined in /usr/include/mach/vm_prot.h. Newly created pages (via vm_allocate) are readable and writable, but not executable. There’s also a copy protection used for copy-on-write behavior, but that’s also for a future post.

And there you have it. Basic kernel-level memory allocation for your process. Now you can get page-aligned allocations of large slabs of memory (sometimes useful for storing GL textures with GL_APPLE_client_storage, among other things). Note that vm_allocate is _not_ faster than malloc/free. It is not wise to use these functions as a replacement for general allocation functions. Standard debugging tools will also not deal with these allocations (guard malloc won’t be able to help you, leaks won’t be able to help you, Instruments probably won’t be able to help you – vmmap will show them though, and you can tag them so that they stand out, using VM_MAKE_TAG as one of the vm_allocate flags).

It can also be useful for initializing some large set of data, and then marking it as read-only to ensure you don’t accidentally mutate it.