Contents

DMTCP implements checkpoint/restore of a process on a library level. This means, that if you want to C/R some application you should launch one with DMTCP library (dynamically) linked from the very beginning. When launched like this, the DMTCP library intercepts a certain amount of library calls from the application, builds a shadow data-base of information about process' internals and then forwards the request down to glibc/kernel. The information gathered is to be used to create an image of the application. With this approach, one can only dump applications known to run successfully with the DMTCP libraries, but the latter doesn't provide proxies for all kernel APIs (for example, inotify() is known to be unsupported). Another implication of this approach is potential performance issues that arise due to proxying of requests.

Restoration of process set is also tricky, as it frequently requires restoring an object with the predefined ID and kernel is known to provide no APIs for several of them. For example, kernel cannot fork a process with the desired PID. To address that, DMTCP fools a process by intercepting the getpid() library call and providing fake PID value to the application. Such behavior is very dangerous, as application might see wrong files in the /proc filesystem if it will try to access one via its PID.

CRIU, on the other hand, doesn't require any libraries to be pre-loaded. It will checkpoint and restore any arbitrary application, as long as kernel provides all needed facilities. Kernel support for some of CRIU features were added recently, essentially meaning that a recent kernel version might be required.

Berkeley Lab Checkpoint/Restart (BLCR) is a part of the Scalable Systems Software Suite ,
developed by the Future Technologies Group at Lawrence Berkeley National Lab under SciDAC
funding from the United States Department of Energy. It is an Open Source, system-level
checkpointer designed with High Performance Computing (HPC) applications in mind: in particular
CPU and memory intensive batch-scheduled MPI jobs. BLCR is implemented as a GPL-licensed
loadable kernel module for Linux 2.4.x and 2.6.x kernels on the x86, x86_64, PPC/PPC64, ARM architectures, and a
small LGPL-licensed library.

PinLIT (Pin-Long Instruction Trace) is a checkpointing tool built on top of Intel's proprietary PIN binary instrumentation tool described on page 48 of Cristiano Pereira's PhD thesis. It records the processor's (big) architectural register state and all pages of memory that contain application and shared library code, optimizing size by only storing memory used during a desired interval.

PinPlay or the Program Record/Replay Toolkit appears to be the successor of or new name for PinLIT.

“looks\seems like yes/no” - i found only unproved message(s) saying “yes”/“no”

“not yet” - it is officially planned or i found no reasons, why it can’t be done.

CRIU

DMTCP

BLCR

OpenVZ

Arch

x86_64, ARM, AArch64, PPC64le

x86, x86_64, ARM

x86, x86_64, PPC/PPC64, ARM

x86, x86_64

OS

Linux

Linux

Linux

Linux

Uses standard kernel?

Yes, provided it's 3.11 or later

Yes

Yes, just needs to load module

No. OpenVZ kernel is required

Can be used without preloading special libraries before app start?

Yes

No

No

Yes

Can be used as non-root user?

Yes, but user can only manipulate tasks belonging to him

Yes

Yes

No

Can run unmodified programs?

Yes

Yes

No. Statically linked and/or threaded apps are unsupported.

Yes

Can run unprepared tasks?

Yes

No. It preloads the DMTCP library. That library runs before the routine main(). It creates a second thread. The checkpoint thread then creates a socket to the DMTCP coordinator and registers itself. The checkpoint thread also creates a signal handler.

No. CR shall notify processes when a checkpoint is to occur (before the kernel takes a checkpoint) to allow the processes to prepare itself accordingly.

Yes, but if all components are the same. Even if prelinked addresses are different, it will not restore, but it can save the whole used libs and localization files to restore program on the different machine

Yes

Containers

Yes, LXC and OpenVZ containers

No. It doesn't support namespaces, so it probably can’t dump containers

Looks like no

Yes

Parallel/distributed computations libraries

No (planned)

Yes. OpenMPI, MPICH2, OpenMP, Cilk are alredy supported and Infiniband is in progress