THREAD MANAGEMENT IN MY HOBBY OS

2015-10-26

I thought I'd write a little more about my home-grown x86-64 OS. This is about my threading mechanism.

The Task list

The global task list contains a list of 16bytes entries that contains the adress of the PML4 base
for the task and the value of the Ring0 RSP. Since task switches always occur during Ring0, only
the RSP0 needs to be saved (RSP3 will have been pushed on stack during privilege change).
Upons switching task, the scheduler will restore cr3 and rsp from the list.

Task State Segment

In 64bit long mode, the TSS doesn't have as much significance than in protected mode when hardware
context switching was available. The TSS is only used to store the RSP0 value. When a privilege
change is performed, from Ring3 to Ring0, the CPU will load the TSS.RSP0 value in RSP and the old
RSP3 value will have been pushed on the Ring0 stack. So the IRET instruction will pop out the Ring3
RSP value (hence why it is not needed in the TSS). The only time that the CPU will look into the TSS
is during a privilege switch from Ring3 to Ring0.

Only one TSS is needed in the system since the value it contains is a virtual address. Every task
will use the same address for RSP3 and RSP0 but it will be mapped differently.

So the TSS is only used to tell the process which address to use for the stack when going from Ring3
to Ring0. Since that address is the same virtual address for every task, only 1 TSS needs to be created.

There is no point in showing the TSS here since only one value is used. It's basically a 0x67 bytes long table
with a 64bit entry at byte index 4 that represents the RSP0 value. the rest of the table can be filled with zeros

Kernel and user tasks

Kernel tasks have the following properties:

Run in ring0.

Their Code Segment Selector is the same as the kernel.

Each kernel task has its own stack.

Code is located in kernel memory which is the identity-mapped portion of virtual memory.

The RSP value that is set in the Ring0 stack (for iret) upon task creation is the ring0 stack pointer.

User tasks have the following properties:

Run in ring3

Their Code Segment Selector is the same for all user thread.

Their Code Segment Selector has DPL=3

Each user task has 2 stacks: one for Ring0 and one for Ring3

Code is relocated at load time and appears to be running at virtual address 0x08000000

The RSP value that is set in the ring0 stack (for iret) upon task creation is the ring3 base stack pointer.

Interupts

When an interrupt occurs, the Trap or Interrupt Gate contains a DPL field and target Selector field. The DPL
will always be set to 3 so that any privilege levels (Ring0-Ring3) can access the interrupt handler. The target
selector is for a Code Segment Descriptor (inthe GDT) with a DPL value of 0. This will effectively run the
handler in Ring0.

Interrupt or task switch during user task

If the interrupt occured during a user task, the task may have been in Ring3 or in Ring0 if in the middle of a
system call. If a privilege change occurs, the value of RSP will be set to the value in the TSS for RSP0. That
value is the same for all tasks but mapped differently to physical memory. The ring3 stack will be left untouched.

If there is no privilege change, then the same stack will be used. The scheduler algorithm doesn't need to change
since the Ring3 stack's RSP has been pushed sometime before when entering Ring0 (ie: when calling a system
function)

Interrupt or task switch during kernel task

If the interrupt occured during a kernel task then nothing special happens. The same stack is being used and same
protection level is applied. Although it would be possible to use the new ITS mechanism to use another stack
during interrupt handling. This would be usefull, for example, when a stack overflow occurs in a kernel thread
and the #PF exception is raised. If there is not stack switch, then the return address can't be pushed on the stack
since the stack if faulty (hence the #PF). So using another stack for interrupts would be a wise choice. But
I'm not implementing this for now.

Task switch

A task switch will always be initiated by the timer iterrupt. The scheduler runs in Ring0 so the
Ring0 stack is used. If the interrupted task was in Ring3, we don't need to worry about its stack
since RSP has been pushed on the Ring0 stack.
The current CPU context will be saved on the Ring0 stack. Therefore, the scheduler will always save the RSP
value in the tasklist's rsp0 entry since it occurs in ring0 code. Upon resuming the task, RSP will be reloaded from
the rsp0 entry, context will be restored and the iret instruction will pop the ring3 RSP to resume execution.

In a multi-cpu system, it would be possible to have less threads running than the number of CPUs. In that case, upon
task-switching, a CPU would just continue to run its current thread, giving 100% of its time to that thread. But for
other CPUs that don't have any task to execute, they should get parked. Once a CPU's thread is terminated, if
there is no other threads to run for him, then it will parked. Normally the CPU would load the context
of the next task but instead it will simply jump to a park function. Since the scheduler is invoked
through an interrupt, we eventually need to "iret". But in the case of parking, we will just "jmp" to a
function and spin indefinitely. The iret instruction would restore flags and pop out the stack a jump to a return
address but in this case we don't want to do any of that. Since there are no flags to restore, no return function
and no stack to clean. The park function does the following:

load kernel PML4 base address in cr3 to restore identity paging

load RSP with a stack unique for the current CPU

acknowledge interrupt (APIC or PIC)

restore interrupt-enable flag (cleared when entering handler)

spin-wait with "hlt" instruction.

parkCPU:
// Set back the kernel page tables,
mov $PML4TABLE,%rax
mov %rax,%cr3
// set back the cpu's stack. This could be
// optimized but kept the way it is for clarity purposes.
// We're parking the CPU... it's not like we need the performance here
mov APIC_BASE+0x20,%eax
shr $24,%eax
mov %rax,%rsp
shl $8,%rsp
add $AP_STACKS,%rsp
// re-enable interrupts and set some registers for debug and wait
mov $0x11111111,%rax
mov $0x22222222,%rax
mov $0x33333333,%rax
call ackAPIC
sti
1: hlt
jmp 1b

Since interrupts are re-enabled and APIC has been ack'd, we will continue to be interrupted by the timer
and we are ready to take on a new task at any time. It would be wise to warn the parked CPU about
newly created tasks (with IPIs) so we don't need to wait until the timer to kick in. But that's for
another day.

Creating a new task

When a new task is created, memory is allocated for its stack and code. An initial context is setup
in the newly created stack. All registers are initialized to 0. Since the task will be scheduled by an interrupt handler,
it needs to have more data initilased in the stack for proper "iret" execution. Of course, the RSP value saved
in the task list will be the bottom of that context in the task's stack.
The initial stack looks as follow:

Stack top

152

SS

0 for kernel thread / Ring3 selector for user thread

144

RSP

Ring0 stack top or Ring3 stack top

136

RFLAGS

0x200202

128

CS

Ring0 Code selector or Ring3 Code selector

120

RIP

entry point for Kernel thread / 0x080000000 for User thread

112

RAX

0

104

RDI

0

96

RBX

0

88

RCX

0

80

RDX

Parameter to pass to thread on load

72

RSI

0

64

RBP

0

56

R8

0

48

R9

0

40

R10

0

32

R11

0

24

R12

0

16

R13

0

8

R14

0

0

R15

0

Stack bottom

Scheduling algorithm

My scheduling algorithm is nothing fancy. Just the good old round-robin way. But on multi-processor, it is probably the worse
thing you could do. Here is an example task list (Note that I should also use a linked list instead of a fixed size table)

CPU0

CPU1

CPU2

CPU3

Task1

Task2

Task3

Task4

Task5

As this table show, each task except Task5 are running on one of the 4 CPUS. At the next schedule(), the task list will look like this:

Task1

Task2

Task3

Task4

Task5

The problem here is that Task 1,2 and 3 are still running. So doing a context switch imposed a large overhead for no reason.
But the fact that they are still running is good, but they are not running on the same CPU anymore. So they will never
fully benefit from the CPU cache since memory will need to be fetched all over again because that CPU had no idea about
the code and data being used by the task. A better algorithm would detect that the task should continue to run on the same CPU.
I need to take the time to think about it.

Ring 3 tasks

To make a task run in ring3, it needs the following

A "non-conforming" code segment descriptor with DPL=3.

63

Irrelevant

1

0

0

0

Irrelevant

48

47

P:1

DPL:3

1

1

C:0

R:1

A:0

Irrelevant

32

31

Irrelevant

16

15

Irrelevant

0

cs must be loaded with RPL=3

15

GDTIndex15:3

TI:0

RPL=3

0

A writable data segment with DPL=3. This is for the stack.

63

Irrelevant

1

0

0

0

Irrelevant

48

47

P:1

DPL:3

1

0

E:0

R:1

A:0

Irrelevant

32

31

Irrelevant

16

15

Irrelevant

0

ss must be loaded with RPL=3

15

GDTIndex15:3

TI:0

RPL=3

0

The task register must be loaded with a valid tss selector. The TSS should be initialzed with a valid RSP0 value
for when the code switches to Ring0 (interrupts). TSS Descriptor:

63

base31:24

1

0

0

0

0

48

47

P:1

DPL:3

0

1

0

0

1

base23:16

32

31

base15:0

16

15

0x67

0

Please refer to Intel 64 and IA-32 Architectures Software Developer's Manual Volume 3 for more information
These values would be on the stack prior to invoking "iret" from the scheduler. The scheduler runs in
ring0, hence why the RPL value in selectors is important

Memory layout

The whole memory is again identity mapped, using 2mb pages, at 0x4000000000.

The kernel's 4k pages is only used to track free pages to be distributed to requesting threads.
Each 4k page entry has its 3 AVL bits cleared. This is how the kernel keeps track of available memory.
A page with non-zero AVL is in use.
So when a process needs to create a new page, it does the following:

look through the kernel 4k pages structure for a free page. Mark is as not-free

look though the process 4k pages structure for a free page. Mark is as not-free

take the physical address of the kernel page (easy since it is identity mapped)
and store it in the process page entry. The process's page now maps to the proper physical page.

The AVL bits are set as follow:

0b000: physical memory pointed to by this page is free to use

0b001: physical memory pointed to by this page is also mapped somewhere else and is a stack page for a thread

0b010: physical memory pointed to by this page is used by the kernel

0b011: physical memory pointed to by this page is used as heap for a thread or the kernel

Creating a process

When creating a new process, the process code will start at 0x8000000. So the kernel will reserve a physical
page (as per the process in step 1 above). The physical address of that page will be used in the process's page
entry for address 0x8000000. This should correspond to Page Directory entry #64. Since the first 64 entries
are 2mb pages and the rest are pointers to page tables, then this virtual address maps to a 4k page.
So all processes will see their entrypoint at 0x08000000 but they will obviously be mapped to different
physical pages. This allows to to compile position dependant code and run it in the OS.

Whole memory identity mapping

Each process have their own virtual address mapping.
If a process calls a system function that needs to manipulate physical
memory this causes a problem. For example, if a process wants to allocate memory, it needs to access it own page tables.
but the pages reside above kernel memory so it is not mapped into process virtual addressing space. The same can happen
when trying to create a process: The system call will reserve a virtual address in its page tables,
which is identity mapped, but then it will attempt to fill in some data in the physical page,
that physical page resides above the kernel memory, so it is not mapped in the process. for example:

Kernel memory ends at 0x08000000. Kernel page tables are 0x00020000.

The kernel page tables is just a list of identity mapped pages above kernel memory
with the AVL field indicating if the page is available.

The first entry maps to physical location 0x08000000. So when searching for a free page, the kernel will
find, for example, an entry at 0x00020B00 with the AVL bits set the "free".

This entry maps to 0x08000000+(4096*(B00/16)) = 0x080B0000.

The process would like that physical address to be mapped to 0x08001000 and fill it with zeros.
Two problem arises:

The process cannot modify its page table to write an entry for 0x08001000 since
its page tables are not mapped.

It cannot fill the page with zeros prior to map it since the page is not mapped and it
cannot access 0x080B0000 directly because this is a physical address and it would be interpreted
as a virtual address by the CPU.

The solution is to keep an indentity mapping of the whole memory at some other address. Lucky for us, with 64bit
addressing we have a very large address range. So identiy mapping will start at 0x4000000000 (256th gig).
That address will map to physical address 0x00000000. So by getting the PLM4 address from cr3 and
adding 0x4000000000 (or simply setting bit38), the process can now access its page tables.

Dynamic stack growth

A user thread is created with a 20mb stack but only the 4 pages are allocated (commited to physical memory).
If the thread attempts an access below that 16k, a Page Fault exception (#PF) will occur since the page will
not be mapped (unless the access went as far down as the heap,in which case corruption would happen). The #PF
handler will then allocate a page for that address

My implementation is rather simple. It allocates
physical pages upon page faults and that's it. The only protection it does is
to check if the address is in the page guard. The page guard is a non-present
page between the heap top and stack bottom. If a stack operation would
fall in that page, then the OS detects this as a stack overflow. But there is
nothing that protects a task to do something like "movq $1, -0x1000000(%rsp)".
This could fall below the page guard and corrupt the heap. Same thing for the heap.
Nothing prevents a task from writting to an address that falls in the stack.
So the the page guard is a best-effort method.

It would be nice if the Exception error code would tell us if this operation was using rbp or rsp,
or any implied usage of the ss segment segment selector.
for example, all these operations should be recognized as stack growth demand:

push %rax

mov %rax,(%rsp)

mov %rax,-5000(%rbp)

mov %rax,-5000(%rbp)

And to some extent: lea -10000(%rsp),%rax; mov %rbx,(%rax)

To make sure that we touch the page guard, we need to do Stack Probing:

// This would be dangerous
void test1()
{
char a[0x2000];
a[0]=1;
}
// This would be safe.
void test1()
{
char a[0x2000];
// the fact that we access a page that is no further than 0x1000 from
// the bottom of the stack will make the #PF correctly create the page
// Doing this is called Stack Probing
a[0x1001] = a[0x1001];
a[0]=1;
}

It's impossible to detect all those conditions. So unfortunately, it is not possible to detect if the
demand is for a legitimate stack growth, stack overflow or heap overflow. So we just blindly allocate the page
and give it to the requesting task.

After trying to find an alternate solution, I found out that apparently Windows and linux are facing the same dillema.
Apparently, my algorithm is good. When compiling a C application
under those OS, the compiler will detect if the program tries to allocate more than one page on the stack and
will generate "stack probing" code. So Stack Probably is a well-known technique to work around that limitation.
But if you write a ASM application under linux or windows, then you need to take care of that.

Another possible way I will eventually explore is to set the stack memory at virtual location 0x1000000000000.
By looking at bit 47of the offending address, I would get a strong hint about this being a stack operation.
This means the OS couldn't support systems with more than 256 terabytes of RAM.... It also has other downsides.

ENABLING MULTI-PROCESSORS IN MY HOBBY OS

2015-10-19

I recently added multi-processor support in my homebrew OS. Here are the technical details. BTW:
Chapter 8 and 10 of the Intel Manual 3 are probably your best resource.

When the system starts, all but one CPU is halted. We must signal the other CPUs to start.
I won't go into the details of how to bootstrap the processor, that step is easy: just
go in protected mode then setup paging and jump to long mode. This is very well covered in the Intel
manuals.

Basically, this is how we switch to protected mode

// Before going any further, you must enable the A-20 line. Not covered in this example
push %cs /* remember, cs is 07C0*/
pop %ds
mov $GDTINFO,%eax
lgdtl (%eax)
mov %cr0,%eax
or $1,%al
mov %eax,%cr0 /* protected mode */
mov $0x08,%bx
// far jump to clear cache
ljmpl $0x10,$PROTECTEDMODE_ENTRY_POINT
GDTINFO:
// GDT INFO
.WORD 0x20
.LONG . + 0x7C04 /*that will be the address of the begining of GDT table*/
// GDT
.LONG 00
.LONG 00
// GDT entry 1. Data segment descriptor used during unreal mode
.BYTE 0xFF
.BYTE 0xFF
.BYTE 0x00
.BYTE 0x00
.BYTE 0x00
.BYTE 0b10010010
.BYTE 0b11001111
.BYTE 0x00
// GDT entry 2. Code segment used during protected mode code execution
.BYTE 0xFF
.BYTE 0xFF
.BYTE 0x00
.BYTE 0x00
.BYTE 0x00
.BYTE 0b10011010
.BYTE 0b11001111
.BYTE 0x00
// GDT entry 3. 64bit Code segment used for jumping to 64bit mode.
// This is just used to turn on 64bit mode. Segmentation will not be used anymore after 64bit code runs.
// We will jump into that segment and it will enable 64bit. But limit and permissions are ignored,
// the CPU will only check for bit D and L in this case because when we will jump in this, we will
// already be in long mode, but in compatibility sub-mode. This means that while in long mode, segments are ignored.
// but not entiorely. Long mode will check for D and L bits when jumping in another segment and will change
// submodes accordingly. So in long mode, segments have a different purpose: to change sub-modes
.BYTE 0xFF
.BYTE 0xFF
.BYTE 0x00
.BYTE 0x00
.BYTE 0x00
.BYTE 0b10011010
.BYTE 0b10101111 // bit 6 (D) must be 0, and bit 5 (L, was reserved before) must be 1
.BYTE 0x00

Detecting the number of CPUs

The first thing to do is to detect the number of CPUs present. This can be done
by looking for the "MP floating pointer" structure. It is located somewhere in in the BIOS
address space and we must find it. I won't go into the details of the structure since it is
very well documented everywhere. The MP structure contains information about the CPUs and IO APIC
on the system. This structure is filled in by the BIOS at boot time. The structure can be at many
places hence why we must search for it in memory. It starts with "_MP_" and contains a checksum, so
by scanning the memory, you will find it. The important thing to know is that you do the following:

Find the structure in memory. According to the specs, it can be in a couple of different places.

Detect number of CPUs and Local APIC address of CPUs

Detect IO APIC address.

For more details on how to find the structure and its format, make a search for "Intel Multi-Processor Specification".

When wandering in the SMP world, you must forget about using the PIC (Programmable Interrupt Controller)
The PIC is an old obsolete device anyway. The new way now is the use the APIC. So we won't be using
the PIC anymore. There is a notion of a local APIC and the IO APIC. The local APIC is an APIC
that is present on each CPU. The local APICs can be use to trigger interrupts from one CPU to
another, as a way of communication. When the system starts, all but one CPU is halted.
We must signal the other CPUs to start. The PIC could not allow us to do that, hence
why we must use the APIC. The local APIC will allow us to trigger an interrupt on the other CPUs
to get them out of their halted state.

We must then setup the local APIC for the current CPU. Each CPU have their own APIC and their
APIC is mapped at the same address for each CPU. The local APIC address is 0xFEE00000.
So when CPU0 read/writes at 0xFEE00000 it is not the same as if CPU1 read/write at 0xFEE00000
since the address maps to each CPU's own APIC. This is nice because it means
you dont need to do something like "What CPU am I? number x? ok, then use address xyz then."
Each CPU only need to write at the same address and they will be guaranteed to write to their
own APIC. It's all transparent so you don't need to worry about it. The address of the IO APIC
maps to the same IO APIC for all CPUs though. But that's also good because all CPUs
want to use the same IO APIC anyway.

The SMP_TRAMPOLINE constant is the address of where I want the APs to jump to when starting. This address must be aligned
on a 4k boundary because we the SIPI message takes the page number as a parameter. Hence why I SHR the address by 12 (div by 4096).
And since the APs will start in 16bit mode, the address must reside under the 1meg barrier. STARTEDCPUS is a 64bit bitfield
that represents the CPUs. Each bit get set by the APs (cpuX sets bit X).

Application processors trampoline code

I decided to put the Application Processor's trampoline code in the bootloader (I've got 512bytes of room, that
should be enough). The bootloader is a good decision beacause it is below the 1meg mark, the source file is
compiled as 16bit code and all the initialisation is done there anyway. But when an AP starts, it will
be given a start address aligned on a 4k page boundary and the bootloader is at 0x7C00. So the bootloader
will copy a "jmp" at 0x1000 to jump to the bootloader AP init function. So the order of execution is:

AP receives SIPI with vector 0x01

AP jumps to 0x1000

Code at 0x1000 will make AP jump to 0x7C0:

AP will switch protected mode and jump to KernelMain

KernelMain will check in MSR[0x1B] if this is an AP or the BST. if BST, then jump to normal initialisation

The system uses a modular architecture now. I like to think of DHAS as a hub for different technologies. Since my home automation system is mix of Insteon devices, ethernet relays and temperature sensors, IP phones, and more, I made a central software that can interface with all those technologies. Each module is responsible for interfacing with one technology. on github, you will find these modules under the src/module folder.
Everytime I need to add a new kind of module, I just create the class and it automatically gets instanciated and used. The modules register their own REST callbacks and the REST engine being self-documenting will show the newly added functions when querying the help API. This way, the system doesn't know anything about its modules. It only knows that it is running modules. So adding new modules becomes very easy because the work is isolated from the rest of the system.

Arduino

Since I am not using the rPi anymore, I needed to find a way to get GPIOs on the server. So I bought an Arduino Leonardo and made a very simple firmware that presents the device as a CDC device using the LUFA library. The source code for the firmware is here:

INSTALLING AND USING DOCKER ON SLACKWARE

2015-06-06

Docker

Imagine you want an asterisk system complete with mariadb and apache but you don't want to
install all that on your day-to-day system. You could create a VM. but vms are heavy and have quite some overhead.
What about backups? you can't copy live running image (not with qemu at least).

Enter Docker. Docker makes this really simple. Docker encloses an environment like a chroot would. But
it acts more like a vm. With docker, the equivalent of a vm is a container. You create your
filesystem and install the needed software in your container and run it. But Docker lets you run 1 command
only. You start the container, the command executes, in its environment, and the container stops. All this using
the host's kernel but separated with namespaces and cgroups. Nothing prevents you from running a script as the command.
So you make a script that starts httpd and asterisk then load bash. the container will run as long as bash doesn't exit.
so if you attach your session on the container and "exit" bash, the container will stop amd asterisk and httpd will shutdown.
The container does not start "init" (unless it's the command you chose to invoke) so it's not like an entire OS is
brought up. Only the command you run is ran in the container's FS but with the host's kernel

Docker allows you export running containers to a tarball. the tarball contains the entire FS of the container (not the memory)
You can then import it anywhere where Docker runs.

Installing docker on slackware

These are the instructions for installing docker 1.7.0 on slackware 14.1 with kernel 3.11.1.
I also tried with slackware 14.0 but the rc.S script does not mount the cgroup hierarchy properly
So I modified it to do it like in 14.1
I could not get it to work with kernel 3.10.17 and did not bother to troubleshoot since I had
a 3.11.1 on hand. 1.7.0-rc1 did not work for me. I needed commit 6cdf8623d52e7e4c5b5265deb5f5b1d33f2e6e95 in.
So I cloned the bleeding edge from git but then 2h later rc2 came out.

Pre-requisites

Before anything, you should make sure that your kernel is compiled with

Download, compile, and prepare the environment

At first, I tried downloading the binaries but docker was complaining about "Udev sync is not supported".
I found out that was because the binary is statically link and it causes some problems that I didn't care to look
into. So I opted for building from source. The first step is to get "go". I didn't want to leave this on my
system so I just put it in a temporary place and then deleted it.

wget https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz
tar -zxf go1.4.2.linux-amd64.tar.gz
mv go /opt
#You should make that permanent if you intend to keep Go after building Docker
PATH=$PATH:/opt/go/bin
GOPATH=/opt/go:/opt/go/src/github.com/docker/docker/vendor
# download docker source.
wget https://github.com/docker/docker/archive/v1.7.0-rc2.tar.gz
tar -zxf v1.7.0-rc2.tar.gz
cd docker-1.7.0-rc2
#docker won't build because there is a header file that won't be found. There is a patch
#for that but let's do it manually here:
sed -i "/ioctl\.h/c\#include \r\n#include \r\n#include \r\n#include \r\n#include \r\n" daemon/graphdriver/btrfs/btrfs.go
#note that DOCKER_GIT_COMMIT needs to match the version you have downloaded
GOROOT=/opt/go AUTO_GOPATH=1 DOCKER_GITCOMMIT="395cced " ./hack/make.sh dynbinary
cp bundles/1.7.0-rc2/dynbinary/docker-1.7.0-rc2 /usr/sbin/docker
cp bundles/1.7.0-rc2/dynbinary/dockerinit-1.7.0-rc2 /usr/sbin/dockerinit
#remove go... or not. It's up to you. If you leave it there, you might want to permanently add it to your PATH
rm -Rf /opt/go

Prepare network bridge

In my case, since I was already using KVM/qemu, I had a bridge alrady setup.
But this is what would be needed

You might want that last example to run at boot time. There is a way to setup a bridge with the init scripts but I just added those lines in rc.local before launching the docker daemon.

There is currently no easy way to assign a static IP to your container. Docker will choose an IP
in the range of you bridge. But this isn't perfect. It seems to pick some addresses that are already used on my network. Issue 6743 on githubis opened for that.
But for the time being, I've hacked the code to make this possible. I won't create a pull request since
they are already working on a more elegant solution. But meanwhile, you can download my fork if you need it.
My fork on github.. That repo contains the patch
to build on slackware and also adds a "--ipv4-adress=A.B.C.D" option to docker run.

Auto start

Finally, you should add this in your rc.local script.

/usr/sbin/docker -d -b br0&

Building a container, running it and doing backups

My use case

What I'm looking to do is to isolate my home automation services in one container
that I can easily transport from one computer to another (or even to a VM). In case
I get a hardware failure, I want to reduce the downtime of my house services. Those include
Asterisk, httpd, MariadDB, CouchDB, DHAS, cron jobs,
and some more. I want to be able to make a daily backup of the container and always be able to
launch it from somehwere else where Docker is installed.

Creating a container

docker run vbatts/slackware:14.1 -ti /bin/bash

That command will create a container from a base image "slackware 14.1" from docker hub.
The image will be downloaded automatically. Then bash will be invoked. the -t -i flags
will give you an interactive TTY attached so you will be able to interact with bash.
From there, install whatever you need in the container. Download gcc, install it, etc. Once you
are done, exit bash. By exiting bash, the container stops. You can now commit your changes to a
new base image

docker commit ContainerID awesomeNewImage

Now you have a base image of your own that you can share with other people. Now create another container
for your real use case from the base image.

docker run -tid --restart=always awesomeNewImage /root/start.sh

The -d flag runs the contain in the background. You can access it using "docker attach".
--restart=always will make the container restart automatically when you exit it and when
the docker daemon starts (after a host reboot for example). When you were setting up
you base image, you could have created a start.sh script that invokes asterisk, httpd, mysqld and couchdb then bash.
To detach from the container without stopping it (leaving your command running) you can Ctrl-P, Ctrl-Q.

Backup/Restore

An easy way to backup is to regularly export the container (Cron job?).

docker export -o backup.tar

This will create a tarball container the entire filesystem of your container.
Then to restore, either locally or on some other machine runner docker: