8. SMP Boot

There are a few SMP related macros, like CONFIG_SMP,
CONFIG_X86_LOCAL_APIC, CONFIG_X86_IO_APIC, CONFIG_MULTIQUAD
and CONFIG_VISWS.
I will ignore code that requires CONFIG_MULTIQUAD
or CONFIG_VISWS,
which most people don't care (if not using IBM high-end multiprocessor
server or SGI Visual Workstation).

IPI (InterProcessor Interrupt), CPU-to-CPU interrupt through local APIC,
is the mechanism used by BSP to trigger APs.

Be aware that "one local APIC per CPU is required" in an
MP-compliant system.
Processors do not share APIC local units address space (physical address
0xFEE00000 - 0xFEEFFFFF), but will share APIC I/O units
(0xFEC00000 - 0xFECFFFFF).
Both address spaces are uncacheable.

Don't confuse start_secondary() with
trampoline_data().
The former is AP "idle" process task struct EIP value, and the latter is
the real-mode code that AP runs after BSP kicks it
(using wakeup_secondary_via_INIT()).

8.3. linux/arch/i386/kernel/trampoline.S

This file contains the 16-bit real-mode AP startup code.
BSP reserved memory space trampoline_base in
start_kernel() -> setup_arch() -> smp_alloc_memory().
Before BSP triggers AP, it copies the trampoline code, between
trampoline_data and
trampoline_end,
to trampoline_base
(in do_boot_cpu() -> setup_trampoline()).
BSP sets up 0:467 to point to trampoline_base,
so that AP will run from here.

Note that BX=1 when AP jumps to
linux/arch/i386/kernel/head.S:startup_32(),
which is different from that of BSP (BX=0).
See Section 6.

8.4. initialize_secondary()

Unlike BSP, at the end of
linux/arch/i386/kernel/head.S:startup_32()
in Section 6.4,
AP will call initialize_secondary() instead of
start_kernel().

/* Everything has been set up for the secondary
* CPUs - they just need to reload everything
* from the task structure
* This function must not return. */
void __init initialize_secondary(void)
{
/* We don't actually need to load the full TSS,
* basically just the stack pointer and the eip. */
asm volatile(
"movl %0,%%esp\n\t"
"jmp *%1"
:
:"r" (current->thread.esp),"r" (current->thread.eip));
}

As BSP called do_boot_cpu() to set
thread.eip to start_secondary(),
control of AP is passed to this function.
AP uses a new stack frame, which was set up by BSP in
do_boot_cpu() -> fork_by_hand() -> do_fork().

8.5. start_secondary()

All APs wait for signal smp_commenced from BSP,
triggered in Section 8.2smp_init() -> smp_commence().
After getting this signal, they will run "idle" processes.