I see that on x86, in a process's VM, shared libraries are loaded between heap and stack, i.e. the mmap area as noted in most of the online articles. But on a PowerPC linux box, I see that all libraries are loaded below where the program itself is loaded. "strace" shows the library load address is pre-determined (assuming by ld) before load/mmap the library.

I wonder whether this is an architecture dependent thing? Are there any online documents about this?

Afaik it depends on the architecture-specific implementation details of the mmap() syscall, which depends on the arch-specific vm implementation.
– peterhSep 11 '15 at 20:36

@peterh it does not look like it is mmap implementation, see the strace log of a cat command I added to the question. The address is passed to mmap.
– weiSep 11 '15 at 20:48

Yes, it is mmap. See: first open libc.so.6, gets fd 3 as result, reads a little bit into its ELF headers, analyzes it, then mmaps fd3 as it is needed. Imho, your question could be tracked down, why mmap works on ppc32 in a different way. And the answer is probably somewhere below arch/ppc/mm.
– peterhSep 11 '15 at 20:54

@peterh, I know it is mmap that loads the library, my question is about what determines the address 0xfe40000 passed to mmap we see mmap(mmap(0xfe40000, 1582324, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
– weiSep 11 '15 at 20:57

2 Answers
2

(Preferred) disposition of the run-time stack differs among hardware architectures. But as for the program proper and shared libraries, for a dynamically linked executable, memory location of mapped regions is determined by the linker. It’s not, generally, a kernel’s job, to decide where components of a userland program should be located. Nor is the order implied by CPU architecture. On the same hardware, and even within one running OS (i. e. kernel) we can imagine different linkers arranging it differently (exec Linux call extracts the name of the linker from the ELF file; see elf_interpreter variable in load_elf_binary() in fs/binfmt_elf.c).

In Linux, default dynamic linker ld-linux is a part of glibc. How it tries to map objects can be seen in the _dl_map_object_from_fd() function at elf/dl-load.c of its source code. Sometimes preferences in the executable are accounted for (that presumedly depend on compiler and linker created the executable), and in some cases disposition of a memory map is decided by the kernel.

There is some googleable information on dynamic linkers and their architecture dependence, such as:

I will answer my own question after some digging.
In short, this is a arch-dependent thing.
PowerPC32 defines a preferred load address function, while most other arches, including x86, does not define this.

/* The idea here is that to conform to the ABI, we are supposed to try
to load dynamic objects between 0x10000 (we actually use 0x40000 as
the lower bound, to increase the chance of a memory reference from
a null pointer giving a segfault) and the program's load address;
this may allow us to use a branch instruction in the PLT rather
than a computed jump. The address is only used as a preference for
mmap, so if we get it wrong the worst that happens is that it gets
mapped somewhere else. */
ElfW(Addr)
__elf_preferred_address (struct link_map *loader, size_t maplength,
ElfW(Addr) mapstartpref)
{
ElfW(Addr) low, high;
struct link_map *l;
Lmid_t nsid;
/* If the object has a preference, load it there! */
if (mapstartpref != 0)
return mapstartpref;
/* Otherwise, quickly look for a suitable gap between 0x3FFFF and
0x70000000. 0x3FFFF is so that references off NULL pointers will
cause a segfault, 0x70000000 is just paranoia (it should always
be superseded by the program's load address). */
low = 0x0003FFFF;
high = 0x70000000;
for (nsid = 0; nsid < DL_NNS; ++nsid)
for (l = GL(dl_ns)[nsid]._ns_loaded; l; l = l->l_next)
{
ElfW(Addr) mapstart, mapend;
mapstart = l->l_map_start & ~(GLRO(dl_pagesize) - 1);
mapend = l->l_map_end | (GLRO(dl_pagesize) - 1);
assert (mapend > mapstart);
/* Prefer gaps below the main executable, note that l ==
_dl_loaded does not work for static binaries loading
e.g. libnss_*.so. */
if ((mapend >= high || l->l_type == lt_executable)
&& high >= mapstart)
high = mapstart;
else if (mapend >= low && low >= mapstart)
low = mapend;
else if (high >= mapend && mapstart >= low)
{
if (high - mapend >= mapstart - low)
low = mapend;
else
high = mapstart;
}
}
high -= 0x10000; /* Allow some room between objects. */
maplength = (maplength | (GLRO(dl_pagesize) - 1)) + 1;
if (high <= low || high - low < maplength )
return 0;
return high - maplength; /* Both high and maplength are page-aligned. */
}