I am trying to understand how a function, say mkdir, works by looking at the kernel source. This is an attempt to understand the kernel internals and navigate between various functions. I know mkdir is defined in sys/stat.h. I found the prototype:

7 Answers
7

System calls aren't handled like regular function calls. It takes special code to make the transition from user space to kernel space, basically a bit of inline assembly code injected into your program at the call site. The kernel side code that "catches" the system call is also low-level stuff you probably don't need to understand deeply, at least at first.

This code is saying mkdir(2) is system call #83. That is to say, system calls are called by number, not by address as with normal functions, because it's not really a function in the way you understand it. The inline assembly glue code I mentioned above uses this to make the transition from user to kernel space, taking your parameters along with it.

Another bit of evidence that things are a little weird here is that there isn't always a strict parameter list for system calls: open(2), for instance, can take either 2 or 3 parameters, a trick C++ knows how to do, but C doesn't, yet the syscall interface is nominally C-compatible.

To answer your first question, there is no single file where mkdir() exists. Linux supports many different file systems and each one has its own implementation of the "mkdir" operation. The abstraction layer that lets the kernel hide all that behind a single system call is called the VFS. So, you probably want to start digging in fs/namei.c, with vfs_mkdir(). The actual implementations of the low-level file system modifying code are elsewhere. For instance, the ext3 implementation is called ext3_mkdir(), defined in fs/ext3/namei.c.

As for your second question, yes there are patterns to all this, but not a single rule. What you actually need is a fairly broad understanding of how the kernel works in order to figure out where you should look for any particular system call. Not all system calls involve the VFS, so their kernel-side call chains don't all start in fs/namei.c. mmap(2), for instance, starts in mm/mmap.c, because it's part of the memory management ("mm") subsystem of the kernel.

Excellent answer, and a great reccomendation in "Understanding the Linux Kernel."
–
Mr. ShickadanceFeb 17 '11 at 18:55

2

Great answer for see the point of kernel. +1
–
Soner GönülAug 7 '11 at 14:30

5

Minor nit: C is very much capable of handling functions with variable number/type of parameters, see e.g. printf(3) and such. This magic is invoked by "..." in the prototype, and handled by va_arg(3) trickery.
–
vonbrandJan 15 '13 at 21:51

This probably doesn't answer your question directly, but I've found strace to be really cool when trying to understand the underlying system calls, in action, that are made for even the simplest shell commands. e.g.

strace -o trace.txt mkdir mynewdir

The system calls for the command mkdir mynewdir will be dumped to trace.txt for your viewing pleasure.

A good place to read the Linux kernel source is the Linux cross-reference (LXR). Searches return typed matches (functions prototypes, variable declarations, etc.) in addition to free text search results, so it's handier than a mere grep (and faster too).

LXR doesn't expand preprocessor definitions. System calls have their name mangled by the preprocessor all over the place. However, most (all?) system calls are defined with one of the SYSCALL_DEFINEx families of macros. Since mkdir takes two arguments, a search for SYSCALL_DEFINE2(mkdir leads to the declaration of the mkdir syscall:

ok, sys_mkdirat means it's the mkdirat syscall, so clicking on it only leads you to the declaration in include/linux/syscalls.h, but the definition is just above.

The main job of mkdirat is to call vfs_mkdir (VFS is the generic filesystem layer). Cliking on that shows two search results: the declaration in include/linux/fs.h, and the definition a few lines above. The main job of vfs_mkdir is to call the filesystem-specific implementation: dir->i_op->mkdir. To find how this is implemented, you need to turn to the implementation of the individual filesystem, and there's no hard-and-fast rule — it could even be a module outside the kernel tree.

Note: the .h file doesn't define the function. It's declared in that .h file and defined (implemented) elsewhere. This allows the compiler to include information about the function's signature (prototype) to allow type checking of arguments and match the return types to any calling contexts in your code.

In general .h (header) files in C are used to declare functions and define macros.

mkdir in particular is a system call. There may be a GNU libc wrapper around that system call (almost certainly is, in fact). The true kernel implementation of mkdir can be found by searching the kernel sources and the system calls in particular.

Note that there will also be an implementation of some sort of directory creation code for each filesystem. The VFS (virtual filesystem) layer provides a common API which the system call layer can call into. Every filesystem must register functions for the VFS layer to call into. This allows different filesystems to implement their own semantics for how directories are structured (for example if they are stored using some sort of hashing scheme to make searching for specific entries more efficient). I mention this because you're likely to trip over these filesystem specific directory creation functions if you're searching the Linux kernel source tree.

The implementation (as described in sys/stat.h) is the business of userland and libc. The kernel internal stuff (how it is really done) is kernel internal business. For all the kernel hackers care, the internal function could be called xyzzy and take 5 parameters. It is libc's job to take the userland call, translate it into whatever kernel incantations are required, ship it off and collect any results.
–
vonbrandJan 16 '13 at 3:42

Please don't post just links to blogs or forums, summarize their contents so that readers can see what they're about, and to have something left if the sites disappear. Also, your first link is about libc, which is off-topic for this question.
–
GillesJan 13 '11 at 22:24