Creating a vDSO: the Colonel's Other Chicken

And Now Some Special Sauce

If the vDSO is operating in userland, how do you
access kernel-land variables?
After all, if vDSOs are supposed to provide kernel information, don't they have to
trip the userland/kernel-land memory segment? And, wouldn't that flip-flopping of
memory context render a vDSO useless? Well, it all depends how the userland
version, the vDSO version, accesses the kernel data. For
gettimeofday(), a
special time variable is mapped into memory where the kernel updates it and the
userland (vDSO version) can read it. The kernel merely copies what it knows
about time into that variable, and when a vDSO is made, that call just reads the
information saving the overhead of crossing memory segments. The addition or
access of a kernel variable is fairly involved as compared to a basic vDSO
function, but because the purpose of a vDSO is to access kernel information, such
as that provided in variables, I probably should give a quick overview
of doing that.

For illustrative purposes, let's add a value that lives in kernel land but
is read from userland. Sure, I said earlier that this mystical number might
change and you should implement a function to return it. Well, you have a
function, but all you know now is the value and not what it might change to in
the future. Let's make the function return a value, nonconstant. Wow, this
use case is becoming really unusual. To elaborate, let's update this variable
as the kernel requests. The kernel will update the vDSO variables
in the update_vsyscall() function located in
linux-2.6.37/arch/x86/kernel.

If you were to declare it const int vnotb = 666;, the value captured there would
not be set (more on this later).

Let's define the value to be, in fact, the mysterious number of the beast
itself, which I will call vnotb. This number will reside in
kernel land, as so many other useful values, such as time, which the efficient
gettimeofday() vDSO will obtain. This is where the true magic of vDSOs lie.

Let's remain in linux-2.6.37/arch/x86/vdso and modify all the
goodies here. First, declare the variable via the
VEXTERN() macro. In
vextern.h, add your declaration alongside all the other declarations:

VEXTERN(vnotb)

This macro will create a variable that is a pointer to the value you care about
and is prefixed with vdso_. In essence, you have declared vnotb as
int *vdso_vnotb;.

vextern.h mentions that:

Any kernel variables used in the vDSO must be exported in the main kernel's
vmlinux.lds.S/vsyscall.h/proper__section and put into vextern.h and be
referenced as a pointer with vdso prefix. The main kernel later fills in
the values (comment in linux-2.6.37/arch/x86/vdso/vextern.h).

Now that you have some of the vDSO code in place, the userland stuff and the
kernel-userland mapping, let's make use of it. In the function
vget_number_of_the_beast(), let's return the value:

notrace int __vdso_number_of_the_beast(void)
{
return *vdso_vnotb;
}

Don't forget to add the header that declares that value,
vextern.h as well as
an additional header that will resolve some data referenced by the latter,
vgtod.h:

#include <asm/vgtod.h>
#include "vextern.h"

To wrap things up, you need to let the kernel know about this variable so it
can pump data into it. You need the kernel to give userland a value.
Well, you
have it mapped at the address specified above, but that is rather
pointless,
unless Mr Sanders, the colonel, doesn't push some data into it. You need to
go up one directory (yes, this isn't the most trivial of processes). Hop
into linux-2.6.37/arch/x86/kernel. You need to let the linker know of this
value, so it can map between kernel and userland, so you probably should rock
that. Modify vmlinux.lds.S, and add the following after the
vgetcpu_mode
piece (note that adding it after or before
vgetcpu_mode isn't necessary, but it's
an easy place to find things):

.vnotb : AT(VLOAD(.vnotb)) {
*(.vnotb)
}
vnotb = VVIRT(.vnotb);

This links the vnotb symbol with the variable
vnotb.
This sets up the variable in the address space for kernel land to access
and write to. The macros above, AT,
VLOAD and VVIRT deal with modifying
addresses so that the proper piece of data, at the
vnotb, is referenced.

Now, you need to declare the value that the kernel land will write to. In
linux-2.6.37/arch/x86/include/asm/vsyscall.h declare this puppy and its
section that will be inserted via the above linker script entry you most recently
added:

In this file, as mentioned, you also will declare the kernel-land variable
to which
the kernel will write. To keep things slightly more readable, associate your
variable next to the vgetcpu_mode declaration:

extern int vnotb;

You also will define a value the kernel can read (I don't use this in my
example, but if the kernel needs to read the value, this is the variable to
read):

extern int __vnotb;

Now let's put this stuff in code and give it a value. The kernel will write the
value via the writable vnotb, and you also can read it from the shared memory
between kernel and userland via __vnotb. You will write the value in the
kernel-land version of the variable, which is writable. In
linux-2.6.37/arch/x86/kernel/vsyscall_64.c, preferably
after all of the #include headers and just after the piece: int __vgetcpu_mode
__section_vgetcpu_mode;, add the following:

int __vnotb __section_vnotb;

Remember, you did a trick with the linker setting the value. If you
set the value globally, as you would for an extern, you would not get a value, the
linker would override it. You need to set this value at runtime and not
statically at compile time. To set this value as the kernel updates, modify
the update_vsyscall() routine in linux-2.6.37/arch/x86/kernel/vsyscall_64.c
with:

vnotb = 666;

This statement is defining the value declared previously in vsyscall.h.

Matt Davis is a software engineer on leave from his job in the US to
pursue a PhD from the Computer Science Department at the University of
Melbourne, where he is focusing his hackery toward the compiler field.

Trending Topics

Upcoming Webinar

Getting Started with DevOps - Including New Data on IT Performance from Puppet Labs 2015 State of DevOps Report

August 27, 2015
12:00 PM CDT

DevOps represents a profound change from the way most IT departments have traditionally worked: from siloed teams and high-anxiety releases to everyone collaborating on uneventful and more frequent releases of higher-quality code. It doesn't matter how large or small an organization is, or even whether it's historically slow moving or risk averse — there are ways to adopt DevOps sanely, and get measurable results in just weeks.