Monday, September 19, 2016

Understanding evdev

This post explains how the evdev protocol works. After reading this post you
should understand what evdev is and how to interpret evdev event dumps to
understand what your device is doing. The post is aimed mainly at users
having to debug a device, I will thus leave out or simplify some of the
technical details. I'll be using the output from evemu-record as example
because that is the primary debugging tool for evdev.

What is evdev?

evdev is a Linux-only generic protocol that the kernel uses to forward
information and events about input devices to userspace. It's not just for mice and keyboards but any device that has any sort of
axis, key or button, including things like webcams and remote controls. Each
device is represented as a device node in the form of
/dev/input/event0, with the trailing number increasing as you add
more devices. The node numbers are re-used after you unplug a device, so
don't hardcode the device node into a script. The device nodes are also
only readable by root, thus you need to run any debugging tools as root too.

evdev is the primary way to talk to input devices on Linux. All X.Org
drivers on Linux use evdev as protocol and libinput as well. Note that
"evdev" is also the shortcut used for xf86-input-evdev, the X.Org
driver to handle generic evdev devices, so watch out for context when you
read "evdev" on a mailing list.

Communicating with evdev devices

Communicating with a device is simple: open the device node and read from
it. Any data coming out is a struct input_event, defined in
/usr/include/linux/input.h:

I'll describe the contents later, but you can see that it's a very simple
struct.

Static information about the device such
as its name and capabilities can be queried with a set of
ioctls. Note
that you should always use
libevdev
to interact with a device, it blunts the few sharp edges evdev has.
See the libevdev
documentation for usage examples.

evemu-record, our primary debugging tool for anything evdev is very
simple. It reads the static information about the device, prints it and then
simply reads and prints all events as they come in. The output is in
machine-readable format but it's annotated with human-readable comments
(starting with #). You can always ignore the non-comment bits. There's a
second command, evemu-describe, that only prints the description and
exits without waiting for events.

Relative devices and keyboards

The top part of an evemu-record output is the device description. This is
a list of static properties that tells us what the device is capable of. For
example, the USB mouse I have plugged in here prints:

The device name is the one (usually) set by the manufacturer and so are the
vendor and product IDs. The bus is one of the "BUS_USB" and similar
constants defined in /usr/include/linux/input.h. The version is often
quite arbitrary, only a few devices have something meaningful here.

We also have a set of supported events, categorised by "event type" and
"event code" (note how type and code are also part of the struct input_event).
The type is a general category, and
/usr/include/linux/input-event-codes.h defines quite a few of those.
The most important types are EV_KEY (keys and buttons), EV_REL (relative
axes) and EV_ABS (absolute axes). In the output above we can see that we
have EV_KEY and EV_REL set.

As a subitem of each type we have the event code. The event codes for this device are
self-explanatory: BTN_LEFT, BTN_RIGHT and BTN_MIDDLE are the left, right and
middle button. The axes are a relative x axis,
a relative y axis and a wheel axis (i.e. a mouse wheel). EV_MSC/MSC_SCAN is
used for raw scancodes and you can usually ignore it.
And finally we have the EV_SYN bits but let's ignore those, they are always
set for all devices.

Note that an event code cannot be on its own, it must be a tuple of (type,
code). For example, REL_X and ABS_X have the same numerical value and
without the type you won't know which one is which.

That's pretty much it. A keyboard will have a lot of EV_KEY
bits set and the EV_REL axes are obviously missing (but not always...).
Instead of BTN_LEFT, a keyboard would have e.g. KEY_ESC, KEY_A, KEY_B, etc.
90% of device
debugging is looking at the event codes and figuring out which ones are
missing or shouldn't be there.

Exercise: You should now be able to read a evemu-record
description from any mouse or keyboard device connected to your computer and
understand what it means. This also applies
to most special devices such as remotes - the only thing that changes are
the names for the keys/buttons. Just run sudo evemu-describe and pick any
device in the list.

The events from relative devices and keyboards

evdev is a serialised protocol. It sends a series of events and then a
synchronisation event to notify us that the preceeding events all belong
together. This synchronisation event is EV_SYN SYN_REPORT, is generated by
the kernel, not the device and hence all EV_SYN codes are always available
on all devices.

Let's have a look at a mouse movement. As explained above, half the line is
machine-readable but we can ignore that bit and look at the human-readable
output on the right.

Mostly the same as button events. But wait, there is one difference: we have
a value of 2 as well. For key events, a value 2 means "key repeat".
If you're on the tty, then this is what generates repeat keys for you. In X
and Wayland we ignore these repeat events and instead use XKB-based key
repeat.

Now look at the keyboard events again and see if you can make sense of the sequence.
We have an Enter release (but no press), then ctrl down (and repeat),
followed by a 'c' press - but no release. The explanation is simple - as
soon as I hit enter in the terminal, evemu-record started recording so it
captured the enter release too. And it stopped recording as soon as ctrl+c
was down because that's when it was cancelled by the terminal. One important
takeaway here: the evdev protocol is not guaranteed to be balanced. You may
see a release for a key you've never seen the press for, and you may be
missing a release for a key/button you've seen the press for (this happens
when you stop recording). Oh, and there's one danger:
if you record your keyboard and you type your password, the keys will show
up in the output. Security experts generally reocmmend not publishing event
logs with your password in it.

Exercise: You should now be able to read a evemu-record
events list from any mouse or keyboard device connected to your computer and
understand the event sequence.This also applies to most special devices such as
remotes - the only thing that changes are the names for the keys/buttons.
Just run sudo evemu-record and pick any device listed.

Absolute devices

Things get a bit more complicated when we look at absolute input devices
like a touchscreen or a touchpad. Yes, touchpads are absolute devices in
hardware and the conversion to relative events is done in userspace by e.g.
libinput. The output of my touchpad is below. Note that I've manually removed a few
bits to make it easier to grasp, they will appear later in the multitouch
discussion.

We have a BTN_LEFT again and a set of other buttons that I'll explain in a
second. But first we look at the EV_ABS output. We have the same naming
system as above. ABS_X and ABS_Y are the x and y axis on the device,
ABS_PRESSURE is an (arbitrary) ranged pressure value.

Absolute axes have a bit more
state than just a simple bit. Specifically, they have a minimum and maximum
(not all hardware has the top-left sensor position on 0/0, it can
be an arbitrary position, specified by the minimum). Notable here is that
the axis ranges are simply the ones announced by the device - there is no
guarantee that the values fall within this range and indeed a lot of
touchpad devices tend to send values slightly outside that range.
Fuzz and flat can be safely ignored, but resolution is interesting. It is
given in units per millimeter and thus tells us the size of the device. in
the above case: (5112 - 1024)/42 means the device is 97mm wide. The
resolution is quite commonly wrong,
a lot of axis
overrides need the resolution changed
to the correct value.

The axis description also has a current value listed. The kernel only sends
events when the value changes, so even if the actual hardware keeps sending
events, you may never see them in the output if the value remains the same.
In other words, holding a finger perfectly still on a touchpad creates
plenty of hardware events, but you won't see anything coming out of the
event node.

Finally, we have properties on this device. These are used to indicate
general information about the device that's not otherwise obvious. In this
case INPUT_PROP_POINTER tells us that we need a pointer for this device (it
is a touchpad after all, a touchscreen would instead have INPUT_PROP_DIRECT
set). INPUT_PROP_BUTTONPAD means that this is a so-called clickpad, it does
not have separate physical buttons but instead the whole touchpad clicks.
Ignore INPUT_PROP_TOPBUTTONPAD because it only applies to the Lenovo *40
series of devices.

Ok, back to the buttons: aside from BTN_LEFT, we have BTN_TOUCH. This one
signals that the user is touching the surface of the touchpad (with some
in-kernel defined minimum pressure value). It's not just for finger-touches,
it's also used for graphics tablet stylus touchpes (so really, it's more
"contact" than "touch" but meh).

The BTN_TOOL_FINGER event tells us that a finger is in detectable range. This
gives us two bits of information: first, we have a finger (a tablet would have
e.g. BTN_TOOL_PEN) and second, we may have a finger in proximity without
touching. On many touchpads, BTN_TOOL_FINGER and BTN_TOUCH come in the same
event, but others can detect a finger hovering over the touchpad too (in which
case you'd also hope for ABS_DISTANCE being available on the touchpad).

Finally, the BTN_TOOL_DOUBLETAP up to BTN_TOOL_QUINTTAP tell us whether the
device can detect 2 through to 5 fingers on the touchpad. This doesn't actually
track the fingers, it merely tells you "3 fingers down" in the case of
BTN_TOOL_TRIPLETAP.

Exercise: Look at your touchpad's description and figure out if the size
of the touchpad is correct based on the axis information [1]. Check how many
fingers your touchpad can detect and whether it can do pressure or distance detection.

The events from absolute devices

Events from absolute axes are not really any different than events from
relative devices which we already covered. The same type/code combination with
a value and a timestamp, all framed by EV_SYN SYN_REPORT events. Here's an
example of me touching the touchpad:

In the first event you see BTN_TOOL_FINGER and BTN_TOUCH set (this touchpad
doesn't detect hovering fingers). An x/y coordinate pair and a pressure value.
The pressure changes in the second event, the third event changes pressure and
location. Finally, we have BTN_TOOL_FINGER and BTN_TOUCH released on finger up,
and the pressure value goes back to 0. Notice how the second event didn't
contain any x/y coordinates? As I said above, the kernel only sends updates on
absolute axes when the value changed.

In the first event, the touchpad detected all three fingers at the same time.
So get BTN_TOUCH, x/y/pressure and BTN_TOOL_TRIPLETAP set. Note that the
various BTN_TOOL_* bits are mutually exclusive. BTN_TOOL_FINGER means
"exactly 1 finger down" and you can't have exactly 1 finger down when you have
three fingers down. In the second event x and pressure update (y has no event,
it stayed the same).

In the event after the break, we switch from three fingers to one finger.
BTN_TOOL_TRIPLETAP is released, BTN_TOOL_FINGER is set. That's very common.
Humans aren't robots, you can't release all fingers at exactly the same time, so
depending on the hardware scanout rate you have intermediate states where one
finger has left already, others are still down. In this case I released two
fingers between scanouts, one was still down. It's not uncommon to see a full
cycle from BTN_TOOL_FINGER to BTN_TOOL_DOUBLETAP to BTN_TOOL_TRIPLETAP on finger
down or the reverse on finger up.

Exercise: test out the pressure values on your touchpad and see how close
you can get to the actual announced range. Check how accurate the multifinger
detection is by tapping with two, three, four and five fingers. (In both cases,
you'll likely find that it's very much hit and miss).

Multitouch and slots

Now we're at the most complicated topic regarding evdev devices. In the
case of multitouch devices, we need to send multiple touches on the same
axes. So we need an additional dimension and that is called multitouch
slots (there is another, older multitouch protocol that doesn't use
slots but it is so rare now that you don't need to bother).

First: all axes that are multitouch-capable are repeated as ABS_MT_foo axis.
So if you have ABS_X, you also get ABS_MT_POSITION_X and both axes have the
same axis ranges and resolutions. The reason here is
backwards-compatibility: if a device only sends multitouch events, older
programs only listening to the ABS_X etc. events won't work. Some axes may
only be available for single-touch (ABS_MT_TOOL_WIDTH in this case).

We have an x and y position for multitouch as well as a pressure axis.
There are also two special multitouch axes that aren't really axes.
ABS_MT_SLOT and ABS_MT_TRACKING_ID. The former specifies which
slot is currently active, the latter is used to track touch points.

Slots are a static property of a device. My touchpad, as you can see above
ony supports 2 slots (min 0, max 1) and thus can track 2 fingers at a
time. Whenever the first finger is set down it's coordinates will be tracked
in slot 0, the second finger will be tracked in slot 1. When the finger in
slot 0 is lifted, the second finger continues to be tracked in slot 1, and
if a new finger is set down, it will be tracked in slot 0. Sounds more
complicated than it is, think of it as an array of possible touchpoints.

The tracking ID is an incrementing number that lets us tell touch
points apart and also tells us when a touch starts and when it ends. The two
values are either -1 or a positive number. Any positive number means "new touch"
and -1 means "touch ended". So when you put two fingers down and lift them
again, you'll get a tracking ID of 1 in slot 0, a tracking ID of 2 in slot
1, then a tracking ID of -1 in both slots to signal they ended. The tracking
ID value itself is meaningless, it simply increases as touches are created.

We have a tracking ID (387) signalling finger down, as well as a position
plus pressure. then some updates and eventually a tracking ID of -1
(signalling finger up). Notice how there is no ABS_MT_SLOT here - the kernel
buffers those too so while you stay in the same slot (0 in this case) you
don't see any events for it. Also notice how you get both single-finger as
well as multitouch in the same event stream. This is for backwards
compatibility [2]

This was a really quick two-finger tap that illustrates the tracking IDs nicely.
In the first event we get a touch down, then an ABS_MT_SLOT event. This
tells us that subsequent events belong to the other slot, so it's the other
finger. There too we get a tracking ID + position. In the next event we get
an ABS_MT_SLOT to switch back to slot 0. Tracking ID of -1 means that touch
ended, and then we see the touch in slot 1 ended too.

Note that "scroll" is something handled in userspace, so what you see here
is just a two-finger move. Everything in there i something we've already
seen, but pay attention to the two middle events: as updates come in for
each finger, the ABS_MT_SLOT changes before the upates are sent. The kernel
filter for identical events is still in effect, so in the third event we
don't get an update for the X position on slot 1. The filtering is
per-touchpoint, so in this case this means that slot 1 position x is still
on 3511, just as it was in the previous event.

That's all you have to remember, really. If you think of evdev as a
serialised way of sending an array of touchpoints, with the slots as the
indices then it should be fairly clear. The rest is then just about actually
looking at the touch positions and making sense of them.

Exercise: do a pinch gesture on your touchpad. See if you can
track the two fingers moving closer together. Then do the same but only move
one finger. See how the non-moving finger gets less updates.

That's it. There are a few more details to evdev but much of that is just
more event types and codes. The few details you really have to worry about
when processing events are either documented in libevdev or abstracted away
completely. The above should be enough to understand what your device does,
and what goes wrong when your device isn't working. Good luck.

[1] If not, file a bug against systemd's hwdb and CC me so we can put
corrections in
[2] We treat some MT-capable touchpads as single-touch devices in libinput
because the MT data is garbage

Just wondering - I know you said most of the stuff should be implemented by the client, but what about the circular scrolling? Does it fall to the client part as well (since libinput already handles edge/two finger scrolling)?

Mitja: note that "client" as seen from the evdev protocol is the userspace process reading i (e.g. libinput), not an X or Wayland client. And yes, with touchpads almost everything including basic pointer motion is implemented in the client

Aidan: that's for in-kernel code, right? If so, it needs to be set separately with input_abs_set_res(), have a look at some of the other drivers. Resolution came quite late (2.6.30) and I guess there's never been a cleanup for those functions to merge them.

Thanks Peter,Done! That integer division does not bring great accuracy. Maybe I should scale up the max_x and max_y then I could get them closer to reality. Hmm, then I will have steps in the data according to whatever scale factor I might use. Any implications in this from an Apps perspective?

Aidan: libinput uses the resolution to convert to physical distances. pointer acceleration and other things are based on the phys. distance, so the more correct the resolution is, the more correct the feel from the device will be. On the X/Wayland client side, I don't think it matters too much but there could be specialised cases where it matters (some CAD app or something maybe). Not for touchpads though, that only affects graphics tablets.

FTR as the next question I am curious about is "doesn't libinput handle gamepads?", the answer is here https://lists.freedesktop.org/archives/wayland-devel/2015-January/019462.html

TL;DR: handling it in libinput makes no sense, and apps like games are apparently using evdev directly.

Also the non-asked subquestion I was wondering about is: how could kernel know the arrangement of buttons on a gamepad to assign to buttons rational names, like it did with BTTN_LEFT for mouse. The answer is: it don't. There seems to be no way to tell, and, although I didn't look yet at evdev output for gamepad, but apparently the names would be pretty random, like js1, js2, js3, etc.

Yagamy: the js1 devices are joydev devices and don't speak the evdev protocol, it's a different kernel driver (similar to evdev0 vs mouse0 - same physical device but different device nodes with different protocols).

The kernel knows the button mappings for some devices, primarily where there's a custom driver. otherwise it relies on HID mappings (+quirks) or guesswork

I set some calibration/settings on input device (turn device wheel to left. I did this with write() function) while this happens input system gathers data. So when I first call libevdev_next_event it gives me old garbage data. But I dont want those.