Thursday, September 12, 2019

An annoying thing about C code is that there are plenty of functions that cannot be unit-tested by some external framework - specifically anything declared as static. Any larger code-base will end up with hundreds of those functions, many of which are short and reasonably self-contained but complex enough to not trust them by looks only. But since they're static I can't access them from the outside (and "outside" is defined as "not in the same file" here).

The approach I've chosen in the past is to move the more hairy ones into separate files or at least declare them normally. That works but is annoying for some cases, especially those that really only get called once. In case you're wondering whether you have at least one such function in your source tree: yes, the bit that parses your commandline arguments is almost certainly complicated and not tested.

Anyway, this week I've finally found the right combination of hacks to make testing static functions easy, and it's:

#include the source file in your test code.

Mock any helper functions you'd need to trick the called functions

Instruct the linker to ignore unresolved symbols

And boom, you can write test cases to only test a single file within your source tree. And without any modifications to the source code itself.

There is no restriction on which test suite you can use. I've started adding a few of test cases based on this approach to libinput and so far it's working well. If you have a better approach or improvements, I'm all ears.

Monday, August 26, 2019

Sounds like déjà vu? Right, I posted a post with an almost identical title 18 months ago or so. This is about Tuhi 0.2, new and remodeled and completely different to that. Sort-of.

Tuhi is an application that supports the Wacom SmartPad devices - Bamboo Spark, Bamboo Slate, Bamboo Folio and Intuos Pro. The Bamboo range are digital notepads. They come with a real pen, you draw normally on the pad and use Bluetooth LE and Wacom's Inkspace application later to sync the files to disk. The Intuos Pro is the same but it's designed as a "normal" tablet with the paper mode available as well.

18 months ago, Benjamin Tissoires and I wrote Tuhi as a DBus session daemon. Tuhi would download the drawings from the file and make them available as JSON files over DBus to be converted to SVG or some other format by ... "clients". We wrote a simple commandline tool to debug Tuhi but no GUI, largely in the hope that maybe someone would be interested in doing that. Fast forward to now and that hasn't happened but I had some spare cycles over the last weeks so I present to you: Tuhi 0.2, now with a GTK GUI:

It's basic but also because it shouldn't do much more than just downloading the drawings and allowing you to save them. This is not an editing UI, it's effectively a file manager for the drawings on the tablet. And since by design those drawings get deleted as you download them, there isn't even much to that (don't worry, Tuhi doesn't really delete files, you can recover almost everything).

Under the hood there were some internal changes too but I suspect they'll be boring to most. The more interesting bits are reworks so we can test the conversions a lot better now and - worst case - recover files if Tuhi crashes. It is largely reverse-engineered after all.

On that note I would like to also extend my thanks to Wacom who have provided us with some of the specs for the protocol (under NDA, we cannot share these with the community, sorry). These specs helped tremendously understanding the protocol bits that were confusing at best and unknown at worst. There are still some corners in the protocol that we don't know but for the most recent generation (i.e. Intuos Pro) we should have correct parsing of the protocol.

Wednesday, July 17, 2019

The average user has approximately one thumb per hand. That thumb comes in handy for a number of touchpad interactions. For example, moving the cursor with the index finger and clicking a button with the thumb. On so-called Clickpads we don't have separate buttons though. The touchpad itself acts as a button and software decides whether it's a left, right, or middle click by counting fingers and/or finger locations. Hence the need for thumb detection, because you may have two fingers on the touchpad (usually right click) but if those are the index and thumb, then really, it's just a single finger click.

libinput has had some thumb detection since the early days when we were still hand-carving bits with stone tools. But it was quite simplistic, as the old documentation illustrates: two zones on the touchpad, a touch started in the lower zone was always a thumb. Where a touch started in the upper thumb area, a timeout and movement thresholds would decide whether it was a thumb. Internally, the thumb states were, Schrödinger-esque, "NO", "YES", and "MAYBE". On top of that, we also had speed-based thumb detection - where a finger was moving fast enough, a new touch would always default to being a thumb. On the grounds that you have no business dropping fingers in the middle of a fast interaction. Such a simplistic approach worked well enough for a bunch of use-cases but failed gloriously in other cases.

Thanks to Matt Mayfields' work, we now have a much more sophisticated thumb detection algorithm. The speed detection is still there but it better accounts for pinch gestures and two-finger scrolling. The exclusion zones are still there but less final about the state of the touch, a thumb can escape that "jail" and contribute to pointer motion where necessary. The new documentation has a bit of a general overview. A requirement for well-working thumb detection however is that your device has the required (device-specific) thresholds set up. So go over to the debugging thumb thresholds documentation and start figuring out your device's thresholds.

As usual, if you notice any issues with the new code please let us know, ideally before the 1.14 release.

Wednesday, June 19, 2019

This is merely an update on the current status quo, if you read this post in a year's time some of the details may have changed

libinput provides an API to handle graphics tablets, i.e. the tablets that are used by artists. The interface is based around tools, each of which can be in proximity at any time. "Proximity" simply means "in detectable range". libinput promises that any interaction is framed by a proximity in and proximity out event pair, but getting to this turned out to be complicated. libinput has seen a few changes recently here, so let's dig into those. Remember that proverb about seeing what goes into a sausage? Yeah, that.

In the kernel API, the proximity events for pens are the BTN_TOOL_PEN bit. If it's 1, we're in proximity, if it's 0, we're out of proximity. That's the theory.

Wacom tablets (or rather the kernel driver) always reset all axes on proximity out. So libinput needs to take care not to send a 0 value to the caller, lest you want a jump to the top left corner every time you move the pen away from the tablet. Some Wacom pens have serial numbers and we use those to uniquely identify a tool. But some devices start sending proximity and axis events before we get the serial numbers which means we can't identify the tool until several ms later. In that case we simply discard the serial. This means we cannot uniquely identify those pens but so far no-one has complained.

A bunch of tablets (HUION) don't have proximity at all. For those, we start getting events and then stop getting events, without any other information. So libinput has a timer - if we don't get events for a given time, we force a proximity out. Of course, this means we also need to force a proximity in when the next event comes in. These tablets are common enough that recently we just enabled the proximity timeout for all tablets. Easier than playing whack-a-mole, doubly so because HUION re-uses USD ids so you can't easily identify them anyway.

Some tablets (HP Spectre 13) have proximity but never send it. So they advertise the capability, just don't generate events for it. Same handling as the ones that don't have proximity at all.

Some tablets (HUION) have proximity, but only send it once per plug-in, after that it's always in proximity. Since libinput may start after the first pen interaction, this means we have to a) query the initial state of the device and b) force proximity in/out based on the timer, just like above.

Some tablets (Lenovo Flex 5) sometimes send proximity out events, but sometimes do not. So for those we have a timer and forced proximity events, but only when our last interaction didn't trigger a proximity event.

The Dell Active Pen always sends a proximity out event, but with a delay of ~200ms. That timeout is longer than the libinput timeout so we'll get a proximity out event, but only after we've already forced proximity out. We can just discard that event.

The Dell Canvas pen (identifies as "Wacom HID 4831 Pen") can have random delays of up to ~800ms in its event reporting. Which would trigger forced proximity out events in libinput. Luckily it always sends proximity out events, so we could quirk out to specifically disable the timer.

The HP Envy x360 sends a proximity in for the pen, followed by a proximity in from the eraser in the next event. This is still an unresolved issue at the time of writing.

That's the current state of things, I'm sure it'll change in a few months time again as more devices decide to be creative. They are artist's tools after all.

The lesson to take away here: all of the above are special cases that need to be implemented but this can only be done on demand. There's no way any one person can test every single device out there and testing by vendors is often nonexistent. So if you want your device to work, don't complain on some random forum, file a bug and help with debugging and testing instead.

We're on the road to he^libinput 1.14 and last week I merged the Dell Canvas Totem support. "Wait, what?" I hear you ask, and "What is that?". Good question - but do pay attention to random press releases more. The Totem (Dell.com) is a round knob that can be placed on the Dell Canvas. Which itself is a pen and touch device, not unlike the Wacom Cintiq range if you're familiar with those (if not, there's always lmgtfy).

The totem's intended use is as secondary device - you place it on the screen while you're using the pen and up pops a radial menu. You can rotate the totem to select items, click it to select something and bang, you're smiling like a stock photo model eating lettuce. The radial menu is just an example UI, there are plenty others. I remember reading papers about bimanual interaction with similar interfaces that dated back to the 80s, so there's a plethora to choose from. I'm sure someone at Dell has written Totem-Pong and if they have not, I really question their job priorities. The technical side is quite simple, the totem triggers a set of touches in a specific configuration, when the firmware detects that arrangement it knows this isn't a finger but the totem.

Pen and touch we already handle well, but the totem required kernel changes and a few new interfaces in libinput. And that was the easy part, the actual UI bits will be nasty.

The kernel changes went into 4.19 and as usual you can throw noises of gratitude at Benjamin Tissoires. The new kernel API basically boils down to the ABS_MT_TOOL_TYPE axis sending MT_TOOL_DIAL whenever the totem is detected. That axis is (like others of the ABS_MT range) an odd one out. It doesn't work as an axis but rather an enum that specifies the tool within the current slot. We already had finger, pen and palm, adding another enum value means, well, now we have a "dial". And that's largely it in terms of API - handle the MT_TOOL_DIAL and you're good to go.

libinput's API is only slightly more complicated. The tablet interface has a new tool type called the LIBINPUT_TABLET_TOOL_TYPE_TOTEM and a new pair of axes for the tool, the size of the touch ellipse. With that you can get the position of the totem and the size (so you know how big the radial menu needs to be). And that's basically it in regards to the API. The actual implementation was a bit more involved, especially because we needed to implement location-based touch arbitration first.

I haven't started on the Wayland protocol additions yet but I suspect they'll look the same as the libinput API (the Wayland tablet protocol is itself virtually identical to the libinput API). The really big changes will of course be in the toolkits and the applications themselves. The totem is not a device that slots into existing UI paradigms, it requires dedicated support. Whether this will be available in your favourite application is likely going to be up to you. Anyway, christmas in July [1] is coming up so now you know what to put on your wishlist.

[1] yes, that's a thing. Apparently christmas with summery temperature, nice weather, sandy beaches is so unbearable that you have to re-create it in the misery of winter. Explains everything you need to know about humans, really.

Thursday, March 21, 2019

I had to work on an image yesterday where I couldn't install anything and the amount of pre-installed tools was quite limited. And I needed to debug an input device, usually done with libinput record. So eventually I found that hexdump supports formatting of the input bytes but it took me a while to figure out the right combination. The various resources online only got me partway there. So here's an explanation which should get you to your results quickly.

By default, hexdump prints identical input lines as a single line with an asterisk ('*'). To avoid this, use the -v flag as in the examples below.

hexdump's format string is single-quote-enclosed string that contains the count, element size and double-quote-enclosed printf-like format string. So a simple example is this:

$ hexdump -v -e '1/2 "%d\n"'
-11643
23698
0
0
-5013
6
0
0

This prints 1 element ('iteration') of 2 bytes as integer, followed by a linebreak. Or in other words: it takes two bytes, converts it to int and prints it. If you want to print the same input value in multiple formats, use multiple -e invocations.

Friday, March 15, 2019

Ho ho ho, let's write libinput. No, of course I'm not serious, because
no-one in their right mind would utter "ho ho ho" without a sufficient
backdrop of reindeers to keep them sane. So what this post is instead is me
writing a nonworking fake libinput in Python, for the sole purpose of
explaining roughly how libinput's architecture looks like. It'll be to the
libinput what a Duplo car is to a Maserati. Four wheels and something
to entertain the kids with but the queue outside the nightclub won't be
impressed.

The target audience are those that need to hack on libinput and where the
balance of understanding vs total confusion is still shifted towards the
latter. So in order to make it easier to associate various bits, here's a
description of the main building blocks.

libinput uses something resembling OOP except that in C you can't have nice
things unless what you want is a buffer overflow\n\80xb1001af81a2b1101.
Instead, we use opaque structs, each with accessor methods and an unhealthy
amount of verbosity. Because Python does have classes, those structs are
represented as classes below. This all won't be actual working Python code, I'm just using the
syntax.

We have two different modes of initialisation, udev and path. The udev
interface is used by Wayland compositors and adds all devices on the given
udev seat. The path interface is used by the X.Org driver and adds only one
specific device at a time. Both interfaces have the dispatch() and
get_events() methods which is how every caller gets events out of
libinput.

In both cases we create a libinput device from the data and create an event
about the new device that bubbles up into the event queue.

But what really are events? Are they real or just a fidget spinner of our
imagination? Well, they're just another object in libinput.

You get the gist. Each event is actually an event of a subtype with a few
common shared fields and a bunch of type-specific ones. The events often
contain some internal value that is calculated on request.
For example, the API for the absolute x/y values returns mm, but
we store the value in device units instead and convert to mm on request.

So, what's a device then? Well, just another
I-cant-believe-this-is-not-a-class with relatively few surprises:

Our evdev device is actually a subclass (well, C, *handwave*) of the
public device and its main function is "read things off the device node".
And it passes that on to a magical interface. Other than that, it's
a collection of generic functions that apply to all devices. The interfaces
is where most of the real work is done.

The interface is decided on by the udev type and is where the
device-specifics happen. The touchpad interface deals with touchpads, the
tablet and switch interface with those devices and the fallback interface is
that for mice, keyboards and touch devices (i.e. the simple devices).

Each interface has very device-specific event processing and can be
compared to the Xorg synaptics vs wacom vs evdev drivers. If you are fixing a touchpad bug, chances are you only need to care about the touchpad interface.

The advantage of this system is twofold. First, the main libinput code only
needs one place where we really care about which acceleration method we
have. And second, the acceleration code can be compiled separately for
analysis and to generate pretty graphs. See the pointer
acceleration docs. Oh, and it also allows us to easily have per-device pointer acceleration methods.

Finally, we have one more building block - configuration options. They're a
bit different in that they're all similar-ish but only to make switching
from one to the next a bit easier.

And that's basically it, those are the building blocks libinput has. The
rest is detail. Lots of it, but if you understand the architecture outline
above, you're most of the way there in diving into the details.

One of the features in the soon-to-be-released libinput 1.13 is
location-based touch arbitration. Touch arbitration is the process of
discarding touch input on a tablet device while a pen is in proximity.
Historically, this was provided by the kernel wacom driver but libinput has
had userspace touch arbitration for quite a while now, allowing for touch
arbitration where the tablet and the touchscreen part are
handled by different kernel drivers.

Basic touch arbitratin is relatively simple: when a pen goes into
proximity, all touches are ignored. When the pen goes out of proximity,
new touches are handled again. There are some extra details (esp. where the kernel handles arbitration too) but let's ignore those for now.

With libinput 1.13 and in preparation for the Dell Canvas Dial Totem, the
touch arbitration can now be limited to a portion of the screen only. On the
totem (future patches, not yet merged) that portion is a square slightly
larger than the tool itself. On normal tablets, that portion is a rectangle,
sized so that it should encompass the users's hand and area around the pen,
but not much more. This enables users to use both the pen and touch input at
the same time, providing for bimanual interaction (where the GUI itself
supports it of course). We use the tilt information of the pen (where
available) to guess where the user's hand will be to adjust the rectangle
position.

There are some heuristics involved and I'm not sure we got all of them right
so I encourage you to give it a try and file an issue where it doesn't
behave as expected.

Thursday, February 14, 2019

In this blog post, I'll explain how to update systemd's hwdb for a new
device-specific entry. I'll focus on input devices, as usual.

What is the hwdb and why do I need to update it?

The hwdb is a binary database sitting at
/etc/udev/hwdb.bin and /usr/lib/udev/hwdb.d. It is usually used to apply udev properties to specific
devices, those properties are then picked up by other processes (udev builtins, libinput, ...) to apply device-specific behaviours.
So you'll need to update the hwdb if you need a specific behaviour from the device.

One of the use-cases I commonly deal with is that some touchpad announces wrong axis ranges or resolutions. With the correct hwdb entry (see the example later) udev can correct these at device initialisation time and every process sees the right axis ranges.

The database is compiled from the various .hwdb files you have
sitting on your system, usually in /etc/udev/hwdb.d and /usr/lib/hwdb.d.
The terser format of the hwdb files makes them easier to update than, say, writing a udev
rule to set those properties.

The full process goes something like this:

The various .hwdb files are installed or modified

The hwdb.bin file is generated from the .hwdb files

A udev rule triggers the udev hwdb builtin. If a match occurs, the builtin prints the to-be properties, and udev captures the output and applies it as udev properties to the device

Some other process (often a different udev builtin) reads the udev property value and does something.

On its own, the hwdb is merely a lookup tool though, it does not modify devices. Think of it as
a virtual filing cabinet, something will need to look at it, otherwise it's just dead weight.

The IMPORT statement translates as "look up the hwdb, import the properties". The RUN statement runs the "keyboard" builtin which may change the device based on the various udev properties now set. The GOTO statement goes to skip the rest of the file.

So again, on its own the hwdb doesn't do anything, it merely prints to-be udev properties to stdout, udev captures those and applies them to the device. And then those properties need to be processed by some other process to actually apply changes.

hwdb file format

The basic format of each hwdb file contains two types of entries, match
lines and property assignments (indented by one space). The match line
defines which device it is applied to.

The match line is the one starting with "evdev", the other two lines are property assignments. Property values are strings, any interpretation to numeric values or others is to be done in the process that requires those properties. Noteworthy here: the hwdb can overwrite previously set properties, but it cannot unset them.

The match line is not specified by the hwdb beyond "it's a glob". The
format to use is defined by the udev rule that invokes the hwdb builtin. Usually the format is:

someprefix:search criteria:

For example, the udev rule that applies for the match above is this one in
60-evdev.rules:

Attentive readers will have noticed that the two entries from
60-evdev.rules I posted here differ. You can have multiple match formats in
the same hwdb file. The hwdb doesn't care, it's just a matching system.

We keep the hwdb files matching the udev rules names for ease of
maintenance so 60-evdev.rules keeps the hwdb files in 60-evdev.hwdb and so on.
But this is just for us puny humans, the hwdb will parse all files it finds into one database.
If you have a hwdb entry in
my-local-overrides.hwdb it will be matched. The file-specific
prefixes are just there to not accidentally match against an unrelated entry.

Applying hwdb updates

The hwdb is a compiled format, so the first thing to do after any changes is
to run

$ systemd-hwdb update

This command compiles the files down to
the binary hwdb that is actually used by udev. Without that update, none of
your changes will take effect.

The second thing is: you need to trigger the udev rules for the device you
want to modify. Either you do this by physically unplugging and re-plugging
the device or by running

$ udevadm trigger

or, better, trigger only the device you care about to avoid accidental side-effects:

$ udevadm trigger /sys/class/input/eventXYZ

In case you also modified the udev rules you should re-load those too. So the
full quartet of commands after a hwdb update is:

That udevadm info command lists all assigned properties, these should now include the modified entries.

Adding new entries

Now let's get down to what you actually want to do, adding a new entry to
the hwdb. And this is where it also get's tricky to have a generic guide
because every hwdb file has its own custom match rules.

The best approach is to open the .hwdb files and the matching .rules
file and figure out what the match formats are and which one is best. For
USB devices there's usually a match format that uses the vendor and product
ID. For built-in devices like touchpads and keyboards there's usually a
dmi-based match format (see /sys/class/dmi/id/modalias). In most cases,
you can just take an existing entry and copy and modify it.

My recommendation is: add an extra property that makes it
easy to verify the new entry is applied. For example do this:

Now run the update commands from above. If FOO=1 doesn't show up, then you know it's the hwdb
entry that's not yet correct. If FOO=1 does show up in the udevadm
info output, then you know the hwdb matches correctly and any issues will be in the next layer.

Increase the value with every change so you can tell whether the most recent
change is applied. And before your submit a pull request, remove the FOO
entry.

Oh, and once it applies correctly, I recommend restarting the system to make sure everything is
in order on a freshly booted system.

Troubleshooting

The reason for adding hwdb entries is always because we want the system to
handle a device in a custom way. But it's hard to figure out what's wrong
when something doesn't work (though 90% of the time it's a typo in the hwdb match).

In almost all cases, the debugging sequence is the following:

does the FOO property show up?

did you run systemd-hwdb update?

did you run udevadm trigger?

did you restart the process that requires the new udev property?

is that process new enough to have support for that property?

If the answer to all these is "yes" and it still doesn't work, you may have
found a bug. But 99% of the time, at least one of those is a sound "no.
oops.".

Your hwdb match may run into issues with some 'special' characters. If your device has e.g. an ® in its device name (some Microsoft devices have this), a bug in systemd caused the match to fail. That bug is fixed now but until it's available in your distribution, replace with an asterisk ('*') in your match line.

Greybeards who have been around since before 2014 (systemd v219) may remember a different tool to update the hwdb: udevadm hwdb --update. This tool still exists, but it does not have the exact same behaviour as systemd-hwdb update. I won't go into details but the hwdb generated by the udevadm tool can provide unexpected matches if you have multiple matches with globs for the same device. A more generic glob can take precedence over a specific glob and so on. It's a rare and niche case and fixed since systemd v233 but the udevadm behaviour remained the same for backwards-compatibility.