Who-T

Wednesday, December 12, 2018

Disclaimer: this is pending for v4.21 and thus not yet in any kernel release.

Most wheel mice have a physical feature to stop the wheel from spinning freely. That feature is called detents, notches, wheel clicks, stops, or something like that. On your average mouse that is 24 wheel clicks per full rotation, resulting in the wheel rotating by 15 degrees before its motion is arrested. On some other mice that angle is 18 degrees, so you get 20 clicks per full rotation.

Of course, the world wouldn't be complete without fancy hardware features. Over the last 10 or so years devices have added free-wheeling scroll wheels or scroll wheels without distinct stops. In many cases wheel behaviour can be configured on the device, e.g. with Logitech's HID++ protocol. A few weeks back, Harry Cutts from the chromium team sent patches to enable Logitech high-resolution wheel scrolling in the kernel. Succinctly, these patches added another axis next to the existing REL_WHEEL named REL_WHEEL_HI_RES. Where available, the latter axis would provide finer-grained scroll information than the click-by-click REL_WHEEL. At the same time I accidentally stumbled across the documentation for the HID Resolution Multiplier Feature. A few patch revisions later and we now have everything queued up for v4.21. Below is a summary of the new behaviour.

The kernel will continue to provide REL_WHEEL as axis for "wheel clicks", just as before. This axis provides the logical wheel clicks, (almost) nothing changes here. In addition, a REL_WHEEL_HI_RES axis is available which allows for finer-grained resolution. On this axis, the magic value 120 represents one logical traditional wheel click but a device may send a fraction of 120 for a smaller motion. Userspace can either accumulate the values until it hits a full 120 for one wheel click or it can scroll by a few pixels on each event for a smoother experience. The same principle is applied to REL_HWHEEL and REL_HWHEEL_HI_RES for horizontal scroll wheels (which these days is just tilting the wheel). The REL_WHEEL axis is now emulated by the kernel and simply sent out whenever we have accumulated 120.

Important to note: REL_WHEEL and REL_HWHEEL are now legacy axes and should be ignored by code handling the respective high-resolution version.

The magic value of 120 is taken directly from Windows. That value was chosen because it has a good number of integer factors, so dividing 120 by whatever multiplier the mouse uses gives you a integer fraction of 120. And because HW manufacturers want it to work on Windows, we can rely on them doing it right, provided we use the same approach.

There are two implementations that matter. Harry's patches enable the high-resolution scrolling on Logitech mice which seem to mostly have a multiplier of 8 (i.e. REL_WHEEL_HI_RES will send eight events with a value of 15 before REL_WHEEL sends 1 click). There are some interesting side-effects with e.g. the MX Anywhere 2S. In high-resolution mode with a multiplier of 8, a single wheel movement does not always give us 8 events, the firmware does its own magic here. So we have some emulation code in place with the goal of making the REL_WHEEL event happen on the mid-point of a wheel click motion. The exact point can shift a bit when the device sends 7 events instead of 8 so we have a few extra bits in place to reset after timeouts and direction changes to make sure the wheel behaviour is as consistent as possible.

The second implementation is for the generic HID protocol. This was all added for Windows Vista, so we're only about a decade behind here. Microsoft got the Resolution Multiplier feature into the official HID documentation (possibly in the hope that other HW manufacturers implement it which afaict didn't happen). This feature effectively provides a fixed value multiplier that the device applies in hardware when enabled. It's basically the same as the Logitech one except it's set through a HID feature instead of a vendor-specific protocol. On the devices tested so far (all Microsoft mice because no-one else seems to implement this) the multipliers vary a bit, ranging from 4 to 12. And the exact behaviour varies too. One mouse behaves correctly (Microsoft Comfort Optical Mouse 3000) and sends more events than before. Other mice just send the multiplied value instead of the normal value, so nothing really changes. And at least one mouse (Microsoft Sculpt Ergonomic) sends the tilt-wheel values more frequently and with a higher value. So instead of one event with value 1 every X ms, we now get an event with value 3 every X/4 ms. The mice tested do not drop events like the Logitech mice do, so we don't need fancy emulation code here. Either way, we map this into the 120 range correctly now, so userspace gets to benefit.

As mentioned above, the Resolution Multiplier HID feature was introduced for Windows Vista which is... not the most recent release. I have a strong suspicion that Microsoft dumped this feature as well, the most recent set of mice I have access to don't provide the feature anymore (they have vendor-private protocols that we don't know about instead). So the takeaway for all this is: if you have a Logitech mouse, you'll get higher-resolution scrolling on v4.21. If you have a Microsoft mouse a few years old, you may get high-resolution wheel scrolling if the device supports it. Any other vendor or a new Microsoft mouse, you don't get it.

Coincidentally, if you know anyone at Microsoft who can provide me with the specs for their custom protocol, I'd appreciate it. We'd love to have support for it both in libratbag and in the kernel. Or any other vendor, come to think of it.

Tuesday, December 11, 2018

This time we're digging into HID - Human Interface Devices and more
specifically the protocol your mouse, touchpad, joystick, keyboard, etc. use to talk to your computer.

Remember the good old days where you had to install a custom driver for
every input device? Remember when PS/2 (the protocol) had to be extended
to accommodate for mouse wheels, and then again for five button mice. And
you had to select the right protocol to make it work. Yeah, me neither, I
tend to suppress those memories because the world is awful enough as it is.

As users we generally like devices to
work out of the box. Hardware manufacturers generally like to add bits and
bobs because otherwise who would buy that new device when last year's device looks identical. This difference in needs can only be
solved by one superhero: Committee-man, with the superpower to survive
endless meetings and get RFCs approved.

Many many moons ago, when USB itself was in its infancy, Committee man and
his sidekick Caffeine boy got the USB consortium agree on a standard for
input devices that is so self-descriptive that operating systems (Win95!)
can write one driver that can handle this year's device, and next year's,
and so on. No need to install extra drivers, your device will just work out
of the box. And so HID was born. This may only be an approximate
summary of history.

Originally HID was designed to work over USB. But just like Shrek
the technology world is obsessed with layers so these days HID works
over different transport layers. HID over USB is what your mouse uses, HID
over i2c may be what your touchpad uses. HID works over Bluetooth and it's
celebrity-diet version BLE. Somewhere, someone out there is very slowly moving a mouse
pointer by sending HID over carrier pigeons just to prove a point. Because there's always that one
guy.

HID is incredibly simple in that the static description of the device can
just be bytes burnt into the ROM like the Australian sun into unprepared English
backpackers. And the event frames are often an identical series of bytes
where every bit is filled in by the firmware according to the
axis/buttons/etc.

HID is incredibly complicated because parsing it is a stack-based mental
overload. Each individual protocol item is simple but getting it right and
all into your head is tricky. Luckily, I'm here for you to make this simpler
to understand or, failing that, at least more entertaining.

As said above, the purpose of HID is to make devices describe themselves in
a generic manner so that you can have a single driver handle any input
device. The idea is that the host parses that standard protocol and knows
exactly how the device will behave. This has worked out great, we only have
around 200 files dealing with vendor- and hardware-specific HID quirks as of
v4.20.

HID messages are Reports. And to know what a Report means and how to
interpret it, you need a Report Descriptor. That Report Descriptor is
static and contains a series of bytes detailing "what" and "where", i.e.
what a sequence of bits represents and where to find those bits in the
Report. So let's try and parse one of Report Descriptors, let's say for a
fictional mouse with a few buttons. How exciting, we're at the forefront of
innovation here.

The Report Descriptor consists of a bunch of Items. A parser reads
the next Item, processes the information within and moves on. Items are
small (1 byte header, 0-4 bytes payload) and generally only apply exactly
one tiny little bit of information. You need to accumulate several items to
build up enough information to actually know what's happening.

The "what" question of the Report Descriptor is answered with the so-called
Usage. This could be something simple like X or Y (0x30
and 0x31) or something more esoteric like System Menu Exit (0x88). A
Usage is 16 bits but all Usages are grouped into so-called Usage
Pages. A Usage Page too is a 16 bit value and together they form the
32-bit value that tells us what the device can do. Examples:

Note how the Usage in the last item is the same as the first one, without
the Usage Page you will mix things up. It helps if you always think
of as the Usage as a 32-bit number. For your kids' bed-time story time, here
are
the HID
Usage Tables from 2004 and the approved HID Usage Table Review Requests
of the last decade. Because nothing puts them to sleep quicker than droning
on about hex numbers associated with remote control buttons.

To successfully interpret a Report from the device, you need to know which
bits have which Usage associated with them. So let's go back to our
innovative mouse. We would want a report descriptor with 6 items like this:

This basically tells the host: X and Y both have 16 bits. So if we get a
4-byte Report from the device, we know two bytes are for X, two for Y.

HID was invented when a time when bits were more expensive than printer ink,
so we can't afford to waste any bits (still the case because who would want to spend an extra penny on more ROM). HID makes use of so-called Global items, once those are set
their value applies to all following items until changed. Usage Page and
Report Size are such Global items, so the above report descriptor is really
implemented like this:

The Report Count just tells us that 2 fields of the current Report Size are
coming up. We have two usages, two fields, and 16 bits each so we know what
to do. The Input item is sort-of the marker for the end of the stack,
it basically tells us "process what you've seen so far", together with a few
flags. Rel in this case means that the Usages are relative. Oh, and
Input means that this is data from device to host. Output would be
data from host to device, e.g. to set LEDs on a keyboard. There's also
Feature which indicates configurable items.

Buttons on a device are generally just numbered so it'd be monumental 16-bits-at-a-time waste to have
HID send Usage (Button1), Usage (Button2), etc. for every button on the
device. HID instead provides a Usage Minimum and Usage Maximum
to sequentially order them. This looks like this:

So we have 5 buttons here and each button has one bit. Note how the buttons are
Abs because a button state is not a relative value, it's either down
or up. HID is quite intolerant to Schrödinger's thought experiments.

Let's put the two things together and we have an almost-correct Report
descriptor:

New here is Cnst. This signals that the bits have a constant value, thus don't need a Usage and basically
don't matter (haha. yeah, right. in theory). Linux does indeed ignore
those. Cnst is used for
padding to align on byte boundaries - 5 bits for buttons plus 3 bits padding
make 8 bits. Which makes one byte as everyone agrees except for granddad
over there in the corner. I don't know how he got in.

Were we to get a 5-byte Report from the device, we'd parse it
approximately like this:

Hooray, we're almost ready. Except not. We may need more info to correctly
interpret the data within those reports.

The Logical Minimum and Logical Maximum specify the value
range of the actual data. We need this to tell us whether the data is signed
and what the allowable range is. Together with the Physical Minimum
and the Physical Maximum they specify what the values really mean.
In the simple case:

A device may use multiple Reports if the hardware doesn't gather all data
within the same hardware bits. Now, you may ask: if I get fifteen reports,
how should I know what belongs together? Good question, and lucky for you
the HID designers are miles ahead of you. Report IDs are grouped into
Collections.

Collections can have multiple types. An Application Collection
describes a set of inputs that make sense as a whole. Usually, every Report
Descriptor must define at least one Application Collection but you may have
two or more. For example, a a keyboard with integrated trackpoint
should and/or would use two. This is how the kernel knows it needs to create
two separate event nodes for the device. Application Collections have a few
reserved Usages that indicate to the host what type of device this is. These
are e.g. Mouse, Joystick, Consumer Control. If you ever
wondered why you have a device named like "Logitech G500s Laser Gaming Mouse
Consumer Control" this is the kernel simply appending the Application
Collection's Usage to the device name.

A Physical Collection indicates that the data is collected at one
physical point though what a point is is a bit blurry. Theoretical
physicists will disagree but a point can be "a mouse". So it's quite common
for all reports on a mouse to be wrapped in one Physical Collections. If you
have a device with two sets of sensors, you'd have two collections to
illustrate which ones go together. Physical Collections also have reserved
Usages like Pointer or Head Tracker.

Finally, a Logical Collection just indicates that some bits of data
belong together, whatever that means. The HID spec uses the example of
buffer length field and buffer data but it's also common for all inputs from
a mouse to be grouped together. A quick check of my mice here shows that
Logitech doesn't wrap the data into a Logical Collection but Microsoft's
firmware does. Because where would we be if we all did the same thing...

Anyway. Now that we know about collections, let's look at a whole report
descriptor as seen in the wild:

We have one Application Collection (Generic Desktop, Mouse) that contains
one Logical Collection (Generic Desktop, Mouse). That contains one Physical
Collection (Generic Desktop, Pointer). Our actual Report (and we have only
one but it has the decimal ID 26) has 5 buttons, two 16-bit axes (x and y)
and finally another 16 bit axis for the Wheel. This device will thus send
8-byte reports and our parser will do:

That's it. Now, obviously, you can't write a parser for every HID descriptor
out there so your actual parsing code needs to be generic. The Linux kernel
does
exactly that and so does everything else that needs to parse HID.
There's a huge variety in devices out there, all with HID descriptors that
may or may not be correct. As with so much in life, correct HID
implementations are often defined by "whatever Windows accepts" so if you
like playing catch, Linux development is for you.

Oh, in case you just got a bit too optimistic about the state of the world:
HID allows for vendor-defined usages. Which does exactly what you'd think it
does, it hides vendor-specific protocol inside what should be a generic
protocol. There are devices with hidden report IDs that you can only
unlock by sending the right magic sequence to the report and/or by defeating
the boss on Level 4. Usually those devices present themselves as
basic/normal devices over HID but if you know the magic sequence you get to
use *gasp* all buttons. Or access the device-specific configuration
features. Logitech's HID++ is just one example here but at least that's one
where we have most of the specs available.

The above describes how to parse the HID report descriptor and interpret the reports. But
what happens once you have a HID report correctly parsed?
In the case of the Linux kernel, once the report descriptor is parsed
evdev nodes are created (one per Application Collection, more or less). As
the Reports come in, they are mapped to evdev codes and the data appears on
the evdev node. That's where userspace like libinput can pick it up. That bit is actually quite simple (mostly anyway).

Monday, December 3, 2018

Last week while reviewing a patch I read that some gaming keyboards have two modes - keyboard mode and gaming mode. When in gaming mode, the keys send out pre-recorded macros when pressed. Presumably (I am not a gamer) this is to record keyboard shortcuts to have quicker access to various functionalities. The macros are stored in the hardware and are thus relatively independent of the host system. Pprovided you have access to the custom protocol, which you probably don't when you're on Linux. But I digress.

I reckoned this could be done in software and work with any 5 dollar USB keyboard. A few hours later, I have this working now: ggkbdd. It sits directly above the kernel and waits for key events. Once the 'mode key' is hit, the keyboard will send pre-configured key sequences for the respective keys. Hitting the mode key again (or ESC) switches back to normal mode.

There's a lot of functionality that is missing such as integration with the desktop (probably via DBus), better security (dropping privs, masking the fd to avoid accidental key logging), better system integration (request fds from logind, possibly through the compositor). And error handling, etc. I think the total time on this spent is somewhere between 3 and 4h, and that includes the time to write this blog post and debug the systemd unit autostartup. There are likely other projects that solve it the same way, or at least in a similar manner. I didn't check.

This was done as proof-of-concept and

I don't know if it's useful and if so, what the use-cases are

I don't know if I will have any time to fix things on this

I don't know if other (better developed) projects already occupy that space

In the grand glorious future and provided this is indeed something generally useful, this would need compositor integration. Not sure we'll ever get to that point. Meanwhile, consider this a code drop for a proof-of-concept and expect that you'll have to fix any bugs yourself.

Tuesday, September 4, 2018

libinput 1.12 was a massive development effort (over 300 patchsets) with a bunch of new features being merged. It'll be released next week or so, so it's worth taking a step back and looking at what actually changed.

The device quirks files replace the previously used hwdb-based udev properties. I've written about this in more detail here but the gist is: we have our own .ini style file format that can match on devices and apply the various quirks devices need. This simplifies debugging a lot, we can now reliably tell users why a quirks file applies or doesn't apply, historically a problem with the hwdb.

The sphinx-based documentation was merged, fixed and added to. We switched to sphinx for the docs and the result is much more user-friendly. Which was the point, it was a switch from a developer-oriented documentation to a user-oriented one. Not that documentation is ever finished.

The usual set of touchpad improvements went in, e.g. the slight motion on finger up is now ignored. We have size-based thumb detection now (useful for Apple touchpads!). And of course various quirks for better pressure ranges, etc. Tripletap on some synaptics touchpads had a tendency to cause multiple taps because of some weird event sequence. Movement in the software button now generates events, the buttons are not just a dead zone anymore. Pointer jump detection is more adaptive now and catches and discards smaller jumps that previously slipped through the cracks. A particularly quirky behaviour was seen on Dell XPS i2c touchpads that exhibit a huge pointer jump, courtesy of the trackpoint controller going to sleep and taking its time to wake up. The delay is still there but the pointer at least lands in the correct location.

We now have improved direction-locking for two-finger scrolling on touchpads. Scrolling up/down should not generate horizontal scroll events anymore as long as the movement is close enough to vertical. This feature is transparent, a diagonal or horizontal movement will immediately disable the direction lock and produce horizontal scroll events as expected.

The trackpoint acceleration has been re-done, see this post for more details and links to the background articles. I've only received one bug report for the new acceleration so it seems to work quite well now. Trackpoints that send events in bursts (e.g. bluetooth ones) are smoothened now to avoid jerky movement.

Velocity averaging was dropped to increase pointer accuracy. Previously we averaged the velocity across multiple events which makes the motion smoother on jittery devices but less accurate on good devices.

We build on FreeBSD now. Presumably this also means it works on FreeBSD :)

libinput now supports palm detection on touchscreens, at least where the ABS_MT_TOOL_TYPE evdev bit is provided.

Thursday, August 16, 2018

This is mostly a request for testing, because I've received zero feedback on the patches that I merged a month ago and libinput 1.12 is due to be out. No comments so far on the RC1 and RC2 either, so... well, maybe this gets a bit broader attention so we can address some things before the release. One can hope.

As the blog posts linked above explain, the trackpoint input data is difficult and largely arbitrary between different devices. The previous pointer acceleration libinput had relied on a fixed reporting rate which isn't true at low speeds, so the new acceleration method switches back to velocity-based acceleration. i.e. we convert the input deltas to a speed, then apply the acceleration curve on that. It's not speed, it's pressure, but it doesn't really matter unless you're a stickler for technicalities.

Because basically every trackpoint has different random data ranges not linked to anything easily measurable, libinput's device quirks now support a magic multiplier to scale the trackpoint range into something resembling a sane range. This is basically what we did before with the systemd POINTINGSTICK_CONST_ACCEL property except that we're handling this in libinput now (which is where acceleration is handled, so it kinda makes sense to move it here). There is no good conversion from the previous trackpoint range property to the new multiplier because the range didn't really have any relation to the physical input users expected.

So what does this mean for you? Test the libinput RCs or, better, libinput from master (because it's stable anyway), or from the Fedora COPR and check if the trackpoint works. If not, check the Trackpoint Configuration page and follow the instructions there.

Thursday, August 9, 2018

libinput made a design decision early on to use physical reference points wherever possible. So your virtual buttons are X mm high/across, the pointer movement is calculated in mm, etc. Unfortunately this exposed us to a large range of devices that don't bother to provide that information or just give us the wrong information to begin with. Patching the kernel for every device is not feasible so in 2015 the 60-evdev.hwdb was born and it has seen steady updates since. Plenty a libinput bug was fixed by just correcting the device's axis ranges or resolution. To take the magic out of the 60-evdev.hwdb, here's a blog post for your perusal, appreciation or, failing that, shaking a fist at. Note that the below is caller-agnostic, it doesn't matter what userspace stack you use to process your input events.

There are four parts that come together to fix devices: a kernel ioctl and a trifecta of udev rules hwdb entries and a udev builtin.

The kernel's EVIOCSABS ioctl

The three values that matter right now: minimum, maximum and resolution. The "value" is just the most recent value on this axis, ignore fuzz/flat for now. The min/max values simply specify the range of values the device will give you, the resolution how many values per mm you get. Simple example: an x axis given at min 0, max 1000 at a resolution of 10 means your devices is 100mm wide. There is no requirement for min to be 0, btw, and there's no clipping in the kernel so you may get values outside min/max. Anyway, your average touchpad looks like this in evemu-record:

This is the information returned by the EVIOCGABS ioctl (EVdev IOCtl Get ABS). It is usually run once on device init by any process handling evdev device nodes.

Because plenty of devices don't announce the correct ranges or resolution, the kernel provides the EVIOCSABS ioctl (EVdev IOCtl Set ABS). This allows overwriting the in-kernel struct with new values for min/max/fuzz/flat/resolution, processes that query the device later will get the updated ranges.

udev rules, hwdb and builtins

The kernel has no notification mechanism for updated axis ranges so the ioctl must be applied before any process opens the device. This effectively means it must be applied by a udev rule. udev rules are a bit limited in what they can do, so if we need to call an ioctl, we need to run a program. And while udev rules can do matching, the hwdb is easier to edit and maintain. So the pieces we have is: a hwdb that knows when to change (and the values), a udev program to apply the values and a udev rule to tie those two together.

In our case the rule is 60-evdev.rules. It checks the 60-evdev.hwdb for matching entries [1], then invokes the udev-builtin-keyboard if any matching entries are found. That builtin parses the udev properties assigned by the hwdb and converts them into EVIOCSABS ioctl calls. These three pieces need to agree on each other's formats - the udev rule and hwdb agree on the matches and the hwdb and the builtin agree on the property names and value format.

But since we want to match for specific use-cases, our udev rule assembles several specific match lines. Have a look at 60-evdev.rules again, the last rule in there assembles a string in the form of "evdev:name:the device name:content of /sys/class/dmi/id/modalias". So your hwdb entry could look like this:

evdev:name:My Touchpad Name:dmi:*svnDellInc*
EVDEV_ABS_00=0:1:3

If the name matches and you're on a Dell system, the device gets the EVDEV_ABS_00 property assigned. The "evdev:" prefix in the match line is merely to distinguish from other match rules to avoid false positives. It can be anything, libinput unsurprisingly used "libinput:" for its properties.

The last part now is understanding what EVDEV_ABS_00 means. It's a fixed string with the axis number as hex number - 0x00 is ABS_X. And the values afterwards are simply min, max, resolution, fuzz, flat, in that order. So the above example would set min/max to 0:1 and resolution to 3 (not very useful, I admit).

Trailing bits can be skipped altogether and bits that don't need overriding can be skipped as well provided the colons are in place. So the common use-case of overriding a touchpad's x/y resolution looks like this:

0x00 and 0x01 are ABS_X and ABS_Y, so we're setting those to 30 units/mm and 20 units/mm, respectively. And if the device is multitouch capable we also need to set ABS_MT_POSITION_X and ABS_MT_POSITION_Y to the same resolution values. The min/max ranges for all axes are left as-is.

The most confusing part is usually: the hwdb uses a binary database that needs updating whenever the hwdb entries change. A call to systemd-hwdb update does that job.

So with all the pieces in place, let's see what happens when the kernel tells udev about the device:

The udev rule assembles a match and calls out to the hwdb,

The hwdb applies udev properties where applicable and returns success,

The udev rule calls the udev keyboard-builtin

The keyboard builtin parses the EVDEV_ABS_xx properties and issues an EVIOCSABS ioctl for each axis,

The kernel updates the in-kernel description of the device accordingly

The udev rule finishes and udev sends out the "device added" notification

The userspace process sees the "device added" and opens the device which now has corrected values

Celebratory champagne corks are popping everywhere, hands are shaken, shoulders are patted in congratulations of another device saved from the tyranny of wrong axis ranges/resolutions

Once you understand how the various bits fit together it should be quite easy to understand what happens. Then the remainder is just adding hwdb entries where necessary but the touchpad-edge-detector tool is useful for figuring those out.

[1] Not technically correct, the udev rule merely calls the hwdb builtin which searches through all hwdb entries. It doesn't matter which file the entries are in.

Wednesday, August 1, 2018

To make testing libinput git master easier, I set up a whot/libinput-git Fedora COPR yesterday. This repo gets the push triggers directly from GitLab so it will rebuild with whatever is currently on git master.

To use the COPR, simply run:

sudo dnf copr enable whot/libinput-git
sudo dnf upgrade libinput

This will give you the libinput package from git. It'll have a date/time/git sha based NVR, e.g. libinput-1.11.901-201807310551git22faa97.fc28.x86_64. Easy to spot at least.

Disclaimer: This is an automated build so not every package is tested. I'm running git master exclusively (from a a ninja install) and I don't push to master unless the test suite succeeds. So the risk for ending up with a broken system is low.

On that note: if you are maintaining a similar repo for other distributions and would like me to add a push trigger in GitLab for automatic rebuilds, let me know.