Wednesday, December 16, 2015

This post only applies to users of the Lenovo x220 laptop experiencing issues when using the touchpad. Specifically, the touchpad is imprecise and "jumpy" after a firmware update, as outlined in Fedora bug 1264453. The cause is buggy touchpad firmware, identifiable by the string "fw: 8.1" in the dmesg output for the touchpad:

If you are experiencing these touchpad issues and your dmesg shows the 8.1 firmware version, please read on for a solution. By default, the x220 shipped with version 8.0 so unless you updated the firmware as part of a Lenovo update, you are not affected by this bug.

The touchpad issues seem identical as the ones seen on the Lenovo x230 model which has the same physical hardware and also ships with a firmware version 8.1. The root cause as seen by libinput is that the touchpad only sends events once the finger moves approximately 50 device units in either direction. The touchpad advertises a resolution of 65 units/mm horizontally and 136 units/mm vertically, but the effective resolution is reduced by roughly 75% and 30% This bugzilla attachment 1082925 shows the recording, you can easily see that while the pressure is upgraded with high granularity, the motion coordinates jump from one position to the next. From what we know this was introduced by the touchpad firmware v8.1, presumably as part of a filter to reduce the jitter some x230 users saw.

libinput automatically
detects the x230 and enables a custom acceleration function for just
that model. That same acceleration function works for the x220 v8.1, but
unfortunately we cannot automatically detect it. As of libinput
1.1.3, libinput recognises a special udev tag,
LIBINPUT_MODEL_LENOVO_X220_TOUCHPAD_FW81, to mark such an updated
x220 and enable a better pointer behaviour. To apply this tag, please do the
following:

Create a new file /etc/udev/hwdb.d/90-libinput-x220-fw8.1.hwdb

Look for X220 in the 90-libinput-model-quirks.hwdb file, copy the match and the property assignment into the file. As of the time of writing, the two lines are as below, but make sure you take the latest from your locally installed libinput version or the link above.

Verify the tag shows up with sudo udevadm test /sys/class/input/event4 (adjust the event node if necessary)

Reboot

The touchpad is now marked as requiring special treatment and libinput will
apply a different pointer acceleration for this touchpad.

Note that any udev property starting with LIBINPUT_MODEL_ is
private API and subject to change at any time. We will never break the
meaning of the LIBINPUT_MODEL_LENOVO_X220_TOUCHPAD_FW81 property, but
the exact behaviour of the property is implementation-dependent and may change
at any time. Do not use it for any other purpose than marking the touchpad
on a Lenovo x220 with an updated touchpad firmware version v8.1.

Wednesday, September 16, 2015

Many modern mice have the ability to store profiles, customize button mappings and actions and switch between several hardware resolutions. A number of those mice are targeted at gamers, but the features are increasingly common in standard mice. Under Linux, support for these device is spotty, though there are a few projects dedicated to supporting parts of the available device range. [1][2][3]

Benjamin Tissoires and I started a new project: libratbag. libratbag is a library to provide a generic interface to these mice,enabling desktop environments to provide configuration tools without having to worry about the device model. As of the time of this writing, we have partial support for the Logitech HID++ 1.0 (G500, G5) and HID++ 2.0 protocols (G303), the Etekcity Scroll Alpha and Roccat Kone XTD. Thomas H. P. Anderson already added the G5, G9 and the M705.

git clone http://github.com/libratbag/libratbag

The internal architecture is fairly simple, behind the library's API we have a couple of protocol-specific drivers that access the mouse. The drivers match a specific product/vendor ID combination and load the data from the device, the library then exports it to the caller as a struct ratbag_device. Each device has at least one profile, each profile has a number of buttons and at least one resolution. Where possible, the resolutions can be queried and set, the buttons likewise can be queried and set for different functions. If the hardware supports it, you can map buttons to other buttons, assign macros, or special functions such as DPI/profile switching. The main goal of libratbag is to unify access to the devices so a configuration application doesn't need different libraries per hardware. Especially short-term, we envision using some of the projects listed above through custom backends.

We're at version 0.1 at the moment, so the API is still subject to change. It looks like this:

libratbag is in a very early state of development. There are a bunch of FIXMEs in the code, the hardware support is still spotty and we'll appreciate any help we can get, especially with the hardware driver backends. There's a TODO in the repo for some things that we already know needs changing.
Feel free to browse the repo
on github and drop us some patches.

Eventually we want this to be
integrated into the desktop environments, either in the respective control
panels or in a standalone application. libratbag already provides SVGs for
some devices we support but we'll need some designer input for the actual
application. Again, any help you want to provide here will be much
appreciated.

Number of buttons

Physically separate buttons

For years this was the default type of touchpads: a touchpad with a
separate set of physical buttons below the touch surface. Such touchpads are
still around, but most newer models are Clickpads now.

Touchpads with physical buttons usually provide two buttons, left and right.
A few touchpads with three buttons exist, and Apple used to have touchpads
with a single physical buttons back in the PPC days. Touchpads with only two
buttons require the software stack to emulate a middle button. libinput does
this when both buttons are pressed simultaneously.

A two-button touchpad, with a two-button pointing stick above.

Note that many Lenovo laptops provide a pointing stick above the touchpad.
This pointing stick has a set of physical buttons just above the touchpad.
While many users use those as substitute touchpad buttons, they logically
belong to the pointing stick. The *40 and *50 series are an exception here,
the former had no physical buttons on the touchpad and required the top
section of the pad to emulate pointing stick buttons, the *50 series has
physical buttons but they are wired to the touchpads. The kernel re-routes
those buttons through the trackstick device.

Clickpads

Clickpads are the most common type of touchpads these days. A Clickpad
has no separate physical buttons, instead the touchpad itself is clickable
as a whole, i.e. a user presses down on the touch area and triggers a
physical click. Clickpads thus only provide a single button,
everything else needs to be software-emulated.

A clickpad on a Lenovo x220t. Just above the touchpad are the three buttons associated with the pointing stick. Faint markings on the bottom of the touchpad hint at where the software buttons should be.

Right and middle clicks are generated either via software
buttons or "clickfinger" behaviour. Software buttons define an area on the touchpad
that is a virtual right button. If a finger is in that area when the click
happens, the left button event is changed to a right button event. A middle click is
either a separate area or emulated when both the left and right virtual
buttons are pressed simultaneously.

When the software stack uses the clickfinger method, the number of fingers
decide the type of click: a one-finger is a left button, a two-finger click is a
right button, a three-finger click is a middle button. The location of the
fingers doesn't matter, though there are usually some limits in how the
fingers can be distributed (e.g. some implementations try to detect a thumb
at the bottom of the touchpad to avoid accidental two-finger clicks when the
user intends a thumb click).

The touchpad on a T440s has no physical buttons for the pointing stick. The marks on the top of the touchpad hint at the software button position for the pointing stick. Note that there are no markings at the bottom of the touchpad anymore.

Clickpads are labelled by the kernel with the INPUT_PROP_BUTTONPAD input property.

Forcepads

One step further down the touchpad evolution, Forcepads are Clickpads
without a physical button. They provide pressure and (at least in Apple's
case) have a vibration element that is software-controlled. Instead of the
satisfying click of a physical button, you instead get a buzz of happiness.
Which apparently feels the same as a click, judging by the reviews I've read
so far. A software-controlled click feel has some advantages, it
can be disabled for some gestures, modified for others, etc. I suspect that
over time Forcepads will become the main touchpad category, but that's a few
years away.

Not much to say on the implementation here. The kernel has some ForcePad
support but everything else is spotty.

Note how Apple's Clickpads have no markings whatsoever, Apple uses the clickfinger method by default.

Touch capabilities

Single-touch touchpads

In the beginning, there was the single-finger touchpad. This touchpad
would simply provide x/y coordinates for a single finger and get mightily
confused when more than one finger was present. These touchpads are now
fighting with dodos for exhibition space in museums, few of those are still
out in the wild.

Pure multi-touch touchpads

Pure multi-touch touchpads are those that can track, i.e. identify
the location of all fingers on the touchpad. Apple's touchpads support 16
touches (iirc), others support 5 touches like the Synaptics touchpads when
using SMBus.

Pure multi-touch touchpads are the easiest to support, we can rely on the
finger locations and use them for scrolling, gestures, etc. These touchpads
usually also provide extra information. In the case of the Apple touchpads
we get an ellipsis and the orientation of the ellipsis for each touch point.
Other touchpads provide a pressure value for each touch point. Though
pressure is a bit of a misnomer, pressure is usually directly related to
contact area.
Since our puny human fingers flatten out as the pressure on the
pad increases, the contact area increases and the firmware then calculates
that back into a (mostly rather arbitrary) pressure reading.

Because pressure is really contact area size, we can use it
to detect accidental palm contact or thumbs though it's fairly unreliable. A
light palm touch or a touch at the very edge of a touchpad will have a low
pressure reading simply because the palm is mostly next to the touchpad and
thus the contact area itself remains small.

Partial multi-touch touchpads

The vast majority of touchpads fall into this category. It's the half-way
point between single-touch and pure multi-touch. These devices can
track N fingers, but detect more than N. The current
Synaptics touchpads fall into that category when they're using the serial
protocol. Most touchpads that fall into this category can track two fingers
and detect up to four or five. So a typical three-finger interaction would
give you the location of two fingers and a separate value telling you that a
third finger is down.

The lack of finger location doesn't matter for some interactions (tapping,
three-finger click) but it can cause issues in some cases. For example, a
user may have a thumb resting on a touchpad while scrolling with two
fingers. Which touch locations you get depends on the order of the fingers
being set down, i.e. this may look like thumb + finger + third touch
somewhere (lucky!) or two fingers scrolling + third touch somewhere
(unlucky, this looks like a three-finger swipe).
So far we've mostly avoided having anything complex enough that requires the
exact location of more than two fingers, these pads are so prevalent that
any complex feature would exclude the majority of users.

Semi-mt touchpads

A sub-class of partial multi-touch touchpads. These touchpads can
technically detect two fingers but the location of both is limited to the
bounding box, i.e. the first touch is always the top-left one and the second
touch is the bottom-right one. Coordinates jump around as fingers move past
each other. Most semi-mt touchpads also have a lower resolution for two
touches than for one, so even things like two-finger scrolling can be very
jumpy.

Semi-mt are labelled by the kernel with the INPUT_PROP_SEMI_MT input property.

Physical properties

External touchpads

USB or Bluetooth touchpads not in a laptop chassis. Think the Apple Magic
Trackpad, the Logitech T650, etc. These are usually clickpads, the biggest
difference is that they can be removed or added at runtime. One interaction
method that is only possible on external touchpads is a thumb resting on the
very edge/immediately next to the touchpad. On the far edge, touchpads don't
always detect the finger location so clicking with a thumb barely touching
the edge makes it hard or impossible to figure out which software button
area the finger is on.

These touchpads also don't need palm detection - since they're not
located underneath the keyboard, accidental palm touches are a non-issue.

A Logitech T650 external touchpad. Note the thumb position, it is possible to click the touchpad without triggering a touch.

Circular touchpads

Yes, used to be a thing. Touchpad shaped in an ellipsis or circle.
Luckily for us they have gone full dodo. The X.Org synaptics driver had to
be aware of these touchpads to calculate the right distance for edge
scrolling - unsurprisingly an edge scroll motion on a circular touchpad
isn't very straight.

Graphics tablets

Touch-capable graphics tablets are effectively external touchpads, with two
differentiators: they are huge compared to normal touchpads and they have no
touchpad buttons whatsoever. This means they can either work like a
Forcepad, or rely on interaction methods that don't require buttons (like
tap-to-click). Since the physical device is shared with the pen input, some
touch arbitration is required to avoid touch input interfering when the pen
is in use.

Dedicated edge scroll area

Mostly on older touchpads before two-finger scrolling became the default method. These touchpads have a marking on the touch area that designates the edge to be used for scrolling. A finger movement in that edge zone should trigger vertical motions. Some touchpads have markers for a horizontal scroll area too at the bottom of the touchpad.

Thursday, July 16, 2015

In a perfect world, any device that advertises absolute x/y axes also advertises the resolution for those axes. Alas, not all of them do. For libinput, this problem is two-fold: parts of the touchscreen API provide data in mm - without knowing the resolution this is a guess at best. But it also matters for touchpads, where a lack of resolution is a lot more common (though the newest generations of major touchpad manufacturers tend to advertise resolutions now).

We have a number of features that rely on the touchpad resolution: from the size of the software button to deciding which type of palm detection we need, it all is calculated based on physical measurements. Until recently, we had code to differentiate between touchpads with resolution and most of the special handling was a matter of magic numbers, usually divided by the diagonal of the touchpad in device units. This made code maintenance more difficult - without testing each device, behaviour could not be guaranteed.

With libinput 0.20, we now got rid of this special handling and instead require the touchpads to advertise resolutions. This requires manual intervention, so we're trying to fix this in multiple places, depending on the confidence of the data. We have hwdb entries for the bcm5974 (Apple) touchpads and the Chromebook Pixel. For Elantech touchpads, a kernel patch is currently waiting for merging. For ALPS touchpads, we ship size hints with libinput's hwdb. If all that fails, we fall back to a default touchpad size of 69x55mm. [1]

All this affects users in two ways: one is that you may notice a slightly different behaviour of your touchpad now. The software-buttons may be bigger or smaller than before, pointer acceleration may be slightly different, etc. Shouldn't be too bad, but you may just notice it. The second noticeable change is that libinput will now log when it falls back to the default size. If you notice a message like that in your log, please file a bug and attach the output of evemu-describe and the physical dimensions of your touchpad. Once we have that information, we can add it to the right place and make sure that everyone else with that touchpad gets the right settings out of the box.

[1] The default size was chosen because it's close enough to what old touchpads used to be, and those are most likely to lack resolution values. This size may change over time as we get better data.

Wednesday, July 15, 2015

The libinput test suite takes somewhere around 35 minutes now for a full run. That's annoying, especially as I'm running it for every commit before pushing. I've tried optimising things, but attempts at making it parallel have mostly failed so far (almost all tests need a uinput device created) and too many tests rely on specific timeouts to check for behaviours. Containers aren't an option when you have to create uinput devices so I started out farming out into VMs.

Ideally, the test suite should run against multiple commits (on multiple VMs) at the same time while I'm working on some other branch and then accumulate the results. And that's where git notes come in. They're a bit odd to use and quite the opposite of what I expected. But in short: a git note is an object that can be associated with a commit, without changing the commit itself. Sort-of like a post-it note attached to the commit. But there are plenty of limitations, for example you can only have one note (per namespace) and merge conflicts are quite easy to trigger. Look at any git notes tutorial to find out more, there's plenty out there.

Anyway, dealing with merge conflicts is a no-go for me here. So after a bit of playing around, I found something that seems to work out well. A script to run make check and add notes to the commit, combined with a repository setup to fetch those notes and display them automatically. The core of the script is this:

Finally, in the main repository, I extended the glob that displays notes to 'everything':

$ git config notes.displayRef "*"

Now git log (and by extension tig) displays all notes attached to a commit automatically. All that's needed is a git fetch --all to fetch everything and it's clear in the logs which commit fails and which one succeeded.

Whenever I look at the log now, I immediately see which commits passed the test suite and which ones didn't (or haven't had it run yet). The only annoyance is that since a note is attached to a commit, amending the commit message or rebasing makes the note "go away". I've copied notes manually after this, but it'd be nice to find a solution to that.

Everything else has been working great so far, but it's quite new so there'll be a bit of polishing happening over the next few weeks. Any suggestions to improve this are welcome.

Thursday, June 25, 2015

One of the bits we are currently finalising in libinput are touchpad gestures. Gestures on a normal touchscreens are left to the compositor and, in extension, to the client applications. Touchpad gestures are notably different though, they are bound to the location of the pointer or the keyboard focus (depending on the context) and they are less context-sensitive. Two fingers moving together on a touchscreen may be two windows being moved at the same time. On a touchpad however this is always a pinch.

Touchpad gestures are a lot more hardware-sensitive than touchscreens where we can just forward the touch points directly. On a touchpad we may have to consider software buttons or just HW-limitations of the touchpad. This prevents the implementation of touchpad gestures in a higher level - only libinput is aware of the location, size, etc. of software buttons.

Hence - touchpad gestures in libinput. The tree is currently sitting here and is being rebased as we go along, but we're expecting to merge this into master soon.

The interface itself is fairly simple: any device that may send gestures will have the LIBINPUT_DEVICE_CAP_GESTURE capability set. This is currently only implemented for touchpads but there is the potential to support this on other devices too. Two gestures are supported: swipe and pinch (+rotate). Both come with a finger count and both follow a Start/Update/End cycle. Gestures have a finger count that remains the same for the gestures, so if you switch from a two-finger pinch to a three-finger pinch you will see one gesture end and the next one start. Note that how to deal with this is up to the caller - it may very well consider this the same gesture semantically.

Swipe gestures have delta coordinates (horizontally and vertically) of the logical center of the gesture, compared to the previous event. A pinch gesture has the delta coordinates too and a delta angle (clockwise, in degrees). A pinch gesture also has the notion of an absolute scale, the Begin event always has a scale of 1.0 and that changes as the fingers move towards each other further apart. A scale of 2.0 means they're now twice as far apart as originally.

Nothing overly exciting really, it's a simple API that provides a couple of basic elements of data. Once integrated into the desktop properly, it should provide for some improved navigation. OS X has had this for a log time now and it's only time we caught up.

Friday, June 5, 2015

libinput provides a number of different out-of-the-box configurations, based on capabilities. For example: middle mouse button emulation is enabled by default if a device only has left and right buttons. On devices with a physical middle button it is available but disabled by default. Likewise, whether tapping is enabled and/or available depends on hardware capabilities. But some requirements cannot be gathered purely by looking at the hardware capabilities.

libinput uses a couple of udev properties, assigned through udev's hwdb, to detect device types. We use the same mechanism to provide us with specific tags to adjust libinput-internal behaviour. The udev properties named LIBINPUT_MODEL_.... tag devices based on a set of udev rules combined with hwdb matches. For example, we tag Chromebooks with LIBINPUT_MODEL_CHROMEBOOK.

Inside libinput, we parse those tags and use them for model-specific configuration. At the time of writing this, we use the chromebook tag to automatically enable clickfinger behaviour on those touchpads (which matches the google defaults on chromebooks). We tag the Lenovo X230 touchpad to give it it's own acceleration method. This touchpad is buggy and the data it sends has a very bad resolution.

In the future these tags will likely expand and encompass more devices that need customised tweaks. But the goal is always that libinput works well out of the box, even if the hardware is quirky. Introducing these tags instead of a sleigh of configuration options has short-term drawbacks: it increases the workload on us maintainers and it may require software updates to get a device to work exactly the way it should. The long-term benefits are maintainability and testability though, as well as us being more aware of what hardware is out there and how it needs to be fixed. Plus the relief of not having to deal with configuration snippets that are years out of date, do all the wrong things but still spread across forums like an STD.

Note: the tags are considered private API and may change at any time, depending what we want or need to do with them. Do not use them for home-made configuration.

Wednesday, June 3, 2015

libinput uses udev tags to determine what a device is. This is a significant difference to the X.Org stack which determines how to deal with a device based on an elaborate set of rules, rules grown over time, matured, but with a slight layer of mould on top by now. In evdev's case that is understandable, it stems from a design where you could just point it at a device in your xorg.conf and it'd automagically work, well before we had even input hotplugging in X. What it leads to now though is that the server uses slightly different rules to decide what a device is (to implement MatchIsTouchscreen for example) than evdev does. So you may have, in theory, a device that responds to MatchIsTouchscreen only to set itself up as keyboard.

libinput does away with this in two ways: it punts most of the decisions on what a device is to udev and its ID_INPUT_... properties. A device marked as ID_INPUT_KEYBOARD will initialize a keyboard interface, an ID_INPUT_TOUCHPAD device will initialize a touchpad backend. The obvious advantage of this is that we only have one place where we have generic device type rules. The second advantage is that where this one place isn't precise enough, it's simple to override with custom rule sets. For example, Wacom tablets are hard to categorise just by looking at the device alone. libwacom generates a udev rule containing the VID/PID of each known device with the right ID_INPUT_TABLET etc. properties.

This is a libinput-internal behaviour. Externally, we are a lot more vague. In fact, we don't tell you at all what a device is, other than what events it will send (pointer, keyboard, or touch). We have thought about implementing some sort of device identifier and the conclusion is that we won't implement this as part of libinput's API because it will simply be wrong some of the time. And if it's wrong, it requires the caller to re-implement something on top of it. At which point the caller may as well implement all of it instead. Why do we expect it to be wrong? Because libinput doesn't know the exact context that requires a device to be labelled as a specific type.

Take a keyboard for example. There are a great many devices that send key events. To the client a keyboard may be any device that can get an XKB layout and is used for typing. But to the compositor, a keyboard may be anything that can send a few specific keycodes. A device with nothing but KEY_POWER? That's enough for the compositor to decide to shut down but that device may not otherwise work as a keyboard. libinput can't know this context. But what libinput provides is the API to query information. libinput_device_pointer_has_button() and libinput_device_keyboard_has_key() are the two candidates here to query about a specific set of buttons and/or keys.

Touchpads, trackpoints and mice all look send pointer events and there is no flag that tells you the device type and that is intentional. libinput doesn't have any intrinsic knowledge about what is a touchpad, we take the ID_INPUT_TOUCHPAD tag. At best, we refuse some devices that were clearly mislabelled but we don't init devices as touchpads that aren't labelled as such. Any device type identification would likely be wrong - for example some Wacom tablets are touchpads internally but would be considered tablets in other contexts.

So in summary, callers are encouraged to rely on the udev information and other bits they can pull from the device to group it into the semantically correct device type. libinput_device_get_udev_device() provides a udev handle for a libinput device and all configurable features are query-able (e.g. "does this device support tapping?"). libinput will not provide a device type because it would likely be wrong in the current context anyway.

Tuesday, June 2, 2015

TLDR: as of libinput 0.16 you can end a touchpad tap-and-drag with a final additional tap

libinput originally only supported single-tap and double-tap. With version 0.15 we now support multi-tap, so you can tap repeatedly to get a triple, quadruple, etc. click. This is quite useful in text editors where a triple click highlights a line, four clicks highlight a paragraph, and 28 clicks order a new touchpad from ebay. Multi-tap also works with drag-and drop, so a triple tap followed by a finger down and hold will send three clicks followed by a single click.

We also support continuous tap-and-drag which is something synaptics didn't support provided with the LockedDrags option: Once the user is in dragging mode (x * tap + hold finger down) they can lift the finger and set it down again without the drag being interrupted. This is quite useful when you have to move across the screen, especially on smaller touchpads or for users that prefer a slow acceleration.

Of course, this adds a timeout to the drag release since we need to wait and see whether the finger comes down again. To help accelerate this, we added a tap-to-release feature (contributed by Velimir Lisec): once in drag mode a final tap will release the button immediately. This is something that OS X has supported for years and after a bit of muscle memory retraining it becomes second nature quickly. So the new timeout-free way to tap-and-drag on a touchpad is now:

Friday, March 6, 2015

Update June 09, 2015: edge scrolling for clickpads has been merged.
Will be availble in libinput 0.20. Consider the rest of this post
obsolete.

libinput supports edge scrolling since version 0.7.0. Whoops, how does the post title go with this statement? Well, libinput supports edge scrolling, but only on some devices and chances are your touchpad won't be one of them. Bug 89381 is the reference bug here.

First, what is edge scrolling? As the libinput documentation illustrates, it is scrolling triggered by finger movement within specific regions of the touchpad - the left and bottom edges for vertical and horizontal scrolling, respectively. This is in contrast to two-finger scrolling, triggered by a two-finger movement, anywhere on the touchpad. synaptics had edge scrolling since at least 2002, the earliest commit in the repo. Back then we didn't have multitouch-capable touchpads, these days they're the default and you'd be struggling to find one that doesn't support at least two fingers. But back then edge-scrolling was the default, and touchpads even had the markings for those scroll edges painted on.

libinput adds a whole bunch of features to the touchpad driver, but those features make it hard to support edge scrolling. First, libinput has quite smart software button support. Those buttons are usually on the lowest ~10mm of the touchpad. Depending on finger movement and position libinput will send a right button click, movement will be ignored, etc. You can leave one finger in the button area while using another finger on the touchpad to move the pointer. You can press both left and right areas for a middle click. And so on. On many touchpads the vertical travel/physical resistance is enough to trigger a movement every time you click the button, just by your finger's logical center moving.

libinput also has multi-direction scroll support. Traditionally we only sent one scroll event for vertical/horizontal at a time, even going as far as locking the scroll direction. libinput changes this and only requires a initial threshold to start scrolling, after that the caller will get both horizontal and vertical scroll information. The reason is simple: it's context-dependent when horizontal scrolling should be used, so a global toggle to disable doesn't make sense. And libinput's scroll coordinates are much more fine-grained too, which is particularly useful for natural scrolling where you'd expect the content to move with your fingers.

Finally, libinput has smart palm detection. The large majority of palm touches are along the left and right edges of the touchpad and they're usually indistinguishable from finger presses (same pressure values for example). Without palm detection some laptops are unusable (e.g. the T440 series).

These features interfere heavily with edge scrolling. Software button areas are in the same region as the horizontal scroll area, palm presses are in the same region as the vertical edge scroll area. The lower vertical edge scroll zone overlaps with software buttons - and that's where you would put your finger if you'd want to quickly scroll up in a document (or down, for natural scrolling). To support edge scrolling on those touchpads, we'd need heuristics and timeouts to guess when something is a palm, a software button click, a scroll movement, the start of a scroll movement, etc. The heuristics are unreliable, the timeouts reduce responsiveness in the UI. So our decision was to only provide edge scrolling on touchpads where it is required, i.e. those that cannot support two-finger scrolling, those with physical buttons. All other touchpads provide only two-finger scrolling. And we are focusing on making 2 finger scrolling good enough that you don't need/want to use edge scrolling (pls file bugs for anything broken)

Now, before you get too agitated: if edge scrolling is that important to you, invest the time you would otherwise spend sharpening pitchforks, lighting torches and painting picket signs into developing a model that allows us to do reliable edge scrolling in light of all the above, without breaking software buttons, maintaining palm detection. We'd be happy to consider it.

This feature got merged for libinput 0.8 but I noticed I hadn't blogged about it. So belatedly, here is a short description of scroll sources in libinput.

Scrolling is a fairly simple concept. You move the mouse wheel and the content moves down. Beyond that the details get quite nitty, possibly even gritty. On touchpads, scrolling is emulated through a custom finger movement (e.g. two-finger scrolling). A mouse wheel moves in discrete steps of (usually) 15 degrees, a touchpad's finger movement is continuous (within the device physical resolution). Another scroll method is implemented for the pointing stick: holding the middle button down while moving the stick will generate scroll events. Like touchpad scroll events, these events are continuous. I'll ignore natural scrolling in this post because it just inverts the scroll direction.
Kinetic scrolling ("fling scrolling") is a comparatively recent feature: when you lift the finger, the final finger speed determines how long the software will keep emulating scroll events. In synaptics, this is done in the driver and causes all sorts of issues - the driver may keep sending scroll events even while you start typing.

In libinput, there is no kinetic scrolling at all, what we have instead are scroll sources. Currently three sources are defined, wheel, finger and continuous. Wheel is obvious, it provides the physical value in degrees (see this post) and in discrete steps. The "finger" source is more interesting, it is the hint provided by libinput that the scroll event is caused by a finger movement on the device. This means that a) there are no discrete steps and b) libinput guarantees a terminating scroll event when the finger is lifted off the device. This enables the caller to implement kinetic scrolling: simply wait for the terminating event and then calculate the most recent speed. More importantly, because the kinetic scrolling implementation is pushed to the caller (who will push it to the client when the Wayland protocol for this is ready), kinetic scrolling can be implemented on a per-widget basis.

Finally, the third source is "continuous". The only big difference to "finger" is that we can't guarantee that the terminating event is sent, simply because we don't know if it will happen. It depends on the implementation. For the caller this means: if you see a terminating scroll event you can use it as kinetic scroll information, otherwise just treat it normally.

For both the finger and the continuous sources the scroll distance provided by libinput is equivalent to "pixels", i.e. the value that the relative motion of the device would otherwise send. This means the caller can interpret this depending on current context too. Long-term, this should make scrolling a much more precise and pleasant experience than the old X approach of "You've scrolled down by one click".

Friday, February 6, 2015

I just pushed a patchset into libinput to introduce the concept of device groups. This post will explain what they are in this context and why they are needed.

libinput exposes kernel devices as an opaque struct libinput_device. It only recognises evdev devices at this point, this may change in the future if we see a need for it. libinput also exposes a few bits of information about the device such as the name, PID/VID and a handle to the struct udev_device that matches this device. The latter enables callers to get more information from the device. libinput also provides a bunch of configuration settings for each device. Pointer devices get acceleration settings, absolute devices have calibration, etc. For most devices this works just fine.

Some devices like Wacom tablets are represented as multiple event nodes. On a 3.19 kernel you'd get three event nodes for an Intuos 5 touch - the pad (i.e. the tablet itself), a touch node and one node for all the tools (stylus, eraser, etc. multiplexed). libinput exposes each of these nodes as separate device, but that is problematic when applying certain configuration settings. For example, applying a left-handed configuration to the tablet means it's rotated by 180 degrees so we need to rotate the coordinates accordingly. Of course, such a rotation would have to apply to both the touch and the stylus devices but now the caller is left with having to figure out which other devices to set.

The original idea was to present such devices as a single, merged struct libinput_device with multiple capabilities, i.e. a single physical device that can do touch, tablet and pad buttons. A configuration setting like left-handed-ness would then apply to all devices transparently. The API is clean, usage is simple, everybody is happy. Except when they aren't - this doesn't actually work particularly well. First, having such merged devices means we require devices to change at runtime, adding/removing capabilities on-the-fly which puts a burden on the callers to handle this correctly. Second, not all configuration options apply to all subdevices. If the Intuos is used as a touchpad you may want natural scrolling enabled on the touchpad but the wheel on the Wacom mouse should probably still work normally. Third, the subdevices may have different PID/VIDs and certainly have different udev devices. So now libinput needs a way to get to those. In short, a merged device looks nice in theory but the implementation of it would make the libinput API cumbersome to use for little benefit.

The solution to this are device groups: each device in libinput is now part of a struct libinput_device_group. This is just an opaque object that doesn't do anything but sit there but it's enough to identify how devices are grouped together. If two devices return the same device group, they logically belong together. The caller can then decide what to do with it, e.g. loop through all devices of a group to apply a certain configuration setting to all devices. The basic approach is thus:

The device groups' lifetime is as you'd expect: it is created for the first device in the group and ceases once the last device in a group is removed. It's not deleted until the last reference was deleted but it won't get recycled. In other words, if you keep unplugging and re-plugging that Intuos tablet, the device group will be new after every plug.

Note that we're intentionally not providing ways to get the devices from a device group, or counting the devices within a group, etc. This avoids race conditions (the view libinput has of the devices isn't the same as the caller has while going through the event queue) but it also makes the API simpler. libinput's callers are mainly compositors which use toolkits with advanced datastructures (glib, Qt, etc.). Using a pointer as key into a hashmap is simpler and less buggy than using whatever hand-crafted hashmap/list implementation we can provide through the libinput API.

Thursday, January 29, 2015

Lenovo released a new set of laptops for 2015 with a new (old) feature: the trackpoint device has the physical buttons back. Lastyear'sexperiment apparently didn't work out so well.

What do we do in Linux with the last generation's touchpads? The kernel marks them with INPUT_PROP_TOPBUTTONPAD based on the PNPID [1]. In the X.Org synaptics driver and libinput we take that property and emulate top software buttons based on it. That took us a while to get sorted and feed into the myriad of Linux distributions out there but at some point last year the delicate balance of nature was restored and the touchpad-related rage dropped to the usual background noise.

Slow-forward to 2015 and Lenovo introduces the new series. In the absence of unnecessary creativity they are called the X1 Carbon 3rd, T450, T550, X250, W550, L450, etc. Lenovo did away with the un(der)-appreciated top software buttons and re-introduced physical buttons for the trackpoint. Now, that's bound to make everyone happy again. However, as we learned from Agent Smith, happiness is not the default state of humans so Lenovo made sure the harvest is safe.

What we expected to happen was that the trackpoint device has BTN_LEFT, BTN_MIDDLE, BTN_RIGHT and the touchpad has BTN_LEFT and is marked with INPUT_PROP_BUTTONPAD (i.e. it is a Clickpad). That is the case on the x220 generation and the T440 generation. Though the latter doesn't actually have trackpoint buttons and we emulated them in software.

On the X1 Carbon 3rd, the trackpoint has BTN_LEFT, BTN_MIDDLE, BTN_RIGHT but they never send events. The touchpad has BTN_LEFT and BTN_0, BTN_1 and BTN_2 [2]. Clicking the left button on the trackpoint generates BTN_0 on the touchpad device, clicking the right button generates BTN_1 on the touchpad device. So in short, Lenovo has decided to wire the newly re-introduced trackpoint buttons to the touchpad, not the trackpoint. [3] The middle button is currently dead, which is a kernel bug. Meanwhile we think of it as security feature - never accidentally paste your password into your IRC session again!

What does this mean for us? Neither synaptics nor evdev nor libinput currently support this so we've been busy apidae and writing patches like crazy.
[Update 19/03/15]: The patches queued in Dmitry's for-linus branch re-route the trackstick buttons in the kernel through the trackstick device. To userspace, the trackstick thus looks the same as in the past and no userspace patches are needed. I've reverted the synaptics patch already, the libinput patch will follow.
The patch goes into the kernel and udev.... The two patches needed go into the kernel and udev, and libinput. No, the three patches needed go into the kernel, udev and libinput, and synaptics. The four patches, no, wait. Amongst the projects needing patches are the kernel, udev, libinput and synaptics. I'll try again:

The kernel needs this patchset here. It removes INPUT_PROP_TOPBUTTONPAD from the device, makes the middle button work and fixes the button behaviour.

synaptics simply mangles the buttons [4] and sends them as left/right/middle buttons accordingly.

With those put together, things pretty much work as they're supposed to. libinput handles middle button scrolling as well this way but synaptics won't, much for the same reason it didn't work in the previous generation: synaptics can't talk to evdev and vice versa. And given that synaptics is on life support or in pallative care, depending how you look at it, I recommend not holding your breath for a fix. Otherwise you may join it quickly.

Note that all the patches are fresh off the presses and there may be a few bits changing before they are done. If you absolutely can't live without the trackpoint's buttons you can work around it by disabling the synaptics kernel driver until the patches have trickled down to your distribution.

The tracking bug for all this is Bug 88609. Feel free to CC yourself on it. Bring popcorn.

Final note: I haven't seen logs from the T450, T550, ... devices yet yet so this is so far only confirmed on the X1 Carbon so far. Given the hardware is essentially identical I expect it to be true for the rest of the series though.

[1] We also apply quirks for the 2013 generation because the firmware was buggy - a problem Synaptics Inc. has since fixed (but currently gives us slight headaches).
[2] It is also marked with INPUT_PROP_TOPBUTTONPAD which is a bug. It uses a new PNPID but one that was in the range we previously believed was for pads without trackpoint buttons. That's an an easy thing to fix.
[3] The reason for that seems to be HW design: this way they can keep the same case/keyboard and just swap the touchpad bits.
[4] synaptics is old enough to support dedicated scroll buttons. Buttons that used to send BTN_0 and BTN_1 and are thus interpreted as scroll up/down event.

Wednesday, January 21, 2015

It seems we can't ever get rid of the issueswiththis series. Daniel Martin filed a kernel bug for the latest series of these devices (Oct 2014) and it looks like they all need manual fixing again.

When the *40 series first came out, the PS/2 firmware was buggy and advertised bogus coordinate ranges for the x/y axes. Since we use those coordinate ranges to set up the size and position of software buttons (very much needed since that series did away with the physical trackpoint buttons) we added kernel patches for each of those laptops. The kernel would match on the PNPID (e.g. LEN0036 on a T440) and fix the min/max range for both axes based actual measurements. Since this requires someone to have a laptop, measure it, file a bug or send a patch, wait for it to get into the kernel, wait for it to get into distros it took quite a while to get all models supported.

Lenovo has updated the series in Oct 2014 and it's starting to get in the hands of users. And as it turns out, the touchpads now have different coordinate ranges but the same PNPID. And the values reported by the firmware are still bogus, so we need the same quirk, again, for each touchpad. Update 22/01/15: looks like the ranges are correct enough, so we don't need to update all ranges separately.

So in short: if you have one of the latest series *40 touchpads, your touchpad software buttons will be off. CC yourself on the kernel bug and if you have a model that's not listed there yet, add the required data. Eventually that will end up in the kernel and then everything is hunky-dory again. Until then, have a drink on behalf of the Synaptics/Lenovo QA departments.

Now the obvious question: why does this work with Windows? They don't use the PS/2 protocol but the SMBus/RMI4 interface and thus PS/2 firmware correctness is apparently not top priority for the QA departments. But the SMBus protocol requires the Host Notify feature, which caused Synaptics to reimplement the i2c driver for Windows. And that's what is shipped/preinstalled as driver. We don't support Host Notify on Linux, so there goes that idea. But there's strong suspicion that's not the only piece of the puzzle that's missing anyway...

Update 22/01/15: The min/max ranges advertised seem to be correct in the newer versions which would indicate that Synaptics has fixed the firmware. That's great (except for re-using the PNPID). Now we need to just detect that and drop the quirks for the newer touchpads. Hans has a good suggestion for how to do this, so with a bit of luck this will end up being only one kernel patch instead of one per device.

Monday, January 19, 2015

A Fedora 22 feature is to use the libinput X.Org driver as default driver for all non-tablet devices.
This replaces the current defaults of synaptics for touchpads and evdev for anything else (tablets usually use the wacom driver, if installed). As expected, changing a default has some repercussions both for users and for developers. These are outlined below, based on the libinput 0.8 release. Future versions may add features, so check with your latest local version.
Generally, the behaviour should roughly stay the same, big changes such as devices not being detected or not working is most likely a bug. Some behaviours are new, e.g. always-on palm detection, top software buttons on specific touchpads, etc. If in doubt, check the libinput documentation for hints on whether something is supposed to work in a particular manner.

Changes visible to Users

Any custom xorg.conf snippets will cease to work, if they are properly stacked. Options set by snippets are almost always exclusive to one particular driver. When the default driver changes, the snippet may not apply to the device anymore. Whether they stop working depends whether the Driver line is present. Consider this example snippet:

This snippet does two things: it assigns the synaptics driver to the "my mouse" device and sets the option TapButton1. The assignment will override the default libinput assignment, i.e. this device won't change behaviour, you just don't get to use any new features. If the Driver line is not present then this snippet won't do anything, the libinput driver does not provide a TapButton1 option. It is safe to leave the snippet in place, options that are not supported by a driver are simply ignored.

The xf86-input-libinput man page has a list of options that can be set. For example, the above snippet would have an equivalent as

Note that this matches on a driver rather than assign the driver. Since options are driver-specific this is the correct approach.

The other visible change is a difference in default pointer speed. We have fine-tuning pointer acceleration on our TODO lists, but we're not quite there yet and any help would be appreciated. In the meantime you may see changes in pointer acceleration.

Finally, you may see certain features have gone the way of the dodo. Specifically the touchpad code exposes a lot less knobs to tweak. Most of that is intentional, some of it may warrant discussion. If there is a particular feature you are missing or doesn't work as you want to, please file a bug.

Changes visible to developers

These changes affect desktop environments, specifically the part that configures input devices. The changes affect three categories: pointer acceleration, button mapping, touchpad disabling and device properties. The property "libinput Send Events Modes Available" exists on all devices, it can be used to determine if a device is handled by the libinput driver.

Pointer acceleration

The X server exposes a variety of knobs for its pointer acceleration code. The oldest knob (and specified in the core protocol) is the XChangePointerControl request. In some environments this is exposed as a single slider, in others it's split into multiple settings (Acceleration and Threshold, for example).

libinput does away with this and only exposes a single 1-value float property "libinput Accel Speed" with a range of -1 (slowest) to 1 (fastest). The XChangePointerControl request has no effect on a libinput device. It is up to you how to map the current speed mappings into the [-1, 1] range.

Button mapping

The X server provides button mapping through the XSetPointerMapping request. This is most commonly used to apply a left-handed configuration and to enable natural scrolling. The call will continue to work with the libinput driver, but better methods are available.

The property "libinput Left Handed Enabled" takes a single boolean 8-bit value to enable and disable left-handed mode. Unlike the X request this will automatically take care of the tapping configuration (and other things in the future). If the property is not available on a device, that device has no left-handed mode.

The property "libinput Natural Scrolling Enabled" takes a single boolean 8-bit value to enable and disable natural scrolling. This applies to smooth scrolling and legacy button scrolling (which the libinput driver doesn't do anyway). If the property is not available on a device, that device has no natural scrolling mode.

Touchpad disabling

In the synaptics driver, disabling the touchpad is usually done with the "Synaptics Off" property. This is used by syndaemon to turn the touchpad off while typing. libinput does this by default, so it is safe to simply ignore this at all. Don't bother starting syndaemon, it won't control the libinput driver.

Device properties

Any code that handles a driver-specific property (prefixed by "evdev" or "synaptics") will stop working. These properties are not exposed by the libinput driver (we tried, it was not viable). KDE's kcm_touchpad module is a particularly bad offender here, it exposes almost every toggle the driver ever had. Make sure the code behaves well if the properties are not present and you're basically good to go.

If you decide to handle libinput-specific properties, the general rule is: if a single-value property is not present, that configuration does not apply to this device. Bitmask-style properties are split into an "libinput foo Available" and "libinput foo Enabled". The former lists the available settings, the latter enables a specific setting. Have a look at the xf86-input-libinput source for details on each property.

Friday, January 16, 2015

libinput 0.8 was released yesterday. One feature I'd like to talk about here: the change to provide mouse wheel events as physical distances.

Mouse wheels are clicks. In the evdev protocol they're sent via the REL_WHEEL and REL_HWHEEL axes, with a value of 1 per click. Spinning the wheel fast enough will give you a higher value per event but it's still just a multiple of the physical clicks. This is a conundrum for libinput.

libinput exports scroll events as "axis" events, the value returned by libinput_event_pointer_get_axis_value() for touchpads and button scrolling is in "pixels". libinput doesn't have a concept of pixels of course but the compositor will likely take the relative motion and apply it to the cursor. Scroll events are in the same coordinate space, i.e. the scrolling for two-finger scrolling has the same feel as moving the pointer. This continuous coordinate space is at odds with the discrete values coming from a wheel. We added axis sources to the API so you can now tell whether an event was generated by the wheel or some other scroll methods. But still, the discrete values from a wheel are at odds with the other sources.

For libinput 0.8, we changed the default reporting mode of the wheel. For the click count, a new call libinput_event_pointer_get_axis_value_discrete() provides that number (positive and negative for both direction). The libinput_event_pointer_get_axis_value() on a wheel now returns the movement of the wheel in degrees. That gives us a continuous coordinate space that also opens up new possibilities: if you know the rotation of a mouse wheel in degrees, you know things like "has the wheel been turned a full rotation". I don't quite know how, but I'm sure there are interesting interfaces you can make from that :)

Of course, the physical properties don't change, the degrees are always a multiple of the click count, and on most mice one click count is a 15 degree movement. The Logitech M325 is a notable exception here with a 20 degree angle. This isn't advertised by the hardware of course so we rely on the udev hwdb to set it where it differs from the default. The patch for this has been pushed to systemd and will soon arrive at a distribution near you.

And to answer a question I'm sure will come up: those mice that support a free spinning scrollwheel don't change the reporting mode. The G500s for example merely moves a physical bit that provides friction and the click feel/noise. The device still reports in 15 degree angle counts. I strongly suspect that other mice with this feature are the same (if not, we can now work this continuous motion into libinput and handle it propertly).