Number of buttons

Physically separate buttons

For years this was the default type of touchpads: a touchpad with a
separate set of physical buttons below the touch surface. Such touchpads are
still around, but most newer models are Clickpads now.

Touchpads with physical buttons usually provide two buttons, left and right.
A few touchpads with three buttons exist, and Apple used to have touchpads
with a single physical buttons back in the PPC days. Touchpads with only two
buttons require the software stack to emulate a middle button. libinput does
this when both buttons are pressed simultaneously.

A two-button touchpad, with a two-button pointing stick above.

Note that many Lenovo laptops provide a pointing stick above the touchpad.
This pointing stick has a set of physical buttons just above the touchpad.
While many users use those as substitute touchpad buttons, they logically
belong to the pointing stick. The *40 and *50 series are an exception here,
the former had no physical buttons on the touchpad and required the top
section of the pad to emulate pointing stick buttons, the *50 series has
physical buttons but they are wired to the touchpads. The kernel re-routes
those buttons through the trackstick device.

Clickpads

Clickpads are the most common type of touchpads these days. A Clickpad
has no separate physical buttons, instead the touchpad itself is clickable
as a whole, i.e. a user presses down on the touch area and triggers a
physical click. Clickpads thus only provide a single button,
everything else needs to be software-emulated.

A clickpad on a Lenovo x220t. Just above the touchpad are the three buttons associated with the pointing stick. Faint markings on the bottom of the touchpad hint at where the software buttons should be.

Right and middle clicks are generated either via software
buttons or "clickfinger" behaviour. Software buttons define an area on the touchpad
that is a virtual right button. If a finger is in that area when the click
happens, the left button event is changed to a right button event. A middle click is
either a separate area or emulated when both the left and right virtual
buttons are pressed simultaneously.

When the software stack uses the clickfinger method, the number of fingers
decide the type of click: a one-finger is a left button, a two-finger click is a
right button, a three-finger click is a middle button. The location of the
fingers doesn't matter, though there are usually some limits in how the
fingers can be distributed (e.g. some implementations try to detect a thumb
at the bottom of the touchpad to avoid accidental two-finger clicks when the
user intends a thumb click).

The touchpad on a T440s has no physical buttons for the pointing stick. The marks on the top of the touchpad hint at the software button position for the pointing stick. Note that there are no markings at the bottom of the touchpad anymore.

Clickpads are labelled by the kernel with the INPUT_PROP_BUTTONPAD input property.

Forcepads

One step further down the touchpad evolution, Forcepads are Clickpads
without a physical button. They provide pressure and (at least in Apple's
case) have a vibration element that is software-controlled. Instead of the
satisfying click of a physical button, you instead get a buzz of happiness.
Which apparently feels the same as a click, judging by the reviews I've read
so far. A software-controlled click feel has some advantages, it
can be disabled for some gestures, modified for others, etc. I suspect that
over time Forcepads will become the main touchpad category, but that's a few
years away.

Not much to say on the implementation here. The kernel has some ForcePad
support but everything else is spotty.

Note how Apple's Clickpads have no markings whatsoever, Apple uses the clickfinger method by default.

Touch capabilities

Single-touch touchpads

In the beginning, there was the single-finger touchpad. This touchpad
would simply provide x/y coordinates for a single finger and get mightily
confused when more than one finger was present. These touchpads are now
fighting with dodos for exhibition space in museums, few of those are still
out in the wild.

Pure multi-touch touchpads

Pure multi-touch touchpads are those that can track, i.e. identify
the location of all fingers on the touchpad. Apple's touchpads support 16
touches (iirc), others support 5 touches like the Synaptics touchpads when
using SMBus.

Pure multi-touch touchpads are the easiest to support, we can rely on the
finger locations and use them for scrolling, gestures, etc. These touchpads
usually also provide extra information. In the case of the Apple touchpads
we get an ellipsis and the orientation of the ellipsis for each touch point.
Other touchpads provide a pressure value for each touch point. Though
pressure is a bit of a misnomer, pressure is usually directly related to
contact area.
Since our puny human fingers flatten out as the pressure on the
pad increases, the contact area increases and the firmware then calculates
that back into a (mostly rather arbitrary) pressure reading.

Because pressure is really contact area size, we can use it
to detect accidental palm contact or thumbs though it's fairly unreliable. A
light palm touch or a touch at the very edge of a touchpad will have a low
pressure reading simply because the palm is mostly next to the touchpad and
thus the contact area itself remains small.

Partial multi-touch touchpads

The vast majority of touchpads fall into this category. It's the half-way
point between single-touch and pure multi-touch. These devices can
track N fingers, but detect more than N. The current
Synaptics touchpads fall into that category when they're using the serial
protocol. Most touchpads that fall into this category can track two fingers
and detect up to four or five. So a typical three-finger interaction would
give you the location of two fingers and a separate value telling you that a
third finger is down.

The lack of finger location doesn't matter for some interactions (tapping,
three-finger click) but it can cause issues in some cases. For example, a
user may have a thumb resting on a touchpad while scrolling with two
fingers. Which touch locations you get depends on the order of the fingers
being set down, i.e. this may look like thumb + finger + third touch
somewhere (lucky!) or two fingers scrolling + third touch somewhere
(unlucky, this looks like a three-finger swipe).
So far we've mostly avoided having anything complex enough that requires the
exact location of more than two fingers, these pads are so prevalent that
any complex feature would exclude the majority of users.

Semi-mt touchpads

A sub-class of partial multi-touch touchpads. These touchpads can
technically detect two fingers but the location of both is limited to the
bounding box, i.e. the first touch is always the top-left one and the second
touch is the bottom-right one. Coordinates jump around as fingers move past
each other. Most semi-mt touchpads also have a lower resolution for two
touches than for one, so even things like two-finger scrolling can be very
jumpy.

Semi-mt are labelled by the kernel with the INPUT_PROP_SEMI_MT input property.

Physical properties

External touchpads

USB or Bluetooth touchpads not in a laptop chassis. Think the Apple Magic
Trackpad, the Logitech T650, etc. These are usually clickpads, the biggest
difference is that they can be removed or added at runtime. One interaction
method that is only possible on external touchpads is a thumb resting on the
very edge/immediately next to the touchpad. On the far edge, touchpads don't
always detect the finger location so clicking with a thumb barely touching
the edge makes it hard or impossible to figure out which software button
area the finger is on.

These touchpads also don't need palm detection - since they're not
located underneath the keyboard, accidental palm touches are a non-issue.

A Logitech T650 external touchpad. Note the thumb position, it is possible to click the touchpad without triggering a touch.

Circular touchpads

Yes, used to be a thing. Touchpad shaped in an ellipsis or circle.
Luckily for us they have gone full dodo. The X.Org synaptics driver had to
be aware of these touchpads to calculate the right distance for edge
scrolling - unsurprisingly an edge scroll motion on a circular touchpad
isn't very straight.

Graphics tablets

Touch-capable graphics tablets are effectively external touchpads, with two
differentiators: they are huge compared to normal touchpads and they have no
touchpad buttons whatsoever. This means they can either work like a
Forcepad, or rely on interaction methods that don't require buttons (like
tap-to-click). Since the physical device is shared with the pen input, some
touch arbitration is required to avoid touch input interfering when the pen
is in use.

Dedicated edge scroll area

Mostly on older touchpads before two-finger scrolling became the default method. These touchpads have a marking on the touch area that designates the edge to be used for scrolling. A finger movement in that edge zone should trigger vertical motions. Some touchpads have markers for a horizontal scroll area too at the bottom of the touchpad.

Thursday, July 16, 2015

In a perfect world, any device that advertises absolute x/y axes also advertises the resolution for those axes. Alas, not all of them do. For libinput, this problem is two-fold: parts of the touchscreen API provide data in mm - without knowing the resolution this is a guess at best. But it also matters for touchpads, where a lack of resolution is a lot more common (though the newest generations of major touchpad manufacturers tend to advertise resolutions now).

We have a number of features that rely on the touchpad resolution: from the size of the software button to deciding which type of palm detection we need, it all is calculated based on physical measurements. Until recently, we had code to differentiate between touchpads with resolution and most of the special handling was a matter of magic numbers, usually divided by the diagonal of the touchpad in device units. This made code maintenance more difficult - without testing each device, behaviour could not be guaranteed.

With libinput 0.20, we now got rid of this special handling and instead require the touchpads to advertise resolutions. This requires manual intervention, so we're trying to fix this in multiple places, depending on the confidence of the data. We have hwdb entries for the bcm5974 (Apple) touchpads and the Chromebook Pixel. For Elantech touchpads, a kernel patch is currently waiting for merging. For ALPS touchpads, we ship size hints with libinput's hwdb. If all that fails, we fall back to a default touchpad size of 69x55mm. [1]

All this affects users in two ways: one is that you may notice a slightly different behaviour of your touchpad now. The software-buttons may be bigger or smaller than before, pointer acceleration may be slightly different, etc. Shouldn't be too bad, but you may just notice it. The second noticeable change is that libinput will now log when it falls back to the default size. If you notice a message like that in your log, please file a bug and attach the output of evemu-describe and the physical dimensions of your touchpad. Once we have that information, we can add it to the right place and make sure that everyone else with that touchpad gets the right settings out of the box.

[1] The default size was chosen because it's close enough to what old touchpads used to be, and those are most likely to lack resolution values. This size may change over time as we get better data.

Wednesday, July 15, 2015

The libinput test suite takes somewhere around 35 minutes now for a full run. That's annoying, especially as I'm running it for every commit before pushing. I've tried optimising things, but attempts at making it parallel have mostly failed so far (almost all tests need a uinput device created) and too many tests rely on specific timeouts to check for behaviours. Containers aren't an option when you have to create uinput devices so I started out farming out into VMs.

Ideally, the test suite should run against multiple commits (on multiple VMs) at the same time while I'm working on some other branch and then accumulate the results. And that's where git notes come in. They're a bit odd to use and quite the opposite of what I expected. But in short: a git note is an object that can be associated with a commit, without changing the commit itself. Sort-of like a post-it note attached to the commit. But there are plenty of limitations, for example you can only have one note (per namespace) and merge conflicts are quite easy to trigger. Look at any git notes tutorial to find out more, there's plenty out there.

Anyway, dealing with merge conflicts is a no-go for me here. So after a bit of playing around, I found something that seems to work out well. A script to run make check and add notes to the commit, combined with a repository setup to fetch those notes and display them automatically. The core of the script is this:

Finally, in the main repository, I extended the glob that displays notes to 'everything':

$ git config notes.displayRef "*"

Now git log (and by extension tig) displays all notes attached to a commit automatically. All that's needed is a git fetch --all to fetch everything and it's clear in the logs which commit fails and which one succeeded.

Whenever I look at the log now, I immediately see which commits passed the test suite and which ones didn't (or haven't had it run yet). The only annoyance is that since a note is attached to a commit, amending the commit message or rebasing makes the note "go away". I've copied notes manually after this, but it'd be nice to find a solution to that.

Everything else has been working great so far, but it's quite new so there'll be a bit of polishing happening over the next few weeks. Any suggestions to improve this are welcome.

Thursday, June 25, 2015

One of the bits we are currently finalising in libinput are touchpad gestures. Gestures on a normal touchscreens are left to the compositor and, in extension, to the client applications. Touchpad gestures are notably different though, they are bound to the location of the pointer or the keyboard focus (depending on the context) and they are less context-sensitive. Two fingers moving together on a touchscreen may be two windows being moved at the same time. On a touchpad however this is always a pinch.

Touchpad gestures are a lot more hardware-sensitive than touchscreens where we can just forward the touch points directly. On a touchpad we may have to consider software buttons or just HW-limitations of the touchpad. This prevents the implementation of touchpad gestures in a higher level - only libinput is aware of the location, size, etc. of software buttons.

Hence - touchpad gestures in libinput. The tree is currently sitting here and is being rebased as we go along, but we're expecting to merge this into master soon.

The interface itself is fairly simple: any device that may send gestures will have the LIBINPUT_DEVICE_CAP_GESTURE capability set. This is currently only implemented for touchpads but there is the potential to support this on other devices too. Two gestures are supported: swipe and pinch (+rotate). Both come with a finger count and both follow a Start/Update/End cycle. Gestures have a finger count that remains the same for the gestures, so if you switch from a two-finger pinch to a three-finger pinch you will see one gesture end and the next one start. Note that how to deal with this is up to the caller - it may very well consider this the same gesture semantically.

Swipe gestures have delta coordinates (horizontally and vertically) of the logical center of the gesture, compared to the previous event. A pinch gesture has the delta coordinates too and a delta angle (clockwise, in degrees). A pinch gesture also has the notion of an absolute scale, the Begin event always has a scale of 1.0 and that changes as the fingers move towards each other further apart. A scale of 2.0 means they're now twice as far apart as originally.

Nothing overly exciting really, it's a simple API that provides a couple of basic elements of data. Once integrated into the desktop properly, it should provide for some improved navigation. OS X has had this for a log time now and it's only time we caught up.

Friday, June 5, 2015

libinput provides a number of different out-of-the-box configurations, based on capabilities. For example: middle mouse button emulation is enabled by default if a device only has left and right buttons. On devices with a physical middle button it is available but disabled by default. Likewise, whether tapping is enabled and/or available depends on hardware capabilities. But some requirements cannot be gathered purely by looking at the hardware capabilities.

libinput uses a couple of udev properties, assigned through udev's hwdb, to detect device types. We use the same mechanism to provide us with specific tags to adjust libinput-internal behaviour. The udev properties named LIBINPUT_MODEL_.... tag devices based on a set of udev rules combined with hwdb matches. For example, we tag Chromebooks with LIBINPUT_MODEL_CHROMEBOOK.

Inside libinput, we parse those tags and use them for model-specific configuration. At the time of writing this, we use the chromebook tag to automatically enable clickfinger behaviour on those touchpads (which matches the google defaults on chromebooks). We tag the Lenovo X230 touchpad to give it it's own acceleration method. This touchpad is buggy and the data it sends has a very bad resolution.

In the future these tags will likely expand and encompass more devices that need customised tweaks. But the goal is always that libinput works well out of the box, even if the hardware is quirky. Introducing these tags instead of a sleigh of configuration options has short-term drawbacks: it increases the workload on us maintainers and it may require software updates to get a device to work exactly the way it should. The long-term benefits are maintainability and testability though, as well as us being more aware of what hardware is out there and how it needs to be fixed. Plus the relief of not having to deal with configuration snippets that are years out of date, do all the wrong things but still spread across forums like an STD.

Note: the tags are considered private API and may change at any time, depending what we want or need to do with them. Do not use them for home-made configuration.

Wednesday, June 3, 2015

libinput uses udev tags to determine what a device is. This is a significant difference to the X.Org stack which determines how to deal with a device based on an elaborate set of rules, rules grown over time, matured, but with a slight layer of mould on top by now. In evdev's case that is understandable, it stems from a design where you could just point it at a device in your xorg.conf and it'd automagically work, well before we had even input hotplugging in X. What it leads to now though is that the server uses slightly different rules to decide what a device is (to implement MatchIsTouchscreen for example) than evdev does. So you may have, in theory, a device that responds to MatchIsTouchscreen only to set itself up as keyboard.

libinput does away with this in two ways: it punts most of the decisions on what a device is to udev and its ID_INPUT_... properties. A device marked as ID_INPUT_KEYBOARD will initialize a keyboard interface, an ID_INPUT_TOUCHPAD device will initialize a touchpad backend. The obvious advantage of this is that we only have one place where we have generic device type rules. The second advantage is that where this one place isn't precise enough, it's simple to override with custom rule sets. For example, Wacom tablets are hard to categorise just by looking at the device alone. libwacom generates a udev rule containing the VID/PID of each known device with the right ID_INPUT_TABLET etc. properties.

This is a libinput-internal behaviour. Externally, we are a lot more vague. In fact, we don't tell you at all what a device is, other than what events it will send (pointer, keyboard, or touch). We have thought about implementing some sort of device identifier and the conclusion is that we won't implement this as part of libinput's API because it will simply be wrong some of the time. And if it's wrong, it requires the caller to re-implement something on top of it. At which point the caller may as well implement all of it instead. Why do we expect it to be wrong? Because libinput doesn't know the exact context that requires a device to be labelled as a specific type.

Take a keyboard for example. There are a great many devices that send key events. To the client a keyboard may be any device that can get an XKB layout and is used for typing. But to the compositor, a keyboard may be anything that can send a few specific keycodes. A device with nothing but KEY_POWER? That's enough for the compositor to decide to shut down but that device may not otherwise work as a keyboard. libinput can't know this context. But what libinput provides is the API to query information. libinput_device_pointer_has_button() and libinput_device_keyboard_has_key() are the two candidates here to query about a specific set of buttons and/or keys.

Touchpads, trackpoints and mice all look send pointer events and there is no flag that tells you the device type and that is intentional. libinput doesn't have any intrinsic knowledge about what is a touchpad, we take the ID_INPUT_TOUCHPAD tag. At best, we refuse some devices that were clearly mislabelled but we don't init devices as touchpads that aren't labelled as such. Any device type identification would likely be wrong - for example some Wacom tablets are touchpads internally but would be considered tablets in other contexts.

So in summary, callers are encouraged to rely on the udev information and other bits they can pull from the device to group it into the semantically correct device type. libinput_device_get_udev_device() provides a udev handle for a libinput device and all configurable features are query-able (e.g. "does this device support tapping?"). libinput will not provide a device type because it would likely be wrong in the current context anyway.

Tuesday, June 2, 2015

TLDR: as of libinput 0.16 you can end a touchpad tap-and-drag with a final additional tap

libinput originally only supported single-tap and double-tap. With version 0.15 we now support multi-tap, so you can tap repeatedly to get a triple, quadruple, etc. click. This is quite useful in text editors where a triple click highlights a line, four clicks highlight a paragraph, and 28 clicks order a new touchpad from ebay. Multi-tap also works with drag-and drop, so a triple tap followed by a finger down and hold will send three clicks followed by a single click.

We also support continuous tap-and-drag which is something synaptics didn't support provided with the LockedDrags option: Once the user is in dragging mode (x * tap + hold finger down) they can lift the finger and set it down again without the drag being interrupted. This is quite useful when you have to move across the screen, especially on smaller touchpads or for users that prefer a slow acceleration.

Of course, this adds a timeout to the drag release since we need to wait and see whether the finger comes down again. To help accelerate this, we added a tap-to-release feature (contributed by Velimir Lisec): once in drag mode a final tap will release the button immediately. This is something that OS X has supported for years and after a bit of muscle memory retraining it becomes second nature quickly. So the new timeout-free way to tap-and-drag on a touchpad is now: