Number of buttons

Physically separate buttons

For years this was the default type of touchpads: a touchpad with a
separate set of physical buttons below the touch surface. Such touchpads are
still around, but most newer models are Clickpads now.

Touchpads with physical buttons usually provide two buttons, left and right.
A few touchpads with three buttons exist, and Apple used to have touchpads
with a single physical buttons back in the PPC days. Touchpads with only two
buttons require the software stack to emulate a middle button. libinput does
this when both buttons are pressed simultaneously.

A two-button touchpad, with a two-button pointing stick above.

Note that many Lenovo laptops provide a pointing stick above the touchpad.
This pointing stick has a set of physical buttons just above the touchpad.
While many users use those as substitute touchpad buttons, they logically
belong to the pointing stick. The *40 and *50 series are an exception here,
the former had no physical buttons on the touchpad and required the top
section of the pad to emulate pointing stick buttons, the *50 series has
physical buttons but they are wired to the touchpads. The kernel re-routes
those buttons through the trackstick device.

Clickpads

Clickpads are the most common type of touchpads these days. A Clickpad
has no separate physical buttons, instead the touchpad itself is clickable
as a whole, i.e. a user presses down on the touch area and triggers a
physical click. Clickpads thus only provide a single button,
everything else needs to be software-emulated.

A clickpad on a Lenovo x220t. Just above the touchpad are the three buttons associated with the pointing stick. Faint markings on the bottom of the touchpad hint at where the software buttons should be.

Right and middle clicks are generated either via software
buttons or "clickfinger" behaviour. Software buttons define an area on the touchpad
that is a virtual right button. If a finger is in that area when the click
happens, the left button event is changed to a right button event. A middle click is
either a separate area or emulated when both the left and right virtual
buttons are pressed simultaneously.

When the software stack uses the clickfinger method, the number of fingers
decide the type of click: a one-finger is a left button, a two-finger click is a
right button, a three-finger click is a middle button. The location of the
fingers doesn't matter, though there are usually some limits in how the
fingers can be distributed (e.g. some implementations try to detect a thumb
at the bottom of the touchpad to avoid accidental two-finger clicks when the
user intends a thumb click).

The touchpad on a T440s has no physical buttons for the pointing stick. The marks on the top of the touchpad hint at the software button position for the pointing stick. Note that there are no markings at the bottom of the touchpad anymore.

Clickpads are labelled by the kernel with the INPUT_PROP_BUTTONPAD input property.

Forcepads

One step further down the touchpad evolution, Forcepads are Clickpads
without a physical button. They provide pressure and (at least in Apple's
case) have a vibration element that is software-controlled. Instead of the
satisfying click of a physical button, you instead get a buzz of happiness.
Which apparently feels the same as a click, judging by the reviews I've read
so far. A software-controlled click feel has some advantages, it
can be disabled for some gestures, modified for others, etc. I suspect that
over time Forcepads will become the main touchpad category, but that's a few
years away.

Not much to say on the implementation here. The kernel has some ForcePad
support but everything else is spotty.

Note how Apple's Clickpads have no markings whatsoever, Apple uses the clickfinger method by default.

Touch capabilities

Single-touch touchpads

In the beginning, there was the single-finger touchpad. This touchpad
would simply provide x/y coordinates for a single finger and get mightily
confused when more than one finger was present. These touchpads are now
fighting with dodos for exhibition space in museums, few of those are still
out in the wild.

Pure multi-touch touchpads

Pure multi-touch touchpads are those that can track, i.e. identify
the location of all fingers on the touchpad. Apple's touchpads support 16
touches (iirc), others support 5 touches like the Synaptics touchpads when
using SMBus.

Pure multi-touch touchpads are the easiest to support, we can rely on the
finger locations and use them for scrolling, gestures, etc. These touchpads
usually also provide extra information. In the case of the Apple touchpads
we get an ellipsis and the orientation of the ellipsis for each touch point.
Other touchpads provide a pressure value for each touch point. Though
pressure is a bit of a misnomer, pressure is usually directly related to
contact area.
Since our puny human fingers flatten out as the pressure on the
pad increases, the contact area increases and the firmware then calculates
that back into a (mostly rather arbitrary) pressure reading.

Because pressure is really contact area size, we can use it
to detect accidental palm contact or thumbs though it's fairly unreliable. A
light palm touch or a touch at the very edge of a touchpad will have a low
pressure reading simply because the palm is mostly next to the touchpad and
thus the contact area itself remains small.

Partial multi-touch touchpads

The vast majority of touchpads fall into this category. It's the half-way
point between single-touch and pure multi-touch. These devices can
track N fingers, but detect more than N. The current
Synaptics touchpads fall into that category when they're using the serial
protocol. Most touchpads that fall into this category can track two fingers
and detect up to four or five. So a typical three-finger interaction would
give you the location of two fingers and a separate value telling you that a
third finger is down.

The lack of finger location doesn't matter for some interactions (tapping,
three-finger click) but it can cause issues in some cases. For example, a
user may have a thumb resting on a touchpad while scrolling with two
fingers. Which touch locations you get depends on the order of the fingers
being set down, i.e. this may look like thumb + finger + third touch
somewhere (lucky!) or two fingers scrolling + third touch somewhere
(unlucky, this looks like a three-finger swipe).
So far we've mostly avoided having anything complex enough that requires the
exact location of more than two fingers, these pads are so prevalent that
any complex feature would exclude the majority of users.

Semi-mt touchpads

A sub-class of partial multi-touch touchpads. These touchpads can
technically detect two fingers but the location of both is limited to the
bounding box, i.e. the first touch is always the top-left one and the second
touch is the bottom-right one. Coordinates jump around as fingers move past
each other. Most semi-mt touchpads also have a lower resolution for two
touches than for one, so even things like two-finger scrolling can be very
jumpy.

Semi-mt are labelled by the kernel with the INPUT_PROP_SEMI_MT input property.

Physical properties

External touchpads

USB or Bluetooth touchpads not in a laptop chassis. Think the Apple Magic
Trackpad, the Logitech T650, etc. These are usually clickpads, the biggest
difference is that they can be removed or added at runtime. One interaction
method that is only possible on external touchpads is a thumb resting on the
very edge/immediately next to the touchpad. On the far edge, touchpads don't
always detect the finger location so clicking with a thumb barely touching
the edge makes it hard or impossible to figure out which software button
area the finger is on.

These touchpads also don't need palm detection - since they're not
located underneath the keyboard, accidental palm touches are a non-issue.

A Logitech T650 external touchpad. Note the thumb position, it is possible to click the touchpad without triggering a touch.

Circular touchpads

Yes, used to be a thing. Touchpad shaped in an ellipsis or circle.
Luckily for us they have gone full dodo. The X.Org synaptics driver had to
be aware of these touchpads to calculate the right distance for edge
scrolling - unsurprisingly an edge scroll motion on a circular touchpad
isn't very straight.

Graphics tablets

Touch-capable graphics tablets are effectively external touchpads, with two
differentiators: they are huge compared to normal touchpads and they have no
touchpad buttons whatsoever. This means they can either work like a
Forcepad, or rely on interaction methods that don't require buttons (like
tap-to-click). Since the physical device is shared with the pen input, some
touch arbitration is required to avoid touch input interfering when the pen
is in use.

Dedicated edge scroll area

Mostly on older touchpads before two-finger scrolling became the default method. These touchpads have a marking on the touch area that designates the edge to be used for scrolling. A finger movement in that edge zone should trigger vertical motions. Some touchpads have markers for a horizontal scroll area too at the bottom of the touchpad.

Thursday, July 16, 2015

In a perfect world, any device that advertises absolute x/y axes also advertises the resolution for those axes. Alas, not all of them do. For libinput, this problem is two-fold: parts of the touchscreen API provide data in mm - without knowing the resolution this is a guess at best. But it also matters for touchpads, where a lack of resolution is a lot more common (though the newest generations of major touchpad manufacturers tend to advertise resolutions now).

We have a number of features that rely on the touchpad resolution: from the size of the software button to deciding which type of palm detection we need, it all is calculated based on physical measurements. Until recently, we had code to differentiate between touchpads with resolution and most of the special handling was a matter of magic numbers, usually divided by the diagonal of the touchpad in device units. This made code maintenance more difficult - without testing each device, behaviour could not be guaranteed.

With libinput 0.20, we now got rid of this special handling and instead require the touchpads to advertise resolutions. This requires manual intervention, so we're trying to fix this in multiple places, depending on the confidence of the data. We have hwdb entries for the bcm5974 (Apple) touchpads and the Chromebook Pixel. For Elantech touchpads, a kernel patch is currently waiting for merging. For ALPS touchpads, we ship size hints with libinput's hwdb. If all that fails, we fall back to a default touchpad size of 69x55mm. [1]

All this affects users in two ways: one is that you may notice a slightly different behaviour of your touchpad now. The software-buttons may be bigger or smaller than before, pointer acceleration may be slightly different, etc. Shouldn't be too bad, but you may just notice it. The second noticeable change is that libinput will now log when it falls back to the default size. If you notice a message like that in your log, please file a bug and attach the output of evemu-describe and the physical dimensions of your touchpad. Once we have that information, we can add it to the right place and make sure that everyone else with that touchpad gets the right settings out of the box.

[1] The default size was chosen because it's close enough to what old touchpads used to be, and those are most likely to lack resolution values. This size may change over time as we get better data.

Wednesday, July 15, 2015

The libinput test suite takes somewhere around 35 minutes now for a full run. That's annoying, especially as I'm running it for every commit before pushing. I've tried optimising things, but attempts at making it parallel have mostly failed so far (almost all tests need a uinput device created) and too many tests rely on specific timeouts to check for behaviours. Containers aren't an option when you have to create uinput devices so I started out farming out into VMs.

Ideally, the test suite should run against multiple commits (on multiple VMs) at the same time while I'm working on some other branch and then accumulate the results. And that's where git notes come in. They're a bit odd to use and quite the opposite of what I expected. But in short: a git note is an object that can be associated with a commit, without changing the commit itself. Sort-of like a post-it note attached to the commit. But there are plenty of limitations, for example you can only have one note (per namespace) and merge conflicts are quite easy to trigger. Look at any git notes tutorial to find out more, there's plenty out there.

Anyway, dealing with merge conflicts is a no-go for me here. So after a bit of playing around, I found something that seems to work out well. A script to run make check and add notes to the commit, combined with a repository setup to fetch those notes and display them automatically. The core of the script is this:

Finally, in the main repository, I extended the glob that displays notes to 'everything':

$ git config notes.displayRef "*"

Now git log (and by extension tig) displays all notes attached to a commit automatically. All that's needed is a git fetch --all to fetch everything and it's clear in the logs which commit fails and which one succeeded.

Whenever I look at the log now, I immediately see which commits passed the test suite and which ones didn't (or haven't had it run yet). The only annoyance is that since a note is attached to a commit, amending the commit message or rebasing makes the note "go away". I've copied notes manually after this, but it'd be nice to find a solution to that.

Everything else has been working great so far, but it's quite new so there'll be a bit of polishing happening over the next few weeks. Any suggestions to improve this are welcome.