In progress: This Work in progress. documents one or more features whose implementation are in progress.

TODO

Check for correct English

Intro

In this document we describe the algorithms we use to make
the touchscreen behave well. By "well" we mean that we can:

Get reliable clicks with the stylus and with the finger

Avoid further filtering in user space. It might be useful in some scenarios, for instance in a based dual boot menu.

Do linear transformation in kernel-space. X (and other programs) should be able to gather data from /dev/input/eventX. This step is optional and you can use tslib for this task if you prefer to do so.

Openmoko developers and contributors have tried different approaches
that have helped improve the touchscreen performance. As a result now
we have a touchscreen filtering framework that we are using in Linux
2.6. We describe the current working solution as of December 2008 as
it is in the andy-tracking branch of the Openmoko GIT repository.

We don't use pressure information

The only information we are using from the hardware is the reported X and
Y coordinates. We are not using pressure information. We could get
the area of the contact to the touchscreen and we could tell between stylus,
fingernail or thumb. The area information is reliable but we didn't manage
to normalize the results across the screen as they varied widely depending
on which corner of the screen is pressed, in a non-linear way.

Why are we doing filtering in kernel space?

In the latest driver we decided to add a generic filtering framework
that could be used by more touchscreen drivers.

We are aware of tslib and we use it with
the current stable kernel (2.6.24) but we also think that doing
filtering in the kernel is a good idea, and in the latest kernel
(2.6.28 - next stable) we are doing it and as a side effect we no
longer need to depend on tslib (but we can use it if we need it).

Let's say we would like to deliver a TS event to user space each
10 milliseconds. In the GTA02 with the current configuration the
Analog/Digital conversion time of a sample is 0.4697 milliseconds,
thus can get 18 samples for each event that we send to user-space.
Sending the event (with 18 samples) takes us about 8.45
milliseconds. Sometimes we even decide that the event should not be
sent to user-space (because the hardware didn't provide reliable data),
and our tests show that it's the right thing to do.
In previous versions of the driver light taps would confuse the driver
(that would report bad clicks) and this is no longer an issue.

Briefly stated: We think it's better if the driver sends only
good data to the applications. If there are multiple distributions
for a device (there are quite a few for the GTA02) they can share
a tuned in-kernel configuration. We also save a lot of kernel mode/user
mode transitions and data transmission
allowing us to consider more samples for each reported event.

clear: We call this operation when we need the filter to be in a clean state.
For instance, when we initialize the filter or when we have a pen-up event.

process: This function is called when we need to send a
sample to the filter, in our driver this happens when an A/D
conversion ends. After that one of three things
can happen:

If the filter needs more samples to be gathered it returns a 0.

If it is ready to return a sample and there are no filters after itself, it returns 1.

If it is ready to return a sample and there are filters after itself it will pass the resulting sample to the next filter and it's up to that filter to decide what it returns. If a filter reaches an error it returns -1. In the GTA01/GTA02 this happens when many inputs are discarded because they are not good samples and we decide to give up trying to get good samples.

scale: This function is called for each filter after the filter chain returns 1.
It is used to save expensive operations (such as divisions) in those cases when
you can get away with doing them with aggregated data.

Group filter

This filter is useful to reject samples that are not reliable. We consider
that a sample is not reliable if it deviates form the Majority.

(1) First we collect a set of S samples. (2) We make a copy of the values
of each coordinate and process each dimension separately. For each
dimension (X,Y):

Sort the points

Points that are "close enough" are considered to be in the same set.

Choose the set with more elements. If more than "threshold" points are in this set we use the first and the last point of the set to define the valid range for this dimension [min, max], otherwise we discard all the points and go to step 1.

We consider the unsorted S samples and try to feed them to the next filter in the chain in the same order they arrived. If one of the points of each sample is not in the allowed range for its dimension, we discard the sample.

Median averaging filter

We sort incoming raw samples into an array of MEDIAN_SIZE length, discarding the
oldest sample each time once we are full. We then return the sum of the middle three
samples for X and Y.

When this filter is cleared (via the clear function) it consumes MEDIAN_SIZE points before the first output takes place.
After that it consumes a few samples for each output and this number of samples depends on how fast the pointer is moving.
The idea is to discard more samples from the array if the pointer is moving fast.

This strongly rejects brief excursions away from a central point that is
sticky in time compared to the excursion duration.

Mean filter

Mean works well if the input data is already good quality, reducing + / - 1
sample jitter when the stylus is still, or moving very slowly, without
introducing abrupt transitions or reducing ability to follow larger
movements. If you set the threshold higher than the dynamic range of the
coordinates, you can just use it as a simple mean average.

This filter has no effect if the samples are changing by more that the
threshold set by averaging_threshold in the configuration.
However while samples come in that don't go outside this threshold from
the last reported sample, Mean replaces the samples with a simple mean
of a configurable number of samples (set by bits_filter_length in config,
which is 2^n, so 5 there makes 32 sample averaging).

In the current configuration we are using a mean with 4 points. It means that
when the filter is cleared we need 4 samples for each output and then we
only use 1 sample for each output until the filter is cleared.

Linear calibration filter

This filter allow us to scale touchscreen values using calibration data. It also exposes the calibration constants using sysfs.
By default it does not scale the coordinates.

How things fit together

The following configuration that we are using for the GTA02 lives in the file arch/arm/mach-s3c2440/mach-gta02.c. This configuration is not in the touchscreen driver, and the same driver can be used with different settings in other devices. The filters can also be used in other drivers.

In progress: This Work in progress. documents one or more features whose implementation are in progress.

TODO

Check for correct English

Intro

In this document we describe the algorithms we use to make
the touchscreen behave well. By "well" we mean that we can:

Get reliable clicks with the stylus and with the finger

Avoid further filtering in user space. It might be useful in some scenarios, for instance in a based dual boot menu.

Do linear transformation in kernel-space. X (and other programs) should be able to gather data from /dev/input/eventX. This step is optional and you can use tslib for this task if you prefer to do so.

Openmoko developers and contributors have tried different approaches
that have helped improve the touchscreen performance. As a result now
we have a touchscreen filtering framework that we are using in Linux
2.6. We describe the current working solution as of December 2008 as
it is in the andy-tracking branch of the Openmoko GIT repository.

We don't use pressure information

The only information we are using from the hardware is the reported X and
Y coordinates. We are not using pressure information. We could get
the area of the contact to the touchscreen and we could tell between stylus,
fingernail or thumb. The area information is reliable but we didn't manage
to normalize the results across the screen as they varied widely depending
on which corner of the screen is pressed, in a non-linear way.

Why are we doing filtering in kernel space?

In the latest driver we decided to add a generic filtering framework
that could be used by more touchscreen drivers.

We are aware of tslib and we use it with
the current stable kernel (2.6.24) but we also think that doing
filtering in the kernel is a good idea, and in the latest kernel
(2.6.28 - next stable) we are doing it and as a side effect we no
longer need to depend on tslib (but we can use it if we need it).

Let's say we would like to deliver a TS event to user space each
10 milliseconds. In the GTA02 with the current configuration the
Analog/Digital conversion time of a sample is 0.4697 milliseconds,
thus can get 18 samples for each event that we send to user-space.
Sending the event (with 18 samples) takes us about 8.45
milliseconds. Sometimes we even decide that the event should not be
sent to user-space (because the hardware didn't provide reliable data),
and our tests show that it's the right thing to do.
In previous versions of the driver light taps would confuse the driver
(that would report bad clicks) and this is no longer an issue.

Briefly stated: We think it's better if the driver sends only
good data to the applications. If there are multiple distributions
for a device (there are quite a few for the GTA02) they can share
a tuned in-kernel configuration. We also save a lot of kernel mode/user
mode transitions and data transmission
allowing us to consider more samples for each reported event.

clear: We call this operation when we need the filter to be in a clean state.
For instance, when we initialize the filter or when we have a pen-up event.

process: This function is called when we need to send a
sample to the filter, in our driver this happens when an A/D
conversion ends. After that one of three things
can happen:

If the filter needs more samples to be gathered it returns a 0.

If it is ready to return a sample and there are no filters after itself, it returns 1.

If it is ready to return a sample and there are filters after itself it will pass the resulting sample to the next filter and it's up to that filter to decide what it returns. If a filter reaches an error it returns -1. In the GTA01/GTA02 this happens when many inputs are discarded because they are not good samples and we decide to give up trying to get good samples.

scale: This function is called for each filter after the filter chain returns 1.
It is used to save expensive operations (such as divisions) in those cases when
you can get away with doing them with aggregated data.

Group filter

This filter is useful to reject samples that are not reliable. We consider
that a sample is not reliable if it deviates form the Majority.

(1) First we collect a set of S samples. (2) We make a copy of the values
of each coordinate and process each dimension separately. For each
dimension (X,Y):

Sort the points

Points that are "close enough" are considered to be in the same set.

Choose the set with more elements. If more than "threshold" points are in this set we use the first and the last point of the set to define the valid range for this dimension [min, max], otherwise we discard all the points and go to step 1.

We consider the unsorted S samples and try to feed them to the next filter in the chain in the same order they arrived. If one of the points of each sample is not in the allowed range for its dimension, we discard the sample.

Median averaging filter

We sort incoming raw samples into an array of MEDIAN_SIZE length, discarding the
oldest sample each time once we are full. We then return the sum of the middle three
samples for X and Y.

When this filter is cleared (via the clear function) it consumes MEDIAN_SIZE points before the first output takes place.
After that it consumes a few samples for each output and this number of samples depends on how fast the pointer is moving.
The idea is to discard more samples from the array if the pointer is moving fast.

This strongly rejects brief excursions away from a central point that is
sticky in time compared to the excursion duration.

Mean filter

Mean works well if the input data is already good quality, reducing + / - 1
sample jitter when the stylus is still, or moving very slowly, without
introducing abrupt transitions or reducing ability to follow larger
movements. If you set the threshold higher than the dynamic range of the
coordinates, you can just use it as a simple mean average.

This filter has no effect if the samples are changing by more that the
threshold set by averaging_threshold in the configuration.
However while samples come in that don't go outside this threshold from
the last reported sample, Mean replaces the samples with a simple mean
of a configurable number of samples (set by bits_filter_length in config,
which is 2^n, so 5 there makes 32 sample averaging).

In the current configuration we are using a mean with 4 points. It means that
when the filter is cleared we need 4 samples for each output and then we
only use 1 sample for each output until the filter is cleared.

Linear calibration filter

This filter allow us to scale touchscreen values using calibration data. It also exposes the calibration constants using sysfs.
By default it does not scale the coordinates.

How things fit together

The following configuration that we are using for the GTA02 lives in the file arch/arm/mach-s3c2440/mach-gta02.c. This configuration is not in the touchscreen driver, and the same driver can be used with different settings in other devices. The filters can also be used in other drivers.