First
announcement and call for expression of interest

International Symbol Recognition Contest
at GREC 2003

Computer Vision Center
Barcelona, Catalonia, SpainJuly
29, 2003

An
international contest on symbol recognition will be held during the 5th
IAPR workshop on graphics recognition (GREC’2003) in Barcelona, July 2003.
The idea is to compare various methods for recognizing linear symbols, i.e.
symbols made of lines, arcs, and simple geometric primitives. Because of the
large variety of possible symbols and of application domains, this first announcement
aims primarily at explaining the organizers’ views on the way the contest should
be run, to get feedback from possible
participants.

Application
domains: the idea is to have
symbols from various domains (electronics, architecture …) and to let contestants
choose to participate in all domains or only in some of them.

Input
data format: the symbols to be
recognized will be available in 3 formats – vector format (in a format inspired
by the VEC format of previous contests), scanned binary image, scanned grey-level
image.

Ground truth: models for each symbol
to be recognized will be made available as perfect symbols in VEC format, “perfect”
scans at 600 dpi, both grey-level and binary, and a number of degraded versions
(see below).

Segmentation: basically, the symbols will be pre-segmented,
i.e. disconnected from their context. An optional step will be to provide a
few complete drawings, containing a number of different symbols, to assess the
methods’ ability to segment or extract and recognize non-segmented symbols.

Degradation: the “perfect” vector symbols will be
progressively degraded using a vector deformation model. At each level of vector
degradation, scans will be performed at 600 dpi, 300 dpi, 200 dpi, 100 dpi and
maybe even lower resolutions. In addition, photocopies, folding and other physical
degradations will be applied to printed symbols before new scans are performed.
When invariance to rotation makes sense, rotated instances of symbols will also
be provided.

Scalability: for each application domain chosen
for the contest, there will be at least two sets of possible symbols, one being
a subset of the other, to test the scalability of recognition methods when the
number of symbols to recognize increases.

Training: all models appearing in the actual contest,
and a number of degraded instances of these models, will be made available to
the contestants sufficiently early (a couple of months before) to allow for
all training to be performed in advance of the contest.

The
contest itself: on the day of the
contest, each participant will run his/her methods on her/his own laptop computer.
There may be a special day – one day before the workshop – for running all the
tests.

Performance
criteria to be measured: we think of assessing the methods’ robustness
to noise, their scalability (discrimination potential) when the number of symbols
increases, their complexity (simply measured as the increase of computation
time when noise increases or when the number of models increases), and, optionally,
their ability to segment symbols which are not disconnected. In this first edition
of the contest, we will limit ourselves to assessing discrete responses of symbol
recognition methods (correct recognition, wrong recognition, reject) and not
confidence values or scores.

Presentation
of results: the idea is not
to have as outcome something like “the winner is…” but rather a number of tables,
where the various criteria (robustness, scalability, complexity, segmentation)
are presented, with a different table for each application domain, and for each
type of input data (vector, binary image, grey level image), and the possibility
to compete only in a subset of all the possible combinations. Ultimately, the
result obtained at the workshop should be the starting point for a thorough
collaboration between the teams involved, and the organizers of the contest,
resulting in a deeper understanding of the characteristics, pros and cons of
various approaches to symbol recognition, to be published in one or several
journal articles (a special issue of one of our reference journals could be
one of the possibilities).

More
information, declaration of interest: to help us define precisely the application domains
and the precise scope of the contest, we need your input! Symbol recognition
has a very wide scope, and we do not pretend having thought of all possible
areas and methods. Potential participants are kindly requested to pre-register
by filling out a detailed questionnaire, available at the feedback
section of this site, before October 31, 2002.