Introduction

This project has been started from my desire to create a small program
on a surface computer (window 8 or Android tablet) which can recognize what my
5 years old daughter draws on it and helps her to study numbers and alphabet
characters. I know it is very hard work relating to machine learning and pattern recognition. The program may not be
completed until my daughter finishes her secondary school program but it is
good reason to me to spend my free time on it. At the present, the project has
achieved several good results such as: a
library for manipulating UNIPEN database, a library for creating a neural
network dynamically on runtime and some classes for character segmentation etc.
These archives have encouraged me to continue to develope the project as well as
to share it to community in order to help juniors easier to study pattern
recognition techniques in general and online handwriting
recognition techniques in particular.

Background

Part 1: UNIPEN – online handwriting training database library: it has several classes manipulating UNIPEN database, one of the most popular handwriting database over the world.

<o:p />

Part 2: Convolution neural network library: the library is organized based on neural network’s objects including: network, layer, neuron, weight, connection, activation function, forward propagation, back propagation classes. It is simple to a junior to create not only a traditional neural network but also a convolution network with smallest effort. Especially, the library also supports creating a network on runtime. So we can create or change different networks when the program is running.

Part 3: Image segmentation library: it is some functions for image pre-processing and segmentation. It is in developing process.

The UNIPEN and its format

In a large collaborative effort, a wide number of research institutes
and industry have generated the UNIPEN standard and database.
Originally hosted by NIST, the data was divided into two distributions, dubbed
the trainset and devset. Since 1999, the International UNIPEN Foundation (iUF)
hosts the data, with the goal to safeguard the distribution of the trainset and
to promote the use of online handwriting in research and applications. In the
last years, dozens of researchers have used the trainset and described
experimental performance results. Many researchers have reported well
established research with proper recognition rates, but all applied some
particular configuration of the data. In most cases the data were decomposed,
using some specific procedure, into three subsets for training, testing and
validation. Therefore, although the same source of data was used, recognition results
cannot really be compared as different decomposition techniques were employed. <o:p />

For some time now, it has been the goal of the iUF to organize a
benchmark on the remaining data set, the devset. Although the devset is
available to some of the original contributors to UNIPEN, it has not officially
been released to a broad audience yet. I have been no luck to work on
it.<o:p />

Due to UNIPEN trainset is collection of particular datasets from
different research institutes, these datasets are
decomposed using some specific procedure. However, my approach is a little bit
different; I tried to find some general points in the structure of these
datasets to create a procedure which can decompose all datasets in the trainset
correctly in most cases. <o:p />

The UNIPEN format is described in here. The format is thought of as
a sequence of pen coordinates, annotated with various information, including segmentation
and labeling. The pen trajectory is encoded as a sequence of components .PEN
DOWN and .PEN UP, containing pen coordinates (e.g. XY or XY T as declared in
.COORD). The instruction .DT permits précising the elapsed time between two components.
The database is divided into one or several data sets starting with .START SET.
Within a set, components are implicitly numbered, starting from zero. Segmentation
and labeling are provided by the .SEGMENT instruction. Component numbers are
used by .SEGMENT to delineate sentences, words, characters. A segmentation
hierarchy (e.g. SENTENCE WORD CHARACTER) is declared with .HIERARCHY . Because
components are referred by a unique combination of set name and order number in that set, it
is possible to separate the .SEGMENT from the data itself.

<o:p />

<o:p />

In general, the format of a UNIPEN data file has KEYWORDS which are
divided to several groups like: Mandatory
declarations, Data documentation, Alphabet, Lexicon, Data layout, Unit system,
Pen trajectory¸ Data annotations. In order to get the information and
categorize these keywords, I built a collection of classes based on the above
groups which can help me to get and categorize all necessary information from data file.

<o:p />

Although the UNIPEN format based on KEYWORD
but it not fix in a specific order. I created a DataSet class like a storage racks, when a KEYWORD is found it will be categorized and put
to a correspondent rack. In the normal, each UNIPEN file contains
one or several Datasets. But, in most cases there is a DataSet in a file. My
library now focuses on this case only.

Getting training patterns (Pen trajectory bitmaps) from trainset using the library is very simple as follows:

Convolution
neural network

Theory of convolution network has been described in
my previous article and several others on Codeproject. In this article, I will only focus on
what development in this library compares to the previous program.

<o:p />

This library has been re-written
completely to fit my current requirement: easy to use to juniors who do not
need a deep knowledge on neural network; creating a neural network simply, changing network parameters without changing code and especially is the capacity of exchanging different networks on runtime. <o:p />

In the current version, if I want to create a network which can recognize not only 10 digits but also alphabets (62 outputs total). I simply add some other layers and change some parameters as follows:

We can change all network parameters such as: number of layers, input pattern size, number of feature map, kernel size in convolution network, number of neuron in a layer, number of output...etc. to have the best network for us. Changing network is not influent to forward propagation or back propagation
classes.

<o:p />

Experiment
with the library:<o:p />

The demo program presents two main functions of the library: UNIPEN data browser and Convolution neural network training and testing. Of course the in put data is UNIPEN trainset which can be downloaded on the website: http://unipen.nici.kun.nl/. In order to the demo program can run correctly, the trainset folder have to be renamed to UnipenData.

Picture 4: UNIPEN data browser

We can simply select Data folder in UnipenData to browse all data. The recognition function can be active by loading a network parameters file. Depend on the network file the program can recognize digits only or all capital letters plus digits.

Picture 5: Convolution network training

The default convolution network is 62 outputs network. You can change the network by loading the attached network parameters files. In order to get corrected training data, for example to a 36 outputs network (a network for capital letters and digits) you should delete all folders in the Data folder except 1a,1b (a folder of digit and capital letters).

In my experiment, results are rather good with 88% accuracy to the collection of capital letter and digits or 97% to digits. I can not to do the experiment to 62 outputs network because my laptop was nearly burn when I trained the network.

Points of Interest

As a human brain, an artificial intelligent system can not create a unique neural network with billions neurons inside to solve different problems. It will contains several small networks which can solve seperated problems. My library has this capacity. So I do hope that it can be applied not only to my daughter's program but also to a real system in some day.

At the moment, this project is sponsored by my university as an annual small research. I am finding a donation or scholarship to continue it. It will be highly appreciated if someone interested in this project and can help it more developed.