2008

Discovery of knowledge from geometric graph databases is of particular importance in chemistry and
biology, because chemical compounds and proteins are represented as graphs with 3D geometric coordinates. In
such applications, scientists are not interested in the statistics of the whole database. Instead they need information
about a novel drug candidate or protein at hand, represented as a query graph. We propose a polynomial-delay
algorithm for geometric frequent subgraph retrieval. It enumerates all subgraphs of a single given query graph
which are frequent geometric epsilon-subgraphs under the entire class of rigid geometric transformations in a database.
By using geometric epsilon-subgraphs, we achieve tolerance against variations in geometry. We compare the proposed
algorithm to gSpan on chemical compound data, and we show that for a given minimum support the total number
of frequent patterns is substantially limited by requiring geometric matching. Although the computation time per
pattern is larger than for non-geometric graph mining, the total time is within a reasonable level even for small
minimum support.

We investigate an implicit method to compute a piecewise linear representation of a surface from a
set of sample points. As implicit surface functions we use the weighted sum of piecewise linear kernel functions.
For such a function we can partition Rd in such a way that these functions are linear on the subsets of the partition.
For each subset in the partition we can then compute the zero level set of the function exactly as the intersection of
a hyperplane with the subset.

We introduce a family of unsupervised algorithms, numerical taxonomy clustering, to simultaneously
cluster data, and to learn a taxonomy that encodes the relationship between the clusters. The algorithms
work by maximizing the dependence between the taxonomy and the original data. The resulting taxonomy is a
more informative visualization of complex data than simple clustering; in addition, taking into account the relations
between different clusters is shown to substantially improve the quality of the clustering, when compared
with state-of-the-art algorithms in the literature (both spectral clustering and a previous dependence maximization
approach). We demonstrate our algorithm on image and text data.

In this report we present new algorithms for non-negative matrix approximation (NMA),
commonly known as the NMF problem. Our methods improve upon the well-known methods of Lee &
Seung [19] for both the Frobenius norm as well the Kullback-Leibler divergence versions of the problem.
For the latter problem, our results are especially interesting because it seems to have witnessed much
lesser algorithmic progress as compared to the Frobenius norm NMA problem. Our algorithms are
based on a particular block-iterative acceleration technique for EM, which preserves the multiplicative
nature of the updates and also ensures monotonicity. Furthermore, our algorithms also naturally apply
to the Bregman-divergence NMA algorithms of Dhillon and Sra [8]. Experimentally, we show that our
algorithms outperform the traditional Lee/Seung approach most of the time.

The Euclidean K-means problem is fundamental to clustering and over the years it has been
intensely investigated. More recently, generalizations such as Bregman k-means [8], co-clustering [10],
and tensor (multi-way) clustering [40] have also gained prominence. A well-known computational difficulty
encountered by these clustering problems is the NP-Hardness of the associated optimization task,
and commonly used methods guarantee at most local optimality. Consequently, approximation algorithms
of varying degrees of sophistication have been developed, though largely for the basic Euclidean
K-means (or `1-norm K-median) problem. In this paper we present approximation algorithms for several
Bregman clustering problems by building upon the recent paper of Arthur and Vassilvitskii [5]. Our algorithms
obtain objective values within a factor O(logK) for Bregman k-means, Bregman co-clustering,
Bregman tensor clustering, and weighted kernel k-means. To our knowledge, except for some special
cases, approximation algorithms have not been considered for these general clustering problems. There
are several important implications of our work: (i) under the same assumptions as Ackermann et al. [1]
it yields a much faster algorithm (non-exponential in K, unlike [1]) for information-theoretic clustering,
(ii) it answers several open problems posed by [4], including generalizations to Bregman co-clustering,
and tensor clustering, (iii) it provides practical and easy to implement methodsin contrast to several
other common approximation approaches.

We study the question of activity classification in videos and present a novel approach for recognizing
human action categories in videos by combining information from appearance and motion of human body parts.
Our approach uses a tracking step which involves Particle Filtering and a local non - parametric clustering step.
The motion information is provided by the trajectory of the cluster modes of a local set of particles. The statistical
information about the particles of that cluster over a number of frames provides the appearance information. Later
we use a Bag ofWords model to build one histogram per video sequence from the set of these robust appearance
and motion descriptors. These histograms provide us characteristic information which helps us to discriminate
among various human actions and thus classify them correctly.
We tested our approach on the standard KTH and Weizmann human action datasets and the results were comparable
to the state of the art. Additionally our approach is able to distinguish between activities that involve the
motion of complete body from those in which only certain body parts move. In other words, our method discriminates
well between activities with gross motion like running, jogging etc. and local motion like waving,
boxing etc.

This paper proposes a framework for single-image super-resolution and JPEG artifact removal.
The underlying idea is to learn a map from input low-quality images (suitably preprocessed low-resolution or
JPEG encoded images) to target high-quality images based on example pairs of input and output images. To
retain the complexity of the resulting learning problem at a moderate level, a patch-based approach is taken such
that kernel ridge regression (KRR) scans the input image with a small window (patch) and produces a patchvalued
output for each output pixel location. These constitute a set of candidate images each of which reflects
different local information. An image output is then obtained as a convex combination of candidates for each
pixel based on estimated confidences of candidates. To reduce the time complexity of training and testing for
KRR, a sparse solution is found by combining the ideas of kernel matching pursuit and gradient descent. As a
regularized solution, KRR leads to a better generalization than simply storing the examples as it has been done
in existing example-based super-resolution algorithms and results in much less noisy images. However, this may
introduce blurring and ringing artifacts around major edges as sharp changes are penalized severely. A prior model
of a generic image class which takes into account the discontinuity property of images is adopted to resolve this
problem. Comparison with existing super-resolution and JPEG artifact removal methods shows the effectiveness
of the proposed method. Furthermore, the proposed method is generic in that it has the potential to be applied to
many other image enhancement applications.

Unsupervised time-series segmentation in the general scenario in which the number of segment-types
and segment boundaries are a priori unknown is a fundamental problem in many applications and requires an accurate segmentation model as well as a way of determining an appropriate number of segment-types.
In most approaches, segmentation and determination of number of segment-types are addressed
in two separate steps, since the segmentation model assumes a predefined number of segment-types.
The determination of number of segment-types is thus achieved by training and comparing several separate models. In this paper, we take a Bayesian approach to a segmentation model based on linear Gaussian state-space models to achieve structure selection within the model. An appropriate prior distribution on the parameters is used to enforce a sparse parametrization, such that the model automatically selects the smallest number of underlying dynamical systems that explain the data well and a parsimonious structure for each dynamical system. As the resulting model is computationally intractable, we introduce a variational approximation, in which a reformulation of the problem enables to use an efficient inference algorithm.

This report summarizes the theory and some main applications of a new non-monotonic algorithm for
maximizing a Poisson Likelihood, which for Positron Emission Tomography (PET) is equivalent to minimizing
the associated Kullback-Leibler Divergence, and for Transmission Tomography is similar to maximizing the dual
of a maximum entropy problem. We call our method non-monotonic maximum likelihood (NMML) and show
its application to different problems such as tomography and image restoration. We discuss some theoretical
properties such as convergence for our algorithm. Our experimental results indicate that speedups obtained via our
non-monotonic methods are substantial.

We propose a framework for analyzing and comparing distributions, allowing us to design statistical tests to determine if two samples are drawn from different distributions. Our test statistic is the largest difference in expectations over functions in the unit ball of a reproducing kernel Hilbert space (RKHS). We present two tests based on large deviation bounds for the test statistic, while a third is based on the asymptotic distribution of this statistic. The test statistic can be computed in quadratic time, although efficient linear time
approximations are available. Several classical metrics on distributions are recovered when the function space used to compute the difference in expectations is allowed to be more general (eg.~a Banach space). We apply our two-sample tests to a variety of problems, including attribute matching for databases using the Hungarian marriage method, where they perform strongly. Excellent performance is also obtained when comparing distributions over graphs, for which these are the first such tests.

This technical report is merely an extended version of the appendix of Steinke et.al. "Manifold-valued Thin-Plate Splines with Applications in
Computer Graphics" (2008) with complete proofs,
which had to be omitted due to space restrictions. This technical report requires a basic knowledge of differential
geometry. However, apart from that requirement the technical report is self-contained.

Real-time control of the endeffector of a humanoid robot in external coordinates requires
computationally efficient solutions of the inverse kinematics problem. In this context, this
paper investigates methods of resolved motion rate control (RMRC) that employ optimization
criteria to resolve kinematic redundancies. In particular we focus on two established techniques,
the pseudo inverse with explicit optimization and the extended Jacobian method. We prove that
the extended Jacobian method includes pseudo-inverse methods as a special solution. In terms of
computational complexity, however, pseudo-inverse and extended Jacobian differ significantly in
favor of pseudo-inverse methods. Employing numerical estimation techniques, we introduce a
computationally efficient version of the extended Jacobian with performance comparable to the
original version. Our results are illustrated in simulation studies with a multiple degree-offreedom
robot, and were evaluated on an actual 30 degree-of-freedom full-body humanoid robot.

Our goal is to understand the principles of Perception, Action and Learning in autonomous systems that successfully interact with complex environments and to use this understanding to design future systems