Computational Measurement of Intelligence

This corresponds to a project spanning the period 1996-2000. The last update of this page was made in 2000!
This page contains much secondary material and digressions, so if you want to start reading something more concise and round,
please try first the most representative publication of this period: "Beyond the Turing Test", J. of Logic, Language and Information, 2000.
For a more recent project and publications, go to the
anYnt project
.

Presentation:

This page describes the theory and results of the measurement of cognitive
abilities by the use of tests generated from computational constructs derived
from descriptional complexity (more commonly known as Kolmogorov complexity).
Although there are still many problems to solve, its current state
of the art (see e.g. Hernández-Orallo 2000d)
allows the generation of grounded tests of an important factor of intelligence,
the ability to comprehend, which correlates with the g-factor.
The implications of a successful computational measurement of intelligence
are many-fold:

its non-anthropomorofic character would allow to measure in a more equal
way human and non-human machines,

AI could evaluate its progress objectively in many different areas (according
to different factorial tests) and, finally,

many fascinating questions about the nature and limits of intelligence
are posed ahead to be solved.

Current and Past Tests:

The following tests were generated according to the theory developed from
some publications (see below). For its complete understanding, please read
the corresponding paper first.
Some of them are compared with the European Intelligence Test (EIQtest,
see e.g. European IQtest)

This test measures the comprehension ability, and
it is measured in two slightly different forms: predictive and abductive
(forward and backwards respectively). These closely related factors are
measured by two separate tests: the IT-test and the AIT-test.

We evaluated the test without penalising the
errors, i.e., the function hit evaluated the same for blanks than
for mistakes. We gave all the questions the same value, independently of
its difficulty (e=0). IQ-correlations are illustrated in Table 1.

Prediction

Abduction

Both (Induction)

High-School

0.31

0.38

0.42

University

0.51

0.42

0.56

Both Groups

0.73

0.68

0.77

Table 1. Correlations with EIQ test

The correlation for induction (prediction + abduction)
is of the same order as the usual correlation for induction tests made
by psychologists. The correlation between the abduction and prediction
tests was 0.61, less than expected, which suggests that even problems constructed
by the same generator can be more or less difficult depending on its presentation
(abductive or predictive). The correlation between induction and similarity
was 0.51, which supports the thesis that "the ability of compression" is
different from "the ability of comprehension". Finally, we think that an
analogy test based on our theory would surely round off the study.

With these data and our amateur methods we are
not in conditions to assert more things about the relation between C-tests
and IQ-tests. There is only a thing that has no discussion in the light
of the results, the k-hardness matches fairly well with the difficulty
people found on them, as it is seen in Figure 1:

This test attempts to measure the analogical ability.
It is called AT-test. However, the problems are quite simple, and the problems
are contaminated by both inductive and deductive factors, thus it is not
much meaningful.