Sign up to receive free email alerts when patent applications with chosen keywords are publishedSIGN UP

Abstract:

Disclosed is a method for assigning the content of a digital image to a
class of a classification system. Said method comprises the following
steps: --a predetermined number of F numerical shape characteristics
ψm are determined; --the value of each shape characteristic of
the F numerical shape characteristics determined for the image is
compared to the value filed in a table for the respective shape
characteristic, values for the individual numerical shape characteristics
being allocated to each class in the table; --the class in which the F
numerical shape characteristics determined for said image correspond best
to the values of the numerical shape characteristics indicated in the
table for said class is output as the class into which the image that is
to be recognized is classified.

Claims:

1. A method for associating the content of a digital image with a class of
a classification system, wherein the image is represented by N pixels, a
each pixel being located at the position (xi, yj) in a
predetermined coordinate system said image extending from the coordinates
(0, 0) to (ximax, yjmax) and imax being the maximum number of
pixels in the direction of the x-coordinate and jmax is the maximum
number of pixels in the direction of the y-coordinate and wherein at
least one numerical content attribute data [j, i] is associated with each
pixel, said method comprising the steps of:determining at least one group
of a predetermined number of F numerical shape attributes ψm
with m as a running index, wherein ψm is a transformed
expression of the moment ρm, and ρm is derived from
ρ m _ = k m R m _ ##EQU00005## with k m =
( m + 2 ) 2 ( π A ) m 2 ##EQU00005.2## A = m
0 , 0 = Δ a Δ b j = 1 j
max i = 1 i max data [ j , i ]
##EQU00005.3## R m _ = υ m υ 0 = υ
m m 0 , 0 ##EQU00005.4## υ m = Δ a
Δ b j = 1 j max i = 1 i max
( R ( j , i ) ) m data [ j , i ]
##EQU00005.5## R ( j , i ) = ( ( i - 0 , 5 )
Δ a - x _ ) 2 + ( ( j = 0 , 5 ) Δ
b - y _ ) 2 ##EQU00005.6## x _ = m 1 , 0 m 0 ,
0 ##EQU00005.7## y _ = m 0 , 1 m 0 , 0 ##EQU00005.8##
m 1 , 0 = ( Δ a ) 2 Δ b j
= 1 j max i = 1 i max ( i - 0 , 5 )
data [ j , i ] ##EQU00005.9## m 0 , 1 = ( Δ
b ) 2 Δ a j = 1 j max i = 1
i max ( j - 0 , 5 ) data [ j , i ]
##EQU00005.10## whereinΔa=width of the pixel in the x-coordinate
direction,Δb=width of the pixel in the y-coordinate direction,data
[j, i]=content attribute of the pixel at the position (yj,
xi)m=a number running from 1 to F as a counter of the shape
attributes;comparing the value of each shape attribute of the F numeric
shape attributes intended for the picture in the at least one group with
the value saved in a table for the respective shape attribute of this
group, wherein values for the individual numerical shape attributes of
this group in the table are associated with each class; andoutputting the
class as an association class in which the image to be recognized is
classified, in which the F numeric shape attributes intended for the
picture best correspond to the values of the numeric shape attributes
recorded in the table for this class.

3. The method of claim 1, further comprising the step of determining the
number F of the numeric shape attributes ψm from at least 29
samples per class of the classification system, wherein the number F is
increased until the values for the shape attributes ψm obtained
for the samples of one class are different for at least one shape
attribute ψm from the numerical values of this shape attribute
ψm of the other classes.

4. The method of claim 1, further comprising the steps of associating
errors of cast parts to error classes defined in an industrial standard,
and generating the digital image by radioscopy.

5. The method of claim 1, further comprising the steps of associating
errors of weld seams to error classes defined in an industrial standard,
and generating the digital image by radioscopy.

6. The method of claim 1, further comprising the steps of associating
objects reproduced on paper to classes defined for objects reproduced on
paper, and generating the digital image with a scanner.

7. The method of claim 6, wherein said classes are defined for objects
that include elements of a text.

8. The method of claim 6, wherein said classes are defined for objects
that include elements of a musical score.

9. The method of claim 1, wherein individual values of individual color
representations are included in content attributes data [i, j] as a
vector.

10. The method of claim 1, further comprising the step of a parallel
execution of said steps of determining, comparing and outputting for an
image having more than a binary representation, so that each of said
parallel executions provides a respective class for a respective
representation of the image.

11. The method of claim 1, further comprising the step of generating an
optical image as the digital image.

12. The method of claim 1, further comprising the step of generating an
image of a person and a least squares method is used to determine whether
the person has been recognized.

13. The method of claim 12, wherein the image of a person is an image of a
fingerprint.

14. The method of claim 12, wherein the image of a person is an image of
an iris.

15. The method of claim 1, further comprising the step of repeating said
steps of determining, comparing and outputting for an image having more
than a binary representation, so that each repetition provides a
respective class for a respective representation of the image.

16. The method of claim 1, further comprising the step of determining the
number F of the numeric shape attributes ψm using a rotational
ellipse.

17. The method of claim 1, further comprising the step of generating an
optical image as the digital image.

18. The method of claim 10, wherein the optical image is an optical image
of the surface of a component.

19. The method of claim 1, further comprising the step of generating a
radioscopic image as the digital image.

20. The method of claim 1, further comprising the step of generating an
ultrasound image as the digital image.

21. The method of claim 1, further comprising the step of generating a
nuclear spin tomography image as the digital image.

22. The method of claim 1, further comprising the step of generating a
synthetic image as the digital image.

Description:

[0001]The invention relates to a method for associating a digital image
with a class of a classification system.

[0002]Automating error recognition based on optical analysis methods has
become increasingly important with the increasing automation of
industrial processes. Optical error recognition methods were performed in
the past by quality assurance personnel, who inspected the object to be
tested or an image representation of the object to be tested and
identified possible errors. For example, x-ray images of weld seams are
checked based on error types, such as for example tears, inadequate
continuous welds, adhesion errors, slag, slag lines, pores, tubular
pores, root notches, root errors, heavy-metal inclusions and edge offset.
It is also known to inspect radioscopic images of cast parts to identify
errors in the cast part, for example inclusion of impurities, inclusion
of gases, bubbles, such as axial pores or spongy pores, fissures or
chaplets. Because of these errors are of similar type, but may be
different in their appearance and shape, more recent approaches in
industrial error evaluation now associate errors with different classes,
wherein the respective class contains errors of the same type. The
industry standard EN 1435 describes, for example, the classification
system for weld seam errors. According to this standard, the errors
occurring in weld seams and identified by x-ray images are divided into
the 30 different classes, for example classes for the error tear, such as
longitudinal care or transverse tear, inadequate continuous welds,
adhesion errors, foreign inclusions, such as slag, slag lines, gas
inclusions, such as pores or tubular pores, or heavy-metal inclusions,
undercuts, root notches, root errors, and edge offset. With increasing
automation of these processes, there is now a push to achieve optical
recognition of errors and association of these errors with predetermined
classes through image analysis based on images that are recorded and
stored using digital image recording techniques. Conventional automated
error recognition methods based on digital images use a so-called
"heuristic approach." With this approach, reference images are saved in
an image processing unit and an attempt is made to through image
comparison to associate the content of a digital image with one of these
reference patterns.

[0003]In other technical fields, image content is associated with classes
of a classification system, for example, for character recognition. In
this case, for example, each letter forms its own class, so that for the
capital letter alphabet there exist, for example, 26 classes, namely for
the characters (A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S,
T, U, V, W, X, Y, Z). The OCR technologies (Optical Character
Recognition) analyze the digital image of a printed page generated by a
scanner and associate the individual letter symbols with the
predetermined classes. As a result, the OCR technology "recognizes" the
text and can transfer the classified characters to a text processing
program as an editable sequence of letters. The granted European patents
0 854 435 B1 and 0 649 113 B1 are directed, for example, to the technical
field of character recognition (Optical Character Recognition).

[0004]The technique of image processing can be more and more divided into
areas with different sub-processes, whose technologies develop
independent of each other. These areas are frequently organized into
image preprocessing, image analysis, analysis of image sequences, image
archiving and the so-called Imaging.

[0005]Image preprocessing is defined as the computer-aided improvement of
the quality (processing: noise elimination, smoothing) of the
corresponding digital image to facilitate visual recognition of the
information content of this image by the viewer.

[0006]Image analysis is defined as the computer-aided evaluation of the
information content of the corresponding digital image by automated and
reproducible structuring, identification and comprehension of this image.

[0007]Analysis of image sequences is defined as the computer-aided
evaluation of the information content of the respective sequence of
digital images by automated and reproducible structuring, identification
and comprehension of all individual images of this sequence and by
automated and reproducible comprehension of the context of the sequence
of individual images of this image sequence.

[0008]Image archiving is defined as the computer-aided compression and
storage of the digital images together with indexed search descriptors
from a controlled vocabulary.

[0009]Imaging is defined as the computer-aided generation of synthetic
graphics and digital images for visualizing and describing the
information content of complex processes on an image and symbol plane for
the human observer.

[0010]The technique of associating the content of digital images with a
class of the classification system is one method of image analysis, which
can be divided into three subareas: segmentation, object recognition and
image comprehension.

[0011]Segmentation is defined as of the automated and reproducible
structuring of the respective digital images by separating the objects
that are relevant for the analysis of the image from each other and from
the image background. Object recognition is defined as the automated and
reproducible classification of the separated objects. Image comprehension
can be interpreted as the automated and reproducible interpretation of
the respective digital image by context evaluation of the classified,
separated objects. The technique of associating digital images with a
class of a classification system is a method of object recognition.

[0012]Object recognition can be viewed as a subarea of pattern
recognition, namely as the subarea of the pattern recognition which
recognizes as patterns only two-dimensional objects in images.

[0013]Images are typically displayed as an image composed of pixels,
whereby to display the image, the content of each pixel and its position
in the image must be known. Depending on the content attribute, the image
is can be divided into color images, grayscale images and binary images,
wherein binary images have as content attribute, for example, only the
values 0 and 1 for black and white, respectively.

[0014]One method frequently used in this technology for associating a
digital image with a class of a classification system, which was used
successfully for decades for distinguishing military aircraft (friend-foe
identification), is known from M. K. Hu: "Visual Pattern Recognition by
Moment Invariants", IRE Trans. Info. Theory, vol. IT-8, 1962, pp. 179-187
and R. C. Gonzalez, R. E. Woods: "Digital Image Processing",
Addison-Wesley Publishing Company, 1992, pp. 514-518. Based on the
so-called normalized centralized axial moments obtained through image
analysis techniques from the image display, a finite sequence
{φ1} of 7 dimensionless shape attributes can be generated for an
arbitrary, separated, in limited, two-dimensional object in a binary
image by scaling. If the 7 sequential elements ΦI
(0≦I≦I0=7) are viewed as the coordinates of an
attribute vector Φ=(Φ1, Φ2, Φ3,
Φ4, Φ5, Φ6, Φ7) which is an element
of a 7-dimensional Euclidian attribute space M7, then this method
induces an object recognition in this 7-dimensional attribute space
M7. The method has the advantage, compared with object recognition
by heuristic attributes, that classification occurs exclusively with
attribute vectors Φ=(Φ1, Φ2, Φ3,
Φ4, Φ5, Φ6, Φ7) whose coordinates are
dimensionless shape attributes, so that in particular size differences
between the objects to be recognized and the objects used for generating
the comparison table become unimportant. In addition, a unique sequential
order with respect to the relevance of the attributes for the object
recognition and the digital image processing is defined within the set of
the dimensionless shape attributes φ1 through the coordinate
reference to the attribute vector Φ so that it is immediately clear
that the first attribute Φ1 is the most important.

[0015]However, this method still has disadvantages because the number of
the available dimensionless shape attributes is limited to 7 and a
misclassification can therefore occur with complex objects, if two
different classes have identical values for the 7 dimensionless shape
attributes.

[0016]In view of this background information, it is an object of the
invention to propose a method for associating the content of a digital
image with a class of a classification system which makes it possible to
reliably recognize also symbols having a more complex shape.

[0017]This object is solved with the method according to claim 1.
Advantageous embodiments are recited in the dependent claims.

[0018]The invention is based on the concept of determining for the image
to be analyzed a predetermined number of numerical shape attributes
ψm wherein m is a running index having values from 1 to F,
wherein ψm is a transformed expression of the dimensionless,
scaled, normalized, centralized, polar moment ρm. For
associating the content of the digital image, these mutually independent
shape attributes ψm can be compared with values for these shape
attributes saved in a table. If the values of all determined F shape
attributes ψm are identical to the F shape attributes
ψm saved in the table, then the image content of the analyzed
image belongs to this class. Because of the digitization, it is
preferable to work with approximate values, so that a class association
is already displayed when the computed F shape attributes ψm
agree approximately with the saved F shape attributes ψm.

[0019]Unlike the conventional method which is limited to 7 shape
attributes, the numerical shape attributes ψm proposed for image
analysis in the present invention are independent of each other in such a
way that a large number of shape attributes can be defined, without
creating an interdependence of the shape attributes. In this way, an
unambiguous association of the image contents to be recognized with a
predetermined class can be achieved.

[0020]In particular, the method of the invention is independent of the
relative position of the content to be recognized with respect to the
acquisition device. Even objects rotated by, for example, 60° or
180° can be uniquely associated.

[0021]The method is based on computing is sequence of F functionally
independent, dimensionless attributes of the separated, limited content
in the presented image.

[0022]The image is conventionally represented by N pixels, wherein a pixel
in a predetermined coordinate system is located at the position (xi,
yi) and the image extends from the coordinates (0, 0) to
(ximax, yjmax) and imax is the maximum number of pixels in the
direction of the x-coordinate and ymax is the maximum number of pixels in
the direction of the y-coordinate, and wherein a content attribute data
[j, i] is associated with each pixel.

[0023]The content attribute for a binary image, where the corresponding
image pixel content assumes, for example, the value 1 or 0 for black or
white, is for example a single value saved in a table, and data [j,i] is
representative for the value in this table at the position associated
with the pixel. In color images, where the content attribute of each
pixel is composed, for example, of three values for the 3 color
representation "red, green, blue" (RGB representation), the content
attribute data [j,i] is, for example, representative of a vector which
has these three values for the respective pixel. Data [j,i] can also be
representative of other vectors, if other color representations are used,
e.g., grayscale representations. Data [j,i] can also be representative of
the magnitude of such vector, when a multi-color representation is
converted from a multi-color representation, for example an RGB
representation, into a grayscale or even a binary representation before
employing the classification method of the invention.

[0024]In a color representation, for example an RGB representation, data
[j,i] can also represent the individual value of the red representation,
or the green representation, or the blue representation in the pixel. The
classification method is then performed, for example, exclusively based
on one representation, for example the red representation, whereby the
method is here performed identical to the preceding method for binary
representations. In this case, binary values 1 and 0 can also be used for
data [j,i] at the pixel, wherein for example 1 indicates red and 0 empty.
The classification method can also be performed in parallel for the
different color representations, i.e., in parallel for a binary red
representation, a binary green representation and a binary blue
representation. This increases the accuracy of the classification.

[0025]The moment ρm transformed into the numerical shape
attribute ψm is computed from

Δa=width of the pixel in the x-coordinate direction,Δb=width
of the pixel in the y-coordinate direction,data [j, i]=content attribute
of the pixel at the position (yj, xi)m=a sequential number from
1 to F.

[0026]In a particularly preferred embodiment, the predetermined coordinate
system is a Cartesian coordinate system, because the majority of digital
images defines the pixels with reference to a Cartesian coordinate
system. However, other coordinate systems, for example polar coordinates
systems, can also be employed.

[0027]While presently digital images can be rendered typically with
between 1 and 3 million image dots (pixels), it can be expected that the
number N will increase with advances of image acquisition and image
processing techniques, so that the afore-described sum functions will
approach integral functions.

[0028]More particularly, an image content is defined by the arrangement of
pixels having the same content attribute.

[0030]The classification system can be, for example, a predetermined
industry standard, for example EN 1435. For identification of persons,
for example, each person can form an individual class. In this case, the
F shape attributes ψm representative of the fingerprint or the
iris image of the person to be identified are then saved in the
comparison table. For identification of persons, the image of the iris
acquired by the image acquisition unit, for example a camera, is analyzed
with the method of the invention, whereby the F shape attributes
ψm of the recorded iris are computed and compared with the shape
attribute values saved in the table. If there is an (approximate)
agreement with all values of the shape attributes ψm of a class,
then the system has recognized the person characterized by this class.
Preferably, a least-squares method, for example a method according to
Gauss, can be used to establish the approximate agreement.

[0031]If a digital image is recognized that is represented in a
representation different from a binary representation, then the
aforementioned method steps can be performed for several groups with F
numerical shape attributes ψm, for example, for one group for
values of a red representation, for one group for values of a green
representation, and for one group for values of a blue representation.
Alternatively, the aforementioned method steps can also be performed on
content attributes data [j,i] which contain the individual values of the
individual color representations as a vector. Computational division
operations are then preferably performed on the magnitudes of the
vectors.

[0032]In a preferred embodiment, the shape attribute ψm is
determined by the transformation

ψ m = 1 ρ m _ m . ##EQU00002##

However, other transformations as can also be used to transform
ψm to ρm, and ψm may even be ρm.

[0033]The shape attribute to be compared with the values stored in the
table is preferably the shape attribute ψm obtained with the
aforementioned transformation. Before the comparison with the table
values or in the transformation from ρm, the sequential order of
the F shape attributes can be subjected to an orthogonalization method,
for example as performed in E. Schmidt. In this approach, the shape
attributes to be compared can be recomputed so as to yield for a circle a
sequence of F shape attributes ψ1, ψ2, ψ3,
ψ4, ψ5 . . . ψF with values of 1, 0, 0, 0, 0 .
. . 0.

[0034]For defining the number F of the numerical shape attributes
ψm, the number F can be increased, starting with F=1, from
several, in particular more than 29 samples per class of the
classification system, until the values for the respective shape
attribute ψm determined for the samples of a class are different
in at least one numerical value for at least one shape attribute
ψm from the numerical value of this shape attribute ψm
of the other class. In a particularly preferred embodiment, the number F
of the shape attributes is increased until the values of the shape
attributes with the highest ordinal numbers m in all classes decrease
with increasing ordinal number. The values of the corresponding shape
attribute ψm determined for the at least 29 samples per class
can be arithmetically averaged in order to determine a value to be
inserted for this class for this shape attribute.

[0035]The table reproduced below, which is intended only to illustrate the
freely selected numerical values, shows that for determining the weld
seam error in relation to the error classes "tear", "pore", "tubular
pore", a number F=1 of the numerical shape attributes ψm is not
sufficiently precise, because ψ1 assumes almost identical values
for the tear class as for the tubular pore classes. The association only
becomes unique by including the second numerical shape attribute
ψ2. As can be seen, in spite of the similar numerical values for
ψ2 in the class "pore" and "tubular pore", this system
consisting of only two shape attributes ψ1, ψ2 is
suitable to precisely classify the 3 errors.

[0036]The number F can also be determined by a method based on a
rotational ellipse. Such "Cluster Methods" are described, for example, in
H. Niemann, Klassifikation von Mustern (Pattern Classification), Springer
Verlag, Berlin, 1983, page 200ff.

[0037]The method of the invention for associating the content of a digital
image with a class of a classification system is employed preferably in
the optical inspection of components, in particular in optical surface
inspection. The method can also be used in quality assurance, texture,
shape and contour analysis, photogrammetry, symbol and text recognition,
personnel recognition, robotic vision or evaluation of radiographic or
radioscopic images, ultrasound images and nuclear spin tomography.

[0038]It is thereby unimportant if the images having objects to be
recognized are "optical" images in the spectral range of visible light or
radiographic or radioscopic images, or even synthetic images from the
technical field Imaging. The method can therefore be used in the field of
optical surface inspection as well as in quality assurance, texture,
shape and contour analysis, photogrammetry, symbol and text recognition,
personnel recognition, robotic vision or evaluation of radiographic or
radioscopic images, ultrasound images and nuclear spin tomography.

[0039]When a concrete problem of object recognition is approached in the
context of this broad range of possible applications, then the degree of
complexity of the problem is defined from the beginning:

[0040]It is known into how many different object classes K the objects to
be recognized are to be sorted. Unlike with classification based on
heuristic attributes, in the new algorithmic method the number of degrees
of freedom of the shape can be experimentally determined with respect to
each object class based on a representative random sampling of test
objects. The classification is performed exclusively with attribute
vectors ψ=(ψ1, ψ2, ψ3, ψ4,
ψ5, . . . , ψF). The attribute vector of an arbitrary
separated, limited, two-dimensional object in the image is located inside
a limited, normalized F-dimensional subarea ("unit hypercube") of an
F-dimensional attribute space. The pattern classification is performed by
a problem-specific clustering of the interior of this F-dimensional unit
hypercube.

[0041]The invention will now be described with reference to a drawing
depicting a single exemplary embodiment. It is shown in:

[0042]FIG. 1 three different representations of a first symbol to be
recognized;

[0043]FIG. 2 three representations of a second symbol to be recognized;
and

[0044]FIG. 3 three representations of a third symbol to be recognized.

[0045]FIGS. 1, 2 and 3 show the letters A, B, and C in three
representations i) normal, ii) normal, but rotated by 90°, iii)
same orientation as normal, but smaller. In addition to the central
orientation depicted in the Figures, positioning to the left and
positioning to the right were also investigated.

[0046]The following Table shows the values for ψ1, wherein
ψ1 is computed from the relation

[0052]With the afore-described relationship and the respective data fields
for the respective representations, where content attributes are saved at
the positions (yj, xi), the values reproduced in the following
table are obtained:

[0054]As can be seen, the value ψ1 for the letter A assumes
values of about 0.57, for the letter B values of about 0.6, and for the
letter C values of about 0.44. A previously defined symbol can therefore
be uniquely recognized with the method of the invention independent of
the actual position and size of the letter.