Sign up to receive free email alerts when patent applications with chosen keywords are publishedSIGN UP

Abstract:

In one aspect, a method to identify a candidate object includes receiving
an image of the candidate object and projecting the received image onto
an image subspace. The image subspace is formed from images of known
objects of a class. The method also includes determining whether the
candidate object is in the object class based on the received image and
the image subspace using a likelihood ratio. The likelihood ratio
includes a first probability density indicating a probability an object
is in the object class and a second probability density indicating a
probability an object is not in the class. The first probability density
and the second probability are each a function of a distance of the
received image to the image subspace.

Claims:

1. A method to identify whether a candidate object is from an object
class, comprising:receiving an image of the candidate object;projecting
the received image onto an image subspace, the image subspace being
formed from images of known objects of the object class; anddetermining
whether the candidate object is in the object class based on the received
image and the image subspace using a likelihood ratio, the likelihood
ratio including a first probability density indicating a probability an
object is in the object class and a second probability density indicating
a probability an object is not in the object class,wherein the first
probability density and the second probability are each a function of a
distance of the received image to the image subspace.

2. The method of claim 1, further comprising performing shape
normalization on the received image.

3. The method of claim 1, further comprising:receiving images from the
known objects from the object class;performing shape normalization on the
received images of the known objects in the object class; andgenerating
the image subspace from the shape-normalized received images of known
objects in the object class.

4. The method of claim 3, further comprising storing each received image
as a polygon having vertices.

5. The method of claim 4 wherein storing each received image as a polygon
having vertices comprises storing each received image as a polygon having
six vertices.

6. The method of claim 4 wherein generating the image subspace comprises
generating the image subspace based on a subset of the vertices.

8. A method to identify whether a candidate object is a mine;
comprising:receiving a sonar image of a candidate object;projecting the
received image onto a mine image subspace, the mine image subspace being
formed from sonar images of known mines; anddetermining whether the
candidate object is a mine based on the received image and the mine image
subspace using a likelihood ratio including a first probability density
indicating a probability an object is a mine and a second probability
density indicating a probability an object is not a non-mine,wherein the
first probability density and the second probability density are each a
function of a distance of the received image to the mine image subspace.

9. The method of claim 8, further comprising performing shape
normalization of the received sonar image.

10. The method of claim 8, further comprising:receiving images from known
mines;performing shape normalization on the received images of known
mines; andgenerating the mine image subspace from the shape normalized
mine images.

11. The method of claim 10, further comprising storing each received image
as a polygon having vertices.

12. The method of claim 11 wherein storing each received image as a
polygon having vertices comprises storing each received image as a
polygon having six vertices.

13. The method of claim 11 wherein generating the image subspace comprises
generating the image subspace based on a subset of the vertices
comprising one of at least a portion of a head of the mine and at least a
portion of a tail of the mine.

14. An article comprising a machine-readable medium that stores executable
instructions to identify whether a candidate object is a mine, the
instructions causing a machine to:receive images from known mines;perform
shape normalization on the received images of known mines;generate a mine
image subspace from the shape normalized mine images;receive the sonar
image of a candidate object;performing shape normalization of the
received sonar image;project the received image onto the mine image
subspace, the mine image subspace being formed from sonar images of known
mines; anddetermine whether the candidate object is a mine based on the
received image and the mine image subspace using a likelihood ratio
including a first probability density indicating a probability an object
is a mine and a second probability density indicating a probability an
object is not a non-mine,wherein the first probability density and the
second probability density are each a function of a distance of the
received image to the mine image subspace.

15. The article of claim 14, further comprising instructions causing a
machine to store each received image as a polygon having vertices.

16. The article of claim 15 wherein instructions causing a machine to
store each received image as a polygon having vertices comprises
instructions causing a machine to store each received image as a polygon
having six vertices.

17. The article of claim 15 wherein instructions causing a machine to
generate the image subspace comprises instructions causing a machine to
generate the image subspace based on a subset of the vertices comprising
one of at least a portion of a head of the mine and at least a portion of
a tail of the mine.

18. An apparatus to identify whether a candidate object is a mine,
comprising:circuitry to:receive images from known mines;store each
received image as a polygon having vertices;perform shape normalization
on the received images of known mines;generate a mine image subspace from
the shape normalized mine images;receive the sonar image of a candidate
object;perform shape normalization of the received sonar image;project
the received image onto the mine image subspace, the mine image subspace
being formed from sonar images of known mines;determine whether the
candidate object is a mine based on the received image and the mine image
subspace using a likelihood ratio including a first probability density
indicating a probability an object is a mine and a second probability
density indicating a probability an object is not a non-mine,wherein the
first probability density and the second probability density are each a
function of a distance of the received image to the mine image subspace.

19. The apparatus of claim 18 wherein the circuitry comprises at least one
of a processor, a memory, programmable logic and logic gates.

20. The apparatus of claim 18 wherein circuitry to store each received
image as a polygon having vertices comprises circuitry to store each
received image as a polygon having six vertices.

21. The apparatus of claim 18 wherein circuitry to generate the image
subspace comprises circuitry to generate the image subspace based on a
subset of the vertices comprising one of at least a portion of a head of
the mine and at least a portion of a tail of the mine.

[0002]Traditionally, mine detection using sonar imagery has used image
processing and detection methods that often rely on preset segmentation
and predetermined feature extraction methods followed by classification
methods (e.g., neural networks, Bayesian networks and so forth). These
classification methods have the disadvantage of not providing good
detection or classification accuracy.

SUMMARY

[0003]In one aspect, a method to identify whether a candidate object is
from an object class includes receiving an image of the candidate object
and projecting the received image onto an image subspace. The image
subspace is formed from images of known objects of a class. The method
also includes determining whether the candidate object is in the object
class based on the received image and the image subspace using a
likelihood ratio. The likelihood ratio includes a first probability
density indicating a probability an object is in the object class and a
second probability density indicating a probability an object is not in
the class. The first probability density and the second probability are
each a function of a distance of the received image to the image
subspace.

[0004]In another aspect, a method to identify whether a candidate object
is a mine includes receiving a sonar image of a candidate object,
projecting the received image onto a mine image subspace, the mine image
subspace being formed from sonar images of known mines and determining
whether the candidate object is a mine based on the received image and
the mine image subspace using a likelihood ratio including a first
probability density indicating a probability an object is a mine and a
second probability density indicating a probability an object is not a
non-mine. The first probability density and the second probability
density are each a function of a distance of the received image to the
mine image subspace.

[0005]In a further aspect, an article includes a machine-readable medium
that stores executable instructions to identify whether a candidate
object is a mine. The instructions cause a machine to receive images from
known mines, perform shape normalization on the received images of known
mines, generate a mine image subspace from the shape normalized mine
images, receive the sonar image of a candidate object, performing shape
normalization of the received sonar image, project the received image
onto the mine image subspace, the mine image subspace being formed from
sonar images of known mines and determine whether the candidate object is
a mine based on the received image and the mine image subspace using a
likelihood ratio including a first probability density indicating a
probability an object is a mine and a second probability density
indicating a probability an object is not a non-mine. The first
probability density and the second probability density are each a
function of a distance of the received image to the mine image subspace.

[0006]In a still further aspect, an apparatus to identify whether a
candidate object is a mine includes circuitry to receive images from
known mines, store each received image as a polygon having vertices,
perform shape normalization on the received images of known mines,
generate a mine image subspace from the shape normalized mine images,
receive the sonar image of a candidate object, perform shape
normalization of the received sonar image, project the received image
onto the mine image subspace, the mine image subspace being formed from
sonar images of known mines and determine whether the candidate object is
a mine based on the received image and the mine image subspace using a
likelihood ratio including a first probability density indicating a
probability an object is a mine and a second probability density
indicating a probability an object is not a non-mine. The first
probability density and the second probability density are each a
function of a distance of the received image to the mine image subspace.

DESCRIPTION OF THE DRAWINGS

[0007]FIG. 1 is a block diagram of an example of a mine detection system.

[0008]FIG. 2 is a flowchart of an example of a process to identify mines.

[0009]FIG. 3 is a representation of determining a distance between a
candidate mine image patch associated with a candidate object and its
projection onto a mine image subspace.

[0011]FIG. 5 is an example of a representation of the mine image subspace
having mine image patches.

[0012]FIG. 6 is a flowchart of an example of a process to generate the
mine image subspace.

[0013]FIG. 7 is a flowchart of an example of a process to determine if the
candidate mine image patch is a mine.

[0014]FIG. 8 is a diagram showing probability density curves of an object
being a mine and an object not being a mine.

[0015]FIG. 9 depicts tables for using different vertices of the mine
polygon to determine if the candidate object is a mine.

[0016]FIG. 10 is a block diagram of an example of an identification
processing system used in mine identification.

DETAILED DESCRIPTION

[0017]In contrast to prior art techniques, described herein is an approach
to identifying mines; however, this approach may be applied to
identifying objects other than mines.

[0018]Referring to FIG. 1, a mine identification system 10 includes an
identification processing system (IPS) 12 and a sonar sensor system 14
connected to the IPS by a network 16. The sonar sensor system 14 sends
sonar signals 17 that are reflected off an object such as a mine 18. The
sonar sensor system 14 detects the object based on the reflected signals
19 received from the object. The IPS 12 determines if the detected object
is a mine.

[0019]In one example, the sonar sensor system 14 may include one or more
sonar sensors, such as sonar buoys. In another example, the sonar sensor
system 14 is a sonar sensor located aboard a submarine. In one example,
the network 16 is a wireless network.

[0020]The IPS 12 may be located on a ground-based platform (e.g., in a
building, in a vehicle and so forth), a space-based platform (e.g., a
satellite, a space-vehicle and so forth), a sea-based platform (e.g., a
ship, a submarine, a buoy, an anchored sea structure, a torpedo, an
undersea robot vehicle and so forth) or on an air-based platform (e.g.,
an aircraft, a helicopter, a missile and so forth).

[0021]In one example, the IPS 12 may be co-located (i.e. on the same
platform) with the sonar sensor system 14. In other examples, the IPS 12
is not co-located with the sonar sensor system 14.

[0022]As will be shown below, a received image of a candidate object is
converted to a candidate mine image patch which is used to determine
whether the candidate object is a mine or not based on a mine image
subspace formed from mine image patches of known mines. In particular, a
distance, d, is determined between a candidate mine image patch 100 (FIG.
3) and a mine image subspace 96 (FIG. 3). The closer the candidate mine
image patch 100 (FIG. 5) is to the mine image subspace 96 (FIG. 3), the
greater likelihood that the candidate object is a mine.

[0023]Referring to FIG. 2, a process 60 is an example of a process to
identify an object and in particular identifying whether an object is a
mine. Process 60 generates a mine image subspace (64). The mine subspace
is formed using a set of known mines. For example, a set of known mines
is used to record their sonar images. Each item in the set of known mines
is saved as a mine image patch. The mine image subspace is used instead
of the entire real image space because the mine image subspace is much
smaller space than the real image space so that using a mine image
subspace reduces processing time.

[0024]The following is mathematical support that a mine image subspace, in
particular an eigen-subspace (based on eigenvalues and eigenvectors), may
be used to model mines rather than using an entire space of images. For
example, let an image, M(x,y), be a two-dimensional N by N array of
intensity values, that is, a vector of dimension N2. Mine image
patches are treated as squares for the sake of simplicity here, although
generally mine images are "shape-normalized" to rectangular grid
dimensions of 32 by 128, describing a vector of dimension 4096. Of note
is that the shape-normalized images of mines, being similar in overall
configuration, will not be randomly distributed in this high dimensional
space but instead form a smaller subspace. Principal Components Analysis
(PCA) may be used to identify a small number of basis vectors that best
capture the distribution of mine images within the larger image space.

[0025]Let {Mi}i=1 . . . T with Miε be a sample set
of normalized mine images. The mean mine image from the set is defined by

ψ ##EQU00001##

where T is the number of data points. Each mine image deviates from the
average by mi=Mi-ψ. PCA is applied to the mean subtracted
set described by {mi}i=1 . . . T in search for a set of
orthonormal basis vectors {ui}i=1 . . . T and associated
eigenvalues {λi}i=1 . . . T that optimally, in a least
squares sense, describe the distribution of the random variable
mε The basis vectors ui and scalars λi are
eigenvectors and eigenvalues of the covariance matrix described as

##EQU00002##

where A=[m1 m2 m3 . . . mn]/ {square root over (T)} is
a block compositional matrix whose column i corresponds to vector mi
and A* is the conjugate transpose of the matrix A. The covariance matrix
C is an N2×N2 matrix. Note that eigenvector computation
for this size matrix is not a computationally feasible task. However,
assuming that the number of data points T in the sample space is far less
than N2, the covariance matrix will have a maximum of T-1 distinct
and meaningful eigenvectors.

[0026]Now consider the eigenvectors vi of A*A such that
A*Avi=μivi. Pre-multiplying both sides by A results in
AA*Avi=μiAvi, from which ui=Avi are the
eigenvectors of the original Covariance matrix C=AA*. Thus, an
alternative matrix L=A*A may be constructed and the T eigenvectors
vi of L may be determined. This transposition analysis reduces the
computation from an intractable task to a feasible task. In reality the
number of samples is far smaller than the dimension of the image space,
so this is a useful step in image-based eigenspace analysis.

[0027]For example, if the eigen-subspace is represented as Un where n
is the number of dimensions and the total space is represented by where
m is the number of dimensions, then Un.OR right. and n<<m.
Thus, eigenvectors of L span a basis set which describe the normalized
mine images. For example, of this set, picking the top 8 eigenvectors
(n=8) associated with the largest 8 eigenvalues forms a lower dimensional
subspace U8 to model the mine image subspace than the total space.

[0028]Based on the preceding mathematical support above for using a mine
image subspace rather than the entire higher dimensional space, the mine
image subspace may be constructed using known images of mines.

[0029]Referring back to FIG. 2, process 60 receives a sonar image (68).
For example, the identification processing system 12 receives a sonar
image from the sonar sensor system 14 containing an unidentified object.

[0030]Process 60 performs shape-normalization on the received image (72).
For example, IPS 12 normalizes the shape of the received image to fit
within the boundaries of a mine image patch (see, FIG. 4, for example) to
form the candidate mine image patch 100 (FIG. 5).

[0032]Process 60 determines if the candidate mine image patch includes a
mine (82). In one example, the candidate mine image patch 100 and the
mine subspace image 96 are used to determine if the received image
includes a mine (FIG. 3).

[0033]Referring to FIG. 4, in one example, a mine image patch 90 includes
a polygon 92 representing a mine and having vertices, for example, six
vertices (a vertex 94a, a vertex 94b, a vertex 94c, a vertex 94d, a
vertex 94e and a vertex 94f). In this example, the vertices 94a, 94b, 94f
form the head of the mine while the vertices 94b-94f form a tail of the
mine. In other embodiments, a mine may be represented by any polygon
having three or more vertices.

[0034]In one example, the set of known mines is generated using side-scan
sonar images of actual mines. Side scan refers to scan configuration
where the sonar sensor is along a horizontal strip on the side of the
under water vehicle. In one particular example, over two hundred mine
images are identified and segmented out of a set of sonar imagery. Each
mine image is shape-normalized and hand-annotated with the six vertices
94a-94f describing a convex polygonal shape of the mine contained within
the mine patch 90. The shape-normalized mine patches are used to
construct the mine image subspace. One example of a mine image subspace
representation is a mine image subspace 96 having mine image patches
90a-90i shown in FIG. 3.

[0035]Referring to FIG. 6, a process 200 is one example to generate a mine
image subspace in processing block 64 (see FIG. 2). Process 200 performs
shape normalization (204). For example, the side-scan sonar imagery for
each known mine is centered with a uniform area, for example, a rectangle
or square, which is the boundary of the mine image patch. In one example,
each mine is morphed into fitting within the boundaries of a mine image
patch 90 (FIG. 4). For example, an elongated mine that extends beyond the
boundary of the mine image patch may be morphed to appear wider in order
to fit with the boundaries of the mine image patch.

[0036]In another example, each mine is shaped as a polygon. In one
example, the mine has six vertices 94a-94f (FIG. 4). In other examples,
the mine may be described using more or less than six vertices. In
further examples, a subset of the vertices may be used such as the
vertices describing the tail of the mine or the vertices that describe
the head of the mine.

[0037]Process 200 performs subspace identification (208). For example, a
distribution is determined from a collection of known mine image patches.
By determining the distribution, it is possible to determine what
subspace from the entire image space mine image patches occupy. This
subspace becomes the mine image subspace.

[0038]Process 200 performs subspace distance statistical analysis (212).
For example, a distance threshold is determined by measuring statistics
between image patches that are known to be mines and image patches that
are known not to be mines to determine a distance threshold. As will be
shown below, a distance between a candidate mine image patch 100 and the
mine image subspace determines the likelihood that the candidate mine
image patch contains a mine. For example, moving closer towards the mine
image subspace from a far distance, what distance do objects identified
as non-mines become identified as mines. In one example, a probability
density is determined that the object is a mine and a probability
distance is determined that the object is a non-mine, each probability
density is a function of distance from the mine image subspace 96.

[0039]Referring to FIG. 7, a process 220 is one example of a process to
determine a distance of the candidate image patch 100 to the mine image
subspace 96 (224). For the candidate mine image patch 100 (FIG. 3), the
vector representation, ω=[w1, w2, w3, . . . ,
w8]T, may be used to determine the distance of the candidate
mine image 100 to the mine image subspace 96.

[0040]For example, referring to FIG. 3, consider, y, representing a point
in space of the candidate mine image patch 100 where yε and
consider, yproj, representing a point on the image mine subspace
where the candidate image is projected and where yproj=

##EQU00003##

where ukε, uk is an eigenvector and k is the index.

[0041]The candidate mine image 100 is separated from the mine subspace 96
by a distance, d. The distance, d=∥y-yproj∥,
provides a good measure of determining whether the candidate mine image
patch 100 contains a mine or not. For example, if the distance is small
then there is a higher likelihood that the candidate mine image patch 100
contains a mine. Likewise, a very large distance indicates a less
likelihood that the candidate mine image patch 100 contains a mine.

[0042]Referring FIGS. 7 and 8, process 220 uses likelihood ratios (228).
The probability density for a mine, π.sub.θ(d), is depicted in a
curve 270 and the probability density for a non-mine,
π.sub.θ'(d), is depicted in a curve 275, where
π.sub.θ(d) and π.sub.θ'(d) are functions of the
distance of the candidate mine image 100 from the mine subspace 96. If a
likelihood ratio, π.sub.θ(d)/π.sub.θ'(d), is greater
than or equal to 1, the candidate object is more likely to be a mine and
if the likelihood ratio is less than 1 then the candidate object is more
likely not a mine.

[0043]One difference between the approach described herein and the prior
art is that the features, such as the shapes of polygons and image pixel
content (colors) corresponding to within those polygonal regions, and
model for a mine is determined directly from the known mine image data.
Therefore, the selected features and the model are optimal in the sense
that they best represent a given data set (e.g., a particular mine);
whereas pre-fixed features and models may be optimal in a general sense
but not for a particular data set (e.g., a particular mine). The approach
previously described, which extends the process of modeling with the
addition of shape parameterization, has the advantage of better modeling
appearance variations due to changing mine geometry and thereby improving
the detection/classification accuracy.

[0044]Referring to FIG. 9, as shown above determining a mine using all six
vertices 94a-94f (FIG. 3) may not be the most optimal solution. For
example, using some of the vertices may generate different results. In
one experiment shown in table 282, using five vertices 94b-94f that
make-up the tail of a mine yielded a correct positive of 91.3%, a false
positive of 3.4%, a false negative of 8.7% and a correct negative of
96.6%. In a second experiment shown in table 284, using all six vertices
94a-94f that make-up the mine yielded a correct positive of 88.9%, a
false positive of 13.8%, a false negative of 11.1% and a correct negative
of 86.2%. In a third experiment shown in table 286, using three vertices
94a, 94b, 94f that make-up the head of the mine yielded a correct
positive of 88.1%, a false positive of 24.1%, a false negative of 11.9%
and a correct negative of 75.9%. In a fourth experiment shown in table
288, using three vertices 94c-94e that make-up a portion of the tail of
the mine yielded a correct positive of 92.5%, a false positive of 17.2%,
a false negative of 7.5% and a correct negative of 82.8%.

[0045]FIG. 10 shows an example of the IPS 12 (FIG. 1). The IPS 12 includes
a processor 422, a volatile memory 424, a non-volatile memory 426 (e.g.,
hard disk) and a network transceiver 425 to communicate with sensor
system 14 through the network 16. Non-volatile memory 426 includes
computer instructions 432, an operating system 436 and mine data 438. The
computer instructions 432 included mine identifying instructions 434
which are executed out of volatile memory 424 to perform the process 60.
In one example, the mine data 438 includes the mine image subspace 96.

[0046]Process 60 is not limited to use with the hardware and software of
FIG. 10; it may find applicability in any computing or processing
environment and with any type of machine or set of machines that is
capable of running a computer program. Process 60 may be implemented in
hardware, software, or a combination of the two. Process 60 may be
implemented in computer programs executed on programmable
computers/machines that each includes a processor, a storage medium or
other article of manufacture that is readable by the processor (including
volatile and non-volatile memory and/or storage elements), at least one
input device, and one or more output devices. Program code may be applied
to data entered using an input device to perform process 60 and to
generate output information.

[0047]The system may be implemented, at least in part, via a computer
program product, (i.e., a computer program tangibly embodied in an
information carrier (e.g., in a machine-readable storage device), for
execution by, or to control the operation of, data processing apparatus
(e.g., a programmable processor, a computer, or multiple computers)).
Each such program may be implemented in a high level procedural or
object-oriented programming language to communicate with a computer
system. However, the programs may be implemented in assembly or machine
language. The language may be a compiled or an interpreted language and
it may be deployed in any form, including as a stand-alone program or as
a module, component, subroutine, or other unit suitable for use in a
computing environment. A computer program may be deployed to be executed
on one computer or on multiple computers at one site or distributed
across multiple sites and interconnected by a communication network. A
computer program may be stored on a storage medium or device (e.g.,
CD-ROM, hard disk, or magnetic diskette) that is readable by a general or
special purpose programmable computer for configuring and operating the
computer when the storage medium or device is read by the computer to
perform process 60. Process 60 may also be implemented as a
machine-readable storage medium, configured with a computer program,
where upon execution, instructions in the computer program cause the
computer to operate in accordance with process 60.

[0048]The processes described herein are not limited to the specific
embodiments described herein. For example, the processes are not limited
to the specific processing order of FIGS. 2, 6 and 7, respectively.
Rather, any of the processing blocks of FIGS. 2, 6 and 7 may be
re-ordered, combined or removed, performed in parallel or in serial, as
necessary, to achieve the results set forth above.

[0049]Even though FIG. 3 is shown in three dimensions, FIG. 5 is only a
representation of a projection onto a mine image subspace with dimensions
greater than 3. One skilled in the art would recognize that showing the
actual representation is impossible.

[0050]The processing blocks in FIGS. 2, 6 and 7 associated with
implementing the system may be performed by one or more programmable
processors executing one or more computer programs to perform the
functions of the system. All or part of the system may be implemented as,
special purpose logic circuitry (e.g., an FPGA (field programmable gate
array) and/or an ASIC (application-specific integrated circuit)).

[0051]Elements of different embodiments described herein may be combined
to form other embodiments not specifically set forth above. Other
embodiments not specifically described herein are also within the scope
of the following claims.