Patent application title: SUPERIMPOSED DISPLAY OF IMAGE CONTOURS

Abstract:

Disclosed are computer-implemented methods, systems, and computer program
products for displaying superimposed image contours. A first centroid or
center-of-mass of a contour extracted from a first image that depicts a
moving object as the object appeared at a first time is calculated. A
second centroid or center-of-mass of a second contour extracted from a
second image that depicts the object as the object appeared at a second
time is also calculated. The second image depicts the object in
substantially the same plane as the first image. The first and second
contours are displayed with the respective first and second centroids or
centers-of-mass positioned at a common location. In some implementations,
the images are ultrasound images depicting, for example, a human heart.
The images can be transgastric short axis views of the human heart.

Claims:

1. A computer-implemented method comprising:calculating a first centroid
or center-of-mass of a first contour extracted from a first image
depicting a moving object as the object appeared at a first
time;calculating a second centroid or center-of-mass of a second contour
extracted from a second image depicting the object as the object appeared
at a second time, the second image depicting the object in substantially
the same plane as the first image; anddisplaying the first and second
contours with the respective first and second centroids or
centers-of-mass positioned at a common location.

2. The method of claim 1, wherein the first and second contours are
extracted automatically by a contour-finding algorithm operating on the
respective first and second images.

3. The method of claim 1, wherein the first and second images comprise
ultrasound images.

4. The method of claim 3, wherein the moving object comprises a chamber of
a human heart and the ultrasound images comprise transgastric short axis
views of the chamber.

5. The method of claim 1, further comprising associating a first
identifying feature with the first contour and associating a second
identifying feature with the second contour.

6. The method of claim 5, wherein the identifying features are colors or
type of line.

7. The method of claim 1, wherein the first and second contours are
associated with a left ventricle end diastolic area (LVEDA) or a left
ventricle end systolic area (LVESA) of a human heart.

8. The method of claim 1, wherein the first and second contours are
displayed over a background pattern different from a first or second
background pattern of the respective first or second images.

9. The method of claim 8, wherein the background pattern is a solid color.

10. The method of claim 1, wherein displaying the first and second
contours comprises;determining a value for the common
location;transforming a first set of data representative of the first
contour using the value of the common location;illustrating the first
contour based on the first set of transformed data;transforming a second
set of data representative of the second contour using the value of the
common location; andillustrating the second contour based on the second
set of transformed data.

11. The method of claim 10, wherein illustrating the first and second
contours further comprises processing a first cineloop associated with
the first contour and a second cineloop associated with the second
contour to display within a user interface window.

12. The method of claim 1, further comprisingdisplaying the first and
second images in a first display window, wherein displaying the first and
second contours comprises displaying the first and second contours in a
second display window.

13. A system comprising:a computing apparatus comprising a processor, a
memory, and an image capturing module;a calculation module to calculate a
first centroid or center-of-mass of a first contour extracted from a
first image depicting a moving object as the object appeared at a first
time and to calculate a second centroid or center-of-mass of a second
contour extracted from a second image depicting the object as the object
appeared at a second time, the second image depicting the object in
substantially the same plane as the first image;a first display window in
communication with the processor to display images from the image
capturing module; anda second display window in communication with the
processor and the calculation module to display the first and second
contours of the object with the respective first and second centroids or
centers-of-mass positioned at a common location.

14. The system of claim 13, further comprising a transesophageal
echocardiography probe in communication with the processor.

15. The system of claim 13, further comprising a transformation module to
transform the first and second contours based on the value of the common
location.

16. The system of claim 13, wherein the first and second images are
displayed side-by-side in the first display window.

17. The system of claim 13, wherein the first and second contour are
displayed superimposed in the second display window.

18. The system of claim 13, wherein the second display window is
superimposed over the first display window.

19. The system of claim 13, wherein the second display window is displayed
in response to a command by a user.

20. The system of claim 19, wherein the command is available in response
to a first cineloop and a second cineloop being populated with a
respective first set of data associated with the first contour and a
second set of data associated with the second contour.

21. A computer program product, tangibly embodied in an information
carrier, the computer program product including instructions being
operable to cause data processing apparatus to:calculate a first centroid
or center-of-mass of a first contour extracted from a first image
depicting a moving object as the object appeared at a first
time;calculate a second centroid or center-of-mass of a second contour
extracted from a second image depicting the object as the object appeared
at a second time, the second image depicting the object in substantially
the same plane as the first image; anddisplay the first and second
contours of the object with the respective first and second centroids or
centers-of-mass positioned at a common location.

Description:

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001]This application claims the benefit of U.S. Patent Application No.
60/986,647 titled "Superimposed Display of Image Contours," which was
filed on Nov. 9, 2007, and the contents of which are hereby incorporated
herein by reference.

TECHNICAL FIELD

[0002]The description generally relates to image processing and display
and more specifically to superimposed displays of image contours.

BACKGROUND

[0003]Digital images often contain objects of interest and can be
displayed at various resolutions and at various points in time. In
addition, digital images often contain noise or less relevant features in
various amounts depending on the device used to collect the data
associated with the digital image and the level of resolution. In
particular, obtaining a digital image using inferential techniques such
as radiographic, echographic, or ultrasonic techniques can be especially
susceptible to blurring between the object of interest and noise (or less
relevant features) due to, e.g., electromagnetic effects. In some
applications, an ultrasound technician captures an image using an image
capture device that captures still images or sequences of still image
frames (e.g., video). The ultrasound technician can annotate the captured
image using an input device to mark, for example, landmarks within the
image, distances between points in the image, or other features.

[0004]In certain circumstances, useful information can be obtained by
comparing an image of a moving object captured at one time t1 with
an image of the object captured at a different, e.g., later, time
t2. For example, the effectiveness of a treatment regimen can be
monitored by comparing an ultrasound image of the transgastric short axis
view (TGSAV) of a human heart under observation at time t1 with an
ultrasound image of the TGSAV of the same heart at time t2.
Typically, comparisons between such ultrasound images are facilitated by
displaying an image and objects therein at time t1 alongside a
second image at time t2 on a single display monitor (e.g., the
images are displayed side-by-side).

[0005]One drawback to displaying ultrasound images side-by-side is that
differences between the images or objects therein can be obscured due to
noise or presence of less relevant features in the images. Additionally,
side-by-side comparison does not readily facilitate comparison between
objects present in each image when the image capture device or the
objects may have moved between time t1 and time t2 or when the
objects appear out of scale between the images. Moreover, visually
comparing two objects from different images in different fields of view
can be time consuming for the ultrasound technician or medical or
healthcare personnel (e.g., a doctor) seeking to visually assess the
efficacy of a course of treatment.

SUMMARY

[0006]The concepts described herein address the drawbacks associated with
presenting two images side-by-side by displaying information extracted
about the objects of interest from each of the images at substantially
the same time in a field of view free from noise or less relevant
features in the images. For example, where the object is human heart, and
the images are ultrasound images, a contour of the heart is determined
and extracted from each image, thereby producing two contours. These two
contours are displayed, centered about a common value, in a different
field of view from either of the two images. Superimposing the contours
of objects from the image in a common field of view allows differences
between the contours (and thus, the objects the contours represent) to be
assessed visually and relatively quickly.

[0007]In general, in one aspect, the invention relates to a
computer-implemented method. The method involves calculating a first
centroid or center-of-mass of a first contour that is extracted from a
first image that depicts a moving object as the object appeared at a
first time. The method also involves calculating a second centroid or
center-of-mass of a second contour that is extracted from a second image
that depicts the object as the object appeared at a second time. The
second image depicts the image in substantially the same plane as the
first image. The method also involves displaying the first and second
contours with the respective first and second centroids or
centers-of-mass positioned at a common location.

[0008]In some embodiments, the first and second contours are extracted
automatically by a contour-finding algorithm operating on the respective
first and second images. The first and second images can be ultrasound
images. In some embodiments, the moving object is a chamber of a human
heart (e.g., a ventricle or ventricle wall), and the ultrasound images
can be transgastric short axis views of the chamber.

[0009]In some embodiments, the method further involves associating a first
identifying feature with the first contour and associating a second
identifying feature with the second contour. The identifying features can
be, for example, colors, line thickness, or types of lines. Some
embodiments of the invention feature the first and second contours being
associated with a left ventricle end diastolic area (LVEDA) or a left
ventricle end systolic area (LVESA) of a human heart. The first and
second contours, in some implementations, are displayed over a background
pattern that is different from a first or second background pattern of
the respective first or second images. An example of a background pattern
is a solid color (e.g., black).

[0010]Some implementations involve a method where displaying the first and
second images involves determining a value for the common location,
transforming a first set of data representative of the first contour
using the value of the common location, and illustrating the first
contour based on the first set of transformed data. Similarly, a second
set of data representative of the second contour is transformed using the
value of the common location, and the second contour is illustrated based
on the value of the transformed data. Illustrating the first and second
contours can involve processing respective first and second cineloops to
display the contours within a user interface window. In some embodiments,
the method also involves displaying the first and second images in a
first display window and displaying the first and second contours
(superimposed) in a second display window.

[0011]In another aspect, the invention relates to a system. The system
includes a computing apparatus that includes a processor, a memory, and
an image capturing module. The system also includes a calculation module
to calculate a first centroid or center-of-mass of a first contour
extracted from a first image that depicts a moving object as the object
appeared at a first time and to calculate a second centroid or
center-of-mass of a second contour extracted from a second image that
depicts the object as the object appeared at a second time. The second
image depicts the object in substantially the same plane as the first
image. The system also includes a first display window in communication
with the processor to display images from the image capturing module and
a second display window in communication with the processor and the
calculation module to display the first and second contours of the object
with the respective first and second centroids or centers-of-mass
positioned at a common location.

[0012]In some embodiments, the system includes a transesophageal
echocardiography probe (e.g., an imaging probe) in communication with the
processor. The system can also include a transformation module to
transform the first and second contours based on the value of the common
location. In the first display window, the first and second images can be
displayed side-by-side. Some implementations feature the first and second
contour displayed superimposed in the second display window. The second
display window can be superimposed over the first display window. The
second display window can also be displayed in response to a command by
or issued by a user. Such a command is available in response to a first
cineloop and a second cineloop being populated with a respective first
set of data associated with the first contour and a second set of data
associated with the second contour.

[0013]In still another aspect, the invention relates to a computer program
product, tangibly embodied in an information carrier. The computer
program product includes instructions operable to cause data processing
apparatus to calculate a first centroid or center-of-mass of a first
contour extracted from a first image that depicts a moving object as the
object appeared at a first time and to calculate a second centroid or
center-of-mass of a second contour extracted from a second image that
depicts the object as the object appeared at a second time. The second
image depicts the object in substantially the same plane as the first
image. The computer program product also includes instructions operable
to cause data processing apparatus to display the first and second
contours of the object with the respective first and second centroids or
centers-of-mass positioned at a common location.

[0014]In some implementations, any of the above aspects can include all of
the above features and/or advantages. These and other features will be
more fully understood by reference to the following description and
drawings, which are illustrative and not necessarily to scale.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015]FIG. 1 depicts a block diagram of a system that embodies the
invention.

[0016]FIG. 2 is an exemplary display window illustrating images and
superimposed contours of an object shown in the images.

[0017]FIG. 3 is an enlarged view of the display window of FIG. 2,
illustrating superimposed contours of the object.

[0018]FIG. 4A is a flow chart illustrating an exemplary process by which
the display window of FIG. 3 is displayed.

[0019]FIG. 4B is an exemplary user interface of a prompt that embodies and
illustrates the process of FIG. 4A.

[0021]FIG. 6 is a flow chart illustrating a process for displaying image
contours with a common centroid or center-of-mass.

DETAILED DESCRIPTION

[0022]FIG. 1 depicts a block diagram of a system 100 that embodies the
invention. The system 100 includes a computing apparatus or computer 105
that is in communication with a display 110. The computer 105 includes a
processor 115, a memory 120, and an image capturing module 125. The image
capturing module 125 is in communication with the memory 120 and the
display 110 for storing and displaying images (and data associated
therewith), respectively.

[0023]The system includes an imaging probe 130 in communication with the
processor 115. The imaging probe 130 can be an ultrasound probe. In some
embodiments, the imaging probe is a transesophageal echocardiography
probe. An example of a suitable transesophageal echocardiography probe is
disclosed in U.S. patent application Ser. Nos. 10/997,059 and 10/996,816,
each titled "Transesophageal Ultrasound Using a Narrow Probe" by Roth et
al., filed Nov. 24, 2004, the entire contents of which are hereby
incorporated herein by reference.

[0024]The system 100 includes an input device 135 in communication with
the processor 115 for communicating information to the processor 115. For
example, the input device 135 can include a mouse or pointer to identify
and mark information displayed on the display 110, and the information
can be used for further processing, as discussed in more detail below.
The input device 135 can also control the processor 115, memory 120,
image capturing module 125, and/or the display 110 by facilitating a user
issuing commands the processor 115 carries out.

[0025]The image capturing module 125 collects and processes image data
from the imaging probe 130 via the processor 115. For example, the image
capturing module 125 captures still images and/or video images (e.g.,
sequences of still images at a particular shutter speed). The processor
115 can store information captured by the image capturing module 125 in
the memory 120, e.g., as a data file, an image file, a cineloop file, or
other type of suitable data structure. A cineloop file is a file
containing multiple images, stored digitally as a sequence of individual
frames. In some embodiments, the number of images or frames in a cineloop
is pre-determined (e.g., 150 frames), but this is not required.

[0026]The processor 115 can include sub-modules or sub-routines for
carrying out, implementing, or performing various operations and tasks.
As illustrated, the processor 115 includes a calculation module 140 and a
transformation module 145 in communication with the calculation module
140. The processor 115 utilizes the calculation module 140 and the
transformation module 145 to facilitate display of image contours on the
display 110, as discussed in more detail below.

[0027]The display 110 displays different types of information about the
system 100. For example, the display 110 displays image data captured
from the imaging probe 130 by the image capturing module 125. The display
110 also displays information calculated or determined by the processor
115, the calculation module 140, and/or the transformation module 145.
The display 110 can also display information from the input device 135.
In some embodiments, the display 110 displays a cursor, landmarks, and/or
menus called or selected by the input device 135. In some embodiments,
the display 110 features a user interface, e.g., a Windows-based user
interface. Those of skill will appreciate other user interfaces suitable
for carrying out the principles described herein.

[0028]FIG. 2 is an exemplary display window 200 illustrating image data
and superimposed contours of an object within the images. In some
embodiments, the window 200 is displayed, for example, on the display 110
of the system 100 of FIG. 1. The window 200 includes a first area 204a
and a second area 204b. The first area 204a and the second area 204b are
displayed side-by-side in the window 200. The first area 204a includes a
first image 208a, and the second area 204b includes a second image 208b.

[0029]As illustrated, the first image 208a depicts a moving object (e.g.,
a patient's human heart) as the object appeared at a first time t1.
The first area 204a displays the time t1 within the region 212a of
the first area 204a. The time t1 at which the first image 208a
depicts the human heart is labeled "11:42:53 AM" on "Nov. 5, 2007" as
shown in the first region 212a. The second image 208b depicts the moving
object (e.g., the same human heart depicted in the first image 208a) as
the object appeared at a second time t2. The second area 204b
displays the time t2 within the region 212b of the second area 204b.
The time t2 at which the second image 208b depicts the human heart
is labeled "03:14:05 PM" on "Nov. 6, 2007" as shown in the second region
212b. Though FIG. 2 is described with reference to a human heart, one of
skill will appreciate that the concepts described herein apply to other
types of images or images of other objects as well.

[0030]The first image 208a and the second image 208b represent a
projection of the object in substantially the same plane, which allows,
for example, meaningful comparison between the images. In some
embodiments, an imaging probe (e.g., the imaging probe 130 of FIG. 1)
emits a fan- or conical-shaped beam (not shown) such that the plane of
the beam defines the image plane of the images 208a-208b. Where the
imaging probe is aimed upwardly through the left ventricle of the human
heart (e.g., by positioning the imaging probe in the fundus of a human
stomach), the imaging probe can provide a trans-gastric short axis view
of the heart represented in the images 208a-208b. The trans-gastric short
axis view of the left ventricle is useful for monitoring the heart's
operation because the view allows medical personnel to directly visualize
the left ventricle, which is the main pumping chamber of the heart.
Positioning the imaging probe in other locations, as would be apparent to
one of skill, allows visualization of the majority of the relevant
cardiac anatomy (e.g., other chambers of the heart, blood vessels, and
chest cavity). The positioning of the imaging probe does not affect the
operation of the principles herein since the images 208a-208b depict the
object projected onto substantially the same image plane.

[0031]A comparison between the images of the human heart at time t1
and time t2 can provide valuable information, for example, to a
healthcare professional such as a doctor. For example, the comparative
information can be used to assess the efficacy or effectiveness of a
course of treatment and/or to monitor the heart's reaction to
administration of treatment. However, the first image 208a and the second
image 208b include less relevant features (or even noise) associated with
ultrasonic imaging and/or electromagnetic phenomena.

[0032]One way to effectively compare the first image 208a of the patient's
heart with the second image of the patient's heart 208b is to determine a
contour of the heart appearing in the first and second images 208a-208b.
The first image 208a includes a first contour 216a that approximates the
outline or contour of the patient's heart (or chamber or portion
thereof), as the heart appeared at the time t1 the first image 208a
was captured. The first contour 216a includes a set of interconnected
points 220a positioned relative to the first image 208a. The second image
208b includes a second contour 216b that approximates the outline or
contour of the patient's heart, as the heart appeared at the time t2
the second image was captured. Like the first contour 216a, the second
contour 216b includes a set of interconnected points 220b positioned
relative to the second image 208b.

[0033]The first set of points 220a is used to create or draw the first
contour 216a, e.g., by connecting the points 220a to form a first closed
polygon representative of the outline of the patient's heart in the first
image 208a, at time t1. The first closed polygon has N1
vertices, each vertex representative of one of the points 220a.
Similarly, the second set of points 220b is used to create or draw the
second contour 216b, e.g., by connecting the points 220b to form a second
closed polygon representative of the outline of the patient's heart in
the second image 208, at time t2. The second closed polygon has
N2 vertices, each vertex representative of one of the points 220b.
In some embodiments, the number of points 220a used to extract the first
contour 216a (e.g., N1) is the same as the number of points 220b
used to extract the second contour 216b (N2), but this is not
required.

[0034]In some embodiments, the location (and thus coordinate values) of
the first and second sets of points 220a-220b are determined by a user.
For example, an ultrasound technician or healthcare provider can use the
input device 135 of the system 100 of FIG. 1 to identify the first and
second sets of points 220a-220b (e.g., by "clicking" on the image to
position the points). In some embodiments, the ultrasound technician
manually connects the points 220a-220b within the respective first and
second sets, e.g., using the input device 135 to draw lines between
points or by allowing the user to drag a point to its location on the
image. Alternatively, an algorithm can connect the points 220a-220b
within the respective first and second sets, e.g., using a connection
algorithm that draws lines (e.g., straight-line segments) between
adjacent points until the algorithm generates a closed polygon where the
first vertex (x1, y1) is also the Nth vertex (xN,
yN).

[0035]In some embodiments, the need for an ultrasound technician or
healthcare professional to determine of the sets of points 220a-220b that
generates the first and second contours 216a-216b is eliminated by using
an automated contour-finding algorithm operating on the respective first
and second images 208a-208b. For example, an edge or feature detector can
be applied to the first and second images 208a-208b determine sets of the
respective contours 216a-216b, either before or after the first or second
image 208a-208b is filtered or sub-sampled if such pre-processing steps
occur. The algorithm can operate either by locating the set of points
220a-220b within the respective images 208a-208b and drawing the
respective contours 216a-216b from the points 220a-220b, or,
alternatively, by drawing the respective contours 216a-216b and
locating/selecting points along the contours 216a-216b to populate the
respective sets 220a-220b. A combination of these algorithms is also
contemplated.

[0036]The sets of points 220a-220b are, in some embodiments, data
structures that can be stored (e.g., in the memory 120 of the system 100
of FIG. 1). In some embodiments, the sets of points 220a-220b are
associated with the respective images 208a-208b, e.g., as cineloop data.
In some embodiments, a data extraction algorithm generates a separate
data structure representative of the first and second contours 216a-216b
based on the sets of points 220a-220b, and the data structure
representative of the first and second contours 216a-216b is stored
(e.g., rather than points 220a-220b themselves). According to any of the
described examples, the first and second contours 216a-216b are extracted
from the respective first and second images 208a-208b for subsequent
processing.

[0037]After the contours 216a-216b have been extracted from the respective
images 208a-208b, the centroid or center-of-mass (not shown) for each
contour is calculated, as discussed in more detail below. The window 200
includes a second window 230. The second window 230 displays the first
contour 216a and the second contour 216b with the respective centroids or
centers-of-mass positioned/aligned at the common location. The second
window 230 is discussed in more detail below with respect to FIG. 3. The
second window 230 is superimposed over the first and second images
208a-208b in the window 200.

[0038]FIG. 3 is an enlarged view of the display window 230 of FIG. 2,
illustrating superimposed contours 216a-216b of the object. The window
230 includes a background 234 on which the contours 216a-216b are
displayed. As illustrated, the background 234 is a solid color, e.g.,
black. Presenting the background 234 as a solid color provides greater
contrast with the contours 216a-216b than the background (not shown) of
the respective images 208a-208b. The greater contrast is due, in part, to
the absence in the background 234 of the less relevant features present
in the images 208a-208b. After the contours 216a-216b have been extracted
from the respective images 208a-208b, the centroid or center-of-mass (not
shown) of each contour 216a-216b is determined, as discussed in more
detail with respect to FIGS. 5-6.

[0039]Within the window 230, the centroid or center-of-mass of the first
contour 216a is positioned at a common location, illustrated as 238. The
centroid or center-of-mass of the second contour 216b is also positioned
at the common location 238. Aligning the centroids or centers-of-mass of
the first and second contours 216a-216b allows the contours 216a-216b to
be displayed superimposed relative to each other in the window 230.
Superimposing the contours 216a-216b using the same location for the
centroid or center-of-mass for each contour 216a-216b allows for quick
visual comparison between the contours 216a-216b, and correspondingly,
between the images 208a-208b of the moving object as it appeared at time
t1 and time t2.

[0040]In some embodiments, the first contour 216a and/or the second
contour 216b are transformed to a common scale. Examples of
transformations include scale transformations, aspect transformations,
skew transformations, translation transformations, rotational
transformations, or any combination thereof of the contours 216a-216b.
Some of these transformations fall into a category of transformations
known as affine transformations.

[0041]The window 230 features a description of the view or plane
represented by the contours 216a-216b. For example, as illustrated, the
contours 216a-216b represent an end diastole transgastric short axis view
("TGSAV") of the left ventricle (or ventricular wall) of a moving heart,
as indicated by the words "End Diastole" 242 in the window 230. The
window 230 also includes an identifying characteristic or feature 246a
associated with the first contour 216a. The feature 246a in window 230a
indicates "Left" and the text is represented as a particular color that
corresponds to a line color with which the first contour 216a is
displayed. The "Left" designation refers to the relative position of the
image 208a from which the contour 216a was extracted. Specifically,
"Left" indicates that the contour 216a represents the image 208a in the
first area 204a of the window 200 of FIG. 2 (i.e., the image on the
left-side of the side-by-side display).

[0042]Similarly, the window includes an identifying characteristic or
feature 246b associated with the second contour 216b. The feature 246b in
window 230 indicates "Right" and a particular color (different from the
color associated with feature 246a) that corresponds to a line color with
which the second contour 216b is displayed in the window 230. The "Right"
designation refers to the relative position of the image 208b from which
the contour 216b was extracted; specifically, "Right" indicates that the
contour 216b represents the image 208b in the second area 204b of the
window 200 in FIG. 2.

[0043]In some embodiments, the features 246a-246b are not line color, but
some other characteristic for visually distinguishing the respective
contours 216a-216b and/or indicating a correspondence between the contour
216a-216b and the underlying image 208a-208b from which the respective
contour 216a-216b was extracted. Examples of suitable identifying
characteristics include line-type (e.g., dotted or dashed, of varying
lengths and frequency) or line thickness. In another implementation, the
contours 216a-216b can be distinguished using labeled arrows (not shown)
pointing to the contour 216a or 216b that corresponds to the respective
image 208a-208b.

[0044]The window 230 also includes a first button 250a and a second button
250b. The first button 250a is labeled "ED," which is representative of
"End Diastole." The second button 250b is labeled "ES," which is
representative of "End Systole." A user (not shown) can toggle between
the end diastole view and the end systole view of the heart by selecting
the first button 250a or the second button 250b, respectively. In some
embodiments, if sufficient data does not exist to populate the window 230
for a particular view (e.g., the end systole view), the respective button
(e.g., the second button 250b) will not be available for user selection.
The availability of the first or second button 250a-250b for user
selection indicates sufficient data exists to superimpose the contours
216a-216b of the end diastole view or the contours (not shown) of the end
systole view in the window.

[0045]In some embodiments, the first and second contours 216a-216b are
extracted from a sequence or series of images, rather than from a single
image 208a-208b. Accordingly, the first and second contours 216a-216b can
change shape and evolve over time, appearing as an animation. For each
such evolution or change to the contours 216a-216b, the centroid or
center-of-mass is calculated and positioned at the common location 238.
In such an embodiment, the first and second contours 216a-216b will give
the appearance of moving (e.g., via animation or time-evolution) in the
window 230, while the centroid or center-of-mass of each moving centroid
appears at the same location 238 in the window. Such embodiments
facilitate comparison between moving images of the object and permit
assessment of the time evolution of the object, e.g., to visually
determine a rate of volumetric change in the ventricle of a heart over a
period of time T1 relative to a second period of time T2.

[0046]FIG. 4A is a flow chart 400 illustrating an exemplary process by
which the display window 230 of FIG. 3 is displayed. In general, the
window 230 of FIG. 3 is displayed in response to a user command or
directive via, e.g., a user menu or prompt. The window 230 is populated
with contours (e.g., contours 216a-216b) extracted from images (e.g.,
images 208a-208b). To illustrate the contours 216a-216b in the window,
sufficient data about the extracted contour 216a-216b is used to control
the display (e.g., data associated with the contour is accessed by a
display algorithm to illustrate the contour on the background 234 in the
window 230).

[0047]At step 405, the process queries and determines whether image data
and contour data exists for a first image and contour, for example, as
the currently displayed cineloop or as a previously-displayed and stored
cineloop. If image and/or contour data for the first image and/or contour
does not exist, the prompt for displaying the window 230 is unavailable
(step 410). If, on the other hand, the first image and/or contour data
does exist, the process (step 415) then queries and determines whether
image data and contour data for a second image and contour exists. If
image and/or contour data for the second image and/or contour does not
exist, the prompt for displaying the window 230 is unavailable (step
410). If data representative of both the first and second images and the
first and second contours exists, the user prompt is available (step
420).

[0048]When the user prompt is available (step 420), the user can select
whether to open the window 230 (step 425) by, for example, issuing a
command through an input device such as the input device 135 of the
system 100 of FIG. 1. If the user does not issue a command, query or
request (step 425), the process is terminated (step 430), and the window
230 is not displayed. If the user, in response to prompt, opts to open
the window 230 (step 425), the user indicates the preference, and the
window 230 opens (step 435). Upon the window opening, the data (e.g.,
cineloop data) associated with the first and second contours (e.g.,
contours 216a-216b) is loaded and displayed in the window 230. When the
window 230 opens, the window is superimposed over the existing display
(e.g., over the window 200 of FIG. 2).

[0049]FIG. 4B is an exemplary user interface 450 of a prompt that embodies
the process of FIG. 4A. The user interface 450 includes a command 455
called "Compare ED/ES" that corresponds to step 420 of the process of
FIG. 4A. "Compare" is associated with a visual comparison between image
contours, while "ED" refers to the end diastole view and "ES" refers to
the end systole view, as discussed above. In some embodiments, the user
interface 450 appears in response to action by the user (e.g., a right
mouse-click). The user interface 450 can also appear automatically in
response to the existence of data associated with the first and second
images and first and second contours. Similarly, in the absence of data
associated with either the first or second image or first or second
contour (e.g., step 410 of FIG. 4A), the command 455 is unavailable for
selection by the user.

[0050]FIG. 5 is a flow chart 500 illustrating an exemplary method for
displaying superimposed image contours. The method involves capturing a
first image at a first time ti (step 504). As discussed above, a first
contour of an object of interest (e.g., a moving object, captured as the
object appeared at time t1) is extracted from the first image (step
508). After the contour is extracted, the centroid or center-of-mass of
the first contour is determined or calculated (step 512). Optionally,
data associated with the centroid or center-of-mass and the first contour
is stored in a computer memory (step 516).

[0051]The centroid or center-of-mass of the first contour is determined
using a centroid-finding or center-finding algorithm applied to the
contour. As discussed above, the first contour includes a set of points
of the image. In some embodiments, the points are pixel addresses or
values in a pixel grid, and as such, are represented as an ordered pair
or coordinate value (xi, yi). The contour forms a closed
polygon of connected vertices where each vertex is a point or ordered
coordinate pair in the set of points that define the contour.

[0052]Various algorithms for determining the centroid of a closed polygon
can be used. In some embodiments, the area of the polygon (e.g., the
first contour) is calculated, and the area is used to determine the
coordinates of the centroid or center-of-mass (e.g., by treating the
polygon as a plate of uniform mass density).

[0053]The area, A, of a non-overlapping, closed polygon that is defined by
N vertices (xi, yi), is governed by Equation 1, below. In
Equation 1, the Nth or notional vertex (e.g., (xN, yN)) is set
equal to the zeroth vertex (e.g., (x0, y0)), indicating a
closed polygon.

A = 1 2 i = 0 N - 1 x i y i + 1 - x i +
1 y i Equation 1 ##EQU00001##

[0054]After the area, A, of the contour has been determined, the x and y
coordinates of the centroid or center-of-mass (Cx, Cy) are
determined using Equations 2A and 2B, where "A" in Equations 2A and 2B
are determined from Equation 1. Like Equation 1, the Nth or notional
vertex (e.g., (xN, yN)) is set equal to the zeroth vertex
(e.g., (x0, y0)).

[0055]Mathematically, a centroid (or barycenter) of an object in an
n-dimensional space represents the intersection of all hyperplanes that
divide the object into two parts of equal moment about the hyperplane.
The centroid is sometimes referred to as the "average" of all points in
the object. In situations in which the object has a uniform density, the
location of the centroid coincides with the object's center-of-mass. As
discussed herein, the "centroid" and "center-of-mass" are used
interchangeably since the calculation involves determination of a
centroid of a polygonal shape that is deemed or assumed to have uniform
mass density.

[0056]In some embodiments, the centroid or center-of-mass is calculated
using the so-called "triangle method." According to the triangle method,
the polygon is partitioned into triangles (or "triangulated"), each
having three vertices: (x1, y1), (x2, y2), and
(x3, y3). The centroid of each triangle is the average of the
vertices, so the coordinate of the centroid (xc, yc) is given
by Equation 3:

[0057]The coordinate of the centroid (xc, yc) of each of the
triangles into which the polygon was partitioned is calculated. The
coordinates of the centroids are added or summed, weighted by the area of
each triangle. Specifically, the abscissa (xc) values are summed and
weighted to determine an abscissa value for the centroid (xc), and
the ordinate (yc) values are summed and weighted to determine an
ordinate value for the centroid (yc). The centroid's abscissa and
ordinate values are then normalized using the total polygonal area to
determine the coordinate of the centroid (Cx, Cy). In some
embodiments, the coordinate of the centroid is rounded to the nearest
pixel value for each of the abscissa and/or ordinate. The coordinate can
also be rounded to the nearest sub-pixel value where pixels are
partitioned (e.g., in quarter-pixel increments). In some embodiments, the
coordinate of the centroid is a real-valued number and is not rounded to
the nearest integer pixel.

[0058]The method involves capturing a second image at a second time
t2 (step 520). A second contour is extracted from the second image
(step 524) as discussed above. Similarly, a second centroid of the second
contour is calculated and determined (step 528) (e.g., using one of the
centroid-finding techniques discussed above). Optionally, data associated
with the second contour and centroid can be stored (step 532).

[0059]In some embodiments, the steps of the process can be completed in a
different order. For example, extracting the first contour (step 508)
does not necessarily precede capturing the second image at the second
time t2. Specifically, the contours can be extracted (step 508 and
524) near the same time, after both images have been captured (steps 504
and 520).

[0060]After the centroids for the first and second contours have been
determined, the position of a common location for displaying the contours
is determined (step 536). In some embodiments, the common location is
selected to be an integer pixel coordinate or sub-pixel coordinate in a
pixel grid that will be displayed in a window (e.g., the window 230 of
FIG. 2). The common location can also be a real-valued coordinate pair.
The common location is used as a reference point or reference value for
both the first contour and the second contour. As discussed above, when
the first and second contours are displayed in the window, the respective
first and second centroids (or centers-of-mass) are positioned at the
common location. The first and second contours are redrawn or illustrated
in the window centered about the common location since the respective
centroids are aligned at the central location (steps 540 and 544). In
some embodiments, the centroids are assigned a value that is the same as
the value of the common location, and the values of the points that
define the contour are transformed based on the value of the common
location.

[0061]FIG. 6 is a flow chart 600 illustrating a process for displaying
image contours with a common centroid or center-of-mass. The process
involves determining the position of the common location (step 536 of
FIG. 5). After the position of the common location has been determined,
data representative of the first contour is transformed based on the
coordinate value of the common location (step 604). In some embodiments,
each point in the set of points that define the contour are associated
with a vector that identifies a distance and a direction of the
particular point relative to the centroid. When the centroid (Cx,
Cy) is positioned at the common location (xCL, yCL), the
centroid undergoes a linear transformation L (xCL-Cx,
yCL-Cy) that relates the common location to the centroid. The
linear transformation L can be applied, e.g., via matrix transformation,
to each of the vectors representative of the points defining the first
contour. In some embodiments, more complicated transformations can be
applied to the set of vectors, e.g., scale transformations, rotational
transformations, skew transformations, aspect transformations, or
combinations of these. The transformed contour can be stored for later
use, or subsequently processed and/or displayed without storage. Other
representations of the points that define a contour (e.g., other than a
vector representation) will be apparent to one of skill.

[0062]After the vectors associated with the points defining the first
contour have been transformed, the first centroid is positioned at the
common location (step 608), and the first contour is illustrated,
displayed, drawn, or redrawn with the first centroid at the common
location (step 612). Specifically, the transformed data is used to
illustrate the contour in the window about the common location (e.g.,
rather than using the nontransformed data).

[0063]A similar operation occurs with respect to the second contour.
Specifically, data associated with or representative of the second
contour is transformed based on information about the common location
(step 616). The second centroid is positioned at the common location
(step 620), and the second contour is illustrated, displayed, drawn, or
redrawn with the second centroid at the common location (step 624), based
on the transformed data. In this way, the image contours can be displayed
in a common field of view (e.g., against a solid-color background) in a
window for quick assessment and comparison. In some embodiments,
illustrating the first and second contours (steps 612 and 624) involves
processing a cineloop associated with the respective contours. The
cineloops can have the transformation data stored or associated
therewith, or the transformation calculation can be implemented
on-the-fly as the contour is illustrated.

[0064]In some embodiments, the transformation steps (steps 604 and 616)
are carried out by a transformation module (e.g., the transformation
module 145 of FIG. 1), based on the value of the common location.

[0065]The above-described techniques can be implemented in digital
electronic circuitry, or in computer hardware, firmware, software, or in
combinations of them. The implementation can be as a computer program
product, e.g., a computer program tangibly embodied in an information
carrier, e.g., in a machine-readable storage device or in a propagated
signal, for execution by, or to control the operation of, data processing
apparatus, e.g., a programmable processor, a computer, or multiple
computers. A computer program can be written in any form of programming
language, including compiled or interpreted languages, and it can be
deployed in any form, including as a stand-alone program or as a module,
component, subroutine, or other unit suitable for use in a computing
environment. A computer program can be deployed to be executed on one
computer or on multiple computers at one site or distributed across
multiple sites and interconnected by a communication network.

[0066]Method steps can be performed by one or more programmable processors
executing a computer program to perform functions of the technology by
operating on input data and generating output. Method steps can also be
performed by, and apparatus can be implemented as, special purpose logic
circuitry, e.g., an FPGA (field programmable gate array) or an ASIC
(application-specific integrated circuit). Modules can refer to portions
of the computer program and/or the processor/special circuitry that
implements that functionality.

[0067]Processors suitable for the execution of a computer program include,
by way of example, both general and special purpose microprocessors, and
any one or more processors of any kind of digital computer. Generally, a
processor receives instructions and data from a read-only memory or a
random access memory or both. The essential elements of a computer are a
processor for executing instructions and one or more memory devices for
storing instructions and data. Generally, a computer will also include,
or be operatively coupled to receive data from or transfer data to, or
both, one or more mass storage devices for storing data, e.g., magnetic,
magneto-optical disks, or optical disks. Data transmission and
instructions can also occur over a communications network. Information
carriers suitable for embodying computer program instructions and data
include all forms of non-volatile memory, including by way of example
semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory
devices; magnetic disks, e.g., internal hard disks or removable disks;
magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and
the memory can be supplemented by, or incorporated in special purpose
logic circuitry.

[0068]The terms "module" and "function," as used herein, mean, but are not
limited to, a software or hardware component which performs certain
tasks. A module may advantageously be configured to reside on addressable
storage medium and configured to execute on one or more processors. A
module may be fully or partially implemented with a general purpose
integrated circuit ("IC"), FPGA, or ASIC. Thus, a module may include, by
way of example, components, such as software components, object-oriented
software components, class components and task components, processes,
functions, attributes, procedures, subroutines, segments of program code,
drivers, firmware, microcode, circuitry, data, databases, data
structures, tables, arrays, and variables. The functionality provided for
in the components and modules may be combined into fewer components and
modules or further separated into additional components and modules.
Additionally, the components and modules may advantageously be
implemented on many different platforms, including computers, computer
servers, data communications infrastructure equipment such as
application-enabled switches or routers, or telecommunications
infrastructure equipment, such as public or private telephone switches or
private branch exchanges ("PBX"). In any of these cases, implementation
may be achieved either by writing applications that are native to the
chosen platform, or by interfacing the platform to one or more external
application engines.

[0069]To provide for interaction with a user, the above described
techniques can be implemented on a computer having a display device,
e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor,
for displaying information to the user and a keyboard and a pointing
device, e.g., a mouse or a trackball, by which the user can provide input
to the computer (e.g., interact with a user interface element). Other
kinds of devices can be used to provide for interaction with a user as
well; for example, feedback provided to the user can be any form of
sensory feedback, e.g., visual feedback, auditory feedback, or tactile
feedback; and input from the user can be received in any form, including
acoustic, speech, or tactile input.

[0070]The above described techniques can be implemented in a distributed
computing system that includes a back-end component, e.g., as a data
server, and/or a middleware component, e.g., an application server,
and/or a front-end component, e.g., a client computer having a graphical
user interface and/or a Web browser through which a user can interact
with an example implementation, or any combination of such back-end,
middleware, or front-end components. The components of the system can be
interconnected by any form or medium of digital data communications,
e.g., a communications network. Examples of communications networks, also
referred to as communications channels, include a local area network
("LAN") and a wide area network ("WAN"), e.g., the Internet, and include
both wired and wireless networks. In some examples, communications
networks can feature virtual networks or sub-networks such as a virtual
local area network ("VLAN"). Unless clearly indicated otherwise,
communications networks can also include all or a portion of the PSTN,
for example, a portion owned by a specific carrier.

[0071]The computing system can include clients and servers. A client and
server are generally remote from each other and typically interact
through a communications network. The relationship of client and server
arises by virtue of computer programs running on the respective computers
and having a client-server relationship to each other.

[0072]Various embodiments are depicted as in communication or connected by
one or more communication paths. A communication path is not limited to a
particular medium of transferring data. Information can be transmitted
over a communication path using electrical, optical, acoustical,
physical, thermal signals, or any combination thereof. A communication
path can include multiple communication channels, for example,
multiplexed channels of the same or varying capacities for data flow.

[0073]Multiple user inputs can be used to configure parameters of the
depicted user interface features. Examples of such inputs include
buttons, radio buttons, icons, check boxes, combo boxes, menus, text
boxes, tooltips, toggle switches, buttons, scroll bars, toolbars, status
bars, windows, or other suitable icons or widgets associated with user
interfaces for allowing a user to communicate with and/or provide data to
any of the modules or systems described herein. In some embodiments, a
touchscreen is used to facilitate an interface or interaction between
components hereof and the user.

[0074]While the invention has been particularly shown and described with
reference to specific embodiments, it should be understood by those
skilled in the art that various changes in form and detail may be made
therein without departing from the spirit and scope of the invention as
defined by the appended claims.