Sign up to receive free email alerts when patent applications with chosen keywords are publishedSIGN UP

Abstract:

Various embodiments herein provide for imager focusing based on
intraoperative data. Ideally, an imaging plane of an ultrasound
transducer would be truly planar. That is not the case, though. Instead,
ultrasound transducers image a volume that is closer to a rectangular
volume, but that has focal depths or areas in the imaged volume that are
"thinner" and provide a better resolution. In general, embodiments herein
may include determining the pose of an imager, such as an ultrasound
transducer, and the pose of a location of interest. Based on those poses,
a focal adjustment may be determined in order to, for example, better
focus the imager on the object of interest. Then data is generated and
the focus of the imager is adjusted. Additionally, imaging data and/or
the object of interest may be displayed. In other embodiments, estimated
projections of medical devices are displayed to allow for better
intraoperative planning.

Claims:

1. A method for imager focusing based on intraoperative data, implemented
on one or more computing devices, comprising: determining pose
information for a focusable imager used in a medical scene; determining
pose information for at least one object of interest in the medical
scene; determining, using the one or more computing devices, a focal
adjustment for the focusable imager based on the pose information for the
at least one object of interest and the pose information for the imager;
and generating data to be sent to the focusable imager to adjust the
focus of the focal imager based on the determined focal adjustment for
the focusable imager.

2. The method of claim 1, wherein the method further comprises adjusting
focus of the focusable imager based on the data generated to adjust the
focus of the focal imager.

3. The method of claim 1, wherein the method further comprises displaying
a 3D graphics representation of the object of interest.

4. The method of claim 1, wherein determining the pose information for
the focusable imager comprises receiving tracking information for the
focusable imager

5. The method of claim 1, wherein determining the pose information for
the at least one object of interest comprises accessing stored pose
information for an object whose pose was previously indicated.

6. The method of claim 5, wherein accessing stored pose information for
an already-known object of interest comprises accessing stored pose
information for a highlight, marking, or annotation of an already-known
object of interest.

7. The method of claim 1, wherein determining the pose for the at least
one object of interest comprises receiving tracking information for at
least one device associated with the object of interest.

8. The method of claim 7, wherein the at least one device comprises a
tracked ablation needle.

9. The method of claim 1, wherein determining the pose for the at least
one object of interest comprises determining the pose of a body part of
an operator.

10. The method of claim 1, wherein determining the focal adjustment based
on the pose of the at least one object of interest comprises determining
a projection of a device onto an imaging plane associated with the
imager.

11. The method of claim 1, wherein determining the focal adjustment based
on the pose of the at least one object of interest comprises determining
a series of focal adjustments to be used over time based on the pose of
the at least one object of interest.

12. The method of claim 1, wherein determining the focal adjustment based
on the pose of the at least one object of interest comprises determining
a depth of focus for the focusable imager based on the pose of the at
least one object of interest.

13. The method of claim 1, wherein determining the focal adjustment based
on the pose of the at least one object of interest comprises determining
an angle for directing an imaging element of the focusable imager.

14. The method of claim 1, wherein the at least one object of interest
comprises two or more objects of interest.

15. The method of claim 14, wherein determining the focal adjustment
based on the pose of the at least one object of interest comprises
determining two or more focal adjustments based on the two or more
objects of interest.

16. A non-transient computer-readable medium comprising
computer-executable instructions for imager focusing based on
intraoperative data, said computer-executable instructions, when running
on one or more computing devices, performing a method comprising:
determining pose information for a focusable imager used in a medical
scene; determining pose information for at least one object of interest
in the medical scene; determining a focal adjustment for the focusable
imager based on the pose information for the at least one object of
interest and the pose information for the imager; and generating data to
be sent to the focusable imager to adjust the focus of the focal imager
based on the determined focal adjustment for the focusable imager.

17. The non-transient computer-readable medium of claim 16, wherein the
method further comprises displaying a 3D graphics representation of the
object of interest.

18. The method of claim 1, wherein determining the pose information for
the at least one object of interest comprises accessing stored pose
information for an object whose pose was previously indicated.

19. A system for imager focusing based on intraoperative data, comprising
one or more computing devices, said computing devices being configured
to: determine pose information for a focusable imager used in a medical
scene; determine pose information for at least one object of interest in
the medical scene; determine a focal adjustment for the focusable imager
based on the pose information for the at least one object of interest and
the pose information for the imager; and generate data to be sent to the
focusable imager to adjust the focus of the focal imager based on the
determined focal adjustment for the focusable imager.

20. The system of claim 19, wherein determining the pose for the at least
one object of interest comprises receiving tracking information for at
least one device associated with the object of interest, wherein the at
least one device comprises a tracked ablation needle.

21. The system of claim 19, wherein determining the focal adjustment
based on the pose of the at least one object of interest comprises
determining a projection of a device onto an imaging plane associated
with the imager.

22. The system of claim 19, wherein determining the focal adjustment
based on the pose of the at least one object of interest comprises
determining a series of focal adjustments to be used over time based on
the pose of the at least one object of interest.

Description:

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Application
No. 61/265,517, entitled "Interactive Control and Display of Ultrasound
Imaging Focus," filed Dec. 1, 2009 and U.S. Provisional Application No.
61/265,521, entitled "Medical Imaging and Needle Manipulation," filed
Dec. 1, 2009, each of which is incorporated by reference herein in its
entirety for all purposes.

BACKGROUND

[0002] Various medical imaging modalities are available, such as X-rays,
CTs, ultrasound, MRIs, etc. These imaging modalities may be used
intraoperatively, during the procedure, or preoperatively, before the
procedure starts. X-rays may be used before a procedure starts for
preoperative diagnosis--such as diagnosing a fracture of a bone.
Ultrasound may be used during a medical procedure to see or find
structures inside a patient's body. Consider, for example, the use of
ultrasound during prenatal analysis to view the fetus.

[0003] Many imagers used in medical procedures may have a focal depth.
That is, a depth at which they have better image quality or more focused
resolution. A problem with current systems is, however, that areas
outside the area of highest focus are produced at lower resolutions.
Issues with such imagers may result, such as multiple structures or
tissues appearing coincident in the image when they are not actually or
truly coincident "in real life", as discussed more below.

SUMMARY

[0004] Various embodiments of the systems, methods, computer-readable
storage media, and techniques described herein overcome some of these
shortcomings of the prior art and provide for imager focusing based on
intraoperative data.

[0005] Presented herein are methods, systems, devices, computer-readable
media, kits, compositions, techniques, and teachings for imager focusing
based on intraoperative data. This summary in no way limits the invention
herein, but instead is provided to summarize a few of the embodiments.
Embodiments include determining pose information for a focusable imager
used in a medical scene and pose information for at least one object of
interest in the medical scene (simultaneously or in any order). A focal
adjustment for the focusable imager is also determined, in some
embodiments, based on the pose information for the at least one object of
interest and the pose information for the imager. Data to be sent to the
focusable imager to adjust the focus of the focal imager may be
determined based on the determined focal adjustment for the focusable
imager. Numerous other embodiments are described throughout herein,
including embodiments where there is also adjusting of the focus of the
focusable imager based on the data generated to adjust the focus of the
focal imager and/or display of a 3D graphics representation of the object
of interest. Embodiments might also include receiving tracking
information for the focusable imager or other devices in the scene. In
some embodiments, determining the pose information for the at least one
object of interest includes accessing stored pose information for an
object whose pose was previously indicated (by, e.g., an operator marking
a tumor). Many additional embodiments are described below.

[0006] Many of the advantages of certain embodiments for imager focusing
are described herein. Of course, it is to be understood that not
necessarily all such advantages need to be achieved in accordance with
any particular embodiment. Thus, for example, those skilled in the art
will recognize that the invention may be embodied or carried out in a
manner that achieves or optimizes one advantage or group of advantages as
taught or suggested herein without necessarily achieving other advantages
as may be taught or suggested herein.

[0007] All of these embodiments are intended to be within the scope of the
invention herein disclosed. These and other embodiments will become
readily apparent to those skilled in the art from the following detailed
description having reference to the attached figures, the invention not
being limited to any particular disclosed embodiment(s).

[0022] Imagers typically have imaging planes. The term "imaging plane" as
used herein is a broad term and includes its plain and ordinary meaning,
and further includes, but is not limited to the plane, area, or volume
that is imaged by an imager. Ideally, the imaging plane of an ultrasound
transducer would be truly planar. That is not the case, though. Instead,
ultrasound transducers, and other imagers, image a volume that is closer
to a rectangular volume (as depicted by volumes 1056 in FIGS. 10A-10C).
Additionally, ultrasound transducers (and other imagers) have focal
depths or areas in the imaged volume that are "thinner" and provide a
better resolution. For example, looking to FIG. 1A, we see an abstract
depiction of the cross section of an ultrasound transducer 155 and the
volume of tissue 150 that is imaged. It is generally the case that the
volume of tissue 150 is wider in certain areas and narrower in other
areas. For example, the volume 150 of tissue that is imaged may be
narrower at a focal depth or focal area 120 and wider in other areas. The
ultrasound imager 155 may also include an imaging element 157. The
imaging element 157 may include a one-dimensional phased array
transducer, two-dimensional phased array transducer or any other imaging
element. In some embodiments, discussed more below, the phased array
transducer may be focusable. Focusing the imaging element 157 may change
the focal depth 120 of the area or volume 150 of tissue to be imaged.

[0023] As noted above, FIG. 1A depicts a cross section of an imager 155
and a volume 150 of tissue to be imaged. Also depicted in FIG. 1A is a
cross section of a first object 130, such as a blood vessel 130, and a
second object 140, such as an ablation needle 140. The blood vessel 130
and the ablation needle 140 are both within the volume of tissue 150 that
is captured by the imager 155. The image 156 (in FIG. 1C) produced by the
ultrasound system contains a cross section of the blood vessel 130 and
the ablation needle 140 seen together as if the ablation needle 140 is
inside the blood vessel 130. As is clear in FIG. 1A, however, the
ablation needle 140 is not inside the blood vessel 130, but because both
the blood vessel 130 and the ablation needle 140 are inside the imaging
volume 150, the image 156 produced collocates the two objects, and one
cannot decipher whether or not one object is inside the other.

[0024] FIG. 1B depicts a cross section of an imager 155 with an imaging
element 157 and an imaging volume 150. In FIG. 1B, the focal plane 120 is
through the ablation needle 140 and the blood vessel 130. Therefore, the
area of highest focus and resolution passes through the blood vessel 130.
Therefore, as depicted in FIG. 1D, the blood vessel 130 appears on the
image 156 produced by the imager, but the ablation needle 140 from FIG.
1B does not show up in the image 156, letting an operator of the
ultrasound transducer 155 (FIG. 1B) determine that the ablation needle
140 is not inside the blood vessel 130.

[0025] The various method, systems, techniques, and computer-readable
media disclosed herein allow for automatic refocusing of imagers, such as
ultrasounds, MRIs and the like, based on objects of interest within the
medical scene. The term "medical scene" as used herein is a broad term
and includes its plain and ordinary meaning, and further includes, but is
not limited to, the site or scene where a medical procedure takes place;
a medical site; a medical diagnostic and therapy site; and/or a virtual
representation of any of those sites or places. For example, as a surgeon
changes the projection of an ablation needle or other device or tool,
that projection's intersection with the imaging plane can be used to
define the focal plane of the imager. For example, in FIG. 4A there is a
patient undergoing a procedure with an ablation needle 445 and ultrasound
455 being used together. These are displayed on display 420 where we see
the 3D model of the ablation needle 446 and the image 456 produced by the
imager 455. Also visible thereon is an area or object of interest 461. In
some embodiments, the area or object of interest 461 may be defined by
the intersection of the projection of the ablation needle (or other tool)
and the imaging plane or volume of the ultrasound (or other imager). In
various embodiments herein, the area or object of interest 461 may be
used to define the focal plane of the imager. Therefore, whatever the
needle is pointing at will be in highest focus.

[0026] In various other embodiments, if the needle is not near the plane
of the imager, the projection of the needle may provide the guidance, as
depicted in FIG. 4B where the 3D model of the needle 446 is distant from
the plane of the imager 456, yet there is a target 461 at the
intersection of the ray (not pictured) extending from the ablation needle
446 to the image plane 456.

[0027] These techniques and teachings herein may be used with any kind of
device in any kind of operation. The term "device" as used herein is a
broad term and includes its plain and ordinary meaning, and further
includes, but is not limited to a medical instrument or an instrument or
tool usable in a medical procedure. For example, the device may be an
ablation needle, a catheter, cryoablation needle, CUSA dissection tool, a
cauterization tool, a knife or scalpel, or any other instrument or tool.
Further, the system could use, in some embodiments, a Nintendo Wii or
similar controller or use the operator's hand, finger, or eye gaze
direction where the hand, finger, or eye gaze are tracked or determined.
For example, an operator of the system could point to or look at the
image 456 on the display 420 or point to or look at a physical location
on the patient to indicate the target 461. Speech recognition could also
be used to allow an operator to indicate the location of a target 461.
The targets may also be obtained preoperatively and marked, annotated, or
highlighted (e.g., such as marking a tumor preoperatively on an MRI or CT
scan). Further, there may be multiple objects of interest.

[0028] In some embodiments, the imager may be an ultrasound, a cone bean
CT, an MRI, optical tomography, confocal microscopy, or any other
appropriate imager. The term "focusable imager," as used herein is a
broad term that encompasses the plain and ordinary meaning of the term,
including without limitation an ultrasound wand that can be focused to
have a particular focal depth, or a cone bean CT, MRI, optical
tomography, confocal microscopy or other device or tool that can be
focused.

[0029] The embodiments described herein can be used with preoperative
data, including imaging data, data from another procedure, data from
another concurrent procedure, and/or with different instrumentation or
devices. For example, preoperative data may be displayed along with the
imager data on a screen displayed to an operator. In some embodiments,
other operative tools or devices may be used and/or may be tracked and
displayed to the operator. Various other techniques, embodiments,
systems, methods, kits, and computer-readable media are described more
below.

System for Imager Focusing Based on Intraoperative Data

[0030] FIG. 2 depicts embodiments of a system 200 configured for imager
focusing based on intraoperative data. There are numerous other possible
embodiments of system 200. For example, numerous of the depicted modules
may be joined together to form a single module and may even be
implemented in a single computer, machine, or computing device. Further,
the position sensing units 210 and 240 may be combined and track all
relevant tracked units 245 and movable imaging units 255, as discussed in
more detail below. Tracking units may be attached to a medical device 245
(e.g., an ablation needle). Additionally, imaging unit 250 may be
excluded and only imaging data from the image guidance unit 230 may be
shown on display unit 220. These and other possible embodiments are
discussed in more detail below. Numerous other embodiments will be
apparent to those skilled in the art and are part of the embodiments
herein.

[0031] In the pictured embodiment, the system 200 comprises a first
position sensing unit 210, a display unit 220, and the second position
sensing unit 240 all coupled to an image guidance unit 230. In some
embodiments, the first position sensing unit 210, the displaying unit
220, the second position sensing unit 240, and the image guidance unit
230 are all physically connected to stand 270. The image guidance unit
230 may be used to produce images 225 that are displayed on display unit
220. As discussed more below, the images 225 produced on the display unit
220 by the image guidance unit 230 may be made based on imaging data,
such as a CT scan, MRI, open-magnet MRI, optical coherence tomography,
positron emission tomography ("PET") scans, fluoroscopy, ultrasound,
and/or other preoperative or intraoperative anatomical imaging data and
3D anatomical imaging data. The images 225 produced may also be based on
intraoperative or realtime data obtained using a movable imaging unit
255, which is coupled to imaging unit 250. The term "realtime" as used
herein is a broad term and has its ordinary and customary meaning,
including without limitation instantaneously or nearly instantaneously.
The use of the term realtime may also mean that actions are performed or
data is obtained with the intention to be used immediately, upon the next
cycle of a system or control loop, or any other appropriate meaning.

[0032] Imaging unit 250 may be coupled to image guidance unit 230. In some
embodiments, imaging unit 250 may be coupled to a second display unit
251. The second display unit 251 may display imaging data from imaging
unit 250. The imaging data displayed on display unit 220 and displayed on
second display unit 251 may be, but are not necessarily, the same. In
some embodiments, the imaging unit 250 is an ultrasound machine 250, the
movable imaging device 255 is an ultrasound transducer 255 or ultrasound
probe 255, and the second display unit 251 is a display associated with
the ultrasound machine 250 that displays the ultrasound images from the
ultrasound machine 250.

[0033] The first position sensing unit 210 may be used to track the
position of movable imaging unit 255. Tracking the position of movable
imaging unit 255 allows for the determination of the relative pose of
imaging data received using the movable imaging unit 255 and imaging unit
250 with that data being sent to image guidance unit 230. For example,
image guidance unit 230 may contain CT data which is being updated and
deformed based on the relative poses of tracking units as received by the
second position sensing unit 240. In such embodiments, the image guidance
unit 230 may take in the poses of the tracking units and, from the poses,
determine an updated 3D graphics stored in image guidance unit 230.
Further, image guidance unit 230 may produce images based on the current
ultrasound or other imaging data coming from imaging unit 250 and an
updated model determined based on the poses of tracking units. The images
produced 225 may be displayed on display unit 220. An example image 225
is shown in FIG. 2.

[0034] In some embodiments, a movable imaging unit 255 may not be
connected directly to an imagining unit 250, but may instead be connected
to image guidance unit 230. The movable imaging unit 255 may be useful
for allowing a user to indicate what portions of a first set of imaging
data should be displayed. For example, the movable imaging unit 255 may
be an ultrasound transducer 255 or a tracked operative needle or other
device 255, for example, and may be used by a user to indicate what
portions of imaging date, such as a pre-operative CT scan, to show on a
display unit 220 as image 225. Further, in some embodiments, there could
be a third set of pre-operative imaging data that could be displayed with
the first set of imaging data. Additionally, in some embodiments, each of
the first and third sets of imaging data could be deformed based on
updated positions of the tracking units and the updated, deformed
versions of the two sets of imaging data could be shown together or
otherwise provide image guidance images 225 for display on display 220.

[0035] First position sensing unit 210 may be an optical tracker, a
magnetic tracker, or any other appropriate type of position sensing
device. For example, in various embodiments, first position sensing unit
210 may be an Ascension Flock of Birds, Nest of Birds, driveBAY, medSAFE,
trakSTAR, miniBIRD, MotionSTAR, or pciBIRD. In some embodiments, the
first position sensing unit may be an Aurora® Electromagnetic
Measurement System using sensor coils. In some embodiments, the first
position sensing unit 210 may also be an optical 3D tracking system such
as the NDI Polaris Spectra, Vicra, Certus, PhaseSpace IMPULSE, Vicon MX,
InterSense IS-900, NaturalPoint OptiTrack, Polhemus FastTrak, IsoTrak, or
Claron MicronTracker2. In some embodiments, the first position sensing
unit 210 may also be an inertial 3D tracking system comprising a compass,
accelerometer, tilt sensor and/or gyro, such as the InterSense
InertiaCube. The first position sensing unit 210 may sense the position
of movable imaging unit 255. If first position sensing unit 210 is an
optical tracker, then movable imaging unit 255 may have fiducials placed
thereon to make visual position and/or orientation detection possible. If
first position sensing unit 210 is a magnetic tracker, then movable
imaging unit 255 they have placed thereon magnetic tracking units.

[0036] The second position sensing unit 240 and tracking units on tracked
device 245 the may together comprise a magnetic tracking system, an
optical tracking system, or any other appropriate tracking system. The
second position sensing unit 240 and tracking units may be used to track
a medical device 245, the deformation of tissue at a target anatomical
site on patient 260, or any other appropriate position or device. Patient
260 may be in an operating room, lying on an operating table, such as
operating table 280, or in any other appropriate place or position. In
various embodiments, second position sensing unit 240 may be an Ascension
Flock of Birds, Nest of Birds, driveBAY, medSAFE, trakSTAR, miniBIRD,
MotionSTAR, or pciBIRD and tracking units may be magnetic tracking coils.
In some embodiments, the second position sensing unit 240 may be an
Aurora® Electromagnetic Measurement System using sensor coils for
tracking units. In some embodiments, the second position sensing unit 240
may also be an optical 3D tracking system using fiducials as tracking
units. Such optical 3D tracking systems may include the NDI Polaris
Spectra, Vicra, Certus, PhaseSpace IMPULSE, Vicon MX, InterSense IS-900,
NaturalPoint OptiTrack, Polhemus FastTrak, IsoTrak, or Claron
MicronTracker2. In some embodiments, the second position sensing unit 240
may also be an inertial 3D tracking system comprising a compass,
accelerometer, tilt sensor and/or gyro, such as the InterSense
InertiaCube.

[0037] "Tracking unit" as used herein is a broad term encompassing its
plain and ordinary meaning and includes without limitation all types of
magnetic coils or other magnetic field sensing devices for use with
magnetic trackers, fiducials or other optically detectable markers for
use with optical trackers, such as those discussed above and below.
Tracking units could also include optical position sensing devices such
as the HiBall tracking system and the first and second position sensing
units 210 and 240 may be part of a HiBall tracking systems. Tracking
units may also include a GPS device or signal emitting device that would
allow for tracking of the position and, optionally, orientation of the
tracking unit. In some embodiments, a signal emitting device might
include a radio-frequency identifier (RFID). In such embodiments, the
first and/or second position sensing unit 210 and 240 may take in the GPS
coordinates of the tracking units or may, for example, triangulate the
radio frequency signal being emitted by the RFID associated with tracking
units.

[0038] In some embodiments, the display unit 220 displays 3D images to a
user. This can be accomplished by a stereoscopic display, a lenticular
display, or any other appropriate type of display. In some embodiments,
an operator may wear head-mounted display in order to receive 3D images
from the image guidance unit 230. In such embodiments, display unit 220
may be omitted.

[0039] In some undepicted embodiments, there is no first position sensing
unit 210 and the poses of both the movable imaging unit 255 and tracked
device 245 are determined using the second position sensing unit 240.
Similarly, in some embodiments, the first position sensing unit 210 may
track the poses of the movable imaging unit 255 and tracked device 245
and the second position sensing unit 240 may not be present. The image
guidance may also be performed at least in part using the techniques
described in U.S. patent application Ser. No. 11/828,826, filed Jul. 26,
2007, U.S. Pat. No. 7,728,868, U.S. patent application Ser. No.
12/399,899, U.S. patent application Ser. No. 12/483,099, U.S. patent
application Ser. No. 12/893,123, U.S. patent application Ser. No.
12/842,261, and/or U.S. patent application Ser. No. 12/703,118, each of
which is incorporated by reference herein in its entirety for all
purposes.

Processes and Methods for Imager Focusing Based on Intraoperative Data

[0040] FIG. 3 depicts embodiments of a process or method 300 for imager
focusing based on intraoperative data. In general, the method may include
determining the pose of an imager, such as an ultrasound transducer,
(block 310) and the pose of a location of interest (block 320). From
there, a focal adjustment may be determined (block 330) in order to, for
example, better focus the imager on the object of interest. Then data is
generated (block 340) and the focus of the imager may be adjusted (block
350). Additionally, imaging data (e.g., a visual representation of
position of the in-focus region, the image obtained by the imager, etc.)
and/or the object of interest may be displayed (block 360). In operation,
various of the blocks presented in process or method 300 in FIG. 3 may be
omitted, extra steps may be added, and steps may be performed in
different order.

[0041] In block 310, pose information for a focusable imager is
determined. Determining the pose information for a focusable imager may
include receiving tracker information on the pose, position, and/or
orientation of a focusable imager, such as imager 255 depicted in FIG. 2.
The term "pose information" as used herein includes its plain and
ordinary meaning, including position, orientation and/or a combination of
the two. "Pose information" can also mean location. As noted above, pose
information may be received via optical tracking, magnetic tracking, GPS,
triangulation, or any other technique.

[0042] In block 320, pose of at least one object of interest is
determined. In various embodiments, different objects may be of interest.
For example, the object of interest may be an ablation needle,
cauterizer, scalpel, catheter, or other device or tool. The pose
information may be determined via tracking information (e.g., from the
tracked device), as described above. The pose information may be used to
determine where an ablation needle, cauterizer or other device or tool is
pointing and the intersection of the device or tool's projection and the
imager plane. This projected intersection may be the object of interest.
For example, turning to FIG. 4A, we see an object of interest 461 as the
projection of the ablation needle 446 as it intersects with the imaging
plane 456.

[0043] In some embodiments, as an ablation needle (or other device or
tool) is moved, the projection of the ablation needle and its
intersection with the imager plane may also move. This may cause the
focal plane of the imager to move to follow the projection. For example,
if an operator, surgeon or other user would like to change the focal
depth of an imager, that person may be able to modify the pose of the
ablation needle in order to change the pose of the object of interest,
which is the intersection of the projection of the ablation needle with
the imager plane. As depicted in FIGS. 4A and 4B, in some embodiments,
the object of interest, namely the intersection of the projection of the
ablation needle and the imager plane, may be displayed as a box, X,
circle or any other appropriate marking. In some embodiments, as depicted
in FIG. 6, the object of interest 661 may be the projection of the
ablation needle 646 onto the imaging volume 656, and it may be displayed
as multiple boxes, X's, circles or other indicia 661 on the volume 656.
This is depicted as an example, in FIG. 6, as three squares on the
closest surface, in the middle, and on the farthest surface of the
imaging volume 656.

[0044] In some embodiments, the object of interest may be a previously
marked, annotated, or highlighted feature within the medical scene. For
example, surgeons may mark, circle, or otherwise annotate or indicate
tumors, blood vessels, or other features of a patient's anatomy using
various techniques. This data for the annotation, highlights, and
markings may be used within the medical scene to indicate various areas
of interest. Consider, for example, a liver with a single tumor. That
tumor may be highlighted, marked, or annotated in a way that the pose of
that tumor is known. As the operator moves the imager (such as the
ultrasound transducer) around the outside of the patient's body, the
focal plane for the imager may be modified to match or closely match the
position of the marked tumor. In this way, the highest resolution and
best focus of the imager will always be at or near the object of
interest, in this case the tumor. There may also be multiple objects of
interest, such as multiple tumors, that are each marked, highlighted,
etc. In some embodiments, there may be objects of interest that are
marked, annotated, or highlighted in addition to and/or instead of the
area of interest indicated by the operator using an ablation needle,
cauterizing tool, catheter, finger, eye gaze, etc. That is, for example,
there may be three tumors and a vein marked, and the operator may be able
to point using a cauterizer to indicate another area of interest. Some or
all of these poses may be determined in block 320.

[0045] In block 330, a focal adjustment for the imager is determined based
on the pose or the poses of the at least one object of interest. For
example, if there is a single object of interest and the desire is to
have the focus on that object of interest, then based on the pose of the
object of interest the focal plane may be defined. The focal plane may,
for example, be defined to pass through the center of the object of
interest or to pass near the object of interest. Turning back to FIG. 1B,
for example, if the object of interest 140 is the ablation needle or its
projection, then the focal plane 120 may be moved to pass through the
center of that object of interest 140. Turning to FIGS. 4A and 4B, if the
object of interest is the intersection of the projected ray (not
depicted) from the ablation needle 446 and the imaging plane 456, said
object of interest being projection 461, then the focal plane may be
determined to pass through or near the object of interest 461. Turning to
FIG. 2, if an operator is manipulating the imager 255 and an ablation
needle 245, and the object of interest is the intersection of the
projection of the ablation needle with the imager, then as the operator
manipulates one or both of the imager 255 and the ablation needle 245,
the object of interest will move and the focus of the imager 255 will be
changed.

[0046] In some embodiments, determining a focal adjustment for the imager
based on one or more poses comprises determining a series of focal
adjustments to be used sequentially over time. For example, if there are
multiple objects of interest, then the imager may be focused on each of
them in series and, thereby receiving at least some images that are
focused on each object. For example, if there are three tumors in a
volume that is being imaged by the imager, then the imager may first
focus on the first tumor, then focus on the second, and finally focus on
the third. In that way the imager will obtain a high resolution image for
each of the three objects of interest (tumors) in turn.

[0047] In some embodiments, a series of focal adjustments may be
determined to get focus in areas around the object of interest. For
example, if there is a single object of interest, a first image may be
focused just above the object of interest, a second through the object of
interest, and a third below the object of interest. Some embodiments will
continually scan the volume of interest by changing the focus in fixed
intervals and/or for fixed distances. For example, if the image volume is
four inches tall, it may first focus one inch down, then focus two inches
down, and then focus three inches down, and cycle through that pattern to
provide a varying focus over time. In some embodiments, this cycling may
be modified so that more of the steps of the cycling are focused at or
near the object of interest and fewer of these steps of the cycle are
focused away from the object of interest.

[0048] In some embodiments, the angle of projection of the ultrasound
waves within the plane of the imager may be modified. For example, FIG.
10A illustrates the depiction of the ultrasound waves 1081 leaving the
imager 1050 via the imaging element 1057 to produce an imaging volume or
plane 1056 in order to capture an image at least in part of an ablation
needle 1046. Typically, the sound waves of an ultrasound will travel
perpendicular to the imaging element 1057. The imaging element 1057 may
be, for example, a one-dimensional phased array transducer. In some
embodiments, the one-dimensional phased array transducer can be
manipulated so that the waves traveling from the transducer can be
focused onto an object of interest.

[0049] FIG. 10C illustrates that the imaging element 1057 can be
reconfigured to direct the imaging waves (e.g., pressure or energy), in a
different direction, as indicated by arrows 1081, in order to change the
angle at which the rays from the element 1057 are hitting the object of
interest, in this case, an ablation needle 1046. One reason to modify the
angle at which the waves from an imaging element are hitting an object of
interest is so that the angle of propagation at or more closely
approximating a perpendicular angle (e.g., around 90°) to the
surface of the object. In some embodiments and with some imagers, the
closer imaging is to a perpendicular angle, the more vividly or better
quality the resulting images will be, and the more likely the user will
detect the object in the resulting image The embodiments discussed with
respect to FIGS. 10A-10C help overcome some issues associated with
objects at oblique angles by refocusing or changing the angle at which an
object of interest is being imaged, thereby improving the quality of the
image that can be obtained. For example, looking to FIG. 10B, the rays
1081 from imager 1050 are focused towards the intersection of the
ablation needle 1046 with the ultrasound volume 1056. In this way, more
of the resolution of the ultrasound will be focused on the object of
interest 1046.

[0050] In some embodiments, similar techniques can be used for biplane or
2D arrays of transducers. For example, all of the transducers in a 2D
array of transducers could be focused towards an object of interest or
one or two dimensions in the 2D array of transducers could be modified,
in an angular sense, in order to better image an object of interest. For
a biplane imager, one or both of the planes could each be separately
modified as discussed above with respect to FIGS. 10A-10C.

[0051] After the focal adjustments have been determined in block 330, then
in block 340, data is generated to alter the focal adjustments of the
imager. For example, in some embodiments the imager may have an API or
application program interface, an electronic interface, etc. Data can be
generated to conform to that interface in order to alter the focal
adjustment of the imager.

[0052] In some embodiments, the data generated in block 340 may be sent to
the imager or the imager's interface, and the imager or the imager's
interface may interact with the imager and/or the imaging elements in
order to alter the focal adjustments, as depicted in block 350 of FIG. 3.
After generating the data to alter the focal adjustments of the imager in
block 340, or altering the focal adjustments of the imager in block 350,
method 300 may start again from block 310, 320 (not pictured), or any of
the other blocks (not pictured). For example, after generating the data
to alter the focal adjustments (block 340) or determining a focal
adjustment for the imager based on the poses (block 330), the method 300
may again return to determine new pose information for the focusable
imager and determine the pose of at least one object of interest, if
there is any new pose information.

[0053] Additionally, after determining the poses of the imager and/or the
objects of interest at any iteration of method 300, a representation of
the imager and the object of interest may be displayed in block 360
(depicted as occurring after block 310 or 320 in FIG. 3).

[0054] Various embodiments of the types of displays that can be used in
block 360 are depicted in the figures herein. For example, as discussed
above, in FIGS. 4A and 4B there is an ablation needle 446 being depicted
on a display 420 as well as the image from the imager 456 and the object
of interest 461. In FIG. 5, display 520 displays an object of interest
561 along with the ablation needled 546 and the imaging volume 556. Here
the imaging volume 556 is being displayed with an indication 521 of the
focal depth of the imager. FIG. 5 may be an example of the imager
iterating over a series of focal depths 521 and, in this particular
instance, the focal depth 521 is above the object of interest 561. FIG. 6
depicts a display 620 showing device or tool 646. Object of interest 661
is displayed in the center of the imaging volume 656 as well as on the
surfaces of the imaging volume 656--as three squares on the display 620.

Distance Indicators

[0055] In some embodiments, it can be helpful to an operator to have bars
or other indicators showing the depth or distance between a device 746
and the image plane 756. These are depicted as bars 771 on display 720.
In some embodiments, the bars between the device 746 and the image plane
756 are shorter when the device is closer to the image 756, and the bars
771 are longer when the device 746 is further away from the imager 756.

[0056] In some embodiments, as depicted in FIG. 8, the indicators 871 may
be more complex than simple bars. In FIG. 8, indicators 871 are thicker
outside of the image volume 856 (e.g., between the image volume 856 and
the cauterizer 846). Inside the image volume 856, the indicators 871 are
narrower. This differentiation of the size of the indicators 871 may be
useful to show the operator both the distance of the cauterizer 846 from
the imaging volume 856 as well as the thickness of the imaging volume 856
at various points in the imaging volume. FIG. 9 shows multiple different
indicators 971-975, each of which may be used as described above. Various
other embodiments, techniques, methods and systems will be clear from the
disclosure herein and are considered part of the embodiments disclosed
herein.

Projecting Placement

[0057] In some cases, particularly with the rapidly growing number of
obese patients the needle or other surgical device may not be long enough
to reach the target or object of interest when approaching from an "easy"
or convenient angle, and so the physician or other operator must find
creative ways of approaching the target. In some embodiments herein, the
system may make it clear to an operator whether the device, when
inserted, will reach a desired feature in the ultrasound image. In
certain procedures and embodiments, there may be prediction information
related to the surgical instruments. In the context of scalpel movement,
this may be the location that the scalpel will hit if a physician
continues to move the scalpel in a particular direction. In the context
of ablation, this may be the projected needle placement if it is driven
along its central axis. FIG. 11A illustrates the projected drive
trajectory 1147 of a needle 1146. If a physician is driving an ablation
needle into tissue (said tissue not pictured in FIG. 11A), then she may
want to know where the needle will be driven. In some embodiments, the
projected drive 1147 of a needle 1146 may be depicted on the display 1120
and may show the physician the projected path 1147 that the needle will
take if it is driven along its central axis--and the depth it may travel.

[0058] In some embodiments, the system may draw the trajectory of the
needle extending beyond the tip, extending approximately one
needle-length beyond the tip (See, e.g., FIG. 11B). Then when the
physician aims the needle 1146 (scalpel, cauterizer, or any other device
or tool) toward the target 1161, resting the tip on the patients' skin or
organ surface 1148 before driving the needle 1146, the trajectory
indicator 1147 will indicate whether the needle 1146 will reach the
target 1161 if driven fully in the same direction. In some embodiments,
in order to aid the physician in placing or orienting a needle, an image
guidance system, such as that depicted in FIG. 2, may draw a number of
rings about the axis of the needle shaft, extrapolated beyond its tip, as
depicted in FIG. 11A.

[0059] A physician may view and manipulate the position and orientation of
the needle 1146 and its expected drive projection (via its displayed
projected trajectory 1147) before it enters the patient's tissue. In some
embodiments, this is accomplished by the doctor positioning the virtual
rings in the drive projection such that they are co-incident (or pass
through) the ultrasound representation of a target, such as a tumor that
the doctor has spotted in the ultrasound. This may allow the physician to
verify that the needle is properly aimed at the target and can drive the
needle forward into the tissue such that it reaches its desired target or
destination. For example, if the doctor spotted a tumor 1161 in the
ultrasound image on display 1120 in FIG. 11B, she may be able to
reposition or align the ablation needle 1146 such that the drive
projection rings on display 1120 intersected or otherwise indicate that
the needle, if driven straight, will reach the tumor. In the example of
FIG. 11B, because the projection 1147 does not reach the tumor 1161 it
appears that the needle 1146 would not reach the tumor 1161 when driven,
as depicted in FIG. 11C.

[0060] The rings of a projection 1147 may be spaced at regular (e.g., 0.5,
1, or 2 cm) intervals to provide the physician with visual cues regarding
the distance from the needle tip to the targeted anatomy. In some
embodiments, the spacing of the rings may indicate other aspects of the
data, such as the drive speed of the needle, the density of the tissue,
the distance to a landmark, such as a target 1161, or any other
appropriate guidance data or property. In some embodiments, the rings or
other trajectory indicator may extend beyond the needle tip, by a
distance equal to the length of the needle-shaft. This way, the user
knows if the needle is long enough to reach the target--even before the
tip enters the patient. That is, in some embodiments, if the rings do not
reach the target with the tip still outside the body, then the tip won't
reach the target when the entire length shaft is inserted into the body.

[0061] Other display markers may be used to show trajectory, such as a
dashed, dotted, or solid line, transparent needle shaft, point cloud,
wire frame, etc. In some embodiments, three-dimensional rings may be used
and provide depth cues and obscure little of the ultrasound image.
Virtual rings or other virtual markers may be displayed
semi-transparently, so that they obscure less of the ultrasound image
than an opaque marker would.

[0062] Other prediction information may also be displayed. For example, if
a scalpel is being tracked by the image guidance system, then a cutting
plane corresponding to the scalpel may be displayed (not pictured). Such
a cutting plan may be coplanar with the blade of the scalpel and may
project from the blade of the scalpel. For example, the projected cutting
plane may show where the scalpel would cut if it were the doctor were to
advance the scalpel. Similar prediction information may be estimable or
determinable for cauterizers, lasers, and numerous other surgical
instruments.

Other Embodiments

[0063] The processes and systems described herein may be performed on or
encompass various types of hardware, such as computing devices. In some
embodiments, position sensing units 210 and 240, display unit 220, image
guidance unit 230, second display unit 251, and/or any other module or
unit of embodiments herein may each be separate computing devices,
applications, or processes or may run as part of the same computing
devices, applications, or processes--or one of more may be combined to
run as part of one application or process--and/or each or one or more may
be part of or run on a computing device. Computing devices may include a
bus or other communication mechanism for communicating information, and a
processor coupled with the bus for processing information. The computing
devices may have a main memory, such as a random access memory or other
dynamic storage device, coupled to the bus. The main memory may be used
to store instructions and temporary variables. The computing devices may
also include a read-only memory or other static storage device coupled to
the bus for storing static information and instructions. The computer
systems may also be coupled to a display, such as a CRT, LCD monitor,
projector, or stereoscopic display. Input devices may also be coupled to
the computing devices. These input devices may include a mouse, a
trackball, foot pedals, touch screen or tablet, drawing tablet, or cursor
direction keys.

[0064] Each computing device may be implemented using one or more physical
computers, processors, embedded devices, field programmable gate arrays
(FPGAs) or computer systems or a combination or portions thereof. The
instructions executed by the computing device may also be read in from a
computer-readable medium. The computer-readable medium may be
non-transitory, such as a CD, DVD, optical or magnetic disk, flash
memory, laserdisc, carrier wave, or any other medium that is readable by
the computing device. In some embodiments, hardwired circuitry may be
used in place of or in combination with software instructions executed by
the processor. Communication among modules, systems, devices, and
elements may be over a direct or switched connections, and wired or
wireless networks or connections, via directly connected wires, or any
other appropriate communication mechanism. Transmission of information
may be performed on the hardware layer using any appropriate system,
device, or protocol, including those related to or utilizing Firewire,
PCI, PCI express, CardBus, USB, CAN, SCSI, IDA, RS232, RS422, RS485,
802.11, etc. The communication among modules, systems, devices, and
elements may include handshaking, notifications, coordination,
encapsulation, encryption, headers, such as routing or error detecting
headers, or any other appropriate communication protocol or attribute.
Communication may also messages related to HTTP, HTTPS, FTP, TCP, IP,
ebMS OASIS/ebXML, DICOM, DICOS, secure sockets, VPN, encrypted or
unencrypted pipes, MIME, SMTP, MIME Multipart/Related Content-type, SQL,
etc.

[0065] Any appropriate 3D graphics processing may be used for displaying
or rendering, including processing based on OpenGL, Direct3D, Java 3D,
etc. Whole, partial, or modified 3D graphics packages may also be used,
such packages including 3DS Max, SolidWorks, Maya, Form Z, Cybermotion
3D, or any others. In some embodiments, various parts of the needed
rendering may occur on traditional or specialized graphics hardware. The
rendering may also occur on the general CPU, on programmable hardware, on
a separate processor, be distributed over multiple processors, over
multiple dedicated graphics cards, or using any other appropriate
combination of hardware or technique.

[0066] As will be apparent, the features and attributes of the specific
embodiments disclosed above may be combined in different ways to form
additional embodiments, all of which fall within the scope of the present
disclosure.

[0067] Conditional language used herein, such as, among others, "can,"
"could," "might," "may," "e.g.," and the like, unless specifically stated
otherwise, or otherwise understood within the context as used, is
generally intended to convey that certain embodiments include, while
other embodiments do not include, certain features, elements, and/or
states. Thus, such conditional language is not generally intended to
imply that features, elements and/or states are in any way required for
one or more embodiments or that one or more embodiments necessarily
include logic for deciding, with or without author input or prompting,
whether these features, elements, and/or states are included or are to be
performed in any particular embodiment.

[0068] Any process descriptions, elements, or blocks in the processes,
methods, and flow diagrams described herein and/or depicted in the
attached figures should be understood as potentially representing
modules, segments, or portions of code which include one or more
executable instructions for implementing specific logical functions or
steps in the process. Alternate implementations are included within the
scope of the embodiments described herein in which elements or functions
may be deleted, executed out of order from that shown or discussed,
including substantially concurrently or in reverse order, depending on
the functionality involved, as would be understood by those skilled in
the art.

[0069] All of the methods and processes described above may be embodied
in, and fully automated via, software code modules executed by one or
more general purpose computers or processors, such as those computer
systems described above. The code modules may be stored in any type of
computer-readable medium or other computer storage device. Some or all of
the methods may alternatively be embodied in specialized computer
hardware.

[0070] It should be emphasized that many variations and modifications may
be made to the above-described embodiments, the elements of which are to
be understood as being among other acceptable examples. All such
modifications and variations are intended to be included herein within
the scope of this disclosure and protected by the following claims.