Abstract:

Waypoints for a steerable medical device are stored as the steerable
medical device is moved within a patient. The stored waypoints are an
ordered sequence of locations. The ordered sequence of locations defines
a safe path within the patient for moving an articulatable portion of the
steerable medical device. The articulatable portion of the steerable
medical device is constrained to follow the safe path as the
articulatable portion moves within the patient. For example, the
articulatable portion of the steerable medical device is constrained to
remain within a boundary region enclosing the safe path as the
articulatable portion of the steerable medical device follows the safe
path.

Claims:

1. A method comprising:storing waypoints of a steerable medical device,
wherein said stored waypoints comprise an ordered sequence of locations,
and further wherein said ordered sequence of locations defines a safe
path for moving an articulatable portion of said steerable medical device
within a patient;constraining said articulatable portion of said
steerable medical device to follow said safe path as said articulatable
portion moves within said patient.

2. The method of claim 1 wherein said constraining further
comprises:constraining said articulatable portion of said steerable
medical device to remain within a boundary region enclosing said safe
path as said articulatable portion of said steerable medical device
follows said safe path.

3. The method of claim 2, wherein said constraining further
comprises:retrieving, by a processor, said ordered sequence of locations;
andgenerating, by said processor using said ordered sequence of
locations, a configuration of said articulatable portion of said
steerable medical device.

4. The method of claim 3 where said constraining further comprises:sending
at least one command, based on said configuration, to a controller to
constrain links, within said articulatable portion of said steerable
medical device.

5. The method of claim 1 wherein said articulatable portion of said
steerable medical device comprises a segment of said steerable medical
device and said segment comprises a plurality of links.

6. The method of claim 2 wherein said boundary region comprises a tube
having a cross-section.

7. The method of claim 6 wherein said cross-section comprises one of a
circular cross-section, an oblate cross-section, and a rectangular cross
cross-section.

8. The method of claim 2 wherein said articulatable portion is included in
a plurality of segments of said steerable medical device.

9. The method of claim 8 wherein said constraining said articulatable
portion of said steerable medical device to remain within a boundary
region enclosing said safe path further comprises:generating position and
orientation data for each of said plurality of segments using said
ordered sequence of locations and a kinematic model of said plurality of
segments of said steerable medical device.

10. The method of claim 9 wherein said generating further
comprises:minimizing a cost function to generate said position and
orientation data for each of said plurality of segments.

11. The method of claim 10 wherein said minimizing a cost function further
comprises:minimizing the sum of the absolute values of relative joint
angles, with an additional constraint that link positions not deviate by
more than a distance Δ at each of the waypoints in said ordered
sequence of locations.

12. A method comprising:transmitting, articulatable segment by
articulatable segment, a proximal roll from a proximal end of each
articulatable segment of a medical device to a distal end of said each
articulatable segment; andmaintaining a shape of said each articulatable
segment during said transmitting.

13. The method of claim 12 wherein said transmitting further
comprises:generating a new pitch angle α' and a new yaw angle
β' for an articulatable segment while maintaining an approximate
bend angle φ and a direction θ when a base of said
articulatable segment rolls though an angle -.DELTA.γ,wherein said
articulatable segment prior to said roll had a pitch angle α and a
yaw angle β;said approximate bend angle φ is,φ= {square root
over (α2+β2)}said direction θ
isθ=arctan(α/β).

14. The method of claim 12 wherein said transmitting further
comprises:performing an iterative solution, using a kinematic model, on
an articulatable segment by articulatable segment basis to generate
orientation data transmitting the distal roll for each articulatable
segment.

15. The method of claim 14 further wherein said orientation data comprises
pitch and yaw angles.

16. The method of claim 15 further comprising:sending at least one command
to a controller for said medical device to configure said medical device
so that each articulatable segment has said yaw and pitch angles for that
articulatable segment.

17. An apparatus comprising:a steerable medical device having an
articulatable portion; anda controller, coupled to said steerable medical
device, comprising:a processor;a memory coupled to said processor; anda
medical device controller coupled to said processor and to said steerable
medical device; andwherein said processor stores in said memory an
ordered sequence of locations; andsaid processor analyzes said ordered
sequence of locations and outputs to said medical device controller at
least one command to constrain said articulatable portion within a
boundary region enclosing a safe path defined by said ordered sequence of
locations.

18. The apparatus of claim 17 wherein said articulatable portion of said
steerable medical device comprises a segment of said steerable medical
device and said segment comprises a plurality of links.

19. The apparatus of claim 18 wherein said boundary region comprises a
tube having a cross-section.

20. The apparatus of claim 19 wherein said cross-section comprises one of
a circular cross-section, an oblate cross-section, and a square cross
cross-section.

[0003]Aspects of this invention are related to medical device path
planning, and more particularly are related to path planning for an
articulatable multi-segment medical device.

[0004]2. Related Art

[0005]An endoscope is a medical device for visualizing the interior of a
patient's body. Endoscopes have been used for a variety of medical
diagnostic procedures and for a variety of medical interventional
procedures.

[0006]Many different types of endoscopes are known. For example, one
steerable endoscope has an elongated body with a steerable distal portion
and an automatically controlled proximal portion. Such an endoscope is
described in U.S. Pat. No. 6,468,203 B2, entitled "Steerable Endoscope
and Improved Method of Insertion," of Amir Belson issued on Oct. 22,
2002, which is incorporated herein by reference in its entirety.

[0008]Unlike prior art methods that required path planning prior to use of
a steerable medical device, waypoints of a steerable medical device are
stored as the device is moved within a patient. The stored waypoints are
an ordered sequence of locations. The ordered sequence of locations
defines a safe path within the patient for moving an articulatable
portion of the steerable medical device.

[0009]Thus, the articulatable portion of the steerable medical device is
constrained to follow the safe path as the articulatable portion moves
within the patient or as the device is inserted into the patient. In one
aspect, the articulatable portion of the steerable medical device is
constrained to remain within a boundary region enclosing the safe path.

[0010]In one aspect, a process of constraining the steerable medical
device includes retrieving, by a processor, the stored ordered sequence
of locations. The processor next generates, using the ordered sequence of
locations, a configuration of the articulatable portion of the steerable
medical device so that the device follows the safe path. Based on the
configuration, the processor sends at least one command to a controller
to actuate the articulatable portion of the steerable medical device
based on the configuration.

[0011]In one embodiment, the articulatable portion of the steerable
medical device includes a segment of the steerable medical device. The
segment, in turn, includes a plurality of links.

[0012]In another aspect, the boundary region is a tube having a
cross-section. Examples of cross-sections include, but are not limited
to, a circular cross-section, an oblate cross-section, and a rectangular
cross cross-section.

[0013]In still yet another aspect, constraining the articulatable portion
of the steerable medical device to remain within a boundary region
enclosing the safe path includes generating position and orientation data
for each of the plurality of segments. This process uses the ordered
sequence of locations and a kinematic model of the plurality of segments
of the steerable medical device.

[0014]The process of generating position and orientation data for each of
the plurality of segments includes minimizing a cost function. In one
aspect, minimizing a cost function further includes minimizing the sum of
the absolute values of relative joint angles, with an additional
constraint that link positions must remain within some distance of the
safe path formed by the waypoints.

[0015]In another embodiment, a process maintains articulatable segments of
a medical device within a boundary region while transmitting a proximal
roll to the distal end of the device. This is done on a per segment
basis, from a proximal end of each articulatable segment of the medical
device to a distal end of the each articulatable segment. A shape of the
each articulatable segment is maintained during the transmitting so that
each articulatable segment remains in the boundary region.

[0016]In one aspect, the roll is transmitted segment by segment by
generating a new pitch angle α' and a new yaw angle β' for an
articulatable segment while maintaining an approximate bend angle φ
and a direction θ when a base of the articulatable segment rolls
though an angle -Δγ. The articulatable segment prior to the
roll had a pitch angle α and a yaw angle β. The approximate
bend angle φ is

φ= {square root over (α2+β2)}.

The direction θ is

θ=arctan(α/β),

where arctan represents the arctangent function.

[0017]In another aspect, the roll is transmitted by performing an
iterative solution on a processor using a kinematic model. The iterative
solution is done on an articulatable segment by articulatable segment
basis to generate orientation data corresponding to the desired distal
roll for a given proximal roll for each articulatable segment. The
orientation data includes pitch and yaw angles, in one aspect. The
processor sends at least one command to a controller of the medical
device to configure the medical device so that each articulatable segment
has the yaw and pitch angles for that articulatable segment.

[0018]An apparatus includes a steerable medical device having an
articulatable portion. The articulatable portion includes at least one
articulatable segment. The apparatus also includes a controller, coupled
to the steerable medical device. The controller includes a processor, and
a medical device controller coupled to the processor and to the steerable
medical device. The processor stores in the memory an ordered sequence of
locations of waypoints. The processor also analyzes the ordered sequence
of locations and outputs to the medical device controller at least one
command to constrain the articulatable portion within a boundary region
enclosing a safe path defined by the ordered sequence of location.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019]FIG. 1 is a diagrammatic view of a steerable medical device
including an articulatable portion, and a safe path.

[0020]FIG. 2 is a diagrammatic view of an apparatus used in one aspect of
the processes and operations described herein.

[0022]FIG. 4 is a process flow diagram for one embodiment of a method of
configuring a steerable medical device to follow a safe path.

[0023]FIG. 5 is a diagrammatic view of elements of a kinematic model that
can be used in the method of FIG. 4.

[0024]FIG. 6 is an illustration of an approximate change in joint angles
due to roll.

[0025]FIG. 7 is a diagrammatic view of elements of a kinematic model that
can be used in the method of FIG. 8.

[0026]FIG. 8 is a process flow diagram for one embodiment of a method
transferring roll distally in a medical device with articulatable
segments.

[0027]FIG. 9 is a diagrammatic view of another apparatus used in one
aspect of the processes and operations described herein.

[0028]In the drawings, the first digit of a figure number indicates the
figure in which the element with that figure number first appeared.

DETAILED DESCRIPTION

[0029]In one aspect, a steerable medical device 100 (FIG. 1), sometimes
referred to as medical device 100, includes a plurality of segments
101-1, 101-2, 101-3 . . . . In the embodiment of FIG. 1, at least three
segments 101-1, 101-2, 101-3 include a plurality of movable links 102-1,
102-2 and 102-3, respectively. In one aspect, a tip of steerable medical
device 100 is at distal end 120 and typically includes at least a camera.

[0030]The use of a steerable medical device with at least three segments
each having a plurality of movable links is illustrative only and in not
intended to be limiting. In view of this disclosure, a steerable medical
device may include from at least one segment having a plurality of links
to a number of segments with movable links needed to provide the required
functionality for the steerable medical device.

[0031]An example of a steerable medical device 100 is a steerable
endoscope. In the following description, steerable medical device 100 is
referred to as steerable endoscope 100. The use of steerable endoscope
100 as an example of a steerable medical device is illustrative only and
is not intended to be limiting to this specific steerable medical device.
An example of a steerable endoscope suitable for use in this disclosure
is described in commonly assigned, U.S. Patent Application Publication
No. 2007/0249901 A1 (filed Mar. 28, 2006; disclosing "Instrument Having
Radio Frequency Identification Systems and Methods for Use."), which is
incorporated herein by reference in its entirety.

[0032]The plurality of movable links in a segment allows that segment to
be articulated as steerable endoscope 100 moves into and out of a
patient. Thus, such segments are referred to as articulatable segments.

[0033]In one aspect, an operator manipulates a direction controller 245
(FIG. 2) to move the tip of steerable endoscope 100 in a particular
direction within the patient. In response to commands from a processor in
processor module 250, as described more completely below, and a signal
from direction controller 245, an endoscope controller 240 (FIG. 2) for
steerable endoscope 100 configures each of the links so that each segment
is constrained to move along, e.g., to follow, a known safe path 150
(FIG. 1) that is defined, in one aspect, by movement of the tip of
endoscope 100.

[0034]In one embodiment, as the operator steers the tip of steerable
endoscope 100 through the patient, a safe path is generated. More
specifically, as the operator inserts steerable endoscope 100 into a
patient and moves steerable endoscope 100 towards a desired location, an
ordered sequence of locations {Pk, Pk+1, Pk+2, Pk+3,
Pk+4} (FIG. 1) of the tip of endoscope 100 are recorded in a memory
230 (FIG. 2) as waypoint data 231, in this example.

[0035]The ordered sequence of locations {Pk, Pk+1, Pk+2,
Pk+3, Pk+4} (FIG. 1) represent the motion of the tip of
steerable endoscope 100 through the patient. Thus, the ordered sequence
of locations {Pk, Pk+1, Pk+2, Pk+3, Pk+4}
represent a safe path 150, which has been chosen by the operator, for
steerable endoscope 100 to follow as steerable endoscope 100 is inserted
into and withdrawn from the patient.

[0036]Using the tip movement to generate an ordered sequence of locations
that define safe path 150 is illustrative only and is not intended to be
limiting. Alternatively, steerable endoscope 100 could be inserted in a
passive state, and then all the inserted articulatable segments activated
at one. The positions of the links in the articulatable segments at the
time of activation provide an ordered sequence of locations that define
safe path 150 for additional insertion and for retraction of steerable
endoscope 100.

[0037]In one aspect, as described more completely below, all trailing
segments 101-1, 101-2, 101-03 . . . of steerable endoscope 100 are
constrained to follow safe path 150 that is defined by ordered sequence
of locations {Pk, Pk+1, Pk+2, Pk+3, Pk+4}.
However, trailing segments 101-1, 101-2, 101-03 . . . may not follow safe
path 150 exactly. In one aspect, the operator can define a boundary
region around safe path 150. The boundary region (See FIGS. 3A to 3C) is
a volume inside which steerable endoscope 100 can move safely. In one
aspect, all trailing segments 101-1, 101-2, 101-03 . . . are constrained
to remain within this boundary region as steerable endoscope 100 moves
into the patient.

[0038]Also, in another aspect, as medical device 100 is withdrawn from the
patient, articulatable segments 101-1, 101-2, 101-03 . . . are
constrained to remain within this boundary region. Hence, unlike prior
art techniques that required advance planning, a safe route for medical
device 100 into and out of a patient is generated based on waypoint data
231 recorded as steerable endoscope 100 moves into the patient.

[0039]When segment 101-1 is considered a trailing segment, as just
described, segment 101-1 is controlled in the same way as the other
trailing articulatable segments. However, in another insertion mode,
segment 101-1 remains directly controlled by the operator and does not
change shape with insertion. In this insertion mode, segment 101-1 would
not be considered a trailing segment.

[0040]FIGS. 3A to 3C are illustrations of boundary regions 310A to 310C,
with different cross-sectional shapes, for safe path 150 defined by
ordered sequence of locations {Pk, Pk+1, Pk+2, Pk+3,
Pk+4}. In FIG. 3A, boundary region 310A has a circular
cross-section. In FIG. 3B, boundary region 310B has an oblate
cross-section. In FIG. 3C, boundary region 310C has a rectangular
cross-section, and in particular, a specific instance of a rectangular
cross-section, a square cross-section.

[0041]In one aspect, each location Pk is a point in space (xk,
yk, zk) relative to a fixed reference frame. Thus, each of
boundary regions 310A, 310B, 310C defines a three-dimensional volume
enclosing safe path 150. This volume is referred to as a tube having a
cross section to convey the three-dimensional characteristics of the
volume. Tube, as used here, is not a physical tube inserted in the
patient to define the volume.

[0042]The particular cross-sectional shapes of the boundary regions
illustrated in FIGS. 3A to 3C are illustrative only and are not intended
to be limiting. In view of this disclosure, for a particular medical
procedure and steerable medical device, an appropriately shaped boundary
region or regions can be selected.

[0043]It is not necessary that the boundary region have the same
cross-sectional shape over the entire extent of the safe path. For
example, a first portion of a boundary region could have a first
cross-sectional shape, and a second portion of the boundary region could
have a second cross-sectional shape that is different from the first
cross-sectional shape.

[0044]FIG. 4 is a process flow diagram for one embodiment of a method 400
used with a steerable medical device such as steerable endoscope 100. In
one aspect, instructions in a METHOD 400 module 232 (FIG. 2) in memory
230 are executed on a processor in processor module 250 to perform at
least some of the operations described more completely below.

[0045]In method 400, STORE WAYPOINTS operation 420, as described more
completely below, saves waypoints of the tip of steerable endoscope 100
as an ordered sequence of locations. The ordered sequence of locations
represents the trajectory of the tip as steerable endoscope 100 is
inserted into the patient. As described above, the ordered sequence of
locations defines a safe path 150 that has been selected by the operator
of steerable endoscope 100.

[0046]The stored ordered sequence of locations is retrieved and analyzed
in FOLLOW SAFE PATH process 430. FOLLOW SAFE PATH process 430 configures
the articulatable portion of endoscope 100 so that the articulatable
portion of endoscope 100 is constrained to follow safe path 150 as the
articulatable portion moves within the patient. Herein, following the
safe path does not mean that the safe path is followed exactly, but
rather that the articulatable portion steerable endoscope 100 stays at
least within a safe region, sometimes called a boundary region, about
safe path 150.

[0047]In FIG. 4, one embodiment of STORE WAYPOINTS operation 420 and
FOLLOW SAFE PATH process 430 are illustrated and described more
completely below. This embodiment is illustrative only and is not
intended to be limiting to the specific operations and processes
described. In view of this disclosure, one knowledgeable in the field can
utilize other techniques to move the articulatable portion of a steerable
medical into and out of a patient while following the safe path described
herein.

[0049]In ESTABLISH WAYPOINT operation 402, a waypoint on the safe path of
the endoscope is established. This can be done in a variety of ways.
Accordingly, check operations 410 to 412, as discussed more completely
below, are illustrative only and are not intended to be limiting to this
particular implementation.

[0050]In one aspect, as the operator inserts steerable endoscope 100 into
a body cavity (for instance, through an incision in the stomach into the
abdominal cavity) insertion is paused periodically. When motion of
endoscope 100 is paused, DEVICE MOTION PAUSED check operation 410
transfers processing to DEVICE MOTION RESUMED check operation 411.

[0051]If steerable endoscope 100 is paused to determine a safe trajectory
for the tip, the operator uses the camera and illumination at the tip of
steerable endoscope 100 to determine a safe trajectory inside the body
cavity. When viewing the interior of the body cavity and determining a
safe direction for farther insertion, the endoscope tip may be oriented
in any direction to provide views of the body cavity.

[0052]Once a safe trajectory is determined, the tip is oriented in the
direction that the operator intends steerable endoscope 100 to travel
upon farther insertion. The operator then inserts endoscope 100 farther
into the patient. However, the operator could also retract endoscope 100
after the pause. Thus, upon resumption of motion of endoscope 100, DEVICE
MOTION RESUMED check operation 411 transfers processing to INSERTION
check operation 412.

[0054]Again, the combination of operations 410 to 412 is one way to allow
an operator to establish a waypoint. Alternatively, the operator could,
for example, push a button when a safe trajectory has been identified and
the operator is about to move the endoscope tip on that trajectory.

[0055]SAVE WAYPOINT operation 403 records the previous location of the tip
as a waypoint in WAYPOINT DATA 403a, which in FIG. 2 is shown as waypoint
data 231. SAVE WAYPOINT operation 403, upon completion, transfers
processing to NEW LINK INSERTED check operation 404. SAVE WAYPONT
operation 403 also transfers back to the start of ESTABLISH WAYPOINT
operation 402 in case the operator decides to change the direction of the
safe trajectory.

[0056]In this example, insertion of a link is used as a trigger to record
another waypoint for the tip. However, other criteria can be used so long
as locations of the tip are saved with sufficient resolution to permit
determining a safe path. As steerable endoscope 100 is moved further into
the patient along the safe trajectory, as each new link is inserted in
the patient, the location of the tip is saved as another waypoint. One
way of determining when a link is inserted in described in commonly
assigned U.S. patent application Ser. No. 12/613,698, entitled "METHOD
AND SYSTEM FOR MEASURING INSERTED LENGTH OF A MEDICAL DEVICE USING
INTERNAL REFERENCED SENSORS" of Caitlin Donhowe, filed on Nov. 6, 2009,
which is incorporated herein by reference in its entirety. Another way of
determining when a link is inserted is to use an external position
sensor. In this example, when a new link is inserted, NEW LINK INSERTED
check operation 404 transfers processing to SAVE WAYPOINT operation 403,
which was described above, and to UPDATE check operation 409.

[0057]UPDATE check operation 409 is shown with a dotted line in FIG. 4
because check operation 409 is optional. In one aspect, the configuration
of the trailing segments of endoscope 100 is updated only after a
predefined number of new waypoints have been saved. In this aspect,
UPDATE check operation 409 does not transfer processing to process 405
until the predefined number of new waypoints is available in WAYPOINT
DATA 403A. Also, UPDATE check operation 409 checks the states of the
articulatable segments and if none of the articulatable segments have
state articulating, UPDATE check operation 409 does not transfer
processing to process 405. Alternatively, the configuration of the
trailing segments of endoscope 100 is updated for each new waypoint and
so UPDATE check operation 409 is not needed.

[0058]In this embodiment, FOLLOW SAFE PATH process 430 includes MAINTAIN
SEGMENTS IN BOUNDARY REGION process 405 and SEND COMMAND process 406.
MAINTAIN SEGMENTS IN BOUNDARY REGION process 405 uses the saved waypoints
to generate a configuration for each of the inserted links so that
trailing segments 101-1 to 101-3 follow safe path 150 of the tip as
defined by the waypoints. As noted above, for some steerable medical
devices, it may not be possible to configure the trailing segments to
follow exactly safe path 150. Thus, in one aspect, a boundary region
about the ordered sequence of locations {Pk}k=1N is
generated. Here, Pk denotes the Cartesian coordinates of the
endoscope tip when the number of the inserted links is k. After the
endoscope has been inserted for N links, the path trajectory of the
endoscope tip is recorded as ordered sequence of locations
{Pk}k=1N in memory 230 (FIG. 2).

[0059]As described above, the boundary region defines a volume around safe
path 150, i.e., a volume that encloses safe path 150. The trailing
segments can safely pass through any part of the boundary region as the
trailing segments move towards the surgical site, or move away from the
surgical site.

[0060]In one aspect, a kinematic model of steerable endoscope 100, such as
model 500 in FIG. 5, is used in MAINTAIN SEGMENTS IN BOUNDARY REGION
process 405. In kinematic model 500, (FIG. 5), a first level 501 of model
500 defines the number of articulatable segments in a plurality of
articulatable segments of the steerable endoscope. In this example, the
steerable endoscope has four articulatable segments Seg 1, Seg 2, Seg 3,
and Seg 4.

[0061]A second level 502 of model 500 is a representation of links and
joints in each of four articulatable segments Seg 1, Seg 2, Seg 3, and
Seg 4. In this example, each segment includes eight links, where adjacent
links are separated by a joint. The joints are numbered sequentially from
the distal end to the proximal end of the endoscope, starting with a
value one. (This numbering sequence is for convenience only. In other
aspects, the numbering may increase in going from a proximal location
towards a distal location.) Each link has a fixed length and the
associated joint can rotate in a single plane, either the x-plane or the
y-plane. The joints alternate so adjacent links rotate in different
planes.

[0062]In model 500, each of four articulatable segments Seg 1, Seg 2, Seg
3, and Seg 4 has an angle of rotation about the x-axis and an angle of
rotation about the y-axis. For articulatable segment Seg 1, the angle of
rotation about the x-axis is θy1, (sometimes referred to
as pitch) and the angle of rotation about the y-axis is
θx1 (sometimes referred to as yaw). For articulatable
segment Seg 2, the angle of rotation about the x-axis is
θy2, and the angle of rotation about the y-axis is
θx2. For articulatable segment Seg 3, the angle of
rotation about the x-axis is θy3 and the angle of
rotation about the y-axis is θx3. For articulatable
segment Seg 4, the angle of rotation about the x-axis is
θy4, and the angle of rotation about the y-axis is
θx4.

[0063]Row 505 in model 500 gives the initial constraint on the joint
angles. In this example, the joint angles for a segment are assumed to be
equal. Thus, each joint angle is one-fourth of the corresponding segment
angle. Row 506 in model 500 gives the roll matrix for each joint.

[0064]In this model, y-links are links that cause motion in the
y-direction rather than links that rotate about the y-direction. This
definition results in Rx(θy). Similarly, in this model,
x-links are links that cause motion in the x-direction rather than links
that rotate about the x-direction. This definition results in
Ry(θx) Those knowledgeable in the field understand that
alternative definitions can be used in the kinematic model, where x-links
are links that rotate about the x-direction and y-links are links that
rotate about the y-direction. Irrespective of the kinematic model
definitions, the results are equivalent.

[0065]Thus, model 500 is illustrative only and is not intended to be
limiting. Model 500 is used in one example that is developed more
completely below.

[0066]Using stored ordered sequence of locations {Pk}k=1N
and kinematic model 500, a cost function is minimized to generate
position and orientation data of the endoscope segments in MAINTAIN
SEGMENTS IN BOUNDARY REGION process 405. Specifically, in one aspect,
positions and orientations are generated for each of the links in the
articulatable portion of endoscope 100. The positions and orientations
are constrained within the boundary region.

[0067]The cost function allows the position and orientation to be
optimized for a variety of criteria, for example, but not limited to: 1)
maximum tip controllability; 2) maximum distance of the segments from
inverse-kinematic singularities; 3) maximum distance of the segments from
articulation limits; 4) maximum distance from user-defined body-cavity
boundaries (organs, adhesions, etc.); 5) minimum distance from the
defined path.

[0068]In process 405, the cost function may be a dynamically changing
function of any number of criteria. However, the number of criteria is
selected so the cost function can be minimized and the segments
positioned to achieve the constrained movement within an endoscope
movement time acceptable to the operator. Conventional techniques are
used by a processor to minimize the cost function selected subject to the
position constraints imposed.

[0069]In an additional aspect, an embodiment of process 405 determines the
optimal configuration of the segments (relative segment angles) to
satisfy the constraint that the segment links stay "near to" the path
defined by the tip forward motion and any additional constraints on the
link (or segment) positions.

[0070]For example, in process 405, the segments may be allowed to assume
positions within a tubular bounding region 310A (FIG. 3A) around the tip
path while encouraging tip manipulability. One way to encourage tip
manipulability in process 405 is to minimize the sum of the absolute
values of the relative joint angles, with the additional constraint that
the link positions not deviate by more than a distance Lx at each of the
waypoints in ordered sequence of locations {Pk}k=1N. In
one aspect, distance Δ is specified by the operator.

[0071]Upon determining the configuration for each of the articulatable
segments, MAINTAIN SEGMENTS IN BOUNDARY REGION process 405 transfers
processing to SEND COMMAND process 406 (FIG. 4). In SEND COMMAND process
406, the processor in processor module 250 (FIG. 2) sends at least one
command to the motors in endoscope controller 240 to configure the
articulatable segments based on the locations and orientations generated
in process 405, i.e., based on the configuration generated in process
405. This constrains the articulatable segments within the boundary
region as endoscope 100 moves further into the patient.

[0074]In the example of FIG. 2, in response to the command, endoscope
controller 240 configures the segments so that as the segments move
forward, the segments are constrained to stay within the boundary region.
Thus, the segments may not follow the trajectory of the tip exactly, but
the segments follow a path within the boundary region that is safe.

[0075]When the operator starts to withdraw the endoscope from the patient,
the waypoints in the stored ordered sequence of locations
{Pk}k=1N decrease by one as each link is withdrawn.
Processes 405 and 406 are used as steerable endoscope is withdrawn to
maintain endoscope 100 with the boundary region.

[0076]As a further example of process 405, the control of the joint angles
at constant endoscopic length increments is considered such that the
endoscope configuration is kept within a safety area, the boundary region
described above, during the process of insertion and withdrawal. In this
example, a configuration of the trailing links that is closest, as
measured by Euclidean norm, to the tip trajectory is generated in process
405, since the tip trajectory represents a safe route selected by the
operator.

[0077]As described above Pk denotes the Cartesian coordinate of the
endoscope tip when the number of the inserted links is k. After the
endoscope has been inserted for N links, the path trajectory of the
endoscope tip is recorded in a series {Pk}k=1N. Process
405 controls the joint angles to achieve an endoscope link configuration
close to the waypoints in recorded series {Pk}k=1N. This
is formulated as the following optimization, where J is the cost function
discussed above:

min θ J = 1 2 k = 1 N p k - P k 2
( 1 ) ##EQU00001##

subject to

θmin≦θ≦θmax, (2)

where pk is the coordinate of each link determined by the kinematics
model.

[0078]To apply an iterative procedure to solve this problem, assume that
at the n-th step, a solution is θn and the next increment
Δθn is to be determined. The new link coordinate is
approximated by

[0094]In the above examples, the sequence of ordered locations and
six-degrees of freedom were considered in configuring articulatable
segments to remain within a boundary region, sometimes called a safe
region. However, in some applications, consideration of only two-degrees
of freedom may be sufficient. The x-y position could be controlled, or
the pitch and the yaw could be controlled. In these applications, each
waypoint only includes two degrees of freedom corresponding to an
insertion depth and the above processes become less computationally
intensive while the safe path is followed.

[0095]The above processes are applicable to a mode of endoscope operation
in which the operator can directly articulate only the tip segment, e.g.,
segment 101-1, insertion causes new safe waypoints to be created along
the path traced by the tip. The above techniques are used in configuring
the trailing segments to follow the safe path.

[0096]In another mode of operation, the operator can make Cartesian
commands at the tip and the rest of the segments make coordinated moves
to accomplish that move within their respective safe regions, referred to
as a boundary region above. The workspace at the tip is quite limited due
to the safe region constraints, but a roll-only mode of operation would
fall into this category of operation. In one aspect, the roll-only mode
of operation could be achieved by simply commanding a roll at the tip
subject to the safe region constraints as discussed above. However, an
alternative approach for the roll-only mode of operation that utilizes
bend angle and bend direction waypoints is described more completely
below.

[0097]For example, a segment state that prevents articulation but allows
roll to propagate from the base of the steerable medical device (proximal
end) to the tip of the steerable medical device (distal end) may be
desirable. In one aspect to propagate roll, annular joint limits are
generated that prevent the combined bend angle of a segment from changing
more than a few degrees, but allow individual joint angles (yaw and
pitch) to vary. However, this alone does not force the shape of the
steerable medical device to be maintained as articulatable segments can
move independently within their respective annuli.

[0098]Ideally, when a pure roll is commanded, all the segments in
steerable medical device 100 would maintain their shape and simply
transmit proximal roll distally. As long as transmitting the proximal
roll segment by segment does not change the shape of steerable medical
device 100, steerable medical device 100 would remain within the boundary
region. However, when a steerable medical device has articulatable
segments with redundant degrees of freedom, the minimum joint velocity
solution can change the shape of that medical device. In one aspect, a
steerable medical device with more than two articulatable segments,
insertion, and roll has redundant degrees of freedom.

[0099]In one aspect, the roll is transmitted distally while maintaining
each articulatable segment within the boundary region by determining
changes in joint angles due to roll on a per-segment basis, which ensures
that the shape of each articulatable segment is maintained and so each
articulatable segment follows the safe path. When each of the
articulatable segments has been rolled, a final iteration can done to
correct any deviation in the position and orientation of the tip.

[0100]FIG. 6 is an illustration of one method to approximate changes in
segment joint angles as roll is propagated on a per segment basis. In
this method, when the base of a segment rolls though an angle
-Δγ, new pitch angle α' and new yaw angle β' are
generated for that segment by maintaining approximate bend angle φ
and direction θ. From FIG. 6, for a current pitch angle αand
a current yaw angle β, approximate bend angle φ is defined as,

φ= {square root over (α2+β2)}

while direction θ is defined as,

θ=arctan(α/β)

and new direction θ' is

θ'=θ+Δγ.

With these definitions, new pitch angle α' and new yaw angle β'
are

α'=φ*sin(θ')

β'=φ*cos(θ').

New pitch angle α' and new yaw angle β' for the segment are
implemented, in one aspect, by assuming equal joint angles for the yaw
joints and equal angles for the pitch joints in the segment.

[0101]Thus, this method takes an initial roll -Δγ, current
pitch angle α and current yaw angle β for each segment and
propagates that roll segment by segment from the proximal end to the
distal end of steerable medical device 100. This method of decoupling the
segments and determining the roll segment by segment without feedback is
useful when there is more than one pair of yaw and pitch joints in a
segment.

[0102]An alternative to the above approach without feedback is to perform
an iterative solution on a segment by segment basis to transmit the roll
distally from the base of the medical device while maintaining the
current shape of each segment. The iterative analysis is performed until
convergence is reached on a per-segment basis, i.e., a segment is
iterated on until converged and then the next segment is processed.

[0103]A kinematic model of a segment is used in this iterative solution.
FIG. 7 is a diagrammatic drawing of a general kinematic model 700 for a
representative segment j of steerable endoscope 100 with an indication of
the joint angle and the appropriate roll matrix. Here, j is an integer
ranging from one to the number of articulatable segments. In determining
the orientation of each of the links, a transform is defined for each
link.

Note that the rotations in kinematic model 700 are defined differently
than those in kinematic model 500 above.

[0108]In this aspect, as noted above, only the yaw and pitch are of
interest in determining the orientation. Thus, the iterative solution is
obtained using the roll matrices Rx and Ry, e.g., three rows
corresponding to the orientation and three columns corresponding to
pitch, yaw, and a virtual roll at the distal end of the segment. The
virtual roll should be equal to the base roll at the proximal end of the
segment, because this kinematic model does not allow roll along the
segment.

[0109]In the approach that used kinematic model 700, orientation data was
utilized to transmit roll distally. In another aspect, x-y position data
with a kinematic model could be used to maintain the shape of steerable
medical device on a segment by segment basis for a displacement in the
z-direction. Utilizing, both position and orientation data at the same
time over-constrains the determination.

[0110]FIG. 8 is a process flow diagram of one embodiment of a process 800
for transmitting proximal roll to a tip of steerable endoscope 100. In
one aspect, instructions in a METHOD 800 module 933 in memory 230A of
system controller 220A (FIG. 9) are executed on a processor in processor
module 250 to perform at least some of the operations described more
completely below.

[0112]Processing remains in ROLL COMMAND check operation 802 until a roll
command is received from the operator of steerable endoscope 100. Upon
receiving a roll command, ROLL COMMAND check operation 802 transfers
processing to USE KINEMATIC MODEL TO GENERATE LINK YAW AND PITCH FOR
SEGMENT process 803. Check operation 802 should not be interpreted as
requiring polling. Check operation 802, for example, could be implemented
as part of an event handler and when a roll command event occurs, process
803 is launched.

[0113]USE KINEMATIC MODEL TO GENERATE LINK YAW AND PITCH FOR SEGMENT
process 803 retrieves the current yaw and pitch for each articulatable
segment from ROLL AND SEGMENT CURRENT PITCH AND YAW data 803A for use in
kinematic model 700 (FIG. 7). Starting with the most proximal
articulatable segment, the desired orientation for that segment is set
equal to the original orientation. The base of the segment, e.g., the
most proximal portion of the segment, is rotated by the amount of the
commanded roll around the x-axis of the segment base.

[0114]In one aspect, the kinematic model is seeded with an approximate
solution, such as that generated using FIG. 6, to reduce the number of
iterations required for convergence. When the analysis for a segment is
completed, yaw data and pitch data are available for each link in that
segment.

[0115]Upon completion of processing the segment, process 803 transfers to
LAST SEGMENT CHECK operation 804. LAST SEGMENT CHECK operation 804
determines whether all the segments have been processed by the kinematic
model. If the necessary segments have been processed, check operation 804
transfers to SEND COMMAND operation 805 and otherwise returns to process
803.

[0116]When the configuration for each of the articulatable segments is
determined, LAST SEGMENT check operation 804 transfers processing to SEND
COMMAND operation 805. In SEND COMMAND operation 805, the processor in
processor module 250 (FIG. 9) sends the at least one command to the
motors in endoscope controller 240 to configure the articulatable
segments based upon the yaw and pitch orientations generated in process
803. In response to the command or commands, endoscope controller 240
configures the articulatable segments so that as these segments roll, the
segments are constrained to stay within the boundary region.

[0117]The resulting configuration of the articulatable segments maintains
the shape of endoscope 100 while the proximal roll is transmitted
distally. Method 800 assures that steerable endoscope 100 stays within
the boundary region discussed above as the proximal roll is transmitted.

[0118]In FIG. 9, elements with the same reference numeral as elements in
FIG. 2 are the same or equivalent elements. Accordingly, the description
of those elements with respect to FIG. 2 is incorporated herein by
reference for FIG. 9.

[0119]System controller 220 (FIG. 2) and system controller 220A (FIG. 9)
are illustrated as unified structures for ease of illustration and
understanding. This is illustrative only and is not intended to be
limiting. The various component of system controllers 220 and 220A can be
located apart and still perform the functions described.

[0120]The above description and the accompanying drawings that illustrate
aspects and embodiments of the present inventions should not be taken as
limiting--the claims define the protected inventions. Various mechanical,
compositional, structural, electrical, and operational changes may be
made without departing from the spirit and scope of this description and
the claims. In some instances, well-known circuits, structures, and
techniques have not been shown or described in detail to avoid obscuring
the invention.

[0121]Further, this description's terminology is not intended to limit the
invention. For example, spatially relative terms--such as "beneath",
"below", "lower", "above", "upper", "proximal", "distal", and the
like--may be used to describe one element's or feature's relationship to
another element or feature as illustrated in the figures. These spatially
relative terms are intended to encompass different positions (i.e.,
locations) and orientations (i.e., rotational placements) of the device
in use or operation in addition to the position and orientation shown in
the figures. For example, if the device in the figures is turned over,
elements described as "below" or "beneath" other elements or features
would then be "above" or "over" the other elements or features. Thus, the
exemplary term "below" can encompass both positions and orientations of
above and below. The device may be otherwise oriented (rotated 90 degrees
or at other orientations) and the spatially relative descriptors used
herein interpreted accordingly. Likewise, descriptions of movement along
and around various axes include various special device positions and
orientations.

[0122]The singular forms "a", "an", and "the" are intended to include the
plural forms as well, unless the context indicates otherwise. The terms
"comprises", "comprising", "includes", and the like specify the presence
of stated features, steps, operations, elements, and/or components but do
not preclude the presence or addition of one or more other features,
steps, operations, elements, components, and/or groups. Components
described as coupled may be electrically or mechanically directly
coupled, or they may be indirectly coupled via one or more intermediate
components.

[0123]The term "flexible" in association with a mechanical structure or
component should be broadly construed. In essence, it means the structure
or component can be bent without harm. For example, a flexible mechanical
structure may include a series of closely spaced components that are
similar to "vertebrae" in a snake-like arrangement. In such an
arrangement, each component is a short link in a kinematic chain, and
movable mechanical constraints (e.g., pin hinge, cup and ball, and the
like) between each link may allow one (e.g., pitch) or two (e.g., pitch
and yaw) degrees of freedom (DOF) of relative movement between the links.
As another example, a flexible mechanical structure may be continuous,
such as a closed bendable tube (e.g., nitinol, polymer, and the like) or
other bendable piece (e.g., kerf-cut tube, helical coil, and the like).
Accordingly, a short, flexible structure may serve as, and be modeled as,
a single mechanical constraint (joint) providing one or more DOFs between
two links in a kinematic chain, even though the structure itself may be a
kinematic chain made of several coupled links.

[0124]While the memory in FIGS. 2 and 9 is illustrated as a unified
structure, this should not be interpreted as requiring that all memory is
at the same physical location. All or part of the memory can be in a
different physical location than a processor. Memory refers to a volatile
memory, a non-volatile memory, or any combination of the two.

[0125]A processor is coupled to a memory containing instructions executed
by the processor. This could be accomplished within a computer system, or
alternatively via a connection to another computer via modems and analog
lines, or digital interfaces and a digital carrier line.

[0126]Herein, a computer program product comprises a medium configured to
store computer readable code needed for any one or any combination of the
operations described with respect to module 232, module 933 or in which
computer readable code for any one or any combination of operations
described with respect to modules 232 and 933 is stored. Some examples of
computer program products are CD-ROM discs, DVD discs, flash memory, ROM
cards, floppy discs, magnetic tapes, computer hard drives, servers on a
network and signals transmitted over a network representing computer
readable program code. A tangible computer program product comprises a
tangible medium configured to store computer readable instructions for
any one of, or any combination of operations described with respect to
modules 232 and 933 or in which computer readable instructions for any
one of, or any combination of operations described with respect to
modules 232 and 933 are stored. Tangible computer program products are
CD-ROM discs, DVD discs, flash memory, ROM cards, floppy discs, magnetic
tapes, computer hard drives and other physical storage mediums.

[0127]In view of this disclosure, instructions used in any one of, or any
combination of operations described with respect to modules 232 and 933
can be implemented in a wide variety of computer system configurations
using an operating system and computer programming language of interest
to the user.

[0128]All examples and illustrative references are non-limiting and should
not be used to limit the claims to specific implementations and
embodiments described herein and their equivalents. The headings are
solely for formatting and should not be used to limit the subject matter
in any way, because text under one heading may cross reference or apply
to text under one or more headings. Finally, in view of this disclosure,
particular features described in relation to one aspect or embodiment may
be applied to other disclosed aspects or embodiments of the invention,
even though not specifically shown in the drawings or described in the
text.