Sign up to receive free email alerts when patent applications with chosen keywords are publishedSIGN UP

Abstract:

Provided is an image stabilizing apparatus and method for correcting an
image that is shaken due to a movement of a camera. The image stabilizing
apparatus includes a characterizing point checking region setting unit
including: a sample frame extract unit which extracts a plurality of
image frames obtained for a certain period of time in image data obtained
by photographing an object; and a frame analyzing unit which detects a
plurality of characterizing points in the extracted plurality of image
frames, and sets a characterizing point checking region which is used to
check characterizing points in a currently input image frame.

Claims:

1. A characterizing point checking region setting unit comprising: a
sample frame extract unit which extracts a plurality of image frames,
obtained for a certain period of time, from image data obtained by
photographing an object; and a frame analyzing unit which detects a
plurality of characterizing points in the extracted plurality of image
frames, and sets a characterizing point checking region, which is used to
check characterizing points in a currently input image frame.

2. The characterizing point checking region setting unit of claim 1,
wherein the plurality of image frames are obtained by continuously
photographing a same object.

3. The characterizing point checking region setting unit of claim 1,
wherein the image data input in the sample frame extract unit is obtained
by a camera fixed on a position.

4. The characterizing point checking region setting unit of claim 1,
wherein the frame analyzing unit comprises: a characterizing point
detector which receives the plurality of image frames from the sample
frame extract unit, and detects the plurality of characterizing points in
the plurality of image frames; a characterizing point classification unit
which classifies the plurality of characterizing points into a plurality
of clusters for each of the image frames; a center point detector which
detects a centroid point of the characterizing points in the plurality of
image frames; and a check region determination unit which sets the
characterizing point checking region including a major cluster among the
plurality of clusters based on the centroid point.

5. The characterizing point checking region setting unit of claim 4,
wherein a cluster of the plurality of clusters is classified as the major
cluster if the cluster comprises 50% or more of the characterizing
points, and as a minor cluster if the cluster comprises less than 50% of
the characterizing points.

6. The characterizing point checking region setting unit of claim 5,
wherein the center point detector calculates an average of a centroid
point in a major cluster of each of the plurality of image frames and of
a centroid point in a minor cluster of the each of the plurality of image
frames, sums the average values, and divides the summed average values by
a number of the plurality of image frames to detect the centroid point of
the characterizing points in the plurality of image frames.

7. The characterizing point checking region setting unit of claim 4,
wherein the check region determination unit defines the characterizing
point checking region as a square or a circle.

8. The characterizing point checking region setting unit of claim 4,
further comprising a checking region adjusting unit which determines
whether the characterizing point checking region includes a standard
level of characterizing points of the image frames extracted for the
certain period of time or greater.

9. The characterizing point checking region setting unit of claim 8,
wherein the standard level is 80% of the characterizing points.

10. A characterizing point checking region setting method comprising:
receiving a plurality of image frames captured for a certain period of
time; and detecting a plurality of characterizing points in the plurality
of image frames, and setting a characterizing point checking region for
checking characterizing points in a currently input image frame.

11. The characterizing point checking region setting method of claim 10,
wherein the setting the characterizing point checking region comprises:
detecting a plurality of characterizing points in each of the plurality
of image frames; classifying the plurality of characterizing points into
a plurality of clusters for each of the image frames; detecting a
representative centroid point representing the plurality of image frames;
and setting the characterizing point checking region including a major
cluster among the plurality of clusters, based on the representative
centroid point.

12. The characterizing point checking region setting method of claim 11,
wherein a cluster of the plurality of clusters is classified as the major
cluster if the cluster comprises 50% or more of the characterizing
points, and as a minor cluster if the cluster comprises less than 50% of
the characterizing points.

13. The characterizing point checking region setting method of claim 12,
wherein the representative centroid point is detected by detecting
centroid points of the major clusters and the minor clusters, calculating
averages values of the centroid points in the clusters, and dividing the
average values by the number of the plurality of image frames to detect
the representative centroid point.

14. The characterizing point checking region setting method of claim 11,
further comprising checking whether the characterizing point checking
region includes a standard level of characterizing points of the image
frames extracted for the certain period of time or greater.

15. An image stabilizing apparatus comprising: the characterizing point
checking region setting unit of claim 1; and an image adjusting unit
which sets the characterizing point checking region in an image frame
that is currently input, compares the currently input image frame with a
reference image that is preset, and adjusts the currently input image
frame as much as a shaking amount when it is determined that the current
image frame is shaken.

16. The image stabilizing apparatus of claim 15, further comprising
reference image setting unit which extracts an image frame that is the
least shaken among the plurality of image frames taken for a certain
period of time, and sets the extracted image frame as the reference image

17. The image stabilizing apparatus of claim 16, wherein the reference
image setting unit extracts the least shaken image frame by determining
distances between a plurality of characterizing points and a center point
in each of the image frames.

18. The image stabilizing apparatus of claim 17, wherein the reference
image setting unit comprises: a center point detector which receives the
plurality of image frames and detects the center point in each of the
image frames; a characterizing point detector which detects the plurality
of characterizing points in each of the image frames; a frame average
calculator which calculates a plurality of frame averages each of which
is an average of the distances between the center point and the plurality
of characterizing points in each of the image frames; a frame comparison
value calculator which calculates a plurality of frame comparison values
each of which is obtained by summing up absolute values each of which is
obtained by subtracting, from the frame average of each of the frame, a
frame average of another image frame; and a reference frame selector
which selects an image frame having the smallest frame average value as
the reference image.

19. An image stabilizing method comprising: the characterizing point
checking region setting method of claim 10; setting the characterizing
point checking region in an image frame that is currently input;
comparing the currently input image frame with a reference image that is
preset; and adjusting the currently input image frame as much as a
shaking amount when it is determined that the current image frame is
shaken.

20. The image stabilizing method of claim 19, further comprising:
extracting an image frame that is the least shaken among the plurality of
image frames taken for a certain period of time; and setting the
extracted image frame as the reference image.

Description:

CROSS-REFERENCE TO RELATED PATENT APPLICATION

[0001] This application claims priority from Korean Patent Application No.
10-2012-0003450, filed on Jan. 11, 2012 in the Korean Intellectual
Property Office, the disclosure of which is incorporated herein in its
entirety by reference.

BACKGROUND

[0002] 1. Field

[0003] Apparatuses and methods consistent with exemplary embodiments
relate to image stabilization, and more particularly, to a characterizing
point checking region setting apparatus and method, and an image
stabilizing apparatus including the characterizing point checking region
setting apparatus.

[0004] 2. Description of the Related Art

[0005] In order to exactly detect a certain object, in particular, a
moving object, by using a camera, each image has to be stabilized.
However, it may be difficult to detect a certain object because captured
images are shaken due to various external causes. For example, when a
certain object is photographed in a state where a camera is exposed to an
outside environment, the camera may slightly move due to, for example,
wind or external shock. In addition, when the camera is mounted on a
movable apparatus, the camera may be shaken according to movement of the
movable apparatus. The shaking of images becomes severe as more external
shock is applied to the camera, and eventually the object may not be
detected exactly. An image stabilization technology is used to detect an
object exactly by stabilizing the shaken images.

[0006] A Korean patent (KR 2008-0083525; Method for stabilizing digital
image which can correct the horizontal shear distortion and vertical
scale distortion) discloses a related art image stabilization method.
According to the related art image stabilization method, a current frame
is corrected by using characterizing points extracted from the current
frame and characterizing points extracted from a previous frame.
According to this image stabilization method, however, if a shaking
degree of the image increases, image correction may not be stably
performed.

SUMMARY

[0007] One or more exemplary embodiments provide an apparatus and method
of setting an optimal characterizing point checking region, and an image
stabilizing apparatus for correcting and stabilizing shaking images by
using the characterizing point checking region.

[0008] According to an aspect of an exemplary embodiment, there is
provided a characterizing point checking region setting unit including: a
sample frame extract unit which extracts a plurality of image frames,
obtained for a certain period of time, from image data obtained by
photographing an object; and a frame analyzing unit which detects a
plurality of characterizing points in the extracted plurality of image
frames, and sets a characterizing point checking region which is used to
check characterizing points in a currently input image frame.

[0009] The frame analyzing unit may include: a characterizing point
detector which receives the plurality of image frames from the sample
frame extract unit, and detects the plurality of characterizing points in
the plurality of image frames; a characterizing point classification unit
which classifies the plurality of characterizing points into a plurality
of clusters for each of the image frames; a center point detector which
detects a centroid point of the characterizing points in the plurality of
image frames; and a check region determination unit which sets the
characterizing point checking region including a major cluster among the
plurality of clusters based on the centroid point.

[0010] According to an aspect of another exemplary embodiment, there is
provided a characterizing point checking region setting method including:
receiving a plurality of image frames captured for a certain period of
time; and detecting a plurality of characterizing points in the plurality
of image frames, and setting a characterizing point checking region for
checking the detected characterizing points, in a currently input image
frame.

[0011] The setting the characterizing point checking region may include:
detecting a plurality of characterizing points in each of the plurality
of image frames; classifying the plurality of characterizing points into
a plurality of clusters for each of the image frames; detecting a
representative centroid point representing the plurality of image frames;
and setting the characterizing point checking region including a major
cluster among the plurality of clusters, based on the representative
centroid point.

[0012] According to an aspect still another exemplary embodiment, there is
provided an image stabilizing apparatus including: the above
characterizing point checking region setting unit; and an image adjusting
unit which sets the characterizing point checking region in an image
frame that is currently input, compares the currently input image frame
with a reference image that is preset, and adjusts the currently input
image frame as much as a shaking amount when it is determined that the
current image frame is shaken.

[0013] The image stabilizing apparatus may further include: a reference
image setting unit which extracts an image frame that is the least shaken
among the plurality of image frames taken for a certain period of time,
and sets the extracted image frame as the reference image.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other aspects will become more apparent by describing
in detail exemplary embodiments with reference to the attached drawings,
in which:

[0015] FIG. 1 is a block diagram of an image stabilizing apparatus
according to an exemplary embodiment;

[0016] FIG. 2 is a detailed block diagram of a reference image setting
unit shown in FIG. 1, according to an exemplary embodiment;

[0018] FIG. 4 is a flowchart illustrating a method of setting a reference
image performed by the reference image setting unit shown in FIG. 2,
according to an exemplary embodiment;

[0019]FIG. 5 is a flowchart illustrating an operation of the method shown
in FIG. 4 in detail, according to an exemplary embodiment;

[0020] FIG. 6 is a detailed block diagram of a characterizing point
checking region setting unit shown in FIG. 1, according to an exemplary
embodiment;

[0021]FIG. 7 is a diagram showing examples of detected centroid points
according to an exemplary embodiment;

[0022] FIGS. 8A and 8B are diagrams illustrating a method of setting a
characterizing point checking region, according to an exemplary
embodiment;

[0023] FIGS. 9A and 9B are diagrams of set optimal characterizing point
checking regions according to an exemplary embodiment;

[0024] FIG. 10 is a flowchart illustrating a method of setting a
characterizing point checking region performed by the characterizing
point checking region setting unit shown in FIG. 6, according to an
exemplary embodiment;

[0025] FIG. 11 is a flowchart illustrating a second operation of the
method shown in FIG. 10 in detail, according to an exemplary embodiment;

[0026] FIG. 12 is a detailed block diagram of an image adjusting apparatus
shown in FIG. 1, according to an exemplary embodiment;

[0027] FIG. 13 is an image showing an example of an optical flow according
to an exemplary embodiment;

[0028]FIG. 14 is a diagram showing representative directions of the
optical flow according to an exemplary embodiment;

[0029]FIG. 15 is an image showing a state where an image is adjusted
according to an exemplary embodiment;

[0030] FIGS. 16A and 16B are graphs showing shaken degrees of image
frames, according to an exemplary embodiment; and

[0031]FIG. 17 is a flowchart illustrating a method of adjusting an image
performed by the image adjusting apparatus of FIG. 12, according to an
exemplary embodiment.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

[0032] Hereinafter, exemplary embodiments will be described in detail with
reference to accompanying drawings. Like reference numerals denote like
elements.

[0033] FIG. 1 is a block diagram of an image stabilizing apparatus 100
according to an exemplary embodiment. The image stabilizing apparatus 100
receives image data P1 that is obtained by a camera (not shown)
photographing an object, and stabilizes images included in the image data
P1. When the object is continuously photographed by the camera in a state
of being fixed, obtained images are stabilized. However, if the object is
photographed in a state where the camera is shaken, obtained images are
also shaken, and accordingly the photographed object may not exactly be
distinguished from other objects or an environment. When the images are
shaken as described above, the image stabilizing apparatus 100 stabilizes
the image by moving the shaking object to an original position in the
image.

[0035] The reference image setting unit 111 extracts an image frame that
is shaken least among a plurality of image frames included in the image
data P1 obtained by photographing the object, and then sets the extracted
image frame as a reference image. The reference image setting unit 111
outputs a signal P2 indicating the reference image to the image adjusting
unit 131. The reference image setting unit 111 is described in more
detail below with reference to FIGS. 2 through 5.

[0036] The characterizing point checking region setting unit 121 receives
the image data P1 input from outside, and sets a characterizing point
checking region. The characterizing point checking region setting unit
121 generates a signal P3 indicating the characterizing point checking
region, and outputs the signal P3 to the image adjusting unit 131. The
characterizing point checking region setting unit 121 will be described
in detail with reference to FIGS. 6 through 11.

[0037] The image adjusting unit 131 receives the signals P2 and P3. The
image adjusting unit 131 sets the characterizing point checking region in
the image included in the image data P1 that is currently input, and
compares the currently input image with the reference image so as to
adjust and stabilize the currently input image according to a shaken
degree of the image when the currently input image is shaken. The image
adjusting unit 131 is described in more detail below with reference to
FIGS. 12 through 17.

[0039] The sample frame extract unit 211 receives the image data P1 from
outside. The image data P1 is obtained by continuously photographing an
object with the camera. The image data P1 includes a plurality of image
frames. For example, the image data P1 includes a plurality of image
frames as shown in FIGS. 3A, 3B and 3C, each including a building located
on a right side of the image frame. The image frames in FIG. 3A and 3C
show states where images are shaken vertically relative to the image
frame in FIG. 3B. The sample frame extract unit 211 extracts a plurality
of image frames taken for a certain time period from the image data P1.
The image data P1 may include hundreds to tens of thousands of image
frames per second according to performance of the camera. Therefore, the
certain period of time may be set as 1 second or shorter if a shutter
speed of the camera is fast, and may be set to be longer than 1 second if
the shutter speed of the camera is slow. However, the present embodiment
is not limited to this example.

[0040] The reference frame extract unit 221 receives the plurality of
image frames extracted by the sample frame extract unit 211, and compares
the received image frames with each other to extract the most stabilized
image frame and sets the most stabilized image frame as a reference
image. The most stabilized image frame is an image frame of which a
shaking degree is the least.

[0041] The reference frame extract unit 221 includes a center point
detector 231, a characterizing point detector 232, a frame average
calculator 233, a frame comparison value calculator 234, and a reference
frame selector 235.

[0042] The center point detector 231 receives the plurality of image
frames from the sample frame extract unit 211, and detects center points
of the plurality of image frames. That is, the center point detector 231
detects one center point from each of the plurality of image frames. The
center point is located at a center of the image frame and may be
represented as coordinates.

[0043] The characterizing point detector 232 receives the plurality of
image frames from the sample frame extract unit 211 and detects a
plurality of characterizing points in the plurality of image frames. That
is, the characterizing point detector 232 detects the plurality of
characterizing points in each of the plurality of image frames. The
plurality of characterizing points may be represented as coordinates. The
image frame includes various characterizing elements, some of which may
be detected as the characterizing points according to needs of a user. In
order to detect the characterizing points of the image frame, a Harris'
corner detection method, a scale invariant feature transform (SIFT)
algorithm, or a speeded-up robust feature (SURF) algorithm may be used.

[0044] The frame average calculator 233 receives the plurality of center
points detected by the center point detector 231 and the plurality of
characterizing points detected by the characterizing point detector 232,
and calculates a plurality of frame averages. The plurality of frame
averages may be obtained by averaging distances between the center points
and the plurality of characterizing points in corresponding image frames.
When the number of image frames is N (N is an integer greater than zero),
N frame averages may be calculated.

[0045] The frame comparison value calculator 234 receives the plurality of
frame averages from the frame average calculator 233 and calculates a
plurality of frame comparison values. The plurality of frame comparison
values may be obtained by summing up absolute values, which are obtained
by subtracting the frame averages of other image frames from the frame
average of the corresponding image frame. If the number of the frame
averages is N, the number of the frame comparison values is also N. The
frame comparison value Pk (k is an integer) of each of the plurality of
image frames may be calculated by the following equation 1.

Pk=abs{Rk-R0}+abs{Rk-R1}+ . . . +abs{Rk-Rn} (1),

[0046] where abs denotes an absolute value.

[0047] For example, if the number of extracted image frames for the
certain period of time is 5, five frame averages R0 to R4 are calculated,
and five frame comparison values P0 to P4 may be obtained as the
following equation 2.

P0=abs{R0-R1}+abs{R0-R2}+abs{R0-R3}+abs{R0-R4}

P1=abs{R1-R0}+abs{R1-R2}+abs{R1-R3}+abs{R1-R4}

P2=abs{R2-R0}+abs{R2-R1}+abs{R2-R3}+abs{R2-R4}

P3=abs{R3-R0}+abs{R3-R1}+abs{R3-R2}+abs{R3-R4}

P4=abs{R4-R0}+abs{R4-R1}+abs{R4-R2}+abs{R4-R3} (2)

[0048] The reference frame selector 235 receives the plurality of frame
comparison values and selects an image frame having the smallest frame
comparison value among the plurality of frame comparison values. The
image frame having the smallest value is set as the reference image. The
smallest frame comparison value represents that the image is least
shaken.

[0049] As described above, the reference image setting unit 111 extracts
the plurality of image frames for the certain period of time from the
image data P1 input from outside and detects an image frame having the
least degree of shaking among the extracted image frames and sets this
image frame as the reference image.

[0050] FIG. 4 is a flowchart illustrating a method of setting the
reference frame by the reference image setting unit 111 of FIG. 2.
Referring to FIG. 2, the method of setting the reference image includes
operation S411 and operation S421.

[0051] In operation S411, the reference image setting unit 111 extracts
the plurality of image frames taken for a certain period of time among
the plurality of image frames included in the image data P1 input from
outside.

[0052] In operation S421, the reference image setting unit 111 compares
the plurality of extracted image frames with each other to detect and set
the image frame that is the most stabilized as the reference image. The
most stabilized image frame denotes an image frame, of which a shaking
degree is the least among the image frames.

[0053]FIG. 5 is a flowchart illustrating the operation S421 of FIG. 4 in
more detail. Referring to FIG. 5, the operation S421 of FIG. 4 includes
four sub-operations S511 through S541.

[0054] In operation S511, the reference image setting unit 111 extracts
the center point and the plurality of characterizing points from each of
the plurality of extracted image frames.

[0055] In operation S521, the reference image setting unit 111 calculates
an average of distances between the center points and the plurality of
characterizing points in each of the image frames, that is, a frame
average.

[0056] In operation S531, the reference image setting unit 111 calculates
a sum of absolute values that are obtained by subtracting other frame
averages from the frame average of each image frame, that is, a frame
comparison value. That is, the reference image setting unit 111
calculates the plurality of frame comparison values by using equation 1
above.

[0057] In operation S541, the reference image setting unit 111 detects an
image frame having the smallest frame comparison value among the
plurality of frame comparison values and sets the detected image frame as
the reference image.

[0058] Therefore, the reference image setting unit 111 detects the image
frame of which a shaking degree is the least among the plurality of image
frames included in the image data P1 and then sets the detected image
frame as the reference image.

[0060] The sample frame extract unit 611 receives the image data P1 from
outside. The image data P1 includes a plurality of image frames that are
obtained by photographing an object continuously. The sample frame
extract unit 611 extracts a plurality of image frames taken for a certain
time period among the plurality of image frames included in the image
data P1. The image data P1 obtained by photographing the object with the
camera may include hundreds to tens of thousands of image frames per
second according to performance of the camera. Therefore, the certain
period of time may be set as 1 second or shorter if the shutter speed of
the camera is fast, and may be set to be longer than 1 second if a
shutter speed of the camera is slow. However, the present embodiment is
not limited to this example.

[0061] The frame analyzing unit 621 receives the plurality of image frames
that are extracted for the certain period of time from the sample frame
extract unit 611. The frame analyzing unit 621 detects a plurality of
characterizing points in the plurality of image frames, and sets an
optimal characterizing point checking region by using the plurality of
characterizing points. The frame analyzing unit 621 outputs a signal P3
representing the characterizing point checking region.

[0062] The frame analyzing unit 621 includes a characterizing point
detector 631, a characterizing point classification unit 632, a center
point detector 633, a checking region setting unit 634, and a checking
region adjusting unit 635.

[0063] The characterizing point detector 631 receives the plurality of
image frames extracted for the certain period of time from the sample
frame extract unit 611 and detects a plurality of characterizing points
(723, 733 of FIG. 7) in each of the plurality of image frames. That is,
the characterizing point detector 631 detects a plurality of
characterizing points (723, 733 of FIG. 7) in each of the plurality of
image frames. Each of the plurality of characterizing points (723, 733 of
FIG. 7) may be represented as coordinates. Each of the image frames
includes various characterizing elements, some of which may be detected
as the characterizing points (723, 733 of FIG. 7) according to setting by
a user. In order to detect the characterizing points (723, 733 of FIG. 7)
of the image frame, a Harris' corner detection method, a SIFT algorithm,
or an SURF algorithm may be used.

[0064] The characterizing point classification unit 632 classifies the
plurality of characterizing points (723, 733 of FIG. 7) detected by the
characterizing point detector 631 as a plurality of clusters (721 and 731
of FIG. 7), for example, a major cluster (721 of FIG. 7) and a minor
cluster (731 of FIG. 7), for each of the image frames (711 of FIG. 7).
The major cluster (721 of FIG. 7) includes 50% or more of the
characterizing points, and the minor cluster (731 of FIG. 7) includes
less than 50% of the characterizing points. As described above, since the
major cluster (721 of FIG. 7) includes more characterizing points than
the minor cluster (731 of FIG. 7), the major cluster 721 may be wider
than the minor cluster 731 as shown in FIG. 7. In order to classify the
characterizing points (723, 733 of FIG. 7) as a plurality of clusters
(721 and 731 of FIG. 7), a k-mean clustering method and a support vector
machine (SVM) method may be used as an example.

[0065] The center point detector 633 detects a centroid point (741 of FIG.
7) of the characterizing points in the plurality of image frames. To do
this, the center point detector 633 detects a centroid point (723 of FIG.
7) of the major cluster 721 and a centroid point (733 of FIG. 7) of the
minor cluster 731 that are classified by the characterizing point
classification unit 632 in each of the image frames (711 of FIG. 7). The
center point detector 633 calculates an average between the centroid
point (723 of FIG. 7) of the major cluster 721 and the centroid point
(733 of FIG. 7) of the minor cluster 731 to detect the centroid point
(741 of FIG. 7) in each of the image frames. The centroid point (741 of
FIG. 7) in each of the image frames is generally adjacent to the major
cluster (721 of FIG. 7) as shown in FIG. 7. The center point detector 633
calculates an average of the centroid points (741 of FIG. 7) of the
plurality of image frames, and detects a representative centroid point
(751 of FIGS. 8A and 8B) of the plurality of image frames, as shown in
FIGS. 8A and 8B. The average of the centroid points (741 of FIG. 7) in
the plurality of image frames may be calculated by summing the centroid
points (741 of FIG. 7) of the plurality of image frames, and dividing the
sum by the number of image frames. The centroid points described above
may be represented as coordinates.

[0066] The checking region setting unit 634 sets a characterizing point
checking region (811 of FIG. 8A or 821 of FIG. 8B) including all of the
major clusters (721 of FIG. 7) of the plurality of image frames based on
the representative centroid point (751 of FIGS. 8A and 8B) detected by
the center point detector 633, as shown in FIGS. 8A and 8B. The
characterizing point checking region 811 or 821 may be formed in various
shapes, for example, may be formed as a region 811 denoted by a circle as
shown in FIG. 8A or may be formed as a region 821 denoted by a square as
shown in FIG. 8B.

[0067] The checking region adjusting unit 635 determines whether the
characterizing point checking region 811 or 821 includes a standard level
or greater of the characterizing points (921 of FIG. 9B) of the image
frames extracted for the certain period of time. The standard level may
be set as 80% of the characterizing points 921. The checking region
adjusting unit 635 expands the characterizing point checking region 811
or 821 so as to include the standard level of characterizing points, if
the characterizing points included in the characterizing point checking
region 811 or 821 are less than the standard level. FIG. 9B shows a state
where the adjustment is finished and an optimal characterizing point
checking region 911 is set, and FIG. 9A shows one of the plurality of
image frames.

[0068] As described above, since the characterizing point checking region
setting unit 121 sets the optimal characterizing point checking region
911, a time taken to test the characterizing points of the image
stabilizing apparatus 100 may be greatly reduced.

[0069] FIG. 10 is a flowchart illustrating a method of setting the
characterizing point checking region performed by the characterizing
point checking region setting unit 121 shown in FIG. 6, according to an
exemplary embodiment. Referring to FIG. 10, the method includes operation
S1011 and operation S1021.

[0070] In operation S1011, the characterizing point checking region
setting unit (121 of FIG. 6) extracts a plurality of image frames taken
for a certain period of time among the plurality of image frames included
in the image data (P1 of FIG. 6) input from outside.

[0071] In operation S1021, the characterizing point checking region
setting unit 121 detects a plurality of characterizing points (921 of
FIG. 9B) in the plurality of extracted image frames and sets the optimal
characterizing point checking region (911 of FIG. 9B) by using the
plurality of characterizing points 921.

[0072] FIG. 11 is a flowchart illustrating the operation S1021 shown in
FIG. 10 in more detail. Referring to FIG. 11, the operation S1021 shown
in FIG. 10 includes sub-operations S1111 through 1151.

[0073] In operation S1111, the characterizing point setting unit 121
extracts a plurality of characterizing points (723, 733 of FIG. 7) from
each of the plurality of extracted image frames.

[0074] In operation S1121, the characterizing point checking region
setting unit 121 classifies the plurality of detected polarizing points
(723, 733 of FIG. 7) as a plurality of clusters (721 and 731 of FIG. 7),
for example, the major cluster 721 and the minor cluster 731, for each of
the image frames. The major cluster 721 is set to include 50% or greater
of the characterizing points, and the minor cluster 731 is set to include
less than 50% of the characterizing points.

[0075] In operation S1131, the characterizing point checking region
setting unit 121 detects the representative centroid point (751 of FIGS.
8A and 8B) of the plurality of image frames. That is, the characterizing
point checking region setting unit 121 detects the centroid points (723
and 733 of FIG. 7) from each of the plurality of clusters 721 and 731,
and calculates the average of the centroid points 723 and 733 of the
plurality of clusters 721 and 731 for each of the image frames to detect
the centroid point 741 of each of the image frames. In addition, the
centroid points 741 of the plurality of image frames are summed, and the
sum is divided by the number of image frames to detect the representative
centroid point 751 of the plurality of image frames.

[0076] In operation S1141, the characterizing point checking region
setting unit 121 sets the characterizing point checking region (811 of
FIG. 8A or 821 of FIG. 8B) that includes all of the major clusters 721
based on the representative centroid point 751.

[0077] In operation S1151, the characterizing point checking region
setting unit 121 determines whether the characterizing point checking
region 811 or 821 includes the standard level of characterizing points
921 of the image frames extracted for the certain period of time or
greater. When an amount of the characterizing points included in the
characterizing point checking region 811 or 821 is less than the standard
level, the characterizing point checking region setting unit 121 expands
the characterizing point checking region 811 or 821 to include the
standard level of characterizing points. The standard level may be set as
80% of the characterizing points 921. Therefore, the optimal
characterizing point checking region 911 may be set.

[0078] As described above, the characterizing point checking region
setting unit 121 sets the optimal characterizing point checking region
911 by using the plurality of image frames included in the image data P1
input from outside, and thus, a time that is taken to check the
characterizing points of the image frames is greatly reduced.

[0080] The image analyzing unit 1201 compares a current image frame
included in the image data P1 input from outside with the predetermined
reference image included in the reference image signal P2 and extracts a
representative direction and a representative magnitude of the shaking if
the current image frame is shaken.

[0082] The optical flow calculator 1211 compares the current image frame
with the reference image to calculate an optical flow (1321 of FIG. 13)
in the characterizing point checking region 911. As shown in FIG. 13, the
optical flow 1321 has a direction and a magnitude. A method of
calculating the optical flow 1321 is well known in the art, and thus
detailed descriptions thereof are not provided here. The reference image
is an image frame of which a shaking degree is the least among the
plurality of image frames taken for the certain period of time. The
optical flow calculator 1211 may receive the reference image from the
reference image setting unit shown in FIG. 2. The method of setting the
reference image is described above with reference to FIGS. 2 through 5.

[0083] The representative direction extractor 1221 inputs the optical flow
1321 calculated by the optical flow calculator 1211. The representative
direction extractor 1221 extracts a representative shaking direction of
the currently input image frame from the optical flow 1321. The shaking
direction of the image may be set in eight (8) directions, for example,
east direction, west direction, south direction, north direction,
south-east direction, north-east direction, south-west direction, and
north-west direction. The representative direction extractor 1221
determines which one of the eight directions is the representative
direction of the optical flow 1321 and sets the direction as the
representative direction of the currently input image frame. The shaking
direction of the image may be divided in more detail, for example, 12
directions, 24 directions, or 36 directions.

[0084] The representative magnitude extractor 1231 inputs the optical flow
1321 calculated by the optical flow calculator 1211. The representative
magnitude extractor 1231 extracts a representative shaking magnitude of
the currently input image frame from the optical flow 1321. The
representative shaking magnitude of the image frame may be obtained by
converting magnitudes of the optical flow having the representative
shaking direction into a histogram, and averaging vectors included in a
range having the largest number of bins in the histogram.

[0085] The image moving unit 1241 moves the currently input image frame as
much as the representative magnitude extracted by the representative
magnitude extractor 1231 in an opposite direction to the representative
direction extracted by the representative direction extractor 1221. That
is, the image moving unit 1241 moves the image frame as much as the
magnitudes of Table 1 below in the directions shown in Table 1. In Table
1, minus (-) denotes the opposite direction, and the representative
directions are the directions shown in FIG. 14.

[0086] Referring to Table 1, the image moving unit 1241 moves the current
image frame on the X-axis as much as the representative magnitude in the
opposite direction to the representative direction, when the
representative direction is an X-axis (1, 5). In addition, the image
moving unit 1241 moves the current image frame on the Y-axis as much as
the representative magnitude in the opposite direction to the
representative direction when the representative direction is a Y-axis
(3, 7). However, when the representative direction is a diagonal
direction (2, 4, 6, 8), the image moving unit 1241 moves the current
image frame in a diagonal line as much as (representative magnitude/
{square root over (2)}) in the opposite direction by using trigonometric
functions. After that, four sides of the moved current image frame are
trimmed in consideration of the representative direction and the
representative magnitude. Therefore, the image may be stabilized as shown
in FIG. 15. The image moving unit 1241 outputs a signal P4 representing
the stabilized image.

[0088] Referring to FIG. 16A, large deviation is shown between locations
of pixels in the image frames. That is, FIG. 16A shows a state where an
image is severely shaken and is unstable.

[0089] Referring to FIG. 16B, small deviation is shown between locations
of the pixels in the image frames. That is, FIG. 16B shows a state where
the image is stabilized.

[0090] As described above, the optical flow of the image input to the
image adjusting unit 131 is calculated to extract the representative
direction and the representative magnitude of the image. Then, if the
image is shaken, the image is moved as much as the representative
magnitude in the opposite direction to the representative direction.
Thus, the shaking may be corrected and the image may be stabilized.

[0091]FIG. 17 is a flowchart illustrating a method of adjusting the image
by the image adjusting unit 131 shown in FIG. 12, according to an
embodiment. Referring to FIG. 17, the method of adjusting the image
includes operations S1711 through S1731.

[0092] In operation S1711, the image adjusting unit 131 compares the
current image frame input from outside with the preset reference image to
calculate the optical flow (1321 of FIG. 13). The reference image is
input from outside to the image adjusting unit 131.

[0093] In operation S1721, the image adjusting unit 131 extracts the
representative direction and the representative magnitude of the shaking
of the currently input image from the optical flow 1321.

[0094] In operation S1731, the image adjusting unit 131 moves the image
frame that is currently input as much as the representative magnitude in
the opposite direction to the representative direction. In more detail,
if the representative direction is an X-axis direction, the image
adjusting unit 131 moves the current image frame as much as the
representative magnitude in the opposite direction to the representative
direction on the X-axis. If the representative direction is a Y-axis
direction, the image adjusting unit 131 moves the current image frame as
much as the representative magnitude in the opposite direction to the
representative direction on the Y-axis. However, when the representative
direction is a diagonal direction, the image adjusting unit 131 moves the
current image frame as much as (representative magnitude/ {square root
over (2)}) in the opposite direction to the representative direction on
the diagonal line. After that, the four sides of the moved current image
frame are trimmed in consideration of the representative direction and
the representative magnitude. Therefore, the shaking is corrected, and
the stabilized image may be obtained as shown in FIG. 15.

[0095] According to the exemplary embodiments, the optimal characterizing
point checking region is set by using the image frames extracted for a
certain period of time, and thus a time to check the characterizing
points of the currently input image frame may be reduced greatly.

[0096] In addition, since the shaking of the currently input image frame
is corrected by using the optimal characterizing point checking region,
the image correction time may be reduced and the image may be optimally
stabilized.

[0097] While the inventive concept has been particularly shown and
described with reference to exemplary embodiments thereof, it will be
understood by those of ordinary skill in the art that various changes in
form and details may be made therein without departing from the spirit
and scope of the inventive concept as defined by the following claims.