Sign up to receive free email alerts when patent applications with chosen keywords are publishedSIGN UP

Abstract:

According to aspects of embodiments, a method of determining an aperture
window in a video deinterlacer includes: computing a first mean absolute
difference (MAD) value for a pixel based on a 1×5 window; computing
a second MAD value for the pixel using a 3×5 window; and
selectively blending the first and second MAD values to form a resultant
motion value.

Claims:

1. A method of determining an aperture window, comprising:computing a
first mean absolute difference (MAD) value for a pixel based on a
1.times.5 window;computing a second MAD value for the pixel using a
3.times.5 window;andselectively blending the first and second MAD values
to form a resultant motion value.

[0003]The present invention relates to motion detection used in connection
with deinterlacing fields of an interlaced video signal.

[0004]2. Discussion of Related Art

[0005]Video images are displayed on monitors or display screens of various
types as a time-sequence of frames shown quickly enough to deceive the
human eye into perceiving that sequence of frames as a continuously
moving picture. Each frame consists of a collection of lines of pixels,
the precise number of which depend on the technology implemented. For
historical reasons, for example reasons which took advantage of older
technologies to reduce perceived flicker arising from such an
arrangement, each frame was further divided into two fields, each of
which contained half of the lines comprising the frame. One field
contained every other line of the frame, e.g. the even lines, and the
other field contained the lines not included in the first field, e.g. the
odd lines. Fields were displayed in even, odd, even, odd, etc. sequence,
at twice the frame rate, thus reducing flicker and displaying the
sequence of frames at the frame rate. Such an arrangement of fields to
form frames is referred to as an interlaced scan display.

[0006]Another way of displaying frames is to simply display each line of a
frame in sequence, and with such timing as to display frames at the
desired frame rate. Such an arrangement of display is referred to as a
progressive scan display.

[0007]When video images are broken down into a sequence of serial elements
so as to form a serial signal that can be transmitted from a video image
source to a video image display, the two-field arrangement is
conventionally still used. Therefore, video image display devices that
employ progressive scan display de-interlace the signal so it can then be
used to produce entire fields at one time.

[0008]In conventional deinterlacer devices, for example, the Supra HD780
available from Zoran Corporation, frame motion detection is used as shown
in FIG. 1. Conventional frame motion detection compares fields of the
same polarity, i.e., even-even or odd-odd.

[0009]A video image stream 101 is loaded into a buffer 103. At time t-1,
frame motion detection is performed by a detector 105 between Field 0 and
Field 2. A motion value 107 is computed for each corresponding pair of
pixels based on a finite window of pixels around that for which the value
is being computed. This motion value is then quantized into a 2-bit
motion value, k, by comparing a quantitative measure of motion detected
to thresholds. This motion value, k, is then written to memory 109, which
would be read back at time t. Also at time t-1, motion values 108
representing motion detected between Field 1 and Field 3 which were
previously computed and stored in memory 109, or simultaneously computed
by detector 111 and stored in memory 109 are read from memory 109. Hence,
at time t-1, two motion values are available. Conventionally, one frame
motion detector (frame motion detector 1-3) 111 detects motion between
field 1 and field 3, while another (frame motion detector 0-2) 105
detects motion between field 0 and field 2. (See also FIG. 2). A goal of
this process is to predict the values of pixels in a frame which are not
part of the current field, i.e., the current missing pixels.

[0010]Processing by a processor 113, then proceeds as follows: The buffer
103 is read out 115 and an output stream 117 either weaved or
interpolated are described. To calculate the final k value for a
particular current missing pixel, the following is done:

[0017]The motivation for utilizing a 3×5 window is to suppress the
effects of noise.

[0018]However, because such a big window is used for motion detection,
motion that is far away from the missing pixel will affect the amount of
motion detected at the missing pixel, resulting in the detection of
motion where there is none.

[0019]The MAD values computed by the motion detector are compared to 3
programmable thresholds to create a 2-bit `k` value:

[0020]This final result from (`k` value) is used by the output blender to
generate the missing lines of the output frame, at each field time. The
output frame is either weaved from successive odd/even field,
interpolated from a single field, or a combination of the two. If motion
in the scene is significant, weaving between fields is more likely to
produce a field tearing artifact. A 2-bit `k` motion value allows for
four blending ratios in the output blender. The values currently used are
summarized in the table below:

[0022]In the Supra HD780, field motion detectors detect motion on two
fields of opposite polarities, in addition to frame motion detection
based on fields of the same polarity. When the frame motion detection and
the field motion detection disagree substantially, a recursive frame
motion algorithm is used to detect motion. A counter is used to keep
track of situations when the results of the frame motion detector and
field motion detector conflict. The counter value is compared to a
threshold (moving_threshold below) and used to modify the `k` motion
value.

[0024]The recursive motion method embodies in the Supra HD780 requires a
4-bit per pixel field buffer to store the historical counter value, based
on using a 2-bit k motion value. This method still produces artifacts in
static high vertical frequency images, where single-time details are
present.

SUMMARY OF INVENTION

[0025]According to aspects of embodiments of the invention, a method
includes: computing a first mean absolute difference (MAD) value for a
pixel based on a 1×5 window; computing a second MAD value for the
pixel using a 3×5 window; and selectively blending the first and
second MAD values to form a resultant motion value. In some variations,
the method may further include selectively blending according to:

[0027]where mad_core_thresh and mad_core_normalize are programmable
registers, and mad_curr_cored is made to saturate at a value of 16.
Forcing mad_curr_cored to saturate at 16 may be done by any suitable
means. The following illustrates the transfer function of the blending
factor:

BRIEF DESCRIPTION OF DRAWINGS

[0028]The accompanying drawings are not intended to be drawn to scale. In
the drawings, each identical or nearly identical component that is
illustrated in various figures is represented by a like numeral. For
purposes of clarity, not every component may be labeled in every drawing.
In the drawings:

[0029]FIG. 1 is a block diagram of a prior art system for frame motion
detection;

[0030]FIG. 2 is a representation of frame memory contents compared using
the prior art system;

[0031]FIG. 3A is an overview block diagram of aspects of an embodiment;

[0032]FIG. 3B is a lower-level block diagram of aspects of an embodiment;

[0033]FIG. 4 is a representation of frame memory contents compared using
the system;

[0034]FIG. 5 is a flow diagram of a method according to an embodiment of
aspects of the invention;

[0035]FIG. 6 is a graph of the blending function; and

[0036]FIG. 7 is a flow diagram of a recursion method according to an
embodiment of aspects of the invention.

DETAILED DESCRIPTION

[0037]This invention is not limited in its application to the details of
construction and the arrangement of components set forth in the following
description or illustrated in the drawings. The invention is capable of
other embodiments and of being practiced or of being carried out in
various ways. Also, the phraseology and terminology used herein is for
the purpose of description and should not be regarded as limiting. The
use of "including," "comprising," or "having," "containing", "involving",
and variations thereof herein, is meant to encompass the items listed
thereafter and equivalents thereof as well as additional items. The
invention is illustrated by the following description of aspects of
embodiments thereof. Some aspects and embodiments relate to different
parts of a larger aspect or embodiment, and represent alternatives that
may be combined variously to form variations on a larger embodiment.

[0038]The exemplary aspects and embodiments relate to deinterlacers used
in high definition television (HDTV) displays, although other
applications will occur to the skilled artisan.

[0039]A high-level overview of the system according to aspects of some
embodiments is shown in FIG. 3A. The elements shown may be implemented as
software executing on a suitable general purpose processor, special
purpose image processor, digital signal processor (DSP), or the like, or
as specialized hardware implementing the various functions in logic, or
as a combination of these. In some implementations, where a function is
called for to process different input values, only one instance of the
function may be incorporated into the implementation, and reused at
different points in time, or, alternatively, multiple instances of the
function may be incorporated into the implementation, and the multiple
instances used in parallel.

[0040]The system includes a memory, 321, in which input buffers, output
buffers, temporary storage registers and the like are defined to hold
various values, such as the values comprising the serial elements of the
serial signal. Connected to the memory, 321, are four frame motion
detectors, 323a, 323b, 323c and 323d, two of which detect motion between
fields 0 and 2, 323a and 323b, and two of which detect motion between
fields 0 and 4, 323c and 323d. Two of the frame motion detectors, 323a
and 323c, employ a three-line by five-pixel window, while the other two
frame motion detectors, 323b and 323d, employ a one-line by five pixel
window. The results of frame motion detection are pair-wise inputs to
selective blending, 325a and 325b, to produce motion values corresponding
to one-frame, frame02, and two-frame, frame04, intervals. The blended
results corresponding to one-frame, frame02, and two-frame, frame04,
intervals are further processed, 327 and 329, as described in connection
with FIG. 5, below.

[0041]A field motion detector, 331, receives frames 0, 1 and 2, and
produces a field motion result. Frame and field motion results are used
by a recursive motion module, 333, to determine whether recursive motion
detection is required by the nature of the serial signal. Finally, the
blend factor computed by selective blending, 325a and 325b, is applied to
a weaver/interpolator, 335, which employs a degree of weaving and an
inverse degree of interpolating determined by the blend factor to create
predicted pixels in an output field.

[0042]The operation of the system whose overview has been presented in
connection with FIG. 3A is now described in connection with FIGS. 3B, 4,
5, 6 and 7.

[0043]According to aspects of embodiments shown in FIG. 3B, a serial
signal 101 representing in digital form an interlaced video signal is
received into a buffer memory 301. A processor, which may be
special-purpose processor, or a general-purpose processor, or a
digital-signal processor reads the signal from the buffer memory,
performs various method steps and writes a result to a buffer memory or
other output. The method includes frame motion detection, windowing and
recursive frame motion detection as described below.

Basic Motion Detection

[0044]According to aspects of an embodiment, frame motion detection is
done between three pairs of fields. Frame motion detection is done
between fields 0 and 2 by detector 303 and fields 1 and 3, by detector
305 as well as between fields 0 and 4 by detector 307. This improves the
overall frame motion detector's ability to detect motion over that of
conventional detectors, hence reducing feathering artifacts. (See also
FIG. 4).

[0045]Detector results are stored in memory 309, after which they are
processed by processor 311 then produces an output stream 313 from the
series of fields 315 read from buffer 301. In accordance with the method
according to aspects of an embodiment, as shown in FIG. 5, to calculate
the final motion value, the following is done:

[0051]When an object is moving within the 5-field window, the motion value
of frame04 will be about twice that of frame02, and so the maximum of
either frame02 or frame04/2 is used, 509. However, when the object is
fast moving (i.e. it moves beyond the 5-field window), the motion values
of frame04 and frame02 will be the same. The pseudocode above provides
that the value of frame02 is then used, 511.

[0052]With this new current frame motion value, the same conditions apply
as before, to calculate the final motion value:

[0053]In contrast with conventional systems, aspects of an embodiment use
the minimum value of frame13_top and frame13_bot, 515, instead of using
the maximum. According to this aspect, the most accurate motion value is
computed, instead of biasing the value to the maximum.

[0054]Biasing the value to the maximum was conventionally thought to be
superior because doing so forced interpolation to be performed more often
in instances where the determination of motion remained uncertain, thus
avoiding feathering artifacts that would result from improperly blending
when interpolation should be called for. The conventional approach is
based on an assumption that the motion detector cannot detect all motion,
and therefore should be biased towards detecting more motion. This
approach also, however, increases flickering or reduced detail artifacts
by interpolating excessively, when blending should have been done
instead.

[0055]Using the most accurate motion value as described above has now been
found to be superior, particularly in connection with the described
aspects of embodiments because these aspects of embodiments detect more
of the motion actually present, thus increasing, without an artifice such
as biasing, the number of instances in which the motion detector detects
motion actually present and therefore indicates interpolation should be
performed rather than blending. By better matching the use of
interpolation and blending to the actual presence or absence of motion in
an image, feathering, flickering and reduced detail artifacts are all
avoided.

Interpolation and Blending

[0056]Interpolation or blending is performed as indicated after performing
a Mean Absolute Difference (MAD) computation on a rectangular window of
pixels as follows. The windows may be 3×5 or 1×5 as now
described.

[0057]An error that can occur is called `false interpolation`. False
interpolation occurs when interpolating is performed on pixels for which
it should not be done. False interpolation occurs due to detection of
motion where none is present, and is most visible on objects with
horizontal edge (e.g. a horizontal line). It is not desirable to use a
MAD 3×5 window at the edges of a moving horizontal line since false
motion will be detected and false interpolation will result, creating
visibly obvious artifacts. When a horizontal line appears in such a
3×5 window, away from the pixel under consideration at the center
of the window, the horizontal line may appear to signify motion at the
pixel under consideration, similarly to noise as described in connection
with conventional systems.

[0058]Hence, a 1×5 window should be used at horizontal edges to
determine the motion value. Of course, the choice of the precise size of
the two windows is left to the skilled artisan, with one window having
multiple lines and a multiple pixel width, while the other window is
vertically narrower to avoid noise and horizontal line artifacts.

[0059]The results of 3×5 window MAD and 1×5 window MAD are
blended based on the difference between the top/mid lines and mid/bottom
lines in the next and previous fields:

where topbot_thresh, blend_res are programmable registers;topbot_thresh
signifies the difference between pixel values that can represent
differentobjects in a picture;blend is the blend factor; andblend_res is
the blending resolution.

[0060]To make the 1×5 window MAD value comparable to the 3×5
window MAD, the 1×5 MAD, the sums of absolute differences for the
detector and window (e.g., sad02_win1×5 and sad02_win3×5, for
frame02) must be multiplied by 3 before being divided by 16 so they can
be blended, as follows.

[0063]where mad_core_thresh and mad_core_normalize are programmable
registers, and mad_curr_cored is made to saturate at a value of 16.
Forcing mad_curr_cored to saturate at 16 may be done by any suitable
means. The following illustrates the transfer function of the blend
factor:

[0064]This blend factor is used to blend between the interpolated line and
the weaved line. The higher the blend factor (i.e. the higher the MAD
value), the more weight is given to the interpolated line. (See FIG. 6).

[0065]It is desired to have a 0 response for low MAD values to mask out
noise. Hence, for one or more values of MAD near or at 0, the blend
factor is forced to saturate at a value of 0 at the low end.

Recursion

[0066]The system performs a recursive computation under specified
conditions, as shown in FIG. 7. This is called recursive motion mode. In
order to perform the method shown in FIG. 7, in addition to basic motion
detection as described above, the system includes two field detectors
(FIG. 3B, 317 and 319), whose outputs are stored and processed (FIG. 3B,
309 and 311) to provide a field motion result which is used together with
the frame motion result as described now. The results of the individual
field motion detectors (FIG. 3B, 317 and 319) are combined, either
algebraically or logically to form the field motion result. Any suitable
function can be used to form the combination, such as taking the maximum
of the outputs of the two field motion detectors (FIG. 3B, 317 and 319).
The field motion detectors are shown in FIG. 3B as separate elements
within the overall motion detector structure; however, skilled artisans
will recognize that the detectors, memory and processor shown may share
or reuse elements, or may be constructed using independent elements, as
various implementations may require for purposes of speed or economy of
hardware.

[0067]The system compares, at 701, the field motion detector result 703
with the frame motion detector result 705. If the field result 703 is
greater than the frame result 705, the system enters into 707 recursive
motion mode 709.

[0068]While in the recursive motion mode 709, the field result 703,
divided, at 711, by a programmable number of steps 713, is written back,
at 715, to a memory location or register used as an accumulator. Once the
accumulator value becomes greater, at 717, than the current field motion
detector result 719, the image is considered to be static, at 721--and
weaving, 723 is done. Otherwise, at 721, while inside the recursive
motion mode 709, the field motion detector result 703 is used as the
blend factor to the final blender 725.

[0069]Recursive motion mode ends, at 727, when the frame motion detector
result is greater than the field motion detector result.

[0071]where steps is a programmable register (power of 2) defining the
number of steps that the deinterlacer will go into weave mode.

[0072]recursive_motion is a one-bit indicator that is written back to
memory.

[0073]recursive_motion_mode is on if either the previous top or previous
bottom line of data has recursive_motion_mode on.

[0074]prev_motion is the motion data in memory. If both previous top and
previous bottom recursive_motion_mode are on, the maximum of the top and
bottom motion data is used. Otherwise, the corresponding motion read back
data is used.

[0075]The motion detector described can be applied to the luminance or
chroma components of an image signal, or to both, in accordance with
various embodiments.

[0076]When film is converted to video, various artifacts peculiar to that
process must be taken into account. They are related to the difference in
frame rates and the accuracy with which physical film frames are aligned
with those digitized counterparts. Therefore, a film detection mode is
used. In film detection mode, a dedicated k motion value is used to
detect cadences, i.e., those variations that may be film conversion
artifacts. This k motion value will also be determined by using the
3×5 MAD window, in an exemplary embodiment.

[0077]Due to bandwidth constraints, an extra field of memory read may not
be possible in certain usage scenarios. Therefore, there will be an
option to use a 3-field deinterlacer instead of 5.

[0078]Since embodiments described above perform 3 field reads from memory,
bandwidth to accomplish cadence detection over an entire image becomes a
major concern. One approach to reduce memory bandwidth is to downsample a
field before writing to memory and upsample when reading back. According
to an exemplary embodiment, this would be done for field4 only. Even
though this requires an extra memory write client, the memory read and
memory write sizes can be significantly reduced, hence reducing both
memory footprint and memory bandwidth.

[0079]The film mode detector mentioned above is, of course, designed for
detecting cadences that occur on the entire image. This solution is best
suited for use when a film source fills each image frame entirely,
however, for overlayed video, where the video from different sources have
different cadence sequences, the film mode detector will fail. One way to
address this issue is to limit cadence detection to predetermined or
other limited regions, including those as small as a single pixel.

[0080]Having thus described several aspects of at least one embodiment of
this invention, it is to be appreciated various alterations,
modifications, and improvements will readily occur to those skilled in
the art. Such alterations, modifications, and improvements are intended
to be part of this disclosure, and are intended to be within the spirit
and scope of the invention. Accordingly, the foregoing description and
drawings are by way of example only.