Easy To Use Patents Search & Patent Lawyer Directory

At Patents you can conduct a Patent Search, File a Patent Application, find a Patent Attorney, or search available technology through our Patent Exchange. Patents are available using simple keyword or date criteria. If you are looking to hire a patent attorney, you've come to the right place. Protect your idea and hire a patent lawyer.

A process and circuit for blending a foreground image (B) with a
background image (A), said foreground and background images being
arranged in pixels and having color representations (R, G, B). The
foreground foreground image (A) has a transparency parameter (T(x,y)) in
accordance with a so-called alpha plane representative of the
transparency profile to apply to the foreground image. The process
involves the steps of:-applying a dithering method on said alpha plane in
order to convert said transparency parameter (T) into a one-bit
transparency parameter (T');-use said one-bit transparency parameter (T')
for controlling a multiplexing unit having two inputs respectively
receiving the foreground image (A) and the background image (B). In one
embodiment, the one-bit transparency parameter T' into the two extreme
values of a range of continuous values, for instance coded on 8 bits. The
process then applies a four-pixel interpolation method to the foreground
image (A) for the purpose of creating a five level transparency parameter
in the blending process, and then uses the five level transparency
parameter for controlling a multiplexing circuit for the purpose of
achieving blending of the foreground image with the background image.

Inventors:

Coupe; David; (Valbonne, FR); Spinelli; Gilles; (Le Tignet, FR)

Serial No.:

142563

Series Code:

13

Filed:

December 29, 2009

PCT Filed:

December 29, 2009

PCT NO:

PCT/EP09/09313

371 Date:

August 29, 2011

Current U.S. Class:

345/592

Class at Publication:

345/592

International Class:

G09G 5/377 20060101 G09G005/377

Foreign Application Data

Date

Code

Application Number

Dec 31, 2008

EP

06368026.4

Claims

1-11. (canceled)

12. A method of blending a foreground image with a background image, said
foreground and background images being arranged in pixels and having
color representations defining the color of each pixel, the foreground
image having further an N-bit transparency parameter associated with each
pixel, the transparency parameters defining an alpha plane representative
of a transparency to apply to the foreground image, the method
comprising, for each pixel: applying a dithering operation on the alpha
plane to convert the N-bit transparency parameter into a one-bit
transparency parameter; multiplexing between the foreground image and the
background image using the one-bit transparency parameter; converting the
one-bit transparency parameter into one of two extreme values of a
predetermined range of continuous values; applying a four-pixel
interpolation method to the foreground image to create a five level
transparency parameter; and multiplexing predetermined fractional
mixtures of the foreground image and the background image using the five
level transparency parameter.

13. The method of claim 12 wherein the predetermined range of continuous
values is represented by an 8-bit value, and wherein the one-bit
transparency parameter is converted into one of hexadecimal values 00 and
FF.

15. The method of claim 12 wherein the dithering operation comprises one
of Halftoning and dithered order operations.

16. A circuit operative to blending a foreground image with a background
image, the foreground and background images being arranged in pixels and
having color representations defining the color of said pixel, the
foreground image further having an N-bit transparency parameter
associated with each pixel, the transparency parameters defining an alpha
plane representative of a transparency to apply to the foreground image,
the circuit comprising, for one or more pixels: a dithering circuit
operative to apply a dithering operation on the alpha plane in order to
convert the N-bit transparency parameter into a one-bit transparency
parameter; a multiplexer operative to select the color components from
the background image or the foreground image, in response to the one-bit
transparency parameter value; a converter operative to convert the
one-bit transparency parameter into one of two extreme values of a
predetermined range of continuous values; an interpolator operative to
perform a four-pixel interpolation method to the foreground image to
create a five level transparency parameter; and a multiplexer operative
to select predetermined fractional mixtures of the foreground image and
the background image using the five level transparency parameter.

17. The circuit of claim 16 wherein the predetermined range of continuous
values is represented by an 8-bit value, and wherein the converter is
operative to convert the one-bit transparency parameter into one of the
hexadecimal values 00 and FF.

18. The circuit of claim 16 wherein the dithering circuit is operative to
apply a Floyd Steinberg dithering operation on the alpha plane.

19. The circuit of claim 16 wherein the dithering circuit is operative to
apply one of a Halftoning and dithered order dithering operation on the
alpha plane.

20. A portable communication device, comprising: a display operative to
output a blended foreground image and background image, the foreground
and background images being arranged in pixels and having color
representations defining the color of said pixel, the foreground image
further having an N-bit transparency parameter associated with each
pixel, the transparency parameters defining an alpha plane representative
of a transparency to apply to the foreground image; and a blending
circuit performing a blending operation for output by the display, the
blending circuit comprising, for one or more pixels, a dithering circuit
operative to apply a dithering operation on the alpha plane in order to
convert the N-bit transparency parameter into a one-bit transparency
parameter; a multiplexer operative to select the color components from
the background image or the foreground image, in response to the one-bit
transparency parameter value; a converter operative to convert the
one-bit transparency parameter into one of two extreme values of a
predetermined range of continuous values; an interpolator operative to
perform a four-pixel interpolation method to the foreground image to
create a five level transparency parameter; and a multiplexer operative
to select predetermined fractional mixtures of the foreground image and
the background image using the five level transparency parameter.

Description

TECHNICAL FIELD

[0001] The present invention relates to digital image processing and more
particularly to a process for blending images.

STATE OF THE ART

[0002] With the advent of digital wireless communications and the
development of mobile telephone, graphic and video processing is taking a
great importance for mobile phone consumers.

[0003] More and more mobile phones get new capabilities where deeper
colour schemes are used as well as video objects can be displayed. Beside
the construction of objects to display, there is a real challenge in
combining and displaying them in the most efficient way.

[0004] The functions to achieve consist is assembling several sources of
graphic or video with the following transformations: [0005] Colour
conversion [0006] Geometric transformation. Ex affine transform:
Translation, Rotation, Scaling, Shearing, Mirroring, and Symmetry. [0007]
Selection and Mixing. The various sources called "plane" will be layered
from background to foreground with simple overlay or more complex
transparency effect. [0008] When a true blending function is applied, the
foreground plane information is weighted by a decimal factor whose value
is in [0,1] range while the background plane is weight by one minus this
factor. The transparency information can come from several sources:
[0009] Global transparency information per layer. [0010] Transparency
information from colour value (also called chroma keying) [0011]
Transparency information from pixel value

[0012] All above functions are generally implemented in the display
composition unit or the graphic renderer. Albeit, the transparency
management is generally the last item integrated in HW since the
transparency management at pixel level is quite consuming in HW resources
and memory footprint.

[0013] Indeed up to colour keying complexity of the underlying hardware is
quite simple since relying on a simple selection whose criterion source
is either a constant value for the whole "plane" or a specific colour
value declared as the transparent one. Supporting a "per pixel" blending
composition capability require a true multiplication and accumulation
function as well as a dedicated transparency information per pixel stored
in system memory.

[0014] These sophisticated blending schemes implicitly discard older
generation HW when it is time to evolve the man machine interface
outlook.

[0015] With respect to FIG. 1A, there is recalled the very classic
chroma-Keying selection process for blending one foreground image A with
a background image B. The mechanism is based, for each pixel to be
displayed, on a multiplexor 10 which receives at a first input the color
(R, B, G component) of an image A which has to be displayed on the
foreground, while a second input receives the color component of an image
B considered as the background. Multiplexor 10 is controlled by a
comparator 11 which compares the component value of the foreground image
with a reference value (CHROMA Keying) which, upon equality, causes
multiplexor 10 to output the background component B.

[0016] The great advantage of this first prior art mechanism results from
the very simple and low cost structure which is advantageous in a small
device such as a mobile telephone. However, the drawback results in the
fact that there is no blending of the images, nor control on the
transparency provided the limited graphical hardware.

[0017] FIG. 1B illustrates another prior art technique which provides
better control on the blending process of the two foreground and
background images. The mechanism is based on one adder 20 which two
inputs respectively receive the output of one multiplier (resp. 21 and
22). Multiplier 21 computes the value A.times.(1-.alpha.) while
multiplier 22 yields .alpha..times.B. There is therefore generated the
value A.times.(1-.alpha.)+a.times.B which provides improved and
"continuous" control on the blending effect and, therefore, more
efficient graphic display.

[0018] The drawback of this second prior art technique lies in the fact
that, firstly, a complex hardware circuitry is necessary for embodying
the two multipliers.

[0019] In addition, more storage space would be required for embodying a 8
bits blending process where the .alpha. value can be different for each
pixel.

[0020] For the sake of illustration, one may consider the following
example simple colour scheme based on indirect colour information access
are quite efficient to store a bitmap. A QVGA image using a colour lookup
table mechanism of 256 colour entries will required 320.times.240 bytes
to store the picture content plus the colour lookup table (CLUT) itself.
Supposing the CLUT entry is made of 3 bytes (Red, Green and Blue channel
coded on 8 bits), the whole image will take:
320.times.240+256.times.3=77568 bytes. Multiple levels of transparencies
are effectively required; it is typically 16, 64 or even 256 levels.
Beyond 16 levels and because of byte alignment constraint, a full
transparency byte will be required for each pixel. In case of the former
QVGA example, it will simply double the storage requirement. Considering
now a "true colour" description where each pixel gets its colour
information directly coded as 16 bits, 18 bits or 24 bits for a 65K, 262K
and 16M colours scheme respectively; the storage is increased by 2 or 3.
Adding the "per pixel blending" is further increasing by a factor of 4
the storage requirement.

[0021] It is desirable to get a graphical rendering as close as possible
to the one obtained with the process of FIG. 1B, but with a combination
of hardware circuitry which is as close as possible to the selection
circuit of FIG. 1A.

[0022] The benefit would be clear: higher reuse of available graphical
circuits and, furthermore, the reduction in the memory footprint
allocated to image storage area.

SUMMARY OF THE INVENTION

[0023] It is an object of the present invention to provide a process for
performing blending of two images with a limited hardware circuitry and
high rendering graphical blending.

[0024] It is a further object of the present invention to provide with a
circuit for controlling transparency of a foreground image with respect
to a background image which requires limited hardware and is cheap to
manufacture.

[0025] It is still a further object of the present invention to carry out
a process and device for controlling blending of two images which are
suitable for mobile telephone equipments.

[0026] These and other objects are achieved by means of a process for
blending a foreground image (B) with a background image (A) in accordance
with a transparency parameter being is coded over N bits.

The foreground and background images being arranged in pixels and have
color representations (e.g. R, G, B) . The transparency parameter
(T(x,y)) defines a so-called alpha plane representative of the
transparency profile to apply to the foreground image. The process
involves the steps of: [0027] applying a dithering method on said alpha
plane in order to convert said transparency parameter (T) into a one-bit
transparency parameter (T'); [0028] using said one-bit transparency
parameter (T') for controlling a multiplexing unit having two inputs
respectively receiving the foreground image (A) and the background image
(B).

[0029] In one embodiment, the one-bit transparency parameter T' is
converted into the two extreme values of a range of continuous values,
for instance coded on 8 bits.

[0030] In one embodiment, the process applies a four-pixel interpolation
method to the foreground image (A) for the purpose of creating a five
level transparency parameter and then uses the results five levels
transparency parameter for controlling a selection circuit based on a
simple multiplexor.

[0031] There is thus achieved a sophisticated (5-level) blending process
with very simple hardware circuitry, based on multliplexing circuit.

[0032] In one embodiment, the one-bit transparency parameter T' is
converted into either 00 or FF in a 8-bit representation format.

[0034] Alternatively, the dithering method may be a Halftoning or dithered
order. Actually any available dithering algorithm can fit.

[0035] The invention also provides a graphical circuit for blending a
foreground image (B) with a background image (A) in accordance with a
transparency parameter (T(x,y)) coder on N-bits and defining an alpha
plane representative of the transparency to apply to the foreground
image. The circuit includes multiplexing means for multiplexing either
the color components from background image (B) or foreground image (A)
and is characterized by [0036] means for applying a dithering method on
said alpha plane in order to convert said transparency parameter (T) into
a one-bit transparency parameter (T'); [0037] means for applying said
one-bit transparency parameter (T') to said multiplexing means.

[0038] In one embodiment, the circuit further includes: [0039] means for
converting said one-bit transparency parameter T' into the two extreme
values of a range of continuous values, for instance coded on 8 bits;
[0040] means for performing a four-pixel interpolation method to the
foreground image (A) for the purpose of creating a five level
transparency parameter in the blending process; [0041] means for applying
said five level transparency parameter for controlling said multiplexing
means so as to achieve improved blending of the foreground image is with
the background image.

[0042] The circuit of the invention is particularly suitable for carrying
out graphical circuits of handheld or mobile telephone displays, but can
encompass other domains where similar needs are required.

DESCRIPTION OF THE DRAWINGS

[0043] Other features of one or more embodiments of the invention will
best be understood by reference to the following detailed description
when read in conjunction with the accompanying drawings.

[0049] FIG. 5 illustrates the principle of the interpolation mechanism
used in the embodiment for generating the interpolated pixel components.

[0050] FIG. 6 illustrates the general structure of one circuit for
embodying the blending process of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0051] The embodiment which is disclosed below is particularly suitable
for the design of a graphical display circuit for small devices such as
handheld, mobile telephones etc. . . . which have limited resources.

[0052] However, it should be clear that this is only one particular
embodiment of the process and circuit of the invention and that the
latter is likely to be adapted to a wide range of applications.

[0053] It is assumed that foreground image A is defined for each pixel
(X,Y) by the three basic components R.sub.A(x,y), G.sub.A(x,y) and
B.sub.A(x,y) band also by a transparency factor T(x,y) which is used for
controlling the blending process. It should be noticed that the system of
representation R-G-B is only given as an example and that another system
of presentation may be considered.

[0054] In the preferred embodiment, each pixel has a representation which
is coded with 32 bits, defining the parameters:

(T(x,y),R.sub.A(x,y),G.sub.A(x,y),B.sub.A(x, y))

[0055] The T(x,y) parameter defines the profile of transparency which will
be used in the blending process, which can be coded with N=8 bits, and
may be represented by a so-called ALPHA PLANE represented in FIG. 3A.

[0056] or (T, R.sub.A, G.sub.A, B.sub.A) in short.

[0057] Similarly, the background image B is assumed to be represented, for
each pixel (x,y) by the components: (R.sub.B, G.sub.B, B.sub.B)

[0058] With respect to FIG. 2A, there is now described one embodiment of
the process for blending a foreground image A with a background image B.

[0059] the process starts with a step 21 which consists in the application
of a dithering method on the alpha plane representative of the
transparency profile of the foreground image to blend, for the purpose of
reducing the T(x,y) transparency parameter to a 1 bit parameter.

[0060] Many dithering techniques may be considered for achieving this 8
bits to 1 bit conversion process, such as, for instance and without any
limitation, a halftoning, a dithered ordered or, in the preferred
embodiment, a Floyd-Steinberg dithering technique.

[0061] Dithering techniques are well known in the art and, for the sake of
conciseness, there will not be further elaborated on. Regarding the
preferred embodiment, the Floyd-Steinberg technique, it suffices to
recall that the Floyd-Steinberg algorithm can be applied to anything
quantization process and particularly to a reduction to a single bit.
FIG. 4 recalls the general process which is based on the distribution of
the dithering error, weighted by a set of values .alpha.= 7/16, .beta.=
3/16; .gamma.= 5/16 and .delta.= 1/16 on the four close neighbouring
pixels, respectively at the left edge, the bottom, the right edge of row
j+1, and the pixel at the right of row j, in accordance with the
following scheme: [0062] The initial quantity is quantized according to
the output number of levels allocated for data representation [0063] The
error value is calculated by subtracting the quantized value to the
original quantity [0064] The error is added to neighbours pixel according
to the following pattern and weight: [0065] The picture scan can be done
in normal raster scan, but some algorithm alternate line scanning from
left-to-right with right-to-line depending on the line number parity

[0066] The process uses the dithering method for the purpose of converting
T transparency parameter (coded with N bits) to a one-bit transparency
parameter T' in a step 22.

[0067] Then, in a step 23, the process uses the computed T' parameter as a
control signal for controlling multiplexing circuits, having two inputs
respectiving receiving the color components of the foreground image (A)
and the background image (B).

[0068] It has been observed, and this is a very interesting effect, that a
aesthetical blending effect is simply achieved by means of this process.

[0069] The result of the application of the dithering process to the ALPHA
PLANE is illustratively shown in FIG. 3B and results in the conversion of
the representation of the foreground pixel transparency T into its new
representation T', with the interesting result that the blending can be
achieved with only one multiplexor multiplexing either (R.sub.A, G.sub.A,
B.sub.A) or (R.sub.B, G.sub.B, B.sub.B) in accordance with the value of
T'.

[0070] There is achieved the possibility to display a blended foreground
image A with a background image B without requiring the need of any
multipliers, as in the technique of FIG. 1B, what is very advantageous.

[0071] With respect to FIG. 2B, there is now described a second embodiment
which significantly improves the blending effect, without additional
complex circuitry.

[0072] Steps 31 and 32 of FIG. 2B are identical to steps 21 and 22 of FIG.
2A.

[0073] The second embodiment, then differs from the first embodiment in
the next following steps:

[0074] The process then proceeds with a step 33 where, with the assumption
of a foreground picture A made of a standard representation (24 bits per
pixel) with an N=8 bit alpha plane, the one bit quantized and dithered
information of the original 8 bits transparency value is converted as
follows:

T''=0xFF if T'=1;

T''=0x00 if the value of T'=0.

[0075] value as the MSB byte

[0076] There is thus provided again a representation of the color of the
picture which incorporates a transparency parameter coded onto 8 bits.

[0077] Now, in a step 34, the process proceeds with a 4-pixels
interpolation of the representation (T'', R.sub.A, G.sub.A, B.sub.A)
(with T'' being the MSB) of the foreground image so as to produce an
interpolated representation (T''.sup.i, R.sub.A.sup.i, G.sub.A.sup.i,
B.sub.A.sup.i) which takes into account of the information of four
neighbouring pixels.

[0078] It should be noticed that, regarding the particular parameter T'',
this 4-pixel interpolation results that five exact blending levels can be
produced, corresponding to the following sequence:

[0079] 0, 1/4, 1/2, 3/4 and 1.

[0080] With the significant technical effect that such blending values can
easily be achieved with minimum hardware circuitry.

[0081] Indeed, in a step 35, the parameter T''' is simply used as a
control signal for controlling the multiplexing circuit allowing
selection between five weighted sum of the pixel of image A and pixel of
image B.

[0082] Such selection requires very limited hardware circuits.

[0083] Indeed, a dividing by 2 is easily carried out by a single
bit-shifting towards the LSB.

[0084] A dividing by 4 is easily produced by two consecutive bit-shifting
towards the LSB.

[0085] At last, a level of 3/4 is easily achieved by means of adding the
two preceding levels 1/4 and 1/2.

[0086] It can be seen, and this is a very advantageous result that without
requiring complex multipliers, a five level blending process can be
achieved while still using a selection multiplexor as used in the basic
process of FIG. 1A.

[0087] In one particular embodiment, the interpolation process of FIG. 5
is applied for the purpose of generating the interpolated representation
(T''.sup.i, R.sub.A.sup.i, G.sub.A.sup.i, B.sub.A.sup.i) in accordance
with the following formulas:

T''.sup.i(x,y)=1/4(T''(x,y)+T''(x+1,y)+T''(x,y+1)+T''(x+1,y+1))

[0088] And

[0089] R.sub.A.sup.i, G.sub.A.sup.i, B.sub.A.sup.i being interpolated in a
similar fashion:

[0090] With respect to FIG. 6 there is now described one embodiment of a
circuit which can be used for performing the blending process, and which
is based on a five input multiplexing block 100, controlled by the five
level T''.sup.I parameter. The first and the fifth inputs of the latter
respectively receive the component of foreground pixel A on a lead 110
and background pixel B on a lead 120.

[0091] Two one-bit shifting circuits, respectively 131 and 133 connected
to lead 110 (pixel A) and 120 (pixel B), for performing a divide by two
operation.

[0092] Two two-bit shifting circuits, respectively 132 and 134 connected
to lead 110 (pixel A) and 120 (pixel B), for performing a divide by 4
operation.

[0093] Three adders 141,142 and 143 have an output which are respectively
connected to the second, the third and the fourth input of selection
block 100.

[0094] Adder 141 has a first input connected to the output of 2-bit
shifting circuit 132, a second input connected to one-bit shifting
circuit 133 and a third input connected to two-bit shifting circuit 134.
That adder 141 produces the value of 1/4 of the component of A plus 3/4
of the component of B.

[0095] Adder 142 has a first input connected to the output of one-bit
shifting circuit 131 and a second input connected to one-bit shifting
circuit 133 so as to produce the value of 1/2 the component of A and 1/2
that of B, that is to say a pure blending.

[0096] Adder 143 has a first input connected to the output of one-bit
shifting circuit 131, a second input connected to two-bit shifting
circuit 132 and a third input connected to two-bit shifting circuit 134.

[0097] The invention achieves a 8 bit-blending scheme with a single bit
selection overlay associated to a fractional pixel interpolation process.
In practice, it has been observed that very little penalty results from
the dithering of the alpha plane and thus the one-bit reduction of the
original transparency parameter T. The Floyd Steinberg dithering
algorithm is traditionally considered as a quite computational intensive
algorithm since often associate to large resolution object (printer
hundreds of DPI resolution on a full A4 format sheet). Considering the
mobile phone environment application we have considered, the screen size
is either QCIF+(220.times.176) or QVGA (320.times.240). Consequently the
required CPU effort needs to be very reasonable and furthermore the
transparency information applies to man machine interface content, which
does not vary very fast even considering small graphic animations.