H04N5/23222—Computer-aided capture of images, e.g. transfer from script file into camera, camera control checks quality of taken pictures, gives advices how to arrange picture composition or decides when to take image

Abstract

A system and method for structured-light dimensioning is disclosed. The method includes combining multiple images using different camera settings to provide all of the information necessary dimensioning. What results is an improved ability to sense a light pattern reflected from an object's surfaces, especially when the lighting and/or object color make imaging all surfaces simultaneously difficult.

Description

FIELD OF THE INVENTION

[0001]

The present invention relates generally to a dimension detection system, and more particularly to a structured-light dimensioning system and method for improving the performance thereof.

BACKGROUND

[0002]

Generally speaking, a structured-light dimensioning system may project a light pattern onto an object as part of a dimension measurement (i.e., dimensioning). In the best of circumstances, a high-quality image of the light pattern, with easily recognizable patterns on all dimensioning surfaces, is captured. Often, however, the captured light-pattern on one or more surfaces is unsuitable due to the lighting and/or the object's color. Under these circumstances, some adjustment of the structured-light dimensioning system may be necessary.

[0003]

Neither the time nor the skill is available in some dimensioning applications (e.g., handheld dimensioning) to adjust the structured-light dimensioning system carefully. What is more, in many cases an optimal pattern image, with resolvable light patterns on all dimensioning surfaces, is unobtainable using a single camera setting. Therefore, a need exists for a structured-light dimensioning system with an improved ability to automatically capture a resolvable light pattern on all dimensioning surfaces, especially when the lighting and/or object color make imaging all dimensioning surfaces simultaneously otherwise difficult.

SUMMARY

[0004]

Accordingly, in one aspect, the present invention embraces a structured-light dimensioning system. The system includes a projector subsystem for projecting a light pattern onto an object that has dimensioning surfaces. The system also includes a camera subsystem for capturing pattern images of the light pattern on the dimensioning surfaces. The system further includes a control subsystem that is communicatively coupled to the projector subsystem and the camera subsystem. The control subsystem includes a processor and a memory. The processor analyzes and processes pattern images, while the memory allows for the storage of pattern images and a software program. When executed by the processor, the software program configures the control subsystem to enable the camera subsystem to capture multiple pattern images using different camera-subsystem settings for each pattern image. The software program also configures the control subsystem to incorporate the multiple pattern images into an image composite with a resolvable light pattern on the dimensioning surfaces.

[0005]

In an exemplary embodiment, the structured-light dimensioning system is handheld and the image composite includes multiple pattern images aligned and combined to remove differences caused by hand motion.

[0006]

In another aspect, the present invention embraces a method to compute an object's dimensions using a structured-light dimensioning system. In the method, a pattern image of a light pattern, projected onto the object, is captured. Within the pattern image, dimensioning surfaces on the object are selected, and from these, the dimensioning surfaces that have a requisite pattern for dimensioning are identified. The pattern image is then incorporated into an image composite. Next, the image composite is checked, and if all of the dimensioning surfaces in the image composite have the requisite pattern, then the image composite is processed to compute the object's dimensions. Otherwise, the camera subsystem's settings are adjusted and the process of capturing a pattern image, selecting dimensioning surfaces, identifying the dimensioning surfaces that have a requisite pattern, and incorporating the pattern image into the image composite are repeated. This repetition continues until all of the dimensioning surfaces in the image composite have the requisite pattern for dimensioning, at which point the image composite is processed to compute the object's dimensions.

[0007]

In an exemplary embodiment, at least three dimensioning surfaces are selected and the object's dimension is the object's volume.

[0008]

In another exemplary embodiment, the image composite includes multiple pattern images.

[0009]

The foregoing illustrative summary, as well as other exemplary objectives and/or advantages of the invention, and the manner in which the same are accomplished, are further explained within the following detailed description and its accompanying drawings.

Dimensioning is the process of remotely measuring an object's dimensions using a dimensioning system (i.e., dimensioner). Typically, the object analyzed is a cubic package and the dimension measured is the object's volume. Measuring a package's volume is especially important in the shipping and warehousing industries, which may have space and/or weight restrictions. For example, the cost to ship a package has historically been based on the package's weight. Charging by weight alone, however, may cause a shipment of a lightweight package with a large volume to become unprofitable. As a result, dimension measurements are often required to compute shipping costs.

[0012]

Dimensioning systems may be static in the sense that a package is unmoving during a measurement. Alternatively, a dimensioning system may be dynamic where the package moves during measurement (e.g., moving along a conveyor). In both of these cases, the dimensioning system is mounted in a fixed position and the imaging environment is carefully controlled. The most promising dimensioning system configuration, however, is handheld, which could adapt to almost any environment.

[0013]

Handheld dimensioning is a challenging problem. In handheld applications, the environment (e.g., the lighting) is uncontrolled and the dimensioner must accommodate non-ideal imaging conditions (e.g., motion associated with being handheld). In addition, handheld applications typical have low tolerance for excessive measurement times or alignment complexities. Thus, the sensing technology chosen for such applications must accommodate these issues.

[0014]

A variety of sensing technologies have been employed for dimensioning. The present invention embraces a dimensioning system using structured-light sensing. Structure light is the process of projecting a light pattern (e.g., dots, grids, bars, etc.) onto a scene (i.e., field of view). The light pattern may be invisible or may be pulsed at an high rate so as not to interfere with other computer vision tasks, such as barcode scanning or optical character recognition (OCR).

[0015]

The scene with the projected light pattern is imaged and the pattern image is examined. Objects in the field of view will cause the light pattern in the pattern image to appear deformed when compared to a reference pattern. This deformation may be mathematically processed to compute distance (i.e., range) information. This range information may, in turn, be used to compute an object's dimensions (e.g., the object's volume).

[0016]

The image quality of the light pattern is key to structured light dimensioning. In some applications, however, lighting variations and/or object color, may negatively affect the pattern image on one, or more, of the object's surfaces. This is especially true for handheld systems that may be operated in a wide range of environments (e.g., in direct sunlight). A pattern image with uneven illumination may not have resolvable pattern elements (e.g., dots) on all surfaces of the object. For example, bright areas may be saturated, washing out the pattern elements, while dark areas may be underexposed, lacking the intensity to resolve the pattern elements.

[0017]

Uneven lighting may be caused by unfavorable illumination and/or the shape of the object. For example, if a box is illuminated with a beam of light, then the sides of the box opposite to the light source will be in shadow while the sides toward the light source will be fully illuminated. Since the accuracy (or even the possibility) of dimensioning depends on having pattern images with good pattern-element visibility on all dimensioning surfaces, it is important to capture pattern images of suitable quality. One camera setting may not be suitable, however, for obtaining good pattern images on each surface. For example, the correct exposure for one surface may overexpose another surface. Moving to a different location may help the imaging problem but may not be convenient in some applications. The present invention embraces the idea of capturing multiple pattern images using different camera settings (e.g., shutter speed or aperture size), so that when combined (i.e., mathematical combination) or used in combination, provide all of the necessary dimensioning information.

[0018]

As mentioned previously, a handheld dimensioner may project a light pattern onto an object and then capture an image (i.e., a pattern image) of the object with the overlaid light pattern. The pattern in the image may then be compared with a reference pattern and variations between the two patterns may be used to calculate the distance of various pattern elements (e.g., sets of dots) on a surface. Clearly, this method will not work when the pattern elements on the surface cannot be imaged. An important aspect of this invention is the realization that while dimensioning needs well-imaged patterns from a plurality of surfaces, these patterns need not be imaged simultaneously, nor do they need to be imaged with the same camera settings (e.g., exposure time).

[0019]

Unlike conventional images that capture all of the visible details of a scene, pattern images are used for sensing pattern distortions caused by the scene. This information may be used to compute the range of small groups of pattern elements on the surface. Once this calculation has been performed for a surface, this surface need not be imaged again unless the dimensioner has been moved appreciably. Pattern images for other surfaces may then be captured (e.g., using different camera settings) to give a composite image with a requisite pattern for range calculations. Once all of the surfaces have been imaged with the requisite pattern and their range calculations have been performed, then the object's volume may be computed.

[0020]

Small motion variations between the pattern images may be compensated by aligning the pattern images. If, for example, three dimensioning-surfaces are required, then at least three pattern-images may be captured. The three pattern images may then be aligned (e.g., aligning the object edges in each image) and overlaid. Alignment between the images can be accomplished in an exemplary embodiment by using a method called iterative closest point (i.e., ICP). This method can correct the small differences. Only small difference between pattern images during a dimensioning measurement is expected since multiple (e.g., three) surfaces may be acquired quickly (e.g., within a quarter of a second).

[0021]

The fastest way to use the pattern images is to acquire the first set of dots on a dimensioning surface and then ignore that dimensioning surface on future image acquisitions until the image composite has a useable pattern. Alternately, surfaces with two or more independent sets of visible pattern elements can be used to make multiple range measurements for each surface. These multiple measurements can then be averaged or otherwise mathematically combined to produce a more accurate range measurement for that side. The present invention, therefore, also offers an improved method for acquiring the information necessary for dimensioning since the same pattern may be acquired multiple times during a dimensioning measurement and thus may have an improved signal-to-noise ratio (i.e., SNR).

[0022]

A block diagram representing an exemplary method to compute an object's dimensions using a structured-light dimensioning system (e.g., handheld structured-light dimensioning system) is shown in Figure 1. The method 1 starts with an object 10 for dimensioning. This object 10 is typically a package and the goal of the method 1 is typically to compute the volume of this package. The object size is typically constrained within a prescribe range, and the object typically must be placed roughly at a prescribed distance from the dimensioner during measurement. The exact prescribed values are related to a projector subsystem that projects the light pattern and a camera subsystem that captures a pattern image of the light pattern reflected from the object.

[0023]

A small item, for example, placed a faraway from the dimensioner will not have a reflected pattern density (i.e., the density of pattern elements reflected from an object's surface) sufficient for a range calculation. Likewise, a large item placed too close to the dimensioner will fill the scene to such an extent that only a portion of a single surface may be imaged. Thus adjusting the field of view for the projector and camera subsystems as well as the pattern density and the image resolution are important factors in determining the operating range of the dimensioner.

[0024]

The dimensioner captures a pattern image 11 by projecting a light pattern onto the object and then capturing an image of the object with the overlaid light pattern. The light projected may be infrared (IR), visible, or ultraviolet

[0025]

(UV) but is typically IR. The light may be pulsed or continuous during the dimensioning process. The pattern may be dynamic or static and may consist of projected patterns including geometrical shapes (e.g., hexagons or line grids), dot arrays, and/or Du Bruijn diagrams. In an exemplary dot pattern (i.e., the pattern elements are dots) the size, shape, and distribution of the dot pattern are known and may be stored in memory as a reference pattern.

[0026]

The captured image is analyzed and dimensioning surfaces are selected 12. The detection and selection of dimensioning surfaces may be accomplished with an image analysis algorithm known as image segmentation. For example, a box may have three surfaces selected for dimensioning. These three surfaces can be used then to determine the box's volume.

[0027]

Once the dimensioning surfaces are selected 12 the light pattern's quality on each surface is be examined. Surfaces with the requisite pattern for dimensioning are identified 13. A requisite pattern is a pattern that is discernable in a pattern image. For example, a dot pattern on a surface that is saturated is not discernable and neither is a dot pattern with an insufficient reflected intensity resulting from an object that has a low reflectance (i.e., low with respect to the light pattern's wavelength).

[0028]

Images with a requisite surface patterns may be incorporated into an image composite 14. An image composite may be a set of individual images, a resulting image from the combination of one or more images, or a portion of an image or images (i.e., segmented surface data).

[0029]

The composite image must have a requisite number of dimensioning surfaces with discernable patterns for dimensioning. As a result, a condition 15 is included in the method. The condition 15 has two alternatives: (i) if all the selected dimensioning surfaces have the requisite pattern, then the image composite is processed 16 resulting in the object's dimension 17, otherwise (ii) the camera settings are adjusted 18 and a new pattern image is acquired and the process repeats.

[0030]

Collecting pattern images until all selected dimensioning surfaces in the image composite have the requisite pattern for dimensioning (i.e., composing the image composite) can be executed in a variety of ways. The fastest way to compose the image composite is to store the first set of pattern elements on a dimensioning surface and then ignore that dimensioning surface on subsequently acquired pattern images until the image composite has a useable pattern on all dimensioning surfaces. Alternately, dimensioning surfaces with two or more independent pattern images can be processed and these multiple measurements can be averaged or otherwise combined to reach a more accurate measurement of that surface.

[0031]

An exemplary embodiment of a structured-light dimensioning system 100 (i.e., dimensioner) block diagram is shown in Figure 2. A dimensioner 100 may be positioned with an object 10 in its field of view 24. The object may have its dimensions (e.g., volume) measured remotely. To accomplish this measurement, the dimensioner 100 utilizes a variety of subsystems.

[0032]

A projector subsystem 20 projects a light pattern 23 onto an object's dimensioning surfaces. The object 10 is positioned within a dimensioning region 19. The projector subsystem 20 has a light source 25 to generate and radiate light. The light source 25 may be a laser diode (LD) or a light emitting diode (LED), either of which may generate light radiation in the ultraviolet (UV), visible (VIS) or infra-red (IR) portions of the spectrum. An optical subassembly 26 is included in the projector subsystem 20 to focus and/or filter the light. A focusing element in the optical subassembly may be a lens or a diffractive optical element (DOE). A light pattern mask 27 is typically used to create the light pattern 23.

[0033]

A camera subsystem 30 captures pattern images of the object 10 and the projected light pattern 23. To accomplish this, the camera subsystem 30 may use an imaging lens 31 to render a real image of the imaging lens's field of view 24 onto an image sensor 32. This imaging lens field of view 24 overlaps at least partially with the projected light pattern 23. The image sensor 32 may be a charge coupled device (i.e., CCD) or a sensor using complementary metal oxide semiconductor (i.e., CMOS) technology. The image sensor 32 includes a plurality of pixels that sample the real image and convert the real-image intensity into an electronic signal. An imager digital signal processor (i.e., DSP) 33 is typically included to convert the electronic signals from the image sensor 32 into a digital signal. A control subsystem 40 is communicatively coupled to the projector subsystem 20 and the camera subsystem 30 via an interconnection system (e.g., bus) 50, which interconnects all of the dimensioners subsystems. The control subsystem 40 includes one or more processors 42 (e.g., one or more controllers, digital signal processor (DSP), application specific integrated circuit (ASIC), programmable gate array (PGA), and/or programmable logic controller (PLC)) to configure subsystems for the generation and capturing processes and then perform the processing necessary on pattern images and the image composite necessary for dimensioning. The processor 42 is typically configured by a software program stored in memory 41 (e.g., read only memory (ROM), flash memory, random access memory (RAM), and/or a hard-drive). The software program, when executed by the processor 42 configures the control subsystem to enable the camera subsystem 20 to: (i) capture multiple pattern images, each pattern image captured using different camera-subsystem settings and (ii) incorporate the multiple pattern images into an image composite with a resolvable light pattern on all dimensioning surfaces.

[0034]

The dimensioner 100 may also include a user interface 70 to display dimension measurements (e.g., linear dimension or volume) results. In some embodiments, the user interface 70 may also facilitate the selection of dimensioning surfaces.

[0035]

The dimensioner 100 may also include a communication subsystem 60 for transmitting and receiving information to/from a separate computing device or storage device. This communication subsystem may be wired or wireless and may enable communication with a variety of protocols (e.g., IEEE 802.11, including WI-FI®, BLUETOOTH®, CDMA, TDMA, or GSM).

[0036]

The subsystems in the dimensioner 100 are electrically connected via a couplers (e.g., wires or fibers), buses, and control lines to form an interconnection system 50. The interconnection system 50 may include power buses or lines, data buses, instruction buses, address buses, etc., which allow operation of the subsystems and interaction there between.

[0037]

To supplement the present disclosure, this application incorporates entirely by reference the following commonly assigned patents, patent application publications, and patent applications:

U.S. Patent Application No. 14/370,237 for WEB-BASED SCAN-TASK ENABLED SYSTEM AND METHOD OF AND APPARATUS FOR DEVELOPING AND DEPLOYING THE SAME ON A CLIENT-SERVER NETWORK filed 07/02/2014 (Chen et al.);

In the specification and/or figures, typical embodiments of the invention have been disclosed. The present invention is not limited to such exemplary embodiments. The use of the term "and/or" includes any and all combinations of one or more of the associated listed items. The figures are schematic representations and so are not necessarily drawn to scale. Unless otherwise noted, specific terms have been used in a generic and descriptive sense and not for purposes of limitation.

Claims (15)

A structured-light dimensioning system, comprising:

a projector subsystem for projecting a light pattern onto an object, said object having dimensioning surfaces;

a camera subsystem for capturing pattern images, the pattern images comprising the light pattern on the dimensioning surfaces;

a control subsystem communicatively coupled to the projector subsystem and the camera subsystem, the control subsystem comprising (i) a processor for analyzing and processing the pattern images and (ii) a memory for storing the pattern images and a software program, wherein

the software program when executed by the processor configures the control subsystem to (i) enable the camera subsystem to capture multiple pattern images, each pattern image captured using different camera-subsystem settings and (ii) incorporate the multiple pattern images into an image composite with a resolvable light pattern on the dimensioning surfaces.

The structured-light dimensioning system according to claim 1, wherein the structured-light dimensioning system is handheld.

The structured-light dimensioning system according to claim 2, wherein the image composite comprises multiple pattern images aligned and combined to remove differences caused by hand motion.

The structured-light dimensioning system according to claim 1, wherein the projected light pattern is an infrared (IR) pattern of dots.

The structured-light dimensioning system according to claim 1, wherein the resolvable light pattern comprises at least one visible pattern element.

The structured-light dimensioning system according to claim 1, wherein the composite image comprises segmented pattern images.

The structured-light dimensioning system according to claim 1, wherein the composite image comprises a single image that is a mathematical combination of pattern images.

The structured-light dimensioning system according to claim 1, wherein the camera-subsystem settings comprise an image-sensor shutter speed.

The structured-light dimensioning system according to claim 1, wherein the camera-subsystem settings comprise an imaging lens aperture size.

The structured-light dimensioning system according to claim 1, wherein the software program configures the control subsystem to identify dimensioning surfaces in a pattern image.

The structured-light dimensioning system according to claim 1, wherein the software program configures the control subsystem to compute a range using the image composite.

The structured-light dimensioning system according to claim 1, wherein the software program configures the control subsystem to compute an object volume using the image composite.

A method to compute an object's dimensions using a structured-light dimensioning system, the method comprising:

capturing a pattern image of a light pattern projected onto the object;

selecting dimensioning surfaces on the object in the pattern image;

identifying dimensioning surfaces that have a requisite pattern for dimensioning;

incorporating the pattern image into an image composite;

adjusting a camera subsystem's settings and repeating the capturing, selecting, identifying, and incorporating until the image composite has the requisite pattern for dimensioning on all dimensioning surfaces; and

processing the image composite to compute the object's dimensions.

The method according to claim 13, wherein at least three dimensioning surfaces are selected and the object's dimension is the object's volume.

The method according to claim 13, wherein the image composite comprises at least two pattern images.

Method and apparatus for increasing the SNR at the RF antennas of wireless end-devices on a wireless communication network, while minimizing the RF power transmitted by the wireless coordinator and routers

Wireless dual-function network device dynamically switching and reconfiguring from a wireless network router state of operation into a wireless network coordinator state of operation in a wireless communication network

Laser scanning system using laser beam sources for producing long and short wavelengths in combination with beam-waist extending optics to extend the depth of field thereof while resolving high resolution bar code symbols having minimum code element widths

Automated method of and system for dimensioning objects transported through a work environment using contour tracing, vertice detection, corner point detection, and corner point reduction methods on two-dimensional range data maps captured by an amplitude modulated laser scanning beam

Method of speckle-noise pattern reduction and apparatus therefore based on reducing the temporal-coherence of the planar laser illumination beam before it illuminates the target object by applying temporal phase modulation techniques during the transmission of the plib towards the target

Automated method of and system for dimensioning objects over a conveyor belt structure by applying contouring tracing, vertice detection, corner point detection, and corner point reduction methods to two-dimensional range data maps of the space above the conveyor belt captured by an amplitude modulated laser scanning beam

Method and apparatus for increasing the SNR at the RF antennas of wireless end-devices on a wireless communication network, while minimizing the RF power transmitted by the wireless coordinator and routers

Wireless dual-function network device dynamically switching and reconfiguring from a wireless network router state of operation into a wireless network coordinator state of operation in a wireless communication network

Laser scanning system using laser beam sources for producing long and short wavelengths in combination with beam-waist extending optics to extend the depth of field thereof while resolving high resolution bar code symbols having minimum code element widths