WO1998017970A1 - Method and apparatus for three-dimensional color scanning - Google Patents
Method and apparatus for three-dimensional color scanning Download PDFInfo
- Publication number
- WO1998017970A1 WO1998017970A1 PCT/US1997/019331 US9719331W WO9817970A1 WO 1998017970 A1 WO1998017970 A1 WO 1998017970A1 US 9719331 W US9719331 W US 9719331W WO 9817970 A1 WO9817970 A1 WO 9817970A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- images
- dimensional
- trackable
- patch
- image
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/579—Depth or shape recovery from multiple images from motion
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0007—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
- H04N13/221—Image signal generators using stereoscopic image cameras using a single 2D image sensor using the relative movement between cameras and objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/246—Calibration of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/254—Image signal generators using stereoscopic image cameras in combination with electromagnetic radiation sources for illuminating objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/189—Recording image signals; Reproducing recorded image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/194—Transmission of image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/243—Image signal generators using stereoscopic image cameras using three or more 2D image sensors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Definitions
- This invention relates generally to optical scanners, and more particularly to optical scanners for providing a digital representation of three dimensional objects.
- One such method is a system which uses a laser range finder to scan the object and record the distance between the known three dimensional location of the range finder and the measured location of the surface of the object.
- the result of using this method or other methods of generating three dimensional surface models is a set of three dimensional points which accurately represent the surface of the object.
- a characteristic of this method and other methods of obtaining a three dimensional surface model is that it is inherently monochromatic, that is, no color information is obtained in the process. If three dimensional color information is desired, then it must be generated by somehow combining or conformally mapping the two dimensional color information onto the three dimensional surface model.
- the problem of conformally mapping the two dimensional color information onto the three dimensional surface model is difficult and it is common for mismatching of color information with the three dimensional points to occur.
- the problem may be visualized by imagining a white statue or bust of a person's head and a color photograph of the same person's face. The photograph cannot simply be projected onto the bust to transfer the correct color information to the correct points on the bust or significant distortion will occur. A significant amount of judgment must be exercised in order to correctly associate the color information from the photograph with the correct surface points on the bust. Similarly, it is difficult to accurately associate color information obtained from two dimensional optical detectors with the correct points on a three dimensional surface model.
- the present invention provides a system and method for using the color information from a series of two dimensional color images to derive the three dimensional location in space of the surface points which produced the color images. Because the color information itself is used to derive the three dimensional location of the surface points, there is no need to conformally map separately generated color information onto the derived three dimensional surface points. The points are derived from color information and so are already associated with the correct color information. Also, the use of the color information increases the accuracy of the three dimensional location of the surface points.
- the present invention provides a three dimensional digital scanner which includes a multiple view detector which is responsive to a broad spectrum of visible light.
- the multiple view detector is operative to develop a plurality of images of a three dimensional object which is being scanned.
- the plurality of images are taken from a plurality of relative angles with respect to the object, and the plurality of images depict a plurality of surface portions of the object .
- a digital processor including a computational unit is coupled to the detector and is responsive to the plurality of images so that it develops 3-D coordinate positions and related image information for the plurality of surface portions of the object.
- a three dimensional image of the object to be scanned is thus developed by the digital processor.
- the data developed includes both shape and surface image color information.
- a three dimensional color digital scanner in another embodiment, includes a color detector responsive to a broad spectrum of visible light to develop a plurality of images of a three dimensional object .
- a rotary object support having an axis of rotation allows the detector to develop a plurality of images of a three dimensional object.
- the plurality of images depict a plurality of surface portions of the object.
- a digital computer is coupled to the detector. The computer tracks patches of the surface portions of the object to determine coordinates of the patches as a function of the rotation of the rotary object support and determines radii of the patches from the axis of rotation.
- a method for scanning a three , dimensional object includes developing a plurality of images of a three dimensional object taken from a plurality of relative angles with respect to the object.
- the plurality of images depict a plurality of surface portions of the object to be scanned.
- 3-D coordinate positions and related image information about the plurality of surface portions of the object is computed from the plurality of images such that a three dimensional image of the object is developed that includes both shape and surface image information.
- a method for determining three dimensional coordinates of a surface portion of an object includes obtaining a plurality of images of the surface portion of the object and identifying a trackable patch of the surface portion in an initial image. An initial set of two dimensional coordinates of the trackable patch in the initial image is determined along with at least one additional set of two dimensional coordinates of the trackable patch in another of the images. A radial coordinate of the trackable patch is determined and then a set of three dimensional coordinates of the trackable patch are determined from the radial coordinate of the trackable patch.
- a method for determining three dimensional coordinates of a surface portion of an object includes rotating the object about an axis of rotation so that a plurality of images of the surface portion of the object are obtained as the object is rotates about the axis of rotation.
- a trackable patch is identified and the two dimensional coordinates of the trackable patch are determined.
- the movement of the trackable patch is tracked as a function of the rotation of the object.
- a radial distance of the trackable patch from the axis of rotation is determined based on the movement of the trackable patch as a function of the rotation of the object and three dimensional coordinates of the surface portion of the object are derived from the coordinates of the trackable patch and the radial distance of the trackable patch from the axis of rotation.
- the present inventions provides a system and method for obtaining 3-D surface information that is linked to color information without the need to conformally map 2-D color data onto a 3-D surface.
- the accuracy of the system is enhanced by the use of color data and the cost of the system is reduced because the 3-D surface is derived from a series of 2-D images.
- Figure 1 illustrates a system for obtaining a series of two dimensional color images of an object and processing those images to obtain a three dimensional model of the surface of the object.
- Figure 1 A illustrates an alternative embodiment of the present invention which enables the top and bottom portions of an object to be scanned.
- Figure IB illustrates another embodiment of the present invention which produces enhanced shading of an object.
- Figure IC illustrates an arrangement where a detector is translated about a stationary object.
- Figure ID illustrates an embodiment of the present invention which uses a multiple number of detectors instead of moving a single detector.
- Figure 2 illustrates in detail an architecture of an image acquisition system.
- Figure 3A is a flow diagram illustrating a process of obtaining multiple images of a rotating object.
- Figure 3B is a flow diagram illustrating a process for generating three dimensional surface data from the two dimensional images of the object.
- Figure 4 is a flow diagram illustrating a process performed on the images before they are stored.
- Figure 5A illustrates the vector nature of the color data obtained.
- Figure 5B illustrates an example of blue color data at times 0, 1, 2, and 3 for a line of pixels.
- Figure 5C illustrates how the data can be compressed by recording only the changes in the color data.
- Figure 6 is a flow diagram illustrating a process for identifying the silhouette of the object in each image.
- Figure 7 is a flow diagram illustrating a process for finding silhouette edges along each scan line.
- Figure 8 is a flow diagram illustrating a process for determining e a set of trackable patches.
- Figure 8A illustrates how to search an image for trackable patches.
- Figure 9 is a flow diagram illustrating a process for determining the radius of the location of patches on the surface of the object as the object is rotated.
- Figure 9A illustrates a set of patch tracking limits.
- Figure 9B illustrates the motion of trackable patches in different images with different angular displacements.
- Figure 9C illustrates the determination of an exact position of the patch in an image.
- Figure 9D is a graph which illustrates the filtering of raw data points.
- Figure 9E is a graph which illustrates how the radius is determined from the points representing the path of the trackable patch across angularly displaced images.
- Figure 10 is a flow diagram illustrating t he post processing that occurs once the radius of the trackable patch is known.
- an embodiment of the present invention includes a system for obtaining a series of two dimensional color images of an object and processing those images to obtain a three dimensional model of the surface of the object.
- An object 100 which is to be digitized is placed on a rotatable platform 102.
- a motor 104 is provided to drive rotatable platform 102 via a shaft 106.
- a position encoder 108 detects the angular position of rotatable platform 102 and generates an electrical signal which represents the angular position of rotatable platform 102.
- An optical detector 110 e.g. a color video camera
- detector 1 10 captures a series of color images of object 100. Each color image taken at a different time is associated with an angular rotation of object 100 about an axis of rotation, "A" which runs through shaft 106. Information about the angular position of object 100 is obtained from position encoder 108. Thus, each "snapshot" or image of object 100 taken by detector 110 from a different view is associated with data about the angle of rotation of object 100 with respect to detector 110.
- An image input processing system 120 (“computer") controls the image acquisition process and records the acquired images along with the associated angular position data.
- processing system 120 is connected to detector 110 and receives data for each image or snapshot taken of object 100 from detector 110, and position encoder 108 sends angular position information to processing system 120, so that processing system 120 can associate the image data from detector 110 with the angular position data taken at the same time.
- detector 110 is a film camera and processing system 120 receives data from a digitizer which digitizes the film images from detector 110.
- Processing system 120 includes a processing unit 122 and a monitor 124 and also controls motor 104.
- a monitor 124 can display a current image 126 being captured by detector 110 or other information about the capturing process.
- Image processor 130 can receive data from processing system 120 in a number of different ways. Image processor 130 can be directly connected to processing system 120 via direct connection 132, or data from processing system 120 can be transferred to a removable storage medium such as disk 134 which may be read by image processor 130. Processing system 120 may also transfer data to image processor 130 via the Internet or a modem connection. Image processor 130 includes processing unit 136 and also includes monitor 138.
- processing system 120 and image processor 130 are combined on a single computer.
- the advantage of separating the functions of processing system 120 and image processor 130 is that the data acquisition and storage function performed by processing system 120 and control of the data acquisition system does not require a complex or powerful processor.
- image processor 130 receives data representing a series of two dimensional images and perform complex and computationally intensive operations on that data to produce a three dimensional surface model.
- Image processor 130 is therefore, given current technology, likely to be a more powerful (and costly ) computer than processing system 120. If that is the case, then it is economically beneficial to utilize a large number of relatively cheap processors for data acquisition and temporary storage and send data from those relatively cheap systems to a smaller number of image processors which generate the three dimensional surface model from the set of two dimensional color images.
- Figure IA illustrates an alternative embodiment of the present invention which enables the top and bottom portions of an object to be scanned.
- object 100 is supported by rotatable platform 102 which is driven by motor 104.
- shaft 107 engages the edge of rotatable platform 102, so that motor 104 and shaft 107 do not obscure the image of the bottom of object 100.
- Rotatable platform 102 is made from a transparent material so that the bottom of object 100 may be viewed through rotatable platform 102.
- a set of mirrors 109 are placed within the field of view of detector 110 so that images of the top and bottom surfaces of object 100 are captured by detector 110 in addition to the side views.
- Figure IB illustrates another embodiment of the present invention which is designed to produce contrast enhancing shading of object 100.
- object 100 is supported by rotatable platform 102 which is driven by a motor 104 via a shaft 106.
- a second motor 142 also drives a rotatable platform 144 via shaft 146.
- Encoder 148 generates data representative of the rotational position of rotatable platform 144 and transmits that data to processing system 120.
- motor 142 receives control commands from processing system 120.
- a light 150 is mounted on rotatable platform 144 to provide illumination of object 100. Light 150 is oriented to provide contrasting illuminated and shaded portions on object 100 which aid in the tracking of features on the surface of object 100.
- Figures 1, IA, and IB each depict embodiments where in the object being imaged is rotated.
- the object remains stationary and the detector moves around the object.
- Figure IC illustrates an arrangement where a detector is translated about a stationary object. It should be noted that as the detector 110 is moved, the optics 111 remain pointed at the object 100.
- Detector 1 10 can be move in many ways and object 100 can be supported in many ways. In one embodiment, an unobstructed view of object 100 is obtained by suspending it from very thin wires. Detector 1 10 is translated about object 100. If object 100 is very large, detector 110 could be mounted on, for example, a helicopter and flown around object 100. It is not necessary that the motion of detector 1 10 be exactly circular around object 100.
- the angular and radial components of the motion of detector 1 10 with respect to object 100 can be computationally analyzed, as will be appreciated by those skilled in the art. As long as the position of detector 110 is measured and recorded, the relative angular position of detector 110 with respect to object 100 can be determined for each image taken by detector 1 10. Methods of determining the position of detector 1 10 include using GPS or a laser positioning system. Once the angular component of the motion is analyzed and the radial component is calculated, the system compensates for the radial component and the images generated by detector 110 can be processed similarly to the images generated by a system that includes a rotating object and a stationary detector. Figure ID illustrates an embodiment of the present invention which uses a multiple number of detectors instead of moving a single detector.
- a top view of object 100 is shown and set of detectors 110 are provided at different angular displacements with respect to object 100.
- the advantage of this embodiment is that no motion is required and the need for motors, encoders, and rotatable supports is limited.
- the image of object 100 captured by each detector is angularly displaced with respect the images captured by the other detectors and so the images may be processed in a similar manner as successive images taken by one moving detector.
- the cost of multiple detectors 110 may be less than the cost of a rotatable drive or a mechanism for moving detector 1 10 and recording the position of detector 1 10.
- Another advantage of this approach is that all of the images of object 100 can be created simultaneously.
- Figures 1 through ID depict various embodiments for creating multiple images of object 100 with object 100 and detector 1 10 at different relative angular displacements.
- Each of these systems provide two dimensional color images of object 100 observed at different angles. This two dimensional information is converted into a three dimensional surface model of object 100 by the process and apparatus of the present invention.
- FIG. 2 illustrates in detail the architecture of processing system 120 used in some embodiments.
- a microprocessor 200 is connected to a memory bus 202 and memory bus 202 is connected to a RAM 204 and a ROM 206.
- Microprocessor 200 is also connected to an input/output ("I/O") bus 208.
- a video interface 210 is coupled to I/O bus 208 to control monitor 124, as is detector interface 212.
- Detector interface 212 buffers and processes data from the detector and also carries output commands to the detector from microprocessor 200. In certain embodiments where a moving detector is used, the detector provides its own control and records its own position. In such embodiments, the detector/processor interface need only be capable of transferring data from the detector, including both image and detector position data, to the processor storage system.
- Mass storage 214 (such as a hard disk drive) is also connected to input/output bus 208 and provides storage capacity for the multiple images generated by the optical system.
- Removable storage 216 (such as a floppy disk drive) also provides a way of transferring data files to and from processing system 120 and another processing system.
- communications interface 218 can be used to transfer files as welL Communications interface 218 may be connected to a local area network ("LAN”) or wide area network (“WAN”) for communication with other workstations.
- Position controller 220 is connected to input/output bus 208 and provides control to a motor in embodiments where processing system 120 provides control commands for rotating object 100. In such embodiments, position detector 222 receives data from an encoder so that processing system 120 may keep track of the position of object 100.
- Lighting control 224 is also connected to input/output bus 208 and is used to control the position of lights which may be moved with respect to object 100. Lighting control 224 also controls the intensity of those lights.
- the architecture shown for processing system 120 in Figure 2 is capable of supporting any of the embodiments shown in Figures 1-1D. If the object is to be rotated, position controller 220 and position detector 222 provide control of the rotation. Position information about object 100 can be integrated with image data from interface 212 and stored in mass storage 214. Movement and intensity control of the light is controlled by lighting control 224. If an autonomous detector is used, data about the detector position and images captured by the detector can be transferred to processing system 120 via communications interface 218 or removable storage 216.
- Multiple detector interfaces are provided to control a multiple number of detectors in embodiments which use more than one detector.
- a three dimensional surface model can be computed using microprocessor 200 and the data contained in mass storage 214, or, alternatively, the data in mass storage 214 can be transferred to a more powerful image processing system.
- Figure 3A is a flow diagram for the process of the present invention of obtaining multiple images of a rotating object.
- the method is implemented on a processing system 120.
- the process starts at step 300, and the user places the object on the rotatable platform in step 302.
- the object begins to rotate while it is being imaged by a detector.
- the processor checks whether the required of number images have already been captured or taken. If the required number of images have been captured, then the process is finished at step 306.
- the two dimensional image data is then ready to be taken to an image processor for generation of a three dimensional surface model. If more images are to be captured, then control is transferred to step 308, and a command is sent to the detector to capture an image.
- FIG. 3A illustrates the process implemented on image processor 130 for generating three dimensional surface data from the two dimensional images of the object developed by the process illustrated in Figure 3 A which was implemented on processing system 120.
- the process begins at step 350 and in a step 352, the processor identifies the silhouette of the object in each of the two dimensional color images of the object provided by the image acquisition system. This silhouette information will be used to reduce the area in each image that must be processed.
- a set of tracking points is determined within the silhouettes of the object. These tracking points are associated with surface features of the object which are identifiable in different images of the object taken by the optical detector.
- the portion of the image of the object which corresponds to the identifiable feature is referred to as a trackable patch.
- Each trackable patch consists of a group of pixels surrounding a geometric point on the surface which is being scanned. The trackable patch is thus a feature associated with and proximal to the geometric point. The patch is distinguishable from adjacent surface locations and therefore trackable.
- the trackable patch is tracked through a series of images of the object taken at different relative angular displacements between the object and the detector.
- Step 356 develops a radius from the axis of rotation for each trackable patch which will account for the motion of the trackable patch in the images. The radius of the trackable patch is used to approximate the radius to the geometric point associated with the trackable patch.
- step 358 a three dimensional surface consisting of the points which correspond to each trackable patch and interpolations between those points has been generated.
- step 358 coordinates of the points corresponding to the modeled surface are output in a standard format.
- the raw trackable patch points are output.
- points are interpolated between the raw trackable patch points, so that a regular array of points representing a three dimensional surface is generated.
- Each trackable patch point is already associated with a color value because the trackable patch points are associated with the trackable patch color information. The color value of the trackable, patch is simply attributed to the point determined for that patch.
- Step 358 outputs the coordinates and their color values. The process is completed at 360.
- the process of Figure 3B generates three-dimensional surface data from a set of a two dimensional images in a manner that associates color information with the three dimensional points during the generation of those points. Indeed, the color information is used to identify the points which are tracked thereby enhancing the tracking process.
- the system does not identify the surface of the object independent of color image information and then match the surface points to color information.
- optical detectors obtain color images of the object.
- the 3-D surface points of the object are not determined independently from that color image information.
- the surface points of the object are not directly determined, rather, the locations of the features located on the surface are determined.
- Those surface features include color information. Therefore, the locations of color features are directly determined and do not need to be mapped onto a surface model which is independently generated.
- the regularly spaced points of the surface model are then generated from the points representing the color surface features by interpolating between the surface feature points.
- Figure 4 illustrates a process 310 that can be performed on the images before they are stored.
- the process starts at 400.
- Step 402 analyzes the difference between the current acquired image and the previous acquired image.
- Step 404 applies a compression technique to the data to reduce the amount of storage space taken up by the data.
- the process is finished in step 406.
- the data is compressed for more compact digital storage.
- Figures 5 A, 5B, and 5C further illustrate the color image pixel data and a compression technique implemented in one embodiment of the invention.
- Figure 5A illustrates the vector nature of the color data obtained. In the example, shown, red-green-blue (“RGB”) color data is used. This or any other color scheme such as cyan-magenta-yellow black (“CMYK”) can be implemented in accordance with the present invention.
- RGB red-green-blue
- CMYK cyan-magenta-yellow black
- the three axes shown in Figure 5 A are labeled R, G, and B.
- Color values can be represented as vectors directed to points in this color space.
- vector 500 represents the color value of a pixel.
- the values of R, G, and B are represented by the point to which vector 500 is directed.
- Vector _500 is the sum of the green component vector 502, the blue component vector 504, and the red component vector 506. Thus, for each point, there is a vector in color space describing the color of the point. The position or pixel location of the point is stored along with the three dimensions of color information which represent the intensities of the three color signals, RGB.
- Figure 5B illustrates an example of what the color data for the color blue looks like at times 0, 1, 2, and 3 for a line of pixels.
- no blue color surface feature is being imaged by the line of pixels.
- a surface feature with a blue component has moved into the position being imaged by the line of pixels, so that 8 blue pixels are observed after X number of no blue pixels are observed.
- Y pixels of no color are again observed.
- only one intensity of blue is shown for the purpose of simplicity. In an actual example, varying intensities of blue could also be observed and recorded.
- the surface feature has moved to the right and now X + 2 pixels are observed with no blue and 8 pixels are observed with blue again.
- Y- 2 pixels of no color are observed next.
- the blue feature has moved to the right one more pixel and so X + 3 pixels of no color are observed followed by the 8 blue pixels followed by Y-3 pixels of no color.
- Figure 5C illustrates how the data can be compressed by recording only the changes in the color data, and then using run length encoding for those changes. Run length encoding techniques are well known to those skilled in the art. At time 0, no color is observed, and so the data stored is simply no color or black. At time 1 , X pixels are still no color, so that number of pixels is recorded and associated with no change. Next, 8 pixels have changed color to blue and so 8 pixels are recorded as having a positive blue change. Finally, the remaining Y pixels have not changed, so Y pixels, no change is recorded.
- the blue feature has moved one more pixel to the right.
- the first X+2 pixels in the linear array had no color at time 2 and still have no color at time 3. Therefore, no change is recorded for the first X+2 pixels.
- the next pixel was formerly blue but, since the blue feature has moved one pixel to the right, that pixel is now has no color. Therefore a negative blue color change is recorded for that pixel.
- the next 7 pixels were blue at time 2 and remain blue at time 3. Therefore, no color change is recorded for those 7 pixels.
- the next pixel had no blue color at time 2 and has a blue color component at time 3. A color change of positive blue is therefore recorded for that pixel.
- the Y-l remaining pixels were formerly blank at time 2 and remain blank at time 3 and so no change is recorded for those Y-l pixels.
- FIG. 6 illustrates the process 352 that identifies the silhouette of the object in each image. The process starts at 600. Step 602 tests whether the process is finished with the images. If the processor is finished with the images, then the process terminates at step 604. If more images remain to be processed, step 606 applies a filter kernel to the image which is herein inco ⁇ orated by reference.
- specularity, emissivity, and absorption can be derived from the surface geometry and the orientation of the local surface in multiple views at multiple angles with respect to the surface. In certain embodiments, these attributes also aid in distinguishing trackable patches.
- Figure 6 illustrates the process 352 that identifies the silhouette of the object in each image. The process starts at 600. Step 602 tests whether the process is finished with the images. If the processor is finished with the images, then the process terminates at step 604. If more images remain to be processed, step 606 applies a filter kernel to the image which is herein inco ⁇ orated by reference.
- a filter kernel is a set of coefficients which is convolved with an image (an XY array of pixels).
- Certain filter kernels such as the Canny and Sobel kernels, highlight edges by producing a high correlation at locations which represent edges in images. Edge detection using filter kernels is described in: “On Detecting Edges” by Nalwi, PAMI, Chapter 8, pp. 699-714; “Finding Lines and Edges in Images” by Canny, MIT Technical Report TR-720, 1984; “On Calibrating Computer Controlled Cameras for Perceiving 3D Scenes” by Sobel Artificial Intelligence, Vol. 5, pp.
- step 608 By correlating the filter kernel with the image at all points, it is possible to determine the edge of the object for the object silhouette, by analyzing the correlation value.
- the silhouette edges are determined in step 608. Control is transferred back to step 602 to check whether there remain other images to be processed for silhouettes.
- the purpose of determining the silhouette of the image is to constrain the amount of area over which surface features are searched for and identified for the object.
- a flying insect or a noise spike recorded by the detector may exist outside the area of the object shown in a given image. Noise spikes or insects may appear to be a good candidate to a be a trackable surface feature, but if the silhouette or edge of the object has been determined and the insect or noise spike is outside of that edge, then it will not be designated as a possible trackable surface feature.
- the search for surface features is further limited to an area within the silhouette as is described below.
- FIG. 7 illustrates further detail regarding process 608 of Figure 6 wherein silhouette edges are found along each scan line.
- the process starts at 700.
- the processor begins at the left edge of the image and correlates the kernel with the pixels of the image to find a correlation maximum which corresponds to a potential left edge.
- the processor moves in from the right edge of the image and correlates the kernel with the image data to find potential right edges.
- a heuristic algorithm is used to determine left and right edges of the object.
- edges are distinguished from bugs or blips by the nature of the edge. In one embodiment this is accomplished by distinguishing between the continuous nature of an edge verses the isolated spatial nature of a spec or blip. Any alternative heuristic algorithm which exists for determining edges in such instances may be used within the scope of the present invention.
- Figure 8 illustrates the process 354 of Figure 3B implemented to determine a set of trackable patches.
- the process starts at 800.
- Step 802 determines whether the processor is finished with all images. If the processor is finished, the process ends at step 804. If the processor is not finished, control is transferred to step 806 and the processor locates the vertical center line of the image being processed. Next, the processor steps through the image to look for trackable patches starting at the center of the image and searching outward.
- the size and dimensional nature of the trackable patches varies in different embodiments.
- trackable patches are searched for in individual scan lines, so that the trackable patches are one dimensional. In other embodiments, the trackable patches are searched for across several scan lines at once so that the patches are two dimensional.
- the size of the trackable patch searched for is fixed in some embodiments at a certain pixel area, for example 10x10. In other embodiments, the area of the trackable patches is not predetermined and may vary in size for different areas of the image. This enables the system to be flexible and select trackable patches of an optimal size for the type of trackable features found on the object. In general, when an "area" is referred to as being checked for trackable patches, any size or dimension area is intended, including a line of pixels. Step 808 checks whether the processor is finished testing all parts of the image.
- step 802. If the processor is finished, then control is transferred back to step 802. If the processor is not finished, then the next unsearched area of the image is searched and analyzed to determine if it contains any trackable patches in step 810.
- a good trackable patch is a series of pixels which have RGB vector values which are readily identifiable and distinguishable from the RGB vector values in other areas of the image. Therefore, a distinctive coloration of a patch is an asset to its trackability.
- the trackable patch RGB vector values will be correlated with the RGB values in other images which represent other views of the object. If the trackable patch is readily distinguishable, then the correlation will yield a distinct spike at the new location of the trackable patch in the other images and the patch will be successfully found or tracked.
- One method is to compute the variance values of the pixels within the area.
- a high variance value will indicate that the pixel values are changing and thus will at least be readily distinguishable from a solid or unchanging color background.
- a more sophisticated method is to convolve different kernels with the image area being tested to judge the quality of the surface features and its likelihood of being trackable.
- Standard kernels such as those defined in Canny and Sobel, supra, can be defined as a standard kernel used to check for trackable patches.
- a learning or smart system in some embodiments has the capability of developing customized kernels on the fly which are optimized for the type of surface features found on the object being scanned.
- the size of trackable patches, and the kernels used to identify them may be held constant or may be varied.
- step 806 locates the vertical center line of the image.
- the system begins at the vertical center line of the image and the search proceeds outward for trackable patches.
- the reason that trackable patches are first looked for at the center line of the image is that surface features are most likely to be completely and accurately sensed without distortion when they are observed head-on by the detector. However, areas to the left or right of the centerline can also be used.
- surface features rotate away from the center line of the object, they will eventually pass out of the view of the detector. Before passing out of the view of the detector, the surface features will begin to become partially obscured and distorted as they are viewed obliquely by the detector. For example, at a 45 degree angle, a circle on the surface of the object will appear to the detector to be an oval. Obscured and distorted surface features are more difficult to accurately track because they change between images and it is also difficult to define a consistent location point for them.
- step 812 the processor marks the trackable patches found in the area searched. This includes storing the color data for the pixels in the patch and storing the location of the patch.
- step 814 the processor increments a distance away from the center line to define a new area to search and control is transferred to step 808.
- step 808 determines if the image is finished being tested. In one embodiment, this is done by checking whether the entire area within a maximum distance from the center has been searched. The distance from the center line over which the processor will search for trackable patches can also be varied.
- Figure 8A illustrates an image being searched for trackable patches by the process 810 of Figure 8. The top portion of Figure 8 illustrates object 100 and area 850 which is magnified below.
- Area 850 consists of a number of areas or sections 852 which, in turn, consists of pixels 854. Area 850 is located at the center line of object 100 and so will be tested by the processor on its first pass. Each section 852 of area 850 will be checked to determine whether it is a good candidate for a trackable patch. The vector color data from the pixels 854 is checked to see if it contains characteristics which are expected to be readily identifiable and distinguishable from other areas.
- the processor begins at the center line, C, and checks outward for good trackable patches because features are most likely to be identifiable at the center.
- the computer can check for patches starting with a location other than the center of the image.
- the trackable patches are shown as squares which have a predetermined area or number of pixels, in this case, five by five.
- the trackable patches have variable or irregular sized areas. Different predefined sized areas can also be used. The important feature is that patches on the surface are identified in the image so that the points corresponding to those patches can be determined and tracked in other images.
- Figure 9 illustrates the process 356 of Figure 3 implemented to determine the radius of the location of patches on the surface of the object as the object is rotating.
- a set of two dimensional images of the object are acquired as the object rotates. These images include surface features of the object located in a two-dimensional space and it is desired to derive the three-dimensional location of the features from the two-dimensional information in the images.
- Features which are likely to be trackable are designated as trackable patches.
- a three dimensional surface is derived by locating each of the patches. This is done by tracking the patch across a number of views which represent angularly rotated images of the object. As the object is rotated (or virtually rotated if multiple detectors or a movable detector are used— whenever the object is described as being, rotated, it should be understood that virtual rotation is also included), features which are located a longer distance from the axis of rotation move a greater distance than features which are located a shorter distance from the axis of rotation.
- a radius from the axis of rotation of the object is derived for each trackable patch which will account for the movement of the trackable patch in the different images.
- Step 910 determines whether the processor is finished with all patches. If the processor is finished with all patches, then control is transferred to step 912 and the process ends. If more patches remain to be processed, then control is transferred to step 913 and a patch is chosen, together with an initial image in which the patch is found.
- step 914 the processor checks whether the patch lies outside of the tracking area within the image. If the patch is not outside the tracking limit, then the processor calculates the expected path of the patch in the image in step 916. If the image is the initial image to be processed, then the expected path will simply be the initial location of the patch in that image.
- the processor finds the exact position of the patch in the image and stores that position for that image.
- the processor updates the patch kernel.
- the reason for updating the patch kernel is that the patch will change as it moves across the image from the center towards the tracking limit.
- the tracking limit defines the area of the image in which patches will be tracked. This area is bounded in one embodiment across a 90 degree range. Although the tracking limit is set so that features within the limit remain relatively constant, it is possible near the edges of limit that the image of the patch will change more and more rapidly with angular displacement. The changing of the image of the patch makes it difficult to track the patch and increases the possibility of error.
- the patch may change considerably from the center of the region to the edge of the region, it is fortunate that, in many cases, the patch will change in a relatively continuous manner. Therefore, although the patch when viewed at the edge of the patch tracking region will appear very different than the patch when viewed at the center of the patch tracking region, the patches viewed in areas of the patch tracking region which are close to each other will be similar.
- the kernel used to search for the patch is changed with each successive image so that the patch searched for in each image is actually the patch which was found in the last image, not the initial patch which was defined in the center image. This makes it much easier to correlate the patch and to successfully track the patch.
- step 940 the data is filtered or smoothed. Filtering and smoothing consists of applying known standard statistical and filtering techniques for cleaning up the data and perhaps rejecting bad data points.
- the data consists of the positions determined across a plurality of images for each patch. This data is associated with the color pixel information for each patch.
- step 942 a recursive least square (“RLS”) algorithm is run to fit the points to a curve.
- step 944 the radius from the axis of rotation of the object to the surface of the object at the trackable patch is determined which will account for the motion of the trackable path across the images. Control is then transferred to step 910 which proceeds to the next patch or, if all patches have been processed, control is transferred to step 912 and the process ends.
- three-dimensional surface information about the object has been obtained. That three-dimensional surface information is integrally linked to color data because the vector color data of the trackable patch was used to find the patch in each successive image so that the motion of the patch could be determined as the object rotated in different images.
- the radius of the surface portion containing the trackable patch combined with the two-dimensional pixel information obtained from the optical detector is then be transformed to provide a three-dimensional representation of the point in any desired coordinate system.
- Figure 9A illustrates a set of patch tracking limits as used by step 914 of Figure 9. .
- Detector 110 views object 100 straight-on at point 900 on the surface of object 100, i.e. along the centerline C t of the object.
- surface features and trackable patches are viewed with the least amount of distortion at this head-on angle.
- the trackable patch is rotated away from point 900, it appears to detector 110 to change shape and becomes distorted.
- the judgment is made that it is not worthwhile to attempt to track trackable patches beyond point 902 and point 904, which are located at a 45 degree angular limit from point 900.
- the tracking limit in each image will be a vertical line corresponding to this angle. In other embodiments, other limits are chosen. Limits can also be learned by the system or determined based on the nature of the data so that no predetermined limits need to be set.
- Figure 9B illustrates the motion of trackable patches across different images with different angular displacements as calculated in step 916 of Figure 9.
- Object 100 in this image is a symmetrically shaped vase so that the silhouette of object 100 remains constant but features on the surface of object 100 move as object 100 is rotated.
- Object 100 is shown with a trackable patch 950, a trackable patch 952 and a trackable patch 954.
- Trackable patch 954 is shown located at the exact intersection of the horizontal and vertical center lines of the detector view.
- Trackable patch 954 is expected to move along a line 956 and so the search for trackable patch 954 in a successive image of object 100 should be executed along line 956.
- line 956 is straight because trackable patch 954 is located at the center point of the detector view in the first image.
- trackable patch 952 is expected to follow a line 958 in images in which trackable patch 952 is angularly displaced, and a line 959 is the path which trackable patch 950 is expected to follow in angularly displaced images.
- Line 958 is curved, unlike straight line 956, because line 958 does not pass through the center C d of the detector view.
- Features not located in the center are expected to follow epipolar trajectories as is known to those skilled in the art.
- Line 959 is even more curved than line 958 because line 959 is further from the center of the detector view than line 958.
- the distance the trackable patch moves along its expected path for a given angular displacement, or the speed of movement of each trackable patch as the object rotates, is not known.
- the change in position will vary with the radial distance of the trackable patch from the axis of rotation of the object. This is how the radius from the axis of rotation of the object is determined.
- Figure 9C illustrates the determination of an exact position of the patch in an image as determined by step 918 of Figure 9.
- Trackable patch 960 was observed in a former image and its location is defined by its center point 961.
- the expected path 962 of center point 961 is known.
- the processor searches for a good correlation of trackable patch 960 with the image data for patches along expected path 962.
- Patch 964 represented by a dashed line, does not present a good a correlation as patch 966, which is represented by a solid line.
- Patch 966 is determined to be the patch and the location of the trackable patch in the new image is determined to be point 968 which represents the location of patch 966.
- the center point of the patch is used to represent the location of the entire patch. This approach works as long as the patch is small. In other embodiments, multiple points are defined for a patch or a weighted average point which is based on the colors of the pixels which make up the patch is defined for the patch.
- the location of the patch in the different images and the motion of the patch in the different images which are angularly displaced is represented by the motion of the center point 961 which represents the center of trackable patch 960 to point 968 which represents the center of the angularly displaced patch 966.
- Figure 9D illustrates the filtering of the raw data points as performed in step 940 of Figure
- Point 970 is shown as being inaccurately derived because it is located away from the expected curve. After the filtering process, point 970 has been moved to point 972 which lies in the expected path.
- a number of standard filtering techniques can be used. The frequency of the rotation of the object is known and all surface feature motion should occur at a spatial frequency corresponding to that frequency. In one embodiment, a brick wall filter is used to filter the data to remove motion occurring at other frequencies.
- the spatial frequency of the occurrence of features in images will not correspond to a frequency of rotation of the object.
- the raw data is filtered using a modified Z-transform method. Modified Z-transform methods are used to filter data when the displacement is non-uniform.
- Figure 9E illustrates how the radius is determined from the points representing the path of the trackable patch across angularly displaced images and corresponds to steps 940 and 942 of Figure 9.
- Figure 9E is a plot of curve 980 which represents the first derivative of the distance, or the speed of the patch as it is angularly displaced from the center of the image versus the angular displacement of the patch from the center.
- curve 980 represents the first derivative of the distance, or the speed of the patch as it is angularly displaced from the center of the image versus the angular displacement of the patch from the center.
- the radial distance, R, of the patch from the axis of rotation of the object can be determined from a graph of the speed of the patch as shown on curve 980. Note that the frequency of this curve is well known so the data can be filtered and the amplitude of the curve can be accurately determined. After fitting the actual rate of displacement of the trackable patch to the cosine curve, the amplitude of the cosine curve is determined. The radius of the trackable patch is derived from the amplitude. Other processing methods are used in different embodiments to derive the radius. In one embodiment, the displacement of the patch, not the velocity is fit to a sine curve. An embodiment can use any processing method available to derive a radius which accounts for the patch motion.
- Figure 10 illustrates the post processing process that can occurs once the radius of the trackable patch is known.
- the process begins at step 1000.
- step 1010 the radial distance data for the trackable patches is combined with the two-dimensional trackable patch data from the images and converted to a desired coordinate system.
- step 1020 the color data associated with the trackable patch is converted to a desired color system for storage.
- Step 1030 performs interpolation and decimation of the points. In one embodiment, all of the trackable patch points are interpolated so that a regular array of surface points can be defined to model the surface of the object.
- step 1040 the points derived in step 1030 and the color values associated with those points in the form determined by step 1020 are stored in a data structure. The process is completed at 105.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Electromagnetism (AREA)
- Image Processing (AREA)
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU50880/98A AU5088098A (en) | 1996-10-25 | 1997-10-24 | Method and apparatus for three-dimensional color scanning |
JP51966498A JP2001503514A (en) | 1996-10-25 | 1997-10-24 | Three-dimensional color scanning method and apparatus |
CA002308619A CA2308619A1 (en) | 1996-10-25 | 1997-10-24 | Method and apparatus for three-dimensional color scanning |
EP97913769A EP0934502A1 (en) | 1996-10-25 | 1997-10-24 | Method and apparatus for three-dimensional color scanning |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/738,437 | 1996-10-25 | ||
US08/738,437 US5864640A (en) | 1996-10-25 | 1996-10-25 | Method and apparatus for optically scanning three dimensional objects using color information in trackable patches |
Publications (1)
Publication Number | Publication Date |
---|---|
WO1998017970A1 true WO1998017970A1 (en) | 1998-04-30 |
Family
ID=24968016
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1997/019331 WO1998017970A1 (en) | 1996-10-25 | 1997-10-24 | Method and apparatus for three-dimensional color scanning |
Country Status (7)
Country | Link |
---|---|
US (4) | US5864640A (en) |
EP (1) | EP0934502A1 (en) |
JP (1) | JP2001503514A (en) |
CN (1) | CN1247955C (en) |
AU (1) | AU5088098A (en) |
CA (1) | CA2308619A1 (en) |
WO (1) | WO1998017970A1 (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2000003356A1 (en) * | 1998-07-10 | 2000-01-20 | Koninklijke Philips Electronics N.V. | Method and device for optically determining depth |
WO2000004508A1 (en) * | 1998-07-20 | 2000-01-27 | Geometrix, Inc. | Automated 3d scene scanning from motion images |
WO2000058913A2 (en) * | 1999-03-26 | 2000-10-05 | Steffen Setzer | Device for pictorially depicting three-dimensional objects |
DE20301177U1 (en) | 2003-01-25 | 2003-05-22 | Massen, Robert, Prof. Dr.-Ing., 78337 Öhningen | Photogrammetric foot measurement unit has integrated camera track |
FR2978840A1 (en) * | 2011-08-02 | 2013-02-08 | Ortery Technologies Inc | Photographic studio for taking e.g. three-dimensional stereoscopic image of object, has motor driving pulley to rotate synchronization belt so as to cause rotation of other pulleys for starting/braking round transparent glass plate |
Families Citing this family (93)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1480903A (en) * | 1996-08-29 | 2004-03-10 | ������������ʽ���� | Specificity information assignment, object extraction and 3-D model generation method and appts thereof |
US5864640A (en) * | 1996-10-25 | 1999-01-26 | Wavework, Inc. | Method and apparatus for optically scanning three dimensional objects using color information in trackable patches |
US6377700B1 (en) * | 1998-06-30 | 2002-04-23 | Intel Corporation | Method and apparatus for capturing stereoscopic images using image sensors |
US20030158786A1 (en) * | 1999-02-26 | 2003-08-21 | Skyline Software Systems, Inc. | Sending three-dimensional images over a network |
US6744914B1 (en) * | 2000-04-28 | 2004-06-01 | Orametrix, Inc. | Method and system for generating a three-dimensional object |
AU4221000A (en) * | 1999-04-08 | 2000-10-23 | Internet Pictures Corporation | Remote controlled platform for camera |
CA2267519A1 (en) | 1999-04-13 | 2000-10-13 | Inspeck Inc. | Optical full human body 3d digitizer |
JP4288753B2 (en) * | 1999-05-31 | 2009-07-01 | コニカミノルタセンシング株式会社 | 3D data input device |
EP1074943A3 (en) * | 1999-08-06 | 2004-03-24 | Canon Kabushiki Kaisha | Image processing method and apparatus |
US7359070B1 (en) * | 1999-09-30 | 2008-04-15 | Sharp Laboratories Of America, Incorporated | Multi-function peripheral with camera |
US6484066B1 (en) | 1999-10-29 | 2002-11-19 | Lockheed Martin Corporation | Image life tunnel scanner inspection system using extended depth of field technology |
US6343184B1 (en) * | 2000-03-09 | 2002-01-29 | Roger F. Huebner | Controlled environment photo studio |
GB2368117A (en) * | 2000-05-08 | 2002-04-24 | Neutral Ltd | Optical viewing arrangement |
EP1185112B1 (en) * | 2000-08-25 | 2005-12-14 | Fuji Photo Film Co., Ltd. | Apparatus for parallax image capturing and parallax image processing |
US7625335B2 (en) * | 2000-08-25 | 2009-12-01 | 3Shape Aps | Method and apparatus for three-dimensional optical scanning of interior surfaces |
AU9081001A (en) * | 2000-09-13 | 2002-03-26 | Nextengine Inc | Imaging system monitored or controlled to ensure fidelity of file captured |
US7358986B1 (en) * | 2000-09-13 | 2008-04-15 | Nextengine, Inc. | Digital imaging system having distribution controlled over a distributed network |
US6856407B2 (en) | 2000-09-13 | 2005-02-15 | Nextengine, Inc. | Method for depth detection in 3D imaging providing a depth measurement for each unitary group of pixels |
US6639684B1 (en) * | 2000-09-13 | 2003-10-28 | Nextengine, Inc. | Digitizer using intensity gradient to image features of three-dimensional objects |
ITBO20000608A1 (en) * | 2000-10-18 | 2002-04-18 | Gd Spa | METHOD AND AUTOMATIC MACHINE FOR THE PROCESSING OF A PRODUCT |
TW492558U (en) * | 2001-01-18 | 2002-06-21 | Chitai Technology Co Ltd | Passive three-dimensional taking picture control apparatus |
US7233351B1 (en) | 2001-02-23 | 2007-06-19 | Nextengine, Inc. | Method for high resolution incremental imaging |
US8032337B2 (en) * | 2001-03-02 | 2011-10-04 | 3Shape A/S | Method for modeling customized earpieces |
DE10111919A1 (en) * | 2001-03-13 | 2002-09-19 | Boegl Max Bauunternehmung Gmbh | guideway beams |
US7365672B2 (en) * | 2001-03-16 | 2008-04-29 | Battelle Memorial Institute | Detection of a concealed object |
US7405692B2 (en) * | 2001-03-16 | 2008-07-29 | Battelle Memorial Institute | Detecting concealed objects at a checkpoint |
US6507309B2 (en) | 2001-03-16 | 2003-01-14 | Battelle Memorial Institute | Interrogation of an object for dimensional and topographical information |
AU2002323027A1 (en) * | 2001-08-07 | 2003-02-24 | Ernest A. Franke | Apparatus and methods of generation of textures with depth buffers |
US7253832B2 (en) * | 2001-08-13 | 2007-08-07 | Olympus Corporation | Shape extraction system and 3-D (three dimension) information acquisition system using the same |
GB0120277D0 (en) | 2001-08-18 | 2001-10-17 | Snap On Equipment Ltd | Three-dimensional mapping systems for automotive vehicles and other articles |
JP2003065736A (en) * | 2001-08-24 | 2003-03-05 | Sanyo Electric Co Ltd | Three-dimensional modeling apparatus |
US7116338B2 (en) * | 2001-09-26 | 2006-10-03 | Canon Kabushiki Kaisha | Color information processing apparatus and method |
KR20030071104A (en) * | 2002-02-27 | 2003-09-03 | 다인디지털(주) | Method of Digital Data Build for Buddhist Sutra |
DE50201004D1 (en) * | 2002-03-01 | 2004-10-21 | Brainlab Ag | Operating lamp with camera system for 3D referencing |
JP4043258B2 (en) * | 2002-03-13 | 2008-02-06 | オリンパス株式会社 | 3D imaging device |
US7068303B2 (en) * | 2002-06-03 | 2006-06-27 | Microsoft Corporation | System and method for calibrating a camera with one-dimensional objects |
US6969836B2 (en) * | 2002-10-25 | 2005-11-29 | General Electric Company | Space carving for measurement of high-curvature areas on a mechanical part |
CA2507642C (en) * | 2002-11-29 | 2010-06-01 | Obe Ohnmacht & Baumgaertner Gmbh & Co. Kg | Method and device for optical shape measurement and/or evaluation |
GB0301775D0 (en) * | 2003-01-25 | 2003-02-26 | Wilson John E | Device and method for 3Dimaging |
US7995831B2 (en) * | 2003-01-30 | 2011-08-09 | Express Scripts, Inc. | Prescription bottle imaging system and method |
PT1600008E (en) * | 2003-02-27 | 2008-06-12 | T Mobile Deutschland Gmbh | Method for the compressed transmission of image data for three-dimensional representation of scenes and objects |
JP3700707B2 (en) * | 2003-03-13 | 2005-09-28 | コニカミノルタホールディングス株式会社 | Measuring system |
DE10315923A1 (en) * | 2003-04-08 | 2004-10-28 | Tbs Holding Ag | Procedure to detect data of uneven surfaces for biometric data, using non-contact optical sensing of surface |
US20050035246A1 (en) * | 2003-07-28 | 2005-02-17 | Coleman Ludlow Peter | Remotely controllable revolving support for speaker |
US20050088515A1 (en) * | 2003-10-23 | 2005-04-28 | Geng Z. J. | Camera ring for three-dimensional (3D) surface imaging |
CN1922613A (en) * | 2004-01-14 | 2007-02-28 | 国际条形码公司 | System and method for compensating for bar code image distortions |
US7711179B2 (en) * | 2004-04-21 | 2010-05-04 | Nextengine, Inc. | Hand held portable three dimensional scanner |
ATE406850T1 (en) | 2004-06-17 | 2008-09-15 | Cadent Ltd | METHOD AND DEVICE FOR COLOR IMAGE FORMING OF A THREE-DIMENSIONAL STRUCTURE |
EP1941843B1 (en) | 2004-06-17 | 2021-11-17 | Align Technology, Inc. | Method and apparatus for colour imaging a three-dimensional structure |
JP2006004158A (en) * | 2004-06-17 | 2006-01-05 | Olympus Corp | Image processing program, image processing method, image processor, and recording medium |
US20060034491A1 (en) * | 2004-08-16 | 2006-02-16 | Manish Sharma | System for visualising a surface structure |
US20060045174A1 (en) * | 2004-08-31 | 2006-03-02 | Ittiam Systems (P) Ltd. | Method and apparatus for synchronizing a transmitter clock of an analog modem to a remote clock |
US7167252B2 (en) * | 2004-11-23 | 2007-01-23 | Kevin Gallup | Method and apparatus for creating cavities in packaging materials for artifacts, art objects and fragile or other valuable items |
DE102005018336A1 (en) * | 2005-02-28 | 2006-08-31 | Osram Opto Semiconductors Gmbh | Optical fiber for use in optoelectronic component e.g. light emitting diode, has uneven light aperture, in which large portion of electromagnetic radiation generated by light source enters into fiber due to unevenness of aperture |
WO2007030026A1 (en) * | 2005-09-09 | 2007-03-15 | Industrial Research Limited | A 3d scene scanner and a position and orientation system |
TWI273509B (en) * | 2005-09-13 | 2007-02-11 | Ind Tech Res Inst | Method for rapidly building image space relation using plane filtering limitation |
CN100346745C (en) * | 2005-09-27 | 2007-11-07 | 四川大学 | Human cranlofacial 3-D measurer and measuring method |
US7995834B1 (en) | 2006-01-20 | 2011-08-09 | Nextengine, Inc. | Multiple laser scanner |
FR2897954B3 (en) * | 2006-02-27 | 2008-02-01 | Ortery Technologies Inc | COMPUTER-CONTROLLED SYSTEM FOR SYNCHRONIZING PHOTOGRAPHING BETWEEN 3-D ROTATING TRAY AND IMAGE CAPTURE DEVICE WITH AUTOMATIC IMAGE FORMAT CONVERSION |
US7526105B2 (en) * | 2006-03-29 | 2009-04-28 | Mark Dronge | Security alarm system |
US7844081B2 (en) * | 2006-05-15 | 2010-11-30 | Battelle Memorial Institute | Imaging systems and methods for obtaining and using biometric information |
US7589310B2 (en) * | 2006-06-05 | 2009-09-15 | Blaise Laurent Mouttet | Image correlation sensor |
US7855732B2 (en) * | 2006-09-05 | 2010-12-21 | Pc Connection, Inc. | Hand producer for background separated images |
US7953277B2 (en) | 2006-09-05 | 2011-05-31 | Williams Robert C | Background separated images for print and on-line use |
ITBO20060624A1 (en) * | 2006-09-05 | 2008-03-06 | Crea Solution S R L | METHOD AND MACHINE FOR CATALOGING DIGITAL ITEMS |
US7931380B2 (en) | 2006-09-05 | 2011-04-26 | Williams Robert C | Imaging apparatus for providing background separated images |
JP4325653B2 (en) * | 2006-09-08 | 2009-09-02 | セイコーエプソン株式会社 | Liquid ejector |
US7834894B2 (en) * | 2007-04-03 | 2010-11-16 | Lifetouch Inc. | Method and apparatus for background replacement in still photographs |
JP2009125835A (en) * | 2007-11-21 | 2009-06-11 | Toppan Printing Co Ltd | Punch hole punching apparatus and bag manufacturing machine with the same |
US20100321475A1 (en) * | 2008-01-23 | 2010-12-23 | Phillip Cox | System and method to quickly acquire three-dimensional images |
WO2010111615A1 (en) | 2009-03-27 | 2010-09-30 | Hard Rock Cafe International (Usa), Inc. | Apparatus and method for performing a timed and controlled movement and positioning of an object |
US8823775B2 (en) * | 2009-04-30 | 2014-09-02 | Board Of Regents, The University Of Texas System | Body surface imaging |
JP5170154B2 (en) * | 2010-04-26 | 2013-03-27 | オムロン株式会社 | Shape measuring apparatus and calibration method |
TWI420440B (en) * | 2010-08-16 | 2013-12-21 | Hon Hai Prec Ind Co Ltd | Object exhibition system and method |
WO2012048420A1 (en) * | 2010-10-15 | 2012-04-19 | Silonex Inc. | Optical positioning system and method |
US8942917B2 (en) | 2011-02-14 | 2015-01-27 | Microsoft Corporation | Change invariant scene recognition by an agent |
US9053571B2 (en) | 2011-06-06 | 2015-06-09 | Microsoft Corporation | Generating computer models of 3D objects |
US9258550B1 (en) | 2012-04-08 | 2016-02-09 | Sr2 Group, Llc | System and method for adaptively conformed imaging of work pieces having disparate configuration |
CN103377469B (en) * | 2012-04-23 | 2016-05-04 | 宇龙计算机通信科技(深圳)有限公司 | Terminal and image processing method |
US9857470B2 (en) | 2012-12-28 | 2018-01-02 | Microsoft Technology Licensing, Llc | Using photometric stereo for 3D environment modeling |
US9940553B2 (en) | 2013-02-22 | 2018-04-10 | Microsoft Technology Licensing, Llc | Camera/object pose from predicted coordinates |
CN103307998B (en) * | 2013-05-27 | 2016-03-23 | 深圳先进技术研究院 | A kind of 3 D scanning system and method |
CN103885279A (en) * | 2014-03-27 | 2014-06-25 | 大连理工大学 | Space three-dimensional image acquisition system |
WO2016040466A1 (en) * | 2014-09-10 | 2016-03-17 | Hasbro, Inc. | Toy system with manually operated scanner |
DK3291730T3 (en) | 2015-05-04 | 2023-11-27 | T&W Eng A/S | EEG MONITOR WITH A HOUSING INDIVIDUALLY ADAPTED TO A PERSON |
DK3436773T3 (en) | 2016-04-01 | 2021-07-12 | Lego As | Toy scanner |
CN106973226B (en) * | 2017-03-30 | 2020-01-24 | 上海明殿文化传播有限公司 | Shooting method and terminal |
WO2019185993A1 (en) * | 2018-03-28 | 2019-10-03 | Engemma Oy | Extended focal plane imaging by using a gemstone manipulator |
DE102018216458A1 (en) * | 2018-09-26 | 2020-03-26 | Hochschule für angewandte Wissenschaften Kempten | Process for recording the surface of a rotating body |
JP7460532B2 (en) | 2018-10-10 | 2024-04-02 | 株式会社Preferred Networks | systems, methods and devices |
JP7193425B2 (en) * | 2019-07-18 | 2022-12-20 | 株式会社ミマキエンジニアリング | 3D data generation device, 3D data generation method, and molding system |
CN113820752B (en) * | 2021-09-14 | 2023-12-19 | 航天科工哈尔滨风华有限公司 | Rotary swinging mechanism of direct-drive type microwave scanning reflecting plate |
US20230147392A1 (en) * | 2021-11-10 | 2023-05-11 | Motional Ad Llc | Two-axis test fixture |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4969106A (en) * | 1989-02-27 | 1990-11-06 | Camsys, Inc. | Computerized method of determining surface strain distributions in a deformed body |
Family Cites Families (35)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4175862A (en) * | 1975-08-27 | 1979-11-27 | Solid Photography Inc. | Arrangement for sensing the geometric characteristics of an object |
US4298800A (en) * | 1978-02-27 | 1981-11-03 | Computome Corporation | Tomographic apparatus and method for obtaining three-dimensional information by radiation scanning |
US4645347A (en) * | 1985-04-30 | 1987-02-24 | Canadian Patents And Development Limited-Societe Canadienne Des Brevets Et D'exploitation Limitee | Three dimensional imaging device |
DE3578241D1 (en) * | 1985-06-19 | 1990-07-19 | Ibm | METHOD FOR IDENTIFYING THREE-DIMENSIONAL OBJECTS BY MEANS OF TWO-DIMENSIONAL IMAGES. |
US4737032A (en) * | 1985-08-26 | 1988-04-12 | Cyberware Laboratory, Inc. | Surface mensuration sensor |
FR2589242B1 (en) | 1985-10-25 | 1988-11-25 | Oreal | PROCESS FOR EXAMINING THE SURFACE OF A SAMPLE AND APPARATUS FOR IMPLEMENTING SAME |
DE3624959A1 (en) * | 1986-07-23 | 1988-01-28 | Mannesmann Ag | AUTOMATIC METHOD FOR CONTACTLESS THREE-DIMENSIONAL MEASUREMENT OF OBJECTS OF LARGE EXPANSION |
JPH077444B2 (en) * | 1986-09-03 | 1995-01-30 | 株式会社東芝 | Connected component extractor for 3D images |
US4792696A (en) | 1987-06-05 | 1988-12-20 | Trustees Of Columbia University In The City Of New York | Method and an apparatus for determining surface shape utilizing object self-shadowing |
DE3817559C1 (en) * | 1988-05-24 | 1989-12-07 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung Ev, 8000 Muenchen, De | |
JPH02138805A (en) * | 1988-08-31 | 1990-05-28 | Canon Inc | Smoothness measuring apparatus and recorder therewith |
US4912336A (en) | 1989-02-21 | 1990-03-27 | Westinghouse Electric Corp. | Surface shape and reflectance extraction system |
JPH0834532B2 (en) * | 1990-05-15 | 1996-03-29 | キヤノン株式会社 | Color image reader |
US5226120A (en) * | 1990-05-21 | 1993-07-06 | Synoptics Communications, Inc. | Apparatus and method of monitoring the status of a local area network |
US5261044A (en) * | 1990-09-17 | 1993-11-09 | Cabletron Systems, Inc. | Network management system using multifunction icons for information display |
US5179554A (en) * | 1991-04-08 | 1993-01-12 | Digital Equipment Corporation | Automatic association of local area network station addresses with a repeater port |
US5321695A (en) * | 1991-05-01 | 1994-06-14 | Hewlett-Packard Company | Port arrival identification for computer network packets |
US5528194A (en) * | 1991-05-13 | 1996-06-18 | Sony Corporation | Apparatus and method for performing geometric transformations on an input image |
US5577130A (en) * | 1991-08-05 | 1996-11-19 | Philips Electronics North America | Method and apparatus for determining the distance between an image and an object |
US5402364A (en) * | 1993-01-15 | 1995-03-28 | Sanyo Machine Works, Ltd. | Three dimensional measuring apparatus |
JPH0779233A (en) * | 1993-06-29 | 1995-03-20 | Synoptics Commun Inc | Apparatus for establishing topology, method and apparatus for communicating topology information |
US5511153A (en) | 1994-01-18 | 1996-04-23 | Massachusetts Institute Of Technology | Method and apparatus for three-dimensional, textured models from plural video images |
JPH07234927A (en) * | 1994-02-22 | 1995-09-05 | Toshiba Medical Eng Co Ltd | Three-dimensional image display device |
US5611000A (en) | 1994-02-22 | 1997-03-11 | Digital Equipment Corporation | Spline-based image registration |
US5684796A (en) * | 1994-05-03 | 1997-11-04 | Bay Networks Group, Inc. | Method and apparatus for determining and maintaining agent topology information in a multi-segment network |
US5561526A (en) * | 1994-05-26 | 1996-10-01 | Lockheed Missiles & Space Company, Inc. | Three-dimensional measurement device and system |
US5559334A (en) | 1995-05-22 | 1996-09-24 | General Electric Company | Epipolar reconstruction of 3D structures |
US5675377A (en) * | 1995-06-30 | 1997-10-07 | Telefonaktiebolaget Lm Ericsson | True three-dimensional imaging and display system |
US5706440A (en) * | 1995-08-23 | 1998-01-06 | International Business Machines Corporation | Method and system for determining hub topology of an ethernet LAN segment |
US5737084A (en) * | 1995-09-29 | 1998-04-07 | Takaoka Electric Mtg. Co., Ltd. | Three-dimensional shape measuring apparatus |
US5818959A (en) | 1995-10-04 | 1998-10-06 | Visual Interface, Inc. | Method of producing a three-dimensional image from two-dimensional images |
US6009188A (en) | 1996-02-16 | 1999-12-28 | Microsoft Corporation | Method and system for digital plenoptic imaging |
US5864640A (en) * | 1996-10-25 | 1999-01-26 | Wavework, Inc. | Method and apparatus for optically scanning three dimensional objects using color information in trackable patches |
US5917937A (en) | 1997-04-15 | 1999-06-29 | Microsoft Corporation | Method for performing stereo matching to recover depths, colors and opacities of surface elements |
US6009190A (en) | 1997-08-01 | 1999-12-28 | Microsoft Corporation | Texture map construction method and apparatus for displaying panoramic image mosaics |
-
1996
- 1996-10-25 US US08/738,437 patent/US5864640A/en not_active Expired - Fee Related
-
1997
- 1997-10-24 AU AU50880/98A patent/AU5088098A/en not_active Abandoned
- 1997-10-24 CA CA002308619A patent/CA2308619A1/en not_active Abandoned
- 1997-10-24 EP EP97913769A patent/EP0934502A1/en not_active Withdrawn
- 1997-10-24 CN CNB971809399A patent/CN1247955C/en not_active Expired - Fee Related
- 1997-10-24 WO PCT/US1997/019331 patent/WO1998017970A1/en not_active Application Discontinuation
- 1997-10-24 JP JP51966498A patent/JP2001503514A/en active Pending
-
1999
- 1999-01-25 US US09/236,727 patent/US6288385B1/en not_active Expired - Fee Related
-
2001
- 2001-08-31 US US09/945,133 patent/US20020043632A1/en not_active Abandoned
-
2002
- 2002-07-30 US US10/208,971 patent/US6747260B2/en not_active Expired - Fee Related
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4969106A (en) * | 1989-02-27 | 1990-11-06 | Camsys, Inc. | Computerized method of determining surface strain distributions in a deformed body |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2000003356A1 (en) * | 1998-07-10 | 2000-01-20 | Koninklijke Philips Electronics N.V. | Method and device for optically determining depth |
WO2000004508A1 (en) * | 1998-07-20 | 2000-01-27 | Geometrix, Inc. | Automated 3d scene scanning from motion images |
WO2000058913A2 (en) * | 1999-03-26 | 2000-10-05 | Steffen Setzer | Device for pictorially depicting three-dimensional objects |
WO2000058913A3 (en) * | 1999-03-26 | 2001-04-05 | Steffen Setzer | Device for pictorially depicting three-dimensional objects |
DE20301177U1 (en) | 2003-01-25 | 2003-05-22 | Massen, Robert, Prof. Dr.-Ing., 78337 Öhningen | Photogrammetric foot measurement unit has integrated camera track |
FR2978840A1 (en) * | 2011-08-02 | 2013-02-08 | Ortery Technologies Inc | Photographic studio for taking e.g. three-dimensional stereoscopic image of object, has motor driving pulley to rotate synchronization belt so as to cause rotation of other pulleys for starting/braking round transparent glass plate |
Also Published As
Publication number | Publication date |
---|---|
US6747260B2 (en) | 2004-06-08 |
AU5088098A (en) | 1998-05-15 |
JP2001503514A (en) | 2001-03-13 |
US20020043632A1 (en) | 2002-04-18 |
CA2308619A1 (en) | 1998-04-30 |
CN1247955C (en) | 2006-03-29 |
US6288385B1 (en) | 2001-09-11 |
US5864640A (en) | 1999-01-26 |
US20020190190A1 (en) | 2002-12-19 |
EP0934502A1 (en) | 1999-08-11 |
CN1242074A (en) | 2000-01-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US5864640A (en) | Method and apparatus for optically scanning three dimensional objects using color information in trackable patches | |
US7098435B2 (en) | Method and apparatus for scanning three-dimensional objects | |
US6858826B2 (en) | Method and apparatus for scanning three-dimensional objects | |
CN111462213B (en) | Equipment and method for acquiring 3D coordinates and dimensions of object in motion process | |
CN111292364B (en) | Method for rapidly matching images in three-dimensional model construction process | |
US6781618B2 (en) | Hand-held 3D vision system | |
US6750873B1 (en) | High quality texture reconstruction from multiple scans | |
JP4245963B2 (en) | Method and system for calibrating multiple cameras using a calibration object | |
CN111292239B (en) | Three-dimensional model splicing equipment and method | |
CN109215111B (en) | Indoor scene three-dimensional modeling method based on laser range finder | |
WO1999053427A1 (en) | Face recognition from video images | |
KR20160019497A (en) | Interactive and automatic 3-d object scanning method for the purpose of database creation | |
CN112465960A (en) | Dimension calibration device and method for three-dimensional model | |
CN111340959B (en) | Three-dimensional model seamless texture mapping method based on histogram matching | |
EP1580684B1 (en) | Face recognition from video images | |
JPH0766436B2 (en) | 3D model construction device using continuous silhouette images | |
Chidambaram | Edge Extraction of Color and Range Images | |
CN113538552A (en) | 3D information synthesis image matching method based on image sorting | |
Rawlinson | Design and implementation of a spatially enabled panoramic virtual reality prototype | |
Christensen et al. | Hybrid approach to the construction of triangulated 3D models of building interiors | |
AU2004212509B2 (en) | Face recognition from video images | |
Guerrero et al. | Camera motion from brightness on lines. Combination of features and normal flow | |
Gurram et al. | A segment-based mesh design for building parallel-perspective stereo mosaics | |
Lang et al. | Active object modeling with VIRTUE | |
Smith et al. | Automatic feature correspondence for scene reconstruction |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 97180939.9 Country of ref document: CN |
|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE GH HU ID IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG UZ VN YU ZW AM AZ BY KG KZ MD RU TJ TM |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH KE LS MW SD SZ UG ZW AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
ENP | Entry into the national phase |
Ref document number: 1998 519664 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1997913769 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 1997913769 Country of ref document: EP |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
ENP | Entry into the national phase |
Ref document number: 2308619 Country of ref document: CA Ref document number: 2308619 Country of ref document: CA Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: CA |
|
WWW | Wipo information: withdrawn in national office |
Ref document number: 1997913769 Country of ref document: EP |