WO2009098894A1 - 電子カメラおよび画像処理方法 - Google Patents
電子カメラおよび画像処理方法 Download PDFInfo
- Publication number
- WO2009098894A1 WO2009098894A1 PCT/JP2009/000477 JP2009000477W WO2009098894A1 WO 2009098894 A1 WO2009098894 A1 WO 2009098894A1 JP 2009000477 W JP2009000477 W JP 2009000477W WO 2009098894 A1 WO2009098894 A1 WO 2009098894A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- update
- tracking
- feature
- region
- image
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
- G02B7/28—Systems for automatic generation of focusing signals
- G02B7/36—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
- G02B7/365—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals by analysis of the spatial frequency components of the image
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B13/00—Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
- G03B13/32—Means for focusing
- G03B13/34—Power focusing
- G03B13/36—Autofocus systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0007—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
- H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Definitions
- the present invention relates to an electronic camera that tracks an object in an image and presents a tracking result on a display and a method thereof.
- a face detection function is mounted on an electronic camera as an alignment means for AF (Auto-Focus), AE (Auto-Exposure), and backlight compensation function.
- AF Auto-Focus
- AE Auto-Exposure
- backlight compensation function By using the face detection function, it is possible to automatically focus on the subject, which is an effective means for assisting the user in photographing.
- FIG. 1 shows an image processing apparatus disclosed in Patent Document 1.
- FIG. 2 is a diagram illustrating an example of cutting out a partial image.
- the partial image cutout unit 900 cuts out the partial image 1000 from the input image 1001.
- the partial image 1000 is cut out by scanning a plurality of size windows from the upper left of the image to the lower right of the image while sequentially shifting an appropriate pixel (for example, one pixel) rightward or downward (see FIG. 2).
- cutout refers to reading out only the image data of a corresponding part.
- Feature amount evaluation unit 1 is composed of a combination of a plurality of discriminators.
- the discriminator calculates a feature amount based on a rectangular feature (to be referred to as an adjacent difference filter hereinafter) at a position designated using a parameter learned using the boosting technique. If the weighted linear sum of the output values of the discriminator is lower than the threshold value calculated based on learning, the feature quantity evaluating unit 1 (901) discriminates the face as a non-face and ends the partial image discrimination process. On the other hand, if it is equal to or greater than the threshold, the face is identified, and the process moves to the next feature quantity evaluation unit 2 (902).
- the feature amount evaluation unit 2 (902) performs evaluation using parameters different from the learning parameters used in the feature amount evaluation unit 1 (901). As described above, the evaluation value is calculated using a plurality of feature amount evaluation units, and the face and the non-face are discriminated based on the calculated evaluation value.
- FIG. 3A, 3B, 3C, and 3D illustrate the adjacent difference filter
- FIG. 3E illustrates an example in which the adjacent difference filter (b) is applied to the image.
- the adjacent difference filter is indicated by the adjacent white rectangle and black rectangle, and outputs the difference between the pixel average value in the white rectangle and the pixel average value in the black rectangle.
- the feature of the facial part is specified by the large difference in the pixel average value output from the adjacent difference filter, and this is a region where the difference in the pixel value of the region between the adjacent regions is large, such as the eyes and mouth. Means that a high feature value is output. For example, in FIG.
- the feature amount is calculated based on the difference between the sum of black rectangular pixels at the forehead position and the sum of white rectangular pixels at the eyebrow position. Since this feature amount is a difference between adjacent regions, it strongly reacts to a local feature (for example, a line component) in the image, and outputs a characteristic value in facial parts such as eyes, eyebrows, and mouth. Note that the adjacent difference filter is generally referred to as a “Har-like feature”.
- Conventionally used object tracking methods include face tracking by re-detecting only the vicinity region of the face detection result position, template matching using previous frame neighborhood search based on correlation calculation, and active search aimed at speeding up There are a method based on this, a particle filter that performs motion prediction using statistical information based on probability distribution, and a method based on condensation.
- the initial feature amount (color histogram of color or brightness, template image itself, shape / contour information, etc.) of an object to be tracked using some method.
- Object tracking is performed by searching for a location similar to the registered feature in the image using the registered feature.
- an initial feature amount is created in advance, and the feature amount and a feature amount extracted based on each position of the image are matched.
- Patent Document 2 uses a method of sequentially updating templates. According to this method, even when the appearance of the tracking target changes, the template is updated in accordance with this change. For this reason, it becomes possible to track an object whose appearance changes.
- the present invention eliminates an error in tracking an area where an object exists by determining an update area using a feature capable of stably detecting the position of the object. Further, when the deviation from the correct position becomes large as a result of the tracking, the tracking is stably performed by correcting to the correct position.
- the electronic camera of the present invention has an electronic camera having a function of specifying and displaying a target area in which a target object to be tracked is captured in each frame of continuously captured images. Then, using a first registered feature quantity quantitatively indicating the feature of the object, a search is made within a predetermined range in the frame, and the region within the region obtained as the search result A first evaluation value indicating a degree of coincidence between the image and the image of the object is calculated, and based on the calculated first evaluation value, a target region that is estimated that the image of the object exists The tracking processing unit to be identified is different from the first feature value, and a second feature value quantitatively indicating the feature of the object is used to determine a predetermined value in the frame.
- a second evaluation value indicating the degree of coincidence with the image of the object is calculated for the image in the region obtained as the search result, and the first evaluation value is calculated based on the calculated second evaluation value.
- An update area calculation unit that specifies an update area for updating the feature amount in the frame, the first evaluation value calculated by the tracking processing unit, and the second value calculated by the update area calculation unit
- An update determination unit that determines whether or not to update the first feature amount by examining whether or not at least one of the evaluation values satisfies a predetermined condition; and the update determination unit
- a registered feature value update unit that updates the first feature value with the new first feature value extracted in the update region, and the update Before by judgment part
- the target region specified in the tracking processing unit is specified as the target region, information about the target region is drawn, and the update determining unit
- a tracking result drawing unit that draws information related to the update region by specifying the update region specified by the update region calculation unit as the target region
- the tracking processing unit specifies a target area where the image of the target object is estimated to exist using the color information of the target object as the first feature amount, and the update area calculation unit includes As the second feature amount, the update area may be specified using shape information of the object.
- the update determination unit is configured to calculate the update region when the second evaluation value calculated by the update region calculation unit is larger than the first evaluation value calculated by the tracking processing unit.
- the first feature value may be determined to be updated when the second evaluation value calculated in the unit is greater than a predetermined first threshold value.
- the first feature amount needs to be updated every time. The amount of processing can be reduced.
- the update determination unit sets a larger value for the first threshold as the average saturation for each pixel of the image of the object is higher, and sets the first threshold as the average saturation of the object is lower.
- the update determination unit that sets a small value is when the second evaluation value calculated by the update region calculation unit is larger than the first evaluation value calculated by the tracking processing unit, or
- the tracking processing unit that determines to update the first feature value when the second evaluation value calculated by the update region calculation unit is greater than a predetermined first threshold value is the first feature value
- the target area where the image of the object is estimated to be present is identified using the color information of the object, and the update area calculation unit uses the shape information of the object as the second feature amount. Special update area It may be used to.
- the distance between the target region specified in the tracking processing unit and the update region specified in the update region calculation unit is equal to or greater than a predetermined second threshold. Sometimes, it may be determined to update.
- the first feature amount can be updated when the position of the tracking processing result by the tracking processing unit and the position of the updating region detection result by the updating region calculation unit is greatly shifted, even when tracking fails, The tracking process can be restored using the updated first feature value, and the tracking performance is stabilized.
- the electronic camera further includes an end determination unit that determines whether or not the tracking of the object cannot be continued, and the end determination unit includes the first evaluation value and the second evaluation value. When both are smaller than a predetermined third threshold, it may be determined that tracking of the object cannot be continued.
- the tracking result drawing unit may finish drawing the target area and the update area when it is determined that the tracking of the target cannot be continued.
- the tracking result drawing unit may draw an image that indicates to the user that tracking is impossible when it is determined that tracking of the target cannot be continued.
- the tracking result drawing unit may draw an image that prompts the user to set the initial position again when it is determined that the tracking of the target cannot be continued.
- the tracking result drawing unit may set the initial position by detecting a new face when it is determined that the tracking of the target cannot be continued.
- the electronic camera further includes a face detection circuit that detects a face in an image in each frame, and a face orientation recognition circuit that recognizes the orientation of the face detected by the face detection circuit.
- the calculation unit calculates an update reference region that is a face region facing directly sideways in the update region calculation unit. Thereafter, the update region in the update reference region may be calculated based on a predetermined positional relationship between the face portion and the hair portion.
- the location to be updated changes in advance according to the update region detection result, for example, the location to be updated changes according to the face orientation of the face detection result, or more depending on the size of the face detection Updating in various positional relationships, such as updating using a large size or updating in a form that includes clothes, is possible, and tracking accuracy is improved.
- this makes it possible to track the tracking object stably, for example, when the appearance does not change even if it rotates like a circular shape.
- the electronic camera further adjusts the operation of the electronic camera based on one of the target area specified by the tracking processing unit and the update area specified by the update area calculation unit.
- a camera control unit that changes the camera parameters may be included.
- the camera control unit controls the operation of at least one of a housing and a pan head of the electronic camera based on the camera parameter, and the specified object as a whole or a part thereof in a predetermined position in a frame. Control may be performed to match the size.
- the electronic camera may further include a target area initial setting unit that determines an initial position of the target area based on an input from a user or using a predetermined method.
- the target area initial setting unit may determine the detection position of either a person or a face as the initial position.
- the target area initial setting unit may determine a position focused by an AF (autofocus) function as an initial position.
- the present invention can be realized not only as an apparatus but also as a method using steps as processing units constituting the apparatus, as a program for causing a computer to execute the steps, or as a computer read recording the program. It can be realized as a possible recording medium such as a CD-ROM, or as information, data or a signal indicating the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.
- FIG. 1 is a flowchart of conventional face detection processing.
- FIG. 2 is an explanatory diagram of face detection processing from a conventional image.
- 3A to 3E are explanatory diagrams of a conventional adjacent difference filter.
- FIG. 4 is a block diagram of the image processing apparatus according to the first and second embodiments of the present invention.
- FIGS. 5A and 5B are explanatory diagrams of an example in which tracking in the conventional method fails.
- 6A and 6B are explanatory diagrams of color histogram search.
- FIGS. 7A and 7B are explanatory views of similarity calculation of color histograms.
- FIGS. 8A and 8B are explanatory views of a profile image of a profile.
- FIG. 9 is a flowchart of the first embodiment.
- 10A to 10C are relationship diagrams between the tracking feature amount and the update region detection result.
- FIG. 11 is a flowchart of the second embodiment.
- Image processing device (electronic camera) DESCRIPTION OF SYMBOLS 101 Lens 102 Shutter 103 Image pick-up element 104 AD converter 105 Timing generation circuit 106 DA converter 107 Memory control circuit 108 System control circuit 109 Image processing circuit 110 Image display memory 111 Memory 112 Resizing circuit 113 Flash 114 Distance measurement control part 115 Zoom control Unit 116 barrier control unit 117 protection unit 118 memory 119 display unit 120 nonvolatile memory 121 mode dial 123 shutter switch 124 recording unit 125 power source control unit 126 power source unit 127 connector 128 power source unit 129 interface 130 interface 131 connector 132 connector 133 optical finder 134 Communication unit 135 Antenna 136 Initial feature amount extraction unit 137 Tracking processing unit 38 Tracking start frame designation circuit 139 Update area detection circuit 140 Update determination circuit 141 End determination circuit 142 Position correction circuit 143 Registered feature amount update circuit 144 Tracking result drawing circuit 145 Camera control circuit 146 Face detection circuit 147 Face direction recognition circuit 148 Image display Section 149 Exposure control section 200 Tracking result in
- the image processing apparatus (electronic camera) according to the first embodiment specifies a target area where an object to be tracked is to be present, using conventional face detection and face orientation recognition. Then, every time the target area is specified for each frame, the feature amount of the color of the object is extracted and updated. When face detection becomes impossible, the target area is specified using the updated color feature of the object, and the registered feature amount of the color is updated.
- FIG. 4 is a diagram showing a configuration of the electronic camera according to the first embodiment of the present invention.
- reference numeral 100 denotes an electronic camera.
- Reference numeral 101 denotes a lens, 102 a shutter having a diaphragm function, 103 an image sensor that converts an optical image into an electrical signal, and 104 an AD converter (Analog Digital Converter) that converts an analog signal output from the image sensor 103 into a digital signal.
- AD converter Analog Digital Converter
- the timing generation circuit 105 supplies a clock signal and a control signal to the image sensor 103, the AD converter 104, and the DA converter (Digital Analog Converter) 106.
- the timing generation circuit 105 is controlled by the memory control circuit 107 and the system control circuit 108.
- the image processing circuit 109 performs predetermined image interpolation processing and color conversion processing on the data from the AD converter 104 or the data from the memory control circuit 107.
- the image processing circuit 109 performs predetermined calculation processing using the captured image data, and the system control circuit 108 controls the exposure control unit 149 and the distance measurement control unit 114 based on the obtained calculation result. I do.
- the memory control circuit 107 controls the AD converter 104, the timing generation circuit 105, the image processing circuit 109, the image display memory 110, the DA converter 106, the memory 111, and the resizing circuit 112.
- the data of the AD converter 104 is written into the image display memory 110 or the memory 111 via the image processing circuit 109 and the memory control circuit 107, or the data of the AD converter 104 is directly passed through the memory control circuit 107.
- 110 is an image display memory
- 106 is a DA converter
- 148 is a TFT (Thin Film Transistor) -LCD (Liquid Crystal Display), etc.
- image data for display written in the image display memory 110 is The image is displayed by the image display unit 148 via the DA converter 106.
- the resize circuit 112 generates a resized image that is a low resolution image based on the frame image of the captured moving image.
- the resizing circuit 112 is used when it is desired to record image data on a recording medium using a pixel number (size) different from the pixel number of the image sensor 103.
- the image display unit 148 since the image display unit 148 has a considerably smaller number of pixels than the image sensor 103, the image display unit 148 is also used to generate a display image when displaying on the image display unit 148.
- the resolution of the resized image is configured to be selected from a plurality of predetermined resolutions based on the application.
- the resizing circuit 112 reads an image stored in the memory 111, performs resizing processing, and writes the processed data in the memory 111.
- the exposure control unit 149 controls the shutter 102 having a diaphragm function, and has a flash light control function in cooperation with the flash 113.
- the ranging control unit 114 controls the focusing of the lens 101.
- the zoom control unit 115 controls zooming of the lens 101.
- the barrier control unit 116 controls the operation of the protection unit 117 that is a barrier.
- the flash 113 also has an AF auxiliary light projecting function and a flash light control function.
- the system control circuit 108 controls the entire image processing apparatus (electronic camera) 100, and the memory 118 records constants, variables, programs, and the like for operation of the system control circuit 108.
- the display unit 119 is a display unit such as a liquid crystal display device or a speaker that displays an operation state or a message using characters, images, sounds, or the like according to execution of a program in the system control circuit 108.
- One or a plurality of display units 119 are installed in a device that is easily visible near the operation unit of the image processing apparatus 100.
- the display unit 119 includes, for example, a combination of an LCD, a light emitting diode (LED), a sounding element, or the like.
- the non-volatile memory 120 is an electrically proof / recordable memory, and for example, an EEPROM or the like is used.
- the mode dial 121 can switch and set each function mode such as an automatic shooting mode, a shooting mode, a panoramic shooting mode, and a live mode.
- the shutter switch 123 is a shutter switch SW1 and is turned on during the operation of a shutter button (not shown) to instruct the start of operations such as AF processing, AE processing, AWB (Auto White Balance) processing, and the like.
- the shutter switch 123 is a shutter switch SW2 and is turned on when an operation of a shutter button (not shown) is completed, and an exposure process for writing a signal read from the image sensor 103 into the memory 111 via the AD converter 104 and the memory control circuit 107. Instructs the start of a series of processing operations such as development processing using computations in the image processing circuit 109 and the memory control circuit 107 and recording processing for reading image data from the memory 111 and writing the image data in the recording unit 124.
- the power control unit 125 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, detects whether or not a battery is installed, the type of battery, and the remaining battery level.
- the DC-DC converter is controlled based on an instruction from the system control circuit 108, and a necessary voltage is supplied to each part including the recording medium for a necessary period.
- the power supply unit 128 is a power supply including a connector 127, a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, and an AC adapter 126 or the like.
- Interfaces 129 and 130 are interfaces with recording media such as memory cards and hard disks, and connectors 131 and 132 connect recording media such as memory cards and hard disks with the main body.
- the protection unit 117 is a barrier that prevents the imaging unit from being stained or damaged by covering the imaging unit including the lens 101 of the image processing apparatus (electronic camera) 100.
- the optical finder 133 can perform photographing using only the optical finder without using the electronic finder function of the image display unit 148.
- the communication unit 134 has various communication functions such as RS232C, USB (Universal Serial Bus), IEEE 1394, modem, LAN, and wireless communication.
- the antenna 135 is a connector for connecting the image processing apparatus (electronic camera) 100 to another device by the communication unit 134 or an antenna in the case of wireless communication.
- the initial feature quantity extraction unit 136 extracts the initial feature quantity from the image data stored in the memory 111 and writes it in the memory 111.
- the coordinates for extracting the initial feature amount can be designated by the user on the touch panel or automatically set by the face detection position or the AF area set by pressing the shutter switch SW1.
- the tracking processing unit 137 reads the registered feature amount from the memory 111, performs tracking processing, and writes the tracking result (coordinate data, evaluation value) in the memory 111.
- the evaluation value is a value indicating the degree of similarity between the registered feature quantity and the target area feature quantity.
- the evaluation value is more probable, that is, the degree of coincidence is high will be described as an example.
- the tracking head frame designation circuit 138 determines whether or not it is the head frame to be tracked among the image data, and writes the determination result in the memory 111.
- the update area detection circuit 139 detects the update area and writes the update area detection result (coordinate data, evaluation value) in the memory 111.
- the evaluation value is a value indicating the degree of similarity between the registered feature quantity and the feature quantity of the update area. In the following, the evaluation value is more probable, that is, the degree of coincidence is high as an example. explain.
- the update determination circuit 140 determines whether or not to update the registered feature value stored in the memory 111 and writes the determination result in the memory 111.
- the end determination circuit 141 writes a determination result on whether or not to end the tracking in the memory 111.
- the position correction circuit 142 corrects the tracking result stored in the memory 111 based on the positional relationship information DB between the update area detection result and the tracking result held in the memory 111 in advance.
- the registered feature quantity update circuit 143 updates the registered feature quantity recorded in the memory 111 by using the position feature of the update area detection result when the tracking result and the update area detection result have a relationship described later.
- the tracking result drawing circuit 144 knows that the image data for display written in the image display memory 110 has been tracked at the position of the tracking result in order to display the tracking result written in the memory 111 on the LCD.
- Information for example, processing such as tracking frame, mosaic, character, display color change, blur, etc. is performed.
- the tracking result drawing circuit 144 displays the tracking result written in the memory 111 on the LCD, the position of the tracking result or the update region detection result for the display image data written in the image display memory 110. For example, processing such as tracking frame, mosaic, character, display color change, blur, etc. is performed.
- the camera control circuit 145 determines that the target object is a certain position / size of the image (for example, the face is centered or the whole body is displayed. Control the camera to zoom.
- the face detection circuit 146 detects a face from an image and writes a face detection result (position, size, evaluation value) in the memory 111.
- the face orientation recognition circuit 147 recognizes whether the face detected by the face detection circuit 146 is facing up, down, left, or right, and writes the recognition result in the memory 111.
- the system control circuit 108 performs tracking and update processing using a flow described later by software processing. May be.
- the system control circuit 108 performs exposure processing for writing the captured image data into the memory 111 via the image sensor 103, the AD converter 104, the image processing circuit 109, and the memory control circuit 107, and the memory control circuit 107 and as necessary.
- the image processing circuit 109 is used to execute a photographing process including a developing process for reading the image data written in the memory 111 and performing various processes.
- the system control circuit 108 When the photographing process is completed, the system control circuit 108 generates a display image for displaying the photographed image on the image display unit 148 using the resizing circuit 112 for the image data written in the memory 111. Similarly, the system control circuit 108 resizes the image to an image size (QVGA (Quarter
- QVGA Quadrater
- the system control circuit 108 performs face detection processing on the face detection image data stored in the memory 111 using the conventional technique by the face detection circuit 146, and stores the processing result in the memory 111.
- system control circuit 108 executes a face orientation recognition process using the conventional technique by the face orientation recognition circuit 147 and stores the processing result in the memory 111.
- the initial feature quantity extraction unit 136 refers to the face detection and face orientation recognition processing results stored in the memory 111, extracts the registered feature quantity, The registered feature value is written in 111.
- the initial feature quantity extraction unit 136 extracts a face color histogram as a color feature quantity based on the face detection result and face orientation information stored in the memory 111 and registers the face color histogram in the memory 111. Then, using the feature amount registered in the memory 111 by the initial feature amount extraction unit 136, the tracking processing unit 137 performs tracking processing.
- a color histogram of this area (reference area) is created. Specifically, on the horizontal axis of the histogram, the H value (maximum 360) of the HSV (Hue Saturation Value) color space of each pixel obtained from the reference region is assigned to the region divided into twenty. The number assigned to each section is the frequency. Which region is included can be calculated by Hi * 18/360, where Hi of each pixel. Then, the frequency of each section is normalized by being divided by the number of pixels in the reference area so that the total frequency becomes 1. Then, as shown in FIG.
- a region 301 that is expanded in a certain range from the detection region becomes a search region, and the search is performed while changing the size and position of the tracking window 302.
- the tracking processing unit 137 creates a color histogram 303 of the tracking window area while searching.
- FIG. 7A shows a reference image, and a color histogram calculated from the reference image is 401.
- FIG. 7B is an input image, and 402 shows a color histogram of the search window.
- the similarity that is, the evaluation value is calculated based on the overlapping degree of the respective color histograms.
- the similarity calculation is performed by (Equation 1).
- Ri is the frequency of the i-th segment in the color histogram of the reference area
- Ii is the frequency of the i-th segment in the color histogram of the tracking window area.
- i takes a value from 0 to 20.
- a value of 20 has been described, but any value may be used as long as the value is 1 or more.
- This value changes depending on the complexity of the target. When a plurality of colors are included, this value is increased, and the accuracy can be improved by seeing the difference in fine sections. On the other hand, for an object with a small number of colors, this value can be reduced and tracking can be performed using a small amount of memory by making a rough difference.
- the face detection position is used as the reference area designation method, this may be designated by the user using a touch panel or a pointer, or may be designated using audio information or the like.
- by performing color histogram correction on these templates as preprocessing it is possible to make them less susceptible to the influence of luminance.
- any method may be used as long as it is a method for obtaining the distance in general, such as a method for minimizing the least square distance or a method using normalized correlation.
- a search method a method of simply scanning an area has been described, but a search method based on a probability distribution such as a particle filter may be used.
- the face detection area 501 includes both skin color information and hair color information.
- the hair color information is acquired by setting the occipital region about 20% in the face detection region as the reference region 502 by the face detection process, the object is moved to the back. Even when facing, it is possible to track using color based on hair color information.
- the tracking can be stabilized by changing this ratio according to the case where it is easy to face the front or the case where it is easy to face the back.
- the hair has been described as a hair, but this method functions effectively even when wearing a hat or clothing that has little color variation.
- the update area detection circuit 139 sets a flag for determining update and writes it in the memory 111. Then, the face detection position is corrected by the position correction circuit 142.
- the registered feature value is extracted based on the face detection position, and the registered feature value is updated by the registered feature value update circuit 143 and written in the memory 111.
- the registered feature amount is extracted in the same manner as described above.
- the position correction circuit 142 corrects the position based on the face detection result and updates the tracking result in the memory 111, thereby enabling stable face tracking.
- face detection may detect only the inside of the face. Therefore, stable tracking is possible by multiplying the detection result coordinates by a constant so as to include the head.
- UI User Interface
- blurring the tracking frame display and tracking result position in the tracking result drawing circuit even in the backward facing state as well as AE / AF
- tracking processing is performed using the color feature amount, and a tracking evaluation value related to the color feature amount is calculated. If the evaluation value of the tracking result is also equal to or less than the threshold value, the tracking process ends (S611). If the evaluation value of the tracking result based on the color feature amount is higher than the threshold value, the color feature represented using the color histogram is extracted from the initial position in the target area, and the initial registered feature amount is updated (S612). Control shifts to display of a tracking frame and camera control (S609). If face detection is possible, a location where the initial registered feature value is updated is selected based on the result of face orientation detection, and the initial registered feature value is updated (S608). Then, the updated tracking result position, that is, the LCD display of the target area and camera control are performed. (S609) Thereafter, the process returns to the tracking process (S605) and the process is repeated.
- Emodiment 2 In the second embodiment, tracking using information that is invariant to rotation and color information, such as circular shape information, and tracking of the head (including not only the face facing the front but also the back facing) is possible.
- An electronic camera is described. That is, in the second embodiment, the position of the update region for updating the color feature is determined using circular information instead of the face orientation information.
- the tracking process will be described as the case where the similarity, that is, the evaluation value is calculated and tracked using color histogram matching as in the first embodiment.
- the initial position of the object is determined based on the face detection information and the instruction from the user (touch panel, pointer, etc.), and the initial feature value is registered.
- the second embodiment shows an example in which the update region detection circuit 139 detects the update region using the shape information.
- any invariant feature may be used as long as it is an invariant feature with respect to the appearance of the object.
- Edge detection is performed in the vicinity of the tracking result stored in the memory 111. Edge detection uses a Sobel operator or the like. Alternatively, there is a method of using a difference point by an inter-frame difference. Hough transform is performed on the point group (feature point group) obtained here, and the radius and center of the circle are obtained. The method for obtaining the radius and center of the circle can be obtained, for example, by a method described in the document “HeadFinder: Person Tracking Based on Difference between Frames, Baba et al.” (Non-Patent Document: 1). The technique described in the above document will be briefly described.
- the circle can be determined by using the constants of the central coordinates a and b and the radius r using (Equation 2).
- the number of unknowns to be calculated is 3, so the voting space becomes three-dimensional. Since this calculation amount is enormous, real-time operation is difficult. Therefore, the size is limited to a certain radius r. Then, the radius parameters a and b are obtained by voting using the feature point group included in a plurality of radii having widths. Thereby, the circular shape of the head is estimated. At this time, how many feature point groups are included on the circle is an evaluation value representing the degree of circular fitting. The circular position, radius information and evaluation value are written in the memory 111.
- the head has a low evaluation value but can be detected stably with respect to the direction change, and the color information has a lower evaluation value when the direction changes.
- the evaluation value is lowered by color information due to the change in appearance, the position is corrected at the detection position of the head circle shape, and the registered feature value is corrected, so that stable tracking with respect to the change in appearance is achieved. The method is described.
- the update determination circuit 140 reads the evaluation value of the tracking result by color and the evaluation value of the update area detection result from the memory 111, and determines whether to update when the evaluation value of the tracking result falls below the threshold th1. And the determination result is written in the memory 111.
- the relationship between the evaluation value and the time axis at this time is shown in FIG.
- FIG. 10 (a) is updated when the evaluation value of the update area detection result exceeds the evaluation value of the tracking result.
- 10A, 10B, and 10C the solid line represents the evaluation value of the tracking result, and the dotted line represents the evaluation value of the update area detection result.
- the appearance of the target object changes and the evaluation value decreases. This is a phenomenon that occurs and can be updated so that it can be recovered when a tracking failure occurs.
- FIG. 10B shows the case where the update is performed when the evaluation value of the update area detection result becomes larger than the threshold value th1.
- This case is when the stability in the update area detection result is guaranteed.
- the position of the tracking result is corrected to the position of the update area detection result.
- the system shown in FIG. 10B can be used when the update area detection result can be sufficiently tracked, and either can be successfully selected for objects that are not, so that a stable system can be constructed.
- the change in the color evaluation value varies depending on the registered color and the complexity of the color of the object. For example, when the average saturation is high, the evaluation value for tracking by color is high, and the evaluation value tends to decrease slowly as the appearance changes.
- the updating method is described based on the relationship between the evaluation values.
- the updating is performed using the distance relationship between the tracking result position by color, that is, the target area, and the circle detection result position, that is, the updating area.
- the distance is not limited to a multiple of a circle, and any distance such as a Euclidean distance may be used. In this case, even if the evaluation value of tracking by the color feature is high and the evaluation value by circle detection is also high, there is a high possibility that the tracking by the color feature is not an object but an erroneous tracking.
- the position correction circuit 142 replaces the coordinates and size of the target area of the tracking result with the coordinates and size of the update area detection result, and stores them in the memory 111. At this time, since the center and radius of the circle are stored in the update area detection result, the value obtained by subtracting the radius from the center so as to obtain the upper left coordinate and the height and width of the rectangular frame is the upper left coordinate, and is twice the radius. Is the height and width of the rectangular frame.
- the tracking end determination circuit 141 stores the determination result for ending tracking when the evaluation value of the tracking result and the evaluation value of the update region detection result are below the threshold th2. 111.
- the tracking result drawing circuit not only performs UI control such as display of the tracking frame and blurring of the tracking result position, but also control of AE / AF, etc.
- UI control such as display of the tracking frame and blurring of the tracking result position
- AE / AF etc.
- Face detection is performed on the photographed input image (S801).
- An initial registered feature is extracted for the face detection result (S803).
- the tracking process is performed using the color feature registered in step S803, and the target area is specified.
- circle detection is performed using a circle that is a shape feature, and an update area is specified (S805).
- Whether or not to end the tracking process is determined based on the evaluation value of the tracking process based on the color feature and the evaluation value of the circle detection using the circle that is the shape characteristic (S806).
- step 806 when the evaluation value of the tracking process based on the color feature amount and the evaluation value based on the circle detection are both below the threshold th2, it is determined that the tracking is to be ended. If it is determined that the tracking process is to be ended, the tracking process is ended and the display of the tracking frame is ended, or the end of the process is presented to the user by an alarm or the like. Thereafter, a display prompting the user to set the initial position again may be displayed on the display, or the initial position setting may be performed by face detection and automatic return may be performed (S810).
- the threshold value th1 is used as shown in FIG. 10B, that is, if the evaluation value by the circle detection exceeds the threshold value th1 regardless of the evaluation value of the tracking processing by the color feature, the color feature Or when the evaluation value of the update region detection result by circle detection exceeds the evaluation value of the tracking feature using the color feature as shown in FIG. It is determined to update (S807).
- the position / feature amount of the tracking result is updated using the position / feature amount of the update region detection result (S808).
- the LCD display of the updated tracking result position and camera control are performed (S809). Thereafter, the process returns to the tracking process in step S804, and the process is repeated in steps S804 to S810.
- the tracking processing method and the similarity calculation method are not limited to the color histogram matching method, but may be a method based on the least square distance or a method based on normalized correlation.
- the search method may be a method based on a particle filter instead of simple scanning.
- the color feature is a tracking feature
- the shape feature is an update region detection feature.
- the color feature may be the update region detection result feature and the shape feature may be the tracking feature.
- the face detection position is used as the initial position, but the AF position may be used as the initial position.
- each functional block in the block diagram (such as FIG. 4) is typically realized as an LSI which is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them. For example, the functional blocks other than the memory may be integrated into one chip.
- LSI is used, but it may be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.
- the method of circuit integration is not limited to LSI, and implementation using a dedicated circuit or a general-purpose processor is also possible.
- An FPGA Field Programmable Gate Array
- a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.
- only the means for storing the data to be encoded or decoded may be configured separately instead of being integrated into one chip.
- the electronic camera and the image processing method according to the present invention have a target tracking function that changes the appearance of an object.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Optics & Photonics (AREA)
- Studio Devices (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Description
101 レンズ
102 シャッター
103 撮像素子
104 AD変換器
105 タイミング発生回路
106 DA変換器
107 メモリ制御回路
108 システム制御回路
109 画像処理回路
110 画像表示メモリ
111 メモリ
112 リサイズ回路
113 フラッシュ
114 測距制御部
115 ズーム制御部
116 バリア制御部
117 保護部
118 メモリ
119 表示部
120 不揮発性メモリ
121 モードダイアル
123 シャッタースイッチ
124 記録部
125 電源制御部
126 電源部
127 コネクタ
128 電源部
129 インタフェース
130 インタフェース
131 コネクタ
132 コネクタ
133 光学ファインダ
134 通信部
135 アンテナ
136 初期特徴量抽出部
137 追跡処理部
138 追跡先頭フレーム指定回路
139 更新領域検知回路
140 更新判定回路
141 終了判定回路
142 位置補正回路
143 登録特徴量更新回路
144 追跡結果描画回路
145 カメラ制御回路
146 顔検出回路
147 顔向き認識回路
148 画像表示部
149 露光制御部
200 フレーム(a)での追尾結果
201 フレーム(b)での追尾結果
202 フレーム(b)での正解位置
301 探索領域
302 追跡ウィンドウ
303 追跡ウィンドウ領域の色ヒストグラム
401 リファレンス画像の色ヒストグラム
402 入力画像の色ヒストグラム
501 顔検出領域
502 リファレンス領域
S601 入力画像ステップ
S602 顔検出ステップ
S603 顔向き認識ステップ
S604 初期登録特徴抽出ステップ
S605 追跡処理ステップ
S606 顔検出ステップ
S607 顔検出判定ステップ
S608 初期登録特徴更新ステップ
S609 追尾枠表示部・カメラ制御ステップ
S610 追跡評価値比較ステップ
S611 追跡終了ステップ
S612 初期登録特徴量更新ステップ
S801 入力画ステップ
S802 顔検出ステップ
S803 初期登録特徴抽出ステップ
S804 追跡処理ステップ
S805 円検出ステップ
S806 追跡終了判定ステップ
S807 特徴量更新判定ステップ
S808 特徴量更新ステップ
S809 追尾枠表示部・カメラ制御ステップ
S810 追跡終了ステップ
900 部分画像切り出し部
901 特徴量評価部1
902 特徴量評価部2
903 特徴量評価部3
1000 部分画像
1001 入力画像
本実施の形態1では、顔検出と色情報を利用した追跡を併用することによって、顔検出だけでは不可能な、後ろを向いた頭部の追跡も可能とする画像処理装置について述べる。すなわち、実施の形態1の画像処理装置(電子カメラ)は、従来の顔検出および顔向き認識を用いて、追尾撮影の対象となる対象物が存在するであろうターゲット領域を特定する。そして、毎フレーム、ターゲット領域が特定される都度、対象物の色の特徴量を抽出して更新する。顔検出ができなくなった場合には、更新された対象物の色特徴を用いてターゲット領域を特定し、色の登録特徴量を更新する。
実施の形態2では、円形状情報のように回転に不変な情報と色情報とを利用した追跡、頭部(正面向きの顔だけでなく顔が見えない後ろ向きの場合も含む)の追跡も可能とする電子カメラについて述べる。すなわち、実施の形態2では、顔向き情報の代わりに円形状の情報を用いて、色特徴を更新するための更新領域の位置を決定する。
Claims (19)
- 連続して撮影する画像の各フレーム内において追尾の対象となる対象物が写っている対象領域を特定して表示する機能を持つ電子カメラであって、
前記対象物の特徴を定量的に示すあらかじめ登録された第一の特徴量を用いて、フレーム内のあらかじめ定められた範囲内を探索し、前記探索結果として得られた領域内の画像に対して前記対象物の画像との一致度を示す第一の評価値を算出し、算出した前記第一の評価値に基づいて、前記対象物の画像が存在すると推定されるターゲット領域を特定する追跡処理部と、
前記第一の特徴量とは異なる特徴量であり、かつ、前記対象物の特徴を定量的に示す第二の特徴量を用いて、前記フレーム内のあらかじめ定められた範囲内を探索し、前記探索結果として得られた領域内の画像に対して前記対象物の画像との一致度を示す第二の評価値を算出し、算出した前記第二の評価値に基づいて、前記第一の特徴量を更新するための更新領域を前記フレーム内において特定する更新領域算出部と、
前記追跡処理部において算出された前記第一の評価値と、前記更新領域算出部において算出された前記第二の評価値とのうち少なくとも一つが、あらかじめ定められた条件を満足するか否かを調べることにより、前記第一の特徴量を更新するか否かを判定する更新判定部と、
前記更新判定部によって前記第一の特徴量を更新すると判定された場合、前記更新領域内において抽出された新たな第一の特徴量により、前記第一の特徴量を更新する登録特徴量更新部と、
前記更新判定部によって前記第一の特徴量を更新しないと判定された場合、前記追跡処理部において特定された前記ターゲット領域を前記対象領域と特定して前記ターゲット領域に関する情報を描画し、前記更新判定部によって前記第一の特徴量を更新すると判定された場合、前記更新領域算出部において特定された前記更新領域を前記対象領域と特定して前記更新領域に関する情報を描画する追跡結果描画部とを備え、
前記追跡処理部は、あらかじめ登録された前記第一の特徴量が更新された場合、更新後の第一の特徴量を用いて、新たなフレーム内の新たなターゲット領域を特定する
電子カメラ。 - 前記追跡処理部は、前記第一の特徴量として、対象物の色情報を用いて、前記対象物の画像が存在すると推定されるターゲット領域を特定し、
前記更新領域算出部は、前記第二の特徴量として、対象物の形状情報を用いて前記更新領域を特定する
請求項1記載の電子カメラ。 - 前記更新判定部は、前記更新領域算出部において算出された前記第二の評価値が、前記追跡処理部において算出された前記第一の評価値よりも大きいとき、または、前記更新領域算出部において算出された前記第二の評価値があらかじめ定められた第一の閾値より大きいときに前記第一の特徴量を更新すると判定する
請求項1記載の電子カメラ。 - 前記更新判定部は、前記対象物の画像の画素ごとの平均彩度が高いほど前記第一の閾値に大きな値を設定し、前記対象の平均彩度が低いほど前記第一の閾値に小さな値を設定する
請求項3記載の電子カメラ。 - 前記更新判定部は、さらに、前記追跡処理部において特定された前記ターゲット領域と前記更新領域算出部において特定された前記更新領域との距離があらかじめ定められた第二の閾値以上になったときに更新すると判定する
請求項1記載の電子カメラ。 - 前記電子カメラは、さらに、
前記対象物の追跡が継続不可能か否かを判定する終了判定部を備え、
前記終了判定部は、前記第一の評価値と前記第二の評価値との両方が、あらかじめ定められた第三の閾値より小さいときに前記対象の追跡が継続不可能と判定する
請求項1記載の電子カメラ。 - 前記追跡結果描画部は、前記対象の追跡が継続不可能と判定されたときに、ターゲット領域および更新領域の描画を終了する
請求項6記載の電子カメラ。 - 前記追跡結果描画部は、前記対象の追跡が継続不可能と判定されたときに、追跡不能となったことをユーザに提示する画像を描画する
請求項6記載の電子カメラ。 - 前記追跡結果描画部は、前記対象の追跡が継続不可能と判定されたときに、ユーザに対して再度初期位置設定を促す画像を描画する
請求項6記載の電子カメラ。 - 前記追跡結果描画部は、前記対象の追跡が継続不可能と判定されたときに、新たな顔検出により初期位置設定を行う
請求項6記載の電子カメラ。 - 前記電子カメラは、さらに、
各フレーム内の画像内において顔を検出する顔検出回路と、
前記顔検出回路によって検出された顔の向きを認識する顔向き認識回路とを備え、
前記更新領域算出部は、前記顔向き認識回路によって前記対象物である顔が横を向いていると認識されたとき、真横を向いている顔の領域である更新基準領域を前記更新領域算出部において算出した後、顔部分と髪の毛部分とのあらかじめ指定された位置関係に基づき、前記更新基準領域内における前記更新領域を算出する
請求項1記載の電子カメラ。 - 前記電子カメラは、さらに、前記追跡処理部において特定されたターゲット領域、および、前記更新領域算出部において特定された前記更新領域のいずれかに基づいて、前記電子カメラの動作を調整するためのカメラパラメータを変更するカメラ制御部を有する
請求項1記載の電子カメラ。 - 前記カメラ制御部は、前記カメラパラメータに基づいて、前記電子カメラの筐体および雲台の少なくとも一方の動作を制御し、特定された前記対象物の全体、または一部をフレーム内の所定の位置、およびサイズに合わせるように制御を行う
請求項12記載の電子カメラ。 - 前記電子カメラは、さらに、前記ターゲット領域の初期位置を、ユーザからの入力に基づいて、または、あらかじめ定められた方法を用いて決定するターゲット領域初期設定部を有する
請求項1記載の電子カメラ。 - 前記ターゲット領域初期設定部は、人物、もしくは、顔のいずれか一方の検出位置を初期位置と決定する
請求項14記載の電子カメラ。 - 前記ターゲット領域初期設定部は、AF(オートフォーカス)機能により焦点を合わせた場所を初期位置と決定する
請求項14記載の電子カメラ。 - 連続して撮影する画像の各フレーム内において追尾の対象となる対象物が写っている対象領域を特定して表示する機能を持つ電子カメラにおける画像処理方法であって、
追跡処理部が、前記対象物の特徴を定量的に示すあらかじめ登録された第一の特徴量を用いて、フレーム内のあらかじめ定められた範囲内を探索し、前記探索結果として得られた領域内の画像に対して前記対象物の画像との一致度を示す第一の評価値を算出し、算出した前記第一の評価値に基づいて、前記対象物の画像が存在すると推定されるターゲット領域を特定し、
更新領域算出部が、前記第一の特徴量とは異なる特徴量であり、かつ、前記対象物の特徴を定量的に示す第二の特徴量を用いて、前記フレーム内のあらかじめ定められた範囲内を探索し、前記探索結果として得られた領域内の画像に対して前記対象物の画像との一致度を示す第二の評価値を算出し、算出した前記第二の評価値に基づいて、前記第一の特徴量を更新するための更新領域を前記フレーム内において特定し、
更新判定部が、前記追跡処理部において算出された前記第一の評価値と、前記更新領域算出部において算出された前記第二の評価値とのうち少なくとも一つが、あらかじめ定められた条件を満足するか否かを調べることにより、前記第一の特徴量を更新するか否かを判定し、
登録特徴量更新部が、前記更新判定部によって前記第一の特徴量を更新すると判定された場合、前記更新領域内において抽出された新たな第一の特徴量により、前記第一の特徴量を更新し、
追跡結果描画部が、前記更新判定部によって前記第一の特徴量を更新しないと判定された場合、前記追跡処理部において特定された前記ターゲット領域を前記対象領域と特定して前記ターゲット領域に関する情報を描画し、前記更新判定部によって前記第一の特徴量を更新すると判定された場合、前記更新領域算出部において特定された前記更新領域を前記対象領域と特定して前記更新領域に関する情報を描画し、
前記追跡処理部は、あらかじめ登録された前記第一の特徴量が更新された場合、更新後の第一の特徴量を用いて、新たなフレーム内の新たなターゲット領域を特定する
画像処理方法。 - 連続して撮影する画像の各フレーム内において追尾の対象となる対象物が写っている対象領域を特定して表示する機能を持つ集積回路であって、
前記対象物の特徴を定量的に示すあらかじめ登録された第一の特徴量を用いて、フレーム内のあらかじめ定められた範囲内を探索し、前記探索結果として得られた領域内の画像に対して前記対象物の画像との一致度を示す第一の評価値を算出し、算出した前記第一の評価値に基づいて、前記対象物の画像が存在すると推定されるターゲット領域を特定する追跡処理部と、
前記第一の特徴量とは異なる特徴量であり、かつ、前記対象物の特徴を定量的に示す第二の特徴量を用いて、前記フレーム内のあらかじめ定められた範囲内を探索し、前記探索結果として得られた領域内の画像に対して前記対象物の画像との一致度を示す第二の評価値を算出し、算出した前記第二の評価値に基づいて、前記第一の特徴量を更新するための更新領域を前記フレーム内において特定する更新領域算出部と、
前記追跡処理部において算出された前記第一の評価値と、前記更新領域算出部において算出された前記第二の評価値とのうち少なくとも一つが、あらかじめ定められた条件を満足するか否かを調べることにより、前記第一の特徴量を更新するか否かを判定する更新判定部と、
前記更新判定部によって前記第一の特徴量を更新すると判定された場合、前記更新領域内において抽出された新たな第一の特徴量により、前記第一の特徴量を更新する登録特徴量更新部と、
前記更新判定部によって前記第一の特徴量を更新しないと判定された場合、前記追跡処理部において特定された前記ターゲット領域を前記対象領域と特定して前記ターゲット領域に関する情報を描画し、前記更新判定部によって前記第一の特徴量を更新すると判定された場合、前記更新領域算出部において特定された前記更新領域を前記対象領域と特定して前記更新領域に関する情報を描画する追跡結果描画部とを備え、
前記追跡処理部は、あらかじめ登録された前記第一の特徴量が更新された場合、更新後の第一の特徴量を用いて、新たなフレーム内の新たなターゲット領域を特定する
集積回路。 - 連続して撮影する画像の各フレーム内において追尾の対象となる対象物が写っている対象領域を特定して表示する機能を持つ画像処理プログラムが記録されたコンピュータ読み取り可能な記録媒体であって、
追跡処理部が、前記対象物の特徴を定量的に示すあらかじめ登録された第一の特徴量を用いて、フレーム内のあらかじめ定められた範囲内を探索し、前記探索結果として得られた領域内の画像に対して前記対象物の画像との一致度を示す第一の評価値を算出し、算出した前記第一の評価値に基づいて、前記対象物の画像が存在すると推定されるターゲット領域を特定し、
更新領域算出部が、前記第一の特徴量とは異なる特徴量であり、かつ、前記対象物の特徴を定量的に示す第二の特徴量を用いて、前記フレーム内のあらかじめ定められた範囲内を探索し、前記探索結果として得られた領域内の画像に対して前記対象物の画像との一致度を示す第二の評価値を算出し、算出した前記第二の評価値に基づいて、前記第一の特徴量を更新するための更新領域を前記フレーム内において特定し、
更新判定部が、前記追跡処理部において算出された前記第一の評価値と、前記更新領域算出部において算出された前記第二の評価値とのうち少なくとも一つが、あらかじめ定められた条件を満足するか否かを調べることにより、前記第一の特徴量を更新するか否かを判定し、
登録特徴量更新部が、前記更新判定部によって前記第一の特徴量を更新すると判定された場合、前記更新領域内において抽出された新たな第一の特徴量により、前記第一の特徴量を更新し、
追跡結果描画部が、前記更新判定部によって前記第一の特徴量を更新しないと判定された場合、前記追跡処理部において特定された前記ターゲット領域を前記対象領域と特定して前記ターゲット領域に関する情報を描画し、前記更新判定部によって前記第一の特徴量を更新すると判定された場合、前記更新領域算出部において特定された前記更新領域を前記対象領域と特定して前記更新領域に関する情報を描画し、
前記追跡処理部は、あらかじめ登録された前記第一の特徴量が更新された場合、更新後の第一の特徴量を用いて、新たなフレーム内の新たなターゲット領域を特定する
プログラムを記録した記録媒体。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP09708197.0A EP2242253B1 (en) | 2008-02-06 | 2009-02-06 | Electronic camera and image processing method |
JP2009552414A JP5090474B2 (ja) | 2008-02-06 | 2009-02-06 | 電子カメラおよび画像処理方法 |
CN2009801044907A CN101939980B (zh) | 2008-02-06 | 2009-02-06 | 电子摄像机和图像处理方法 |
US12/865,917 US8253819B2 (en) | 2008-02-06 | 2009-02-06 | Electronic camera and image processing method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2008-026160 | 2008-02-06 | ||
JP2008026160 | 2008-02-06 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2009098894A1 true WO2009098894A1 (ja) | 2009-08-13 |
Family
ID=40951973
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2009/000477 WO2009098894A1 (ja) | 2008-02-06 | 2009-02-06 | 電子カメラおよび画像処理方法 |
Country Status (5)
Country | Link |
---|---|
US (1) | US8253819B2 (ja) |
EP (1) | EP2242253B1 (ja) |
JP (1) | JP5090474B2 (ja) |
CN (1) | CN101939980B (ja) |
WO (1) | WO2009098894A1 (ja) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110033086A1 (en) * | 2009-08-06 | 2011-02-10 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
JP2011053959A (ja) * | 2009-09-02 | 2011-03-17 | Canon Inc | 画像処理装置、被写体判別方法、プログラム及び記憶媒体 |
CN102055908A (zh) * | 2009-11-10 | 2011-05-11 | 奥林巴斯映像株式会社 | 图像拍摄装置以及图像拍摄方法 |
CN102521580A (zh) * | 2011-12-21 | 2012-06-27 | 华平信息技术(南昌)有限公司 | 实时目标匹配跟踪方法及*** |
JP2013196199A (ja) * | 2012-03-16 | 2013-09-30 | Fujitsu Ltd | 利用者検知装置、方法及びプログラム |
KR101455018B1 (ko) * | 2011-02-03 | 2014-10-27 | 가부시키가이샤 리코 | 촬상 장치, 촬상 방법, 및 컴퓨터 판독 가능한 기록 매체 |
JP2015041383A (ja) * | 2013-08-23 | 2015-03-02 | 株式会社リコー | 対象追跡方法及び対象追跡装置 |
JP2016057409A (ja) * | 2014-09-08 | 2016-04-21 | キヤノン株式会社 | 撮像装置および撮像装置の制御方法 |
JP2021132362A (ja) * | 2020-02-19 | 2021-09-09 | キヤノン株式会社 | 被写体追尾装置、被写体追尾方法、コンピュータプログラム及び記憶媒体 |
JP2021184564A (ja) * | 2020-05-22 | 2021-12-02 | パナソニックIpマネジメント株式会社 | 画像追尾装置 |
US11553136B2 (en) | 2020-02-19 | 2023-01-10 | Canon Kabushiki Kaisha | Subject tracking device, subject tracking method, and storage medium |
Families Citing this family (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4869270B2 (ja) * | 2008-03-10 | 2012-02-08 | 三洋電機株式会社 | 撮像装置及び画像再生装置 |
US20110102570A1 (en) * | 2008-04-14 | 2011-05-05 | Saar Wilf | Vision based pointing device emulation |
WO2010044214A1 (ja) * | 2008-10-14 | 2010-04-22 | パナソニック株式会社 | 顔認識装置および顔認識方法 |
KR101051365B1 (ko) * | 2009-07-16 | 2011-07-22 | 주식회사 유니온커뮤니티 | 카메라를 구비한 출입관리장치 및 그 출입관리방법 |
GB2474536B (en) | 2009-10-13 | 2011-11-02 | Pointgrab Ltd | Computer vision gesture based control of a device |
JP5427577B2 (ja) * | 2009-12-04 | 2014-02-26 | パナソニック株式会社 | 表示制御装置及び表示画像形成方法 |
KR101652261B1 (ko) * | 2009-12-24 | 2016-08-30 | 삼성전자주식회사 | 카메라를 이용한 오브젝트 검출 방법 |
US8705813B2 (en) * | 2010-06-21 | 2014-04-22 | Canon Kabushiki Kaisha | Identification device, identification method, and storage medium |
CN102075686B (zh) * | 2011-02-10 | 2013-10-30 | 北京航空航天大学 | 一种鲁棒的实时在线摄像机跟踪方法 |
US8766153B2 (en) * | 2011-02-17 | 2014-07-01 | Mitutoyo Corporation | Vision measuring device and auto-focusing control method |
KR20120118383A (ko) * | 2011-04-18 | 2012-10-26 | 삼성전자주식회사 | 이미지 보정 장치 및 이를 이용하는 이미지 처리 장치와 그 방법들 |
CN103004179B (zh) * | 2011-06-29 | 2015-11-25 | 奥林巴斯映像株式会社 | 跟踪装置和跟踪方法 |
JP5888890B2 (ja) * | 2011-07-21 | 2016-03-22 | オリンパス株式会社 | フォーカス制御装置 |
JP6034197B2 (ja) * | 2011-08-25 | 2016-11-30 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | 画像処理装置、3次元撮像装置、画像処理方法、および画像処理プログラム |
CN103299613B (zh) * | 2011-11-17 | 2017-08-15 | 松下电器(美国)知识产权公司 | 图像处理装置、摄像装置及图像处理方法 |
KR101989152B1 (ko) * | 2012-02-10 | 2019-09-30 | 삼성전자주식회사 | 동영상 촬영 또는 재생 중, 정지 영상을 캡쳐하는 장치 및 방법 |
US9049364B2 (en) * | 2012-02-13 | 2015-06-02 | Htc Corporation | Focus adjusting method and image capture device thereof |
US9256781B2 (en) * | 2012-05-10 | 2016-02-09 | Pointguard Ltd. | System and method for computer vision based tracking of an object |
US8938124B2 (en) * | 2012-05-10 | 2015-01-20 | Pointgrab Ltd. | Computer vision based tracking of a hand |
US8441548B1 (en) * | 2012-06-15 | 2013-05-14 | Google Inc. | Facial image quality assessment |
CN103516972B (zh) * | 2012-06-19 | 2017-03-29 | 联想(北京)有限公司 | 摄像头控制方法及电子设备 |
JP2014116716A (ja) * | 2012-12-07 | 2014-06-26 | Samsung Display Co Ltd | 追尾装置 |
JP2014139734A (ja) * | 2013-01-21 | 2014-07-31 | Sony Corp | 情報処理装置および方法、並びにプログラム |
JP2014207659A (ja) * | 2013-03-19 | 2014-10-30 | パナソニック株式会社 | 画像処理装置及び画像処理方法及び撮像装置 |
KR101316805B1 (ko) | 2013-05-22 | 2013-10-11 | 주식회사 파이브지티 | 자동 얼굴 위치 추적 및 얼굴 인식 방법 및 시스템 |
US10474921B2 (en) * | 2013-06-14 | 2019-11-12 | Qualcomm Incorporated | Tracker assisted image capture |
WO2014203026A1 (en) * | 2013-06-17 | 2014-12-24 | Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirketi | A method for object tracking |
JP6268960B2 (ja) * | 2013-11-15 | 2018-01-31 | オムロン株式会社 | 画像認識装置及び画像認識装置に対するデータ登録方法 |
US9437000B2 (en) * | 2014-02-20 | 2016-09-06 | Google Inc. | Odometry feature matching |
CN105518702B (zh) * | 2014-11-12 | 2018-06-26 | 深圳市大疆创新科技有限公司 | 一种对目标物体的检测方法、检测装置以及机器人 |
JP6521626B2 (ja) | 2014-12-25 | 2019-05-29 | キヤノン株式会社 | 被写体追跡装置、方法およびプログラム |
US9934443B2 (en) * | 2015-03-31 | 2018-04-03 | Daon Holdings Limited | Methods and systems for detecting head motion during an authentication transaction |
EP3376470B1 (en) * | 2015-11-13 | 2021-01-13 | Panasonic Intellectual Property Management Co., Ltd. | Moving body tracking method, moving body tracking device, and program |
TWI585394B (zh) * | 2015-12-09 | 2017-06-01 | 由田新技股份有限公司 | 動態式自動追焦系統 |
JP6656987B2 (ja) * | 2016-03-30 | 2020-03-04 | 株式会社エクォス・リサーチ | 画像認識装置、移動体装置、及び画像認識プログラム |
CN106934395B (zh) * | 2017-02-15 | 2020-06-30 | 国家计算机网络与信息安全管理中心 | 一种采用surf特征和颜色特征相融合的刚体目标跟踪方法 |
CN107181976B (zh) * | 2017-04-28 | 2021-01-29 | 华为技术有限公司 | 一种弹幕显示方法及电子设备 |
CN110059521B (zh) * | 2018-01-18 | 2022-05-13 | 浙江宇视科技有限公司 | 目标跟踪方法及装置 |
US20190244030A1 (en) * | 2018-02-07 | 2019-08-08 | Hitachi, Ltd. | Object tracking in video using better object area |
US11164015B2 (en) * | 2018-05-08 | 2021-11-02 | Ford Global Technologies, Llc | Simultaneous diagnosis and shape estimation from a perceptual system derived from range sensors |
JP6779474B2 (ja) * | 2019-01-16 | 2020-11-04 | みこらった株式会社 | 浮揚移動台 |
CN110517614B (zh) * | 2019-08-28 | 2022-11-22 | 苏州精速智能科技有限公司 | 一种液晶模组导电粒子不良的检测方法 |
US11308923B2 (en) * | 2019-10-09 | 2022-04-19 | Motorola Mobility Llc | Electronic device with face orienting user interface |
US11076111B1 (en) * | 2019-11-13 | 2021-07-27 | Twitch Interactive, Inc. | Smart color-based background replacement |
CN111062965B (zh) * | 2019-12-26 | 2023-01-17 | 西华大学 | 一种基于流水线的低复杂度双门限多分辨率嘴部检测方法 |
US11004212B1 (en) * | 2020-01-02 | 2021-05-11 | Hong Kong Applied Science and Technology Research Institute Company Limited | Object tracking method and system using iterative template matching |
CN115222817B (zh) * | 2022-08-31 | 2022-11-29 | 成都千嘉科技股份有限公司 | 一种指针式压力表的指针定位方法 |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10304348A (ja) * | 1997-05-01 | 1998-11-13 | Atr Chinou Eizo Tsushin Kenkyusho:Kk | 動物体追跡方法 |
JPH11167455A (ja) * | 1997-12-05 | 1999-06-22 | Fujitsu Ltd | 手形状認識装置及び単色物体形状認識装置 |
JP2004362210A (ja) * | 2003-06-04 | 2004-12-24 | Nippon Telegr & Teleph Corp <Ntt> | 物体追跡装置と方法、及びそのプログラムとそのプログラムを記録した記録媒体 |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH056434A (ja) * | 1991-06-27 | 1993-01-14 | Matsushita Electric Ind Co Ltd | 自動追尾装置 |
US6687386B1 (en) * | 1999-06-15 | 2004-02-03 | Hitachi Denshi Kabushiki Kaisha | Object tracking method and object tracking apparatus |
US6859552B2 (en) * | 2000-11-07 | 2005-02-22 | Minolta Co., Ltd. | Image retrieving apparatus |
JP2002157599A (ja) | 2000-11-17 | 2002-05-31 | Nippon Telegr & Teleph Corp <Ntt> | 物体検出認識方法,物体検出認識プログラムを記録した記録媒体および物体監視追跡装置 |
US7099510B2 (en) * | 2000-11-29 | 2006-08-29 | Hewlett-Packard Development Company, L.P. | Method and system for object detection in digital images |
US7298412B2 (en) * | 2001-09-18 | 2007-11-20 | Ricoh Company, Limited | Image pickup device, automatic focusing method, automatic exposure method, electronic flash control method and computer program |
KR100459893B1 (ko) * | 2002-01-08 | 2004-12-04 | 삼성전자주식회사 | 동영상에서 칼라 기반의 객체를 추적하는 방법 및 그 장치 |
JP4140567B2 (ja) * | 2004-07-14 | 2008-08-27 | 松下電器産業株式会社 | 物体追跡装置および物体追跡方法 |
US7315631B1 (en) * | 2006-08-11 | 2008-01-01 | Fotonation Vision Limited | Real-time face tracking in a digital image acquisition device |
JP4824411B2 (ja) * | 2005-01-20 | 2011-11-30 | パナソニック株式会社 | 顔抽出装置、半導体集積回路 |
ATE546800T1 (de) * | 2005-07-05 | 2012-03-15 | Omron Tateisi Electronics Co | Tracking-vorrichtung |
JP2007072520A (ja) * | 2005-09-02 | 2007-03-22 | Sony Corp | 映像処理装置 |
JP4657934B2 (ja) * | 2006-01-23 | 2011-03-23 | 富士フイルム株式会社 | 顔検出方法および装置並びにプログラム |
GB2452512B (en) * | 2007-09-05 | 2012-02-29 | Sony Corp | Apparatus and method of object tracking |
TW200929005A (en) * | 2007-12-26 | 2009-07-01 | Altek Corp | Human face detection and tracking method |
JP4730431B2 (ja) * | 2008-12-16 | 2011-07-20 | 日本ビクター株式会社 | 目標追尾装置 |
JP5383366B2 (ja) * | 2009-07-28 | 2014-01-08 | キヤノン株式会社 | 画像処理装置、画像処理方法、および、そのプログラム |
JP5662670B2 (ja) * | 2009-10-27 | 2015-02-04 | キヤノン株式会社 | 画像処理装置、画像処理方法、及びプログラム |
-
2009
- 2009-02-06 EP EP09708197.0A patent/EP2242253B1/en active Active
- 2009-02-06 US US12/865,917 patent/US8253819B2/en active Active
- 2009-02-06 CN CN2009801044907A patent/CN101939980B/zh active Active
- 2009-02-06 WO PCT/JP2009/000477 patent/WO2009098894A1/ja active Application Filing
- 2009-02-06 JP JP2009552414A patent/JP5090474B2/ja active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10304348A (ja) * | 1997-05-01 | 1998-11-13 | Atr Chinou Eizo Tsushin Kenkyusho:Kk | 動物体追跡方法 |
JPH11167455A (ja) * | 1997-12-05 | 1999-06-22 | Fujitsu Ltd | 手形状認識装置及び単色物体形状認識装置 |
JP2004362210A (ja) * | 2003-06-04 | 2004-12-24 | Nippon Telegr & Teleph Corp <Ntt> | 物体追跡装置と方法、及びそのプログラムとそのプログラムを記録した記録媒体 |
Non-Patent Citations (1)
Title |
---|
See also references of EP2242253A4 * |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8538075B2 (en) * | 2009-08-06 | 2013-09-17 | Canon Kabushiki Kaisha | Classifying pixels for target tracking, apparatus and method |
US20110033086A1 (en) * | 2009-08-06 | 2011-02-10 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
JP2011053959A (ja) * | 2009-09-02 | 2011-03-17 | Canon Inc | 画像処理装置、被写体判別方法、プログラム及び記憶媒体 |
CN102055908A (zh) * | 2009-11-10 | 2011-05-11 | 奥林巴斯映像株式会社 | 图像拍摄装置以及图像拍摄方法 |
CN102055908B (zh) * | 2009-11-10 | 2013-07-24 | 奥林巴斯映像株式会社 | 图像拍摄装置以及图像拍摄方法 |
US9336606B2 (en) | 2011-02-03 | 2016-05-10 | Ricoh Company, Ltd. | Image capturing apparatus, image capturing method, and computer program product |
KR101455018B1 (ko) * | 2011-02-03 | 2014-10-27 | 가부시키가이샤 리코 | 촬상 장치, 촬상 방법, 및 컴퓨터 판독 가능한 기록 매체 |
CN102521580A (zh) * | 2011-12-21 | 2012-06-27 | 华平信息技术(南昌)有限公司 | 实时目标匹配跟踪方法及*** |
JP2013196199A (ja) * | 2012-03-16 | 2013-09-30 | Fujitsu Ltd | 利用者検知装置、方法及びプログラム |
JP2015041383A (ja) * | 2013-08-23 | 2015-03-02 | 株式会社リコー | 対象追跡方法及び対象追跡装置 |
JP2016057409A (ja) * | 2014-09-08 | 2016-04-21 | キヤノン株式会社 | 撮像装置および撮像装置の制御方法 |
JP2021132362A (ja) * | 2020-02-19 | 2021-09-09 | キヤノン株式会社 | 被写体追尾装置、被写体追尾方法、コンピュータプログラム及び記憶媒体 |
US11553136B2 (en) | 2020-02-19 | 2023-01-10 | Canon Kabushiki Kaisha | Subject tracking device, subject tracking method, and storage medium |
JP7214681B2 (ja) | 2020-02-19 | 2023-01-30 | キヤノン株式会社 | 被写体追尾装置、被写体追尾方法、コンピュータプログラム及び記憶媒体 |
JP2021184564A (ja) * | 2020-05-22 | 2021-12-02 | パナソニックIpマネジメント株式会社 | 画像追尾装置 |
Also Published As
Publication number | Publication date |
---|---|
CN101939980A (zh) | 2011-01-05 |
EP2242253B1 (en) | 2019-04-03 |
US20110001840A1 (en) | 2011-01-06 |
EP2242253A1 (en) | 2010-10-20 |
EP2242253A4 (en) | 2014-05-14 |
JP5090474B2 (ja) | 2012-12-05 |
US8253819B2 (en) | 2012-08-28 |
JPWO2009098894A1 (ja) | 2011-05-26 |
CN101939980B (zh) | 2012-08-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5090474B2 (ja) | 電子カメラおよび画像処理方法 | |
US8836777B2 (en) | Automatic detection of vertical gaze using an embedded imaging device | |
JP6106921B2 (ja) | 撮像装置、撮像方法および撮像プログラム | |
JP5054063B2 (ja) | 電子カメラ、画像処理装置及び画像処理方法 | |
US8306262B2 (en) | Face tracking method for electronic camera device | |
CN107707871B (zh) | 图像处理设备、摄像设备、图像处理方法和存储介质 | |
JP2008310775A (ja) | 表情認識装置及び方法、並びに撮像装置 | |
JP2015104016A (ja) | 被写体検出装置、撮像装置、被写体検出装置の制御方法、被写体検出装置の制御プログラムおよび記憶媒体 | |
JP6157165B2 (ja) | 視線検出装置及び撮像装置 | |
CN106603968B (zh) | 信息处理装置及信息处理方法 | |
US20210256713A1 (en) | Image processing apparatus and image processing method | |
JPH10307923A (ja) | 顔部品抽出装置および顔の向き検出装置 | |
JP2012068948A (ja) | 顔属性推定装置およびその方法 | |
JP5471130B2 (ja) | 画像処理装置及び方法 | |
CN113610865A (zh) | 图像处理方法、装置、电子设备及计算机可读存储介质 | |
JP5539565B2 (ja) | 撮像装置及び被写体追跡方法 | |
US10140503B2 (en) | Subject tracking apparatus, control method, image processing apparatus, and image pickup apparatus | |
JP2009163422A (ja) | 画像処理装置、画像処理方法、及び画像処理プログラム | |
JP5247419B2 (ja) | 撮像装置および被写体追跡方法 | |
JP2007025899A (ja) | 画像処理装置、画像処理方法 | |
JP5336798B2 (ja) | 画像処理装置および画像処理方法、プログラム | |
US10943094B2 (en) | Information processing apparatus for estimating age and method thereof | |
US20230065506A1 (en) | Image processing apparatus, control method thereof, and image capturing apparatus | |
US20240037985A1 (en) | Cascaded detection of facial attributes | |
US20240070877A1 (en) | Image processing apparatus, method for controlling the same, imaging apparatus, and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200980104490.7 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09708197 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2009552414 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 12865917 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2009708197 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |