WO2023221141A1 - 拍摄装置以及对焦控制程序 - Google Patents
拍摄装置以及对焦控制程序 Download PDFInfo
- Publication number
- WO2023221141A1 WO2023221141A1 PCT/CN2022/094276 CN2022094276W WO2023221141A1 WO 2023221141 A1 WO2023221141 A1 WO 2023221141A1 CN 2022094276 W CN2022094276 W CN 2022094276W WO 2023221141 A1 WO2023221141 A1 WO 2023221141A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- camera unit
- focus
- optical system
- image
- focus control
- Prior art date
Links
- 230000003287 optical effect Effects 0.000 claims abstract description 90
- 238000003384 imaging method Methods 0.000 claims description 93
- 239000007788 liquid Substances 0.000 claims description 8
- 238000012545 processing Methods 0.000 description 21
- 238000010586 diagram Methods 0.000 description 19
- 238000006243 chemical reaction Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 238000001514 detection method Methods 0.000 description 8
- 230000007246 mechanism Effects 0.000 description 8
- 238000003860 storage Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 238000000034 method Methods 0.000 description 6
- 230000015654 memory Effects 0.000 description 5
- 230000003936 working memory Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005401 electroluminescence Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 238000010079 rubber tapping Methods 0.000 description 2
- 206010047513 Vision blurred Diseases 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
- H04N23/672—Focus control based on electronic image sensor signals based on the phase difference signals
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B3/00—Simple or compound lenses
- G02B3/12—Fluid-filled or evacuated lenses
- G02B3/14—Fluid-filled or evacuated lenses of variable focal length
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
Definitions
- the present invention relates to a photographing device and a focus control program.
- an imaging device that performs automatic focus control using a phase difference signal obtained on the image plane of an imaging element.
- An example of an imaging element that outputs such a phase difference signal is an imaging element that is two-dimensionally arranged with normal pixels that output an image signal dedicated to generating an image and phase pixels that output a phase difference signal exclusively used for autofocus ( For example, refer to Patent Document 1).
- an imaging element Compared with an imaging element in which all pixels are pupil-divided for each microlens, such an imaging element has two photoelectric conversion sections and can switch between the output of an image signal and the output of a phase difference signal. Lower cost manufacturing is an advantage.
- Patent document 1 Japanese Patent Application Publication No. 2016-90785
- phase pixels that exclusively output phase difference signals do not output image signals for generating images
- the pixel values in the addresses where the phase pixels are arranged are generated by interpolation processing based on the pixel values of surrounding pixels. . Therefore, if the ratio of phase pixels among the pixels forming the imaging element is increased, the quality of the generated image will be reduced.
- the phase pixels are discretely arranged so as to be surrounded by the normal pixels, and that their ratio is also preferably small.
- phase pixels are arranged as described above, it is difficult to detect a phase difference signal that reflects the subject relatively small relative to the entire image captured by the imaging element, and the subject may not be focused.
- the subject may not be focused.
- the present invention has been made to solve this problem, and provides a photographing device, etc., which can accurately automatically focus on the main subject even when a wide-angle image in which the main subject is relatively small is generated. .
- the imaging device in the first aspect of the present invention includes: a first camera unit having a first optical system; a second camera unit configured as a second optical system facing the same direction as the first camera unit; and a focus control A unit that performs focus control of the first camera unit and the second camera unit.
- the first camera unit and the second camera unit each have a photographing element, and the photographing element is composed of a common sensor that outputs an image signal for forming an image. Pixels and phase pixels that are discretely arranged and surrounded by the ordinary pixels and output a phase difference signal for detecting focus are two-dimensionally arranged, and the focus control section uses the above-mentioned image sensor output from the imaging element of the first camera unit.
- focus control of the first optical system is performed with reference to the second defocus information acquired from the phase difference signal output from the imaging element of the second camera unit.
- the focus control program in the second aspect of the present invention performs focus control on the first camera unit and the second camera unit of the photographing device.
- the photographing device includes: a first camera unit having a first optical system; and a second camera. unit, which is configured as a second optical system facing the same direction as the first camera unit, the first camera unit and the second camera unit each having a photographing element, the photographing element is composed of an image sensor that outputs an image signal for forming an image.
- Ordinary pixels and phase pixels that are discretely arranged and surrounded by the ordinary pixels and output a phase difference signal for detecting focus are two-dimensionally arranged, and are formed using the image signal output from the imaging element of the first camera unit.
- the computer is caused to execute the following steps: an acquisition step of acquiring second defocus information based on the phase difference signal output from the imaging element of the second camera unit; and a driving step of referring to the second defocus information. to drive the focus lens of the above-mentioned first optical system.
- a photographing device or the like that can accurately automatically focus on a main subject even when a wide-angle image in which the main subject is relatively small is generated.
- FIG. 1 is a diagram showing the appearance of the imaging device according to this embodiment.
- FIG. 2 is a diagram showing the main hardware configuration of the imaging device.
- FIG. 3 is a diagram illustrating the pixel arrangement of the imaging element.
- FIG. 4 is a diagram showing an example of a scene to be photographed.
- FIG. 5 is a diagram showing an example of an image obtained from the first camera unit when autofocus control is performed by a phase difference signal from the first camera unit.
- FIG. 6 is a diagram showing an example of an image obtained from the second camera unit when autofocus control is performed on the same scene by a phase difference signal from the second camera unit.
- FIG. 7 is a diagram showing an example of an image obtained from the first camera unit when autofocus control of the first optical system is performed with reference to second defocus information.
- FIG. 8 is a diagram explaining the correspondence relationship between two focus areas.
- FIG. 9 is a diagram showing a processing procedure until a wide-angle image is generated.
- FIG. 1 is a diagram showing the appearance of the imaging device 100 according to this embodiment.
- (A) of FIG. 1 is a diagram mainly showing the first side of the imaging device 100
- (B) of FIG. 1 is a diagram mainly showing the second side opposite to the first side.
- the imaging device 100 of this embodiment is a so-called smartphone.
- it is a smartphone that also functions as an imaging device.
- the photographing function related to the present invention will be described, and other functions of the smartphone that utilize image data generated by photographing will be omitted.
- the imaging device 100 is described using a smartphone as an example.
- the imaging device may be a stand-alone camera or may be incorporated into a device with a imaging function such as a tablet terminal.
- the imaging device 100 includes a first camera unit 110 and a second camera unit 120 arranged in the same direction on the first surface side.
- the first camera unit 110 is a camera unit for generating a wide-angle image.
- the second camera unit 120 is a camera unit for generating telephoto images.
- the user designates the first camera unit for shooting, and when the user wants to obtain a telephoto image, the user designates the second camera unit for shooting.
- the first camera unit 110 and the second camera unit 120 are arranged parallel to the long side of the photographing device 100 in the figure.
- the arrangement of the two camera units is not limited to this. For example, they may also be arranged along a straight line obliquely intersecting the long side. .
- the positions of the first camera unit 110 and the second camera unit 120 may be opposite to those in the figure.
- the imaging device 100 includes a display 130 on the second side.
- the display 130 is, for example, a display device using an organic EL (Electro Luminescence; electroluminescence) panel, and displays a real-time image of the subject before shooting (live view display) or displays an image after shooting.
- a camera unit for selfies that is independent of the first camera unit 110 and the second camera unit 120 may be provided on the second surface side.
- a shutter button 161 is provided on the side of the imaging device 100 .
- the user can provide a photographing instruction to the photographing device 100 by pressing the shutter button 161 .
- a touch panel 162 is provided to overlap the display 130 .
- the user can also provide a shooting instruction to the photographing device 100 by tapping the shutter button displayed on the display 130 .
- users can also tap any part of the subject image displayed in live view to designate a certain area including that part as the focus area.
- the user can also switch between the first camera unit 110 and the second camera unit 120 or select displayed menu items through contact actions such as tapping.
- FIG. 2 is a diagram showing the main hardware configuration of the imaging device 100 .
- the imaging device 100 is composed of the first camera unit 110 , the second camera unit 120 , and the display 130 , as well as a system control unit 150 that controls these, and peripheral elements that cooperate with the system control unit 150 .
- the first camera unit 110 is a camera unit for generating wide-angle images, and mainly includes a first optical system 111 , a first drive mechanism 112 , a first imaging element 113 and a first analog front end (AFE) 114 .
- the first optical system 111 is an optical system for imaging the incident subject light beam on the imaging surface of the first imaging element 113 . It is represented by one lens in the figure, but it is generally composed of multiple lenses, and at least part of it is a focusing lens that can advance and retreat along the optical axis direction.
- the first drive mechanism 112 is a drive mechanism for moving the focus lens of the first optical system 111 along the optical axis direction, and includes an actuator that operates in accordance with instructions from the system control unit 150 .
- the first imaging element 113 is, for example, a CMOS image sensor.
- the first imaging element 113 will be described in detail later.
- the first imaging element 113 transmits pixel signals (image signals and phase difference signals to be described later) as output signals to the first analog front end 114 in accordance with instructions from the system control unit 150 .
- the first analog front end 114 adjusts the level of the pixel signal according to the gain instructed by the system control unit 150 , A/D converts the pixel signal into digital data, and transmits it to the working memory 151 .
- the second camera unit 120 is a camera unit for generating telephoto images, and mainly includes a second optical system 121 , a second driving mechanism 122 , a second imaging element 123 and a second analog front end (AFE) 124 .
- the second optical system 121 is an optical system for imaging the incident subject light beam on the imaging surface of the second imaging element 123 .
- the second optical system 121 is generally composed of a plurality of lenses like the first optical system 111, and is a focusing lens in which at least a part thereof can advance and retreat along the optical axis direction.
- the second drive mechanism 122 is a drive mechanism for moving the focus lens of the second optical system 121 along the optical axis direction, and includes an actuator that operates in accordance with instructions from the system control unit 150 .
- the second imaging element 123 is, for example, a CMOS image sensor.
- the second imaging element 123 and the first imaging element 113 will be described in detail later.
- the second imaging element 123 transmits the pixel signal as an output signal to the second analog front end 124 according to the instruction of the system control unit 150 .
- the second analog front end 124 adjusts the level of the pixel signal according to the gain instructed by the system control unit 150 , A/D converts the pixel signal into digital data, and transmits it to the working memory 151 .
- the first optical system 111 and the second optical system 121 are both single-focus optical systems with a fixed focal length, but they may also be at least one variable-focus optical system (zoom lens) that can change the focal length.
- the focal length of the first optical system 111 may be set shorter than the focal length of the second optical system 121 .
- the angle of view of the second optical system 121 is set to a telephoto angle of view compared to the angle of view of the first optical system 111 .
- the system control unit 150 is a processor (CPU: Central Processing Unit) that directly or indirectly controls each element constituting the imaging device 100 .
- the system control unit 150 functions as a various function control unit according to the control program to be executed. For example, when executing the focus control of the first camera unit 110 and the second camera unit 120 , it functions as a focus control unit and when shooting, the system control unit 150 functions as a focus control unit. When the received image is displayed on the display 130, it functions as a display control unit.
- the imaging device 100 mainly includes a work memory 151 , an image processing unit 152 , an operation unit 160 , a storage unit 170 , and a communication interface 180 as peripheral elements that cooperate with the system control unit 150 .
- the working memory 151 is a volatile high-speed memory, and is composed of, for example, SRAM (Static Random Access Memory).
- the working memory 151 receives pixel data sequentially converted from the first analog front end 114 and the second analog front end 124 respectively, and if the pixel data is data converted from an image signal, it is collectively stored in one frame of frame data. In addition, if the data is data converted from a phase difference signal, it is stored collectively as two waveform data.
- the work memory 151 transfers frame data to the image processing unit 153 and waveform data to the system control unit 150 .
- the work memory 151 is also appropriately used as a temporary storage area during the processing stage when the image processing unit 153 performs image processing or the processing stage when the system control unit 150 performs focus processing.
- the image processing unit 153 is composed of, for example, an ASIC (Application Specific Integrated Circuit) that specifically performs image processing. It performs various image processing such as interpolation processing on the received frame data to generate image data conforming to a predetermined format. If the generated image data is used for storage, it is stored in the storage unit 170 , and if it is used for display, it is displayed on the display 130 .
- ASIC Application Specific Integrated Circuit
- the operation unit 160 is an input device including the shutter button 161 or the touch panel 162 , and is a member that is operated when the user provides an instruction to the imaging device 100 .
- the operation unit 160 may further include a microphone.
- the storage unit 170 is a non-volatile memory, and is composed of, for example, an SSD (Solid State Drive).
- the storage unit 170 not only stores image data generated by photography, but also stores constants, variables, setting values, control programs, and the like required for operating the imaging device 100 .
- the communication interface 180 may include a communication unit of a 5G line or a wireless LAN. The communication interface 180 is used to transmit generated image data to an external device, etc.
- FIG. 3 is a diagram illustrating the pixel arrangement of the first imaging element 113 .
- the second imaging element 114 is also the same as the first imaging element 113, the first imaging element 113 will be described here.
- the first imaging element 113 is an imaging element that is two-dimensionally arranged from normal pixels 210 that output image signals exclusively for generating images, and phase pixels 220 that output phase difference signals exclusively for detecting focus.
- the normal pixel 210 is a substantially square pixel in which one photoelectric conversion unit is arranged without displacement relative to one microlens. In the normal pixel 210, a color filter of either RGB is arranged between the microlens and the photoelectric conversion unit.
- the phase pixel 220 is a substantially rectangular photoelectric conversion portion arranged for one microlens, which is similar to the shape of one of the photoelectric conversion portions of an ordinary pixel divided into two portions. Shifted pixels.
- the phase pixel 220 does not have a color filter disposed between the microlens and the photoelectric conversion unit.
- all adjacent pixels are normal pixels 210. In other words, each phase pixel 220 is surrounded by the normal pixels 210 and arranged discretely from each other.
- the structure of the photoelectric conversion part can be the same as that of the ordinary pixel 210, and a light-shielding mask with a shift opening is disposed between the microlens and the photoelectric conversion part, and the shift opening generates a signal related to the photoelectric conversion. The same effect occurs when the part is shifted as described above.
- the phase pixels 220 include a first phase pixel 221 in which the photoelectric conversion part is shifted in the first direction (the lower side in the figure), and a second phase pixel 221 in which the photoelectric conversion part is shifted in the opposite direction to the first direction (the upper side in the figure).
- the first phase pixels 221 and the second phase pixels 222 are respectively arranged in a predetermined pattern. Specifically, a plurality of detection lines are set along the shift direction of the shifted pixels (the up and down direction in the figure), and the first phase pixels 221 are periodically arranged on one side of each detection line (the right side in the figure). , the second phase pixels 222 are arranged out of phase with the same period on the other side (the left side in the figure).
- the first phase waveform is formed based on the phase difference signal that is the output signal of the first phase pixel 221
- the second phase waveform is formed based on the phase difference signal that is the output signal of the second phase pixel 222 .
- the system control unit 150 calculates the defocus amount which is the relative shift amount of the first phase waveform and the second phase waveform, the defocus direction which is the shift direction, and the degree of overlap of the two waveforms. Focus evaluation value, etc., to obtain out-of-focus information.
- the system control unit 150 performs focus processing based on the acquired out-of-focus information to focus on a predetermined subject. The focus processing will be described in detail later.
- first imaging element 113 and the second imaging element 123 are two-dimensionally arrayed imaging elements composed of normal pixels and phase pixels discretely arranged surrounded by the normal pixels, they do not need to be the same imaging elements.
- the respective imaging elements may differ from each other in, for example, the overall number of pixels or the arrangement pattern of phase pixels.
- the detection lines are set along one axis which is the up-down direction, but the detection lines may also be set in the orthogonal direction (the left-right direction in the example of the drawing), and the arrangement may be suitable for It detects the phase pixels of the line (phase pixels shifted to the right and phase pixels shifted to the left in the example of the figure). In this case, each phase pixel is preferably surrounded by normal pixels. If the detection lines are set in two orthogonal axes in this manner, focusing accuracy can be improved.
- phase pixels that exclusively output phase difference signals do not output image signals for generating images
- the pixel value of the pixel to which the address of the phase pixel is arranged is generally It is generated by interpolation processing based on the pixel values of surrounding pixels. Therefore, if the ratio of phase pixels among the pixels forming the imaging element is increased, the quality of the generated image will be reduced.
- the phase pixels are discretely arranged so as to be surrounded by ordinary pixels, and the ratio thereof is preferably small.
- the total number of phase pixels 220 is less than 5% of the total number of pixels.
- the focus control unit cannot obtain accurate defocus information that reflects the subject relatively small relative to the entire image captured by the imaging element.
- Unable to focus on the subject For example, in a scene where there is a long and slender main subject in front of a background with many high-frequency components, it is easy to fail to focus on the main subject and focus on the background side. This kind of failure is more likely to occur when an optical system using a wide-angle viewing angle is used for reflection.
- the imaging device 100 in this embodiment refers to the image signal output from the second imaging element 123 of the second camera unit 120.
- the second defocus information obtained by the phase difference signal performs focus control of the first optical system 111 .
- FIG. 4 is a diagram showing an example of a scene to be photographed. Specifically, a case is shown in which the user attempts to photograph a scene in which the forest 920 spreads behind the person 910 as the main subject through the photographing device 100 . Here, it is assumed that the user wants to focus on the character 910 . The user can determine the composition while confirming the live view image continuously acquired by the first camera unit 110 or the second camera unit 120 and displayed on the display 130 .
- FIG. 5 is a diagram showing an example of an image obtained from the first camera unit 110 when autofocus control of the first optical system 111 is performed only by the phase difference signal from the first camera unit 110 .
- the forest 920 as the background of the scene is a collection of many trees. Therefore, within the angle of view captured by the first camera unit 110, the image area (background area) of the forest 920 is an area with high spatial frequency.
- the image area (main area) of the person 910 as the main subject only occupies a very small part of the angle of view captured by the first camera unit 110, and is an area with relatively low spatial frequency.
- the focus control unit calculates out-of-focus information for the background area with a high spatial frequency.
- the focus lens of the first optical system 111 is driven based on the defocus information calculated in this way, the first optical system 111 focuses on the forest 920 as the background, and the wide-angle image 301 generated after the focusing, as shown in the figure, serves as the main subject.
- the subject person 910 is a blurred image.
- phase pixels 220 arranged along the detection line are discrete, the number of phase pixels 220 included in a small main area becomes smaller, and it is more difficult for the focus control section to base the operation on the phase pixels included in the main area.
- the defocus information is calculated from the waveform formed by the phase difference signal output by 220.
- FIG. 6 is a diagram showing an example of an image obtained from the second camera unit 120 when autofocus control of the second optical system 121 is performed on the same scene by a phase difference signal from the second camera unit 120 .
- the second optical system 121 of the second camera unit 120 has a telephoto viewing angle compared to the first optical system 111. Therefore, the image area (main area) of the person 910 accounts for a larger proportion of the entire area than in the case of FIG. 5 . Then, the number of phase pixels 220 included in the main area also increases, and more detailed parts of the person 910 are resolved, so the spatial frequency thereof becomes higher.
- the focus control unit can calculate defocus information for the main area.
- the focus lens of the second optical system 121 is driven based on the defocus information calculated in this way, the second optical system 121 focuses on the person 910 as the main subject, and as shown in the figure, the telephoto image generated after this focusing 302 is an image in which the character 910 is focused.
- the first optical system 111 can be improved It also focuses on the possibility of the character 910.
- the focus control unit drives the second imaging element 123 to cause the phase pixels 220 to output a phase difference signal and acquire second defocus information.
- the focus control unit moves the focus lens of the first optical system 111 to focus on the person 910 by referring to the second defocus information.
- the second defocus information includes area information of the second focus area 320 as an area for focus evaluation in the second imaging element 123, and the defocus amount, defocus direction, and focus evaluation value in the second focus area.
- FIG. 7 is a diagram showing an example of an image obtained from the first camera unit 110 when autofocus control of the first optical system 111 is performed with reference to the second defocus information.
- the focus control unit determines the first focus area 310 as the focus area in the first camera unit 110 based on the area information of the second focus area 320 included in the second defocus information.
- the first camera unit 110 and the second camera unit 120 are arranged close to each other, and the optical axis of the first optical system 111 and the optical axis of the second optical system 121 are parallel to each other. Therefore, for simplicity, the depth of the subject may not be considered.
- the focus control unit uses this conversion formula or reference table to determine the first focus area 310 based on the area information of the second focus area 320 .
- the focus control unit executes focus control of the first optical system 111 on the premise that the main subject exists in the area. Specifically, among the phase difference signals output by the first imaging element 113 , a phase waveform is generated only for the phase difference signal included in the first focus area 310 . Furthermore, the focus range (that is, the depth range in which the main subject is assumed to exist) is limited by referring to the defocus amount and the defocus direction of the second defocus information so that it is not affected by the background area. Based on the conditions defined in this way, the defocus amount and the defocus direction as the first defocus information are determined. The focus control unit determines the movement direction and movement amount of the focus lens of the first optical system 111 based on the determined defocus amount and defocus direction. By moving in this manner, it is possible to focus on the person 910 as the main subject.
- the focus control unit causes the first imaging element 113 to output the phase difference signal again, evaluates its phase waveform, and determines whether it is in a focused state.
- photographing processing is performed to generate a wide-angle image 301 .
- the first defocus information can be obtained again to correct the position of the focus lens.
- the position of the focus lens may be corrected while wobbling the focus lens so that the contrast of the partial image generated based on the image signal output from the ordinary pixels included in the first focus area 310 is the highest. The latter is what is called contrast AF.
- the focus control unit may perform contrast AF using the first focus area 310 as a target immediately after determining the first focus area 310 based on the second focus area 320 , without acquiring the second focus area 310 from the first imaging element 113 .
- An out-of-focus message may be performed.
- the focus control unit may refer to the second defocus information to execute focus control of the first optical system. That is, when it is determined that the focus control of the first optical system 111 is to be executed based on the first defocus information, the focus control of the first optical system may be executed without referring to the second defocus information. For example, whether it is difficult to perform focus control of the first optical system 111 based on the first defocus information can be determined based on whether the focus evaluation value in the first defocus information is less than a threshold or above the threshold.
- the second defocus information may be acquired in parallel with the acquisition of the first defocus information, and the judgment may be made by comparing the respective defocus amounts and defocus directions. Specifically, if the depth of the subject calculated based on the defocus amount and the defocus direction included in the first defocus information, and the subject's depth calculated based on the defocus amount and the defocus direction included in the second defocus information, If the depth is within a fixed range, it is assumed that the same subject is captured, and therefore it is determined that the focus control of the first optical system 111 can be performed based on the first defocus information. If it is not within the fixed range, assuming that different subjects are respectively captured, it is determined that it is difficult to perform focus control of the first optical system 111 based on the first defocus information.
- FIG. 8 is an explanatory diagram illustrating the correspondence relationship between two focus areas. Specifically, this figure shows a situation where the first camera unit 110 and the second camera unit 120 respectively capture a person existing at a distance d1 and a person existing at a distance d2 from the shooting device 100, as well as the telephoto images 302 and 302 thus captured. Figure of wide angle image 301.
- the person existing at the near distance d1 and the person existing at the far distance d2 are both close to the second camera unit 120 side.
- the telephoto image 302 generated from the second camera unit 120 is compared with the wide-angle image 301 generated from the first camera unit, the person present at the close distance d1 is captured when the person is photographed.
- the distance between the second focus area 320 and the first focus area 310 is greater than the distance between the second focus area 320 and the first focus area 310 that captures a person existing at a far distance d2 when the person is photographed.
- the first focus area 310 is shifted to the right when the person exists at the distance d1 than when the person exists at the distance d2.
- This correspondence can be calculated through triangulation. Specifically, when calculating the distance between the second focus area 320 and the person as the main subject in the telephoto image 302, the baseline length which is the distance between the optical axes of the two optical systems and the two optical systems can be used The angle of view ratio is used to determine the first focus area 310 in the wide-angle image 301 . In addition, the distance to the person can be calculated based on the defocus amount and defocus direction in the second defocus information, and the focus lens position at that time.
- the focus control section determines the first focus area 310 corresponding to the second focus area 320 more accurately in this manner, the accuracy of the focus control of the first optical system 111 can be further improved.
- the focus control unit may use a general near point priority (subjects close to the shooting device are given priority) or center priority (subjects near the center of the angle of view are given priority). Algorithm to determine the main subject to focus on. In this case, the subject calculated based on the defocus amount and the defocus direction included in the first defocus information is located closer than the subject calculated based on the defocus amount and the defocus direction included in the second defocus information. When the subject is at a far position or when it exists at a position far from the center, it may be determined that it is difficult to perform focus control of the first optical system 111 based on the first defocus information.
- this area may be used as the first focus area 310 .
- this area may be used as the first focus area 310 . Even in this case, if it is determined that it is difficult to perform focus control of the first optical system 111 based on the first defocus information, it is possible to perform the second focus area 320 with reference to the second defocus information in the area. Focus control of the first optical system 111.
- FIG. 9 is a flowchart showing the main processing procedure until the system control section 150 generates a wide-angle image. The flow starts, for example, from the time when the user presses the shutter button 161 .
- step S101 the focus control unit acquires first defocus information. Specifically, as described above, the first imaging element 113 is driven to cause the phase pixel 220 to output a phase difference signal, and various operations are performed to obtain the first defocus information.
- step S102 the focus control unit acquires the second defocus information in the same manner as the first defocus information. The process of step S102 may be performed in parallel with step S101.
- step S103 the focus control section determines whether focus control of the first optical system 111 can be performed based on the first defocus information. If it is determined that it is possible, step S104 is skipped and step S105 is entered. If it is determined that it is not possible, step S104 is entered.
- the focus control unit determines the first focus area 310 in the first imaging element 113 with reference to the second defocus information acquired in step S102, and proceeds to step S105.
- step S105 When entering step S105, if step S104 is skipped, the focus control unit moves the focus lens of the first optical system 111 based on the first defocus information to focus on the main subject.
- step S104 After step S104 is passed, in the determined first focus area 310, as described above, for example, by applying limiting conditions, the first defocus information is obtained again, and the first optical system is configured based on the first defocus information.
- the focus lens of 111 moves to focus on the main subject.
- step S106 causes the first imaging element 113 to output a phase difference signal again, evaluates its phase waveform, and determines whether it is in a focused state. If it is determined that the camera is in a focused state, step S107 is skipped and step S108 is entered. If it is determined that it is not in a focused state, step S107 is entered.
- step S107 the focus control unit executes contrast AF, corrects the position of the focus lens, and focuses on the main subject. After that, step S108 is entered.
- the system control unit 150 drives the first imaging element 113 to cause the normal pixels to output image signals, and causes the image processing unit 153 to generate image data.
- the system control unit 150 stores the generated image data in the storage unit 170 or displays it on the display 130 according to preset instructions, or sends it to an external device through the communication interface 180 to complete a series of processes.
- the first camera unit 110 captures a still image, but the same focus control can be performed even when a moving image is captured. For example, if frame images are continuously generated based on the image signals among the pixel signals output by the first imaging element 113 and the first defocus information is generated based on the phase difference signal, even during shooting of a moving image, it is possible to refer to the second image generated in parallel.
- the same focus control as above is performed while receiving out-of-focus information.
- the imaging device 100 includes two camera units.
- the same focus control can be performed even in an imaging device including three or more camera units.
- the focus control of the camera unit with a wide-angle angle of view can refer to the parameters obtained from the camera unit with a standard angle of view and the camera unit with a telephoto angle of view respectively.
- focus information may refer to the defocus information obtained from the camera unit with a telephoto viewing angle.
- focus control is performed based on one piece of defocus information acquired by each focus lens of the first optical system 111 and the second optical system 121 in an arbitrary state.
- the focus control may also be performed in an arbitrary state. Focus control is performed by acquiring multiple pieces of defocus information while changing the position of the focus lens. For example, when using an optical system with a small opening F value or an optical system with a long focal length, the position of the focusing lens can be changed multiple times according to its characteristics to obtain defocus information each time.
- the movement range of the focus lens can be determined based on the applied voltage. That is, since such a liquid lens can be a high-magnification zoom lens, for example, in the telephoto area, it is possible to focus on a subject 1.5m away and only allow the focus lens to move within this range. Shorten the acquisition time of out-of-focus information.
- the liquid lens may be a wide-angle lens or a telephoto lens. Since the liquid lens requires a motor to change the thickness of the liquid unit when focusing, a longer stroke or a higher moving speed of the motor is required during focusing. Slower, which may affect focusing speed. At this time, the optical system including the liquid lens can be assisted to perform focus control based on the defocus information of other optical systems. For example, multiple focus distance intervals are determined so that the liquid lens only needs to focus within a smaller range, thereby increasing the focusing speed.
- 100...shooting device 110...first camera unit, 111...first optical system, 112...first drive mechanism, 113...first shooting element, 114...first analog front end (AFE), 120...second camera unit, 121... second optical system, 122... second drive mechanism, 123... second imaging element, 124... second analog front end (AFE), 130... display, 150... system control unit, 151... working memory, 152...
- image processing Department 160...operation part, 161...shutter button, 162...touch panel, 170...storage part, 180...communication interface, 210...normal pixels, 220...phase pixels, 221...first phase pixels, 222...second phase pixels , 301...wide-angle image, 302...telephoto image, 310...first focus area, 320...second focus area, 910...people, 920...forest.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Studio Devices (AREA)
- Lens Barrels (AREA)
- Cameras In General (AREA)
- Automatic Focus Adjustment (AREA)
- Focusing (AREA)
Abstract
Description
Claims (8)
- 一种拍摄装置,其特征在于,具备:第一摄像头单元,其具有第一光学***;第二摄像头单元,其配置为与所述第一摄像头单元朝向同一方向;以及对焦控制部,其执行所述第一摄像头单元以及所述第二摄像头单元的对焦控制,所述第一摄像头单元以及所述第二摄像头单元分别具有拍摄元件,所述拍摄元件由输出用于形成图像的图像信号的普通像素和被所述普通像素包围且离散地配置的、输出用于检测焦点的相位差信号的相位像素二维排列而成,所述对焦控制部在使用从所述第一摄像头单元的所述拍摄元件输出的所述图像信号形成所述图像的情况下,参照从由所述第二摄像头单元的所述拍摄元件输出的所述相位差信号获取的第二失焦信息,执行所述第一光学***的对焦控制。
- 根据权利要求1所述的拍摄装置,其中,所述对焦控制部基于由所述第一摄像头单元和所述第二摄像头单元的配置确定的基线长度来确定所述第一光学***的对焦控制中的对焦区域。
- 根据权利要求1或2所述的拍摄装置,其中,如果所述对焦控制部确定需要所述第二失焦信息来协助进行第一光学***的对焦控制,则基于所述第二失焦信息,执行对所述第一光学***的对焦控制。
- 根据权利要求3所述的拍摄装置,其中,所述对焦控制部并行地获取所述第一失焦信息和所述第二失焦信息。
- 根据权利要求4所述的拍摄装置,其中,所述对焦控制部通过对所述第一失焦信息与所述第二失焦信息进行比较来判断是否难以执行所述第一光学***的对焦控制。
- 根据权利要求1~5中的任意一项所述的拍摄装置,其中,所述对焦控制部在参照所述第二失焦信息驱动所述第一光学***的对焦透镜后,基于使用从所述第一摄像头单元的所述拍摄元件输出的所述图像信号算出的对比度信息,对所述对焦透镜的位置进行校正。
- 根据权利要求1~6中的任意一项所述的拍摄装置,其中,所述第一光学***包含能够根据电信号进行对焦的液态透镜,所述对焦控制部基于通过第一失焦信息和/或第二失焦信息确定的所述电信号来控制所述第一光学***进行对焦。
- 一种对焦控制程序,其特征在于,进行拍摄装置的第一摄像头单元以及第二摄像头单元的对焦控制,所述拍摄装置具备:所述第一摄像头单元,其具有第一光学***;以及所述第二摄像头单元,其配置为与所述第一摄像头单元朝向同一方向的第二光学***,所述第一摄像头单元以及所述第二摄像头单元分别具有拍摄元件,所述拍摄元件由输出用于形成图像的图像信号的普通像素和被所述普通像素包围且离散地配置的、输出用于检测焦点的相位差信号的相位像素二维排列而成,在使用从所述第一摄像头单元的所述拍摄元件输出的所述图像信号形成所述图像的情况下,使计算机执行如下步骤:获取步骤,基于从所述第二摄像头单元的所述拍摄元件输出的所述相位差信号获取第二失焦信息;以及驱动步骤,参照所述第二失焦信息来驱动所述第一光学***的对焦透镜。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2022/094276 WO2023221141A1 (zh) | 2022-05-20 | 2022-05-20 | 拍摄装置以及对焦控制程序 |
JP2022538156A JP2024522409A (ja) | 2022-05-20 | 2022-05-20 | 撮像装置およびフォーカス制御プログラム |
CN202280001794.6A CN117546475A (zh) | 2022-05-20 | 2022-05-20 | 拍摄装置以及对焦控制程序 |
KR1020237037882A KR20230167074A (ko) | 2022-05-20 | 2022-05-20 | 촬영 장치 및 포커싱 제어 프로그램 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2022/094276 WO2023221141A1 (zh) | 2022-05-20 | 2022-05-20 | 拍摄装置以及对焦控制程序 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023221141A1 true WO2023221141A1 (zh) | 2023-11-23 |
Family
ID=88834417
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/094276 WO2023221141A1 (zh) | 2022-05-20 | 2022-05-20 | 拍摄装置以及对焦控制程序 |
Country Status (4)
Country | Link |
---|---|
JP (1) | JP2024522409A (zh) |
KR (1) | KR20230167074A (zh) |
CN (1) | CN117546475A (zh) |
WO (1) | WO2023221141A1 (zh) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016090785A (ja) | 2014-11-04 | 2016-05-23 | キヤノン株式会社 | 撮像装置及びその制御方法 |
CN106576143A (zh) * | 2014-07-25 | 2017-04-19 | 三星电子株式会社 | 图像拍摄装置和图像拍摄方法 |
CN106603911A (zh) * | 2016-11-29 | 2017-04-26 | 广东欧珀移动通信有限公司 | 控制方法、控制装置及电子装置 |
CN112004026A (zh) * | 2020-09-01 | 2020-11-27 | 北京小米移动软件有限公司 | 相位对焦装置、方法、拍摄方法、装置、终端设备及介质 |
WO2021106529A1 (ja) * | 2019-11-29 | 2021-06-03 | 富士フイルム株式会社 | 情報処理装置、情報処理方法、及びプログラム |
CN114079734A (zh) * | 2015-11-24 | 2022-02-22 | 三星电子株式会社 | 数字拍摄装置及其操作方法 |
-
2022
- 2022-05-20 WO PCT/CN2022/094276 patent/WO2023221141A1/zh active Application Filing
- 2022-05-20 CN CN202280001794.6A patent/CN117546475A/zh active Pending
- 2022-05-20 KR KR1020237037882A patent/KR20230167074A/ko unknown
- 2022-05-20 JP JP2022538156A patent/JP2024522409A/ja active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106576143A (zh) * | 2014-07-25 | 2017-04-19 | 三星电子株式会社 | 图像拍摄装置和图像拍摄方法 |
JP2016090785A (ja) | 2014-11-04 | 2016-05-23 | キヤノン株式会社 | 撮像装置及びその制御方法 |
CN114079734A (zh) * | 2015-11-24 | 2022-02-22 | 三星电子株式会社 | 数字拍摄装置及其操作方法 |
CN106603911A (zh) * | 2016-11-29 | 2017-04-26 | 广东欧珀移动通信有限公司 | 控制方法、控制装置及电子装置 |
WO2021106529A1 (ja) * | 2019-11-29 | 2021-06-03 | 富士フイルム株式会社 | 情報処理装置、情報処理方法、及びプログラム |
CN112004026A (zh) * | 2020-09-01 | 2020-11-27 | 北京小米移动软件有限公司 | 相位对焦装置、方法、拍摄方法、装置、终端设备及介质 |
Also Published As
Publication number | Publication date |
---|---|
KR20230167074A (ko) | 2023-12-07 |
JP2024522409A (ja) | 2024-06-21 |
CN117546475A (zh) | 2024-02-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP3992992B2 (ja) | 被写体像取得装置 | |
JP5388544B2 (ja) | 撮像装置およびそのフォーカス制御方法 | |
JP4874669B2 (ja) | オートフォーカスユニット及びデジタルカメラ | |
JP2010093422A (ja) | 撮像装置 | |
JP4094458B2 (ja) | 画像入力装置 | |
US8830383B2 (en) | Image pickup apparatus and control method thereof | |
JP2009192774A (ja) | 焦点調節装置および撮像装置 | |
JP5896763B2 (ja) | 光学機器および自動焦点調節を行う方法 | |
JP2008262049A (ja) | オートフォーカス装置、撮像装置及びオートフォーカス方法 | |
JP2014153509A (ja) | 撮像装置及び撮像方法 | |
JP2019117395A (ja) | 撮像装置 | |
JP2008298943A (ja) | 焦点調節装置および撮像装置 | |
JP6801658B2 (ja) | 撮像装置、撮像方法、及びプログラム | |
JP6187617B2 (ja) | 撮像装置 | |
WO2023221141A1 (zh) | 拍摄装置以及对焦控制程序 | |
JP2016142924A (ja) | 撮像装置及びその制御方法、プログラム、記憶媒体 | |
JP4786734B2 (ja) | カメラ | |
JP5018932B2 (ja) | 撮像装置 | |
JP4535462B2 (ja) | 被写体像取得装置 | |
JP6900228B2 (ja) | 撮像装置、撮像システム、撮像装置の制御方法、および、プログラム | |
JP2002277730A (ja) | 電子カメラの自動焦点制御方法、装置及びプログラム | |
JP2010224165A (ja) | 撮像装置 | |
JP2011176457A (ja) | 電子カメラ | |
JP2017134322A (ja) | レンズ装置 | |
JP4308494B2 (ja) | カメラ、被写体像取得装置、自動合焦システムおよび自動合焦方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 202280001794.6 Country of ref document: CN |
|
ENP | Entry into the national phase |
Ref document number: 2022538156 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20237037882 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020237037882 Country of ref document: KR |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202347078568 Country of ref document: IN |