WO2018045592A1 - 拍摄图像方法、装置和终端 - Google Patents
拍摄图像方法、装置和终端 Download PDFInfo
- Publication number
- WO2018045592A1 WO2018045592A1 PCT/CN2016/098751 CN2016098751W WO2018045592A1 WO 2018045592 A1 WO2018045592 A1 WO 2018045592A1 CN 2016098751 W CN2016098751 W CN 2016098751W WO 2018045592 A1 WO2018045592 A1 WO 2018045592A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- focus
- point
- quadrilateral
- rectangular
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 101
- 238000012937 correction Methods 0.000 claims abstract description 49
- 239000002131 composite material Substances 0.000 claims description 47
- 238000009826 distribution Methods 0.000 claims description 37
- 230000015572 biosynthetic process Effects 0.000 claims description 27
- 238000003786 synthesis reaction Methods 0.000 claims description 27
- 230000008569 process Effects 0.000 claims description 20
- 230000002194 synthesizing effect Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims description 11
- 238000004458 analytical method Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 description 33
- 238000010586 diagram Methods 0.000 description 30
- 238000012545 processing Methods 0.000 description 16
- PCTMTFRHKVHKIS-BMFZQQSSSA-N (1s,3r,4e,6e,8e,10e,12e,14e,16e,18s,19r,20r,21s,25r,27r,30r,31r,33s,35r,37s,38r)-3-[(2r,3s,4s,5s,6r)-4-amino-3,5-dihydroxy-6-methyloxan-2-yl]oxy-19,25,27,30,31,33,35,37-octahydroxy-18,20,21-trimethyl-23-oxo-22,39-dioxabicyclo[33.3.1]nonatriaconta-4,6,8,10 Chemical compound C1C=C2C[C@@H](OS(O)(=O)=O)CC[C@]2(C)[C@@H]2[C@@H]1[C@@H]1CC[C@H]([C@H](C)CCCC(C)C)[C@@]1(C)CC2.O[C@H]1[C@@H](N)[C@H](O)[C@@H](C)O[C@H]1O[C@H]1/C=C/C=C/C=C/C=C/C=C/C=C/C=C/[C@H](C)[C@@H](O)[C@@H](C)[C@H](C)OC(=O)C[C@H](O)C[C@H](O)CC[C@@H](O)[C@H](O)C[C@H](O)C[C@](O)(C[C@H](O)[C@H]2C(O)=O)O[C@H]2C1 PCTMTFRHKVHKIS-BMFZQQSSSA-N 0.000 description 15
- 238000003384 imaging method Methods 0.000 description 14
- 238000001514 detection method Methods 0.000 description 11
- 239000011159 matrix material Substances 0.000 description 11
- 230000005484 gravity Effects 0.000 description 10
- 238000003860 storage Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000009466 transformation Effects 0.000 description 6
- 238000003708 edge detection Methods 0.000 description 5
- 238000012546 transfer Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 241001156002 Anthonomus pomorum Species 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 238000010187 selection method Methods 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000699670 Mus sp. Species 0.000 description 1
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000012905 input function Methods 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- SBNFWQZLDJGRLK-UHFFFAOYSA-N phenothrin Chemical compound CC1(C)C(C=C(C)C)C1C(=O)OCC1=CC=CC(OC=2C=CC=CC=2)=C1 SBNFWQZLDJGRLK-UHFFFAOYSA-N 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/30003—Arrangements for executing specific machine instructions
- G06F9/3004—Arrangements for executing specific machine instructions to perform operations on memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/10—Selection of transformation methods according to the characteristics of the input images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/17—Image acquisition using hand-held instruments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/242—Aligning, centring, orientation detection or correction of the image by image rotation, e.g. by 90 degrees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
- H04N1/3876—Recombination of partial images to recreate the original image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
- H04N23/675—Focus control based on electronic image sensor signals comprising setting of focusing regions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/958—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging
- H04N23/959—Computational photography systems, e.g. light-field imaging systems for extended depth of field imaging by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V10/235—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on user input or interaction
Definitions
- the present invention relates to the field of computers, and in particular, to a method, an apparatus, and a terminal for capturing an image.
- Step S1 First, the camera focuses on the center of the field of view and takes a picture. Typically, the point of focus is the default image center.
- Step S2 Next, the processor acquires the photo and detects the position of the side or corner of the quadrilateral image therein.
- Step S3 The processor maps the image pixels in the detected quadrilateral to the area of the rectangular target according to the aspect ratio of the preset rectangle, and the newly added pixels in the mapping process are interpolated by the existing pixels in the original image. .
- Step S4 Generate a corrected new image according to the pixels in the transformed rectangle, and output and save the new image.
- the rectangular document corrected by the prior art can solve the trapezoidal distortion problem well when the image precision is not high, and the utility model is very practical, but it is not suitable for those applications that require high image precision.
- FIG. 2 is a schematic diagram showing an imaging abstraction of a tilt-shooting rectangular target.
- the physical plane W has a rectangular target, and when the photographing device observes the rectangular target, the tilt of the physical plane W and the imaging plane C The angle is ⁇ , and the rectangular target is imaged as a quadrilateral image on the imaging plane C.
- the rectangular targets are not all located within the depth of field of the lens focus, thereby causing the quadrilateral image to be partially clear near the focus, and the portion away from the focus to be blurred; in the quadrilateral image Keystone correction to obtain the stage of the corrected rectangular image.
- the rectangular target is inclined, the rectangular image corresponding to the distal end of the rectangular target is relative to The partial image of the rectangular image corresponding to the near end requires more pixels to be added by interpolation.
- the resolution of the initial image obtained by the shooting is often insufficient at the far end, so that the interpolation process in the trapezoidal correction stage does not have enough information source, resulting in trapezoidal correction.
- FIG. 4 is a schematic diagram showing the target image after the rectangular correction of the initial image acquired in FIG. 3, specifically, in conjunction with FIG. 4, when the actual rectangular target is photographed, relative to the distal end.
- the corresponding target image portion A has a poor definition and its sharpness is significantly lower than the near-end corresponding target image portion B.
- the embodiments of the present invention provide a method, an apparatus, and a terminal for capturing an image, which are intended to solve the problem in the prior art that the corrected rectangular document may have poor locality and poor user experience. problem.
- a method for capturing an image in a terminal with a camera includes: entering a shooting mode, detecting a quadrilateral image in the preview image; Selecting at least one focus within the quadrilateral image and focusing at least one initial image with the at least one focus, wherein the quadrilateral image comprises a first set of opposite sides and a second set of opposite sides, the first set The angle of the opposite side of the opposite side is smaller than the angle of the opposite side of the second set of opposite sides, the first side is the shorter one of the first set of opposite sides, and the at least one focus includes the first focus, a first focus in the quadrilateral image and in a partial region at one end of the first side; correcting a quadrilateral image in each of the at least one initial image to a rectangular image; based on the rectangular image, obtaining a final image and outputting .
- the method performs a focus-capture initial image by selecting a point in an area of one end of the first side corresponding to the distal end of the photographed rectangular target as a focus, and can improve the far-end of the rectangular target in the initial image.
- the resolution of the corresponding partial region is such that the source pixel of the corrected interpolation of the partial region is increased during the correction process, so that the sharpness of the captured image can be improved.
- the step of selecting at least one focus within the quadrilateral image and focusing at least one initial image with the at least one focus comprises: selecting the within the quadrilateral image The first focus captures an initial image at the first focus.
- the step of selecting the first focus within the quadrilateral image comprises: determining a first point and a second point on a second side and a third side of the quadrilateral image, respectively The second side and the third side are adjacent to the first side, and the distance from the first point to the vertex where the first side intersects the second side is less than or equal to the a vertex where the second side intersects the fourth side, a distance from the second point to a vertex where the first side intersects the third side is less than or equal to a vertex that intersects the third side and the fourth side
- the fourth side is an opposite side of the first side; determining a vertex where the first side intersects the second side, a vertex where the first side intersects the third side, the first A first area formed by a point and the second point connection is a partial area of one end of the first side; a point in the selected one of the first areas is the first focus.
- the step of selecting a point in the first region to be the first focus includes any one of: selecting an intersection of diagonal lines of the first region as The first focus; the intersection of the midpoint connecting the opposite side of the first region is the first focus.
- the step of selecting a point in the first region to be the first focus includes any one of: selecting a dynamic determination based on a luminance distribution in the first region a point is the first focus; a point selected to be dynamically determined based on the target to be tracked in the first area is the first focus; and a center point of gravity determined based on the uneven density distribution in the first area is selected For the first focus.
- selecting a center of gravity point determined based on the uneven density distribution in the first region as the first focus includes: detecting a quadrilateral image in the preview image; according to a shooting inclination of the quadrilateral image, Different parts of the quadrilateral image are assigned different densities, wherein the density ratio is set to be inversely proportional to the side length of the slice; according to the density ratio and the area of the quadrilateral image, the center of gravity of the quadrilateral image is calculated as the first focus.
- the step of obtaining a final image based on the rectangular image and outputting comprises: outputting the rectangular image as a final image.
- the step of selecting at least one focus within the quadrilateral image and focusing at least one initial image with the at least one focus comprises: selecting at least two within the quadrilateral image a focus, the at least two focuss including at least one of the first focus and the at least one second focus, the second focus being different from the first focus, and respectively photographing at least two of the at least two focuss Initial image.
- the second focus is a selected point in a partial region of the first edge-to-edge end within the quadrilateral image.
- the method for selecting the second focus includes: determining a third point and a fourth point on the second side and the third side of the quadrilateral image, wherein the second side and The third side is adjacent to the first side, and the distance from the third point to the vertex where the first side intersects the second side is greater than or equal to the intersection of the second side and the fourth side a vertex, a distance from the fourth point to a vertex where the first side intersects the third side is greater than or equal to a vertex where the third side intersects the fourth side, the fourth side is the a pair of opposite sides of the first side; determining a vertex where the fourth side intersects the second side, a vertex where the fourth side intersects the third side, the third point, and the fourth point
- the second area composed of the lines is a partial area of one end of the first side opposite side; the points in the selected one of the second areas are respectively selected as the second focus.
- the step of correcting the quadrilateral image in each of the at least one initial image into a rectangular image comprises: correcting the quadrilateral image in the at least two initial images to at least two rectangles respectively An image based on the rectangular image, the final image is obtained and outputted: combining the at least two rectangular images to obtain a final image; and outputting the final image.
- the step of synthesizing the at least two rectangular images to obtain a final image includes: performing feature point alignment on the at least two rectangular images; and each of the aligned rectangular images based on the alignment a sharpness distribution, determining a plurality of composite regions in the final image; synthesizing each of the composite regions to obtain a final image based on the sharpness of each of the rectangular images.
- the step of synthesizing each of the composite regions to obtain a final image includes at least one of: obtaining one from each of the rectangular images and the synthesis region The domain corresponds to the region with the highest resolution, and is stitched to obtain a final image; at least two regions corresponding to the composite region are obtained from each of the rectangular images, and the pixel is merged with the sharpness as a weight to obtain a final image. .
- the terminal has at least two cameras, and the step of focusing and capturing at least one initial image with the at least one focus includes: controlling each of the cameras to perform with different focus points respectively. Focus on each of at least one initial image.
- the method before the step of photographing at least one initial image by the focus, the method further comprises: determining an initial to be acquired according to the number of times the user continuously inputs the shooting instruction within the specified time The number of images.
- the method performs, by the device, selecting at least one focus corresponding to a proximal end of the rectangular target and a focus corresponding to a distal end of the rectangular target in the detected quadrilateral image during shooting, respectively.
- selecting a portion with higher definition from each rectangular image to form a composite image, and outputting the composite image as a final image by selecting different positions
- the focus is focused, the initial image with different local sharpness can be obtained, and the local sharpness of the corrected rectangular image is also different.
- the method further includes determining the number of initial images to be captured before the step of capturing at least one initial image in focus with the at least one focus.
- the determining the number of initial images to be captured includes: according to an angle between opposite sides of a first set of opposite sides of the quadrilateral image detected in the preview image, or within the quadrilateral image
- the number of initial images to be captured is determined from a sharpness profile in the direction from the first side to the first side.
- an apparatus for capturing an image in a terminal with a camera includes: a detecting module, configured to enter a shooting mode, and detect a quadrilateral image in the preview image a photographing module, configured to select at least one focus within the quadrilateral image based on a photographing instruction, and focus at least one initial image with the at least one focus, wherein the quadrilateral image includes a first set of opposite sides and The second set of opposite sides, the opposite angle of the opposite side of the first set of opposite sides is small An angle between opposite sides of the second set of opposite sides, a first side being a shorter one of the first set of opposite sides, the at least one focus comprising a first focus, and the first focus being in the quadrilateral a correction module for correcting a quadrilateral image in each of the at least one initial image into a rectangular image; and an output module for obtaining a final image based on the rectangular image Image and output.
- a detecting module configured to enter a shooting mode, and detect a quadrilateral image
- the photographing module includes: a first photographing sub-module for selecting the first focus within the quadrilateral image and taking an initial image with the first focus.
- the first shooting sub-module includes: a first side determining sub-module, configured to determine a first point and a second point on the second side and the third side of the quadrilateral image, respectively, wherein The second side and the third side are adjacent to the first side, and the distance from the first point to the vertex where the first side intersects the second side is less than or equal to the second side a vertex intersecting the fourth side, a distance from the second point to a vertex where the first side intersects the third side is less than or equal to a vertex that intersects the third side and the fourth side, The fourth side is the opposite side of the first side; the first area determining sub-module is configured to determine a vertex where the first side intersects the second side, the first side and the third side a first region formed by the intersecting vertices, the first point and the second point connecting line is a partial region of the first side end; the first focus determining submodule is configured to select one of the first regions The point in the area is
- the first focus determination sub-module includes at least one of the following: a first selection unit, configured to select an intersection of diagonal lines of the first area as the first focus a second selected unit, configured to select an intersection of a pair of midpoint midpoints of the first region as the first focus;
- the first focus determination sub-module includes at least one of: a third selected unit, configured to select a point dynamically determined based on a luminance distribution in the first region as the first a focus; a fourth selected unit for selecting a point dynamically determined based on the target to be tracked in the first area as the first focus; a fifth selected unit for selecting at the first The center of gravity point determined based on the uneven density distribution in the region is the first focus.
- the first focus determination sub-module includes at least one of the following: the output module includes: a first output sub-module for outputting the rectangular image as a final image.
- the first focus determination submodule includes at least one of the following:
- the photographing module includes: a second photographing sub-module for selecting at least two focus points in the quadrilateral image, the at least two focus points including at least one of the first focus and at least one second focus, The second focus is different from the first focus, and at least two initial images are captured in focus with the at least two focus points, respectively.
- the second focus determination sub-module includes at least one of the following: the second focus is a selected point in a partial region of the first edge-to-edge end within the quadrilateral image. .
- the second focus determining submodule includes at least one of the following: the second shooting submodule includes: a second side determining submodule for using the second side of the quadrilateral image Determining a third point and a fourth point on the third side, wherein the second side and the third side are adjacent to the first side, the third point to the first side and the a distance at which the vertices of the second side intersect is greater than or equal to a vertex that intersects the second side and the fourth side, and a distance from the fourth point to a vertex where the first side intersects the third side is greater than or equal to a vertex intersecting the third side with the fourth side, the fourth side is an opposite side of the first side; a second area determining submodule for determining the fourth side and the second side a second region formed by the intersecting vertices, the vertices at which the fourth side intersects the third side, and the third point and the fourth point line are partial regions of one end of the first side a second focus
- the correction module includes: a correction submodule, configured to respectively correct quadrilateral images in the at least two initial images into at least two rectangular images; and the output module includes: a synthesis submodule, For synthesizing the at least two rectangular images to obtain a final image; and a second output sub-module for outputting the final image.
- the synthesis sub-module includes: an analysis sub-module for performing sharpness analysis and feature point alignment on the rectangular image; and an area dividing sub-module for each of the aligned rectangular images a sharpness distribution, a plurality of composite regions are determined; and a sub-module is executed for synthesizing each of the composite regions based on the sharpness of each of the rectangular images.
- the execution sub-module includes at least one of the following: a first synthesis sub-unit, configured to obtain, from each of the rectangular images, a highest resolution corresponding to the composite region. a region, splicing to obtain a final image; a second compositing subunit, configured to acquire at least two regions corresponding to the compositing region from each of the rectangular images, and perform pixel merging with a sharpness as a weight to obtain Final image
- the shooting module when the terminal has at least two cameras, the shooting module includes: a control sub-module, configured to control each of the cameras to focus on at least one initial image with different focus.
- the apparatus further includes: a selection module, configured to determine a number of initial images to be captured before the at least one initial image is captured in focus by the at least one focus.
- the selected module includes: a selected sub-module, configured to select an angle of a pair of opposite sides of the first pair of sides of the quadrilateral image detected in the preview image, or an image within the quadrilateral image A sharpness distribution curve in a direction from the first side to the first side to the side, determining the number of initial images to be photographed.
- a terminal includes: at least one camera; a display unit; one or more processors; a memory; a plurality of applications; the memory is used to store a computer program;
- the processor running the computer program, performing the following process: entering a shooting mode, detecting a quadrilateral image in the preview image; selecting at least one focus within the quadrilateral image based on the shooting instruction, and using the at least one focus Focusing on at least one initial image, wherein the quadrilateral image comprises a first set of opposite sides and a second set of opposite sides, the opposite sides of the first set of opposite sides being smaller than the opposite sides of the second set of opposite sides An angle, a first side being a shorter one of the first set of opposite sides, the at least one focus comprising a first focus, the first focus being within the quadrilateral image and a partial area at one end of the first side Correcting a quadrilateral image in each of the at least one initial image to a rectangular image; based
- the terminal can perform the process in conjunction with the method provided by the embodiments of the present invention in the first aspect.
- FIG. 1 is a schematic flowchart diagram of an image acquisition method in the prior art
- Figure 2 is a schematic illustration of an imaging abstraction of a tilted rectangular object
- FIG. 3 is a schematic diagram showing an oblique acquisition of a rectangular object in an image acquisition process according to an image acquisition method of the prior art
- FIG. 4 is a schematic diagram showing a target image after rectangular correction of the initial image acquired in FIG. 3;
- FIG. 5 is a schematic structural diagram of a terminal according to some embodiments.
- FIG. 6 is a schematic diagram showing functions of modules in an Android framework provided according to some embodiments.
- FIG. 7 is a flow chart showing a method of capturing an image according to some embodiments.
- FIG. 8 shows a schematic diagram of a quadrilateral image in a preview image provided in accordance with some embodiments
- FIG. 9 illustrates a schematic diagram of detecting the sharpness of a quadrilateral image in a preview image provided in accordance with some embodiments.
- 10a-10e illustrate schematic diagrams of methods of selecting a focus during a photographing process, in accordance with some embodiments
- FIG. 11 is a schematic diagram showing a schematic diagram of a method for performing pixel interpolation in rectangular correction according to some embodiments
- FIG. 12 is a schematic flow chart showing a method according to some captured images
- 13a-13c are schematic diagrams showing before and after feature point alignment of two quadrilateral images according to some embodiments.
- 14a-14g illustrate a simplified schematic diagram of a composite region based on sharpness distribution partitioning of two corrected quadrilateral images, provided in accordance with some embodiments;
- FIG. 15 is a schematic structural diagram of a terminal having two cameras according to some embodiments.
- FIG. 16 is a schematic diagram showing a schematic diagram of focus selection when shooting with the terminal shown in FIG. 15 according to some embodiments;
- FIG. 17 shows a schematic flow diagram of capturing an image in accordance with some embodiments.
- Figure 18 shows a schematic flow diagram of capturing an image in accordance with some embodiments
- 19a-19c illustrate schematic diagrams of obtaining an initial image and a corrected image by a method of capturing an image, in accordance with an embodiment of the present invention
- FIG. 20 is a block diagram showing the structure of an apparatus for capturing an image according to some embodiments.
- FIG. 21 is a schematic structural diagram of a first shooting sub-module according to some embodiments.
- FIG. 22 is a schematic structural diagram of a second shooting sub-module provided according to some embodiments.
- FIG. 23 shows a block diagram of a synthesis sub-module provided in accordance with some embodiments.
- the technical solution of the embodiments of the present invention may be used for a terminal capable of capturing an image
- the terminal may be all electronic devices having a photographing function and an image processing function
- the electronic device may be a smart camera, a mobile phone, Tablet Personal Computer, Laptop Computer, Digital Camera, Digital Camera, Projection Equipment, Personal Digital Assistant (PDA), e-book reader (English: e-book reader) Or a wearable device or the like, which is not further limited in this embodiment of the present invention.
- FIG. 5 shows a schematic structural diagram of a terminal that can be used to perform one of the methods of the embodiment according to some embodiments.
- the terminal can be a terminal device including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), an on-board computer, a digital camera, etc.
- the terminal 500 includes an RF (Radio). Frequency, radio frequency circuit 510, memory 520, input unit 530, display unit 540, sensor 550, audio circuit 560, WiFi (wireless fidelity) module 570, processor 580, and power supply 590 and the like.
- RF Radio
- Frequency radio frequency circuit 510, memory 520, input unit 530, display unit 540, sensor 550, audio circuit 560, WiFi (wireless fidelity) module 570, processor 580, and power supply 590 and the like.
- WiFi wireless fidelity
- the components of the terminal 500 will be specifically described below with reference to FIG. 5:
- the RF circuit 510 can be used for receiving and transmitting signals during the transmission or reception of information or during a call. Specifically, after receiving the downlink information of the base station, it is processed by the processor 580. In addition, the uplink data is designed to be sent to the base station.
- RF circuits include, but are not limited to, an antenna, at least one amplifier, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, and the like.
- RF circuitry 510 can also communicate with the network and other devices via wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access).
- GSM Global System of Mobile communication
- GPRS General Packet Radio Service
- CDMA Code Division Multiple Access
- the terminal for communication may include an RF circuit, and for a non-communication terminal such as a digital camera, the RF circuit may not be included.
- the memory 520 can be used to store software programs and modules, and the processor 580 executes various functional applications and data processing of the terminal 500 by running software programs and modules stored in the memory 520.
- the memory 520 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored according to The data created by the use of the terminal 500 (such as audio data, phone book, etc.) and the like.
- memory 520 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
- the input unit 530 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the terminal 500.
- the input unit 530 may include a touch panel 531, at least one camera 532, and other input devices 533.
- the touch panel 531 also referred to as a touch screen, can collect touch operations on or near the user (such as the user using a finger, a stylus, or the like on the touch panel 531 or near the touch panel 531. Operation), and drive the corresponding connecting device according to a preset program.
- the touch panel 531 can include touch detection. Two parts of the device and the touch controller.
- the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information.
- the processor 580 is provided and can receive commands from the processor 580 and execute them.
- the touch panel 531 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves.
- the input unit 530 further includes at least one camera 532, wherein the camera 532 can perform a photographing operation based on the touch information received by the touch device, and then transmit the acquired captured image to the processor 580 for corresponding processing.
- the camera 532 can be a digital camera or an analog camera.
- the imaging color can be color, black and white, grayscale, etc.
- the camera 532 can be directly mounted on the terminal or externally connected through the data line.
- the input unit 530 may further include other input devices 533.
- other input devices 533 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like.
- the display unit 540 can be used to display information input by the user or information provided to the user and various menus of the terminal 500.
- the display unit 540 can include a display panel 541.
- the display panel 541 can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.
- the touch panel 531 can cover the display panel 541. When the touch panel 531 detects a touch operation on or near it, the touch panel 531 transmits to the processor 580 to determine the type of the touch event, and then the processor 580 according to the touch event. The type provides a corresponding visual output on display panel 541.
- the touch panel 531 and the display panel 541 are used as two independent components to implement the input and input functions of the terminal 500, in some embodiments, the touch panel 531 may be integrated with the display panel 541. The input and output functions of the terminal 500 are implemented.
- Terminal 500 can also include at least one type of sensor 550, such as a light sensor, motion sensor, and other sensors.
- the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 541 according to the brightness of the ambient light, and the proximity sensor may close the display panel 541 when the terminal 500 moves to the ear. / or backlight.
- Accelerometer sensors can detect the magnitude of acceleration in all directions (usually three axes). When stationary, the magnitude and direction of gravity can be detected. It can be used to identify the attitude of the terminal (such as horizontal and vertical screen switching, related games, magnetometer posture).
- vibration recognition related functions such as pedometer, tapping
- other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which can also be configured in the terminal 500, are not described herein.
- Audio circuit 560, speaker 561, and microphone 562 can provide an audio interface between the user and terminal 500.
- the audio circuit 560 can transmit the converted electrical data of the received audio data to the speaker 561, and convert it into a sound signal output by the speaker 561.
- the microphone 562 converts the collected sound signal into an electrical signal, and the audio circuit 560 is used by the audio circuit 560. After receiving, it is converted into audio data, and then processed by the audio data output processor 580, transmitted to the terminal, for example, via the RF circuit 510, or outputted to the memory 520 for further processing.
- WiFi is a short-range wireless transmission technology
- the terminal 500 can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 570, which provides wireless broadband Internet access for users.
- FIG. 5 shows the WiFi module 570, it can be understood that it does not belong to the essential configuration of the terminal 500, and may be omitted as needed within the scope of not changing the essence of the invention.
- Processor 580 is the control center of terminal 500, which connects various portions of the entire terminal using various interfaces and lines, by running or executing software programs and/or modules stored in memory 520, and recalling data stored in memory 520, The various functions and processing data of the terminal 500 are performed to perform overall monitoring of the terminal.
- the processor 580 may include one or more processing units; preferably, the processor 580 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application, and the like.
- the modem processor primarily handles wireless communications. It will be appreciated that the above described modem processor may also not be integrated into the processor 580.
- the terminal 500 also includes a power source 590 (such as a battery) that supplies power to the various components.
- a power source 590 such as a battery
- the power source can be logically coupled to the processor 580 through a power management system to manage functions such as charging, discharging, and power management through the power management system.
- the terminal 500 may further include a Bluetooth module or the like, and details are not described herein again.
- the processor included in the terminal has the following functions: entering a shooting mode, Detecting a quadrilateral image in the preview image; capturing at least one initial image based on the photographing instruction, wherein the initial image includes at least one point in a partial region of the quadrilateral image near the shorter side as the first focus a captured image; correcting a quadrilateral image in the initial image into a rectangular image; and having a plurality of initial images, synthesizing a plurality of the rectangular images into clear based on the corrected sharpness distribution of each of the rectangular images The final image.
- Figure 6 shows the architecture in the Android system.
- the document correction module is a function related to photographing, and the algorithm and operation of the camera are implemented in the framework layer (software framework layer).
- the application layer mainly provides a camera application, controls the camera to obtain a preview image, and captures an initial image; in the application frame layer, detects a quadrilateral image in the preview image, and corrects the initial image into a rectangular image based on the document correction algorithm, when there are multiple In the rectangular image, feature alignment and compositing are performed to form the final image;
- the camera driver in the kernel is a program that controls the action of the specific camera, such as controlling the focus, photographing, etc. of the camera, and the drivers of different manufacturers and models are different.
- the camera driver is the final execution module of the application layer for previewing, focusing, taking photos, adjusting camera parameters, and directly controlling the camera hardware.
- the hardware abstraction layer encapsulates the kernel driver and provides an interface to shield the low-level implementation details. The purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system to have Hardware-independent, portable on multiple platforms.
- the hardware abstraction layer is a programming layer that allows the operating system to interact with hardware devices at the logical layer rather than the hardware layer.
- the operating system core or hardware driver can call the hardware abstraction layer. In either case, the calling program does not need to know the specific design details of the hardware, just the parameters required by the abstraction layer.
- the hardware abstraction layer is a bridge connecting the application framework layer and the camera driver. It is a proxy that transfers the control commands of the application layer and the application framework layer to the camera to the correct hardware driver to perform actions and returns the data obtained from the hardware.
- Application layer and application framework layer is a bridge connecting the application framework layer and the camera driver. It is a proxy that transfers the control commands of the application layer and the application framework layer to the camera to the correct hardware driver to perform
- FIG. 7 illustrates a method of capturing an image, which may be applied to a shooting mode of document correction (or trapezoidal correction) for improving the clarity of a corrected image in document correction, in accordance with some embodiments.
- the document correction is a method of detecting, transforming, and processing a quadrilateral image
- the input is an image
- the output is also an image, which is used to help the user correct image distortion caused by tilting of the shooting angle when shooting a rectangular target.
- the final image of the rectangle is finally output.
- the rectangular object generally refers to a rectangular image containing information, such as: a file, an invoice, a book, a business card, a document, a handout, a photo, an advertisement, a display, a television, a movie, a screen, etc., of course, those skilled in the art It should be understood that the rectangular targets described above are merely examples, and when other rectangular objects are captured as images, the methods described herein can be used for shooting and correction.
- the method provided in accordance with some embodiments includes: step S11, step S12, step S13, and step S14.
- step S11 a shooting mode is entered to detect a quadrilateral image in the preview image; then, in step S12, at least one focus is selected within the quadrilateral image based on the shooting instruction, and the at least one Focusing on at least one initial image, wherein the quadrilateral image comprises a first set of opposite sides and a second set of opposite sides, the opposite sides of the first set of opposite sides being smaller than the pair of opposite pairs of the second set An angle between the sides, the first side being the shorter one of the first set of opposite sides, the at least one focus comprising a first focus, the first focus being within the quadrilateral image and at the end of the first side In the region; then, in step S13, the quadrilateral image in each of the at least one initial image is corrected to a rectangular image; thereafter, in step S14, based on the rectangular image, a
- the terminal selects to take an initial image, and in the step S12, the terminal selects a point in a partial region of the first side of the first edge in the quadrilateral image based on the shooting instruction. a focus, and photographing an initial image with the first focus, wherein the first side is the middle of the two opposite sides of the opposite sides of the quadrilateral image having a smaller angle of the opposite sides a short side; then, in the step S13, the terminal corrects the quadrilateral image in the initial image to a rectangular image; subsequently, in the step S14, the terminal uses the rectangular image as a final image Output. Outputting the final image may include displaying the final image directly on the display unit, or saving to the storage space of the photographed image, and displaying the final image on the display unit according to the user's instruction.
- the method performs a focus shooting initial image by selecting a point in an area of one end of the first side corresponding to the distal end of the photographed rectangular target as the first focus, and can improve the initial image.
- the resolution of the partial region corresponding to the distal end of the rectangular target in the image is to increase the source pixel of the corrected interpolation of the partial region during the correction process, thereby improving the sharpness of the final image.
- the terminal starts a photographing device (for example, a camera or the like) according to a corresponding operation of the user, and enters a photographing mode.
- a photographing device for example, a camera or the like
- the user's corresponding operations may include any identifiable operation by the user on the device, and may include at least one of: any specified touch operation (click, slide, etc.) on the touch device of the device, the device Recognizable voice control operations, gesture operations, etc.
- the shooting mode may be a shooting mode corrected according to a document selected by a user.
- the terminal acquires a preview image through the camera and detects a quadrilateral image in the preview image.
- the focus of the focus may be a point (eg, a center point) of the central region of the preview image, and is continuously focused as the framing position changes.
- the focus of the focus may also be a point (eg, a human eye center point, etc.) that is dynamically determined by the target to be tracked.
- the terminal acquires an image acquired by the camera in real time and presents the acquired image as a preview image, which may be displayed on a display device (eg, a display screen) of the terminal.
- a display device eg, a display screen
- the image of the rectangular target is presented in the preview image.
- the plane of the camera is not parallel to the plane where the rectangular target is located, but is inclined to the plane where the rectangular target is photographed.
- the rectangular object presents a quadrilateral image in the preview image. Generally, the greater the tilt of the shot, the larger the deformation of the quadrilateral image presented by the rectangular object.
- the terminal detects a quadrilateral image in the preview image, and the position of the edge and the corner of the quadrilateral image can be detected by some detection methods.
- the method for detecting the position of the edge and the angle is, for example, identified by the Canny operator edge detection method.
- Hough transform detection technique parametrimeter estimation technique using voting principle
- the method includes a step S121, in the step S121, based on the photographing instruction, the selected point is a first focus in a partial region of the first side of the first side of the quadrilateral image, and The first focus focuses on the initial image, wherein the first side is the shorter one of the two pairs of sides of the quadrilateral image having a smaller angle of the opposite side.
- the shooting instruction may be automatically triggered by the terminal after entering the shooting mode.
- the initial image is automatically triggered; or may be determined according to a user operation input.
- the initial image is an image acquired by the camera acquisition based on the shooting instruction, and is not rectangular corrected and/or synthesized.
- the two opposite sides with smaller angles on the sides are the two sides with larger difference in shooting distance, and based on the principle of near-large and far-distance, the shorter sides of the two opposite sides with smaller angles That is, the first side corresponds to the farthest end of the rectangular target, and the longer side, that is, the first side opposite side (fourth side) corresponds to the edge of the nearest end of the rectangular target.
- the determining method of the first side specifically includes: first, determining four sides of the quadrilateral image; and then determining an angle between the two opposite sides, wherein the two pairs of sides having the smaller angle of the opposite sides are The shorter side is the first side corresponding to the most distal end of the rectangular target, and the longer side that is also opposite the first side is the side corresponding to the nearest end of the rectangular target.
- the angle between the opposite sides is the angle between the two opposite side extension lines, or the angle formed by moving one side in parallel to intersect the opposite side.
- the quadrilateral image ABCD in the preview image is detected, and the angle formed by the opposite side is the angle formed by the opposite side, such as the angle between the edge AB and the opposite side of the edge CD ⁇ 1, the edge AD and The opposite side angle ⁇ 2 of the side BC, where ⁇ 1 is smaller than ⁇ 2, and the side AB is smaller than the side CD, it is determined that the side AB of the shorter side is the first side, and the side CD is the first side opposite side.
- the angle of the opposite side is considered to be 0°.
- a partial region at one end of the first side AB of the quadrilateral image ABCD corresponds to a partial region of the distal end of the rectangular target, and the selected portion in the partial region at the end of the first side AB in the quadrilateral image ABCD is first.
- the focus is to set the first focus to the farther end corresponding to the rectangular target.
- the specific step of determining a partial region of one end of the first side may include: In step S121a, a first point and a second point are respectively determined on the second side and the third side of the quadrilateral image, and the distance from the first point to the vertex where the first side intersects the second side a distance equal to or less than a vertex that intersects the second side and the fourth side, and a distance from the second point to a vertex where the first side intersects the third side is less than or equal to the third side and the first a vertex intersecting at four sides, the fourth side being an opposite side of the first side; in step S121b, a vertex at which the first side intersects the second side, the first side and the first side A first region formed by the apex of the three sides intersecting, the first point, and the second point connecting line forms a partial region of one end of the first side.
- a partial region at one end of the first side AB in the quadrilateral image ABCD may be along a certain point E on the second side AD of the quadrilateral image ABCD (this point E is compared to the first
- the four-sided CD is closer to the first side AB) to the third side BC, closer to the first side AB, a point F (the point F is closer to the first side AB than the fourth side CD)
- a partial area ABFE at the first end may be along a certain point E on the second side AD of the quadrilateral image ABCD (this point E is compared to the first
- the four-sided CD is closer to the first side AB) to the third side BC, closer to the first side AB, a point F (the point F is closer to the first side AB than the fourth side CD)
- a partial area ABFE at the first end may be along a certain point E on the second side AD of the quadrilateral image ABCD (this point E is compared to the first
- the four-sided CD is
- selecting the first focus in the first area is step S121c.
- the first focus is selected by selecting the intersection of the diagonal lines of the first area as Depicting a first focus; selecting an intersection of a pair of midpoints of the first region as the first focus; selecting a point dynamically determined based on a luminance distribution in the first region as the first focus Selecting, in the first region, a point dynamically determined based on the target to be tracked as the first focus; selecting a center of gravity point determined based on the uneven density distribution in the first region as the first focus; and many more.
- the midpoint F and the midpoint E of the second side AD and the third side BC are determined, and the first area ABEF is determined. Then, a point is selected as the first focus in the first area ABEF.
- the intersection G1 of the intersection of two or two pairs of midpoints in the first region ABEF or the intersection G2 of the intersection of the intersections of the two diagonals is selected as The first focus and so on.
- a point may also be dynamically determined as the first focus in the first area ABEF according to the brightness distribution, the target to be tracked, and the like.
- the target to be tracked may be, for example, a target (eg, a human face, etc.) highlighted in the quadrilateral image.
- the third points F' and E' on the second side AD and the third side BC near the first side AB may also be determined, and the first area ABE'F' and the like may be determined.
- the manner of selecting the first focus may also be, in the quadrilateral image ABCD, based on the centroid calculation of the uneven density distribution, the specific steps include: (1) detecting the preview image Quadrilateral image ABCD; (2) According to the shooting inclination of the quadrilateral image, different density is assigned to different parts of the quadrilateral image ABCD, for example, in the horizontal direction of the preview image, the density ratio is set from the first side AB to the fourth side CD.
- the density ratio is also set to be inversely proportional to the side length of the slice ( Or a higher power ratio), in FIG. 10c, the angle between the opposite sides of the first side AB and the fourth side CD is 0°, and the direction from the second side AD to the third side BC in the vertical direction of the preview image
- the density ratio is uniform; (3)
- the center of gravity of the quadrilateral image ABCD is calculated as the first focus G according to the density ratio and the area of the quadrilateral image ABCD.
- the center of gravity calculated using the center of gravity of the uneven density distribution is also a point in the region near the end of the first side.
- the terminal maximizes the distal end of the quadrilateral image corresponding to the rectangular target by selecting a point in the quadrilateral image that is close to a partial region of the distal end of the rectangular target.
- the sharpness of the partial region to compensate for the insufficient resolution of the partial region corresponding to the distal end of the rectangular target in the quadrilateral image, improve the sharpness of the initial image at the appropriate depth of field position, thereby improving the overall image of the final image. Sharpness and visual effects.
- the terminal corrects the quadrilateral image in the initial image to a rectangular image.
- the method of performing the correction may be, first, generating a matrix M with respect to the pixel point position coordinates (x, y) based on the pixel distribution of the quadrilateral image, according to the size and sum of the quadrilateral image and the rectangular image to be corrected.
- the shape correspondence relationship determines the transformation matrix of the quadrilateral image to the rectangular image, and performs operations on the matrix M and the transformation matrix to calculate the corresponding position (x', y') of the pixel in the new image.
- the pixel value of the pixel point is taken as the pixel value of the integer coordinate; if the position of the pixel point does not fall on the integer coordinate, the pixel points adjacent to the integer coordinate are used.
- Interpolation calculation obtains the pixel value of the integer coordinate, generates a new pixel point, generates a matrix T according to the new pixel point position coordinate, and generates a corrected rectangular image according to the matrix T.
- the interpolation method can adopt the linear interpolation method, and the linear function is the interpolation method of the interpolation function.
- Two-dimensional images usually use bilinear interpolation, which is a linear interpolation extension of an interpolation function with two variables, obtained by linear interpolation in two directions. Referring to FIG.
- the values of 4 pixels are respectively ⁇ 10, 20, 30, 40 ⁇ as shown in (a), and if enlarged to (b) 9 pixels shown, 9 pixels can be reduced to cover 4 pixels, each pixel is combined according to the area of the original pixel covered, with different weights, as shown in (c), where a, c , g, i only contains one pixel, the pixel value is equal to the pixel covered under it; b, d, f, h spans two pixels, the equal weight is equal, the values of two pixels are fused by weight; e covers 4 The pixels are equal in area, the weights are equal, and four pixels are blended by the mean value; finally, the nine pixel values as shown in (d) are ⁇ 10, 15, 20, 20, 25, 30, 30, 35, 40 ⁇ .
- image interpolation methods such as nearest interpolation, linear interpolation, bilinear interpolation, cubic convolution interpolation, bicubic interpolation, etc., as applied to some embodiments of the present invention, although not described again, can also be cited It is used in some embodiments of the invention.
- the region image other than the quadrilateral image in the initial image may be deleted.
- the region image other than the quadrilateral image in the initial image may be retained and adjusted accordingly according to the change of the quadrilateral image;
- the image of the area other than the quadrilateral image in the initial image may continue to be adjusted according to the alignment and composition process of the rectangular image in the subsequent steps, and remains in the final image for output.
- the image of the area other than the quadrilateral image in the initial image may also be deleted in a subsequent step or deleted before being corrected to a rectangular image.
- the terminal selectively captures a plurality of the initial images and performs rectangular correction, synthesis, and output on the initial images.
- a shooting mode is entered, and a quadrilateral image is detected in the preview image; in the step S12, the terminal selects at least two focus points in the quadrilateral image based on the shooting instruction.
- the at least two focuss include at least one of the first focus and the at least one second focus, and respectively photographing at least two initial images with the focus, wherein the focus includes at least one within the quadrilateral image Part of the first end a selected point in the region and at least one selected point in a partial region of the first edge opposite end of the quadrilateral image, the first edge being a smaller angle of the quadrilateral image And the shorter side of the two pairs of edges; in the step S13, the terminal corrects the quadrilateral image in the at least two initial images into a rectangular image; in the step S14, the step S141 and the step are included S142, in the step S141, the terminal synthesizes a plurality of the rectangular images to obtain a final image, and in the step S142, outputs the final image. Outputting the final image may include displaying the final image directly on the display unit, or saving to the storage space of the photographed image, and displaying the final image on the display unit according to the user's instruction.
- the method separately captures at least one second focus and a first focus corresponding to a distal end of the rectangular target in the detected quadrilateral image during the shooting by the terminal, and acquires multiple pieces.
- the second focus is a focus corresponding to the proximal end of the rectangular target.
- the content of the step S11 is the same as or substantially the same as the content of the step S11 shown in FIG. 7.
- the content of the step S11 shown in FIG. 7. For the sake of brevity, it will not be described again, and is included herein by reference.
- the terminal captures a plurality of initial images based on the photographing instruction.
- the plurality of initial images include not only at least one selected image in a partial region of one end of the first side of the quadrilateral image, but also at least one image in the quadrilateral image.
- the selected point in the partial area at one end of the edge is the image of the second focus focus shooting.
- the number of initial images to be photographed is not limited, and may be, for example, one, two, three, four, eight, and the like.
- the number of initial images to be taken may be pre-programmed by the program or determined by a user selected operation.
- the program is automatically set to be used every time the user enters a shot. (for example, clicking the shooting button), two initial images are taken; for example, each time the document-corrected shooting mode is entered, the user is provided with a selection prompt to select the number of initial images to be taken by the user, and then whenever Each time the user enters a shooting instruction (for example, clicking a shooting button), a corresponding number of initial images are taken.
- the number of shots of the initial image is determined according to a user's shooting instruction (a single shot or an instruction to take multiple shots).
- the method further includes determining the number of initial images to be captured prior to capturing the initial image.
- the number of initial images to be captured may determine the number of initial images to be captured according to the imaging condition of the quadrilateral image in the preview image, wherein the imaging situation of the quadrilateral image in the preview image may be but not limited to: preview The degree of deformation of the quadrilateral image in the image, or the difference in sharpness of the quadrilateral image in the preview image. If the deformation of the quadrilateral image is smaller or the difference in sharpness after correction is smaller, it is determined to take a smaller number of initial images, and if the quadrilateral image is deformed more or the sharpness difference is larger, it is determined to take a larger number of initials. Images, such as two, four or even more.
- analyzing a sharpness profile in a direction from the first side to the first side of the quadrilateral image in the preview image; and then determining an initial to be photographed according to the sharpness profile The number of images.
- Shooting at the focus position enables more clear local areas to be obtained, thereby improving the overall sharpness of the final image in subsequent stages of synthesis.
- determining the number of initial images to be captured according to the deformation condition of the quadrilateral image may be determined according to the contrast angle of the quadrilateral image, and specifically includes: (1) first detecting the quadrilateral in the preview image. Image ABCD; (2) Find the angle of the quadrilateral image ABCD to the side; (3) For example, if any of the angles ⁇ is greater than the preset angle threshold, the preset angle threshold may be 10° to 20°, for example At 15°, it can be presumed that the inclination of the shooting angle is large, and it is determined that two or more initial images are taken, and the larger the angle between the two pairs of edges, the number of initial images can be increased.
- analyzing a sharpness profile in a direction from the first side to the first side of the quadrilateral image in the preview image; and then determining an initial to be photographed according to the sharpness profile The number of images.
- Shooting at the focus position enables more clear local areas to be obtained, thereby improving the overall sharpness of the final image in subsequent stages of synthesis. Referring to FIG.
- the process of determining the number of shots according to the definition distribution includes: (1) detecting a quadrilateral image ABCD in the preview image; (2) finding a preview image along the first direction X (ie, the first side AB to the fourth direction) Edge CD direction), the quadrilateral image ABCD sharpness curve; (3) If within the range of the quadrilateral image, based on the range of the sharpness curve, determine the number of shots of the initial image, the range of variation is the sharpness maximum and clarity For the difference of the minimum value, take 100 full marks as an example. When the range is 30 to 40, it is determined to take two initial images. When the range is 40 to 50, it is determined to take three initial images. When the range is 50 to 80, the shooting is determined. The initial image, .
- the relevant threshold of the shooting tilt can be specifically set according to different shooting conditions, different shooting conditions, and shooting requirements.
- the imaging of the quadrilateral image in the preview image can be used to determine the number of shots of the initial image, and can also be used to select a reference for each focus position during the shooting of the initial image.
- the number of focus corresponding to the number of sheets of the initial image to be acquired is selected.
- the method of selecting a point as the first focus in the partial region of the first side of the first side of the quadrilateral image refer to the selected method in step S12 shown in FIG. 7, and details are not described herein again.
- a method of selecting a point as a second focus in a partial region of one end of the first side to the side (ie, the fourth side) within the quadrilateral image may include: at the second side and the first Determining a third point and a fourth point on each of the three sides, wherein a distance from the third point to a vertex where the first side intersects the second side is greater than or equal to the second side and the fourth a vertex of the intersection, a distance from the fourth point to a vertex where the first side intersects the third side is greater than or equal to a vertex that intersects the third side and the fourth side; determining the fourth a second region formed by a vertex intersecting the second side, a vertex where the fourth side intersects the third side, and a third line connecting the third point and the fourth point is the first a partial region at one end of the edge; at least one of the points in the second region is selected as the second focus.
- the third point may coincide with a first point, which may coincide with the first point.
- the position of the reselected focus may be not limited to the first region or the determination of the point as the focus in the first region or the second region. In the second area, it is preferred to select different focus positions each time to obtain clear images of different local areas.
- the manner in which the second focus is selected in the second region may refer to the manner in which the first focus is selected in the first region.
- the midpoints of the second side AD and the third side BC intersecting the fourth side CD are point F and point E, respectively.
- two three-point points F1 and F2 on the second side AD, and two three-point points E1 and E2 on the third side BC divide the quadrilateral area into three partial areas ABE1F1.
- F1E1E2F2 and F2E2CD take the intersection of the opposite midpoint connection points in the three partial regions, and determine the three focal points G1, G2 and G3.
- the number of initial images to be captured may also determine the number of initial images to be acquired according to the user continuously inputting a shooting instruction (for example, a continuous shooting manner by long pressing the photo button) within a specified time. Since the number of initial images taken is dynamic, in the case where the number of initial images is unknown in advance, the first point in the quadrilateral image may be selected as the first point in the partial region near the shorter side of the quadrilateral image. Focus, when the second sheet is taken, the point in the partial region near the longer side of the quadrilateral image is selected as the second focus, and in the subsequent shooting of each photo, the quadrilateral image may be alternated into the shorter side. A point in a partial area and a point in a partial area of the quadrilateral image that is closer to the longer side are used as the focus, but each selected focus is different from the focus of the previous shot.
- a shooting instruction for example, a continuous shooting manner by long pressing the photo button
- the image area near the focus is clearer, and the more different focus is selected, the more clear local areas of the quadrilateral image can be obtained, which can help improve the final The sharpness of the image.
- step S13 the quadrilateral image in each of the initial images is corrected to a rectangular image.
- step S13 for the correction of each initial image, reference may be made to the specific content of step S13 shown in FIG. 7, and details are not described herein again.
- the step S141 includes the following steps: step S141a, performing feature point alignment on the rectangular image; and step S141b, determining a plurality of composite regions based on the sharpness distribution of each of the aligned rectangular images Step S141c, synthesizing each of the composite regions based on the sharpness of each of the rectangular images.
- the step of synthesizing each of the composite regions based on the sharpness of each of the rectangular images is: acquiring, from each of the rectangular images, an area having the highest resolution corresponding to the composite region, and performing Splicing to obtain a final image; or, acquiring at least two regions corresponding to the composite region from each of the rectangular images, and performing pixel fusion with sharpness as a weight to obtain a final image.
- the edges (including edges and corners) of the rectangular target detected by performing edge detection on the quadrilateral image may be different each time, and thus the quadrilateral image in different initial images is displaced. If the size and the rotation angle are slightly different, each rectangular image corrected by the initial image needs to be further aligned based on the feature points of the rectangular image to facilitate the subsequent synthesis step.
- all the corrected rectangular images are image-aligned (Image Stiching), and at least two rectangular images have translational relationships such as translation, rotation, and affine.
- the transformation matrix can be calculated.
- aligning the feature points of at least two rectangular images, and the rectangular images are aligned.
- the alignment process includes:
- the other rectangular images are aligned with the reference image two feature points, and extracting and describing the feature points of the two rectangular images includes: extracting a series of feature points, and the extraction method may include: Scale-invariant feature transform (SIFT), SURF algorithm (Speeded Up Robust Features), Harris corner detection algorithm, etc., other feature point extraction methods well known to those skilled in the art, if applicable
- SIFT Scale-invariant feature transform
- SURF algorithm Speeded Up Robust Features
- Harris corner detection algorithm etc.
- the rectangular image T1 of FIG. 13a is used as a reference image, and the rectangular image T2 of FIG. 13b is aligned.
- the feature points of the rectangular image T1 and the rectangular image T2 are matched to find out Feature matching points X1, X2, X3, and X4, and corresponding X1', X2', X3', and X4', wherein the feature matching point is an abstract mark of a pixel point, indicating a position and an eigenvalue in the rectangular image
- the feature point may be a position mark such as a corner, a fork, or an edge in the rectangular image;
- the coordinates of the feature matching points X1, X2, X3, and X4 are: (211, 224), (577, 240), (209, 466), (545, 511), and in the rectangular image T2,
- the coordinates of the feature matching points X1', X2', X3', and X4' are: (213, 215), (579, 261), (192, 450), (525, 527),
- one of the rectangular images T2 is transformed, rotated, scaled, affine transformed, etc., and aligned with another rectangular image T1 to form an aligned rectangular image T2', the rectangular image T1 and the aligned image
- the rectangular image T2' is as shown in Fig. 13c.
- edges including edges and corners
- the corrected aspect ratio is also within an error range. Then the rectangular image corrected for each initial image can be aligned.
- the composite area may be a plurality of splicing areas, or a combination of splicing areas and fused areas.
- the rectangular correction and feature alignment are performed on the initial images M1 and M2 of FIG. 14a to form two corrected rectangular images T1 and T2 of FIG. 14b (T1 and T2 may also be aligned rectangular images),
- T1 and T2 may also be aligned rectangular images
- the sharpness of the two rectangular images T1 and T2 shown in Fig. 14b is analyzed, and the sharpness curves L1 and L2 shown in Fig. 14c are obtained, L1 is the sharpness curve of the rectangular image T1, and L2 is the sharpness curve of the rectangular image T2.
- the evaluation method of the image sharpness may adopt a Brenner gradient function (Brunner gradient function), specifically, a definition evaluation by calculating the square of the gray difference of two adjacent pixels, and the gradient function is defined as follows:
- f(x, y) represents the gray value of the pixel f (x, y) corresponding to the image f
- D(f) is the result of the image sharpness calculation.
- FIG. 14d is a target image in which the rectangular images T1 and T2 are combined, and the rectangular and rectangular images T1 and T2 in 14d are identical in size, and the rectangle in the initial image is 14d.
- the middle may be blank, and the splicing line MN divides the rectangle in FIG. 14d into two composite regions Q1 and Q2.
- the splicing line is not necessarily a straight line, and the traverse is determined according to the distribution of the sharpness.
- the area corresponding to the rectangular image with higher definition can be directly selected for splicing, that is, the area of the corrected rectangular image corresponding to the sharpness curve L1 is selected in the combined area Q1 and the synthesized area Q2
- the area of the corrected rectangular image corresponding to the sharpness curve L2 is selected for splicing. It is possible to divide into regions of high definition and sharpness in accordance with the splicing line MN in the rectangular images T1 and T2, and the regions with high definition are copied into the corresponding composite regions; or based on the pixels in the composite region divided in FIG. 14d.
- the point coordinates are used to find the pixel value of the pixel coordinate in the rectangular image with high definition as the pixel value of the pixel in the composite area.
- the three composite regions Q1, Q2, and Q3 are divided based on the sharpness curves L1 and L2, wherein the sharpness curves L1 and L2 of the two corrected rectangular images corresponding to the composite region Q2 are The sharpness difference is not large, so the composite area Q2 can be determined as a fusion area, in which the two corrected rectangular images are used, and the sharpness of each pixel is used as a weight for fusion processing.
- FIG. 14f when there are three corrected rectangular images, obtaining sharpness distribution curves L1, L2, and L3, and dividing three composite regions Q1, Q2, and Q3 based on three sharpness distribution curves, at the time of synthesis Selecting the region of the corrected rectangular image corresponding to the sharpness curve L1 in the combined region Q1, selecting the region of the corrected rectangular image corresponding to the sharpness curve L2 in the combined region Q2, and selecting the sharpness curve in the combined region Q3
- the area of the corrected rectangular image corresponding to L3 is spliced to form an image with higher definition.
- FIG. 14g when there are three corrected rectangular images, the sharpness distribution curves L1, L2, and L3 are obtained, and the five composite regions Q1, Q2, Q3, and Q4 are divided based on the three sharpness distribution curves. And Q5, wherein the composite regions Q2 and Q4 are determined as fusion regions, wherein the fusion region Q2 is fused based on the rectangular images corresponding to the sharpness curves L1 and L2, and the fusion region Q4 is fused based on the rectangular images corresponding to the sharpness curves L2 and L3.
- the composite regions Q1, Q3, and Q5 respectively select regions of the corrected rectangular image corresponding to the sharpness curve L1, the sharpness curve L2, and the sharpness curve L3.
- the step of acquiring at least one initial image includes: controlling at least two cameras, respectively, with different The focus is focused, and each shot takes at least one initial image.
- the terminal is a binocular camera phone having two identical pixels.
- the shooting positions are different, so that it is possible to simultaneously focus on different positions and take two initial images.
- the two initial images Due to the deviation of the shooting position, the two initial images have slight aberrations.
- the position of the quadrilateral image in the initial image is deviated.
- (a) in FIG. 16 is a preview image of the camera 1 and is calculated based on a preset focus selection method.
- the two candidate focus points are G1 and G2
- (b) in Fig. 16 is a preview image of the camera 2, and based on the same focus selection method, the calculated two candidate focus points are H1 and H2. Since there are two cameras that can focus at the same time, (G1, H2) and (G2, H1) become two sets of candidate focus pairs.
- the two cameras simultaneously acquire the initial image.
- the corrected two rectangular images are obtained by rectangular correction (also referred to as trapezoidal correction), and then the alignment and splicing fusion process is performed on the corrected rectangular image by the method in the foregoing embodiment.
- rectangular correction also referred to as trapezoidal correction
- the terminal may also be a binocular camera mobile phone having two different pixels.
- the preview images of the two cameras also have parallax.
- resolution has become a major consideration in determining the focus position.
- the focus of the high-pixel camera can be determined at the small end portion of the detected quadrilateral image, that is, the far-end side of the captured rectangular object, and the focus of the low-pixel camera is determined to be detected.
- the big end of the quadrilateral image which is the proximal side of the rectangular target being photographed.
- the resolution is also different.
- the difference in sharpness is also shown as follows: When analyzing the sharpness of the two images, the maximum value and the range of the resolution of the two images are greatly different. A position with equal resolution can be used as the stitching position, which may use more images from a high resolution camera. This embodiment is also applicable to the method of splicing region fusion of the first embodiment to reduce image mutation near the splicing line.
- a method for capturing an image includes the following steps, in accordance with an embodiment of the present invention:
- the camera is turned on, enters the document correction mode, acquires the preview image Ip, and detects the quadrilateral image Qp in the current Ip;
- a shooting instruction is detected
- Correction Q is a rectangular image Ir, which is the final corrected image Ir;
- the number of shots is automatically selected according to the imaging condition of the quadrilateral image detected in the preview image, such as the angle of the edge or the sharpness distribution curve.
- the automatic selection of the number of shots may be performed before the initial image is taken, that is, may be performed before the photographing instruction is detected.
- 19a to 19c are diagrams showing an initial image obtained and a final output image in the process of capturing an image and processing an image using the method of capturing an image as described in the present application.
- the quadrilateral image is detected in the preview image, and the shorter side of the two opposite sides of the quadrilateral image having the smaller angle is the first side (corresponding to the distal edge of the rectangular target being photographed)
- a plurality of initial images are taken, and at least one initial image taken as a focus is obtained as a point of a partial region close to one end of the first side as shown in FIG. 19a, as shown in FIG. 19b.
- the initial image taken as the focus of the point on the side of the first side of the opposite side is subjected to rectangle correction and synthesis processing to form a final image as shown in Fig. 19c, wherein the sharpness of the area A' is compared to the prior art. Significantly improved, while the difference in sharpness from the area B' is small, improving the overall sharpness of the final image.
- the apparatus includes: a detection module, a photographing module, a correction module, and an output module, first, the detection module enters a shooting mode, and the preview image Detecting a quadrilateral image; then, the photographing module selects at least one focus within the quadrilateral image based on a photographing instruction, and photographs at least one initial image in focus with the at least one focus, wherein the quadrilateral image includes a pair of opposite sides and a second set of opposite sides, the opposite sides of the first set of opposite sides are smaller than the opposite sides of the second set of opposite sides, and the first side is the first pair of opposite sides a short side, the at least one focus comprising a first focus, the first focus being within the quadrilateral image and in a partial region at one end of the first side; then, the correction module will each of the at least one The quadrilateral image in the initial image is corrected to a rectangular image; thereafter, the output module
- FIG. 20 illustrates an apparatus 1 for capturing an image in a terminal including a camera and a plurality of applications, wherein the apparatus 1 includes a detection module 11, a photographing module 12, a correction module 13, and an output module 14 according to some embodiments.
- the detecting module 11 enters a shooting mode to detect a quadrilateral image in the preview image; then, the shooting module 12 selects at least one focus within the quadrilateral image based on the shooting instruction, and the at least one focus Focusing on at least one initial image, wherein the quadrilateral image comprises a first set of opposite sides and a second set of opposite sides, the opposite sides of the first set of opposite sides being smaller than the opposite sides of the second set of opposite sides An angle, a first side being a shorter one of the first set of opposite sides, the at least one focus comprising a first focus, the first focus being within the quadrilateral image and a partial area at one end of the first side
- the correction module 13 corrects the quadrilateral image in each of the at least one
- the apparatus for photographing an image of the present invention can be applied to a photographing mode of document correction (or trapezoidal correction) for improving the sharpness of a corrected image in document correction, wherein the document correction is a quadrilateral image Detection, transformation, processing, input is an image, and the output is also an image, which is used to help the user correct the image distortion caused by the tilt of the shooting angle when shooting a rectangular target, and finally output the moment Shape the final image.
- a photographing mode of document correction or trapezoidal correction
- the rectangular object generally refers to a rectangular image containing information, such as: a file, an invoice, a book, a business card, a document, a handout, a photo, an advertisement, a display, a television, a movie, a screen, etc., of course, those skilled in the art It should be understood that the rectangular targets described above are merely examples, and when other rectangular objects are captured as images, the methods described herein can be used for shooting and correction.
- the device may be hardware, software or a combination of software and hardware installed in a terminal having a camera and a plurality of applications, wherein the terminal may be all electronic devices having a photographing function and an image processing function.
- the electronic device can be a mobile phone, a tablet personal computer, a laptop computer, a digital camera, a digital camera, a projection device, a personal digital assistant (PDA), an e-book.
- PDA personal digital assistant
- the embodiment of the present invention is not limited to the embodiment of the present invention, such as an e-book reader or a wearable device.
- step S11 For a specific implementation of the detecting module 11, refer to the specific content of step S11 shown in FIG. 7.
- the implementation method of the shooting module refers to the specific content of step S12 shown in FIG. 7, and details are not described herein for brevity.
- the photographing module 12 includes: a first photographing sub-module 121, configured to select the first focus in the quadrilateral image, and photograph the first focus The initial image.
- the first side determining submodule 121a, the first area determining submodule 121b, and the first focus refers to the contents of the steps S121a, S121b, and S121c in the foregoing embodiment, and the contents of the specific embodiments described in FIG. 10a and FIG. 10b, and details are not described herein again.
- the implementation method of the calibration module refers to the specific content of step S13 shown in FIG. 7.
- the implementation method of the output module refers to the specific content of step S14 shown in FIG. 7.
- details are not described herein again. The way is included here.
- the output module outputs the rectangular image as a final image through a first output sub-module (not shown).
- the output module includes a synthesis sub-module 141 and a second output sub-module (not shown).
- the process performed by the device 11 includes: first, the detecting module 12 enters a shooting mode, and detects a quadrilateral image in the preview image; then, the shooting module 12 includes a second shooting sub-module 122, The second photographing sub-module 122 selects at least two focus points within the quadrilateral image, the at least two focus points including at least one of the first focus and at least one second focus, the second focus being different from Depicting a first focus and respectively capturing at least two initial images with the at least two focuss; respectively; subsequently, the correction module corrects the quadrilateral image in the initial image to a rectangular image.
- the detection module refer to the specific content of step S11' shown in FIG. 12
- the implementation method of the imaging module refers to the specific content of step S12' shown in FIG. 12, and details are not described herein for brevity.
- the second focus is a selected point in a partial region of the first edge-to-edge end within the quadrilateral image.
- the second shooting submodule includes: a second side determining submodule, a second area determining submodule, and a second focus determining submodule.
- the second side determining sub-module respectively determines a third point and a fourth point on the second side and the third side of the quadrilateral image, wherein the second side and the third side are opposite to the first side Adjacent to a side, a distance from the third point to a vertex where the first side intersects the second side is greater than or equal to a vertex that intersects the second side and the fourth side, the fourth point to the The distance between the apex of the first side and the third side is greater than or equal to the intersection of the third side and the fourth side a vertex, the fourth side is an opposite side of the first side; the second area determining submodule determines a vertex, the fourth side, and the fourth side of the fourth side intersecting the second side a second region formed by the vertices of the three sides intersecting, the third point and the fourth point connection is a partial region of one end of the first edge opposite side; the second focus determining submodule selects a The points in the second area are respectively taken as the second focus
- the correction module respectively corrects the quadrilateral image in the at least two initial images into at least two rectangular images by using a correction submodule;
- the output module includes: a synthesis submodule and a second output submodule, the synthesis submodule Combining the at least two rectangular images to obtain a final image; the second output sub-module outputs the final image.
- the number of initial images to be captured is not limited, and may be, for example, one, two, three, four, eight, and the like.
- the number of initial images to be taken may be pre-programmed by the program or determined by a user selected operation.
- the program is automatically set to take two initial images each time the user inputs a shooting instruction (for example, clicking a shooting button); and, for example, provides a selection prompt to the user each time the document-corrected shooting mode is entered.
- the number of initial images taken by the user is selected, and then each time the user inputs a shooting instruction (for example, clicking a shooting button), a corresponding number of initial images are taken.
- the number of initial images to be captured may also determine the number of initial images to be captured according to the imaging condition of the quadrilateral image in the preview image, wherein the imaging of the quadrilateral image in the preview image may be For example, but not limited to, the case where the tilt angle is roughly determined according to the degree of deformation of the quadrilateral image in the preview image, or the difference of the sharpness of each part after the square image is corrected to a rectangular image according to the pre-detected preview image. .
- the quadrilateral image has a smaller tilt angle or a smaller resolution difference after correction, it is determined to take a smaller number of initial images, and if the tilt angle is larger or the sharpness difference is larger, it is determined that a larger number of shots are taken.
- the initial image for example, two, four or even more.
- the number of shots of the initial image is determined according to the shooting tilt, and the greater the shooting tilt, the increased number of captured images to enhance the sharpness of the original image.
- the apparatus also includes a selected module that is capable of capturing at least one initial image after focusing with the at least one focus The number of initial images to be taken. Further, the selected module includes a selected sub-module according to an angle of a pair of opposite sides of the first pair of sides of the quadrilateral image detected in the preview image, or an image within the quadrilateral image The sharpness distribution curve in the direction from the first side to the first side is determined, and the number of initial images to be photographed is determined.
- the shooting tilt is predicted according to a diagonal angle of the opposite side of the quadrilateral image or a sharpness distribution curve in a direction from the first side to the first side opposite side in the quadrilateral image;
- the number determination module determines the number of initial images to be photographed according to the tilt threshold range to which the photographing tilt belongs, wherein the larger the photographing tilt, the more the determined number of initial images to be photographed.
- the implementation method of the pre-judging module and the quantity determining module refers to the judging process shown in FIG. 8, and will not be described again for the sake of brevity.
- the number of shots of the initial image is determined according to a user's shooting instruction (a single shot or an instruction to take multiple shots).
- the photographing module further includes: a quantity determining sub-module, the number determining sub-module determining, according to the number of times the user continuously inputs the photographing instruction within a specified time, before the step of capturing the at least two initial images The number of initial images to be acquired.
- the implementation method of the quantity determining sub-module refers to the judging process shown in FIG. 9, and will not be described again for the sake of brevity.
- the quadrilateral image detected in the preview image is based on an opposite side angle of the quadrilateral image detected in the preview image, or from the first side to the first side in the quadrilateral image
- the sharpness distribution curve in the direction of the edge is used to predict the shooting tilt; then, the number of initial images to be captured is determined according to the tilt threshold range to which the shooting tilt belongs, and the greater the shooting tilt, the determined to be photographed The more the number of initial images, the greater the tilt of the shot, the greater the difference in sharpness between the far end and the near end of the photographed rectangular target.
- the prediction of the tilt angle of the shooting angle can be used to determine the number of shots of the initial image, and can also be used for selection. A reference to each focus position during the shooting of the initial image.
- the first photographing module 121 includes a first side determining sub-module 121a, a first area determining sub-module 121b, and a first focus determining sub-module 121c.
- the first Determining the sub-module 121a to determine a first point and a second point on the second side and the third side of the quadrilateral image, respectively, wherein the second side and the third side are adjacent to the first side a distance from the first point to a vertex where the first side intersects the second side is less than or equal to a vertex that intersects the second side and the fourth side, and the second point is to the first side a distance from a vertex intersecting the third side is less than or equal to a vertex at which the third side intersects the fourth side, the fourth side being an opposite side of the first side; the first area is determined
- the sub-module 121b determines a vertex where the first side intersects the second side, a vertex where the first side intersects
- the first side determining sub-module 121a the first area determining sub-module 121b, and the first focus determining sub-module 121c, refer to the contents and diagrams of step S121, step S122, and step S123 in the foregoing embodiment.
- the contents of the specific embodiments described in FIG. 10a and FIG. 10b are not described again.
- the second imaging module 122 includes: a second edge determining submodule 122a, a second region determining submodule 122b, and a second focus determining submodule 122c, wherein the The two-edge determining sub-module 122a determines a third point and a fourth point on the second side and the third side, respectively, wherein the third point intersects the first side and the second side The distance of the vertex is greater than or equal to the vertex that the second side intersects the fourth side, and the distance from the fourth point to the vertex where the first side intersects the third side is greater than or equal to the third a vertex intersecting the fourth side; the second area determining submodule 122b determines a vertex where the fourth side intersects the second side, and a vertex where the fourth side intersects the third side a second area formed by the third point and the fourth point connection is a partial area of one end of the first edge opposite side; the second focus determination submodule 122c select
- the implementation methods of the second edge determining submodule 122a, the second region determining submodule 122b, and the second focus determining submodule 122c are similar to the first edge determining submodule 121a, and the first region is determined.
- the implementation method of the sub-module 121b and the first focus determination sub-module 121c is not described again for the sake of brevity.
- the output module 14 includes a synthesis sub-module 141, and the synthesis sub-module 131 synthesizes a plurality of the rectangular images to obtain a final image.
- the synthesis sub-module 141 includes an analysis sub-module 141a, a region dividing sub-module 141b, and an execution sub-module 141c.
- the analysis sub-module 141a performs sharpness analysis and feature point alignment on the rectangular image;
- the area dividing sub-module 141b determines a plurality of synthesized regions based on the sharpness distribution of each of the aligned rectangular images;
- the module 141c synthesizes each of the composite regions based on the sharpness of each of the rectangular images.
- step S141a the area division sub-module, and the synthesis sub-module
- step S141b the specific contents of step S141a, step S141b, and step S141c in the foregoing embodiment, and for brevity, details are not described herein again.
- the execution sub-module comprises a first synthesis sub-unit (not shown in the drawing) and/or a second synthesis sub-unit (not shown in the drawing), the first synthesis sub-unit from each Acquiring a region with the highest resolution corresponding to the composite region in the rectangular image, and performing splicing to obtain a final image; the second synthesis subunit acquiring at least two corresponding to the composite region from each of the rectangular images The area with higher definition, with the weight as the weight, performs pixel fusion to get the final image.
- the embodiment of the synthesis process of the first synthesis subunit and/or the second synthesis subunit is described with reference to FIGS. 14a to 14c and their corresponding descriptions, and details are not described herein again.
- the shooting module includes: a control sub-module, and the control sub-module controls each of the cameras to focus on at least one initial image with different focus.
- the specific implementation method of the control sub-module is shown in FIG. 15 and FIG. 16 and its corresponding description, and details are not described herein again.
- embodiments of the present application can be provided as a method, system, or computer program product.
- the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware.
- the application can be packaged in one or more of them.
- a computer containing computer usable program code may be in the form of a computer program product embodied on a memory channel, including but not limited to disk storage and optical storage.
- the present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG.
- the computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine instruction for generating instructions executed by a processor of a computer or other programmable data processing device Means for implementing the functions specified in one or more flows of the flowchart or in a block or blocks of the flowchart.
- the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
- the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
- These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
- the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
一种拍摄图像的方法,基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;然后校正为矩形图像后,得到最终图像并输出。通过选定与被拍摄的矩形目标远端对应的部分区域中的点作为焦点进行对焦拍摄初始图像,能够提高初始图像中与所述矩形目标远端对应的部分区域的分辨率,以增加该部分区域的校正插值的来源像素,提高最终图像的清晰度。
Description
本发明涉及计算机领域,尤其涉及一种拍摄图像的方法、装置和终端。
近年来,手机或智能相机的拍照功能在快速的发展,涌现出了很多新的拍照技术,如全景拍照、全焦模式、文档校正等。
结合图1,在现有的文档校正过程的方案中,包括以下步骤:步骤S1:首先,摄像头对焦到视野中心,拍摄一张照片。通常,对焦的点为默认的图像中心。步骤S2:接着,处理器获取该照片,检测其中的四边形图像的边或角的位置。步骤S3:处理器根据预设的矩形的宽高比,把检测到的四边形内的图片像素,映射到矩形目标的区域中,映射过程中的新增像素通过原图像中的已有像素插值得到。步骤S4:根据变换后的矩形内的像素生成校正后的新图像,输出并保存新图像。
现有技术校正后的矩形文档,对图像精度要求不高的时候,可以很好的解决梯形失真问题,实用性非常强,但对于那些对图像精度要求较高的应用则不甚适宜。
图2示出倾斜拍摄矩形目标的成像抽象示意图,在拍摄过程中,结合图2,实物平面W中具有一个矩形目标,当拍摄设备观察该矩形目标时,实物平面W与成像平面C呈现的倾斜角度为α,矩形目标在成像平面C上成像为四边形图像。
当实物平面W与成像平面C倾斜角度α过大时,矩形目标并不是都位于镜头对焦的景深范围内,因而造成四边形图像在焦点附近的部分清晰,远离焦点的部分模糊;在对四边形图像进行梯形校正,以获得校正后的矩形图像的阶段,若矩形目标倾斜较大,则矩形目标的远端对应的矩形图像部分区域相对于
近端对应的矩形图像部分区域,需要更多像素是插值添加,然而,拍摄获得的初始图像在远端的分辨率往往不够,使得梯形校正阶段的插值过程,没有足够的信息来源,造成梯形校正后图像的部分区域清晰度明显下降,图4示出对图3获取的初始图像进行矩形校正后的目标图像的示意图,具体地,结合图4,当拍摄实际的矩形目标时,相对于远端对应的目标图像部分A的清晰度较差,其清晰度明显低于近端对应的目标图像部分B。
因此,现有技术在校正后的矩形文档,会出现-四边形的一侧的清晰度,明显低于四边形的另一侧的清晰度,甚至会出现文字看不清楚的情况,达不到文档校正的目的,用户体验不佳。
发明内容
有鉴于此,本发明的实施例提供了拍摄图像的方法、装置及终端,旨在解决现有技术中,在校正后的矩形文档,会出现局部清晰度局部较差、导致用户体验不佳的问题。
第一方面,根据本发明一些实施例提供的一种在带有摄像头的终端中拍摄图像的方法,其中,所述方法包括:进入拍摄模式,在预览图像中检测四边形图像;基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;将每个所述至少一张初始图像中的四边形图像校正为矩形图像;基于所述矩形图像,得到最终图像并输出。
根据一些实施例,所述方法通过选定与被拍摄的矩形目标远端对应的第一边一端的区域中的点作为焦点进行对焦拍摄初始图像,能够提高初始图像中与所述矩形目标远端对应的部分区域的分辨率,以在校正过程中增加该部分区域的校正插值的来源像素,从而能够提高拍摄图像的清晰度。
在一些具体实现方式中,所述的在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:在所述四边形图像内选定所述第一焦点,以所述第一焦点拍摄一张初始图像。
在一些具体实现方式中,所述的在所述四边形图像内选定所述第一焦点的步骤包括:在所述四边形图像的第二边和第三边上分别确定第一点和第二点,其中,所述第二边和所述第三边与所述第一边相邻,所述第一点到所述第一边与所述第二边相交的顶点的距离小于等于到所述第二边与第四边相交的顶点,所述第二点到所述第一边与所述第三边相交的顶点的距离小于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;确定所述第一边与所述第二边相交的顶点、所述第一边与所述第三边相交的顶点、所述第一点及所述第二点连线所组成的第一区域为所述第一边一端的部分区域;选定一个所述第一区域中的点为所述第一焦点。
在一些具体实现方式中,所述的选定一个所述第一区域中的点为所述第一焦点的步骤包括以下任一项:选定所述第一区域的对角连线的交点为所述第一焦点;选定所述第一区域的对边中点连线的交点为所述第一焦点。
在一些具体实现方式中,所述的选定一个所述第一区域中的点为所述第一焦点的步骤包括以下任一项:选定在所述第一区域中基于亮度分布动态确定的点为所述第一焦点;选定在所述第一区域中基于待跟踪目标动态确定的点为所述第一焦点;选定在所述第一区域中基于不均匀密度分布确定的重心点为所述第一焦点。
在一些具体实现方式中,选定在所述第一区域中基于不均匀密度分布确定的重心点为所述第一焦点包括:检测预览图像中的四边形图像;根据四边形图像的拍摄倾斜度,为四边形图像不同部位分配不同的密度,其中把密度比例设置为与切面的边长成反比例;根据密度比例及四边形图像的面积,计算出四边形图像的重心,作为第一焦点。
在一些具体实现方式中,所述的基于所述矩形图像,得到最终图像并输出的步骤包括:将所述矩形图像作为最终图像输出。
在一些具体实现方式中,所述在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:在所述四边形图像内选定至少两个焦点,所述至少两个焦点包括至少一个所述第一焦点和至少一个第二焦点,所述第二焦点不同于所述第一焦点,并分别以所述至少两个焦点对焦拍摄至少两张初始图像。
在一些具体实现方式中,所述第二焦点为在所述四边形图像内所述第一边对边一端的部分区域中选定的点。
在一些具体实现方式中,所述第二焦点的选定方法包括:在所述四边形图像的第二边和第三边上分别确定第三点和第四点,其中,所述第二边和所述第三边与所述第一边相邻,所述第三点到所述第一边与所述第二边相交的顶点的距离大于等于到所述第二边与第四边相交的顶点,所述第四点到所述第一边与所述第三边相交的顶点的距离大于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;确定所述第四边与所述第二边相交的顶点、所述第四边与所述第三边相交的顶点、所述第三点及所述第四点连线所组成的第二区域为所述第一边对边一端的部分区域;选定一个所述第二区域中的点分别作为第二焦点。
在一些具体实现方式中,所述将每个所述至少一张初始图像中的四边形图像校正为矩形图像的步骤包括:将所述至少两张初始图像中的四边形图像分别校正为至少两个矩形图像;所述的基于所述矩形图像,得到最终图像并输出的步骤包括:对所述至少两个矩形图像进行合成以得到最终图像;输出所述最终图像。
在一些具体实现方式中,所述的对所述至少两个矩形图像进行合成以得到最终图像的步骤包括:对所述至少两个矩形图像进行特征点对齐;基于对齐后的各所述矩形图像的清晰度分布,确定所述最终图像中的多个合成区域;基于各所述矩形图像的清晰度,对每一所述合成区域进行合成以得到最终图像。
在一些具体实现方式中,所述的对每一所述合成区域进行合成以得到最终图像的步骤包括至少以下任一项:从各所述矩形图像中获取一个与所述合成区
域对应清晰度最高的区域,进行拼接,以得到最终图像;从各所述矩形图像中获取至少两个与所述合成区域对应的区域,以清晰度为权重,进行像素融合,以得到最终图像。
在一些具体实现方式中,所述终端具有至少两个摄像头,则所述的并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:控制各所述摄像头,分别以不同的焦点进行对焦各拍摄至少一张初始图像。
在一些具体实现方式中,在所述的以所述焦点对焦拍摄至少一张初始图像步骤之前,所述方法还包括:根据用户在指定时间内连续输入的拍摄指令的次数,确定待获取的初始图像的数量。
在一些实施例中,所述方法通过所述装置在拍摄过程中在所检测四边形图像中,至少选择一个对应于矩形目标近端的焦点和一个对应于矩形目标远端的焦点,分别进行拍摄,获取多张初始图像,并将若干所述初始图像校正为矩形图像后,从各矩形图像中选择清晰度较高的局部,形成合成图像,并将合成图像作为最终图像输出,通过选择不同位置的焦点对焦,能够获得的局部清晰度不同的初始图像,校正后的矩形图像的局部清晰度也不同,通过对各矩形图像的最清晰局部进行合成,进一步提升最终图像的清晰度。
在一些具体实现方式中,在所述的以所述至少一个焦点对焦拍摄至少一张初始图像步骤之前,所述方法还包括:确定待拍摄的初始图像的数量。
在一些具体实现方式中,所述的确定待拍摄的初始图像的数量的步骤包括:根据所述预览图像中检测的四边形图像的第一组对边的对边夹角、或所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线,确定待拍摄的初始图像的数量。
第二方面,根据本发明一些实施例提供的一种在带有摄像头的终端中中拍摄图像的装置,其中,所述装置包括:检测模块,用于进入拍摄模式,在预览图像中检测四边形图像;拍摄模块,用于基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小
于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;校正模块,用于将每个所述至少一张初始图像中的四边形图像校正为矩形图像;输出模块,用于基于所述矩形图像,得到最终图像并输出。
在一些具体实现方式中,所述拍摄模块包括:第一拍摄子模块,用于在所述四边形图像内选定所述第一焦点,以所述第一焦点拍摄一张初始图像。
在一些具体实现方式中,所述第一拍摄子模块包括:第一边确定子模块,用于在所述四边形图像的第二边和第三边上分别确定第一点和第二点,其中,所述第二边和所述第三边与所述第一边相邻,所述第一点到所述第一边与所述第二边相交的顶点的距离小于等于到所述第二边与第四边相交的顶点,所述第二点到所述第一边与所述第三边相交的顶点的距离小于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;第一区域确定子模块,用于确定所述第一边与所述第二边相交的顶点、所述第一边与所述第三边相交的顶点、所述第一点及所述第二点连线所组成的第一区域为所述第一边一端的部分区域;第一焦点确定子模块,用于选定一个所述第一区域中的点为所述第一焦点。
在一些具体实现方式中,所述第一焦点确定子模块包括以下至少任一项:第一选定单元,用于选定所述第一区域的对角连线的交点为所述第一焦点;第二选定单元,用于选定所述第一区域的对边中点连线的交点为所述第一焦点;
在一些具体实现方式中,所述第一焦点确定子模块包括以下至少任一项:第三选定单元,用于选定在所述第一区域中基于亮度分布动态确定的点为所述第一焦点;第四选定单元,用于选定在所述第一区域中基于待跟踪目标动态确定的点为所述第一焦点;第五选定单元,用于选定在所述第一区域中基于不均匀密度分布确定的重心点为所述第一焦点。
在一些具体实现方式中,所述第一焦点确定子模块包括以下至少任一项:所述输出模块包括:第一输出子模块,用于将所述矩形图像作为最终图像输出。
在一些具体实现方式中,所述第一焦点确定子模块包括以下至少任一项:
所述拍摄模块包括:第二拍摄子模块,用于在所述四边形图像内选定至少两个焦点,所述至少两个焦点包括至少一个所述第一焦点和至少一个第二焦点,所述第二焦点不同于所述第一焦点,并分别以所述至少两个焦点对焦拍摄至少两张初始图像。
在一些具体实现方式中,所述第二焦点确定子模块包括以下至少任一项:所述第二焦点为在所述四边形图像内所述第一边对边一端的部分区域中选定的点。
在一些具体实现方式中,所述第二焦点确定子模块包括以下至少任一项:所述第二拍摄子模块包括:第二边确定子模块,用于在所述四边形图像的第二边和第三边上分别确定第三点和第四点,其中,所述第二边和所述第三边与所述第一边相邻,所述第三点到所述第一边与所述第二边相交的顶点的距离大于等于到所述第二边与第四边相交的顶点,所述第四点到所述第一边与所述第三边相交的顶点的距离大于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;第二区域确定子模块,用于确定所述第四边与所述第二边相交的顶点、所述第四边与所述第三边相交的顶点、所述第三点及所述第四点连线所组成的第二区域为所述第一边对边一端的部分区域;第二焦点确定子模块,用于选定一个所述第二区域中的点分别作为第二焦点。
在一些具体实现方式中,所述校正模块包括:校正子模块,用于将所述至少两张初始图像中的四边形图像分别校正为至少两个矩形图像;所述输出模块包括:合成子模块,用于对所述至少两个矩形图像进行合成以得到最终图像;第二输出子模块,用于输出所述最终图像。
在一些具体实现方式中,所述合成子模块包括:分析子模块,用于对所述矩形图像进行清晰度分析及特征点对齐;区域划分子模块,用于基于对齐后的各所述矩形图像的清晰度分布,确定若干合成区域;执行子模块,用于基于各所述矩形图像的清晰度,对每一所述合成区域进行合成。
在一些具体实现方式中,所述执行子模块包括至少以下任一项:第一合成子单元,用于从各所述矩形图像中获取一个与所述合成区域对应清晰度最高的
区域,进行拼接,以得到最终图像;第二合成子单元,用于从各所述矩形图像中获取至少两个与所述合成区域对应的区域,以清晰度为权重,进行像素融合,以得到最终图像
在一些具体实现方式中,所述终端具有至少两个摄像头时,所述拍摄模块包括:控制子模块,用于控制各所述摄像头,分别以不同的焦点进行对焦各拍摄至少一张初始图像。
在一些具体实现方式中,所述装置还包括:选定模块,用于在以所述至少一个焦点对焦拍摄至少一张初始图像之前,确定待拍摄的初始图像的数量。
在一些具体实现方式中,所述选定模块包括:选定子模块,用于根据所述预览图像中检测的四边形图像的第一组对边的对边夹角、或所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线,确定待拍摄的初始图像的数量。
第三方面,根据本发明一些实施例提供的一种终端,包括:至少一个摄像头;一个显示单元;一个或多个处理器;存储器;多个应用程序;所述存储器用于存储计算机程序;所述处理器,运行所述计算机程序,执行下述流程:进入拍摄模式,在预览图像中检测四边形图像;基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;将每个所述至少一张初始图像中的四边形图像校正为矩形图像;基于所述矩形图像,得到最终图像并输出。
在一些具体实现方式中,所述终端能够结合第一方面中本发明各实施例所提供的方法执行流程。
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所
需要使用的附图作简要地介绍。显而易见地,下面描述中的附图仅仅是本发明的一些实施例,而非全部。对于本领域普通技术人员来讲,在没有付出创造性劳动的前提下,还可以根据这些附图获得其它的附图。
图1示出现有技术中一种图像获取方法的流程示意图;
图2示出倾斜拍摄矩形目标的成像抽象示意图;
图3示出基于现有技术的一种图像获取方法在图像获取过程中倾斜拍摄矩形目标的示意图;
图4示出对图3获取的初始图像进行矩形校正后的目标图像的示意图;
图5示出根据一些实施例提供的终端的结构示意图;
图6示出根据一些实施例提供的Android框架中的模块功能示意图;
图7示出根据一些实施例提供的拍摄图像的方法流程示意图;
图8示出根据一些实施例提供的预览图像中四边形图像的示意图;
图9示出根据一些实施例提供的预览图像中检测四边形图像清晰度的示意图;
图10a~图10e示出根据一些实施例提供的拍摄过程中选定焦点的方法示意图;
图11示出根据一些实施例提供的一种矩形校正时进行像素插值的方法示意图的方法示意图;
图12示出根据一些拍摄图像的方法流程示意图;
图13a~图13c示出根据一些实施例提供的对两张四边形图像进行特征点对齐前后的简要示意图;
图14a~图14g示出根据一些实施例提供的基于两张校正后的四边形图像的清晰度分布划分的合成区域的简要示意图;
图15示出根据一些实施例提供的一种具有两个摄像头的终端的结构示意图;
图16示出根据一些实施例提供的利用图15所示终端进行拍摄时焦点选择示意图的简要示意图;
图17示出根据一些具体实施例中拍摄图像的简要流程示意图;
图18示出根据一些具体实施例中拍摄图像的简要流程示意图;
图19a~图19c示出根据一些实施例具体实施例的拍摄图像的方法获得初始图像及校正后图像的示意图;
图20示出根据一些实施例提供的拍摄图像的装置的结构示意图;
图21示出根据一些实施例提供的第一拍摄子模块的结构示意图;
图22示出根据一些实施例提供的第二拍摄子模块的结构示意图;
图23示出根据一些实施例提供的合成子模块的结构示意图。
为使本发明的目的、技术方案和优点更加清楚,下面将结合附图对本发明实施方式做出进一步地详细描述。对于本领域普通技术人员在没有付出创造性劳动的前提下所获得的其它所有实施例,都属于本发明保护的范围。
在一些实施例中,本发明实施例的技术方案可用于能够拍摄图像的终端,所述终端可为所有具有拍照功能以及图像处理功能的电子设备,所述电子设备可为智能相机、移动电话、平板电脑(Tablet Personal Computer)、膝上型电脑(Laptop Computer)、数码相机、数字摄影机、投影设备、个人数字助理(personal digital assistant,简称PDA)、电子书阅读器(英文:e-book reader)或可穿戴式设备(Wearable Device)等,对此本发明实施例不做进一步限定。
图5示出了根据一些实施例提供了一种终端的结构示意图,所述终端可用于执行实施例一种所述的方法。
该终端可以为包括手机、平板电脑、PDA(Personal Digital Assistant,个人数字助理)、POS(Point of Sales,销售终端)、车载电脑、数码相机等终端设备,参考图5,终端500包括RF(Radio Frequency,射频)电路510、存储器520、输入单元530、显示单元540、传感器550、音频电路560、WiFi(wireless fidelity,无线保真)模块570、处理器580、以及电源590等部件。本领域技术人员可以理解,图5中示出的终端结构只做实现方式的举例,并不构成对终端的限定,
可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。
下面结合图5对终端500的各个构成部件进行具体的介绍:
RF电路510可用于收发信息或通话过程中,信号的接收和发送,特别地,将基站的下行信息接收后,给处理器580处理;另外,将设计上行的数据发送给基站。通常,RF电路包括但不限于天线、至少一个放大器、收发信机、耦合器、LNA(Low Noise Amplifier,低噪声放大器)、双工器等。此外,RF电路510还可以通过无线通信与网络和其他设备通信。所述无线通信可以使用任一通信标准或协议,包括但不限于GSM(Global System of Mobile communication,全球移动通讯***)、GPRS(General Packet Radio Service,通用分组无线服务)、CDMA(Code Division Multiple Access,码分多址)、WCDMA(Wideband Code Division Multiple Access,宽带码分多址)、LTE(Long Term Evolution,长期演进)、电子邮件、SMS(Short Messaging Service,短消息服务)等。对于通信用的终端可包含RF电路,对于非通信终端,如数码相机,可不包含RF电路。
存储器520可用于存储软件程序以及模块,处理器580通过运行存储在存储器520的软件程序以及模块,从而执行终端500的各种功能应用以及数据处理。存储器520可主要包括存储程序区和存储数据区,其中,存储程序区可存储操作***、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等;存储数据区可存储根据终端500的使用所创建的数据(比如音频数据、电话本等)等。此外,存储器520可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
输入单元530可用于接收输入的数字或字符信息,以及产生与终端500的用户设置以及功能控制有关的键信号输入。具体地,输入单元530可包括触控面板531、至少一个摄像头532以及其他输入设备533。触控面板531,也称为触摸屏,可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板531上或在触控面板531附近的操作),并根据预先设定的程式驱动相应的连接装置。可选的,触控面板531可包括触摸检测
装置和触摸控制器两个部分。其中,触摸检测装置检测用户的触摸方位,并检测触摸操作带来的信号,将信号传送给触摸控制器;触摸控制器从触摸检测装置上接收触摸信息,并将它转换成触点坐标,再送给处理器580,并能接收处理器580发来的命令并加以执行。此外,可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板531。除了触控面板531,输入单元530还包括至少一个摄像头532,其中,所述摄像头532可以基于触控设备接收的触摸信息执行拍摄工作,再将所获取的拍摄图像传送给处理器580进行相应处理,其中,所述摄像头532可以是数字摄像头、模拟摄像头,其成像色彩可以是彩色、黑白、灰度等,摄像头532可以是直接安装于终端上的,也可以是通过数据线外接的。此外,除了触控面板531和摄像头532,输入单元530还可以包括:其他输入设备533。具体地,其他输入设备533可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆等中的一种或多种。
显示单元540可用于显示由用户输入的信息或提供给用户的信息以及终端500的各种菜单。显示单元540可包括显示面板541,可选的,可以采用LCD(Liquid Crystal Display,液晶显示器)、OLED(Organic Light-Emitting Diode,有机发光二极管)等形式来配置显示面板541。进一步的,触控面板531可覆盖显示面板541,当触控面板531检测到在其上或附近的触摸操作后,传送给处理器580以确定触摸事件的类型,随后处理器580根据触摸事件的类型在显示面板541上提供相应的视觉输出。虽然在图5中,触控面板531与显示面板541是作为两个独立的部件来实现终端500的输入和输入功能,但是在某些实施例中,可以将触控面板531与显示面板541集成而实现终端500的输入和输出功能。
终端500还可包括至少一种传感器550,比如光传感器、运动传感器以及其他传感器。具体地,光传感器可包括环境光传感器及接近传感器,其中,环境光传感器可根据环境光线的明暗来调节显示面板541的亮度,接近传感器可在终端500移动到耳边时,关闭显示面板541和/或背光。作为运动传感器的一
种,加速计传感器可检测各个方向上(一般为三轴)加速度的大小,静止时可检测出重力的大小及方向,可用于识别终端姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等;至于终端500还可配置的陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器,在此不再赘述。
音频电路560、扬声器561,传声器562可提供用户与终端500之间的音频接口。音频电路560可将接收到的音频数据转换后的电信号,传输到扬声器561,由扬声器561转换为声音信号输出;另一方面,传声器562将收集的声音信号转换为电信号,由音频电路560接收后转换为音频数据,再将音频数据输出处理器580处理后,经RF电路510以发送给比如另一终端,或者将音频数据输出至存储器520以便进一步处理。
WiFi属于短距离无线传输技术,终端500通过WiFi模块570可以帮助用户收发电子邮件、浏览网页和访问流式媒体等,它为用户提供了无线的宽带互联网访问。虽然图5示出了WiFi模块570,但是可以理解的是,其并不属于终端500的必须构成,完全可以根据需要在不改变发明的本质的范围内而省略。
处理器580是终端500的控制中心,利用各种接口和线路连接整个终端的各个部分,通过运行或执行存储在存储器520内的软件程序和/或模块,以及调用存储在存储器520内的数据,执行终端500的各种功能和处理数据,从而对终端进行整体监控。可选的,处理器580可包括一个或多个处理单元;优选的,处理器580可集成应用处理器和调制解调处理器,其中,应用处理器主要处理操作***、用户界面和应用程序等,调制解调处理器主要处理无线通信。可以理解的是,上述调制解调处理器也可以不集成到处理器580中。
终端500还包括给各个部件供电的电源590(比如电池),优选的,电源可以通过电源管理***与处理器580逻辑相连,从而通过电源管理***实现管理充电、放电、以及功耗管理等功能。
尽管未示出,终端500还可以包括蓝牙模块等,在此不再赘述。
在本发明实施例中,该终端所包括处理器具有以下功能:进入拍摄模式,
在预览图像中检测四边形图像;基于拍摄指令,拍摄至少一张初始图像,其中,所述初始图像包括至少一张以所述四边形图像内靠近较短边的部分区域中的点为第一焦点所拍摄的图像;将所述初始图像中的四边形图像校正为矩形图像;当具有多张初始图像时,基于校正后的各所述矩形图像的清晰度分布,将若干所述矩形图像合成为清晰的最终图像。
该终端实现上述功能可以采用本申请中所提及的前述实施例的具体方法实现,为简明起见,不再赘述,并以引用的方式包含此。
图6示出,在Android***中的架构,文档校正模块是与拍照相关的一个功能,算法和对摄像头的操作在framework层(软件框架层)实现。
从上至下分为应用层、应用框架层、硬件抽象层和内核。应用层主要提供摄像头应用,控制摄像头获取预览图像,及拍摄初始图像;在应用框架层中,在预览图像中检测四边形图像,并基于文档校正算法,将初始图像校正为矩形图像,当有多张矩形图像时进行特征对齐和合成,形成最终图像;内核中的摄像头驱动是控制具体摄像头动作的程序,比如控制摄像头的对焦、拍照等动作,不同厂商和型号的摄像头,驱动程序是不同的。摄像头驱动是应用层的预览、对焦、拍照、调整拍照参数的最终执行模块,直接控制摄像头硬件。硬件抽象层是对内核驱动程序的封装,向上提供接口,屏蔽低层的实现细节,其目的在于将硬件抽象化,它隐藏了特定平台的硬件接口细节,为操作***提供虚拟硬件平台,使其具有硬件无关性,可在多种平台上进行移植。硬件抽象层是一个编程层,允许操作***在逻辑层而不是硬件层与硬件设备交互,操作***核心或者硬件驱动程序都可以调用硬件抽象层。无论哪种情况,调用程序都不用了解硬件的具体设计细节,只需要给出抽象层所需的参数即可。硬件抽象层是连接应用框架层与摄像头驱动程序的桥梁,是一个代理,把应用层和应用框架层对摄像头的控制指令,转给正确的硬件驱动去执行动作,并返回从硬件获取的数据给应用层和应用框架层。
图7示出根据一些实施例提供的拍摄图像的方法,所述方法可应用于文档校正(或称为梯形校正)的拍摄模式,用于提升文档校正中校正后图像的清晰
度,其中,所述的文档校正是一种通过对四边形图像的检测、变换、处理,输入是图像,输出也是图像,用于帮助用户在拍摄矩形目标时,校正由于拍摄角度倾斜导致图像变形,最终输出矩形最终图像。在此,所述矩形目标泛指矩形的含有信息的图像,例如:文件、***、书本、名片、证件、讲义、照片、广告、展板、电视、电影、屏幕等等,当然,本领域技术人员应当能够理解,上述矩形目标仅为举例,其他矩形物体被拍摄为图像时,都可以应用本申请所述方法进行拍摄和校正。
结合图7,根据一些实施例所提供的方法包括:步骤S11、步骤S12、步骤S13和步骤S14。结合图7,在步骤S11中,进入拍摄模式,在预览图像中检测四边形图像;接着,在步骤S12中,基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;然后,在步骤S13中,将每个所述至少一张初始图像中的四边形图像校正为矩形图像;之后,在步骤S14中,基于所述矩形图像,得到最终图像并输出。
在一些实施例中,所述终端选择拍摄一张初始图像,则在所述步骤S12中,所述终端基于拍摄指令,在所述四边形图像内第一边一端的部分区域中选定点为第一焦点,并以所述第一焦点对焦拍摄一张初始图像,其中,所述第一边为所述四边形图像的两个对边夹角中具有较小对边夹角的两对边中较短的一边;然后,在所述步骤S13中,所述终端将所述初始图像中的四边形图像校正为矩形图像;随后,在所述步骤S14中,所述终端将所述矩形图像作为最终图像输出。输出最终图像可以包括直接在显示单元显示最终图像,或者保存到拍照图像的存储空间中,根据用户指示在显示单元上显示最终图像。
根据一些实施例,所述方法通过选定与被拍摄的矩形目标远端对应的第一边一端的区域中的点作为第一焦点,进行对焦拍摄初始图像,能够提高初始图
像中与所述矩形目标远端对应的部分区域的分辨率,以在校正过程中增加该部分区域的校正插值的来源像素,从而能够提高最终图像的清晰度。
具体地,在所述步骤S11中,所述终端根据用户相应操作,启动拍摄设备(例如摄像头等),进入拍摄模式。
在一些实施例中,用户相应的操作可以包括用户对设备进行任何可识别的操作,可以包括至少以下任一项:对设备的触控设备的任何指定的触摸操作(点击、滑动等)、设备可识别的声控操作、手势操作等。
在一些实施例中,所述拍摄模式可以是根据用户选择的文档校正的拍摄模式。
接着,在拍摄模式下,所述终端通过摄像头获取预览图像,并在预览图像中检测四边形图像。在一些实施例中,终端在获取预览图像时,其对焦的焦点可以是预览图像的中央区域的点(例如中心点),并且是随着取景位置的变化,持续的对焦的。在一些实施例中,对焦的焦点还可以是待跟踪目标动态确定的点(例如人眼中心点等)。
在一些实施例中,所述终端实时地获取摄像头采集的图像,并将所获取的图像呈现为预览图像,所述预览图像可以显示在终端的显示设备(例如显示屏)上。当所述摄像头对向矩形目标时,则预览图像中呈现所述矩形目标的图像,通常,摄像头所在平面并非平行于矩形目标所在的平面,而是倾斜于被拍摄的矩形目标所在的平面,所述矩形目标在预览图像中则呈现四边形图像。通常拍摄倾斜度越大,矩形目标呈现的四边形图像的形变越大。
在一些实施例中,所述终端在预览图像中检测四边形图像,可以通过一些检测方法检测四边形图像的边和角的位置,检测边和角的位置的方法例如,采用Canny算子边缘检测方法标识出四边形图像中的实际边缘,其中,所述Canny算子边缘检测方法基于边缘检测计算理论(Computational theory of edge detection)、Hough变换检测技术(使用表决原理的参数估计技术)检测直线,哈里斯角点检测(Harris Conner Points)等等。本领域技术人员应当能够理解,其他对四边形结构的边和角的位置的检测方法如适用于本发明的实施例,虽不
再赘述,仍能够以引用的方式包含于本发明的实施例中。
接着,在所述步骤S12中,包括步骤S121,在所述步骤S121中,基于拍摄指令,在所述四边形图像内第一边一端的部分区域中选定点为第一焦点,并以所述第一焦点对焦拍摄初始图像,其中,所述第一边为所述四边形图像的、具有较小的对边夹角的两对边中、较短的一边。
在此,所述的拍摄指令可以是进入拍摄模式后由终端自动触发,如在文档校正模式中,当在预览图像中检测到四边形图像时自动触发拍摄初始图像;也可以是根据用户操作输入确定。所述初始图像是基于拍摄指令由摄像头采集所获取的、未进行矩形校正和/或合成的图像。
基于倾斜拍摄的原理,对边夹角较小的两个对边为拍摄距离相差较大的两边,并基于近大远小原理,对边夹角较小的两个对边中,较短边,即所述第一边对应所述矩形目标最远端的边,较长边,即所述第一边对边(第四边)对应所述矩形目标最近端的边。
在一些实施例中,所述第一边的确定方法具体包括:首先,确定四边形图像的四边;接着,确定两个对边夹角,其中,具有较小的对边夹角的两对边中,较短的一边即为与矩形目标最远端边相对应的所述第一边,较长的、也是与所述第一边相对的一边即为与矩形目标最近端边相对应的边。对边夹角是两个对边延长线的夹角,或者平行移动一个边使其与对边相交产生的夹角。
结合图8,在一具体实施例中,检测到预览图像中的四边形图像ABCD,对边夹角为对边所形成的夹角,如边AB与边CD的对边夹角β1,边AD与边BC的对边夹角β2,其中,β1小于β2,且边AB小于边CD,则确定较短边的边AB为第一边,边CD为第一边对边。在一些实施例中,四边形图像两对边平行时,则认为对边夹角为0°。所述四边形图像ABCD内第一边AB一端的部分区域对应于所述矩形目标远端的部分区域中,则在所述四边形图像ABCD内第一边AB一端的部分区域中选定点为第一焦点,即为将第一焦点设定在对应所述矩形目标的较远的一端。
在一些实施例中,确定第一边一端的部分区域的具体步骤可以包括:在步
骤S121a中,在所述四边形图像的第二边和第三边上分别确定第一点和第二点,所述第一点到所述第一边与所述第二边相交的顶点的距离小于等于到所述第二边与第四边相交的顶点,所述第二点到所述第一边与所述第三边相交的顶点的距离小于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;在步骤S121b中,所述第一边与所述第二边相交的顶点、所述第一边与所述第三边相交的顶点、所述第一点及所述第二点连线所组成的第一区域形成所述第一边一端的部分区域。
继续参考图8,在一些实施例中,所述四边形图像ABCD内第一边AB一端的部分区域可以是沿着四边形图像ABCD的第二边AD上的某点E(该点E相比于第四边CD更靠近第一边AB)到第三边BC上的更靠近第一边AB某点F(该点F相比于第四边CD更靠近第一边AB)的连线切割的、第一边一端的部分区域ABFE。
然后,在所述第一区域中选择第一焦点为步骤S121c,在一些优选的实施例中,选定第一焦点的方式是:选定所述第一区域的对角连线的交点为所述第一焦点;选定所述第一区域的对边中点连线的交点为所述第一焦点;选定在所述第一区域中基于亮度分布动态确定的点为所述第一焦点;选定在所述第一区域中基于待跟踪目标动态确定的点为所述第一焦点;选定在所述第一区域中基于不均匀密度分布确定的重心点为所述第一焦点;等等。
在一些实施例中,参考图10a,确定第二边AD和第三边BC的中点F和中点E,确定第一区域ABEF。然后,在第一区域ABEF中选择点作为第一焦点。结合图10a和图10b,在一些实施例中,在第一区域ABEF中选定两两对边中点连线的交点G1、或两条对角线的连线的交点连线的交点G2作为第一焦点等等。在一些实施例中,还可以在第一区域ABEF中,根据亮度分布、待跟踪目标等等动态确定一个点作为第一焦点。其中,所述待跟踪目标可以是,例如四边形图像中重点标示的目标(例如人脸等)。在其他实施例中,还可以确定第二边AD和第三边BC上靠近第一边AB的三分点F’和E’,并确定第一区域ABE’F’等等。
在一些实施例中,如图10c所示,选定第一焦点的方式还可以是,在四边形图像ABCD中,基于不均匀密度分布的重心计算,具体步骤包括:(1)检测预览图像中的四边形图像ABCD;(2)根据四边形图像的拍摄倾斜度,为四边形图像ABCD不同部位分配不同的密度,例如,沿预览图像的水平方向,从第一边AB向第四边CD,把密度比例设置为与切面的边长成反比例(或者更高次幂的比例),再沿预览图像的垂直方向从第二边AD向第三边BC,同样把密度比例设置为与切面的边长成反比例(或者更高次幂的比例),图10c中,第一边AB与第四边CD的对边夹角为0°,沿预览图像的垂直方向从第二边AD向第三边BC方向上的密度比例一致;(3)根据密度比例及四边形图像ABCD的面积,计算出四边形图像ABCD的重心,作为第一焦点G。在此,根据重心原理,采用不均匀密度分布的重心计算的重心点同样是靠近第一边一端的区域中的点。
根据一些实施例,在所述步骤S12中,所述终端通过在所述四边形图像中选定靠近与所述矩形目标远端的部分区域的点,尽量提高所述四边形图像对应矩形目标的远端的部分区域的清晰度,以弥补所述四边形图像中对应所述矩形目标远端的部分区域的分辨率不足的情况,提升所述初始图像在适当景深位置的清晰度,进而提高最终图像的整体清晰度和视觉效果。
接着,在所述步骤S13中,所述终端将所述初始图像中的四边形图像校正为矩形图像。
在一些实施例中,进行校正的方法可以是,首先,基于四边形图像的像素分布生成关于像素点位置坐标(x,y)的矩阵M,根据所述四边形图像和待校正的矩形图像的尺寸和形状对应关系,确定四边形图像到矩形图像的变换矩阵,并通过矩阵M与变换矩阵进行运算,计算出像素点在新图像中对应的位置(x’,y’)。若像素点的位置落在整数坐标上,则将该像素点的像素值作为该整数坐标的像素值;若像素点的位置不是落在整数坐标上,则利用该整数坐标邻近的若干像素点进行插值计算,获得该整数坐标的像素值,生成新的像素点,根据新的像素点位置坐标生成矩阵T,根据矩阵T生成校正后的矩形图像。
其中,插值方法可以采用线性插值法,以线性函数为插值函数的插值方法。
二维图像通常使用双线性插值,双线性插值是有两个变量的插值函数的线性插值扩展,通过在两个方向分别进行一次线性插值获得。结合图11,以从4个像素点插值到9个像素点为例:4个像素的值分别为如(a)所示{10,20,30,40},若放大到如(b)所示的9个像素,可把9个像素缩小覆盖到4个像素上,每个像素按照覆盖到的原像素的面积,取不同的权重来组合,如(c)所示,其中,a,c,g,i只含有一个像素,像素值等于其下覆盖的像素;b,d,f,h跨越了两个像素,面积相等则权重相等,两个像素的值按权重融合;e覆盖了4个像素,面积相等,权重相等,取四个像素按均值融合;最终如(d)所示的9个像素值为{10,15,20,20,25,30,30,35,40}。
此外,其他图像插值方法,例如最临近插值、线性插值、双线性插值、立方卷积插值、双三次插值等,如适用本发明一些实施例的,虽不再赘述,也能够以引用的方式应用于本发明一些实施例中。
在一些实施例中,在步骤S13中,将初始图像中的四边形图像校正为矩形图像后,可以将初始图像中的四边形图像以外的区域图像删除。
在一些实施例中,在步骤S13中,将初始图像中的四边形图像校正为矩形图像后,初始图像中的四边形图像以外的区域图像可以保留,并根据四边形图像的变化进行相应调整;在一些实施例中,初始图像中的四边形图像以外的区域图像,可以在后续步骤中继续根据矩形图像的对齐和合成过程继续进行相应调整,并保留在最终图像中输出。
在一些实施例中,初始图像中的四边形图像以外的区域图像也可以在后续某一步骤中删除或者在校正为矩形图像之前删除。
在一些实施例中,所述终端选择拍摄多张所述初始图像,并对所述初始图像进行矩形矫正、合成和输出。具体地,结合图12,在步骤S11中,进入拍摄模式,在预览图像中检测四边形图像;在所述步骤S12中,所述终端基于拍摄指令,在所述四边形图像内选定至少两个焦点,所述至少两个焦点包括至少一个所述第一焦点和至少一个第二焦点,并分别以所述焦点对焦拍摄至少两张初始图像,其中,所述焦点包括至少一个在所述四边形图像内第一边一端的部分
区域中选定的点和至少一个在所述四边形图像内所述第一边对边一端的部分区域中选定的点,所述第一边为所述四边形图像的、具有较小的夹角的两对边中、较短的一边;在所述步骤S13中,所述终端将所述至少两张初始图像中的四边形图像校正为矩形图像;在所述步骤S14中,包括步骤S141和步骤S142,在所述步骤S141中,所述终端对多张所述矩形图像进行合成,以得到最终图像,在所述步骤S142中,输出所述最终图像。输出最终图像可以包括直接在显示单元显示最终图像,或者保存到拍照图像的存储空间中,根据用户指示在显示单元上显示最终图像。
在一些实施例中,所述方法通过所述终端在拍摄过程中在所检测四边形图像中,至少选择一个第二焦点和一个对应于矩形目标远端的第一焦点,分别进行拍摄,获取多张初始图像,并将若干所述初始图像校正为矩形图像后,从各矩形图像中选择清晰度较高的局部,形成合成图像,并将合成图像作为最终图像输出,第二焦点和第一焦点不同,通过选择不同位置的焦点对焦,能够获得的局部清晰度不同的初始图像,校正后的矩形图像的局部清晰度也不同,通过对各矩形图像的最清晰局部进行合成,进一步提升最终图像的清晰度。可选地,第二焦点为对应于矩形目标近端的焦点。
在此,所述步骤S11的内容与图7所示的步骤S11的内容相同或基本相同,为简明起见,不再赘述,并以引用的方式包含于此。
接着,根据一些实施例,在所述步骤S12中,在图7所示的步骤S12的基础上,所述终端基于拍摄指令,拍摄多张初始图像。多张初始图像中,不仅包括至少一张在所述四边形图像内第一边一端的部分区域中选定点为第一焦点对焦拍摄的图像,还包括至少一张在所述四边形图像内第一边对边一端的部分区域中选定点为第二焦点对焦拍摄的图像。
其中,待拍摄的初始图像是数量不被限定,例如可以是一张、两张、三张、四张、八张等等。
在一些实施例中,待拍摄的初始图像的数量,可以是程序预先设定的,或由根据用户选定操作确定。例如,程序自动设定为每当用户每输入一次拍摄指
令(例如单击拍摄按键),则拍摄两张初始图像;再例如,在每次进入文档校正的拍摄模式时,向用户提供选择提示,以由用户选定拍摄初始图像的数量,随后每当用户每输入一次拍摄指令(例如单击拍摄按键),则拍摄相应数量的初始图像。
在一些实施例中,根据用户拍摄指令(拍摄单张还是连拍多张的指令)确定初始图像的拍摄数量。
在一些实施例中,所述方法还包括,在拍摄初始图像之前,先确定待拍摄的初始图像的数量。其中,待拍摄的初始图像的数量可以根据预览图像中所述四边形图像的成像情况,确定待拍摄的初始图像的数量,其中,预览图像中所述四边形图像的成像情况可以为但不限于:预览图像中所述四边形图像的形变程度,或预览图像中所述四边形图像的清晰度差异。若所述四边形图像形变越小或校正后清晰度差异越小,则确定拍摄较少数量的初始图像,若所述四边形图像形变越大或清晰度差异越大,则确定拍摄较多数量的初始图像,例如拍摄两张、四张甚至更多张。
在一些实施例中,分析预览图像中所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线;然后,根据清晰度分布曲线确定待拍摄的初始图像的数量。通常,清晰度分布曲线的变化范围越大,其拍摄倾斜度越大,被拍摄的矩形目标远端和近端的清晰度差别越大,则通过增加初始图像的拍摄数量,并结合选择不同的焦点位置进行拍摄,能够获得更多清晰的局部区域,从而在后续合成阶段提高最终图像的整体清晰度。
在一些实施例中,结合图8,根据四边形图像的形变情况确定待拍摄的初始图像的数量,可以为根据四边形图像的对比夹角来确定,具体包括:(1)首先检测预览图像中的四边形图像ABCD;(2)求出四边形图像ABCD对边夹角;(3)例如,若任一夹角β大于预设夹角阈值时,其中,预设夹角阈值可取10°~20°,例如15°,则可推测拍摄角度的倾斜度较大,则可确定拍摄两张或更多张初始图像,两两对边沿线的夹角越大,可以增加初始图像的拍摄数量的。可选地,可以判断较小的对边夹角是否大于预设夹角阈值。
在一些实施例中,分析预览图像中所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线;然后,根据清晰度分布曲线确定待拍摄的初始图像的数量。通常,清晰度分布曲线的变化范围越大,其拍摄倾斜度越大,被拍摄的矩形目标远端和近端的清晰度差别越大,则通过增加初始图像的拍摄数量,并结合选择不同的焦点位置进行拍摄,能够获得更多清晰的局部区域,从而在后续合成阶段提高最终图像的整体清晰度。结合图9,根据清晰度分布判断拍摄张数的过程包括:(1)检测预览图像中的四边形图像ABCD;(2)求出预览图像沿第一方向X上(即第一边AB向第四边CD方向),四边形图像ABCD中清晰度曲线;(3)若在四边形图像的范围内,基于清晰度曲线的变化范围,确定拍摄初始图像的张数,变化范围为清晰度最大值和清晰度最小值的差值,以100满分为例,变化范围在30~40时确定拍摄两张初始图像,变化范围在40~50时确定拍摄三张初始图像,变化范围在50~80时确定拍摄四张初始图像,。在具体场景实施例中,拍摄倾斜度的相关阈值可以根据不同拍摄场景下,不同的拍摄条件和拍摄要求进行具体设定。
此外,预览图像中所述四边形图像的成像情况,可以用于确定拍摄初始图像的张数、还可以用于选择初始图像的拍摄过程中各焦点位置的参考等。
当确定待拍摄的初始图像的张数后,选定与待获取的初始图像的张数对应数量的焦点。在所述四边形图像内第一边一端的部分区域中选定点为第一焦点的方法可参考图7所示的步骤S12中的选定方法,在此不再赘述。
在一些实施例中,在所述四边形图像内第一边对边(即第四边)一端的部分区域中选定点为第二焦点的方法可以包括:在所述第二边和所述第三边上分别确定第三点和第四点,其中,所述第三点到所述第一边与所述第二边相交的顶点的距离大于等于到所述第二边与所述第四边相交的顶点,所述第四点到所述第一边与所述第三边相交的顶点的距离大于等于到所述第三边与所述第四边相交的顶点;确定所述第四边与所述第二边相交的顶点、所述第四边与所述第三边相交的顶点、所述第三点及所述第四点连线所组成的第二区域为所述第一边对边一端的部分区域;选定至少一个所述第二区域中的点分别作为第二焦点。
在一些实施例中,所述第三点可以与第一点重合,所述第四点可以与第一点重合。
在一些实施例中,若需拍摄三张或者更多张的初始图像时,除在第一区域或第二区域中确定点作为焦点外,再选择的焦点的位置可以不限于在第一区域或第二区域内,优选每次选择不同的焦点位置,以获取不同局部地区的清晰图像。
在一些实施例中,在第二区域中选定第二焦点的方式可以参考在所述第一区域中选定第一焦点的方式。
具体地,例如,参考图10d,在拍摄两张初始图像时,对于四边形图像ABCD,与第四边CD相交的第二边AD和第三边BC的中点分别为点F和点E,则选择选择靠近第一边AB的部分四边形区域ABEF和靠近第四边CD的部分四边形区域CDFE。
在另一实施例中,参考图10e,第二边AD上两个三分点F1和F2,第三边BC上两个三分点E1和E2,将四边形区域划分为三个部分区域ABE1F1、F1E1E2F2及F2E2CD,分别取三个部分区域中的对边中点连线的交点,确定三个焦点G1、G2和G3。
此外,所述待拍摄的初始图像的数量也可以根据用户在指定时间内连续输入拍摄指令(例如长按拍照按键以进行的连拍方式),确定待获取的初始图像的数量。由于拍摄的初始图像的数量是动态的,在预先未知初始图像数量的情况下,可以在拍摄第一张时,选定以所述四边形图像内靠近较短边的部分区域中的点为第一焦点,在拍摄第二张时,选定以所述四边形图像中靠近较长边的部分区域中的点为第二焦点,在后续拍摄每张照片时,可以交替四边形图像内靠近较短边的部分区域中的点和所述四边形图像中靠近较长边的部分区域中的点作为焦点,但每次选定的焦点与前任一次拍摄的焦点不同。
通过以多个不同的焦点进行对焦拍摄多张初始图像,焦点附近的图像区域较为清晰,选择越多不同的焦点,越能够获得所述四边形图像的更多清晰的局部区域,进而能够帮助提高最终图像的清晰度。
在步骤S13中,将每张所述初始图像中的四边形图像校正为矩形图像。所述步骤S13的对每张初始图像进行校正的具体内容可以参考图7所示的步骤S13的具体内容,不再赘述。
接着,在一些实施例中,所述步骤S141包括以下步骤:步骤S141a,对所述矩形图像进行特征点对齐;步骤S141b,基于对齐后的各所述矩形图像的清晰度分布,确定若干合成区域;步骤S141c,基于各所述矩形图像的清晰度,对每一所述合成区域进行合成。
其中,所述终端基于各所述矩形图像的清晰度,对每一所述合成区域进行合成的步骤采用:从各所述矩形图像中获取一个与所述合成区域对应清晰度最高的区域,进行拼接,以得到最终图像;或,从各所述矩形图像中获取至少两个与所述合成区域对应区域,以清晰度为权重,进行像素融合,以得到最终图像。
在一些实施例中,当拍摄多张初始图像时,每次拍摄对四边形图像进行边缘检测检测到的矩形目标的边缘(包括边、角)可能不同,因此不同的初始图像中的四边形图像存在位移、尺寸、旋转角度略有不同的情况,则每张初始图像校正后的矩形图像需要基于矩形图像的特征点进行进一步对齐,以利于后续合成步骤。
具体地,将所有校正后的矩形图像进行图像对齐(Image Stiching),至少两个矩形图像之间会有平移、旋转、仿射等变换关系,根据特征点的相对位置关系,可以计算出变换矩阵,把至少两个矩形图像的特征点对齐,矩形图像也就对齐了。在一些实施例中,对齐过程包括:
(1)以任一张矩形图像为基准图像,其他矩形图像与基准图像两两特征点对齐,对两两矩形图像进行特征点提取和描述包括:提取一系列特征点,其提取方法可以包括:尺度不变特征变换法(SIFT,Scale-invariant feature transform),SURF算法(Speeded Up Robust Features)、哈里斯(Harris)角点检测算法等,其他本领域技术人员熟知的特征点提取方法,如适用本申请的,仍能够以引用的方式包含于本发明的思想范围之内;
(2)结合图13a和图13b,以图13a的矩形图像T1为基准图像,对图13b的矩形图像T2进行对齐,具体地,将矩形图像T1和矩形图像T2的特征点进行匹配,找出特征匹配点X1、X2、X3和X4,及对应X1’、X2’、X3’和X4’,其中,所述特征匹配点是像素点的抽象的标记,表示在矩形图像中的位置和特征值,特征点可以是矩形图像中拐角、分叉、边缘等位置标记;
(3)根据特征匹配点,计算两个矩形图像T1和T2的相对转移矩阵,所述相对转移矩阵里表示的是每个特征匹配点进行对齐变换的旋转、平移、缩放的变换参数,例如,在一具体场景的实施例中,矩形图像T1中,特征匹配点X1、X2、X3和X4的坐标分别为:(211,224),(577,240),(209,466),(545,511),矩形图像T2中,特征匹配点X1’、X2’、X3’和X4’的坐标分别为:(213,215),(579,261),(192,450),(525,527),
则对应的相对转移矩阵为:
(4)根据相对转移矩阵,把其中一矩形图像T2进行平移、旋转、缩放、仿射等变换,与另一矩形图像T1对齐,形成对齐后的矩形图像T2’,矩形图像T1和对齐后的矩形图像T2’如图13c所示。
在一些实施例中,当拍摄多张初始图像时,对四边形图像进行边缘检测检测到的矩形目标的边缘(包括边、角)在误差范围内,并且校正的宽高比例也在误差范围内,则每张初始图像校正后的矩形图像可以无需进行对齐。
接着,分析每一矩形图像的清晰度分布情况,根据清晰度分布情况,确定至少一条拼接线或融合区域,基于拼接线的划分,从各矩形图像中选择清晰度较高的局部图像进行拼接,或在所述融合区域中,基于所述矩形图像的清晰度,将相应矩形图像的像素进行融合,从而获得清晰的最终图像,输出。其中,合成区域可以是若干拼接区域,或拼接区域与融合区域的结合。
具体地,对图14a的初始图像M1和M2进行矩形校正和特征对齐后形成图14b两张校正后的矩形图像T1和T2(T1和T2也可以是对齐后的矩形图像),
分析图14b所示两幅矩形图像T1和T2的清晰度,得到的如图14c所示清晰度曲线L1和L2,L1是矩形图像T1的清晰度曲线,L2是矩形图像T2的清晰度曲线。
在此,图像清晰度的评价方法,可以采用Brenner梯度函数(布伦纳梯度函数),具体地,通过计算相邻两个像素灰度差的平方进行清晰度评价,该梯度函数定义如下:
D(f)=ΣyΣx|f(x+2,y)-f(x,y)|2
其中:f(x,y)表示图像f对应像素点(x,y)的灰度值,D(f)为图像清晰度计算结果。
此外,基于TenenGrad函数、基于Brenner函数、基于方差函数、基于平方梯度函数、基于Vollath函数、基于加窗梯度函数、基于熵函数、基于区域对比度、以及无参考图像的清晰度评价方法(ΔDoM)等,均可以应用于本发明所示方法中。
接着,基于图14b和图14c的清晰度分布图,可知两张校正和对齐后的矩形图像T1和T2的清晰度分布曲线分别为L1和L2,基于清晰度分布曲线L1和L2对应的位置,生成如图14d所示的用于划分合成区域的拼接线MN,图14d是将矩形图像T1和T2合成的目标图像,14d中的矩形和矩形图像T1、T2尺寸一致,初始图14d中的矩形中可以是空白的,拼接线MN将图14d中的矩形划分出两个合成区域Q1和Q2,在此,所述拼接线不一定是直线,其具体根据清晰度的分布确定走向。
接着,在拼接区域中,可以直接选择对应清晰度较高的矩形图像的区域进行拼接即可,即,在合成区域Q1选择清晰度曲线L1对应的校正后的矩形图像的区域与在合成区域Q2选择清晰度曲线L2对应的校正后的矩形图像的区域进行拼接。可以在矩形图像T1和T2中按照拼接线MN划分为清晰度高和清晰度地的区域,则清晰度高的区域复制到对应的合成区域中;或者基于图14d中划分的合成区域内的像素点坐标,查找所述像素点坐标在清晰度高的矩形图像中的像素值,作为合成区域内该像素点的像素值。
在另一实施例中,结合图14e,基于清晰度曲线L1和L2划分为三个合成区域Q1、Q2和Q3,其中,合成区域Q2对应的两张校正后矩形图像的清晰度曲线L1和L2的清晰度差距不大,因此合成区域Q2可以确定为融合区,在这一区域中,利用两张校正后的矩形图像,以每个像素点的清晰度为权重进行融合处理。
在另一实施例中,图14f,当有三张校正后的矩形图像,获得清晰度分布曲线L1、L2和L3,基于三条清晰度分布曲线划分三个合成区域Q1、Q2和Q3,在合成时,在合成区域Q1中选择清晰度曲线L1对应的校正后的矩形图像的区域、在合成区域Q2中选择清晰度曲线L2对应的校正后的矩形图像的区域及在合成区域Q3中选择清晰度曲线L3对应的校正后的矩形图像的区域进行拼接,形成清晰度较高的图像。
接着,在另一实施例中,图14g,当有三张校正后的矩形图像,获得清晰度分布曲线L1、L2和L3,基于三条清晰度分布曲线划分五个合成区域Q1、Q2、Q3、Q4和Q5,其中,合成区域Q2和Q4确定为融合区域,其中融合区域Q2基于清晰度曲线L1和L2对应的矩形图像进行融合,融合区域Q4基于清晰度曲线L2和L3对应的矩形图像进行融合,以获得融合图像,合成区域Q1、Q3和Q5分别选择选择清晰度曲线L1、清晰度曲线L2及清晰度曲线L3对应的校正后的矩形图像的区域。
结合图15,根据本申请一实施例所述的方法中,当所述终端具有至少两个摄像头,则所述的获取至少一张初始图像的步骤包括:控制至少两个摄像头,分别以不同的焦点进行对焦,各拍摄至少一张初始图像。
在一具体实施例中,所述终端为具有两个同像素的双目摄像头手机。与前述实施例的不同,在与有两个参数相同的摄像头,仅仅拍摄位置有不同,因此可以同时对焦在不同的位置,拍摄两张初始图像。
由于拍摄位置的偏差,两张初始图像略有像差,如图16所示,从两个摄像头分别得到的图像中,四边形图像在初始图像中的位置有偏差。在一些实施例中,图16中(a)是摄像头一的预览图像,基于预设的焦点选择方法,计算出
的两个候选的焦点为G1和G2,图16中(b)是摄像头二的预览图像,基于同样的焦点选择方法,计算出的两个候选的焦点为H1和H2。由于有两个摄像头,可同时对焦,因此(G1,H2)和(G2,H1)成为两组候选焦点对。分别计算两组候选焦点对中的焦点与初始图像的中心的距离的和或者方差,选出离中心最近的一组候选焦点对,作为后续两摄像头同时拍照时分别对焦的焦点,例如,图16中较优的一组焦点位置是(G1,H2)。
接着,基于拍摄指令,两个摄像头同时获取初始图像。
通过矩形校正(也称为梯形校正),获得校正后的两矩形图像,接着,采用前述实施例中的方法,对校正后的矩形图像进行对齐和拼接融合过程。
根据本申请另一实施例所述的方法,所述终端还可以是具有两个不同像素的双目摄像头手机。基于前述实施例,两个摄像头的预览图像同样有视差。不过分辨率成为确定焦点位置主要考虑的因素。
为了提升远端图像的分辨率,可以将高像素摄像头的焦点确定在检测出的四边形图像的小端部分,即被拍摄的矩形目标的远端一侧,而低像素摄像头的焦点确定在检测出的四边形图像的大端,即被拍摄的矩形目标的近端一侧。
经过四边形校正后的校正图像,分辨率也是不同的,如下还表示出了清晰度的差异:当分析两个图像的清晰度时,两图像清晰度的最大值和取值范围都会差距较大。可取清晰度相等的位置作为拼接位置,这个位置可能会采用更多来自高分辨率摄像头的图像。该实施例同样可适用实施例一的拼接区域融合的方法,来减少拼接线附近的图像突变。
结合图17,根据本发明一具体实施例,在一具体场景中,拍摄图像的方法包括如下步骤:
1.摄像头打开,进入文档校正模式,获取预览图像Ip,并检测当前Ip中的四边形图像Qp;
2.检测到拍摄指令;
接着,判断需要拍摄原始图片的数量,若仅拍摄一张则:
3.1根据Qp确定一个焦点F;
3.2根据焦点F拍摄照片Io(即初始图像);
3.3检测Io中的四边形图像Q;
3.4校正Q为矩形图像Ir,为最终校正图像Ir;
若拍摄多张则:
4.1根据Qp确定至少两个焦点{F1,F2,…};
4.2根据{F1,F2,…}拍摄照片{Io1,Io2,...}(即初始图像);
4.3根据{Io1,Io2,…},分别检测其中的四边形图像{Q1,Q2,...};
4.4分别校正{Q1,Q2,…}为矩形图像{Ir1,Ir2,...},并对矩形图像{Ir1,Ir2,...}进行特征对齐{Ir1’,Ir2’,...};
4.5统一对{Ir1’,Ir2’,...}进行局部图像清晰度分析,确定至少一条拼接线{Lm1,…}或融合区域{Rm1,...};
4.6根据{Lm1,…}或{Rm1,...},把{Ir1,Ir2,…}合成为最终校正图像Im。
最后,9.将步骤3.4或步骤4.6获得的最终校正图像Ir或Im作为最终图像输出。
结合图18,在另一实施例中,在图17所示的具体场景基础上,根据预览图像中检测的四边形图像的成像情况,如对边夹角或清晰度分布曲线,自动选择拍摄张数。在一些实施例中,自动选择拍摄张数在拍摄初始图像之前执行即可,即可以在检测到拍摄指令之前执行。
图19a~图19c示出了采用本申请所述拍摄图像的方法,在拍摄图像和处理图像过程中,所获得的初始图像及最终输出的图像的示意图。在拍摄矩形目标时,在预览图像中检测四边形图像,确定四边形图像中对边夹角较小的两两对边中较短边为第一边(与被拍摄的矩形目标的远端边相对应),接着,基于拍摄指令,拍摄多张初始图像,至少获得有一张如图19a所示的以靠近第一边对边一端的部分区域的点作为焦点拍摄的初始图像,如图19b所示的以靠近第一边对边一端的部分区域的点作为焦点拍摄的初始图像,进行矩形校正和合成处理,形成如图19c所示的最终图像,其中,相比于现有技术,区域A’的清晰度明显提高,同时与区域B’的清晰度差别较小,提高了最终图像的整体的清晰度。
在以上各实施例提及的附图,均为便于理解各实施例的示例性的附图,其尺寸及比例并不被限制,且各附图间的比例关系并非必然关联。
根据一些实施方式提供的在带有摄像头的终端中拍摄图像的装置,其中,所述装置包括:检测模块、拍摄模块、校正模块及输出模块,首先,所述检测模块进入拍摄模式,在预览图像中检测四边形图像;接着,所述拍摄模块基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;然后,所述校正模块将每个所述至少一张初始图像中的四边形图像校正为矩形图像;之后,所述输出模块基于所述矩形图像,得到最终图像并输出。
图20示出根据一些实施例提供的在包括摄像头和多个应用程序的终端中拍摄图像的装置1,其中,所述装置1包括:检测模块11、拍摄模块12、校正模块13及输出模块14,首先,所述检测模块11进入拍摄模式,在预览图像中检测四边形图像;然后,所述拍摄模块12基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;接着,所述校正模块13将每个所述至少一张初始图像中的四边形图像校正为矩形图像;随后,所述输出模块14基于所述矩形图像,得到最终图像并输出。
本发明拍摄图像的装置,可应用于文档校正(或称为梯形校正)的拍摄模式,用于提升文档校正中校正后图像的清晰度,其中,所述的文档校正是一种通过对四边形图像的检测、变换、处理,输入是图像,输出也是图像,用于帮助用户在拍摄矩形目标时,校正由于拍摄角度倾斜导致图像变形,最终输出矩
形最终图像。在此,所述矩形目标泛指矩形的含有信息的图像,例如:文件、***、书本、名片、证件、讲义、照片、广告、展板、电视、电影、屏幕等等,当然,本领域技术人员应当能够理解,上述矩形目标仅为举例,其他矩形物体被拍摄为图像时,都可以应用本申请所述方法进行拍摄和校正。
所述装置可以是安装于具有摄像头及多个应用程序的终端中的具有处理能力的硬件、软件或软件与硬件的结合,其中,所述终端可为所有具有拍照功能以及图像处理功能的电子设备,所述电子设备可为移动电话、平板电脑(Tablet Personal Computer)、膝上型电脑(Laptop Computer)、数码相机、数字摄影机、投影设备、个人数字助理(personal digital assistant,简称PDA)、电子书阅读器(英文:e-book reader)或可穿戴式设备(Wearable Device)等,对此本发明实施例不做进一步限定。
所述检测模块11的一种具体实现方式参考图7所示的步骤S11的具体内容,所述拍摄模块的实现方法参考图7所示的步骤S12的具体内容,为简明起见,不再赘述。
如图21所示,在一些实施例中,所述拍摄模块12包括:第一拍摄子模块121,用于在所述四边形图像内选定所述第一焦点,以所述第一焦点拍摄一张初始图像。具体地,所述第一拍摄模块121包括:第一边确定子模块121a、第一区域确定子模块121b和第一焦点确定子模块121c,所述第一边确定子模块121a在所述四边形图像的第二边和第三边上分别确定第一点和第二点,其中,所述第二边和所述第三边与所述第一边相邻,所述第一点到所述第一边与所述第二边相交的顶点的距离小于等于到所述第二边与第四边相交的顶点,所述第二点到所述第一边与所述第三边相交的顶点的距离小于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;所述第一区域确定子模块121b确定所述第一边与所述第二边相交的顶点、所述第一边与所述第三边相交的顶点、所述第一点及所述第二点连线所组成的第一区域为所述第一边一端的部分区域;所述第一焦点确定子模块121c选定一个所述第一区域中的点为所述第一焦点。所述第一边确定子模块121a、第一区域确定子模块121b和第一焦
点确定子模块121c的具体实现参考前述实施例中的步骤S121a、步骤S121b和步骤S121c的内容及图10a和图10b所述的具体实施例的内容,不再赘述。
所述校正模块的实现方法参考图7所示的步骤S13的具体内容,所述输出模块的实现方法参考图7所示的步骤S14的具体内容,为简明起见,不再赘述,并以引用的方式包含于此。
随后,所述输出模块通过第一输出子模块(图中未示出),将所述矩形图像作为最终图像输出。
在一些实施例中,所述输出模块包括合成子模块141和第二输出子模块(图中未示出)。
结合图20和图22,所述装置11执行的过程包括:首先,所述检测模块12进入拍摄模式,在预览图像中检测四边形图像;然后,所述拍摄模块12包括第二拍摄子模块122,所述第二拍摄子模块122在所述四边形图像内选定至少两个焦点,所述至少两个焦点包括至少一个所述第一焦点和至少一个第二焦点,所述第二焦点不同于所述第一焦点,并分别以所述至少两个焦点对焦拍摄至少两张初始图像;随后,所述校正模块将所述初始图像中的四边形图像校正为矩形图像。所述检测模块的实现方法参考图12所示的步骤S11’的具体内容,所述拍摄模块的实现方法参考图12所示的步骤S12’的具体内容,为简明起见,不再赘述。
其中,选定所述第一焦点的具体实现方式参考图21所示的第一拍摄子模块选定所述第一焦点的具体实现方式,不再赘述。
在一些实施例中,所述第二焦点为在所述四边形图像内所述第一边对边一端的部分区域中选定的点。所述第二拍摄子模块包括:第二边确定子模块、第二区域确定子模块和第二焦点确定子模块。所述第二边确定子模块在所述四边形图像的第二边和第三边上分别确定第三点和第四点,其中,所述第二边和所述第三边与所述第一边相邻,所述第三点到所述第一边与所述第二边相交的顶点的距离大于等于到所述第二边与第四边相交的顶点,所述第四点到所述第一边与所述第三边相交的顶点的距离大于等于到所述第三边与所述第四边相交的
顶点,所述第四边为所述第一边的对边;所述第二区域确定子模块确定所述第四边与所述第二边相交的顶点、所述第四边与所述第三边相交的顶点、所述第三点及所述第四点连线所组成的第二区域为所述第一边对边一端的部分区域;所述第二焦点确定子模块选定一个所述第二区域中的点分别作为第二焦点。
所述校正模块通过校正子模块将所述至少两张初始图像中的四边形图像分别校正为至少两个矩形图像;所述输出模块包括:合成子模块和第二输出子模块,所述合成子模块对所述至少两个矩形图像进行合成以得到最终图像;所述第二输出子模块输出所述最终图像。
在一些具体的实现方式中,待拍摄的初始图像是数量不被限定,例如可以是一张、两张、三张、四张、八张等等。
在一些实施例中,待拍摄的初始图像的数量,可以是程序预先设定的,或由根据用户选定操作确定。例如,程序自动设定为每当用户每输入一次拍摄指令(例如单击拍摄按键),则拍摄两张初始图像;再例如,在每次进入文档校正的拍摄模式时,向用户提供选择提示,以由用户选定拍摄初始图像的数量,随后每当用户每输入一次拍摄指令(例如单击拍摄按键),则拍摄相应数量的初始图像。
在一些实施例中,待拍摄的初始图像的数量,也可以根据预览图像中所述四边形图像的成像情况,确定待拍摄的初始图像的数量,其中,预览图像中所述四边形图像的成像情况可以为但不限于:根据预览图像中所述四边形图像的形变程度大致判断的拍摄倾斜角度的情况,或根据预检测预览图像中所述四边形图像在校正为矩形图像后各局部的清晰度差异情况确定。例如,若所述四边形图像倾斜角度越小或校正后清晰度差异越小,则确定拍摄较少数量的初始图像,若其倾斜角度越大或清晰度差异越大,则确定拍摄较多数量的初始图像,例如拍摄两张、四张甚至更多张。
在一些实施例中,初始图像的拍摄数量根据拍摄倾斜度确定,拍摄倾斜度越大,增加拍摄图像数量,以提升初始图像的清晰度。所述装置还包括选定模块,所述选定模块在以所述至少一个焦点对焦拍摄至少一张初始图像之前,确
定待拍摄的初始图像的数量。进一步地,所述选定模块包括选定子模块,所述选定子模块根据所述预览图像中检测的四边形图像的第一组对边的对边夹角、或所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线,确定待拍摄的初始图像的数量。
具体地,根据所述四边形图像的对边夹角或所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线,预判所述拍摄倾斜度;数量确定模块根据拍摄倾斜度所属的倾斜度阈值范围确定待拍摄的初始图像的数量,其中,所述拍摄倾斜度越大,所确定的待拍摄的初始图像的数量越多。预判模块和数量确定模块的实现方法参考图8所示的判断过程,为简明起见,不再赘述。
在一些实施例中,根据用户拍摄指令(拍摄单张还是连拍多张的指令)确定初始图像的拍摄数量。在一些实施例中,所述拍摄模块还包括:数量确定子模块,所述数量确定子模块在拍摄至少两张初始图像的步骤之前,根据用户在指定时间内连续输入的拍摄指令的次数,确定待获取的初始图像的数量。数量确定子模块的实现方法参考图9所示的判断过程,为简明起见,不再赘述。
在一些实施例中,对预览图像中所检测的四边形图像根据所述预览图像中检测的四边形图像的对边夹角、或所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线,预判所述拍摄倾斜度;然后,根据拍摄倾斜度所属的倾斜度阈值范围确定待拍摄的初始图像的数量,拍摄倾斜度越大,所确定的待拍摄的初始图像的数量可以越多,拍摄倾斜度越大,被拍摄的矩形目标远端和近端的清晰度差别越大,通过增加初始图像的拍摄数量,并结合选择不同的焦点位置进行拍摄,能够获得更多清晰的局部区域,从而在后续合成阶段提高最终图像的整体清晰度,此外,对拍摄角度的倾斜角度的预判,可以用于确定拍摄初始图像的张数、还可以用于选择初始图像的拍摄过程中各焦点位置的参考等。
如图21所示,在一些实施例中,所述第一拍摄模块121包括:第一边确定子模块121a、第一区域确定子模块121b和第一焦点确定子模块121c。所述第
一边确定子模块121a在所述四边形图像的第二边和第三边上分别确定第一点和第二点,其中,所述第二边和所述第三边与所述第一边相邻,所述第一点到所述第一边与所述第二边相交的顶点的距离小于等于到所述第二边与第四边相交的顶点,所述第二点到所述第一边与所述第三边相交的顶点的距离小于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;所述第一区域确定子模块121b确定所述第一边与所述第二边相交的顶点、所述第一边与所述第三边相交的顶点、所述第一点及所述第二点连线所组成的第一区域为所述第一边一端的部分区域;所述第一焦点确定子模块121c选定所述第一区域中的点为焦点。
所述第一边确定子模块121a、所述第一区域确定子模块121b和所述第一焦点确定子模块121c的实现方法参考前述实施例中的步骤S121、步骤S122和步骤S123的内容及图10a和图10b所述的具体实施例的内容,不再赘述。
如图22所示,在一些实施例中,所述第二拍摄模块122包括:第二边确定子模块122a、第二区域确定子模块122b和第二焦点确定子模块122c,其中,所述第二边确定子模块122a在所述第二边和所述第三边上分别确定第三点和第四点,其中,所述第三点到所述第一边与所述第二边相交的顶点的距离大于等于到所述第二边与所述第四边相交的顶点,所述第四点到所述第一边与所述第三边相交的顶点的距离大于等于到所述第三边与所述第四边相交的顶点;所述第二区域确定子模块122b确定所述第四边与所述第二边相交的顶点、所述第四边与所述第三边相交的顶点、所述第三点及所述第四点连线所组成的第二区域为所述第一边对边一端的部分区域;所述第二焦点确定子模块122c选定至少一个所述第二区域中的点分别作为焦点。
所述第二边确定子模块122a、所述第二区域确定子模块122b和所述第二焦点确定子模块122c的实现方法类似于所述第一边确定子模块121a、所述第一区域确定子模块121b和所述第一焦点确定子模块121c的实现方法,为简明起见,不再赘述。
接着,结合图23,所述输出模块14包括合成子模块141,所述合成子模块
131对多张所述矩形图像进行合成,以得到最终图像。
如图23所示,在一些实施例中,所述合成子模块141包括:分析子模块141a、区域划分子模块141b和执行子模块141c。所述分析子模块141a对所述矩形图像进行清晰度分析及特征点对齐;所述区域划分子模块141b基于对齐后的各所述矩形图像的清晰度分布,确定若干合成区域;所述合成子模块141c基于各所述矩形图像的清晰度,对每一所述合成区域进行合成。所述分析子模块、区域划分子模块和合成子模块的实现方法参考前述实施例中步骤S141a、步骤S141b和步骤S141c的具体内容,为简明起见,不再赘述。
在一些实施例中,所述执行子模块包括第一合成子单元(附图未示出)和/或第二合成子单元(附图未示出),所述第一合成子单元从各所述矩形图像中获取一个与所述合成区域对应清晰度最高的区域,进行拼接,以得到最终图像;所述第二合成子单元从各所述矩形图像中获取至少两个与所述合成区域对应清晰度较高的区域,以清晰度为权重,进行像素融合,以得到最终图像。第一合成子单元和/或第二合成子单元的合成过程的实施例参考图14a~图14c所示及其对应描述,不再赘述。
若所述终端具有至少两个摄像头时,所述拍摄模块包括:控制子模块,控制子模块控制各所述摄像头,分别以不同的焦点进行对焦各拍摄至少一张初始图像。所述控制子模块的具体实现方法图15和图16所示及其对应描述,不再赘述。
在本发明实施例中使用的术语是仅仅出于描述特定实施例的目的,而非旨在限制本发明。在本发明实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式,除非上下文清楚地表示其他含义。还应当理解,本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
本领域内的技术人员应明白,本申请的实施例可提供为方法、***、或计算机程序产品。因此,本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本申请可采用在一个或多个其中包
含有计算机可用程序代码的计算机可用存储信道(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器指令,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然,本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样,倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内,则本申请也意图包含这些改动和变型在内。
综上所述,以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照上述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对上述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。
Claims (42)
- 一种在带有摄像头的终端中拍摄图像的方法,其中,所述方法包括:进入拍摄模式,在预览图像中检测四边形图像;基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;将每个所述至少一张初始图像中的四边形图像校正为矩形图像;基于所述矩形图像,得到最终图像并输出。
- 根据权利要求1所述的方法,其中,所述的在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:在所述四边形图像内选定所述第一焦点,以所述第一焦点拍摄一张初始图像。
- 根据权利要求2所述的方法,其中,所述的在所述四边形图像内选定所述第一焦点的步骤包括:在所述四边形图像的第二边和第三边上分别确定第一点和第二点,其中,所述第二边和所述第三边与所述第一边相邻,所述第一点到所述第一边与所述第二边相交的顶点的距离小于等于到所述第二边与第四边相交的顶点,所述第二点到所述第一边与所述第三边相交的顶点的距离小于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;确定所述第一边与所述第二边相交的顶点、所述第一边与所述第三边相交的顶点、所述第一点及所述第二点连线所组成的第一区域为所述第一边一端的部分区域;选定一个所述第一区域中的点为所述第一焦点。
- 根据权利要求3所述的方法,其中,所述的选定一个所述第一区域中的点为所述第一焦点的步骤包括以下任一项:选定所述第一区域的对角连线的交点为所述第一焦点;选定所述第一区域的对边中点连线的交点为所述第一焦点。
- 根据权利要求2至4中任一项所述的方法,其中,所述的基于所述矩形图像,得到最终图像并输出的步骤包括:将所述矩形图像作为最终图像输出。
- 根据权利要求1所述的方法,其中,所述在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:在所述四边形图像内选定至少两个焦点,所述至少两个焦点包括至少一个所述第一焦点和至少一个第二焦点,所述第二焦点不同于所述第一焦点,并分别以所述至少两个焦点对焦拍摄至少两张初始图像。
- 根据权利要求6所述的方法,其中,所述第二焦点为在所述四边形图像内所述第一边对边一端的部分区域中选定的点。
- 根据权利要求6或7所述的方法,其中,所述第二焦点的选定方法包括:在所述四边形图像的第二边和第三边上分别确定第三点和第四点,其中,所述第二边和所述第三边与所述第一边相邻,所述第三点到所述第一边与所述第二边相交的顶点的距离大于等于到所述第二边与第四边相交的顶点,所述第四点到所述第一边与所述第三边相交的顶点的距离大于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;确定所述第四边与所述第二边相交的顶点、所述第四边与所述第三边相交的顶点、所述第三点及所述第四点连线所组成的第二区域为所述第一边对边一端的部分区域;选定一个所述第二区域中的点分别作为第二焦点。
- 根据权利要求6至8中任一项所述的方法,其中,所述将每个所述至少一张初始图像中的四边形图像校正为矩形图像的步骤包括:将所述至少两张初始图像中的四边形图像分别校正为至少两个矩形图像;所述的基于所述矩形图像,得到最终图像并输出的步骤包括:对所述至少两个矩形图像进行合成以得到最终图像;输出所述最终图像。
- 根据权利要求9所述的方法,其中,所述的对所述至少两个矩形图像进行合成以得到最终图像的步骤包括:对所述至少两个矩形图像进行特征点对齐;基于对齐后的各所述矩形图像的清晰度分布,确定所述最终图像中的多个合成区域;基于各所述矩形图像的清晰度,对每一所述合成区域进行合成以得到最终图像。
- 根据权利要求10所述的方法,其中,所述的对每一所述合成区域进行合成以得到最终图像的步骤包括至少以下任一项:从各所述矩形图像中获取一个与所述合成区域对应清晰度最高的区域,进行拼接,以得到最终图像;从各所述矩形图像中获取至少两个与所述合成区域对应的区域,以清晰度为权重,进行像素融合,以得到最终图像。
- 根据权利要求6至11中任一项所述的方法,其中,所述终端具有至少两个摄像头,则所述的并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:控制各所述摄像头,分别以不同的焦点进行对焦各拍摄至少一张初始图像。
- 根据权利要求1至12中任一项所述的方法,其中,在所述的以所述至少一个焦点对焦拍摄至少一张初始图像步骤之前,所述方法还包括:确定待拍摄的初始图像的数量。
- 根据权利要求13所述的方法,其中,所述的确定待拍摄的初始图像的数量的步骤包括:根据所述预览图像中检测的四边形图像的第一组对边的对边夹角、或所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线,确定待拍摄的初始图像的数量。
- 一种在带有摄像头的终端中中拍摄图像的装置,其中,所述装置包括:检测模块,用于进入拍摄模式,在预览图像中检测四边形图像;拍摄模块,用于基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;校正模块,用于将每个所述至少一张初始图像中的四边形图像校正为矩形图像;输出模块,用于基于所述矩形图像,得到最终图像并输出。
- 根据权利要求15所述的装置,其中,所述拍摄模块包括:第一拍摄子模块,用于在所述四边形图像内选定所述第一焦点,以所述第一焦点拍摄一张初始图像。
- 根据权利要求16所述的装置,其中,所述第一拍摄子模块包括:第一边确定子模块,用于在所述四边形图像的第二边和第三边上分别确定第一点和第二点,其中,所述第二边和所述第三边与所述第一边相邻,所述第一点到所述第一边与所述第二边相交的顶点的距离小于等于到所述第二边与第四边相交的顶点,所述第二点到所述第一边与所述第三边相交的顶点的距离小于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;第一区域确定子模块,用于确定所述第一边与所述第二边相交的顶点、所述第一边与所述第三边相交的顶点、所述第一点及所述第二点连线所组成的第一区域为所述第一边一端的部分区域;第一焦点确定子模块,用于选定一个所述第一区域中的点为所述第一焦点。
- 根据权利要求17所述的装置,其中,所述第一焦点确定子模块用于以下至少任一项:选定所述第一区域的对角连线的交点为所述第一焦点;选定所述第一区域的对边中点连线的交点为所述第一焦点。
- 根据权利要求16至18所述的装置,其中,所述输出模块包括:第一输出子模块,用于将所述矩形图像作为最终图像输出。
- 根据权利要求15所述的装置,其中,所述拍摄模块包括:第二拍摄子模块,用于在所述四边形图像内选定至少两个焦点,所述至少两个焦点包括至少一个所述第一焦点和至少一个第二焦点,所述第二焦点不同于所述第一焦点,并分别以所述至少两个焦点对焦拍摄至少两张初始图像。
- 根据权利要求20所述的装置,其中,所述第二焦点为在所述四边形图像内所述第一边对边一端的部分区域中选定的点。
- 根据权利要求20或21所述的装置,其中,所述第二拍摄子模块包括:第二边确定子模块,用于在所述四边形图像的第二边和第三边上分别确定第三点和第四点,其中,所述第二边和所述第三边与所述第一边相邻,所述第三点到所述第一边与所述第二边相交的顶点的距离大于等于到所述第二边与第四边相交的顶点,所述第四点到所述第一边与所述第三边相交的顶点的距离大于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;第二区域确定子模块,用于确定所述第四边与所述第二边相交的顶点、所述第四边与所述第三边相交的顶点、所述第三点及所述第四点连线所组成的第二区域为所述第一边对边一端的部分区域;第二焦点确定子模块,用于选定一个所述第二区域中的点分别作为第二焦点。
- 根据权利要求20至22中任一项所述的装置,其中,所述校正模块包括:校正子模块,用于将所述至少两张初始图像中的四边形图像分别校正为至少两个矩形图像;所述输出模块包括:合成子模块,用于对所述至少两个矩形图像进行合成以得到最终图像;第二输出子模块,用于输出所述最终图像。
- 根据权利要求23所述的装置,其中,所述合成子模块包括:分析子模块,用于对所述矩形图像进行清晰度分析及特征点对齐;区域划分子模块,用于基于对齐后的各所述矩形图像的清晰度分布,确定若干合成区域;执行子模块,用于基于各所述矩形图像的清晰度,对每一所述合成区域进行合成。
- 根据权利要求24所述的装置,其中,所述执行子模块包括至少以下任一项:第一合成子单元,用于从各所述矩形图像中获取一个与所述合成区域对应清晰度最高的区域,进行拼接,以得到最终图像;第二合成子单元,用于从各所述矩形图像中获取至少两个与所述合成区域对应的区域,以清晰度为权重,进行像素融合,以得到最终图像
- 根据权利要求20至25中任一项所述的装置,其中,所述终端具有至少两个摄像头时,所述拍摄模块包括:控制子模块,用于控制各所述摄像头,分别以不同的焦点进行对焦各拍摄至少一张初始图像。
- 根据权利要求15至26中任一项所述的装置,其中,所述装置还包括:选定模块,用于在以所述至少一个焦点对焦拍摄至少一张初始图像之前,确定待拍摄的初始图像的数量。
- 根据权利要求27所述的装置,其中,所述选定模块包括:选定子模块,用于根据所述预览图像中检测的四边形图像的第一组对边的对边夹角、或所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线,确定待拍摄的初始图像的数量。
- 一种终端,包括:至少一个摄像头;一个显示单元;一个或多个处理器;存储器;多个应用程序;及所述存储器用于存储计算机程序;所述处理器,运行所述计算机程序,执行下述流程:进入拍摄模式,在预览图像中检测四边形图像;基于拍摄指令,在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像,其中,所述四边形图像包括第一组对边和第二组对边,所述第一组对边的对边夹角小于所述第二组对边的对边夹角,第一边为所述第一组对边中较短的一边,所述至少一个焦点包括第一焦点,所述第一焦点在所述四边形图像内且在第一边一端的部分区域中;将每个所述至少一张初始图像中的四边形图像校正为矩形图像;基于所述矩形图像,得到最终图像并输出。
- 根据权利要求29所述的终端,其中,所述的在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:在所述四边形图像内选定所述第一焦点,以所述第一焦点拍摄一张初始图像。
- 根据权利要求30所述的终端,其中,所述的在所述四边形图像内选定所述第一焦点的步骤包括:在所述四边形图像的第二边和第三边上分别确定第一点和第二点,其中,所述第二边和所述第三边与所述第一边相邻,所述第一点到所述第一边与所述第二边相交的顶点的距离小于等于到所述第二边与第四边相交的顶点,所述第二点到所述第一边与所述第三边相交的顶点的距离小于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;确定所述第一边与所述第二边相交的顶点、所述第一边与所述第三边相交的顶点、所述第一点及所述第二点连线所组成的第一区域为所述第一边一端的部分区域;选定一个所述第一区域中的点为所述第一焦点。
- 根据权利要求31所述的终端,其中,所述的选定一个所述第一区域中的点为所述第一焦点的步骤包括以下任一项:选定所述第一区域的对角连线的交点为所述第一焦点;选定所述第一区域的对边中点连线的交点为所述第一焦点。
- 根据权利要求30至32中任一项所述的终端,其中,所述的基于所述矩形图像,得到最终图像并输出的步骤包括:将所述矩形图像作为最终图像输出。
- 根据权利要求29所述的终端,其中,所述在所述四边形图像内选定至少一个焦点,并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:在所述四边形图像内选定至少两个焦点,所述至少两个焦点包括至少一个所述第一焦点和至少一个第二焦点,所述第二焦点不同于所述第一焦点,并分别以所述至少两个焦点对焦拍摄至少两张初始图像。
- 根据权利要求34所述的终端,其中,所述第二焦点为在所述四边形图像内所述第一边对边一端的部分区域中选定的点。
- 根据权利要求34或35所述的终端,其中,选定所述第二焦点的步骤包括:在所述四边形图像的第二边和第三边上分别确定第三点和第四点,其中,所述第二边和所述第三边与所述第一边相邻,所述第三点到所述第一边与所述第二边相交的顶点的距离大于等于到所述第二边与第四边相交的顶点,所述第四点到所述第一边与所述第三边相交的顶点的距离大于等于到所述第三边与所述第四边相交的顶点,所述第四边为所述第一边的对边;确定所述第四边与所述第二边相交的顶点、所述第四边与所述第三边相交的顶点、所述第三点及所述第四点连线所组成的第二区域为所述第一边对边一端的部分区域;选定一个所述第二区域中的点分别作为第二焦点。
- 根据权利要求34至36中任一项所述的终端,其中,所所述将每个所述 至少一张初始图像中的四边形图像校正为矩形图像的步骤包括:将所述至少两张初始图像中的四边形图像分别校正为至少两个矩形图像;所述的基于所述矩形图像,得到最终图像并输出的步骤包括:对所述至少两个矩形图像进行合成以得到最终图像;输出所述最终图像。
- 根据权利要求37所述的终端,其中,所述的对所述至少两个矩形图像进行合成以得到最终图像的步骤包括:对所述至少两个矩形图像进行特征点对齐;基于对齐后的各所述矩形图像的清晰度分布,确定所述最终图像中的多个合成区域;基于各所述矩形图像的清晰度,对每一所述合成区域进行合成以得到最终图像。
- 根据权利要求38所述的终端,其中,所述的对每一所述合成区域进行合成以得到最终图像的步骤包括至少以下任一项:从各所述矩形图像中获取一个与所述合成区域对应清晰度最高的区域,进行拼接,以得到最终图像;从各所述矩形图像中获取至少两个与所述合成区域对应的区域,以清晰度为权重,进行像素融合,以得到最终图像。
- 根据权利要求34至39中任一项所述的终端,其中,所述终端具有至少两个摄像头,则所述的并以所述至少一个焦点对焦拍摄至少一张初始图像的步骤包括:控制各所述摄像头,分别以不同的焦点进行对焦各拍摄至少一张初始图像。
- 根据权利要求29至40中任一项所述的终端,其中,在所述的以所述至少一个焦点对焦拍摄至少一张初始图像步骤之前,所述处理器,运行所述计算机程序,执行下述流程:确定待拍摄的初始图像的数量。
- 根据权利要求41所述的终端,其中,所述的确定待拍摄的初始图像的 数量的步骤包括:根据所述预览图像中检测的四边形图像的第一组对边的对边夹角、或所述四边形图像内从所述第一边到所述第一边对边的方向上的清晰度分布曲线,确定待拍摄的初始图像的数量。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/332,701 US10863077B2 (en) | 2016-09-12 | 2016-09-12 | Image photographing method, apparatus, and terminal |
PCT/CN2016/098751 WO2018045592A1 (zh) | 2016-09-12 | 2016-09-12 | 拍摄图像方法、装置和终端 |
CN201680089115.XA CN109691080B (zh) | 2016-09-12 | 2016-09-12 | 拍摄图像方法、装置和终端 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2016/098751 WO2018045592A1 (zh) | 2016-09-12 | 2016-09-12 | 拍摄图像方法、装置和终端 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018045592A1 true WO2018045592A1 (zh) | 2018-03-15 |
Family
ID=61562448
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2016/098751 WO2018045592A1 (zh) | 2016-09-12 | 2016-09-12 | 拍摄图像方法、装置和终端 |
Country Status (3)
Country | Link |
---|---|
US (1) | US10863077B2 (zh) |
CN (1) | CN109691080B (zh) |
WO (1) | WO2018045592A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112752017A (zh) * | 2020-06-23 | 2021-05-04 | 长沙中天电子设计开发有限公司 | 拍照答题器的答题方法、拍照方法、拍照答题器及*** |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2018052096A1 (ja) | 2016-09-14 | 2018-03-22 | ソニー株式会社 | センサ、入力装置および電子機器 |
US10909719B2 (en) * | 2016-12-21 | 2021-02-02 | Huawei Technologies Co., Ltd. | Image processing method and apparatus |
CN112104796B (zh) * | 2019-06-18 | 2023-10-13 | Oppo广东移动通信有限公司 | 图像处理方法和装置、电子设备、计算机可读存储介质 |
CN110766615B (zh) * | 2019-09-09 | 2023-07-14 | 北京美院帮网络科技有限公司 | 一种图片矫正方法、装置、终端及计算机可读存储介质 |
CN110807729B (zh) * | 2019-10-30 | 2023-06-23 | 口碑(上海)信息技术有限公司 | 图像数据的处理方法及装置 |
CN113657258A (zh) * | 2021-08-16 | 2021-11-16 | 前景智能交通科技(苏州)有限公司 | 一种基于图像处理的接触网故障识别***和识别方法 |
CN115690149B (zh) * | 2022-09-27 | 2023-10-20 | 江苏盛利智能科技有限公司 | 显示器的图像融合处理***及方法 |
CN115620321B (zh) * | 2022-10-20 | 2023-06-23 | 北京百度网讯科技有限公司 | 表格识别方法及装置、电子设备和存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080187242A1 (en) * | 2007-02-07 | 2008-08-07 | Oki Data Corporation | Image reading apparatus |
CN101634795A (zh) * | 2009-08-20 | 2010-01-27 | 上海合合信息科技发展有限公司 | 相机自动聚焦方法及*** |
CN101996392A (zh) * | 2009-08-07 | 2011-03-30 | 夏普株式会社 | 摄像图像处理***以及图像输出方法 |
CN104735350A (zh) * | 2015-03-02 | 2015-06-24 | 联想(北京)有限公司 | 一种信息处理方法及电子设备 |
CN105574813A (zh) * | 2015-12-31 | 2016-05-11 | 青岛海信移动通信技术股份有限公司 | 一种图像处理方法和装置 |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6975352B2 (en) * | 2000-12-18 | 2005-12-13 | Xerox Corporation | Apparatus and method for capturing a composite digital image with regions of varied focus and magnification |
US20060210192A1 (en) * | 2005-03-17 | 2006-09-21 | Symagery Microsystems Inc. | Automatic perspective distortion detection and correction for document imaging |
US7760962B2 (en) * | 2005-03-30 | 2010-07-20 | Casio Computer Co., Ltd. | Image capture apparatus which synthesizes a plurality of images obtained by shooting a subject from different directions, to produce an image in which the influence of glare from a light is reduced |
US8483473B2 (en) * | 2008-01-18 | 2013-07-09 | Mitek Systems, Inc. | Systems and methods for obtaining financial offers using mobile image capture |
US8995012B2 (en) * | 2010-11-05 | 2015-03-31 | Rdm Corporation | System for mobile image capture and processing of financial documents |
JP5708051B2 (ja) * | 2011-03-07 | 2015-04-30 | 株式会社リコー | 映像処理装置、映像処理システム、テレビ会議システム、遠方監視システム、映像処理方法、及び撮像装置 |
WO2014033886A1 (ja) * | 2012-08-30 | 2014-03-06 | 富士通株式会社 | 画像処理装置、画像処理方法、及びプログラム |
WO2015013947A1 (zh) | 2013-08-01 | 2015-02-05 | 华为终端有限公司 | 一种拍照方法及装置 |
JP6347675B2 (ja) * | 2014-06-06 | 2018-06-27 | キヤノン株式会社 | 画像処理装置、撮像装置、画像処理方法、撮像方法及びプログラム |
US9160946B1 (en) * | 2015-01-21 | 2015-10-13 | A2iA S.A. | Systems and methods for capturing images using a mobile device |
JP6644641B2 (ja) * | 2016-05-31 | 2020-02-12 | オリンパス株式会社 | 撮像装置、制御方法、及びプログラム |
-
2016
- 2016-09-12 WO PCT/CN2016/098751 patent/WO2018045592A1/zh active Application Filing
- 2016-09-12 US US16/332,701 patent/US10863077B2/en active Active
- 2016-09-12 CN CN201680089115.XA patent/CN109691080B/zh active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080187242A1 (en) * | 2007-02-07 | 2008-08-07 | Oki Data Corporation | Image reading apparatus |
CN101996392A (zh) * | 2009-08-07 | 2011-03-30 | 夏普株式会社 | 摄像图像处理***以及图像输出方法 |
CN101634795A (zh) * | 2009-08-20 | 2010-01-27 | 上海合合信息科技发展有限公司 | 相机自动聚焦方法及*** |
CN104735350A (zh) * | 2015-03-02 | 2015-06-24 | 联想(北京)有限公司 | 一种信息处理方法及电子设备 |
CN105574813A (zh) * | 2015-12-31 | 2016-05-11 | 青岛海信移动通信技术股份有限公司 | 一种图像处理方法和装置 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112752017A (zh) * | 2020-06-23 | 2021-05-04 | 长沙中天电子设计开发有限公司 | 拍照答题器的答题方法、拍照方法、拍照答题器及*** |
Also Published As
Publication number | Publication date |
---|---|
CN109691080B (zh) | 2020-10-09 |
CN109691080A (zh) | 2019-04-26 |
US20190222744A1 (en) | 2019-07-18 |
US10863077B2 (en) | 2020-12-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018045592A1 (zh) | 拍摄图像方法、装置和终端 | |
US10789671B2 (en) | Apparatus, system, and method of controlling display, and recording medium | |
KR100796849B1 (ko) | 휴대 단말기용 파노라마 모자이크 사진 촬영 방법 | |
CN107659769B (zh) | 一种拍摄方法、第一终端及第二终端 | |
US20170034410A1 (en) | Method and electronic device for stabilizing video | |
KR102124617B1 (ko) | 이미지 합성 방법 및 그 전자 장치 | |
KR102512889B1 (ko) | 이미지 융합 프로세싱 모듈 | |
US20210048891A1 (en) | Using natural movements of a hand-held device to manipulate digital content | |
US20090227283A1 (en) | Electronic device | |
CN107948505B (zh) | 一种全景拍摄方法及移动终端 | |
CN108776822B (zh) | 目标区域检测方法、装置、终端及存储介质 | |
CN111064895B (zh) | 一种虚化拍摄方法和电子设备 | |
TWI545508B (zh) | 用於執行臉部追蹤功能的方法及其電子裝置 | |
CN114298902A (zh) | 一种图像对齐方法、装置、电子设备和存储介质 | |
CN114445315A (zh) | 图像质量增强方法和电子设备 | |
CN114782296B (zh) | 图像融合方法、装置及存储介质 | |
CN110086998B (zh) | 一种拍摄方法及终端 | |
US20240031667A1 (en) | Image Processing Method, Mobile Terminal, and Storage Medium | |
WO2013136602A1 (ja) | プロジェクタ付き撮影装置及びその撮影制御方法 | |
US20230012826A1 (en) | Method for correcting shaking at high magnification and electronic device therefor | |
JP7025237B2 (ja) | 画像処理装置およびその制御方法ならびにプログラム | |
JP6777507B2 (ja) | 画像処理装置および画像処理方法 | |
Huang et al. | Using homography relationship for auto-calibration in mobile smart-project device system | |
CN112383708B (zh) | 拍摄方法、装置、电子设备及可读存储介质 | |
CN114390206A (zh) | 拍摄方法、装置和电子设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16915526 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16915526 Country of ref document: EP Kind code of ref document: A1 |