WO2020014881A1 - 一种图像校正方法和终端 - Google Patents

一种图像校正方法和终端 Download PDF

Info

Publication number
WO2020014881A1
WO2020014881A1 PCT/CN2018/096039 CN2018096039W WO2020014881A1 WO 2020014881 A1 WO2020014881 A1 WO 2020014881A1 CN 2018096039 W CN2018096039 W CN 2018096039W WO 2020014881 A1 WO2020014881 A1 WO 2020014881A1
Authority
WO
WIPO (PCT)
Prior art keywords
corrected
coordinate
distortion
coordinate point
position coordinates
Prior art date
Application number
PCT/CN2018/096039
Other languages
English (en)
French (fr)
Inventor
张磊
廖川
刘万程
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201880086820.3A priority Critical patent/CN111656391B/zh
Priority to PCT/CN2018/096039 priority patent/WO2020014881A1/zh
Publication of WO2020014881A1 publication Critical patent/WO2020014881A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction

Definitions

  • the present application relates to the technical field of terminals, and in particular, to an image correction method and a terminal.
  • the camera function of the terminal has become one of the functions frequently used by users, and users have increasingly higher requirements for the quality of captured images.
  • FIG. 1 is a schematic diagram of lens distortion.
  • the imaging point is at point A (point A is at a distance of O from the imaging center). The distance is a), and in (b) of Fig. 1, because the convex lens is tilted, after the light is imaged through the convex lens, the imaging point is at point B (the distance between point B and the point O of the imaging center is b, and b is greater than a) Therefore, the imaging in FIG. 1 (b) is distorted relative to the imaging in FIG. 1 (a).
  • the present application provides an image correction method and terminal for correcting a distorted image and improving the quality of a captured image.
  • an embodiment of the present application provides an image correction method suitable for a terminal.
  • the method includes: in response to a user operation, the terminal starts a camera application, opens a camera, and collects an original image through the camera; the terminal Determining the ideal position coordinates of each coordinate point according to the original position coordinates and the distortion transformation formula of each coordinate point among all coordinate points on the original image; wherein the distortion transformation formula is used to represent the original of the coordinate points A conversion relationship between position coordinates and ideal position coordinates; the terminal determines the degree of distortion of each coordinate point according to the original position coordinates and ideal position coordinates of each coordinate point; the terminal determines The degree of distortion of the coordinate points and the threshold of the degree of distortion determine the coordinate points that need to be corrected and the coordinate points that do not need to be corrected among all the coordinate points; and the terminal determines all of The correction weights of the coordinate points to be corrected are described, wherein the continuous function is used to represent the degree of distortion and The conversion relationship between the positive weights; the terminal determines the corrected coordinates of the
  • the terminal may judge the coordinate points on the original image, and determine the coordinate points that need to be corrected and the coordinate points that do not need to be corrected. For coordinate points that do not need to be corrected, the terminal can keep the pixel values of these coordinate points unchanged. For the coordinate points that need to be corrected, the terminal determines the correction weight of each coordinate point that needs to be corrected and a correction formula corresponding to the correction weight, and corrects the coordinate points that need to be corrected by using the correction formula. Because the correction weight of each coordinate point to be corrected is different, the correction formula of each coordinate point to be corrected is also different. In this way, on the one hand, the terminal does not need to correct all the coordinate points, which saves calculation and improves efficiency. On the other hand, for different coordinate points that need to be corrected, the terminal uses different correction formulas to correct, which helps to improve the quality of the captured image.
  • the execution step includes: determining each coordinate point The distance between the original position coordinates of the point and the ideal position coordinates, the distance being the degree of distortion of each coordinate point.
  • the terminal determines the distance between the original position coordinates and the ideal position coordinates of each coordinate point, which is the degree of distortion of each coordinate point.
  • the near coordinate point means that the coordinate point does not need to be corrected. In this way, the terminal does not need to correct all coordinate points, which saves calculation and improves efficiency.
  • the execution step includes: determining all the coordinates The first coordinate point in the point where the degree of distortion is greater than the distortion degree threshold, the first coordinate point is the coordinate point that needs to be corrected; determining the second coordinate in all the coordinate points whose degree of distortion is less than or equal to the distortion degree threshold Point, the second coordinate point is the coordinate point that does not need to be corrected.
  • the terminal may store a distortion degree threshold. For coordinate points whose distortion degree is greater than the distortion degree threshold, that is, coordinate points that need to be corrected, and for coordinate points whose distortion degree is less than or equal to the distortion degree threshold, that is, coordinate points that do not need to be corrected. . In this way, the terminal does not need to correct all the coordinate points, which helps to save calculation and improve efficiency.
  • the terminal determines the correction of the coordinate point that needs to be corrected according to the degree of distortion and the continuous function of the coordinate point that needs to be corrected.
  • Weights including:
  • the terminal substitutes the d into the following continuous function:
  • the distortion degree threshold is d0, and the correction weight f of the coordinate point that needs to be corrected is obtained.
  • a continuous function is stored in the terminal.
  • the terminal determines that the distortion degree of a coordinate point that needs to be corrected is d
  • the terminal substitutes d into the continuous function to obtain the coordinate point that needs to be corrected.
  • Correction weight f the degree of distortion of each coordinate point that needs to be corrected is different
  • the correction weight of each coordinate point that needs to be determined through the continuous function is different, that is, each coordinate point that needs to be corrected has a different weight.
  • the correction formula is different. In this way, using a different correction formula for each coordinate point that needs to be corrected helps to improve the quality of the captured image.
  • a correction formula corresponding to the correction weight is obtained, and the corrected position coordinate of the coordinate point to be corrected is calculated by the correction formula. Since the correction weight of each coordinate point that needs to be corrected is different, the correction formula of each coordinate point that needs to be corrected is different. In this way, using different correction formulas for different coordinate points that need to be corrected can help improve the quality of the captured image.
  • the terminal when the terminal sets the pixel value corresponding to the original position coordinate of the coordinate point that needs to be corrected on the original image to the pixel value corresponding to the corrected position coordinate, the terminal specifically The execution step includes: if the corrected position coordinates are integers, determining a pixel value corresponding to an original position coordinate of the coordinate point to be corrected on the original image is a pixel value corresponding to the corrected position coordinate; If the corrected position coordinates are non-integer, the pixel values corresponding to the original position coordinates of the coordinate points that need to be corrected are interpolated into the horizontal and vertical directions near the corrected position coordinates by a pixel interpolation algorithm. Coordinates are integers at other positions.
  • the terminal uses a pixel interpolation algorithm to interpolate the pixel values located at the original position coordinates to the horizontal and vertical coordinates near the corrected position coordinates. Other position coordinates that are integers. In this way, the terminal adopts different measures for each coordinate point that needs to be corrected, which helps to improve the quality of the captured image.
  • an embodiment of the present application provides a terminal, including a processor and a memory.
  • the memory is used to store one or more computer programs; when the one or more computer programs stored in the memory are executed by a processor, the terminal can implement the first aspect or any one of the possible design methods of the first aspect.
  • an embodiment of the present application further provides a terminal, where the terminal includes a module / unit that executes the first aspect or a method of any possible design of the first aspect.
  • modules / units can be implemented by hardware, and can also be implemented by hardware executing corresponding software.
  • an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium includes a computer program, and when the computer program runs on a terminal, causes the terminal to execute the first aspect or the foregoing first Any one of the possible design methods.
  • an embodiment of the present application further provides a computer program product that, when the computer program product runs on a terminal, causes the terminal to execute the first aspect or any one of the foregoing possible designs of the first aspect. method.
  • FIG. 1 is a schematic diagram of lens distortion according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of camera imaging provided by an embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of a mobile phone according to an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a mobile phone according to an embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of an image correction method according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of radial distortion and switching distortion according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of a black and white grid for camera calibration provided by an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a continuous function in a two-dimensional coordinate system according to an embodiment of the present invention.
  • FIG. 9 is a schematic diagram of relative positions between coordinate points according to an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of a pixel interpolation process according to an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of a terminal according to an embodiment of the present invention.
  • the pixel involved in the embodiment of the present application is the smallest imaging unit on an image.
  • One pixel can correspond to one coordinate point on the image.
  • a pixel can include one parameter (such as grayscale) or a set of multiple parameters (such as grayscale, brightness, color, etc.). If the pixel includes a parameter, the pixel value is the value of the parameter. If the pixel is a collection of multiple parameters, the pixel value includes the value of each parameter in the set.
  • the image plane coordinate system involved in the embodiment of the present application is a coordinate system established on a camera imaging plane.
  • the imaging plane please refer to FIG. 2, which is a schematic diagram of camera imaging provided by an embodiment of the present application. As shown in FIG. 2, when the camera captures a person, an image of the person is collected through a lens, and the collected image is presented on an imaging plane.
  • the original image involved in the embodiment of the present application that is, the input image of the image correction algorithm provided in the embodiment of the present application, that is, when the terminal runs the code of the image correction algorithm, the terminal performs image correction processing on the input image.
  • the original image may be an image sensor in the camera generates an image of the object to be captured after the lens group in the camera collects the light signal reflected by the object to be captured.
  • the original image may also be an output image of the image sensor in the camera after generating the image of the object to be captured through other algorithms according to the light signal, which is not limited in this embodiment of the present invention.
  • the corrected image involved in the embodiment of the present application that is, the output image after the terminal runs the code of the image correction algorithm provided in the embodiment of the present application to correct the original image.
  • the original position coordinates involved in the embodiments of the present application that is, the coordinate positions of each pixel point on the original image in the image plane coordinate system.
  • the original position coordinates are represented by (x 0 , y 0 ).
  • the ideal position coordinates involved in the embodiments of the present application that is, ideally, each pixel point on the original image should be a coordinate position in an image plane coordinate system.
  • the ideal position coordinates are represented by (x, y).
  • the corrected position coordinates involved in the embodiments of the present application that is, the coordinate positions of each pixel point in the image plane coordinate system in the corrected image.
  • the corrected position coordinates are represented by (x ', y'). .
  • the multiple involved in the embodiments of the present application refers to two or more.
  • the electronic device may be a portable electronic device that further includes other functions such as a personal digital assistant and / or a music player function, such as a mobile phone, a tablet computer, and a wearable device with a wireless communication function (such as a smart watch). )Wait.
  • a portable electronic device include, but are not limited to, a portable electronic device equipped with iOS, Android, Microsoft, or other operating systems.
  • the aforementioned portable electronic device may also be other portable electronic devices, such as a laptop computer having a touch-sensitive surface (for example, a touch panel). It should also be understood that, in some other embodiments of the present application, the above electronic device may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (such as a touch panel).
  • the electronic device in the embodiment of the present application may be a mobile phone 300.
  • the embodiment will be specifically described below with the mobile phone 300 as an example. It should be understood that the illustrated mobile phone 300 is only an example of an electronic device, and the mobile phone 300 may have more or fewer parts than those shown in the figure, two or more parts may be combined, or With different component configurations.
  • the various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.
  • the mobile phone 300 may specifically include: one or more processors 301, a radio frequency (RF) circuit 302, a memory 303, a touch screen 304, a Bluetooth device 305, one or more sensors 306, and Wi-Fi
  • These components can communicate via one or more communication buses or signal lines (not shown in FIG. 3).
  • RF radio frequency
  • the mobile phone 300 may include more or fewer components than shown in the figure, or combine certain components, or arrange different components. .
  • the processor 301 is a control center of the mobile phone 300, and uses various interfaces and lines to connect various parts of the mobile phone 300.
  • the processor 301 runs or executes an application program (Application, App for short) stored in the memory 303, and calls the stored program in the memory 303.
  • Data and instructions execute various functions of the mobile phone 300 and process data.
  • the processor 301 may include one or more processing units.
  • the processor 301 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, and a video. Codec, digital signal processor (DSP), baseband processor, and / or neural-network processing unit (NPU), etc.
  • AP application processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the different processing units included in the processor 301 may be independent devices or integrated in the same processor.
  • the processor 301 may integrate a GPU and a DSP, or may integrate an AP and an ISP.
  • the processor 301 integrated application processor and modem processor as an example.
  • the application processor mainly deals with the operating system, user interface, and application programs, etc.
  • the modem processor mainly deals with wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 301.
  • the processor 301 may further include a fingerprint verification chip for verifying the collected fingerprint.
  • the radio frequency circuit 302 may be used for receiving and transmitting wireless signals during information transmission or communication. Specifically, the radio frequency circuit 302 may receive the downlink data of the base station and process it to the processor 301; in addition, send the uplink data to the base station. Generally, the radio frequency circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency circuit 302 can also communicate with other devices through wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to a global mobile communication system, a general packet wireless service, code division multiple access, wideband code division multiple access, long-term evolution, email, short message service, and the like.
  • the memory 203 is used to store application programs and data, and the processor 301 executes various functions and data processing of the mobile phone 300 by running the application programs and data stored in the memory 303.
  • the memory 303 mainly includes a storage program area and a storage data area, where the storage program area can store an operating system and at least one application required by a function (such as a sound playback function, an image playback function, etc.); the storage data area can store data according to the mobile phone used Data created at 300 (such as audio data, phone book, etc.).
  • the memory 303 may include a high-speed random access memory, and may also include a non-volatile memory, such as a magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • the memory 303 can store various operating systems, such as the IOS operating system developed by Apple Inc. and the Android operating system developed by Google Inc. Exemplarily, the storage 303 stores application programs related to the embodiments of the present application, such as Taskcard store, Twitter, phone book, Weibo, and the like.
  • the touch screen 304 may include a touch-sensitive surface 304-1 and a display 304-2.
  • the touch-sensitive surface 304-1 (such as a touch panel) can capture touch events on or near the user of the mobile phone 300 (for example, the user uses a finger, a stylus, or any suitable object on the touch-sensitive surface 304-1). Or operations near the touch-sensitive surface 304-1), and send the collected touch information to other devices such as the processor 301.
  • the user's touch event near the touch-sensitive surface 304-1 can be referred to as hovering touch; hovering touch can mean that the user does not need to directly touch the touch in order to select, move or drag the target (such as the App icon, etc.) Board without the user having to be near the electronic device in order to perform the desired function.
  • the touch-sensitive surface 304-1 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch position, and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into contact coordinates, and When sent to the processor 301, the touch controller may also receive and execute instructions sent by the processor 301.
  • the display also referred to as a display screen
  • the display 304-2 may be used to display information input by the user or information provided to the user and various menus of the mobile phone 300.
  • the display 304-2 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.
  • the touch-sensitive surface 304-1 may be overlaid on the display 304-2.
  • the touch-sensitive surface 304-1 When the touch-sensitive surface 304-1 detects a touch event on or near it, it is transmitted to the processor 301 to determine the type of the touch event, and then the processor 301 may provide corresponding visual output on the display 304-2 according to the type of touch event.
  • the touch-sensitive surface 304-1 and the display screen 304-2 are implemented as two separate components to implement the input and output functions of the mobile phone 300, in some embodiments, the touch-sensitive surface 304- 1 Integrated with the display screen 304-2 to implement the input and output functions of the mobile phone 300.
  • the touch screen 304 may further include a series of pressure sensor arrays, which may enable the mobile phone to sense the pressure applied to the touch screen 304 by a touch event.
  • the mobile phone 300 may further include a Bluetooth device 305 for implementing data exchange between the mobile phone 300 and other short-range electronic devices (such as a mobile phone, a smart watch, etc.).
  • the Bluetooth device in the embodiment of the present application may be an integrated circuit or a Bluetooth chip.
  • the mobile phone 300 may further include at least one sensor 306, such as a light sensor, a motion sensor, and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor.
  • the ambient light sensor may adjust the brightness of the display of the touch screen 304 according to the brightness of the ambient light.
  • the proximity sensor may turn off the power of the display when the mobile phone 300 is moved to the ear.
  • the accelerometer sensor can detect the magnitude of acceleration in various directions (usually three axes), and can detect the magnitude and direction of gravity when it is stationary. It can be used for applications that recognize the attitude of mobile phones (such as horizontal and vertical screen switching, related Games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tap), etc.
  • the sensor 306 may further include a fingerprint sensor.
  • the fingerprint sensor 312 may be arranged on the back of the mobile phone 300 (for example, under the rear camera), or the fingerprint sensor 312 may be arranged on the front of the mobile phone 300 (for example, under the touch screen 304).
  • the fingerprint recognition function can also be implemented by configuring the fingerprint sensor 312 in the touch screen 304, that is, the fingerprint sensor 312 can be integrated with the touch screen 304 to implement the fingerprint recognition function of the mobile phone 300.
  • the fingerprint sensor 312 may be configured in the touch screen 304, may be a part of the touch screen 304, or may be configured in the touch screen 304 in other ways.
  • the fingerprint sensor 312 can also be implemented as a full-board fingerprint sensor. Therefore, the touch screen 304 can be regarded as a panel where fingerprint collection can be performed at any position.
  • the fingerprint sensor 312 may send the collected fingerprint to the processor 301, so that the processor 301 processes the fingerprint (for example, fingerprint verification, etc.).
  • the fingerprint sensor 312 in the embodiment of the present application may use any type of sensing technology, including but not limited to optical, capacitive, piezoelectric, or ultrasonic sensing technologies.
  • the mobile phone 300 other sensors, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, may not be described herein.
  • sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, may not be described herein.
  • Wi-Fi device 307 is used to provide mobile phone 200 with network access that complies with Wi-Fi related standard protocols.
  • Mobile phone 300 can access Wi-Fi access points through Wi-Fi device 307 to help users send and receive email, Browse the web and access streaming media, etc., it provides users with wireless broadband Internet access.
  • the Wi-Fi device 307 can also be used as a Wi-Fi wireless access point, and can provide Wi-Fi network access for other electronic devices.
  • the positioning device 308 is configured to provide a geographic location for the mobile phone 300. It can be understood that the positioning device 308 may specifically be a receiver of a positioning system such as a global positioning system (GPS) or a Beidou satellite navigation system.
  • GPS global positioning system
  • Beidou satellite navigation system a positioning system
  • the audio circuit 309, the speaker 313, and the microphone 314 may provide an audio interface between the user and the mobile phone 300.
  • the audio circuit 309 may transmit the received electrical data converted electric signal to the speaker 313, and the speaker 313 converts it into a sound signal for output.
  • the microphone 314 converts the collected sound signal into an electric signal, and the audio circuit 309 After receiving, it is converted into audio data, and then the audio data is output to the RF circuit 302 for transmission to, for example, a mobile phone, or the audio data is output to the memory 303 for further processing.
  • the peripheral interface 310 is used to provide various interfaces for external input / output devices (such as a keyboard, a mouse, an external display, an external memory, a user identification module card, etc.). For example, it is connected to a mouse through a universal serial bus interface, and connected to a subscriber identity module (SIM) card provided by a telecommunications operator through a metal contact on the card slot of the subscriber identity module.
  • SIM subscriber identity module
  • the peripheral interface 310 may be used to couple the above-mentioned external input / output peripherals to the processor 301 and the memory 303.
  • the mobile phone 300 may further include a power supply device 311 (such as a battery and a power management chip) for supplying power to various components.
  • the battery may be logically connected to the processor 301 through the power management chip, so as to manage charging, discharging, and power consumption management through the power supply device 311 And other functions.
  • the mobile phone 300 may further include a camera 315, where the camera 315 includes a lens group 315-1 and an image sensor 315-2, and the lens group 315-1 includes a plurality of lenses (convex lenses or concave lenses) for collecting reflections of objects to be photographed.
  • the optical signal is transmitted to the image sensor 315-2.
  • the image sensor 315-2 generates an original image of an object to be captured according to the light signal.
  • the image sensor 315-2 sends the generated original image to the processor 301, and the processor 301 runs the code of the image correction algorithm provided in the embodiment of the present application to correct the original image to obtain a corrected image.
  • the camera 315 may be a front camera or a rear camera.
  • the front camera may also be used to capture facial feature information, and the processor 301 may perform face recognition on the facial feature information, and then perform subsequent processing.
  • the mobile phone 300 may further include a flash, a micro-projection device, a near field communication (NFC) device, and the like, and details are not described herein.
  • NFC near field communication
  • the following embodiments can all be implemented in an electronic device (such as a mobile phone 300, a tablet computer, etc.) having the above-mentioned hardware structure.
  • Image correction algorithm see Figure 4 for details. It should be noted that in FIG. 4, the processor 301 integrates the application processor 301-1 and the ISP 301-2 as an example.
  • obtaining a corrected image by using the mobile phone 300 shown in FIG. 4 may be the following process:
  • the lens group 315-1 in the camera 315 collects the light signals reflected by the object to be captured, and transmits the collected light signals to the image sensor 315-2.
  • the image sensor 315-2 generates an original image of the object to be captured according to the light signal, and the original image is distorted.
  • the image sensor 315-2 sends the original image to the ISP 301-2.
  • the application processor 301-1 executes the code of the image correction algorithm provided by the embodiment of the application (for example, the code of the image correction algorithm provided by the embodiment of the application is stored in the memory 303, and the application processor 301 -1 runs the code of the image correction algorithm stored in the memory 303) to obtain a value between the original position coordinates (x 0 , y 0 ) of each coordinate point and the corrected position coordinates (x ', y'). Mapping relations.
  • the application processor 301-1 may send the mapping relationship to the ISP 301-2.
  • the ISP301-2 After the ISP301-2 receives the original image and the mapping relationship, it performs image correction on the original image according to the mapping relationship, that is, moves the pixel value on (x 0 , y 0 ) on the original image to the corrected position. At the position coordinates (x ', y') of the, obtain a corrected image, and send the corrected image to the application processor 301-1.
  • the display 304-2 displays the corrected image in a finder frame of a finder interface.
  • the mapping relationship may also be stored in the memory 303.
  • the ISP 301-2 obtains the mapping relationship from the memory 303. Then, ISP301-2 moves the pixel value on (x 0 , y 0 ) on the original image to the corrected position coordinate (x ', y') according to the mapping relationship, to obtain the corrected image.
  • ISP301-2 can also run the code of the image correction algorithm provided in the embodiment of the present application by itself to obtain the mapping relationship between the original position coordinates (x 0 , y 0 ) and the corrected position coordinates (x ', y'). .
  • ISP301-2 moves the pixel value on (x 0 , y 0 ) on the original image to the corrected position coordinate (x ', y') according to the mapping relationship.
  • a corrected image is obtained, and the corrected image is sent to the application processor 301-1.
  • the display 304-2 displays the corrected image in a finder frame of a finder interface.
  • the display 304-2 displays the corrected image, that is, the preview image, that is, from the launch of the camera application to the presentation of the preview image, the mobile phone 300 performs the above process again.
  • the camera stores the corrected image.
  • the application processor 301-1 can detect whether the collected original image changes in real time or periodically. Whenever the original image changes, the ISP 301-2 can perform image correction on the changed original image according to the mapping relationship, that is, As much as possible, ensure that the preview image is a corrected image. There can be multiple ways for the application processor 301-1 to detect whether the collected original image has changed.
  • the image sensor 315-2 can send the original image collected each time to the application processor 315-1, and the application processor 315- 1 Judging that the similarity of the original image received two times in a row is small, indicating that the original image has been changed; or, the motion sensor in the mobile phone 200 can detect whether the current image is in a moving state. If so, the original image changes, and of course, there may be In other manners, this embodiment of the present application is not limited to this.
  • the mobile phone 300 can perform image correction on the captured original image in real time, and as far as possible, ensure that the preview image seen by the user is a corrected image, which helps to improve the user experience.
  • obtaining a corrected image by using the mobile phone 300 shown in FIG. 4 may also be the following process:
  • the display 304-2 displays the original image during the photo preview process, and no correction is performed on the original image at this time.
  • the application processor 301-1 determines the user's photographing operation, it triggers the ISP 301-2 to correct the original image according to the mapping relationship. That is, in this embodiment, during the process of obtaining the captured image by the mobile phone 300, the image calibration process is performed only once, and the calculation amount is small.
  • FIG. 5 is a schematic flowchart of an image correction method according to an embodiment of the present application. As shown in FIG. 5, the application processor 201 runs the code of the image correction algorithm to perform the following process:
  • the first step determine the ideal position coordinates (x, y) of each coordinate point according to the original position coordinates (x 0 , y 0 ), the distortion coefficient and the distortion conversion formula of each coordinate point on the original image.
  • which conversion formula is stored in the mobile phone 300
  • which conversion formula is used. This is because, after the camera is designed, before the mobile phone 200 is shipped from the factory, the designer generally calibrates the camera in the mobile phone 300. Camera calibration, that is, the process of determining the distortion coefficient of the camera. Because after the camera is designed, it is not clear what type of distortion will occur when taking images through the camera. Because there are many types of image distortion, the corresponding conversion formulas for each type of distortion are different. Therefore, the designer may choose to calibrate the camera of the mobile phone 300 by using a certain type of distortion conversion formula. What kind of distortion type conversion formula does the designer use to calibrate the camera of the mobile phone 300, and what kind of distortion type conversion formula is stored in the mobile phone 300.
  • the distortion coefficient and the conversion formula used are stored in the mobile phone 300. After the mobile phone 300 leaves the factory, when the camera takes an image, the distortion coefficient and the conversion formula can be used to perform image correction to obtain a corrected image.
  • FIG. 6 shows a schematic diagram of radial distortion and tangential distortion. The following describes the process by which the mobile phone 300 obtains the distortion coefficient through camera calibration.
  • One possible calibration method is that the designer uses a conversion formula corresponding to radial distortion for calibration.
  • the mobile phone 300 captures the black-and-white checkered picture shown in FIG. 7 to obtain a captured image, which may be distorted.
  • the coordinates of each intersection on the black and white grid shown in FIG. 7 can be set in advance (for example, the designer can set it by himself), and the coordinates of the intersection on the captured image are in the image plane coordinates.
  • the mobile phone 300 can determine the original position coordinates of each intersection in the image plane coordinate system. Because the coordinates of the intersection on the black and white grid picture are not in the image plane coordinate system, the designer can set a reference coordinate system (such as the world coordinate system) to convert the coordinates of the intersection on the black and white grid picture to the reference coordinate. At the same time, the coordinates of the intersection point on the captured image are converted into the reference coordinate system. Therefore, in the reference coordinate system, the coordinates of each intersection on the black and white grid picture shown in FIG.
  • the mobile phone 300 can calculate the distortion coefficient by using a conversion formula corresponding to the radial distortion.
  • the conversion formula corresponding to radial distortion is as follows:
  • k 1 , k 2 and k 3 are distortion coefficients.
  • Formula (1) is only an example of the conversion formula corresponding to radial distortion.
  • the conversion formula corresponding to radial distortion can also be in other forms, such as the following formula:
  • the mobile phone 300 needs to calculate three distortion coefficient values of k 1 , k 2 , and k 3 , so at least three equations are needed. Therefore, the mobile phone 300 can determine the coordinates of two intersection points on the object to be photographed (that is, the black and white checkered pattern shown in FIG. 4), for example, (x, y) are (1, 2) and (1, 4), respectively.
  • the mobile phone 200 determines that the coordinates corresponding to (1, 2) on the distorted black-and-white checkered pattern obtained by shooting are (1.5, 2.7), and the coordinates corresponding to (1, 4) are (1.8, 4.2). Then the terminal 300 brings these four coordinates into formula (1), and obtains the following equations:
  • the values of k 1 , k 2 , and k 3 can be obtained.
  • the value of k 1 is 0.00604, the value of k 2 is -0.01833, and the value of k 3 is 0.00000.
  • the calibration process ends, that is, the mobile phone 300 brings the values of k 1 , k 2 , and k 3 into formula (1) to obtain the conversion formula (3 ) To store the conversion formula (3) for use.
  • the mobile phone 300 captures the black-and-white checkered picture shown in FIG. 7 to obtain a captured image, which may have distortion.
  • the coordinates of each intersection on the black and white grid picture shown in FIG. 7 can be used as the ideal position coordinates (x, y), and the coordinates of each intersection on the captured image can be used as the original Position coordinates (x 0 , y 0 ).
  • the mobile phone 300 calculates a distortion coefficient by using a transformation formula of tangential distortion.
  • the transformation formula of tangential distortion is as follows:
  • p 1 and p 2 are distortion coefficients.
  • Formula (4) is only an example of a transformation formula for tangential distortion.
  • the transformation formula for tangential distortion may also be in other forms, which is not limited in the embodiment of the present application.
  • the mobile phone 300 needs to calculate two distortion coefficients of p 1 and p 2 , so at least two equations are needed. Therefore, the mobile phone 300 can determine the coordinates of at least one intersection on the object to be photographed (ie, the black and white checkered pattern shown in FIG. 7), for example, (x, y) is (1, 2). The mobile phone 200 determines that the coordinates corresponding to (1, 2) on the black and white checkered pattern of the specific distortion obtained are (1.5, 2.7). Then the mobile phone 200 brings these two coordinates into the formula (4), and obtains the following equations:
  • the values of p 1 and p 2 can be obtained.
  • the value of p 1 is -0.00037
  • the value of p 2 is -0.00095.
  • Another possible calibration method is that the designer uses a formula obtained by combining a conversion formula corresponding to radial distortion and a conversion formula for tangential distortion for calibration. Similarly, the designer takes the black-and-white checkered picture shown in FIG. 7 as the object to be photographed of the mobile phone 300 and shoots to obtain a photographed image, which may have distortion.
  • the mobile phone 300 calculates a distortion coefficient by using a formula obtained by combining a conversion formula corresponding to radial distortion and a conversion formula for tangential distortion.
  • the formula obtained by combining the transformation formula corresponding to the radial distortion and the transformation formula for the tangential distortion is as follows:
  • k 1 , k 2 , k 3 , p 1 , and p 2 are distortion coefficients.
  • Formula (6) is only an example.
  • the formula obtained by combining the transformation formula corresponding to the radial distortion and the transformation formula for the tangential distortion can also be in other forms, for example, it can also be in the following form:
  • the mobile phone 300 needs to calculate the values of five distortion coefficients of k 1 , k 2 , k 3 , p 1 , and p 2 , so at least 5 equations are needed, that is, The coordinates of at least three intersections on the black-and-white checkered picture shown in FIG. 7, for example, (x, y) are (1, 2), (1, 4), and (1, 6), respectively.
  • the mobile phone 300 determines that the coordinates corresponding to (1, 2) on the black and white checkered pattern with distortion obtained are (1.5, 2.7), the coordinates corresponding to (1, 4) are (1.8, 4.2), and (1, 6) The corresponding coordinates are (1.6, 6). Then the terminal 200 brings these two coordinates into the formula (5), and obtains the following equations:
  • five distortion coefficient values of k 1 , k 2 , k 3 , p 1 , and p 2 can be obtained.
  • the calibration process ends, that is, the mobile phone 200 divides k 1 , k 2 , k 3 , p 1 , and p 2 into five.
  • the values of the distortion coefficients are brought into formula (7) to obtain the conversion formula, and the conversion formula is stored for use.
  • the formula (3) is stored in the mobile phone 300, that is, the application processor 301-1 can use the formula (3) and the original position coordinates (x 0 , y 0 ), determine the ideal position coordinates (x, y) of each coordinate point.
  • the application processor 301-1 may determine ideal position coordinates (x, y) of all coordinate points on the original image. For example, the camera of the mobile phone 300 is 1920 * 1080 pixels, then the application processor 301-1 can calculate all the coordinate points from (0, 0), (0, 1) to (1920, 1080) on the original image. The ideal position coordinates. Taking one of the coordinate points (0, 1) as an example, the application processor 301-1 brings the coordinate point (0, 1) into the conversion formula (5), and then can obtain a corresponding to the coordinate point (0, 1). Ideal position coordinates. In this way, the application processor 301-1 obtains the ideal position coordinates (x, y) corresponding to the original position coordinates (x 0 , y 0 ) of each coordinate point on the original image.
  • Step 2 Determine the degree of distortion of each coordinate point according to the original position coordinates (x 0 , y 0 ) and ideal position coordinates (x, y) of each coordinate point.
  • the degree of distortion can be used to indicate the distance between the original position coordinates and the ideal position coordinates of each coordinate point on the imaging plane.
  • the degree of distortion can be expressed by d, then
  • the application processor 301-1 can obtain the degree of distortion d of each coordinate point.
  • the third step is to determine the coordinate points that need to be corrected and the coordinate points that do not need to be corrected according to the distortion degree d and the distortion degree threshold value d 0 of each coordinate point. Please continue to refer to FIG. 5. For coordinate points that need to be corrected, perform steps 4-7, and for coordinate points that do not need to be corrected, perform step 8.
  • the application processor 301-1 to obtain the distortion level of each coordinate point d, the distortion and the degree of distortion of the threshold value d 0 d each coordinate point may be compared, if a coordinate point of distortion If the degree is greater than the distortion degree threshold, the coordinate point needs to be corrected. If the distortion degree of a coordinate point is less than or equal to the distortion degree threshold, the coordinate point does not need to be corrected. For example, there are a total of 1,000 coordinate points, and the distortion level of 300 of them is less than or equal to the distortion threshold. Then, the mobile phone 200 may not correct the 300 coordinate points, that is, perform the eighth step on the 300 coordinate points.
  • d 0 degree i.e. distortion threshold value may be based on experience of the designer or obtained experimentally, it is stored for use in the mobile phone 300.
  • Step 4 Determine the correction weight corresponding to the coordinate point that needs to be corrected according to the degree of distortion and the continuous function of the coordinate point that needs to be corrected.
  • the mobile phone 200 stores a function regarding the correction weight and the degree of distortion, that is, the two variables of the function are the correction weight and the distortion imaging, respectively.
  • a function regarding the correction weight and the degree of distortion that is, the two variables of the function are the correction weight and the distortion imaging, respectively.
  • d represents the degree of distortion of a coordinate point
  • d 0 is the threshold of the degree of distortion
  • f is the correction weight of the coordinate point
  • the application processor 301-1 may bring the distortion degree d and the distortion threshold d 0 of the coordinate points to be corrected obtained in the second step into formula (8) to obtain the coordinate points that need to be corrected.
  • Correction weight f may be brought the distortion degree d and the distortion threshold d 0 of the coordinate points to be corrected obtained in the second step into formula (8) to obtain the coordinate points that need to be corrected.
  • FIG. 8 is a curve relationship of the function (8) in a two-dimensional coordinate system, where the abscissa is the degree of distortion d and the ordinate is the correction weight f.
  • the application processor 301-1 may also determine the ordinate corresponding to the distortion degree d on the curve shown in FIG. 8 according to the distortion degree d of the coordinate point that needs to be corrected obtained in the second step, that is, the coordinate point that needs to be corrected. Correction weight.
  • the degree of distortion of the coordinate point to be corrected is d1
  • it is determined that the ordinate corresponding to the degree of distortion d1 on the curve shown in FIG. 8 is f1.
  • Step 5 Determine the corrected position coordinates (x ', y') corresponding to the coordinate points to be corrected according to the correction weight f and the correction formula of the coordinate points to be corrected.
  • the application processor 301-1 may determine the correction of the coordinate points to be corrected according to the original position coordinates (x 0 , y 0 ) and the correction weight f of the coordinate points that need to be corrected, and the correction formula (9). Post position coordinates (x ', y').
  • A indicates the original position coordinate point, that is, the point A coordinate is (x 0 , y 0 )
  • B indicates the corrected position coordinate point, that is, the point B coordinate is (x ', y')
  • C indicates the ideal position coordinate point.
  • the coordinate of point C is (x, y).
  • f is the correction weight of a coordinate point obtained in the fourth step.
  • FIG. 9 shows a schematic diagram of relative positions between various coordinate points provided in the embodiment of the present application.
  • the image plane coordinate system includes eight original position coordinate points, where the original position coordinate points are represented by white points.
  • the ideal position coordinate point in FIG. 9 is indicated by a black point, and the corrected position coordinate point is indicated by a grid point.
  • the application processor 301-1 brings the coordinates of point A into the conversion formula (3) to obtain the coordinates of the ideal position coordinate point corresponding to point A, that is, point C.
  • the application processor 301-1 brings the coordinates of the D point into the conversion formula (3), and obtains the ideal position coordinates corresponding to the D point, that is, the coordinates of the E point.
  • the application processor 301-1 determines the degree of distortion dA of point A based on the original coordinate position and ideal position coordinates of point A, and determines the degree of distortion dD of point D based on the original coordinate position of point D and the ideal position coordinates.
  • the application processor 301-1 brings the distortion degree dA and the distortion threshold d0 of point A into formula (8) to obtain the correction weight fA of point A.
  • the application processor 301-1 determines, according to formula (9), the coordinate point after correction corresponding to the coordinates of point A, that is, the coordinates of point B.
  • the ratio between the distance L between the points A and B and the distance dA between the points A and C (the degree of distortion of the point A) is equal to the correction weight fA of the point A.
  • the correction weight of each coordinate point that needs to be corrected is different.
  • the application processor 301-1 may determine the corrected position coordinates of each coordinate point that needs to be corrected according to the correction weight of each coordinate point that needs to be corrected.
  • the application processor 301-1 can obtain the corrected position coordinates (x 'y') of each coordinate point that needs to be corrected.
  • the application processor 301-1 can establish a mapping relationship between the original position coordinates (x 0 , y 0 ) of the coordinate points to be corrected and the corrected position coordinates (x 'y').
  • the application processor 301-1 obtains the mapping relationship between the original position coordinates (x 0 , y 0 ) and the corrected position coordinates (x 'y'), it can send the mapping relationship to the ISP 301 -2.
  • the ISP 301-2 receives the original image and the mapping relationship, it performs image correction on the original image according to the mapping relationship.
  • the ISP301-2 maps the original image on the original image based on the mapping relationship between the original position coordinates (x 0 , y 0 ) of the coordinate points to be corrected and the corrected position coordinates (x 'y').
  • the pixel values on the original position coordinates (x 0 , y 0 ) of the corrected coordinate point are set as the pixel values of the corrected position coordinates (x 'y').
  • the image sensor 315-2 corresponds the pixel value to the original position coordinates of each coordinate point on the imaging plane, that is, the original image is obtained. Therefore, ISP301-2 can map the original image to the coordinates that need to be corrected according to the mapping relationship between the original position coordinates (x 0 , y 0 ) of the coordinate points that need to be corrected and the corrected position coordinates (x 'y'). The pixel value corresponding to the point is moved to the corrected position coordinate.
  • ISP301-2 corresponds a pixel value to a coordinate point
  • the horizontal and vertical coordinate values of the coordinate point are required to be integers. Therefore, if the corrected position coordinate (x'yy ') of a coordinate point to be corrected is an integer, ISP301-2 determines that the pixel value corresponding to the original position coordinate of the coordinate point to be corrected is the corrected position coordinate. The pixel value. Since the corrected position coordinates (x'yy ') are calculated through the foregoing five steps, the corrected position coordinates (x'yy') may not be integers. Therefore, before the ISP301-2 obtains the corrected image, it can perform the seventh step described below.
  • Step 7 If the corrected position coordinates (x 'y') are not integers, the ISP301-2 can use the pixel interpolation algorithm to interpolate the pixel values corresponding to the original position coordinates (x 0 , y 0 ) into the corrected position coordinates. At the integer coordinates near (x 'y'), a corrected image is obtained.
  • the two coordinate points on the original image are (8, 19) and (9, 20).
  • the corrected position coordinate corresponding to (8, 19) is (8.8 , 20.2).
  • ISP301-2 determines that the pixel value corresponding to (8,19) is the first pixel value, and ISP301-2 corresponds the first pixel value to (8.8, 20.2). Because (8.8, 20.2) is not an integer coordinate, ISP301- 2 Need to use the pixel interpolation algorithm to find the pixel values at four locations a (8, 20), b (9, 20), c (8, 21), and d (9, 21) near (8.8, 20.2).
  • pixel interpolation algorithms such as the nearest neighbor element method, bilinear interpolation method, cubic interpolation method, and so on.
  • the nearest neighbor element method when ISP301-2 solves the pixel values of a, b, c, and d, it depends on the nearby pixel values. For example, d (9,21) is closer to (8.8,20.2), so the pixel value of d (9,21) can be the pixel value of (8.8,20.2), that is, the pixel value of (8,19).
  • the corrected position coordinate corresponding to (9, 20) is (8.3, 20.1).
  • ISP301-2 determines that the pixel value corresponding to (9,20) is the second pixel value, and ISP301-2 corresponds to the second pixel value to (8.3, 20.1). Because a (8,20) is located near (8.3,20.1), the pixel value of a (8,20) can be the pixel value corresponding to (8.3,20.1), that is, the pixel value corresponding to (9,20).
  • the ISP301-2 keeps the pixel values corresponding to the original position coordinates of these points unchanged.
  • the execution order between the eighth step and the fourth to seventh steps is not limited.
  • Steps 6-7 the ISP obtains the corrected position coordinates and pixel values of the coordinate points that need to be corrected.
  • Step 8 the ISP obtains the original position coordinates and pixel values of the coordinate points that do not need to be corrected.
  • the corrected position coordinates and pixel values of the coordinate points that need to be corrected, and the original coordinate positions and pixel values of the coordinate points that do not need to be corrected are used to generate a corrected image, that is, the ninth step.
  • scenario 1 a scene in which a camera in a mobile phone starts to capture an image.
  • scenario 2 When the screen of the mobile phone is locked, a user's face image is collected through a face recognizer and unlocked.
  • the mobile phone can run the code of the image correction algorithm provided in the embodiment of the present application to perform image correction on a captured user face image to obtain a corrected user face image. Then, the mobile phone determines the corrected user face Whether the partial image can be unlocked (for example, whether the face image of the user after correction matches the pre-stored image).
  • Scenario 3 When a mobile phone makes a video call (WeChat, QQ video call) with other mobile phones, each time the mobile phone acquires an image, it can run the image correction algorithm code provided in the embodiment of this application to perform image correction on the image. Then, the one mobile phone sends the corrected image to other mobile phones.
  • a video call WeChat, QQ video call
  • the terminal may include a hardware structure and / or a software module, and implement the foregoing functions in the form of a hardware structure, a software module, or a hardware structure and a software module. Whether one of the above functions is executed by a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application of the technical solution and design constraints.
  • FIG. 11 shows a terminal 1100 provided by the present application.
  • the terminal 1100 may include: an input device 1101 and a camera 1102, one or more processors 1103, a memory 1104, and one or more computer programs.
  • the above devices may be connected through one or more communication buses 1106.
  • the one or more computer programs 1105 are stored in the memory 1104 and configured to be executed by the one or more processors 1103.
  • the one or more computer programs 1105 include instructions. 5 and the corresponding steps in the respective embodiments.
  • An embodiment of the present invention further provides a computer storage medium.
  • the storage medium may include a memory, and the memory may store a program.
  • the terminal executes the method including the method described in the method embodiment shown in FIG. 5. All steps performed by the terminal.
  • An embodiment of the present invention further provides a computer program product that, when the computer program product runs on a terminal, causes the terminal to execute all operations performed by the terminal, as described in the method embodiment shown in FIG. 5 above. step.
  • Computer-readable media includes computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
  • a storage media may be any available media that can be accessed by a computer.
  • computer-readable media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), read-only memory (EEPROM), compact disc-read-only memory (CD-ROM) ROM) or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer. Also. Any connection is properly a computer-readable medium.
  • disks and discs include compact discs (CDs), laser discs, optical discs, digital video discs (DVDs), floppy discs, and Blu-ray discs, among which Discs usually reproduce data magnetically, while discs use lasers to reproduce data optically.
  • CDs compact discs
  • DVDs digital video discs
  • floppy discs floppy discs
  • Blu-ray discs among which Discs usually reproduce data magnetically, while discs use lasers to reproduce data optically. The above combination should also be included in the protection scope of the computer-readable medium.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

一种图像校正方法和终端。该方法包括：终端根据每个坐标点的畸变程度和畸变程度阈值，确定需要校正的坐标点和不需要校正的坐标点；根据需要校正的坐标点的畸变程度和连续函数，确定需要校正的坐标点的校正权重；根据需要校正的坐标点的原始位置坐标、理想位置坐标以及与校正权重对应的校正公式，确定需要校正的坐标点的校正后的位置坐标；将原始图像上位于需要校正的坐标点的原始位置坐标对应的像素值设置为校正后的位置坐标对应的像素值，并保持原始图像上的不需要校正的坐标点的像素值不变。通过这种方式，终端无需对所有坐标点都校正，节省计算量，且对于每个需要校正的坐标点采用不同的校正公式进行校正，有助于提高拍摄图像的质量。

Description

一种图像校正方法和终端 技术领域
本申请涉及终端技术领域，尤其涉及一种图像校正方法和终端。
背景技术
随着终端技术的进步，终端的各种功能不断丰富。其中，终端的拍照功能已经成为用户使用频率较高的功能之一，而且用户对拍摄图像的质量要求越来越高。
目前，影响图像质量的一个重要因素是镜头畸变。产生镜头畸变的原因有多种，比如，镜头在装配过程中导致镜头倾斜。请参见图1所示，为一种镜头畸变的示意图，在图1中的(a)中，在理想情况下，光线通过凸透镜成像后，成像点在A点(A点距离成像中心O点的距离为a)，而在图1中的(b)中，由于凸透镜发生倾斜，光线通过该凸透镜成像后，成像点在B点(B点距离成像中心O点的距离为b，b大于a)，所以图1(b)中的成像相对于图1(a)的成像发生畸变。
由此可见，如何减小镜头畸变以提升拍摄图像的质量，是有待解决的问题。
发明内容
本申请提供一种图像校正方法和终端，用以校正发生畸变的图像，提升拍摄的图像的质量。
第一方面，本申请实施例提供一种图像校正方法，该方法适用于终端，所述方法包括：终端响应于用户操作，启动相机应用，打开摄像头，通过所述摄像头采集原始图像；所述终端根据所述原始图像上的所有坐标点中每个坐标点的原始位置坐标和畸变转换公式，确定所述每个坐标点的理想位置坐标；其中，所述畸变转变公式用于表示坐标点的原始位置坐标和理想位置坐标之间的转换关系；所述终端根据所述每个坐标点的原始位置坐标和理想位置坐标，确定所述每个坐标点的畸变程度；所述终端根据所述每个坐标点的畸变程度和畸变程度阈值，确定所述所有坐标点中需要校正的坐标点和不需要校正的坐标点；所述终端根据所述需要校正的坐标点的畸变程度和连续函数，确定所述需要校正的坐标点的校正权重；其中，所述连续函数用于表示畸变程度和校正权重之间的转换关系；所述终端根据所述需要校正的坐标点的原始位置坐标、理想位置坐标以及与所述校正权重对应的校正公式，确定所述需要校正的坐标点的校正后的位置坐标；其中，所述校正公式用于指示所述需要校正的坐标点的原始位置坐标、理想位置坐标以及校正后的位置坐标之间的相对位置关系；所述终端将所述原始图像上位于所述需要校正的坐标点的原始位置坐标对应的像素值设置为所述校正后的位置坐标对应的像素值，并；保持所述原始图像上的不需要校正的坐标点的像素值不变；所述终端根据所述需要校正的坐标点的校正后的位置坐标和像素值，以及所述不需要校正的坐标点的坐标位置和像素值，生成校正后的图像。
在本申请实施例中，终端可以对原始图像上的坐标点进行判断，确定出需要校正的坐标点和不需要校正的坐标点。对于不需要校正的坐标点，终端可以保持这些坐标点的像素值不变。对于需要校正的坐标点，终端确定每个需要校正的坐标点的校正权重以及与该校正权重对应的校正公式，并通过该校正公式对需要校正的坐标点进行校正。由于每个需要校正的坐标点的校正权重不同，所以每个需要校正的坐标点的校正公式也不同。通过这种 方式，一方面，终端无需对所有的坐标点都校正，节省计算量，提高效率。另一方面，对于不同的需要校正的坐标点，终端采用不同的校正公式进行校正，有助于提高拍摄图像的质量。
在一种可能的设计中，所述终端在根据所述每个坐标点的原始位置坐标和理想位置坐标，确定所述每个坐标点的畸变程度时，执行步骤包括：确定所述每个坐标点的原始位置坐标和理想位置坐标之间的距离，所述距离为所述每个坐标点的畸变程度。
在本申请实施例中，终端确定每个坐标点的原始位置坐标和理想位置坐标之间的距离，该距离即每个坐标点的畸变程度，对于原始位置坐标和理想位置坐标之间的距离较近的坐标点，说明该坐标点无需校正。通过这种方式，终端无需对所有的坐标点都校正，节省计算量，提高效率。
在一种可能的设计中，所述终端在根据所述每个坐标点的畸变程度和畸变程度阈值，确定所述所有坐标点中需要校正的坐标点时，执行步骤包括：确定所述所有坐标点中畸变程度大于畸变程度阈值的第一坐标点，所述第一坐标点即为所述需要校正的坐标点；确定所述所有坐标点中畸变程度小于等于所述畸变程度阈值的第二坐标点，所述第二坐标点即为所述不需要校正的坐标点。
在本申请实施例中，终端中可以存储畸变程度阈值，对于畸变程度大于畸变程度阈值的坐标点即需要校正的坐标点，对于畸变程度小于等于畸变程度阈值的坐标点即不需要校正的坐标点。通过这种方式，终端无需对所有坐标点进行校正，有助于节省计算量，提高效率。
在一种可能的设计中，若所述需要校正的坐标点的畸变程度为d，所述终端根据所述需要校正的坐标点的畸变程度和连续函数，确定所述需要校正的坐标点的校正权重，包括：
所述终端将所述d代入如下的连续函数中：
Figure PCTCN2018096039-appb-000001
其中，畸变程度阈值为d0，得到所述需要校正的坐标点的校正权重f。
在本申请实施例中，终端中存储有连续函数，当终端确定一个需要校正的坐标点的畸变程度为d时，终端将该d代入到所述连续函数中，得到该需要校正的坐标点的校正权重f。在这种方式中，由于每个需要校正的坐标点的畸变程度不同，所以通过所述连续函数，确定出的每个需要校正的坐标点的校正权重不同，即每个需要校正的坐标点的校正公式不同。通过这种方式，对于每个需要校正的坐标点采用不同的校正公式进行校正，有助于提高拍摄图像的质量。
在一种可能的设计中，若需要校正的坐标点的原始位置坐标点为A、理想位置坐标点为B、校正后的位置坐标点为C，校正权重为f，则与所述校正权重f对应的校正公式为：
Figure PCTCN2018096039-appb-000002
其中，
Figure PCTCN2018096039-appb-000003
表示从A点到B点的矢量，
Figure PCTCN2018096039-appb-000004
表示从A点到C点的矢量。
在本申请实施例中，终端确定需要校正的坐标点的校正权重后，得到与所述校正权重对应的校正公式，通过该校正公式计算需要校正的坐标点的校正后的位置坐标。由于每个需要校正的坐标点的校正权重不同，所以每个需要校正的坐标点的校正公式不同。通过这种方式，对不同的需要校正的坐标点采用不同的校正公式进行校正，有助于提高拍摄图像的质量。
在一种可能的设计中，所述终端在将所述原始图像上位于所述需要校正的坐标点的原 始位置坐标对应的像素值设置为所述校正后的位置坐标对应的像素值时，具体执行步骤包括：若所述校正后的位置坐标为整数，确定所述原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值为所述校正后的位置坐标对应的像素值；若所述校正后的位置坐标为非整数，通过像素插值算法将原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值插值到所述校正后的位置坐标附近的、横纵坐标均为整数的其它位置坐标处。
在本申请实施例中，若所述校正后的位置坐标为整数，则所述原始图像上位于所述需要校正的坐标点的原始位置坐标对应的像素值即为所述校正后的位置坐标对应的像素值；若所述校正后的位置坐标为非整数，所述终端通过像素插值算法将位于所述原始位置坐标处的像素值插值到所述校正后的位置坐标附近的、横纵坐标均为整数的其它位置坐标处。通过这种方式，终端对每个需要校正的坐标点采用不同的措施，有助于提高拍摄图像的质量。
第二方面，本申请实施例提供一种终端，包括处理器和存储器。其中，存储器用于存储一个或多个计算机程序；当存储器存储的一个或多个计算机程序被处理器执行时，使得终端能够实现第一方面或者第一方面的任意一种可能的设计的方法。
第三方面，本申请实施例还提供了一种终端，所述终端包括执行第一方面或者第一方面的任意一种可能的设计的方法的模块/单元。这些模块/单元可以通过硬件实现，也可以通过硬件执行相应的软件实现。
第四方面，本申请实施例中还提供一种计算机可读存储介质，所述计算机可读存储介质包括计算机程序，当计算机程序在终端上运行时，使得所述终端执行第一方面或上述第一方面的任意一种可能的设计的方法。
第五方面，本申请实施例还提供一种包含计算机程序产品，当所述计算机程序产品在终端上运行时，使得所述终端执行第一方面或上述第一方面的任意一种可能的设计的方法。
附图说明
图1为本发明一实施例提供的一种镜头畸变的示意图；
图2为本发明一实施例提供的一种相机成像的示意图；
图3为本发明一实施例提供的一种手机的结构示意图；
图4为本发明一实施例提供的一种手机的结构示意图；
图5为本发明一实施例提供的一种图像校正方法的流程示意图；
图6为本发明一实施例提供的一种径向畸变和切换畸变的示意图；
图7为本发明一实施例提供的一种用于摄像头标定的黑白方格示意图；
图8为本发明一实施例提供的连续函数在二维坐标系中的曲线示意图；
图9为本发明一实施例提供的各个坐标点之间的相对位置的示意图；
图10为本发明一实施例提供的一种像素插值过程的示意图；
图11为本发明一实施例提供的一种终端的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。
以下，对本申请实施例中的部分用语进行解释说明，以便于本领域技术人员理解。
本申请实施例涉及的像素，为一张图像上的最小成像单元。一个像素可以对应图像上 的一个坐标点。像素可以包括一个参数(比如灰度)，也可以是多个参数的集合(比如灰度、亮度、颜色等)。如果像素包括一个参数，那么像素值就是该参数的取值，如果像素是多个参数的集合，那么像素值包括所述集合中每个参数的取值。
本申请实施例涉及的像平面坐标系，即建立于相机成像平面上的坐标系。关于成像平面，请参见图2所示，为本申请实施例提供的一种相机成像的示意图。如图2所示，相机拍摄人物时，通过镜头采集人物图像，并将采集的图像呈现在成像平面上。
本申请实施例涉及的原始图像，即本申请实施例提供的图像校正算法的输入图像，即终端在运行图像校正算法的代码时，对该输入图像进行图像校正处理。作为一种示例，原始图像可以是摄像头中的镜头组采集待拍摄物体反射的光信号后，摄像头中的图像传感器根据所述光信号生成待拍摄物体的图像。当然，原始图像还可以是摄像头中的图像传感器根据所述光信号生成待拍摄物体的图像经过其它算法之后的输出图像，本发明实施例对此不做限定。
本申请实施例涉及的校正后的图像，即终端运行本申请实施例提供的图像校正算法的代码对原始图像进行校正后的输出图像。
本申请实施例涉及的原始位置坐标，即原始图像上的每个像素点在像平面坐标系中的坐标位置，在下文中，原始位置坐标用(x 0，y 0)表示。
本申请实施例涉及的理想位置坐标，即理想情况下，原始图像上的每个像素点应当在像平面坐标系中的坐标位置，在下文中，理想位置坐标用(x，y)表示。
本申请实施例涉及的校正后的位置坐标，即校正后的图像上的每个像素点在像平面坐标系中的坐标位置，在下文中，校正后的位置坐标用(x’，y’)表示。
本申请实施例涉及的多个，是指大于或等于两个。
另外，需要理解的是，在本申请的描述中，“第一”、“第二”等词汇，仅用于区分描述的目的，而不能理解为指示或暗示相对重要性，也不能理解为指示或暗示顺序。
以下介绍电子设备、用于这样的电子设备的图形用户界面(graphical user interface，GUI)、和用于使用这样的电子设备的实施例。在本申请一些实施例中，电子设备可以是还包含其它功能诸如个人数字助理和/或音乐播放器功能的便携式电子设备，诸如手机、平板电脑、具备无线通讯功能的可穿戴设备(如智能手表)等。便携式电子设备的示例性实施例包括但不限于搭载iOS、Android、Microsoft或者其它操作***的便携式电子设备。上述便携式电子设备也可以是其它便携式电子设备，诸如具有触敏表面(例如触控面板)的膝上型计算机(Laptop)等。还应当理解的是，在本申请其他一些实施例中，上述电子设备也可以不是便携式电子设备，而是具有触敏表面(例如触控面板)的台式计算机。
示例性地，如图3所示，本申请实施例中的电子设备可以为手机300。下面以手机300为例对实施例进行具体说明。应该理解的是，图示手机300仅是电子设备的一个范例，并且手机300可以具有比图中所示出的更多的或者更少的部件，可以组合两个或更多的部件，或者可以具有不同的部件配置。图中所示出的各种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
如图3所示，手机300具体可以包括：一个或多个处理器301、射频(radio frequency，RF)电路302、存储器303、触摸屏304、蓝牙装置305、一个或多个传感器306、Wi-Fi装置307、定位装置308、音频电路309、外设接口310以及电源装置311等部件。这些部件可通过一根或多根通信总线或信号线(图3中未示出)进行通信。本领域技术人员可以 理解，图3中示出的硬件结构并不构成对手机300的限定，手机300可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
下面结合图3对手机300的各个部件进行具体的介绍：
处理器301是手机300的控制中心，利用各种接口和线路连接手机300的各个部分，通过运行或执行存储在存储器303内的应用程序(Application，简称App)，以及调用存储在存储器303内的数据和指令，执行手机300的各种功能和处理数据。在一些实施例中，处理器301可包括一个或多个处理单元。处理器301可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(Neural-network Processing Unit，NPU)等。其中，处理器301包括的不同的处理单元可以是独立的器件，也可以是集成在同一个处理器中。比如，处理器301可以集成GPU和DSP，或者可以集成AP和ISP。以处理器301集成应用处理器和调制解调处理器为例；其中，应用处理器主要处理操作***、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器301中。在本申请其他一些实施例中，上述处理器301还可以包括指纹验证芯片，用于对采集到的指纹进行验证。
射频电路302可用于在收发信息或通话过程中，无线信号的接收和发送。具体地，射频电路302可以将基站的下行数据接收后，给处理器301处理；另外，将涉及上行的数据发送给基站。通常，射频电路包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器、双工器等。此外，射频电路302还可以通过无线通信和其他设备通信。所述无线通信可以使用任一通信标准或协议，包括但不限于全球移动通讯***、通用分组无线服务、码分多址、宽带码分多址、长期演进、电子邮件、短消息服务等。
存储器203用于存储应用程序以及数据，处理器301通过运行存储在存储器303的应用程序以及数据，执行手机300的各种功能以及数据处理。存储器303主要包括存储程序区以及存储数据区，其中，存储程序区可存储操作***、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)；存储数据区可以存储根据使用手机300时所创建的数据(比如音频数据、电话本等)。此外，存储器303可以包括高速随机存取存储器，还可以包括非易失存储器，例如磁盘存储器件、闪存器件或其他非易失性固态存储器件等。存储器303可以存储各种操作***，例如苹果公司所开发的IOS操作***，谷歌公司所开发的Android操作***等。示例性地，存储器303中存储了与本申请实施例相关的应用程序，例如Taskcard store、推特、电话本、微博等。
触摸屏304可以包括触敏表面304-1和显示器304-2。其中，触敏表面304-1(例如触控面板)可采集手机300的用户在其上或附近的触摸事件(比如用户使用手指、触控笔等任何适合的物体在触敏表面304-1上或在触敏表面304-1附近的操作)，并将采集到的触摸信息发送给其他器件例如处理器301。其中，用户在触敏表面304-1附近的触摸事件可以称之为悬浮触控；悬浮触控可以是指，用户无需为了选择、移动或拖动目标(例如App图标等)而直接接触触控板，而只需用户位于电子设备附近以便执行所想要的功能。触敏表面304-1可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再发送给处理器301，触摸控制器还可以 接收处理器301发送的指令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型来实现触敏表面304-1。显示器(也称为显示屏)304-2可用于显示由用户输入的信息或提供给用户的信息以及手机300的各种菜单。可以采用液晶显示器、有机发光二极管等形式来配置显示器304-2。触敏表面304-1可以覆盖在显示器304-2之上，当触敏表面304-1检测到在其上或附近的触摸事件后，传送给处理器301以确定触摸事件的类型，随后处理器301可以根据触摸事件的类型在显示器304-2上提供相应的视觉输出。虽然在图3中，触敏表面304-1与显示屏304-2是作为两个独立的部件来实现手机300的输入和输出功能，但是在某些实施例中，可以将触敏表面304-1与显示屏304-2集成而实现手机300的输入和输出功能。在本申请其他一些实施例中，触摸屏304还可以包括一系列的压力传感器阵列，可以使得手机感测触摸事件所施加给触摸屏304的压力。
手机300还可以包括蓝牙装置305，用于实现手机300与其他短距离的电子设备(例如手机、智能手表等)之间的数据交换。本申请实施例中的蓝牙装置可以是集成电路或者蓝牙芯片等。
手机300还可以包括至少一种传感器306，比如光传感器、运动传感器以及其他传感器。具体地，光传感器可包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节触摸屏304的显示器的亮度，接近传感器可在手机300移动到耳边时，关闭显示器的电源。作为运动传感器的一种，加速计传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向，可用于识别手机姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等。
在本申请一些实施例中，传感器306还可以包括指纹传感器。例如，可以在手机300的背面(例如后置摄像头的下方)配置指纹传感器312，或者在手机300的正面(例如触摸屏304的下方)配置指纹传感器312。另外，也可以通过在触摸屏304中配置指纹传感器312来实现指纹识别功能，即指纹传感器312可以与触摸屏304集成在一起来实现手机300的指纹识别功能。在这种情况下，该指纹传感器312可以配置在触摸屏304中，可以是触摸屏304的一部分，也可以以其他方式配置在触摸屏304中。另外，该指纹传感器312还可以被实现为全面板指纹传感器，因此，可以把触摸屏304看成是任何位置都可以进行指纹采集的一个面板。该指纹传感器312可以将采集到的指纹发送给处理器301，以便处理器301对该指纹进行处理(例如指纹验证等)。本申请实施例中的指纹传感器312可以采用任何类型的感测技术，包括但不限于光学式、电容式、压电式或超声波传感技术等。另外，关于本申请实施例中在触摸屏中集成指纹传感器的具体技术方案，可以参见美国专利与商标局公告的申请号为US 2015/0036065 A1，名称为“在电子设备中的指纹传感器”的专利申请，其全部控件通过引用结合在本申请各个实施例中。
至于手机300还可配置的陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器，在此不予赘述。
Wi-Fi装置307，用于为手机200提供遵循Wi-Fi相关标准协议的网络接入，手机300可以通过Wi-Fi装置307接入到Wi-Fi接入点，进而帮助用户收发电子邮件、浏览网页和访问流媒体等，它为用户提供了无线的宽带互联网访问。在其他一些实施例中，该Wi-Fi装置307也可以作为Wi-Fi无线接入点，可以为其他电子设备提供Wi-Fi网络接入。
定位装置308，用于为手机300提供地理位置。可以理解的是，该定位装置308具体 可以是全球定位***(global positioning system，GPS)、北斗卫星导航***等定位***的接收器。
音频电路309、扬声器313、麦克风314可提供用户与手机300之间的音频接口。音频电路309可将接收到的音频数据转换后的电信号，传输到扬声器313，由扬声器313转换为声音信号输出；另一方面，麦克风314将收集的声音信号转换为电信号，由音频电路309接收后转换为音频数据，再将音频数据输出至RF电路302以发送给比如一个手机，或者将音频数据输出至存储器303以便进一步处理。
外设接口310，用于为外部的输入/输出设备(例如键盘、鼠标、外接显示器、外部存储器、用户识别模块卡等)提供各种接口。例如通过通用串行总线接口与鼠标连接，通过用户识别模块卡卡槽上的金属触点与电信运营商提供的用户识别模块(subscriber identity module，SIM)卡连接。外设接口310可以被用来将上述外部的输入/输出***设备耦接到处理器301和存储器303。
手机300还可以包括给各个部件供电的电源装置311(比如电池和电源管理芯片)，电池可以通过电源管理芯片与处理器301逻辑相连，从而通过电源装置311实现管理充电、放电、以及功耗管理等功能。
手机300还可以包括摄像头315，其中，摄像头315包括镜头组315-1和图像传感器315-2，其中，镜头组315-1包括多个透镜(凸透镜或凹透镜)，用于采集待拍摄物体反射的光信号，并将采集的光信号传递给图像传感器315-2。图像传感器315-2根据所述光信号生成待拍摄物体的原始图像。图像传感器315-2将生成的原始图像发送给处理器301，处理器301运行本申请实施例提供的图像校正算法的代码对原始图像进行校正，得到校正后的图像。
其中，摄像头315可以是前置摄像头或后置摄像头。其中，前置摄像头还可以用于捕捉人脸特征信息，处理器301可以对该人脸特征信息进行人脸识别，进而进行后续处理。手机300还可以包括闪光灯、微型投影装置、近场通信(near field communication，NFC)装置等，在此不予赘述。
以下实施例均可以在具有上述硬件结构的电子设备(例如手机300、平板电脑等)中实现。
为了方便描述本申请实施例提供的图像校正算法，从图3中示出的手机300中的所有部件中提取出与所述图像校正算法相关的部件，下文将通过这些部件介绍本申请实施例的图像校正算法，具体请参见图4。需要说明的是，在图4中，以处理器301集成应用处理器301-1和ISP301-2为例。
在本申请一些实施例中，通过图4所示的手机300拍摄得到一张校正后的图像可以是如下过程：
用户通过触敏表面304-1点击相机应用的图标，触发应用处理器301-1启动相机应用，打开摄像头315，显示器304-2显示相机应用的界面，例如取景界面。摄像头315中的镜头组315-1采集待拍摄物体反射的光信号，并将采集的光信号传递给图像传感器315-2。图像传感器315-2根据所述光信号生成待拍摄物体的原始图像，该原始图像是有畸变的。图像传感器315-2将原始图像发送给ISP301-2。
在本申请一些实施例中，应用处理器301-1运行本申请实施例提供的图像校正算法的代码(比如，本申请实施例提供的图像校正算法的代码存储在存储器303中，应用处理器 301-1运行存储在存储器303中的所述图像校正算法的代码)，得到每个坐标点的原始位置坐标(x 0，y 0)和校正后的位置坐标(x’，y’)之间的映射关系。应用处理器301-1可以将所述映射关系发送给ISP301-2。ISP301-2接收到原始图像和所述映射关系之后，根据所述映射关系对原始图像进行图像校正，即将所述原始图像上位于(x 0，y 0)上的像素值移动到所述校正后的位置坐标(x’，y’)处，得到校正后的图像，并将所述校正后的图像发送给应用处理器301-1。显示器304-2显示在取景界面的取景框中显示所述校正后的图像。
当然，应用处理器301-1得到原始位置坐标(x 0，y 0)和校正后的位置坐标(x’ y’)之间的映射关系之后，也可以将该映射关系存储在存储器303中，ISP301-2接收到原始图像后，从存储器303中获取所述映射关系。然后，ISP301-2根据所述映射关系将所述原始图像上位于(x 0，y 0)上的像素值移动到所述校正后的位置坐标(x’，y’)处，得到校正后的图像。
当然，ISP301-2也可以自己运行本申请实施例提供的图像校正算法的代码，得到原始位置坐标(x 0，y 0)和校正后的位置坐标(x’，y’)之间的映射关系。ISP301-2接收到原始图像之后，根据所述映射关系将所述原始图像上位于(x 0，y 0)上的像素值移动到所述校正后的位置坐标(x’，y’)处，得到校正后的图像，并将所述校正后的图像发送给应用处理器301-1。显示器304-2显示在取景界面的取景框中显示所述校正后的图像。
显示器304-2显示所述校正后的图像即预览图像，也就是说，从相机应用启动到预览图像呈现，手机300执行一遍上述过程。当用户触发相机中的拍摄按钮时，相机将所述校正后的图像存储下来。
通常，用户在启动相机应用后，会移动位置以寻找合适的拍摄角度，所以摄像头315采集到的原始图像是发生变化的。因此，应用处理器301-1可以实时的或者周期性的检测采集的原始图像是否发生变化，每当原始图像发生变化，ISP301-2可以根据所述映射关系对变化的原始图像进行图像校正，即尽可能的保证预览图像是校正后的图像。应用处理器301-1检测采集的原始图像是否发生变化的方式可以有多种，比如，图像传感器315-2可以将每次采集的原始图像发送给应用处理器315-1，应用处理器315-1判断连续两次接收到的原始图像的相似度较小，说明原始图像发送变化；或者，手机200中的运动传感器可以检测当前是否处于运动状态，若是，则原始图像发生变化，当然还可以有其他方式，本申请实施例对此不限定。在这个实施例中，手机300可以实时的对采集的原始图像进行图像校正，尽可能的保证用户看到的预览图像为校正后的图像，有助于提高用户体验。
在本申请另一些实施例中，通过图4所示的手机300拍摄得到一张校正后的图像还可以是如下过程：
显示器304-2在拍照预览过程中显示所述原始图像，此时不对所述原始图像进行校正。当应用处理器301-1判断用户的拍照操作时，才触发ISP301-2根据所述映射关系对所述原始图像进行校正。也就是说，在这个实施例中，手机300得到拍摄图像的过程中，只进行一次图像校准过程，计算量较小。
下面介绍应用处理器301-1运行本申请实施例提供的图像校正算法的代码，得到原始位置坐标(x 0，y 0)和校正后的位置坐标(x’，y’)之间的映射关系的过程。请参见图5所示，为本申请实施例提供的图像校正方法的流程示意图。如图5所示，应用处理器201运行图像校正算法的代码以执行如下过程：
第一步：根据原始图像上每个坐标点的原始位置坐标(x 0，y 0)，畸变系数和畸变的转 换公式，确定每个坐标点的理想位置坐标(x，y)。
在本申请一些实施例中，手机300中存储有哪一种转换公式，便采用哪种转换公式。这是因为，在摄像头设计好之后，手机200在出厂之前，设计人员一般会对手机300中的摄像头进行摄像头标定。摄像头标定，即，摄像头的畸变系数的确定过程。因为，摄像头设计好之后，并不清楚通过该摄像头拍摄图像的话会发生何种类型畸变。由于图像的畸变类型有多种，每种畸变类型对应的转换公式不同。因此，设计人员可以选择采用某种畸变类型的转换公式对手机300的摄像头进行标定。设计人员采用哪种畸变类型的转换公式对手机300的摄像头进行标定，便在手机300中存储哪种畸变类型的转换公式。因此，通过摄像头标定过程确定该摄像头的畸变系数后，将畸变系数和采用的转换公式存储在手机300中。手机300出厂之后，相机拍摄一张图像时，可以利用畸变系数和所述转换公式进行图像校正，得到校正后的图像。
通常，图像的畸变类型可以分为径向畸变和切向畸变。图6示出了一种径向畸变和切向畸变的示意图。下面介绍手机300通过摄像头标定，得到畸变系数的过程。
一种可能的标定方式为，设计人员采用径向畸变对应的转换公式进行标定。手机300拍摄图7所示的黑白方格图片，得到拍摄图像，该拍摄图像可能存在畸变。
需要说明的是，图7所示的黑白方格上的每个交叉点的坐标可以提前设定好(比如，设计人员可以自行设定)，而拍摄图像上的交叉点的坐标在像平面坐标系中，手机300可以在像平面坐标系中确定每个交叉点的原始位置坐标。由于黑白方格图片上的交叉点的坐标不在像平面坐标系中，所以设计人员可以设置一参考坐标系(比如世界坐标系)，将黑白方格图片上的交叉点的坐标转换到该参考坐标系中，同时将拍摄图像上交叉点的坐标转换到该参考坐标系中。因此，在参考坐标系中，图7所示的黑白方格图片上每个交叉点的坐标可以作为理想位置坐标(x，y)，而拍摄图像上的每个交叉点的坐标可以作为原始位置坐标(x 0，y 0)。然后，手机300可以通过径向畸变对应的转换公式计算得到畸变系数。其中，径向畸变对应的转换公式如下：
Figure PCTCN2018096039-appb-000005
其中，
Figure PCTCN2018096039-appb-000006
k 1、k 2、k 3为畸变系数。公式(1)只是径向畸变对应的转换公式的一种示例，径向畸变对应的转化公式还可以是其它形式，比如还可以下面的公式：
Figure PCTCN2018096039-appb-000007
以转化公式是公式(1)为例，手机300需要计算出k 1、k 2、k 3三个畸变系数的取值，所以需要至少3个方程。因此，手机300可以确定待拍摄物体(即图4所示的黑白方格图形)上的两个交叉点的坐标，比如(x，y)分别为(1，2)和(1，4)。手机200确定拍摄得到的具体畸变的黑白方格图形上与(1，2)对应的坐标为(1.5，2.7)，与(1，4)对应的坐标为(1.8，4.2)。那么终端300将这四个坐标带入到公式(1)中，得到如下的方程组：
Figure PCTCN2018096039-appb-000008
通过求解上述方程组，可以得到k 1、k 2、k 3的取值，比如k 1的取值为0.00604，k 2的 取值为-0.01833，k 3的取值为0.00000。手机300得到k 1、k 2、k 3的取值后，即标定过程结束，即手机300将k 1、k 2、k 3的取值带入到公式(1)中，得到转换公式(3)，将转换公式(3)存储以便使用。
Figure PCTCN2018096039-appb-000009
另一种可能的标定方式为，设计人员采用切向畸变对应的转换公式进行标定。手机300拍摄图7所示的黑白方格图片，得到拍摄图像，该拍摄图像可能具有畸变。类似的，在参考坐标系中，图7所示的黑白方格图片上每个交叉点的坐标可以作为理想位置坐标(x，y)，而拍摄图像上的每个交叉点的坐标可以作为原始位置坐标(x 0，y 0)。然后，手机300通过切向畸变的转化公式计算得到畸变系数。其中，切向畸变的转化公式如下：
Figure PCTCN2018096039-appb-000010
其中，
Figure PCTCN2018096039-appb-000011
p 1、p 2是畸变系数。公式(4)只是切向畸变的转化公式的一种示例，切向畸变的转化公式还可以是其它形式，本申请实施例不做限定。
在这种标定方式中，手机300需要计算出p 1、p 2两个畸变系数的取值，所以需要至少两个方程。因此，手机300可以确定待拍摄物体(即图7所示的黑白方格图形)上的至少一个交叉点的坐标，比如(x，y)为(1，2)。手机200确定拍摄得到的具体畸变的黑白方格图形上与(1，2)对应的坐标为(1.5，2.7)。那么手机200将这两个坐标带入到公式(4)中，得到如下的方程组：
Figure PCTCN2018096039-appb-000012
通过求解上述方程组，可以得到p 1、p 2的取值。比如p 1的取值为-0.00037，p 2的取值为-0.00095。手机300得到p 1、p 2的取值之后，即标定过程结束，即手机300将p 1、p 2的取值带入到公式(4)中，得到转换公式(5)，将转换公式(5)存储以便使用。
Figure PCTCN2018096039-appb-000013
又一种可能的标定方式为，设计人员采用由径向畸变对应的转换公式和切向畸变的转换公式结合得到的公式进行标定。类似的，设计人员将图7所示的黑白方格图片作为手机300的待拍摄物体，进行拍摄，得到拍摄图像，该拍摄图像可能具有畸变。手机300通过由径向畸变对应的转换公式和切向畸变的转换公式结合得到的公式计算得到畸变系数。作为一种示例，由径向畸变对应的转换公式和切向畸变的转换公式结合得到的公式如下：
Figure PCTCN2018096039-appb-000014
其中，
Figure PCTCN2018096039-appb-000015
k 1、k 2、k 3、p 1、p 2是畸变系数。公式(6)只是一种示例，由径向畸变对应的转换公式和切向畸变的转换公式结合得到的公式还可以是其它形式，比如，还可以是如下形式：
Figure PCTCN2018096039-appb-000016
以公式(7)为例，在这种标定方式中，手机300需要计算出k 1、k 2、k 3、p 1、p 2五个畸变系数的取值，所以需要至少5个方程，即图7所示的黑白方格图片上的至少三个交叉点的坐标，比如(x，y)分别为(1，2)、(1，4)、(1、6)。手机300确定拍摄得到的具有畸变的黑白方格图形上与(1，2)对应的坐标为(1.5，2.7)，与(1，4)对应的坐标为(1.8，4.2)，与(1，6)对应的坐标为(1.6，6)。那么终端200将这两个坐标带入到公式(5)中，得到如下的方程组：
Figure PCTCN2018096039-appb-000017
通上述方程组可以求出k 1、k 2、k 3、p 1、p 2五个畸变系数的取值。手机200得到k 1、k 2、k 3、p 1、p 2五个畸变系数的取值之后，即标定过程结束，即手机200将k 1、k 2、k 3、p 1、p 2五个畸变系数的取值带入到公式(7)中，得到转换公式，将转换公式存储以便使用。
以设计人员采用上述第一种摄像头标定方式为例，即手机300中存储公式(3)，即应用处理器301-1可以通过公式(3)和每个坐标点的原始位置坐标(x 0，y 0)，确定每个坐标点理想位置坐标(x，y)。
在本申请一些实施例中，应用处理器301-1可以确定原始图像上所有坐标点的理想位置坐标(x，y)。比如，手机300的摄像头是1920*1080的像素，那么应用处理器301-1可以计算原始图像上从(0，0)、(0，1)一直到(1920，1080)的所有坐标点各自对应的理想位置坐标。以其中一个坐标点(0，1)为例，应用处理器301-1将该坐标点(0，1)带入转换公式(5)中便可以得到与该坐标点(0，1)对应的理想位置坐标。通过这种方式，应用处理器301-1得到原始图像上每个坐标点的原始位置坐标(x 0，y 0)对应的理想位置坐标(x，y)。
第二步：根据每个坐标点的原始位置坐标(x 0，y 0)和理想位置坐标(x，y)，确定每个坐标点的畸变程度。
在本申请实施例中，畸变程度可以用于指示成像平面上每个坐标点的原始位置坐标和理想位置坐标之间的距离，所述距离越大，畸变程度越大，所述距离越小，畸变程度越小。比如，畸变程度可以用d表示，则
Figure PCTCN2018096039-appb-000018
即，在第二步中，应用处理器301-1可以得到每个坐标点的畸变程度d。
第三步，根据每个坐标点的畸变程度d和畸变程度阈值d 0，确定需要校正的坐标点和不需要校正的坐标点。请继续参见图5，对于需要校正的坐标点，执行第四-七步，对于不需要校正的坐标点，执行第八步。
在本申请一些实施例中，应用处理器301-1得到每个坐标点的畸变程度d后，可以将每个坐标点的畸变程度d与畸变程度阈值d 0进行比较，如果一个坐标点的畸变程度大于畸变程度阈值，那么该坐标点需要校正，如果一个坐标点的畸变程度小于等于畸变程度阈值，那么该坐标点不需要校正。比如，总共有1000个坐标点，其中300个坐标点的畸变程度 均小于等于畸变程度阈值，那么手机200可以不对这300个坐标点进行校正，即对这300个坐标点执行第八步。
其中，d 0即畸变程度阈值的取值可以是的设计人员根据经验或者实验获得的，存储在手机300中以便使用。
由此可见，在本申请实施例中，无需对一张图像上的每个坐标点进行校正，有助于节省工作量，提高效率。
第四步：根据需要校正的坐标点的畸变程度和连续函数，确定与所述需要校正的坐标点对应的校正权重。
在本申请实施例中，手机200中存储一个关于校正权重和畸变程度的函数，即该函数的两个变量分别为校正权重和畸变成像。示例性的，所述函数的一种可能的形式如下：
Figure PCTCN2018096039-appb-000019
其中，d表示一个坐标点的畸变程度，d 0是畸变程度阈值，f即该坐标点的校正权重。
作为一种示例，应用处理器301-1可以将在第二步中得到的需要校正的坐标点的畸变程度d和畸变程度阈值d 0带入到公式(8)中，得到需要校正的坐标点的校正权重f。
作为另一种示例，请参见图8所示，为函数(8)在二维坐标系中的曲线关系，其中，横坐标是畸变程度d，纵坐标是校正权重f。应用处理器301-1还可以根据第二步中得到的需要校正的坐标点的畸变程度d在图8所示的曲线上确定与所述畸变程度d对应的纵坐标，即需要校正的坐标点的校正权重。比如，需要校正的坐标点的畸变程度是d1，在图8所示的曲线上确定与畸变程度d1对应的纵坐标是f1。
第五步：根据需要校正的坐标点的校正权重f和校正公式，确定需要校正的坐标点对应的校正后的位置坐标(x’，y’)。
在本申请实施例中，应用处理器301-1可以根据需要校正的坐标点的原始位置坐标(x 0，y 0)和校正权重f，以及校正公式(9)确定需要校正的坐标点的校正后的位置坐标(x’，y’)。
Figure PCTCN2018096039-appb-000020
其中，A表示原始位置坐标点，即A点坐标为(x 0，y 0)，B表示校正后的位置坐标点，即B点坐标为(x’，y’)，C表示理想位置坐标点，即C点坐标为(x，y)。
Figure PCTCN2018096039-appb-000021
表示从A点到B点的矢量，
Figure PCTCN2018096039-appb-000022
表示从A点到C点的矢量。其中，f是在第四步中得到的一个坐标点的校正权重。
通过公式(9)可知，校正后的位置坐标(x’，y’)与原始位置坐标(x 0，y 0)之间的距离L和畸变程度d之间的距离的比值为校正权重f。
示例性的，图9示出了本申请实施例提供的各个坐标点之间的相对位置的示意图。
如图9所示，在像平面坐标系中包括8个原始位置坐标点，其中，原始位置坐标点用白色点表示。图9中的理想位置坐标点用黑色点表示，校正后的位置坐标点用方格点表示。
以原始坐标点是A点和D点为例，应用处理器301-1将A点的坐标带入转换公式(3)中，得到与A点对应的理想位置坐标点即C点的坐标。应用处理器301-1将D点的坐标带入转换公式(3)中，得到与D点对应的理想位置坐标即E点的坐标。应用处理器301-1根据A点的原始坐标位置和理想位置坐标确定A点的畸变程度dA，根据D点的原始坐标位置和理想位置坐标确定D点的畸变程度dD
假设A点的畸变程度dA大于畸变程度阈值d 0，所以A点需要校正。假设D点的畸变程度dD小于等于畸变程度阈值d 0，所以D点不需要校正。
以原始位置坐标点是A点为例，应用处理器301-1将A点的畸变程度dA和畸变程度阈值d0带入公式(8)中，得到A点的校正权重fA。应用处理器301-1根据公式(9)确定与A点的坐标对应的校正后的坐标点即B点的坐标。其中，A点与B点之间的距离L与A点和C点之间的距离dA(A点的畸变程度)之间的比值等于A点的校正权重fA。
通过以上描述可知，在本申请实施例中，每个需要校正的坐标点的校正权重不同。应用处理器301-1可以根据每个需要校正的坐标点的校正权重确定每个需要校正的坐标点的校正后的位置坐标。
通过上述五步，应用处理器301-1可以得到每个需要校正的坐标点的校正后的位置坐标(x’ y’)。应用处理器301-1可以建立需要校正的坐标点的原始位置坐标(x 0，y 0)与校正后的位置坐标(x’ y’)之间的映射关系。
如前述内容可知，应用处理器301-1得到原始位置坐标(x 0，y 0)与校正后的位置坐标(x’ y’)之间的映射关系之后，可以将所述映射关系发送给ISP301-2。ISP301-2接收到原始图像和所述映射关系之后，根据所述映射关系对原始图像进行图像校正。下面介绍ISP301-2根据所述映射关系对原始图像进行图像校正的过程，请继续参见图5所示，具体如下：
第六步，ISP301-2根据需要校正的坐标点的原始位置坐标(x 0，y 0)与校正后的位置坐标(x’ y’)之间的映射关系，将原始图像上位于所述需要校正的坐标点的原始位置坐标(x 0，y 0)上的像素值设置为校正后的位置坐标(x’ y’)的像素值。
需要说明的是，当用户通过手机200拍照时，图像传感器315-2将像素值对应到成像平面上的每个坐标点的原始位置坐标上，即得到原始图像。因此，ISP301-2可以根据需要校正的坐标点的原始位置坐标(x 0，y 0)与校正后的位置坐标(x’ y’)之间的映射关系，将原始图像上与需要校正的坐标点对应的像素值移动到校正后的位置坐标处。
通常，ISP301-2将像素值对应到一个坐标点时，要求该坐标点的横纵坐标取值是整数。因此，如果一个需要校正的坐标点的校正后的位置坐标(x’ y’)是整数，ISP301-2确定该需要校正的坐标点的原始位置坐标对应的像素值为所述校正后的位置坐标的像素值。由于校正后的位置坐标(x’ y’)是通过前述五步计算得到的，所以校正后的位置坐标(x’ y’)可能不是整数。因此，ISP301-2得到校正后的图像之前，还可以进行下述的第七步。
第七步：若校正后的位置坐标(x’ y’)不是整数，ISP301-2可以通过像素插值算法，将原始位置坐标(x 0，y 0)对应的像素值插值到校正后的位置坐标(x’ y’)附近的整数坐标处，得到校正后的图像。
举例来说，请参见图10所示，原始图像上的两个坐标点分别为(8,19)和(9,20).其中，(8,19)对应的校正后的位置坐标为(8.8，20.2)。ISP301-2确定(8,19)对应的像素值为第一像素值，ISP301-2将该第一像素值对应到(8.8，20.2)，由于(8.8，20.2)不是整数坐标，所以，ISP301-2需要通过像素插值算法求出位于(8.8，20.2)附近的a(8，20)、b(9，20)、c(8，21)、d(9，21)四个位置的像素值。其中，像素插值算法可以有多种，比如最邻近元法、双线性内插法、三次内插法等等。以最邻近元法为例，ISP301-2求解a、b、c、d四个位置的像素值时，依靠其附近像素值。比如，d(9，21)距离(8.8，20.2)较近，那么d(9，21)的像素值可以是(8.8，20.2)的像素值，即(8,19)的像素值。
对于a、b、c也可以采用类似的方式。比如(9,20)对应的校正后的位置坐标为(8.3，20.1)。ISP301-2确定(9,20)对应的像素值为第二像素值，ISP301-2将该第二像素值对应到(8.3，20.1)。由于a(8，20)位于(8.3，20.1)附近，所以a(8，20)的像素值可以是(8.3，20.1)对应的像素值，即(9,20)对应的像素值。
第八步，对于原始图像上不需要校正的坐标点，ISP301-2保持这些点的原始位置坐标对应的像素值不变。
在本申请实施例中，不限定第八步和第四-七步之间的执行顺序。
由第六-七步，ISP得到需要校正的坐标点的校正后的位置坐标和像素值；由第八步，ISP得到不需要校正的坐标点的原始位置坐标和像素值；所以，ISP根据所述需要校正的坐标点的校正后的位置坐标和像素值，以及所述不需要校正的坐标点的原始坐标位置和像素值，生成校正后的图像，即第九步。
在本申请一些实施例中，图5所示的图像校正方法的流程可以应用于多种场景。比如，场景一：手机中的相机启动拍摄图像的场景。场景二：手机锁屏时，通过脸部识别器采集用户脸部图像，进行解锁的场景。这种场景下，手机可以运行本申请实施例提供的图像校正算法的代码对采集到一张用户脸部图像进行图像校正，得到校正后的用户脸部图像，然后，手机判断校正之后的用户脸部图像是否能够解锁(比如，校正之后的用户脸部图像是否和预存的图像相匹配)。场景三：一个手机与其它手机进行视频通话(微信、QQ视频通话)时，所述一个手机每采集一张图像，可以运行本申请实施例提供的图像校正算法的代码对该图像进行图像校正，然后，所述一个手机将校正后的图像发送给其它手机。
本申请的各个实施方式可以任意进行组合，以实现不同的技术效果。
上述本申请提供的实施例中，从终端作为执行主体的角度对本申请实施例提供的方法进行了介绍。为了实现上述本申请实施例提供的方法中的各功能，终端可以包括硬件结构和/或软件模块，以硬件结构、软件模块、或硬件结构加软件模块的形式来实现上述各功能。上述各功能中的某个功能以硬件结构、软件模块、还是硬件结构加软件模块的方式来执行，取决于技术方案的特定应用和设计约束条件。
基于相同的构思，图11所示为本申请提供的一种终端1100。如图11所示，本申请另外一些实施例公开了一种终端1100，该终端1100可以包括：输入设备1101和摄像头1102，一个或多个处理器1103；存储器1104；以及一个或多个计算机程序1105，上述各器件可以通过一个或多个通信总线1106连接。其中该一个或多个计算机程序1105被存储在上述存储器1104中并被配置为被该一个或多个处理器1103执行，该一个或多个计算机程序1105包括指令，上述指令可以用于执行如图5及相应实施例中的各个步骤。
本发明实施例还提供一种计算机存储介质，该存储介质可以包括存储器，该存储器可存储有程序，该程序被执行时，使得终端执行包括如前的图5所示的方法实施例中记载的终端所执行的全部步骤。
本发明实施例还提供一种包含计算机程序产品，当所述计算机程序产品在终端上运行时，使得所述终端执行包括如前的图5所示的方法实施例中记载的终端所执行的全部步骤。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到本申请实施例可以用硬件实现，或固件实现，或它们的组合方式来实现。当使用软件实现时，可以将上述功能存储在计算机可读介质中或作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机存储介质和通信介质，其中通信介质包括便于从一个地方 向另一个地方传送计算机程序的任何介质。存储介质可以是计算机能够存取的任何可用介质。以此为例但不限于：计算机可读介质可以包括RAM、ROM、电可擦可编程只读存储器(electrically erasable programmable read only memory，EEPROM)、只读光盘(compact disc read-Only memory，CD-ROM)或其他光盘存储、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质。此外。任何连接可以适当的成为计算机可读介质。例如，如果软件是使用同轴电缆、光纤光缆、双绞线、数字用户线(digital subscriber line，DSL)或者诸如红外线、无线电和微波之类的无线技术从网站、服务器或者其他远程源传输的，那么同轴电缆、光纤光缆、双绞线、DSL或者诸如红外线、无线和微波之类的无线技术包括在所属介质的定影中。如本申请实施例所使用的，盘(disk)和碟(disc)包括压缩光碟(compact disc，CD)、激光碟、光碟、数字通用光碟(digital video disc，DVD)、软盘和蓝光光碟，其中盘通常磁性的复制数据，而碟则用激光来光学的复制数据。上面的组合也应当包括在计算机可读介质的保护范围之内。
总之，以上所述仅为本申请的实施例而已，并非用于限定本申请的保护范围。凡根据本申请的揭露，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (14)

  1. 一种图像校正方法，其特征在于，所述方法包括：
    终端响应于用户操作，启动相机应用，打开摄像头，通过所述摄像头采集原始图像；
    所述终端根据所述原始图像上的所有坐标点中每个坐标点的原始位置坐标和畸变转换公式，确定所述每个坐标点的理想位置坐标；其中，所述畸变转变公式用于表示坐标点的原始位置坐标和理想位置坐标之间的转换关系；
    所述终端根据所述每个坐标点的原始位置坐标和理想位置坐标，确定所述每个坐标点的畸变程度；
    所述终端根据所述每个坐标点的畸变程度和畸变程度阈值，确定所述所有坐标点中需要校正的坐标点和不需要校正的坐标点；
    所述终端根据所述需要校正的坐标点的畸变程度和连续函数，确定所述需要校正的坐标点的校正权重；其中，所述连续函数用于表示畸变程度和校正权重之间的转换关系；
    所述终端根据所述需要校正的坐标点的原始位置坐标、理想位置坐标以及与所述校正权重对应的校正公式，确定所述需要校正的坐标点的校正后的位置坐标；其中，所述校正公式用于指示所述需要校正的坐标点的原始位置坐标、理想位置坐标以及校正后的位置坐标之间的相对位置关系；
    所述终端将所述原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值设置为所述校正后的位置坐标对应的像素值，并；
    保持所述原始图像上与不需要校正的坐标点的原始位置坐标对应的像素值不变；
    所述终端根据所述需要校正的坐标点的校正后的位置坐标和像素值，以及所述不需要校正的坐标点的原始坐标位置和像素值，生成校正后的图像。
  2. 如权利要求1所述的方法，其特征在于，所述终端根据所述每个坐标点的原始位置坐标和理想位置坐标，确定所述每个坐标点的畸变程度；包括：
    所述终端确定所述每个坐标点的原始位置坐标和理想位置坐标之间的距离，所述距离为所述每个坐标点的畸变程度。
  3. 如权利要求1或2所述的方法，其特征在于，所述终端根据所述每个坐标点的畸变程度和畸变程度阈值，确定所述所有坐标点中需要校正的坐标点和不需要校正的坐标点；包括：
    所述终端确定所述所有坐标点中畸变程度大于畸变程度阈值的第一坐标点，所述第一坐标点即为所述需要校正的坐标点；
    所述终端确定所述所有坐标点中畸变程度小于等于所述畸变程度阈值的第二坐标点，所述第二坐标点即为所述不需要校正的坐标点。
  4. 如权利要求1-3任一所述的方法，其特征在于，若所述需要校正的坐标点的畸变程度为d，所述终端根据所述需要校正的坐标点的畸变程度和连续函数，确定所述需要校正的坐标点的校正权重，包括：
    所述终端将所述d代入如下的连续函数中：
    Figure PCTCN2018096039-appb-100001
    其中，畸变程度阈值为d0，得到所述需要校正的坐标点的校正权重f。
  5. 如权利要求1-4任一所述的方法，其特征在于，若需要校正的坐标点的原始位置坐标点为A、理想位置坐标点为B、校正后的位置坐标点为C，校正权重为f，则与所述校正权重f对应的校正公式为：
    Figure PCTCN2018096039-appb-100002
    其中，
    Figure PCTCN2018096039-appb-100003
    表示从A点到B点的矢量，
    Figure PCTCN2018096039-appb-100004
    表示从A点到C点的矢量。
  6. 如权利要求1-5任一所述的方法，其特征在于，所述终端将所述原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值设置为所述校正后的位置坐标对应的像素值，包括：
    若所述校正后的位置坐标为整数，所述终端确定所述原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值为所述校正后的位置坐标对应的像素值；
    若所述校正后的位置坐标为非整数，所述终端通过像素插值算法将原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值插值到所述校正后的位置坐标附近的、横纵坐标均为整数的其它位置坐标处。
  7. 一种终端，其特征在于，包括输入设备、摄像头、处理器和存储器；
    所述输入设备：用于接收用于启动摄像头的操作；
    所述摄像头：用于采集原始图像；
    所述存储器用于存储一个或多个计算机程序；当所述存储器存储的一个或多个计算机程序被所述处理器执行时，使得所述终端执行：
    根据所述原始图像上的所有坐标点中每个坐标点的原始位置坐标和畸变转换公式，确定所述每个坐标点的理想位置坐标；其中，所述畸变转变公式用于表示坐标点的原始位置坐标和理想位置坐标之间的转换关系；
    根据所述每个坐标点的原始位置坐标和理想位置坐标，确定所述每个坐标点的畸变程度；
    根据所述每个坐标点的畸变程度和畸变程度阈值，确定所述所有坐标点中需要校正的坐标点和不需要校正的坐标点；
    根据所述需要校正的坐标点的畸变程度和连续函数，确定所述需要校正的坐标点的校正权重；其中，所述连续函数用于表示畸变程度和校正权重之间的转换关系；
    根据所述需要校正的坐标点的原始位置坐标、理想位置坐标以及与所述校正权重对应的校正公式，确定所述需要校正的坐标点的校正后的位置坐标；其中，所述校正公式用于指示所述需要校正的坐标点的原始位置坐标、理想位置坐标以及校正后的位置坐标之间的相对位置关系；
    将所述原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值设置为所述校正后的位置坐标对应的像素值，并；
    保持所述原始图像上与不需要校正的坐标点的原始位置坐标对应的像素值不变；
    根据所述需要校正的坐标点的校正后的位置坐标和像素值，以及所述不需要校正的坐标点的原始坐标位置和像素值，生成校正后的图像。
  8. 如权利要求7所述的终端，其特征在于，当所述存储器存储的一个或多个计算机程序被所述处理器执行时，还使得所述终端执行：
    确定所述每个坐标点的原始位置坐标和理想位置坐标之间的距离，所述距离为所述每个坐标点的畸变程度。
  9. 如权利要求7或8所述的终端，其特征在于，当所述存储器存储的一个或多个计 算机程序被所述处理器执行时，还使得所述终端执行：
    确定所述所有坐标点中畸变程度大于畸变程度阈值的第一坐标点，所述第一坐标点即为所述需要校正的坐标点；
    确定所述所有坐标点中畸变程度小于等于所述畸变程度阈值的第二坐标点，所述第二坐标点即为所述不需要校正的坐标点。
  10. 如权利要求7-9任一所述的终端，其特征在于，若所述需要校正的坐标点的畸变程度为d，当所述存储器存储的一个或多个计算机程序被所述处理器执行时，还使得所述终端执行：
    所述终端将所述d代入如下的连续函数中：
    Figure PCTCN2018096039-appb-100005
    其中，畸变程度阈值为d0，得到所述需要校正的坐标点的校正权重f。
  11. 如权利要求7-10任一所述的终端，其特征在于，若需要校正的坐标点的原始位置坐标点为A、理想位置坐标点为B、校正后的位置坐标点为C，校正权重为f，则与所述校正权重f对应的校正公式为：
    Figure PCTCN2018096039-appb-100006
    其中，
    Figure PCTCN2018096039-appb-100007
    表示从A点到B点的矢量，
    Figure PCTCN2018096039-appb-100008
    表示从A点到C点的矢量。
  12. 如权利要求7-11任一所述的终端，其特征在于，当所述存储器存储的一个或多个计算机程序被所述处理器执行时，还使得所述终端执行：
    若所述校正后的位置坐标为整数，确定所述原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值为所述校正后的位置坐标对应的像素值；
    若所述校正后的位置坐标为非整数，通过像素插值算法将原始图像上与所述需要校正的坐标点的原始位置坐标对应的像素值插值到所述校正后的位置坐标附近的、横纵坐标均为整数的其它位置坐标处。
  13. 一种计算机存储介质，其特征在于，所述计算机可读存储介质包括计算机程序，当计算机程序在终端上运行时，使得所述终端执行如权利要求1至6任一所述的方法。
  14. 一种包含指令的计算机程序产品，其特征在于，当所述指令在计算机上运行时，使得所述计算机执行如权利要求1-6任一项所述的方法。
PCT/CN2018/096039 2018-07-17 2018-07-17 一种图像校正方法和终端 WO2020014881A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880086820.3A CN111656391B (zh) 2018-07-17 2018-07-17 一种图像校正方法和终端
PCT/CN2018/096039 WO2020014881A1 (zh) 2018-07-17 2018-07-17 一种图像校正方法和终端

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/096039 WO2020014881A1 (zh) 2018-07-17 2018-07-17 一种图像校正方法和终端

Publications (1)

Publication Number Publication Date
WO2020014881A1 true WO2020014881A1 (zh) 2020-01-23

Family

ID=69164202

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/096039 WO2020014881A1 (zh) 2018-07-17 2018-07-17 一种图像校正方法和终端

Country Status (2)

Country Link
CN (1) CN111656391B (zh)
WO (1) WO2020014881A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114697518A (zh) * 2020-12-29 2022-07-01 浙江宇视科技有限公司 摄像头的镜头畸变下的遮挡方法、装置、介质及电子设备
CN116862980A (zh) * 2023-06-12 2023-10-10 上海玉贲智能科技有限公司 图像边缘的目标检测框位置优化校正方法、***、介质及终端

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112215782B (zh) * 2020-10-29 2022-10-14 中国科学院长春光学精密机械与物理研究所 一种卷帘快门成像装置图像校正方法及装置
CN114283095B (zh) * 2021-12-30 2023-07-25 爱芯元智半导体(上海)有限公司 一种图像畸变校正方法、***、电子设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103996172A (zh) * 2014-05-08 2014-08-20 东北大学 一种基于多步校正的鱼眼图像校正方法
WO2014176971A1 (zh) * 2013-11-22 2014-11-06 中兴通讯股份有限公司 畸变图像校正复原与分析报警的方法和装置
WO2017043125A1 (ja) * 2015-09-07 2017-03-16 株式会社ソニー・インタラクティブエンタテインメント 情報処理システム、情報処理装置、出力装置、プログラム及び記録媒体
CN106815823A (zh) * 2017-02-22 2017-06-09 广东工业大学 一种透镜畸变标定校正方法及其装置
CN106875341A (zh) * 2015-12-11 2017-06-20 宁波舜宇光电信息有限公司 畸变图像校正方法及其定位方法
CN107424126A (zh) * 2017-05-26 2017-12-01 广州视源电子科技股份有限公司 图像校正方法、装置、设备、***及摄像设备和显示设备
CN107871329A (zh) * 2017-12-18 2018-04-03 横琴峰云视觉技术有限公司 一种相机光学中心的快速标定方法及装置

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1996389A (zh) * 2007-01-09 2007-07-11 北京航空航天大学 基于共线特征点的摄像机畸变快速校正方法
JP2010218226A (ja) * 2009-03-17 2010-09-30 Suzuki Motor Corp 計測マップ生成装置及び走行環境確認装置
JP4877402B2 (ja) * 2010-04-05 2012-02-15 ソニー株式会社 画像処理装置、画像処理方法、撮像装置、プログラム、及び記録媒体
JP2012147281A (ja) * 2011-01-13 2012-08-02 Ricoh Co Ltd 画像処理装置
CN103247031B (zh) * 2013-04-19 2016-03-09 华为技术有限公司 一种畸变图像校正的方法、终端及***
CN103426149B (zh) * 2013-07-24 2016-02-03 玉振明 大视角图像畸变的校正处理方法
US10262400B2 (en) * 2014-10-31 2019-04-16 Huawei Technologies Co., Ltd. Image processing method and device using reprojection error values
CN105354796B (zh) * 2015-10-12 2018-12-04 北京鑫洋泉电子科技有限公司 用于行车辅助的图像处理方法及***
WO2017092631A1 (zh) * 2015-11-30 2017-06-08 宁波舜宇光电信息有限公司 鱼眼图像的畸变图像校正方法及鱼眼相机的标定方法
CN107680060A (zh) * 2017-09-30 2018-02-09 努比亚技术有限公司 一种图像畸变校正方法、终端及计算机可读存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014176971A1 (zh) * 2013-11-22 2014-11-06 中兴通讯股份有限公司 畸变图像校正复原与分析报警的方法和装置
CN103996172A (zh) * 2014-05-08 2014-08-20 东北大学 一种基于多步校正的鱼眼图像校正方法
WO2017043125A1 (ja) * 2015-09-07 2017-03-16 株式会社ソニー・インタラクティブエンタテインメント 情報処理システム、情報処理装置、出力装置、プログラム及び記録媒体
CN106875341A (zh) * 2015-12-11 2017-06-20 宁波舜宇光电信息有限公司 畸变图像校正方法及其定位方法
CN106815823A (zh) * 2017-02-22 2017-06-09 广东工业大学 一种透镜畸变标定校正方法及其装置
CN107424126A (zh) * 2017-05-26 2017-12-01 广州视源电子科技股份有限公司 图像校正方法、装置、设备、***及摄像设备和显示设备
CN107871329A (zh) * 2017-12-18 2018-04-03 横琴峰云视觉技术有限公司 一种相机光学中心的快速标定方法及装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114697518A (zh) * 2020-12-29 2022-07-01 浙江宇视科技有限公司 摄像头的镜头畸变下的遮挡方法、装置、介质及电子设备
CN116862980A (zh) * 2023-06-12 2023-10-10 上海玉贲智能科技有限公司 图像边缘的目标检测框位置优化校正方法、***、介质及终端
CN116862980B (zh) * 2023-06-12 2024-01-23 上海玉贲智能科技有限公司 图像边缘的目标检测框位置优化校正方法、***、介质及终端

Also Published As

Publication number Publication date
CN111656391B (zh) 2023-09-22
CN111656391A (zh) 2020-09-11

Similar Documents

Publication Publication Date Title
CN110581948B (zh) 提供质量定制图像的电子装置及其控制方法、服务器
CN109891874B (zh) 一种全景拍摄方法及装置
US11558553B2 (en) Electronic device for stabilizing image and method for operating same
CN109788189B (zh) 将相机与陀螺仪融合在一起的五维视频稳定化装置及方法
WO2020014881A1 (zh) 一种图像校正方法和终端
US11222477B2 (en) Electronic device for providing augmented reality service and operating method thereof
CN112602111A (zh) 模糊基于深度信息组合多个图像而获得的图像的电子设备及驱动该电子设备的方法
US20140038560A1 (en) System for and method of transmitting communication information
CN107948505B (zh) 一种全景拍摄方法及移动终端
WO2022033272A1 (zh) 图像处理方法以及电子设备
EP3742721A1 (en) Electronic device having camera module capable of switching line of sight and method for recording video
CN109302563B (zh) 防抖处理方法、装置、存储介质及移动终端
WO2021104265A1 (zh) 电子设备及对焦方法
US10270963B2 (en) Angle switching method and apparatus for image captured in electronic terminal
US11164388B2 (en) Electronic device and method for providing augmented reality object therefor
US11877057B2 (en) Electronic device and focusing method
CN111488895A (zh) 对抗数据生成方法、装置、设备及存储介质
CN111147745B (zh) 拍摄方法、装置、电子设备及存储介质
CN111757146B (zh) 视频拼接的方法、***及存储介质
US10902265B2 (en) Imaging effect based on object depth information
CN110443841B (zh) 地面深度的测量方法、装置及***
CN114868180A (zh) 用于改变显示器的指定区域中的显示的电子装置及其操作方法
CN116052235B (zh) 注视点估计方法及电子设备
CN215344778U (zh) 摄像模组及移动终端
CN114630085B (zh) 图像投影方法、装置、存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18927069

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18927069

Country of ref document: EP

Kind code of ref document: A1