CN113763437B

CN113763437B - Image processing method, device, electronic equipment and storage medium

Info

Publication number: CN113763437B
Application number: CN202111088907.4A
Authority: CN
Inventors: 权威
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-09-16
Filing date: 2021-09-16
Publication date: 2023-12-05
Anticipated expiration: 2041-09-16
Also published as: CN113763437A

Abstract

The application discloses an image processing method, an image processing device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring registration parameters between the first image and the second image; acquiring an offset of a center point of the second image with respect to a center point of the first image in the longitudinal direction as a first offset, a maximum offset of the second image with respect to the first image in the positive direction in the longitudinal direction as a second offset, and a maximum offset of the second image with respect to the first image in the negative direction in the longitudinal direction as a third offset; based on the first offset, the second offset and the third offset, loading a plurality of rows of pixel points corresponding to the pixel points of the target row in the first image into a row cache; obtaining pixel points matched with each pixel point in the pixel points of the target row from the row cache as matched pixel points corresponding to each pixel point; and registering the matched pixel points corresponding to each pixel point based on the registration parameters. The method can reduce the line cache required in image registration.

Description

Image processing method, device, electronic equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image processing method, an image processing device, an electronic device, and a storage medium.

Background

In image processing, multi-frame image processing is widely used for effective verification and mass use in scenes such as professional photography, continuous shooting, industrial detection, medical imaging and the like. Among them, image registration is one of the key technologies in multi-frame image processing. However, when image registration is performed, more line buffers are occupied to reach the required registration range, and thus, a large hardware cost is brought.

Disclosure of Invention

In view of the above, the present application provides an image processing method, an image processing apparatus, an electronic device, and a storage medium.

In a first aspect, an embodiment of the present application provides an image processing method, applied to an electronic device, where the method includes: acquiring registration parameters between a first image and a second image, wherein the first image is a previous frame image of the second image in an image sequence; acquiring an offset of a center point of the second image with respect to a center point of the first image in a longitudinal direction as a first offset, a maximum offset of the second image with respect to the first image in a positive direction in the longitudinal direction as a second offset, and a maximum offset of the second image with respect to the first image in a negative direction in the longitudinal direction as a third offset; when correcting the target line pixel point of the second image, loading a plurality of lines of pixel points corresponding to the target line pixel point in the first image into a line cache based on the first offset, a first difference value between the second offset and the first offset, and a second difference value between the third offset and the first offset, wherein each line of pixel point in the plurality of lines of pixel points corresponds to a line cache, and the target line of pixel point is any line of pixel point in the second image; obtaining pixel points matched with each pixel point in the target line pixel point from line pixel points cached in the line cache as matched pixel points corresponding to each pixel point; and based on the registration parameters, registering the matched pixel points corresponding to each pixel point to obtain registered pixel points serving as corrected pixel points of each pixel point.

In a second aspect, an embodiment of the present application provides an image processing apparatus, applied to an electronic device, including: the device comprises a parameter acquisition module, an offset acquisition module, a pixel buffer module, a pixel determination module and a pixel correction module, wherein the parameter acquisition module is used for acquiring registration parameters between a first image and a second image, and the first image is a previous frame image of the second image in an image sequence; the offset acquisition module is used for acquiring an offset of a center point of the second image relative to a center point of the first image in a longitudinal direction as a first offset, a maximum offset of the second image relative to the first image in a positive direction in the longitudinal direction as a second offset, and a maximum offset of the second image relative to the first image in a negative direction in the longitudinal direction as a third offset; the pixel buffer module is configured to load a plurality of rows of pixel points corresponding to a target row of pixel points in the first image into a row buffer based on the first offset, a first difference value between the second offset and the first offset, and a second difference value between the third offset and the first offset when correcting the target row of pixel points in the second image, where each row of pixel points in the plurality of rows of pixel points corresponds to a row buffer, and the target row of pixel points is a pixel point of any row in the second image; the pixel determining module is used for obtaining a pixel point matched with each pixel point in the target line pixel point from line pixel points cached in the line cache as a matched pixel point corresponding to each pixel point; the pixel correction module is used for carrying out registration on the matched pixel points corresponding to each pixel point based on the registration parameters to obtain registered pixel points serving as pixel points after correction of each pixel point.

In a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors; a memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more application programs configured to perform the image processing method provided in the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having stored therein program code that is callable by a processor to perform the image processing method provided in the first aspect described above.

According to the scheme provided by the application, by acquiring the registration parameter between the first image and the second image, wherein the first image is the previous frame image of the second image in the image sequence, before image registration is carried out based on the registration parameter, the offset of the center point of the second image relative to the center point of the first image in the longitudinal direction is acquired as a first offset, the maximum offset of the second image relative to the first image in the positive direction in the longitudinal direction is acquired as a second offset, and the maximum offset of the second image relative to the first image in the negative direction in the longitudinal direction is acquired as a third offset. When correcting the target line pixel point of the second image, loading a plurality of lines of pixel points corresponding to the target line pixel point in the first image into a line buffer based on the first offset, the first difference value between the second offset and the first offset, and the second difference value between the third offset and the first offset, wherein each line of pixel point in the plurality of lines of pixel points corresponds to a line buffer, the target line pixel point is a pixel point of any line in the second image, the pixel point matched with each pixel point in the target line pixel point is obtained from the line pixel points buffered in the line buffer and used as a matched pixel point corresponding to each pixel point, and then the matched pixel point corresponding to each pixel point is registered based on registration parameters to obtain a registered pixel point which is used as a pixel point corrected for each pixel point. Therefore, when the line pixel points of the image are loaded into the line cache, partial offset of the maximum offset of the image in the longitudinal direction can be preset based on the offset of the central point in the longitudinal direction, and the corresponding line pixel points are loaded into the line cache based on the offset, so that the line cache required during image registration is effectively reduced, and the hardware cost is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 shows a flowchart of an image processing method according to an embodiment of the present application.

Fig. 2 shows a schematic diagram of image offset according to an embodiment of the present application.

Fig. 3 shows a schematic diagram of image imaging provided by an embodiment of the present application.

Fig. 4 shows another schematic diagram of image offset provided by an embodiment of the present application.

Fig. 5 shows a flowchart of an image processing method according to another embodiment of the present application.

Fig. 6 shows still another schematic diagram of image offset provided by an embodiment of the present application.

Fig. 7 shows a flowchart of an image processing method according to still another embodiment of the present application.

Fig. 8 shows a schematic diagram of an image processing method according to still another embodiment of the present application.

Fig. 9 shows a flowchart of an image processing method according to still another embodiment of the present application.

Fig. 10 shows a block diagram of an image processing apparatus according to an embodiment of the present application.

Fig. 11 is a block diagram of an electronic device for performing an image processing method according to an embodiment of the present application.

Fig. 12 is a storage unit for storing or carrying program code for implementing an image processing method according to an embodiment of the present application.

Detailed Description

In order to enable those skilled in the art to better understand the present application, the following description will make clear and complete descriptions of the technical solutions according to the embodiments of the present application with reference to the accompanying drawings.

Multi-frame image processing is a widely used technique that is effectively verified and used in large numbers in professional photography, industrial inspection, and cell phone applications. For example, in capturing multiple frames of images, due to the movement of the capturing device, there is a slight difference in content of the captured images, so that global registration of the images is often required in processing multiple frames of images, and then other processing is performed on the images, such as multiple frames of noise reduction, high-Dynamic Range (HDR), exposure fusion, and the like. Image registration (Image registration) is a process of matching and overlaying two or more images acquired at different times, with different sensors (imaging devices) or under different conditions (weather, illuminance, camera position and angle, etc.).

The inventors have long studied and found that when implementing an image registration algorithm by using hardware (such as application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable Gate Array, FPGA), etc.), a certain line buffer (line buffer) needs to be reserved to avoid processing delay in the final image deformation registration (warp) stage. For example, when the maximum longitudinal offset supported in image registration is ±20 lines of pixels (pixels), 40 lines need to be reserved to complete the image registration operation. For 1080p resolution video, which is equivalent to an alignment range of about + -20/1920 (approximately 1.04%), normal handheld devices such as cameras, mobile phones and the like are photographed, and a correction range of about 1% -2% is required to be ensured to align most of multi-frame images, so that the overhead of image registration by a visible hardware scheme is very large.

In order to solve the above problems, the inventor proposes the image processing method, the device, the electronic equipment and the storage medium provided by the embodiment of the application, when the line pixel points of the image are loaded into the line cache, the partial offset of the maximum offset of the image in the longitudinal direction can be preset based on the offset of the central point in the longitudinal direction, and the corresponding line pixel points are loaded into the line cache based on the offset, so that the line cache required during image alignment is effectively reduced, and the hardware cost is reduced. The specific image processing method is described in detail in the following embodiments.

Referring to fig. 1, fig. 1 is a flowchart illustrating an image processing method according to an embodiment of the application. In a specific embodiment, the image processing method is applied to an image processing apparatus 400 as shown in fig. 9 and an electronic device 100 (fig. 10) provided with the image processing apparatus 400. In the following, the specific flow of the present embodiment will be described by taking an electronic device as an example, and it will be understood that the electronic device applied in the present embodiment may be a smart phone, a tablet computer, a smart watch, smart glasses, a notebook computer, etc., which is not limited herein. The following will describe the flowchart shown in fig. 1 in detail, and the image processing method specifically may include the following steps:

step S110: and acquiring registration parameters between a first image and a second image, wherein the first image is a previous frame image of the second image in the image sequence.

In the embodiment of the application, when the electronic device performs registration on the images in the image sequence, a first image and a second image can be acquired, wherein the first image is the previous frame image of the second image in the image sequence. After the first image and the second image are acquired, a registration parameter between the first image and the second image may be acquired to correct the second image based on the registration parameter. It will be appreciated that after continuously capturing images, a current frame image may be corrected using a previous frame image after obtaining continuous multi-frame images, that is, if the captured images are ordered in the order of 1 st, 2 nd, and 3 rd frames … M, where M is a positive integer, the ordering number of the second image is greater than 1, so that the second image may be corrected using the previous frame image. For example, in a scene of video shooting, each frame image may be acquired, and correction may be performed for each frame image starting from the 2 nd frame image to correct a difference in content due to movement at the time of shooting, and of course, a specific application scene may not be limited.

In some embodiments, the first image and the second image may be images obtained by capturing by the electronic device, for example, the captured images are corrected in real time when the electronic device captures video; the first image and the second image may be images captured by other devices, and after the capturing is completed, the electronic device completes corresponding processing to correct differences in image content caused by movement or the like during the capturing.

In some implementations, the registration parameter may be a transformation matrix. For the first image and the second image, the electronic device may perform feature detection processing to obtain feature points; then, matching feature point pairs are found through similarity measurement, and feature matching is completed; next, the image space coordinate transformation parameters are obtained through the matched feature point pairs. The coordinate transformation parameters are used for image registration to obtain corrected images.

As a possible implementation, the transformation parameters may be affine transformation (affine) matrices. The affine transformation is one of the transformation modes in the image transformation, the image transformation is based on matrix operation, and the corresponding relation of different images can be quickly found through the matrix operation. Affine transformation refers to a process of performing a linear transformation in a vector space once, transforming to another vector space. When the affine transformation matrix is acquired, feature detection can be performed on feature areas (such as texture areas, corner points and the like) of the first image and the second image; based on the detected characteristics, screening out the characteristics with stronger characteristics, performing block matching, and obtaining a point pair sequence after the block matching; based on the sequence of point pairs and the formula of affine transformation, an affine transformation matrix can be obtained.

As another possible embodiment, since the movement at the time of photographing may also be a rotation of the apparatus, it is also possible to determine a rotation matrix based on the rotation parameters between the first image and the second image, and use the rotation matrix as the registration parameter. Alternatively, since the rotation parameter includes a plurality of rotation amounts corresponding to a plurality of spatial directions, one of which corresponds to one rotation amount, the rotation matrix may be generated using the plurality of rotation amounts. The obtained rotation matrix can be used for carrying out rotation processing on the first image so as to finish registration of the first image and the second image, thereby correcting content difference caused by rotation. Of course, the specific registration parameters may not be limiting.

Step S120: the offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction is acquired as a first offset, the maximum offset of the second image with respect to the first image in the positive direction in the longitudinal direction is acquired as a second offset, and the maximum offset of the second image with respect to the first image in the negative direction in the longitudinal direction is acquired as a third offset.

Since the maximum longitudinal offset of the line determines the reserved line buffer size during image registration, a larger line buffer is required to be occupied, and thus a larger overhead is caused. For example, referring to fig. 2, the deformed image (right image) is shifted from the original image (left image), the maximum shift in the longitudinal direction of the whole image is Δy2, the shift in the downward direction is Δy1, if the upward direction is negative, the shift in the upward direction in the longitudinal direction may be denoted as- Δy2, and if the downward direction is positive, the shift in the downward direction in the longitudinal direction may be denoted as Δy1; since the matching pixel point needs to be found in the other image for registration by using the registration parameter when the correction is performed for each row of pixels, so as to obtain a new pixel point (i.e., the corrected pixel point), the row pixels in the maximum offset ranges of the positions of the pixels in the same row of pixels in the other image are required to be oriented and cached from the maximum offset ranges of the positions of the pixels in the same row of pixels in the other image, so as to ensure that the pixels can be matched with the corresponding pixel points. That is, for the above offsets of Δy2 and Δy1, when correction is performed for each row of pixels of the second image, a Δy2+Δy1 row buffer needs to be reserved to buffer the pixel points of Δy2+Δy1 rows in the first image, so as to ensure that the process does not block. The negative direction and the positive direction in the longitudinal direction may be a negative direction in an upward direction in the longitudinal direction, and a positive direction in a downward direction in the longitudinal direction; the downward direction in the longitudinal direction may be a negative direction, and the upward direction in the longitudinal direction may be a positive direction.

In addition, the maximum shift amount of the image in the longitudinal direction is affected by two factors. One of these is the longitudinal offset of the center of the image, for example, the longitudinal offset of the center point o' as shown in fig. 3; another factor is the amount of rotation of the image plane about the z-axis of the camera coordinate system, which determines the offset of the image in the longitudinal direction. For example, as shown in fig. 4, the image shift by the longitudinal movement and the shift by the rotation determine the maximum shift amounts of the final image in both directions in the longitudinal direction as a whole.

Therefore, in the embodiment of the present application, the offset amount of the center point of the second image with respect to the center point of the first image in the longitudinal direction may be acquired and taken as the first offset amount; acquiring the maximum offset of the second image relative to the first image in the positive direction of the longitudinal direction, and taking the maximum offset as the second offset; and acquiring the maximum offset of the second image relative to the first image in the longitudinal negative direction, and taking the maximum offset as a third offset. Thus, when the pixel points of the line are cached based on the second offset and the third offset, the partial offset of the maximum offset of the image in the longitudinal direction can be preset based on the first offset, so that the required line cache is reduced.

In some implementations, the electronic device may determine the above respective offsets based on the above acquired registration parameters; the electronic device can also calculate the offset according to the coordinates of the corresponding pixel points in the first image and the second image directly; if the electronic device is a device for capturing the first image and the second image, and the electronic device is provided with a gyroscope, the offset amounts can be determined according to detection data of the gyroscope when the first image and the second image are captured. Of course, the manner of specifically acquiring the above respective offsets may not be limited.

Step S130: and when correcting the target line pixel point of the second image, loading a plurality of lines of pixel points corresponding to the target line pixel point in the first image into a line cache based on the first offset, a first difference value between the second offset and the first offset, and a second difference value between the third offset and the first offset, wherein each line of pixel point in the plurality of lines of pixel points corresponds to a line cache, and the target line of pixel point is any line of pixel point in the second image.

In the embodiment of the application, after acquiring the offset of the center point of the second image relative to the center point of the first image in the longitudinal direction, the maximum offset of the second image relative to the first image in the positive direction of the longitudinal direction, and the maximum offset of the second image relative to the first image in the negative direction of the longitudinal direction, when correcting each row of pixels of the second image based on the above registration parameters, the electronic device may buffer the corresponding row of pixels in the second image based on the first offset, the second offset, and the third offset. Because the offset of the image center point, that is, the first offset, is already known, when the corresponding line of pixels in the second image are cached, the corresponding line of pixels can be taken in the positive direction and the negative direction in the longitudinal direction from the preset line position after the first offset is preset, based on the first difference value between the second offset and the first offset and the second difference value between the third offset and the first offset. That is, it has been known that the image is necessarily offset by the first offset amount in the longitudinal direction, when the line pixels are cached, the part of the line pixels of the first offset amount can be drained from the line pixels that originally need to be cached, and thus, the required line cache can be saved.

Step S140: and obtaining pixel points matched with each pixel point in the target line pixel point from the line pixel points cached in the line cache as matched pixel points corresponding to each pixel point.

In the embodiment of the application, when the target line pixel point of the second image is corrected, each pixel point in the target line pixel point and the pixel point cached in the line cache can be matched from the line pixel points cached in the line cache, so that the pixel point matched with each pixel point in the target line pixel point can be found from the pixel points in the line cache. The obtained pixel points matched with each pixel point in the target line pixel points can be used as the matched pixel points corresponding to each pixel point in the target line pixel points, namely the pixel points for registering through configuration parameters.

Step S150: and based on the registration parameters, registering the matched pixel points corresponding to each pixel point to obtain registered pixel points serving as corrected pixel points of each pixel point.

In the embodiment of the application, when the electronic device corrects the target line pixel point of the second image, after the matching pixel point corresponding to each pixel point in the target line pixel point is obtained, the above obtained registration parameters can be used for registering each matching pixel point, so as to obtain registered pixel points, and the registered pixel points are used as pixel points corrected for each pixel point.

In some embodiments, when the registration parameter is an affine transformation matrix, the pixel coordinates of each matched pixel point may be substituted into an affine transformation formula, so as to obtain the pixel coordinates of the pixel points after the registration of each matched pixel point, and form the registered pixel points. Of course, the specific registration manner may be determined according to the acquired registration parameters, which is not limited herein.

In some embodiments, after the correction of one row of pixels in the second image is completed, the electronic device may return to step S130 to correct pixels in other rows, and the process is repeated continuously, so that correction of all rows of pixels in the second image can be completed, thereby implementing correction of the whole image of the second image, and obtaining an image after correction of the second image. The corrected image may be used for replacing the second image, or may be used for performing noise reduction processing, high dynamic illumination rendering, exposure fusion and other processing on the second image, which is not limited herein.

According to the image processing method provided by the embodiment of the application, when the image is corrected based on image registration, besides the offset of the image global in the longitudinal positive direction and the image global in the longitudinal negative direction, the offset of the image center point in the longitudinal direction is obtained, so that when the corresponding line pixel points are cached, the offset of the center point can be preset, and then the corresponding line pixel points are cached, so that the part of line pixel points which originally need to be cached and have the first offset are discharged, the line caching quantity is reduced, and the hardware cost is reduced. Further, since the line buffer amount required for the same registration range is reduced, a larger registration range can be achieved.

Referring to fig. 5, fig. 5 is a flowchart illustrating an image processing method according to another embodiment of the application. The image processing method is applied to the electronic device, and will be described in detail below with respect to the flowchart shown in fig. 5, where the image processing method specifically includes the following steps:

step S210: and acquiring registration parameters between a first image and a second image, wherein the first image is a previous frame image of the second image in the image sequence.

Step S220: the offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction is acquired as a first offset, the maximum offset of the second image with respect to the first image in the positive direction in the longitudinal direction is acquired as a second offset, and the maximum offset of the second image with respect to the first image in the negative direction in the longitudinal direction is acquired as a third offset.

In the embodiment of the present application, the step S210 and the step S220 may refer to the contents of other embodiments, which are not described herein.

Step S230: and when correcting the target row pixel points of the second image, acquiring row pixel points, of which the positions are matched with the target row pixel points, in the first image, as matched row pixel points.

In the embodiment of the application, when the electronic device corrects the target line pixel point of the second image, and caches the corresponding line pixel point in the first image, the line pixel point of which the position is matched with the target line pixel point in the first image can be determined and used as the matched line pixel point. For example, if the target line pixel point is a T-th line pixel point in the second image and T is a positive integer, the T-th line pixel point in the first image is obtained as a matching line pixel point.

Step S240: and acquiring the row position of the pixel points of the matched row according to the row position shifted by the first offset as a target row position.

In the embodiment of the application, since the offset of the center point in the longitudinal direction is already known, that is, the partial offset of the maximum offset of the second image relative to the first image is known, when a plurality of rows of pixels corresponding to the target row of pixels in the first image are taken, the initial row position in the first image can be redetermined. The electronic device may acquire a line position of the pixel point of the matching line after the line position is offset according to the first offset, and use the line position as the target line position. For example, referring to fig. 6, in the longitudinal direction, the center point of the second image is shifted from the center point of the first image by an amount of shift by Δy downward; the electronic device may obtain the shifted row position after shifting Δy downward at the row position of the matched row pixel point.

Step S250: and acquiring pixel points of a target line number along a longitudinal positive direction and a longitudinal negative direction respectively from the position of the target line in the first image based on the difference value between the second offset and the first offset and the difference value between the third offset and the first offset, so as to obtain a plurality of rows of pixel points.

In the embodiment of the application, after the obtained target line position of the matched line pixel point is offset according to the first offset, the pixel point of the target line number can be obtained from the target line position in the first image along the longitudinal positive direction and the longitudinal negative direction respectively based on the difference value between the second offset and the first offset and the difference value between the third offset and the first offset, so as to obtain a plurality of lines of pixel points. The target line number may be determined based on a difference between the second offset and the first offset, and a difference between the third offset and the first offset. It will be appreciated that since the start line position (i.e., the target line position) is redetermined based on the offset of the center point, only the pixel points of the line are taken from the position of the target line position along the positive direction and the negative direction in the longitudinal direction based on the difference between the second offset and the first offset and the difference between the third offset and the first offset, so that the offset of the image in the longitudinal direction can be covered, and the pixel points matched with the pixel points in the target line pixel points can be acquired when correction is ensured.

In some embodiments, based on the difference between the second offset and the first offset and the difference between the third offset and the first offset, obtaining the pixel points of the target line number from the target line position in the first image along the positive direction and the negative direction in the longitudinal direction, to obtain a plurality of rows of pixel points may include: acquiring a difference value between the second offset and the first offset as a first difference value, and acquiring a difference value between the third offset and the first offset as a second difference value; acquiring a first line number of pixel points from the target line position in the first image along the longitudinal positive direction, wherein the first line number is matched with the first difference value; and acquiring a second number of pixel points along the longitudinal negative direction from the target line position in the first image, wherein the second number of pixel points is matched with the second difference value, and the sum of the first number of pixel points and the second number of pixel points is the target line number.

In this embodiment, when the row pixel of the target row number is taken at the target row position based on the difference (first difference) between the second offset and the first offset, and the difference (second difference) between the third offset and the first offset, since the first difference corresponds to the positive direction in the longitudinal direction and the second difference corresponds to the negative direction in the longitudinal direction, the row pixel of the first difference can be taken in the positive direction in the longitudinal direction from the target row position, and the row pixel of the second difference can be taken in the negative direction in the longitudinal direction from the target row position, so that the taken row pixel can cover the offset, and the pixel matched with the pixel in the target row pixel can be obtained.

In other embodiments, based on the difference between the second offset and the first offset and the difference between the third offset and the first offset, obtaining pixel points of the target line number from the target line position in the first image along the positive direction and the negative direction in the longitudinal direction, to obtain a plurality of rows of pixel points may include: acquiring a difference value between the second offset and the first offset as a first difference value, and acquiring a difference value between the third offset and the first offset as a second difference value; obtaining the maximum value of the first difference value and the second difference value; and acquiring pixel points of a specified line number along the positive direction and the negative direction in the longitudinal direction respectively from the target line position in the first image to obtain a plurality of lines of pixel points, wherein the specified line number is matched with the maximum value.

In this embodiment, when the pixel is cached, it is considered that the pixel of the same line number in the positive direction and the negative direction in the vertical direction, that is, the pixel of the ±specified line number is covered, and therefore, in order to enable the offset to be covered by the line pixel to be fetched, the maximum value of the first difference value and the second difference value may be taken as the specified line number. That is, the pixel points of the designated line number are acquired from the target line position in the first image along the positive direction and the negative direction in the longitudinal direction, so that the line pixel points of the required cache can be obtained.

Step S260: and loading each row of pixel points in the plurality of rows of pixel points into a row of cache.

In the embodiment of the application, after the above rows of pixel points are obtained, each row of pixel points in the rows of pixel points can be loaded into a row of cache. Referring again to fig. 6, in the case where the line pixels covering the same number of lines in the positive direction and the negative direction in the vertical direction are considered, if the above first offset Δy is not considered in caching the line pixels, the number of line pixels to be cached is 2×max (Δy2, Δy1), but with the implementation provided by the embodiment of the present application, only the number of line pixels to be cached is 2×max (Δy2- Δy, Δy1- Δy), and the required number is less than 2×max (Δy2, Δy1), so that the consumption of the line cache can be effectively reduced.

Step S270: and obtaining pixel points matched with each pixel point in the target line pixel point from the line pixel points cached in the line cache as matched pixel points corresponding to each pixel point.

Step S280: and based on the registration parameters, registering the matched pixel points corresponding to each pixel point to obtain registered pixel points serving as corrected pixel points of each pixel point.

In the embodiment of the present application, the step S270 and the step S280 may refer to the contents of other embodiments, which are not described herein.

According to the image processing method provided by the embodiment of the application, when the image is corrected based on image registration, besides the offset of the image global in the longitudinal positive direction and the image global in the longitudinal negative direction, the offset of the image center point in the longitudinal direction is obtained, when the corresponding line pixel points are cached, the initial line position is preset and offset is performed based on the offset of the center point, and then the corresponding line pixel points are cached, so that the part of line pixel points which originally need to be cached and have the first offset are discharged, the line caching quantity is reduced, and the hardware cost is effectively reduced.

Referring to fig. 7, fig. 7 is a flowchart illustrating an image processing method according to another embodiment of the application. The image processing method is applied to the electronic device, and will be described in detail with respect to the flowchart shown in fig. 7, and the image processing method specifically includes the following steps:

step S310: and acquiring registration parameters between a first image and a second image, wherein the first image is a previous frame image of the second image in the image sequence.

In the embodiment of the present application, step S310 may refer to the content of other embodiments, which is not described herein.

Step S320: and acquiring the offset of the position point after the center point of a third image is registered relative to the center point of the third image in the longitudinal direction based on the transformation parameters obtained when the first image is corrected, wherein the third image is the previous frame image of the first image, and the offset is taken as the offset of the center point of the second image relative to the center point of the first image in the longitudinal direction.

In the embodiment of the present application, the electronic device may correct the continuous multi-frame image by using the image processing method provided in the embodiment of the present application, so before correcting the second image, the correction of the previous frame image of the second image is completed, that is, the correction of the first image is completed, in order to increase the processing speed, the registration parameter obtained when correcting the first image may be cached, and based on the registration parameter, the offset of the position point after registering the center point of the third image with respect to the center point of the third image in the longitudinal direction may be calculated, and the offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction may be used as the offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction. That is, considering that most of the motion has a certain inertia, the offset of the center point of the previous frame image (first image) may be used as the offset of the center point of the current frame image (second image).

In some implementations, the registration parameters may include an affine transformation matrix. The obtaining, based on the transformation parameters obtained when the first image is corrected, an offset of a position point of the third image after the center point of the third image is registered with respect to the center point of the third image, as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction, may include: performing affine transformation on the central point of the third image based on an affine transformation matrix obtained when the first image is corrected, so as to obtain the central point after affine transformation as a target position point; an offset of the target position point with respect to the center point of the third image in the longitudinal direction is acquired as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction. In this embodiment, after affine transformation is performed on the center point of the previous frame image of the first image using the affine transformation matrix obtained when the first image is corrected, the offset of the center point after affine transformation with respect to the center point before affine transformation, that is, the offset corresponding to the center point of the second image, may be obtained and used as the offset of the center point of the second image.

For example, referring to fig. 8, when each frame of image is corrected, after image feature point detection and block matching, global movement may be calculated, so as to obtain registration parameters, and image registration is performed, and an offset of a center point may be determined based on the calculated global movement, and the offset may be preset for use in a buffer process when correcting the next frame of image. In fig. 8, N is a positive integer, and N is greater than 2.

Step S330: and taking the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction as a first offset.

Step S340: based on registration parameters between the first image and the second image, a maximum offset of the second image relative to the first image in a positive direction of a longitudinal direction is obtained as a second offset, and a maximum offset of the second image relative to the first image in a negative direction of the longitudinal direction is obtained as a third offset.

In the embodiment of the present application, when the global maximum offset in the longitudinal direction is obtained, the maximum offset in the positive direction and the negative direction of the longitudinal direction of the pixel point after the registration of each pixel point of the first image may be calculated based on the pixel point after the registration, relative to the pixel point before the registration.

Step S350: and when correcting the target line pixel point of the second image, loading a plurality of lines of pixel points corresponding to the target line pixel point in the first image into a line cache based on the first offset, a first difference value between the second offset and the first offset, and a second difference value between the third offset and the first offset, wherein each line of pixel point in the plurality of lines of pixel points corresponds to a line cache, and the target line of pixel point is any line of pixel point in the second image.

Step S360: and obtaining pixel points matched with each pixel point in the target line pixel point from the line pixel points cached in the line cache as matched pixel points corresponding to each pixel point.

Step S370: and based on the registration parameters, registering the matched pixel points corresponding to each pixel point to obtain registered pixel points serving as corrected pixel points of each pixel point.

In the embodiment of the present application, the steps S350 to S370 may refer to the content of other embodiments, which are not described herein.

According to the image processing method provided by the embodiment of the application, when the image is corrected based on image registration, besides the offset of the image global in the longitudinal positive direction and the image global in the longitudinal negative direction, the offset of the image center point in the longitudinal direction is obtained, when the corresponding line pixel points are cached, the corresponding line pixel points are cached after the preset offset is carried out based on the offset of the center point, so that the part of line pixel points which originally need to be cached and have the first offset are discharged, the line buffering quantity is reduced, and the hardware cost is effectively reduced. In addition, when the offset of the center point is acquired, the offset of the center point of the previous frame image is calculated as the offset of the center point of the current frame image according to the registration parameter obtained when the previous frame image is corrected, so that delay can be reduced, and the processing speed can be improved.

Referring to fig. 9, fig. 9 is a flowchart illustrating an image processing method according to still another embodiment of the present application. The image processing method is applied to the electronic equipment, and the electronic equipment comprises a gyroscope. The following will describe the flowchart shown in fig. 9 in detail, and the image processing method specifically may include the following steps:

step S410: an affine transformation matrix between a first image and a second image is acquired, wherein the first image is a previous frame image of the second image in an image sequence.

In the embodiment of the present application, step S410 may refer to the content of the foregoing embodiment, and is not described herein.

Step S420: and acquiring the offset of the position point after the center point of the third image is registered relative to the center point of the third image in the longitudinal direction as a fourth offset based on the registration parameter obtained when the first image is corrected.

In the embodiment of the application, when the electronic device determines the offset of the center point of the second image relative to the center point of the first image in the longitudinal direction, the electronic device may acquire the offset of the position point of the third image after the center point of the third image is registered relative to the center point of the third image in the longitudinal direction based on the registration parameter obtained when the first image is corrected, and use the offset as the fourth offset. The specific acquiring manner may refer to the content of the foregoing embodiment, and is not described herein.

Step S430: and acquiring an offset corresponding to the rotation of the X axis of the gyroscope as a fifth offset based on the detection data of the gyroscope when the first image is acquired and the detection data of the gyroscope when the second image is acquired.

In the embodiment of the application, the offset of the center point of the previous frame image is taken as the offset of the center point of the current frame image, so that errors are caused. Therefore, the electronic device can acquire the offset corresponding to the rotation of the X axis of the gyroscope by using the detection data of the gyroscope when the first image is acquired and the detection data of the gyroscope when the second image is acquired. It will be appreciated that the offset of the centre point is mainly influenced by the rotation of the X-axis of the gyroscope, and that by registering the angular velocity (angular_x) of the gyroscope in the X-axis direction, the longitudinal offset due to rotation about the X-axis can be predicted.

In some embodiments, the electronic device may integrate the angular velocity along the time axis direction after performing difference smoothing on the detection data of the gyroscope when the first image is acquired and the detection data of the gyroscope when the second image is acquired, so as to obtain the rotation angle r; then the view angle (fov) of the image acquisition device of the electronic equipment in the y direction is obtained and is recorded as fov _y The method comprises the steps of carrying out a first treatment on the surface of the The pixel resolution h in the vertical direction is obtained, so that the angular resolution θ=h/fov in the vertical direction can be calculated _y The method comprises the steps of carrying out a first treatment on the surface of the After the angular resolution and the rotation angle are obtained,the longitudinal offset due to rotation about the X-axis, i.e., Δy, can be calculated based on the angular resolution and the rotation angle _r ＝θ×r。

Step S440: an offset of a center point of the second image with respect to a center point of the first image in a longitudinal direction is determined based on the fourth offset and the fifth offset.

In the embodiment of the application, after the fourth offset and the fifth offset are obtained, the offset of the center point of the second image in the longitudinal direction relative to the center point of the first image can be determined based on the fourth offset and the fifth offset, so as to improve the accuracy of the obtained offset of the center point.

In some embodiments, determining the offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction based on the fourth offset and the fifth offset may include: acquiring a difference value between the fourth offset and the fifth offset; if the absolute value of the difference value is larger than a preset threshold value, taking the fifth offset as the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction; and if the absolute value of the difference is smaller than or equal to a preset threshold value, taking the fourth offset as the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction. In this embodiment, when the absolute value of the difference is greater than the preset threshold, it indicates that the offset calculated by the registration parameter of the previous frame is greatly different from the offset calculated by the gyroscope, so the fifth offset may be used instead to improve accuracy.

In some embodiments, when determining the offset of the center point of the second image in the longitudinal direction relative to the center point of the first image according to the fourth offset and the fifth offset, the electronic device may also perform weighted summation on the fourth offset and the fifth offset according to weights corresponding to the preset fourth offset and the fifth offset, so as to obtain the offset of the center point of the second image in the longitudinal direction relative to the center point of the first image. Wherein, the sum of the weights corresponding to the fourth offset and the fifth offset is 1. Of course, the specific manner of determining the offset of the center point of the second image with respect to the center point of the first image according to the offsets (fourth offset and fifth offset) determined in the above two manners may not be limited.

Step S450: and taking the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction as a first offset.

Step S460: based on registration parameters between the first image and the second image, a maximum offset of the second image relative to the first image in a positive direction of a longitudinal direction is obtained as a second offset, and a maximum offset of the second image relative to the first image in a negative direction of the longitudinal direction is obtained as a third offset.

Step S470: and when correcting the target line pixel point of the second image, loading a plurality of lines of pixel points corresponding to the target line pixel point in the first image into a line cache based on the first offset, a first difference value between the second offset and the first offset, and a second difference value between the third offset and the first offset, wherein each line of pixel point in the plurality of lines of pixel points corresponds to a line cache, and the target line of pixel point is any line of pixel point in the second image.

Step S480: and obtaining pixel points matched with each pixel point in the target line pixel point from the line pixel points cached in the line cache as matched pixel points corresponding to each pixel point.

Step S490: and based on the affine transformation matrix, registering the matched pixel points corresponding to each pixel point to obtain registered pixel points serving as corrected pixel points of each pixel point.

In the embodiment of the present application, the steps S450 to S390 can refer to the content of other embodiments, which are not described herein.

According to the image processing method provided by the embodiment of the application, when the image is corrected based on image registration, besides the offset of the image global in the longitudinal positive direction and the image global in the longitudinal negative direction, the offset of the image center point in the longitudinal direction is obtained, when the corresponding line pixel points are cached, the corresponding line pixel points are cached after the preset offset is carried out based on the offset of the center point, so that the part of line pixel points which originally need to be cached and have the first offset are discharged, the line buffering quantity is reduced, and the hardware cost is effectively reduced. In addition, when the offset of the center point is acquired, the offset of the center point of the previous frame image is calculated according to registration parameters obtained when the previous frame image is corrected, the offset of the center point is calculated through detection data of a gyroscope, the offset of the center point of the current frame image is comprehensively determined based on the offset calculated in two modes, and the accuracy of the acquired offset of the center point can be improved.

Referring to fig. 10, a block diagram of an image processing apparatus 400 according to an embodiment of the application is shown. The image processing apparatus 400 is applied to the above-described electronic device, and the image processing apparatus 400 includes: parameter acquisition module 410, offset acquisition module 420, pixel buffer module 430, pixel determination module 440, and pixel correction module 450. The parameter obtaining module 410 is configured to obtain registration parameters between a first image and a second image, where the first image is a previous frame image of the second image in the image sequence; the offset obtaining module 420 is configured to obtain, as a first offset, an offset of a center point of the second image with respect to a center point of the first image in a longitudinal direction, as a second offset, a maximum offset of the second image with respect to the first image in a positive direction of the longitudinal direction, and as a third offset, a maximum offset of the second image with respect to the first image in a negative direction of the longitudinal direction; the pixel buffer module 430 is configured to load a plurality of rows of pixels in the first image corresponding to the target row of pixels into a row buffer based on the first offset, a first difference between the second offset and the first offset, and a second difference between the third offset and the first offset when correcting the target row of pixels in the second image, where each row of pixels in the plurality of rows of pixels corresponds to a row buffer, and the target row of pixels is a pixel of any row in the second image; the pixel determining module 440 is configured to obtain, from the line pixel points cached in the line cache, a pixel point that is matched with each pixel point in the target line pixel point, as a matched pixel point corresponding to each pixel point; the pixel correction module 450 is configured to register the matched pixel point corresponding to each pixel point based on the registration parameter, so as to obtain a registered pixel point, which is used as a corrected pixel point for each pixel point.

In some embodiments, the pixel buffer module 430 may be specifically configured to: when correcting the target line pixel point of the second image, acquiring a line pixel point, the position of which is matched with the target line pixel point, in the first image as a matched line pixel point; acquiring the row position of the pixel points of the matched row, which is offset according to the first offset, as a target row position; acquiring pixel points of a target line number along a longitudinal positive direction and a longitudinal negative direction respectively from the position of the target line in the first image based on the difference value between the second offset and the first offset and the difference value between the third offset and the first offset, so as to obtain a plurality of rows of pixel points; and loading each row of pixel points in the plurality of rows of pixel points into a row of cache.

As a possible implementation manner, the pixel buffer module 430 obtains a plurality of rows of pixel points from the target row position in the first image along the positive direction and the negative direction of the longitudinal direction based on the difference value between the second offset and the first offset and the difference value between the third offset and the first offset, where the obtaining the plurality of rows of pixel points may include: acquiring a difference value between the second offset and the first offset as a first difference value, and acquiring a difference value between the third offset and the first offset as a second difference value; acquiring a first line number of pixel points from the target line position in the first image along the longitudinal positive direction, wherein the first line number is matched with the first difference value; and acquiring a second number of pixel points along the longitudinal negative direction from the target line position in the first image, wherein the second number of pixel points is matched with the second difference value, and the sum of the first number of pixel points and the second number of pixel points is the target line number.

As a possible implementation manner, the pixel buffer module 430 obtains a plurality of rows of pixel points from the target row position in the first image along the positive direction and the negative direction of the longitudinal direction based on the difference value between the second offset and the first offset and the difference value between the third offset and the first offset, where the obtaining the plurality of rows of pixel points may include: acquiring a difference value between the second offset and the first offset as a first difference value, and acquiring a difference value between the third offset and the first offset as a second difference value; obtaining the maximum value of the first difference value and the second difference value; and acquiring pixel points of a specified line number along the positive direction and the negative direction in the longitudinal direction respectively from the target line position in the first image to obtain a plurality of lines of pixel points, wherein the specified line number is matched with the maximum value.

In some embodiments, the offset acquisition module 420 may be specifically configured to: based on registration parameters obtained when the first image is corrected, acquiring offset of a position point obtained after the center point of a third image is registered relative to the center point of the third image in the longitudinal direction, wherein the third image is a previous frame image of the first image, and the offset is used as offset of the center point of the second image relative to the center point of the first image in the longitudinal direction; taking the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction as a first offset; based on registration parameters between the first image and the second image, a maximum offset of the second image relative to the first image in a positive direction of a longitudinal direction is obtained as a second offset, and a maximum offset of the second image relative to the first image in a negative direction of the longitudinal direction is obtained as a third offset.

As a possible implementation, the registration parameters include an affine transformation matrix. The offset obtaining module 420 obtains, based on a registration parameter obtained when the first image is corrected, an offset of a position point obtained by registering a center point of a third image with respect to the center point of the third image in a longitudinal direction, as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction, and may include: performing affine transformation on the central point of the third image based on an affine transformation matrix obtained when the first image is corrected, so as to obtain the central point after affine transformation as a target position point; an offset of the target position point with respect to the center point of the third image in the longitudinal direction is acquired as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction.

As a possible implementation, the electronic device comprises a gyroscope. The offset obtaining module 420 obtains, based on a registration parameter obtained when the first image is corrected, an offset of a position point obtained by registering a center point of a third image with respect to the center point of the third image in a longitudinal direction, as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction, and may include: acquiring offset of a position point after the center point of a third image is registered relative to the center point of the third image in the longitudinal direction based on registration parameters obtained when the first image is corrected, and taking the offset as a fourth offset; acquiring an offset corresponding to X-axis rotation of the gyroscope as a fifth offset based on detection data of the gyroscope when the first image is acquired and detection data of the gyroscope when the second image is acquired; an offset of a center point of the second image with respect to a center point of the first image in a longitudinal direction is determined based on the fourth offset and the fifth offset.

Optionally, the determining, by the offset obtaining module 420, an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction based on the fourth offset and the fifth offset may include: acquiring a difference value between the fourth offset and the fifth offset; if the absolute value of the difference value is larger than a preset threshold value, taking the fifth offset as the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction; and if the absolute value of the difference is smaller than or equal to a preset threshold value, taking the fourth offset as the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction.

In some embodiments, the image processing apparatus 400 may further include: and returning to the module. The return module is configured to register, based on the registration parameter, the matched pixel point corresponding to each pixel point to obtain a registered pixel point, and after the registered pixel point is used as a corrected pixel point for each pixel point, return to the step of loading a plurality of rows of pixel points corresponding to the target row of pixel points in the first image into a row buffer memory until the correction is completed for each row of pixel points in the second image to obtain an image corrected for the second image when the target row of pixel points in the second image are corrected.

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus and modules described above may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.

In several embodiments provided by the present application, the coupling of the modules to each other may be electrical, mechanical, or other.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules.

In summary, according to the scheme provided by the application, by acquiring the registration parameter between the first image and the second image, the first image is the previous frame image of the second image in the image sequence, before performing image registration based on the registration parameter, the offset of the center point of the second image relative to the center point of the first image in the longitudinal direction is acquired as the first offset, the maximum offset of the second image relative to the first image in the positive direction in the longitudinal direction is acquired as the second offset, and the maximum offset of the second image relative to the first image in the negative direction in the longitudinal direction is acquired as the third offset. When correcting the target line pixel point of the second image, loading a plurality of lines of pixel points corresponding to the target line pixel point in the first image into a line buffer based on the first offset, the first difference value between the second offset and the first offset, and the second difference value between the third offset and the first offset, wherein each line of pixel point in the plurality of lines of pixel points corresponds to a line buffer, the target line pixel point is a pixel point of any line in the second image, the pixel point matched with each pixel point in the target line pixel point is obtained from the line pixel points buffered in the line buffer and used as a matched pixel point corresponding to each pixel point, and then the matched pixel point corresponding to each pixel point is registered based on registration parameters to obtain a registered pixel point which is used as a pixel point corrected for each pixel point. Therefore, when the line pixel points of the image are loaded into the line cache, partial offset of the maximum offset of the image in the longitudinal direction can be preset based on the offset of the central point in the longitudinal direction, and the corresponding line pixel points are loaded into the line cache based on the offset, so that the line cache required during image registration is effectively reduced, and the hardware cost is reduced.

Referring to fig. 11, a block diagram of an electronic device according to an embodiment of the present application is shown. The electronic device 100 may be an electronic device capable of running an application program, such as a smart phone, a tablet computer, a smart watch, smart glasses, a notebook computer, etc. The electronic device 100 of the present application may include one or more of the following components: a processor 110, a memory 120, and one or more application programs, wherein the one or more application programs may be stored in the memory 120 and configured to be executed by the one or more processors 110, the one or more program(s) configured to perform the method as described in the foregoing method embodiments.

Processor 110 may include one or more processing cores. The processor 110 utilizes various interfaces and lines to connect various portions of the overall electronic device 100, perform various functions of the electronic device 100, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and invoking data stored in the memory 120. Alternatively, the processor 110 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 110 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for being responsible for rendering and drawing of display content; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 110 and may be implemented solely by a single communication chip.

The Memory 120 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Memory 120 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described below, etc. The storage data area may also store data created by the electronic device 100 in use (e.g., phonebook, audiovisual data, chat log data), and the like.

Referring to fig. 12, a block diagram of a computer readable storage medium according to an embodiment of the present application is shown. The computer readable medium 800 has stored therein program code which can be invoked by a processor to perform the methods described in the method embodiments described above.

The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 comprises a non-volatile computer readable medium (non-transitory computer-readable storage medium). The computer readable storage medium 800 has storage space for program code 810 that performs any of the method steps described above. The program code can be read from or written to one or more computer program products. Program code 810 may be compressed, for example, in a suitable form.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be appreciated by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not drive the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, applied to an electronic device, comprising:

acquiring registration parameters between a first image and a second image, wherein the first image is a previous frame image of the second image in an image sequence;

based on registration parameters obtained when the first image is corrected, acquiring offset of a position point obtained after the center point of a third image is registered relative to the center point of the third image in the longitudinal direction, wherein the third image is a previous frame image of the first image, and the offset is used as offset of the center point of the second image relative to the center point of the first image in the longitudinal direction;

Taking the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction as a first offset;

based on registration parameters between the first image and the second image, acquiring a maximum offset of the second image relative to the first image in a positive longitudinal direction as a second offset and a maximum offset of the second image relative to the first image in a negative longitudinal direction as a third offset;

when correcting the target line pixel point of the second image, acquiring a line pixel point, which is matched with the target line pixel point in position in the first image, as a matched line pixel point, wherein the target line pixel point is a pixel point of any line in the second image;

acquiring the row position of the pixel points of the matched row, which is offset according to the first offset, as a target row position;

acquiring pixel points of a target line number along a longitudinal positive direction and a longitudinal negative direction respectively from the position of the target line in the first image based on the difference value between the second offset and the first offset and the difference value between the third offset and the first offset, so as to obtain a plurality of rows of pixel points;

Loading each row of pixel points in the plurality of rows of pixel points into a row of cache;

obtaining pixel points matched with each pixel point in the target line pixel point from line pixel points cached in the line cache as matched pixel points corresponding to each pixel point;

and based on the registration parameters, registering the matched pixel points corresponding to each pixel point to obtain registered pixel points serving as corrected pixel points of each pixel point.

2. The method according to claim 1, wherein the obtaining a plurality of rows of pixel points from the target row position in the first image based on the difference between the second offset and the first offset and the difference between the third offset and the first offset, the pixel points respectively along the positive direction and the negative direction in the longitudinal direction includes:

acquiring a difference value between the second offset and the first offset as a first difference value, and acquiring a difference value between the third offset and the first offset as a second difference value;

acquiring a first line number of pixel points from the target line position in the first image along the longitudinal positive direction, wherein the first line number is matched with the first difference value;

And acquiring a second number of pixel points along the longitudinal negative direction from the target line position in the first image, wherein the second number of pixel points is matched with the second difference value, and the sum of the first number of pixel points and the second number of pixel points is the target line number.

3. The method according to claim 1, wherein the obtaining a plurality of rows of pixel points from the target row position in the first image based on the difference between the second offset and the first offset and the difference between the third offset and the first offset, the pixel points respectively along the positive direction and the negative direction in the longitudinal direction includes:

obtaining the maximum value of the first difference value and the second difference value;

and acquiring pixel points of a specified line number along the positive direction and the negative direction in the longitudinal direction respectively from the target line position in the first image to obtain a plurality of lines of pixel points, wherein the specified line number is matched with the maximum value.

4. The method according to claim 1, wherein the registration parameters include an affine transformation matrix, and the acquiring, based on the registration parameters obtained when the first image is corrected, an offset of a position point of a third image after the center point of the third image is registered with respect to the center point of the third image in the longitudinal direction as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction includes:

Performing affine transformation on the central point of the third image based on an affine transformation matrix obtained when the first image is corrected, so as to obtain the central point after affine transformation as a target position point;

an offset of the target position point with respect to the center point of the third image in the longitudinal direction is acquired as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction.

5. The method according to claim 1, wherein the electronic device includes a gyroscope, and the acquiring, based on a registration parameter obtained when the first image is corrected, an offset of a position point obtained by registering a center point of a third image with respect to the center point of the third image in a longitudinal direction as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction includes:

acquiring offset of a position point after the center point of a third image is registered relative to the center point of the third image in the longitudinal direction based on registration parameters obtained when the first image is corrected, and taking the offset as a fourth offset;

acquiring an offset corresponding to X-axis rotation of the gyroscope as a fifth offset based on detection data of the gyroscope when the first image is acquired and detection data of the gyroscope when the second image is acquired;

An offset of a center point of the second image with respect to a center point of the first image in a longitudinal direction is determined based on the fourth offset and the fifth offset.

6. The method of claim 5, wherein determining an offset of a center point of the second image relative to a center point of the first image in a longitudinal direction based on the fourth offset and the fifth offset comprises:

acquiring a difference value between the fourth offset and the fifth offset;

if the absolute value of the difference value is larger than a preset threshold value, taking the fifth offset as the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction;

and if the absolute value of the difference is smaller than or equal to a preset threshold value, taking the fourth offset as the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction.

7. The method according to any one of claims 1-6, wherein after the registering the matching pixel point corresponding to each pixel point based on the registration parameter, to obtain a registered pixel point, the method further includes:

And returning to the step of obtaining the line pixel points matched with the target line pixel points in the first image as the matched line pixel points when the target line pixel points of the second image are corrected, until the correction is completed on each line of pixel points of the second image, and obtaining the corrected image of the second image.

8. An image processing apparatus, characterized by being applied to an electronic device, comprising: a parameter acquisition module, an offset acquisition module, a pixel buffer module, a pixel determination module and a pixel correction module, wherein,

the parameter acquisition module is used for acquiring registration parameters between a first image and a second image, wherein the first image is a previous frame image of the second image in an image sequence;

the offset obtaining module is configured to obtain, based on a registration parameter obtained when the first image is corrected, an offset of a position point obtained after the center point of a third image is registered with respect to the center point of the third image in a longitudinal direction, as an offset of the center point of the second image with respect to the center point of the first image in the longitudinal direction, where the third image is a previous frame image of the first image; taking the offset of the central point of the second image relative to the central point of the first image in the longitudinal direction as a first offset; based on registration parameters between the first image and the second image, acquiring a maximum offset of the second image relative to the first image in a positive longitudinal direction as a second offset and a maximum offset of the second image relative to the first image in a negative longitudinal direction as a third offset;

The pixel buffer module is used for acquiring row pixel points, as matched row pixel points, of which the positions are matched with the target row pixel points in the first image when correcting the target row pixel points in the second image, wherein the target row pixel points are pixel points of any row in the second image; acquiring the row position of the pixel points of the matched row, which is offset according to the first offset, as a target row position; acquiring pixel points of a target line number along a longitudinal positive direction and a longitudinal negative direction respectively from the position of the target line in the first image based on the difference value between the second offset and the first offset and the difference value between the third offset and the first offset, so as to obtain a plurality of rows of pixel points; loading each row of pixel points in the plurality of rows of pixel points into a row of cache;

the pixel determining module is used for obtaining a pixel point matched with each pixel point in the target line pixel point from line pixel points cached in the line cache as a matched pixel point corresponding to each pixel point;

the pixel correction module is used for carrying out registration on the matched pixel points corresponding to each pixel point based on the registration parameters to obtain registered pixel points serving as pixel points after correction of each pixel point.

9. An electronic device, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the method of any of claims 1-7.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a program code, which is callable by a processor for executing the method according to any one of claims 1-7.