CN117241142A

CN117241142A - Dynamic correction method and device for pitch angle of pan-tilt camera, equipment and storage medium

Info

Publication number: CN117241142A
Application number: CN202311249013.8A
Authority: CN
Inventors: 刘建德; 杨吉团; 贾承晖; 庞国安
Original assignee: Shenzhen Consys Technology Co ltd
Current assignee: Shenzhen Consys Technology Co ltd
Priority date: 2023-09-22
Filing date: 2023-09-22
Publication date: 2023-12-15

Abstract

The embodiment of the application provides a method and a device for dynamically correcting a pitch angle of a pan-tilt camera, equipment and a storage medium, and belongs to the technical field of aerial photographing equipment. The method comprises the following steps: acquiring a video frame image pair and current pitching angle, current shooting height and current shooting position information of a video frame image pair of a pan-tilt camera; performing pixel displacement measurement on the video frame image pair to obtain current pixel displacement; performing physical displacement measurement on the current shooting position information of the video frame image pair to obtain current position displacement; carrying out displacement prediction on the ratio of the current pixel displacement to the preset pixel position to obtain a predicted position displacement; performing difference calculation on the current position displacement and the predicted position displacement to obtain a position displacement difference; according to the position displacement difference, the current pitching angle and the current shooting height, carrying out pitch angle prediction to obtain a corrected pitching angle; and correcting the pitch angle of the pan-tilt camera according to the corrected pitch angle. The embodiment of the application can realize dynamic correction of the pitch angle without adding additional equipment.

Description

Dynamic correction method and device for pitch angle of pan-tilt camera, equipment and storage medium

Technical Field

The application relates to the technical field of aerial photographing equipment, in particular to a method and a device for dynamically correcting a pitch angle of a pan-tilt camera, equipment and a storage medium.

Background

Aerial photo holder head camera has very important application in agriculture and forestry plant protection, geographical investigation, electric power inspection, traffic inspection, non-civil fields, etc. Along with the development of aerial photo holder cloud platform cameras, people are more and more high to aerial photo holder cloud platform camera's control accuracy. The pitch angle of the aerial camera is easy to accumulate mechanical errors in high-altitude flight, and the stepping motor cannot perform angle compensation under the condition of lost steps, so that the errors are larger and larger, and therefore the pitch angle needs to be corrected. In the related art, two main modes for correcting the pitch angle are adopted, wherein the first mode is to use a standard instrument for static correction during initialization before aerial photography, and the second mode is to use a laser radar for auxiliary correction. The first correction method is only effective for fixed scenes, and cannot solve the accumulated error of pitch angle in the motion process. The second correction method needs to add a laser radar on aerial photographing equipment, and the traditional laser radar cannot meet the requirement of high altitude and long distance. Therefore, how to dynamically adjust the pitch angle of the aerial photographing device becomes a technical problem to be solved.

Disclosure of Invention

The embodiment of the application mainly aims to provide a method, a device, equipment and a storage medium for dynamically correcting a pitch angle of a pan-tilt camera, aiming at realizing dynamic pitch angle correction without adding additional equipment.

In order to achieve the above objective, a first aspect of an embodiment of the present application provides a method for dynamically correcting a pitch angle of a pan-tilt camera, which is applied to a pan-tilt camera, and the method includes:

acquiring current video frame image pairs shot by the cradle head camera and image shooting information of the video frame image pairs; wherein the image photographing information includes: current pitching angle, current shooting height and current shooting position information;

performing pixel displacement measurement according to the video frame image pair to obtain current pixel displacement;

performing physical displacement measurement on the current shooting position information of the video frame image pair to obtain current position displacement;

performing position displacement prediction according to the ratio of the current pixel displacement to the preset pixel position to obtain predicted position displacement; wherein the pixel position ratio characterizes a ratio between pixel displacement and position displacement;

performing difference calculation according to the current position displacement and the predicted position displacement to obtain a position displacement difference;

According to the position displacement difference, the current pitching angle and the current shooting height, carrying out pitch angle prediction to obtain a corrected pitching angle;

and correcting the pitch angle of the pan-tilt camera according to the corrected pitch angle.

In some embodiments, the video frame image pair comprises: a current video frame image and a selected video frame image, the selected video frame image being a previous video frame image of the current video frame image; the measuring pixel displacement according to the video frame image pair to obtain the current pixel displacement comprises the following steps:

selecting a central area of the current video frame image to obtain a selected area;

extracting the central position of the selected area in the current video frame image to obtain current central position information;

extracting the central position of the selected area in the selected video frame image to obtain selected central position information;

and calculating the offset of the current center position information and the selected center position information to obtain the current pixel displacement.

In some embodiments, the performing physical displacement measurement on the current shooting position information of the video frame image pair to obtain a current position displacement includes:

Acquiring the current shooting position information of the current video frame image to obtain current position information;

acquiring the current shooting position information of the selected video frame image to obtain selected position information;

and carrying out displacement calculation according to the current position information and the selected position information to obtain the current position displacement.

In some embodiments, before the estimating of the position displacement according to the ratio of the current pixel displacement to the preset pixel position, the method further includes:

the construction of the preset pixel position ratio specifically comprises the following steps:

acquiring at least two historical video frame images shot by the cradle head camera and historical shooting position information of each historical video frame image;

performing pixel displacement measurement on the historical video frame images of two adjacent frames to obtain at least two historical pixel displacements;

performing position displacement measurement on the historical shooting position information of the historical video frame images of two adjacent frames to obtain at least two historical position displacements;

calculating the ratio of the historical pixel displacement to the corresponding historical position displacement to obtain a candidate pixel position ratio;

And carrying out average value calculation on the at least two candidate pixel position ratios to obtain the preset pixel position ratio.

In some embodiments, the predicting the pitch angle according to the position displacement difference, the current pitch angle and the current shooting height to obtain a corrected pitch angle includes:

performing pitch angle prediction on the position displacement difference, the current pitch angle and the current shooting height through a preset pitch angle estimation model to obtain candidate pitch angles;

performing deviation measurement according to the candidate pitch angle, the current pitch angle, a preset average deviation and a preset smoothing coefficient to obtain a pitch angle deviation value;

and summing according to the current pitching angle and the pitching angle deviation value to obtain the corrected pitching angle.

In some embodiments, before the performing deviation measurement according to the candidate pitch angle, the current pitch angle, a preset average deviation and a preset smoothing coefficient to obtain a pitch angle deviation value, the method further includes: the construction of the preset average deviation specifically comprises the following steps:

acquiring historical pitching angles of at least two historical video frame images;

performing difference value calculation on the historical pitching angles of each historical video frame image and a preset verification pitching angle to obtain pitching angle differences;

And carrying out average value calculation on at least two pitching angle differences to obtain the preset average deviation.

In some embodiments, after the capturing the image capturing information of the current video frame image pair and the video frame image pair captured by the pan-tilt camera, the method includes:

acquiring the image shooting information of the current video frame image to obtain current shooting information;

acquiring the image shooting information of the selected video frame image to obtain the last shooting information;

comparing the current shooting information with the last shooting information to obtain shooting information difference;

and if the shooting information difference is larger than a preset threshold value, stopping calculation of the corrected pitch angle of the pan-tilt camera.

In order to achieve the above object, a second aspect of the embodiments of the present application provides a dynamic correction device for a pitch angle of a pan-tilt camera, the device including:

be applied to cloud platform camera, the device includes:

the acquisition module is used for acquiring the current video frame image pair shot by the cradle head camera and the image shooting information of the video frame image pair; wherein the image photographing information includes: current pitching angle, current shooting height and current shooting position information;

The pixel displacement measurement module is used for measuring the pixel displacement according to the video frame image pair to obtain the current pixel displacement;

a physical displacement measurement, configured to perform a physical displacement measurement on the current shooting position information of the video frame image pair, to obtain a current position displacement;

the position prediction module is used for carrying out position displacement prediction according to the ratio of the current pixel displacement to the preset pixel position to obtain predicted position displacement; wherein the pixel position ratio characterizes a ratio between pixel displacement and position displacement;

the displacement difference calculation module is used for carrying out difference calculation according to the current position displacement and the predicted position displacement to obtain a position displacement difference;

the angle prediction module is used for predicting a pitch angle according to the position displacement difference, the current pitch angle and the current shooting height to obtain a corrected pitch angle;

and the correction module is used for correcting the pitch angle of the pan-tilt camera according to the corrected pitch angle.

To achieve the above object, a third aspect of the embodiments of the present application proposes an electronic device, including a memory storing a computer program and a processor implementing the method according to the first aspect when the processor executes the computer program.

To achieve the above object, a fourth aspect of the embodiments of the present application proposes a computer-readable storage medium storing a computer program which, when executed by a processor, implements the method of the first aspect.

According to the dynamic correction method, the device, the equipment and the storage medium for the pitch angle of the pan-tilt camera, the current pixel displacement and the current position displacement of two adjacent frames of video frame images are obtained, and then the theoretical position displacement is calculated to be used as the predicted position displacement based on the current pixel displacement and the preset pixel position ratio. And then determining a position displacement difference based on the predicted position displacement and the current position displacement to judge whether the pitch angle has deviation according to the position displacement difference. And recalculating the real pitching angle based on the position displacement difference, the current pitching angle and the current shooting height to serve as a corrected pitching angle, so as to correct the pitching angle of the pan-tilt camera according to the corrected pitching angle, and realize real-time correction of the pan-tilt camera in the dynamic shooting process. Therefore, the correction pitch angle is calculated by adopting the relation among the pixel displacement, the position displacement and the shooting height, so that the correction pitch angle is accurately and simply calculated, the correction pitch angle is not required to be statically corrected in advance, and external equipment is not required to be added, and the pan-tilt camera can have higher control precision in high-altitude long-distance shooting.

Drawings

Fig. 1 is a flowchart of a method for dynamically correcting a pitch angle of a pan-tilt camera according to an embodiment of the present application;

fig. 2 is a flowchart of a method for dynamically correcting a pitch angle of a pan-tilt camera according to another embodiment of the present application;

fig. 3 is a flowchart of step S102 in fig. 1;

fig. 4 is a flowchart of step S103 in fig. 1;

fig. 5 is a flowchart of a method for dynamically correcting a pitch angle of a pan-tilt camera according to another embodiment of the present application;

fig. 6 is a flowchart of step S106 in fig. 1;

fig. 7 is a schematic diagram of an optical axis and the ground of a pan-tilt camera in the dynamic correction method of a pitch angle of the pan-tilt camera according to the embodiment of the present application;

fig. 8 is a flowchart of a method for dynamically correcting a pitch angle of a pan-tilt camera according to another embodiment of the present application;

fig. 9 is a schematic diagram illustrating a distribution of tilt angle differences of a pan-tilt camera according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a dynamic correcting device for a pitch angle of a pan-tilt camera according to an embodiment of the present application;

fig. 11 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

It should be noted that although functional block division is performed in a device diagram and a logic sequence is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.

First, several nouns involved in the present application are parsed:

cradle head camera: the traditional video-level cloud platform is reduced to a size capable of being held by hands, and the consumer group is the common public. The tripod head camera adopts an integrated high-integration design, and is highly integrated with a triaxial mechanical stability-enhancing tripod head and high-definition camera used on a professional unmanned aerial vehicle.

Pitch angle: and an included angle between the X axis of the machine body coordinate system and the horizontal plane.

Sliding coefficient: the magnitude of the sliding friction is proportional to the positive pressure between objects in contact with each other: i.e., f=μn, where μ is a proportionality constant called "coefficient of sliding friction", which is a value without units. The coefficient of sliding friction is related to the material contacting the object, the degree of surface smoothness, the degree of dryness, the surface temperature, the relative movement speed, etc.

Aerial photo holder head camera has very important application in the fields of agricultural plant protection, remote sensing mapping, geographical investigation, electric power inspection, traffic inspection, non-civil use and the like. Along with the development of unmanned aerial vehicle technology, control accuracy requirements for a tripod head camera are higher and higher, mechanical errors are easy to accumulate when a pitching angle of the tripod head camera flies at high altitude, angle compensation cannot be performed under the condition that a stepping motor loses steps, and the errors are larger and larger. In order to accurately control the pan-tilt camera, the pitch angle needs to be corrected. In the related art, there are two main methods for correcting the pitch angle, the first method is that the pan-tilt camera is initialized before aerial photography, and a standard instrument is used for static correction during initialization. The second approach is to use lidar to assist in rectification. The first method, although simple in logic, is effective for static scene fixed deviation, and cannot solve accumulated errors in the motion process. The second method uses laser radar to assist in correction, equipment is required to be additionally arranged on the unmanned aerial vehicle, the unmanned aerial vehicle is inconvenient, and general laser equipment cannot meet the high-altitude long-distance requirement.

Based on the above, the embodiment of the application provides a dynamic correction method, a device, equipment and a storage medium for the pitch angle of a tripod head camera, which are characterized in that the position information, the height and the theoretical pitch angle of each frame of video frame image are acquired by the tripod head camera, then the pixel displacement between two adjacent video frame images is analyzed, the position displacement of the two video frame images is calculated, the predicted position displacement is calculated according to the ratio of the pixel displacement to the preset pixel position, the predicted position displacement is compared with the actual position displacement to determine a displacement difference, and then the actual pitch angle is calculated according to the displacement difference, the height and the theoretical pitch angle to be used for correcting the pitch angle of the tripod head camera according to the corrected pitch angle, so that the pitch angle correction of the tripod head camera in the motion process can be completed without adding external equipment, the cost is saved, and the tripod head camera can have higher control precision in the motion process.

The method, the device, the equipment and the storage medium for dynamically correcting the pitch angle of the pan-tilt camera provided by the embodiment of the application are specifically described through the following embodiment, and the method for dynamically correcting the pitch angle of the pan-tilt camera in the embodiment of the application is described first.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.

Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

The embodiment of the application provides a dynamic correction method for a pitch angle of a pan-tilt camera, and relates to the technical field of aerial photographing equipment. The dynamic correction method for the pitch angle of the pan-tilt camera provided by the embodiment of the application can be applied to aerial photographing equipment and also can be applied to software of the aerial photographing equipment; the software may be an application for implementing a dynamic correction method of the pitch angle of the pan-tilt camera, but is not limited to the above form.

It should be noted that, in each specific embodiment of the present application, when related processing is required according to user information, user behavior data, user history data, user location information, and other data related to user identity or characteristics, permission or consent of the user is obtained first, and the collection, use, processing, and the like of the data comply with related laws and regulations and standards. In addition, when the embodiment of the application needs to acquire the sensitive personal information of the user, the independent permission or independent consent of the user is acquired through popup or jump to a confirmation page and the like, and after the independent permission or independent consent of the user is definitely acquired, the necessary relevant data of the user for enabling the embodiment of the application to normally operate is acquired.

Fig. 1 is an optional flowchart of a method for dynamically correcting a pitch angle of a pan-tilt camera according to an embodiment of the present application, and the method for dynamically correcting a pitch angle of a pan-tilt camera is applied to a pan-tilt camera, where the method in fig. 1 may include, but is not limited to, steps S101 to S107.

Step S101, obtaining image shooting information of a current video frame image pair and a video frame image pair shot by a cradle head camera; wherein, the image photographing information includes: current pitching angle, current shooting height and current shooting position information;

Step S102, pixel displacement measurement is carried out according to video frame image pairs, and current pixel displacement is obtained;

step S103, physical displacement measurement is carried out on the current shooting position information of the video frame image pair to obtain current position displacement;

step S104, position displacement estimation is carried out according to the ratio of the current pixel displacement to the preset pixel position, and predicted position displacement is obtained; wherein the pixel position ratio characterizes a ratio between the pixel displacement and the position displacement;

step S105, calculating a difference value according to the current position displacement and the predicted position displacement to obtain a position displacement difference;

step S106, pitch angle prediction is carried out according to the position displacement difference, the current pitch angle and the current shooting height, and a corrected pitch angle is obtained;

and step S107, correcting the pitch angle of the pan-tilt camera according to the corrected pitch angle.

In the steps S101 to S107 shown in the embodiment of the present application, the current pitch angle, the current shooting height and the current shooting position information of the video frame image pair are acquired by acquiring the video frame image pair currently shot by the pan-tilt camera. And carrying out pixel displacement measurement on the video frame image pair to obtain current pixel displacement, and carrying out displacement measurement on the current shooting position information of the video frame image pair to obtain current position displacement. In order to judge whether the pitch angle has errors, the position displacement is estimated according to the ratio of the current pixel displacement to the preset pixel position to obtain the predicted position displacement. If the predicted position displacement and the current position displacement are consistent, representing that the acquired current pitching angle is correct, and if the predicted position displacement and the current position displacement are inconsistent, representing that the acquired current pitching angle has deviation, calculating the difference between the predicted position displacement and the current position displacement to obtain a position displacement difference. If the position displacement difference is not zero, representing that the current pitching angle has an error, predicting the pitching angle according to the position displacement difference, the current pitching angle and the current shooting height to obtain a corrected pitching angle, wherein the corrected pitching angle is the real pitching angle of the pan-tilt camera, and correcting the pitching angle of the pan-tilt camera according to the corrected pitching angle. Therefore, the pitch angle correction is carried out on the moving tripod head camera based on the pixel displacement, the shooting height and the shooting position of the video frame image, the static correction in advance is not needed, and external equipment is not needed to be added, so that the tripod head camera can have higher control precision when shooting at high altitude and long distance.

In step S101 of some embodiments, the pair of video frame images includes: the camera comprises a current video frame image and a selected video frame image, wherein the current video frame image is a video frame image currently shot by a holder camera, and the selected video frame image is a video frame image of a frame above the current video frame image. And judging the pixel displacement of the cradle head camera in the motion process by acquiring adjacent video frame images. It should be noted that, after the current video frame image and the selected video frame image are obtained, the current shooting position information of the current video frame image is obtained as the current position information, and the current shooting position information of the selected video frame image is used as the selected position information, so as to determine the moving distance of the pan-tilt camera on the ground according to the current position information and the selected position information. It should be further noted that the current shooting height is a vertical height of shooting the current video frame image, that is, a current height of the unmanned aerial vehicle. The current pitching angle is also the pitching angle reading when shooting the current video frame image, and belongs to the pitching angle automatically detected by the pan-tilt camera.

Referring to fig. 2, in some embodiments, after step S101, the method for dynamically correcting the pitch angle of the pan-tilt camera may further include, but is not limited to, steps S201 to S204:

Step S201, obtaining image shooting information of a current video frame image to obtain current shooting information;

step S202, obtaining image shooting information of a selected video frame image to obtain the last shooting information;

step S203, comparing the current shooting information with the last shooting information to obtain shooting information difference;

and step S204, if the shooting information difference is larger than a preset threshold value, stopping calculation of the corrected pitch angle of the pan-tilt camera.

In steps S201 to S202 of some embodiments, the image capturing information of the current video frame image is taken as current capturing information, the image capturing information of the selected video frame image is taken as last capturing information, and whether the pan-tilt camera is moving too much can be analyzed through the current capturing information and the last capturing information. If the movement is not too large, the image shooting information collected by the characterization tripod head camera is faulty, and the pitch angle is not required to be corrected.

In step S203 of some embodiments, the current photographing information and the last photographing information are compared. Note that, the current shooting information includes: current pitching angle, current shooting height and current position information, the last shooting information comprises: last pitch angle, last shooting height and selected position information. And comparing the current pitching angle with the last pitching angle to obtain a pitching angle difference value, comparing the current shooting height with the last shooting height to obtain a height difference value, and comparing the current position information with the selected position information to obtain a position difference value. Therefore, the difference values of the plurality of comparisons are determined as photographing information differences, and the photographing information differences can determine the movement condition of the pan-tilt camera.

In step S204 of some embodiments, if the difference between the photographing information is greater than the preset threshold, it is indicated that the collected image photographing information has a fault, so the pan-tilt camera correction is stopped, that is, the steps S102 to S107 are stopped, and the steps S102 to S107 are continued to be executed when the difference between the photographing information is less than the preset threshold. It should be noted that, the preset threshold includes: and if the height difference value is larger than the height threshold value, the pitch angle difference value is larger than the pitch angle threshold value, and the position difference value is larger than the displacement threshold value, the shooting information is excessively large, and the steps S102 to S107 are executed after waiting again.

For example, if the height threshold is 1m, the pitch angle threshold is 0.5, the displacement threshold is 2m, it is determined whether the height difference is greater than 1m, the pitch angle difference is greater than 0.5, and the position difference is greater than 2m, so as to determine whether the difference of the photographed information is too large.

In steps S201 to S204 illustrated in this embodiment, the correction of the pan-tilt camera is determined by calculating the difference between the image capturing information of the previous video frame image and the current video frame image according to whether the difference between the image capturing information is too large, so as to prevent the pan-tilt camera from being still subjected to pitch angle correction when the image capturing information is not collected, so as to ensure that the pan-tilt camera is subjected to pitch angle correction again under the condition that the image capturing information is accurately collected, and the pitch angle correction of the pan-tilt camera is more accurate.

Referring to fig. 3, in some embodiments, step S102 may include, but is not limited to, steps S301 to S304:

step S301, selecting a central area of a current video frame image to obtain a selected area;

step S302, extracting the central position of a selected area in a current video frame image to obtain current central position information;

step S303, extracting the central position of the selected area in the selected video frame image to obtain the selected central position information;

step S304, calculating the offset of the current center position information and the selected center position information to obtain the current pixel displacement.

In step S301 of some embodiments, a center region is selected from the current video frame image as a selected region, and the selected region is used to measure pixel displacement of different video frame images. For example, if the pixels of the current video frame image are 1080×1920, a center region of 50×50 is selected in the current video frame image.

In step S302 of some embodiments, a center is selected in a selected area of the current video frame image, that is, a position of the selected area center in the current video frame image is acquired to obtain current center position information. For example, if the selected area is 50×50, the center position information of the selected area in the current video frame image is determined to be (540, 960).

In step S303 of some embodiments, a center position of the selected area is selected in the selected video frame image, that is, a position of the center of the selected area in the selected video frame image is acquired to obtain the selected center position information. For example, the selected center position information of the selected region in the selected video frame image is determined to be (500, 980).

In step S304 of some embodiments, because the current center position information is the horizontal plane coordinate position information, the selected center position information is also the horizontal plane coordinate position information. And obtaining the current pixel displacement by calculating the displacement between the current center position information and the selected center position information. For example, based on the current center position information being (540, 960), the selected center position information is (500, 980). The current pixel displacement is obtained by calculating the difference between the two (-40, 20) and is defined as pixel-offset.

In this embodiment, the tracking algorithm is used to identify the current center position information and the selected center position information to determine the current pixel displacement. Specifically, the tracking algorithm adopts STC algorithm (Fast Tracking via Dense Spatio-Temporal Context Learning) to ensure the reliability of the calculated current pixel displacement.

In steps S301 to S304 illustrated in the present embodiment, the center positions of two video frame images in the same area are determined, so that the pixel displacement between two adjacent video frame images is determined according to the difference between the center positions, so that the current pixel displacement calculation is easy and accurate.

Referring to fig. 4, in some embodiments, step S103 may include, but is not limited to, steps S401 to S403:

step S401, obtaining current shooting position information of a current video frame image, and obtaining current position information;

step S402, obtaining current shooting position information of a selected video frame image to obtain selected position information;

step S403, performing displacement calculation according to the current position information and the selected position information to obtain the current position displacement.

In steps S401 to S402 of some embodiments, the current shooting position information of the current video frame image is taken as the current position information, and the current position information is GPS position information, and is determinedMeaning Gps _k . By using the current position information of the selected video frame image as the selected position information, and the selected position information is also GPS position information, the method is defined as Gps ₀ . The longitude and latitude position Gps of the cradle head camera in shooting the current video frame image can be known through the current position information and the selected position information _k The longitude and latitude position Gps of the cradle head camera in shooting the selected video frame image can be known ₀ 。

In step S403 of some embodiments, since the current position information and the selected position information are both latitude and longitude positions, displacement calculation is performed on the current position information and the selected position information, that is, a distance between the two positions is calculated as the current position displacement, and the current position displacement is defined as physical_offset.

In steps S401 to S404 illustrated in this embodiment, by acquiring the latitude and longitude positions between two captured video frame images, the displacement of the current position is used as the displacement of the current position according to the distance between the two latitude and longitude positions, and the displacement of the current position can determine the movement condition of the pan-tilt camera on the horizontal plane, so as to analyze the accuracy of the calculation of the pitch angle.

In some embodiments, before step S104, the method for dynamically correcting the pitch angle of the pan-tilt camera may further include constructing a preset pixel position ratio. The preset pixel position ratio represents the relation between the pixel displacement and the position displacement, the current pitching angle is determined according to pitch angle readings on the pan-tilt camera, and then the ratio is calculated according to a plurality of pitch angle readings and the position displacement, so that the position displacement theoretically corresponding to the current pitching angle can be measured.

Referring to fig. 5, in some embodiments, constructing the preset pixel position ratio may include, but is not limited to, steps S501 to S505:

step S501, acquiring at least two historical video frame images shot by a cradle head camera and historical shooting position information of each historical video frame image;

step S502, performing pixel displacement measurement on historical video frame images of two adjacent frames to obtain at least two historical pixel displacements;

step S503, performing position displacement measurement on the historical shooting position information of the historical video frame images of two adjacent frames to obtain at least two historical position displacements;

step S504, calculating the ratio of the historical pixel displacement to the corresponding historical position displacement to obtain a candidate pixel position ratio;

step S505, performing average value calculation on the position ratio of at least two candidate pixels to obtain a preset pixel position ratio.

In step S501 of some embodiments, the historical photographing information is photographing information of each historical video frame image photographed by the pan-tilt camera, and the historical photographing information includes: historical shooting position information, historical shooting height and historical pitching angle. It should be noted that, the historical pitching angle is pitch angle reading corresponding to each historical video frame image, and the unmanned aerial vehicle can record shooting information of the pan-tilt camera in real time in the flight process, so as to find corresponding shooting information as historical shooting information according to the time point of each historical video frame image.

In step S502 of some embodiments, historical video frame images of two adjacent frames are combined to form a historical image pair, and pixel position measurement is performed on the historical image pair to obtain a historical pixel displacement. Because there are at least two historical image pairs, there are also at least two historical pixel displacements. It should be noted that, the operation process of performing the pixel displacement measurement on the historical image pair is the same as the pixel displacement measurement of the video frame image pair, and will not be described herein. In particular, at least two historical pixel displacements are characterized as pixel_offset1, pixel_offset2, pixel_offset3.

In step S503 of some embodiments, the historical positional displacement is obtained by performing distance calculation on the historical photographing positional information of the historical image pair, and there are at least two historical image pairs, so there are also at least two historical positional displacements. It should be noted that, the calculation manner of the historical position displacement is the same as the current position displacement, which is not described herein, and at least two historical position displacements are defined as physical_offse1, physical_offse2, and.

In steps S504 to S505 of some embodiments, a candidate pixel position ratio is obtained by performing ratio calculation on each historical pixel position and the corresponding historical position displacement, and there are a plurality of candidate pixel position ratios. And calculating the average value of the plurality of candidate pixel ratios to obtain a preset pixel position ratio, so that the calculation of the preset pixel position ratio is accurate. It should be noted that, the calculation of the preset pixel position ratio is shown in formula (1):

Rate＝(pixel_offset1/physical_offse1+pixel_offset2/physical_offse2+……+pixel_offsetn/physical_offsen)/n (1)

in steps S501 to S504 illustrated in the present embodiment, by recording the history shooting position information of each history video frame image, the history position displacement is obtained by calculating the history pixel displacement of any two-frame history video frame images and calculating the shooting position displacement between any two-frame history video frame images. And calculating the ratio of the historical pixel displacement to the corresponding historical position displacement to obtain a plurality of candidate pixel position ratios, and then calculating the average value of the plurality of candidate pixel position ratios as a preset pixel position ratio to ensure that the preset pixel position ratio is accurately calculated, and the position displacement corresponding to each pixel displacement can be predicted according to the preset pixel position ratio.

In step S104 of some embodiments, before the preset pixel position ratio is determined, the current pixel displacement and the preset pixel position ratio are divided to obtain a theoretical position displacement, that is, a predicted position displacement. For example, if the current pixel displacement is pixel_offset ', the predicted position displacement is pre_physical_offset=pixel_offset'/Rate.

In step S105 of some embodiments, a position displacement difference is obtained by performing a difference calculation on the theoretical predicted position displacement and the actual current position displacement, so as to indirectly determine whether the current pitch angle is correct according to the position displacement difference. It should be noted that, if the position displacement difference is not 0, it indicates that there is a deviation in the current pitch angle, and if the position displacement difference is equal to 0, it indicates that the current pitch angle is accurate. For example, the current position displacement is physical_offset, the predicted position displacement is pre_physical_offset, and the position displacement difference value d=pre_physical_offset-physical_offset is calculated.

Referring to fig. 6, in some embodiments, step S106 may include, but is not limited to, steps S601 to S603:

step S601, carrying out pitch angle prediction on the position displacement difference, the current pitch angle and the current shooting height through a preset pitch angle prediction model to obtain candidate pitch angles;

Step S602, performing deviation measurement according to a candidate pitch angle, a current pitch angle, a preset average deviation and a preset smoothing coefficient to obtain a pitch angle deviation value;

and step S603, summing according to the current pitching angle and the pitch angle deviation value to obtain a corrected pitching angle.

In step S601 of some embodiments, if the position displacement difference is not equal to 0, it is determined that there is a deviation in the current pitch angle, and a candidate pitch angle needs to be constructed according to the position displacement difference, the current pitch angle, and the current shooting height. Note that, referring to fig. 7, fig. 7 depicts a relationship between a real pitch angle and a current pitch angle. In fig. 7, H is the current shooting height, camera is the position of the pan-tilt Camera, and H is the height of the pan-tilt Camera perpendicular to the ground. The intersection point of the optical axis of the tripod head camera and the ground is A, and the current pitching angle has deviation, so that A is only the intersection point of the optical axis of the tripod head camera and the ground in theory, and actually the intersection point of the optical axis of the tripod head camera and the ground is B point, so that the position displacement difference is d. As can be seen from fig. 3, the current pitch angle is α, and the candidate pitch angle is β. Therefore, the construction of the candidate pitch angle by the current pitch angle, the current shooting height, and the position displacement difference is as shown in formula (2):

β＝arctan((H*tan(α)+d)/H) (2)

In step S602 of some embodiments, the preset average deviation is calculated according to the historical pitch angle readings and the actual pitch angle, and is used to characterize the deviation degree between the actual pitch angle and the directly detected pitch angle. The preset smoothing coefficient is a conventional coefficient. It should be noted that, according to the data distribution situation between the real pitch angle and the pitch angle reading, the pitch angle deviation value can be calculated as shown in formula (3):

pitch_offset_avg＝λ*pitch_offset_avg+(1-λ)*(β-α) (3)

wherein λ is a preset sliding coefficient, λ=0.7, β is a candidate pitch angle, α is a current pitch angle, and pitch_offset_avg is a preset average deviation.

In step S603 of some embodiments, after the pitch angle deviation value is calculated, a corrected pitch angle is obtained by performing sum calculation according to the pitch angle deviation value and the current pitch angle, so that the corrected pitch angle can accurately represent the real pitch angle of the pan-tilt camera.

In steps S601 to S603 illustrated in this embodiment, a pitch angle estimation model with a geometric relationship is constructed according to a displacement difference, a pitch angle and a shooting height, then a position displacement difference, a current pitch angle and a current shooting height are input to a pitch angle estimation model to obtain candidate pitch angles, the candidate pitch angles are not final pitch angles, pitch angle deviation values are constructed according to the candidate pitch angles, the current pitch angles, preset average deviation and preset smoothing coefficients, then the pitch angle deviation values are added to the current pitch angles to obtain corrected pitch angles, so that pitch angle calculation is accurate, no additional equipment is required, no initialization adjustment in a static state is required, and pitch angle correction of a pan-tilt camera in a motion process is simple.

In some embodiments, before step S602, the method for dynamically correcting a pitch angle of a pan-tilt camera further includes: and constructing a preset average deviation. It should be noted that, because the preset average deviation needs to be constructed according to multiple pitch angle readings and the pitch angle obtained by the real test, the accuracy of pitch angle calculation is increased.

Referring to fig. 8, in some embodiments, constructing the pitch angle deviation value may include, but is not limited to, including steps S801 to S803:

step S801, acquiring historical pitching angles of at least two historical video frame images;

step S802, performing difference calculation on the historical pitching angles of each historical video frame image and a preset verification pitching angle to obtain pitching angle differences;

step S803, carrying out average value calculation on at least two pitching angle differences to obtain preset average deviation.

In steps S801 to S802 of some embodiments, since the pitch angle fluctuation approaches a gaussian distribution during the flight of the unmanned aerial vehicle, as shown in fig. 9, fig. 9 shows a graph of the relationship between the predicted historical pitch angle and the true measured verification pitch angle at different times. Wherein different grayscales represent historical pitch angles at different moments, i.e. pitch angle readings at different moments, and the abscissa represents 50 measurement samples. The pitch angle on the ordinate is the deviation degree of the verified pitch angle, so that the deviation rule of the pitch angle reading can be determined through fig. 8, that is, a certain deviation is predicted. Therefore, the difference between the historical pitching angle and the verification pitching angle of each historical video frame image is calculated to obtain a pitching angle difference, and the pitching angle difference is shown in fig. 9, so that the distribution situation of the pitching angle difference can be seen.

In step S803 of some embodiments, a plurality of pitch angle differences are averaged, and the average is taken as a preset average deviation, so that the preset average deviation is accurately calculated. Therefore, the pitch angle deviation value is calculated according to the preset average deviation, and further the pitch angle is calculated more accurately.

In steps S801 to S803 illustrated in this embodiment, a difference value is calculated by collecting a plurality of historical pitch angles and corresponding verification pitch angles to obtain a pitch angle difference, and then a mean value is calculated on a plurality of pitch angles to obtain a preset mean deviation, so that the preset mean deviation is accurately calculated.

Referring to fig. 10, the embodiment of the application further provides a dynamic correction device for a pitch angle of a pan-tilt camera, which can implement the dynamic correction method for the pitch angle of the pan-tilt camera, and is applied to the pan-tilt camera, and the device comprises:

an obtaining module 1001, configured to obtain image capturing information of a current video frame image pair and a video frame image pair captured by a pan-tilt camera; wherein, the image photographing information includes: current pitching angle, current shooting height and current shooting position information;

a pixel displacement measurement module 1002, configured to perform pixel displacement measurement according to the video frame image pair, to obtain a current pixel displacement;

A physical displacement measurement 1003, configured to perform a physical displacement measurement on current shooting position information of the video frame image pair, to obtain a current position displacement;

the position prediction module 1004 is configured to perform position displacement prediction according to a current pixel displacement and a preset pixel position ratio, so as to obtain a predicted position displacement; wherein the pixel position ratio characterizes a ratio between the pixel displacement and the position displacement;

the displacement difference calculation module 1005 is configured to perform difference calculation according to the current position displacement and the predicted position displacement, so as to obtain a position displacement difference;

the angle prediction module 1006 is configured to predict a pitch angle according to the position displacement difference, the current pitch angle, and the current shooting height, so as to obtain a corrected pitch angle;

and the correction module 1007 is configured to correct the pitch angle of the pan-tilt camera according to the corrected pitch angle. The specific implementation of the dynamic correcting device for the pitch angle of the pan-tilt camera is basically the same as the specific embodiment of the dynamic correcting method for the pitch angle of the pan-tilt camera, and is not repeated here.

The embodiment of the application also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the dynamic correction method of the pitch angle of the pan-tilt camera when executing the computer program. The electronic equipment can be any intelligent terminal including a tablet personal computer, a vehicle-mounted computer and the like.

Referring to fig. 11, fig. 11 illustrates a hardware structure of an electronic device according to another embodiment, the electronic device includes:

the processor 1101 may be implemented by a general purpose CPU (central processing unit), a microprocessor, an application specific integrated circuit (ApplicationSpecificIntegratedCircuit, ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solution provided by the embodiments of the present application;

the memory 1102 may be implemented in the form of read-only memory (ReadOnlyMemory, ROM), static storage, dynamic storage, or random access memory (RandomAccessMemory, RAM). The memory 1102 may store an operating system and other application programs, and when the technical solution provided in the embodiments of the present disclosure is implemented by software or firmware, relevant program codes are stored in the memory 1102, and the processor 1101 invokes a dynamic correction method for a pitch angle of a pan-tilt camera to execute the embodiments of the present disclosure;

an input/output interface 1103 for implementing information input and output;

the communication interface 1104 is configured to implement communication interaction between the device and other devices, and may implement communication in a wired manner (e.g. USB, network cable, etc.), or may implement communication in a wireless manner (e.g. mobile network, WIFI, bluetooth, etc.);

Bus 1105 transmits information between the various components of the device (e.g., processor 1101, memory 1102, input/output interface 1103, and communication interface 1104);

wherein the processor 1101, memory 1102, input/output interface 1103 and communication interface 1104 enable communication connection therebetween within the device via bus 1105.

The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the dynamic correction method of the pitch angle of the pan-tilt camera when being executed by a processor.

The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The embodiment of the application provides a dynamic correction method, a device, equipment and a storage medium for a pitching angle of a tripod head camera, which are characterized in that the position information, the height and the theoretical pitching angle of each frame of video frame image shot by the tripod head camera are acquired, then the pixel displacement between two adjacent video frame images is analyzed, the position displacement of the two video frame images is calculated, the predicted position displacement is calculated according to the pixel displacement and the preset pixel position ratio, the predicted position displacement is compared with the actual position displacement to determine a displacement difference, and then the actual pitching angle is calculated according to the displacement difference, the height and the theoretical pitching angle to be used for correcting the pitching angle of the tripod head camera according to the corrected pitching angle, so that the pitching angle correction of the tripod head camera in the moving process can be completed, no external equipment is required to be added, the cost is saved, and the tripod head camera can have higher control precision in the moving process.

The embodiments described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

It will be appreciated by persons skilled in the art that the embodiments of the application are not limited by the illustrations, and that more or fewer steps than those shown may be included, or certain steps may be combined, or different steps may be included.

The above described apparatus embodiments are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those of ordinary skill in the art will appreciate that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.

The terms "first," "second," "third," "fourth," and the like in the description of the application and in the above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the above-described division of units is merely a logical function division, and there may be another division manner in actual implementation, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including multiple instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing a program.

The preferred embodiments of the present application have been described above with reference to the accompanying drawings, and are not thereby limiting the scope of the claims of the embodiments of the present application. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the embodiments of the present application shall fall within the scope of the claims of the embodiments of the present application.

Claims

1. The method for dynamically correcting the pitch angle of the pan-tilt camera is characterized by being applied to the pan-tilt camera, and comprises the following steps:

2. The method of claim 1, wherein the pair of video frame images comprises: a current video frame image and a selected video frame image, the selected video frame image being a previous video frame image of the current video frame image; the measuring pixel displacement according to the video frame image pair to obtain the current pixel displacement comprises the following steps:

3. The method according to claim 2, wherein said physically displacing the current shooting position information of the video frame image pair to obtain a current position displacement includes:

4. The method of claim 2, wherein prior to said estimating the position displacement based on the ratio of the current pixel displacement to a predetermined pixel position, the method further comprises:

5. The method of claim 4, wherein said predicting a pitch angle based on said position displacement difference, said current pitch angle, and said current shooting height, results in a corrected pitch angle, comprising:

6. The method of claim 5, wherein prior to said deriving a pitch angle deviation value from said candidate pitch angle, said current pitch angle, a preset average deviation, and a preset smoothing factor, said method further comprises: the construction of the preset average deviation specifically comprises the following steps:

7. The method according to any one of claims 2 to 6, comprising, after the acquiring the image capturing information of the pan-tilt camera capturing the current video frame image pair and the video frame image pair:

8. A dynamic correction device for a pitch angle of a pan-tilt camera, the device comprising:

9. An electronic device comprising a memory and a processor, the memory storing a computer program, the processor implementing the dynamic correction method of the pitch angle of the pan-tilt camera according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the dynamic correction method of the pitch angle of the pan-tilt camera according to any one of claims 1 to 7.