WO2021217372A1 - 可移动平台的控制方法和设备 - Google Patents
可移动平台的控制方法和设备 Download PDFInfo
- Publication number
- WO2021217372A1 WO2021217372A1 PCT/CN2020/087326 CN2020087326W WO2021217372A1 WO 2021217372 A1 WO2021217372 A1 WO 2021217372A1 CN 2020087326 W CN2020087326 W CN 2020087326W WO 2021217372 A1 WO2021217372 A1 WO 2021217372A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- target object
- distance
- distance correction
- movable platform
- correction deviation
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000003068 static effect Effects 0.000 claims abstract description 76
- 238000012937 correction Methods 0.000 claims description 197
- 238000004590 computer program Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 14
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 3
- 125000004122 cyclic group Chemical group 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000002304 esc Anatomy 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/12—Target-seeking control
Definitions
- the embodiments of the present application relate to the technical field of movable platforms, and in particular, to a method and equipment for controlling a movable platform.
- the drone can fly with reference to the target. For example, the drone can move according to a fixed trajectory relative to the target. Common scenarios are: automatic surround shooting, automatic inspection, autonomous monitoring, etc. Scenes.
- the drone needs to observe the position of the target and fly around the target according to the position of the target.
- the position of the observation target can be determined in the following way: the drone runs a multi-view geometric algorithm through the multi-frame images collected by the camera to determine the position of the target.
- this method is suitable when the target is at rest. If the target is in a moving state and the above method is still used, the determined position of the target is not accurate.
- the embodiments of the present application provide a method and device for controlling a movable platform, which are used to accurately determine the position of a target object.
- an embodiment of the present application provides a method for controlling a movable platform, the movable platform includes a camera, and the method includes:
- a multi-view geometric algorithm is run based on the multi-frame images collected by the shooting device to determine the first position of the target object, and the movable platform is controlled according to the first position.
- the target object performs work tasks
- the second position of the target object is determined according to the size of the image area of the target object in the image captured by the shooting device, and the movable platform is controlled according to the second position Perform the work task on the target object.
- an embodiment of the present application provides a control device for a movable platform, the movable platform includes a photographing device, and the control device for the movable platform includes a memory and at least one processor;
- the memory is used to store program code
- the at least one processor is configured to execute the program code for:
- a multi-view geometric algorithm is run based on the multi-frame images collected by the shooting device to determine the first position of the target object, and the movable platform is controlled according to the first position.
- the target object performs work tasks
- the second position of the target object is determined according to the size of the image area of the target object in the image captured by the shooting device, and the movable platform is controlled according to the second position Perform the work task on the target object.
- an embodiment of the present application provides a movable platform that includes a camera and a control device of the movable platform described in the embodiment of the present application in the first aspect.
- an embodiment of the present application provides a computer-readable storage medium with a computer program stored on the computer-readable storage medium; when the computer program is executed, the implementation of the The control method of the movable platform.
- an embodiment of the present application provides a program product, the program product includes a computer program, the computer program is stored in a readable storage medium, and at least one processor can read the A computer program, and the at least one processor executes the computer program to implement the control method of the movable platform according to the embodiment of the present application in the first aspect.
- the mobile platform control method and device run a multi-view geometric algorithm according to the multi-frame images collected by the camera when the target object is in a static state to determine the first position of the target object
- the second position of the target object is determined according to the size of the image area of the target object in the image collected by the shooting device, and the position of the target object is determined in different ways according to the different states of the target object, which improves Determine the accuracy of the location of the target object.
- the performance of the work tasks is better.
- Fig. 1 is a schematic architecture diagram of an unmanned aerial system according to an embodiment of the present application
- Figure 2 is a schematic diagram of an application scenario provided by an embodiment of the application
- FIG. 3 is a flowchart of a method for controlling a movable platform according to an embodiment of the application
- FIG. 4 is a schematic diagram of running a multi-view geometric algorithm based on multiple frames of images collected by a photographing device to determine the position of a stationary target object according to an embodiment of the application;
- FIG. 5 is a schematic diagram of running a multi-view geometric algorithm to determine the position of a target object in a moving state according to a multi-frame image collected by a photographing device according to an embodiment of the application;
- FIG. 6 is a schematic diagram of determining that the target object is in a static state according to an embodiment of the application
- FIG. 7 is a schematic diagram of determining that a target object is in a motion state according to an embodiment of the application.
- FIG. 8 is a schematic structural diagram of a control device for a movable platform provided by an embodiment of the application.
- FIG. 9 is a schematic structural diagram of a movable platform provided by an embodiment of this application.
- FIG. 10 is a schematic structural diagram of a movable platform provided by another embodiment of this application.
- a component when referred to as being "fixed to” another component, it can be directly on the other component or a centered component may also exist. When a component is considered to be “connected” to another component, it can be directly connected to the other component or there may be a centered component at the same time.
- the embodiments of the present application provide a method and equipment for controlling a movable platform.
- the movable platform can be unmanned aerial vehicles, unmanned vehicles, unmanned ships, robots, handheld PTZ, etc.
- the following description of the mobile platform of this application uses drones as an example. It will be obvious to those skilled in the art that other types of drones can be used without restriction, and the embodiments of the present application can be applied to various types of drones.
- the drone can be a small or large drone.
- the drone may be a rotorcraft, for example, a multi-rotor drone propelled by multiple propulsion devices through the air, and the embodiments of the present application are not limited thereto.
- Fig. 1 is a schematic architecture diagram of an unmanned aerial system according to an embodiment of the present application.
- a rotary wing drone is taken as an example for description.
- the unmanned aerial system 100 may include a drone 110, a display device 130, and a control terminal 140.
- the UAV 110 may include a power system 150, a flight control system 160, a frame, and a pan/tilt 120 carried on the frame.
- the drone 110 can wirelessly communicate with the control terminal 140 and the display device 130.
- the drone 110 further includes a battery (not shown in the figure), and the battery provides electrical energy for the power system 150.
- the UAV 110 may be an agricultural UAV or an industrial application UAV, and there is a need for cyclic operation.
- the battery also has the need for cyclic operation.
- the frame may include a fuselage and a tripod (also called a landing gear).
- the fuselage may include a center frame and one or more arms connected to the center frame, and the one or more arms extend radially from the center frame.
- the tripod is connected with the fuselage and used for supporting the UAV 110 when it is landed.
- the power system 150 may include one or more electronic governors (referred to as ESCs) 151, one or more propellers 153, and one or more motors 152 corresponding to the one or more propellers 153, wherein the motors 152 are connected to Between the electronic governor 151 and the propeller 153, the motor 152 and the propeller 153 are arranged on the arm of the UAV 110; the electronic governor 151 is used to receive the driving signal generated by the flight control system 160 and provide driving according to the driving signal Current is supplied to the motor 152 to control the speed of the motor 152.
- the motor 152 is used to drive the propeller to rotate, thereby providing power for the flight of the drone 110, and the power enables the drone 110 to realize one or more degrees of freedom of movement.
- the drone 110 may rotate about one or more rotation axes.
- the aforementioned rotation axis may include a roll axis (Roll), a yaw axis (Yaw), and a pitch axis (pitch).
- the motor 152 may be a DC motor or an AC motor.
- the motor 152 may be a brushless motor or a brushed motor.
- the flight control system 160 may include a flight controller 161 and a sensing system 162.
- the sensing system 162 is used to measure the attitude information of the UAV, that is, the position information and state information of the UAV 110 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration, and three-dimensional angular velocity.
- the sensing system 162 may include, for example, at least one of sensors such as a gyroscope, an ultrasonic sensor, an electronic compass, an inertial measurement unit (IMU), a vision sensor, a global navigation satellite system, and a barometer.
- the global navigation satellite system may be the Global Positioning System (GPS).
- the flight controller 161 is used to control the flight of the drone 110, for example, it can control the flight of the drone 110 according to the attitude information measured by the sensor system 162. It should be understood that the flight controller 161 can control the drone 110 according to pre-programmed program instructions, and can also control the drone 110 by responding to one or more remote control signals from the control terminal 140.
- the pan/tilt head 120 may include a motor 122.
- the pan/tilt is used to carry a load, and the load may be, for example, the camera 123.
- the flight controller 161 can control the movement of the pan/tilt 120 through the motor 122.
- the pan/tilt head 120 may further include a controller for controlling the movement of the pan/tilt head 120 by controlling the motor 122.
- the pan-tilt 120 may be independent of the drone 110 or a part of the drone 110.
- the motor 122 may be a DC motor or an AC motor.
- the motor 122 may be a brushless motor or a brushed motor.
- the pan/tilt may be located on the top of the drone or on the bottom of the drone.
- the photographing device 123 may be, for example, a device for capturing images, such as a camera or a video camera, and the photographing device 123 may communicate with the flight controller and take pictures under the control of the flight controller.
- the imaging device 123 of this embodiment at least includes a photosensitive element, and the photosensitive element is, for example, a Complementary Metal Oxide Semiconductor (CMOS) sensor or a Charge-coupled Device (CCD) sensor. It can be understood that the camera 123 can also be directly fixed to the drone 110, so the pan/tilt 120 can be omitted.
- CMOS Complementary Metal Oxide Semiconductor
- CCD Charge-coupled Device
- the display device 130 is located on the ground end of the unmanned aerial vehicle 100, can communicate with the drone 110 in a wireless manner, and can be used to display the attitude information of the drone 110.
- the image photographed by the photographing device 123 may also be displayed on the display device 130. It should be understood that the display device 130 may be an independent device or integrated in the control terminal 140.
- the control terminal 140 is located on the ground end of the unmanned aerial vehicle 100, and can communicate with the drone 110 in a wireless manner for remote control of the drone 110.
- Fig. 2 is a schematic diagram of an application scenario provided by an embodiment of the application.
- Fig. 2 shows a drone 201 and a control terminal 202 of the drone.
- the control terminal 202 of the drone 201 may be one or more of a remote control, a smart phone, a desktop computer, a laptop computer, and a wearable device (watch, bracelet).
- the control terminal 202 is the remote controller 2021 and the terminal device 2022 as an example for schematic description.
- the terminal device 2022 is, for example, a smart phone, a wearable device, a tablet computer, etc., but the embodiment of the present application is not limited thereto.
- the drone 201 includes a fuselage 2011 and a gimbal 2012 connected to the fuselage 2011, and the gimbal 2012 is used to carry a load 2013.
- the load 2013 includes a camera
- the drone transmits the image captured by the camera to the control terminal 202
- the control terminal 202 displays the image captured by the camera.
- the drone 201 needs to determine the position of the target object. Therefore, the present application proposes that the method of observing the position of the target object when the target object is in a stationary state is different from the method of observing the position of the target object when the target object is in a moving state.
- the position of the target object is determined according to the multi-view geometric algorithm; when the target object is in a moving state, the position of the target object is determined according to the image area size of the target object in the collected image. In order to accurately determine the location of the target when the target is in different states.
- FIG. 3 is a flowchart of a method for controlling a movable platform according to an embodiment of the application.
- the method of this embodiment can be applied to a control device of a movable platform.
- the control device of the movable platform may be arranged on the movable platform; or, a part of the control device of the movable platform is arranged on the movable platform, and the other part is arranged on the control terminal of the movable platform.
- the control device of the movable platform is set on the movable platform as an example.
- the method of this embodiment may include:
- S301 Determine whether the target object is in a static state.
- the movable platform includes a photographing device, which is used to photograph the target object so that the target object is located in the image collected by the photographing device.
- the movable platform can determine whether the target object photographed by the camera is in a static state. If it is determined that the target object is in a static state, S302 is executed. If it is determined that the target object is not in a static state, for example, the target object is in a moving state, S303 is executed.
- S302 When the target object is in a static state, run a multi-view geometric algorithm according to the multi-frame images collected by the shooting device to determine the first position of the target object, and control the movable platform to perform work tasks on the target object according to the first position.
- the photographing device may photograph an image including the target object.
- the movable platform runs the multi-view geometric algorithm according to the multi-frame images collected by the camera to determine the position of the target object, which is called the first position.
- the first position can be used to guide the movable platform to perform corresponding work tasks.
- the movable platform is controlled to perform work tasks on the target object.
- one way to implement multi-view geometry to determine the position of the target object based on the multi-frame images collected by the camera is: when the target object is in a static state, according to the principle of epipolar geometry, as shown in A in Figure 4, One frame of image collected by the camera is taken as the first frame of image.
- the shooting device can determine the position of the target object through the observation of a single frame of image. On a ray connecting the shooting device and the target object, the direction of the target object can be determined, but the first frame image cannot determine the distance between the shooting device and the target object. the distance.
- the next frame image of the first frame image collected by the camera is called the second frame image.
- a ray that characterizes the direction of the target object can be determined.
- the position of the target object is obtained, that is, the position of the target object determined according to the first frame and the second frame image .
- the camera continues to capture the third frame of image, and according to the position of the target object determined above, the target object is projected into the first frame image, the second frame image, and the third frame image to obtain 3 projection positions respectively.
- the projection position (for example, the sum of the reprojection errors corresponding to the three projection positions is the smallest) optimizes the position of the above-mentioned target object, and obtains the optimized position of the target object.
- the camera continues to capture the fourth frame of image, and according to the above optimized target object position, the target object is respectively projected into the first frame image, the second frame image, the third frame image, and the fourth frame image to obtain 4 projection positions respectively.
- Optimize the position of the target object according to the four projection positions for example, the sum of the reprojection errors corresponding to the four projection positions is the smallest
- the position of the target object is optimized, so that the accurate position of the target object when it is in a static state can be obtained through multiple frames of images.
- the above-mentioned method may also be referred to as a triangulation ranging method.
- S303 When the target object is in a moving state, determine the second position of the target object according to the size of the image area of the target object in the image captured by the shooting device, and control the movable platform to perform work tasks on the target object according to the second position.
- the multi-view geometric algorithm When the target object is in a moving state, if the multi-view geometric algorithm is run to determine the position of the target object based on the multi-frame images collected by the camera, as shown in Figure 5, two rays are determined according to the two frames of images collected by the camera. The intersection of the two rays is not located at the position of the target object, so the position of the target object in the moving state cannot be obtained by the above-mentioned method when the target object is in a static state.
- the photographing device may also photograph an image including the target object.
- the target object is located in the image collected by the photographing device and occupies a part of the image area in the image.
- the movable platform determines the position of the target object according to the size of the image area of the target object in the image collected by the shooting device, and this position is called the second position.
- the second position can be used to guide the movable platform to perform corresponding work tasks. Then, according to the second position, the movable platform is controlled to perform work tasks on the target object.
- the single frame image of the shooting device can determine the ray that characterizes the direction of the target object;
- the size of the image area in the image collected by the camera and the priori information of the target object (such as the prior size, for example, the priori size of the target object is a car and the target object is different), to determine the distance between the target object and the camera , According to the distance and the direction of the target object, the position of the target object can be determined.
- the first position of the target object is determined by running the multi-view geometric algorithm according to the multi-frame images collected by the shooting device when the target object is in a static state.
- the size of the image area in the obtained image determines the second position of the target object, and the position of the target object is determined in different ways for different states of the target object, which improves the accuracy of the determined position of the target object.
- the performance of the work tasks is better.
- the work task is at least one of a circle flight task and a tracking task. If the work task surrounds the flight task, the movable platform is controlled to fly around the target object according to the first position or the second position.
- the trajectory of the movable platform flying around the target object may be an involute line, or a spiral line, or an arc line with a constant radius (or a circular line). This embodiment can accurately surround the target object and can also quickly follow the target object.
- the first position of the target object when the target object is in a static state, after the first position of the target object is obtained, the first position of the target object is no longer updated. That is, the multi-view geometric algorithm is no longer run based on the multiple frames of images collected by the camera to update the first position of the target object. Even if the camera continues to capture images, the first position of the target object is no longer optimized based on the image. It not only saves processing resources, avoids the influence of the noise of a certain frame of image on the shooting effect, but also ensures that when the target object is in a static state, the movable platform is controlled according to the same fixed first position to perform work tasks on the target object to improve the work task Implementation effect. For example, the center of the movable platform around the target object is fixed to ensure the smoothness of the surrounding path.
- the second position of the target object is updated. In order to improve the accuracy of the position determined when the target object is in motion, the performance of the work task is improved.
- a possible implementation of S302 above is: running a multi-view geometric algorithm based on the multi-frame images collected by the shooting device to determine the first distance between the target object and the shooting device, and according to the first distance Determine the first position of the target object.
- the distance between the target object and the shooting device can be determined by running the multi-view geometric algorithm according to the multiple frames of images collected by the shooting device, and this distance is called the first distance.
- the first distance is determined as the first position of the target object, where the position can be represented by the magnitude of the distance.
- the shooting device captures the target object
- the target object occupies an image area in the image captured by the shooting device
- the direction of the target object can be determined according to the image area, and the direction of the target object and the first distance are determined. The first position.
- the distance between the target object and the shooting device can also be determined according to the size of the image area of the target object in the image captured by the shooting device, which is called the second distance.
- the specific implementation process can be referred to the above The description of, I won’t repeat it here.
- the image size of the target object in the image collected by the shooting device is smaller, it indicates that the distance between the target object and the shooting device is farther, on the contrary, if the target object is in the image collected by the shooting device The larger the image size in, the closer the distance between the target object and the camera.
- the a priori size of the target object needs to be used in determining the distance between the target object and the shooting device, there may be a deviation between the a priori size and the actual size of the target object. In comparison, it is closer to the actual distance between the target object and the camera. Therefore, there is a distance deviation between the first distance and the second distance obtained in different ways.
- the movable platform of this embodiment can also obtain a distance correction deviation, which is used to characterize the distance deviation between the first distance and the second distance.
- a possible implementation of S303 above is: determining the third distance between the target object and the shooting device according to the size of the image area of the target object in the image captured by the shooting device, and correcting the deviation according to the third distance and the distance Determine the second position of the target object.
- the distance between the target object and the photographing device is determined according to the size of the image area of the target object in the image collected by the photographing device, and this distance is called the third distance.
- the a priori size of the target object needs to be referred to in the process of determining the third distance, so there is an error between the third distance and the distance between the target object and the photographing device.
- the multi-view geometric algorithm is run based on the multi-frame images collected by the shooting device to determine that the distance between the target object and the shooting device is closer to the distance between the target object and the shooting device, the deviation is corrected according to the above-mentioned distance, and the third distance
- the corrected distance is closer to the distance between the target object and the photographing device, and then the second position of the target object is determined according to the corrected distance. Since the deviation of the third distance is corrected in this embodiment, the correction accuracy is higher, so that the obtained second position is closer to the actual position of the target object, which improves the accuracy of determining the position of the target object when the target object is in motion. sex.
- the corrected distance is determined as the second position of the target object, where the position can be represented by the magnitude of the distance.
- the shooting device captures the target object
- the target object occupies an image area in the image captured by the shooting device, and the direction of the target object can be determined based on the image area, and the direction of the target object and the corrected distance are determined.
- the second position because the shooting device captures the target object, the target object occupies an image area in the image captured by the shooting device, and the direction of the target object can be determined based on the image area, and the direction of the target object and the corrected distance are determined. The second position.
- the second distance between the target object and the shooting device is determined according to the size of the image area of the target object in the image collected by the shooting device; according to the first distance and the second distance The distance determines the distance to correct the deviation.
- the multi-view geometric algorithm is run based on the multi-frame images collected by the camera to determine the first distance between the target object and the camera, which is closer to the target object when the target object is in a stationary state The distance from the camera.
- the distance deviation from the distance between the target object and the shooting device determined according to the size of the image area in this embodiment, when the target object is in a static state, The second distance between the target object and the shooting device is determined according to the size of the image area of the target object in the image collected by the shooting device. Then, according to the distance between the target object and the shooting device obtained in different ways under the same state of the target object, that is, the above-mentioned distance correction deviation is determined according to the first distance and the second distance.
- the first distance is subtracted from the second distance to obtain the above-mentioned distance correction deviation.
- the above-mentioned corrected distance is, for example, equal to the sum of the third distance and the distance correction deviation.
- the first distance is compared to the second distance to obtain the above-mentioned distance correction deviation.
- the above-mentioned corrected distance is, for example, equal to the product of the third distance and the distance corrected deviation.
- the first distance is subtracted from the second distance and then compared to the second distance to obtain the aforementioned distance correction deviation.
- the above-mentioned corrected distance is, for example, equal to the sum of the third distance and (the product of the third distance and the distance corrected deviation).
- the target object when the target object is in a static state, multiple frames of images collected by the shooting device are acquired, and the second distance between the target object and the shooting device is determined according to the size of the image area of the target object in each frame of the image collected by the shooting device, according to The first distance and the multiple second distances determine the above-mentioned distance correction deviation. For example: determine the average value of a plurality of second positions, and then determine the above-mentioned distance correction deviation according to the first distance and the average value. Alternatively, the distance correction deviation is determined according to the first distance and each second distance, and then the mean value of the multiple distance correction deviations is determined to be the final distance correction deviation.
- the above-mentioned distance correction deviation is calculated, and the distance correction deviation more accurately reflects the deviation between the target object and the photographing device obtained in different ways under the same state of the target object. Then, when the target object is in a motion state, the deviation is corrected according to the distance to obtain a more accurate position of the target object.
- one or more reference distance correction deviations are pre-stored in the local storage device of the movable platform.
- the movable platform of this embodiment may obtain at least one reference distance correction deviation from the local storage device of the movable platform, and then determine the aforementioned distance correction deviation according to the obtained at least one reference distance correction deviation. If a reference distance correction deviation is obtained from the local storage device of the movable platform, the reference distance correction deviation is determined as the aforementioned distance correction deviation. If multiple reference distance correction deviations are obtained from the local storage device of the movable platform, the mean value, or the maximum value, or the minimum value, or the median of the multiple reference distance correction deviations is determined as the above-mentioned distance Correct the deviation.
- the distance correction deviation of this embodiment is pre-stored in the local storage device of the movable platform, the distance correction deviation can be obtained from the local storage device, and there is no need to calculate and obtain it when the target object is in a static state, which saves processing resources.
- the local storage device of the movable platform pre-stores reference distance correction deviations corresponding to multiple object types, and the multiple object types include cars, people, and so on.
- Each object type corresponds to a reference distance correction deviation.
- each object type corresponds to multiple reference distance correction deviations.
- the reference distance correction deviation corresponding to different object types may be different.
- the movable platform determines the object type of the target object according to the image collected by the camera, and obtains the reference distance matching the object type of the target object from the local storage device of the movable platform according to the object type of the target object. Correct the deviation, and determine the reference distance correction deviation as the above-mentioned distance correction deviation.
- the object type of the target object is a car as an example.
- the reference distance correction deviation corresponding to the car is obtained from multiple reference distance correction deviations corresponding to cars, people, etc. pre-stored in the local storage device of the movable platform. . If the reference distance correction deviation corresponding to the car obtained from the local storage device of the movable platform is one, the reference distance correction deviation is determined as the aforementioned distance correction deviation. If there are multiple reference distance correction deviations corresponding to the car obtained from the local storage device of the movable platform, then the average or maximum or minimum value or median of the multiple reference distance correction deviations is determined as the aforementioned distance correction deviation .
- multiple reference distance correction deviations corresponding to multiple reference distances are pre-stored in the local storage device of the movable platform.
- Each reference distance corresponds to a reference distance correction deviation.
- each reference distance corresponds to multiple reference distance correction deviations.
- the reference distance correction deviations corresponding to different reference distances may be different.
- the movable platform determines the first distance between the target object and the photographing device through the foregoing solutions when the target object is in a static state. Then, the reference distance correction deviation corresponding to the at least one reference distance is acquired from the local storage device of the movable platform according to the first distance, and the distance correction deviation is determined according to the reference distance correction deviation. If the multiple reference distances include the first distance, one or more pre-stored reference distance correction deviations corresponding to the first distance are obtained from the local storage device of the movable platform. If the first distance is not included in the multiple reference distances, in this embodiment, the pre-stored one or more reference distance correction deviations corresponding to the reference distance closest to the first distance are obtained from the local storage device of the movable platform. Or, if the first distance is not included in the multiple reference distances, in this embodiment, one or more pre-stored two reference distances adjacent to the first distance are obtained from the local storage device of the movable platform. Correct the deviation with reference to the distance.
- the reference distance correction deviation obtained from the local storage device of the movable platform according to the first distance is one, the reference distance correction deviation is determined as the aforementioned distance correction deviation. If there are multiple reference distance correction deviations corresponding to the car obtained from the local storage device of the movable platform according to the first distance, then the average or maximum value or minimum value or median of the multiple reference distance correction deviations is determined as the above The distance correction deviation.
- the movable platform determines the second distance between the target object and the shooting device according to the size of the image area of the target object in the image captured by the shooting device .
- the reference distance correction deviation corresponding to at least one reference distance is acquired from the local storage device of the movable platform according to the second distance, and the distance correction deviation is determined according to the reference distance correction deviation.
- the specific implementation process please refer to the above-mentioned related description about the first distance to determine the distance correction deviation, which will not be repeated here.
- the movable platform determines the third distance between the target object and the shooting device in the above-mentioned manner when the target object is in a motion state. Then, the reference distance correction deviation corresponding to the at least one reference distance is acquired from the local storage device of the movable platform according to the third distance, and the distance correction deviation is determined according to the reference distance correction deviation.
- the reference distance correction deviation corresponding to the at least one reference distance is acquired from the local storage device of the movable platform according to the third distance, and the distance correction deviation is determined according to the reference distance correction deviation.
- a possible implementation manner for determining whether the target object is in a static state is to obtain multiple frames of images containing the target object collected by the photographing device, and determine whether the target object is in a static state according to the multiple frames of images.
- the photographing device may collect multiple frames of images, and the multiple frames of images include the target object. Then, according to the multi-frame image, it is determined whether the target object is in a static state. Therefore, it can quickly and stably detect whether the target object is in a static state or a moving state, and at the same time, it can avoid the influence of the noise of a certain frame of image on the result.
- a multi-view geometric algorithm is run based on multiple frames of images to determine the fourth position of the target object at multiple moments; it is determined whether the multiple fourth positions meet the preset position convergence conditions; when they are satisfied, Determine that the target object is in a static state, otherwise, determine that the target object is in a moving state.
- each frame of image may correspond to a moment
- a possible implementation of running a multi-view geometric algorithm based on multiple frames of images to determine the fourth position of the target object at multiple moments is: the photographing device sequentially collects multiple frames of images, After the second frame of image is captured by the imaging device at the second time, the fourth position of the target object at the second time can be determined according to the first frame of image and the second frame of image. After the imaging device at the third time captures the third frame of image, the fourth position of the target object at the third time can be determined according to the first frame of image, the second frame of image, and the third frame of image.
- the fourth position of the target object at the fourth time can be determined according to the first frame of image, the second frame of image, the third frame of image, and the fourth frame of image.
- the fourth position of the target object at multiple moments can be obtained.
- the preset position convergence condition for example, whether the multiple fourth positions converge to the same position, for example, it is determined that the multiple fourth positions are located within a preset range area, and the preset range area can indicate The covariance of the fourth position convergence.
- the multiple fourth positions meet the preset position convergence condition, it indicates that the position of the target object has basically not changed at these moments, and it is determined that the target object is in a static state.
- the multiple fourth positions do not satisfy the preset position convergence condition, it indicates that the position of the target object is changing, and it is determined that the target object is in a moving state.
- each frame of image may correspond to a moment
- another possible implementation manner for determining the fourth position of the target object at multiple moments is: in this implementation manner, a cyclic triangulation ranging scheme is proposed, that is, multiple instantiations at one time.
- a triangulation rangefinder is instantiated based on two adjacent frames of images.
- a multi-view geometric algorithm For example, run a multi-view geometric algorithm to obtain a position based on at least two frames of images, and obtain multiple positions in the same way.
- the following takes two frames of images to obtain a position as an example, but this embodiment is not limited to two frames of images.
- the photographing device sequentially collects multiple frames of images, taking 8 frames of images photographed by the photographing device at 8 moments as an example.
- a fourth position of the target object can be determined based on the first frame image and the second frame image (for example, see Figure 4 Shown in B).
- a fourth position of the target object can be determined according to the third frame image and the fourth frame image.
- the fifth frame of image is collected at the 5th moment and the 6th frame of image is collected at the 6th moment
- a fourth position of the target object can be determined according to the 5th frame of image and the 6th frame of image.
- the 7th frame image is collected at the 7th moment and the 8th frame image is collected at the 8th moment
- a fourth position of the target object can be determined based on the 7th frame image and the 8th frame image.
- the position of the target object is basically unchanged. As shown in FIG. 6, it can be determined that the four fourth positions of the target object converge within the same preset range area, which indicates that the target object is in a static state.
- the position of the target object changes. As shown in FIG. 7, it can be determined that the fourth position of the target object cannot converge within the same preset range area at the above 4 moments, which indicates that the target object is in a motion state.
- each frame of the image may include the target object, and the target object occupies an image area in each frame of the image.
- the target object may be included in each frame of the image.
- the size of the image area determines whether the target object is stationary. For example, if the target object is moving, the image size of the target object in the image is different. Therefore, it can be judged whether the difference between the size of the image area of the target object in each frame of the image is smaller than the preset difference, if so, the target object is determined to be in a static state, otherwise the target object is determined to be in a moving state.
- the movable platform determines whether the target object is switched from a stationary state to a moving state during the process of controlling the movable platform to perform work tasks on the target object according to the first position described above.
- the target object when the target object is in a static state, after the first position is determined, the target object can be moved to perform work tasks on the target object according to the first position control. Because the first position is used for the movable platform to perform work tasks when the target object is stationary. If the target object is in a motion state, the first position can not be used to control the movable platform to perform the work task, otherwise the movable platform cannot perform the above-mentioned work task on the target object. Therefore, in the process of controlling the movable platform to perform work tasks on the target object according to the first position, it is determined whether the target object is switched from a stationary state to a moving state.
- the target object has not switched from the stationary state to the moving state, it means that the target object is still in the stationary state, and then the movable platform is continuously controlled to perform work tasks on the target object according to the first position. If it is determined that the target object is switched from a static state to a moving state, it means that the target object is in a moving state, the second position of the target object is determined according to the size of the image area of the target object in the image captured by the photographing device, and the second position is determined according to the second position Control the movable platform to perform work tasks on the target object.
- one possible implementation of determining whether the target object is switched from a static state to a moving state is: acquiring a multi-frame image collected by a photographing device; and identifying the position of the target object in the multi-frame image; The object is projected into multiple frames of images to obtain the projection position of the target object in the image; according to the recognized position and projection position, it is determined whether the target object is switched from a static state to a moving state.
- the process of controlling the movable platform to perform work tasks on the target object according to the first position multiple frames of images collected by the shooting device are acquired, and then the position of the target object in each frame of the image is recognized (where, how To identify the position of the target object in the image, please refer to the description in the neural network model or image tracking and other related technologies, which will not be repeated here), and project the target object into each frame of image according to the above-mentioned first position to obtain the target object Is the projection position in each frame of the image.
- determine the deviation between the recognized position and the projection position which can be called re-projection error.
- the re-projection errors ie multiple re-projection errors
- multiple reprojection errors are preset error convergence conditions, that is, whether these multiple reprojection errors converge to the same error. For example, it is determined that the multiple reprojection errors are within a preset error range. If multiple reprojection errors meet the preset error convergence conditions, it means that the position of the target object has basically not changed, and it is determined that the target object is still in a static state. When multiple reprojection errors do not meet the preset error convergence condition, it means that the position of the target object is beginning to change, it is determined that the target object starts to move, and the target object switches from a static state to a moving state.
- the target object starts to move. Or, for example, some of the multiple re-projection errors are small, and some of the re-projection errors are large, then it is determined that the target object starts to move. Thus, it is determined that the target object is switched from a stationary state to a moving state.
- the target object can be detected in real time and quickly that the target object is switched from a stationary state to a moving state. Then it can seamlessly switch to: determine the second position of the target object according to the size of the image area of the target object in the image captured by the shooting device, and control the movable platform to perform the work task on the target object according to the second position . Avoid sudden changes in the determined position when switching from a stationary state to a moving state. Therefore, the path is smooth when the movable platform performs work tasks on the target object, and there is no sudden jitter.
- the photographing device in the foregoing embodiments may be a monocular camera.
- An embodiment of the present application also provides a computer storage medium, the computer storage medium stores program instructions, and the program execution may include some or all of the steps of the movable platform control method in any of the above embodiments. .
- FIG. 8 is a schematic structural diagram of a control device for a movable platform provided by an embodiment of the application.
- the movable platform includes a photographing device.
- the control device 800 for the movable platform includes: at least one processor 801.
- a processor 801 is shown as an example in the figure.
- the at least one processor 801 is configured to:
- a multi-view geometric algorithm is run based on the multi-frame images collected by the shooting device to determine the first position of the target object, and the movable platform is controlled according to the first position.
- the target object performs work tasks
- the second position of the target object is determined according to the size of the image area of the target object in the image captured by the shooting device, and the movable platform is controlled according to the second position Perform the work task on the target object.
- the first position of the target object is no longer updated.
- the at least one processor 801 runs a multi-view geometric algorithm according to the multi-frame images collected by the camera to determine the first position of the target object, it is specifically configured to:
- the at least one processor 801 is further configured to: obtain a distance correction deviation, where the distance correction deviation is used to characterize the distance deviation between the first distance and the second distance, and the second distance is the distance between the first distance and the second distance.
- the distance between the target object and the photographing device is determined according to the size of the image area of the target object in the image collected by the photographing device when the object is in a static state.
- the at least one processor 801 is specifically configured to: when determining the second position of the target object according to the size of the image area of the target object in the image captured by the photographing device:
- the third distance between the target object and the shooting device is determined according to the size of the image area of the target object in the image captured by the shooting device, and the third distance between the target object and the shooting device is determined according to the third distance and the distance correction deviation.
- the second position of the target object is determined according to the size of the image area of the target object in the image captured by the shooting device, and the third distance between the target object and the shooting device is determined according to the third distance and the distance correction deviation.
- the at least one processor 801 is specifically configured to: when the target object is in a static state, determine the target object according to the size of the image area of the target object in the image captured by the photographing device The second distance from the camera; the distance correction deviation is determined according to the first distance and the second distance.
- one or more reference distance correction deviations are pre-stored in the local storage device of the movable platform, and the at least one processor 801 is specifically configured to:
- the local storage device of the movable platform pre-stores reference distance correction deviations corresponding to multiple object types
- the at least one processor 801 is further configured to: determine according to the image collected by the photographing device The object type of the target object.
- the at least one processor 801 when acquiring at least one reference distance correction deviation from the local storage device of the movable platform, and determining the distance correction deviation according to the at least one reference distance correction deviation, is specifically configured to:
- a plurality of reference distance correction deviations corresponding to the reference distance are pre-stored in the local storage device of the movable platform.
- the at least one processor 801 is specifically configured to: obtain a reference distance correction deviation corresponding to at least one reference distance from a local storage device of the movable platform according to the first distance, and determine the reference distance correction deviation according to the reference distance correction deviation The distance correction deviation.
- the distance correction deviation corresponding to a plurality of pre-stored reference distances is acquired from a local storage device of the movable platform.
- the at least one processor 801 is further configured to: when the target object is in a static state, determine the difference between the target object and the shooting device according to the size of the image area of the target object in the image captured by the shooting device The second distance between.
- the at least one processor 801 when acquiring at least one reference distance correction deviation from the local storage device of the movable platform, and determining the distance correction deviation according to the at least one reference distance correction deviation, is specifically configured to:
- the distance correction deviation corresponding to a plurality of pre-stored reference distances is acquired from a local storage device of the movable platform.
- the at least one processor 801 is specifically configured to: obtain a reference distance correction deviation corresponding to at least one reference distance from a local storage device of the movable platform according to the third distance, and determine the reference distance correction deviation according to the reference distance correction deviation The distance correction deviation.
- the at least one processor 801 is specifically configured to: acquire a multi-frame image including a target object collected by the photographing device; and determine whether the target is in a static state according to the multi-frame image.
- the at least one processor 801 is specifically configured to: run a multi-view geometric algorithm according to the multi-frame images to determine the fourth position of the target object at multiple moments; and determine whether the multiple fourth positions satisfy The preset position convergence condition; when it is met, it is determined that the target object is in a static state, otherwise, it is determined that the target object is in a moving state.
- the at least one processor 801 is further configured to:
- the at least one processor 801 is specifically configured to: acquire a multi-frame image collected by a photographing device; identify the position of a target object in the multi-frame image; Projecting into the multi-frame image to obtain the projection position of the target object in the image; determining whether the target object is switched from a static state to a moving state according to the recognized position and the projection position.
- the work task is at least one of a circle flight task and a tracking task.
- control device 800 of the movable platform of this embodiment may further include a memory 802.
- the memory 802 is used to store program codes.
- the at least one processor 801 calls the program code, and when the program code is executed, it is used to implement the foregoing methods.
- control device of the movable platform of this embodiment can be used to implement the technical solutions of the foregoing method embodiments of the present application, and the implementation principles and technical effects are similar, and will not be repeated here.
- FIG. 9 is a schematic structural diagram of a movable platform provided by an embodiment of the application.
- the movable platform 900 includes a camera 901 and at least one processor 902.
- the figure shows one processor 902 as an example.
- the at least one processor 902 is configured to: determine whether the target object is in a static state; when the target object is in a static state, run a multi-view geometric algorithm according to the multi-frame images collected by the photographing device 901 to determine the The first position of the target object, and according to the first position, the movable platform 900 is controlled to perform work tasks on the target object; when the target object is in a moving state, the shooting device 901 captures the target object according to the target object. The size of the image area in the obtained image determines the second position of the target object, and the movable platform 900 is controlled to perform the work task on the target object according to the second position.
- the first position of the target object is no longer updated.
- the at least one processor 902 runs a multi-view geometric algorithm according to the multi-frame images collected by the photographing device 901 to determine the first position of the target object, it is specifically configured to:
- the at least one processor 902 is further configured to: obtain a distance correction deviation, where the distance correction deviation is used to characterize the distance deviation between the first distance and the second distance, and the second distance is the distance between the first distance and the second distance.
- the distance between the target object and the photographing device 901 is determined according to the size of the image area of the target object in the image collected by the photographing device 901 when the object is in a static state.
- the at least one processor 902 is specifically configured to determine the second position of the target object according to the size of the image area of the target object in the image captured by the photographing device 901:
- the at least one processor 902 is specifically configured to: when the target object is in a static state, determine the target according to the size of the image area of the target object in the image captured by the photographing device 901 The second distance between the object and the camera 901; the distance correction deviation is determined according to the first distance and the second distance.
- the mobile platform 900 in this embodiment locally further includes a storage device 903.
- One or more reference distance correction deviations are pre-stored in the local storage device 903 of the movable platform 900, and the at least one processor 902 is specifically configured to:
- At least one reference distance correction deviation is acquired from the local storage device 903 of the movable platform 900, and the distance correction deviation is determined according to the at least one reference distance correction deviation.
- the local storage device 903 of the movable platform 900 pre-stores reference distance correction deviations corresponding to multiple object types, and the at least one processor 902 is further configured to: The image determines the object type of the target object.
- the at least one processor 902 is specifically configured to obtain at least one reference distance correction deviation from the local storage device 903 of the movable platform 900, and to determine the distance correction deviation according to the at least one reference distance correction deviation :
- the reference distance correction deviation matching the object type of the target object is obtained from the local storage device 903 of the movable platform 900 according to the object type of the target object, and the reference distance correction deviation is determined as the distance correction deviation.
- the local storage device 903 of the movable platform 900 pre-stores reference distance correction deviations corresponding to multiple reference distances.
- the at least one processor 902 is specifically configured to: obtain a reference distance correction deviation corresponding to at least one reference distance from the local storage device 903 of the movable platform 900 according to the first distance, and correct the deviation according to the reference distance Determine the distance correction deviation.
- the local storage device 903 of the movable platform 900 obtains the distance correction deviation corresponding to multiple pre-stored reference distances, and the at least one processor 902 is further configured to:
- the second distance between the target object and the photographing device 901 is determined according to the size of the image area of the target object in the image collected by the photographing device 901.
- the at least one processor 902 is specifically configured to obtain at least one reference distance correction deviation from the local storage device 903 of the movable platform 900, and to determine the distance correction deviation according to the at least one reference distance correction deviation :
- the reference distance correction deviation corresponding to at least one reference distance is acquired from the local storage device 903 of the movable platform 900 according to the second distance, and the distance correction deviation is determined according to the reference distance correction deviation.
- the local storage device 903 of the movable platform 900 obtains the distance correction deviation corresponding to multiple pre-stored reference distances.
- the at least one processor 902 is specifically configured to: obtain a reference distance correction deviation corresponding to at least one reference distance from the local storage device 903 of the movable platform 900 according to the third distance, and correct the deviation according to the reference distance Determine the distance correction deviation.
- the at least one processor 902 is specifically configured to: acquire a multi-frame image including a target object collected by the photographing device 901; and determine whether the target is in a static state according to the multi-frame image.
- the at least one processor 902 is specifically configured to: run a multi-view geometric algorithm according to the multi-frame images to determine the fourth position of the target object at multiple moments; and determine whether the multiple fourth positions satisfy The preset position convergence condition; when it is met, it is determined that the target object is in a static state, otherwise, it is determined that the target object is in a moving state.
- the at least one processor 902 is further configured to:
- the at least one processor 902 is specifically configured to: acquire a multi-frame image captured by the photographing device 901; identify the position of the target object in the multi-frame image; The object is projected into the multi-frame image to obtain the projection position of the target object in the image; according to the identified position and the projection position, it is determined whether the target object is switched from a static state to a motion state.
- the work task is at least one of a circle flight task and a tracking task.
- the movable platform 900 of this embodiment may further include a memory (not shown in the figure).
- the memory is used to store program codes.
- the at least one processor 902 calls the program code, and when the program code is executed, it is used to implement the foregoing methods.
- the memory and the aforementioned storage device 903 may be the same component or different components.
- the movable platform of this embodiment can be used to implement the technical solutions of the foregoing method embodiments of the present application, and its implementation principles and technical effects are similar, and will not be repeated here.
- FIG. 10 is a schematic structural diagram of a movable platform provided by another embodiment of the application.
- the movable platform 1000 includes a camera 1001 and a control device 1002 of the movable platform.
- control device 1002 of the movable platform may adopt the structure of the device embodiment shown in FIG. 8, and correspondingly, it may execute the technical solution provided by any of the foregoing method embodiments, which will not be repeated here.
- the movable platform 1000 further includes a storage device 1003.
- the storage device 1003 is configured to pre-store one or more reference distance correction deviations.
- a person of ordinary skill in the art can understand that all or part of the steps in the above method embodiments can be implemented by a program instructing relevant hardware.
- the foregoing program can be stored in a computer readable storage medium. When the program is executed, it is executed. Including the steps of the foregoing method embodiment; and the foregoing storage medium includes: read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks, etc., which can store program codes Medium.
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Studio Devices (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
一种可移动平台的控制方法和设备,该方法包括:确定目标对象是否为静止状态(S301);在目标对象为静止状态时,根据拍摄装置采集到的多帧图像运行多视图几何算法以确定目标对象的第一位置,并根据第一位置控制可移动平台对目标对象执行工作任务(S302);在目标对象为运动状态时,根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象的第二位置,并根据第二位置控制可移动平台对目标对象执行工作任务(S303),提高了确定出的不同状态下目标对象的位置的准确性。而且根据上述位置控制可移动平台对目标对象执行工作任务,工作任务的执行效果更佳。
Description
本申请实施例涉及可移动平台技术领域,尤其涉及一种可移动平台的控制方法和设备。
无人机选中目标物后,无人机可以参考该目标物进行飞行,比如无人机可以按照相对于目标物的固定轨迹运动,常见的场景为:自动环绕拍摄、自动巡检、自主监控等场景。
以主动环绕拍摄为例,无人机需要观测目标物的位置,并根据目标物的位置环绕目标物飞行。观测目标物的位置可以通过如下方式确定:无人机通过拍摄装置采集到的多帧图像运行多视图几何算法确定目标物的位置。但是,这种方式适用于目标物处于静止状态。如果目标物处于运动状态,仍采用上述方式,则确定出的目标物的位置不准确。
发明内容
本申请实施例提供一种可移动平台的控制方法和设备,用于准确确定出目标对象的位置。
第一方面,本申请实施例提供一种可移动平台的控制方法,所述可移动平台包括拍摄装置,所述方法包括:
确定目标对象是否为静止状态;
在所述目标对象为静止状态时,根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置,并根据所述第一位置控制可移动平台对所述目标对象执行工作任务;
在所述目标对象为运动状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置,并根据所述第二位置控制可移动平台对所述目标对象执行所述工作任务。
第二方面,本申请实施例提供一种可移动平台的控制装置,所述可移动 平台包括拍摄装置,所述可移动平台的控制装置包括:存储器和至少一个处理器;
所述存储器,用于存储程序代码;
所述至少一个处理器,用于执行所述程序代码,以用于:
确定目标对象是否为静止状态;
在所述目标对象为静止状态时,根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置,并根据所述第一位置控制可移动平台对所述目标对象执行工作任务;
在所述目标对象为运动状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置,并根据所述第二位置控制可移动平台对所述目标对象执行所述工作任务。
第三方面,本申请实施例提供一种可移动平台,所述可移动平台包括拍摄装置和如第一方面本申请实施例所述的可移动平台的控制装置。
第四方面,本申请实施例提供一种计算机可读存储介质,所述计算机可读存储介质上存储有计算机程序;所述计算机程序在被执行时,实现如第一方面本申请实施例所述的可移动平台的控制方法。
第六方面,本申请实施例提供一种程序产品,所述程序产品包括计算机程序,所述计算机程序存储在可读存储介质中,至少一个处理器可以从所述可读存储介质读取所述计算机程序,所述至少一个处理器执行所述计算机程序以实施如第一方面本申请实施例所述的可移动平台的控制方法。
综上所述,本申请实施例提供的可移动平台的控制方法和设备,通过在目标对象为静止状态时根据拍摄装置采集到的多帧图像运行多视图几何算法以确定目标对象的第一位置,在目标对象为运动状态时根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象的第二位置,通过针对目标对象的不同状态采用不同的方式确定目标对象的位置,提高了确定出的目标对象的位置的准确性。而且根据上述位置控制可移动平台对目标对象执行工作任务,工作任务的执行效果更佳。
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实 施例或现有技术描述中所需要使用的附图作一简单地介绍,显而易见地,下面描述中的附图是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1是根据本申请的实施例的无人飞行***的示意性架构图;
图2为本申请实施例提供的应用场景示意图;
图3为本申请一实施例提供的可移动平台的控制方法的流程图;
图4为本申请一实施例提供的根据拍摄装置采集到的多帧图像运行多视图几何算法以确定为静止状态的目标对象的位置的示意图;
图5为本申请一实施例提供的根据拍摄装置采集到的多帧图像运行多视图几何算法以确定为运动状态的目标对象的位置的示意图;
图6为本申请一实施例提供的确定目标对象为静止状态的示意图;
图7为本申请一实施例提供的确定目标对象为运动状态的示意图;
图8为本申请一实施例提供的可移动平台的控制装置的结构示意图;
图9为本申请一实施例提供的可移动平台的结构示意图;
图10为本申请另一实施例提供的可移动平台的结构示意图。
为使本申请实施例的目的、技术方案和优点更加清楚,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
需要说明的是,当组件被称为“固定于”另一个组件,它可以直接在另一个组件上或者也可以存在居中的组件。当一个组件被认为是“连接”另一个组件,它可以是直接连接到另一个组件或者可能同时存在居中组件。
除非另有定义,本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中在本申请的说明书中所使用的术语只是为了描述具体的实施例的目的,不是旨在于限制本申请。本文所使用的术语“及/或”包括一个或多个相关的所列项目的任意的和所有的组合。
本申请的实施例提供了可移动平台的控制方法和设备。其中,可移动平台可以是无人机、无人车、无人船、机器人、手持云台等。以下对本申请可移动平台的描述使用无人机作为示例。对于本领域技术人员将会显而易见的是,可以不受限制地使用其他类型的无人机,本申请的实施例可以应用于各种类型的无人机。例如,无人机可以是小型或大型的无人机。在某些实施例中,无人机可以是旋翼无人机(rotorcraft),例如,由多个推动装置通过空气推动的多旋翼无人机,本申请的实施例并不限于此。
图1是根据本申请的实施例的无人飞行***的示意性架构图。本实施例以旋翼无人机为例进行说明。
无人飞行***100可以包括无人机110、显示设备130和控制终端140。其中,无人机110可以包括动力***150、飞行控制***160、机架和承载在机架上的云台120。无人机110可以与控制终端140和显示设备130进行无线通信。其中,无人机110还包括电池(图中未示出),电池为动力***150提供电能。无人机110可以是农业无人机或行业应用无人机,有循环作业的需求。相应的,电池也有循环作业的需求。
机架可以包括机身和脚架(也称为起落架)。机身可以包括中心架以及与中心架连接的一个或多个机臂,一个或多个机臂呈辐射状从中心架延伸出。脚架与机身连接,用于在无人机110着陆时起支撑作用。
动力***150可以包括一个或多个电子调速器(简称为电调)151、一个或多个螺旋桨153以及与一个或多个螺旋桨153相对应的一个或多个电机152,其中电机152连接在电子调速器151与螺旋桨153之间,电机152和螺旋桨153设置在无人机110的机臂上;电子调速器151用于接收飞行控制***160产生的驱动信号,并根据驱动信号提供驱动电流给电机152,以控制电机152的转速。电机152用于驱动螺旋桨旋转,从而为无人机110的飞行提供动力,该动力使得无人机110能够实现一个或多个自由度的运动。在某些实施例中,无人机110可以围绕一个或多个旋转轴旋转。例如,上述旋转轴可以包括横滚轴(Roll)、偏航轴(Yaw)和俯仰轴(pitch)。应理解,电机152可以是直流电机,也可以交流电机。另外,电机152可以是无刷电机,也可以是有刷电机。
飞行控制***160可以包括飞行控制器161和传感***162。传感*** 162用于测量无人机的姿态信息,即无人机110在空间的位置信息和状态信息,例如,三维位置、三维角度、三维速度、三维加速度和三维角速度等。传感***162例如可以包括陀螺仪、超声传感器、电子罗盘、惯性测量单元(Inertial Measurement Unit,IMU)、视觉传感器、全球导航卫星***和气压计等传感器中的至少一种。例如,全球导航卫星***可以是全球定位***(Global Positioning System,GPS)。飞行控制器161用于控制无人机110的飞行,例如,可以根据传感***162测量的姿态信息控制无人机110的飞行。应理解,飞行控制器161可以按照预先编好的程序指令对无人机110进行控制,也可以通过响应来自控制终端140的一个或多个遥控信号对无人机110进行控制。
云台120可以包括电机122。云台用于携带负载,负载例如可以是拍摄装置123。飞行控制器161可以通过电机122控制云台120的运动。可选的,作为另一实施例,云台120还可以包括控制器,用于通过控制电机122来控制云台120的运动。应理解,云台120可以独立于无人机110,也可以为无人机110的一部分。应理解,电机122可以是直流电机,也可以是交流电机。另外,电机122可以是无刷电机,也可以是有刷电机。还应理解,云台可以位于无人机的顶部,也可以位于无人机的底部。
拍摄装置123例如可以是照相机或摄像机等用于捕获图像的设备,拍摄装置123可以与飞行控制器通信,并在飞行控制器的控制下进行拍摄。本实施例的拍摄装置123至少包括感光元件,该感光元件例如为互补金属氧化物半导体(Complementary Metal Oxide Semiconductor,CMOS)传感器或电荷耦合元件(Charge-coupled Device,CCD)传感器。可以理解,拍摄装置123也可直接固定于无人机110上,从而云台120可以省略。
显示设备130位于无人飞行***100的地面端,可以通过无线方式与无人机110进行通信,并且可以用于显示无人机110的姿态信息。另外,还可以在显示设备130上显示拍摄装置123拍摄的图像。应理解,显示设备130可以是独立的设备,也可以集成在控制终端140中。
控制终端140位于无人飞行***100的地面端,可以通过无线方式与无人机110进行通信,用于对无人机110进行远程操纵。
应理解,上述对于无人飞行***各组成部分的命名仅是出于标识的目的, 并不应理解为对本申请的实施例的限制。
图2为本申请实施例提供的应用场景示意图,如图2所示,图2中示出了无人机201、无人机的控制终端202。无人机201的控制终端202可以是遥控器、智能手机、台式电脑、膝上型电脑、穿戴式设备(手表、手环)中的一种或多种。本申请实施例以控制终端202为摇控器2021和终端设备2022为例来进行示意性说明。该终端设备2022例如是智能手机、可穿戴设备、平板电脑等,但本申请实施例并限于此。
无人机201的包括机身2011和连接在所述机身2011上的云台2012,云台2012用于承载负载2013。其中,负载2013包括有拍摄装置,无人机将拍摄装置拍摄到的图像传输给控制终端202,控制终端202显示拍摄装置拍摄到的图像。在无人机201跟踪目标对象的场景中或者无人机201环绕目标对象飞行的场景中,无人机201需要确定目标对象的位置。因此,本申请提出了在目标对象处于静止状态下观测目标对象的位置的方式与目标对象处于运动状态下观测目标对象的位置的方式不同。比如在目标对象为静止状态时,根据多视图几何算法确定目标对象的位置;在目标对象为运动状态时根据目标对象在采集到的图像中的图像区域尺寸确定目标对象的位置。以便在目标物处于不同状态下准确确定出目标物的位置。
图3为本申请一实施例提供的可移动平台的控制方法的流程图,本实施例的方法可以应用于可移动平台的控制装置中。该可移动平台的控制装置可以设置在可移动平台;或者,该可移动平台的控制装置的一部分设置在可移动平台上,另一部分设置在可移动平台的控制终端上。本实施例以可移动平台的控制装置设置在可移动平台为例,如图3所示,本实施例的方法可以包括:
S301、确定目标对象是否为静止状态。
本实施例中,可移动平台包括拍摄装置,该拍摄装置用于对目标对象进行拍摄,以便目标对象位于拍摄装置采集的图像中。可移动平台可以确定拍摄装置拍摄的目标对象是否为静止状态。若确定目标对象为静止状态,则执行S302。若确定目标对象不为静止状态,比如目标对象为运动状态,则执行S303。
S302、在目标对象为静止状态时,根据拍摄装置采集到的多帧图像运行 多视图几何算法以确定目标对象的第一位置,并根据第一位置控制可移动平台对目标对象执行工作任务。
本实施例中,当目标对象为静止状态时,拍摄装置可以拍摄包括目标对象的图像。可移动平台根据拍摄装置采集到的多帧图像运行多视图几何算法,确定目标对象的位置,该位置称为第一位置。该第一位置可以用于指导可移动平台执行相应的工作任务。然后根据该第一位置控制可移动平台对目标对象执行工作任务。
其中,根据拍摄装置采集到的多帧图像运行多视图几何确定目标对象的位置的一种实现方式为:当目标对象静止状态时,根据对极几何原理,如图4中的A所示,将拍摄装置采集到的一帧图像作为第1帧图像。拍摄装置通过单帧图像的观测能够确定目标对象的位置在拍摄装置与目标对象连线的一条射线上,即可确定目标对象的方向,但根据第1帧图像无法确定拍摄装置与目标对象之间的距离。而如图4中的B所示,拍摄装置采集到第1帧图像的下一帧图像,称为第2帧图像。根据第2帧图像又能够确定一个表征目标对象的方向的射线,根据确定的两次射线的交点,就获得了目标对象的位置,即根据第1帧和第2帧图像确定的目标对象的位置。拍摄装置继续拍摄到第3帧图像,根据上述确定的目标对象的位置将目标对象分别投影到第1帧图像、第2帧图像、第3帧图像中分别获得3个投影位置,根据这3个投影位置(比如这3个投影位置分别对应的重投影误差之和最小)对上述目标对象的位置进行优化,获得优化后的目标对象的位置。拍摄装置继续拍摄到第4帧图像,根据上述优化的目标对象的位置将目标对象分别投影到第1帧图像、第2帧图像、第3帧图像、第4帧图像中分别获得4个投影位置,根据这4个投影位置(比如这4个投影位置分别对应的重投影误差之和最小)对上述目标对象的位置进行优化,再次获得优化后的目标对象的位置。以此类推,对目标对象的位置进行优化,从而可以通过多帧图像获得目标对象为静止状态时准确的位置。可选的,上述的方式也可称为三角测距法。
S303、在目标对象为运动状态时,根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象的第二位置,并根据第二位置控制可移动平台对目标对象执行工作任务。
在目标对象为运动状态时,如果根据拍摄装置采集到的多帧图像运行多 视图几何算法确定目标对象的位置,如图5所示,根据拍摄装置采集的两帧图像确定两条射线,但是该两条射线的交点并不位于目标对象的位置上,因此采用目标对象为静止状态时的上述方式无法得到运动状态的目标对象的位置。
而本实施例中,当目标对象为运动状态时,拍摄装置也可以拍摄包括目标对象的图像,目标对象位于拍摄装置采集的图像中,并且占据该图像中的一部分图像区域。可移动平台根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象的位置,该位置称为第二位置。该第二位置可以用于指导可移动平台执行相应的工作任务。然后根据该第二位置控制可移动平台对目标对象执行工作任务。
其中,根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象的位置的一种实现方式为:通过拍摄装置的单帧图像可以确定表征目标对象的方向的射线;根据目标对象在拍摄装置采集到的图像中的图像区域尺寸以及目标对象的先验信息(比如先验尺寸,例如目标对象为车与目标对象为人的先验尺寸不同),确定目标对象与拍摄装置之间的距离,根据该距离与目标对象的方向,可以确定目标对象的位置。
本实施例中,通过在目标对象为静止状态时根据拍摄装置采集到的多帧图像运行多视图几何算法以确定目标对象的第一位置,在目标对象为运动状态时根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象的第二位置,通过针对目标对象的不同状态采用不同的方式确定目标对象的位置,提高了确定出的目标对象的位置的准确性。而且根据上述位置控制可移动平台对目标对象执行工作任务,工作任务的执行效果更佳。
在一些实施例中,工作任务为环绕飞行任务和跟踪任务中的至少一种。如果工作任务环绕飞行任务,则根据第一位置或第二位置控制可移动平台环绕目标对象飞行。其中,该可移动平台环绕目标对象飞行的轨迹可以是渐开线,或者,螺旋线,或者,半径不变的圆弧线(也可以是圆周线)。本实施例能够准确环绕目标对象,也能够快速跟随目标对象。
在一些实施例中,在目标对象为静止状态时,在获得目标对象的第一位置之后,不再更新目标对象的第一位置。也就是,不再根据拍摄装置采集到的多帧图像运行多视图几何算法更新目标对象的第一位置,即使拍摄装置继 续采集到图像,也不再根据该图像优化目标对象的第一位置。既能节省处理资源,避免某一帧图像的噪声对拍摄效果的影响,也能保证目标对象为静止状态时,根据同一固定的第一位置控制可移动平台对目标对象执行工作任务,改善工作任务执行效果。比如可移动平台环绕目标对象的圆心固定,保证环绕路径的平滑性。
在一些实施例中,在目标对象为运动状态时,在获得目标对象的第二位置之后,更新目标对象的第二位置。以提高目标对象运动时确定的位置的准确性,改善工作任务执行效果。
在一些实施例中,上述S302的一种可能的实现方式为:根据拍摄装置采集到的多帧图像运行多视图几何算法以确定目标对象与拍摄装置之间的第一距离,并根据第一距离确定目标对象的第一位置。
本实施例中,根据拍摄装置采集到的多帧图像运行多视图几何算法可以确定目标对象与拍摄装置之间的距离,该距离称为第一距离。
可选的,将第一距离确定为目标对象的第一位置,其中,位置可以通过距离大小来表示。或者,由于拍摄装置拍摄到目标对象,目标对象在拍摄装置拍摄到的图像中占据图像区域,根据该图像区域可以确定目标对象的方向,并根据该目标对象的方向和上述第一距离,确定上述的第一位置。
另外,在目标对象为静止状态时,也可以根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象与拍摄装置之间的距离,称为第二距离,具体实现过程可以参见上述的描述,此处不再赘述。其中,针对同一目标对象,如果目标对象在拍摄装置采集到的图像中的图像尺寸越小,则表明目标对象与拍摄装置之间的距离越远,反之,如果目标对象在拍摄装置采集到的图像中的图像尺寸越大,表明目标对象与拍摄装置之间的距离越近。而本实施例中,在确定目标对象与拍摄装置之间的距离由于需要采用目标对象的先验尺寸,先验尺寸与目标对象的实际尺寸之间可能存在偏差,上述第一距离与第二距离相比,更接近目标对象与拍摄装置之间的实际距离。所以采用不同方式获得的第一距离与第二距离之间存在距离偏差。
所以,本实施例的可移动平台还可以获取距离校正偏差,该距离校正偏差用于表征上述第一距离与上述第二距离之间存在的距离偏差。
相应的,上述S303的一种可能的实现方式为:根据目标对象在拍摄装置 采集到的图像中的图像区域尺寸确定目标对象与拍摄装置之间的第三距离,根据第三距离和距离校正偏差确定目标对象的第二位置。
本实施例中,当目标对象为运动状态时,根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象与拍摄装置之间的距离,该距离称为第三距离。但是如上所述,在确定第三距离的过程中需要参考目标对象的先验尺寸,所以第三距离与目标对象与拍摄装置之间的距离存在误差。又由于根据拍摄装置采集到的多帧图像运行多视图几何算法确定目标对象与拍摄装置之间的距离更接近目标对象与拍摄装置之间的距离,所以根据上述的距离校正偏差,将第三距离变换为在目标对象为运动状态时相当于根据拍摄装置采集到的多帧图像运行多视图几何算法确定的目标对象与拍摄装置之间的距离,该距离称为校正后的距离。该校正后的距离更接近目标对象与拍摄装置之间的距离,然后根据该校正后的距离确定目标对象的第二位置。由于本实施例中校正了第三距离的偏差,校正精度更高,从而获得的第二位置更加接近目标对象的实际位置,提高了在目标对象为运动状态时,确定的目标对象的位置的准确性。
可选的,将校正后的距离确定为目标对象的第二位置,其中,位置可以通过距离大小来表示。或者,由于拍摄装置拍摄到目标对象,目标对象在拍摄装置拍摄到的图像中占据图像区域,根据该图像区域可以确定目标对象的方向,并根据目标对象的方向和上述校正后的距离,确定上述的第二位置。
下面对获取距离校正偏差的几种可能的实现方式进行举例说明。
在一种可能的实现方式中,在目标对象为静止状态时,根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象与拍摄装置的第二距离;根据第一距离和第二距离确定距离校正偏差。在目标对象为静止状态时,根据拍摄装置采集到的多帧图像运行多视图几何算法以确定目标对象与拍摄装置之间的第一距离,该第一距离更接近目标对象为静止状态时目标对象与拍摄装置之间的距离。为了获得多视图几何算法确定的目标对象与拍摄装置的距离,相对于,根据图像区域尺寸确定的目标对象与拍摄装置之间的距离,的距离偏差,本实施例在目标对象为静止状态时,根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定目标对象与拍摄装置之间的第二距离。然后根据目标对象处于同一状态下依据不同方式得到的目标对象与拍摄装置 之间的距离,也就是根据第一距离与第二距离确定上述的距离校正偏差。
可选的,将第一距离减去第二距离获得上述的距离校正偏差。相应地,上述校正后的距离例如等于第三距离与距离校正偏差之和。
可选的,将第一距离比上第二距离获得上述的距离校正偏差。相应地,上述校正后的距离例如等于第三距离与距离校正偏差的乘积。
可选的,将第一距离减去第二距离再比上第二距离,获得上述的距离校正偏差。相应地,上述校正后的距离例如等于第三距离与(第三距离与距离校正偏差的乘积)之和。
需要说明的是,距离校正偏差的表现形式,并不限于此。
可选的,在目标对象为静止状态时,获取拍摄装置采集的多帧图像,根据目标对象在拍摄装置采集到的每帧图像中的图像区域尺寸确定目标对象与拍摄装置的第二距离,根据第一距离和多个第二距离,确定上述的距离校正偏差。例如:确定多个第二位置的均值,然后根据第一距离和该均值,确定上述的距离校正偏差。或者,根据第一距离和每个第二距离,确定距离校正偏差,然后确定多个距离校正偏差的均值为最终的距离校正偏差。
本实施例中,在目标对象为静止状态时,计算获得上述的距离校正偏差,该距离校正偏差更加准确地反映目标对象同一状态下不同方式获得的目标对象与拍摄装置之间的偏差。然后在目标对象为运动状态时,根据该距离校正偏差,获得更准确的目标对象的位置。
在另一种可能的实现方式中,可移动平台的本地存储装置中预先存储有一个或多个参考距离校正偏差。本实施例的可移动平台可以从可移动平台的本地存储装置中获取至少一个参考距离校正偏差,然后根据获取的至少一个参考距离校正偏差确定上述的距离校正偏差。如果从可移动平台的本地存储装置中获取到一个参考距离校正偏差,则将该参考距离校正偏差确定为上述的距离校正偏差。如果从可移动平台的本地存储装置中获取到多个参考距离校正偏差,则将该多个参考距离校正偏差的均值,或者,最大值,或者,最小值,或中位数确定为上述的距离校正偏差。
由于本实施例的距离校正偏差预先存储在可移动平台的本地存储装置中,可从本地存储装置中获取到距离校正偏差,无需目标对象为静止状态时再计算获取,节省处理资源。
下面对从可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据至少一个参考距离校正偏差确定距离校正偏差的几种实现方式进行举例说明。
在一种可能的实现方式中,可移动平台的本地存储装置中预先存储有多个对象类型对应的参考距离校正偏差,该多个对象类型包括:汽车、人等。每个对象类型对应一个参考距离校正偏差,可选的,也有可能每个对象类型对应多个参考距离校正偏差。不同对象类型对应的参考距离校正偏差可能不同。本实施例中,可移动平台根据拍摄装置采集到的图像确定目标对象的对象类型,根据所述目标对象的对象类型从可移动平台的本地存储装置中获取与目标对象的对象类型匹配的参考距离校正偏差,将该参考距离校正偏差确定为上述的距离校正偏差。此处以该目标对象的对象类型是汽车为例,本实施例中从可移动平台的本地存储装置中预先存储的汽车、人等对应的多个参考距离校正偏差中获取汽车对应的参考距离校正偏差。如果从可移动平台的本地存储装置获取的汽车对应的参考距离校正偏差为一个,则将该参考距离校正偏差确定为上述的距离校正偏差。如果从可移动平台的本地存储装置获取的汽车对应的参考距离校正偏差为多个,则将该多个参考距离校正偏差的均值或者最大值或最小值或中位数确定为上述的距离校正偏差。
在另一种可能的实现方式中,可移动平台的本地存储装置中预先存储有多个参考距离对应的参考距离校正偏差。每个参考距离对应一个参考距离校正偏差,可选的,也有可能每个参考距离对应多个参考距离校正偏差。不同参考距离对应的参考距离校正偏差可能不同。
可选的,可移动平台在目标对象为静止状态时通过上述各方案确定目标对象与拍摄装置之间的第一距离。然后根据第一距离从可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据参考距离校正偏差确定距离校正偏差。如多个参考距离中包括第一距离,则从可移动平台的本地存储装置中获取预先存储的与第一距离对应的一个或多个参考距离校正偏差。如果多个参考距离中未包括第一距离,则本实施例中从可移动平台的本地存储装置中获取预先存储的与第一距离最接近的参考距离对应的一个或多个参考距离校正偏差。或者,如果多个参考距离中未包括第一距离,则本实施例中从可移动平台的本地存储装置中获取预先存储的与第一距离相 邻的两个参考距离分别对应的一个或多个参考距离校正偏差。
如果根据第一距离从可移动平台的本地存储装置获取的参考距离校正偏差为一个,则将该参考距离校正偏差确定为上述的距离校正偏差。如果根据第一距离从可移动平台的本地存储装置获取的汽车对应的参考距离校正偏差为多个,则将该多个参考距离校正偏差的均值或者最大值或最小值或中位数确定为上述的距离校正偏差。
可选的,本实施例中,可移动平台在目标对象为静止状态时根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定目标对象与拍摄装置之间的第二距离。根据第二距离从可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据参考距离校正偏差确定距离校正偏差。具体实现过程可以参见上述有关第一距离确定距离校正偏差的相关描述,此处不再赘述。
可选的,本实施例中,可移动平台在目标对象为运动状态时通过上述方式确定目标对象与拍摄装置之间的第三距离。然后根据第三距离从可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据参考距离校正偏差确定距离校正偏差。具体实现过程可以参见上述有关第一距离确定距离校正偏差的相关描述,此处不再赘述。
在上述任一实施例中,确定目标对象是否为静止状态的一种可能的实现方式:获取拍摄装置采集到的包含目标对象的多帧图像,根据多帧图像确定目标对象是否为静止状态。本实施例中,拍摄装置可以采集到多帧图像,该多帧图像中包括目标对象。然后根据该多帧图确定目标对象是否为静止状态。从而快速、稳定地检测出目标对象为静止状态还是运动状态,同时能够避免某一帧图像的噪声对结果的影响。
下面对如何根据所述多帧图像确定所述目标是否为静止状态的实现方式进行举例说明。
在一种可能的实现方式中,根据多帧图像运行多视图几何算法以确定多个时刻的目标对象的第四位置;确定多个第四位置是否满足预设的位置收敛条件;当满足时,确定目标对象为静止状态,否则,确定目标对象为运动状态。
可选的,每帧图像可以对应一个时刻,根据多帧图像运行多视图几何算 法以确定多个时刻的目标对象的第四位置的一种可能的实现方式为:拍摄装置依次采集多帧图像,当第2时刻拍摄装置采集第2帧图像后,根据第1帧图像与第2帧图像可以确定第2时刻的目标对象的第四位置。当第3时刻拍摄装置采集第3帧图像后,根据第1帧图像、第2帧图像、第3帧图像可以确定第3时刻的目标对象的第四位置。当第4时刻拍摄装置采集第4帧图像后,根据第1帧图像、第2帧图像、第3帧图像、第4帧图像可以确定第4时刻的目标对象的第四位置。以此类推,可以获得多个时刻的目标对象的第四位置。具体如何获得上述第四位置可以参见图3所示实施例中有关S302的相关描述,此处不再赘述。然后确定多个第四位置是否满足预设的位置收敛条件,比如这多个第四位置是否收敛于同一位置,例如判断这多个第四位置位于预设范围区域内,预设范围区域可以表示第四位置收敛的协方差。当多个第四位置满足预设的位置收敛条件时,说明目标对象的位置在这些时刻基本没有变化,确定目标对象为静止状态。当多个第四位置不满足预设的位置收敛条件时,说明目标对象的位置在发生变化,确定目标对象为运动状态。
可选的,每帧图像可以对应一个时刻,确定多个时刻的目标对象的第四位置的另一种可能的实现方式为:本实现方式中提出循环三角测距方案,即一次性实例化多个三角测距器,随着时间更新,实时删除最旧的三角测距器,同时补充最新的三角测距器。具体地,根据相邻两帧图像实例化一个三角测距器。
比如根据至少两帧图像运行多视图几何算法获得一个位置,按照相同的方式可以获得多个位置。下面以两帧图像获得一个位置为例,但本实施例并不限于两帧图像。
拍摄装置依次采集多帧图像,以拍摄装置在8个时刻拍摄到的8帧图像为例。拍摄装置在第1时刻采集第1帧图像以及在第2时刻采集第2帧图像后,根据第1帧图像与第2帧图像可以确定目标对象的一个第四位置(比如可参见图4中的B所示)。在第3时刻采集第3帧图像以及在第4时刻采集第4帧图像后,根据第3帧图像与第4帧图像可以确定目标对象的一个第四位置。在第5时刻采集第5帧图像以及在第6时刻采集第6帧图像后,根据第5帧图像与第6帧图像可以确定目标对象的一个第四位置。在第7时刻采集第7帧图像以及在第8时刻采集第8帧图像后,根据第7帧图像与第8帧 图像可以确定目标对象的一个第四位置。
当目标对象为静止状态时,目标对象的位置基本不变。如图6所示,可以确定目标对象的上述4个第四位置收敛于同一预设范围区域内,表示目标对象为静止状态。
当目标对象为运动状态,目标对象的位置发生变化。如图7所示,可以确定上述4个时刻的目标对象的第四位置无法收敛于同一预设范围区域内,表示目标对象为运动状态。
在另一种可能的实现方式中,获取到多帧图像之后,可以每帧图像中包括目标对象,而且目标对象在每帧图像占据有图像区域,本实施例可以根据目标对象在每帧图像中的图像区域尺寸确定目标对象是否为静止状态。比如如果目标对象在运动,则该目标对象在图像中的图像尺寸不同。所以可以判断目标对象在每帧图像中的图像区域尺寸间的差异是否小于预设差异,若是,则确定目标对象为静止状态,否则确定目标对象为运动状态。
在上述任一实施例中,可移动平台在根据上述第一位置控制可移动平台对目标对象执行工作任务的过程中,确定目标对象是否从静止状态切换为运动状态。
本实施例中,在目标对象为静止状态时,确定第一位置之后,根据第一位置控制可移动对目标对象执行工作任务。由于第一位置用于目标对象为静止状态下可移动平台执行工作任务。如果目标对象为运动状态,则不能使用第一位置控制可移动平台执行工作任务,否则可移动平台无法实现对目标对象执行上述工作任务。因此,在根据第一位置控制可移动平台对目标对象执行工作任务的过程中,确定目标对象是否从静止状态切换为运动状态。如果确定目标对象未从静止状态切换为运动状态,说明目标对象仍为静止状态,则继续根据第一位置控制可移动平台对目标对象执行工作任务。如果确定目标对象从静止状态切换为运动状态,说明目标对象为运动状态,则根据目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定目标对象的第二位置,并根据第二位置控制可移动平台对目标对象执行工作任务。
其中,确定目标对象是否从静止状态切换为运动状态的一种可能的实现方式为:获取拍摄装置采集得到的多帧图像;以及识别目标对象在多帧图像中的位置;根据第一位置将目标对象投影到多帧图像中以获取目标对象在图 像中的投影位置;根据识别的位置和投影位置确定目标对象是否从静止状态切换为运动状态。
本实施例中,在根据第一位置控制可移动平台对目标对象执行工作任务的过程中,获取拍摄装置采集得到的多帧图像,然后识别目标对象分别在每帧图像中的位置(其中,如何识别目标对象在图像中的位置可以参见神经网络模型或者图像跟踪等相关技术中的描述,此处不再赘述),以及根据上述的第一位置将目标对象投影到每帧图像中,获得目标对象是每帧图像中的投影位置。针对每帧图像,确定识别到的位置与投影位置之间的偏差,可以称为重投影误差。根据多帧图像对应的重投影误差(即多个重投影误差),确定目标对象是否从静止状态切换为运动状态。
比如判断多个重投影误差是否预设的误差收敛条件,也就是这多个重投影误差是否收敛于同一误差。例如判断这多个重投影误差位于预设误差范围内。如果多个重投影误差满足预设的误差收敛条件时,说明目标对象的位置基本没有变化,确定目标对象仍处于静止状态。当多个重投影误差不满足预设的误差收敛条件时,说明目标对象的位置开始在发生变化,确定目标对象开始运动,目标对象从静止状态切换为运动状态。
例如根据多个重投影误差确定重投影误差越来越大,则确定目标对象开始运动。或者,例如多个重投影误差中一些重投影误差小,一些重投影误差大,则确定目标对象开始运动。从而确定目标对象从静止状态切换为运动状态。
因此,通过本实施例的上述各方式可以实时、快速检测到目标对象由静止状态切换为运动状态。然后即可无缝切换为:根据目标对象在拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置,并根据第二位置控制可移动平台对目标对象执行所述工作任务。避免了从静止状态切换为运动状态时确定的位置出现突变。所以在可移动平台对目标对象执行工作任务的过程中路径平滑,不会出现突然的抖动。
可选的,上述各实施例中的拍摄装置可以是单目相机。
本申请实施例中还提供了一种计算机存储介质,该计算机存储介质中存储有程序指令,所述程序执行时可包括如上述任一实施例中的可移动平台的控制方法的部分或全部步骤。
图8为本申请一实施例提供的可移动平台的控制装置的结构示意图,所述可移动平台包括拍摄装置,如图8所示,所述可移动平台的控制装置800包括:至少一个处理器801,图中以一个处理器801为例示出。
所述至少一个处理器801,用于:
确定目标对象是否为静止状态;
在所述目标对象为静止状态时,根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置,并根据所述第一位置控制可移动平台对所述目标对象执行工作任务;
在所述目标对象为运动状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置,并根据所述第二位置控制可移动平台对所述目标对象执行所述工作任务。
可选的,在所述目标对象为静止状态时,不再更新所述目标对象的第一位置。
可选的,所述至少一个处理器801在根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置时,具体用于:
根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象与所述拍摄装置之间的第一距离,并根据所述第一距离确定所述目标对象的第一位置。
所述至少一个处理器801还用于:获取距离校正偏差,其中,所述距离校正偏差用于表征所述第一距离第二距离之间的距离偏差,所述第二距离为在所述目标对象为静止状态时根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定的所述目标对象与拍摄装置之间的距离。
所述至少一个处理器801,在根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置时,具体用于:
根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象与所述拍摄装置之间的第三距离,根据所述第三距离和所述距离校正偏差确定所述目标对象的第二位置。
可选的,所述至少一个处理器801,具体用于:在所述目标对象为静止状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象与拍摄装置的第二距离;根据所述第一距离和所述第二距 离确定所述距离校正偏差。
可选的,所述可移动平台的本地存储装置中预先存储有一个或多个参考距离校正偏差,所述至少一个处理器801,具体用于:
从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差。
可选的,所述可移动平台的本地存储装置中预先存储有多个对象类型对应的参考距离校正偏差,所述至少一个处理器801,还用于:根据所述拍摄装置采集到的图像确定所述目标对象的对象类型。
所述至少一个处理器801,在从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差时,具体用于:
根据所述目标对象的对象类型从所述可移动平台的本地存储装置中获取与目标对象的对象类型匹配的参考距离校正偏差,将所述参考距离校正偏差确定为所述距离校正偏差。
可选的,所述可移动平台的本地存储装置中预先存储有多个参考距离对应的参考距离校正偏差。所述至少一个处理器801,具体用于:根据所述第一距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
可选的,所述可移动平台的本地存储装置中获取预先存储的多个参考距离对应的所述距离校正偏差。所述至少一个处理器801,还用于:在所述目标对象为静止状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象与拍摄装置之间的第二距离。
所述至少一个处理器801,在从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差时,具体用于:
根据所述第二距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
可选的,所述可移动平台的本地存储装置中获取预先存储的多个参考距离对应的所述距离校正偏差。
所述至少一个处理器801,具体用于:根据所述第三距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
可选的,所述至少一个处理器801,具体用于:获取所述拍摄装置采集到的包含目标对象的多帧图像;根据所述多帧图像确定所述目标是否为静止状态。
可选的,所述至少一个处理器801,具体用于:根据所述多帧图像运行多视图几何算法以确定多个时刻的目标对象的第四位置;确定所述多个第四位置是否满足预设的位置收敛条件;当满足时,确定所述目标对象为静止状态,否则,确定所述目标对象为运动状态。
可选的,所述至少一个处理器801,还用于:
在根据所述第一位置控制可移动平台对所述目标对象执行工作任务的过程中,确定所述目标对象是否从静止状态切换为运动状态。
可选的,所述至少一个处理器801,具体用于:获取拍摄装置采集得到的多帧图像;识别目标对象在所述多帧图像中的位置;根据所述第一位置将所述目标对象投影到所述多帧图像中以获取目标对象在所述图像中的投影位置;根据所述识别的位置和所述投影位置确定所述目标对象是否从静止状态切换为运动状态。
可选的,所述工作任务为环绕飞行任务和跟踪任务中的至少一种。
可选的,本实施例的可移动平台的控制装置800还可以包括存储器802。存储器802,用于存储程序代码。所述至少一个处理器801,调用所述程序代码,当程序代码被执行时,用于实施上述各方法。
本实施例的可移动平台的控制装置,可以用于执行本申请上述各方法实施例的技术方案,其实现原理和技术效果类似,此处不再赘述。
图9为本申请一实施例提供的可移动平台的结构示意图,所述可移动平台900包括拍摄装置901和至少一个处理器902,图中以一个处理器902为例示出。
所述至少一个处理器902,用于:确定目标对象是否为静止状态;在所述目标对象为静止状态时,根据所述拍摄装置901采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置,并根据所述第一位置控制可 移动平台900对所述目标对象执行工作任务;在所述目标对象为运动状态时,根据所述目标对象在所述拍摄装置901采集到的图像中的图像区域尺寸确定所述目标对象的第二位置,并根据所述第二位置控制可移动平台900对所述目标对象执行所述工作任务。
可选的,在所述目标对象为静止状态时,不再更新所述目标对象的第一位置。
可选的,所述至少一个处理器902在根据所述拍摄装置901采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置时,具体用于:
根据所述拍摄装置901采集到的多帧图像运行多视图几何算法以确定所述目标对象与所述拍摄装置901之间的第一距离,并根据所述第一距离确定所述目标对象的第一位置。
所述至少一个处理器902还用于:获取距离校正偏差,其中,所述距离校正偏差用于表征所述第一距离第二距离之间的距离偏差,所述第二距离为在所述目标对象为静止状态时根据所述目标对象在所述拍摄装置901采集到的图像中的图像区域尺寸确定的所述目标对象与拍摄装置901之间的距离。
所述至少一个处理器902,在根据所述目标对象在所述拍摄装置901采集到的图像中的图像区域尺寸确定所述目标对象的第二位置时,具体用于:
根据所述目标对象在所述拍摄装置901采集到的图像中的图像区域尺寸确定所述目标对象与所述拍摄装置901之间的第三距离,根据所述第三距离和所述距离校正偏差确定所述目标对象的第二位置。
可选的,所述至少一个处理器902,具体用于:在所述目标对象为静止状态时,根据所述目标对象在所述拍摄装置901采集到的图像中的图像区域尺寸确定所述目标对象与拍摄装置901的第二距离;根据所述第一距离和所述第二距离确定所述距离校正偏差。
可选的,本实施例的可移动平台900本地还包括存储装置903。所述可移动平台900的本地存储装置903中预先存储有一个或多个参考距离校正偏差,所述至少一个处理器902,具体用于:
从所述可移动平台900的本地存储装置903中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差。
可选的,所述可移动平台900的本地存储装置903中预先存储有多个对 象类型对应的参考距离校正偏差,所述至少一个处理器902,还用于:根据所述拍摄装置901采集到的图像确定所述目标对象的对象类型。
所述至少一个处理器902,在从所述可移动平台900的本地存储装置903中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差时,具体用于:
根据所述目标对象的对象类型从所述可移动平台900的本地存储装置903中获取与目标对象的对象类型匹配的参考距离校正偏差,将所述参考距离校正偏差确定为所述距离校正偏差。
可选的,所述可移动平台900的本地存储装置903中预先存储有多个参考距离对应的参考距离校正偏差。
所述至少一个处理器902,具体用于:根据所述第一距离从所述可移动平台900的本地存储装置903中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
可选的,所述可移动平台900的本地存储装置903中获取预先存储的多个参考距离对应的所述距离校正偏差,所述至少一个处理器902,还用于:
在所述目标对象为静止状态时,根据所述目标对象在所述拍摄装置901采集到的图像中的图像区域尺寸确定所述目标对象与拍摄装置901之间的第二距离。
所述至少一个处理器902,在从所述可移动平台900的本地存储装置903中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差时,具体用于:
根据所述第二距离从所述可移动平台900的本地存储装置903中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
可选的,所述可移动平台900的本地存储装置903中获取预先存储的多个参考距离对应的所述距离校正偏差。所述至少一个处理器902,具体用于:根据所述第三距离从所述可移动平台900的本地存储装置903中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
可选的,所述至少一个处理器902,具体用于:获取所述拍摄装置901 采集到的包含目标对象的多帧图像;根据所述多帧图像确定所述目标是否为静止状态。
可选的,所述至少一个处理器902,具体用于:根据所述多帧图像运行多视图几何算法以确定多个时刻的目标对象的第四位置;确定所述多个第四位置是否满足预设的位置收敛条件;当满足时,确定所述目标对象为静止状态,否则,确定所述目标对象为运动状态。
可选的,所述至少一个处理器902,还用于:
在根据所述第一位置控制可移动平台900对所述目标对象执行工作任务的过程中,确定所述目标对象是否从静止状态切换为运动状态。
可选的,所述至少一个处理器902,具体用于:获取拍摄装置901采集得到的多帧图像;识别目标对象在所述多帧图像中的位置;根据所述第一位置将所述目标对象投影到所述多帧图像中以获取目标对象在所述图像中的投影位置;根据所述识别的位置和所述投影位置确定所述目标对象是否从静止状态切换为运动状态。
可选的,所述工作任务为环绕飞行任务和跟踪任务中的至少一种。
可选的,本实施例的可移动平台900还可以包括存储器(图中未示出)。存储器,用于存储程序代码。所述至少一个处理器902,调用所述程序代码,当程序代码被执行时,用于实施上述各方法。
存储器与上述存储装置903可以为同一部件,也可以为不同的部件。
本实施例的可移动平台,可以用于执行本申请上述各方法实施例的技术方案,其实现原理和技术效果类似,此处不再赘述。
图10为本申请另一实施例提供的可移动平台的结构示意图,所述可移动平台1000包括拍摄装置1001和可移动平台的控制装置1002。
其中,可移动平台的控制装置1002可以采用图8所示装置实施例的结构,对应,可以执行上述任一方法实施例提供的技术方案,此处不再赘述。
可选的,可移动平台1000还包括存储装置1003。存储装置1003用于预先存储有一个或多个参考距离校正偏差。
本领域普通技术人员可以理解:实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成,前述的程序可以存储于一计算机可读取存储介质中,该程序在执行时,执行包括上述方法实施例的步骤;而前述 的存储介质包括:只读内存(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是:以上各实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述各实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分或者全部技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。
Claims (30)
- 一种可移动平台的控制方法,其特征在于,所述可移动平台包括拍摄装置,所述方法包括:确定目标对象是否为静止状态;在所述目标对象为静止状态时,根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置,并根据所述第一位置控制可移动平台对所述目标对象执行工作任务;在所述目标对象为运动状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置,并根据所述第二位置控制可移动平台对所述目标对象执行所述工作任务。
- 根据权利要求1所述的方法,其特征在于,在所述目标对象为静止状态时,不再更新所述目标对象的第一位置。
- 根据权利要求1或2所述的方法,其特征在于,所述根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置,包括:根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象与所述拍摄装置之间的第一距离,并根据所述第一距离确定所述目标对象的第一位置;所述方法还包括:获取距离校正偏差,其中,所述距离校正偏差用于表征所述第一距离和第二距离之间的距离偏差,所述第二距离为在所述目标对象为静止状态时根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定的所述目标对象与拍摄装置之间的距离;所述根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置,包括:根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象与所述拍摄装置之间的第三距离,根据所述第三距离和所述距离校正偏差确定所述目标对象的第二位置。
- 根据权利要求3所述的方法,其特征在于,所述获取距离校正偏差包括:在所述目标对象为静止状态时,根据所述目标对象在所述拍摄装置采集 到的图像中的图像区域尺寸确定所述目标对象与拍摄装置的第二距离;根据所述第一距离和所述第二距离确定所述距离校正偏差。
- 根据权利要求3所述的方法,其特征在于,所述可移动平台的本地存储装置中预先存储有一个或多个参考距离校正偏差,所述获取距离校正偏差包括:从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差。
- 根据权利要求5所述的方法,其特征在于,所述可移动平台的本地存储装置中预先存储有多个对象类型对应的参考距离校正偏差,所述方法还包括:根据所述拍摄装置采集到的图像确定所述目标对象的对象类型;所述从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差,包括:根据所述目标对象的对象类型从所述可移动平台的本地存储装置中获取与目标对象的对象类型匹配的参考距离校正偏差,将所述参考距离校正偏差确定为所述距离校正偏差。
- 根据权利要求5所述的方法,其特征在于,所述可移动平台的本地存储装置中预先存储有多个参考距离对应的参考距离校正偏差;从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差,包括:根据所述第一距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
- 根据权利要求5所述的方法,其特征在于,所述可移动平台的本地存储装置中获取预先存储的多个参考距离对应的所述距离校正偏差,所述方法还包括:在所述目标对象为静止状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象与拍摄装置之间的第二距离;从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差,包括:根据所述第二距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
- 根据权利要求5所述的方法,其特征在于,所述可移动平台的本地存储装置中获取预先存储的多个参考距离对应的所述距离校正偏差;从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差,包括:根据所述第三距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
- 根据权利要求1-9任一项所述的方法,其特征在于,所述确定目标对象是否为静止状态,包括:获取所述拍摄装置采集到的包含目标对象的多帧图像;根据所述多帧图像确定所述目标是否为静止状态。
- 根据权利要求10所述的方法,其特征在于,所述根据所述多帧图像确定所述目标是否为静止状态,包括:根据所述多帧图像运行多视图几何算法以确定多个时刻的目标对象的第四位置;确定所述多个第四位置是否满足预设的位置收敛条件;当满足时,确定所述目标对象为静止状态,否则,确定所述目标对象为运动状态。
- 根据权利要求1-11任一项所述的方法,其特征在于,所述方法还包括:在根据所述第一位置控制可移动平台对所述目标对象执行工作任务的过程中,确定所述目标对象是否从静止状态切换为运动状态。
- 根据权利要求12所述的方法,其特征在于,所述确定所述目标对象是否从静止状态切换为运动状态,包括:获取拍摄装置采集得到的多帧图像;识别目标对象在所述多帧图像中的位置;根据所述第一位置将所述目标对象投影到所述多帧图像中以获取目标对 象在所述图像中的投影位置;根据所述识别的位置和所述投影位置确定所述目标对象是否从静止状态切换为运动状态。
- 根据权利要求1-13任一项所述的方法,其特征在于,所述工作任务为环绕飞行任务和跟踪任务中的至少一种。
- 一种可移动平台的控制装置,其特征在于,所述可移动平台包括拍摄装置,所述可移动平台的控制装置包括:存储器和至少一个处理器;所述存储器,用于存储程序代码;所述至少一个处理器,用于执行所述程序代码,以用于:确定目标对象是否为静止状态;在所述目标对象为静止状态时,根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置,并根据所述第一位置控制可移动平台对所述目标对象执行工作任务;在所述目标对象为运动状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置,并根据所述第二位置控制可移动平台对所述目标对象执行所述工作任务。
- 根据权利要求15所述的控制装置,其特征在于,在所述目标对象为静止状态时,不再更新所述目标对象的第一位置。
- 根据权利要求15或16所述的控制装置,其特征在于,所述至少一个处理器,在根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象的第一位置时,具体用于:根据所述拍摄装置采集到的多帧图像运行多视图几何算法以确定所述目标对象与所述拍摄装置之间的第一距离,并根据所述第一距离确定所述目标对象的第一位置;所述至少一个处理器还用于:获取距离校正偏差,其中,所述距离校正偏差用于表征所述第一距离和第二距离之间的距离偏差,所述第二距离为在所述目标对象为静止状态时根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定的所述目标对象与拍摄装置之间的距离;所述至少一个处理器,在根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象的第二位置时,具体用于:根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象与所述拍摄装置之间的第三距离,根据所述第三距离和所述距离校正偏差确定所述目标对象的第二位置。
- 根据权利要求17所述的控制装置,其特征在于,所述至少一个处理器,具体用于:在所述目标对象为静止状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象与拍摄装置的第二距离;根据所述第一距离和所述第二距离确定所述距离校正偏差。
- 根据权利要求17所述的控制装置,其特征在于,所述可移动平台的本地存储装置中预先存储有一个或多个参考距离校正偏差,所述至少一个处理器,具体用于:从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差。
- 根据权利要求19所述的控制装置,其特征在于,所述可移动平台的本地存储装置中预先存储有多个对象类型对应的参考距离校正偏差,所述至少一个处理器,还用于:根据所述拍摄装置采集到的图像确定所述目标对象的对象类型;所述至少一个处理器,在从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差时,具体用于:根据所述目标对象的对象类型从所述可移动平台的本地存储装置中获取与目标对象的对象类型匹配的参考距离校正偏差,将所述参考距离校正偏差确定为所述距离校正偏差。
- 根据权利要求19所述的控制装置,其特征在于,所述可移动平台的本地存储装置中预先存储有多个参考距离对应的参考距离校正偏差;所述至少一个处理器,具体用于:根据所述第一距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
- 根据权利要求19所述的控制装置,其特征在于,所述可移动平台的本地存储装置中获取预先存储的多个参考距离对应的所述距离校正偏差,所 述至少一个处理器,还用于:在所述目标对象为静止状态时,根据所述目标对象在所述拍摄装置采集到的图像中的图像区域尺寸确定所述目标对象与拍摄装置之间的第二距离;所述至少一个处理器,在从所述可移动平台的本地存储装置中获取至少一个参考距离校正偏差,根据所述至少一个参考距离校正偏差确定所述距离校正偏差时,具体用于:根据所述第二距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
- 根据权利要求19所述的控制装置,其特征在于,所述可移动平台的本地存储装置中获取预先存储的多个参考距离对应的所述距离校正偏差;所述至少一个处理器,具体用于:根据所述第三距离从所述可移动平台的本地存储装置中获取至少一个参考距离对应的参考距离校正偏差,根据所述参考距离校正偏差确定所述距离校正偏差。
- 根据权利要求15-23任一项所述的控制装置,其特征在于,所述至少一个处理器,具体用于:获取所述拍摄装置采集到的包含目标对象的多帧图像;根据所述多帧图像确定所述目标是否为静止状态。
- 根据权利要求24所述的控制装置,其特征在于,所述至少一个处理器,具体用于:根据所述多帧图像运行多视图几何算法以确定多个时刻的目标对象的第四位置;确定所述多个第四位置是否满足预设的位置收敛条件;当满足时,确定所述目标对象为静止状态,否则,确定所述目标对象为运动状态。
- 根据权利要求15-25任一项所述的控制装置,其特征在于,所述至少一个处理器,还用于:在根据所述第一位置控制可移动平台对所述目标对象执行工作任务的过程中,确定所述目标对象是否从静止状态切换为运动状态。
- 根据权利要求26所述的控制装置,其特征在于,所述至少一个处理 器,具体用于:获取拍摄装置采集得到的多帧图像;识别目标对象在所述多帧图像中的位置;根据所述第一位置将所述目标对象投影到所述多帧图像中以获取目标对象在所述图像中的投影位置;根据所述识别的位置和所述投影位置确定所述目标对象是否从静止状态切换为运动状态。
- 根据权利要求15-27任一项所述的控制装置,其特征在于,所述工作任务为环绕飞行任务和跟踪任务中的至少一种。
- 一种可移动平台,其特征在于,所述可移动平台包括拍摄装置和如权利要求15-28任一项所述的可移动平台的控制装置。
- 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有计算机程序;所述计算机程序在被执行时,实现如权利要求1-14任一项所述的可移动平台的控制方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202080029224.9A CN114096931A (zh) | 2020-04-27 | 2020-04-27 | 可移动平台的控制方法和设备 |
PCT/CN2020/087326 WO2021217372A1 (zh) | 2020-04-27 | 2020-04-27 | 可移动平台的控制方法和设备 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/087326 WO2021217372A1 (zh) | 2020-04-27 | 2020-04-27 | 可移动平台的控制方法和设备 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021217372A1 true WO2021217372A1 (zh) | 2021-11-04 |
Family
ID=78373878
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/087326 WO2021217372A1 (zh) | 2020-04-27 | 2020-04-27 | 可移动平台的控制方法和设备 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN114096931A (zh) |
WO (1) | WO2021217372A1 (zh) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106295459A (zh) * | 2015-05-11 | 2017-01-04 | 青岛若贝电子有限公司 | 基于机器视觉和级联分类器的车辆检测和预警方法 |
CN106813649A (zh) * | 2016-12-16 | 2017-06-09 | 北京远特科技股份有限公司 | 一种图像测距定位的方法、装置及adas |
CN108351653A (zh) * | 2015-12-09 | 2018-07-31 | 深圳市大疆创新科技有限公司 | 用于uav飞行控制的***和方法 |
CN109407697A (zh) * | 2018-09-20 | 2019-03-01 | 北京机械设备研究所 | 一种基于双目视觉测距的无人机跟踪运动目标***及方法 |
CN110222581A (zh) * | 2019-05-13 | 2019-09-10 | 电子科技大学 | 一种基于双目相机的四旋翼无人机视觉目标跟踪方法 |
CN110262565A (zh) * | 2019-05-28 | 2019-09-20 | 深圳市吉影科技有限公司 | 应用于水下六推无人机的目标跟踪运动控制方法及装置 |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102156481B (zh) * | 2011-01-24 | 2013-06-05 | 广州嘉崎智能科技有限公司 | 无人飞行器的智能追踪控制方法及*** |
CN108762310A (zh) * | 2018-05-23 | 2018-11-06 | 深圳市乐为创新科技有限公司 | 一种基于视觉的无人机跟随飞行的控制方法及*** |
-
2020
- 2020-04-27 WO PCT/CN2020/087326 patent/WO2021217372A1/zh active Application Filing
- 2020-04-27 CN CN202080029224.9A patent/CN114096931A/zh active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106295459A (zh) * | 2015-05-11 | 2017-01-04 | 青岛若贝电子有限公司 | 基于机器视觉和级联分类器的车辆检测和预警方法 |
CN108351653A (zh) * | 2015-12-09 | 2018-07-31 | 深圳市大疆创新科技有限公司 | 用于uav飞行控制的***和方法 |
CN106813649A (zh) * | 2016-12-16 | 2017-06-09 | 北京远特科技股份有限公司 | 一种图像测距定位的方法、装置及adas |
CN109407697A (zh) * | 2018-09-20 | 2019-03-01 | 北京机械设备研究所 | 一种基于双目视觉测距的无人机跟踪运动目标***及方法 |
CN110222581A (zh) * | 2019-05-13 | 2019-09-10 | 电子科技大学 | 一种基于双目相机的四旋翼无人机视觉目标跟踪方法 |
CN110262565A (zh) * | 2019-05-28 | 2019-09-20 | 深圳市吉影科技有限公司 | 应用于水下六推无人机的目标跟踪运动控制方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN114096931A (zh) | 2022-02-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20200346753A1 (en) | Uav control method, device and uav | |
WO2019223271A1 (zh) | 飞行器偏航角修正方法、装置及飞行器 | |
WO2020172800A1 (zh) | 可移动平台的巡检控制方法和可移动平台 | |
WO2019227441A1 (zh) | 可移动平台的拍摄控制方法和设备 | |
WO2021217371A1 (zh) | 可移动平台的控制方法和装置 | |
US11798172B2 (en) | Maximum temperature point tracking method, device and unmanned aerial vehicle | |
WO2019227289A1 (zh) | 延时拍摄控制方法和设备 | |
WO2020042159A1 (zh) | 一种云台的转动控制方法、装置及控制设备、移动平台 | |
US11272105B2 (en) | Image stabilization control method, photographing device and mobile platform | |
WO2022016534A1 (zh) | 无人机的飞行控制方法和无人机 | |
WO2021052334A1 (zh) | 一种无人飞行器的返航方法、装置及无人飞行器 | |
WO2021168819A1 (zh) | 无人机的返航控制方法和设备 | |
CN111344651B (zh) | 无人机的控制方法和无人机 | |
WO2020048365A1 (zh) | 飞行器的飞行控制方法、装置、终端设备及飞行控制*** | |
WO2021081774A1 (zh) | 一种参数优化方法、装置及控制设备、飞行器 | |
CN113795803B (zh) | 无人飞行器的飞行辅助方法、设备、芯片、***及介质 | |
WO2020019260A1 (zh) | 磁传感器校准方法、控制终端以及可移动平台 | |
WO2020062089A1 (zh) | 磁传感器校准方法以及可移动平台 | |
WO2019183789A1 (zh) | 无人机的控制方法、装置和无人机 | |
WO2021168821A1 (zh) | 可移动平台的控制方法和设备 | |
WO2020019175A1 (zh) | 图像处理方法和设备、摄像装置以及无人机 | |
JP6910785B2 (ja) | 移動撮像装置およびその制御方法、ならびに撮像装置およびその制御方法、無人機、プログラム、記憶媒体 | |
WO2021217372A1 (zh) | 可移动平台的控制方法和设备 | |
WO2020062255A1 (zh) | 拍摄控制方法和无人机 | |
CN112985359A (zh) | 影像获取方法及影像获取设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20933378 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20933378 Country of ref document: EP Kind code of ref document: A1 |