WO2019061111A1 - Path adjustment method and unmanned aerial vehicle - Google Patents

Path adjustment method and unmanned aerial vehicle Download PDF

Info

Publication number
WO2019061111A1
WO2019061111A1 PCT/CN2017/103808 CN2017103808W WO2019061111A1 WO 2019061111 A1 WO2019061111 A1 WO 2019061111A1 CN 2017103808 W CN2017103808 W CN 2017103808W WO 2019061111 A1 WO2019061111 A1 WO 2019061111A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
return
drone
path
returning
Prior art date
Application number
PCT/CN2017/103808
Other languages
French (fr)
Chinese (zh)
Inventor
周游
刘洁
武志远
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201780009908.0A priority Critical patent/CN108700892A/en
Priority to PCT/CN2017/103808 priority patent/WO2019061111A1/en
Publication of WO2019061111A1 publication Critical patent/WO2019061111A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10—Simultaneous control of position or course in three dimensions
    • G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft

Definitions

  • the present invention relates to the field of electronic technologies, and in particular, to a path adjustment method and a drone.
  • GPS Global Positioning System
  • BDS BeiDou Navigation Satellite System
  • the embodiment of the invention discloses a path adjustment method and a drone, which can realize automatic and accurate return of the drone when there is no positioning signal.
  • a first aspect of the embodiments of the present invention discloses a path adjustment method, including: when a positioning signal is lost, determining a return path according to the recorded location information;
  • Determining location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image;
  • the flight path is adjusted according to the position information of the first position and the return path.
  • the specific when the drone returns in accordance with the return route, the specific includes:
  • the vision module includes a visual odometer (Visual Odometery, VO)
  • a second aspect of an embodiment of the present invention discloses a drone, including: a memory and a processor;
  • the memory is configured to store program instructions
  • the processor is configured to execute the program instructions stored by the memory, when the program instructions are executed, for:
  • the return path is determined according to the recorded position information
  • Determining location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image;
  • the flight path is adjusted according to the position information of the first position and the return path.
  • the drone may determine the return path when the positioning signal is lost, and collect the returning image at the first position when returning according to the return path, and the target mark image is matched, according to the Determining the returning image, the target marking image, and the pose information associated with the target marking image to determine position information of the first position, and adjusting the flight path according to the position information of the first position and the returning path, when the positioning signal is lost.
  • FIG. 1 is a schematic diagram of a scenario for path adjustment according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a path adjustment method according to an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of another path adjustment method according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of another scenario for path adjustment according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a drone according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a system according to an embodiment of the present invention.
  • drones use positioning systems (such as GPS systems, BDS systems, etc.) to achieve positioning, but the positioning system sometimes fails, resulting in loss of positioning signals.
  • positioning systems such as GPS systems, BDS systems, etc.
  • the drone can be used in the following two ways:
  • the first way is that when the positioning signal is lost, the drone stops at the position where the signal is lost, waiting for the operator to find it. However, once the drone has traveled a long distance (for example, 3km, 5km, etc.), the operator takes a long time to reach the stop position of the drone, and the convenience is low. In addition, taking the drone as an example, the value of the drone itself is very high, and the equipment carried by the drone is even more valuable. If the drone crashes in the place where the signal is lost, it crashes inadvertently. Will cause huge losses.
  • the second way is when the positioning signal is lost, the operator manually controls the drone to return.
  • manual control has higher professional requirements for operators, and for non-professional operators, manual operation steps are cumbersome and difficult to operate.
  • the operators of agricultural drones are usually ordinary farmers, and their operation level usually does not meet professional requirements, and agricultural drones are often at low altitudes (for example 5 meters, 10 meters, etc., once the operation is not proper, the drone is very easy to crash, resulting in huge losses.
  • FIG. 1 is a schematic diagram of a scenario for path adjustment according to an embodiment of the present invention.
  • the black line is the navigation path of the drone.
  • the drone starts from the takeoff point A and follows the navigation route planned by the black line.
  • the drone can record a series of The location information and the mark image at the location information are collected, and the location information and the corresponding mark image are saved in the image database, wherein the circle shown in FIG. 1 represents the flight point recorded by the drone, and the flight point corresponds to Location information.
  • the location information may be location information recorded by the location system.
  • the drone can record the acquired location as the location information through a GPS positioning system.
  • the drone may select a return point from the recorded flight points when the lost GPS point B loses the GPS signal (shown in Figure 1 by a black circle).
  • the return point may be a flight point located on the return path, that is, the return point may be a subset of the recorded flight points.
  • the drone can determine a plurality of flight points between the takeoff point A and the lost GPS point B.
  • Each flight point can be associated with a corresponding GPS signal, a corresponding marker image, and a corresponding inertial measurement.
  • Unit (IMU) pose information wherein the associated information is merely an example, not an exhaustive one, and a flight point may also be associated with a plurality of other information.
  • the drone may select a plurality of return points from a plurality of flight points and connect the plurality of return points to obtain a return path.
  • the return path is indicated by a broken line ( Wherein, the portion coincident with the navigation path is not shown in FIG. 1).
  • a polygon approximation strategy such as an RDP algorithm (RamerDouglas Peuckeralgorithm), etc.
  • RDP algorithm RamerDouglas Peuckeralgorithm
  • the drone can utilize the VO to capture the return image and estimate the current location of arrival in real time, as well as determine the flight point within the preset range of the location (eg, within 1 meter, within 3 meters, etc.).
  • the drone is estimated to have reached the vicinity of the return point C (the actual arrived position is represented by the first position), and the returning image n can be collected at the first position, and the returning image is acquired.
  • n a mark image c associated with the return point C and a mark image corresponding to each of the flight points within the preset range of the return point C (for example, within 1 meter, within 3 meters, etc.) (here, the mark image c of the return point C will be
  • the mark image in the preset range of the return point C is represented by the collection m) to perform matching processing.
  • the drone can determine that the marker image c in the collection m matches the return image m according to the matching result of the collection m and the returning image n (where the return point C and the flight point C are the same one) Point, the marker image c is a corresponding picture of the flight point C), and the drone can acquire the GPS signal corresponding to the flight point C, the corresponding marker image (ie, the marker image c) and the corresponding IMU posture information.
  • the drone may be based on the GPS signal corresponding to the flight point C, the corresponding mark image (ie, the mark image c) and the corresponding IMU posture information, and the return image n and the captured image at the first position.
  • the corresponding IMU posture information when the returning image n is captured determines the deviation of the first position from the returning point C.
  • the drone can adjust the flight path to the next return point of the return point C based on the deviation.
  • the position information of the first position indicates that 0.5 meters to the left of the return point C, then the drone can travel 0.5 meters to the right side to return to the return point C for returning, or the drone It is also possible to calculate the distance value from the return point D based on the current position information, and then fly to the return point d according to the distance value to continue to return according to the return path.
  • the drone can further obtain a depth map corresponding to the returning image n and a depth map of the marker image c to obtain a camera pose between the marker image c and the returning image n.
  • the pose relationship can include rotation and displacement.
  • the UAV can use the IMU unit to acquire the posture of the IMU corresponding to the marker image c and the IMU posture corresponding to the return image n, and calculate the pose relationship between the marker image c and the return image n. .
  • the drone can determine that the determination process of the marker image c is correct, and the location information corresponding to the marker image c can be used to represent the first Location location information.
  • the flying height of the drone when the drone returns in accordance with the return path, is lower than a preset height.
  • the preset height can be less than 30 meters. Alternatively, the preset height can be less than 20 meters. Alternatively, the preset height can be less than 10 meters.
  • FIG. 2 is a schematic flowchart diagram of a path adjustment method according to an embodiment of the present invention.
  • the path adjustment method described in this embodiment includes:
  • execution body of the embodiment of the present invention may be an unmanned aerial vehicle, and the unmanned aerial vehicle may be an unmanned aerial vehicle.
  • the drone may be an industrial-grade drone, such as an agricultural sprinkler, etc.
  • the agricultural sprinkler may be flying at a low altitude (for example, a flying height of less than 30 m, or When the flying height is less than 20 m, or the flying height is less than 10 m, etc., the method provided by the embodiment of the present invention is performed.
  • the positioning signal may be a positioning module of the drone (eg, a GPS module, etc.)
  • the obtained signal can be used to record the position information of the drone.
  • the location information of the drone may include positioning information, such as GPS information and the like.
  • the drone may lose the positioning signal. When the positioning signal is lost, the drone can determine the return path according to the position information recorded by the positioning module.
  • the method when the location signal is lost, before determining the return path according to the recorded location information, the method further includes: generating the image database, where the image database includes a plurality of marker images and is associated with the marker image Pose information.
  • the mark image may refer to an image taken corresponding to the position information for marking the position where the drone has flown.
  • the drone flies to position a, where the marker image a is taken, and the marker image a can be used to identify the location a.
  • the pose information may include location information and posture information.
  • the posture information may include Rotation and Translation when the marker image is captured.
  • the generating an image database includes: when the UAV is flying according to the positioning signal, recording a mark image during flight and pose information associated with the mark image during flight; The marker image at the time of flight and the pose information associated with the marker image at the time of flight generate an image database.
  • the drone can fly according to the indication of the positioning signal when the positioning signal is not lost, and record the returning point every time (for example, 1 minute, 10 minutes, etc.) or in real time during the flight. And recording the position information of the return point, and calling the imaging device to capture the mark image at the position information, and recording the posture information when the mark image is captured, and finally storing the mark image, the position information, and the posture information in the image database.
  • the return path refers to the path planned by the drone.
  • the drone can preferentially select a straight flight according to the position information recorded immediately before the lost positioning signal and the position information of the drone when starting, and the path through the more returning point is Return route.
  • the drone can also determine the position information recorded by the two positions based on the position information recorded immediately before the lost positioning signal and the position information when the drone starts.
  • the line segment serves as the return path.
  • the drone may adopt a polygon approximation strategy, such as an RDP algorithm, to determine the return path.
  • a polygon approximation strategy such as an RDP algorithm
  • the flying height of the drone is lower than a preset height when returning according to the return path.
  • the preset height may be 30 meters or 20 meters, which is not limited by the embodiment of the present invention.
  • the preset height can be less than 20 meters. For example, 19 meters, 13 meters, 10 meters, 5 meters and so on.
  • the drone can use a visual odometer to capture the returning image.
  • the drone can also use other means to capture the returning image.
  • the VO can realize the return of the drone according to the return path, but the VO usually pays attention to the motion at the local time (for example, the motion between two moments), when the VO samples the time at some interval, It is possible to estimate the motion of a moving object within each time interval. Since this estimation is affected by noise, image brightness, flying height, etc., it may cause the estimation error of the previous time, which will be added to the motion of the latter time, causing drift ( The Drift phenomenon causes the VO not to return completely in accordance with the return path, and there is a certain error with the return path.
  • the return path includes position information of a plurality of return points; the first position is a point at which the distance from the return point is less than a preset distance.
  • the preset distance may be any preset distance of 1 meter, 5 meters, 10 meters, etc., and the present invention does not impose any limitation.
  • the first position may be a position where the aircraft actually arrives, and the first position may have deviated from the return path.
  • the return path includes a plurality of return points, including the return point 1 as an example.
  • the drone flies to the return point 1, it can be estimated that the return point 1 has been reached, but it is actually possible There is a deviation from the position of the return point 1, which may be within the preset distance.
  • the target mark image may be a mark map that most closely matches the return image. image.
  • the drone may select a plurality of marker images within the preset distance range of the return point 1 (for example, centering on the position of the returned flight 1 to preset The distance range is selected for the coverage area, and the plurality of marker images are matched with the return image to determine the target marker image.
  • the matching processing the returning image with the recorded marking image comprises: performing matching processing according to image similarity between the returned image and the recorded marked image.
  • the drone can extract the image features in the returning image and the image features of the respective marker images if the similarity of the image features of one of the marker images a reaches a preset condition (eg, the similarity is 80% or more) ), it can be determined that the mark image a is a target mark image.
  • a preset condition eg, the similarity is 80% or more
  • S204 Determine location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image.
  • the drone can determine the location information corresponding to the target marker image as the location information of the first location.
  • the drone can determine the difference between the returning image and the target mark image according to the return image and the pose information associated with the target mark image, and calculate the first according to the difference.
  • the deviation of the position from the target mark image can determine the position information of the first position according to the position information corresponding to the target mark image and the deviation between the first position and the target mark image.
  • the flight path refers to the path in which the drone actually flies.
  • the drone can determine the return point 1 and the return point 2, and calculate the distance value D between the return point 1 and the return point 2, and when returning according to the return path, the unmanned person
  • the machine can use VO to determine that it has moved to the vicinity of the return point 1 (but may actually deviate from the position of the return point 1 and the actual position is represented by the first position), and the return image can be collected according to the first position. And matching the return image with the mark image in the image data set to determine the target mark image, and then determining the first position according to the return image, the target mark image, and the pose information associated with the target mark image Location information (for example, at the location of the return point 1) 5 meters to the left)).
  • the UAV can calculate a distance and an orientation between the first location and the returning point 2 according to the location information of the first location, and adjust the flight path based on the distance and the orientation. So that you can fly to the location where the return point 2 is located.
  • the drone may further calculate a distance and an orientation between the first location and the returning point 1 according to the location information of the first location, and may first adjust the flight path to be able to reach The location where the return point 1 is located, and then you can continue to fly so that you can reach the location where the return point 2 is located.
  • the drone may further detect the positioning signal when returning according to the return path; if the positioning signal is detected, return according to the positioning signal and the return path.
  • the drone can preset two flight modes, one can be a mode 1 according to a positioning signal, and the other is a mode 2 according to a visual flight, and the mode of the visual flight is a mode in which a marker image needs to be collected.
  • mode 1 fails, the drone can automatically switch to mode 2.
  • the mode 1 returns to the normal state (that is, the state that the drone can detect the positioning signal), the drone can switch back from mode 2 to mode 1. And can return according to the positioning signal and the determined return path.
  • the drone can determine the return path when the positioning signal is lost, and collect the return image at the first position when the return path is returned according to the return path, and the target mark image is matched. Determining the location information of the first location according to the returning image, the target marker image, and the pose information associated with the target marker image, and adjusting the flight path according to the location information of the first location and the return route, may be lost in positioning When the signal is transmitted, it can automatically return to the air, and during the returning process, the deviation between the actual flight path and the return path can be adjusted, which satisfies the need for automation and intelligence of the drone.
  • FIG. 3 is a schematic flowchart diagram of another path adjustment method according to an embodiment of the present invention. The method described in this embodiment includes:
  • the description information of the returning image may refer to the information generated based on the description of the image.
  • the description information may include an image content profile, a content classification, and the like in the returning image. This is not subject to any restrictions.
  • the drone can determine the description information by a bag of words model (BOW model).
  • BOW model bag of words model
  • the drone may extract an image feature corresponding to the return image, determine a descriptor corresponding to the image feature corresponding to the return image, and then generate description information according to the descriptor.
  • the descriptor corresponding to the image feature may be a description term for the image feature, and the description term may be used to describe the classification, characteristics, and the like of the image feature.
  • the drone may employ clustering in unsupervised ML to classify individual image features to corresponding descriptor descriptors.
  • the generating the description information according to the descriptor comprises: determining a weight value corresponding to the descriptor; and generating description information according to the descriptor and a weight value corresponding to the descriptor.
  • the weight value corresponding to the descriptor may be determined according to the importance degree of the descriptor. Specifically, the degree of importance may be determined according to the proportion of the image content represented by the descriptor in the entire image, the representation in the entire image, and the like.
  • the UAV can use the K-Means++ algorithm (which can guarantee a uniform averaged algorithm), and can represent the description information in a tree structure, and can use the word frequency-inverse document frequency (The term frequency-inverse document frequency, tf-idf) is used to set the weight value of each descriptor.
  • K-Means++ algorithm which can guarantee a uniform averaged algorithm
  • tf-idf word frequency-inverse document frequency
  • the tree structure may be a structure of a K-tree.
  • FIG. 4 is another schematic diagram of a scenario for path adjustment according to an embodiment of the present invention.
  • a solid black line indicates a path when an image feature is searched, and a circle indicates a path.
  • Node also known as a descriptor).
  • the UAV can use the returning image as a root node, and can extract image features in the returned image, determine descriptors corresponding to each image feature, and determine the number of collation layers corresponding to each descriptor, by layer Classification, get the first layer classification, the second layer classification, until the kth layer classification, wherein the kth layer classification can be the last layer classification, the kth layer classification includes the leaf node descriptor That is, the descriptor corresponding to the image content of the image itself, further, the drone can assign a weight value to each node, and the tree structure shown in FIG. 4 can be established, and the content represented by the entire tree structure can be used as Description of the image.
  • the drone can use the returning image a as a root node, and the drone can determine the image feature of the returning image a.
  • Each descriptor in the first layer categorization (indicated by a circle) may each correspond to a weight value, and each descriptor in the second layer categorization may respectively correspond to a weight value, and each description in the third layer categorization
  • the symbols may each correspond to a weight value, wherein the descriptor in the third layer classification may be a leaf node descriptor, and the content represented by the tree structure shown in FIG. 4 may be used as the description information of the return image a.
  • the description information associated with each of the mark images is recorded, including: extracting image features corresponding to the respective mark images, and determining descriptors corresponding to the image features; and generating description information according to the descriptors.
  • the generating the description information according to the descriptor comprises: determining a weight value corresponding to the descriptor; and generating description information according to the descriptor and a weight value corresponding to the descriptor.
  • the drone can determine the corresponding description information by using the same processing manner for each of the return image and each of the marker images, and set a weight value for each descriptor. Specifically, the UAV determines the description information and the manner of setting the weight value according to the image features corresponding to the image, and the parameters of the foregoing step S303 and the corresponding description of FIG. 4 are not described herein.
  • the drone can compare the tree structure determined by the returning image and the tree structure determined by each of the marker images to determine the similarity of the tree structures of the two.
  • the drone may reach a preset similarity condition if the similarity between the tree structure of the return image and the tree structure of the candidate marker image is determined (eg, the similarity is 90% or more) Then, the drone can determine the candidate mark image as the target mark image.
  • a preset similarity condition if the similarity between the tree structure of the return image and the tree structure of the candidate marker image is determined (eg, the similarity is 90% or more) Then, the drone can determine the candidate mark image as the target mark image.
  • the drone can determine that the marker image f is the target marker image.
  • the determining the target mark image according to the comparison result comprises: determining the first mark image according to the comparison result; comparing the depth map corresponding to the first mark image with the depth map corresponding to the return image If the comparison result is that the two depth maps match, the first marker image is used as the target marker image.
  • the drone can utilize a binocular vision system to obtain a depth map of the first marker image and a depth map of the flyback image.
  • comparing the depth map corresponding to the first marker image with the depth map corresponding to the return image includes: a depth map corresponding to the first marker image and a corresponding map of the return image Depth map, obtaining relative pose information between the two depth maps; obtaining a relative rotation relationship between the two images according to the first marker image and the returning image; and comparing the relative rotation relationship with the relative The pose information is compared.
  • the depth information of the screen contents of the two depth images may be different due to the difference in the shooting posture, and the difference of the depth information may be used as the relative of the two images. Rotation relationship.
  • the relative position information between the first mark image and the return image may be a difference between a position and a posture when the first mark image is captured and a position and a posture when the return image is captured.
  • the drone can obtain the two depth images through a binocular vision system, and obtain relative pose information (including position information and posture information) between the two depth images.
  • the location information can be Including rotation
  • the attitude information may include displacement).
  • the drone can also obtain a relative rotation relationship between the returning image and the first marker image through the inertial measurement unit, and match the relative rotation relationship with the relative pose information, and if the matching is performed, the first marker can be The position information corresponding to the image determines the position information of the first position.
  • S306. Determine location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image.
  • the target return point may be the point closest to the first position, or the target return point may be the next return point that the drone should arrive after reaching the first position according to the return path. .
  • the drone can return to the return path of the return point 1 to the return point 2, and after reaching the return point 1 (the actual position may be offset from the return point 1 and the actual position is represented by the first position), it can be determined
  • the return point 1 is reached, and the exact position of the first position is determined by matching the return image with the marker image, and the distance value between the first position and the return point 1 (ie, the target return point) is determined.
  • the UAV can further determine the first position and the return point 2 (ie, the target return point) after determining the exact position of the first position by matching the return image with the marked image. Distance value.
  • the drone can correct the current flight path according to the distance value, so that the drone can return to the return path and fly according to the return path.
  • the unmanned aerial vehicle can determine the return path when the positioning signal is lost, and collect the returning image when returning according to the return path, determine the description information of the returning image, and describe the returning image.
  • the information is compared with the description information of each mark image, and the target mark image is determined according to the comparison result, and the position information of the first position is determined according to the target mark image, and finally the distance value between the first position and the target return point is determined.
  • the flight path can be adjusted, and the position of the drone can be continuously corrected by the method of matching the returning image and the marked image during the returning process, so that the drone can return according to the returning path, thereby realizing the unmanned Automatic and accurate return of the machine when there is no positioning signal.
  • FIG. 5 is a schematic structural diagram of a drone according to an embodiment of the present invention.
  • the drone described in this embodiment includes: a memory 501 and a processor 502;
  • the memory 501 is configured to store program instructions
  • the processor 502 is configured to execute the program instructions stored in the memory, when the program instructions are executed, to:
  • the return path is determined according to the recorded position information
  • Determining location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image;
  • the flight path is adjusted according to the position information of the first position and the return path.
  • the processor 502 when the processor 502 is configured to lose the positioning signal, before determining the return path according to the recorded location information, the processor 502 is further configured to:
  • the image database is generated, the image database including a plurality of marker images and pose information associated with the marker images.
  • processor 502 when the processor 502 generates an image database, it is specifically used to:
  • An image database is generated based on the mark image at the time of flight and the pose information associated with the mark image at the time of flight.
  • the return path includes location information of a plurality of return points
  • the first position is a point where the distance from the return point is less than a preset distance.
  • the return path refers to: the path planned by the drone;
  • the flight path refers to the path in which the drone actually flies.
  • the processor 502 when the processor 502 adjusts the flight path according to the location information of the first location and the return path, the processor 502 is specifically configured to:
  • the flight path is adjusted based on the distance value to cause the drone to fly to the target return point.
  • the processor 502 performs matching processing on the return image with the mark image in the image database, and determines the target mark image according to the result of the matching process, specifically for:
  • processor 502 when the processor 502 records the description information associated with each of the tag images, it is specifically used to:
  • Descriptive information is generated based on the descriptor.
  • Descriptive information is generated based on the descriptor and a weight value corresponding to the descriptor.
  • the processor 502 determines the target mark image according to the comparison result, it is specifically used to:
  • the first marker image is used as the target marker image.
  • the processor 502 compares the depth map corresponding to the first marker image with the depth map corresponding to the return image, the processor 502 is specifically configured to:
  • the relative rotational relationship is compared to the relative pose information.
  • the processor 502 when the processor 502 performs matching processing on the return image and the recorded mark image, the processor 502 is specifically configured to:
  • a matching process is performed according to the image similarity between the returning image and the recorded mark image.
  • the processor 502 is further configured to:
  • the processor 502 is further configured to:
  • the flying height of the drone is lower than a preset height.
  • the preset height is less than 20 meters.
  • FIG. 6 is a schematic structural diagram of a system according to an embodiment of the present invention. As shown in FIG. 6, the system includes a camera 601 and a drone 602.
  • the UAV 602 is the UAV 602 disclosed in the foregoing embodiment of the present invention, and the principles and implementations are similar to the foregoing embodiments, and details are not described herein again.
  • the camera device 601 can be disposed on the drone 562 for capturing a marker image and a return image for marking a position during flight.
  • the imaging device 60 can be mounted on the main body of the drone by a pan/tilt or other mounted device.
  • the camera device is used for image or video shooting during flight of the drone, including but not limited to multi-spectral imager, hyperspectral imager, visible light camera and infrared camera, VO, etc., and the camera device can be one or one the above.
  • the drone can control the camera to capture images in real time during flight.
  • the drone 602 can determine the return path according to the recorded position information when the positioning signal is lost, and collect the return image at the first position when returning according to the return path, and mark the return image and the image database. Performing matching processing on the image, and determining a target mark image according to the result of the matching process, and determining position information of the first position according to the return image, the target mark image, and pose information associated with the target mark image, according to the first The location information of the location and the return path adjust the flight path.
  • UAV 602 can be used to perform the path adjustment method shown in the foregoing method embodiment, and the specific implementation process can refer to the method embodiment, and details are not described herein.
  • the program can be stored in a computer readable storage medium, and the storage medium can include: Flash disk, Read-Only Memory (ROM), Random Access Memory (RAM), disk or optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Navigation (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

A path adjustment method and an unmanned aerial vehicle. The method comprises: when a positioning signal is lost, determining a return flight path according to recorded position information; during a return flight according to the return flight path, collecting a return flight image at a first position; carrying out matching processing on the return flight image and marker images in an image database, and determining a target marker image according to a matching processing result; determining position information about the first position according to the return flight image, the target marker image and pose information associated with the target marker image; and adjusting a flight path according to the position information about the first position and the return flight path, so that an automatic and accurate return flight can be carried out when the unmanned aerial vehicle loses a positioning signal.

Description

一种路径调整方法及无人机Path adjustment method and drone 技术领域Technical field
本发明涉及电子技术领域，尤其涉及一种路径调整方法及无人机。The present invention relates to the field of electronic technologies, and in particular, to a path adjustment method and a drone.
背景技术Background technique
随着电子技术的不断发展，无人机也得到了很大发展。With the continuous development of electronic technology, drones have also been greatly developed.
目前，无人机大多数都是依靠定位***(例如全球定位***(Global Positioning System，GPS)、北斗卫星导航***(BeiDou Navigation Satellite System，BDS)等)来实现定位，在丢失定位信号时，操作者可以手动控制该无人机来进行返航。At present, most of the drones rely on positioning systems (such as Global Positioning System (GPS), BeiDou Navigation Satellite System (BDS), etc.) to achieve positioning, when the positioning signal is lost, the operation The drone can be manually controlled to return.
然而，上述操作者手动控制返航的方式，操作步骤十分繁琐，且对操作者的专业要求较高，对于非专业的操作者来说，容易出现操作失误的情况，导致返航过程中，无人机容易出现跌落、摔坏的问题，造成不必要的损失。However, the above-mentioned operator manually controls the way of returning, the operation steps are very cumbersome, and the professional requirements of the operator are high, and for non-professional operators, the operation error is easy to occur, resulting in the returning process, the drone It is prone to problems of falling and breaking, causing unnecessary losses.
发明内容Summary of the invention
本发明实施例公开了一种路径调整方法及无人机，可以实现无人机在无定位信号时的自动精准返航。The embodiment of the invention discloses a path adjustment method and a drone, which can realize automatic and accurate return of the drone when there is no positioning signal.
本发明实施例第一方面公开了一种路径调整方法，包括：丢失定位信号时，根据已记录的位置信息确定返航路径；A first aspect of the embodiments of the present invention discloses a path adjustment method, including: when a positioning signal is lost, determining a return path according to the recorded location information;
在按照所述返航路径返航时，在第一位置处采集返航图像；When returning according to the return path, collecting a returning image at the first position;
将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像；Performing matching processing on the return image and the mark image in the image database, and determining the target mark image according to the result of the matching process;
根据所述返航图像、所述目标标记图像以及与所述目标标记图像关联的位姿信息确定所述第一位置的位置信息；Determining location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image;
根据所述第一位置的位置信息和所述返航路径调整飞行路径。The flight path is adjusted according to the position information of the first position and the return path.
其中，所述无人机在按照所述返航路径返航时，具体包括：Wherein, when the drone returns in accordance with the return route, the specific includes:
利用视觉模块按照该返航路径返航；其中，该视觉模块包括视觉里程计(Visual Odometery，VO) Returning to the return path using the vision module; wherein the vision module includes a visual odometer (Visual Odometery, VO)
本发明实施例第二方面公开了一种无人机，包括：存储器和处理器；A second aspect of an embodiment of the present invention discloses a drone, including: a memory and a processor;
所述存储器，用于存储程序指令；The memory is configured to store program instructions;
所述处理器，用于执行所述存储器存储的程序指令，当程序指令被执行时，用于：The processor is configured to execute the program instructions stored by the memory, when the program instructions are executed, for:
丢失定位信号时，根据已记录的位置信息确定返航路径；When the positioning signal is lost, the return path is determined according to the recorded position information;
在按照所述返航路径返航时，在第一位置处采集返航图像；When returning according to the return path, collecting a returning image at the first position;
将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像；Performing matching processing on the return image and the mark image in the image database, and determining the target mark image according to the result of the matching process;
根据所述返航图像、所述目标标记图像以及与所述目标标记图像关联的位姿信息确定所述第一位置的位置信息；Determining location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image;
根据所述第一位置的位置信息和所述返航路径调整飞行路径。The flight path is adjusted according to the position information of the first position and the return path.
在本发明实施例中，无人机可以在丢失定位信号时，确定出返航路径，并在按照所述返航路径返航时，在第一位置处采集返航图像，将匹配出目标标记图像，根据所述返航图像、该目标标记图像以及与该目标标记图像关联的位姿信息确定该第一位置的位置信息，根据该第一位置的位置信息和该返航路径调整飞行路径，可以在丢失定位信号时自动返航，一定程度上满足了对无人机自动化、智能化需求。In the embodiment of the present invention, the drone may determine the return path when the positioning signal is lost, and collect the returning image at the first position when returning according to the return path, and the target mark image is matched, according to the Determining the returning image, the target marking image, and the pose information associated with the target marking image to determine position information of the first position, and adjusting the flight path according to the position information of the first position and the returning path, when the positioning signal is lost The automatic return to navigation has satisfied the automation and intelligent needs of the drone to a certain extent.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without paying for creative labor.
图1是本发明实施例提供的一种用于路径调整的情景示意图；1 is a schematic diagram of a scenario for path adjustment according to an embodiment of the present invention;
图2是本发明实施例提供的一种路径调整方法的流程示意图；2 is a schematic flowchart of a path adjustment method according to an embodiment of the present invention;
图3是本发明实施例提供的另一种路径调整方法的流程示意图；3 is a schematic flowchart of another path adjustment method according to an embodiment of the present invention;
图4是本发明实施例提供的另一种用于路径调整的情景示意图；FIG. 4 is a schematic diagram of another scenario for path adjustment according to an embodiment of the present invention; FIG.
图5是本发明实施例提供的一种无人机的结构示意图；FIG. 5 is a schematic structural diagram of a drone according to an embodiment of the present invention; FIG.
图6是本发明实施例提供的一种***的结构示意图。 FIG. 6 is a schematic structural diagram of a system according to an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present invention will be clearly and completely described in the following with reference to the accompanying drawings.
大多数无人机采用的是定位***(例如GPS***，BDS***等)来实现定位，但是定位***有时会出现故障，导致定位信号丢失。在这种情况下，无人机可以采用以下两种方式：Most drones use positioning systems (such as GPS systems, BDS systems, etc.) to achieve positioning, but the positioning system sometimes fails, resulting in loss of positioning signals. In this case, the drone can be used in the following two ways:
第一种方式是在定位信号丢失时，该无人机就在丢失信号的位置原地停止，以等待操作者来寻找。然而，一旦无人机已经驶出很远的距离(例如3km、5km等)，操作者就需要花费很长时间才能到达无人机停止的位置，便捷性较低。另外，以无人机为例，无人机本身价值很高，再加上无人机搭载的设备，更是价值不菲，如果无人机在丢失信号的位置原地紧急迫降时不慎坠毁，将会造成巨大的损失。The first way is that when the positioning signal is lost, the drone stops at the position where the signal is lost, waiting for the operator to find it. However, once the drone has traveled a long distance (for example, 3km, 5km, etc.), the operator takes a long time to reach the stop position of the drone, and the convenience is low. In addition, taking the drone as an example, the value of the drone itself is very high, and the equipment carried by the drone is even more valuable. If the drone crashes in the place where the signal is lost, it crashes inadvertently. Will cause huge losses.
第二种方式是在定位信号丢失时，操作者手动控制该无人机进行返航。然而，手动控制对操作者的专业要求较高，对于非专业的操作者来说，手动操作步骤十分繁琐，且操作困难。举例来说，以无人机为农业无人机为例，农业无人机的操作者通常为普通农户，其操作水平通常达不到专业要求，且农业无人机又常常是在低空(例如5米、10米等)作业，一旦操作不当，无人机十分容易坠毁，导致巨大的损失。The second way is when the positioning signal is lost, the operator manually controls the drone to return. However, manual control has higher professional requirements for operators, and for non-professional operators, manual operation steps are cumbersome and difficult to operate. For example, taking drones as agricultural drones, the operators of agricultural drones are usually ordinary farmers, and their operation level usually does not meet professional requirements, and agricultural drones are often at low altitudes (for example 5 meters, 10 meters, etc., once the operation is not proper, the drone is very easy to crash, resulting in huge losses.
因此，如何保证无人机在丢失定位信号时能够自动、精准的返航，成为了一个亟待解决的问题。Therefore, how to ensure that the drone can return automatically and accurately when the positioning signal is lost becomes an urgent problem to be solved.
为了解决上面提出的技术问题，本发明实施例提供了一种路径调整方法。请参阅图1，为本发明实施例提供的一种用于路径调整的情景示意图。如图1所示，黑线为无人机的导航路径，无人机从起飞点A出发，并按照黑线规划的导航路径前行，在前行过程中，该无人机可以记录一系列的位置信息以及采集该位置信息处的标记图像，并将该位置信息与对应的标记图像保存在图像数据库，其中，图1所示的圆圈表示该无人机记录的飞行点，该飞行点对应着位置信息。In order to solve the technical problem presented above, an embodiment of the present invention provides a path adjustment method. FIG. 1 is a schematic diagram of a scenario for path adjustment according to an embodiment of the present invention. As shown in Figure 1, the black line is the navigation path of the drone. The drone starts from the takeoff point A and follows the navigation route planned by the black line. During the forward process, the drone can record a series of The location information and the mark image at the location information are collected, and the location information and the corresponding mark image are saved in the image database, wherein the circle shown in FIG. 1 represents the flight point recorded by the drone, and the flight point corresponds to Location information.
在一个实施例中，该位置信息可以是通过定位***记录下来的位置信息。 例如，该无人机可以通过GPS定位***将获取到的定位记录为该位置信息。In one embodiment, the location information may be location information recorded by the location system. For example, the drone can record the acquired location as the location information through a GPS positioning system.
在一个实施例中，该无人机在丢失GPS点B丢失GPS信号时，可以从记录的飞行点中选择出返航点(图1用黑色圆圈表示)。具体的，该返航点可以是位于返航路径上的飞行点，也就是说，返航点可以为记录的飞行点的子集。In one embodiment, the drone may select a return point from the recorded flight points when the lost GPS point B loses the GPS signal (shown in Figure 1 by a black circle). Specifically, the return point may be a flight point located on the return path, that is, the return point may be a subset of the recorded flight points.
在一个实施例中，该无人机可以在起飞点A到丢失GPS点B之间，确定出多个飞行点，每一个飞行点可以关联对应的GPS信号，对应的标记图像以及对应的惯性测量单元(IMU)姿态信息，其中，上述关联的信息只是举例，而非穷举，一个飞行点还可以关联多个其他信息。In one embodiment, the drone can determine a plurality of flight points between the takeoff point A and the lost GPS point B. Each flight point can be associated with a corresponding GPS signal, a corresponding marker image, and a corresponding inertial measurement. Unit (IMU) pose information, wherein the associated information is merely an example, not an exhaustive one, and a flight point may also be associated with a plurality of other information.
在一个实施例中，该无人机可以从多个飞行点中选择出多个返航点，并将该多个返航点连线，得到返航路径，在图1中，该返航路径由虚线表示(其中，与导航路径重合部分在图1中未示出)。In one embodiment, the drone may select a plurality of return points from a plurality of flight points and connect the plurality of return points to obtain a return path. In FIG. 1, the return path is indicated by a broken line ( Wherein, the portion coincident with the navigation path is not shown in FIG. 1).
其中，该无人机确定返航路径时，可以采用多边形近似的策略，如RDP算法(RamerDouglas Peuckeralgorithm)等等，本发明对此不作任何限制。Wherein, when the UAV determines the return path, a polygon approximation strategy, such as an RDP algorithm (RamerDouglas Peuckeralgorithm), etc., may be adopted, and the present invention does not impose any limitation.
在一个实施例中，该无人机可以利用VO采集返航图像，并实时预估当前到达的位置，以及确定在该位置预设范围内(例如1米内，3米内等等)的飞行点。In one embodiment, the drone can utilize the VO to capture the return image and estimate the current location of arrival in real time, as well as determine the flight point within the preset range of the location (eg, within 1 meter, within 3 meters, etc.).
在一个实施例中，该无人机预估当前到达了返航点C的附近(实际到达的位置用第一位置表示)，就可以在该第一位置处采集返航图像n，并将该返航图像n与该返航点C相关联的标记图像c以及在该返航点C预设范围内(例如1米内、3米内等)的飞行点各自对应的标记图像(此处将返航点C的标记图像c以及返航点C预设范围内的标记图像用合集m表示)进行匹配处理。In one embodiment, the drone is estimated to have reached the vicinity of the return point C (the actual arrived position is represented by the first position), and the returning image n can be collected at the first position, and the returning image is acquired. n a mark image c associated with the return point C and a mark image corresponding to each of the flight points within the preset range of the return point C (for example, within 1 meter, within 3 meters, etc.) (here, the mark image c of the return point C will be And the mark image in the preset range of the return point C is represented by the collection m) to perform matching processing.
在一个实施例中，该无人机可以根据该合集m以及返航图像n的匹配结果，确定出合集m中的标记图像c与返航图像m匹配(其中，返航点C和飞行点C是同一个点，标记图像c是飞行点C的对应图片)，该无人机就可以获取该飞行点C对应的GPS信号，对应的标记图像(即标记图像c)以及对应的IMU姿态信息。In an embodiment, the drone can determine that the marker image c in the collection m matches the return image m according to the matching result of the collection m and the returning image n (where the return point C and the flight point C are the same one) Point, the marker image c is a corresponding picture of the flight point C), and the drone can acquire the GPS signal corresponding to the flight point C, the corresponding marker image (ie, the marker image c) and the corresponding IMU posture information.
在一个实施例中，该无人机可以根据该飞行点C对应的GPS信号、对应的标记图像(即标记图像c)以及对应的IMU姿态信息，以及在第一位置拍摄到的返航图像n和拍摄该返航图像n时对应的IMU姿态信息，确定该第一位置距离返航点C的偏差。 In one embodiment, the drone may be based on the GPS signal corresponding to the flight point C, the corresponding mark image (ie, the mark image c) and the corresponding IMU posture information, and the return image n and the captured image at the first position. The corresponding IMU posture information when the returning image n is captured determines the deviation of the first position from the returning point C.
在一个实施例中，该无人机可以根据该偏差调整飞到返航点C的下一个返航点的飞行路径。例如，第一位置的位置信息表示在返航点C正左方0.5米、那么该无人机可以向正右方行进0.5米，以回到该返航点C上进行返航，或者，该无人机还可以根据当前位置信息计算出距离返航点D的距离值，然后根据该距离值飞行到返航点d，以继续按照该返航路径返航。In one embodiment, the drone can adjust the flight path to the next return point of the return point C based on the deviation. For example, the position information of the first position indicates that 0.5 meters to the left of the return point C, then the drone can travel 0.5 meters to the right side to return to the return point C for returning, or the drone It is also possible to calculate the distance value from the return point D based on the current position information, and then fly to the return point d according to the distance value to continue to return according to the return path.
在一个实施例中，该无人机还可以将该返航图像n对应的深度图以及标记图像c的深度图，得到该标记图像c与返航图像n之间的位姿关系(camera pose)，该位姿关系可以包括旋转和位移。In an embodiment, the drone can further obtain a depth map corresponding to the returning image n and a depth map of the marker image c to obtain a camera pose between the marker image c and the returning image n. The pose relationship can include rotation and displacement.
在一个实施例中，该无人机可以利用IMU单元获取该标记图像c对应的IMU的姿态以及返航图像n对应的IMU姿态，并计算得到该标记图像c与返航图像n之间的位姿关系。In an embodiment, the UAV can use the IMU unit to acquire the posture of the IMU corresponding to the marker image c and the IMU posture corresponding to the return image n, and calculate the pose relationship between the marker image c and the return image n. .
在一个实施例中，如果通过上述两种方式计算得到的位姿关系匹配，那么该无人机可以确定该标记图像c的确定过程正确，该标记图像c对应的位置信息可以用于表示第一位置的位置信息。In an embodiment, if the pose relationship is calculated by the above two methods, the drone can determine that the determination process of the marker image c is correct, and the location information corresponding to the marker image c can be used to represent the first Location location information.
在一个实施例中，所述无人机在按照所述返航路径返航时，所述无人机的飞行高度低于预置高度。该预置高度可以小于30米。或者，该预置高度可以小于20米。或者，该预置高度可以小于10米。In one embodiment, when the drone returns in accordance with the return path, the flying height of the drone is lower than a preset height. The preset height can be less than 30 meters. Alternatively, the preset height can be less than 20 meters. Alternatively, the preset height can be less than 10 meters.
为了更好的说明，下面描述本申请的方法实施例。For a better description, the method embodiments of the present application are described below.
请参阅图2，为本发明实施例提供的一种路径调整方法的流程示意图。本实施例中所描述的路径调整方法，包括：FIG. 2 is a schematic flowchart diagram of a path adjustment method according to an embodiment of the present invention. The path adjustment method described in this embodiment includes:
S201、丢失定位信号时，根据已记录的位置信息确定返航路径。S201. When the positioning signal is lost, the return path is determined according to the recorded location information.
需要说明的是，本发明实施例的执行主体可以为无人机，该无人机可以是指无人驾驶飞行器。It should be noted that the execution body of the embodiment of the present invention may be an unmanned aerial vehicle, and the unmanned aerial vehicle may be an unmanned aerial vehicle.
在一些可行的实施方式中，该无人机可以是工业级无人机，例如农业喷洒无人机等，进一步的，该农业喷洒无人机可以是在低空飞行(例如飞行高度小于30m，或者飞行高度小于20m，或者飞行高度小于10m等等)时，执行本发明实施例所提供的方法。In some feasible embodiments, the drone may be an industrial-grade drone, such as an agricultural sprinkler, etc. Further, the agricultural sprinkler may be flying at a low altitude (for example, a flying height of less than 30 m, or When the flying height is less than 20 m, or the flying height is less than 10 m, etc., the method provided by the embodiment of the present invention is performed.
需要说明的是，该定位信号可以是该无人机的定位模块(例如GPS模块等) 取得的信号，可用于记录该无人机的位置信息。It should be noted that the positioning signal may be a positioning module of the drone (eg, a GPS module, etc.) The obtained signal can be used to record the position information of the drone.
具体的，该无人机的位置信息可以包括定位信息，例如GPS信息等等。Specifically, the location information of the drone may include positioning information, such as GPS information and the like.
需要说明的是，定位模块出现故障或供能不足时，该无人机可能会丢失定位信号，在丢失定位信号时，该无人机可以根据定位模块记录的位置信息，来确定出返航路径。It should be noted that when the positioning module fails or the power supply is insufficient, the drone may lose the positioning signal. When the positioning signal is lost, the drone can determine the return path according to the position information recorded by the positioning module.
在一个实施例中，所述丢失定位信号时，根据已记录的位置信息确定返航路径之前，还包括：生成所述图像数据库，所述图像数据库中包括多个标记图像和与所述标记图像关联的位姿信息。In an embodiment, when the location signal is lost, before determining the return path according to the recorded location information, the method further includes: generating the image database, where the image database includes a plurality of marker images and is associated with the marker image Pose information.
需要说明的是，该标记图像可以是指对应于位置信息拍摄的图像，用于标记无人机飞行过的位置。例如，该无人机飞行到位置a，并在该位置a拍摄该标记图像a，该标记图像a就可以用于标识该位置a。It should be noted that the mark image may refer to an image taken corresponding to the position information for marking the position where the drone has flown. For example, the drone flies to position a, where the marker image a is taken, and the marker image a can be used to identify the location a.
需要说明的是，该位姿信息，可以包括位置信息以及姿态信息。It should be noted that the pose information may include location information and posture information.
其中，该姿态信息可以包括拍摄该标记图像时的旋转(Rotation)和位移(Translation)。The posture information may include Rotation and Translation when the marker image is captured.
在一个实施例中，所述生成图像数据库，包括：在所述无人机根据定位信号飞行时，记录飞行时的标记图像以及与所述飞行时的标记图像关联的位姿信息；根据所述飞行时的标记图像以及与所述飞行时的标记图像关联的位姿信息生成图像数据库。In one embodiment, the generating an image database includes: when the UAV is flying according to the positioning signal, recording a mark image during flight and pose information associated with the mark image during flight; The marker image at the time of flight and the pose information associated with the marker image at the time of flight generate an image database.
举例来说，该无人机可以在未丢失定位信号时，根据定位信号的指示进行飞行，并在飞行过程中，每隔一段时间(例如1分钟、10分钟等)或者实时记录一次返航点，并记录该返航点的位置信息，并调用摄像装置在该位置信息处拍摄标记图像，以及记录拍摄该标记图像时的姿态信息，最后将上述标记图像、位置信息、姿态信息保存在图像数据库中。For example, the drone can fly according to the indication of the positioning signal when the positioning signal is not lost, and record the returning point every time (for example, 1 minute, 10 minutes, etc.) or in real time during the flight. And recording the position information of the return point, and calling the imaging device to capture the mark image at the position information, and recording the posture information when the mark image is captured, and finally storing the mark image, the position information, and the posture information in the image database.
在一个实施例中，所述返航路径是指：所述无人机规划的路径。In one embodiment, the return path refers to the path planned by the drone.
在一些可行的实施方式中，无人机可以根据丢失定位信号前一瞬间记录的位置信息，以及无人机出发时的位置信息，优先选择直线飞行，且经过较多的返航点的路径为该返航路径。In some feasible implementation manners, the drone can preferentially select a straight flight according to the position information recorded immediately before the lost positioning signal and the position information of the drone when starting, and the path through the more returning point is Return route.
在一些可行的实施方式中，无人机也可以根据丢失定位信号前一瞬间记录的位置信息，以及无人机出发时的位置信息，将由两个位置信息确定出来的直 线段作为该返航路径。In some feasible implementation manners, the drone can also determine the position information recorded by the two positions based on the position information recorded immediately before the lost positioning signal and the position information when the drone starts. The line segment serves as the return path.
具体实现中，该无人机可以采用多边形近似策略，如RDP算法来确定该返航路径，当然，上述方式只是举例，而非穷举，包含但不限于上述可选方式。In a specific implementation, the drone may adopt a polygon approximation strategy, such as an RDP algorithm, to determine the return path. Of course, the foregoing manner is only an example, not an exhaustive one, including but not limited to the above optional manner.
S202、在按照所述返航路径返航时，在第一位置处采集返航图像。S202. Collect a returning image at the first position when returning according to the return path.
在一个实施例中，在按照所述返航路径返航时，所述无人机的飞行高度低于预置高度。该预置高度可以为30米，也可以为20米，本发明实施例对此不作任何限制。In one embodiment, the flying height of the drone is lower than a preset height when returning according to the return path. The preset height may be 30 meters or 20 meters, which is not limited by the embodiment of the present invention.
在一些可行的实施方式中，该预置高度可以小于20米。例如为19米，13米，10米，5米等等。In some possible implementations, the preset height can be less than 20 meters. For example, 19 meters, 13 meters, 10 meters, 5 meters and so on.
在一些可行的实施例中，该无人机可以使用视觉里程计来采集返航图像。或者，该无人机也可以采用其他方式来采集返航图像。In some possible embodiments, the drone can use a visual odometer to capture the returning image. Alternatively, the drone can also use other means to capture the returning image.
需要说明的是，通过VO可以实现无人机根据返航路径返航，但VO通常关注局部时间上的运动(例如两个时刻之间的运动)，当VO以某种间隔对时间进行采样时，就可估计运动物体在各时间间隔之内的运动，由于这个估计受噪声影响、图像亮度、飞行高度等因素，可能会导致先前时刻的估计误差，会累加到后面时间的运动之上，造成漂移(Drift)现象，使VO并不完全按照该返航路径返航，与该返航路径存在一定的误差。It should be noted that the VO can realize the return of the drone according to the return path, but the VO usually pays attention to the motion at the local time (for example, the motion between two moments), when the VO samples the time at some interval, It is possible to estimate the motion of a moving object within each time interval. Since this estimation is affected by noise, image brightness, flying height, etc., it may cause the estimation error of the previous time, which will be added to the motion of the latter time, causing drift ( The Drift phenomenon causes the VO not to return completely in accordance with the return path, and there is a certain error with the return path.
在一个实施例中，所述返航路径包括多个返航点的位置信息；所述第一位置为与所述返航点的距离小于预置距离的点。In one embodiment, the return path includes position information of a plurality of return points; the first position is a point at which the distance from the return point is less than a preset distance.
需要说明的是，该预置距离可以为1米、5米、10米等任意预设的距离，本发明对此不作任何限制。It should be noted that the preset distance may be any preset distance of 1 meter, 5 meters, 10 meters, etc., and the present invention does not impose any limitation.
其中，该第一位置可以是飞行器实际到达的位置，且该第一位置可以已偏离该返航路径。Wherein, the first position may be a position where the aircraft actually arrives, and the first position may have deviated from the return path.
还需要说明的是，该返航路径中包括多个返航点，以包括返航点1为例，该无人机在飞行到返航点1时，可以预估已到达返航点1的附近，但实际可能与返航点1的位置存在偏差，该偏差值可以在该预置距离之内。It should also be noted that the return path includes a plurality of return points, including the return point 1 as an example. When the drone flies to the return point 1, it can be estimated that the return point 1 has been reached, but it is actually possible There is a deviation from the position of the return point 1, which may be within the preset distance.
S203、将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像。S203. Perform matching processing on the return image and the mark image in the image database, and determine a target mark image according to the result of the matching process.
需要说明的是，该目标标记图像可以为与该返航图像最为匹配的标记图 像。It should be noted that the target mark image may be a mark map that most closely matches the return image. image.
举例来说，该无人机在确定飞行到返航点1的位置时，可以选取该返航点1预设距离范围内的多个标记图像(例如以该返航的1的位置为中心，以预设距离范围为覆盖面积进行选取)，并将该多个标记图像与该返航图像进行匹配处理，确定出目标标记图像。For example, when determining the position of the flight to the return point 1 , the drone may select a plurality of marker images within the preset distance range of the return point 1 (for example, centering on the position of the returned flight 1 to preset The distance range is selected for the coverage area, and the plurality of marker images are matched with the return image to determine the target marker image.
在一个实施例中，所述将所述返航图像与已记录的标记图像进行匹配处理，包括：根据所述返航图像与已记录的标记图像之间的图像相似度进行匹配处理。In one embodiment, the matching processing the returning image with the recorded marking image comprises: performing matching processing according to image similarity between the returned image and the recorded marked image.
举例来说，该无人机可以提取该返航图像中的图像特征以及各个标记图像的图像特征，如果其中一个标记图像a的图像特征的相似度达到预设条件(例如相似度在80％及以上)，就可以确定该标记图像a为目标标记图像。For example, the drone can extract the image features in the returning image and the image features of the respective marker images if the similarity of the image features of one of the marker images a reaches a preset condition (eg, the similarity is 80% or more) ), it can be determined that the mark image a is a target mark image.
S204、根据所述返航图像、所述目标标记图像以及与所述目标标记图像关联的位姿信息确定所述第一位置的位置信息。S204. Determine location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image.
举例来说，该无人机可以将该目标标记图像所对应的位置信息确定为该第一位置的位置信息。For example, the drone can determine the location information corresponding to the target marker image as the location information of the first location.
又举例来说，该无人机还可以根据返航图像以及与目标标记图像关联的位姿信息，确定出该返航图像与该目标标记图像的差异度，并可以根据该差异都计算出该第一位置与该目标标记图像的偏差，根据该目标标记图像对应的位置信息以及该第一位置与该目标标记图像的偏差，便可以确定出该第一位置的位置信息。For example, the drone can determine the difference between the returning image and the target mark image according to the return image and the pose information associated with the target mark image, and calculate the first according to the difference. The deviation of the position from the target mark image can determine the position information of the first position according to the position information corresponding to the target mark image and the deviation between the first position and the target mark image.
S205、根据所述第一位置的位置信息和所述返航路径调整飞行路径。S205. Adjust a flight path according to the location information of the first location and the return path.
在一个实施例中，所述飞行路径是指：所述无人机实际飞行的路径。In one embodiment, the flight path refers to the path in which the drone actually flies.
在一些可行的实施方式中，该无人机可以确定出返航点1以及返航点2，并计算出返航点1与返航点2之间的距离值D，在按照返航路径返航时，该无人机可以利用VO确定已飞行到返航点1的位置附近(但实际可能会与返航点1的位置存在偏差，实际到达的位置用第一位置表示)，就可以根据在第一位置处采集返航图像，并将该返航图像与图像数据集中的标记图像进行匹配出来，确定出目标标记图像，然后根据该返航图像、该目标标记图像以及与该目标标记图像关联的位姿信息，确定该第一位置的位置信息(例如位于该返航点1位置的 正左方5米处)。In some feasible implementation manners, the drone can determine the return point 1 and the return point 2, and calculate the distance value D between the return point 1 and the return point 2, and when returning according to the return path, the unmanned person The machine can use VO to determine that it has moved to the vicinity of the return point 1 (but may actually deviate from the position of the return point 1 and the actual position is represented by the first position), and the return image can be collected according to the first position. And matching the return image with the mark image in the image data set to determine the target mark image, and then determining the first position according to the return image, the target mark image, and the pose information associated with the target mark image Location information (for example, at the location of the return point 1) 5 meters to the left)).
在一些可行的实施方式中，该无人机可以根据该第一位置的位置信息，计算出该第一位置与返航点2之间的距离以及方位，并可以基于该距离以及方位调整该飞行路径，以便于可以飞行到该返航点2所在的位置。In some feasible implementation manners, the UAV can calculate a distance and an orientation between the first location and the returning point 2 according to the location information of the first location, and adjust the flight path based on the distance and the orientation. So that you can fly to the location where the return point 2 is located.
在一些可行的实施方式中，该无人机还可以根据该第一位置的位置信息，计算出该第一位置与返航点1之间的距离以及方位，并可以首先调整飞行路径以便于能够到达返航点1所在的位置，然后可以继续飞行，以便于到达返航点2所在的位置。In some possible implementations, the drone may further calculate a distance and an orientation between the first location and the returning point 1 according to the location information of the first location, and may first adjust the flight path to be able to reach The location where the return point 1 is located, and then you can continue to fly so that you can reach the location where the return point 2 is located.
在一个实施例中，所述无人机还可以在按照所述返航路径返航时，检测所述定位信号；若检测到所述定位信号，则根据所述定位信号以及所述返航路径返航。In an embodiment, the drone may further detect the positioning signal when returning according to the return path; if the positioning signal is detected, return according to the positioning signal and the return path.
具体实现中，无人机可以预置两个飞行模式，一种可以是按定位信号飞行的模式1，一种是按视觉飞行的模式2，该视觉飞行的模式即为需要采集标记图像的模式。当模式1出现故障时，无人机可以自动切换至模式2，当该模式1恢复正常状态(即无人机可以检测到定位信号的状态)，无人机便可以从模式2切换回模式1，并可以根据定位信号以及确定出的返航路径进行返航。In a specific implementation, the drone can preset two flight modes, one can be a mode 1 according to a positioning signal, and the other is a mode 2 according to a visual flight, and the mode of the visual flight is a mode in which a marker image needs to be collected. . When mode 1 fails, the drone can automatically switch to mode 2. When the mode 1 returns to the normal state (that is, the state that the drone can detect the positioning signal), the drone can switch back from mode 2 to mode 1. And can return according to the positioning signal and the determined return path.
可见，在本发明实施例中，无人机可以在丢失定位信号时，确定出返航路径，并在按照所述返航路径返航时，在第一位置处采集返航图像，将匹配出目标标记图像，根据所述返航图像、该目标标记图像以及与该目标标记图像关联的位姿信息确定该第一位置的位置信息，根据该第一位置的位置信息和该返航路径调整飞行路径，可以在丢失定位信号时，能够自动返航，且在返航过程中可以实施调整实际飞行路径与返航路径之间的偏差，一定程度上满足了对无人机自动化、智能化需求。It can be seen that, in the embodiment of the present invention, the drone can determine the return path when the positioning signal is lost, and collect the return image at the first position when the return path is returned according to the return path, and the target mark image is matched. Determining the location information of the first location according to the returning image, the target marker image, and the pose information associated with the target marker image, and adjusting the flight path according to the location information of the first location and the return route, may be lost in positioning When the signal is transmitted, it can automatically return to the air, and during the returning process, the deviation between the actual flight path and the return path can be adjusted, which satisfies the need for automation and intelligence of the drone.
请参阅图3，为本发明实施例提供的另一种路径调整方法的流程示意图。本实施例中所描述的方法，包括：FIG. 3 is a schematic flowchart diagram of another path adjustment method according to an embodiment of the present invention. The method described in this embodiment includes:
S301、丢失定位信号时，根据已记录的位置信息确定返航路径。S301. When the positioning signal is lost, the return path is determined according to the recorded location information.
S302、在按照所述返航路径返航时，在第一位置处采集返航图像。 S302. Collecting a returning image at the first position when returning according to the return path.
需要说明的是，本发明实施例所示的S301步骤以及S302步骤的具体实现方式可参照前述方法实施例的S201步骤以及S202步骤，在此不作赘述。It should be noted that, in the specific implementation manners of the steps S301 and S302 in the embodiment of the present invention, reference may be made to the steps S201 and S202 in the foregoing method embodiments, and details are not described herein.
S303、确定所述返航图像的描述信息。S303. Determine description information of the returning image.
需要说明的是，该返航图像的描述信息，可以是指基于对图像的描述产生的信息，具体的，该描述信息可以包括返航图像中的图像内容简介、内容分类等等，本发明实施例对此不作任何限制。It should be noted that the description information of the returning image may refer to the information generated based on the description of the image. Specifically, the description information may include an image content profile, a content classification, and the like in the returning image. This is not subject to any restrictions.
在一些可行的实施方式中，该无人机可以通过词袋模型(bag of words model，BOW model)来确定该描述信息。In some possible implementations, the drone can determine the description information by a bag of words model (BOW model).
在一个实施例中，该无人机可以提取返航图像对应的图像特征，并确定该返航图像对应的图像特征所对应的描述符，然后根据该描述符生成描述信息。In one embodiment, the drone may extract an image feature corresponding to the return image, determine a descriptor corresponding to the image feature corresponding to the return image, and then generate description information according to the descriptor.
还需要说明的是，该图像特征对应的描述符，可以是对该图像特征的描述词语，该描述词语可以用于描述该图像特征的分类、特点等等。It should also be noted that the descriptor corresponding to the image feature may be a description term for the image feature, and the description term may be used to describe the classification, characteristics, and the like of the image feature.
在一些可行的实施方式中，无人机可以采用无监督机器学习(unsupervised ML)中的聚类(clustering)来将各个图像特征归类得到对应的描述符(feature descriptors)。In some possible implementations, the drone may employ clustering in unsupervised ML to classify individual image features to corresponding descriptor descriptors.
在一个实施例中，所述根据所述描述符生成描述信息，包括：确定所述描述符对应的权重值；根据所述描述符以及所述描述符对应的权重值生成描述信息。In an embodiment, the generating the description information according to the descriptor comprises: determining a weight value corresponding to the descriptor; and generating description information according to the descriptor and a weight value corresponding to the descriptor.
需要说明的是，该描述符对应的权重值可以是根据描述符的重要程度确定的。具体的，该重要程度可以根据描述符所表示的图像内容在整个图像中的占比范围、在整个图像中的代表性等等来确定。It should be noted that the weight value corresponding to the descriptor may be determined according to the importance degree of the descriptor. Specifically, the degree of importance may be determined according to the proportion of the image content represented by the descriptor in the entire image, the representation in the entire image, and the like.
在一些可行的实施方式中，该无人机可以使用K-Means++算法(能保证归类均匀的均值算法)，并可以用树形结构来表示该描述信息，并可以采用词频-逆文档频率(term frequency-inverse document frequency，tf-idf)来设置每个描述符的权重值。In some feasible embodiments, the UAV can use the K-Means++ algorithm (which can guarantee a uniform averaged algorithm), and can represent the description information in a tree structure, and can use the word frequency-inverse document frequency ( The term frequency-inverse document frequency, tf-idf) is used to set the weight value of each descriptor.
进一步的，该树形结构可以是K叉树的结构。Further, the tree structure may be a structure of a K-tree.
为了更好的说明，请参阅图4，为本发明实施例提供的另一种用于路径调整的情景示意图，在图4中，黑色实线表示查找图像特征时的路径，圆圈表示路径上的节点(也即描述符)。 For better explanation, please refer to FIG. 4, which is another schematic diagram of a scenario for path adjustment according to an embodiment of the present invention. In FIG. 4, a solid black line indicates a path when an image feature is searched, and a circle indicates a path. Node (also known as a descriptor).
具体的，该无人机可以将返航图像作为根节点，并可以提取该返航图像中的图像特征，确定各个图像特征对应的描述符，并确定各个描述符对应的归类层数，通过层层归类，得到第一层归类，第二层归类，直至第k层归类，其中，第k层归类可以为最后一层归类，该第k层归类包括叶子节点的描述词，也就是图像本身的图像内容所对应的描述符，进一步，该无人机可以对每一个节点赋予权重值，就可以建立图4所示的树形结构，整个树形结构表示的内容可以作为该图像的描述信息。Specifically, the UAV can use the returning image as a root node, and can extract image features in the returned image, determine descriptors corresponding to each image feature, and determine the number of collation layers corresponding to each descriptor, by layer Classification, get the first layer classification, the second layer classification, until the kth layer classification, wherein the kth layer classification can be the last layer classification, the kth layer classification includes the leaf node descriptor That is, the descriptor corresponding to the image content of the image itself, further, the drone can assign a weight value to each node, and the tree structure shown in FIG. 4 can be established, and the content represented by the entire tree structure can be used as Description of the image.
举例来说，如图4所示，该无人机可以将返航图像a作为根节点，该无人机可以确定该返航图像a的图像特征。根节点为返航图像a，可以分为3层归类(k＝3)。第一层归类中的各个描述符(以圆圈表示)可以各自对应着一个权重值，第二层归类中的各个描述符可以各自对应着一个权重值，第三层归类中的各个描述符可以各自对应着一个权重值，其中，第三层归类中的描述符可以为叶子节点描述词，整个图4所示的树形结构表示的内容可以作为该返航图像a的描述信息。For example, as shown in FIG. 4, the drone can use the returning image a as a root node, and the drone can determine the image feature of the returning image a. The root node is the return image a, which can be divided into three layers (k=3). Each descriptor in the first layer categorization (indicated by a circle) may each correspond to a weight value, and each descriptor in the second layer categorization may respectively correspond to a weight value, and each description in the third layer categorization The symbols may each correspond to a weight value, wherein the descriptor in the third layer classification may be a leaf node descriptor, and the content represented by the tree structure shown in FIG. 4 may be used as the description information of the return image a.
S304、根据所述返航图像的描述信息与图像数据库中的已记录的各个标记图像关联的描述信息，进行相似度比较。S304. Perform similarity comparison according to the description information of the returned image and the description information associated with each recorded mark image in the image database.
在一个实施例中，记录各个标记图像关联的描述信息，包括：提取各个标记图像对应的图像特征，并确定所述图像特征对应的描述符；根据所述描述符生成描述信息。In one embodiment, the description information associated with each of the mark images is recorded, including: extracting image features corresponding to the respective mark images, and determining descriptors corresponding to the image features; and generating description information according to the descriptors.
在一个实施例中，所述根据所述描述符生成描述信息，包括：确定所述描述符对应的权重值；根据所述描述符以及所述描述符对应的权重值生成描述信息。In an embodiment, the generating the description information according to the descriptor comprises: determining a weight value corresponding to the descriptor; and generating description information according to the descriptor and a weight value corresponding to the descriptor.
需要说明的是，无人机可以将返航图像与各个标记图像均利用相同的处理方式，来确定对应的描述信息，并对各个描述符设置权重值。具体的，该无人机根据图像对应的图像特征来确定出描述信息和设置权重值的方式，可参数前述S303步骤，以及图4的对应描述部分，在此不作赘述。It should be noted that the drone can determine the corresponding description information by using the same processing manner for each of the return image and each of the marker images, and set a weight value for each descriptor. Specifically, the UAV determines the description information and the manner of setting the weight value according to the image features corresponding to the image, and the parameters of the foregoing step S303 and the corresponding description of FIG. 4 are not described herein.
在一些可行的实施方式中，该无人机可以将该返航图像确定出来的树形结构，以及各个标记图像确定出来的树形结构进行对比，确定二者的树形结构的相似度。 In some feasible embodiments, the drone can compare the tree structure determined by the returning image and the tree structure determined by each of the marker images to determine the similarity of the tree structures of the two.
S305、根据比较结果确定出目标标记图像。S305. Determine a target mark image according to the comparison result.
在一些可行的实施方式中，该无人机可以如果确定返航图像的树形结构与待选标记图像的树形结构的相似度达到预设相似度条件(例如相似度在90％及其以上)，那么该无人机可以确定该待选标记图像为目标标记图像。In some feasible implementation manners, the drone may reach a preset similarity condition if the similarity between the tree structure of the return image and the tree structure of the candidate marker image is determined (eg, the similarity is 90% or more) Then, the drone can determine the candidate mark image as the target mark image.
举例来说，如果该比较结果表示返航图像的树形结构与标记图像f的树形结构相似度在99％，那么该无人机可以确定该标记图像f为目标标记图像。For example, if the comparison result indicates that the tree structure of the returning image is 99% similar to the tree structure of the marker image f, the drone can determine that the marker image f is the target marker image.
在一个实施例中，所述根据比较结果确定出目标标记图像，包括：根据比较结果确定出第一标记图像；将所述第一标记图像对应的深度图，与返航图像对应的深度图进行比较；若比较结果为两幅深度图之间相匹配，则将所述第一标记图像作为目标标记图像。In one embodiment, the determining the target mark image according to the comparison result comprises: determining the first mark image according to the comparison result; comparing the depth map corresponding to the first mark image with the depth map corresponding to the return image If the comparison result is that the two depth maps match, the first marker image is used as the target marker image.
在一些可行的实施方式中，该无人机可以利用双目视觉***得到第一标记图像的深度图以及返航图像的深度图。In some possible implementations, the drone can utilize a binocular vision system to obtain a depth map of the first marker image and a depth map of the flyback image.
在一个实施例中，所述将所述第一标记图像对应的深度图，与返航图像对应的深度图进行比较，包括：根据所述第一标记图像对应的深度图以及所述返航图像对应的深度图，得到两幅深度图之间的相对位姿信息；根据所述第一标记图像以及所述返航图像，得到两幅图像之间的相对旋转关系；将所述相对旋转关系与所述相对位姿信息进行比较。In one embodiment, comparing the depth map corresponding to the first marker image with the depth map corresponding to the return image includes: a depth map corresponding to the first marker image and a corresponding map of the return image Depth map, obtaining relative pose information between the two depth maps; obtaining a relative rotation relationship between the two images according to the first marker image and the returning image; and comparing the relative rotation relationship with the relative The pose information is compared.
需要说明的是，摄像装置在拍摄上述两幅深度图像时，由于拍摄姿态的差异，可能导致两幅深度图像的画面内容的深度信息具有差异，该深度信息的差异可以作为该两幅图像的相对旋转关系。It should be noted that, when the image capturing apparatus captures the two depth images, the depth information of the screen contents of the two depth images may be different due to the difference in the shooting posture, and the difference of the depth information may be used as the relative of the two images. Rotation relationship.
还需要说明的是，该第一标记图像以及该返航图像之间的相对位姿信息，可以是拍摄第一标记图像时的位置和姿态，与拍摄返航图像时的位置和姿态之间的差异。It should be noted that the relative position information between the first mark image and the return image may be a difference between a position and a posture when the first mark image is captured and a position and a posture when the return image is captured.
举例来说，该无人机可以通过双目视觉***得到该两幅深度图像，并解得两幅深度图像之间的相对位姿信息(包括位置信息和姿态信息，具体的，该位置信息可以包括旋转，该姿态信息可以包括位移)。该无人机还可以通过惯性测量单元得到返航图像与第一标记图像的相对旋转关系，并将该相对旋转关系与该相对位姿信息进行匹配，如果匹配上了，则可以根据该第一标记图像对应的位置信息，确定出该第一位置的位置信息。 For example, the drone can obtain the two depth images through a binocular vision system, and obtain relative pose information (including position information and posture information) between the two depth images. Specifically, the location information can be Including rotation, the attitude information may include displacement). The drone can also obtain a relative rotation relationship between the returning image and the first marker image through the inertial measurement unit, and match the relative rotation relationship with the relative pose information, and if the matching is performed, the first marker can be The position information corresponding to the image determines the position information of the first position.
S306、根据所述返航图像、所述目标标记图像以及与所述目标标记图像关联的位姿信息确定所述第一位置的位置信息。S306. Determine location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image.
S307、确定所述第一位置为与所述返航路径中的目标返航点的距离值。S307. Determine that the first location is a distance value from a target return point in the return path.
需要说明的是，该目标返航点可以是指距离该第一位置最近的点，或者，该目标返航点还可以是该无人机按照返航路径到达第一位置之后，应该到达的下一个返航点。It should be noted that the target return point may be the point closest to the first position, or the target return point may be the next return point that the drone should arrive after reaching the first position according to the return path. .
举例来说，该无人机可以按照返航点1到返航点2的返航路径返航，在到达返航点1后(实际位置可能偏移该返航点1，实际位置用第一位置表示)，可以确定到达返航点1，并通过返航图像与标记图像匹配的方式，确定该第一位置的准确位置，并确定该第一位置与返航点1(即目标返航点)之间的距离值。For example, the drone can return to the return path of the return point 1 to the return point 2, and after reaching the return point 1 (the actual position may be offset from the return point 1 and the actual position is represented by the first position), it can be determined The return point 1 is reached, and the exact position of the first position is determined by matching the return image with the marker image, and the distance value between the first position and the return point 1 (ie, the target return point) is determined.
又举例来说，该无人机还可以在通过返航图像与标记图像匹配的方式，确定该第一位置的准确位置之后，进一步确定该第一位置与返航点2(即目标返航点)之间的距离值。For another example, the UAV can further determine the first position and the return point 2 (ie, the target return point) after determining the exact position of the first position by matching the return image with the marked image. Distance value.
S308、根据所述距离值调整飞行路径，以使所述无人机飞行到所述目标返航点。S308. Adjust a flight path according to the distance value, so that the drone flies to the target return point.
需要说明的是，该无人机可以根据该距离值修正当前的飞行路径，以使该无人机可以回到返航路径上，并按照该返航路径飞行。It should be noted that the drone can correct the current flight path according to the distance value, so that the drone can return to the return path and fly according to the return path.
可见，在本发明实施例中，该无人机可以在丢失定位信号时，确定出返航路径，并在按照返航路径返航时采集返航图像，确定该返航图像的描述信息，并将返航图像的描述信息与各个标记图像的描述信息进行相似度比较，根据比较结果确定出目标标记图像，并根据该目标标记图像确定该第一位置的位置信，最后确定该第一位置与目标返航点的距离值，按照该距离值调整飞行路径，可以在返航过程中，通过返航图像和标记图像匹配的方式，不断修正无人机的位置，以使该无人机可以按照该返航路径返航，实现了无人机在无定位信号时的自动精准返航。It can be seen that, in the embodiment of the present invention, the unmanned aerial vehicle can determine the return path when the positioning signal is lost, and collect the returning image when returning according to the return path, determine the description information of the returning image, and describe the returning image. The information is compared with the description information of each mark image, and the target mark image is determined according to the comparison result, and the position information of the first position is determined according to the target mark image, and finally the distance value between the first position and the target return point is determined. According to the distance value, the flight path can be adjusted, and the position of the drone can be continuously corrected by the method of matching the returning image and the marked image during the returning process, so that the drone can return according to the returning path, thereby realizing the unmanned Automatic and accurate return of the machine when there is no positioning signal.
请参阅图5，为本发明实施例提供的一种无人机的结构示意图。本实施例中所描述的无人机，包括：存储器501和处理器502；FIG. 5 is a schematic structural diagram of a drone according to an embodiment of the present invention. The drone described in this embodiment includes: a memory 501 and a processor 502;
所述存储器501，用于存储程序指令； The memory 501 is configured to store program instructions;
所述处理器502，用于执行所述存储器存储的程序指令，当程序指令被执行时，用于：The processor 502 is configured to execute the program instructions stored in the memory, when the program instructions are executed, to:
丢失定位信号时，根据已记录的位置信息确定返航路径；When the positioning signal is lost, the return path is determined according to the recorded position information;
在按照所述返航路径返航时，在第一位置处采集返航图像；When returning according to the return path, collecting a returning image at the first position;
将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像；Performing matching processing on the return image and the mark image in the image database, and determining the target mark image according to the result of the matching process;
根据所述返航图像、所述目标标记图像以及与所述目标标记图像关联的位姿信息确定所述第一位置的位置信息；Determining location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image;
根据所述第一位置的位置信息和所述返航路径调整飞行路径。The flight path is adjusted according to the position information of the first position and the return path.
在一个实施例中，所述处理器502用于丢失定位信号时，根据已记录的位置信息确定返航路径之前，还用于：In an embodiment, when the processor 502 is configured to lose the positioning signal, before determining the return path according to the recorded location information, the processor 502 is further configured to:
生成所述图像数据库，所述图像数据库中包括多个标记图像和与所述标记图像关联的位姿信息。The image database is generated, the image database including a plurality of marker images and pose information associated with the marker images.
在一个实施例中，所述处理器502生成图像数据库时，具体用于：In one embodiment, when the processor 502 generates an image database, it is specifically used to:
在所述无人机根据定位信号飞行时，记录飞行时的标记图像以及与所述飞行时的标记图像关联的位姿信息；Recording a marker image during flight and pose information associated with the marker image during flight when the drone is flying according to the positioning signal;
根据所述飞行时的标记图像以及与所述飞行时的标记图像关联的位姿信息生成图像数据库。An image database is generated based on the mark image at the time of flight and the pose information associated with the mark image at the time of flight.
在一个实施例中，所述返航路径包括多个返航点的位置信息；In one embodiment, the return path includes location information of a plurality of return points;
所述第一位置为与所述返航点的距离小于预置距离的点。The first position is a point where the distance from the return point is less than a preset distance.
在一个实施例中，所述返航路径是指：所述无人机规划的路径；In one embodiment, the return path refers to: the path planned by the drone;
所述飞行路径是指：所述无人机实际飞行的路径。The flight path refers to the path in which the drone actually flies.
在一个实施例中，所述处理器502根据所述第一位置的位置信息和所述返航路径调整飞行路径时，具体用于：In an embodiment, when the processor 502 adjusts the flight path according to the location information of the first location and the return path, the processor 502 is specifically configured to:
确定所述第一位置为与所述返航路径中的目标返航点的距离值；Determining that the first location is a distance value from a target return point in the return path;
根据所述距离值调整飞行路径，以使所述无人机飞行到所述目标返航点。The flight path is adjusted based on the distance value to cause the drone to fly to the target return point.
在一个实施例中，所述处理器502将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像时，具体用于：In an embodiment, the processor 502 performs matching processing on the return image with the mark image in the image database, and determines the target mark image according to the result of the matching process, specifically for:
确定所述返航图像的描述信息； Determining description information of the returning image;
根据所述返航图像的描述信息与图像数据库中的已记录的各个标记图像关联的描述信息，进行相似度比较；Performing a similarity comparison according to the description information of the returned image and the description information associated with each recorded mark image in the image database;
根据比较结果确定出目标标记图像，其中，该目标标记图像关联的描述信息与所述返航图像的描述信息满足预设相似度条件。Determining the target mark image according to the comparison result, wherein the description information associated with the target mark image and the description information of the return flight image satisfy a preset similarity condition.
在一个实施例中，所述处理器502记录各个标记图像关联的描述信息时，具体用于：In one embodiment, when the processor 502 records the description information associated with each of the tag images, it is specifically used to:
提取各个标记图像对应的图像特征，并确定所述图像特征对应的描述符；Extracting image features corresponding to the respective tag images, and determining descriptors corresponding to the image features;
根据所述描述符生成描述信息。Descriptive information is generated based on the descriptor.
在一个实施例中，所述处理器502根据所述描述符生成描述信息时，具体用于：In an embodiment, when the processor 502 generates the description information according to the descriptor, specifically:
确定所述描述符对应的权重值；Determining a weight value corresponding to the descriptor;
根据所述描述符以及所述描述符对应的权重值生成描述信息。Descriptive information is generated based on the descriptor and a weight value corresponding to the descriptor.
在一个实施例中，所述处理器502根据比较结果确定出目标标记图像时，具体用于：In one embodiment, when the processor 502 determines the target mark image according to the comparison result, it is specifically used to:
根据比较结果确定出第一标记图像；Determining a first mark image according to the comparison result;
将所述第一标记图像对应的深度图，与返航图像对应的深度图进行比较；Comparing the depth map corresponding to the first marker image with the depth map corresponding to the return image;
若比较结果为两幅深度图之间相匹配，则将所述第一标记图像作为目标标记图像。If the comparison result is that the two depth maps match, the first marker image is used as the target marker image.
在一个实施例中，所述处理器502将所述第一标记图像对应的深度图，与返航图像对应的深度图进行比较时，具体用于：In an embodiment, when the processor 502 compares the depth map corresponding to the first marker image with the depth map corresponding to the return image, the processor 502 is specifically configured to:
根据所述第一标记图像对应的深度图以及所述返航图像对应的深度图，得到两幅深度图之间的相对位姿信息；Obtaining relative pose information between the two depth maps according to the depth map corresponding to the first mark image and the depth map corresponding to the return image;
根据所述第一标记图像以及所述返航图像，得到两幅图像之间的相对旋转关系；Obtaining a relative rotation relationship between the two images according to the first mark image and the return image;
将所述相对旋转关系与所述相对位姿信息进行比较。The relative rotational relationship is compared to the relative pose information.
在一个实施例中，所述处理器502将所述返航图像与已记录的标记图像进行匹配处理时，具体用于：In an embodiment, when the processor 502 performs matching processing on the return image and the recorded mark image, the processor 502 is specifically configured to:
根据所述返航图像与已记录的标记图像之间的图像相似度进行匹配处理。A matching process is performed according to the image similarity between the returning image and the recorded mark image.
在一个实施例中，所述处理器502还用于： In an embodiment, the processor 502 is further configured to:
在按照所述返航路径返航时，检测所述定位信号；Detecting the positioning signal when returning according to the return path;
若检测到所述定位信号，则根据所述定位信号以及所述返航路径返航。If the positioning signal is detected, returning according to the positioning signal and the return path.
在一个实施例中，所述处理器502还用于：In an embodiment, the processor 502 is further configured to:
在按照所述返航路径返航时，所述无人机的飞行高度低于预置高度。When returning according to the return path, the flying height of the drone is lower than a preset height.
在一个实施例中，所述预置高度小于20米。In one embodiment, the preset height is less than 20 meters.
本发明实施例提供一种***。图6是本发明实施例提供的一种***的结构示意图。如图6所示，该***包括：摄像装置601和无人机602。Embodiments of the present invention provide a system. FIG. 6 is a schematic structural diagram of a system according to an embodiment of the present invention. As shown in FIG. 6, the system includes a camera 601 and a drone 602.
其中，该无人机602为上述本发明实施例中公开的无人机602，原理和实现方式均与上述实施例类似，此处不再赘述。The UAV 602 is the UAV 602 disclosed in the foregoing embodiment of the present invention, and the principles and implementations are similar to the foregoing embodiments, and details are not described herein again.
其中，该摄像装置601可以设置在该无人机562上，用于在飞行时拍摄用于标记位置的标记图像以及返航图像。The camera device 601 can be disposed on the drone 562 for capturing a marker image and a return image for marking a position during flight.
具体地，摄像装置60可通过云台或其他搭载设备搭载于无人机的主体上。摄像装置用于在无人机的飞行过程中进行图像或视频拍摄，包括但不限于多光谱成像仪、高光谱成像仪、可见光相机及红外相机、VO等，并且该摄像装置可以为一个或者一个以上。具体的，无人机可以在飞行时，实时控制摄像装置拍摄图像。Specifically, the imaging device 60 can be mounted on the main body of the drone by a pan/tilt or other mounted device. The camera device is used for image or video shooting during flight of the drone, including but not limited to multi-spectral imager, hyperspectral imager, visible light camera and infrared camera, VO, etc., and the camera device can be one or one the above. Specifically, the drone can control the camera to capture images in real time during flight.
其中，无人机602可以在丢失定位信号时，根据已记录的位置信息确定返航路径，在按照该返航路径返航时，在第一位置处采集返航图像，将该返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像，根据该返航图像、该目标标记图像以及与该目标标记图像关联的位姿信息确定该第一位置的位置信息，根据该第一位置的位置信息和该返航路径调整飞行路径。The drone 602 can determine the return path according to the recorded position information when the positioning signal is lost, and collect the return image at the first position when returning according to the return path, and mark the return image and the image database. Performing matching processing on the image, and determining a target mark image according to the result of the matching process, and determining position information of the first position according to the return image, the target mark image, and pose information associated with the target mark image, according to the first The location information of the location and the return path adjust the flight path.
需要说明的是，该无人机602可用于执行前述方法实施例所示的路径调整方法，其具体实现过程可参照该方法实施例，在此不作赘述。It should be noted that the UAV 602 can be used to perform the path adjustment method shown in the foregoing method embodiment, and the specific implementation process can refer to the method embodiment, and details are not described herein.
需要说明的是，对于前述的各个方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述的动作顺序的限制，因为依据本发明，某一些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优 选实施例，所涉及的动作和模块并不一定是本发明所必须的。It should be noted that, for the foregoing various method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. Secondly, those skilled in the art should also know that the embodiments described in the specification are excellent. In selected embodiments, the actions and modules involved are not necessarily required by the present invention.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。A person skilled in the art can understand that all or part of the steps of the foregoing embodiments can be completed by a program to instruct related hardware. The program can be stored in a computer readable storage medium, and the storage medium can include: Flash disk, Read-Only Memory (ROM), Random Access Memory (RAM), disk or optical disk.
以上对本发明实施例所提供的一种路径调整方法及无人机进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。 The path adjustment method and the UAV provided by the embodiments of the present invention are described in detail above. The principles and implementation manners of the present invention are described in the following. The description of the above embodiments is only used to help understanding. The method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and the scope of application. It is understood to be a limitation of the invention.

Claims (30)

  1. 一种路径调整方法，其特征在于，应用于无人机，所述方法包括：A path adjustment method is characterized in that it is applied to a drone, and the method includes:
    丢失定位信号时，根据已记录的位置信息确定返航路径；When the positioning signal is lost, the return path is determined according to the recorded position information;
    在按照所述返航路径返航时，在第一位置处采集返航图像；When returning according to the return path, collecting a returning image at the first position;
    将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像；Performing matching processing on the return image and the mark image in the image database, and determining the target mark image according to the result of the matching process;
    根据所述返航图像、所述目标标记图像以及与所述目标标记图像关联的位姿信息确定所述第一位置的位置信息；Determining location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image;
    根据所述第一位置的位置信息和所述返航路径调整飞行路径。The flight path is adjusted according to the position information of the first position and the return path.
  2. 根据权利要求1所述的方法，其特征在于，所述丢失定位信号时，根据已记录的位置信息确定返航路径之前，还包括：The method according to claim 1, wherein before the returning path is determined according to the recorded position information, the method further comprises:
    生成所述图像数据库，所述图像数据库中包括多个标记图像和与所述标记图像关联的位姿信息。The image database is generated, the image database including a plurality of marker images and pose information associated with the marker images.
  3. 根据权利要求2所述的方法，其特征在于，所述生成图像数据库，包括：The method of claim 2, wherein the generating an image database comprises:
    在所述无人机根据定位信号飞行时，记录飞行时的标记图像以及与所述飞行时的标记图像关联的位姿信息；Recording a marker image during flight and pose information associated with the marker image during flight when the drone is flying according to the positioning signal;
    根据所述飞行时的标记图像以及与所述飞行时的标记图像关联的位姿信息生成图像数据库。An image database is generated based on the mark image at the time of flight and the pose information associated with the mark image at the time of flight.
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述返航路径包括多个返航点的位置信息；The method according to any one of claims 1 to 3, wherein the return path includes position information of a plurality of return points;
    所述第一位置为与所述返航点的距离小于预置距离的点。The first position is a point where the distance from the return point is less than a preset distance.
  5. 根据权利要求4所述的方法，其特征在于，所述返航路径是指：所述无人机规划的路径；The method according to claim 4, wherein the return route refers to: a path planned by the drone;
    所述飞行路径是指：所述无人机实际飞行的路径。 The flight path refers to the path in which the drone actually flies.
  6. 根据权利要求1所述的方法，其特征在于，所述根据所述第一位置的位置信息和所述返航路径调整飞行路径，包括：The method according to claim 1, wherein the adjusting the flight path according to the position information of the first position and the return path comprises:
    确定所述第一位置为与所述返航路径中的目标返航点的距离值；Determining that the first location is a distance value from a target return point in the return path;
    根据所述距离值调整飞行路径，以使所述无人机飞行到所述目标返航点。The flight path is adjusted based on the distance value to cause the drone to fly to the target return point.
  7. 根据权利要求1所述的方法，其特征在于，所述将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像，包括：The method according to claim 1, wherein the matching the returning image with the marked image in the image database, and determining the target marking image according to the result of the matching processing comprises:
    确定所述返航图像的描述信息；Determining description information of the returning image;
    根据所述返航图像的描述信息与图像数据库中的已记录的各个标记图像关联的描述信息，进行相似度比较；Performing a similarity comparison according to the description information of the returned image and the description information associated with each recorded mark image in the image database;
    根据比较结果确定出目标标记图像，其中，该目标标记图像关联的描述信息与所述返航图像的描述信息满足预设相似度条件。Determining the target mark image according to the comparison result, wherein the description information associated with the target mark image and the description information of the return flight image satisfy a preset similarity condition.
  8. 根据权利要求7所述的方法，其特征在于，记录各个标记图像关联的描述信息，包括：The method according to claim 7, wherein the description information associated with each of the tag images is recorded, including:
    提取各个标记图像对应的图像特征，并确定所述图像特征对应的描述符；Extracting image features corresponding to the respective tag images, and determining descriptors corresponding to the image features;
    根据所述描述符生成描述信息。Descriptive information is generated based on the descriptor.
  9. 根据权利要求8所述的方法，其特征在于，所述根据所述描述符生成描述信息，包括：The method according to claim 8, wherein the generating the description information according to the descriptor comprises:
    确定所述描述符对应的权重值；Determining a weight value corresponding to the descriptor;
    根据所述描述符以及所述描述符对应的权重值生成描述信息。Descriptive information is generated based on the descriptor and a weight value corresponding to the descriptor.
  10. 根据权利要求7所述的方法，其特征在于，所述根据比较结果确定出目标标记图像，包括：The method according to claim 7, wherein the determining the target mark image according to the comparison result comprises:
    根据比较结果确定出第一标记图像；Determining a first mark image according to the comparison result;
    将所述第一标记图像对应的深度图，与返航图像对应的深度图进行比较；Comparing the depth map corresponding to the first marker image with the depth map corresponding to the return image;
    若比较结果为两幅深度图之间相匹配，则将所述第一标记图像作为目标标 记图像。If the comparison result is that the two depth maps match, the first marker image is used as the target Record the image.
  11. 根据权利要求10所述的方法，其特征在于，所述将所述第一标记图像对应的深度图，与返航图像对应的深度图进行比较，包括：The method according to claim 10, wherein comparing the depth map corresponding to the first marker image with the depth map corresponding to the return image comprises:
    根据所述第一标记图像对应的深度图以及所述返航图像对应的深度图，得到两幅深度图之间的相对位姿信息；Obtaining relative pose information between the two depth maps according to the depth map corresponding to the first mark image and the depth map corresponding to the return image;
    根据所述第一标记图像以及所述返航图像，得到两幅图像之间的相对旋转关系；Obtaining a relative rotation relationship between the two images according to the first mark image and the return image;
    将所述相对旋转关系与所述相对位姿信息进行比较。The relative rotational relationship is compared to the relative pose information.
  12. 根据权利要求1-11任一项所述的方法，其特征在于，所述将所述返航图像与已记录的标记图像进行匹配处理，包括：The method according to any one of claims 1 to 11, wherein the matching the returning image with the recorded marking image comprises:
    根据所述返航图像与已记录的标记图像之间的图像相似度进行匹配处理。A matching process is performed according to the image similarity between the returning image and the recorded mark image.
  13. 根据权利要求12所述的方法，其特征在于，所述方法还包括：The method of claim 12, wherein the method further comprises:
    在按照所述返航路径返航时，检测所述定位信号；Detecting the positioning signal when returning according to the return path;
    若检测到所述定位信号，则根据所述定位信号以及所述返航路径返航。If the positioning signal is detected, returning according to the positioning signal and the return path.
  14. 根据权利要求1-11所述的方法，其特征在于，所述无人机在按照所述返航路径返航时，所述无人机的飞行高度低于预置高度。The method according to any of claims 1-11, characterized in that, when the drone returns in accordance with the return path, the flying height of the drone is lower than a preset height.
  15. 根据权利要求1-11所述的方法，其特征在于，所述预置高度小于20米。The method of claims 1-11 wherein said preset height is less than 20 meters.
  16. 一种无人机，其特征在于，包括：存储器和处理器；An unmanned aerial vehicle, comprising: a memory and a processor;
    所述存储器，用于存储程序指令；The memory is configured to store program instructions;
    所述处理器，用于执行所述存储器存储的程序指令，当程序指令被执行时，用于：The processor is configured to execute the program instructions stored by the memory, when the program instructions are executed, for:
    丢失定位信号时，根据已记录的位置信息确定返航路径；When the positioning signal is lost, the return path is determined according to the recorded position information;
    在按照所述返航路径返航时，在第一位置处采集返航图像； When returning according to the return path, collecting a returning image at the first position;
    将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像；Performing matching processing on the return image and the mark image in the image database, and determining the target mark image according to the result of the matching process;
    根据所述返航图像、所述目标标记图像以及与所述目标标记图像关联的位姿信息确定所述第一位置的位置信息；Determining location information of the first location according to the returning image, the target marker image, and pose information associated with the target marker image;
    根据所述第一位置的位置信息和所述返航路径调整飞行路径。The flight path is adjusted according to the position information of the first position and the return path.
  17. 根据权利要求16所述的无人机，其特征在于，所述处理器用于丢失定位信号时，根据已记录的位置信息确定返航路径之前，还用于：The drone according to claim 16, wherein the processor is configured to: when the positioning signal is lost, before determining the return path according to the recorded position information,
    生成所述图像数据库，所述图像数据库中包括多个标记图像和与所述标记图像关联的位姿信息。The image database is generated, the image database including a plurality of marker images and pose information associated with the marker images.
  18. 根据权利要求17所述的无人机，其特征在于，所述处理器生成图像数据库时，具体用于：The drone according to claim 17, wherein when the processor generates an image database, it is specifically used to:
    在所述无人机根据定位信号飞行时，记录飞行时的标记图像以及与所述飞行时的标记图像关联的位姿信息；Recording a marker image during flight and pose information associated with the marker image during flight when the drone is flying according to the positioning signal;
    根据所述飞行时的标记图像以及与所述飞行时的标记图像关联的位姿信息生成图像数据库。An image database is generated based on the mark image at the time of flight and the pose information associated with the mark image at the time of flight.
  19. 根据权利要求16-18任一项所述的无人机，其特征在于，所述返航路径包括多个返航点的位置信息；The drone according to any one of claims 16 to 18, wherein the return path includes position information of a plurality of return points;
    所述第一位置为与所述返航点的距离小于预置距离的点。The first position is a point where the distance from the return point is less than a preset distance.
  20. 根据权利要求19所述的无人机，其特征在于，所述返航路径是指：所述无人机规划的路径；The drone according to claim 19, wherein the return path refers to: a path planned by the drone;
    所述飞行路径是指：所述无人机实际飞行的路径。The flight path refers to the path in which the drone actually flies.
  21. 根据权利要求19或20所述的无人机，其特征在于，所述处理器根据所述第一位置的位置信息和所述返航路径调整飞行路径时，具体用于：The UAV according to claim 19 or 20, wherein when the processor adjusts the flight path according to the position information of the first position and the return path, the method is specifically configured to:
    确定所述第一位置为与所述返航路径中的目标返航点的距离值； Determining that the first location is a distance value from a target return point in the return path;
    根据所述距离值调整飞行路径，以使所述无人机飞行到所述目标返航点。The flight path is adjusted based on the distance value to cause the drone to fly to the target return point.
  22. 根据权利要求16所述的无人机，其特征在于，所述处理器将所述返航图像与图像数据库中的标记图像进行匹配处理，并根据匹配处理的结果确定出目标标记图像时，具体用于：The drone according to claim 16, wherein the processor performs matching processing on the return image and the mark image in the image database, and determines the target mark image according to the result of the matching process, specifically to:
    确定所述返航图像的描述信息；Determining description information of the returning image;
    根据所述返航图像的描述信息与图像数据库中的已记录的各个标记图像关联的描述信息，进行相似度比较；Performing a similarity comparison according to the description information of the returned image and the description information associated with each recorded mark image in the image database;
    根据比较结果确定出目标标记图像，其中，该目标标记图像关联的描述信息与所述返航图像的描述信息满足预设相似度条件。Determining the target mark image according to the comparison result, wherein the description information associated with the target mark image and the description information of the return flight image satisfy a preset similarity condition.
  23. 根据权利要求22所述的无人机，其特征在于，所述处理器记录各个标记图像关联的描述信息时，具体用于：The drone according to claim 22, wherein when the processor records the description information associated with each of the tag images, it is specifically used to:
    提取各个标记图像对应的图像特征，并确定所述图像特征对应的描述符；Extracting image features corresponding to the respective tag images, and determining descriptors corresponding to the image features;
    根据所述描述符生成描述信息。Descriptive information is generated based on the descriptor.
  24. 根据权利要求23所述的无人机，其特征在于，所述处理器根据所述描述符生成描述信息时，具体用于：The drone according to claim 23, wherein when the processor generates the description information according to the descriptor, it is specifically used to:
    确定所述描述符对应的权重值；Determining a weight value corresponding to the descriptor;
    根据所述描述符以及所述描述符对应的权重值生成描述信息。Descriptive information is generated based on the descriptor and a weight value corresponding to the descriptor.
  25. 根据权利要求22所述的无人机，其特征在于，所述处理器根据比较结果确定出目标标记图像时，具体用于：The drone according to claim 22, wherein when the processor determines the target mark image according to the comparison result, it is specifically used for:
    根据比较结果确定出第一标记图像；Determining a first mark image according to the comparison result;
    将所述第一标记图像对应的深度图，与返航图像对应的深度图进行比较；Comparing the depth map corresponding to the first marker image with the depth map corresponding to the return image;
    若比较结果为两幅深度图之间相匹配，则将所述第一标记图像作为目标标记图像。If the comparison result is that the two depth maps match, the first marker image is used as the target marker image.
  26. 根据权利要求25所述的无人机，其特征在于，所述处理器将所述第一 标记图像对应的深度图，与返航图像对应的深度图进行比较时，具体用于：A drone according to claim 25, wherein said processor is said first The depth map corresponding to the mark image is compared with the depth map corresponding to the return image, and is specifically used for:
    根据所述第一标记图像对应的深度图以及所述返航图像对应的深度图，得到两幅深度图之间的相对位姿信息；Obtaining relative pose information between the two depth maps according to the depth map corresponding to the first mark image and the depth map corresponding to the return image;
    根据所述第一标记图像以及所述返航图像，得到两幅图像之间的相对旋转关系；Obtaining a relative rotation relationship between the two images according to the first mark image and the return image;
    将所述相对旋转关系与所述相对位姿信息进行比较。The relative rotational relationship is compared to the relative pose information.
  27. 根据权利要求16-26任一项所述的无人机，其特征在于，所述处理器将所述返航图像与已记录的标记图像进行匹配处理时，具体用于：The unmanned aerial vehicle according to any one of claims 16 to 26, wherein when the processor performs matching processing on the returning image and the recorded mark image, it is specifically used for:
    根据所述返航图像与已记录的标记图像之间的图像相似度进行匹配处理。A matching process is performed according to the image similarity between the returning image and the recorded mark image.
  28. 根据权利要求27所述的无人机，其特征在于，所述处理器还用于：The drone according to claim 27, wherein the processor is further configured to:
    在按照所述返航路径返航时，检测所述定位信号；Detecting the positioning signal when returning according to the return path;
    若检测到所述定位信号，则根据所述定位信号以及所述返航路径返航。If the positioning signal is detected, returning according to the positioning signal and the return path.
  29. 根据权利要求16-26所述的无人机，其特征在于，所述处理器还用于：The drone according to any one of claims 16-26, wherein the processor is further configured to:
    在按照所述返航路径返航时，所述无人机的飞行高度低于预置高度。When returning according to the return path, the flying height of the drone is lower than a preset height.
  30. 根据权利要求16-26所述的无人机，其特征在于，所述预置高度小于20米。 A drone according to any of claims 16-26, wherein said preset height is less than 20 meters.
PCT/CN2017/103808 2017-09-27 2017-09-27 Path adjustment method and unmanned aerial vehicle WO2019061111A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201780009908.0A CN108700892A (en) 2017-09-27 2017-09-27 A kind of path method of adjustment and unmanned plane
PCT/CN2017/103808 WO2019061111A1 (en) 2017-09-27 2017-09-27 Path adjustment method and unmanned aerial vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/103808 WO2019061111A1 (en) 2017-09-27 2017-09-27 Path adjustment method and unmanned aerial vehicle

Publications (1)

Publication Number Publication Date
WO2019061111A1 true WO2019061111A1 (en) 2019-04-04

Family

ID=63843838

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/103808 WO2019061111A1 (en) 2017-09-27 2017-09-27 Path adjustment method and unmanned aerial vehicle

Country Status (2)

Country Link
CN (1) CN108700892A (en)
WO (1) WO2019061111A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114442672A (en) * 2022-02-16 2022-05-06 中国电子科技集团公司第五十四研究所 Full-autonomous cruising method of unmanned aerial vehicle

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11055875B2 (en) * 2018-12-20 2021-07-06 The Boeing Company Computer-vision-based autonomous or supervised- autonomous landing of aircraft
CN110597297A (en) * 2019-10-21 2019-12-20 深圳市道通智能航空技术有限公司 Aircraft return control method and device, aircraft and storage medium
CN111367266A (en) * 2020-03-12 2020-07-03 北京三快在线科技有限公司 Unmanned equipment route adjusting method and device and unmanned equipment system
CN117991817A (en) * 2020-07-24 2024-05-07 深圳市大疆创新科技有限公司 Unmanned aerial vehicle flight control method and unmanned aerial vehicle
CN112783208A (en) * 2020-12-31 2021-05-11 广州极飞科技股份有限公司 Unmanned equipment return control method and device and unmanned equipment
CN113985910B (en) * 2021-09-09 2023-07-18 常州希米智能科技有限公司 Unmanned aerial vehicle control method and system combining mobile network and satellite positioning
CN115409437B (en) * 2022-11-03 2023-03-24 成都沃飞天驭科技有限公司 Low-altitude flight emergency processing method and device, aircraft and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105302043A (en) * 2015-11-17 2016-02-03 辽宁天行健航空科技有限公司 Safety control system and control method of unmanned plane
CN105786018A (en) * 2016-04-19 2016-07-20 清远市巨劲科技有限公司 Automatic return flight laser positioning system for unmanned plane and unmanned plane
CN105867181A (en) * 2016-04-01 2016-08-17 腾讯科技（深圳）有限公司 Control method and apparatus of unmanned aerial vehicle
CN105955291A (en) * 2016-04-29 2016-09-21 深圳市哈博森科技有限公司 Unmanned plane flight route track recording and automatic flight control mode
CN106352872A (en) * 2016-09-14 2017-01-25 北京理工大学 Autonomous navigation system for drone and navigation method of autonomous navigation system
CN106527481A (en) * 2016-12-06 2017-03-22 重庆零度智控智能科技有限公司 Unmanned aerial vehicle flight control method, device and unmanned aerial vehicle
US20170233071A1 (en) * 2016-02-15 2017-08-17 Skyyfish, LLC System and Method for Return-Home Command in Manual Flight Control

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104807456B (en) * 2015-04-29 2018-04-17 深圳市保千里电子有限公司 A kind of method maked a return voyage automatically during GPS no signals
CN106027896A (en) * 2016-06-20 2016-10-12 零度智控（北京）智能科技有限公司 Video photographing control device and method, and unmanned aerial vehicle
JP6020872B1 (en) * 2016-06-24 2016-11-02 株式会社アドインテ Analysis system and analysis method
CN106774402A (en) * 2016-12-28 2017-05-31 湖南省道通科技有限公司 The method and device positioned to unmanned plane
CN106774404B (en) * 2016-12-29 2020-06-26 哈密创动科技有限公司 Unmanned aerial vehicle return control method and device and unmanned aerial vehicle
CN107077153A (en) * 2017-01-22 2017-08-18 深圳市大疆创新科技有限公司 Control method, control system and the movable fixture of movable fixture
CN106873619B (en) * 2017-01-23 2021-02-02 上海交通大学 Processing method of flight path of unmanned aerial vehicle
CN107065925B (en) * 2017-04-01 2020-04-07 成都通甲优博科技有限责任公司 Unmanned aerial vehicle return method and device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105302043A (en) * 2015-11-17 2016-02-03 辽宁天行健航空科技有限公司 Safety control system and control method of unmanned plane
US20170233071A1 (en) * 2016-02-15 2017-08-17 Skyyfish, LLC System and Method for Return-Home Command in Manual Flight Control
CN105867181A (en) * 2016-04-01 2016-08-17 腾讯科技（深圳）有限公司 Control method and apparatus of unmanned aerial vehicle
CN105786018A (en) * 2016-04-19 2016-07-20 清远市巨劲科技有限公司 Automatic return flight laser positioning system for unmanned plane and unmanned plane
CN105955291A (en) * 2016-04-29 2016-09-21 深圳市哈博森科技有限公司 Unmanned plane flight route track recording and automatic flight control mode
CN106352872A (en) * 2016-09-14 2017-01-25 北京理工大学 Autonomous navigation system for drone and navigation method of autonomous navigation system
CN106527481A (en) * 2016-12-06 2017-03-22 重庆零度智控智能科技有限公司 Unmanned aerial vehicle flight control method, device and unmanned aerial vehicle

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114442672A (en) * 2022-02-16 2022-05-06 中国电子科技集团公司第五十四研究所 Full-autonomous cruising method of unmanned aerial vehicle

Also Published As

Publication number Publication date
CN108700892A (en) 2018-10-23

Similar Documents

Publication Publication Date Title
WO2019061111A1 (en) Path adjustment method and unmanned aerial vehicle
US20230236611A1 (en) Unmanned Aerial Vehicle Sensor Activation and Correlation System
US20210109548A1 (en) Heading generation method and system of unmanned aerial vehicle
CN109765930B (en) Unmanned aerial vehicle vision navigation
CN105790155B (en) A kind of autonomous cruising inspection system of power transmission line unmanned machine and method based on differential GPS
CN109324337B (en) Unmanned aerial vehicle route generation and positioning method and device and unmanned aerial vehicle
CN109596118B (en) Method and equipment for acquiring spatial position information of target object
CN109002055B (en) High-precision automatic inspection method and system based on unmanned aerial vehicle
CN108702448B (en) Unmanned aerial vehicle image acquisition method, unmanned aerial vehicle and computer readable storage medium
WO2017075964A1 (en) Unmanned aerial vehicle photographing control method, unmanned aerial vehicle photographing method, mobile terminal and unmanned aerial vehicle
KR101329583B1 (en) Air observations using the rotor structure construction method and system for terrain data
WO2020103108A1 (en) Semantic generation method and device, drone and storage medium
WO2018121247A1 (en) Method and device for controlling flight of unmanned aerial vehicle, and unmanned aerial vehicle
CN106292126B (en) A kind of intelligence aerial survey flight exposal control method, unmanned aerial vehicle (UAV) control method and terminal
WO2020103109A1 (en) Map generation method and device, drone and storage medium
WO2018209898A1 (en) Information processing device, aerial photographing path generation method, aerial photographing path generation system, program and recording medium
US20240179398A1 (en) Method, control apparatus and control system for controlling an image capture of movable device
JPWO2018198281A1 (en) Information processing device, aerial photography route generation method, aerial photography route generation system, program, and recording medium
CN115861860B (en) Target tracking and positioning method and system for unmanned aerial vehicle
US20230359204A1 (en) Flight control method, video editing method, device, uav and storage medium
CN117036989A (en) Miniature unmanned aerial vehicle target recognition and tracking control method based on computer vision
CN114689030A (en) Unmanned aerial vehicle auxiliary positioning method and system based on airborne vision
WO2020024211A1 (en) Unmanned aerial vehicle landing method and apparatus, and unmanned aerial vehicle
WO2020006732A1 (en) Unmanned aerial vehicle landing method and apparatus, and unmanned aerial vehicle
CN117950422B (en) Unmanned aerial vehicle inspection system and inspection method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17927677

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17927677

Country of ref document: EP

Kind code of ref document: A1