CN110703802A - Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation - Google Patents
Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation Download PDFInfo
- Publication number
- CN110703802A CN110703802A CN201911065609.6A CN201911065609A CN110703802A CN 110703802 A CN110703802 A CN 110703802A CN 201911065609 A CN201911065609 A CN 201911065609A CN 110703802 A CN110703802 A CN 110703802A
- Authority
- CN
- China
- Prior art keywords
- bridge
- unmanned aerial
- aerial vehicle
- task
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 165
- 238000012545 processing Methods 0.000 claims abstract description 92
- 201000010099 disease Diseases 0.000 claims abstract description 65
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 65
- 238000000034 method Methods 0.000 claims abstract description 45
- 238000005457 optimization Methods 0.000 claims abstract description 14
- 238000005516 engineering process Methods 0.000 claims abstract description 10
- 238000004891 communication Methods 0.000 claims description 31
- 230000002159 abnormal effect Effects 0.000 claims description 23
- 238000004458 analytical method Methods 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 14
- 238000007726 management method Methods 0.000 claims description 13
- 238000012216 screening Methods 0.000 claims description 10
- 230000005856 abnormality Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 6
- 230000009191 jumping Effects 0.000 claims description 6
- BULVZWIRKLYCBC-UHFFFAOYSA-N phorate Chemical compound CCOP(=S)(OCC)SCSCC BULVZWIRKLYCBC-UHFFFAOYSA-N 0.000 claims description 3
- 238000005067 remediation Methods 0.000 claims 1
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 5
- 239000000725 suspension Substances 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 229910000831 Steel Inorganic materials 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003707 image sharpening Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 238000002922 simulated annealing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Traffic Control Systems (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention belongs to the technical field of unmanned aerial vehicles and bridge detection, and particularly relates to a bridge automatic detection method and system based on multi-unmanned aerial vehicle cooperative operation, aiming at solving the problems that the traditional bridge detection operation is difficult, and a single unmanned aerial vehicle consumes long time and has high error rate. The method comprises the following steps: analyzing bridge environment information and dividing a bridge detection task into a plurality of subtasks; acquiring a task grouping optimization function and dividing a subtask group; respectively planning the unmanned aerial vehicle tracks corresponding to the subtask groups, and distributing the subtask groups to corresponding unmanned aerial vehicles; and transmitting the picture shot by the unmanned aerial vehicle to an image processing unit to extract bridge disease characteristics, and generating a bridge disease report. According to different bridge tasks, different shooting schemes are designed, bridge detection is carried out through mutual cooperation of multiple unmanned aerial vehicles, each unmanned aerial vehicle simultaneously executes different subtasks, and a redundancy fault-tolerant technology is adopted for exception remedy, so that the method is high in efficiency, high in task continuity and integrity, high in precision and low in error rate.
Description
Technical Field
The invention belongs to the technical field of unmanned aerial vehicles and bridge detection, and particularly relates to an automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation.
Background
At the present stage, unmanned aerial vehicle technology has penetrated into a plurality of fields such as agriculture, electric power, traffic and the like, and its targeted application has become one of the important subjects of the current research. In the field of bridge detection, the current bridge management and maintenance department mainly adopts modes such as artificial visual detection, telescope detection and the like. In the conventional detection process, the uniqueness of the detection equipment is a significant problem. When the bridge is in a special structure bridge such as a suspension bridge or a long-span high-pier bridge, the traditional bridge detection mode has the limitations of high operation difficulty, high danger and the like for places which are difficult to touch beyond a certain range, such as high tower columns, bridge bottom plates, tower top structures, cable-stayed steel cables and the like.
At the present stage, unmanned aerial vehicle technology has penetrated into a plurality of fields such as agriculture, electric power, traffic and the like, and its targeted application has become one of the important subjects of the current research. In the field of bridge detection, the current bridge management and maintenance department mainly adopts modes such as artificial visual detection, telescope detection and the like. In the conventional detection process, the uniqueness of the detection equipment is a significant problem. When the bridge is in a special structure bridge such as a suspension bridge or a long-span high-pier bridge, the traditional bridge detection mode has the limitations of high operation difficulty, high danger and the like for places which are difficult to touch beyond a certain range, such as high tower columns, bridge bottom plates, tower top structures, cable-stayed steel cables and the like.
Because the complexity of bridge detection task and the limitation of unmanned aerial vehicle performance, single unmanned aerial vehicle has shown certain limitation in the ability of carrying out the task, specifically includes: due to the limitation of an airborne sensor and communication equipment, the sensing capability of a single unmanned aerial vehicle to the task environment is limited, and the task environment is difficult to be grasped in all directions; the unmanned aerial vehicle has limited cruising ability due to the limitation of self power storage equipment and does not have the ability of high-strength continuous operation; once the single unmanned aerial vehicle is affected by abnormal faults, the task execution efficiency is greatly reduced, even the task is terminated, and the fault tolerance is poor.
Generally speaking, the traditional bridge detection operation difficulty is big, with high costs, and single unmanned aerial vehicle bridge detection is long consuming time, the fault tolerance is poor, the limited error rate that leads to of acquisition information is high, with high costs.
Disclosure of Invention
In order to solve the problems in the prior art, namely the problems that the traditional bridge detection operation is high in difficulty and cost, and the bridge detection of a single unmanned aerial vehicle is long in time consumption, high in error rate and high in cost, the invention provides an automatic bridge detection method based on the cooperative operation of multiple unmanned aerial vehicles, which comprises the following steps:
step S10, according to the obtained bridge model to be detected, analyzing bridge environment information and dividing a bridge detection task into a plurality of bridge detection subtasks; the bridge detection subtask is an unmanned aerial vehicle hovering shooting point;
step S20, acquiring a task clustering optimization function based on the plurality of bridge detection subtasks and each unmanned aerial vehicle executing the task;
step S30, dividing the bridge detection subtasks into a set number of subtask groups based on the task grouping optimization function, and acquiring the grade of each subtask group according to the preset importance of each subtask group;
step S40, respectively planning the flight path and the flight path grade of the unmanned aerial vehicle corresponding to each subtask group based on the acquired parameters of each unmanned aerial vehicle and the grade of each subtask group, and distributing each subtask group to the corresponding unmanned aerial vehicle;
step S50, each unmanned aerial vehicle carries out bridge detection shooting according to the corresponding unmanned aerial vehicle track and track grade, and transmits the shot picture to the corresponding image processing unit according to the input image processing unit selection information;
and step S60, extracting bridge disease features corresponding to the pictures shot by the unmanned aerial vehicle through a bridge automatic detection image processing algorithm preset by the image processing unit, screening the bridge disease features by combining a preset threshold value, and generating a bridge disease report after obtaining effective bridge disease information.
In some preferred embodiments, the task clustering optimization function is:
JA＝ω2λ+ω3η-ω1J
wherein J is a task group distribution index, lambda is a task quantity balance index, eta is an unmanned aerial vehicle cruise time balance index, and omega1、ω2、ω3The weights are respectively a task group distribution index, a task quantity balance index and an unmanned aerial vehicle cruising time balance index.
In some preferred embodiments, the task group distribution index is:
wherein A isiRepresents the ith sub-task group in the set number of sub-task groups, S represents the number of sub-task groups,
represents AiT represents a bridge detection subtask,
representing bridge detection subtasks T and AiCluster center of
The euclidean distance of (c).
In some preferred embodiments, the task number balance index is:
wherein, muSkThe kth unmanned aerial vehicle representing the S-th sub-task group executes the number of sub-tasks, S represents the number of sub-task groups, and M represents the number of sub-tasks.
In some preferred embodiments, the unmanned aerial vehicle cruise time balance index is:
wherein S represents the number of subtask groups, ψi(i ═ 1,2, …, S) represents the optimal cruising path of the drone for the ith subtask group, t (ψ)i) And the cruising shooting time required by the optimal cruising path of the unmanned plane representing the ith subtask group.
In some preferred embodiments, step S50 shows "the pre-established communication network" is one or more of Wi-Fi, ZigBee and GPRS.
In some preferred embodiments, the bridge automatic detection method further includes a step of remedying the abnormality of the unmanned aerial vehicle, and the method includes:
and setting the abnormal times threshold of each unmanned aerial vehicle through redundancy fault-tolerant technology, if the abnormal times of any one unmanned aerial vehicle is greater than the threshold, descending the unmanned aerial vehicle and sending abnormal descending information, wherein the uncompleted subtasks are used as a subtask group, and after the unmanned aerial vehicle flight path planning is carried out again, the uncompleted subtasks are distributed to standby unmanned aerial vehicles for execution.
On the other hand, the invention provides an automatic bridge detection system based on multi-unmanned aerial vehicle cooperative operation, which comprises a ground control station and a plurality of bridge detection execution ends;
the ground control station comprises a central control unit, a network management unit, an image processing unit, a bridge model analysis unit, a task allocation unit, a flight scheduling unit and an information display unit, and is configured to analyze bridge environment information and divide a bridge detection task into a plurality of bridge detection subtasks according to an acquired bridge model to be detected, allocate each subtask group to each bridge detection execution end after combining each unmanned aerial vehicle executing the task to perform task grouping, unmanned aerial vehicle track planning and track grade division, receive photos sent by the bridge detection execution ends, extract bridge disease characteristics through a preset bridge automatic detection image processing algorithm, screen the bridge disease characteristics by combining a preset threshold value, and generate a bridge disease report after acquiring effective bridge disease information;
the bridge detection execution end comprises a micro-processing module, a flight power module, a communication module, an image acquisition module, a positioning module and an obstacle avoidance module, and is configured to carry out bridge detection shooting according to the subtask group distributed by the ground control station and transmit a shot picture to a corresponding image processing unit according to input image processing unit selection information.
In some preferred embodiments, the central control unit includes a signal receiving module, a signal processing module, and an instruction transceiving module, and is configured to acquire states of the units of the bridge detection execution ends and transmit data information between the units of the ground control station.
In some preferred embodiments, the bridge model analysis unit comprises a bridge model storage module, a bridge model analysis module;
the bridge model storage module is configured to store a relational database or a non-relational database of bridge models of various categories;
the bridge analysis module is configured to analyze bridge environment information and divide a bridge detection task into a plurality of bridge detection subtasks according to a bridge model to be detected.
In some preferred embodiments, the ground control station is further provided with an exception handling unit;
the abnormity processing unit is configured to receive abnormal landing information sent by the unmanned aerial vehicle with the abnormal times larger than a set threshold value, and the central control unit controls the task distribution unit and the flight scheduling unit to distribute the standby unmanned aerial vehicle to execute the bridge detection task.
In some preferred embodiments, the positioning module adopts one or more of GPS positioning, IMU positioning, compass positioning and altimeter positioning.
In some preferred embodiments, the bridge detection execution end is further provided with an image processing module;
the image processing module is used for extracting bridge disease characteristics of the photos shot by the unmanned aerial vehicle in the corresponding bridge detection execution end in real time through a preset bridge automatic detection image processing algorithm, screening the bridge disease characteristics by combining a preset threshold value, and generating a bridge disease report after obtaining effective bridge disease information.
In some preferred embodiments, the obstacle avoidance module adopts one or more of laser radar obstacle avoidance, binocular vision obstacle avoidance, ultrasonic ranging obstacle avoidance, and infrared ranging obstacle avoidance.
In some preferred embodiments, the obstacle avoidance module is provided with a method for controlling collision avoidance of the unmanned aerial vehicle, and the method includes:
step B10, in the flight process of the current unmanned aerial vehicle, acquiring the flight positions of other unmanned aerial vehicles in real time, and calculating the distance delta L between the current unmanned aerial vehicle and other unmanned aerial vehicles in real timet；
Step B20, judging whether there is delta LtLess than or equal to L0 and
if the judgment result is yes, jumping to step B30; otherwise, continue step B20; wherein L is0Is a set safe distance;
step B30, comparing the flight path grades corresponding to the two unmanned aerial vehicles at the collision edge, and if the flight path grade of the current unmanned aerial vehicle is high, continuing to execute the task according to the flight path of the current unmanned aerial vehicle; otherwise, jumping to step B40;
step B40, hovering the current unmanned aerial vehicle until delta Lt＞L0Currently, the drone continues to perform tasks according to its flight path.
In some preferred embodiments, the method for operating the automatic bridge detection system includes:
step A10, the bridge model analysis unit analyzes the bridge environment information and divides the bridge task, and transmits the result to the central control unit, and the central control unit sends the work order and the analysis result to the task allocation unit;
step A20, the task allocation unit carries out sub-task grouping, unmanned aerial vehicle track planning and unmanned aerial vehicle track grading, and returns grouping, track planning and track grading results to the central control unit, and the central control unit transmits the results and the working instructions to the flight scheduling unit;
step A30, the flight scheduling unit allocates the subtask group to each unmanned aerial vehicle according to the performance index of each unmanned aerial vehicle, generates scheduling and flight instructions, and transmits the scheduling and flight instructions to the central control unit;
step A40, the central control unit dispatches the network management unit to establish a communication network, the execution end unmanned aerial vehicle set joins the network, the network management unit transmits the flight shooting instruction to the corresponding unmanned aerial vehicle, and monitors the network communication state;
step A50, the unmanned aerial vehicle receives the flight instruction through the communication module and transmits the flight instruction to the micro-processing module;
step A60, the micro-processing module sends a flight instruction to the flight power module, so that the unmanned aerial vehicle flies according to the planned path; the micro-processing module sends a shooting instruction to the image acquisition module, and the image acquisition module shoots the bridge image;
step A70, in the flight process of the unmanned aerial vehicle, the microprocessing module judges whether the unmanned aerial vehicle is abnormal in real time, if the abnormal times exceed the set threshold, the step A20 is returned; otherwise, go to step A80;
step A80, the microprocessor module obtains the real-time position of other unmanned aerial vehicles, calculates the distance delta L between the other unmanned aerial vehiclestAnd determining whether Δ L is presentt≤L0And is
If yes, further judging whether the level of the local computer is lower, if yes, hovering the local computer, and returning to the step A80; otherwise, go to step A90;
step A90, when the unmanned aerial vehicle acquires one image, the image processing unit of the ground workstation or the image processing module on the unmanned aerial vehicle extracts the image bridge disease characteristics;
step A100, a micro-processing module judges whether an unmanned aerial vehicle image acquisition task is finished, if so, step A110 is executed; otherwise, return to step A60;
and A110, screening the disease information according to the bridge disease characteristics, and generating a disease report from the effective disease information.
The invention has the beneficial effects that:
(1) according to the automatic bridge detection method based on multi-unmanned aerial vehicle cooperative operation, bridge detection is performed through mutual cooperation of the multiple unmanned aerial vehicles, a complete bridge detection task is split into a set of multiple subtasks, and each unmanned aerial vehicle in the unmanned aerial vehicle cluster executes different subtasks simultaneously, so that the task execution efficiency of the unmanned aerial vehicles is improved, and the bridge detection duration is shortened.
(2) According to the automatic bridge detection method based on multi-unmanned aerial vehicle cooperative operation, an unmanned aerial vehicle cluster system does not depend on an individual, the redundancy fault-tolerant technology is adopted to ensure that the unmanned aerial vehicle can still normally work after the occurrence of the abnormality within the set times, and when the abnormality exceeds the set times, the system can call other individuals to make up the abnormality, so that the continuity and the integrity of task execution are ensured, and the error rate is reduced.
(3) The automatic bridge detection method based on the multi-unmanned aerial vehicle cooperative operation can adopt a corresponding image processing scheme according to the speed and precision required to be detected, and the real-time performance and the accuracy are balanced according to the requirements of bridge detection tasks.
(4) The bridge automatic detection system based on the cooperative operation of the multiple unmanned aerial vehicles has the advantages of simple structure, light volume, good robustness and suitability for various bridge detection tasks and unmanned aerial vehicles.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a schematic flow chart of an automatic bridge detection method based on multi-UAV cooperative operation according to the present invention;
FIG. 2 is a schematic diagram of a ground control station framework of a bridge detection system according to an embodiment of the method for automatically detecting a bridge based on cooperative operation of multiple unmanned aerial vehicles;
FIG. 3 is a schematic diagram of a bridge detection execution end framework of a bridge detection system according to an embodiment of the bridge automatic detection method based on multi-UAV cooperative operation of the present invention;
fig. 4 is a schematic diagram of a bridge detection process of an embodiment of the bridge automatic detection method based on multi-unmanned aerial vehicle cooperative work of the present invention.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
The invention discloses an automatic bridge detection method based on multi-unmanned aerial vehicle cooperative operation, which comprises the following steps:
step S10, according to the obtained bridge model to be detected, analyzing bridge environment information and dividing a bridge detection task into a plurality of bridge detection subtasks; the bridge detection subtask is an unmanned aerial vehicle hovering shooting point;
step S20, acquiring a task clustering optimization function based on the plurality of bridge detection subtasks and each unmanned aerial vehicle executing the task;
step S30, dividing the bridge detection subtasks into a set number of subtask groups based on the task grouping optimization function, and acquiring the grade of each subtask group according to the preset importance of each subtask group;
step S40, respectively planning the flight path and the flight path grade of the unmanned aerial vehicle corresponding to each subtask group based on the acquired parameters of each unmanned aerial vehicle and the grade of each subtask group, and distributing each subtask group to the corresponding unmanned aerial vehicle;
step S50, each unmanned aerial vehicle carries out bridge detection shooting according to the corresponding unmanned aerial vehicle track and track grade, and transmits the shot picture to the corresponding image processing unit according to the input image processing unit selection information;
and step S60, extracting bridge disease features corresponding to the pictures shot by the unmanned aerial vehicle through a bridge automatic detection image processing algorithm preset by the image processing unit, screening the bridge disease features by combining a preset threshold value, and generating a bridge disease report after obtaining effective bridge disease information.
In order to more clearly describe the method for automatically detecting a bridge based on cooperative operation of multiple unmanned aerial vehicles, the following describes in detail the steps in the embodiment of the method of the present invention with reference to fig. 1.
The bridge automatic detection method based on the multi-unmanned aerial vehicle cooperative work comprises the steps of S10-S60, and the steps are described in detail as follows:
step S10, according to the obtained bridge model to be detected, analyzing bridge environment information and dividing a bridge detection task into a plurality of bridge detection subtasks; the bridge detection subtask is an unmanned aerial vehicle hovering shooting point.
In order to ensure that all-dimensional multi-angle shooting is carried out on the diseases of the bridge, bridge detection subtasks are divided according to specific bridge detection tasks and bridge types, and the three conditions are divided:
first, bridge deck sections, conical slopes, beam/abutment exteriors, etc. are tested: the front-view, back-view, left-view, right-view and down-view pictures in the target range are respectively shot.
Secondly, carrying out local key detection on the upper structures of the arch bridge, the suspension bridge and the cable-stayed bridge: and (5) flying around the local detection part to obtain a multi-view picture.
Thirdly, locally and mainly detecting the structural diseases below the bridge deck: local illumination is carried out by using the illuminating equipment, and a multi-view picture of the bridge substructure is shot in a rotating holder mode.
In the embodiment of the invention, the bridge detection task is recorded as T ═ T1,T2,…,TM}，TMThe suspension point shot by the unmanned aerial vehicle is a subtask, and M is the number of the subtasks. The set of drones performing bridge detection is denoted as V ═ V1,V2,…,VNAnd N is the number of unmanned aerial vehicles for executing bridge detection. The set after dividing the subtasks is recorded as A ═ A1,A2,…,ASAnd S is the number of subtask groups. The unmanned plane set division formation is marked as F ═ F1,F2,…,FKAnd S is the formation number of the unmanned aerial vehicles.
Step S20, based on the plurality of bridge detection subtasks and each unmanned aerial vehicle that executes the task, obtaining a task clustering optimization function, as shown in formula (1):
JA＝ω2λ+ω3η-ω1j type (1)
Wherein J is a task group distribution index, lambda is a task quantity balance index, eta is an unmanned aerial vehicle cruise time balance index, and omega1、ω2、ω3The weights are respectively a task group distribution index, a task quantity balance index and an unmanned aerial vehicle cruising time balance index.
The task group distribution index is shown as formula (2):
wherein A isiRepresents the ith sub-task group in the set number of sub-task groups, S represents the number of sub-task groups,
represents AiT represents a bridge detection subtask,
representing bridge detection subtasks T and AiCluster center of
The euclidean distance of (c).
The task quantity balance index is shown as formula (3):
wherein, muSkThe kth unmanned aerial vehicle representing the S-th sub-task group executes the number of sub-tasks, S represents the number of sub-task groups, and M represents the number of sub-tasks.
The task number balance index represents the task number balance rate, and the closer lambda to 1, the more balanced the task number.
The unmanned aerial vehicle cruise time balance index is as shown in formula (4):
wherein S representsNumber of subtask groups, ψi(i ═ 1,2, …, S) represents the optimal cruising path of the drone for the ith subtask group, t (ψ)i) And the cruising shooting time required by the optimal cruising path of the unmanned plane representing the ith subtask group.
A closer η to 1 indicates more uniform cruise time.
And step S30, dividing the bridge detection subtasks into a set number of subtask groups based on the task grouping optimization function, and acquiring the grade of each subtask group according to the preset importance of each subtask group.
And step S40, respectively planning the flight path and the flight path grade of the unmanned aerial vehicle corresponding to each subtask group based on the acquired parameters of each unmanned aerial vehicle and the grade of each subtask group, and distributing each subtask group to the corresponding unmanned aerial vehicle.
And taking the task clustering optimization function as a target function, and seeking a global optimal solution by using target optimization algorithms such as a K-means clustering algorithm improved by a simulated annealing algorithm and the like to realize task clustering and track planning. The task clustering index of the method of the invention is only an example, and for different tasks, a reasonable task clustering index is determined according to specific requirements, and the invention is not detailed herein. The flight path planning comprises planning of a return flight path of the unmanned aerial vehicle, and the optimization of the unmanned aerial vehicle for executing tasks and the return flight total travel is guaranteed.
And step S50, each unmanned aerial vehicle carries out bridge detection shooting according to the corresponding unmanned aerial vehicle track and track grade, and the shot picture is transmitted to the corresponding image processing unit according to the input image processing unit selection information.
The unmanned aerial vehicle cooperative control method can select a centralized or distributed control structure according to specific requirements. The centralized control structure utilizes the global information to perform analysis and decision, can better grasp the global information to realize reasonable scheduling, and has higher requirements on the performance of communication bandwidth and the processing speed and reliability of the control center. The positions of all unmanned aerial vehicles in the cluster system of the distributed control structure are equal, tasks are completed cooperatively in a cooperative mode, meanwhile, the distributed control structure has certain autonomous control and decision-making capacity, information interaction can be carried out with other unmanned aerial vehicles according to the communication condition of a topological network, and the integral control of an unmanned aerial vehicle cluster system is achieved.
The unmanned aerial vehicle shoots the in-process and needs to carry out local location, can adopt GPS location or SLAM location technique. The GPS positioning is only suitable for bridge deck, conical slope and other parts which are not seriously shielded by barriers, and the SLAM positioning technology can realize the positioning of parts with low GPS signal reliability, such as bridge bottom and the like. SLAM localization techniques may optionally employ one or more of a monocular camera, a binocular camera, a depth camera, a lidar, and an inertial measurement unit. The method comprises the following steps of detailing a plurality of unmanned aerial vehicle positioning steps by using an SLAM positioning technology:
firstly, collecting environmental information: the data of various types of sensors can be integrated through the multi-type sensors to make up the defects of the respective data and improve the robustness and the precision of the SLAM;
secondly, extracting environmental features: determining the corresponding relation between the sequence sensor information and the real environment;
thirdly, pose estimation: estimating and correcting pose information by adopting algorithms such as particle filtering and the like;
fourthly, updating the map: constructing a local environment map according to the correction pose information, and storing the local environment map in a map database to realize local map updating;
fifthly, local map feature extraction: determining the corresponding relation between a local map and a global environment;
sixthly, map fusion: and fusing the local maps with certain overlapping degree into a global map by adopting a preset algorithm.
Local map data of each unmanned aerial vehicle are mutually transmitted by adopting a radio wave technology, and different transmission frequency bands, such as 2.4G, 5G and the like, can be selected according to requirements. The frequency of the 5G transmission frequency band is high, the number of carried data is large, and the interference is low.
The communication mode of the pre-established communication network is one or more of Wi-Fi, ZigBee and GPRS.
And step S60, extracting bridge disease features corresponding to the pictures shot by the unmanned aerial vehicle through a bridge automatic detection image processing algorithm preset by the image processing unit, screening the bridge disease features by combining a preset threshold value, and generating a bridge disease report after obtaining effective bridge disease information.
The image processing can be carried out in two modes:
the mode I is that each unmanned aerial vehicle transmits the acquired image and the shooting position to the image processing unit, and the image processing unit performs centralized processing through a preset bridge automatic detection image processing algorithm. The method has higher requirement on communication performance and heavier communication load. But the overall and precision of the image processing is high.
And in the second mode, each unmanned aerial vehicle respectively carries out real-time processing on the images shot by the unmanned aerial vehicle through the image processing module of the unmanned aerial vehicle, and transmits the processing result to the image processing unit. This method has a light communication load, but the accuracy of image processing may be affected, and the global property is difficult to grasp.
In one embodiment of the present invention, the image processing steps are:
first, image acquisition represents: converting the analog image signal into a digital form and representing the digital image;
and secondly, enhancing the image: changing the visual quality of the image, highlighting important features of the image, and comprising the steps of gray level conversion of the image, image sharpening, image edge processing, noise processing of the image, histogram correction and the like;
thirdly, image segmentation: segmenting image features into a plurality of meaningful regions;
fourthly, image analysis: the object features can be extracted and represented using a deep learning approach. For example, the outline trends of the edges of the cracks and the pits can be recorded by adopting a chain code tracking method, and the disease data information can be determined through further calculation. The disease information includes: the length and width of the crack, and the damaged area and depth of the pit.
The bridge automatic detection method is also provided with a step of remedying the abnormality of the unmanned aerial vehicle, and the method comprises the following steps:
and setting the abnormal times threshold of each unmanned aerial vehicle through redundancy fault-tolerant technology, if the abnormal times of any one unmanned aerial vehicle is greater than the threshold, descending the unmanned aerial vehicle and sending abnormal descending information, wherein the uncompleted subtasks are used as a subtask group, and after the unmanned aerial vehicle flight path planning is carried out again, the uncompleted subtasks are distributed to standby unmanned aerial vehicles for execution.
The bridge automatic detection system based on the multi-unmanned aerial vehicle cooperative operation of the second embodiment of the invention comprises a ground control station and a plurality of bridge detection execution ends;
as shown in fig. 2, a schematic diagram of a ground control station framework of a bridge detection system according to an embodiment of the method for automatically detecting a bridge based on cooperative work of multiple unmanned aerial vehicles of the present invention includes a central control unit, a network management unit, an image processing unit, a bridge model analysis unit, a task allocation unit, a flight scheduling unit, an information display unit, and an exception handling unit.
The ground control station analyzes bridge environment information and divides bridge detection tasks into a plurality of bridge detection subtasks according to the acquired bridge model to be detected, and distributes each subtask group to each bridge detection execution end and receives photos sent by the bridge detection execution end after combining each unmanned aerial vehicle executing the tasks to perform task grouping, unmanned aerial vehicle track planning and track grade division, extracts bridge disease characteristics through a preset bridge automatic detection image processing algorithm, screens the bridge disease characteristics by combining a preset threshold value, and generates a bridge disease report after acquiring effective bridge disease information.
And the central control unit comprises a signal receiving module, a signal processing module and an instruction transceiving module and is used for acquiring the states of all the units of the plurality of bridge detection execution ends and transmitting data information among all the units of the ground control station.
The central control unit is located at a central position in a system of the whole ground control station, monitors and controls the whole operation condition of the system, monitors the health state information of all modules (including a flight control module, a communication module, an image acquisition module and the like) on each unmanned aerial vehicle at the bridge detection execution end, such as battery capacity, sensor abnormity, power system abnormity and the like, and is also a bridge for data communication among all units in the system.
The network management unit is used for establishing the unmanned aerial vehicle communication local area network. In addition, the system is also used for monitoring and maintaining the communication states among all the unmanned aerial vehicles in the network and between the unmanned aerial vehicles and the ground control station, and feeding back and processing the sudden communication abnormal problems in the operation. Can receive the real-time location and the image data that unmanned aerial vehicle sent, also send scheduling information to unmanned aerial vehicle.
The image processing unit can carry out image processing and disease identification on the local bridge image information collected and returned by the bridge detection execution end, and meanwhile, a disease report is generated according to the returned detection position information and is automatically written into a disease information database. The user can see the details of the disease by consulting the database. The image processing unit is used for detailed and accurate processing of the returned image, has strict requirements on processing accuracy and has no strict requirements on speed.
The bridge model analysis unit comprises a bridge model storage module and a bridge model analysis module;
the bridge model storage module is configured to store a relational database (MYSQL, Oracle and the like) or a non-relational database (CouchDB, MongoDB and the like) of each type of bridge model.
And the bridge analysis module is configured to analyze bridge environment information and divide the bridge detection task into a plurality of bridge detection subtasks according to the bridge model to be detected, so as to prepare for unmanned aerial vehicle task allocation.
And the task allocation unit groups the bridge detection subtasks according to the task grouping indexes and determines the flight path of the unmanned aerial vehicle for completing each subtask group.
And the flight scheduling unit allocates tasks for each unmanned aerial vehicle according to the performance of the unmanned aerial vehicle and the results of task grouping and track planning, and generates instruction information for controlling the unmanned aerial vehicle to fly.
The information display unit can selectively adopt a CRT display screen or an LCD display screen (liquid crystal display) and is used for displaying the real-time information of the health of each module detected by the central control unit and the network communication state detected by the network management module.
And the abnormality processing unit is configured to receive abnormal landing information sent by the unmanned aerial vehicle with the abnormal times larger than a set threshold value, control the task distribution unit and the flight scheduling unit through the central control unit, and distribute the standby unmanned aerial vehicle to execute the bridge detection task.
As shown in fig. 3, a schematic diagram of a bridge detection execution end framework of a bridge detection system according to an embodiment of the automatic bridge detection method based on cooperative operation of multiple unmanned aerial vehicles of the present invention includes a microprocessor module, a flight power module, a communication module, an image acquisition module, a positioning module, and an obstacle avoidance module. An image processing module is also needed for unmanned planes requiring real-time image processing.
And the bridge detection execution end carries out bridge detection shooting according to the subtask group distributed by the ground control station and transmits the shot picture to the corresponding image processing unit according to the input image processing unit selection information.
And the micro-processing module is used for receiving and executing the instruction, exchanging information with an external memory and a logic component, and scheduling the work of each module of the bridge detection execution end.
The flight power module receives a flight instruction issued by the micro-processing module, provides power and controls the aircraft to fly according to the instruction.
The communication module is responsible for the communication between this machine and other unmanned aerial vehicle, this machine and ground control station. The module receives a flight instruction sent by the ground control station, transmits the flight instruction to the micro-processing module, and the micro-processing module issues a flight command to the flight power module according to the received instruction. And transmitting the position information of the unmanned aerial vehicle generated by the positioning module and the local bridge image information acquired by the image acquisition module to the ground control station.
The image acquisition module comprises a camera and a cloud platform of variable focal length to adjust the angle of camera and change the shooting precision of camera in a flexible way, enlarge the shooting distance of camera, guarantee the stability that the camera was shot. The zoom camera adjusts the focal length according to specific environment needs, and collects bridge image information.
The positioning module adopts one or more of GPS positioning, IMU positioning, compass positioning and altimeter positioning.
The bridge detection execution end is also provided with an image processing module;
and the image processing module is used for extracting the bridge disease characteristics of the photos shot by the unmanned aerial vehicle in the corresponding bridge detection execution end in real time through a preset bridge automatic detection image processing algorithm, screening the bridge disease characteristics by combining a preset threshold value, and generating a bridge disease report after obtaining effective bridge disease information.
The obstacle avoidance module adopts one or more of laser radar obstacle avoidance, binocular vision obstacle avoidance, ultrasonic distance measurement obstacle avoidance and infrared distance measurement obstacle avoidance.
Each unmanned aerial vehicle makes each unmanned aerial vehicle can guarantee not colliding with each other to position mutual induction each other through the ad hoc network technique.
The obstacle avoidance module is provided with an unmanned aerial vehicle flight anti-collision control method, which comprises the following steps:
step B10, in the flight process of the current unmanned aerial vehicle, acquiring the flight positions of other unmanned aerial vehicles in real time, and calculating the distance delta L between the current unmanned aerial vehicle and other unmanned aerial vehicles in real timet；
Step B20, judging whether there is delta LtLess than or equal to L0 andif the judgment result is yes, jumping to step B30; otherwise, continue step B20; wherein L is0Is a set safe distance;
step B30, comparing the flight path grades corresponding to the two unmanned aerial vehicles at the collision edge, and if the flight path grade of the current unmanned aerial vehicle is high, continuing to execute the task according to the flight path of the current unmanned aerial vehicle; otherwise, jumping to step B40;
step B40, hovering the current unmanned aerial vehicle until delta Lt＞L0Currently, the drone continues to perform tasks according to its flight path.
Based on the above-mentioned bridge automatic detection system based on multi-unmanned aerial vehicle cooperative work, as shown in fig. 4, the bridge detection process of an embodiment of the bridge automatic detection method based on multi-unmanned aerial vehicle cooperative work of the present invention includes:
step A10, the bridge model analysis unit analyzes the bridge environment information and divides the bridge task, and transmits the result to the central control unit, and the central control unit sends the work order and the analysis result to the task allocation unit;
step A20, the task allocation unit carries out sub-task grouping, unmanned aerial vehicle track planning and unmanned aerial vehicle track grading, and returns grouping, track planning and track grading results to the central control unit, and the central control unit transmits the results and the working instructions to the flight scheduling unit;
step A30, the flight scheduling unit allocates the subtask group to each unmanned aerial vehicle according to the performance index of each unmanned aerial vehicle, generates scheduling and flight instructions, and transmits the scheduling and flight instructions to the central control unit;
step A40, the central control unit dispatches the network management unit to establish a communication network, the execution end unmanned aerial vehicle set joins the network, the network management unit transmits the flight shooting instruction to the corresponding unmanned aerial vehicle, and monitors the network communication state;
step A50, the unmanned aerial vehicle receives the flight instruction through the communication module and transmits the flight instruction to the micro-processing module;
step A60, the micro-processing module sends a flight instruction to the flight power module, so that the unmanned aerial vehicle flies according to the planned path; the micro-processing module sends a shooting instruction to the image acquisition module, and the image acquisition module shoots the bridge image;
step A70, in the flight process of the unmanned aerial vehicle, the microprocessing module judges whether the unmanned aerial vehicle is abnormal in real time, if the abnormal times exceed the set threshold, the step A20 is returned; otherwise, go to step A80;
step A80, the microprocessor module obtains the real-time position of other unmanned aerial vehicles, calculates the distance delta L between the other unmanned aerial vehiclestAnd determining whether Δ L is presentt≤L0And isIf yes, further judging whether the level of the local computer is lower, if yes, hovering the local computer, and returning to the step A80; otherwise, go to step A90;
step A90, when the unmanned aerial vehicle acquires one image, the image processing unit of the ground workstation or the image processing module on the unmanned aerial vehicle extracts the image bridge disease characteristics;
step A100, a micro-processing module judges whether an unmanned aerial vehicle image acquisition task is finished, if so, step A110 is executed; otherwise, return to step A60;
and A110, screening the disease information according to the bridge disease characteristics, and generating a disease report from the effective disease information.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related descriptions of the method described above may refer to the corresponding process in the foregoing system embodiment, and are not described herein again.
It should be noted that, the bridge automatic detection system based on cooperative operation of multiple unmanned aerial vehicles provided in the above embodiment is only illustrated by dividing the above modules, and in practical applications, the above functions may be allocated by different modules according to needs, that is, the modules in the embodiment of the present invention are further decomposed or combined, for example, the modules in the above embodiment may be combined into one large module, or may be further decomposed into multiple small modules, so as to complete all or part of the above described functions. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (16)
1. A bridge automatic detection method based on multi-unmanned aerial vehicle cooperative work is characterized by comprising the following steps:
step S10, according to the obtained bridge model to be detected, analyzing bridge environment information and dividing a bridge detection task into a plurality of bridge detection subtasks; the bridge detection subtask is an unmanned aerial vehicle hovering shooting point;
step S20, acquiring a task clustering optimization function based on the plurality of bridge detection subtasks and each unmanned aerial vehicle executing the task;
step S30, dividing the bridge detection subtasks into a set number of subtask groups based on the task grouping optimization function, and acquiring the grade of each subtask group according to the preset importance of each subtask group;
step S40, respectively planning the flight path and the flight path grade of the unmanned aerial vehicle corresponding to each subtask group based on the acquired parameters of each unmanned aerial vehicle and the grade of each subtask group, and distributing each subtask group to the corresponding unmanned aerial vehicle;
step S50, each unmanned aerial vehicle carries out bridge detection shooting according to the corresponding unmanned aerial vehicle track and track grade, and transmits the shot picture to the corresponding image processing unit according to the input image processing unit selection information;
and step S60, extracting bridge disease features corresponding to the pictures shot by the unmanned aerial vehicle through a bridge automatic detection image processing algorithm preset by the image processing unit, screening the bridge disease features by combining a preset threshold value, and generating a bridge disease report after obtaining effective bridge disease information.
2. The method for automatically detecting the bridge based on the cooperative work of the multiple unmanned aerial vehicles according to claim 1, wherein the task clustering optimization function is as follows:
JA＝ω2λ+ω3η-ω1J
wherein J is a task group distribution index, lambda is a task quantity balance index, eta is an unmanned aerial vehicle cruise time balance index, and omega1、ω2、ω3The weights are respectively a task group distribution index, a task quantity balance index and an unmanned aerial vehicle cruising time balance index.
3. The automatic bridge detection method based on multi-unmanned aerial vehicle cooperative work of claim 2, wherein the task group distribution index is as follows:
wherein A isiRepresents the ith sub-task group in the set number of sub-task groups, S represents the number of sub-task groups,
represents AiT represents a bridge detection subtask,
representing bridge detection subtasks T and AiCluster center of
The euclidean distance of (c).
4. The automatic bridge detection method based on multi-unmanned-aerial-vehicle cooperative work according to claim 2, wherein the task number balance index is as follows:
wherein, muSkRepresenting the kth task of the S subtask groupThe man-machine executes the number of subtasks, S represents the number of subtask groups, and M represents the number of subtasks.
5. The automatic bridge detection method based on multi-unmanned-aerial-vehicle cooperative work according to claim 2, wherein the unmanned-aerial-vehicle cruise time balance index is as follows:
wherein S represents the number of subtask groups, ψi(i ═ 1,2, …, S) represents the optimal cruising path of the drone for the ith subtask group, t (ψ)i) And the cruising shooting time required by the optimal cruising path of the unmanned plane representing the ith subtask group.
6. The method for automatically detecting the bridge based on the cooperative work of the multiple unmanned aerial vehicles according to claim 1, wherein the communication mode of the pre-established communication network in the step S50 is one or more of Wi-Fi, ZigBee and GPRS.
7. The automatic bridge detection method based on the cooperative work of the multiple unmanned aerial vehicles according to any one of claims 1-6, wherein the automatic bridge detection method is further provided with a step of unmanned aerial vehicle abnormality remediation, and the method comprises the following steps:
and setting the abnormal times threshold of each unmanned aerial vehicle through redundancy fault-tolerant technology, if the abnormal times of any one unmanned aerial vehicle is greater than the threshold, descending the unmanned aerial vehicle and sending abnormal descending information, wherein the uncompleted subtasks are used as a subtask group, and after the unmanned aerial vehicle flight path planning is carried out again, the uncompleted subtasks are distributed to standby unmanned aerial vehicles for execution.
8. A bridge automatic detection system based on multi-unmanned aerial vehicle cooperative operation is characterized by comprising a ground control station and a plurality of bridge detection execution ends;
the ground control station comprises a central control unit, a network management unit, an image processing unit, a bridge model analysis unit, a task allocation unit, a flight scheduling unit and an information display unit, and is configured to analyze bridge environment information and divide a bridge detection task into a plurality of bridge detection subtasks according to an acquired bridge model to be detected, allocate each subtask group to each bridge detection execution end after combining each unmanned aerial vehicle executing the task to perform task grouping, unmanned aerial vehicle track planning and track grade division, receive photos sent by the bridge detection execution ends, extract bridge disease characteristics through a preset bridge automatic detection image processing algorithm, screen the bridge disease characteristics by combining a preset threshold value, and generate a bridge disease report after acquiring effective bridge disease information;
the bridge detection execution end comprises a micro-processing module, a flight power module, a communication module, an image acquisition module, a positioning module and an obstacle avoidance module, and is configured to carry out bridge detection shooting according to the subtask group distributed by the ground control station and transmit a shot picture to a corresponding image processing unit according to input image processing unit selection information.
9. The system according to claim 8, wherein the central control unit comprises a signal receiving module, a signal processing module, and an instruction transceiving module, and is configured to obtain the status of each of the plurality of bridge detection execution ends and transmit data information between the units of the ground control station.
10. The automatic bridge detection system based on multi-unmanned-aerial-vehicle cooperative work of claim 8, wherein the bridge model analysis unit comprises a bridge model storage module and a bridge model analysis module;
the bridge model storage module is configured to store a relational database or a non-relational database of bridge models of various categories;
the bridge analysis module is configured to analyze bridge environment information and divide a bridge detection task into a plurality of bridge detection subtasks according to a bridge model to be detected.
11. The automatic bridge detection system based on multi-unmanned-aerial-vehicle cooperative operation of any one of claims 8-10, wherein the ground control station is further provided with an exception handling unit;
the abnormity processing unit is configured to receive abnormal landing information sent by the unmanned aerial vehicle with the abnormal times larger than a set threshold value, and the central control unit controls the task distribution unit and the flight scheduling unit to distribute the standby unmanned aerial vehicle to execute the bridge detection task.
12. The system of claim 8, wherein the positioning module employs one or more of GPS positioning, IMU positioning, compass positioning, and altimeter positioning.
13. The automatic bridge detection system based on multi-unmanned-aerial-vehicle cooperative work according to claim 8, wherein the bridge detection execution end is further provided with an image processing module;
the image processing module is used for extracting bridge disease characteristics of the photos shot by the unmanned aerial vehicle in the corresponding bridge detection execution end in real time through a preset bridge automatic detection image processing algorithm, screening the bridge disease characteristics by combining a preset threshold value, and generating a bridge disease report after obtaining effective bridge disease information.
14. The automatic bridge detection system based on multi-unmanned-aerial-vehicle cooperative operation of claim 8, wherein the obstacle avoidance module adopts one or more of laser radar obstacle avoidance, binocular vision obstacle avoidance, ultrasonic ranging obstacle avoidance, and infrared ranging obstacle avoidance.
15. The automatic bridge detection system based on multi-unmanned-aerial-vehicle cooperative work of claim 14, wherein the obstacle avoidance module is provided with an unmanned-aerial-vehicle flight collision avoidance control method, comprising:
step B10, in the flight process of the current unmanned aerial vehicle, acquiring the flight positions of other unmanned aerial vehicles in real time, and calculating the distance delta L between the current unmanned aerial vehicle and other unmanned aerial vehicles in real timet；
Step B20, judging whether there is delta Lt≤L0And is
If the judgment result is yes, jumping to step B30; otherwise, continue step B20; wherein L is0Is a set safe distance;
step B30, comparing the flight path grades corresponding to the two unmanned aerial vehicles at the collision edge, and if the flight path grade of the current unmanned aerial vehicle is high, continuing to execute the task according to the flight path of the current unmanned aerial vehicle; otherwise, jumping to step B40;
step B40, hovering the current unmanned aerial vehicle until delta Lt＞L0Currently, the drone continues to perform tasks according to its flight path.
16. An operation method of an automatic bridge detection system, based on the automatic bridge detection system based on the cooperative work of multiple unmanned aerial vehicles according to any one of claims 8 to 15, the operation method of the automatic bridge detection system comprises the following steps:
step A10, the bridge model analysis unit analyzes the bridge environment information and divides the bridge task, and transmits the result to the central control unit, and the central control unit sends the work order and the analysis result to the task allocation unit;
step A20, the task allocation unit carries out sub-task grouping, unmanned aerial vehicle track planning and unmanned aerial vehicle track grading, and returns grouping, track planning and track grading results to the central control unit, and the central control unit transmits the results and the working instructions to the flight scheduling unit;
step A30, the flight scheduling unit allocates the subtask group to each unmanned aerial vehicle according to the performance index of each unmanned aerial vehicle, generates scheduling and flight instructions, and transmits the scheduling and flight instructions to the central control unit;
step A40, the central control unit dispatches the network management unit to establish a communication network, the execution end unmanned aerial vehicle set joins the network, the network management unit transmits the flight shooting instruction to the corresponding unmanned aerial vehicle, and monitors the network communication state;
step A50, the unmanned aerial vehicle receives the flight instruction through the communication module and transmits the flight instruction to the micro-processing module;
step A60, the micro-processing module sends a flight instruction to the flight power module, so that the unmanned aerial vehicle flies according to the planned path; the micro-processing module sends a shooting instruction to the image acquisition module, and the image acquisition module shoots the bridge image;
step A70, in the flight process of the unmanned aerial vehicle, the microprocessing module judges whether the unmanned aerial vehicle is abnormal in real time, if the abnormal times exceed the set threshold, the step A20 is returned; otherwise, go to step A80;
step A80, step A80, the microprocessor module obtains the real-time position of other unmanned aerial vehicles, and calculates the distance delta L between the other unmanned aerial vehiclestAnd determining whether Δ L is presentt≤L0And isIf yes, further judging whether the level of the local computer is lower, if yes, hovering the local computer, and returning to the step A80; otherwise, go to step A90;
step A90, when the unmanned aerial vehicle acquires one image, the image processing unit of the ground workstation or the image processing module on the unmanned aerial vehicle extracts the image bridge disease characteristics;
step A100, a micro-processing module judges whether an unmanned aerial vehicle image acquisition task is finished, if so, step A110 is executed; otherwise, return to step A60;
and A110, screening the disease information according to the bridge disease characteristics, and generating a disease report from the effective disease information.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911065609.6A CN110703802A (en) | 2019-11-04 | 2019-11-04 | Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation |
| PCT/CN2020/083863 WO2021088311A1 (en) | 2019-11-04 | 2020-04-09 | Multi-unmanned aerial vehicle collaborative operation-based automatic inspection method and system for bridges |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911065609.6A CN110703802A (en) | 2019-11-04 | 2019-11-04 | Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110703802A true CN110703802A (en) | 2020-01-17 |
Family
ID=69202365
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911065609.6A Pending CN110703802A (en) | 2019-11-04 | 2019-11-04 | Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110703802A (en) |
| WO (1) | WO2021088311A1 (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111390913A (en) * | 2020-04-09 | 2020-07-10 | 武汉武大卓越科技有限责任公司 | Automatic detection method for bridge bottom structure |
| CN111765924A (en) * | 2020-07-13 | 2020-10-13 | 江苏中科智能制造研究院有限公司 | Atmospheric environment monitoring method and system based on multiple unmanned aerial vehicles |
| CN111964649A (en) * | 2020-08-21 | 2020-11-20 | 武汉原点勘测设计工程有限公司 | Terrain information acquisition method based on unmanned aerial vehicle remote sensing |
| CN112098326A (en) * | 2020-08-20 | 2020-12-18 | 东南大学 | Automatic detection method and system for bridge diseases |
| CN112362068A (en) * | 2020-12-04 | 2021-02-12 | 浙江煤炭测绘院有限公司 | Unmanned aerial vehicle surveying and mapping method, device and system |
| CN112580892A (en) * | 2020-12-29 | 2021-03-30 | 广州极飞科技有限公司 | Collaborative operation route planning method and device, computer equipment and storage medium |
| CN112598187A (en) * | 2020-12-29 | 2021-04-02 | 广州极飞科技有限公司 | Method and device for generating collaborative work route, computer equipment and storage medium |
| WO2021088311A1 (en) * | 2019-11-04 | 2021-05-14 | 中国科学院自动化研究所 | Multi-unmanned aerial vehicle collaborative operation-based automatic inspection method and system for bridges |
| CN114091807A (en) * | 2021-10-12 | 2022-02-25 | 广西综合交通大数据研究院 | Method, device and system for distributing and scheduling tasks of multiple unmanned aerial vehicles and storage medium |
| CN114510073A (en) * | 2022-01-24 | 2022-05-17 | 枣庄易飞航天科技有限公司 | Intelligent high-speed frontier defense inspection system based on composite wing unmanned aerial vehicle |
| CN114531193A (en) * | 2022-01-04 | 2022-05-24 | 无锡市市政设施养护管理有限公司 | Bridge state monitoring method based on unmanned aerial vehicle cellular topology networking and mobile edge calculation |
| CN114550107A (en) * | 2022-04-26 | 2022-05-27 | 深圳联和智慧科技有限公司 | Bridge linkage intelligent inspection method and system based on unmanned aerial vehicle cluster and cloud platform |
| CN115032206A (en) * | 2022-06-15 | 2022-09-09 | 招商局重庆交通科研设计院有限公司 | Intelligent detection method for bridge robot |
| CN115239204A (en) * | 2022-09-19 | 2022-10-25 | 中国电子科技集团公司第十四研究所 | Collaborative task planning method for multi-platform unmanned aerial vehicle-mounted radio frequency system |
| CN117348424A (en) * | 2023-11-30 | 2024-01-05 | 南通大地测绘有限公司 | Unmanned aerial vehicle group collaborative mapping method and system based on self-adaptive algorithm |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117590757B (en) * | 2024-01-19 | 2024-05-07 | 成都航空职业技术学院 | Multi-unmanned aerial vehicle cooperative task allocation method based on Gaussian distribution sea-gull optimization algorithm |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100763851B1 (en) * | 2006-02-03 | 2007-10-05 | 한국도로공사 | Bridge inspecting robot capable of climbing up an obstacle and processing an continuos image |
| US20140336928A1 (en) * | 2013-05-10 | 2014-11-13 | Michael L. Scott | System and Method of Automated Civil Infrastructure Metrology for Inspection, Analysis, and Information Modeling |
| CN105259911A (en) * | 2015-07-22 | 2016-01-20 | 北京佰才邦技术有限公司 | Control method and system of movable carrier and control system of unmanned plane |
| CN105735150A (en) * | 2016-03-04 | 2016-07-06 | 浙江大学 | Movable multi-view visual bridge conventional detection method |
| CN108051450A (en) * | 2017-11-01 | 2018-05-18 | 中铁大桥科学研究院有限公司 | A kind of bridge health monitoring system and method based on unmanned plane |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110703802A (en) * | 2019-11-04 | 2020-01-17 | 中国科学院自动化研究所 | Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation |
-
2019
- 2019-11-04 CN CN201911065609.6A patent/CN110703802A/en active Pending
-
2020
- 2020-04-09 WO PCT/CN2020/083863 patent/WO2021088311A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100763851B1 (en) * | 2006-02-03 | 2007-10-05 | 한국도로공사 | Bridge inspecting robot capable of climbing up an obstacle and processing an continuos image |
| US20140336928A1 (en) * | 2013-05-10 | 2014-11-13 | Michael L. Scott | System and Method of Automated Civil Infrastructure Metrology for Inspection, Analysis, and Information Modeling |
| CN105259911A (en) * | 2015-07-22 | 2016-01-20 | 北京佰才邦技术有限公司 | Control method and system of movable carrier and control system of unmanned plane |
| CN105735150A (en) * | 2016-03-04 | 2016-07-06 | 浙江大学 | Movable multi-view visual bridge conventional detection method |
| CN108051450A (en) * | 2017-11-01 | 2018-05-18 | 中铁大桥科学研究院有限公司 | A kind of bridge health monitoring system and method based on unmanned plane |
Non-Patent Citations (3)
| Title |
|---|
| 尹高扬 等: ""多无人机协同多任务分配研究"", 《电光与控制》 * |
| 王毅 等: ""多无人机协同任务分群方案研究"", 《舰船电子工程》 * |
| 邵壮 等: ""三维动态环境下多无人机编队分布式保持控制"", 《控制与决策》 * |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021088311A1 (en) * | 2019-11-04 | 2021-05-14 | 中国科学院自动化研究所 | Multi-unmanned aerial vehicle collaborative operation-based automatic inspection method and system for bridges |
| CN111390913B (en) * | 2020-04-09 | 2021-07-20 | 武汉光谷卓越科技股份有限公司 | Automatic detection method for bridge bottom structure |
| CN111390913A (en) * | 2020-04-09 | 2020-07-10 | 武汉武大卓越科技有限责任公司 | Automatic detection method for bridge bottom structure |
| CN111765924A (en) * | 2020-07-13 | 2020-10-13 | 江苏中科智能制造研究院有限公司 | Atmospheric environment monitoring method and system based on multiple unmanned aerial vehicles |
| CN112098326A (en) * | 2020-08-20 | 2020-12-18 | 东南大学 | Automatic detection method and system for bridge diseases |
| CN112098326B (en) * | 2020-08-20 | 2022-09-30 | 东南大学 | Automatic detection method and system for bridge diseases |
| CN111964649A (en) * | 2020-08-21 | 2020-11-20 | 武汉原点勘测设计工程有限公司 | Terrain information acquisition method based on unmanned aerial vehicle remote sensing |
| CN112362068A (en) * | 2020-12-04 | 2021-02-12 | 浙江煤炭测绘院有限公司 | Unmanned aerial vehicle surveying and mapping method, device and system |
| CN112580892A (en) * | 2020-12-29 | 2021-03-30 | 广州极飞科技有限公司 | Collaborative operation route planning method and device, computer equipment and storage medium |
| CN112598187A (en) * | 2020-12-29 | 2021-04-02 | 广州极飞科技有限公司 | Method and device for generating collaborative work route, computer equipment and storage medium |
| CN114091807A (en) * | 2021-10-12 | 2022-02-25 | 广西综合交通大数据研究院 | Method, device and system for distributing and scheduling tasks of multiple unmanned aerial vehicles and storage medium |
| CN114531193A (en) * | 2022-01-04 | 2022-05-24 | 无锡市市政设施养护管理有限公司 | Bridge state monitoring method based on unmanned aerial vehicle cellular topology networking and mobile edge calculation |
| CN114531193B (en) * | 2022-01-04 | 2023-11-10 | 无锡市市政设施养护管理有限公司 | Bridge state monitoring method based on unmanned aerial vehicle cellular topology networking and mobile edge calculation |
| CN114510073A (en) * | 2022-01-24 | 2022-05-17 | 枣庄易飞航天科技有限公司 | Intelligent high-speed frontier defense inspection system based on composite wing unmanned aerial vehicle |
| CN114510073B (en) * | 2022-01-24 | 2022-10-25 | 枣庄易飞航天科技有限公司 | Intelligent high-speed frontier defense inspection system based on composite wing unmanned aerial vehicle |
| CN114550107A (en) * | 2022-04-26 | 2022-05-27 | 深圳联和智慧科技有限公司 | Bridge linkage intelligent inspection method and system based on unmanned aerial vehicle cluster and cloud platform |
| CN114550107B (en) * | 2022-04-26 | 2022-07-26 | 深圳联和智慧科技有限公司 | Bridge linkage intelligent inspection method and system based on unmanned aerial vehicle cluster and cloud platform |
| CN115032206A (en) * | 2022-06-15 | 2022-09-09 | 招商局重庆交通科研设计院有限公司 | Intelligent detection method for bridge robot |
| CN115239204A (en) * | 2022-09-19 | 2022-10-25 | 中国电子科技集团公司第十四研究所 | Collaborative task planning method for multi-platform unmanned aerial vehicle-mounted radio frequency system |
| CN115239204B (en) * | 2022-09-19 | 2023-02-14 | 中国电子科技集团公司第十四研究所 | Collaborative task planning method for multi-platform unmanned aerial vehicle-mounted radio frequency system |
| CN117348424A (en) * | 2023-11-30 | 2024-01-05 | 南通大地测绘有限公司 | Unmanned aerial vehicle group collaborative mapping method and system based on self-adaptive algorithm |
| CN117348424B (en) * | 2023-11-30 | 2024-03-08 | 南通大地测绘有限公司 | Unmanned aerial vehicle group collaborative mapping method and system based on self-adaptive algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021088311A1 (en) | 2021-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110703802A (en) | Automatic bridge detection method and system based on multi-unmanned aerial vehicle cooperative operation | |
| CN108196575B (en) | Unmanned aerial vehicle task allocation and route planning method | |
| Rathinam et al. | Vision-based monitoring of locally linear structures using an unmanned aerial vehicle | |
| Luo et al. | A survey of intelligent transmission line inspection based on unmanned aerial vehicle | |
| Scherer et al. | Autonomous landing at unprepared sites by a full-scale helicopter | |
| CN101477169B (en) | Electric power circuit detection method by polling flying robot | |
| CN107992067A (en) | Unmanned plane inspection fault diagnosis system based on integrated gondola and AI technologies | |
| CN106708073B (en) | A kind of quadrotor system of independent navigation power-line patrolling fault detection | |
| CN111555178B (en) | Sky-ground cooperative intelligent inspection method and system for power transmission line | |
| US20230286556A1 (en) | Autonomous drone for railroad track inspection | |
| CN110634104B (en) | Multi-map splicing method and device | |
| Yaqot et al. | Unmanned aerial vehicle (UAV) in precision agriculture: business information technology towards farming as a service | |
| CN103810382B (en) | Method for choosing two-level data fusion strategy of airborne distributed multi-sensor | |
| Mejias et al. | Forced landing technologies for unmanned aerial vehicles: towards safer operations | |
| CN110132242A (en) | Multiple-camera positions and the Triangulation Algorithm and its movable body of map structuring immediately | |
| CN103984355A (en) | Routing inspection flying robot and overhead power line distance prediction and maintaining method | |
| CN106789740B (en) | Multi-platform sensor collaborative management method for sequential auction according to task priority | |
| CN112215416B (en) | Intelligent planning inspection route system and method | |
| CN112817331A (en) | Intelligent forestry information monitoring system based on multi-machine cooperation | |
| Minghui et al. | Deep learning enabled localization for UAV autolanding | |
| CN109631898A (en) | The method and device navigated to ship | |
| CN115980742A (en) | Radar detection method and device for unmanned aerial vehicle | |
| Xu et al. | Development of power transmission line detection technology based on unmanned aerial vehicle image vision | |
| CN117519944A (en) | Unmanned vehicle based on cooperation of computing power perception and edge cloud computing and cooperation method thereof | |
| CN105096661B (en) | Air traffic Workflow Management System and method based on vertical sectional view |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200117 |