CN115100379A - Fan blade transportation supervision method, system, equipment and medium - Google Patents

Fan blade transportation supervision method, system, equipment and medium Download PDF

Info

Publication number
CN115100379A
CN115100379A CN202210674915.5A CN202210674915A CN115100379A CN 115100379 A CN115100379 A CN 115100379A CN 202210674915 A CN202210674915 A CN 202210674915A CN 115100379 A CN115100379 A CN 115100379A
Authority
CN
China
Prior art keywords
road
fan blade
transportation
unmanned aerial
aerial vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210674915.5A
Other languages
Chinese (zh)
Other versions
CN115100379B (en
Inventor
陈旭
许慧青
肖思恒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Energy Group Science And Technology Research Institute Co ltd
Original Assignee
Guangdong Energy Group Science And Technology Research Institute Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Energy Group Science And Technology Research Institute Co ltd filed Critical Guangdong Energy Group Science And Technology Research Institute Co ltd
Priority to CN202210674915.5A priority Critical patent/CN115100379B/en
Publication of CN115100379A publication Critical patent/CN115100379A/en
Application granted granted Critical
Publication of CN115100379B publication Critical patent/CN115100379B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02EREDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
    • Y02E10/00Energy generation through renewable energy sources
    • Y02E10/70Wind energy
    • Y02E10/72Wind turbines with rotation axis in wind direction

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

The invention relates to the technical field of wind power equipment transportation, in particular to a method, a system, equipment and a medium for supervising the transportation of fan blades, which comprises the following steps: extracting road point cloud data according to the transportation scene data, acquiring a road inflection point according to the road point cloud data, establishing a simulation model to simulate the passing safety of vehicles, and determining a target transportation vehicle based on a simulation result and fan blade parameters; determining the flight state of the unmanned aerial vehicle according to the running state of the target transport vehicle and the road inflection point, and acquiring the posture of the fan blade according to the fan blade image shot by the unmanned aerial vehicle; and performing collision detection according to the posture of the fan blade and the road three-dimensional scene model so as to judge the safety of the fan blade in the transportation process. According to the invention, the unmanned aerial vehicle is applied to the monitoring of the fan blade transportation process, so that automatic routing inspection is realized, effective guidance can be provided for emergency situations in fan blade transportation, the transportation safety is improved, and the transportation cost is reduced.

Description

Fan blade transportation supervision method, system, equipment and medium
Technical Field
The invention relates to the technical field of wind power equipment transportation, in particular to a method, a system, equipment and a medium for supervising fan blade transportation based on an unmanned aerial vehicle.
Background
Wind power generation is a renewable clean energy, the demand of China for clean energy is continuously increased, and the wind power industry is rapidly developed. Along with the increase of the single-machine capacity of the fan, the weight and the length of the blades are also continuously increased, the length of the blades of the large-scale fan can reach six seventy meters or even hundreds of meters, however, the areas rich in wind energy resources in China are mostly in mountainous areas, the terrain is far away, roads are tortuous, the ultra-long blades are easy to collide with mountains, trees and the like on two sides of the roads, and higher requirements are provided for the transportation of the blades. The trafficability characteristic of the traditional transportation mode is generally poor, and the road exploration, obstacle clearing, road modification and other work need to be carried out manually before transportation, so that a large amount of manpower, material resources and financial resources are consumed, and the road condition is not scientifically evaluated. At present, wind power equipment transportation researches are carried out by optimizing a transportation device, although the azimuth angle of a fan blade is adjusted through a hydraulic mechanism, the occupied width of a road is reduced, and a buffering or protecting device for fan blade transportation is designed, when a transportation vehicle encounters a turn and a mountain wall, effective guidance can not be provided for emergency situations in fan blade transportation.
Along with the development of technique, unmanned aerial vehicle has obtained extensive application in road survey and drawing and traffic control, simultaneously because fan blade's transportation environment is more complicated, need overcome the influence of turn, gable, fan blade length, consequently, be applied to unmanned aerial vehicle and monitor and manage in fan blade's the transportation, will greatly reduced manpower and raise the efficiency.
Disclosure of Invention
The invention provides a method, a system, equipment and a medium for supervising fan blade transportation based on an unmanned aerial vehicle, and solves the technical problems that the existing wind power equipment transportation method only optimizes a transportation device, cannot realize automatic routing inspection and provide effective guidance for emergency situations in fan blade transportation, and is low in efficiency.
In order to solve the technical problems, the invention provides a method, a system, equipment and a medium for monitoring and managing fan blade transportation.
In a first aspect, the present invention provides a method for supervising fan blade transportation, comprising the steps of:
carrying out three-dimensional reconstruction based on the collected transportation scene data to obtain a road three-dimensional scene model;
extracting road point cloud data according to the transportation scene data, acquiring a road inflection point according to the road point cloud data, and establishing a simulation model;
simulating the passing safety of the vehicle by using the simulation model and the road inflection point to obtain a simulation result;
determining a target transport vehicle based on the simulation result and pre-collected fan blade parameters;
determining the flight state of the unmanned aerial vehicle according to the running state of the target transport vehicle and the road inflection point so as to enable the fan blade to be always positioned in the visual field area of the unmanned aerial vehicle;
acquiring the posture of a fan blade according to a fan blade image shot by an unmanned aerial vehicle, and performing collision detection according to the posture of the fan blade and the road three-dimensional scene model to obtain a collision detection result;
and formulating a fan blade transportation planning strategy based on the collision detection result.
In a further embodiment, the step of performing three-dimensional reconstruction based on the collected transportation scene data to obtain a road three-dimensional scene model includes:
determining a target transportation route according to the position information of the wind power plant and the road condition information, and acquiring laser point cloud data and video image data of the target transportation route and the surrounding environment thereof;
carrying out combined calibration on laser point cloud data and video image data at the same time period, and registering the laser point cloud data to obtain registered laser point cloud data;
performing corner feature extraction on the video image data to obtain video image feature data;
projecting the registered laser point cloud data to the video image characteristic data to obtain fusion data;
and calibrating the fusion data, and performing three-dimensional reconstruction according to the calibrated fusion data to obtain a road three-dimensional scene model.
In a further embodiment, the step of obtaining a road inflection point from the road point cloud data and building a simulation model includes:
determining a track buffer area according to the collected historical track information of the unmanned aerial vehicle;
segmenting the road point cloud data by using a preset segmentation length, and segmenting a ground point cloud;
constructing a combined characteristic value according to the transport road point cloud echo intensity and elevation change, and extracting the ground point cloud to obtain road edge points;
clustering the road edge points to obtain a road boundary;
extracting a road inflection point according to the road boundary;
acquiring road parameters according to the road inflection point and the road point cloud data;
establishing a simulation model according to pre-collected fan transportation device parameters and the road parameters;
the fan transport device parameters include transport vehicle parameters and fan blade parameters, the fan blade parameters include fan blade length and weight, and the road parameters include road width, road inflection radius, and road bearing capacity.
In further embodiments, the step of determining the flight status of the drone as a function of the driving status of the target transport vehicle and the road inflection point comprises:
determining the flight speed of the unmanned aerial vehicle according to the running state of the target transport vehicle so as to enable the unmanned aerial vehicle and the target transport vehicle to keep relatively static;
determining the entrance and exit distance of the road inflection point according to the road inflection point;
calculating the initial minimum flying height of the unmanned aerial vehicle according to the field angles of the unmanned aerial vehicle in different shooting directions, the pre-collected fan blade length and the access distance of the road inflection point;
setting a flying height adjusting threshold value according to the road terrain height change;
and dynamically adjusting the flying height of the unmanned aerial vehicle according to the initial minimum flying height and the flying height adjusting threshold value to obtain the target flying height of the unmanned aerial vehicle.
In a further embodiment, the initial minimum flying height of the drone is calculated as:
Figure BDA0003696080430000031
Figure BDA0003696080430000041
Figure BDA0003696080430000042
Figure BDA0003696080430000043
in the formula I 1 Representing the length of the fan blade, alpha representing the angle between the fan blade and the horizontal plane of the target transport vehicle, l 2 Represents the distance between two points, H, of the road inflection point entrance and the road inflection point exit camera Representing the longitudinal resolution, W, of the image camera Representing the image lateral resolution, h representing the initial minimum flying height of the drone, A hfov Representing the angle of view of the unmanned aerial vehicle camera when shooting vertically, A dfov Shows the angle of view, A, of the unmanned aerial vehicle camera when shooting in the diagonal direction vfov The view angle of the unmanned aerial vehicle camera during horizontal shooting is shown.
In a further embodiment, the step of obtaining the fan blade attitude from the fan blade image captured by the drone comprises:
identifying a fan blade image shot by the unmanned aerial vehicle by using a pre-constructed full convolution neural network to obtain an image identification result;
obtaining the image pixel position of the fan blade according to the image recognition result, and calculating the position coordinates corresponding to the end points of the two ends of the fan blade according to the unmanned aerial vehicle camera parameter and the holder angle parameter;
and acquiring the posture of the fan blade according to the position coordinates corresponding to the end points of the two ends of the fan blade.
In a further embodiment, the step of performing collision detection according to the fan blade attitude and the road three-dimensional scene model to obtain a collision detection result includes:
taking a line segment between end points of two ends of the fan blade as a first line segment;
traversing all planes in the road three-dimensional scene model, and taking a line segment between any two points in the planes as a second line segment;
calculating the shortest spatial distance between the first line segment and the second line segment;
and comparing the shortest space distance with a preset fan blade safety distance threshold value to judge the safety of the fan blade relative to the road three-dimensional scene model.
In a second aspect, the present invention provides a wind turbine blade transportation supervision system, the system comprising:
the three-dimensional reconstruction module is used for performing three-dimensional reconstruction based on the collected transportation scene data to obtain a road three-dimensional scene model;
the road evaluation module is used for extracting road point cloud data according to the transportation scene data, acquiring a road inflection point according to the road point cloud data and establishing a simulation model; the simulation model is used for simulating the passing safety of the vehicle by utilizing the road inflection point to obtain a simulation result;
the unmanned aerial vehicle monitoring module is used for determining a target transport vehicle based on the simulation result and pre-collected fan blade parameters; the system is also used for determining the flight state of the unmanned aerial vehicle according to the running state of the target transport vehicle and the road inflection point so as to enable the fan blade to be always positioned in the visual field area of the unmanned aerial vehicle;
the collision detection module is used for acquiring the posture of the fan blade according to the fan blade image shot by the unmanned aerial vehicle, and performing collision detection according to the posture of the fan blade and the road three-dimensional scene model to obtain a collision detection result; and the collision detection result is used for making a fan blade transportation planning strategy.
In a third aspect, the present invention further provides a computer device, including a processor and a memory, where the processor is connected to the memory, the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory, so that the computer device executes the steps for implementing the method.
In a fourth aspect, the present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the above method.
The invention provides a fan blade transportation supervision method, a system, equipment and a medium based on an unmanned aerial vehicle, wherein the method carries out three-dimensional reconstruction on a transportation road by utilizing the unmanned aerial vehicle to carry a laser radar and a visible light cloud platform camera, and sets a supervision point position of the unmanned aerial vehicle according to a road three-dimensional scene model and fan blade transportation characteristics, so that the investigation efficiency is improved; meanwhile, the invention carries out collision detection on the fan blade so as to check the safety of the fan blade relative to the road three-dimensional scene model, ensure that the fan blade passes through safely and reduce the damage of the fan blade. Compared with the prior art, the method realizes the transportation process of automatically detecting the fan blade by using the unmanned aerial vehicle, reduces the risk of impact of the fan blade, and saves the operation time and cost.
Drawings
FIG. 1 is a schematic flow chart of a method for supervising fan blade transportation according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a three-dimensional reconstruction process provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of a road boundary extraction process provided by an embodiment of the present invention;
FIG. 4 is a schematic view of a transport vehicle shot by an unmanned aerial vehicle according to an embodiment of the invention;
FIG. 5 is a schematic diagram of a safety check of a fan blade relative to a three-dimensional scene model of a road according to an embodiment of the present invention;
FIG. 6 is a block diagram of a wind turbine blade transportation monitoring system provided by an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, which are given solely for the purpose of illustration and are not to be construed as limitations of the invention, including the drawings which are incorporated herein by reference and for illustration only, and are not to be construed as limitations of the invention, since many variations thereof are possible without departing from the spirit and scope of the invention.
Referring to fig. 1, an embodiment of the present invention provides a method for supervising fan blade transportation, as shown in fig. 1, the method includes the following steps:
s1, performing three-dimensional reconstruction based on the collected transportation scene data to obtain a road three-dimensional scene model.
In an embodiment, as shown in fig. 2, the step of performing three-dimensional reconstruction based on the collected transportation scene data to obtain a road three-dimensional scene model includes:
determining a target transportation route according to the position information of the wind power plant and the road condition information, selecting an unmanned aerial vehicle to carry a laser radar and a visible light cloud platform camera, and performing automatic cruise through an unmanned aerial vehicle intelligent scheduling algorithm and path planning to obtain laser point cloud data and video image data of the target transportation route and the surrounding environment thereof;
the laser point cloud data and the video image data in the same time period are jointly calibrated, so that the laser point cloud data can be transplanted into a visual coordinate system, and meanwhile, the video image data can also be transplanted into the laser coordinate system, thereby being more beneficial to the fusion of later data;
registering the laser point cloud data by using positioning navigation data obtained by RTK to obtain registered laser point cloud data; the registration step needs to calibrate the newly added laser point cloud data by using positioning navigation data obtained by RTK, where an iterative algorithm is used for calibration, specifically: in the process of each iteration, each laser point in each frame of point cloud is calculated, and the point with the closest distance is searched in another frame of point cloud, so that the distance of all point cloud pairs between every two points is the minimum.
In order to enable the images of the frames to correspond, performing corner feature extraction on video image data, and matching adjacent images by taking corners as tracking points to obtain video image feature data;
projecting the registration laser point cloud data onto the video image characteristic data to obtain fusion data so as to realize data fusion;
and calibrating the fusion data, and performing three-dimensional reconstruction according to the calibrated fusion data to obtain a road three-dimensional scene model.
And S2, extracting road point cloud data according to the transportation scene data, acquiring a road inflection point according to the road point cloud data, and establishing a simulation model.
In one embodiment, the step of obtaining a road inflection point according to the road point cloud data and building a simulation model comprises the following steps:
determining a track buffer area according to the collected historical track information of the unmanned aerial vehicle;
segmenting the road point cloud data by using a preset segmentation length, and segmenting a ground point cloud;
constructing a combined characteristic value according to the transport road point cloud echo intensity and elevation change, and extracting the ground point cloud to obtain road edge points;
clustering the road edge points to obtain a road boundary;
extracting and obtaining a road inflection point according to the road boundary, wherein the road inflection point is a road turning point;
acquiring road parameters according to the road inflection point and the road point cloud data;
establishing a simulation model according to pre-collected fan transportation device parameters and the road parameters;
the fan transportation device parameters comprise transportation vehicle parameters and fan blade parameters, the fan blade parameters comprise fan blade length and weight, and the road parameters comprise road width, road inflection radius and road bearing capacity.
Specifically, fig. 3 is a schematic diagram of a road boundary extraction process provided in the embodiment of the present invention, in order to ensure efficiency of road boundary extraction, a buffer area with a certain width is set according to historical track information of an unmanned aerial vehicle, interference data far away from two sides of a road are eliminated, meanwhile, obtained road point cloud data are segmented according to a preset length, and in order to ensure continuity of the data, an overlap area with a preset overlap length is further set in the embodiment, and data preprocessing is performed; and then separating ground points and non-ground points by adopting a Cloth Simulation Filter (CSF), removing salt and pepper noise by adopting median filtering, constructing a joint characteristic value according to the echo intensity and the elevation change of the point cloud of the transportation road, extracting road edge points of the point cloud of the ground, finally performing K-means clustering on the preliminarily extracted road edge points, removing the non-edge points extracted by mistake, and refining the extraction result to obtain a road boundary.
And S3, simulating the passing safety of the vehicle by using the simulation model and the road inflection point to obtain a simulation result.
In the embodiment, whether vehicles can safely pass is preliminarily evaluated through a simulation model, if the transport vehicles cannot safely pass, the distribution conditions of the terrain, the mountain wall, trees and the like of turning points are obtained according to a three-dimensional scene model of the road, and the road is reformed or obstacles are removed by combining road parameters; if the transport vehicle can safely pass, the next step is carried out to determine the target transport vehicle.
And S4, determining a target transport vehicle based on the simulation result and the pre-collected fan blade parameters.
In the embodiment, under the condition that the simulated transport vehicle can safely pass through, a target transport vehicle is selected according to the pre-collected fan blade parameters, and meanwhile, the unmanned aerial vehicle is used for monitoring the vehicle for transporting the fan blades; in this embodiment, the fan blade parameters include the size and weight of the fan blade.
And S5, determining the flight state of the unmanned aerial vehicle according to the running state of the target transport vehicle and the road inflection point so as to enable the fan blade to be always positioned in the visual field area of the unmanned aerial vehicle.
In one embodiment, the step of determining the flight status of the drone according to the driving status of the target transport vehicle and the road inflection point comprises:
determining the flight speed of the unmanned aerial vehicle according to the running state of the target transport vehicle so as to keep the unmanned aerial vehicle and the target transport vehicle relatively static, thereby realizing real-time tracking of the vehicle;
determining the entrance and exit distance of the road inflection point according to the road inflection point;
calculating the initial minimum flying height of the unmanned aerial vehicle according to the field angles of the unmanned aerial vehicle in different shooting directions, the length of a fan blade and the entrance-exit distance of the road inflection point;
setting a flying height adjusting threshold value according to the road terrain height change;
and dynamically adjusting the flying height of the unmanned aerial vehicle according to the initial minimum flying height and the flying height adjusting threshold value to obtain the target flying height of the unmanned aerial vehicle.
Specifically, in order to better recognize the driving state and the posture of the fan blade of the target transport vehicle, the present embodiment needs to ensure that the fan blade can still be completely imaged when the distance between the unmanned aerial vehicle and the target transport vehicle is minimum, namely, no matter whether the unmanned aerial vehicle camera shoots transversely (a) vfov ) Longitudinal shooting (A) hfov ) Or shot in the diagonal direction (A) dfov ) During the time, but unmanned aerial vehicle camera visual field homoenergetic enough covers complete haulage vehicle and fan blade, simultaneously because mainly easily take place the collision at road turning point in the fan blade transportation, consequently regard road turning point as the key point position of unmanned aerial vehicle control, promptly, as shown in fig. 4, this embodiment needs to ensure that target haulage vehicle and fan blade are all in the unmanned aerial vehicle field of vision in getting into road inflection entry A and leaving road inflection export B this time quantum, consequently, as shown in fig. 4, this embodiment, target haulage vehicle and fan blade are all in the unmanned aerial vehicle field of vision inIn this embodiment, the target flying height of the unmanned aerial vehicle is calculated according to the road inflection point entrance a and the road inflection point exit B, and the following calculation process is the target flying height of the unmanned aerial vehicle:
suppose a fan blade length of l 1 The included angle between the fan blade and the horizontal plane where the target transport vehicle is located is alpha, and the distance between two points of a road inflection point inlet A and a road inflection point outlet B is l 2 Longitudinal resolution of image H camera Transverse resolution of the image W camera Calculating the initial minimum flying height h of the unmanned aerial vehicle according to the field angle of the camera, wherein the calculation formula is as follows:
Figure BDA0003696080430000091
Figure BDA0003696080430000092
Figure BDA0003696080430000093
Figure BDA0003696080430000094
in the formula, A hfov Representing the angle of view of the unmanned aerial vehicle camera when shooting vertically, A dfov Representing the angle of view of the unmanned aerial vehicle camera when shooting in the diagonal direction, A vfov The view angle of the unmanned aerial vehicle camera during horizontal shooting is shown.
On this basis, in order to make the target transport vehicle not exceed unmanned aerial vehicle's field of vision when changing speed or turning to, have sufficient reaction time to adjust the gesture of cloud platform, can increase certain height (predetermined flying height adjustment threshold value) as unmanned aerial vehicle's target flying height on the basis of the above-mentioned unmanned aerial vehicle initial minimum flying height h who calculates the result.
In the flight process of the unmanned aerial vehicle, if the fact that the transport vehicle is close to a turning point is detected according to position information, the cradle head is controlled through an image control algorithm to lock the transport vehicle, particularly a fan blade, so that the transport vehicle is located in a picture vision center, a target area is automatically focused and amplified for shooting, the unmanned aerial vehicle can better observe the posture of the fan blade, and the posture of the fan blade is timely adjusted according to the conditions around a road; in addition, in order to avoid the transport vehicle from generating side turning or safety accidents, a background system of the unmanned aerial vehicle is used for giving an alarm and prompting before the vehicle reaches a turning point, so that transport personnel can pay attention to the vehicle speed and the turning angle in advance and control the vehicle speed and the turning angle well, and the vehicle can efficiently and safely pass through the road risk point.
S6, acquiring the posture of the fan blade according to the fan blade image shot by the unmanned aerial vehicle, and performing collision detection according to the posture of the fan blade and the three-dimensional scene model of the road to obtain a collision detection result.
In one embodiment, the step of obtaining the posture of the fan blade according to the fan blade image shot by the unmanned aerial vehicle comprises:
identifying a fan blade image shot by the unmanned aerial vehicle by using a pre-constructed full convolution neural network to obtain an image identification result;
obtaining the image pixel position of the fan blade according to the image recognition result, and calculating the position coordinates corresponding to the end points of the two ends of the fan blade according to the unmanned aerial vehicle camera parameter and the holder angle parameter;
and acquiring the postures of the fan blades according to the position coordinates corresponding to the end points of the two ends of the fan blades.
Specifically, in the embodiment, the fan blade is intelligently identified by adopting a full convolution neural network model, information of the fan blade is labeled by a data labeling tool labelme, the information comprises a fan blade picture and a fan blade mask, the fan blade picture and the fan blade mask are used for forming an image-mask pair for training the full convolution neural network model, and a model loss function value is calculated and recorded in a training process; and then, semantically segmenting a fan blade picture acquired in an actual application scene by adopting a trained full convolution neural network model, acquiring a fan blade prediction mask, further identifying the fan blade to obtain an image identification result, calculating the image pixel position of the fan blade according to the image identification result, calculating position coordinates corresponding to end points at two ends of the fan blade by combining parameters such as unmanned aerial vehicle camera parameters and a holder angle, and judging the posture of the fan blade according to the position coordinates corresponding to the end points at the two ends of the fan blade.
In one embodiment, the step of performing collision detection according to the fan blade attitude and the road three-dimensional scene model to obtain a collision detection result includes:
taking a line segment between end points of two ends of the fan blade as a first line segment;
traversing all planes in the road three-dimensional scene model, and taking a line segment between any two points in the planes as a second line segment;
calculating the shortest spatial distance between the first line segment and the second line segment;
and comparing the shortest spatial distance with a preset fan blade safe distance threshold value to judge the safety of the fan blade relative to the road three-dimensional scene model, if all the shortest spatial distances are detected to be greater than the fan blade safe distance threshold value, judging that the fan blade is in a safe range according to a collision result, otherwise, judging that the minimum value of the shortest spatial distances in the traversing process is smaller than the safe distance, and judging that the fan blade collides according to the collision result.
Specifically, as shown in fig. 5, in the present embodiment, safety of a fan blade with respect to a three-dimensional scene model of a road is detected to determine whether the fan blade collides with obstacles such as a mountain wall and trees on both sides of the road, in the present embodiment, a capsule body collision detection method is adopted to calculate safety of the fan blade with respect to the three-dimensional scene model of the road, the identified fan blade is regarded as a capsule, the three-dimensional scene model of the road is regarded as a plurality of planes, and whether the fan blade collides with any plane of the three-dimensional scene model of the road is detected 1 And P 2 To connect the endpoint P 1 And P 2 The connecting line between them is used as the first line segment S 1 Two points P in any plane of the road three-dimensional scene model 3 、P 4 The formed line segment is used as a second line segment S 2 Then calculate S 1 And S 2 I.e., the present embodiment will connect the line segment S 1 And S 2 The end points of the two ends of the shortest line segment S are respectively marked as P 5 And P 6 And comparing the shortest line segment S (shortest spatial distance) with a preset fan blade safety distance threshold value R, and judging the safety of the fan blade relative to the road three-dimensional scene model:
Figure BDA0003696080430000111
for the end points P5 and P6 of the shortest line segment S, the calculation can be performed as follows:
Figure BDA0003696080430000112
Figure BDA0003696080430000113
P 5 =P 3 +S 2 ×min(0,max(1,λ 1 ))
P 6 =P 1 -S 1 ×min(0,max(1,λ 2 ))
and S7, making a fan blade transportation planning strategy based on the collision detection result.
In this embodiment, when detecting that fan blade can collide, instruct the transportation personnel to take corresponding measure through unmanned aerial vehicle, adjust fan blade gesture in order to avoid the barrier, for example: the azimuth angle of the fan blade is adjusted through a hydraulic mechanism of the transport vehicle, and if the obstacle cannot be avoided, the road is reformed or the obstacle is cleared.
The embodiment of the invention provides a fan blade transportation supervision method, which is characterized in that an unmanned aerial vehicle is used for surveying a fan blade transportation road, and supervision points of the unmanned aerial vehicle are designed according to a road three-dimensional scene model and fan blade transportation characteristics, so that the unmanned aerial vehicle can better observe the posture of a fan blade, and simultaneously, the fan blade is subjected to collision detection to check the safety of the fan blade relative to the road three-dimensional scene model, thereby ensuring that the fan blade effectively avoids obstacles, improving the trafficability and the safety and reducing the transportation cost of the fan blade. Compare in current manual work and carry out road reconnaissance technique, this application can realize automatic the cruising, has improved fan blade's conveying efficiency greatly, and can provide effective reference basis for fan blade when meetting the obstacle in the transportation, formulates and demolish the obstacle scheme.
The sequence numbers of the above processes do not mean the execution sequence, and the execution sequence of each process should be determined by the function and the inherent logic, and should not be limited in any way to the implementation process of the embodiment of the present application.
In one embodiment, as shown in FIG. 6, an embodiment of the present invention provides a wind turbine blade transport supervision system, the system comprising:
the three-dimensional reconstruction module 101 is used for performing three-dimensional reconstruction based on the acquired transportation scene data to obtain a road three-dimensional scene model;
the road evaluation module 102 is used for extracting road point cloud data according to the transportation scene data, acquiring a road inflection point according to the road point cloud data, and establishing a simulation model; the simulation model is used for simulating the passing safety of the vehicle by utilizing the road inflection point to obtain a simulation result;
the unmanned aerial vehicle monitoring module 103 is used for determining a target transport vehicle based on the simulation result and pre-collected fan blade parameters; the system is also used for determining the flight state of the unmanned aerial vehicle according to the running state of the target transport vehicle and the road inflection point so as to enable the fan blade to be always positioned in the visual field area of the unmanned aerial vehicle;
the collision detection module 104 is configured to obtain a fan blade attitude according to a fan blade image shot by the unmanned aerial vehicle, and perform collision detection according to the fan blade attitude and the road three-dimensional scene model to obtain a collision detection result; and the system is also used for making a fan blade transportation planning strategy based on the collision detection result.
For specific limitations of a wind turbine blade transportation supervision system, reference may be made to the above limitations of a wind turbine blade transportation supervision method, which are not described herein in detail. Those of ordinary skill in the art will appreciate that the various modules and steps described in connection with the embodiments disclosed herein may be implemented in hardware, software, or a combination of both. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the invention provides a fan blade transportation supervision system, wherein a three-dimensional reconstruction module is used for carrying a laser radar and a visible light cloud platform camera by using an unmanned aerial vehicle to realize three-dimensional reconstruction; the road evaluation module is used for simulating passing safety of vehicles, the unmanned aerial vehicle monitoring module is used for designing an unmanned aerial vehicle supervision point position according to a road three-dimensional scene model and fan blade transportation characteristics, and the collision detection module is used for performing collision detection on the fan blade to check safety of the fan blade relative to the road three-dimensional scene model. Compared with the prior art, the automatic cruise control system has the advantages that automatic cruise is achieved, the transportation efficiency and the safety of the wind turbine generator are improved, and the transportation cost is reduced.
FIG. 7 is a computer device including a memory, a processor, and a transceiver connected via a bus according to an embodiment of the present invention; the memory is used to store a set of computer program instructions and data and may transmit the stored data to the processor, which may execute the program instructions stored by the memory to perform the steps of the above-described method.
Wherein the memory may comprise volatile memory or nonvolatile memory, or may comprise both volatile and nonvolatile memory; the processor may be a central processing unit, a microprocessor, an application specific integrated circuit, a programmable logic device, or a combination thereof. By way of example, and not limitation, the programmable logic device described above may be a complex programmable logic device, a field programmable gate array, general array logic, or any combination thereof.
In addition, the memory may be a physically separate unit or may be integrated with the processor.
It will be appreciated by those of ordinary skill in the art that the architecture shown in fig. 7 is a block diagram of only a portion of the architecture associated with the present solution and is not intended to limit the computing devices to which the present solution may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have the same arrangement of components.
In one embodiment, the present invention provides a computer readable storage medium, on which a computer program is stored, and the computer program implements the steps of the above method when executed by a processor.
According to the fan blade transportation supervision method, the system, the equipment and the medium, a transportation scene is three-dimensionally reconstructed, and the supervision point position of the unmanned aerial vehicle is set according to the road three-dimensional scene model and the fan blade transportation characteristics, so that the unmanned aerial vehicle surveying efficiency is improved; meanwhile, the embodiment of the invention carries out collision detection on the fan blade in the transportation process so as to guide transportation personnel to take corresponding measures in time through the unmanned aerial vehicle, thereby ensuring that the fan blade can effectively avoid obstacles, improving the passing performance and the safety, reducing the damage of the fan blade and reducing the transportation cost of the fan blade.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions according to the embodiments of the present invention are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in, or transmitted from one computer-readable storage medium to another computer-readable storage medium, the computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device including one or more available media.
Those skilled in the art will appreciate that all or part of the processes in the methods according to the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and the computer program can include the processes according to the embodiments of the methods described above when executed.
The above-mentioned embodiments only express some preferred embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and substitutions without departing from the technical principle of the present invention, and such modifications and substitutions should also be considered as the protection scope of the present application. Therefore, the protection scope of the present patent application shall be subject to the protection scope of the claims.

Claims (10)

1. A fan blade transportation supervision method is characterized by comprising the following steps:
carrying out three-dimensional reconstruction based on the collected transportation scene data to obtain a road three-dimensional scene model;
extracting road point cloud data according to the transportation scene data, acquiring a road inflection point according to the road point cloud data, and establishing a simulation model;
simulating the passing safety of the vehicle by using the simulation model and the road inflection point to obtain a simulation result;
determining a target transport vehicle based on the simulation result and pre-collected fan blade parameters;
determining the flight state of the unmanned aerial vehicle according to the running state of the target transport vehicle and the road inflection point, so that the fan blade is always positioned in the visual field area of the unmanned aerial vehicle;
acquiring the posture of a fan blade according to a fan blade image shot by an unmanned aerial vehicle, and performing collision detection according to the posture of the fan blade and the road three-dimensional scene model to obtain a collision detection result;
and formulating a fan blade transportation planning strategy based on the collision detection result.
2. The wind turbine blade transportation supervision method according to claim 1, wherein the step of performing three-dimensional reconstruction based on the collected transportation scene data to obtain a road three-dimensional scene model comprises:
determining a target transportation route according to the position information of the wind power plant and the road condition information, and acquiring laser point cloud data and video image data of the target transportation route and the surrounding environment thereof;
carrying out combined calibration on laser point cloud data and video image data in the same time period, and carrying out registration on the laser point cloud data to obtain registered laser point cloud data;
performing corner feature extraction on the video image data to obtain video image feature data;
projecting the registered laser point cloud data to the video image characteristic data to obtain fusion data;
and calibrating the fusion data, and performing three-dimensional reconstruction according to the calibrated fusion data to obtain a road three-dimensional scene model.
3. The fan blade transportation supervision method according to claim 1, wherein the step of obtaining a road inflection point according to the road point cloud data and establishing a simulation model comprises:
determining a track buffer area according to the collected historical track information of the unmanned aerial vehicle;
segmenting the road point cloud data by using a preset segmentation length, and segmenting a ground point cloud;
constructing a joint characteristic value according to the transport road point cloud echo intensity and elevation change, and extracting the ground point cloud to obtain road edge points;
clustering the road edge points to obtain a road boundary;
extracting a road inflection point according to the road boundary;
acquiring road parameters according to the road inflection point and the road point cloud data;
establishing a simulation model according to pre-collected fan transportation device parameters and the road parameters;
the fan transportation device parameters comprise transportation vehicle parameters and fan blade parameters, the fan blade parameters comprise fan blade length and weight, and the road parameters comprise road width, road inflection radius and road bearing capacity.
4. The wind turbine blade transportation supervision method according to claim 1, wherein the step of determining the flight status of the drone according to the driving status of the target transportation vehicle and the road inflection point comprises:
determining the flight speed of the unmanned aerial vehicle according to the running state of the target transport vehicle so as to enable the unmanned aerial vehicle and the target transport vehicle to keep relatively static;
determining the entrance and exit distance of the road inflection point according to the road inflection point;
calculating the initial minimum flying height of the unmanned aerial vehicle according to the field angles of the unmanned aerial vehicle in different shooting directions, the pre-collected fan blade length and the access distance of the road inflection point;
setting a flying height adjusting threshold value according to the road terrain height change;
and dynamically adjusting the flying height of the unmanned aerial vehicle according to the initial minimum flying height and the flying height adjusting threshold value to obtain the target flying height of the unmanned aerial vehicle.
5. The wind turbine blade transportation regulatory method of claim 4, wherein the initial minimum flying height of the drone is calculated as:
Figure FDA0003696080420000031
Figure FDA0003696080420000032
Figure FDA0003696080420000033
Figure FDA0003696080420000034
in the formula I 1 Representing the length of the fan blade, alpha representing the angle between the fan blade and the horizontal plane of the target transport vehicle, l 2 Represents the distance between two points, H, of the road inflection entrance and the road inflection exit camera Representing the longitudinal resolution, W, of the image camera Representing the image lateral resolution, h representing the initial minimum flying height of the drone, A hfov Representing the angle of view of the unmanned aerial vehicle camera when shooting vertically, A dfov Representing the angle of view of the unmanned aerial vehicle camera when shooting in the diagonal direction, A vfov The view angle of the unmanned aerial vehicle camera during horizontal shooting is shown.
6. The wind turbine blade transportation supervision method according to claim 1, wherein the step of obtaining a wind turbine blade attitude from a wind turbine blade image captured by the drone comprises:
identifying a fan blade image shot by an unmanned aerial vehicle by using a pre-constructed full convolution neural network to obtain an image identification result;
obtaining the image pixel position of the fan blade according to the image recognition result, and calculating the position coordinates corresponding to the end points at the two ends of the fan blade according to the unmanned aerial vehicle camera parameter and the holder angle parameter;
and acquiring the posture of the fan blade according to the position coordinates corresponding to the end points of the two ends of the fan blade.
7. The method of claim 1, wherein the step of performing collision detection based on the fan blade attitude and the three-dimensional scene model of the road comprises:
taking a line segment between end points of two ends of the fan blade as a first line segment;
traversing all planes in the road three-dimensional scene model, and taking a line segment between any two points in the planes as a second line segment;
calculating the shortest spatial distance between the first line segment and the second line segment;
and comparing the shortest space distance with a preset fan blade safety distance threshold value to judge the safety of the fan blade relative to the road three-dimensional scene model.
8. A fan blade transport supervisory system, the system comprising:
the three-dimensional reconstruction module is used for performing three-dimensional reconstruction based on the collected transportation scene data to obtain a road three-dimensional scene model;
the road evaluation module is used for extracting road point cloud data according to the transportation scene data, acquiring a road inflection point according to the road point cloud data and establishing a simulation model; the simulation model is used for simulating the passing safety of the vehicle by utilizing the road inflection point to obtain a simulation result;
the unmanned aerial vehicle monitoring module is used for determining a target transport vehicle based on the simulation result and pre-collected fan blade parameters; the unmanned aerial vehicle flight state detection device is also used for determining the flight state of the unmanned aerial vehicle according to the running state of the target transport vehicle and the road inflection point so as to enable the fan blade to be always positioned in the field of view area of the unmanned aerial vehicle;
the collision detection module is used for acquiring the posture of the fan blade according to the fan blade image shot by the unmanned aerial vehicle, and performing collision detection according to the posture of the fan blade and the road three-dimensional scene model to obtain a collision detection result; and the system is also used for making a fan blade transportation planning strategy based on the collision detection result.
9. A computer device, characterized by: comprising a processor coupled to a memory for storing a computer program and a memory for executing the computer program stored in the memory to cause the computer device to perform the method of any of claims 1 to 7.
10. A computer-readable storage medium characterized by: the computer-readable storage medium has stored thereon a computer program which, when executed, implements the method of any of claims 1 to 7.
CN202210674915.5A 2022-06-15 2022-06-15 Fan blade transportation supervision method, system, equipment and medium Active CN115100379B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210674915.5A CN115100379B (en) 2022-06-15 2022-06-15 Fan blade transportation supervision method, system, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210674915.5A CN115100379B (en) 2022-06-15 2022-06-15 Fan blade transportation supervision method, system, equipment and medium

Publications (2)

Publication Number Publication Date
CN115100379A true CN115100379A (en) 2022-09-23
CN115100379B CN115100379B (en) 2024-06-25

Family

ID=83290077

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210674915.5A Active CN115100379B (en) 2022-06-15 2022-06-15 Fan blade transportation supervision method, system, equipment and medium

Country Status (1)

Country Link
CN (1) CN115100379B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116543322A (en) * 2023-05-17 2023-08-04 深圳市保臻社区服务科技有限公司 Intelligent property routing inspection method based on community potential safety hazards
CN118037964A (en) * 2024-04-11 2024-05-14 中国电建集团江西省水电工程局有限公司 BIM-based wind power equipment transportation virtual model generation method and device
CN118032063A (en) * 2024-04-12 2024-05-14 中国电建集团江西省水电工程局有限公司 Wind power equipment transportation topography measurement system and method based on unmanned aerial vehicle oblique photography
CN118133434A (en) * 2024-05-08 2024-06-04 桉睿数字化科技(北京)有限公司 Digital ocean engineering module transportation simulation method and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113205218A (en) * 2021-05-07 2021-08-03 湖北省电力勘测设计院有限公司 Wind power plant micro-addressing method based on unmanned aerial vehicle camera mapping
CN214608073U (en) * 2020-12-17 2021-11-05 国家电投集团江西水电检修安装工程有限公司 A unmanned aerial vehicle for fan blade detects
US20210358206A1 (en) * 2020-05-14 2021-11-18 Star Institute Of Intelligent Systems Unmanned aerial vehicle navigation map construction system and method based on three-dimensional image reconstruction technology
CN114020002A (en) * 2021-12-20 2022-02-08 复亚智能科技(太仓)有限公司 Method, device and equipment for inspecting fan blade by unmanned aerial vehicle, unmanned aerial vehicle and medium
CN114357843A (en) * 2022-01-11 2022-04-15 中国电建集团江西省水电工程局有限公司 Method for carrying out numerical collision experiment simulation aiming at wind power equipment transportation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210358206A1 (en) * 2020-05-14 2021-11-18 Star Institute Of Intelligent Systems Unmanned aerial vehicle navigation map construction system and method based on three-dimensional image reconstruction technology
CN214608073U (en) * 2020-12-17 2021-11-05 国家电投集团江西水电检修安装工程有限公司 A unmanned aerial vehicle for fan blade detects
CN113205218A (en) * 2021-05-07 2021-08-03 湖北省电力勘测设计院有限公司 Wind power plant micro-addressing method based on unmanned aerial vehicle camera mapping
CN114020002A (en) * 2021-12-20 2022-02-08 复亚智能科技(太仓)有限公司 Method, device and equipment for inspecting fan blade by unmanned aerial vehicle, unmanned aerial vehicle and medium
CN114357843A (en) * 2022-01-11 2022-04-15 中国电建集团江西省水电工程局有限公司 Method for carrying out numerical collision experiment simulation aiming at wind power equipment transportation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
MAHMOOD SHAFIEE: "Unmanned Aerial Drones for Inspection of Offshore Wind Turbines: A Mission-Critical Failure Analysis", 《ROBOTICS》, 1 January 2021 (2021-01-01) *
刘力: "基于无人机的海事立体监管体系框架", 《中国航海》, 25 March 2015 (2015-03-25), pages 68 - 70 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116543322A (en) * 2023-05-17 2023-08-04 深圳市保臻社区服务科技有限公司 Intelligent property routing inspection method based on community potential safety hazards
CN118037964A (en) * 2024-04-11 2024-05-14 中国电建集团江西省水电工程局有限公司 BIM-based wind power equipment transportation virtual model generation method and device
CN118032063A (en) * 2024-04-12 2024-05-14 中国电建集团江西省水电工程局有限公司 Wind power equipment transportation topography measurement system and method based on unmanned aerial vehicle oblique photography
CN118133434A (en) * 2024-05-08 2024-06-04 桉睿数字化科技(北京)有限公司 Digital ocean engineering module transportation simulation method and system

Also Published As

Publication number Publication date
CN115100379B (en) 2024-06-25

Similar Documents

Publication Publication Date Title
CN115100379B (en) Fan blade transportation supervision method, system, equipment and medium
US11885910B2 (en) Hybrid-view LIDAR-based object detection
Zhu et al. 3d lidar point cloud based intersection recognition for autonomous driving
CA2994511C (en) Condition detection using image processing
US10127449B2 (en) Condition detection using image processing
KR102235787B1 (en) Device and method for monitoring a berthing
Zormpas et al. Power transmission lines inspection using properly equipped unmanned aerial vehicle (UAV)
CN106708073B (en) A kind of quadrotor system of independent navigation power-line patrolling fault detection
CN112348034A (en) Crane defect detection system based on unmanned aerial vehicle image recognition and working method
Liu et al. Point cloud segmentation based on Euclidean clustering and multi-plane extraction in rugged field
CN114020015A (en) Unmanned aerial vehicle path planning system and method based on barrier map bidirectional search
CN115127510A (en) Triphibian three-dimensional unmanned multi-platform linkage landslide intelligent patrol system
Savva et al. ICARUS: Automatic autonomous power infrastructure inspection with UAVs
JP7153820B2 (en) Method, System and Apparatus for Forced Landing Path Planning of Aircraft Based on Image Identification
CN113050685B (en) Autonomous inspection method for underground unmanned aerial vehicle of coal mine
CN117406771B (en) Efficient autonomous exploration method, system and equipment based on four-rotor unmanned aerial vehicle
KR102372687B1 (en) Learning method and learning device for heterogeneous sensor fusion by using merging network which learns non-maximum suppression
CN113674355A (en) Target identification and positioning method based on camera and laser radar
KR20210090573A (en) Method and device for monitoring harbor and ship considering sea level
CN117369479A (en) Unmanned aerial vehicle obstacle early warning method and system based on oblique photogrammetry technology
CN117392641A (en) Collision early warning and navigation early warning method and system for aircraft tractor
Hudjakov et al. Aerial imagery terrain classification for long-range autonomous navigation
Muffert et al. Stix-fusion: A probabilistic stixel integration technique
CN113935946A (en) Method and device for detecting underground obstacle in real time
Inoue et al. Autonomous Navigation and Obstacle Avoidance in an Orchard Using Machine Vision Techniques for a Robotic Mower

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant