CN115440050B - Mine unmanned information transmission optimization method and system - Google Patents
Mine unmanned information transmission optimization method and system Download PDFInfo
- Publication number
- CN115440050B CN115440050B CN202211388008.0A CN202211388008A CN115440050B CN 115440050 B CN115440050 B CN 115440050B CN 202211388008 A CN202211388008 A CN 202211388008A CN 115440050 B CN115440050 B CN 115440050B
- Authority
- CN
- China
- Prior art keywords
- video frame
- similarity
- image
- frame image
- unmanned
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
- G08G1/096708—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control
- G08G1/096725—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control where the received information generates an automatic action on the vehicle control
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
- G08G1/096766—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission
- G08G1/096775—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission where the origin of the information is a central station
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0968—Systems involving transmission of navigation instructions to the vehicle
- G08G1/096805—Systems involving transmission of navigation instructions to the vehicle where the transmitted instructions are used to compute a route
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0968—Systems involving transmission of navigation instructions to the vehicle
- G08G1/096833—Systems involving transmission of navigation instructions to the vehicle where different aspects are considered when computing the route
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Atmospheric Sciences (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Chemical & Material Sciences (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Analytical Chemistry (AREA)
- Mathematical Physics (AREA)
- Traffic Control Systems (AREA)
Abstract
Aiming at a semi-closed scene of a mine, the method has the characteristics of relatively fixed driving route, relatively strong road closure and relatively few emergency situations, optimizes the rule of uploading information by the vehicle-mounted camera of the unmanned mine car, judges the similarity between the video frame information acquired by the vehicle-mounted camera in real time and the background video frame information, judges whether an emergency situation occurs according to the similarity, and uploads the video information acquired by the vehicle-mounted camera in real time only when the emergency situation occurs; furthermore, by establishing a burst condition data set, secondarily comparing the initial target video frame information with the burst condition data set, and setting different similarities according to different degrees of influence of each burst condition on unmanned decision making, the uploaded video frame information can better reflect whether the burst condition occurs or not.
Description
Technical Field
The invention relates to the technical field of unmanned driving, in particular to an information transmission optimization method and system for mine unmanned driving.
Background
The unmanned technology is characterized in that the peripheral environment of the unmanned vehicle is sensed by utilizing various technologies such as radar, laser, ultrasonic wave, GPS, odometer and computer vision, obstacles and various identification plates are identified through an advanced computer control system, and a proper path is planned to control the unmanned vehicle to run.
As shown in fig. 1, due to complexity of unmanned decision making, an unmanned vehicle generally transmits data acquired by sensors to a cloud server and an edge node for operation, so as to make a decision, however, since the cloud server controls multiple unmanned vehicles at the same time, a large amount of data is simultaneously uploaded to the cloud server and the edge node by a large amount of unmanned vehicles, which may cause a problem of network congestion, when a vehicle is driving, various sensors such as a camera may acquire data at a millisecond-level frequency, and reports indicate that the data amount acquired by an unmanned vehicle per day is up to 4TB, meanwhile, actual observation finds that data generated in real time by an unmanned vehicle may even approach 2 GB/s, if all vehicles on a certain road section transmit all data to the cloud server, network bandwidth may be exhausted, thereby causing network congestion and data transmission delay, and slightly affecting user experience of an auxiliary system and a vehicle-mounted entertainment system, which may cause that an unmanned system may not execute a calculation task of automatic driving in time, and loses control over vehicles, thereby causing serious potential safety hazards.
When the problem of large data transmission quantity is solved, on one hand, network blockage and transmission delay possibly caused are prevented on hardware by updating an information transmission architecture and increasing network transmission bandwidth; but this will undoubtedly add significant cost to the unmanned system; on the other hand, research is carried out on the aspect of software, data information acquired by a sensor is processed, transmission optimization is carried out on an information source, and the amount of transmitted information is reduced, for example, patent document (CN 112489072A) discloses a transmission load optimization method for vehicle-mounted video sensing information, when video image data of environment sensing information acquired in real time needs to be transmitted to a vehicle control module, a static background image and a dynamic foreground image in a real-time video frame image are separated before transmission each time, the separated static background image is transmitted during initial transmission, and only the separated dynamic foreground image is transmitted during transmission each time later; after receiving the dynamic foreground image, the vehicle end fuses the dynamic foreground image and the initially received static background image to obtain real-time video frame image data required to be transmitted;
the method can greatly reduce the transmitted information amount and can greatly reduce the time delay of information transmission, but the use scene of the method is an urban road and an open scene, while the driving scene of the unmanned mine car is a mining area scene, and the method belongs to a semi-closed scene and has the characteristics of relatively fixed driving route, relatively strong road closure and relatively few emergencies, so the method cannot be well applied to the field of information optimized transmission of the unmanned mine car.
Disclosure of Invention
The invention aims to solve the technical problems that according to the defects of the technical scheme, according to the semi-closed scene of the mine, the mine has the characteristics of relatively fixed driving route, relatively strong road closure and relatively few emergency situations, the scene that preset driving parameters possibly need to be changed suddenly in the driving process of the unmanned mine car is listed as emergency situation information, and the uploading information amount is effectively reduced and the uploading time delay is reduced by identifying the emergency situation video information of the vehicle-mounted camera and only uploading the emergency situation video information.
The concept of the present application will be described first with reference to the drawings. It should be noted that the following descriptions of the concepts are only intended to make the contents of the present application more easily understood, and do not represent limitations on the scope of the present application.
In order to achieve the above object, according to an aspect of the present invention, a mine unmanned information transmission optimization method includes:
step 1: closing a running route of the unmanned mine car, removing obstacles in the road of the running route, and detecting the state of a sensor in the unmanned mine car;
step 2: adjusting the unmanned mine car to be in a driver driving mode, driving the whole driving route according to recommended driving parameters, and taking video data acquired by a vehicle-mounted camera as background video information;
specifically, the recommended driving parameters are driving parameters obtained by a driver considering safety and ore transportation efficiency, and comprise driving speed and driving acceleration;
and step 3: converting the background video information into a background video frame image, and recording the position coordinates of the frame of background video frame image;
and 4, step 4: the unmanned mine car executes a transportation task in an unmanned mode, video information is collected in real time through a vehicle-mounted camera and converted into a real-time video frame image, and a vehicle-mounted GPS records the position coordinate of the unmanned mine car in real time;
and 5: the vehicle-mounted server judges the similarity between the video frame image collected by the vehicle-mounted camera and the background video frame image under the position coordinate; taking the video frame image with the similarity larger than the first similarity as an initial target video frame image, wherein the initial target video frame is a video frame image which is preliminarily determined to be uploaded after being compared with the background video frame image;
furthermore, weather is different, light is clear and dark, and the acquisition time of the camera is different, and the data acquired by the camera and the background video data may have difference in image brightness, so that the similarity judgment is affected, and the video frame image acquired by the vehicle-mounted camera and the background video frame image can be subjected to normalization processing in the same mode by taking a pixel as a unit, so as to avoid interference of the weather and the acquisition time on video information and background video information; the normalization processing formula is as follows:
in the formula, x t Expressing pixel gray value after normalization processing, wherein x is the pixel gray value of the video image shot by the vehicle-mounted camera in real time, x min Is the minimum value, x, of a pixel in the image matrix max Is the maximum value of the pixel in the image matrix, and a is the coefficient;
specifically, the similarity calculation formula is:
wherein F is a similarity function between a real-time video frame image pixel set A and a background video frame image pixel set B,
for the intersection of the set of real-time video frame image pixels a and the set of background video frame image pixels B,
is the union of ase:Sub>A set of pixels of the real-time video frame image A and ase:Sub>A set of pixels of the background video frame image B, A-B is ase:Sub>A feature belonging to the set of pixels of the real-time video frame image A but not to the pixels of the background video frame image B, and B-A is ase:Sub>A feature belonging to the pixels of the background video frame image B but not to the set of pixels of the real-time video frame image A.
In this embodiment, the method further performs screening and determining on the initial target video frame data, screens out video frame information that is unimportant to the unmanned decision in the initial target video frame data, and retains video frame information that is more important to the unmanned decision as target information for determining uploading, and specifically includes:
step 6: acquiring images of all types of vehicles in a mining area through a vehicle-mounted camera of the unmanned mine car as a subset S of images of sudden conditions of the vehicles v And setting images of a plurality of ore obstacles on the road surface as a road obstacle emergent condition image subset S o And taking the image deviating from the center line of the road during the driving process of the unmanned mine car as the subset S of the image of the emergent condition deviating from the center line d And establishes an emergency condition image set S based on the vehicle emergency condition image subset Sv, the obstacle emergency condition image subset So, and the off-center line emergency condition image subset Sd,
wherein S = S v + S o + S d
And 7: respectively comparing the initial target video frame image with each frame image in the burst condition image set S, and further determining uploaded target information;
since there are fewer images in the emergency image set S, the initial target video frame image needs to be compared with each image in the emergency image set S, and the similarity between the initial target video frame image and each frame image needs to be calculated;
specifically, the step 7 includes:
step 7.1: calculating the similarity between the initial target video frame image and each frame image in the burst condition image set S; specifically, at this time, the initial video frame image is defined as a pixel set C, each frame of image in the burst status image set S is defined as a pixel set D, and the initial target video frame image is compared with each image in the burst status image set S to obtain a plurality of similarities;
step 7.2: and determining the uploading target information according to the similarity obtained in the step 7.1.
The step 7.2 specifically comprises:
a. if the highest similarity value obtained in the step 7.1 is the subset S of the initial video frame image and the vehicle emergency image v If the image is obtained by calculation, the initial target frame image is proved to be a vehicle appearing on the front road, at the moment, a second similarity is set, and if the highest value of the similarity is greater than the second similarity, the initial target video frame image is uploaded;
b: if the highest similarity value obtained in the step 7.1 is the initial video frame image and the obstacle emergency image subset S o If the initial target frame image is obtained through calculation, the initial target frame image is proved to be an ore obstacle emergency situation on the road in front, at the moment, a third similarity is set, and if the highest value of the similarity is larger than the third similarity, the initial target video frame image is uploaded;
c: if the highest similarity value obtained in the step 7.1 is the initial video frame image and the off-center line emergency image subset S d If the initial target frame image is obtained by calculation, the initial target frame image is proved to be the deviation road of the unmanned mine carSetting a fourth similarity in case of a center line burst condition, and uploading the initial target video frame image if the highest value of the similarity is greater than the fourth similarity;
setting different similarity values according to different degrees of influence of each emergency on unmanned driving decision, and performing targeted secondary filtering on uploaded data;
when vehicles appear on the road at the front side, because the danger of the emergency situation is small, the information needing to be uploaded is less, and at the moment, the second similarity is set to be a larger numerical value, so that most of the initial target video frame data is filtered for the second time;
when the ore barrier type emergency situation appears on the road in front, the danger coefficient of the emergency situation is higher, so more information needs to be uploaded, and at the moment, the third similarity is set to be a moderate numerical value;
when the unmanned mine car deviates from the road center line type emergency situation, the emergency risk coefficient is the highest, a large amount of information needs to be uploaded for decision making, and therefore the fourth similarity is set to be a smaller numerical value, so that less data are filtered, and most data are uploaded;
it is worth emphasizing that the setting of different similarity values is more helpful for the unmanned control system to make an accurate decision, in this embodiment, the second similarity > the third similarity > the fourth similarity are set;
and executing data transmission operation on the target video frame information.
And the cloud server performs image fusion and replaces the background video information under the coordinate according to the uploaded information and the background video information, and decides the driving parameters of the unmanned mine car by combining with other sensor information.
According to another aspect of the present invention, a mine unmanned information transmission optimization system includes:
the vehicle-mounted camera is used for collecting video data;
the vehicle-mounted server is used for operating the unmanned mine information transmission optimization method;
and the transmission module is used for realizing the transmission of the video frame data.
Based on the technical scheme, the mine unmanned information transmission optimization method and system have the following technical effects:
aiming at a semi-closed scene of a mine, the method has the characteristics of relatively fixed driving route, relatively strong road closure and relatively few emergency situations, optimizes the rule of uploading information by the vehicle-mounted camera of the unmanned mine car, judges the similarity between the video frame information acquired by the vehicle-mounted camera in real time and the background video frame information, judges whether an emergency situation occurs or not according to the similarity, and uploads the video information acquired by the vehicle-mounted camera in real time only when the emergency situation occurs;
by establishing the burst condition data set, carrying out secondary comparison on the initial target video frame information and the burst condition data set, and setting different similarities according to different influence degrees of each burst condition on unmanned decision, the uploaded video frame information can better reflect whether the burst condition occurs or not.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic diagram of information transmission between a cloud server, an edge node and an unmanned vehicle in the prior art;
fig. 2 is a flowchart of an information transmission optimization method for mine unmanned driving according to an embodiment of the present application.
Fig. 3 is a flowchart for determining an upload target information according to different burst conditions according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The concept to which the present application relates will be first explained below with reference to the drawings. It should be noted that the following descriptions of the concepts are only for the purpose of facilitating understanding of the contents of the present application, and do not represent limitations on the scope of the present application.
The invention aims to solve the technical problems that according to the defects of the technical scheme, according to the semi-closed scene of the mine, the vehicle-mounted mining vehicle has the characteristics of relatively fixed driving route, relatively strong road closure and relatively few emergency situations, the information uploaded by the vehicle-mounted camera is optimized, the scene that the preset driving parameters are possibly required to be changed suddenly in the driving process of the unmanned mining vehicle is identified as useful information, only the useful information is uploaded, the amount of the uploaded information is effectively reduced, and the uploading time delay is reduced.
As shown in fig. 2, a mine unmanned information transmission optimization method includes:
step 1: closing a running route of the unmanned mine car, removing obstacles in the road of the running route, and detecting the state of a sensor in the unmanned mine car;
the driving route and the driving parameters of the unmanned vehicle in the mine area are relatively fixed, and after the cloud server for controlling the unmanned mine car sends an operation command, the unmanned mine car generally runs according to the fixed driving route and the fixed driving parameters, so that in the running process of the unmanned mine car, the unmanned mine car interacts with the cloud server or the edge server to realize control aiming at preventing possible dangerous scenes, such as a large amount of ores on the road surface, vehicles in front of the road surface, deviation of the mine car from the road and the like; after a cloud server for controlling the unmanned tramcar sends an operation command, the unmanned tramcar generally travels back and forth in the route for multiple times to complete the transportation task of the operation, aiming at the fixed operation route, firstly, the route can be surveyed, ore barriers scattered on the road surface in the route are removed, the route is temporarily closed, other vehicles are not allowed to drive into the route, and meanwhile, the state of a sensor in the unmanned tramcar, particularly the state of a vehicle-mounted camera, is detected;
step 2: adjusting the unmanned mine car to be in a driver driving mode, driving the whole driving route according to recommended driving parameters, and taking video data acquired by a vehicle-mounted camera as background video information;
specifically, the recommended driving parameters are driving parameters obtained by a driver considering safety and ore transportation efficiency, and comprise driving speed and driving acceleration;
and step 3: converting the background video information into a background video frame image, and recording the position coordinates of the frame image;
and 4, step 4: the unmanned mine car executes a transportation task in an unmanned mode, video information is collected in real time through a vehicle-mounted camera and converted into a real-time video frame image, and a vehicle-mounted GPS records the position coordinate of the unmanned mine car in real time;
and 5: the vehicle-mounted server judges the similarity between the video frame image collected by the vehicle-mounted camera and the background video frame image under the position coordinate; taking a video frame image with the similarity larger than the first similarity as an initial target video frame image, wherein the initial target video frame is a video frame image which is preliminarily determined to be uploaded after being compared with the background video frame image;
furthermore, the weather is different, the light is clear and dark, and the camera acquisition time is different, the data acquired by the camera may be different from the background video data in image brightness, so that the similarity judgment is influenced, and the video information acquired by the vehicle-mounted camera and the background video information can be subjected to normalization processing in the same mode by taking a pixel as a unit, so as to avoid the interference of the weather and the acquisition time on the video information and the background video information; the normalization processing formula is as follows:
in the formula, x t Expressing pixel gray value after normalization processing, wherein x is the pixel gray value of the video image shot by the vehicle-mounted camera in real time, and x is the pixel gray value of the video image shot by the vehicle-mounted camera in real time min Is the minimum value of a pixel in the image matrix, x max Is the maximum value of the pixel in the image matrix, and a is the coefficient;
specifically, the similarity calculation formula is as follows:
wherein F is a similarity function of a real-time video frame information pixel set A and a background video frame information pixel set B,
for the intersection of the set of real-time video frame information pixels a and the set of background video frame information pixels B,
is the union of ase:Sub>A set of real-time video frame information pixels ase:Sub>A and ase:Sub>A set of background video frame information pixels B, ase:Sub>A-B is ase:Sub>A feature that belongs to the set of real-time video frame information pixels ase:Sub>A but not to the set of background video frame information pixels B, and B-ase:Sub>A is ase:Sub>A feature that belongs to the set of background video frame information pixels B but not to the set of real-time video frame information pixels ase:Sub>A.
In this embodiment, the further screening and determining of the initial target video frame data is performed, video frame information that is unimportant to the unmanned decision in the initial target video frame data is screened out, and video frame information that is more important to the unmanned decision is retained as target information for determining uploading, and specifically, the method includes:
step 6: acquiring images of all types of vehicles in a mining area through a vehicle-mounted camera of the unmanned mine car as a subset S of images of sudden conditions of the vehicles v And setting images of a plurality of ore obstacles on the road surface as a road obstacle emergent condition image subset S o And taking the image deviating from the center line of the road during the driving process of the unmanned mine car as the subset S of the image of the emergent condition deviating from the center line d And establishes an emergency condition image set S based on the vehicle emergency condition image subset Sv, the obstacle emergency condition image subset So, and the off-center line emergency condition image subset Sd,
wherein S = S v + S o + S d
As is known, due to the semi-closed nature of mines, the emergency situations often encountered in the roads of unmanned mines are mainly: the method comprises the following steps that (1) normal running of the unmanned mine car is influenced by other vehicles suddenly driving into a target road, (2) more ores are scattered on the road due to jolting of the unmanned mine car driven before, the number of the ores is large, and influence on safe running of the unmanned mine car is large, and (3) the unmanned vehicle deviates from a road center line due to the fact that the road surface is uneven, so that the risk of driving out of the target road exists, and the influence on safety of the unmanned mine car is large;
and 7: comparing the initial target video frame image with each frame image in the burst condition image set S, and further determining uploaded target information;
since there are fewer images in the emergency image set S, the initial target video frame image needs to be compared with each image in the emergency image set S, and the similarity between the initial target video frame image and each image needs to be calculated;
specifically, the step 7 includes:
step 7.1: calculating the similarity between the initial target video frame image and each frame image in the burst condition image set S; specifically, at this time, the initial video frame image is defined as a pixel set C, each frame of image in the burst status image set S is defined as a pixel set D, and the initial target video frame image is compared with each image in the burst status image set S to obtain a plurality of similarities;
and 7.2: and determining the uploading target information according to the similarity obtained in the step 7.1.
As shown in fig. 3, the step 7.2 specifically includes:
a. if the highest similarity value obtained in the step 7.1 is the initial video frame image and the vehicle emergency image subset S v If the image is obtained by calculation, the initial target frame image is proved to be a vehicle appearing on the front road, at the moment, a second similarity is set, and if the highest value of the similarity is greater than the second similarity, the initial target video frame image is uploaded;
b: if the highest similarity value obtained in the step 7.1 is the initial video frame image and the obstacle emergency image subset S o If the initial target frame image is obtained through calculation, the initial target frame image is proved to be an ore obstacle emergency situation on the road in front, at the moment, a third similarity is set, and if the highest value of the similarity is larger than the third similarity, the initial target video frame image is uploaded;
c: if the highest similarity value obtained in the step 7.1 is the initial video frame image and the off-center line emergency image subset S d If the initial target frame image is obtained by calculating the intermediate image, the initial target frame image is proved to be an emergency situation of the unmanned mine car deviating from the center line of the road, at the moment, a fourth similarity is set, and if the highest value of the similarity is greater than the fourth similarity, the initial target video frame image is uploaded;
illustratively, the emergency image set S is provided with 9 images, S v-1 ，S v-2 ，S v-3 ；S o-1 ，S o-2 ，S o-3 ；S d-1 ，S d-2 ，S d-3 Respectively calculating the similarity between the initial video frame image and the 9 images in the burst state image set, wherein in the calculation, in order to generate the discrimination, the numerical value of the similarity takes a decimal pointTwo bits, then 9 similarity values are obtained, 81.57, 81.52, 82.17, 73.14, 74.16, 79.58, 91.16, 92.01, 91.52; the initial video frame image and S d-2 The image similarity is highest, and at the moment, the initial video frame image is considered to be in a sudden situation that the unmanned mine car deviates from the center line of the road at a high probability, namely the unmanned mine car is probably driving deviating from the center line of the road at the moment and has a risk of being out of control;
setting different similarity values according to different influence degrees of each emergency on the unmanned driving decision, and performing targeted secondary filtering on uploaded data;
when vehicles appear on the road at the front side, because the danger of the emergency situation is small, the information needing to be uploaded is less, and at the moment, the second similarity is set to be a larger numerical value, so that most of the initial target video frame data is filtered for the second time;
when the ore barrier type emergency situation appears on the road in front, the danger coefficient of the emergency situation is higher, so more information needs to be uploaded, and at the moment, the third similarity is set to be a moderate numerical value;
when the unmanned mine car deviates from the road center line type emergency situation, the emergency risk coefficient is the highest, a large amount of information needs to be uploaded for decision making, and therefore the fourth similarity is set to be a smaller numerical value, so that less data are filtered, and most data are uploaded;
it is worth emphasizing that the setting of different similarity values is more helpful for the unmanned control system to make an accurate decision, in this embodiment, the second similarity > the third similarity > the fourth similarity are set;
and executing data transmission operation on the target video frame information.
And the cloud server performs image fusion and replaces the background video information under the coordinates according to the uploaded information and the background video information, and decides the driving parameters of the unmanned tramcar by combining with other sensor information.
According to another aspect of the invention, an information transmission optimization system for mine unmanned operation comprises:
the vehicle-mounted camera is used for collecting video data;
the vehicle-mounted server is used for operating the unmanned mine information transmission optimization method;
and the transmission module is used for realizing the transmission of the video frame data.
The method is used for the mine semi-closed scene and has the advantages that a driving route is fixed, road closure is strong, and emergency is less, information uploading rules of a vehicle-mounted camera of the unmanned mine car are optimized, the similarity between video frame information acquired by the vehicle-mounted camera in real time and background video frame information is judged, whether emergency occurs or not is judged according to the similarity, the video information acquired by the vehicle-mounted camera in real time is uploaded only when the emergency occurs, furthermore, the initial target video frame information and the emergency data set are secondarily compared through establishing an emergency data set, different similarities are set according to different impact degrees of unmanned decision according to each emergency, and accordingly the uploaded video frame information can better reflect whether the emergency occurs or not.
The above-described embodiments and/or implementations are only illustrative of the preferred embodiments and/or implementations for implementing the present technology, and are not intended to limit the embodiments of the present technology in any way, and those skilled in the art can make modifications or changes without departing from the scope of the technical means disclosed in the present disclosure, but should be regarded as the technical means or implementations that are substantially the same as the present invention.
Claims (8)
1. A mine unmanned information transmission optimization method is characterized in that: the method comprises the following steps:
step 1: closing a running route of the unmanned mine car, removing mineral obstacles in the road of the running route, and detecting the state of a sensor in the unmanned mine car;
and 2, step: adjusting the unmanned mine car to be in a driver driving mode, driving the whole driving route according to recommended driving parameters, and taking video data acquired by a vehicle-mounted camera as background video information;
and 3, step 3: converting the background video information into background video frame images, and recording the position coordinates of each frame of the background video frame images;
and 4, step 4: the unmanned mine car executes a transportation task in an unmanned mode, video information is collected in real time through the vehicle-mounted camera and converted into a real-time video frame image, and the position coordinate of the unmanned mine car is recorded in real time through a vehicle-mounted GPS;
and 5: the vehicle-mounted server judges the similarity between the real-time video frame image acquired by the vehicle-mounted camera and the background video frame image under the position coordinate; taking the video frame image with the similarity smaller than the first similarity as an initial target video frame image;
and 6: establishing an emergent condition image set S;
and 7: respectively comparing the initial target video frame image with each frame image in the burst condition image set S, and further determining uploaded target information;
the step 7 specifically includes: step 7.1: calculating the similarity between the initial target video frame image and each frame image in the emergency condition image set S, wherein the emergency condition image set S comprises a vehicle emergency condition image subset Sv, an obstacle emergency condition image subset So and an off-center line emergency condition image subset Sd;
step 7.2: determining uploading target information according to the similarity obtained in the step 7.1;
the step 7.2 specifically comprises:
a. if the highest similarity value obtained in the step 7.1 is the initial target video frame image and the vehicle emergency image subset S v If the image is obtained by calculation, setting a second similarity, if the phase is the same as the image, and calculating the similarity of the image and the second similarityIf the highest similarity value is greater than the second similarity, uploading the initial target video frame image;
b: if the highest similarity value obtained in the step 7.1 is the initial target video frame image and the obstacle emergency image subset S o If the highest value of the similarity is larger than the third similarity, uploading the initial target video frame image;
c: if the highest similarity value obtained in the step 7.1 is the initial target video frame image and the off-center line burst condition image subset S d And if the highest value of the similarity is greater than the fourth similarity, uploading the initial target video frame image.
2. The mine unmanned information transmission optimization method according to claim 1, wherein: the step 1 specifically comprises the following steps: after a cloud server for controlling the unmanned mine car sends an operation command, the unmanned mine car surveys the route, ore obstacles scattered on the road surface in the route are removed, the route is temporarily closed, other vehicles are not allowed to drive into the route, and meanwhile, the state of a vehicle-mounted camera in the unmanned mine car is detected.
3. The mine unmanned information transmission optimization method of claim 1, wherein: in the step 2, the recommended driving parameters are driving parameters obtained by considering safety and ore transportation efficiency of a driver, and include driving speed and driving acceleration.
4. The mine unmanned information transmission optimization method of claim 1, wherein: in the step 5, the real-time video frame image and the background video frame image acquired by the vehicle-mounted camera are subjected to normalization processing in the same mode by taking a pixel as a unit.
5. The mine unmanned information transmission optimization method according to claim 4, wherein: the normalization processing formula is as follows:
in the formula, x t Expressing pixel gray value after normalization processing, wherein x is the pixel gray value of the video image shot by the vehicle-mounted camera in real time, and x is the pixel gray value of the video image shot by the vehicle-mounted camera in real time min Is the minimum value, x, of a pixel in the image matrix max Is the maximum value of a pixel in the image matrix and a is the coefficient.
6. The mine unmanned information transmission optimization method according to claim 1, wherein: in step 5, the similarity calculation formula is as follows:
wherein, F is ase:Sub>A similarity function of ase:Sub>A real-time video frame image pixel set A and ase:Sub>A background video frame image pixel set B, A ≧ B is the intersection of the real-time video frame image pixel set A and the background video frame image pixel set B, A $ B is the union of the real-time video frame image pixel set A and the background video frame image pixel set B, A-B is the characteristic belonging to the real-time video frame image pixel set A but not the background video frame image pixel set B, and B-A is the characteristic belonging to the background video frame image pixel set B but not the real-time video frame information image set A.
7. The mine unmanned information transmission optimization method of claim 1, wherein: setting the second similarity > the third similarity > the fourth similarity.
8. An unmanned information transmission optimization system for a mine, comprising:
the vehicle-mounted camera is used for collecting video data;
an on-board server for operating the mine unmanned information transmission optimization method of any one of claims 1 to 7;
and the transmission module is used for realizing the transmission of the video frame data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211388008.0A CN115440050B (en) | 2022-11-08 | 2022-11-08 | Mine unmanned information transmission optimization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211388008.0A CN115440050B (en) | 2022-11-08 | 2022-11-08 | Mine unmanned information transmission optimization method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115440050A CN115440050A (en) | 2022-12-06 |
| CN115440050B true CN115440050B (en) | 2023-02-03 |
Family
ID=84252582
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211388008.0A Active CN115440050B (en) | 2022-11-08 | 2022-11-08 | Mine unmanned information transmission optimization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115440050B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6480769B1 (en) * | 1999-10-05 | 2002-11-12 | Komatsu Ltd. | Vehicle travel control system |
| JP2004046875A (en) * | 2002-07-03 | 2004-02-12 | Iwane Kenkyusho:Kk | Transportation automatic guiding device |
| JP2004362549A (en) * | 2003-05-09 | 2004-12-24 | Matsushita Electric Ind Co Ltd | Onboard video playback system and car navigation device |
| KR20050096995A (en) * | 2004-03-30 | 2005-10-07 | 이규백 | System and method for parking violation control using internet |
| WO2015163028A1 (en) * | 2014-04-25 | 2015-10-29 | 日立建機株式会社 | Road surface estimation device and road surface estimation system for mine work-vehicle |
| CN107330802A (en) * | 2017-06-20 | 2017-11-07 | 山东东山王楼煤矿有限公司 | A kind of intelligent Mine transportation and sales system |
| CN109855619A (en) * | 2017-11-30 | 2019-06-07 | 基本立子(北京)科技发展有限公司 | A method of optimization vehicle driving trace upload information |
| CN109949231A (en) * | 2019-02-02 | 2019-06-28 | 浙江工业大学 | A kind of method and device for urban managing information acquiring and processing |
| CN111866809A (en) * | 2020-07-29 | 2020-10-30 | 江苏徐工工程机械研究院有限公司 | Unmanned mine card remote control driving system and method |
| CN112489072A (en) * | 2020-11-11 | 2021-03-12 | 广西大学 | Vehicle-mounted video perception information transmission load optimization method and device |
| CN113919677A (en) * | 2021-09-29 | 2022-01-11 | 江苏徐工工程机械研究院有限公司 | Unmanned mine operation management system based on cloud platform |
| CN114915646A (en) * | 2022-06-16 | 2022-08-16 | 上海伯镭智能科技有限公司 | Data grading uploading method and device for unmanned mine car |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10816986B2 (en) * | 2016-08-02 | 2020-10-27 | Ge Global Sourcing Llc | Systems for vehicle collision avoidance |
-
2022
- 2022-11-08 CN CN202211388008.0A patent/CN115440050B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6480769B1 (en) * | 1999-10-05 | 2002-11-12 | Komatsu Ltd. | Vehicle travel control system |
| JP2004046875A (en) * | 2002-07-03 | 2004-02-12 | Iwane Kenkyusho:Kk | Transportation automatic guiding device |
| JP2004362549A (en) * | 2003-05-09 | 2004-12-24 | Matsushita Electric Ind Co Ltd | Onboard video playback system and car navigation device |
| KR20050096995A (en) * | 2004-03-30 | 2005-10-07 | 이규백 | System and method for parking violation control using internet |
| WO2015163028A1 (en) * | 2014-04-25 | 2015-10-29 | 日立建機株式会社 | Road surface estimation device and road surface estimation system for mine work-vehicle |
| CN107330802A (en) * | 2017-06-20 | 2017-11-07 | 山东东山王楼煤矿有限公司 | A kind of intelligent Mine transportation and sales system |
| CN109855619A (en) * | 2017-11-30 | 2019-06-07 | 基本立子(北京)科技发展有限公司 | A method of optimization vehicle driving trace upload information |
| CN109949231A (en) * | 2019-02-02 | 2019-06-28 | 浙江工业大学 | A kind of method and device for urban managing information acquiring and processing |
| CN111866809A (en) * | 2020-07-29 | 2020-10-30 | 江苏徐工工程机械研究院有限公司 | Unmanned mine card remote control driving system and method |
| CN112489072A (en) * | 2020-11-11 | 2021-03-12 | 广西大学 | Vehicle-mounted video perception information transmission load optimization method and device |
| CN113919677A (en) * | 2021-09-29 | 2022-01-11 | 江苏徐工工程机械研究院有限公司 | Unmanned mine operation management system based on cloud platform |
| CN114915646A (en) * | 2022-06-16 | 2022-08-16 | 上海伯镭智能科技有限公司 | Data grading uploading method and device for unmanned mine car |
Non-Patent Citations (1)
| Title |
|---|
| 西湾露天煤矿110 t大型矿山卡车无人驾驶***关键技术;白怡明;《露天采矿技术》;20200215(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115440050A (en) | 2022-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101963422B1 (en) | Collision-avoidance system for autonomous-capable vehicles | |
| US11714420B2 (en) | System and method for detecting occluded objects based on image processing | |
| US10663575B2 (en) | Off-road dump truck and obstacle discrimination device | |
| US10328932B2 (en) | Parking assist system with annotated map generation | |
| AU2021313775A1 (en) | Drivable surface identification techniques | |
| US20200250984A1 (en) | Pothole detection system | |
| KR102131893B1 (en) | Apparatus and method for determining the vehicle cover fault automatically | |
| CN112466141A (en) | Vehicle-road-collaboration-oriented intelligent network connection end equipment interaction method, system and storage medium | |
| AU2023202153B2 (en) | Apparatus for analyzing a payload being transported in a load carrying container of a vehicle | |
| CN112930290A (en) | Method for formulating control instructions for a motor vehicle | |
| CN114915646B (en) | Data grading uploading method and device for unmanned mine car | |
| US9939523B2 (en) | Vehicle type radar system, and method for removing an uninterested target | |
| CN115465293A (en) | Multi-sensor safety self-cognition and safety processing device and method | |
| CN115440050B (en) | Mine unmanned information transmission optimization method and system | |
| CN116279527A (en) | Vehicle running control method, control system and vehicle | |
| WO2022267266A1 (en) | Vehicle control method based on visual recognition, and device | |
| CN116152753A (en) | Vehicle information identification method and system, storage medium and electronic device | |
| CN111033169A (en) | Method and control device for estimating vehicle dimensions | |
| CN115131957A (en) | Road condition early warning method and device | |
| CN113232678A (en) | Vehicle control method and device and automatic driving vehicle | |
| CN114954570B (en) | Rail obstacle processing method, device, electronic equipment and storage medium | |
| CN117622228B (en) | Remote control method and device for unmanned automatic operation automobile in automobile | |
| CN114550105B (en) | Goods transportation method and system based on Internet of vehicles | |
| KR102281891B1 (en) | Vehicle control system for providing road icing phenomenon nitification and control method thereof | |
| US20220063559A1 (en) | Work vehicle, door state determination system, and method of determining state of work vehicle door |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: An Optimization Method and System for Information Transmission of Mine Unmanned Driving Effective date of registration: 20230802 Granted publication date: 20230203 Pledgee: Industrial Bank Co.,Ltd. Shanghai Huangpu Sub branch Pledgor: SHANGHAI BOONRAY INTELLIGENT TECHNOLOGY Co.,Ltd. Registration number: Y2023310000429 |