CN117437609A - Method, system, equipment and storage medium for correcting driving deviation of tire crane - Google Patents
Method, system, equipment and storage medium for correcting driving deviation of tire crane Download PDFInfo
- Publication number
- CN117437609A CN117437609A CN202311753037.7A CN202311753037A CN117437609A CN 117437609 A CN117437609 A CN 117437609A CN 202311753037 A CN202311753037 A CN 202311753037A CN 117437609 A CN117437609 A CN 117437609A
- Authority
- CN
- China
- Prior art keywords
- tire crane
- control
- deviation correction
- data
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 239000011159 matrix material Substances 0.000 claims abstract description 105
- 238000012937 correction Methods 0.000 claims abstract description 79
- 238000003062 neural network model Methods 0.000 claims abstract description 30
- 238000001514 detection method Methods 0.000 claims abstract description 11
- 230000004927 fusion Effects 0.000 claims abstract description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/60—Other road transportation technologies with climate change mitigation effect
- Y02T10/72—Electric energy management in electromobility
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Biodiversity & Conservation Biology (AREA)
- Image Analysis (AREA)
Abstract
The application provides a driving deviation rectifying method, a driving deviation rectifying system, driving deviation rectifying equipment and a storage medium of a tire crane, and relates to the technical field of computer vision. The computing equipment inputs the road image in the running direction of the tire crane obtained from the camera into a trained lane line detection model to obtain an image feature matrix describing the position of the lane line; acquiring motor speed, current speed difference and historical control deviation correction quantity which are acquired simultaneously with road images from electrical control equipment of a tire crane; normalizing and dimension-expanding the data acquired from the electrical control equipment to obtain a control data matrix; and after the image feature matrix and the control data matrix are fused, inputting the fused image feature matrix and the control data matrix into a trained neural network model to obtain the current control deviation correction quantity, and adjusting the speeds of a plurality of motors of the tire crane according to the current control deviation correction quantity by using the electric control equipment. The image characteristics and the electrical control data are subjected to multi-mode fusion and then input into the neural network model for reasoning, so that the accuracy of deviation correction control can be effectively improved in different running environments.
Description
Technical Field
The application relates to the technical field of computer vision, in particular to a driving deviation rectifying method, a driving deviation rectifying system, driving deviation rectifying equipment and a storage medium of a tire crane.
Background
Tyre cranes, i.e. tyre cranes, travel with a tyre chassis. Because the tire crane drives the tires by the motors at two sides, the speeds at two sides are different often caused by factors such as a car body counterweight, hoisting weight, ground flatness and the like in the running process of the tire crane, so that route deviation is generated.
At present, a mode of manually correcting and controlling the tire crane has certain error, the correction accuracy is poor, and the tire crane cannot be well adapted to different running environments.
Disclosure of Invention
In order to solve the problems, the application provides a driving deviation rectifying method, a driving deviation rectifying system, driving deviation rectifying equipment and a storage medium of a tire crane, which can effectively improve the accuracy of the deviation rectifying control in different driving environments by carrying out multi-mode fusion on image characteristics and electric control data and then inputting the image characteristics and the electric control data into a neural network model for reasoning.
In a first aspect, the present application provides a driving deviation rectifying method for a tire crane, where the method includes:
s1, acquiring a road image in the running direction of a tire crane from a camera, and inputting the road image into a trained lane line detection model to obtain an image feature matrix of the road image, wherein the image feature matrix describes the positions of lane lines in the road image;
s2, acquiring a current speed difference, a historical control deviation correction amount and a plurality of motor speeds, which are acquired at the same moment as the road image, from electrical control equipment corresponding to the tire crane, wherein the current speed difference is a motor speed difference at two sides of the tire crane;
s3, normalizing the current speed difference, the historical control deviation correction amount and the motor speeds, then splicing the normalized current speed difference, the historical control deviation correction amount and the motor speeds into a data sequence, and performing dimension expansion on the data sequence to obtain a control data matrix in a preset format;
s4, fusing the image feature matrix and the control data matrix in the preset format to obtain a fused data matrix; inputting the fusion data matrix into a trained neural network model, and obtaining the current control deviation correction quantity output by the neural network model;
s5, the current control deviation correction amount is sent to the electric control equipment, so that the electric control equipment adjusts the speeds of a plurality of motors of the tire crane according to the current control deviation correction amount.
In one possible implementation manner, the step S3 includes:
s31, dividing the current speed difference, the historical control deviation correction amount and the motor speeds by corresponding preset maximum values respectively, splicing the obtained normalized current speed difference, the normalized historical control deviation correction amount and the normalized motor speeds to obtain a data sequence, and combining the data sequence into a first data matrix after copying a plurality of data sequences;
s32, expanding the first data matrix into the control data matrix with the preset format according to the input dimension of the neural network model, wherein the expansion mode comprises the following steps:
A. performing numerical value estimation on the first data matrix by adopting a given linear interpolation algorithm to obtain a plurality of second data matrixes which have linear relation with the first data matrix, and combining the plurality of second data matrixes with the first data matrix to obtain the control data matrix in a preset format; or alternatively, the first and second heat exchangers may be,
B. and after the first data matrix is duplicated for a plurality of times, combining to obtain the control data matrix with the preset format.
In one possible embodiment, the trained lane line detection model comprises: a backbone network and an attention module constructed by a convolutional neural network, wherein the step S1 comprises:
s11, acquiring a road image in the travelling direction of the tire crane from a camera of the tire crane;
s12, inputting the road image into the backbone network, and generating a local feature matrix through the backbone network according to the input sample road image; the local feature matrix is pooled and then input into the attention module, the weight relation among all local feature blocks in the local feature matrix is learned through the attention module, and all the local feature blocks in the local feature matrix are weighted and combined according to the learned weight relation, so that a global feature matrix is obtained; and combining the local feature matrix and the global feature matrix to obtain an image feature matrix of the road image.
In one possible implementation manner, the neural network model is a multi-layer perceptron, and the training process of the neural network model includes:
acquiring a set of sample data and tag control correction amounts of the sample data, wherein the set of sample data comprises: sample image characteristics corresponding to the sample road image, sample speed difference acquired at the same time with the sample road image and sample history control deviation correction quantity;
inputting the sample data into the multi-layer perceptron after fusion, and calculating the mean square error loss between the training control deviation correction quantity and the label control deviation correction quantity according to the training control deviation correction quantity output by the multi-layer perceptron;
and according to the mean square error loss, adopting a random gradient descent method and error back propagation to adjust model parameters of the multi-layer perceptron.
In one possible implementation, the current speed difference describes a route offset state of the tire crane, the plurality of motor speeds describe a current travel speed of the tire crane, and the historical control deviation correction amount is used to provide verified guiding information when generating the current control deviation correction amount.
In a second aspect, a driving deviation rectifying system for a tire crane is provided, the system comprising: the device comprises a camera, a computing device and an electrical control device; the camera is used for collecting road images in the running direction of the tire crane; the electrical control equipment is used for collecting the current speed difference, the historical control deviation correction quantity and the motor speeds of the tire crane, wherein the current speed difference is the motor speed difference at two sides of the tire crane; the computing device is used for executing the driving deviation rectifying method of the tyre crane provided by the first aspect.
In a third aspect, there is provided a computing device comprising a memory and a processor, the memory storing at least one program, the at least one program being executable by the processor to implement the method of driving a truck tire as provided in the first aspect.
In a fourth aspect, there is provided a computer-readable storage medium having stored therein at least one program that is executed by a processor to implement the driving deviation rectifying method of a tire crane as provided in the first aspect.
The technical scheme provided by the application at least comprises the following technical effects:
in the driving deviation correcting system of the tire crane, a computing device inputs a road image in the driving direction of the tire crane obtained from a camera into a trained lane line detection model to obtain an image feature matrix describing the position of the lane line; acquiring motor speed, current speed difference and historical control deviation correction quantity which are acquired simultaneously with road images from electrical control equipment of a tire crane; normalizing and dimension-expanding the data acquired from the electrical control equipment to obtain a control data matrix; and after the image feature matrix and the control data matrix are fused, inputting the fused image feature matrix and the control data matrix into a trained neural network model to obtain the current control deviation correction quantity, and adjusting the speeds of a plurality of motors of the tire crane according to the current control deviation correction quantity by using the electric control equipment. The image characteristics and the electrical control data are subjected to multi-mode fusion and then input into the neural network model for reasoning, so that the accuracy of deviation correction control can be effectively improved in different running environments.
Drawings
Fig. 1 is a schematic diagram of a driving deviation correcting system of a tire crane according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a tire crane running provided in an embodiment of the present application;
fig. 3 is a schematic diagram of a driving deviation rectifying method of a tire crane according to an embodiment of the present application;
fig. 4 is a schematic diagram of a lane line detection model according to an embodiment of the present disclosure;
fig. 5 is a schematic hardware structure of a computing device according to an embodiment of the present application.
Detailed Description
To further illustrate the embodiments, the present application provides the accompanying drawings. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments and together with the description, serve to explain the principles of the embodiments. With reference to these matters, one of ordinary skill in the art would understand other possible embodiments and the advantages of the present application. The components in the figures are not drawn to scale and like reference numerals are generally used to designate like components. The term "at least one" means one or more, and the term "plurality" means two or more.
The present application will now be further described with reference to the drawings and detailed description.
The application provides a driving deviation correcting system and method of a tire crane, which are used for accurately adjusting the real-time driving of the tire crane by combining a computer vision technology and multi-mode fusion deep learning thinking and generating control deviation correcting quantity in real time.
Fig. 1 is a schematic diagram of a driving deviation correcting system of a tire crane provided in an embodiment of the present application, referring to fig. 1, the driving deviation correcting system of a tire crane provided in an embodiment of the present application includes: camera, computing device and electrical control device.
The camera is used for collecting road images in the running direction of the tire crane. The camera is disposed on the body, chassis or any position of the tire crane where the lane line in the driving direction can be completely shot. Optionally, the camera can be deployed in a tire crane driving area and can completely shoot any machine position of a lane line, and the source machine position of the camera adopted in driving deviation correction calculation can be dynamically switched along with the driving route of the tire crane. Optionally, the camera may be used as a front-end station to collect road images including front lane lines, or may be used as a rear-end station to collect road images including rear lane lines.
Fig. 2 is a schematic diagram of a traveling of a tire crane according to an embodiment of the present application, and a relative positional relationship among a traveling direction, the tire crane, and a lane line is shown in fig. 2 in a plan view.
Wherein, electrical control equipment is used for controlling the traveling of tire crane. Illustratively, referring to FIG. 1, the electrical control device is a Programmable Logic Controller (PLC), or other application specific integrated circuit that can electrically control the body control unit of the tire crane.
In the embodiment of the application, the electrical control equipment is used for collecting control data of the tire crane, wherein the control data comprises a current speed difference, a historical control deviation correction amount and a plurality of motor speeds, and the current speed difference is a motor speed difference at two sides of the tire crane.
In the driving process of the tire crane, a camera collects road images in front, and at the same time, an electric control device collects motor speed (rotating speed), current speed difference and historical control deviation correction quantity output by a vehicle body control unit. The motor speeds are multiple, and the current speed difference can be obtained by calculating the difference value according to the motor speeds of the two sides of the tire crane.
The historical control deviation correction amount is the control deviation correction amount adopted by the last deviation correction control of the tire crane, and can be used as the input of a model to guide the generation of the control deviation correction amount adopted by the next deviation correction.
In the embodiment of the application, the computing equipment is used for fusing the road image acquired by the camera and the control data acquired by the electrical control equipment to form multi-mode fusion data as input of a neural network model, and the current control deviation rectifying quantity is obtained after calculation of the neural network model and used for adjusting the motor speed of the tire crane so as to realize the deviation rectifying of the driving route. Illustratively, referring to FIG. 1, the computing device is an engineering control computer (industrial personal computer).
The following describes a specific flow of the driving deviation rectifying method of the tire crane. Fig. 3 is a schematic diagram of a driving deviation rectifying method of a tire crane according to an embodiment of the present application, referring to fig. 3, the method includes steps S1 to S4, and the method is executed by a computing device in the system.
S1, the computing equipment acquires a road image in the running direction of the tire crane from the camera, and inputs the road image into a trained lane line detection model to obtain an image feature matrix of the road image.
Wherein the image feature matrix describes lane line locations in the road image.
In this embodiment of the present application, the trained lane line detection model includes: a backbone network constructed by convolutional neural network and an attention module. An embodiment of step S1 is described below in conjunction with fig. 3, and step S1 includes the following steps S11 and S12.
S11, the computing equipment acquires a road image in the running direction of the tire crane from a camera of the tire crane.
S12, the computing equipment inputs the road image into a backbone network, and generates a local feature matrix according to the input sample road image through the backbone network; after the local feature matrix is pooled, the local feature matrix is input into an attention module, the weight relation among all the local feature blocks in the local feature matrix is learned through the attention module, and all the local feature blocks in the local feature matrix are weighted and combined according to the learned weight relation, so that a global feature matrix is obtained; and combining the local feature matrix and the global feature matrix to obtain an image feature matrix of the road image. Alternatively, the lane line detection model may also employ a YOLO series model or a LaneATT model.
Fig. 4 is a schematic diagram of a lane line detection model provided in the embodiment of the present application, referring to fig. 4, a backbone network in the model is used for primarily extracting a local feature matrix from an input image, an attention module is used for weighting according to the local feature matrix to obtain a global feature matrix, and finally, the local feature matrix and the global feature matrix are combined to obtain an image feature matrix.
The image features obtained by combining the local features and the global features in the process can more accurately extract information from other lanes under the condition that the lanes are blocked or no visible lane marks, so that the accuracy of the image features is effectively improved.
S2, the computing equipment acquires the current speed difference, the historical control deviation correction amount and a plurality of motor speeds which are acquired at the same moment as the road image from the electrical control equipment corresponding to the tire crane.
The current speed difference is the motor speed difference at two sides of the tire crane and is used for indicating the motor speed error to be overcome when the correction is performed.
Among them, the speeds of the motors correspond to the running speeds of the tires on both sides of the tire crane, and the speeds of the motor 1 and the motor 2 correspond to the left side of the tire crane and the speeds of the motor 3 and the motor 4 correspond to the right side of the tire crane, among the four collected motor speeds, for example. The current speed difference can be calculated from a plurality of motor speeds.
The historical control deviation correction amount is the control deviation correction amount adopted by the last deviation correction control of the tire crane, and can be used as the input of a model to guide the generation of the control deviation correction amount adopted by the next deviation correction.
In the embodiment of the application, the current speed difference is used for providing a route deviation state of the tire crane, the plurality of motor speeds are used for describing the running speed of the tire crane, and the historical control deviation correcting quantity is used for providing verified guiding information when a new control deviation correcting quantity is generated.
In one possible implementation, the control data obtained from the electrical control device (PLC) also includes other data. Referring to table 1, in addition to the current speed difference, the historical control deviation correction amount, and the plurality of motor speeds, the data transmitted by the PLC includes: the system comprises a heartbeat state, a tire crane moving sign, a tire crane moving direction, a deviation correcting sign bit, a current speed difference direction, a historical control deviation correcting quantity direction, a current control deviation correcting quantity, an automatic deviation correcting preparation state and the like.
TABLE 1
And S3, the computing equipment normalizes the current speed difference, the historical control deviation correction amount and the motor speeds, then splices the normalized current speed difference, the historical control deviation correction amount and the motor speeds into a data sequence, and performs dimension expansion on the data sequence to obtain a control data matrix in a preset format.
In the embodiment of the present application, the multiple types of control data collected in step S2 are standardized through step S3, and are used for inputting a model. This step S3 includes the following steps S31 and S32.
S31, dividing the current speed difference, the historical control deviation correction amount and the motor speeds by corresponding preset maximum values respectively, splicing the obtained normalized current speed difference, the normalized historical control deviation correction amount and the normalized motor speeds to obtain a data sequence, and combining the data sequence into a first data matrix after copying a plurality of copies;
illustratively, 6 data of four motor speeds, current speed differences, and historical control corrections are normalized.
For example, the motor speed a, the motor speed B, the motor speed C, the motor speed D, the current speed difference E, and the history control deviation correction amount F are normalized by a preset maximum value A1, a preset maximum value B1, a preset maximum value C1, a preset maximum value D1, a preset maximum value E1, and a preset maximum value F1, respectively, to obtain a normalized data sequence: {,/>,/>,/>,/>,/>}；
The preset maximum values of the 4 motor speeds are 65535, the preset maximum value of the current speed difference is 100, and the preset maximum value of the historical control deviation correction amount is 100.
S32, expanding the first data matrix into a control data matrix with a preset format according to the input dimension of the neural network model.
Taking 512×6×6 as an example of the input dimension of the neural network model, the data sequence of 1*6 is copied and expanded into a first data matrix of 1×6×6, and then the first data matrix is expanded to obtain a control data matrix (denoted as L) of 512×6×6.
In the embodiment of the present application, the expansion modes include the following a and B.
A. And carrying out numerical value estimation on the first data matrix by adopting a given linear interpolation algorithm to obtain a plurality of second data matrixes which have linear relation with the first data matrix, and combining the plurality of second data matrixes with the first data matrix to obtain a control data matrix in a preset format.
B. And after the first data matrix is duplicated for a plurality of times, combining to obtain a control data matrix with a preset format.
S32, expanding the data sequence into a control data matrix with a preset format according to the input dimension of the neural network model.
Through the process, multiple types of control data are integrated into a standardized data sequence, and then the standardized data sequence is expanded to obtain a control data matrix matched with the input dimension of the neural network model, so that the data alignment is completed, the information quantity of the data can be improved, and the model reasoning is facilitated.
S4, the computing equipment fuses the image feature matrix and the control data matrix in a preset format to obtain a fused data matrix; and inputting the fusion data matrix into a trained neural network model to obtain the current control deviation correction quantity output by the neural network model.
In the embodiment of the application, the current speed difference is used for providing the route deviation state of the tire crane, the plurality of motor speeds are used for describing the running speed of the tire crane, and the image feature matrix can accurately describe the current road condition.
According to the method, influences of factors such as the car body counterweight, the hoisting weight and the ground flatness on the driving deviation are difficult to accurately quantify, so that multiple types of control data (current speed difference, historical control deviation correction amount and multiple motor speeds) and an image feature matrix are adopted to be fused, and the control deviation correction amount can be accurately generated by utilizing the good learning ability of the neural network model according to the current road condition, the tire crane driving speed and the route deviation state.
In the embodiment of the present application, an example is taken in which the input dimension of the neural network model is 512×6×6. The data dimension of the image feature matrix (denoted as X) is 512X 6, corresponding to three dimensions C X H X W, respectively. Wherein C corresponds to the number of channels (channels); h denotes the image height (height), i.e. the number of matrix columns; w denotes an image Width (Width). The fusion process is to add the image feature matrix X and the control data matrix L in a para-position way, and the fusion process is expressed as Y=L+X; how the data dimension of the resulting data matrix is still 512 x 6.
In one possible implementation, the neural network model is a multi-layer perceptron (Multilayer Perceptron, MLP). The input dimension of the MLP model is 512 x 6 as described above, the dimension of the output control deviation correction amount is 1 x 1. The training process of the neural network model comprises the following steps 1 to 3.
Step 1, obtaining a group of sample data and label control deviation correction amount of the sample data, wherein the group of sample data comprises: sample image characteristics corresponding to the sample road image, sample speed difference acquired at the same time with the sample road image and sample history control deviation correction amount.
And 2, inputting the fused sample data into a multi-layer perceptron, and calculating the mean square error loss between the training control deviation correction quantity and the label control deviation correction quantity according to the training control deviation correction quantity output by the multi-layer perceptron.
And 3, adjusting model parameters of the multi-layer perceptron by adopting a random gradient descent method and error back propagation according to the mean square error loss.
And S5, the computing equipment sends the current control deviation correction amount to the electric control equipment so that the electric control equipment adjusts the speeds of a plurality of motors of the tire crane according to the current control deviation correction amount.
In the embodiment of the present application, the current control deviation correction amount may be an adjustment amount for a motor speed at any side of the tire crane. The current control deviation correction amount is input into an electrical control device (PLC), and the PLC modifies one or more output motor speeds so that the tire crane runs according to the corrected motor speeds to gradually correct the deviation relative to the lane line.
In one possible implementation, the data sent by the computing device to the electrical control device is as shown in table 2.
TABLE 2
Through the fields, the current running state, the system control state and the camera state of the tire crane of the vehicle can be further integrated, and more accurate control on the tire crane is realized.
According to the technical scheme, influences of factors such as a car body counterweight, hoisting weight and ground flatness on driving deviation are difficult to accurately quantify, various control data (current speed difference, historical control deviation correction quantity and multiple motor speeds) and an image feature matrix are adopted to be fused, the image features and the electrical control data are input into a neural network model for reasoning after being subjected to multi-mode fusion, and the control deviation correction quantity can be accurately generated by utilizing good learning ability of the neural network model according to current road conditions, tire crane driving speed and route deviation states.
And the control data of multiple types are integrated into a standardized data sequence and then expanded to obtain a control data matrix matched with the input dimension of the neural network model, so that the data alignment is completed, the information quantity of the data can be improved, and the model reasoning is facilitated.
The application also provides a computing device which can execute the driving deviation rectifying method of the tire crane. Fig. 5 is a schematic hardware structure of a computing device provided in an embodiment of the present application, where, as shown in fig. 5, the computing device includes a processor 501, a memory 502, a bus 503, and a computer program stored in the memory 502 and capable of running on the processor 501, where the processor 501 includes one or more processing cores, the memory 502 is connected to the processor 501 through the bus 503, and the memory 502 is used to store program instructions, and when the processor executes the computer program, the processor implements all or part of the steps in the foregoing method embodiments provided in the present application.
Further, as an executable scheme, the computing device may be a computer unit, and the computer unit may be a computing device such as a desktop computer, a notebook computer, a palm computer, and a cloud server. The computer unit may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that the constituent structures of the computer unit described above are merely examples of the computer unit and are not limiting, and may include more or fewer components than those described above, or may combine certain components, or different components. For example, the computer unit may further include an input/output device, a network access device, a bus, etc., which is not limited in this embodiment of the present application.
Further, as an implementation, the processor may be a central processing unit (Central Processing Unit, CPU), other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like that is a control center of the computer unit, connecting various parts of the entire computer unit using various interfaces and lines.
The memory may be used to store the computer program and/or modules, and the processor may implement the various functions of the computer unit by running or executing the computer program and/or modules stored in the memory, and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for a function; the storage data area may store data created according to the use of the cellular phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
The present application also provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the methods described above in the embodiments of the present application.
The modules/units integrated with the computer unit may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each method embodiment described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a software distribution medium, and so forth. It should be noted that the content of the computer readable medium can be appropriately increased or decreased according to the requirements of the legislation and the patent practice in the jurisdiction.
While this application has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application as defined by the appended claims.
Claims (8)
1. A method for correcting a traveling deviation of a tire crane, the method comprising:
s1, acquiring a road image in the running direction of a tire crane from a camera, and inputting the road image into a trained lane line detection model to obtain an image feature matrix of the road image, wherein the image feature matrix describes the positions of lane lines in the road image;
s2, acquiring a current speed difference, a historical control deviation correction amount and a plurality of motor speeds, which are acquired at the same moment as the road image, from electrical control equipment corresponding to the tire crane, wherein the current speed difference is a motor speed difference at two sides of the tire crane;
s3, normalizing the current speed difference, the historical control deviation correction amount and the motor speeds, then splicing the normalized current speed difference, the historical control deviation correction amount and the motor speeds into a data sequence, and performing dimension expansion on the data sequence to obtain a control data matrix in a preset format;
s4, fusing the image feature matrix and the control data matrix in the preset format to obtain a fused data matrix; inputting the fusion data matrix into a trained neural network model, and obtaining the current control deviation correction quantity output by the neural network model;
s5, the current control deviation correction amount is sent to the electric control equipment, so that the electric control equipment adjusts the speeds of a plurality of motors of the tire crane according to the current control deviation correction amount.
2. The method for correcting the traveling deviation of a tire crane according to claim 1, wherein the step S3 comprises:
s31, dividing the current speed difference, the historical control deviation correction amount and the motor speeds by corresponding preset maximum values respectively, splicing the obtained normalized current speed difference, the normalized historical control deviation correction amount and the normalized motor speeds to obtain a data sequence, and combining the data sequence into a first data matrix after copying a plurality of data sequences;
s32, expanding the first data matrix into the control data matrix with the preset format according to the input dimension of the neural network model, wherein the expansion mode comprises the following steps:
A. performing numerical value estimation on the first data matrix by adopting a given linear interpolation algorithm to obtain a plurality of second data matrixes which have linear relation with the first data matrix, and combining the plurality of second data matrixes with the first data matrix to obtain the control data matrix in a preset format; or alternatively, the first and second heat exchangers may be,
B. and after the first data matrix is duplicated for a plurality of times, combining to obtain the control data matrix with the preset format.
3. The method for correcting a traveling deviation of a tire crane according to claim 1, wherein the trained lane line detection model comprises: a backbone network and an attention module constructed by a convolutional neural network, wherein the step S1 comprises:
s11, acquiring a road image in the travelling direction of the tire crane from a camera of the tire crane;
s12, inputting the road image into the backbone network, and generating a local feature matrix through the backbone network according to the input sample road image; the local feature matrix is pooled and then input into the attention module, the weight relation among all local feature blocks in the local feature matrix is learned through the attention module, and all the local feature blocks in the local feature matrix are weighted and combined according to the learned weight relation, so that a global feature matrix is obtained; and combining the local feature matrix and the global feature matrix to obtain an image feature matrix of the road image.
4. The method for correcting the driving deviation of the tire crane according to claim 1, wherein the neural network model is a multi-layer perceptron, and the training process of the neural network model comprises:
acquiring a set of sample data and tag control correction amounts of the sample data, wherein the set of sample data comprises: sample image characteristics corresponding to the sample road image, sample speed difference acquired at the same time with the sample road image and sample history control deviation correction quantity;
inputting the sample data into the multi-layer perceptron after fusion, and calculating the mean square error loss between the training control deviation correction quantity and the label control deviation correction quantity according to the training control deviation correction quantity output by the multi-layer perceptron;
and according to the mean square error loss, adopting a random gradient descent method and error back propagation to adjust model parameters of the multi-layer perceptron.
5. The method of claim 1, wherein the current speed difference describes a route offset state of the tire crane, the plurality of motor speeds describe a current travel speed of the tire crane, and the historical control deviation correction amount is used to provide verified guidance information when generating the current control deviation correction amount.
6. A ride control system for a tire crane, the system comprising: the device comprises a camera, a computing device and an electrical control device; the camera is used for collecting road images in the running direction of the tire crane; the electrical control equipment is used for collecting the current speed difference, the historical control deviation correction quantity and the motor speeds of the tire crane, wherein the current speed difference is the motor speed difference at two sides of the tire crane; the computing device is configured to perform the driving deviation correcting method of the tire crane according to any one of claims 1 to 5.
7. A computing device comprising a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the method of rectifying the ride of a tyre crane according to any one of claims 1 to 5.
8. A computer-readable storage medium, wherein at least one program is stored in the storage medium, and the at least one program is executed by a processor to implement the driving deviation correcting method of the tire crane according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311753037.7A CN117437609B (en) | 2023-12-20 | 2023-12-20 | Method, system, equipment and storage medium for correcting driving deviation of tire crane |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311753037.7A CN117437609B (en) | 2023-12-20 | 2023-12-20 | Method, system, equipment and storage medium for correcting driving deviation of tire crane |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117437609A true CN117437609A (en) | 2024-01-23 |
| CN117437609B CN117437609B (en) | 2024-03-15 |
Family
ID=89551952
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311753037.7A Active CN117437609B (en) | 2023-12-20 | 2023-12-20 | Method, system, equipment and storage medium for correcting driving deviation of tire crane |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117437609B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1570796A (en) * | 2004-05-14 | 2005-01-26 | 李俊 | Rubber-tired crane autopilot and box location management system based on differential GPS technology |
| CN105776027A (en) * | 2016-05-12 | 2016-07-20 | 中南大学 | Deviation rectification control method and system for car of bridge crane in walking process |
| JP2020119407A (en) * | 2019-01-25 | 2020-08-06 | 日立造船株式会社 | Estimating model generation device, estimating model generation method by estimating model generation device, and estimating device |
| CN111776942A (en) * | 2020-06-17 | 2020-10-16 | 深圳元戎启行科技有限公司 | Tire crane running control system, method and device and computer equipment |
| CN115258957A (en) * | 2022-07-29 | 2022-11-01 | 湖南三一塔式起重机械有限公司 | Rotation control method, tower crane, and storage medium |
| CN115830559A (en) * | 2022-12-06 | 2023-03-21 | 博大视野(厦门)科技有限公司 | Method for identifying lane line of tire crane, storage medium and electronic equipment |
-
2023
- 2023-12-20 CN CN202311753037.7A patent/CN117437609B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1570796A (en) * | 2004-05-14 | 2005-01-26 | 李俊 | Rubber-tired crane autopilot and box location management system based on differential GPS technology |
| CN105776027A (en) * | 2016-05-12 | 2016-07-20 | 中南大学 | Deviation rectification control method and system for car of bridge crane in walking process |
| JP2020119407A (en) * | 2019-01-25 | 2020-08-06 | 日立造船株式会社 | Estimating model generation device, estimating model generation method by estimating model generation device, and estimating device |
| CN111776942A (en) * | 2020-06-17 | 2020-10-16 | 深圳元戎启行科技有限公司 | Tire crane running control system, method and device and computer equipment |
| CN115258957A (en) * | 2022-07-29 | 2022-11-01 | 湖南三一塔式起重机械有限公司 | Rotation control method, tower crane, and storage medium |
| CN115830559A (en) * | 2022-12-06 | 2023-03-21 | 博大视野(厦门)科技有限公司 | Method for identifying lane line of tire crane, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117437609B (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3699048A1 (en) | Travelling track prediction method and device for vehicle | |
| EP3699053A1 (en) | Method and apparatus for planning speed of autonomous vehicle, and storage medium | |
| CN108519094B (en) | Local path planning method and cloud processing terminal | |
| EP3822852A2 (en) | Method, apparatus, computer storage medium and program for training a trajectory planning model | |
| CN106558080A (en) | Join on-line proving system and method outside a kind of monocular camera | |
| CN107860391A (en) | Automobile accurate navigation method and device | |
| CN112307978B (en) | Target detection method and device, electronic equipment and readable storage medium | |
| CN110174110B (en) | Map corresponding method and device, electronic equipment and computer readable medium | |
| CN107091648A (en) | A kind of data fusion method and system of laser radar and differential GPS | |
| CN109387210A (en) | Automobile navigation method and its device | |
| CN113011364B (en) | Neural network training, target object detection and driving control method and device | |
| CN114495554A (en) | Intersection guide information construction method and system for intelligent driving | |
| CN111681172A (en) | Method, equipment and system for cooperatively constructing point cloud map | |
| CN117437609B (en) | Method, system, equipment and storage medium for correcting driving deviation of tire crane | |
| CN109829401A (en) | Traffic sign recognition method and device based on double capture apparatus | |
| CN110083158B (en) | Method and equipment for determining local planning path | |
| CN116811898A (en) | Method and device for generating curve automatic driving road network based on lane lines | |
| CN116543271A (en) | Method, device, electronic equipment and medium for determining target detection evaluation index | |
| CN115230715A (en) | Lane change prediction method and device, nonvolatile storage medium and computer equipment | |
| CN113888601B (en) | Target trajectory prediction method, electronic device, and storage medium | |
| CN116070903A (en) | Risk determination method and device for passing through obstacle region and electronic equipment | |
| Chen et al. | A new adaptive region of interest extraction method for two-lane detection | |
| CN114694121A (en) | Lane line correction method, lane line correction device, electronic apparatus, storage medium, and program product | |
| CN115140029A (en) | Safety capability detection method and device for automatic driving automobile | |
| EP3690706A1 (en) | Method and device for detecting lane elements to plan the drive path of autonomous vehicle by using a horizontal filter mask, wherein the lane elements are unit regions including pixels of lanes in an input image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |