CN115497058B - Non-contact vehicle weighing method based on multispectral imaging technology - Google Patents

Non-contact vehicle weighing method based on multispectral imaging technology Download PDF

Info

Publication number
CN115497058B
CN115497058B CN202211076683.XA CN202211076683A CN115497058B CN 115497058 B CN115497058 B CN 115497058B CN 202211076683 A CN202211076683 A CN 202211076683A CN 115497058 B CN115497058 B CN 115497058B
Authority
CN
China
Prior art keywords
tire
pixel
image
temperature
ground
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211076683.XA
Other languages
Chinese (zh)
Other versions
CN115497058A (en
Inventor
吴刚
张皓炜
高康
李胡兵
冯锦鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN202211076683.XA priority Critical patent/CN115497058B/en
Publication of CN115497058A publication Critical patent/CN115497058A/en
Application granted granted Critical
Publication of CN115497058B publication Critical patent/CN115497058B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01G—WEIGHING
    • G01G19/00—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups
    • G01G19/02—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups for weighing wheeled or rolling bodies, e.g. vehicles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/10—Geometric CAD
    • G06F30/17—Mechanical parametric or variational design
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/24—Character recognition characterised by the processing or recognition method
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
    • G06F2119/14—Force analysis or force optimisation, e.g. static or dynamic forces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a non-contact vehicle weighing method based on a multispectral imaging technology, which comprises the following steps: processing pixel temperature matrix data based on an OpenCV image processing algorithm, and detecting by using an image iterative geometric fitting and region growing algorithm to obtain mechanical deformation parameters of the target sample tire; detecting to obtain the stamping characters of the side wall of the target sample tire, and obtaining the tire size and the air pressure information of the target sample tire; and inputting the mechanical deformation parameters and character information of the tire to be detected into a machine learning model, and calculating the load of the tire to be detected. The invention can realize rapid weighing of the vehicle in all-weather multi-environment multi-application scenes, and is used for solving the problems of weak generalization capability, limited applicable environment and the like of the existing vehicle weighing method based on calculation vision, and the technical problems of short service life and high cost of the traditional weighing system.

Description

Non-contact vehicle weighing method based on multispectral imaging technology
Technical Field
The invention belongs to the technical field of image recognition weighing, and particularly relates to a non-contact vehicle weighing method based on a multispectral imaging technology.
Background
The rapid development of the transportation industry has put higher demands on intelligent operation and maintenance of transportation. Currently, infrastructures such as roads, bridges and the like are in an overload and overrun working state for a long time, so that irreversible damage is caused to the infrastructures of the roads and the bridges, and the life safety and the property safety of people are greatly threatened.
The development of computer vision is an effective means for solving the traffic intelligence operation and maintenance. The intelligent transportation operation and maintenance is a novel transportation operation and maintenance concept integrating multiple high and new technologies into a whole through multi-disciplinary fusion and intersection. The intelligent traffic operation and maintenance means are the development direction of future traffic management systems, and are beneficial to long-term service of traffic infrastructures. The non-contact vehicle load identification mode based on computer vision is a novel technology for starting a vehicle, and is also an important part of intelligent traffic.
Advanced non-contact measurement technology based on computer vision has been developed to a certain extent, and is a low-cost, efficient and easy-to-operate vehicle load detection method, which continuously attracts attention from researchers. The traditional vehicle weighing means is mainly a contact type weighing method realized by adopting a vehicle dynamic weighing device and an internal sensor. The weighing equipment adopted by the method is in an overload and overrun working state for a long time, so that the internal sensor is easy to damage, the service life of the equipment is short, and the maintenance cost is high.
At present, a non-contact vehicle weighing method based on computer vision also has certain defects, the vehicle weighing method based on an optical image is weak in generalization capability, the image is easily influenced by a light source and a use environment, and the image processing means cannot be widely applied to various applications. Moreover, the load measurement algorithm corresponding to the non-contact vehicle weighing method based on computer vision is weak in generalization capability, and the load estimation formula based on fitting can only be suitable for vehicle load estimation in a fixed application scene.
Therefore, how to further improve the non-contact measurement accuracy and the application scene based on computer vision through the multi-band invisible light source, overcome the defect that the traditional optical camera cannot measure at night, and overcome the weak generalization capability of the existing load estimation algorithm through the artificial intelligence technology becomes a problem which needs to be solved by the technicians in the field.
Disclosure of Invention
The technical problems to be solved are as follows: the invention discloses a non-contact vehicle weighing method based on a multispectral imaging technology, which can realize rapid vehicle weighing under all-weather multiple environments and multiple application scenes and is used for solving the problems of weak generalization capability, limited applicable environment and the like of the existing vehicle weighing method based on calculation vision, and the technical problems of short service life and high cost of the traditional weighing system.
The technical scheme is as follows:
a non-contact vehicle weighing method based on multispectral imaging technology, the non-contact vehicle weighing method comprising the steps of:
s1, acquiring a side thermal imaging image of a target sample tire in a normal working state; extracting pixel temperature matrix data from the side thermal imaging image to obtain surface temperature information of a target sample tire; processing pixel temperature matrix data based on an OpenCV image processing algorithm, and detecting by using an image iterative geometric fitting and region growing algorithm to obtain mechanical deformation parameters of the target sample tire;
s2, acquiring an optical image of the side wall of the target sample tire; detecting and obtaining stamping characters of the side wall of the target sample tire from the optical image of the side wall of the target sample tire; obtaining tire size and air pressure information of a target sample tire through imprinting characters;
s3, taking the obtained mechanical deformation parameters of the target sample tire and corresponding tire size and air pressure information as training samples of a machine learning model, and training to obtain the machine learning model for predicting the tire load;
s4, inputting the mechanical deformation parameters and the character information of the tire to be detected into a machine learning model, and calculating the load of the tire to be detected.
Further, in step S1, the process of acquiring a side thermal imaging image of the target sample tire in a normal working state includes the following steps:
centering the thermal imaging acquisition assembly to the center of the tire hub by utilizing infrared laser calibration; capturing side thermal imaging information of the tire in a normal working state by adopting a thermal imaging acquisition component; the side thermal imaging information captured by the thermal imaging acquisition assembly is saved in a CSV format.
Further, in step S1, the mechanical deformation parameters include: tire maximum pixel radius R, tire maximum pixel area S 1 The hub pixel radius r, the contact pixel length 1 of the tire and the ground, the pixel distance h from the center of the tire to the ground, and the equivalent pixel area S after the tire is deformed 2 The image pixel area difference deltas before and after the tire deformation, and the tire-to-ground contact parting line pixel length L.
Further, in step S1, the process of processing the pixel temperature matrix data based on the OpenCV image processing algorithm and detecting the mechanical deformation parameters of the target sample tire through the geometric fitting and the region growing algorithm of the image iteration includes the following steps:
s11, generating a temperature image linearly related to the temperature according to the pixel temperature matrix data; calculating pixel gradient amplitude values of a temperature image by adopting a sobel edge detection operator, and performing image segmentation around pixel points with temperature differences larger than a preset temperature difference threshold value in the temperature image;
s12, reserving the gradient amplitude of the first 5% of the segmented image, and taking out the point with the maximum gradient amplitude for color marking; sequentially selecting pixel points marked by colors from the lower direction of the image, taking the selected pixel points as seed points for fitting the outer contour of the tire for the first time, wherein the selected seed points are all pixel points on the interface between the tire and the air;
s13, carrying out region growth on the seed points by using the seed points selected in the step S12 by using a region growing algorithm to find points adjacent to the seed points as new seed points, and carrying out second fitting of the outer contour of the tire;
s14, reserving the seed points selected in the step S12 and the step S13, taking the tire outline fitted for the second time as a reference, finding the gradient amplitude maximum pixel point of the upper half part of the tire from top to bottom, and carrying out third tire outline fitting;
s15, searching a pixel point with the largest gradient amplitude from the center of the tire by taking the fitted tire outer contour in the step S14 as a reference, and fitting to obtain the hub outer contour by taking the pixel point as a seed point for fitting the hub outer contour;
s16, repeating iteration to fit to obtain the tire outer contour and the hub outer contour which meet the preset error standard;
s17, finding a pixel gradient amplitude point at the junction of the tire and the ground in an included angle range of 45 degrees below left and 45 degrees below right of the center of the tire, and drawing a tire and ground parting line by taking the found pixel gradient amplitude point as a reference to obtain an interface between the tire and the ground after the tire is deformed;
s18, carrying out endpoint processing on image pixels of the interface between the tire and the ground from bottom to top, calculating Y coordinate difference values of two adjacent pixel points, and judging that any pixel point is an endpoint of the contact between the tire and the ground when the Y coordinate difference value of the pixel point adjacent to the pixel point is larger than a preset coordinate difference value threshold value, so as to obtain the real contact pixel length of the tire and the ground.
Further, the obtained mechanical deformation parameters of the tire are corrected by adopting a proportion factor alpha:
wherein Rim is the radius of the hub; r is the radius of the hub pixel obtained by fitting;
the corrected mechanical deformation parameters are as follows:
r true ＝α×r
R true ＝α×R
S 1true ＝α 2 ×S 1
l true ＝α×k
h true ＝α×h
S 2true ＝α 2 ×S 2
ΔS true ＝α 2 ×ΔS
L true ＝α×L
wherein r is true 、R true 、s 1true 、l true 、h true 、s 2true 、ΔS true And L true The method comprises the steps of respectively correcting the hub pixel radius, the maximum pixel radius of the tire, the maximum pixel area of the tire, the contact pixel length of the tire and the ground, the pixel distance from the center of the tire to the ground, the equivalent pixel area after the tire is deformed, the image pixel area difference before and after the tire is deformed and the contact parting line pixel length of the tire and the ground; r, S 1 、l、h、S 2 And delta S and L are respectively the maximum pixel radius of the tire, the maximum pixel area of the tire, the contact pixel length of the tire and the ground, the pixel distance from the center of the tire to the ground, the equivalent pixel area after the tire is deformed, the image pixel area difference before and after the tire is deformed and the contact parting line pixel length of the tire and the ground, which are obtained by fitting.
Further, in step S2, based on the deep learning end-to-end OCR character recognition algorithm, performing OCR character recognition on the optical image of the sidewall of the target sample tire, and acquiring the size information and the air pressure information of the tire according to the recognition result; the size information comprises tire section height H, tire section width b, hub radius Rim and air pressure information atm; the tire true air pressure is 1.1 to 1.2 times the maximum air pressure indicated by the tire identifier.
Further, in step S3, each training sample includes 12 mechanical features and 1 tag; the mechanical characteristics are as follows: maximum radius R of tire true Maximum area S of tyre 1true Radius r of hub true Length of contact of tire with ground l true Distance h from center of tire to ground true Equivalent area S after deformation of tire 2true Area difference DeltaS before and after tire deformation true Length L of tire-to-ground contact parting line true Tire section height H, tire section width b, and tire air pressure atm: the label is as follows: tire-ground contact force F;
the machine learning model is an integrated decision tree model reflecting the mapping relation between 12 mechanical features and 1 mechanical response.
Further, the non-contact vehicle weighing method further comprises the following steps:
s5, carrying out temperature correction on the real load of the tire predicted by the machine learning model through a regression coefficient beta:
in the method, in the process of the invention,for the tire real load after temperature correction, F is the tire real load predicted by the machine learning model, < ->Is the average temperature of the tire surface; the value range of the regression coefficient beta is 0.9 to 1.5, when the tire surface temperature reaches 90 percent of the cold tire pressure temperature corresponding to the standard tire pressure, the value is 0.9, the middle is taken by linear interpolation, when the tire surface temperature reaches 150 percent of the cold tire pressure temperature corresponding to the standard tire pressure, the value is 1.5, and the middle is taken by linear interpolation;
s6, predicting the whole vehicle load W according to the number of the axles:
further, after extracting the contour image after the tire is segmented, mapping the pixel points of the contour image back into the pixel temperature matrix data, and calculating the average temperature of the tire surface by using the temperature data in the tire contour
Where n represents the number of temperature pixels captured by thermal imaging of the tire surface, T i The temperature magnitude of the ith temperature data for the tire surface.
The invention also discloses an automatic load prediction system, which comprises a thermal imaging acquisition component, a data processing device, an optical image acquisition component, an OCR recognition device, a machine learning prediction model and a model updating device;
the data processing device comprises a storage unit, a thermal imaging data processing program and an OCR character recognition program;
the storage unit is used for storing the thermal imaging image of the tire to be detected shot by the thermal imaging acquisition assembly and the optical image of the side wall of the tire to be detected shot by the optical image acquisition assembly; the OCR character recognition program is used for recognizing the imprinting characters contained in the optical image of the tire side wall; the data processing device acquires the tire size and the air pressure information of the tire to be detected through the stamping characters and transmits the tire size and the air pressure information to the machine learning prediction model; the thermal imaging data processing program processes the thermal imaging image of the tire to be detected according to the method, calculates mechanical deformation parameters of the tire to be detected, and transmits the mechanical deformation parameters to the machine learning prediction model;
the machine learning prediction model processes the imported mechanical deformation parameters, the tire size and the air pressure information of the tire to be detected, and calculates to obtain the load of the tire to be detected;
the model updating device is used for importing new sample data into the machine learning model and updating the machine learning model.
The beneficial effects are that:
firstly, compared with the existing optical camera detection method, the non-contact vehicle weighing method based on the multispectral imaging technology can detect the deformation information of the tire with extremely high precision according to different radiance and surface temperature of objects, is not limited by light sources and application scenes, and overcomes the defect that an optical camera can only work in specific light source environments such as daytime and cannot work at night.
Secondly, the non-contact vehicle weighing method based on the multispectral imaging technology provided by the invention considers 12 mechanical characteristics based on a machine learning load prediction algorithm, comprehensively considers variables related to tire load, can continuously improve the prediction precision of a model through real-time updating of the model, improves the generalization performance of the model, and is suitable for various tires by means of expanding a data set.
Thirdly, according to the non-contact vehicle weighing method based on the multispectral imaging technology, only the camera is required to be installed outside the lane to shoot the outline of the vehicle, the equipment is convenient to erect, the required number of the cameras is small, the cameras can be replaced according to the precision requirement of the invention, and the requirement benefit maximization is reasonably realized. The newly-erected camera can quickly finish the vehicle weight by only carrying out simple distortion correction, and has strong movable capability and reproduction capability.
Drawings
FIG. 1 is a flow chart of a non-contact vehicle weighing method based on multispectral imaging technology according to an embodiment of the invention;
FIG. 2 is a schematic illustration of an outdoor field deployment of an automated load prediction system of the present invention;
FIG. 3 is a graphical illustration of the tire outer profile captured by the thermal imaging acquisition assembly;
FIG. 4 is a schematic illustration of a tire deformation profile detected from a thermogram;
FIG. 5 is a schematic illustration of the average temperature of the tire exterior surface captured by the thermal imaging acquisition assembly;
FIG. 6 is a schematic illustration of thermal imaging during night operation;
FIG. 7 is an effect diagram of tire character recognition;
fig. 8 is a prediction effect diagram of the automated load prediction system of the present invention.
Detailed Description
The following examples will provide those skilled in the art with a more complete understanding of the invention, but are not intended to limit the invention in any way.
Referring to fig. 1, the embodiment discloses a non-contact vehicle weighing method based on a multispectral imaging technology, which comprises the following steps:
s1, acquiring a side thermal imaging image of a target sample tire in a normal working state; extracting pixel temperature matrix data from the side thermal imaging image to obtain surface temperature information of a target sample tire; and processing pixel temperature matrix data based on an OpenCV image processing algorithm, and detecting by using an image iterative geometric fitting and region growing algorithm to obtain the mechanical deformation parameters of the target sample tire.
S2, acquiring an optical image of the side wall of the target sample tire; detecting and obtaining stamping characters of the side wall of the target sample tire from the optical image of the side wall of the target sample tire; tire size and air pressure information of the target sample tire are obtained by imprinting the characters.
And S3, taking the obtained mechanical deformation parameters of the target sample tire and corresponding tire size and air pressure information as training samples of a machine learning model, and training to obtain the machine learning model for predicting the tire load.
S4, inputting the mechanical deformation parameters and the character information of the tire to be detected into a machine learning model, and calculating the load of the tire to be detected.
The embodiment also discloses an automatic load prediction system based on infrared thermal imaging and machine vision, which comprises a thermal imaging acquisition component, a data processing device, an optical image acquisition component, an OCR recognition device, a machine learning prediction model and a model updating device. The thermal imaging acquisition component is used for acquiring thermal imaging images of all tires of the target vehicle to obtain CSV temperature data; the optical image acquisition assembly is used for acquiring the side wall optical image of the tire. The data processing device includes a storage unit, a thermal imaging data processing program, and an OCR character recognition program. The storage unit is used for storing tire deformation parameters and OCR character recognition information obtained through detection of the thermal imaging image; the machine learning prediction model is used for receiving 12 mechanical characteristics sent by the data processing device and predicting and correcting the tire load; the model updating component is used for retraining a machine learning prediction model after the load is predicted by the system every time, replacing an original model, and taking new measured data into a database to enhance the generalization capability of the model.
In this embodiment, the thermal imaging acquisition component may be a high-resolution high-frame-rate thermal imaging device with a resolution greater than or equal to 640×480, and in this embodiment, a K26HE25 high-speed high-frame-rate infrared thermal imaging acquisition component is used. The optical image acquisition component adopts a Nikon D5600 single-lens reflex camera (hereinafter referred to as a camera), and the rest components adopt notebook computers, as shown in fig. 2, and is an outdoor deployment schematic diagram of an automatic load prediction system.
In this embodiment, the thermal imaging capturing component captures the tire of the target vehicle through thermal imaging, so as to obtain an original CSV temperature data file, extract the temperature information corresponding to each pixel point of the CSV temperature data file, and redraw the temperature image according to the linear relationship, as shown in fig. 3. And then, carrying out edge detection on the temperature image by adopting a sobel operator, calculating the temperature gradient amplitude in the image, finding out the point with the maximum gradient amplitude, reserving the pixel point with the front 5% of the gradient amplitude, and carrying out color marking.
According to priori knowledge, the maximum gradient amplitude point at the contact surface of the lower half part of the tire and the air can be closest to the edge contour of the tire, and the edge detection points at the interface of the tire and the air are sequentially selected from the lower direction of the image to be seed points for fitting the outer contour circle of the tire. And fitting the outer contour of the tire by using the first selected seed points, performing region growing operation on the first selected seed points, selecting the second seed points, and continuing fitting the outer contour of the tire. And selecting seed points for the third time from the upper half part of the tire from top to bottom, and further fitting the outer contour of the tire. And sequentially searching temperature gradient amplitude points towards the circle center by taking the outer contour of the tire as a boundary, and carrying out hub outer contour fitting by taking the temperature gradient amplitude points as edge pixel points of the hub. Finding a pixel gradient amplitude point at the boundary between the tire and the ground within 45 degrees of the left lower part and the right lower part of the center of the tire, taking the point as a reference to serve as a tire and ground dividing line to obtain the boundary between the tire and the ground after the tire is deformed, carrying out endpoint processing on image pixels of the boundary between the tire and the ground from bottom to top, calculating a Y coordinate difference value of two adjacent pixel points, judging the point as an endpoint of the contact between the tire and the ground when the difference value is larger than a certain threshold value to obtain the real contact pixel length of the tire and the ground, and completing the outer contour edge segmentation of the tire and the hub based on thermal imaging data to obtainThe following parameters, tire maximum pixel radius R, tire maximum pixel area S 1 Hub pixel radius r, tire and ground contact pixel length l, tire center-to-ground pixel distance h, tire deformed equivalent pixel area S 2 The image pixel area difference deltas before and after the tire deformation, the tire-to-ground contact parting line pixel length L, is shown in fig. 4.
The average temperature of the outer surface of the tire is calculated based on the thermal information captured by the thermal imaging,as shown in fig. 5.
The optical acquisition component is utilized to acquire an optical image of the tire, and then an OCR recognition algorithm is called, wherein the OCR recognition algorithm adopts a character recognition technology based on deep learning, a PSENET character positioning network and a CRNN character recognition network are trained through a transfer learning method, a two-stage character recognition method is formed, the character recognition method is characterized in that identifier information of the side wall of the tire is used, and tire size information (tire section height H, tire section width b and hub radius Rim) and air pressure information atm are obtained, as shown in fig. 7.
According to the size information of the hub, performing size correction on various deformation parameters measured in thermal imaging to obtain the maximum radius R of the tire true Maximum area S of tire 1true Hub radius r true Length of contact of tire with ground l true Distance h from center of tire to ground true Equivalent area S after tire deformation 2true Area difference deltas before and after tire deformation true Length L of tire-ground contact parting line true 。
According to a data set collected by an indoor test (images of tires with different sizes, different tire pressures, different loads and different temperatures are collected), tire section heights, section widths and tire pressures obtained by tire manufacturers and models are recognized through OCR technology, and a certain amount of sample data is formed), the XGboost model is trained to obtain a load prediction model.
The 12 mechanical characteristics measured by the thermal imaging module and the character recognition module are input into the XGboost load prediction model to obtain the predicted load of a single tire, and the error of the method is less than 5% through outdoor verification, so that the prediction effect is good, as shown in figure 8.
According to the characteristic that the tire pressure receives temperature change, temperature correction is carried out on the tire load to obtain the corrected load of the single tireAnd then further calculating the whole vehicle load according to the number of the vehicle axles to obtain the whole vehicle load
If the embodiment has the condition of measuring the real load of the tire, the tire load measured by the third party equipment is combined with 12 mechanical characteristics provided by the invention to construct a data set for updating the model, thereby realizing an automatic updating and self-learning mechanism of the model and improving the universal capability of the load prediction model.
The non-contact vehicle weighing method based on the multispectral imaging technology can accurately predict the tire-ground contact force of the vehicle under the non-contact condition. The method is innovative in that a thermal imaging device captures the tire deformation outline represented by thermodynamic information contained by multispectral under a normal working state, and an image segmentation algorithm is designed for multispectral imaging captured by a thermal imaging acquisition component device, so that the tire deformation information in normal working is obtained; simultaneously detecting identifiers of the tire side walls by an optical camera to obtain basic size information of the tire; and inputting the tire deformation information and the tire identifier information into an XGboost machine learning prediction model to further realize the prediction of the tire load. Compared with the prior art, the measuring module of the embodiment can more accurately measure the deformation information and thermodynamic information of the tire than an optical camera; compared with the existing calculation model, the load prediction module of the embodiment is more accurate, the error of weighing the whole vehicle is less than 5%, various mechanical characteristics are more comprehensively considered, and the model generalization capability can be enhanced continuously through the model, so that the application range of the model is expanded, and the model has stronger robustness. In addition, the embodiment can realize all-weather real-time weighing of the vehicle at night or in daytime, and has wider applicability. Fig. 6 is a schematic illustration of thermal imaging during night operation.
The foregoing is only a preferred embodiment of the invention, it being noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the present invention, and such modifications and adaptations are intended to be comprehended within the scope of the invention.

Claims (9)

1. A non-contact vehicle weighing method based on a multispectral imaging technology, which is characterized by comprising the following steps:
s1, acquiring a side thermal imaging image of a target sample tire in a normal working state; extracting pixel temperature matrix data from the side thermal imaging image to obtain surface temperature information of a target sample tire; processing pixel temperature matrix data based on an OpenCV image processing algorithm, and detecting by using an image iterative geometric fitting and region growing algorithm to obtain mechanical deformation parameters of the target sample tire;
s2, acquiring an optical image of the side wall of the target sample tire; detecting and obtaining stamping characters of the side wall of the target sample tire from the optical image of the side wall of the target sample tire; obtaining tire size and air pressure information of a target sample tire through imprinting characters;
s3, taking the obtained mechanical deformation parameters of the target sample tire and corresponding tire size and air pressure information as training samples of a machine learning model, and training to obtain the machine learning model for predicting the tire load;
s4, inputting mechanical deformation parameters and character information of the tire to be detected into a machine learning model, and calculating the load of the tire to be detected;
in step S1, the process of processing the pixel temperature matrix data based on the OpenCV image processing algorithm and detecting the mechanical deformation parameters of the target sample tire through the geometric fitting and the region growing algorithm of the image iteration includes the following steps:
s11, generating a temperature image linearly related to the temperature according to the pixel temperature matrix data; calculating pixel gradient amplitude values of a temperature image by adopting a sobel edge detection operator, and performing image segmentation around pixel points with temperature differences larger than a preset temperature difference threshold value in the temperature image;
s12, reserving the gradient amplitude of the first 5% of the segmented image, and taking out the point with the maximum gradient amplitude for color marking; sequentially selecting pixel points marked by colors from the lower direction of the image, taking the selected pixel points as seed points for fitting the outer contour of the tire for the first time, wherein the selected seed points are all pixel points on the interface between the tire and the air;
s13, carrying out region growth on the seed points by using the seed points selected in the step S12 by using a region growing algorithm to find points adjacent to the seed points as new seed points, and carrying out second fitting of the outer contour of the tire;
s14, reserving the seed points selected in the step S12 and the step S13, taking the tire outline fitted for the second time as a reference, finding the gradient amplitude maximum pixel point of the upper half part of the tire from top to bottom, and carrying out third tire outline fitting;
s15, searching a pixel point with the largest gradient amplitude from the center of the tire by taking the fitted tire outer contour in the step S14 as a reference, and fitting to obtain the hub outer contour by taking the pixel point as a seed point for fitting the hub outer contour;
s16, repeating iteration to fit to obtain the tire outer contour and the hub outer contour which meet the preset error standard;
s17, finding a pixel gradient amplitude point at the junction of the tire and the ground in an included angle range of 45 degrees below left and 45 degrees below right of the center of the tire, and drawing a tire and ground parting line by taking the found pixel gradient amplitude point as a reference to obtain an interface between the tire and the ground after the tire is deformed;
s18, carrying out endpoint processing on image pixels of the interface between the tire and the ground from bottom to top, calculating Y coordinate difference values of two adjacent pixel points, and judging that any pixel point is an endpoint of the contact between the tire and the ground when the Y coordinate difference value of the pixel point adjacent to the pixel point is larger than a preset coordinate difference value threshold value, so as to obtain the real contact pixel length of the tire and the ground.
2. The method for weighing a non-contact vehicle based on multispectral imaging technology according to claim 1, wherein in step S1, the process of acquiring a side thermal imaging image of a target sample tire in a normal operating state comprises the following steps:
centering the thermal imaging acquisition assembly to the center of the tire hub by utilizing infrared laser calibration; capturing side thermal imaging information of the tire in a normal working state by adopting a thermal imaging acquisition component; the side thermal imaging information captured by the thermal imaging acquisition assembly is saved in a CSV format.
3. The method of weighing a vehicle in a non-contact manner based on multispectral imaging technology according to claim 1, wherein in step S1, the mechanical deformation parameters include: tire maximum pixel radius R, tire maximum pixel area S 1 The hub pixel radius r, the contact pixel length l between the tire and the ground, the pixel distance h between the center of the tire and the ground, and the equivalent pixel area S after the tire is deformed 2 The image pixel area difference deltas before and after the tire deformation, and the tire-to-ground contact parting line pixel length L.
4. The method for weighing a vehicle without contact based on multispectral imaging technique according to claim 1, wherein the obtained parameters of the mechanical deformation of the tire are modified by a scaling factor α:
wherein Rim is the radius of the hub; r is the radius of the hub pixel obtained by fitting;
the corrected mechanical deformation parameters are as follows:
r true ＝α×r
R true ＝α×R
S 1true ＝α 2 ×S 1
l true ＝α×l
h true ＝α×h
S 2true ＝α 2 ×S 2
ΔS true ＝α 2 ×ΔS
L true ＝α×L
wherein r is true 、R true 、S 1true 、l true 、h true 、S 2true 、ΔS true And L true The method comprises the steps of respectively correcting the hub pixel radius, the maximum pixel radius of the tire, the maximum pixel area of the tire, the contact pixel length of the tire and the ground, the pixel distance from the center of the tire to the ground, the equivalent pixel area after the tire is deformed, the image pixel area difference before and after the tire is deformed and the contact parting line pixel length of the tire and the ground; r, S 1 、l、h、S 2 And delta S and L are respectively the maximum pixel radius of the tire, the maximum pixel area of the tire, the contact pixel length of the tire and the ground, the pixel distance from the center of the tire to the ground, the equivalent pixel area after the tire is deformed, the image pixel area difference before and after the tire is deformed and the contact parting line pixel length of the tire and the ground, which are obtained by fitting.
5. The non-contact vehicle weighing method based on the multispectral imaging technology according to claim 1, wherein in the step S2, based on a deep learning end-to-end OCR character recognition algorithm, OCR character recognition is performed on an optical image of a target sample tire sidewall, and size information and barometric pressure information of the tire are obtained according to the recognition result; the size information comprises tire section height H, tire section width b, hub radius Rim and air pressure information atm; the tire true air pressure is 1.1 to 1.2 times the maximum air pressure indicated by the tire identifier.
6. The method of claim 1, wherein in step S3, each training sample comprises 12 mechanical characteristicsSign and 1 tag; the mechanical characteristics are as follows: maximum radius R of tire true Maximum area S of tyre 1true Hub radius rtrue, length of tire contact with ground l true Distance h from center of tire to ground true Equivalent area S after deformation of tire 2true Area difference DeltaS before and after tire deformation true Length L of tire-to-ground contact parting line true Tire section height H, tire section width b, and tire air pressure atm; the label is as follows: tire-ground contact force F;
the machine learning model is an integrated decision tree model reflecting the mapping relation between 12 mechanical features and 1 mechanical response.
7. The method of claim 1, further comprising the steps of:
s5, carrying out temperature correction on the real load of the tire predicted by the machine learning model through a regression coefficient beta:
in the method, in the process of the invention,for the tire real load after temperature correction, F is the tire real load predicted by the machine learning model, < ->Is the average temperature of the tire surface; the value range of the regression coefficient beta is 0.9 to 1.5, when the tire surface temperature reaches 90 percent of the cold tire pressure temperature corresponding to the standard tire pressure, the value is 0.9, the middle is taken by linear interpolation, when the tire surface temperature reaches 150 percent of the cold tire pressure temperature corresponding to the standard tire pressure, the value is 1.5, and the middle is taken by linear interpolation;
s6, predicting the whole vehicle load W according to the number of the axles:
8. the method for weighing a vehicle in a non-contact manner based on the multispectral imaging technique according to claim 1, wherein after extracting the contour image after the tire is segmented, the pixel points of the contour image are mapped back into the pixel temperature matrix data, and the average temperature of the tire surface is calculated by using the temperature data in the tire contour
Where n represents the number of temperature pixels captured by thermal imaging of the tire surface, T i The temperature magnitude of the ith temperature data for the tire surface.
9. An automated load prediction system, comprising a thermal imaging acquisition component, a data processing device, an optical image acquisition component, an OCR device, a machine learning prediction model and a model updating device;
the data processing device comprises a storage unit, a thermal imaging data processing program and an OCR character recognition program;
the storage unit is used for storing the thermal imaging image of the tire to be detected shot by the thermal imaging acquisition assembly and the optical image of the side wall of the tire to be detected shot by the optical image acquisition assembly; the OCR character recognition program is used for recognizing the imprinting characters contained in the optical image of the tire side wall; the data processing device acquires the tire size and the air pressure information of the tire to be detected through the stamping characters and transmits the tire size and the air pressure information to the machine learning prediction model; the thermal imaging data processing program processes the thermal imaging image of the tire to be detected according to the method of any one of claims 1-8, calculates mechanical deformation parameters of the tire to be detected, and transmits the mechanical deformation parameters to a machine learning prediction model;
the machine learning prediction model processes the imported mechanical deformation parameters, the tire size and the air pressure information of the tire to be detected, and calculates to obtain the load of the tire to be detected;
the model updating device is used for importing new sample data into the machine learning model and updating the machine learning model.
CN202211076683.XA 2022-09-02 2022-09-02 Non-contact vehicle weighing method based on multispectral imaging technology Active CN115497058B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211076683.XA CN115497058B (en) 2022-09-02 2022-09-02 Non-contact vehicle weighing method based on multispectral imaging technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211076683.XA CN115497058B (en) 2022-09-02 2022-09-02 Non-contact vehicle weighing method based on multispectral imaging technology

Publications (2)

Publication Number Publication Date
CN115497058A CN115497058A (en) 2022-12-20
CN115497058B true CN115497058B (en) 2023-09-01

Family

ID=84468846

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211076683.XA Active CN115497058B (en) 2022-09-02 2022-09-02 Non-contact vehicle weighing method based on multispectral imaging technology

Country Status (1)

Country Link
CN (1) CN115497058B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110659649A (en) * 2019-09-29 2020-01-07 哈尔滨工业大学 Image processing and character recognition algorithm based on near infrared light imaging
CN111735524A (en) * 2020-08-27 2020-10-02 湖南大学 Tire load obtaining method based on image recognition, vehicle weighing method and system
CN114092683A (en) * 2021-11-09 2022-02-25 湖南中登科技有限公司 Tire deformation identification method and device based on visual feedback and depth network

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019078612A (en) * 2017-10-24 2019-05-23 コニカミノルタ株式会社 Tire deformation amount calculating device, overload detection system, and tire deformation amount calculation method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110659649A (en) * 2019-09-29 2020-01-07 哈尔滨工业大学 Image processing and character recognition algorithm based on near infrared light imaging
CN111735524A (en) * 2020-08-27 2020-10-02 湖南大学 Tire load obtaining method based on image recognition, vehicle weighing method and system
CN114092683A (en) * 2021-11-09 2022-02-25 湖南中登科技有限公司 Tire deformation identification method and device based on visual feedback and depth network

Also Published As

Publication number Publication date
CN115497058A (en) 2022-12-20

Similar Documents

Publication Publication Date Title
CN112380952B (en) Power equipment infrared image real-time detection and identification method based on artificial intelligence
CN110889398B (en) Multi-modal image visibility detection method based on similarity network
CN110866430B (en) License plate recognition method and device
CN109099929B (en) Intelligent vehicle positioning device and method based on scene fingerprints
CN112434586B (en) Multi-complex scene target detection method based on domain self-adaptive learning
CN111339975B (en) Target detection, identification and tracking method based on central scale prediction and twin neural network
CN111259704B (en) Training method of dotted lane line endpoint detection model
KR101374139B1 (en) Monitoring method through image fusion of surveillance system
CN112861631B (en) Wagon balance human body intrusion detection method based on Mask Rcnn and SSD
CN111582339B (en) Vehicle detection and recognition method based on deep learning
CN113159043B (en) Feature point matching method and system based on semantic information
CN111310662A (en) Flame detection and identification method and system based on integrated deep network
CN113326846B (en) Rapid bridge apparent disease detection method based on machine vision
CN115493679A (en) Toll station vehicle dynamic weighing system based on multi-view-field thermal imaging technology
CN112598066A (en) Lightweight road pavement detection method and system based on machine vision
CN116740758A (en) Bird image recognition method and system for preventing misjudgment
CN117705815B (en) Printing defect detection method based on machine vision
CN113158954B (en) Automatic detection method for zebra crossing region based on AI technology in traffic offsite
CN113221957B (en) Method for enhancing radar information fusion characteristics based on center
CN115497058B (en) Non-contact vehicle weighing method based on multispectral imaging technology
CN111814647A (en) Two-stage bicycle subclass key point confidence coefficient image CNN identification method
CN111476314A (en) Fuzzy video detection method integrating optical flow algorithm and deep learning
CN116310902A (en) Unmanned aerial vehicle target detection method and system based on lightweight neural network
CN115690770A (en) License plate recognition method based on space attention characteristics in non-limited scene
CN114943738A (en) Sensor packaging curing adhesive defect identification method based on visual identification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant