CN111507270A - Vehicle illegal behavior detection system based on block chain and deep learning - Google Patents
Vehicle illegal behavior detection system based on block chain and deep learning Download PDFInfo
- Publication number
- CN111507270A CN111507270A CN202010308506.4A CN202010308506A CN111507270A CN 111507270 A CN111507270 A CN 111507270A CN 202010308506 A CN202010308506 A CN 202010308506A CN 111507270 A CN111507270 A CN 111507270A
- Authority
- CN
- China
- Prior art keywords
- encoder
- license plate
- decoder
- safety belt
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a vehicle illegal behavior detection system based on a block chain and deep learning. The system comprises: the system comprises a shared feature extraction unit, a vehicle line pressing judgment unit, a safety belt wearing judgment unit and a license plate identification unit, wherein all nodes in a cloud server cluster are loaded with parameters and weights required by a vehicle illegal behavior detection deep neural network; and aiming at each neural network reasoning request, selecting a plurality of available nodes from the cloud server cluster, using parameters of network modules respectively distributed in different available nodes as block data, generating a block chain private chain, and executing deep neural network reasoning for vehicle illegal behavior detection, thereby realizing vehicle illegal detection. By utilizing the invention, in the vehicle illegal detection, the safety belt wearing detection and the line pressing detection are realized, and the detection precision and the safety performance of the system are improved.
Description
Technical Field
The invention belongs to the technical field of deep learning and block chains, and particularly relates to a vehicle illegal behavior detection system based on a block chain and deep learning.
Background
With the improvement of social and economic levels and the increasingly developed traffic networks, the holding quantity of urban automobiles is also increased on a large scale. Although the driving behaviors of drivers are restrained by traffic laws and regulations, the illegal behaviors of automobiles influencing the road traffic safety in urban road traffic are still rare, such as illegal lane changing by a solid line and no safety belt fastening of the drivers.
The automobile illegal lane changing and the automobile illegal driving without following the lane line are important reasons for causing local traffic jam, and the detection of the two illegal behaviors is based on the vehicle line pressing detection. The driver can easily and conveniently drive the vehicle to get on the road without fastening a safety belt, and once an accident occurs, the safety of the driver can be fatally influenced. Therefore, on increasingly congested urban roads, efficient management of illegal behaviors of automobiles is also in need.
At present, automobile line pressing detection is realized based on an underground induction coil. The induction coil is easy to damage, the maintenance cost is high, the installation process is complicated, the time consumption is long, and the detection precision is low. There is still no corresponding detection method as to whether the seat belt is worn. Moreover, the server cluster used for calculation is easy to leak information and has low safety performance.
Therefore, the existing field of vehicle illegal behavior detection has the problems of low detection precision, lack of safety belt wearing detection and low system safety performance.
Disclosure of Invention
The invention aims to provide a vehicle illegal behavior detection system based on a block chain and deep learning, aiming at the defects in the prior art, so that the wearing detection and the line pressing detection of a safety belt are realized, and the detection precision and the safety performance of the system are improved.
A block chain and deep learning based vehicle law violation detection system, the system comprising:
the shared feature extraction unit is used for extracting the features of the color image of the monitoring area by using a shared feature encoder;
the vehicle line pressing judging unit is used for analyzing the output of the shared feature encoder to obtain a vehicle line pressing judging result and comprises a global feature decoder, a global feature encoder and a first full-connection network;
the safety belt wearing judgment unit is used for analyzing the output of the shared characteristic encoder to obtain a safety belt wearing judgment result and comprises a safety belt segmentation decoder, a safety belt characteristic encoder and a second full-connection network;
the license plate recognition unit is used for analyzing the output of the shared characteristic encoder to obtain a license plate recognition result, and comprises a license plate characteristic encoder, a license plate position characteristic decoder and a number recognition OCR module;
the system comprises a shared characteristic encoder, a global characteristic decoder, a global characteristic encoder, a first fully-connected network, a safety belt segmentation decoder, a safety belt characteristic encoder, a second fully-connected network, a license plate characteristic encoder and a license plate position characteristic decoder, wherein the shared characteristic encoder, the global characteristic decoder, the first fully-connected network, the safety belt segmentation decoder, the safety belt characteristic encoder, the second fully-connected network, the license plate characteristic encoder and the license plate position characteristic decoder form;
the system also comprises a cloud server cluster, wherein all nodes in the cloud server cluster are loaded with the weight and the parameters required by the vehicle illegal behavior detection deep neural network; according to each vehicle illegal behavior detection deep neural network reasoning request, selecting a plurality of available nodes from a cloud server cluster, taking weights and parameters required by a shared feature encoder, a global feature decoder, a global feature encoder, a first full-connection network, a safety belt segmentation decoder, a safety belt feature encoder, a second full-connection network, a license plate feature encoder and a license plate position feature decoder which are respectively distributed in different available nodes as block data, generating a vehicle illegal behavior detection deep neural network reasoning block chain private chain according to a vehicle illegal behavior detection deep neural network reasoning sequence, and executing vehicle illegal behavior detection deep neural network reasoning.
Further, the block in the private chain of block chains encrypts the neural network inference intermediate result data to be transmitted to the next block, and decrypts the neural network inference intermediate result data received from the previous block.
Further, the global feature decoder is used for decoding the features output by the shared feature encoder;
a global feature encoder for encoding the output of the global feature decoder to further extract features;
and the first full-connection network is used for performing weighted classification on the output of the global feature encoder and outputting a vehicle line pressing judgment result.
Further, the safety belt segmentation decoder is used for decoding the features output by the shared encoder to obtain a safety belt segmentation mask;
the safety belt feature encoder is used for extracting features of the safety belt segmentation mask;
and the second full-connection network is used for carrying out weighted classification on the characteristics of the safety belt characteristic encoder to obtain a judgment result of whether the safety belt is worn.
Further, the license plate feature encoder is used for further extracting features output by the shared encoder;
the license plate position characteristic decoder is used for decoding the characteristics obtained by the license plate characteristic encoder to obtain a license plate segmentation position heat map;
and the number recognition OCR module is used for recognizing the license plate numbers in the license plate positions in the license plate segmentation position heat map.
Further, a shared feature encoder, a global feature decoder, a global feature encoder, a first fully-connected network, a safety belt segmentation decoder, a safety belt feature encoder, a second fully-connected network, a license plate feature encoder and a license plate position feature decoder are respectively and properly subdivided, parameters of each subdivided module which are respectively distributed in different nodes are used as block data, and a vehicle illegal behavior detection deep neural network reasoning block chain private chain is generated according to a vehicle illegal behavior detection deep neural network reasoning sequence.
Further, encryption is performed using an RSA encryption mechanism.
Further, the system further comprises: and the visualization unit is used for displaying the monitoring area image and the vehicle illegal information on a foreground page by combining the Web GIS technology.
Compared with the prior art, the invention has the following beneficial effects:
1. the method adopts the multitask deep neural network to analyze the image of the monitored area to obtain the vehicle line pressing judgment result, the safety belt wearing judgment result and the license plate recognition result.
2. The invention uses the shared characteristic encoder to have faster and more stable convergence, and can not generate the situations encountered by the traditional naive image classification scheme, such as incapability of training, low precision, overfitting and the like.
3. The method is based on the block chain technology, reasonably divides the vehicle illegal behavior detection deep neural network, dynamically generates the block chain private chain aiming at each network reasoning request, and compared with the traditional single-machine execution, not only improves the parallel performance of the system, but also has better fault-tolerant performance.
4. The block chain private chain is generated in real time according to available nodes in the cloud server cluster, is not easy to attack and crack, and improves the security performance of system data.
5. The invention encrypts the data between the private chains of the network inference block chain, prevents the leakage of the transmission data between the private chain blocks of the block chain, ensures the confidentiality of the transmission data, has convenient RSA encryption operation calculation and small calculation amount, and can not increase the system burden while improving the confidentiality and the safety performance.
Drawings
FIG. 1 is a diagram of a neural network architecture of the system of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
At present, the detection method based on the illegal action that a driver does not fasten a safety belt is less, and the method is not applied to urban traffic management in a large scale. Based on the existing snapshot camera, the invention provides a vehicle illegal behavior detection system based on a block chain and deep learning, and detection of vehicle pressing lines and unbelted safety belts of a driver is completed. FIG. 1 is a diagram of a neural network architecture of the system of the present invention. The following description will be made by way of specific examples.
The first embodiment is as follows:
the vehicle illegal behavior detection system based on the block chain and the deep learning comprises:
and the image acquisition unit is used for acquiring a color image of the monitoring area.
The invention adopts the neural network structure based on the image to detect the illegal behavior of the vehicle, so that the image of the monitoring area is firstly obtained. Preferably, the monitoring area is photographed by a color camera to obtain a color image of the monitoring area. Therefore, the vehicle illegal behavior detection system based on the block chain and the deep learning inputs color image information and outputs information on whether the current vehicle is pressed and whether the driver fastens a safety belt and corresponding license plate information.
Specifically, when the vehicle travels to a specified field of view, the camera captures current screen information. It should be noted that the camera is arranged in the area where the line pressing and lane changing are forbidden according to the traffic management department. The practitioner can set the appropriate image capture area at his or her discretion.
In order to realize the detection of the illegal behaviors of the vehicle, the invention designs a mixed deep neural network structure aiming at the tasks to realize related functions, and the neural network structure of the system is shown as figure 1.
In the design of the hybrid deep neural network, in order to enhance the context information, prevent over-fitting and under-fitting and improve the detection precision and the network resource utilization rate, the invention uses the following means: feature encoders, a multi-tasking network, and encoder-decoder mechanisms are shared to improve characterization accuracy and loss function accuracy. It is worth to be noted that the encoder expands the channels of the feature map, and the image size is reduced, i.e. the spatial accuracy is reduced and the number of types of feature intensity is increased, whereas the decoder reduces the number of channels and increases the spatial accuracy of the feature map.
After the image acquisition unit is used for acquiring the color image of the monitoring area, the invention acquires the illegal vehicle behavior information through three functional units, namely a vehicle line pressing judgment unit, a safety belt wearing judgment unit and a license plate number identification unit. The three functional units are all realized on the basis of a neural network, and in order to improve the calculation efficiency of the vehicle illegal behavior detection deep neural network, a shared feature encoder is designed for encoding the color image of the monitoring area acquired by a camera so as to finish primary shared feature extraction.
And the shared feature extraction unit is used for extracting the features of the color image of the monitoring area by using the shared feature encoder.
The features extracted by the shared feature encoder can support the detection of vehicle line pressing, safety belts and license plate position features which are potentially related to the geometric information of the vehicle. The above features are not explicitly obtained by the conventional direct image classification. Explicit derivation of the above features can help the network to take into account the potential relationship of the vehicle driver, the precise location of the vehicle center, and the overall location of the vehicle. The whole network is trained, loss items are more, the hierarchy of the characterization is based on details in the image, and therefore the constraint of the characterization is more accurate and is not over-constrained. Thus, for a given task, the use of a shared feature encoder converges faster and more stably without the inability to train, with low accuracy, over-fit, and the like, encountered with conventional naive image classification schemes.
The encoder and the decoder can be realized in various ways, in order to take the size of a large target into consideration, the invention proposes to adopt an hourglass network to extract features, and an implementer can also select a proper module design in a neural network according to the size of an image and the occupation of a video memory, such as Residual Block, Bottleneck Block, CNN Block and the like. The encoder, the decoder and the full-connection network of the invention adopt which network design, an implementer can select according to the specific implementation requirements, and the modularization idea is the protection content of the invention.
And the output of the shared characteristic encoder is used as the input of a vehicle line pressing judgment unit, a safety belt wearing judgment unit and a license plate number identification unit.
And the vehicle pressing line judging unit is used for analyzing the output of the shared feature encoder to obtain a vehicle pressing line judging result, and comprises a global feature decoder, a global feature encoder and a first full-connection network.
The vehicle line pressing judgment neural network comprises a global feature decoder, a global feature encoder and a first fully-connected network.
A global feature decoder for decoding the output of the shared feature encoder.
And the global feature encoder is used for encoding the output of the global feature decoder so as to further extract features.
And the first full-connection network is used for performing weighted classification on the output of the global feature encoder and outputting a vehicle line pressing judgment result.
Different from the safety belt wearing judgment unit and the license plate recognition unit, the vehicle line pressing judgment unit uses the mixed global features obtained by the global feature decoder and the global feature encoder instead of the filtered features of the safety belt and the license plate branch. The traditional vehicle position information is described by using a boundary box, and the two-dimensional boundary box ignores visual errors caused by perspective, so that the three-dimensional information of the vehicle cannot be considered. The scheme of the three-dimensional bounding box has the problems of large calculation amount and limited precision. The vehicle line pressing judgment is carried out based on the principle of image classification, and does not explicitly specify the lane line and the vehicle position. Therefore, the global feature decoder, the global feature encoder and the first fully-connected network are adopted to carry out direct image classification by utilizing the features of the shared feature encoder of the multitask network, so that the calculation efficiency can be improved, the detection precision can be improved, and the labeling difficulty can be reduced.
The shared feature encoder can support safety belt and license plate position feature detection, so that the context features are richer, and the global feature decoder is designed to decode the features obtained by the shared feature encoder, so that compared with the feature extraction performed by continuous encoding, the spatial domain precision of the feature tensor is improved, the useful features can be further extracted, and further the classification precision of the network is improved. The global feature decoder performs decoding processing such as convolution and up-sampling on the feature diagram output of the shared feature encoder, and finally inputs the feature diagram output to the first full-connection network based on encoding and down-sampling through the global feature encoder. The first fully-connected network uses one-hot coding, namely, confidence of no-line pressing and line pressing are respectively output. The result output of the vehicle line-pressing determination unit is similar to a classical image classification deep neural network, so that Argmax post-processing is used.
It is worth mentioning that the vehicle pressing line generally includes two cases, one is a temporary pressing line in which the driver does not control the vehicle direction, and the distance of pressing the solid line is very small. The other is that the vehicle changes lanes illegally or occupies a lane, and in this case, the vehicle tire is pressed by a large distance of the solid line. Therefore, during training, the implementer marks the vehicle line pressing behavior for the case that the vehicle tire pressure has a large solid line distance.
And the safety belt wearing judging unit is used for analyzing the output of the shared characteristic encoder to obtain a safety belt wearing judging result, and comprises a safety belt segmentation decoder, a safety belt characteristic encoder and a second full-connection network.
The safety belt wearing judging unit is a second functional unit of the system and is responsible for judging whether the driver wears the safety belt. The safety belt wearing judging unit comprises a safety belt segmentation decoder, a safety belt characteristic encoder and a second full-connection network.
And the safety belt segmentation decoder is used for decoding the features output by the shared feature encoder to obtain a safety belt segmentation mask. Based on the semantic segmentation deep neural network, the shared feature encoder and the safety belt segmentation decoder form a structure similar to the semantic segmentation deep neural network, and a safety belt mask is obtained.
In order to make the safety belt wearing judgment unit obtain the classification result, a safety belt characteristic encoder and a second full-connection network are designed. And the safety belt feature encoder is used for extracting features of the safety belt segmentation mask. And the second full-connection network is used for carrying out weighted classification on the characteristics of the safety belt characteristic encoder to obtain a judgment result of whether the safety belt is worn. The second fully-connected network also outputs a one-hot code, i.e. two confidences, namely with and without a belt, which the implementer finally needs to use Argmax to make a decision.
If the judgment result of the vehicle line pressing judgment unit is that the vehicle line pressing and/or the judgment result of the safety belt wearing judgment unit is that the safety belt is not worn, the identity of the vehicle with the traffic violation needs to be identified. Therefore, the invention designs the license plate recognition unit.
And the license plate recognition unit is used for recognizing the license plate number.
The license plate recognition unit comprises a license plate feature encoder, a license plate position feature decoder and a number recognition OCR module. And the license plate feature encoder is used for further extracting features output by the shared encoder. And the license plate position characteristic decoder is used for decoding the characteristics obtained by the license plate characteristic encoder to obtain a license plate segmentation position heat map. The hotspot in the heat map represents the confidence in the presence of the seat belt, with its hotspot shape being the bounding box centered at the anchor point location. And the number recognition OCR module is used for recognizing the license plate numbers in the license plate positions by the license plate segmentation position heat map obtained by the license plate feature decoder.
Because the license plate recognition technology is mature, an implementer can use the existing license plate recognition functional module which is packaged well. Therefore, the license plate number can be obtained only by determining the ROI according to the license plate segmentation position heat map, intercepting the ROI in an original image and sending the ROI into a number recognition OCR module.
The vehicle line-pressing judging unit, the safety belt wearing unit, the license plate recognizing unit and the shared characteristic encoder can be used as a deep neural network for training, the specific training process is that an implementer prepares images of various illegal conditions, marks whether a vehicle is pressed with a line, whether a safety belt is worn and the position of a license plate, and finally trains by using a cross entropy loss function, the network is of a single-input multi-output structure, and although the structure is slightly complex, the training form is consistent with that of a common image classification training method.
It should be noted that, in the specific implementation, the vehicle line pressing determination unit, the safety belt wearing determination unit, and the license plate recognition unit may be parallel, so that after the vehicle is illegal, the illegal and license plate numbers are transmitted to the data center together. On the other hand, in order to save computing resources, an implementer can call the license plate recognition unit to perform license plate recognition on the current vehicle after judging that the vehicle has at least one of illegal behaviors of vehicle pressing lines and safety belts not worn. The premise of the method is that the illegal behaviors are a few of driving behaviors, and the method has the advantages of saving computing resources and simultaneously prolonging the response time of the result.
In order to improve the confidentiality of the system and prevent data leakage, the invention designs the private chain of the block chain by combining the block chain technology.
The present invention is described in detail herein in connection with the block chain technique and the DNN technique. The block chain adopts block division data, a chain data structure is used, the data are used as blocks for verification and storage, the whole data structure is summarized, centralized hardware and management mechanisms do not exist, and decentralization is achieved. The block chain technology of the first generation is mainly applied to a distributed account book, the block chain technology of the second generation mainly realizes an intelligent contract, and the block chain idea and other field technologies can be combined by the block chain technology of the third generation, so that more and more presentation forms exist, and more emphasis is placed on system function service. The block chain private chain completely inherits the characteristics of the public chain, is not bound by a game mechanism, focuses more on data transmission and encryption and decryption processing of practical application, and can be better combined with technologies in other fields. For AI calculations, it is not necessary to store intermediate result data, and the logic of the chain is preserved to match the principle of neural network forward propagation.
The invention considers the problem that the contents of the image data in the plaintext are leaked in the uploading process and the processing process if the data are directly uploaded to the cloud for processing, so that the different modules of the deep neural network are used as blocks in a block chain private chain mode for carrying out dispersed reasoning, and the data transmitted among the blocks are encrypted, thereby realizing excellent performances of parallel reasoning, fault tolerance and data leakage prevention.
Based on the idea, the vehicle illegal behavior detection deep neural network is required to be divided into modules. The shared feature encoder, the global feature decoder, the global feature encoder, the first fully-connected network, the safety belt segmentation decoder, the safety belt feature encoder, the second fully-connected network, the license plate feature encoder and the license plate position feature decoder form a main component module of the vehicle illegal behavior detection deep neural network. The method comprises the following steps of taking a shared feature encoder, a global feature decoder, a global feature encoder, a first fully-connected network, a safety belt segmentation decoder, a safety belt feature encoder, a second fully-connected network, a license plate feature encoder and a license plate position feature decoder as different modules of the network. Thus, according to the inference sequence of the neural network shown in fig. 1, a deep neural network inference chain for detecting vehicle illegal behaviors can be obtained.
The specific inference process can be performed according to a chain logic. The network exists in parts that can be parallel: the safety belt wearing device comprises a vehicle pressing line judging unit, a safety belt wearing judging unit and a license plate recognizing unit. Thus, the feature encoder blocks may be shared before proceeding in parallel.
The system further comprises a cloud server cluster, and all nodes in the cloud server cluster are loaded with the weights and parameters required by the vehicle illegal behavior detection deep neural network. According to each vehicle illegal behavior detection deep neural network reasoning request, selecting a plurality of available nodes from a cloud server cluster, taking weights and parameters required by a shared feature encoder, a global feature decoder, a global feature encoder, a first full-connection network, a safety belt segmentation decoder, a safety belt feature encoder, a second full-connection network, a license plate feature encoder and a license plate position feature decoder which are respectively distributed in different available nodes as block data, and generating a vehicle illegal behavior detection deep neural network reasoning block chain private chain according to a vehicle illegal behavior detection deep neural network reasoning sequence. When selecting available nodes and performing node sorting, preferably, randomly sorting the available nodes in the cloud server cluster, and selecting the same number of computing nodes as the number of the blocks from the available nodes. For example, there are 10 available nodes, 9 nodes are selected from the available nodes, one node is randomly selected, and parameters such as the weight required by the shared feature encoder in the nodes are used as block data; and randomly taking another node, taking parameters such as the weight required by the global feature decoder in the node as block data, linking the block data with the previous block, and so on to generate a corresponding private chain of the vehicle illegal behavior detection deep neural network inference block according to the neural network inference sequence. Therefore, a plurality of vehicle illegal behavior detection deep neural network reasoning block private chains generated aiming at different requests can exist in the cloud server cluster at the same time, and the block private chains are dynamically generated, are not easy to crack by attacks, and have better confidentiality. And after the block chain private chain is obtained according to the inference sequence, performing network inference calculation on the inference request according to the inference sequence.
Furthermore, some modules of the neural network are difficult to be put into one node at a time, that is, the computation workload of some modules is large, such as a shared feature encoder, and it is difficult to complete the computation in a short time. Therefore, the modules of the neural network, namely the encoder, the decoder and the fully-connected network, can be further segmented in advance to obtain a shared feature encoder submodule group, a global feature decoder submodule group, a global feature encoder submodule group, a first fully-connected network submodule group, a safety belt segmentation decoder submodule group, a safety belt feature encoder submodule group, a second fully-connected network submodule group, a license plate feature encoder submodule group and a license plate position feature decoder submodule group. Therefore, according to the inference sequence of the neural network, a more subdivided vehicle illegal behavior detection deep neural network inference chain can be obtained. Correspondingly, aiming at each vehicle illegal behavior detection deep neural network reasoning request, a plurality of available nodes are selected from a cloud server cluster, weights and parameters required by sub-module groups of a shared feature encoder, a global feature decoder, a global feature encoder, a first full-connection network, a safety belt segmentation decoder, a safety belt feature encoder, a second full-connection network, a license plate feature encoder and a license plate position feature decoder which are respectively distributed in different available nodes are used as block data, and a vehicle illegal behavior detection deep neural network reasoning block chain private chain is generated according to a vehicle illegal behavior detection deep neural network reasoning sequence.
Further, in order to ensure the confidentiality of data between private chains of the block chain of the vehicle illegal behavior detection deep neural network inference and prevent data leakage, data transmission between blocks is encrypted. That is, data transmission from a node to a node of the private blockchain needs to use an encryption means, and a block in the private blockchain encrypts the neural network inference intermediate result data to be transmitted to the next block and decrypts the neural network inference intermediate result data received from the previous block. The RSA encryption mechanism is preferably used here.
In addition, the image acquisition unit can be used as a block and added into a block chain private chain, so that the purpose of ensuring that the output of the image acquisition unit is confidential through an encryption strategy and is not easy to intercept and tamper is achieved.
The RSA encryption mechanism employed by the block node of the present invention is described below.
Because the value of each element of the data tensor is mostly floating point number, the invention only encrypts the integer part of the data tensor, and the decimal part is kept equal to the original data.
The encryption key generation mode is as follows: first, two prime numbers are randomly generated, and assuming that p is 3 and q is 11, the product of the two numbers is 33 and f (p, q) is (p-1) (q-1) 20, the value of e and the value of f (p, q) are coprime, and assuming that e is 3, the encryption key (e, n) is (3, 33).
The encryption process is as follows: and if the product of each time is larger than n, dividing the product by n and taking the remainder, and continuing to multiply the product by a by the remainder until the multiplication by e is finished. Assuming that a is 5, the encrypted number is 26.
The decryption key generation mode is as follows: by using the algorithm Extended Euclidean, the input parameters p, q, e, i.e. 3, 11, 3, the decryption parameter (7,33) can be obtained.
The decryption process is similar to the encryption process except for the difference in key values. The value of a is assumed to be 5, the value of a is 26 after the encryption parameter is encrypted, and 5 can be obtained after the decryption parameter is obtained through the rule and the 26 is decrypted, namely, the original data is recovered.
Since the data center receiving the vehicle violation information and the license plate number information is authentic, different p, q may be periodically broadcast by the data center to generate a new key.
The implementer should know how to update the key, i.e. what value of p, q, specifically, as the case may be, when and how to update, there are a number of well-known methods, e.g. timed update, manual update.
And finally, uploading the illegal action information and the license plate information corresponding to the illegal action information to a data center. The data center is a center for monitoring all vehicle illegal behaviors. The data center and the cloud server can be located at one place or can be distributed in different geographic positions. And after the data center receives the data, decrypting to restore the illegal vehicle behavior information.
In order to visually present the traffic information state of the current area, a user visually acquires the vehicle illegal information of the current area in real time, so that an administrator more visually supervises the traffic state and provides early warning information and road traffic safety guarantee. The system also comprises a visualization unit which is used for combining the Web GIS technology, constructing a geographic information database and displaying the image information acquired by the camera in real time, the vehicle illegal information obtained by analysis and the license plate information of the illegal vehicle on a foreground Web page.
The method adopts the multitask deep neural network to analyze the image of the monitored area to obtain the vehicle line pressing judgment result, the safety belt wearing judgment result and the license plate recognition result. The invention uses the shared characteristic encoder to have faster and more stable convergence, and can not generate the situations encountered by the traditional naive image classification scheme, such as incapability of training, low precision, overfitting and the like. The method is based on the block chain technology, reasonably divides the vehicle illegal behavior detection deep neural network, dynamically generates the block chain private chain aiming at each network reasoning request, and compared with the traditional single-machine execution, not only improves the parallel performance of the system, but also has better fault-tolerant performance. The block chain private chain is generated in real time according to available nodes in the cloud server cluster, is not easy to attack and crack, and improves the security performance of system data. The invention encrypts the data between the private chains of the network inference block chain, prevents the leakage of the transmission data between the private chain blocks of the block chain, ensures the confidentiality of the transmission data, has convenient RSA encryption operation calculation and small calculation amount, and can not increase the system burden while improving the confidentiality and the safety performance.
The above embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the present invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. A vehicle illegal behavior detection system based on block chains and deep learning is characterized by comprising:
the shared feature extraction unit is used for extracting the features of the color image of the monitoring area by using a shared feature encoder;
the vehicle line pressing judging unit is used for analyzing the output of the shared feature encoder to obtain a vehicle line pressing judging result and comprises a global feature decoder, a global feature encoder and a first full-connection network;
the safety belt wearing judgment unit is used for analyzing the output of the shared characteristic encoder to obtain a safety belt wearing judgment result and comprises a safety belt segmentation decoder, a safety belt characteristic encoder and a second full-connection network;
the license plate recognition unit is used for analyzing the output of the shared characteristic encoder to obtain a license plate recognition result, and comprises a license plate characteristic encoder, a license plate position characteristic decoder and a number recognition OCR module;
the system comprises a shared characteristic encoder, a global characteristic decoder, a global characteristic encoder, a first fully-connected network, a safety belt segmentation decoder, a safety belt characteristic encoder, a second fully-connected network, a license plate characteristic encoder and a license plate position characteristic decoder, wherein the shared characteristic encoder, the global characteristic decoder, the first fully-connected network, the safety belt segmentation decoder, the safety belt characteristic encoder, the second fully-connected network, the license plate characteristic encoder and the license plate position characteristic decoder form;
the system also comprises a cloud server cluster, wherein all nodes in the cloud server cluster are loaded with parameters required by the vehicle illegal behavior detection deep neural network; according to each vehicle illegal behavior detection deep neural network reasoning request, selecting a plurality of available nodes from a cloud server cluster, taking parameters required by a shared feature encoder, a global feature decoder, a global feature encoder, a first full-connection network, a safety belt segmentation decoder, a safety belt feature encoder, a second full-connection network, a license plate feature encoder and a license plate position feature decoder which are respectively distributed in different available nodes as block data, generating a vehicle illegal behavior detection deep neural network reasoning block chain private chain according to a vehicle illegal behavior detection deep neural network reasoning sequence, and executing vehicle illegal behavior detection deep neural network reasoning.
2. The system of claim 1, wherein a block in the private chain of blocks encrypts neural network inference intermediate result data that it is to transmit to a next block and decrypts neural network inference intermediate result data that it receives from a previous block.
3. The system of claim 2, wherein the encryption is performed using an RSA encryption mechanism.
4. The system of any of claims 1-3, wherein the global feature decoder is to decode a feature output by the shared feature encoder;
a global feature encoder for encoding the output of the global feature decoder to further extract features;
and the first full-connection network is used for performing weighted classification on the output of the global feature encoder and outputting a vehicle line pressing judgment result.
5. A system according to any of claims 1 to 3, wherein the seat belt segmentation decoder is arranged to decode features output by the shared encoder to obtain a seat belt segmentation mask;
the safety belt feature encoder is used for extracting features of the safety belt segmentation mask;
and the second full-connection network is used for carrying out weighted classification on the characteristics of the safety belt characteristic encoder to obtain a judgment result of whether the safety belt is worn.
6. The system of any one of claims 1-3, wherein the license plate feature encoder is configured to perform further feature extraction on features output by the shared encoder;
the license plate position characteristic decoder is used for decoding the characteristics obtained by the license plate characteristic encoder to obtain a license plate segmentation position heat map;
and the number recognition OCR module is used for recognizing the license plate numbers in the license plate positions in the license plate segmentation position heat map.
7. The system according to any one of claims 1 to 3, wherein the shared feature encoder, the global feature decoder, the global feature encoder, the first fully-connected network, the safety belt segmentation decoder, the safety belt feature encoder, the second fully-connected network, the license plate feature encoder, and the license plate position feature decoder are respectively and properly subdivided, parameters of each subdivided module respectively distributed in different nodes are used as block data, and a vehicle illegal behavior detection deep neural network inference block chain private chain is generated according to a vehicle illegal behavior detection deep neural network inference sequence.
8. The system of claim 1, further comprising: and the visualization unit is used for displaying the monitoring area image and the vehicle illegal information on a foreground page by combining the WebGIS technology.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010308506.4A CN111507270A (en) | 2020-04-18 | 2020-04-18 | Vehicle illegal behavior detection system based on block chain and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010308506.4A CN111507270A (en) | 2020-04-18 | 2020-04-18 | Vehicle illegal behavior detection system based on block chain and deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111507270A true CN111507270A (en) | 2020-08-07 |
Family
ID=71864712
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010308506.4A Withdrawn CN111507270A (en) | 2020-04-18 | 2020-04-18 | Vehicle illegal behavior detection system based on block chain and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111507270A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112954399A (en) * | 2021-02-09 | 2021-06-11 | 腾讯科技（深圳）有限公司 | Image processing method and device and computer equipment |
| CN113780132A (en) * | 2021-08-31 | 2021-12-10 | 武汉理工大学 | Lane line detection method based on convolutional neural network |
-
2020
- 2020-04-18 CN CN202010308506.4A patent/CN111507270A/en not_active Withdrawn
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112954399A (en) * | 2021-02-09 | 2021-06-11 | 腾讯科技（深圳）有限公司 | Image processing method and device and computer equipment |
| CN112954399B (en) * | 2021-02-09 | 2021-12-17 | 腾讯科技（深圳）有限公司 | Image processing method and device and computer equipment |
| CN113780132A (en) * | 2021-08-31 | 2021-12-10 | 武汉理工大学 | Lane line detection method based on convolutional neural network |
| CN113780132B (en) * | 2021-08-31 | 2023-11-24 | 武汉理工大学 | Lane line detection method based on convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | Inferring dockless shared bike distribution in new cities | |
| KR20210134638A (en) | autonomous vehicle system | |
| Zhang et al. | City brain: practice of large‐scale artificial intelligence in the real world | |
| WO2021115133A1 (en) | Driving-behavior recognition method, apparatus, electronic device, and storage medium | |
| CN111507270A (en) | Vehicle illegal behavior detection system based on block chain and deep learning | |
| CN111563433A (en) | Wisdom building site is monitored system of overflowing water based on block chain | |
| Awan et al. | Modeling dynamic spatio-temporal correlations for urban traffic flows prediction | |
| Bustos et al. | Explainable, automated urban interventions to improve pedestrian and vehicle safety | |
| Masuduzzaman et al. | UAV-based MEC-assisted automated traffic management scheme using blockchain | |
| US20220021660A1 (en) | Data privacy system | |
| US20160314306A1 (en) | Image tagging device | |
| Talukder et al. | Trajectory-based signal control in mixed connected vehicle environments | |
| CN111414499A (en) | Operation personnel safety belt wearing detection system based on block chain and BIM | |
| CN111524131A (en) | Wall crack detection system based on block chain and BIM | |
| Chen et al. | MDFD: A multi-source data fusion detection framework for Sybil attack detection in VANETs | |
| CN117576655A (en) | Traffic sign detection method and system based on federal learning | |
| CN111459186B (en) | Unmanned aerial vehicle cruise system based on deep neural network and block chain | |
| CN111523434A (en) | Building site personnel abnormal gathering detection system based on block chain and BIM | |
| CN116229334B (en) | Pig farm cross-camera event management method and system based on block chain | |
| CN112667888A (en) | Big data processing system based on block chain | |
| CN111414500A (en) | Steel wire rope breakage early warning system based on block chain and BIM | |
| CN115203354B (en) | Vehicle code track pre-association method and device, computer equipment and storage medium | |
| CN116977484A (en) | Image desensitizing method, device, electronic equipment and storage medium | |
| EP4062579B1 (en) | Anonymous collection of data from a group of entitled members | |
| KR102436142B1 (en) | Method for providing probabilistic prediction service using violation factors photographed by intermittent camera and apparatus using the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200807 |