CN115783919A - Electric vehicle monitoring system and method for elevator - Google Patents
Electric vehicle monitoring system and method for elevator Download PDFInfo
- Publication number
- CN115783919A CN115783919A CN202211497090.0A CN202211497090A CN115783919A CN 115783919 A CN115783919 A CN 115783919A CN 202211497090 A CN202211497090 A CN 202211497090A CN 115783919 A CN115783919 A CN 115783919A
- Authority
- CN
- China
- Prior art keywords
- interest
- feature map
- region
- elevator
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 180
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000001514 detection method Methods 0.000 claims abstract description 58
- 238000009826 distribution Methods 0.000 claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 40
- 238000010586 diagram Methods 0.000 claims description 69
- 239000011159 matrix material Substances 0.000 claims description 63
- 239000013598 vector Substances 0.000 claims description 44
- 238000013527 convolutional neural network Methods 0.000 claims description 39
- 238000012937 correction Methods 0.000 claims description 24
- 238000000605 extraction Methods 0.000 claims description 19
- 238000003062 neural network model Methods 0.000 claims description 19
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 230000004927 fusion Effects 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 5
- 238000013473 artificial intelligence Methods 0.000 abstract description 14
- 238000013135 deep learning Methods 0.000 abstract description 10
- 238000005516 engineering process Methods 0.000 abstract description 8
- 239000000284 extract Substances 0.000 abstract description 8
- 238000009432 framing Methods 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 28
- 238000003860 storage Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000007792 addition Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 230000002708 enhancing effect Effects 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 4
- 238000005065 mining Methods 0.000 description 4
- 208000006440 Open Bite Diseases 0.000 description 2
- 238000004873 anchoring Methods 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000003466 welding Methods 0.000 description 2
- 206010000369 Accident Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
- Maintenance And Inspection Apparatuses For Elevators (AREA)
Abstract
The system and the method adopt an artificial intelligence monitoring technology based on machine vision, carry out target framing on image information about an electric vehicle in an elevator monitoring image, then carry out unblocking processing on an interested area of the electric vehicle in the image, and extract overall content feature distribution information in the processed monitoring image by using an artificial intelligence algorithm based on deep learning so as to detect and judge whether the electric vehicle is carried in the elevator. Therefore, the intelligent monitoring and detection of the electric vehicle in the elevator can be accurately carried out.
Description
Technical Field
The present application relates to the field of elevator monitoring technologies, and more particularly, to a system and a method for monitoring an electric vehicle for an elevator.
Background
With the development of socio-economy, elevators have been fully integrated into our lives as an important means of transportation for people and goods. At present, in a residential building of a residential area, a resident often pushes an electric vehicle to a lift to be transported to a floor for charging, which causes great potential safety hazard, so that the life safety and property safety of the resident in the residential building are affected, however, the existing camera in the lift does not have the function of reminding the resident.
Therefore, an elevator electric vehicle monitoring scheme is desired.
Disclosure of Invention
The present application is proposed to solve the above-mentioned technical problems. The embodiment of the application provides an electric vehicle monitoring system and method for an elevator, which adopt an artificial intelligence monitoring technology based on machine vision to frame a target for image information about an electric vehicle in an elevator monitoring image, then carry out the shielding removal processing of an interested region of the electric vehicle of the image, and extract the whole content feature distribution information in the processed monitoring image by using an artificial intelligence algorithm based on deep learning so as to detect and judge whether the electric vehicle is carried in the elevator. Therefore, the intelligent monitoring and detection of the electric vehicle in the elevator can be accurately carried out.
Accordingly, according to one aspect of the present application, there is provided an electric vehicle monitoring system for an elevator, comprising:
the monitoring module is used for acquiring an elevator monitoring image acquired by a camera deployed in the elevator;
the vehicle target detection module is used for enabling the elevator monitoring image to pass through a vehicle target detection network so as to obtain an interested area image;
a de-occlusion module for passing the region of interest image through a de-occlusion generator based on a challenge generation network to obtain a generated region of interest image;
the local feature extraction module is used for enabling the generated interested region image to pass through a deep convolution neural network model serving as a feature extractor so as to obtain an interested region feature map;
the global feature extraction module is used for enabling the region-of-interest feature map to pass through a non-local neural network model so as to obtain an enhanced region-of-interest feature map;
the characteristic distribution correction module is used for carrying out characteristic distribution correction on each characteristic matrix of the enhanced region-of-interest characteristic diagram along the channel dimension to obtain a corrected enhanced region-of-interest characteristic diagram; and
and the monitoring result generating module is used for enabling the corrected enhanced region of interest characteristic diagram to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the elevator is internally provided with the electric vehicle or not.
In the electric vehicle monitoring system for the elevator, the vehicle target detection network is based on an anchor window, and the target detection network based on the anchor window is Fast R-CNN, fast R-CNN or RetinaNet.
In the above electric vehicle monitoring system for elevator, the countermeasure generation network includes a generator and a discriminator, wherein the de-occlusion module is further configured to input the region-of-interest image into the de-occlusion generator based on the countermeasure generation network to output the generated region-of-interest image by the generator of the de-occlusion generator through deconvolution coding.
In the above-mentioned electric vehicle monitoring system for elevator, the local feature extraction module is further configured to: using each layer of the deep convolutional neural network model to respectively perform in the forward pass of the layer: performing convolution processing on input data to obtain a convolution characteristic diagram; performing mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; and outputting the final layer of the deep convolutional neural network model as the region-of-interest feature map, and inputting the first layer of the deep convolutional neural network model as the generated region-of-interest image.
In the above-mentioned electric vehicle monitoring system for elevator, the global feature extraction module is further configured to: encoding the region-of-interest feature map using the non-local neural network in a manner to obtain the enhanced region-of-interest feature map; wherein the encoding process comprises: respectively carrying out first point convolution processing, second point convolution processing and third point convolution processing on the characteristic diagram of the region of interest to obtain a first characteristic diagram, a second characteristic diagram and a third characteristic diagram; calculating a position-weighted sum between the first feature map and the second feature map to obtain a fused feature map; inputting the fused feature map into a Softmax function so as to map feature values of all positions in the fused feature map into a probability space to obtain a normalized fused feature map; calculating the position-point-based multiplication between the normalized fusion feature map and the third feature map to obtain a re-fusion feature map; embedding a Gaussian similarity function into the re-fused feature map to obtain a global similar feature map; performing fourth point convolution processing on the global similar feature map to adjust the channel number of the global similar feature map so as to obtain a channel adjustment global similar feature map; and calculating a weighted sum of the channel adjustment global similar feature map and the encrypted local feature map according to positions to obtain the enhanced region-of-interest feature map.
In the above-mentioned electric vehicle monitoring system for elevator, the characteristic distribution correction module is further configured to: performing characteristic distribution correction on each characteristic matrix of the characteristic diagram of the enhanced region of interest along the channel dimension by using the following formula to obtain the corrected characteristic diagram of the enhanced region of interest; wherein the formula is:
wherein
A respective feature matrix along a channel dimension representing the enhanced region of interest feature map,
is the eigenvalue of each position of the diagonal matrix obtained by linear transformation of each characteristic matrix along the channel dimension of the characteristic diagram of the enhanced region of interest,
is a two-norm of a vector, and
meaning that each value of the matrix is multiplied by a predetermined value,
indicating that the addition is by position,
a respective feature matrix along a channel dimension representing the corrected enhanced region of interest feature map.
In the above-mentioned electric vehicle monitoring system for elevator, the monitoring result generating module includes: the expansion unit is used for expanding each corrected enhanced region-of-interest feature matrix in the corrected enhanced region-of-interest feature map into one-dimensional feature vectors according to row vectors or column vectors and then cascading the one-dimensional feature vectors to obtain classified feature vectors; a full-connection coding unit, configured to perform full-connection coding on the classification feature vector using a full-connection layer of the classifier to obtain a coded classification feature vector; and the classification result generation unit is used for inputting the encoding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
According to another aspect of the present application, there is also provided an electric vehicle monitoring method for an elevator, including:
acquiring an elevator monitoring image acquired by a camera deployed in an elevator;
passing the elevator monitoring image through a vehicle target detection network to obtain an interested area image;
passing the region of interest image through a de-occlusion generator based on a challenge-generation network to obtain a generated region of interest image;
obtaining a region-of-interest characteristic diagram by passing the generated region-of-interest image through a deep convolutional neural network model serving as a characteristic extractor;
passing the region-of-interest feature map through a non-local neural network model to obtain an enhanced region-of-interest feature map;
performing characteristic distribution correction on each characteristic matrix of the characteristic diagram of the enhanced region of interest along the channel dimension to obtain a corrected enhanced region of interest characteristic diagram; and
and passing the corrected enhanced region-of-interest feature map through a classifier to obtain a classification result, wherein the classification result is used for indicating whether an electric vehicle is carried in the elevator.
In the method for monitoring the electric vehicle for the elevator, the vehicle target detection network is based on an anchor window, and the target detection network based on the anchor window is Fast R-CNN, fast R-CNN or RetinaNet.
In the above method for monitoring an electric vehicle for an elevator, the countermeasure generation network includes a generator and a discriminator, wherein the passing the region-of-interest image through a de-occlusion generator based on the countermeasure generation network to obtain a generated region-of-interest image includes: inputting the region-of-interest image into the resist generation network-based de-occlusion generator to output the generated region-of-interest image by de-convolution encoding by the de-occlusion generator.
In the above method for monitoring an electric vehicle for an elevator, the step of passing the generated image of the region of interest through a deep convolutional neural network model as a feature extractor to obtain a feature map of the region of interest includes: using the layers of the deep convolutional neural network model to respectively perform in the forward pass of the layers: performing convolution processing on input data to obtain a convolution characteristic diagram; performing mean pooling on the convolution feature map based on a local feature matrix to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the deep convolutional neural network model is the region-of-interest feature map, and the input of the first layer of the deep convolutional neural network model is the generated region-of-interest image.
In the above method for monitoring an electric vehicle for an elevator, the passing the feature map of the region of interest through a non-local neural network model to obtain an enhanced feature map of the region of interest includes: encoding the region-of-interest feature map using the non-local neural network in a manner to obtain the enhanced region-of-interest feature map; wherein the encoding process comprises: respectively carrying out first point convolution processing, second point convolution processing and third point convolution processing on the characteristic diagram of the region of interest to obtain a first characteristic diagram, a second characteristic diagram and a third characteristic diagram; calculating a position-weighted sum between the first feature map and the second feature map to obtain a fused feature map; inputting the fusion feature map into a Softmax function to map feature values of various positions in the fusion feature map into a probability space to obtain a normalized fusion feature map; calculating the position-based multiplication between the normalized fusion feature map and the third feature map to obtain a re-fusion feature map; embedding a Gaussian similarity function into the re-fused feature map to obtain a global similar feature map; performing fourth-point convolution processing on the global similar feature map to adjust the channel number of the global similar feature map so as to obtain a channel-adjusted global similar feature map; and calculating a weighted sum according to positions of the channel adjustment global similar feature map and the encrypted local feature map to obtain the enhanced region-of-interest feature map.
In the above method for monitoring an electric vehicle for an elevator, the performing feature distribution correction on each feature matrix along a channel dimension of the enhanced region-of-interest feature map to obtain a corrected enhanced region-of-interest feature map includes: performing feature distribution correction on each feature matrix of the enhanced region-of-interest feature map along the channel dimension by the following formula to obtain the corrected enhanced region-of-interest feature map; wherein the formula is:
wherein
A respective feature matrix along a channel dimension representing the enhanced region of interest feature map,
is the eigenvalue of each position of the diagonal matrix obtained by linear transformation of each characteristic matrix along the channel dimension of the characteristic diagram of the enhanced region of interest,
is a two-norm of a vector, an
Meaning that each value of the matrix is multiplied by a predetermined value,
it is shown that the addition by position,
respective feature matrices along a channel dimension representing the corrected enhanced region of interest feature map.
In the above method for monitoring an electric vehicle for an elevator, the passing the corrected enhanced feature map of the region of interest through a classifier to obtain a classification result, where the classification result is used to indicate whether the electric vehicle is loaded in the elevator, includes: expanding each corrected enhanced region-of-interest feature matrix in the corrected enhanced region-of-interest feature map into one-dimensional feature vectors according to row vectors or column vectors, and then cascading to obtain classified feature vectors; performing full-joint coding on the classification feature vectors by using a full-joint layer of the classifier to obtain coded classification feature vectors; and inputting the encoding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory in which computer program instructions are stored, which, when executed by the processor, cause the processor to perform the method of monitoring a motor vehicle for an elevator as described above.
According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to execute the method of electric vehicle monitoring for elevators as described above.
Compared with the prior art, the system and the method for monitoring the electric vehicle for the elevator adopt an artificial intelligence monitoring technology based on machine vision, perform target framing on image information of the electric vehicle in an elevator monitoring image, perform shielding removal processing on an interested region of the electric vehicle, extract whole content feature distribution information in the processed monitoring image by using an artificial intelligence algorithm based on deep learning, and accordingly perform detection and judgment on whether the electric vehicle is carried in the elevator. Therefore, the intelligent monitoring and detection of the electric vehicle in the elevator can be accurately carried out.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally indicate like parts or steps.
Fig. 1 is a schematic view of a scenario of an electric vehicle monitoring system for an elevator according to an embodiment of the present application.
Fig. 2 is a block diagram of an electric vehicle monitoring system for an elevator according to an embodiment of the present application.
Fig. 3 is a schematic diagram of the architecture of an electric vehicle monitoring system for an elevator according to an embodiment of the present application.
Fig. 4 is a block diagram of a monitoring result generation module in an electric vehicle monitoring system for an elevator according to an embodiment of the present application.
Fig. 5 is a flowchart of an electric vehicle monitoring method for an elevator according to an embodiment of the present application.
Fig. 6 is a block diagram of an electronic device according to an embodiment of the application.
Detailed Description
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only a few embodiments of the present application, and not all embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.
Summary of the application
As the background art mentioned above, with the development of socio-economic, elevators have been fully integrated into our lives as an important means of transportation of people and goods. At present, in a residential building of a residential area, residents often push an electric vehicle into an elevator to transport the elevator to a floor or to charge in an area where the electric vehicle is not allowed to park, such as a lobby, a stairway, a public walkway or a corridor, and the like, if electric vehicle fire disasters and the like possibly occur, great potential safety hazards can be caused, and therefore life safety and property safety of the residents in the residential building are affected, however, an existing camera in the elevator does not have the function of reminding the residents. Therefore, an elevator electric vehicle monitoring scheme is desired.
At present, deep learning and neural networks have been widely used in the fields of computer vision, natural language processing, speech signal processing, and the like. In addition, deep learning and neural networks also exhibit a level close to or even exceeding that of humans in the fields of image classification, object detection, semantic segmentation, text translation, and the like.
In recent years, deep learning and development of neural networks provide new solutions and schemes for intelligent monitoring of electric vehicles in elevators.
Correspondingly, the recognition and detection of the electric vehicle in the elevator or the area where the electric vehicle is not allowed to park, such as a hall and the like, can be performed by collecting the monitoring image through the camera in the elevator, but because the monitoring image has a large amount of information, the vehicle existing in the monitoring image may be a bicycle, and further the recognition of the electric vehicle is misjudged, and the boundary of the vehicle in the image becomes fuzzy due to the influence of environmental factors, so that the accurate recognition of the vehicle is difficult. And when gathering monitored control image by the camera of arranging in the elevator, the people can cause the vehicle to shelter from, and then influences the precision that the electric motor car detected. Meanwhile, the installation position of the camera is usually high, so that the acquired image has large distortion and the detection precision is also influenced. Based on the technical scheme, the method adopts an artificial intelligence monitoring technology based on machine vision, carries out target framing on image information about the electric vehicle in the elevator monitoring image, then carries out shielding removal processing on an interested region of the electric vehicle in the image, and extracts whole content feature distribution information in the processed monitoring image by using an artificial intelligence algorithm based on deep learning so as to detect and judge whether the electric vehicle is carried in the elevator. Like this, can accurately carry out the elevator or not allow the intelligent monitoring of electric motor car in the region that the electric motor car parked like the lobby detects, so as to avoid the electric motor car to get into the elevator or not allow the region that the electric motor car parked like the lobby etc. to cause the potential safety hazard, thereby prevent the elevator or not allow the region that the electric motor car parked like the lobby etc. to take place electric motor car conflagration etc., reduced because of the electric motor car is parked in disorder or get into the elevator etc. and lead to the emergence of the harm scheduling problem of fire incident to personnel or article, and then ensured the life safety and the property safety of resident family in the residential building.
Specifically, in the technical scheme of the application, whether the elevator is internally provided with the electric vehicle is taken as an example, and firstly, a camera arranged in the elevator is used for collecting an elevator monitoring image. It should be understood that consideration is given to the fact that the amount of information in the monitored image is large, and that the information in the monitored image is largeThe existing vehicles may be bicycles, so that the recognition of the electric vehicles is misjudged, and meanwhile, the boundary of the vehicles in the images is blurred due to the influence of environmental factors, so that the vehicles are difficult to recognize. Therefore, when detecting whether there is an electric vehicle in the elevator, the feature distribution information about the vehicle in the monitoring image should be focused. In consideration of the fact that the remaining useless interference feature information can be filtered out when feature mining is performed on the feature information about the vehicle in the monitoring image, it is obvious that the accuracy of electric vehicle identification detection can be improved. Based on this, in the technical scheme of this application, further pass through vehicle object detection network with the elevator monitoring image in order to obtain the region of interest image. In particular, in one particular example, here the vehicle object detection network is an anchor window based object detection network that is Fast R-CNN, or RetinaNet. In particular, a target anchoring layer and an anchor frame of the target detection network are usedBTo perform sliding to process the monitoring image to frame the region of interest of the vehicle, thereby obtaining the welding region of interest.
Further, when the electric vehicle in the elevator is identified, a person may block the vehicle, so that it is difficult to accurately identify and detect whether the vehicle is the electric vehicle. Therefore, in the technical scheme of the application, a de-occlusion generator based on a countermeasure generation network is used for carrying out de-occlusion processing on the interested area image so as to reduce interference of an obstruction on vehicle identification. Specifically, in the technical solution of the present application, the de-occlusion generator is a de-occlusion generator based on a countermeasure generation network, and it should be understood that the countermeasure generation network includes a generator and a discriminator, wherein the generator is used for generating a de-occlusion vehicle interested area image, and the discriminator is used for calculating a difference between the de-occlusion vehicle interested area image and a real vehicle non-occlusion image, and updating a network parameter of the generator through a gradient descent direction propagation algorithm to obtain a generator with a de-occlusion function, that is, the de-occlusion generator.
And then, performing feature mining on the generated interested region image by using a deep convolutional neural network model which has excellent performance in the aspect of implicit feature extraction of the image and serves as a feature extractor to extract local implicit association feature distribution information about the vehicle in the generated interested region image, so as to obtain an interested region feature map.
Then, considering that convolution is a typical local operation, it can only extract image local features, but cannot focus on global feature information, which affects detection accuracy. However, for the local implicit features of the regions of interest of the monitoring image, the local implicit features of the regions of interest have global correlation features, and the correlation between the local implicit feature distributions of the regions of interest generates a foreground object. Therefore, in the technical scheme of the application, in order to more accurately detect and judge whether the electric vehicle is carried in the elevator, the non-local neural network is used for further extracting the features of the image. That is, the region-of-interest feature map is passed through a non-local neural network model to expand the characteristic receptive field through the non-local neural network model, thereby obtaining an enhanced region-of-interest feature map. Particularly, here, the non-local neural network captures hidden dependency information by calculating the similarity between the local implicit features of the regions of interest, further models context features, enables the network to pay attention to the global overall content between the local implicit features of the regions of interest, and further improves the capability of extracting features of a backbone network in classification and detection tasks.
And further, the corrected enhanced region-of-interest feature map is processed by a classifier to obtain a classification result, and the classification result is used for indicating whether the elevator is loaded with the electric vehicle or not. Namely, the corrected enhanced region-of-interest feature map is used as a classification feature map to obtain a classification result for indicating whether the elevator is loaded with the electric vehicle or not through classification processing in a classifier. Like this, can carry the electric motor car to carrying in the elevator and carry out intellectual detection system to avoid the electric motor car to get into the elevator and cause the potential safety hazard.
In particular, in the technical solution of the present application, when the region-of-interest image is obtained through a de-occlusion generator based on a competing generation network, the generated region-of-interest image may have a generated semantic distribution that is slightly different locally from a natural semantic distribution of the image, so that such semantic distribution difference affects monotonicity of image global-local associated feature semantic expression in a feature matrix of the enhanced region-of-interest feature map after being amplified by a deep convolutional neural network model and a non-local neural network model serving as a feature extractor, thereby affecting accuracy of a classification result of the enhanced region-of-interest feature map.
Therefore, each feature matrix of the enhanced region of interest feature map is preferably subjected to a smooth maximum function approximation modulation, expressed as:
Here, by approximately defining the symbolized distance function with a smooth maximum function along the row and column dimensions of the feature matrix of the enhanced region of interest signature, a relatively good union of the convex optimization of the high-dimensional manifold characterized by the feature matrix of the enhanced region of interest signature in the high-dimensional feature space can be achieved, and by modulating the structured feature distribution of the feature matrix of the enhanced region of interest signature therewith, a natural distribution shift of the intrinsic structure of the feature distribution into the spatial feature variation in the feature space can be obtained, enhancing the convex monotonicity preservation of the feature expression of the high-dimensional manifold of the feature matrix of the enhanced region of interest signature, and thus enhancing the accuracy of the classification result of the enhanced region of interest signature. Like this, can accurately carry out the intelligent monitoring of electric motor car in the elevator and detect to avoid the electric motor car to get into the elevator and cause the potential safety hazard, and then ensured the life safety and the property safety of resident family in the residential building.
Based on this, this application provides an electric vehicle monitoring system for elevator, it includes: the monitoring module is used for acquiring an elevator monitoring image acquired by a camera deployed in the elevator; the vehicle target detection module is used for enabling the elevator monitoring image to pass through a vehicle target detection network so as to obtain an interested area image; a de-occlusion module for passing the region of interest image through a de-occlusion generator based on a challenge generation network to obtain a generated region of interest image; the local feature extraction module is used for enabling the generated interesting area image to pass through a deep convolutional neural network model serving as a feature extractor so as to obtain an interesting area feature map; the global feature extraction module is used for enabling the region-of-interest feature map to pass through a non-local neural network model so as to obtain an enhanced region-of-interest feature map; the characteristic distribution correction module is used for carrying out characteristic distribution correction on each characteristic matrix of the enhanced region-of-interest characteristic diagram along the channel dimension to obtain a corrected enhanced region-of-interest characteristic diagram; and the monitoring result generation module is used for enabling the corrected enhanced region of interest characteristic diagram to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the elevator is loaded with the electric vehicle or not.
Fig. 1 is a schematic view of a scene of an electric vehicle monitoring system for an elevator according to an embodiment of the present application. As shown in fig. 1, in an application scenario of the electric vehicle monitoring system for an elevator, an elevator monitoring image acquired by a camera (e.g., C as illustrated in fig. 1) disposed in an elevator (e.g., E as illustrated in fig. 1) is first acquired. Further, the elevator monitoring image is input into a server (e.g., S as illustrated in fig. 1) in which an elevator electric vehicle monitoring algorithm is deployed, wherein the server can process the elevator monitoring image with the elevator electric vehicle monitoring algorithm to obtain a classification result indicating whether an electric vehicle (e.g., el as illustrated in fig. 1) is carried in the elevator.
Having described the general principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.
Exemplary System
Fig. 2 is a block diagram of an electric vehicle monitoring system for an elevator according to an embodiment of the present application. As shown in fig. 2, an electric vehicle monitoring system 100 for an elevator according to an embodiment of the present application includes: the monitoring module 110 is used for acquiring elevator monitoring images acquired by cameras deployed in the elevator; a vehicle target detection module 120, configured to pass the elevator monitoring image through a vehicle target detection network to obtain an image of a region of interest; a de-occlusion module 130 for passing the region of interest image through a de-occlusion generator based on a challenge-generation network to obtain a generated region of interest image; a local feature extraction module 140, configured to pass the generated region-of-interest image through a deep convolutional neural network model as a feature extractor to obtain a region-of-interest feature map; the global feature extraction module 150 is configured to pass the region-of-interest feature map through a non-local neural network model to obtain an enhanced region-of-interest feature map; a feature distribution correction module 160, configured to perform feature distribution correction on each feature matrix of the enhanced region of interest feature map along the channel dimension to obtain a corrected enhanced region of interest feature map; and a monitoring result generating module 170, configured to pass the corrected enhanced region of interest feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether an electric vehicle is loaded in the elevator.
Fig. 3 is a schematic diagram of the architecture of an electric vehicle monitoring system for an elevator according to an embodiment of the present application. As shown in fig. 3, an elevator monitoring image collected by a camera disposed in an elevator is first acquired. And then, passing the elevator monitoring image through a vehicle target detection network to obtain an interested area image. The region of interest image is then passed through a de-occlusion generator based on a challenge-generation network to obtain a generated region of interest image. And then, passing the generated interested region image through a deep convolutional neural network model serving as a feature extractor to obtain an interested region feature map. And then, passing the characteristic map of the region of interest through a non-local neural network model to obtain an enhanced characteristic map of the region of interest. And then, performing characteristic distribution correction on each characteristic matrix of the characteristic diagram of the enhanced region of interest along the channel dimension to obtain a corrected characteristic diagram of the enhanced region of interest. And then, the corrected enhanced interesting area feature map passes through a classifier to obtain a classification result, and the classification result is used for indicating whether the elevator is loaded with the electric vehicle or not.
In the above-mentioned electric vehicle monitoring system 100 for an elevator, the monitoring module 110 is configured to obtain an elevator monitoring image collected by a camera disposed in the elevator. As the background art mentioned above, with the development of socio-economic, elevators have been fully integrated into our lives as an important means of transportation of people and goods. At present, in a residential building of a residential area, a resident often pushes an electric vehicle to a lift to be transported to a floor for charging, which causes great potential safety hazard, so that the life safety and property safety of the resident in the residential building are affected, however, the existing camera in the lift does not have the function of reminding the resident. Therefore, an elevator motor vehicle monitoring scheme is desired.
Correspondingly, the recognition and detection of the electric vehicle in the elevator can be carried out by collecting the monitoring image through the camera in the elevator, but because the monitoring image has more information, the vehicle in the monitoring image can be a bicycle, and the recognition of the electric vehicle is misjudged, and the boundary of the vehicle in the image becomes fuzzy due to the influence of environmental factors, so that the precise recognition of the vehicle is difficult. And when the camera who deploys in the elevator gathers surveillance image, the people can cause the vehicle to shelter from, and then influences the precision that the electric motor car detected. Meanwhile, because the installation position of the camera is usually high, the acquired image has large distortion, and the detection precision is also influenced. Based on the technical scheme, the method adopts an artificial intelligence monitoring technology based on machine vision, carries out target framing on image information about the electric vehicle in the elevator monitoring image, then carries out shielding removal processing on an interested region of the electric vehicle in the image, and extracts whole content feature distribution information in the processed monitoring image by using an artificial intelligence algorithm based on deep learning so as to detect and judge whether the electric vehicle is carried in the elevator. Like this, can accurately carry out the intelligent monitoring of electric motor car in the elevator and detect to avoid the electric motor car to get into the elevator and cause the potential safety hazard, and then ensured the life safety and the property safety of resident family in the residential building. Specifically, in the technical scheme of this application, first, gather elevator control image through the camera of deployment in the elevator.
In the above-mentioned electric vehicle monitoring system 100 for elevator, the vehicle object detection module 120 is configured to pass the elevator monitoring image through a vehicle object detection network to obtain an image of a region of interest. It should be understood that, considering that the monitoring image has a large amount of information, and the vehicle in the monitoring image may be a bicycle, the recognition of the electric vehicle is misjudged, and considering that the vehicle in the image may have its boundary blurred due to the influence of environmental factors, and thus it is difficult to recognize the vehicle. Therefore, when detecting whether there is an electric vehicle in the elevator, the feature distribution information about the vehicle in the monitoring image should be focused. In consideration of the fact that the remaining useless interference feature information can be filtered when feature mining is performed on the feature information about the vehicle in the monitoring image, it is obvious that the accuracy of electric vehicle identification and detection can be improved. Based on this, in the technical scheme of this application, further pass through vehicle object detection network with the elevator monitoring image in order to obtain the region of interest image.
In particular, in a toolIn the example, the vehicle target detection network is an anchor window-based target detection network, and the anchor window-based target detection network is Fast R-CNN, fast R-CNN or RetinaNet. In particular, a target anchoring layer and an anchor frame of the target detection network are usedBTo perform sliding to process the monitoring image to frame the region of interest of the vehicle, thereby obtaining the welding region of interest.
In the above-mentioned electric vehicle monitoring system 100 for elevator, the de-occlusion module 130 is configured to pass the region-of-interest image through a de-occlusion generator based on a challenge generation network to obtain a generated region-of-interest image. When the electric vehicle in the elevator is identified, a person possibly shields the vehicle, so that whether the vehicle is the electric vehicle or not is difficult to accurately identify and detect. Therefore, in the technical scheme of the application, a de-occlusion generator based on a countermeasures generation network is used for carrying out de-occlusion processing on the interested area image so as to reduce interference of an obstruction on vehicle identification. Specifically, in the technical solution of the present application, the de-occlusion generator is a de-occlusion generator based on a countermeasure generation network, and it should be understood that the countermeasure generation network includes a generator and a discriminator, wherein the generator is used for generating a de-occlusion vehicle interested area image, and the discriminator is used for calculating a difference between the de-occlusion vehicle interested area image and a real vehicle non-occlusion image, and updating a network parameter of the generator through a gradient descent direction propagation algorithm to obtain a generator with a de-occlusion function, that is, the de-occlusion generator.
More specifically, in the embodiment of the present application, the de-occlusion module 130 is further configured to input the region of interest image into the de-occlusion generator based on the countermeasure generation network to output the generated region of interest image by the generator of the de-occlusion generator through deconvolution coding.
In the above-mentioned electric vehicle monitoring system 100 for an elevator, the local feature extraction module 140 is configured to pass the generated region-of-interest image through a deep convolutional neural network model as a feature extractor to obtain a region-of-interest feature map. Namely, feature mining is carried out on the generated interested region image by using a deep convolutional neural network model which has excellent performance in implicit feature extraction of the image and is used as a feature extractor, so that local implicit association feature distribution information about vehicles in the generated interested region image is extracted, and an interested region feature map is obtained.
Specifically, in this embodiment of the present application, the local feature extraction module 140 is further configured to: using each layer of the deep convolutional neural network model to respectively perform in the forward pass of the layer: performing convolution processing on input data to obtain a convolution characteristic diagram; performing mean pooling on the convolution feature map based on a local feature matrix to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the deep convolutional neural network model is the region-of-interest feature map, and the input of the first layer of the deep convolutional neural network model is the generated region-of-interest image.
In the above-mentioned electric vehicle monitoring system 100 for elevator, the global feature extraction module 150 is configured to pass the region-of-interest feature map through a non-local neural network model to obtain an enhanced region-of-interest feature map. It is considered that since convolution is a typical local operation, it can only extract image local features, but cannot focus on global feature information, which affects detection accuracy. However, for the local implicit features of the regions of interest of the monitoring image, the local implicit features of the regions of interest have global correlation features, and the correlation between the local implicit feature distributions of the regions of interest generates a foreground object. Therefore, in the technical scheme of the application, in order to more accurately detect and judge whether the electric vehicle is carried in the elevator, the non-local neural network is used for further extracting the features of the image. That is, the region-of-interest feature map is passed through a non-local neural network model to expand the characteristic receptive field through the non-local neural network model, thereby obtaining an enhanced region-of-interest feature map. Particularly, here, the non-local neural network captures hidden dependency information by calculating the similarity between the local implicit features of the regions of interest, and further models context features, so that the network focuses on the global overall content between the local implicit features of the regions of interest, and further improves the feature extraction capability of the backbone network in classification and detection tasks.
Specifically, in this embodiment of the present application, the global feature extraction module 150 is further configured to: encoding the region-of-interest feature map using the non-local neural network in a manner to obtain the enhanced region-of-interest feature map; wherein the encoding process comprises: respectively carrying out first point convolution processing, second point convolution processing and third point convolution processing on the characteristic diagram of the region of interest to obtain a first characteristic diagram, a second characteristic diagram and a third characteristic diagram; calculating a position-weighted sum between the first feature map and the second feature map to obtain a fused feature map; inputting the fusion feature map into a Softmax function to map feature values of various positions in the fusion feature map into a probability space to obtain a normalized fusion feature map; calculating the position-point-based multiplication between the normalized fusion feature map and the third feature map to obtain a re-fusion feature map; embedding the re-fused feature map into a Gaussian similarity function to obtain a global similar feature map; performing fourth-point convolution processing on the global similar feature map to adjust the channel number of the global similar feature map so as to obtain a channel-adjusted global similar feature map; and calculating a weighted sum according to positions of the channel adjustment global similar feature map and the encrypted local feature map to obtain the enhanced region-of-interest feature map.
In the above-mentioned electric vehicle monitoring system 100 for an elevator, the feature distribution correction module 160 is configured to perform feature distribution correction on each feature matrix along the channel dimension of the enhanced region of interest feature map to obtain a corrected enhanced region of interest feature map. Particularly, in the technical solution of the present application, when the region-of-interest image is obtained through a de-occlusion generator based on a countermeasure generation network, the generated region-of-interest image may have a generated semantic distribution that is slightly different from a natural semantic distribution of the image locally, so that such semantic distribution difference affects monotonicity of image global-local associated feature semantic expression in a feature matrix of the enhanced region-of-interest feature map after being amplified by a deep convolutional neural network model and a non-local neural network model serving as a feature extractor, thereby affecting accuracy of a classification result of the enhanced region-of-interest feature map. Therefore, each feature matrix of the enhanced region of interest feature map is preferably subjected to a smooth maximum function approximation modulation.
Specifically, in this embodiment, the feature distribution correction module 160 is further configured to: performing characteristic distribution correction on each characteristic matrix of the characteristic diagram of the enhanced region of interest along the channel dimension by using the following formula to obtain the corrected characteristic diagram of the enhanced region of interest; wherein the formula is:
wherein
A respective feature matrix along a channel dimension representing the enhanced region of interest feature map,
is the eigenvalue of each position of the diagonal matrix obtained by linear transformation of each characteristic matrix along the channel dimension of the enhanced region-of-interest characteristic diagram,
is a two-norm of a vector, and
meaning that each value of the matrix is multiplied by a predetermined value,
it is shown that the addition by position,
a respective feature matrix along a channel dimension representing the corrected enhanced region of interest feature map.
Here, by approximately defining the symbolized distance function by a smooth maximum function along the row and column dimensions of the feature matrix of the enhanced region of interest feature map, a relatively good union of convex optimization of the high-dimensional manifold characterized by the feature matrix of the enhanced region of interest feature map in the high-dimensional feature space can be achieved, and by modulating the structured feature distribution of the feature matrix of the enhanced region of interest feature map therewith, a natural distribution transfer of the intrinsic structure of the feature distribution into a spatial feature variation in the feature space can be obtained, enhancing the convex monotonicity retention of the feature expression of the high-dimensional manifold of the feature matrix of the enhanced region of interest feature map, thereby enhancing the accuracy of the classification result of the enhanced region of interest feature map.
In the above-mentioned elevator electric vehicle monitoring system 100, the monitoring result generating module 170 is configured to pass the corrected enhanced region of interest feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether an electric vehicle is loaded in the elevator. Namely, the corrected enhanced interested area feature map is used as a classification feature map to obtain a classification result for indicating whether the elevator is loaded with the electric vehicle or not through classification processing in a classifier. Like this, can carry the electric motor car to carrying in the elevator and carry out intellectual detection system to avoid the electric motor car to get into the elevator and cause the potential safety hazard, and then ensured resident family's life safety and property safety in the residential building.
Fig. 4 is a block diagram of a monitoring result generation module in an electric vehicle monitoring system for an elevator according to an embodiment of the present application. As shown in fig. 4, the monitoring result generating module 170 includes: an unfolding unit 171, configured to unfold each corrected enhanced region-of-interest feature matrix in the corrected enhanced region-of-interest feature map into one-dimensional feature vectors according to row vectors or column vectors, and then cascade the one-dimensional feature vectors to obtain classified feature vectors; a full-concatenation encoding unit 172, configured to perform full-concatenation encoding on the classification feature vector using a full-concatenation layer of the classifier to obtain an encoded classification feature vector; and a classification result generating unit 173 for inputting the encoded classification feature vector into the Softmax classification function of the classifier to obtain the classification result.
Further, the classification result comprises a first result and a second result, the first result is that the elevator is loaded with the electric vehicle, and the second result is that the elevator is not loaded with the electric vehicle. The above-mentioned electric vehicle monitoring system for elevator 100 may further include a prompt module, the prompt module is configured to output prompt information according to the above-mentioned classification result, and if the prompt module is an alarm device, the alarm device may be installed in the elevator, and if the classification result indicates that the electric vehicle is carried in the elevator, the prompt module issues a sound prompt or an alarm to prompt a user that the electric vehicle cannot be carried in the elevator.
Further, the above-mentioned electric vehicle monitoring system 100 for elevator may further include a communication module, which may be connected to the control module of the elevator, and the communication module is configured to send a control command to the control module of the elevator according to the above-mentioned classification result, if the classification result indicates that the elevator carries an electric vehicle, the communication module sends a control command to stop the operation to the control module of the elevator, and the control module of the elevator may respond to the control command to stop the operation of the elevator, so as to prevent the user from using the elevator to carry the electric vehicle. Furthermore, the communication module may be further communicatively connected to a remote terminal (e.g., a property terminal), and the communication module may send monitoring information to the remote terminal according to the classification result, for example, if the classification result indicates that the elevator carries an electric car, the communication module sends monitoring information that the elevator carries the electric car to the remote terminal, that is, the monitoring information may include whether the elevator carries the electric car, a monitoring picture, a monitoring video, or other relevant information. Furthermore, the electric vehicle monitoring system 100 for an elevator may further include an inquiry module and a storage module, the storage module is configured to store alarm information, monitoring pictures, video information, and the like, and the inquiry module is configured to call corresponding alarm information, corresponding monitoring pictures, video information, and the like from the storage module in response to an inquiry instruction of a user, so as to facilitate elevator maintenance and other work by related personnel.
That is, the electric vehicle monitoring system 100 for an elevator can monitor whether the electric vehicle is parked in an area where parking is not allowed, such as an elevator, a hall, or the like, and if the electric vehicle is parked illegally, i.e., in the area where parking is not allowed, an alarm device sends out an alarm message such as a voice prompt, or pushes the monitoring information to a relevant unit or person, or the like. Further, the electric vehicle monitoring system 100 for the elevator may also query the alarm information or the monitoring information, and the stored screen shots or monitoring videos corresponding to the alarm information, and query the collected statistical information of multiple illegal parking of the same vehicle, so as to provide evidence for fire-fighting law enforcement.
Further, the above-mentioned electric vehicle monitoring system 100 for elevator may also collect and monitor videos of areas where electric vehicles are not parked, such as an elevator hall, an elevator corridor, or a hallway, and monitor whether electric vehicles are parked in these areas, and if it is monitored that electric vehicles are parked in these areas, prompt information or alarm is given, which is not limited herein.
In summary, the electric vehicle monitoring system 100 for an elevator according to the embodiment of the present application is illustrated, which employs an artificial intelligence monitoring technology based on machine vision, performs target framing on image information about an electric vehicle in an elevator monitoring image, performs occlusion removal processing on an area of interest of the electric vehicle, and extracts overall content feature distribution information in the processed monitoring image by using an artificial intelligence algorithm based on deep learning, so as to perform detection and determination on whether the electric vehicle is carried in the elevator. Therefore, the intelligent monitoring and detection of the electric vehicle in the elevator can be accurately carried out.
Therefore, the electric vehicle monitoring system 100 for elevator can accurately perform intelligent monitoring and detection of electric vehicles in elevator or areas where the electric vehicle is not allowed to park, such as hallways, and the like, so as to avoid potential safety hazards caused by the fact that the electric vehicle enters the elevator or the areas where the electric vehicle is not allowed to park, such as hallways, and the like, thereby preventing the electric vehicle fire and the like in the elevator or the areas where the electric vehicle is not allowed to park, such as hallways, and the like, reducing the occurrence of problems of damages of fire accidents to personnel or articles and the like caused by the fact that the electric vehicle is randomly parked or placed in the elevator, and the like, and further ensuring the life safety and property safety of residents in the residential building.
As described above, the electric vehicle monitoring system 100 for an elevator according to the embodiment of the present application can be implemented in various terminal devices, such as a server for electric vehicle monitoring for an elevator, and the like. In one example, the electric vehicle monitoring system 100 for an elevator according to the embodiment of the present application may be integrated into a terminal device as one software module and/or hardware module. For example, the elevator electric vehicle monitoring system 100 may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the elevator electric vehicle monitoring system 100 can also be one of the hardware modules of the terminal device.
Alternatively, in another example, the elevator electric vehicle monitoring system 100 and the terminal device may be separate devices, and the elevator electric vehicle monitoring system 100 may be connected to the terminal device through a wired and/or wireless network and transmit the interactive information according to an agreed data format.
Exemplary method
Fig. 5 is a flowchart of an electric vehicle monitoring method for an elevator according to an embodiment of the present application. As shown in fig. 5, the method for monitoring a motor vehicle for an elevator according to the embodiment of the present application includes: s110, acquiring an elevator monitoring image acquired by a camera arranged in an elevator; s120, enabling the elevator monitoring image to pass through a vehicle target detection network to obtain an interested area image; s130, enabling the region-of-interest image to pass through a de-occlusion generator based on a countermeasure generation network to obtain a generated region-of-interest image; s140, obtaining a characteristic map of the region of interest by passing the generated region of interest image through a deep convolutional neural network model serving as a characteristic extractor; s150, passing the characteristic diagram of the region of interest through a non-local neural network model to obtain an enhanced characteristic diagram of the region of interest; s160, performing characteristic distribution correction on each characteristic matrix of the enhanced region-of-interest characteristic diagram along the channel dimension to obtain a corrected enhanced region-of-interest characteristic diagram; and S170, passing the corrected enhanced region-of-interest feature map through a classifier to obtain a classification result, wherein the classification result is used for indicating whether an electric vehicle is carried in the elevator or not.
In one example, in the above method for monitoring an electric vehicle for an elevator, the vehicle object detection network is an anchor window-based object detection network, and the anchor window-based object detection network is Fast R-CNN, faster R-CNN, or RetinaNet.
In one example, in the above method for monitoring an electric vehicle for an elevator, the countermeasure generation network includes a generator and a discriminator, wherein the passing the region-of-interest image through a de-occlusion generator based on the countermeasure generation network to obtain a generated region-of-interest image includes: inputting the region of interest image into the countermeasures generation network based de-occlusion generator to output the generated region of interest image by a generator of the de-occlusion generator by deconvolution encoding.
In an example, in the above method for monitoring an electric vehicle for an elevator, the passing the generated image of the region of interest through a deep convolutional neural network model as a feature extractor to obtain a feature map of the region of interest includes: using each layer of the deep convolutional neural network model to respectively perform in the forward pass of the layer: performing convolution processing on input data to obtain a convolution characteristic diagram; performing mean pooling on the convolution feature map based on a local feature matrix to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the deep convolutional neural network model is the region-of-interest feature map, and the input of the first layer of the deep convolutional neural network model is the generated region-of-interest image.
In an example, in the above method for monitoring an electric vehicle for an elevator, the passing the region-of-interest feature map through a non-local neural network model to obtain an enhanced region-of-interest feature map includes: encoding the region-of-interest feature map using the non-local neural network in a manner to obtain the enhanced region-of-interest feature map; wherein the encoding process comprises: respectively carrying out first point convolution processing, second point convolution processing and third point convolution processing on the characteristic diagram of the region of interest to obtain a first characteristic diagram, a second characteristic diagram and a third characteristic diagram; calculating a position-weighted sum between the first feature map and the second feature map to obtain a fused feature map; inputting the fused feature map into a Softmax function so as to map feature values of all positions in the fused feature map into a probability space to obtain a normalized fused feature map; calculating the position-point-based multiplication between the normalized fusion feature map and the third feature map to obtain a re-fusion feature map; embedding a Gaussian similarity function into the re-fused feature map to obtain a global similar feature map; performing fourth point convolution processing on the global similar feature map to adjust the channel number of the global similar feature map so as to obtain a channel adjustment global similar feature map; and calculating a weighted sum according to positions of the channel adjustment global similar feature map and the encrypted local feature map to obtain the enhanced region-of-interest feature map.
In one example, in the above method for monitoring an electric vehicle for an elevator, the performing feature distribution correction on each feature matrix along a channel dimension of the enhanced region of interest feature map to obtain a corrected enhanced region of interest feature map includes: performing characteristic distribution correction on each characteristic matrix of the characteristic diagram of the enhanced region of interest along the channel dimension by using the following formula to obtain the corrected characteristic diagram of the enhanced region of interest; wherein the formula is:
wherein
Characterizing the enhanced region of interestOf the respective feature matrices along the channel dimension,
is the eigenvalue of each position of the diagonal matrix obtained by linear transformation of each characteristic matrix along the channel dimension of the characteristic diagram of the enhanced region of interest,
is a two-norm of a vector, an
Meaning that each value of the matrix is multiplied by a predetermined value,
it is shown that the addition by position,
respective feature matrices along a channel dimension representing the corrected enhanced region of interest feature map.
In an example, in the above method for monitoring an electric vehicle for an elevator, the step of passing the corrected enhanced region-of-interest feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether the elevator carries an electric vehicle or not, includes: expanding each corrected enhanced region-of-interest feature matrix in the corrected enhanced region-of-interest feature map into one-dimensional feature vectors according to row vectors or column vectors, and then cascading to obtain classified feature vectors; performing full-concatenation coding on the classification feature vectors by using a full-concatenation layer of the classifier to obtain coded classification feature vectors; and inputting the encoding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
In summary, the method for monitoring the electric vehicle for the elevator is clarified, and the method adopts an artificial intelligence monitoring technology based on machine vision to frame the target of the image information about the electric vehicle in the elevator monitoring image, then carries out the shielding removal processing of the interested region of the electric vehicle, and uses an artificial intelligence algorithm based on deep learning to extract the whole content feature distribution information in the processed monitoring image, thereby carrying out the detection judgment whether the electric vehicle is carried in the elevator. Therefore, the intelligent monitoring and detection of the electric vehicle in the elevator can be accurately carried out.
Exemplary electronic device
Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 6. Fig. 6 is a block diagram of an electronic device according to an embodiment of the application.
As shown in fig. 6, the electronic device 10 includes one or more processors 11 and memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input device 13 may include, for example, a keyboard, a mouse, and the like.
The output device 14 can output various information including the classification result to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 6, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the functions in the method for electric vehicle monitoring for elevators according to the various embodiments of the present application described in the "exemplary methods" section of this specification above.
The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in functions in an electric vehicle monitoring method for elevators according to various embodiments of the present application described in the "exemplary methods" section above in this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The basic principles of the present application have been described above with reference to specific embodiments, but it should be noted that advantages, effects, etc. mentioned in the present application are only examples and are not limiting, and the advantages, effects, etc. must not be considered to be possessed by various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by one skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably herein. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (10)
1. An electric vehicle monitoring system for an elevator, comprising:
the monitoring module is used for acquiring an elevator monitoring image acquired by a camera arranged in the elevator;
the vehicle target detection module is used for enabling the elevator monitoring image to pass through a vehicle target detection network so as to obtain an interested area image;
a de-occlusion module for passing the region of interest image through a de-occlusion generator based on a challenge generation network to obtain a generated region of interest image;
the local feature extraction module is used for enabling the generated interesting area image to pass through a deep convolutional neural network model serving as a feature extractor so as to obtain an interesting area feature map;
the global feature extraction module is used for enabling the region-of-interest feature map to pass through a non-local neural network model so as to obtain an enhanced region-of-interest feature map;
the characteristic distribution correction module is used for carrying out characteristic distribution correction on each characteristic matrix of the enhanced region-of-interest characteristic diagram along the channel dimension to obtain a corrected enhanced region-of-interest characteristic diagram; and
and the monitoring result generating module is used for enabling the corrected enhanced interesting area characteristic diagram to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether an electric car is carried in the elevator or not.
2. The electric vehicle monitoring system for elevator according to claim 1, wherein the vehicle target detection network is an anchor window-based target detection network, and the anchor window-based target detection network is Fast R-CNN, or RetinaNet.
3. The elevator motor vehicle monitoring system according to claim 2, wherein the countermeasure generation network comprises a generator and a discriminator, wherein the de-occlusion module is further configured to input the region of interest image into the countermeasure generation network based de-occlusion generator to output the generated region of interest image by the de-occlusion generator through deconvolution encoding.
4. The elevator motor vehicle monitoring system of claim 3, wherein the local feature extraction module is further configured to:
using the layers of the deep convolutional neural network model to respectively perform in the forward pass of the layers:
performing convolution processing on input data to obtain a convolution characteristic diagram;
performing mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map; and
performing nonlinear activation on the pooled feature map to obtain an activated feature map;
and outputting the final layer of the deep convolutional neural network model as the region-of-interest feature map, and inputting the first layer of the deep convolutional neural network model as the generated region-of-interest image.
5. The elevator electric vehicle monitoring system of claim 4, wherein the global feature extraction module is further configured to:
encoding the region-of-interest feature map using the non-local neural network in a manner to obtain the enhanced region-of-interest feature map;
wherein the encoding process comprises:
respectively carrying out first point convolution processing, second point convolution processing and third point convolution processing on the characteristic diagram of the region of interest to obtain a first characteristic diagram, a second characteristic diagram and a third characteristic diagram;
calculating a position-weighted sum between the first feature map and the second feature map to obtain a fused feature map;
inputting the fused feature map into a Softmax function so as to map feature values of all positions in the fused feature map into a probability space to obtain a normalized fused feature map;
calculating the position-based multiplication between the normalized fusion feature map and the third feature map to obtain a re-fusion feature map;
embedding a Gaussian similarity function into the re-fused feature map to obtain a global similar feature map;
performing fourth-point convolution processing on the global similar feature map to adjust the channel number of the global similar feature map so as to obtain a channel-adjusted global similar feature map; and
and calculating the weighted sum of the channel adjustment global similar feature map and the encrypted local feature map according to the position to obtain the enhanced region-of-interest feature map.
6. The elevator motor vehicle monitoring system of claim 5, wherein the signature distribution correction module is further configured to: performing characteristic distribution correction on each characteristic matrix of the characteristic diagram of the enhanced region of interest along the channel dimension by using the following formula to obtain the corrected characteristic diagram of the enhanced region of interest;
wherein the formula is:
wherein
A respective feature matrix along a channel dimension representing the enhanced region of interest feature map,
is the eigenvalue of each position of the diagonal matrix obtained by linear transformation of each characteristic matrix along the channel dimension of the enhanced region-of-interest characteristic diagram,
is a two-norm of a vector, an
Meaning that each value of the matrix is multiplied by a predetermined value,
it is shown that the addition by position,
a respective feature matrix along a channel dimension representing the corrected enhanced region of interest feature map.
7. The electric vehicle monitoring system for elevator according to claim 6, wherein the monitoring result generating module comprises:
the expansion unit is used for expanding each corrected enhanced region-of-interest feature matrix in the corrected enhanced region-of-interest feature map into one-dimensional feature vectors according to row vectors or column vectors and then cascading the one-dimensional feature vectors to obtain classified feature vectors;
a full-concatenation encoding unit, configured to perform full-concatenation encoding on the classification feature vector using a full-concatenation layer of the classifier to obtain an encoded classification feature vector; and
a classification result generating unit, configured to input the encoded classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
8. An electric vehicle monitoring method for an elevator is characterized by comprising the following steps:
acquiring an elevator monitoring image acquired by a camera deployed in an elevator;
passing the elevator monitoring image through a vehicle target detection network to obtain an interested area image;
passing the region of interest image through a de-occlusion generator based on a challenge-generation network to obtain a generated region of interest image;
passing the generated region-of-interest image through a depth convolution neural network model serving as a feature extractor to obtain a region-of-interest feature map;
passing the region-of-interest feature map through a non-local neural network model to obtain an enhanced region-of-interest feature map;
carrying out characteristic distribution correction on each characteristic matrix of the characteristic diagram of the enhanced region of interest along the channel dimension to obtain a corrected enhanced region of interest characteristic diagram; and
and passing the corrected enhanced region-of-interest characteristic diagram through a classifier to obtain a classification result, wherein the classification result is used for indicating whether an electric car is carried in the elevator or not.
9. The method for monitoring electric vehicles for elevators according to claim 8, wherein the vehicle target detection network is an anchor window-based target detection network, and the anchor window-based target detection network is Fast R-CNN, or RetinaNet.
10. The method for monitoring electric vehicles for elevator according to claim 9, wherein the countermeasure generation network comprises a generator and a discriminator, wherein the passing the region of interest image through a de-occlusion generator based on the countermeasure generation network to obtain a generated region of interest image comprises: inputting the region-of-interest image into the resist generation network-based de-occlusion generator to output the generated region-of-interest image by de-convolution encoding by the de-occlusion generator.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211497090.0A CN115783919A (en) | 2022-11-28 | 2022-11-28 | Electric vehicle monitoring system and method for elevator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211497090.0A CN115783919A (en) | 2022-11-28 | 2022-11-28 | Electric vehicle monitoring system and method for elevator |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115783919A true CN115783919A (en) | 2023-03-14 |
Family
ID=85441949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211497090.0A Pending CN115783919A (en) | 2022-11-28 | 2022-11-28 | Electric vehicle monitoring system and method for elevator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115783919A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116652988A (en) * | 2023-07-28 | 2023-08-29 | 江苏泽宇智能电力股份有限公司 | Intelligent optical fiber wiring robot and control method thereof |
| CN116933041A (en) * | 2023-09-14 | 2023-10-24 | 深圳市力准传感技术有限公司 | Force sensor number checking system and method |
-
2022
- 2022-11-28 CN CN202211497090.0A patent/CN115783919A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116652988A (en) * | 2023-07-28 | 2023-08-29 | 江苏泽宇智能电力股份有限公司 | Intelligent optical fiber wiring robot and control method thereof |
| CN116652988B (en) * | 2023-07-28 | 2023-10-27 | 江苏泽宇智能电力股份有限公司 | Intelligent optical fiber wiring robot and control method thereof |
| CN116933041A (en) * | 2023-09-14 | 2023-10-24 | 深圳市力准传感技术有限公司 | Force sensor number checking system and method |
| CN116933041B (en) * | 2023-09-14 | 2024-05-03 | 深圳市力准传感技术有限公司 | Force sensor number checking system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115783919A (en) | Electric vehicle monitoring system and method for elevator | |
| Vu et al. | Automatic video interpretation: A novel algorithm for temporal scenario recognition | |
| KR101724658B1 (en) | Human detecting apparatus and method | |
| KR102217253B1 (en) | Apparatus and method for analyzing behavior pattern | |
| CN107122743B (en) | Security monitoring method and device and electronic equipment | |
| KR102256802B1 (en) | Unmanned parking illegal tailing management system and method using reflector camera | |
| CN115909260A (en) | Method and system for early warning of workplace intrusion based on machine vision | |
| CN111931862A (en) | Method and system for detecting illegal posted advertisements and electronic equipment | |
| KR101879444B1 (en) | Method and apparatus for operating CCTV(closed circuit television) | |
| KR102090907B1 (en) | vehicle detection method and number cognition method using image enhancement and deep learning, and park systems using the method | |
| KR102346204B1 (en) | Method, device and system for processing cctv image installed in public place based on artificial intelligence | |
| CN117523437A (en) | Real-time risk identification method for substation near-electricity operation site | |
| CN116977129A (en) | Intelligent property management system and method thereof | |
| KR20180093418A (en) | Apparatus and method for detecting pedestrian | |
| KR20210023273A (en) | Apparatus and method for judging tunnel situation | |
| KR102339841B1 (en) | Method and system for License plate recognition corresponding to tailgating in parking control | |
| JP2005284652A (en) | Video monitoring method and apparatus using moving vector | |
| CN114283360A (en) | Method, apparatus, device and medium for detecting electric vehicle in elevator | |
| KR101669885B1 (en) | method of detecting pedestrian collision and IoT apparatus thereof | |
| CN114359839A (en) | Method and system for identifying entrance of electric vehicle into elevator | |
| KR20220121961A (en) | Parking management method and apparatus based on cctv | |
| Ghazal et al. | Real-time vandalism detection by monitoring object activities | |
| CN112580753A (en) | Emergency forced landing position determining method based on texture feature mask | |
| CN117456610B (en) | Climbing abnormal behavior detection method and system and electronic equipment | |
| KR102286675B1 (en) | Method for implementing security system based on motion recognition and security system using the method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20230314 |