CN113283466A - Instrument reading identification method and device and readable storage medium - Google Patents
Instrument reading identification method and device and readable storage medium Download PDFInfo
- Publication number
- CN113283466A CN113283466A CN202110389066.4A CN202110389066A CN113283466A CN 113283466 A CN113283466 A CN 113283466A CN 202110389066 A CN202110389066 A CN 202110389066A CN 113283466 A CN113283466 A CN 113283466A
- Authority
- CN
- China
- Prior art keywords
- instrument
- meter
- image
- meter reading
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000013145 classification model Methods 0.000 claims abstract description 21
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 238000013527 convolutional neural network Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 description 12
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000001965 increasing effect Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000012805 post-processing Methods 0.000 description 3
- 238000005034 decoration Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000036544 posture Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a device for identifying meter reading and a readable storage medium, comprising the following steps: acquiring an instrument image of a reading to be identified; performing image preprocessing on the instrument image by adopting a histogram equalization method; performing instrument type identification on the preprocessed instrument image by adopting an instrument classification model; and acquiring meter reading from the preprocessed meter image according to the obtained meter type. The invention can solve the problems of high requirement on the image quality of the instrument and poor generalization capability of the existing instrument reading identification method, and improves the accuracy of the instrument reading identification.
Description
Technical Field
The invention relates to the field of digital image processing, in particular to a method and a device for recognizing meter reading and a readable storage medium.
Background
Industrial instruments have been widely used in various fields, particularly in power grids, to monitor pressure, electric power, temperature and humidity, etc., in countless numbers. At present, a manual inspection mode is generally adopted in a power grid, and inspection is performed approximately once a week to check whether the meter reading of each device is normal. However, the method is time-consuming and labor-consuming, and is not timely, and if some significant problems occur, manual inspection is not timely in response, so that the intelligent identification of the industrial instrument in the power grid is very important.
At present, the automatic identification of the reading of the pointer instrument mainly adopts a method based on template matching. The method uses a feature matching algorithm to register an image to be recognized to a standard image posture, and then identifies a pointer reading. The method has poor generalization capability, high requirements on picture quality, sensitivity to shooting environments such as illumination and the like and image noise, and is difficult to identify due to the fact that instrument images are shot in different illumination, postures and scales, so that the requirements of practical application are difficult to meet.
Disclosure of Invention
The invention aims to provide a meter reading identification method, a device and a readable storage medium, which are used for solving the problems of high requirement on the image quality of a meter and poor generalization capability of the existing meter reading identification method.
The technical scheme provided by the invention is as follows:
a meter reading identification method comprising: acquiring an instrument image of a reading to be identified; performing image preprocessing on the instrument image by adopting a histogram equalization method; performing instrument type identification on the preprocessed instrument image by adopting an instrument classification model; and acquiring meter reading from the preprocessed meter image according to the obtained meter type.
Further, the image preprocessing of the meter image by using a histogram equalization method includes: acquiring gray level histogram information of the instrument image; and adjusting the gray value of the instrument image by adopting a histogram equalization method according to the gray histogram information to ensure that the distribution of the gray value of the adjusted instrument image is approximately uniform.
Further, the instrument classification model adopts a convolutional neural network based on hole convolution.
Further, the convolutional neural network based on the hole convolution comprises: the basic network uses an Xreception network; replacing a convolution module in the Xception network with a hole convolution; the output layer of the Xception network uses a convolutional layer of 1 × 1,256 channels instead of a fully connected layer.
Further, said obtaining a meter reading from said pre-processed meter image based on said obtained meter type comprises: extracting a meter area from the preprocessed meter image; taking meter readings from the meter area according to the obtained meter type.
Further, said taking meter readings from said meter area based on said obtained meter type comprises: selecting a corresponding key point positioning model according to the obtained instrument type; inputting the instrument area into the key point positioning model to obtain a starting point coordinate and an end point coordinate of a pointer in the instrument, a dial center coordinate and a dial starting coordinate; and obtaining the meter reading according to the starting point coordinate and the end point coordinate of the pointer, the dial center coordinate and the dial starting coordinate.
Furthermore, the key point positioning model is based on a ShuffleNet network, and a shallow feature, a middle feature and a deep feature are respectively extracted from the ShuffleNet network and accumulated on the final output layer.
Further, the steps of extracting a shallow feature, a middle feature and a deep feature from the ShuffleNet network respectively and accumulating the shallow feature, the middle feature and the deep feature to a final output layer include: and respectively extracting a branch from the 5 th layer, the 8 th layer and the 13 th layer of the Shufflenet network, accumulating, and outputting through an embedded layer and a convolution layer of 1x1x 6.
The invention also provides a device for recognizing the reading of the instrument, which comprises: the image acquisition module is used for acquiring an instrument image of a reading to be identified; the image preprocessing module is used for preprocessing the instrument image by adopting a histogram equalization method; the instrument classification module is used for identifying the instrument type of the preprocessed instrument image by adopting an instrument classification model; and the instrument reading identification module is used for acquiring the instrument reading from the preprocessed instrument image according to the obtained instrument type.
The invention also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the meter reading identification method as set forth above.
The meter reading identification method, the meter reading identification device and the readable storage medium provided by the invention can at least bring the following beneficial effects:
1. according to the invention, the pretreatment of the instrument image is carried out by adopting histogram equalization, so that the quality of the instrument image is improved; by using the deep learning algorithm, the generalization capability of the model and the accuracy of meter reading identification are improved.
2. According to the invention, the typical network and the cavity convolution are combined, so that the receptive field of the network characteristic layer is increased, and the accuracy of the instrument classification model is improved.
3. According to the invention, shallow, medium and deep feature layers are extracted, the features are fused, and finally regression of the key points is carried out, so that the accuracy of key point positioning is improved.
Drawings
The above features, technical features, advantages and implementations of a method and apparatus for identifying a meter reading, a readable storage medium, and a computer program product will be further described in the following detailed description of preferred embodiments in a clearly understandable manner, in conjunction with the accompanying drawings.
FIG. 1 is a flow chart of one embodiment of a meter reading identification method of the present invention;
FIG. 2 is a schematic diagram of an embodiment of a meter reading identification apparatus of the present invention;
FIG. 3 is a schematic diagram showing a comparison between an original image of a meter image and an equalized image;
FIG. 4 is a schematic diagram of a network structure of a key point location model;
FIG. 5 is a flow chart illustrating the application of the meter reading identification method of the present invention to an embodiment;
FIG. 6 is a diagram of a generic convolution kernel and a hole convolution kernel.
The reference numbers illustrate:
100. the system comprises an image acquisition module, a 200 image preprocessing module, a 300 meter reading identification module, a 400 meter reading identification module, a 410 meter area extraction unit, a 420 key point positioning unit and a 430 meter reading calculation unit.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will be made with reference to the accompanying drawings. It is obvious that the drawings in the following description are only some examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.
For the sake of simplicity, the drawings only schematically show the parts relevant to the present invention, and they do not represent the actual structure as a product. In addition, in order to make the drawings concise and understandable, components having the same structure or function in some of the drawings are only schematically depicted, or only one of them is labeled. In this document, "one" means not only "only one" but also a case of "more than one".
In an embodiment of the present invention, as shown in fig. 1, a meter reading identification method includes:
step S100 acquires a meter image of a reading to be recognized.
Step S200 performs image preprocessing on the meter image by using a histogram equalization method.
The gray value distribution of the instrument image is adjusted by adopting a histogram equalization method, and the brightness and darkness of the whole image are averaged, so that the quality of the image obtained under strong light and dark light can be improved, and the image identification accuracy of a subsequent model can be improved.
Step S200 specifically includes:
acquiring gray level histogram information of an instrument image;
and adjusting the gray value of the instrument image by adopting a histogram equalization method according to the gray histogram information to ensure that the distribution of the gray value of the adjusted instrument image is approximately uniform.
Generally, the components of the histogram of the dark image are concentrated on the end where the gradation is lower, while the components of the histogram of the bright image are biased toward the end where the gradation is higher. If the gray value distribution of an image is approximately uniform, the image has a larger gray dynamic range and higher contrast, and the details of the image are richer. The gray value of the image can be adjusted by adopting a histogram equalization method, so that the gray value distribution of the equalized image is approximately uniformly distributed, and the contrast of the image is effectively enhanced.
And step S300, performing instrument type identification on the preprocessed instrument image by adopting an instrument classification model.
Preferably, the instrument classification model employs a convolutional neural network based on a hole convolution.
As shown in fig. 6, (a) is a 3 × 3 ordinary convolution kernel, and (b) is a 5 × 5 hollow convolution kernel, which is partially filled with 0. It can be seen that the size of the hole convolution kernel becomes larger, but in practice only 9 points are weighted other than 0, as in the 3 x 3 normal convolution kernel.
Therefore, under the condition of not increasing the calculation amount, the visual field of the convolution kernel is enlarged through the hole convolution (namely, the size of the convolution kernel is enlarged), each convolution output contains information in a large range, and the accuracy of the classification of the convolution neural network is improved.
Fig. 6 (b) is only an example of a convolution kernel in which the hole interval is 1, and the hole interval may have other values, and an appropriate hole interval may be selected as needed.
One specific implementation of the convolutional neural network based on the hole convolution is as follows:
the basic network uses an Xreception network; replacing a convolution module in the Xception network with a hole convolution; the output layer of the Xception network uses a 1x1x256 convolutional layer instead of a fully-connected layer.
The cavity convolution replaces a common convolution module, so that the receptive field can be greatly improved, and the accuracy of network classification is improved.
Fully connected layer tiling results in adjacent information being far apart, with 1x1 convolutional layers being computationally less intensive, faster, and more accurate. Therefore, the accuracy of network classification can be further improved by replacing the fully-connected layer with the convolutional layer of 1x1x 256.
Of course, the underlying network may also adopt other classical convolutional neural network models, such as VGG, *** net, ResNet, etc., and the convolutional module in the classical network is replaced by the hole convolution.
Step S400 obtains a meter reading from the pre-processed meter image according to the obtained meter type.
Step S400 specifically includes:
step S410 extracts a meter region from the preprocessed meter image.
The meter regions can be extracted from the pre-processed meter images using existing target detection models, such as the lightweight networks MobileNetv1, MobileNetv 2.
Step S420 obtains meter readings from the meter area according to the obtained meter type.
Optionally, step S420 includes:
step S421 selects a corresponding key point location model according to the obtained instrument type.
Optionally, the key point location model is based on a shuffle network, and a shallow feature, a middle feature and a deep feature are respectively extracted from the shuffle network and accumulated on the final output layer. Alternatively, a branch is taken from each of layers 5, 8 and 13 of the ShuffleNet network, accumulated, and output via the embedded layer and a 1x1x6 convolutional layer.
Step S422, the instrument area is input into the key point positioning model, and the starting point coordinate and the end point coordinate of the pointer in the instrument, the dial center coordinate and the dial starting coordinate are obtained.
Step S423 obtains the meter reading from the start point coordinate and the end point coordinate of the pointer, the dial center coordinate and the dial start coordinate.
Obtaining a straight line where the current pointer is located according to the starting point coordinate and the end point coordinate of the pointer, obtaining a dial starting line according to the dial center coordinate and the dial starting coordinate, calculating an included angle between the straight line where the current pointer is located and the dial starting line, and obtaining the current meter reading according to the included angle and the perValue (the meter reading represented by each degree) of the meter.
In the embodiment, the quality of the instrument image is improved by adopting histogram equalization to preprocess the instrument image; by using a deep learning algorithm, the generalization capability of the model and the accuracy of meter reading identification are improved; by combining the typical network and the cavity convolution, the receptive field of a network characteristic layer is increased, and the accuracy of an instrument classification model is improved; by extracting shallow, medium and deep feature layers, the features are fused, and finally regression of key points is carried out, so that the accuracy of key point positioning is improved.
In one embodiment of the present invention, as shown in fig. 2, a meter reading recognition apparatus includes:
and the image acquisition module 100 is used for acquiring a meter image of the reading to be identified.
And the image preprocessing module 200 is configured to perform image preprocessing on the instrument image by using a histogram equalization method.
The gray value distribution of the instrument image can be adjusted by adopting a histogram equalization method, the brightness and darkness of the whole image are averaged, the quality of the image obtained under strong light and dark light can be improved, and the image identification accuracy of a subsequent model can be improved.
The image preprocessing module 200 is further configured to obtain gray level histogram information of the instrument image; and adjusting the gray value of the instrument image by adopting a histogram equalization method according to the gray histogram information to ensure that the distribution of the gray value of the adjusted instrument image is approximately uniform.
And the instrument classification module 300 is configured to perform instrument type identification on the preprocessed instrument image by using an instrument classification model.
Preferably, the instrument classification model employs a convolutional neural network based on a hole convolution.
One specific implementation of the convolutional neural network based on the hole convolution is as follows:
the basic network uses an Xreception network; replacing a convolution module in the Xception network with a hole convolution; the output layer of the Xception network uses a 1x1x256 convolutional layer instead of a fully-connected layer.
The cavity convolution replaces a common convolution module, so that the receptive field can be greatly improved, and the accuracy of network classification is improved.
Fully connected layer tiling results in adjacent information being far apart, with 1x1 convolutional layers being computationally less intensive, faster, and more accurate. Therefore, the accuracy of network classification can be further improved by replacing the fully-connected layer with the convolutional layer of 1x1x 256.
In other embodiments, the underlying network may also adopt other classical convolutional neural network models, such as VGG, GoogleNet, ResNet, etc., and replace the convolutional module in the above classical network with a hole convolution.
And a meter reading identification module 400, configured to obtain a meter reading from the preprocessed meter image according to the obtained meter type.
Optionally, the meter reading identification module 400 includes:
a meter region extraction unit 410 for extracting a meter region from the preprocessed meter image;
the meter reading identification module 400 is further configured to obtain a meter reading from the meter area according to the obtained meter type.
Optionally, the meter reading identification module 400 further comprises a key point positioning unit 420 and a meter reading calculation unit 430.
A key point positioning unit 420, configured to select a corresponding key point positioning model according to the obtained instrument type; and inputting the instrument area into the key point positioning model to obtain the starting point coordinate and the end point coordinate of the pointer in the instrument, the central coordinate of the dial plate and the starting coordinate of the dial plate.
And the meter reading calculating unit 430 is used for obtaining the meter reading according to the start point coordinate and the end point coordinate of the pointer, the dial center coordinate and the dial start coordinate.
Optionally, the key point location model is based on a shuffle network, and a shallow feature, a middle feature and a deep feature are respectively extracted from the shuffle network and accumulated on the final output layer. Preferably, a branch is taken from each of layers 5, 8 and 13 of the ShuffleNet network, accumulated, and output via the embedded layer and a 1x1x6 convolutional layer.
Obtaining a straight line where the current pointer is located according to the starting point coordinate and the end point coordinate of the pointer, obtaining a dial starting line according to the dial center coordinate and the dial starting coordinate, calculating an included angle between the straight line where the current pointer is located and the dial starting line, and obtaining the current meter reading according to the included angle and the perValue (the meter reading represented by each degree) of the meter.
In the embodiment, the quality of the instrument image is improved by adopting histogram equalization to preprocess the instrument image; by using a deep learning algorithm, the generalization capability of the model and the accuracy of meter reading identification are improved; by combining the typical network and the cavity convolution, the receptive field of a network characteristic layer is increased, and the accuracy of an instrument classification model is improved; by extracting shallow, medium and deep feature layers, the features are fused, and finally regression of key points is carried out, so that the accuracy of key point positioning is improved.
The embodiment of the meter reading recognition device provided by the invention and the embodiment of the meter reading recognition method provided by the invention are based on the same inventive concept, and the same technical effects can be obtained. Accordingly, other specific contents of the embodiment of the meter reading identification device can refer to the description of the embodiment of the meter reading identification method.
In an embodiment of the present invention, a computer-readable storage medium has stored thereon a computer program which, when executed by a processor, can implement the meter reading identification method as recited in the preceding embodiments. That is, when part or all of the technical solutions of the embodiments of the present invention contributing to the prior art are embodied by means of a computer software product, the computer software product is stored in a computer-readable storage medium. The computer readable storage medium can be any portable computer program code entity apparatus or device. For example, the computer readable storage medium may be a U disk, a removable magnetic disk, a magnetic diskette, an optical disk, a computer memory, a read-only memory, a random access memory, etc.
The invention also provides a specific implementation scenario example, and the method and the device for recognizing the reading of the meter, which are provided by the application, are applied to the reading recognition of the industrial meter, and specifically comprise the following steps:
the system comprises an image preprocessing module, a meter classifying module, a key area positioning module (namely a meter area extracting unit 410), a key point positioning module (namely a key point positioning unit 420), a post-processing module (namely a meter reading calculating unit 430) and a model training module.
An image preprocessing module: the method is used for averaging the brightness of the instrument image. The contrast of the instrument image obtained under strong light or dark light is improved by adjusting the brightness to a uniform level using a histogram equalization method. As shown in fig. 3, the original image is on the left, and the instrument image is too dark, which is likely to cause errors in subsequent instrument classification, key region positioning, and key point positioning; and the right side is the instrument picture after the histogram equalization, the brightness of the picture is improved, and the subsequent analysis is favorably utilized.
An instrument classification module: for identifying the kind of the meter through the meter classification model. The industrial meters are various, and different meters are different greatly, such as a pointer pattern and a dial pattern. The instrument classification module is trained by adopting a network based on cavity convolution, the cavity convolution can greatly improve the receptive field and see more information, and therefore the accuracy of network classification is improved. The difference between the hole convolution and the ordinary convolution is that most of convolution kernels are 0 values, so that the whole network is sparse, and the accuracy is improved.
A key area positioning module: and the dial area is obtained through the key area positioning model. Because in a photo, except the dial plate there is a large amount of background interference, after positioning the dial plate position, take out the dial plate picture, will only send the picture of the dial plate region to the subsequent key point positioning module, can promote the accuracy rate that the key point positions greatly.
The key point positioning module: and the key points are used for acquiring the dial picture through the key point positioning model. Due to the fact that the different instruments have large style difference, the key point positioning models are different according to different instrument types in order to improve the accuracy of instrument reading.
Taking pointer reading identification as an example, it is most critical to acquire the position and angle of the current pointer. The position and angle of the current pointer are acquired by the following positions of key points: the starting point and the end point of the pointer, the central point of the dial plate and the starting point of the dial plate are provided with the points, so that two straight lines (one is the straight line where the current pointer is located, and the other is the starting line of the dial plate) can be obtained, and the included angle of the two straight lines is calculated.
A post-processing module: for calculating the current meter reading. Taking the identification of the pointer as an example, the angle of the pointer can be calculated by taking the starting point of the pointer, the central point of the dial and the starting point of the dial. With the pointer angle, knowing the reading represented by each degree a priori, the reading can be obtained by the following formula:
Result＝currentAngle*perValue；
where currentAngle is the angle of the current pointer and perValue represents the meter reading represented by each degree.
And the model training module is used for finishing the training of the various models. The model training is divided into three stages:
the first stage completes the training of an instrument classification model:
different instrument images are put into different folders, the instruments are of various types, each type is put into one folder, and a corresponding instrument type label is marked on each type of instrument image. The instrument classification model adopts supervised learning.
The network of the instrument classification model is based on an Xception network, and all convolution modules in the Xception are replaced by hollow convolution. So far, the preparation work is finished, the learning rate is 0.001 during training, the final output layer uses 1x1x256 convolution instead of the traditional full connection layer, and the accuracy rate of model classification can be improved.
And the second stage is to complete the training of the key area positioning model:
existing object detection models can be used for critical area localization. Because the dial plate positioning task is relatively simple, the light-weight network MobileNetv2 is selected as the backbone, the model is small, the occupation of the model on resources can be reduced, and the running efficiency of the model is improved.
And the third stage completes the training of the key point positioning model:
on the basis of the ShuffleNet network, a branch is respectively extracted from the 5 th layer, the 8 th layer and the 13 th layer in the network and is accumulated on the final output layer, so that the accuracy rate of key point positioning is improved. The 5 th layer outputs shallow features, the 8 th layer outputs middle features, and the 13 th layer outputs deep features. Shallow, medium and deep features are extracted, the features are fused, and finally regression of key points is carried out, so that the accuracy of the key point positioning model is improved.
The whole training is divided into the following steps:
a. and (4) preparing data. The location of the labeled keypoints is different for different tables. Therefore, the instrument key point positioning model of the corresponding category needs to be called according to the identification result of the instrument type. For example, some meters have only 3 key points and need to mark 3 points, because the circle center coincides with the pointer end point, only three points need to be regressed during regression; some meters have 4 key points, and 4 points, two points on the pointer, the circle center and the starting point need to be marked.
b. And (5) enhancing the image. In order to increase the generalization capability of the model and the accuracy of the positioning of the key points, operations such as brightness, darkness, saturation, random cropping, stretching, rotation, affine transformation, perspective transformation and the like are required to perform data enhancement on the original image so as to obtain more data samples and improve the accuracy of the model.
c. And (4) preparing a network. With the improved shuffle net, the network structure is as shown in fig. 4, outputting corresponding features from the 5 th layer, the 8 th layer and the 13 th layer respectively, fusing the three features together through an Embedding layer (Embedding), and finally following a convolution of 1x1x 6.
d. And (5) training. During training, the learning rate is reduced in stages, each stage is reduced by 0.1, the stages are [10,50,100,120,130], the initial learning rate is 0.1, and the final learning rate is minus 6 th power of 10.
After training the instrument classification model, the key area positioning model and the key point positioning model, the whole calling process is as shown in fig. 5, and the steps are as follows:
step S1 captures an RGB meter picture 1 from the camera.
And step S2, preprocessing the picture to obtain an equalized picture 2.
Step S3 sends the picture 2 to the instrument classification model, and determines which type the instrument belongs to, to obtain the instrument type.
Step S4 sends the picture 2 to the key area location model to obtain the dial position of the picture 2, and deduces the dial area from the original picture to obtain the picture 3.
Step S5 selects a corresponding key point location model according to the type of the meter, and sends the picture 3 to the key point location model to obtain the coordinates of the key point.
The key point coordinates include a pointer start coordinate ptLineStart, an end coordinate ptLineEnd, a dial center coordinate ptCenter, and a dial start coordinate ptStart.
Step S6 is to perform post-processing on the coordinates of the key points to obtain two straight lines (a straight line where the current pointer is located and a dial start line), calculate an included angle between the two straight lines, and then obtain a meter reading according to the included angle and perValue (a meter reading represented by each degree) of the current meter.
According to the embodiment, a set of complete reading identification scheme for the intelligent power grid instrument is provided, and the scheme is stable and reliable, has strong generalization capability and can be directly popularized; the traditional algorithms with extremely high requirements on picture quality, such as template matching, are abandoned, and the generalization capability and accuracy are improved by using a deep learning algorithm; some typical networks are combined with the cavity convolution, so that the receptive field of a network characteristic layer is increased, and the accuracy of a classification model is improved; by extracting shallow, medium and deep feature layers, the features are fused, and finally regression of key points is carried out, so that the accuracy of key point positioning is improved.
It should be noted that the above embodiments can be freely combined as necessary. The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method of identifying a meter reading, comprising:
acquiring an instrument image of a reading to be identified;
performing image preprocessing on the instrument image by adopting a histogram equalization method;
performing instrument type identification on the preprocessed instrument image by adopting an instrument classification model;
and acquiring meter reading from the preprocessed meter image according to the obtained meter type.
2. The meter reading identification method according to claim 1, wherein the image preprocessing of the meter image by histogram equalization comprises:
acquiring gray level histogram information of the instrument image;
and adjusting the gray value of the instrument image by adopting a histogram equalization method according to the gray histogram information to ensure that the distribution of the gray value of the adjusted instrument image is approximately uniform.
3. The meter reading identification method according to claim 1, wherein:
the instrument classification model adopts a convolutional neural network based on cavity convolution.
4. The meter reading identification method of claim 3, wherein the convolutional neural network based on a hole convolution comprises:
the basic network uses an Xreception network;
replacing a convolution module in the Xception network with a hole convolution;
the output layer of the Xconcept network uses a convolution layer of 1x1x256 to replace a full connection layer.
5. The meter reading identification method of claim 1, wherein obtaining the meter reading from the pre-processed meter image based on the obtained meter type comprises:
extracting a meter area from the preprocessed meter image;
taking meter readings from the meter area according to the obtained meter type.
6. The meter reading identification method of claim 5, wherein obtaining the meter reading from the meter area based on the obtained meter type comprises:
selecting a corresponding key point positioning model according to the obtained instrument type;
inputting the instrument area into the key point positioning model to obtain a starting point coordinate and an end point coordinate of a pointer in the instrument, a dial center coordinate and a dial starting coordinate;
and obtaining the meter reading according to the starting point coordinate and the end point coordinate of the pointer, the dial center coordinate and the dial starting coordinate.
7. The meter reading identification method according to claim 6, wherein:
the key point positioning model is based on a ShuffleNet network, and a shallow feature, a middle feature and a deep feature are respectively extracted from the ShuffleNet network and accumulated on the final output layer.
8. The meter reading identification method of claim 7, wherein extracting a shallow feature, a middle feature and a deep feature from the ShuffleNet network respectively and accumulating on a final output layer comprises:
and respectively extracting a branch from the 5 th layer, the 8 th layer and the 13 th layer of the Shufflenet network, accumulating, and outputting through an embedded layer and a convolution layer of 1x1x 6.
9. A meter reading identification device, comprising:
the image acquisition module is used for acquiring an instrument image of a reading to be identified;
the image preprocessing module is used for preprocessing the instrument image by adopting a histogram equalization method;
the instrument classification module is used for identifying the instrument type of the preprocessed instrument image by adopting an instrument classification model;
and the instrument reading identification module is used for acquiring the instrument reading from the preprocessed instrument image according to the obtained instrument type.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the meter reading identification method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110389066.4A CN113283466A (en) | 2021-04-12 | 2021-04-12 | Instrument reading identification method and device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110389066.4A CN113283466A (en) | 2021-04-12 | 2021-04-12 | Instrument reading identification method and device and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113283466A true CN113283466A (en) | 2021-08-20 |
Family
ID=77276573
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110389066.4A Pending CN113283466A (en) | 2021-04-12 | 2021-04-12 | Instrument reading identification method and device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113283466A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113591819A (en) * | 2021-09-30 | 2021-11-02 | 成都交大光芒科技股份有限公司 | Intelligent identification method and device for auxiliary monitoring liquid crystal display instrument of traction substation |
| CN114359552A (en) * | 2022-03-02 | 2022-04-15 | 南京天创电子技术有限公司 | Instrument image identification method based on inspection robot |
| WO2024141157A1 (en) * | 2022-12-27 | 2024-07-04 | Siemens Aktiengesellschaft | System, apparatus and method for processing data from one or more physical devices monitoring one or more assets in a facility |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573702A (en) * | 2014-12-01 | 2015-04-29 | 长沙众治电气技术有限公司 | Method for automatically identifying sulfur hexafluoride pressure instrument image |
| CN108009535A (en) * | 2017-11-21 | 2018-05-08 | 武汉中元华电科技股份有限公司 | A kind of simple pointer meter reading method based on machine vision |
| CN108121985A (en) * | 2017-11-21 | 2018-06-05 | 武汉中元华电科技股份有限公司 | A kind of double pointer meters number reading methods based on machine vision |
| WO2018157862A1 (en) * | 2017-03-02 | 2018-09-07 | 腾讯科技（深圳）有限公司 | Vehicle type recognition method and device, storage medium and electronic device |
| CN109190473A (en) * | 2018-07-29 | 2019-01-11 | 国网上海市电力公司 | The application of a kind of " machine vision understanding " in remote monitoriong of electric power |
| CN110503144A (en) * | 2019-08-15 | 2019-11-26 | 南京天创电子技术有限公司 | A kind of pointer instrument recognition methods for crusing robot |
| CN112036475A (en) * | 2020-08-28 | 2020-12-04 | 江南大学 | Fusion module, multi-scale feature fusion convolutional neural network and image identification method |
| CN112115893A (en) * | 2020-09-24 | 2020-12-22 | 深圳市赛为智能股份有限公司 | Instrument panel pointer reading identification method and device, computer equipment and storage medium |
| WO2021031066A1 (en) * | 2019-08-19 | 2021-02-25 | 中国科学院深圳先进技术研究院 | Cartilage image segmentation method and apparatus, readable storage medium, and terminal device |
| CN112434612A (en) * | 2020-11-25 | 2021-03-02 | 创新奇智(上海)科技有限公司 | Smoking detection method and device, electronic equipment and computer readable storage medium |
| CN112560837A (en) * | 2020-12-02 | 2021-03-26 | 南方电网电力科技股份有限公司 | Reading method, device, equipment and storage medium of pointer instrument |
-
2021
- 2021-04-12 CN CN202110389066.4A patent/CN113283466A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573702A (en) * | 2014-12-01 | 2015-04-29 | 长沙众治电气技术有限公司 | Method for automatically identifying sulfur hexafluoride pressure instrument image |
| WO2018157862A1 (en) * | 2017-03-02 | 2018-09-07 | 腾讯科技（深圳）有限公司 | Vehicle type recognition method and device, storage medium and electronic device |
| CN108009535A (en) * | 2017-11-21 | 2018-05-08 | 武汉中元华电科技股份有限公司 | A kind of simple pointer meter reading method based on machine vision |
| CN108121985A (en) * | 2017-11-21 | 2018-06-05 | 武汉中元华电科技股份有限公司 | A kind of double pointer meters number reading methods based on machine vision |
| CN109190473A (en) * | 2018-07-29 | 2019-01-11 | 国网上海市电力公司 | The application of a kind of " machine vision understanding " in remote monitoriong of electric power |
| CN110503144A (en) * | 2019-08-15 | 2019-11-26 | 南京天创电子技术有限公司 | A kind of pointer instrument recognition methods for crusing robot |
| WO2021031066A1 (en) * | 2019-08-19 | 2021-02-25 | 中国科学院深圳先进技术研究院 | Cartilage image segmentation method and apparatus, readable storage medium, and terminal device |
| CN112036475A (en) * | 2020-08-28 | 2020-12-04 | 江南大学 | Fusion module, multi-scale feature fusion convolutional neural network and image identification method |
| CN112115893A (en) * | 2020-09-24 | 2020-12-22 | 深圳市赛为智能股份有限公司 | Instrument panel pointer reading identification method and device, computer equipment and storage medium |
| CN112434612A (en) * | 2020-11-25 | 2021-03-02 | 创新奇智(上海)科技有限公司 | Smoking detection method and device, electronic equipment and computer readable storage medium |
| CN112560837A (en) * | 2020-12-02 | 2021-03-26 | 南方电网电力科技股份有限公司 | Reading method, device, equipment and storage medium of pointer instrument |
Non-Patent Citations (1)
| Title |
|---|
| 徐发兵;吴怀宇;陈志环;喻汉;: "基于深度学习的指针式仪表检测与识别研究", 高技术通讯, no. 12 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113591819A (en) * | 2021-09-30 | 2021-11-02 | 成都交大光芒科技股份有限公司 | Intelligent identification method and device for auxiliary monitoring liquid crystal display instrument of traction substation |
| CN114359552A (en) * | 2022-03-02 | 2022-04-15 | 南京天创电子技术有限公司 | Instrument image identification method based on inspection robot |
| WO2024141157A1 (en) * | 2022-12-27 | 2024-07-04 | Siemens Aktiengesellschaft | System, apparatus and method for processing data from one or more physical devices monitoring one or more assets in a facility |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110348319B (en) | Face anti-counterfeiting method based on face depth information and edge image fusion | |
| CN113283466A (en) | Instrument reading identification method and device and readable storage medium | |
| CN110298291B (en) | Mask-RCNN-based cow face and cow face key point detection method | |
| CN109101938B (en) | Multi-label age estimation method based on convolutional neural network | |
| CN110232326B (en) | Three-dimensional object recognition method, device and storage medium | |
| CN110263768A (en) | A kind of face identification method based on depth residual error network | |
| CN110147776B (en) | Method and device for determining positions of key points of human face | |
| CN113361542B (en) | Local feature extraction method based on deep learning | |
| CN109376631A (en) | A kind of winding detection method and device neural network based | |
| CN109740572A (en) | A kind of human face in-vivo detection method based on partial color textural characteristics | |
| CN110363072B (en) | Tongue picture identification method, tongue picture identification device, computer equipment and computer readable storage medium | |
| CN107766864B (en) | Method and device for extracting features and method and device for object recognition | |
| CN109461186A (en) | Image processing method, device, computer readable storage medium and electronic equipment | |
| CN110334727B (en) | Intelligent matching detection method for tunnel cracks | |
| CN111784675A (en) | Method and device for processing article texture information, storage medium and electronic equipment | |
| CN112651333B (en) | Silence living body detection method, silence living body detection device, terminal equipment and storage medium | |
| CN112862757A (en) | Weight evaluation system based on computer vision technology and implementation method | |
| CN113011253B (en) | Facial expression recognition method, device, equipment and storage medium based on ResNeXt network | |
| CN109920018A (en) | Black-and-white photograph color recovery method, device and storage medium neural network based | |
| CN116343228A (en) | Intelligent reading method and system for water meter | |
| CN110569784A (en) | Human body size measuring method and system, storage medium and electronic equipment | |
| CN111523342A (en) | Two-dimensional code detection and correction method in complex scene | |
| CN111080754A (en) | Character animation production method and device for connecting characteristic points of head and limbs | |
| CN112926508B (en) | Training method and device of living body detection model | |
| CN112070181B (en) | Image stream-based cooperative detection method and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |