CN112507815A - Artificial intelligence image recognition algorithm and system for pointer instrument panel scale - Google Patents
Artificial intelligence image recognition algorithm and system for pointer instrument panel scale Download PDFInfo
- Publication number
- CN112507815A CN112507815A CN202011332228.2A CN202011332228A CN112507815A CN 112507815 A CN112507815 A CN 112507815A CN 202011332228 A CN202011332228 A CN 202011332228A CN 112507815 A CN112507815 A CN 112507815A
- Authority
- CN
- China
- Prior art keywords
- instrument panel
- pointer
- neural network
- scale
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 23
- 238000013135 deep learning Methods 0.000 claims abstract description 33
- 238000013528 artificial neural network Methods 0.000 claims abstract description 25
- 238000001514 detection method Methods 0.000 claims abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 13
- 238000002372 labelling Methods 0.000 claims abstract description 11
- 238000003062 neural network model Methods 0.000 claims abstract description 11
- 238000005520 cutting process Methods 0.000 claims abstract description 7
- 238000003860 storage Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 28
- 238000004364 calculation method Methods 0.000 claims description 13
- 230000004913 activation Effects 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 3
- 230000007935 neutral effect Effects 0.000 claims description 2
- 210000000988 bone and bone Anatomy 0.000 claims 1
- 239000003086 colorant Substances 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 210000003128 head Anatomy 0.000 description 3
- 238000011897 real-time detection Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003121 nonmonotonic effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/02—Recognising information on displays, dials, clocks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an artificial intelligence image recognition algorithm and system for pointer instrument panel scale, wherein the method comprises the following steps: s1, collecting the photos of the instrument panel, labeling the content of the photos and storing the content; s2, building a deep learning neural network, extracting the characteristics of the marked photos, and storing the trained deep learning neural network model; s3, recognizing the picture to be recognized by using the deep learning neural network model, and finishing cutting and independent storage of the dashboard picture; and S4, completing character detection and identification of the dashboard picture by adopting an open source tesseract algorithm. Has the advantages that: the invention adopts a deep learning neural network mode, can finish detection and identification on various instrument panel photos, greatly improves the identification accuracy of the instrument panel, can be used in various places needing to identify the reading of the instrument panel, can detect the reading of the instrument panel in real time, and gives an alarm when the reading exceeds a certain threshold value.
Description
Technical Field
The invention relates to the technical field of picture recognition, in particular to an artificial intelligence image recognition algorithm and system for instrument panel scales of a pointer instrument.
Background
In daily life and industrial production, most of machine equipment or switch valves display data information by using an instrument panel, and the instrument panel needs to detect, read and analyze information such as measuring range, measuring unit, pointer direction and the like when reading and identifying due to various displayed data and realized functions, and the traditional instrument panel detection method generally comprises the following methods:
1. the visual inspection method is only suitable for real-time observation of human eyes, cannot complete tasks under the conditions of lack of manpower and no human, has relatively high manpower cost, and can cause fatigue of human eyes to cause detection errors after long-time detection.
2. The instrument panel identification method based on the traditional image algorithm is low in identification speed, cannot accurately identify the position of an instrument panel pointer, cannot detect pictures with poor quality or incorrect angles of the instrument panel, can identify other objects on equipment as the instrument panel pointer, is low in accuracy rate, has high requirements for the quality of the images, is low in algorithm robustness, and has more false detection and missed detection.
An effective solution to the problems in the related art has not been proposed yet.
Disclosure of Invention
Aiming at the problems in the related art, the invention provides an artificial intelligent image recognition algorithm and system for the scale of a dial instrument panel, so as to overcome the technical problems in the prior related art.
Therefore, the invention adopts the following specific technical scheme:
according to one aspect of the invention, an artificial intelligence image recognition algorithm for a pointer instrument panel scale is provided, and the method comprises the following steps:
s1, collecting the photos of the instrument panel, labeling the content of the photos and storing the content;
s2, building a deep learning neural network, extracting the characteristics of the marked photos, and storing the trained deep learning neural network model;
s3, recognizing the picture to be recognized by using the deep learning neural network model, and finishing cutting and independent storage of the dashboard picture;
and S4, completing character detection and recognition of the dashboard picture by adopting an open source tesseract algorithm.
Further, the content of the photograph includes dashboard positions x1, y1, x2, y2 and dashboard pointer positions centerx (center abscissa), centery (center ordinate), directx (diameter abscissa), directy (diameter ordinate);
the instrument panel positions x1, y1, x2 and y2 respectively correspond to an abscissa and an ordinate of the upper left corner of the instrument panel and an abscissa and an ordinate of the lower right corner of the instrument panel, and the instrument panel pointer positions centerx, centery, directx and direction respectively correspond to an abscissa and an ordinate of the rotation center of the instrument panel pointer and an abscissa and an ordinate of the pointer pointing end needle point.
Further, the labeling result of the photo is stored as an xml file, and each xml file corresponds to the photo one by one.
Further, a Backbone portion of the deep learning neural network implements cross-stage partial connection by using a CSPNet (cross-stage local network), and a Swish activation function is used, where a function expression of the Swish activation function is:
f(x)＝x*sigmoid(x)。
further, a hack (Neck) part of the deep learning neural network generates an image pyramid by using BiFPN, and image features are mixed and combined, wherein a specific formula is as follows:
Pout7＝Conv(Pin7)；
Pout6＝Conv(Pin6+Resize(Pout7))；
…
Pout3＝Conv(Pin3+Resize(Pout4))；
where Conv is the convolution operation and Resize is the feature map upsampling or downsampling operation.
Further, the Head part of the deep learning neural network outputs 3 stages, and the down-sampling rates of the stages are 8, 16 and 32 corresponding to small, medium and large instrument panels respectively.
Further, the loss function of the deep learning neural network is based on the classification loss of the instrument panel, the regression loss of the instrument panel frame and the regression loss of the key point of the instrument panel pointer, and the specific function is as follows:
L＝Lobj(pi,pi*)+λ1pi*Lbox(ti,ti*)+λ2pi*Lldms(li,li*)；
lobj is the classification loss of the instrument panel, softmax loss of two classifications is adopted, pi represents the probability that the prediction anchor is the instrument panel, pi represents the true value, a positive sample is 1, a negative sample is 0, Lbox represents the regression loss of the instrument panel frame, smooth L1 regression functions are adopted, ti and ti represent the prediction frame position corresponding to the positive sample and the position of a real mark frame, Lldms is the regression loss of the key point of the pointer, a smooth L1 regression function is also adopted, li and li respectively represent the prediction value and the true value of the key point of the pointer on the instrument panel of the positive sample, each value comprises the coordinate information of two key points, and the values of lambda 1 and lambda 2 are 0.25 and 0.1 respectively.
Further, the completing the character detection and identification of the dashboard picture by adopting the open-source tesseract algorithm further comprises:
s401, identifying the measuring range of the instrument panel and the scale position corresponding to the reading number through the instrument panel picture;
s402, determining a scale interval of the instrument according to the value and the position of the number, and correspondingly calculating;
s403, determining the position and the direction of a pointer according to the pointer coordinates output by the deep learning neural network model;
and S404, calculating the pointer reading of the instrument panel according to the pointer slope and the measuring range of the instrument panel.
According to one aspect of the invention, an artificial intelligence image recognition system for instrument panel scale of a pointer instrument is also provided, and the system comprises:
the photo collecting and marking module is used for collecting the photos of the instrument panel, marking the content of the photos and storing the content;
the model training module is used for building a deep learning neural network, extracting the characteristics of the marked photos and storing the trained model;
the picture recognition module is used for recognizing the photo to be recognized by utilizing the model, detecting the positions of the instrument panel and the pointer, cutting the instrument panel part in the photo and realizing the independent storage of the instrument panel picture;
and the identification calculation module is used for carrying out character detection and identification on the instrument panel picture by adopting an open source tesseract algorithm, identifying the measuring range of the instrument panel and the scale position corresponding to the number, determining the scale interval of the instrument according to the value and the position of the number, and completing calculation by adopting a preset method to realize the identification of the reading of the instrument panel.
Further, the model training module comprises a backsbone part, a neutral part, a Head part and a loss function part.
The invention has the beneficial effects that:
1. the invention adopts a deep learning neural network mode, trains according to the characteristics of the collected labeled parts in the labeled pictures, and stores the training result into the model, wherein the quality of the model is related to the labeling quality of the pictures and the number of the labeled pictures, the more accurate the labeling is, the more the labeled instrument panel types are, the more the trained pictures are, the higher the quality of the model is, and the better the recognition effect is. The method can be used in various places where instrument panel reading needs to be identified, can detect the reading of the instrument panel in real time, and sends out alarm information after the reading exceeds a certain threshold value.
2. The invention adopts a neural network with higher speed, can run on edge equipment based on CPU fact, and greatly increases the identification efficiency of the instrument panel; and the position and the angle of the pointer are extracted by adopting a key point detection technology, and the Hough line detection technology is adopted to assist in identification, so that the precision of pointer identification is improved. The invention improves the identification precision, reduces the algorithm calculation requirement, can achieve the effect of real-time detection on edge equipment with low calculation, and can properly save the cost of computing equipment; the accurate identification of the instrument panel can be realized under the condition of low image quality, parameters adapting to the image quality do not need to be adjusted in a circulating and reciprocating mode during image processing, and the time for deploying and debugging the algorithm is saved.
3. The invention has more accurate recognition and reading of the instrument panel, can correct the position of the instrument panel, can perform instrument panel recognition on cameras with larger distortion and photos with bad shooting angles, has lower requirements on the image shooting conditions of the instrument panel to be recognized, can perform recognition under the conditions of different light brightness and colors, and greatly improves the accuracy, thereby avoiding the need of directly shooting the instrument panel every time, reducing the times of photo shooting, improving the recognition efficiency, even if the instrument panel at the edge of the picture can be recognized, having low requirements on the distortion of the camera, and reducing the cost for purchasing the camera. In addition, the type of the instrument panel needing to be identified is not required, and the specified instrument panel does not need to be specially replaced for identifying the instrument panel; and the requirement on the image shooting condition of the instrument panel to be identified is low, and the instrument panel can be identified under the conditions of different light brightness and colors.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flow chart of an artificial intelligence image recognition algorithm for a pointer instrument dashboard scale according to an embodiment of the invention;
FIG. 2 is a block diagram of CSPNet in an artificial intelligence image recognition algorithm with respect to pointer instrument dashboard scales according to an embodiment of the present invention;
FIG. 3 is a structural diagram of BiFPN in an artificial intelligence image recognition algorithm for the scale of a dial instrument dashboard according to an embodiment of the present invention;
fig. 4 is a system block diagram of an artificial intelligence image recognition system for a dial instrument dashboard scale according to an embodiment of the invention.
In the figure:
1. a photo collection and labeling module; 2. a model training module; 3. a picture identification module; 4. a computing module is identified.
Detailed Description
For further explanation of the various embodiments, the drawings which form a part of the disclosure and which are incorporated in and constitute a part of this specification, illustrate embodiments and, together with the description, serve to explain the principles of operation of the embodiments, and to enable others of ordinary skill in the art to understand the various embodiments and advantages of the invention, and, by reference to these figures, reference is made to the accompanying drawings, which are not to scale and wherein like reference numerals generally refer to like elements.
According to the embodiment of the invention, an artificial intelligence image recognition algorithm and system for instrument panel scale of a pointer instrument are provided.
Referring to the drawings and the detailed description, the invention is further explained, as shown in fig. 1, according to an embodiment of the invention, regarding an artificial intelligence image recognition algorithm for a pointer instrument panel scale, the method includes the following steps:
s1, collecting the photos of the instrument panel, labeling the content of the photos and storing the content;
the more the number of the photos is, the more accurate the marked position is, the more the types of the instrument panel are, and the stronger the identification robustness is.
S2, building a deep learning neural network, extracting the characteristics of the marked photos, and storing the trained deep learning neural network model;
s3, recognizing the picture to be recognized by using the deep learning neural network model, and finishing cutting and independent storage of the dashboard picture;
projection transformation is needed to be carried out on some instrument panels with more deviated shooting angles, and the positions of the instrument panels are corrected, so that the calculation of the pointer angles of the instrument panels is more accurate.
And S4, completing character detection and identification of the dashboard picture by adopting an open source tesseract algorithm.
In one embodiment, the content of the photograph includes dashboard positions x1, y1, x2, y2 and dashboard pointer positions centerx, centery, directx, directy;
the instrument panel positions x1, y1, x2 and y2 respectively correspond to an abscissa and an ordinate of the upper left corner of the instrument panel and an abscissa and an ordinate of the lower right corner of the instrument panel, and the instrument panel pointer positions centerx, centery, directx and direction respectively correspond to an abscissa and an ordinate of the rotation center of the instrument panel pointer and an abscissa and an ordinate of the pointer pointing end needle point.
In one embodiment, the labeling result of the photo is saved as an xml file, and each xml file corresponds to the photo one by one.
In one embodiment, as shown in fig. 2, the backhaul part of the deep learning neural network implements cross-phase partial connection using CSPNet and uses a Swish activation function, where the function expression of the Swish activation function is:
f(x)＝x*sigmoid(x)。
the CSPNet can improve the learning capacity of CNN, still can guarantee higher accuracy after model compression and lightweight words, and reduce the calculation cost and memory occupation, the CSPNet has a basic structure diagram as shown in FIG. 2, wherein Part1 does not operate direct concatee, Part2 performs convolution operation, Block is formed by connecting a plurality of basic convolution layers and shortcut, Transition Layer represents a Transition Layer and mainly comprises a bottleneck Layer (1x1 convolution) and a pooling Layer, aggregation and image feature formation on different image fine granularity can be realized through the CSPNet, in order to reduce the calculation amount of the network, the depth of the network and the number of channels of the network are properly reduced, and a Swish activation function is adopted, compared with a Relu activation function, the Swish activation function is smoother and nonmonotonic, and a neural network deeper than Relu can be trained.
In one embodiment, as shown in fig. 3, the Neck part of the deep learning neural network generates an image pyramid by using BiFPN, mixes and combines image features, wherein the BiFPN has a structure as shown in fig. 3, receives the features { P3, P4, P5, P6, P7} of level 3-7 from the backbone network, and repeatedly applies the top-down and bottom-up bi-directional feature fusion, and the specific formula is as follows:
Pout7＝Conv(Pin7)；
Pout6＝Conv(Pin6+Resize(Pout7))；
…
Pout3＝Conv(Pin3+Resize(Pout4))；
where Conv is the convolution operation and Resize is the feature map upsampling or downsampling operation.
In one embodiment, the Head portion of the deep learning neural network outputs 3 stages with down-sampling rates of 8, 16, 32 for small, medium, and large dashboards, respectively.
In the same way, the 16 × 16 field under stride16 is used for detecting a middle instrument panel, and the 8 × 8 field under stride8 is used for detecting a smaller instrument panel. The numerical values in the output channel respectively correspond to the confidence coefficient and the position coordinate value of the instrument panel, meanwhile, the branch with the added key point is used for detecting the pointer, and the pointer endpoint coordinates of the pointer endpoint marking point of the pointer close to the rotation center and the pointer endpoint coordinate of the pointer pointing end needle point are correspondingly output.
In one embodiment, the loss function of the deep learning neural network is based on instrument panel classification loss, instrument panel frame regression loss, and instrument panel pointer key point regression loss, and the specific function is as follows:
L＝Lobj(pi,pi*)+λ1pi*Lbox(ti,ti*)+λ2pi*Lldms(li,li*)；
lobj is the classification loss of the instrument panel, softmax loss of two classifications is adopted, pi represents the probability that the prediction anchor is the instrument panel, pi represents the true value, a positive sample is 1, a negative sample is 0, Lbox represents the regression loss of the instrument panel frame, smooth L1 regression functions are adopted, ti and ti represent the prediction frame position corresponding to the positive sample and the position of a real mark frame, Lldms is the regression loss of the key point of the pointer, a smooth L1 regression function is also adopted, li and li respectively represent the prediction value and the true value of the key point of the pointer on the instrument panel of the positive sample, each value comprises the coordinate information of two key points, and the values of lambda 1 and lambda 2 are 0.25 and 0.1 respectively.
In addition, in the training process, various data enhancement methods such as Mosaic and Random Scale are adopted to improve the generalization capability of the model.
In one embodiment, the completing the character detection and recognition of the dashboard picture by using the open-source tesseract algorithm further includes:
s401, identifying the measuring range of the instrument panel and the scale position corresponding to the reading number through the instrument panel picture;
s402, determining a scale interval of the instrument according to the value and the position of the number, and correspondingly calculating;
s403, determining the position and the direction of a pointer according to the pointer coordinates output by the deep learning neural network model;
and S404, calculating the pointer reading of the instrument panel according to the pointer slope and the measuring range of the instrument panel.
According to another embodiment of the invention, according to the figure, an artificial intelligence image recognition system for instrument panel scale of a pointer instrument is further provided, and the system comprises:
the photo collecting and marking module 1 is used for collecting the photos of the instrument panel, marking the content of the photos and storing the photos;
the model training module 2 is used for building a deep learning neural network, extracting the characteristics of the marked photos and storing the trained model;
the picture recognition module 3 is used for recognizing the pictures to be recognized by utilizing the model, detecting the positions of the instrument panel and the pointer, cutting the instrument panel part in the pictures and realizing the independent storage of the instrument panel pictures;
and the identification calculation module 4 is used for performing character detection and identification on the instrument panel picture by adopting an open source tesseract algorithm, identifying the measuring range of the instrument panel and the scale position corresponding to the indicating number, determining the scale interval of the instrument according to the value and the position of the indicating number, and completing calculation by adopting a preset method to realize the identification of the reading of the instrument panel.
In one embodiment, the model training module includes a Back section, a Neck section, a Head section, and a loss function section.
In summary, by means of the above technical scheme of the present invention, the present invention adopts a deep learning neural network mode, performs training according to the characteristics of the collected labeled part in the labeled picture, and stores the training result into the model, the quality of the model is related to the labeled quality of the picture and the number of labeled pictures, the more accurate the labeling is, the more the labeled instrument panel types are, the more the trained pictures are, the higher the quality of the model is, and the better the recognition effect is. The method can be used in various places where instrument panel reading needs to be identified, can detect the reading of the instrument panel in real time, and sends out alarm information after the reading exceeds a certain threshold value. The invention adopts a neural network with higher speed, can run on edge equipment based on CPU fact, and greatly increases the identification efficiency of the instrument panel; and the position and the angle of the pointer are extracted by adopting a key point detection technology, and the Hough line detection technology is adopted to assist in identification, so that the precision of pointer identification is improved. The invention improves the identification precision, reduces the algorithm calculation requirement, can achieve the effect of real-time detection on edge equipment with low calculation, and can properly save the cost of computing equipment; the accurate identification of the instrument panel can be realized under the condition of low image quality, parameters adapting to the image quality do not need to be adjusted in a circulating and reciprocating mode during image processing, and the time for deploying and debugging the algorithm is saved. The invention has more accurate recognition and reading of the instrument panel, can correct the position of the instrument panel, can perform instrument panel recognition on cameras with larger distortion and photos with bad shooting angles, has lower requirements on the image shooting conditions of the instrument panel to be recognized, can perform recognition under the conditions of different light brightness and colors, and greatly improves the accuracy, thereby avoiding the need of directly shooting the instrument panel every time, reducing the times of photo shooting, improving the recognition efficiency, even if the instrument panel at the edge of the picture can be recognized, having low requirements on the distortion of the camera, and reducing the cost for purchasing the camera. In addition, the type of the instrument panel needing to be identified is not required, and the specified instrument panel does not need to be specially replaced for identifying the instrument panel; and the requirement on the image shooting condition of the instrument panel to be identified is low, and the instrument panel can be identified under the conditions of different light brightness and colors.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. The artificial intelligence image recognition algorithm for the scale of the instrument panel of the pointer instrument is characterized by comprising the following steps of:
s1, collecting the photos of the instrument panel, labeling the content of the photos and storing the content;
s2, building a deep learning neural network, extracting the characteristics of the marked photos, and storing the trained deep learning neural network model;
s3, recognizing the picture to be recognized by using the deep learning neural network model, and finishing cutting and independent storage of the dashboard picture;
and S4, completing character detection and identification of the dashboard picture by adopting an open source tesseract algorithm.
2. The pointer instrument dashboard scale artificial intelligence image recognition algorithm of claim 1, wherein the content of the photograph includes dashboard positions x1, y1, x2, y2 and dashboard pointer positions centerx, centery, directx, direction;
the instrument panel positions x1, y1, x2 and y2 respectively correspond to an abscissa and an ordinate of the upper left corner of the instrument panel and an abscissa and an ordinate of the lower right corner of the instrument panel, and the instrument panel pointer positions centerx, centery, directx and direction respectively correspond to an abscissa and an ordinate of the rotation center of the instrument panel pointer and an abscissa and an ordinate of the pointer pointing end needle point.
3. The artificial intelligence image recognition algorithm for pointer instrument panel scale as recited in claim 2, wherein the labeling result of the photos is saved as xml files, and each xml file corresponds to one of the photos.
4. The artificial intelligence image recognition algorithm for pointer instrument panel scale according to claim 1, wherein a Backbone part of the deep learning neural network adopts CSPNet to realize cross-stage partial connection and adopts a Swish activation function, and a function expression of the Swish activation function is as follows:
f(x)＝x*sigmoid(x)。
5. the artificial intelligence image recognition algorithm for the scale of the pointer instrument dashboard as recited in claim 4, wherein the Neck part of the deep learning neural network adopts BiFPN to generate an image pyramid, and image features are mixed and combined, and the specific formula is as follows:
Pout7＝Conv(Pin7)；
Pout6＝Conv(Pin6+Resize(Pout7))；
…
Pout3＝Conv(Pin3+Resize(Pout4))；
where Conv is the convolution operation and Resize is the feature map upsampling or downsampling operation.
6. The artificial intelligence image recognition algorithm for pointer instrument dashboard scales of claim 5, wherein a Head portion of the deep learning neural network outputs 3 stages with down-sampling rates of 8, 16 and 32 for small, medium and large dashboards, respectively.
7. The artificial intelligence image recognition algorithm for pointer instrument panel scale according to claim 6, wherein the loss function of the deep learning neural network is based on a panel classification loss, a panel frame regression loss and a panel pointer key point regression loss, and the specific functions are as follows:
L＝Lobj(pi,pi*)+λ1pi*Lbox(ti,ti*)+λ2pi*Lldms(li,li*)；
lobj is the classification loss of the instrument panel, softmax loss of two classifications is adopted, pi represents the probability that the prediction anchor is the instrument panel, pi represents the true value, a positive sample is 1, a negative sample is 0, Lbox represents the regression loss of the instrument panel frame, smooth L1 regression functions are adopted, ti and ti represent the prediction frame position corresponding to the positive sample and the position of a real mark frame, Lldms is the regression loss of the key point of the pointer, a smooth L1 regression function is also adopted, li and li respectively represent the prediction value and the true value of the key point of the pointer on the instrument panel of the positive sample, each value comprises the coordinate information of two key points, and the values of lambda 1 and lambda 2 are 0.25 and 0.1 respectively.
8. The artificial intelligence image recognition algorithm for pointer instrument panel scale as recited in claim 1, wherein the performing text detection and recognition on the panel board picture by using an open source tesseract algorithm further comprises:
s401, identifying the measuring range of the instrument panel and the scale position corresponding to the reading number through the instrument panel picture;
s402, determining a scale interval of the instrument according to the value and the position of the number, and correspondingly calculating;
s403, determining the position and the direction of a pointer according to the pointer coordinates output by the deep learning neural network model;
and S404, calculating the pointer reading of the instrument panel according to the pointer slope and the measuring range of the instrument panel.
9. An artificial intelligence image recognition system for instrument panel scale of pointer instrument to realize the artificial intelligence image recognition algorithm for instrument panel scale of pointer instrument as claimed in any one of claims 1-8, characterized in that the system comprises:
the photo collecting and marking module is used for collecting the photos of the instrument panel, marking the content of the photos and storing the content;
the model training module is used for building a deep learning neural network, extracting the characteristics of the marked photos and storing the trained model;
the picture recognition module is used for recognizing the photo to be recognized by utilizing the model, detecting the positions of the instrument panel and the pointer, cutting the instrument panel part in the photo and realizing the independent storage of the instrument panel picture;
and the identification calculation module is used for carrying out character detection and identification on the instrument panel picture by adopting an open source tesseract algorithm, identifying the measuring range of the instrument panel and the scale position corresponding to the number, determining the scale interval of the instrument according to the value and the position of the number, and completing calculation by adopting a preset method to realize the identification of the reading of the instrument panel.
10. The system of claim 9, wherein the model training module comprises a Back bone section, a neutral section, a Head section, and a loss function section.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011332228.2A CN112507815A (en) | 2020-11-24 | 2020-11-24 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011332228.2A CN112507815A (en) | 2020-11-24 | 2020-11-24 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112507815A true CN112507815A (en) | 2021-03-16 |
Family
ID=74958318
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011332228.2A Pending CN112507815A (en) | 2020-11-24 | 2020-11-24 | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112507815A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112990179A (en) * | 2021-04-20 | 2021-06-18 | 成都阿莱夫信息技术有限公司 | Single-pointer type dial reading automatic identification method based on picture processing |
| CN113408542A (en) * | 2021-05-25 | 2021-09-17 | 深圳市富能新能源科技有限公司 | Pointer instrument reading identification method, system, equipment and computer storage medium |
| CN113408551A (en) * | 2021-05-25 | 2021-09-17 | 深圳市富能新能源科技有限公司 | Pointer instrument reading identification method, system, equipment and computer storage medium |
| CN113435300A (en) * | 2021-06-23 | 2021-09-24 | 国网智能科技股份有限公司 | Real-time identification method and system for lightning arrester instrument |
| CN113469178A (en) * | 2021-07-05 | 2021-10-01 | 安徽南瑞继远电网技术有限公司 | Electric power meter identification method based on deep learning |
| CN113506244A (en) * | 2021-06-05 | 2021-10-15 | 北京超维世纪科技有限公司 | Indicator light detection and color identification generalization capability improvement algorithm based on deep learning |
| CN113505808A (en) * | 2021-06-05 | 2021-10-15 | 北京超维世纪科技有限公司 | Detection and identification algorithm for power distribution facility switch based on deep learning |
| CN113705111A (en) * | 2021-09-22 | 2021-11-26 | 百安居信息技术(上海)有限公司 | Fitment furniture automatic layout method and system based on deep learning |
| CN113837178A (en) * | 2021-09-15 | 2021-12-24 | 国网河北省电力有限公司 | Deep learning-based automatic positioning and unified segmentation method for meter of transformer substation |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108764257A (en) * | 2018-05-23 | 2018-11-06 | 郑州金惠计算机***工程有限公司 | A kind of pointer instrument recognition methods of various visual angles |
| US20190095739A1 (en) * | 2017-09-27 | 2019-03-28 | Harbin Institute Of Technology | Adaptive Auto Meter Detection Method based on Character Segmentation and Cascade Classifier |
| CN109948469A (en) * | 2019-03-01 | 2019-06-28 | 吉林大学 | The automatic detection recognition method of crusing robot instrument based on deep learning |
| CN110543878A (en) * | 2019-08-07 | 2019-12-06 | 华南理工大学 | pointer instrument reading identification method based on neural network |
| CN110659636A (en) * | 2019-09-20 | 2020-01-07 | 随锐科技集团股份有限公司 | Pointer instrument reading identification method based on deep learning |
| CN111950330A (en) * | 2019-05-16 | 2020-11-17 | 杭州测质成科技有限公司 | Pointer instrument indicating number detection method based on target detection |
-
2020
- 2020-11-24 CN CN202011332228.2A patent/CN112507815A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190095739A1 (en) * | 2017-09-27 | 2019-03-28 | Harbin Institute Of Technology | Adaptive Auto Meter Detection Method based on Character Segmentation and Cascade Classifier |
| CN108764257A (en) * | 2018-05-23 | 2018-11-06 | 郑州金惠计算机***工程有限公司 | A kind of pointer instrument recognition methods of various visual angles |
| CN109948469A (en) * | 2019-03-01 | 2019-06-28 | 吉林大学 | The automatic detection recognition method of crusing robot instrument based on deep learning |
| CN111950330A (en) * | 2019-05-16 | 2020-11-17 | 杭州测质成科技有限公司 | Pointer instrument indicating number detection method based on target detection |
| CN110543878A (en) * | 2019-08-07 | 2019-12-06 | 华南理工大学 | pointer instrument reading identification method based on neural network |
| CN110659636A (en) * | 2019-09-20 | 2020-01-07 | 随锐科技集团股份有限公司 | Pointer instrument reading identification method based on deep learning |
Non-Patent Citations (6)
| Title |
|---|
| ALEXEY B 等: "YOLOv4: Optimal Speed and Accuracy of Object Detection", 《ARXIV:2004.10934V》, 23 April 2020 (2020-04-23), pages 1 - 17, XP093032778, DOI: 10.48550/arXiv.2004.10934 * |
| TAN MINGXING 等: "EfficientDet: Scalable and Efficient Object Detection", 《2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》, 5 August 2020 (2020-08-05), pages 10778 - 10787 * |
| 何配林: "基于深度学习的工业仪表识别读数算法研究及应用", 《中国优秀硕士学位论文全文数据库 (工程科技Ⅱ辑)》, no. 07, 15 July 2020 (2020-07-15), pages 030 - 83 * |
| 刘丽媛 等: "复杂背景下仪表信息的图像识别研究", 《激光杂志》, vol. 41, no. 04, 25 April 2020 (2020-04-25), pages 66 - 69 * |
| 张骥 等: "一种鲁棒的数显式仪表读数检测识别方法", 《自动化技术与应用》, vol. 38, no. 02, 25 February 2019 (2019-02-25), pages 140 - 144 * |
| 秦善培: "基于机器视觉的汽车仪表读数识别方法研究与应用", 《中国优秀硕士学位论文全文数据库 (工程科技Ⅱ辑)》, no. 08, 15 August 2015 (2015-08-15), pages 035 - 129 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112990179A (en) * | 2021-04-20 | 2021-06-18 | 成都阿莱夫信息技术有限公司 | Single-pointer type dial reading automatic identification method based on picture processing |
| CN113408542A (en) * | 2021-05-25 | 2021-09-17 | 深圳市富能新能源科技有限公司 | Pointer instrument reading identification method, system, equipment and computer storage medium |
| CN113408551A (en) * | 2021-05-25 | 2021-09-17 | 深圳市富能新能源科技有限公司 | Pointer instrument reading identification method, system, equipment and computer storage medium |
| CN113506244A (en) * | 2021-06-05 | 2021-10-15 | 北京超维世纪科技有限公司 | Indicator light detection and color identification generalization capability improvement algorithm based on deep learning |
| CN113505808A (en) * | 2021-06-05 | 2021-10-15 | 北京超维世纪科技有限公司 | Detection and identification algorithm for power distribution facility switch based on deep learning |
| CN113435300A (en) * | 2021-06-23 | 2021-09-24 | 国网智能科技股份有限公司 | Real-time identification method and system for lightning arrester instrument |
| CN113435300B (en) * | 2021-06-23 | 2022-10-14 | 国网智能科技股份有限公司 | Real-time identification method and system for lightning arrester instrument |
| CN113469178A (en) * | 2021-07-05 | 2021-10-01 | 安徽南瑞继远电网技术有限公司 | Electric power meter identification method based on deep learning |
| CN113469178B (en) * | 2021-07-05 | 2024-03-01 | 安徽南瑞继远电网技术有限公司 | Power meter identification method based on deep learning |
| CN113837178A (en) * | 2021-09-15 | 2021-12-24 | 国网河北省电力有限公司 | Deep learning-based automatic positioning and unified segmentation method for meter of transformer substation |
| CN113705111A (en) * | 2021-09-22 | 2021-11-26 | 百安居信息技术(上海)有限公司 | Fitment furniture automatic layout method and system based on deep learning |
| CN113705111B (en) * | 2021-09-22 | 2024-04-26 | 百安居网络技术(上海)有限公司 | Automatic layout method and system for decoration furniture based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112507815A (en) | Artificial intelligence image recognition algorithm and system for pointer instrument panel scale | |
| CN111612763B (en) | Mobile phone screen defect detection method, device and system, computer equipment and medium | |
| CN108647694A (en) | Correlation filtering method for tracking target based on context-aware and automated response | |
| AU2020103716A4 (en) | Training method and device of automatic identification device of pointer instrument with numbers in natural scene | |
| CN112949633B (en) | Improved YOLOv 3-based infrared target detection method | |
| CN102411705B (en) | Method and interface of recognizing user's dynamic organ gesture and electric-using apparatus using the interface | |
| CN109522963A (en) | A kind of the feature building object detection method and system of single-unit operation | |
| CN113487610B (en) | Herpes image recognition method and device, computer equipment and storage medium | |
| CN111046746A (en) | License plate detection method and device | |
| CN117152484B (en) | Small target cloth flaw detection method based on improved YOLOv5s | |
| CN113505808B (en) | Deep learning-based switch detection and identification algorithm for power distribution facilities | |
| CN116385810A (en) | Yolov 7-based small target detection method and system | |
| CN113987251A (en) | Method, system, equipment and storage medium for establishing ship face characteristic database | |
| CN114241469A (en) | Information identification method and device for electricity meter rotation process | |
| CN116434230A (en) | Ship water gauge reading method under complex environment | |
| CN115082922A (en) | Water meter digital picture processing method and system based on deep learning | |
| Wang et al. | Improved YOLOv3 detection method for PCB plug-in solder joint defects based on ordered probability density weighting and attention mechanism | |
| CN112364687A (en) | Improved Faster R-CNN gas station electrostatic sign identification method and system | |
| CN111008634A (en) | Character recognition method and character recognition device based on example segmentation | |
| CN116912872A (en) | Drawing identification method, device, equipment and readable storage medium | |
| CN116385915A (en) | Water surface floater target detection and tracking method based on space-time information fusion | |
| CN114937153A (en) | Neural network-based visual feature processing system and method under weak texture environment | |
| CN114463628A (en) | Deep learning remote sensing image ship target identification method based on threshold value constraint | |
| Chen et al. | EEE-Net: Efficient edge enhanced network for surface defect detection of glass | |
| Yu et al. | Surface Defect inspection under a small training set condition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |