CN112257709B - Signboard photo auditing method and device, electronic equipment and readable storage medium - Google Patents
Signboard photo auditing method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN112257709B CN112257709B CN202011145385.2A CN202011145385A CN112257709B CN 112257709 B CN112257709 B CN 112257709B CN 202011145385 A CN202011145385 A CN 202011145385A CN 112257709 B CN112257709 B CN 112257709B
- Authority
- CN
- China
- Prior art keywords
- photo
- signboard
- sign
- image
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000001514 detection method Methods 0.000 claims abstract description 60
- 238000012549 training Methods 0.000 claims description 42
- 238000003062 neural network model Methods 0.000 claims description 32
- 238000004422 calculation algorithm Methods 0.000 claims description 25
- 230000006870 function Effects 0.000 claims description 19
- 238000013434 data augmentation Methods 0.000 claims description 17
- 238000010606 normalization Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 10
- 102100032202 Cornulin Human genes 0.000 claims description 8
- 101000920981 Homo sapiens Cornulin Proteins 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 7
- 210000004205 output neuron Anatomy 0.000 claims description 3
- 238000012545 processing Methods 0.000 description 13
- 238000012360 testing method Methods 0.000 description 13
- 238000012795 verification Methods 0.000 description 11
- 230000007306 turnover Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 5
- 238000002372 labelling Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 210000002569 neuron Anatomy 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 239000000470 constituent Substances 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000004880 explosion Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000009849 deactivation Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002779 inactivation Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a signboard photo auditing method, which comprises the following steps: and (3) a step of detecting the overturning: detecting whether the signboard photo is a turner photo or not, if not, detecting; a signboard image recognition step: identifying a sign image in the detected sign photo; a signboard character recognition step: identifying text in the sign image; and a signboard character comparison step: and comparing whether the characters in the signboard images are consistent with the corresponding signboard characters, and if so, checking. Further, the present application relates to a sign photo auditing apparatus including: the overturn detecting module is used for detecting whether the signboard photo is an overturn photo or not, and if not, the overturn detecting module passes the detection; a sign image recognition module for recognizing a sign image in the detected sign photo; the signboard character recognition module is used for recognizing characters in the signboard image; and the sign character comparison module is used for comparing whether characters in the sign image are consistent with corresponding sign characters or not, and if so, the sign image is checked. Further, the present application relates to an electronic device and a computer-readable storage medium.
Description
Technical Field
The application belongs to the technical field of sign photo auditing, and particularly relates to a sign photo auditing method, a sign photo auditing device, electronic equipment and a readable storage medium.
Background
When the e-commerce enterprise requires the registration of the merchant, uploading the signboard photo of the store where the merchant is located as qualification proof, and checking the uploaded signboard photo online by an auditor to judge whether the merchant meets the registration requirement.
When uploading a signboard photo, an electronic commerce enterprise can require that only the original camera of a mobile phone can be called, and the shooting place of the signboard photo must be consistent with the uploaded store position information, so that some common cheating means, such as uploading a turned-over photo, are avoided, misleading is easily caused to an auditor, and economic loss is caused to the enterprise by the electronic commerce.
At present, the auditing of the sign photo is mostly manual, the auditing speed is low, auditing personnel cannot audit in time after working, the auditing time of a merchant is prolonged, the merchant is easy to make a behavior of giving up registration after waiting for a long time, and an electronic commerce enterprise is caused to miss users.
The present application has been made in view of the above-described technical drawbacks.
It should be noted that the above disclosure of the background art is only for aiding in understanding the inventive concept and technical solution of the present application, which is not necessarily prior art to the present patent application, and should not be used for evaluating the novelty and creativity of the present application in the case where no clear evidence indicates that the above content has been disclosed at the filing date of the present application.
Disclosure of Invention
It is an object of the present application to provide a method, apparatus, electronic device and readable storage medium for verifying a sign photo, which overcome or alleviate at least one technical disadvantage of the known art.
The technical scheme of the application is as follows:
one aspect provides a sign photo auditing method, comprising:
and (3) a step of detecting the overturning: detecting whether the signboard photo is a turner photo or not, if not, detecting;
a signboard image recognition step: identifying a sign image in the detected sign photo;
A signboard character recognition step: identifying text in the sign image;
and a signboard character comparison step: and comparing whether the characters in the signboard images are consistent with the corresponding signboard characters, and if so, checking.
According to at least one embodiment of the present application, in the above method for auditing a signboard photo, the step of detecting a rollover is specifically:
collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
And (3) carrying out image graying on the signboard photo, detecting whether the processed signboard photo is a turnup photo based on a turnup detection model, and if not, passing the detection.
According to at least one embodiment of the present application, in the above method for auditing a signboard photo, the collecting a plurality of photographs and training a neural network model to obtain a convergence fitting photograph detection model specifically includes:
the collected turner photo is marked with characteristics;
Graying, special data augmentation and size normalization are carried out on the turner photo, and then the turner photo is divided into a training set, a testing set and a verification set;
Training the neural network model based on the turnup photos in the training set, testing based on the turnup photos in the testing set, and verifying through the turnup photos in the verification set to obtain a convergence fitting turnup detection model.
According to at least one embodiment of the present application, in the above method for auditing a signboard photo, the feature labeling of the collected flip-chip photo specifically includes:
and labeling the acquired flip photo with fuzzy characteristics, moire phenomenon characteristics, black edge characteristics, watermark characteristics and mobile phone frame characteristics.
According to at least one embodiment of the present application, in the above method for verifying a signboard photo, the turning photo is subjected to gray-scale processing, specifically:
graying treatment is carried out on the turnup photo according to a conversion formula G=pR+qG+wB, wherein,
G is a single channel of the gray scale map;
RGB is three channels of a color map;
P＝0.340～0.344，q＝0.505～0.509，w＝0.149～0.153，p+q+w＝1。
According to at least one embodiment of the present application, in the above method for verifying a signboard photo, special data augmentation processing is performed on the signboard photo, specifically:
the brightness of the feature marked flip photo is changed to carry out special data augmentation treatment, which can be to increase the brightness of the flip photo to 1.5 times of the original brightness and/or decrease the brightness of the flip photo to 0.7 times of the original brightness.
According to at least one embodiment of the present application, in the above method for auditing a signboard photo, the size normalization processing is performed on the flip-chip photo, specifically:
the size of the turnup photo is normalized to 227 x 227, if the size is too large, the model is difficult to train, if the size is too small, the turnup feature is lost too much, and the accuracy of model training is affected.
According to at least one embodiment of the present application, in the above method for auditing a signboard photo, the dividing and turning the signboard photo into a training set, a test set and a verification set is specifically as follows:
according to 8:1:1 to the training set, the test set and the verification set.
According to at least one embodiment of the present application, in the above method for auditing a signboard photo, the neural network model includes:
A first convolution layer, two basicblock modules and two full connection layers, each basicblock module consisting of 4 channels in parallel, the first channel comprising 3 convolution layers 3*3, the second channel comprising 2 convolution layers 3*3, the third channel comprising one convolution layer 3*3, the fourth channel being a identity module, a pooling layer being added after the first convolution layer and the two basicblock modules.
According to at least one embodiment of the present application, in the above-mentioned signboard photo auditing method, in the neural network model, a dropout layer for performing neuronal inactivation is accessed after a first full-connection layer, and a relu function is adopted as an activation function in the neural network model.
According to at least one embodiment of the present application, in the above-mentioned signboard photo auditing method, a batch normalization layer is provided before the activation function.
According to at least one embodiment of the present application, in the above-mentioned signboard photo auditing method, the detecting whether the processed signboard photo is a photograph to be flipped based on the photograph flipping detection model specifically includes:
And obtaining the turn-over probability of the processed signboard photo based on the turn-over detection model, if the turn-over probability is smaller than the detection threshold value, judging that the signboard photo is not the turn-over photo, otherwise, judging that the signboard photo is the turn-over photo.
According to at least one embodiment of the present application, in the above method for checking a sign photo, the step of identifying the sign image specifically includes:
collecting a plurality of signboard images, and training a yolo_v4 target detection algorithm model to obtain a convergence fitting signboard image recognition model;
The signboard image in the detected signboard photo is identified based on the signboard image identification model.
According to at least one embodiment of the present application, in the above method for auditing a signboard photo, the step of identifying a signboard text specifically includes:
collecting OCR data, training a DB text region detection algorithm and a CRNN text recognition algorithm model to obtain a convergence fitting signboard text recognition model;
words in the sign image are identified based on the sign word recognition model.
According to at least one embodiment of the present application, in the above method for checking a sign photo, the step of comparing the sign text specifically includes:
And comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, checking.
Another aspect provides a sign photo auditing apparatus, comprising:
the overturn detecting module is used for detecting whether the signboard photo is an overturn photo or not, and if not, the overturn detecting module passes the detection;
A sign image recognition module for recognizing a sign image in the detected sign photo;
the signboard character recognition module is used for recognizing characters in the signboard image;
And the sign character comparison module is used for comparing whether characters in the sign image are consistent with corresponding sign characters or not, and if so, the sign image is checked.
According to at least one embodiment of the present application, in the above-mentioned sign photo auditing device, the flip-photo detection module detects whether the sign photo is a flip-photo, and if not, the flip-photo detection module detects that, specifically,
Collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
And (3) carrying out image graying and special data augmentation processing on the signboard photo, detecting whether the processed signboard photo is a flipped photo based on a flipped photo detection model, and if not, passing the detection.
According to at least one embodiment of the present application, in the sign photo auditing apparatus, the sign image recognition module recognizes a sign image in a detected sign photo, specifically,
Collecting a plurality of signboard images, and training a yolo_v4 target detection algorithm model to obtain a convergence fitting signboard image recognition model;
The signboard image in the detected signboard photo is identified based on the signboard image identification model.
According to at least one embodiment of the present application, in the sign photo auditing apparatus, the sign text recognition module recognizes text in a sign image, specifically,
Collecting OCR data, training a DB text region detection algorithm and a CRNN text recognition algorithm model to obtain a convergence fitting signboard text recognition model;
words in the sign image are identified based on the sign word recognition model.
According to at least one embodiment of the present application, in the sign photo auditing apparatus, the sign text comparison module compares whether text in the sign image is consistent with corresponding sign text, and if so, the sign photo auditing apparatus, specifically,
And comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, checking.
A further aspect provides an electronic device, comprising:
A processor;
A memory storing a computer program configured to enable any of the above-described sign photo auditing methods when executed by the processor.
A further aspect provides a computer readable storage medium storing a computer program which when executed by a processor is capable of implementing any of the sign photo auditing methods described above.
Drawings
FIG. 1 is a flow chart of a method for auditing a sign photo provided by an embodiment of the present application;
fig. 2 is a schematic diagram of a sign photo auditing apparatus according to an embodiment of the present application.
For the purpose of better illustrating the embodiments, certain elements of the drawings may be omitted, enlarged or reduced and do not represent the actual product dimensions; further, the drawings are for illustrative purposes, wherein the terms describing the positional relationship are limited to the illustrative description only and are not to be construed as limiting the present patent.
Detailed Description
In order to make the technical solution of the present application and its advantages more clear, the technical solution of the present application will be further and completely described in detail with reference to the accompanying drawings, it being understood that the specific embodiments described herein are only some of the embodiments of the present application, which are for explanation of the present application and not for limitation of the present application. It should be noted that, for convenience of description, only the part related to the present application is shown in the drawings, and other related parts may refer to the general design, and the embodiments of the present application and the technical features of the embodiments may be combined with each other to obtain new embodiments without conflict.
Furthermore, unless defined otherwise, technical or scientific terms used in the description of the application should be given the ordinary meaning as understood by one of ordinary skill in the art to which the application pertains. The use of the terms "a," "an," or "the" and similar referents in the description of the application are not to be construed as limiting the amount absolutely, but rather as existence of at least one. As used in this description of the application, the terms "comprises," "comprising," or the like are intended to cover an element or article that appears before the term as such, but does not exclude other elements or articles from the list of elements or articles that appear after the term.
The application is described in further detail below with reference to fig. 1 to 2.
One aspect provides a sign photo auditing method, comprising:
and (3) a step of detecting the overturning: detecting whether the signboard photo is a turner photo or not, if not, detecting;
a signboard image recognition step: identifying a sign image in the detected sign photo;
A signboard character recognition step: identifying text in the sign image;
and a signboard character comparison step: and comparing whether the characters in the signboard images are consistent with the corresponding signboard characters, namely the signboard names, and if so, checking.
As for the signboard photo auditing method disclosed by the embodiment, as can be understood by those skilled in the art, the method can be used for realizing automatic auditing of uploading photos to merchants by electronic commerce industry, automatically detecting whether the signboard photo is a flip photo, identifying the signboard image in the detected signboard photo, identifying the characters in the signboard image, comparing whether the characters in the signboard image are consistent with the corresponding signboard characters, and completing auditing of the signboard photo with higher efficiency and accuracy.
In some optional embodiments, in the above method for verifying a signboard photo, the step of detecting a rollover includes:
collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
And (3) carrying out image graying and special data augmentation processing on the signboard photo, detecting whether the processed signboard photo is a flipped photo based on a flipped photo detection model, and if not, passing the detection.
For the signboard photo auditing method disclosed in the above embodiment, it can be understood by those skilled in the art that when detecting whether the signboard photo is a photograph to be flipped, the signboard photo is preprocessed by using image graying and special data augmentation to improve the detection accuracy of whether the signboard photo is a photograph to be flipped, and then the neural network model is trained based on a plurality of acquired photographs to obtain a convergence fitting flip detection model, so that the processed signboard photo is detected whether the processed signboard photo is a photograph to be flipped, thereby having higher detection accuracy.
In some optional embodiments, in the above method for auditing a signboard photo, the collecting a plurality of photographs and training a neural network model to obtain a convergence fit photograph detection model specifically includes:
the collected turner photo is marked with characteristics;
Graying, special data augmentation and size normalization are carried out on the turner photo, and then the turner photo is divided into a training set, a testing set and a verification set;
Training the neural network model based on the turnup photos in the training set, testing based on the turnup photos in the testing set, and verifying through the turnup photos in the verification set to obtain a convergence fitting turnup detection model.
In some optional embodiments, in the above method for auditing a signboard photo, the feature labeling of the collected flip-chip photo specifically includes:
and labeling the acquired flip photo with fuzzy characteristics, moire phenomenon characteristics, black edge characteristics, watermark characteristics and mobile phone frame characteristics.
For the method for auditing the signboard photo disclosed in the above embodiment, it can be understood by those skilled in the art that the blur characteristic, the moire phenomenon characteristic, the black edge characteristic, the watermark characteristic and the mobile phone frame characteristic may exist in the flip photo, and the collected flip photo is marked with the characteristics, so that a special data set for detecting the flip photo can be constructed.
In some optional embodiments, in the above method for verifying a signboard photo, the subjecting the flip-chip photo to grayscale processing specifically includes:
graying treatment is carried out on the turnup photo according to a conversion formula G=pR+qG+wB, wherein,
G is a single channel of the gray scale map;
RGB is three channels of a color map;
P＝0.340～0.344，q＝0.505～0.509，w＝0.149～0.153，p+q+w＝1。
for the sign photo auditing method disclosed in the above embodiment, it can be understood by those skilled in the art that the feature gap between the real photo and the flip photo is not related to the color information, and the presence of the color information may introduce noise to affect the detection of the flip photo, and the flip photo is subjected to gray processing according to the above conversion formula, so that the noise generated by the color information can be effectively reduced.
In some optional embodiments, in the above method for verifying a signboard photo, special data augmentation processing is performed on the flip-chip photo, specifically:
the brightness of the feature marked flip photo is changed to carry out special data augmentation treatment, which can be to increase the brightness of the flip photo to 1.5 times of the original brightness and/or decrease the brightness of the flip photo to 0.7 times of the original brightness.
As for the sign photo auditing method disclosed in the above embodiment, it can be understood by those skilled in the art that part of the real photo may be taken in a strong light environment or a night environment, and the part of the photo occupies a relatively low area but has a relatively large influence on classification results, so that the accuracy can be effectively improved by changing the brightness of the photo to simulate the strong light environment and the night environment, the brightness of the photo can be increased to 1.5 times of the original brightness and/or the brightness of the photo can be reduced to 0.7 times of the original brightness, and the data set can be expanded to 3 times of the original brightness, based on which the accuracy of the model can be correspondingly improved.
In some optional embodiments, in the above method for verifying a signboard photo, the size normalization processing is performed on the flip-chip photo, specifically:
the size of the turnup photo is normalized to 227 x 227, if the size is too large, the model is difficult to train, if the size is too small, the turnup feature is lost too much, and the accuracy of model training is affected.
In some optional embodiments, in the above method for auditing a signboard photo, the dividing and turning the signboard photo into a training set, a test set and a verification set is specifically as follows:
according to 8:1:1 to the training set, the test set and the verification set.
For the signboard photo auditing method disclosed in the above embodiment, it can be understood by those skilled in the art that the flip photos in the training set are used for the data samples of model fitting; the verification centralized photograph taking is a sample set independently reserved in the model training process, is used for adjusting the hyper-parameters of the model, is used for carrying out preliminary evaluation on the capacity of the model, is used for searching the optimal network depth of the neural network model, or is used for determining a stopping point of a back propagation algorithm or selecting the number of hidden layer neurons in the neural network; the photographs taken in the test set can be used to evaluate the generalization ability of the model.
In some optional embodiments, in the above method for auditing a signboard photo, the neural network model includes: 1 convolutional layer, two basicblock and two fully-connected layers. Wherein Basicblock consists of 4 channels in parallel, the first channel comprising 3 convolutional layers 3*3 instead of one 7*7, the second channel comprising 2 convolutional layers 3*3 instead of one 5*5, the third channel comprising one 3*3, the fourth channel being a identity module; a pooling layer is added after the first layer convolution and two basicblock. After repeated verification of experiments by using the self-built flip photo data set, the network depth is designed at 9 layers, the recognition effect on the model is not improved when the depth and the width of the network are increased again after 9 layers, and meanwhile, unnecessary calculation amount is increased.
In some alternative embodiments, in the above method for auditing a sign photo, the number of convolution kernels in each layer is 64, 128, 256; each pooling layer adopts a mode of maximum pooling, the pooling size is 2 x 2, and the step length is 2; the number of output neurons of the fully connected layer is 128 and 2 respectively, wherein 2 is the class number of output softmax, namely the two classes of the flip and the normal.
In some optional embodiments, in the above-mentioned method for checking and examining the signboard photo, a dropout layer is connected after the first full-connection layer to improve generalization of the model, so as to avoid overfitting, the dropout layer can randomly deactivate part of neurons in the network during training, and corresponding weights can not participate in forward and backward propagation during a single training process, so that the neural network model can learn general commonality, and overfitting is avoided. The draft parameter of dropout may be set to 0.5, i.e., the probability of deactivation of each neuron is 0.5.
In some optional embodiments, in the above method for auditing a signboard photo, the activation function in the neural network model may be relu functions, and compared with the conventional sigmoid and tanh activation functions, the relu activation function has better performance, and can effectively avoid gradient extinction and gradient explosion, and the calculation formula is as follows:
f(x)＝max(0，x)。
In some alternative embodiments, in the above-mentioned method for checking and examining the pictures of the signboards, a batch normalization layer is added after each convolution layer and the first full-connection layer to speed up model training and improve accuracy. The batch standardization can enable the input value of the nonlinear transformation to fall into a region where the input is sensitive through a standardization means, gradient explosion is avoided, data distribution is consistent, and the model convergence speed is increased. The batch normalization layer is usually placed behind relu when designing the network, but the application finds that the batch normalization layer is placed in front of relu when training and identifying the flap test through a large number of tests and analysis and summary of experimental data.
In some alternative embodiments, in the above-described sign photo auditing method, the two parameters momentum and epsilon of the batch normalization layer may be set to 0.99 and 0.001, respectively.
In some alternative embodiments, in the above-mentioned method for verifying the sign photo, the neural network model may use a cross-validation method to select the final super parameters during training, including the initial learning rate, the learning rate decay interval, batchsize, etc. The accuracy of the model on the verification set separated in advance is used for measuring the performance of the model and the generalization capability of the model, when the model loss is converged, training is stopped when the accuracy is not improved, and the converged model is used as a final-use flap detection model.
In some optional embodiments, in the above method for checking a signboard photo, the detecting whether the processed signboard photo is a rollover photo based on a rollover detection model specifically includes:
and obtaining the turn-over probability of the processed signboard photo based on the turn-over detection model, if the turn-over probability is smaller than the detection threshold value, judging that the signboard photo is not the turn-over photo, otherwise, judging that the signboard photo is the turn-over photo. The size of the detection threshold may be specifically determined by a skilled person according to a specific practical implementation, and may be 50%, for example.
In some optional embodiments, in the above method for checking and examining a sign photo, the step of identifying a sign image specifically includes:
collecting a plurality of signboard images, and training a yolo_v4 target detection algorithm model to obtain a convergence fitting signboard image recognition model;
The signboard image in the detected signboard photo is identified based on the signboard image identification model.
As for the sign photo auditing method disclosed in the above embodiment, it can be understood by those skilled in the art that the sign image recognition model which is converged and fitted is obtained by training the yolo_v4 target detection algorithm model based on a plurality of acquired sign images, so that the sign images in the detected sign photo are recognized, and the recognition accuracy is high.
In some optional embodiments, in the above method for checking and examining a sign photo, the step of identifying a sign text specifically includes:
collecting OCR data, training a DB text region detection algorithm and a CRNN text recognition algorithm model to obtain a convergence fitting signboard text recognition model;
words in the sign image are identified based on the sign word recognition model.
For the signboard photo auditing method disclosed in the above embodiment, it can be understood by those skilled in the art that the collected OCR data is used to train the DB text region detection algorithm and the CRNN text recognition algorithm model to obtain a convergence fit signboard text recognition model, so as to recognize the text in the signboard image, and the recognition accuracy is higher.
In some optional embodiments, in the above method for checking a sign photo, the step of comparing a sign text specifically includes:
And comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold, checking if the specific value is 40%.
As for the sign photo auditing method disclosed in the above embodiment, it can be understood by those skilled in the art that the sign words, that is, the name of the sign, may be uploaded by the merchant at the time of registration, and by comparing the similarity between the words in the sign image and the corresponding sign words, it is determined whether the words in the sign image in the sign photo are consistent with the corresponding sign words, and when the words are consistent, that is, the similarity exceeds a set threshold, the sign photo is audited.
The method for auditing the signboards and the photos disclosed by the embodiment can adopt the following expression:
Wherein,
U is the text content in the sign image;
P represents the character content of the sign;
P n U represents the repeated text content between the two;
d represents the number of words of the text.
With respect to the method for checking the sign photo disclosed in the above embodiment, it will be further understood by those skilled in the art that the specific size of the threshold value may be determined by the relevant person according to the specific practice when the present application is applied, and the factors mainly considered may include the accuracy of identifying the sign image in the detected sign photo and the accuracy of identifying the text in the sign image.
Another aspect provides a sign photo auditing apparatus, comprising:
the overturn detecting module is used for detecting whether the signboard photo is an overturn photo or not, and if not, the overturn detecting module passes the detection;
A sign image recognition module for recognizing a sign image in the detected sign photo;
the signboard character recognition module is used for recognizing characters in the signboard image;
And the sign character comparison module is used for comparing whether characters in the sign image are consistent with corresponding sign characters or not, and if so, the sign image is checked.
For the signboard photo auditing device disclosed in the above embodiment, it can be understood by those skilled in the art that the device can be used for implementing automatic auditing of uploading photos to merchants by electronic commerce industry, automatically detecting whether the signboard photo is a flip photo, identifying the signboard image in the detected signboard photo, identifying the characters in the signboard image, and comparing whether the characters in the signboard image are consistent with the corresponding signboard characters, thereby completing auditing of the signboard photo with higher efficiency and accuracy.
In some optional embodiments, in the above sign photo auditing apparatus, the flip detection module detects whether the sign photo is a flip photo, and if not, the flip detection module detects that, specifically,
Collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
And (3) carrying out image graying and special data augmentation processing on the signboard photo, detecting whether the processed signboard photo is a flipped photo based on a flipped photo detection model, and if not, passing the detection.
For the signboard photo auditing device disclosed in the above embodiment, it can be understood by those skilled in the art that when detecting whether the signboard photo is a photograph to be turned over, the signboard photo is preprocessed by using image graying and special data augmentation to improve the detection accuracy of whether the signboard photo is a photograph to be turned over, and then the neural network model is trained to obtain a convergence fit photograph turning-over detection model based on a plurality of collected photograph turning-over photos, so that whether the processed signboard photo is a photograph to be turned over is detected, and the detection accuracy is higher.
In some optional embodiments, in the above sign photo auditing apparatus, the sign image recognition module recognizes a sign image in the detected sign photo, specifically,
Collecting a plurality of signboard images, and training a yolo_v4 target detection algorithm model to obtain a convergence fitting signboard image recognition model;
The signboard image in the detected signboard photo is identified based on the signboard image identification model.
As for the sign photo auditing device disclosed in the above embodiment, it can be understood by those skilled in the art that the sign image recognition model which is converged and fitted is obtained by training the yolo_v4 target detection algorithm model based on a plurality of acquired sign images, so that the sign images in the detected sign photo are recognized, and the recognition accuracy is high.
In some optional embodiments, in the above sign photo auditing apparatus, the sign text recognition module recognizes text in the sign image, specifically,
Collecting OCR data, training a DB text region detection algorithm and a CRNN text recognition algorithm model to obtain a convergence fitting signboard text recognition model;
words in the sign image are identified based on the sign word recognition model.
As for the sign photo auditing device disclosed in the above embodiment, it can be understood by those skilled in the art that the device trains the DB text region detection algorithm and the CRNN text recognition algorithm model based on the collected OCR data to obtain a convergence fit sign text recognition model, so as to recognize the text in the sign image, and has higher recognition accuracy.
In some optional embodiments, in the above-mentioned sign photo auditing apparatus, the sign text comparison module compares whether the text in the sign image is consistent with the corresponding sign text, and if so, the sign text is audited, specifically,
And comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, checking if the specific numerical value can be 40%.
As for the sign photo auditing device disclosed in the above embodiment, it is understood by those skilled in the art that the sign words, that is, the name of the sign, may be uploaded by the merchant at the time of registration, and by comparing the similarity between the words in the sign image and the corresponding sign words, it is determined whether the words in the sign image in the sign photo are identical to the corresponding sign words, and when the words are identical, that is, the similarity exceeds a set threshold, the sign photo is audited.
The method for auditing the signboards and the photos disclosed by the embodiment can adopt the following expression:
Wherein,
U is the text content in the sign image;
P represents the character content of the sign;
P n U represents the repeated text content between the two;
d represents the number of words of the text.
With respect to the sign photo auditing apparatus disclosed in the above embodiment, it will also be understood by those skilled in the art that the specific size of the threshold value may be determined by the relevant person according to specific practice when applying the present application, and the factors mainly considered may include the accuracy of identifying the sign image in the detected sign photo, and the accuracy of identifying the text in the sign image.
A further aspect provides an electronic device, comprising:
A processor;
A memory storing a computer program configured to enable any of the above-described sign photo auditing methods when executed by the processor.
A further aspect provides a computer readable storage medium storing a computer program which when executed by a processor is capable of implementing any of the sign photo auditing methods described above.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred.
For the signboard photo auditing device disclosed in the above embodiment, because it corresponds to the signboard photo auditing method disclosed in the above embodiment, the description is simpler, the specific relevant points can be described by referring to the signboard photo auditing method part, and the technical effects can also be referred to the technical effects of the signboard photo auditing method part, and are not described herein again.
Moreover, those skilled in the art should appreciate that the various modules and units of the disclosed apparatus can be implemented in electronic hardware, computer software, or combinations of both, and that the application is generally described in terms of functions, whether implemented in hardware or software, depending on the specific application and design constraints of the solution, those skilled in the art can choose to implement the described functions in different ways for each specific application and its practical constraints, but such implementation should not be considered to be beyond the scope of the present application.
A further aspect provides an electronic device, comprising:
A processor;
A memory storing a computer program configured to enable any of the above-described sign photo auditing methods when executed by the processor.
In some alternative embodiments, the processor may be a central processing unit CPU or other form of processing unit having data processing and/or instruction execution capabilities, may be a general purpose or special purpose processor, and may control other components in the compensation electronics to perform the desired functions.
In some alternative embodiments, the memory may include various forms of computer-readable storage media, such as volatile memory, which may be random access memory, RAM, and/or cache memory, and/or nonvolatile memory, which may be read only memory, ROM, hard disk, flash memory, etc. The memory may have stored thereon a computer program that, when executed by the processor, may perform the functions of embodiments of the present application and/or other desired functions, and may further have stored thereon various application programs and various data.
In some alternative embodiments, the processor and memory may be connected by a bus system, which may be a serial, parallel communication bus, or the like.
It should be noted that, for clarity and brevity, not all of the constituent units of the electronic device are shown in the above embodiments, and those skilled in the art may provide and set other constituent units not shown according to specific needs to implement the necessary functions of the electronic device.
For the electronic device disclosed in the above embodiment, since the processor may implement any of the above-mentioned sign photo auditing methods when executing the computer program stored in the memory thereof, the technical effects thereof may be referred to the technical effects of the above-mentioned sign photo auditing method section correspondingly, and will not be described herein in detail.
A further aspect provides a computer readable storage medium storing a computer program which when executed by a processor is capable of implementing any of the sign photo auditing methods described above.
In alternative embodiments, the computer readable storage medium may comprise a memory card of a smart phone, a memory component of a tablet computer, a hard disk of a personal computer, a random access memory RAM, a read-only memory ROM, an erasable programmable read-only memory EPROM, a portable compact disc read-only memory CD-ROM, a flash memory, or any combination of the foregoing storage media, as well as other suitable storage media.
Having thus described the technical aspects of the present application with reference to the preferred embodiments shown in the drawings, it should be understood by those skilled in the art that the scope of the present application is not limited to the specific embodiments, and those skilled in the art may make equivalent changes or substitutions to the related technical features without departing from the principle of the present application, and those changes or substitutions will fall within the scope of the present application.
Claims (9)
1. A sign photo auditing method, comprising:
and (3) a step of detecting the overturning: detecting whether the signboard photo is a turner photo or not, if not, detecting;
a signboard image recognition step: identifying a sign image in the detected sign photo;
A signboard character recognition step: identifying text in the sign image;
And a signboard character comparison step: comparing whether the characters in the signboard images are consistent with the corresponding signboard characters, if so, checking;
The step of detecting the flap comprises the following steps:
collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
performing image graying and special data augmentation treatment on the signboard photo, detecting whether the treated signboard photo is a turnup photo or not based on a turnup detection model, and if not, detecting;
the special data augmentation treatment is carried out on the turner photo, and the specific steps are as follows:
special data augmentation treatment is carried out by changing the brightness of the turned-over photo, so that the brightness of the turned-over photo is increased to 1.5 times of the original brightness or the brightness of the turned-over photo is reduced to 0.7 times of the original brightness;
The neural network model comprises 1 convolution layer, two basicblock and two full connection layers; wherein Basicblock consists of 4 channels in parallel, the first channel comprises 3 convolutional layers of 3*3, the second channel comprises 2 convolutional layers of 3*3, the third channel comprises a 3*3 convolutional layer, and the fourth channel is identity module; adding a pooling layer after the first layer convolution and two basicblock;
The depth of the neural network model network is 9 layers;
in the neural network model, the number of convolution kernels of each layer is 64, 128, 256 and 256 respectively, each pooling layer adopts a maximum pooling mode, the pooling size is 2 x2, the step length is 2, the number of output neurons of the full-connection layer is 128 and 2 respectively, and 2 is the class number of output softmax;
in the neural network model, a dropout layer is connected behind a first full-connection layer;
The activation function in the neural network model is relu functions;
And adding a batch normalization layer after each convolution layer and the first full-connection layer in the neural network model, wherein the batch normalization layer is arranged in front of relu functions.
2. The method of verifying a sign photo of claim 1,
The signboard image identification step specifically comprises the following steps:
collecting a plurality of signboard images, and training a yolo_v4 target detection algorithm model to obtain a convergence fitting signboard image recognition model;
The signboard image in the detected signboard photo is identified based on the signboard image identification model.
3. The method of verifying a sign photo of claim 1,
The signboard character recognition step specifically comprises the following steps:
collecting OCR data, training a DB text region detection algorithm and a CRNN text recognition algorithm model to obtain a convergence fitting signboard text recognition model;
words in the sign image are identified based on the sign word recognition model.
4. The method of verifying a sign photo of claim 1,
The signboard character comparison step specifically comprises the following steps:
And comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, checking.
5. A sign photo auditing apparatus, comprising:
the overturn detecting module is used for detecting whether the signboard photo is an overturn photo or not, and if not, the overturn detecting module passes the detection;
A sign image recognition module for recognizing a sign image in the detected sign photo;
the signboard character recognition module is used for recognizing characters in the signboard image;
The sign character comparison module is used for comparing whether characters in the sign image are consistent with corresponding sign characters or not, and if so, the sign image is checked;
in the overturn detection, a plurality of overturn photo are collected, and training is carried out on a neural network model to obtain a convergence fitting overturn detection model;
performing image graying and special data augmentation treatment on the signboard photo, detecting whether the treated signboard photo is a turnup photo or not based on a turnup detection model, and if not, detecting;
the special data augmentation treatment is carried out on the turner photo, and the specific steps are as follows:
special data augmentation treatment is carried out by changing the brightness of the turned-over photo, so that the brightness of the turned-over photo is increased to 1.5 times of the original brightness or the brightness of the turned-over photo is reduced to 0.7 times of the original brightness;
The neural network model comprises 1 convolution layer, two basicblock and two full connection layers; wherein Basicblock consists of 4 channels in parallel, the first channel comprises 3 convolutional layers of 3*3, the second channel comprises 2 convolutional layers of 3*3, the third channel comprises a 3*3 convolutional layer, and the fourth channel is identity module; adding a pooling layer after the first layer convolution and two basicblock;
The depth of the neural network model network is 9 layers;
in the neural network model, the number of convolution kernels of each layer is 64, 128, 256 and 256 respectively, each pooling layer adopts a maximum pooling mode, the pooling size is 2 x2, the step length is 2, the number of output neurons of the full-connection layer is 128 and 2 respectively, and 2 is the class number of output softmax;
in the neural network model, a dropout layer is connected behind a first full-connection layer;
The activation function in the neural network model is relu functions;
And adding a batch normalization layer after each convolution layer and the first full-connection layer in the neural network model, wherein the batch normalization layer is arranged in front of relu functions.
6. The sign photo auditing device of claim 5,
In the signboard image recognition module, the signboard images in the detected signboard photos are recognized, specifically,
Collecting a plurality of signboard images, and training a yolo_v4 target detection algorithm model to obtain a convergence fitting signboard image recognition model;
The signboard image in the detected signboard photo is identified based on the signboard image identification model.
7. The sign photo auditing device of claim 5,
In the signboard character recognition module, characters in the signboard image are recognized, specifically,
Collecting OCR data, training a DB text region detection algorithm and a CRNN text recognition algorithm model to obtain a convergence fitting signboard text recognition model;
words in the sign image are identified based on the sign word recognition model.
8. An electronic device, comprising:
A processor;
A memory storing a computer program configured to enable the sign photo auditing method of any of claims 1-4 when executed by the processor.
9. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, is capable of implementing the sign photo auditing method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011145385.2A CN112257709B (en) | 2020-10-23 | 2020-10-23 | Signboard photo auditing method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011145385.2A CN112257709B (en) | 2020-10-23 | 2020-10-23 | Signboard photo auditing method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112257709A CN112257709A (en) | 2021-01-22 |
| CN112257709B true CN112257709B (en) | 2024-05-07 |
Family
ID=74263610
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011145385.2A Active CN112257709B (en) | 2020-10-23 | 2020-10-23 | Signboard photo auditing method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112257709B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113486715A (en) * | 2021-06-04 | 2021-10-08 | 广州图匠数据科技有限公司 | Image reproduction identification method, intelligent terminal and computer storage medium |
| CN113435437A (en) * | 2021-06-24 | 2021-09-24 | 随锐科技集团股份有限公司 | Method and device for identifying state of switch on/off indicator and storage medium |
| CN116958795A (en) * | 2023-06-30 | 2023-10-27 | 北京房多多信息技术有限公司 | Method and device for identifying flip image, electronic equipment and storage medium |
| CN117315415B (en) * | 2023-07-13 | 2024-06-11 | 东南大学 | Shop sign health performance evaluation method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558794A (en) * | 2018-10-17 | 2019-04-02 | 平安科技（深圳）有限公司 | Image-recognizing method, device, equipment and storage medium based on moire fringes |
| CN109726743A (en) * | 2018-12-12 | 2019-05-07 | 苏州大学 | A kind of retina OCT image classification method based on Three dimensional convolution neural network |
| CN109886275A (en) * | 2019-01-16 | 2019-06-14 | 深圳壹账通智能科技有限公司 | Reproduction image-recognizing method, device, computer equipment and storage medium |
| CN110046644A (en) * | 2019-02-26 | 2019-07-23 | 阿里巴巴集团控股有限公司 | A kind of method and device of certificate false proof calculates equipment and storage medium |
| CN110349033A (en) * | 2019-05-30 | 2019-10-18 | 阿里巴巴集团控股有限公司 | Project case Methods of initial evaluation of sci-tech and device |
| CN110458802A (en) * | 2019-06-28 | 2019-11-15 | 天津大学 | Based on the projection normalized stereo image quality evaluation method of weight |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10546231B2 (en) * | 2017-01-23 | 2020-01-28 | Fotonation Limited | Method for synthesizing a neural network |
-
2020
- 2020-10-23 CN CN202011145385.2A patent/CN112257709B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558794A (en) * | 2018-10-17 | 2019-04-02 | 平安科技（深圳）有限公司 | Image-recognizing method, device, equipment and storage medium based on moire fringes |
| CN109726743A (en) * | 2018-12-12 | 2019-05-07 | 苏州大学 | A kind of retina OCT image classification method based on Three dimensional convolution neural network |
| CN109886275A (en) * | 2019-01-16 | 2019-06-14 | 深圳壹账通智能科技有限公司 | Reproduction image-recognizing method, device, computer equipment and storage medium |
| CN110046644A (en) * | 2019-02-26 | 2019-07-23 | 阿里巴巴集团控股有限公司 | A kind of method and device of certificate false proof calculates equipment and storage medium |
| CN110349033A (en) * | 2019-05-30 | 2019-10-18 | 阿里巴巴集团控股有限公司 | Project case Methods of initial evaluation of sci-tech and device |
| CN110458802A (en) * | 2019-06-28 | 2019-11-15 | 天津大学 | Based on the projection normalized stereo image quality evaluation method of weight |
Non-Patent Citations (2)
| Title |
|---|
| Image Recapture Detection with Convolutional and Recurrent Neural Networks;Haoliang Li等;IS&T International Symposium on Electronic Imaging 2017;20171231;第87-91页 * |
| 基于并行残差卷积神经网络的多种树叶分类;魏书伟;曾上游;周悦;王新娇;;现代电子技术;20200501(第09期);第96-100页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112257709A (en) | 2021-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112257709B (en) | Signboard photo auditing method and device, electronic equipment and readable storage medium | |
| Rao et al. | A deep learning approach to detection of splicing and copy-move forgeries in images | |
| Bayar et al. | Design principles of convolutional neural networks for multimedia forensics | |
| CN106803069B (en) | Crowd happiness degree identification method based on deep learning | |
| Cheng et al. | Enhanced convolutional neural network for plankton identification and enumeration | |
| Rantzsch et al. | Signature embedding: Writer independent offline signature verification with deep metric learning | |
| CN107133622A (en) | The dividing method and device of a kind of word | |
| CN110532855A (en) | Natural scene certificate image character recognition method based on deep learning | |
| Althafiri et al. | Bahraini paper currency recognition | |
| CN108323209A (en) | Information processing method, system, cloud processing device and computer program product | |
| CN113111880A (en) | Certificate image correction method and device, electronic equipment and storage medium | |
| Jhang et al. | CNN training for face photo based gender and age group prediction with camera | |
| Wencheng et al. | Offline Chinese signature verification based on alexnet | |
| CN112861743A (en) | Palm vein image anti-counterfeiting method, device and equipment | |
| Gaddam et al. | On Sudoku problem using deep learning and image processing technique | |
| CN111242114A (en) | Character recognition method and device | |
| Sharma et al. | Copy and Move Forged Image Detection by Deep Learning | |
| Rasheed et al. | A Novel Method for Signature Verification Using Deep Learning | |
| CN112329634A (en) | Classroom behavior recognition method and device, electronic equipment and storage medium | |
| Fadl et al. | Automatic fake document identification and localization using DE-Net and color-based features of foreign inks | |
| Sundravadivelu et al. | Extensive Analysis of IoT Assisted Fake Currency Detection using Novel Learning Scheme | |
| Yu et al. | Two strategies to optimize the decisions in signature verification with the presence of spoofing attacks | |
| Shukla et al. | Artificial Intelligence Approach for Signature Detection | |
| Wang et al. | A Multi-scale Domain Adaptive Framework for Scene Text Detection | |
| Muttreja et al. | Indian Currency Classification and Counterfeit Detection Using Deep Learning and Image Processing Approach |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |