CN112257709A - Signboard photo auditing method and device, electronic equipment and readable storage medium - Google Patents
Signboard photo auditing method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN112257709A CN112257709A CN202011145385.2A CN202011145385A CN112257709A CN 112257709 A CN112257709 A CN 112257709A CN 202011145385 A CN202011145385 A CN 202011145385A CN 112257709 A CN112257709 A CN 112257709A
- Authority
- CN
- China
- Prior art keywords
- signboard
- photo
- image
- characters
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 238000001514 detection method Methods 0.000 claims abstract description 67
- 238000012795 verification Methods 0.000 claims abstract description 26
- 238000012552 review Methods 0.000 claims abstract description 25
- 238000012549 training Methods 0.000 claims description 35
- 238000004422 calculation algorithm Methods 0.000 claims description 25
- 238000003062 neural network model Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 16
- 238000013434 data augmentation Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 102100032202 Cornulin Human genes 0.000 claims description 8
- 101000920981 Homo sapiens Cornulin Proteins 0.000 claims description 8
- 230000006870 function Effects 0.000 description 13
- 238000012360 testing method Methods 0.000 description 11
- 238000010606 normalization Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 230000004913 activation Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000011176 pooling Methods 0.000 description 5
- 238000002372 labelling Methods 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000004880 explosion Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002779 inactivation Effects 0.000 description 1
- 210000004205 output neuron Anatomy 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a signboard photo auditing method, which comprises the following steps: and (3) copying and detecting: detecting whether the signboard photo is a reproduction photo, and if not, passing the detection; signboard image recognition: identifying a signboard image in the detected signboard photo; signboard character recognition: recognizing characters in the signboard image; signboard text comparison: and comparing whether the characters in the signboard image are consistent with the corresponding signboard characters, and if so, passing the verification. Furthermore, it relates to a signboard photo review device comprising: the copying detection module is used for detecting whether the signboard photo is a copied photo or not, and if not, the signboard photo passes the detection; a signboard image recognition module for recognizing a signboard image in the detected signboard photo; the signboard character recognition module is used for recognizing characters in the signboard image; and the signboard character comparison module is used for comparing whether the characters in the signboard image are consistent with the corresponding signboard characters or not, and if so, the verification is passed. Furthermore, the present invention relates to an electronic device and a computer-readable storage medium.
Description
Technical Field
The application belongs to the technical field of signboard photo auditing, and particularly relates to a signboard photo auditing method and device, electronic equipment and a readable storage medium.
Background
When the electronic commerce and enterprise requires the merchant to register, the signboard photo of the store where the merchant is located is uploaded to serve as a qualification certificate, and an auditor audits the uploaded signboard photo on line to judge whether the merchant meets the registration requirement.
When the signboard photo is uploaded usually, e-commerce enterprises can require that only the original camera of the mobile phone can be called and the shooting place of the signboard photo is required to be consistent with the uploaded store position information, so as to avoid some common cheating means, such as uploading and copying the photo, which easily cause misleading to auditors and bring economic loss to the enterprises.
At present, the examination and verification of the signboard photos are mostly manually examined and verified, the examination and verification speed is low, the examination and verification cannot be timely carried out by an auditor after work, the waiting and verification time of a merchant is prolonged, and the merchant easily gives up registration after waiting for a long time, so that the user of the e-commerce enterprise is missed.
The present application has been made in view of the above-mentioned technical drawbacks.
It should be noted that the above background disclosure is only for the purpose of assisting understanding of the inventive concept and technical solutions of the present invention, and does not necessarily belong to the prior art of the present patent application, and the above background disclosure should not be used for evaluating the novelty and inventive step of the present application without explicit evidence to suggest that the above content is already disclosed at the filing date of the present application.
Disclosure of Invention
An object of the present application is to provide a signboard photo review method, apparatus, electronic device and readable storage medium, so as to overcome or alleviate technical shortcomings of at least one aspect known to exist.
The technical scheme of the application is as follows:
one aspect provides a signboard photo auditing method, which comprises the following steps:
and (3) copying and detecting: detecting whether the signboard photo is a reproduction photo, and if not, passing the detection;
signboard image recognition: identifying a signboard image in the detected signboard photo;
signboard character recognition: recognizing characters in the signboard image;
signboard text comparison: and comparing whether the characters in the signboard image are consistent with the corresponding signboard characters, and if so, passing the verification.
According to at least one embodiment of the present application, in the method for reviewing a signboard photo, the step of detecting the reproduction includes:
collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
and carrying out image graying on the signboard photo, detecting whether the processed signboard photo is the copied photo or not based on the copying detection model, and if not, passing the detection.
According to at least one embodiment of the present application, in the signboard photo auditing method, the acquiring of the plurality of reproduction photos trains the neural network model to obtain a convergence fitting reproduction detection model specifically includes:
carrying out feature labeling on the collected copied photos;
graying, special data augmentation processing and size normalization processing are carried out on the shot photos, and then the shot photos are divided into a training set, a test set and a verification set;
training the neural network model based on the reproduction photos in the training set, testing based on the reproduction photos in the testing set, and verifying through the reproduction photos in the verification set to obtain a reproduction detection model with convergence fitting.
According to at least one embodiment of the present application, in the above signboard photo review method, the characteristic labeling of the captured copied photo specifically includes:
and carrying out fuzzy characteristic marking, moire phenomenon characteristic marking, black edge characteristic marking, watermark characteristic marking and mobile phone frame characteristic marking on the collected copied photos.
According to at least one embodiment of the present application, in the above signboard photo review method, the graying processing is performed on the shot photo, specifically:
graying the photographed picture according to a conversion formula G ═ pR + qG + wB, wherein,
g is a single channel of the gray scale map;
RGB is three channels of a color picture;
P＝0.340～0.344，q＝0.505～0.509，w＝0.149～0.153，p+q+w＝1。
according to at least one embodiment of the present application, in the above signboard photo auditing method, the special data augmentation process is performed on the photo that is shot, specifically:
the brightness of the copied photo with the characteristic labels is changed to carry out special data augmentation processing, and the brightness of the copied photo can be increased to 1.5 times of the original brightness and/or the brightness of the copied photo can be reduced to 0.7 times of the original brightness.
According to at least one embodiment of the present application, in the method for examining and reviewing signboard photos, the size normalization processing is performed on the captured photos, specifically:
the sizes of the copied photos are normalized to 227 x 227, if the sizes of the copied photos are too large, the model is difficult to train, and if the sizes of the copied photos are too small, the copied characters are lost too much, and the accuracy of model training is influenced.
According to at least one embodiment of the present application, in the signboard photo auditing method, the dividing of the copied photos into a training set, a test set and a verification set specifically includes:
according to the following steps of 8: 1: 1, dividing the reproduced photos into a training set, a testing set and a verification set.
According to at least one embodiment of the present application, in the signboard photo review method, the neural network model includes:
a first convolutional layer, two basic block modules and two fully connected layers, each basic block module is composed of 4 channels in parallel, the first channel comprises 3 convolutional layers of 3 x 3, the second channel comprises 2 convolutional layers of 3 x 3, the third channel comprises a convolutional layer of 3 x 3, the fourth channel is an identity module, and a pooling layer is added after the first convolutional layer and the two basic block modules.
According to at least one embodiment of the application, in the signboard photo auditing method, a dropout layer for neuron inactivation is connected after a first full-link layer in the neural network model, and a relu function is used as an activation function in the neural network model.
According to at least one embodiment of the application, in the signboard photo auditing method, a batch standardization layer is arranged before the activation function.
According to at least one embodiment of the present application, in the method for examining and verifying a signboard photo, the detecting whether the processed signboard photo is a copied photo based on the copy detection model specifically includes:
and obtaining the reproduction probability of the processed signboard photo based on the reproduction detection model, if the reproduction probability is smaller than a detection threshold value, judging that the signboard photo is not the reproduced photo, and otherwise, judging that the signboard photo is the reproduced photo.
According to at least one embodiment of the present application, in the signboard picture auditing method, the signboard image recognition step specifically includes:
collecting a plurality of signboard images, training a YOLO _ v4 target detection algorithm model, and obtaining a convergent fitting signboard image recognition model;
the sign image in the detected sign photo is identified based on a sign image recognition model.
According to at least one embodiment of the present application, in the method for examining and verifying a signboard photo, the signboard text recognition step specifically includes:
OCR data are collected, and a DB character region detection algorithm and a CRNN character recognition algorithm model are trained to obtain a convergent fitting signboard character recognition model;
characters in the signboard image are identified based on the signboard character recognition model.
According to at least one embodiment of the present application, in the method for examining and verifying a signboard photo, the signboard text comparison step specifically includes:
and comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, passing the verification.
Another aspect provides a signboard photo review apparatus including:
the copying detection module is used for detecting whether the signboard photo is a copied photo or not, and if not, the signboard photo passes the detection;
a signboard image recognition module for recognizing a signboard image in the detected signboard photo;
the signboard character recognition module is used for recognizing characters in the signboard image;
and the signboard character comparison module is used for comparing whether the characters in the signboard image are consistent with the corresponding signboard characters or not, and if so, the verification is passed.
According to at least one embodiment of the present application, in the apparatus for examining signboard photo, the duplication detection module detects whether the signboard photo is a duplicated photo, and if not, through the detection, specifically,
collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
and carrying out image graying and special data augmentation processing on the signboard photo, detecting whether the processed signboard photo is a reproduction photo or not based on a reproduction detection model, and if not, passing the detection.
According to at least one embodiment of the present application, in the signboard picture auditing apparatus, the signboard image recognition module recognizes the signboard image in the detected signboard picture, and specifically,
collecting a plurality of signboard images, training a YOLO _ v4 target detection algorithm model, and obtaining a convergent fitting signboard image recognition model;
the sign image in the detected sign photo is identified based on a sign image recognition model.
According to at least one embodiment of the present application, in the signboard photo auditing apparatus, the signboard text recognition module recognizes text in the signboard image, and specifically,
OCR data are collected, and a DB character region detection algorithm and a CRNN character recognition algorithm model are trained to obtain a convergent fitting signboard character recognition model;
characters in the signboard image are identified based on the signboard character recognition model.
According to at least one embodiment of the present application, in the apparatus for examining and verifying signboard photo, in the signboard text comparing module, whether the text in the signboard image is consistent with the corresponding signboard text is compared, if yes, the examination is passed, specifically,
and comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, passing the verification.
Yet another aspect provides an electronic device comprising:
a processor;
a memory storing a computer program configured to implement any of the above-described signboard photo review methods when executed by the processor.
Yet another aspect provides a computer readable storage medium storing a computer program which, when executed by a processor, is capable of implementing any of the above-described signboard photograph auditing methods.
Drawings
FIG. 1 is a flow chart of a signboard photo review method provided by an embodiment of the application;
fig. 2 is a schematic diagram of a signboard photo auditing apparatus provided in an embodiment of the present application.
For the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; further, the drawings are for illustrative purposes, and terms describing positional relationships are limited to illustrative illustrations only and are not to be construed as limiting the patent.
Detailed Description
In order to make the technical solutions and advantages of the present application clearer, the technical solutions of the present application will be further clearly and completely described in the following detailed description with reference to the accompanying drawings, and it should be understood that the specific embodiments described herein are only some of the embodiments of the present application, and are only used for explaining the present application, but not limiting the present application. It should be noted that, for convenience of description, only the parts related to the present application are shown in the drawings, other related parts may refer to general designs, and the embodiments and technical features in the embodiments in the present application may be combined with each other to obtain a new embodiment without conflict.
In addition, unless otherwise defined, technical or scientific terms used in the description of the present application shall have the ordinary meaning as understood by one of ordinary skill in the art to which the present application belongs. The use of the terms "a," "an," or "the" and similar referents in the context of describing the application is not to be construed as an absolute limitation on the number, but rather as the presence of at least one. The word "comprising" or "comprises", and the like, when used in this description, is intended to specify the presence of stated elements or items, but not the exclusion of other elements or items.
The present application is described in further detail below with reference to fig. 1-2.
One aspect provides a signboard photo auditing method, which comprises the following steps:
and (3) copying and detecting: detecting whether the signboard photo is a reproduction photo, and if not, passing the detection;
signboard image recognition: identifying a signboard image in the detected signboard photo;
signboard character recognition: recognizing characters in the signboard image;
signboard text comparison: and comparing whether the characters in the signboard image are consistent with the corresponding signboard characters, namely the signboard names, and if so, passing the verification.
For the signboard photo auditing method disclosed in the above embodiment, those skilled in the art can understand that the method can be used for automatically auditing the photos uploaded by merchants in the e-commerce industry, automatically detecting whether the signboard photos are the copied photos, identifying the signboard images in the detected signboard photos, identifying the characters in the signboard images, and comparing whether the characters in the signboard images are consistent with the corresponding signboard characters to complete the auditing of the signboard photos, and has high efficiency and accuracy.
In some optional embodiments, in the method for reviewing a signboard photo, the step of detecting the reproduction includes:
collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
and carrying out image graying and special data augmentation processing on the signboard photo, detecting whether the processed signboard photo is a reproduction photo or not based on a reproduction detection model, and if not, passing the detection.
For the signboard photo auditing method disclosed in the above embodiment, those skilled in the art can understand that when detecting whether the signboard photo is a reproduced photo, the signboard photo is preprocessed by using image graying and special data augmentation to improve the detection accuracy of whether the signboard photo is the reproduced photo, and then a neural network model is trained based on a plurality of acquired reproduced photos to obtain a convergent fitting reproduction detection model, and whether the processed signboard photo is the reproduced photo is detected, so that the signboard photo auditing method has higher detection accuracy.
In some optional embodiments, in the signboard photograph auditing method, the acquiring a plurality of reproduction photographs and training the neural network model to obtain a convergence fitting reproduction detection model specifically include:
carrying out feature labeling on the collected copied photos;
graying, special data augmentation processing and size normalization processing are carried out on the shot photos, and then the shot photos are divided into a training set, a test set and a verification set;
training the neural network model based on the reproduction photos in the training set, testing based on the reproduction photos in the testing set, and verifying through the reproduction photos in the verification set to obtain a reproduction detection model with convergence fitting.
In some optional embodiments, in the method for reviewing a signboard photo, the characteristic labeling of the captured copy photo specifically includes:
and carrying out fuzzy characteristic marking, moire phenomenon characteristic marking, black edge characteristic marking, watermark characteristic marking and mobile phone frame characteristic marking on the collected copied photos.
As for the signboard photo auditing method disclosed in the above embodiment, those skilled in the art can understand that the reproduced photo may have a blur characteristic, a moire phenomenon characteristic, a black edge characteristic, a watermark characteristic, and a mobile phone frame characteristic, and the acquired reproduced photo is labeled with the characteristics, so that a special data set for detecting the reproduced photo can be constructed.
In some optional embodiments, in the above method for examining and verifying a signboard photo, the graying processing is performed on the captured photo, specifically:
graying the photographed picture according to a conversion formula G ═ pR + qG + wB, wherein,
g is a single channel of the gray scale map;
RGB is three channels of a color picture;
P＝0.340～0.344，q＝0.505～0.509，w＝0.149～0.153，p+q+w＝1。
as for the signboard photo auditing method disclosed in the above embodiment, those skilled in the art can understand that the characteristic difference between the real photo and the reproduced photo is not related to the color information, and noise may be introduced into the presence of the color information to affect the detection of the reproduced photo, and the reproduced photo is grayed according to the above conversion formula, so that noise generated by the color information can be effectively reduced.
In some optional embodiments, in the method for reviewing a signboard photo, the special data augmentation process is performed on the photo to be shot, specifically:
the brightness of the copied photo with the characteristic labels is changed to carry out special data augmentation processing, and the brightness of the copied photo can be increased to 1.5 times of the original brightness and/or the brightness of the copied photo can be reduced to 0.7 times of the original brightness.
For the signboard photo auditing method disclosed in the above embodiment, those skilled in the art can understand that part of the real photo may be taken in a highlight environment or a night environment, and the proportion of the part of the photo is low but the influence on the classification result is large, so that the precision can be effectively improved by simulating the highlight environment and the night environment by changing the brightness of the photo, the brightness of the reproduced photo is increased to 1.5 times of the original brightness and/or the brightness of the reproduced photo is decreased to 0.7 times of the original brightness, the data set can be expanded to 3 times of the original brightness, and based on this, the precision of the model can be correspondingly improved.
In some optional embodiments, in the method for reviewing a signboard photo, the size normalization process is performed on the captured photo, and specifically, the size normalization process is performed by:
the sizes of the copied photos are normalized to 227 x 227, if the sizes of the copied photos are too large, the model is difficult to train, and if the sizes of the copied photos are too small, the copied characters are lost too much, and the accuracy of model training is influenced.
In some optional embodiments, in the signboard photo auditing method, the dividing of the copied photos into a training set, a test set, and a verification set specifically includes:
according to the following steps of 8: 1: 1, dividing the reproduced photos into a training set, a testing set and a verification set.
For the signboard photo auditing method disclosed in the above embodiment, those skilled in the art can understand that the copied photos in the training set are used as data samples for model fitting; the verification concentrated reproduction picture is a sample set reserved independently in the model training process, is used for adjusting the hyper-parameters of the model, is used for carrying out preliminary evaluation on the capability of the model, is used for searching the optimal network depth of the neural network model, or determines the stop point of a back propagation algorithm or selects the number of hidden layer neurons in the neural network; the test-focused rendering can be used to evaluate the generalization ability of the model.
In some optional embodiments, in the signboard photo auditing method, the neural network model includes: 1 convolutional layer, two basic blocks and two fully connected layers. Wherein, the Basicblock is composed of 4 channels in parallel, the first channel comprises 3 convolution layers of 3 × 3 to replace one convolution layer of 7 × 7, the second channel comprises 2 convolution layers of 3 × 3 to replace one convolution layer of 5 × 5, the third channel comprises one convolution layer of 3 × 3, and the fourth channel is an identity module; the pooling layer is added after the first layer convolution and two basicblocks. After the use is carried out the experiment and is verified repeatedly from the reproduction photo data set of establishing, the network depth design of this application is at 9 layers at last, and the degree of depth and the width of network increase again behind 9 layers are not promoting the recognition effect of model, can increase unnecessary calculated amount simultaneously.
In some optional embodiments, in the signboard photo review method, the number of convolution kernels per layer is 64, 128, 256; each pooling layer adopts a maximum pooling mode, the pooling size is 2 x 2, and the step length is 2; the number of output neurons of the full connection layer is 128 and 2 respectively, wherein 2 is the number of categories for outputting softmax, namely, the reproduction and the normal category.
In some optional embodiments, in the signboard photo review method, a dropout layer is connected after a first fully-connected layer to improve the generalization of the model and avoid overfitting, the dropout layer randomly deactivates part of neurons of the network during training, and corresponding weights do not participate in forward and backward propagation in a single training process, so that the neural network model can learn common commonality, thereby avoiding overfitting. The rate parameter of dropout can be set to 0.5, i.e., the probability of each neuron being inactive is 0.5.
In some optional embodiments, in the signboard photo auditing method, the activation functions in the neural network model may be relu functions, and the relu activation functions have better performance than conventional sigmoid and tanh activation functions, and can effectively avoid gradient extinction and gradient explosion, and the calculation formula is as follows:
f(x)＝max(0，x)。
in some optional embodiments, in the above signboard photo review method, a batch normalization layer is added after each convolution layer and the first full-link layer to speed up model training and improve accuracy. Batch standardization can enable the input value of nonlinear transformation to fall into an area with sensitive input through a standardization means, thereby avoiding gradient explosion, enabling data distribution to be consistent and accelerating the speed of model convergence. Usually, the batch normalization layer is placed behind relu when the network is designed, but the effect of placing the batch normalization layer in front of relu is better when training and recognition of the copying detection are carried out through a large amount of tests and analysis and summary of experimental data.
In some alternative embodiments, in the above signboard photo review method, the two parameters momentum and epsilon of the batch normalization layer can be set to 0.99 and 0.001, respectively.
In some optional embodiments, in the signboard photo review method, the final hyper-parameter may be selected by using a cross-validation method during the neural network model training, including an initial learning rate, a learning rate decay interval, a batch size, and the like. And measuring the performance of the model and the generalization capability of the model by using the accuracy of the model on a verification set separated in advance, stopping training when the model loses convergence and the accuracy is not improved any more, and taking the converged model as a finally used rephotography detection model.
In some optional embodiments, in the method for reviewing a signboard photo, the detecting whether the processed signboard photo is a copy photo based on the copy detection model specifically includes:
and obtaining the reproduction probability of the processed signboard photo based on the reproduction detection model, if the reproduction probability is smaller than a detection threshold value, judging that the signboard photo is not the reproduced photo, and otherwise, judging that the signboard photo is the reproduced photo. The size of the detection threshold may be determined by a skilled person according to a specific implementation, and may be, for example, 50%.
In some optional embodiments, in the method for reviewing a signboard photo, the signboard image recognition step specifically includes:
collecting a plurality of signboard images, training a YOLO _ v4 target detection algorithm model, and obtaining a convergent fitting signboard image recognition model;
the sign image in the detected sign photo is identified based on a sign image recognition model.
With respect to the signboard photo auditing method disclosed in the above embodiments, those skilled in the art can understand that the method has higher recognition accuracy by training the YOLO _ v4 target detection algorithm model based on a plurality of collected signboard images to obtain a convergent fitting signboard image recognition model, and recognizing the signboard images in the detected signboard photos.
In some optional embodiments, in the method for reviewing a signboard photo, the signboard text recognition step specifically includes:
OCR data are collected, and a DB character region detection algorithm and a CRNN character recognition algorithm model are trained to obtain a convergent fitting signboard character recognition model;
characters in the signboard image are identified based on the signboard character recognition model.
For the signboard photo auditing method disclosed in the above embodiment, those skilled in the art can understand that the method trains a DB character region detection algorithm and a CRNN character recognition algorithm model based on collected OCR data to obtain a convergent fitting signboard character recognition model, recognizes characters in a signboard image, and has high recognition accuracy.
In some optional embodiments, in the method for examining and verifying a signboard photo, the step of comparing a signboard text specifically includes:
and comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, and the specific numerical value can be 40%, passing the examination.
For the method for examining and verifying the signboard photo disclosed in the above embodiment, it can be understood by those skilled in the art that the name of the signboard text, that is, the name of the signboard, can be uploaded by the merchant during registration, whether the text in the signboard image in the signboard photo is consistent with the corresponding signboard text is determined by comparing the similarity between the text in the signboard image and the corresponding signboard text, and when the similarity is consistent, that is, the similarity exceeds a set threshold value, the signboard photo is examined.
The similarity of the signboard photo auditing method disclosed in the above embodiment can adopt the following expression:
u is the text content in the signboard image;
p represents signboard text content;
p ^ N U represents the repeated text content between the two;
d represents the word number of the word.
With respect to the method for examining and verifying the signboard photo disclosed in the foregoing embodiment, it can be further understood by those skilled in the art that the specific size of the threshold can be determined by the relevant person according to the specific practice when applying the present application, and the main factors to be considered may include the accuracy of recognizing the signboard image in the detected signboard photo and the accuracy of recognizing the characters in the signboard image.
Another aspect provides a signboard photo review apparatus including:
the copying detection module is used for detecting whether the signboard photo is a copied photo or not, and if not, the signboard photo passes the detection;
a signboard image recognition module for recognizing a signboard image in the detected signboard photo;
the signboard character recognition module is used for recognizing characters in the signboard image;
and the signboard character comparison module is used for comparing whether the characters in the signboard image are consistent with the corresponding signboard characters or not, and if so, the verification is passed.
For the signboard photo auditing device disclosed in the above embodiment, those skilled in the art can understand that the device can be used for realizing automatic auditing of photos uploaded by merchants in the e-commerce industry, automatically detecting whether the signboard photos are copied photos, identifying signboard images in the detected signboard photos, identifying characters in the signboard images, and comparing whether the characters in the signboard images are consistent with the corresponding signboard characters, thereby completing auditing of the signboard photos, and having higher efficiency and accuracy.
In some optional embodiments, in the apparatus for examining and verifying a signboard photo, the duplication detection module detects whether the signboard photo is a duplicated photo, and if not, the verification module, specifically,
collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
and carrying out image graying and special data augmentation processing on the signboard photo, detecting whether the processed signboard photo is a reproduction photo or not based on a reproduction detection model, and if not, passing the detection.
For the signboard photo auditing device disclosed in the above embodiment, those skilled in the art can understand that when detecting whether the signboard photo is a reproduced photo, the signboard photo is preprocessed by using image graying and special data augmentation to improve the detection accuracy of whether the signboard photo is the reproduced photo, and then the processed signboard photo is detected whether to be the reproduced photo based on a reproduction detection model obtained by training a neural network model based on a plurality of acquired reproduced photos to obtain a convergent fitting, so that the signboard photo auditing device has higher detection accuracy.
In some optional embodiments, in the signboard photo auditing apparatus, the signboard image recognition module recognizes the signboard image in the detected signboard photo, and specifically,
collecting a plurality of signboard images, training a YOLO _ v4 target detection algorithm model, and obtaining a convergent fitting signboard image recognition model;
the sign image in the detected sign photo is identified based on a sign image recognition model.
With respect to the signboard photo auditing device disclosed in the above embodiments, it can be understood by those skilled in the art that the device has higher recognition accuracy by training the YOLO _ v4 target detection algorithm model to obtain a convergent fitting signboard image recognition model based on a plurality of collected signboard images, and recognizing the signboard images in the detected signboard photos.
In some optional embodiments, in the signboard photo auditing apparatus, the signboard text recognition module recognizes text in the signboard image, and specifically,
OCR data are collected, and a DB character region detection algorithm and a CRNN character recognition algorithm model are trained to obtain a convergent fitting signboard character recognition model;
characters in the signboard image are identified based on the signboard character recognition model.
As for the signboard photo auditing device disclosed in the above embodiment, those skilled in the art can understand that the device trains the DB character region detection algorithm and the CRNN character recognition algorithm model based on the collected OCR data to obtain a convergent fitting signboard character recognition model, recognizes characters in a signboard image, and has high recognition accuracy.
In some optional embodiments, in the apparatus for examining and verifying a signboard photo, in the signboard text comparing module, whether the text in the signboard image is consistent with the corresponding signboard text is compared, if yes, the examination is passed, specifically,
and comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, and the specific numerical value can be 40%, passing the examination.
As for the apparatus for examining and verifying a signboard photo disclosed in the above embodiment, it can be understood by those skilled in the art that the name of a signboard text, which is uploaded by a merchant during registration, is determined whether the text in the signboard image in the signboard photo is consistent with the corresponding signboard text by comparing the similarity between the text in the signboard image and the corresponding signboard text, and when the similarity is consistent, that is, when the similarity exceeds a set threshold value, the signboard photo is examined.
The similarity of the signboard photo auditing method disclosed in the above embodiment can adopt the following expression:
u is the text content in the signboard image;
p represents signboard text content;
p ^ N U represents the repeated text content between the two;
d represents the word number of the word.
With respect to the apparatus for examining and verifying signboard photographs disclosed in the above embodiments, it can be understood by those skilled in the art that the specific size of the threshold value can be determined by the relevant person according to the specific practice when applying the present application, and the main factors to be considered may include the accuracy of recognizing the signboard image in the detected signboard photograph and the accuracy of recognizing the characters in the signboard image.
Yet another aspect provides an electronic device comprising:
a processor;
a memory storing a computer program configured to implement any of the above-described signboard photo review methods when executed by the processor.
Yet another aspect provides a computer readable storage medium storing a computer program which, when executed by a processor, is capable of implementing any of the above-described signboard photograph auditing methods.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
For the signboard picture auditing device disclosed in the above embodiment, because it corresponds to the signboard picture auditing method disclosed in the above embodiment, the description is simpler, and specific relevant points can be referred to the explanation of the signboard picture auditing method, and the technical effects can also be referred to the technical effects of the signboard picture auditing method, which are not described herein again.
Furthermore, those skilled in the art should also realize that the various modules, units, and units of the apparatus disclosed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of both, and that for the sake of clarity only explaining the interchangeability of hardware and software, the functions described herein are generally implemented by hardware or software, and that depending on the particular application and design constraints imposed on the technical solution, those skilled in the art can choose different ways to implement the described functions for each particular application and its practical constraints, but such implementation should not be considered as beyond the scope of the present application.
Yet another aspect provides an electronic device comprising:
a processor;
a memory storing a computer program configured to implement any of the above-described signboard photo review methods when executed by the processor.
In some alternative embodiments, the processor may be a central processing unit CPU or other form of processing unit having data processing capabilities and/or instruction execution capabilities, may be a general purpose processor or a special purpose processor, and may control other components in the compensation electronics to perform desired functions.
In some alternative embodiments, the memory may include various forms of computer-readable storage media, such as volatile memory, which may be random access memory, RAM, and/or cache memory, and/or non-volatile memory, which may be read-only memory, ROM, a hard disk, flash memory, and so forth. The memory may store thereon a computer program that is executed by the processor to implement the functions of the embodiments of the present application and/or other desired functions, and may store various application programs and various data.
In alternative embodiments, the processor and memory may be connected by a bus system, which may be a serial, parallel communication bus, or the like.
It should be noted that, for clarity and conciseness of representation, not all the components of the electronic device are shown in the foregoing embodiments, and in order to implement the necessary functions of the electronic device, a person skilled in the art may provide and set other components not shown according to specific needs.
For the electronic device disclosed in the above embodiment, since the processor of the electronic device can implement any one of the above signboard photo auditing methods when executing the computer program stored in the memory of the electronic device, the technical effect of the above signboard photo auditing method can be referred to correspondingly, and details are not repeated herein.
Yet another aspect provides a computer readable storage medium storing a computer program which, when executed by a processor, is capable of implementing any of the above-described signboard photograph auditing methods.
In some alternative embodiments, the computer-readable storage medium may include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a random access memory RAM, a read only memory ROM, an erasable programmable read only memory EPROM, a portable compact disc read only memory CD-ROM, a flash memory, or any combination of the above, as well as other suitable storage media.
Having thus described the present application in connection with the preferred embodiments illustrated in the accompanying drawings, it will be understood by those skilled in the art that the scope of the present application is not limited to those specific embodiments, and that equivalent modifications or substitutions of related technical features may be made by those skilled in the art without departing from the principle of the present application, and those modifications or substitutions will fall within the scope of the present application.
Claims (10)
1. A signboard photo auditing method, comprising:
and (3) copying and detecting: detecting whether the signboard photo is a reproduction photo, and if not, passing the detection;
signboard image recognition: identifying a signboard image in the detected signboard photo;
signboard character recognition: recognizing characters in the signboard image;
signboard text comparison: and comparing whether the characters in the signboard image are consistent with the corresponding signboard characters, and if so, passing the verification.
2. A signboard photo review method according to claim 1,
the reproduction detection step specifically comprises the following steps:
collecting a plurality of reproduction photos, and training a neural network model to obtain a convergence fitting reproduction detection model;
and carrying out image graying and special data augmentation processing on the signboard photo, detecting whether the processed signboard photo is a reproduction photo or not based on a reproduction detection model, and if not, passing the detection.
3. A signboard photo review method according to claim 1,
the signboard image identification step specifically comprises the following steps:
collecting a plurality of signboard images, training a YOLO _ v4 target detection algorithm model, and obtaining a convergent fitting signboard image recognition model;
the sign image in the detected sign photo is identified based on a sign image recognition model.
4. A signboard photo review method according to claim 1,
the signboard character recognition step specifically comprises the following steps:
OCR data are collected, and a DB character region detection algorithm and a CRNN character recognition algorithm model are trained to obtain a convergent fitting signboard character recognition model;
characters in the signboard image are identified based on the signboard character recognition model.
5. A signboard photo review method according to claim 1,
the signboard text comparison step specifically comprises the following steps:
and comparing the similarity between the characters in the signboard image and the corresponding signboard characters, and if the similarity exceeds a set threshold value, passing the verification.
6. A signboard photo review device, comprising:
the copying detection module is used for detecting whether the signboard photo is a copied photo or not, and if not, the signboard photo passes the detection;
a signboard image recognition module for recognizing a signboard image in the detected signboard photo;
the signboard character recognition module is used for recognizing characters in the signboard image;
and the signboard character comparison module is used for comparing whether the characters in the signboard image are consistent with the corresponding signboard characters or not, and if so, the verification is passed.
7. A signboard photo review device according to claim 6,
the signboard image recognition module recognizes the detected signboard image in the signboard photo, and specifically,
collecting a plurality of signboard images, training a YOLO _ v4 target detection algorithm model, and obtaining a convergent fitting signboard image recognition model;
the sign image in the detected sign photo is identified based on a sign image recognition model.
8. A signboard photo review device according to claim 6,
in the signboard character recognition module, characters in a signboard image are recognized, specifically,
OCR data are collected, and a DB character region detection algorithm and a CRNN character recognition algorithm model are trained to obtain a convergent fitting signboard character recognition model;
characters in the signboard image are identified based on the signboard character recognition model.
9. An electronic device, comprising:
a processor;
a memory storing a computer program configured to enable the signboard photo review method according to any one of claims 1 to 5 when executed by the processor.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, is capable of implementing a signboard photo review method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011145385.2A CN112257709B (en) | 2020-10-23 | 2020-10-23 | Signboard photo auditing method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011145385.2A CN112257709B (en) | 2020-10-23 | 2020-10-23 | Signboard photo auditing method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112257709A true CN112257709A (en) | 2021-01-22 |
| CN112257709B CN112257709B (en) | 2024-05-07 |
Family
ID=74263610
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011145385.2A Active CN112257709B (en) | 2020-10-23 | 2020-10-23 | Signboard photo auditing method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112257709B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113435437A (en) * | 2021-06-24 | 2021-09-24 | 随锐科技集团股份有限公司 | Method and device for identifying state of switch on/off indicator and storage medium |
| CN113486715A (en) * | 2021-06-04 | 2021-10-08 | 广州图匠数据科技有限公司 | Image reproduction identification method, intelligent terminal and computer storage medium |
| CN116958795A (en) * | 2023-06-30 | 2023-10-27 | 北京房多多信息技术有限公司 | Method and device for identifying flip image, electronic equipment and storage medium |
| CN117315415A (en) * | 2023-07-13 | 2023-12-29 | 东南大学 | Shop sign health performance evaluation method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180211155A1 (en) * | 2017-01-23 | 2018-07-26 | Fotonation Limited | Method for synthesizing a neural network |
| CN109558794A (en) * | 2018-10-17 | 2019-04-02 | 平安科技（深圳）有限公司 | Image-recognizing method, device, equipment and storage medium based on moire fringes |
| CN109726743A (en) * | 2018-12-12 | 2019-05-07 | 苏州大学 | A kind of retina OCT image classification method based on Three dimensional convolution neural network |
| CN109886275A (en) * | 2019-01-16 | 2019-06-14 | 深圳壹账通智能科技有限公司 | Reproduction image-recognizing method, device, computer equipment and storage medium |
| CN110046644A (en) * | 2019-02-26 | 2019-07-23 | 阿里巴巴集团控股有限公司 | A kind of method and device of certificate false proof calculates equipment and storage medium |
| CN110349033A (en) * | 2019-05-30 | 2019-10-18 | 阿里巴巴集团控股有限公司 | Project case Methods of initial evaluation of sci-tech and device |
| CN110458802A (en) * | 2019-06-28 | 2019-11-15 | 天津大学 | Based on the projection normalized stereo image quality evaluation method of weight |
-
2020
- 2020-10-23 CN CN202011145385.2A patent/CN112257709B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180211155A1 (en) * | 2017-01-23 | 2018-07-26 | Fotonation Limited | Method for synthesizing a neural network |
| CN109558794A (en) * | 2018-10-17 | 2019-04-02 | 平安科技（深圳）有限公司 | Image-recognizing method, device, equipment and storage medium based on moire fringes |
| CN109726743A (en) * | 2018-12-12 | 2019-05-07 | 苏州大学 | A kind of retina OCT image classification method based on Three dimensional convolution neural network |
| CN109886275A (en) * | 2019-01-16 | 2019-06-14 | 深圳壹账通智能科技有限公司 | Reproduction image-recognizing method, device, computer equipment and storage medium |
| CN110046644A (en) * | 2019-02-26 | 2019-07-23 | 阿里巴巴集团控股有限公司 | A kind of method and device of certificate false proof calculates equipment and storage medium |
| CN110349033A (en) * | 2019-05-30 | 2019-10-18 | 阿里巴巴集团控股有限公司 | Project case Methods of initial evaluation of sci-tech and device |
| CN110458802A (en) * | 2019-06-28 | 2019-11-15 | 天津大学 | Based on the projection normalized stereo image quality evaluation method of weight |
Non-Patent Citations (2)
| Title |
|---|
| HAOLIANG LI等: "Image Recapture Detection with Convolutional and Recurrent Neural Networks", IS&T INTERNATIONAL SYMPOSIUM ON ELECTRONIC IMAGING 2017, 31 December 2017 (2017-12-31), pages 87 - 91 * |
| 魏书伟;曾上游;周悦;王新娇;: "基于并行残差卷积神经网络的多种树叶分类", 现代电子技术, no. 09, 1 May 2020 (2020-05-01), pages 96 - 100 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113486715A (en) * | 2021-06-04 | 2021-10-08 | 广州图匠数据科技有限公司 | Image reproduction identification method, intelligent terminal and computer storage medium |
| CN113435437A (en) * | 2021-06-24 | 2021-09-24 | 随锐科技集团股份有限公司 | Method and device for identifying state of switch on/off indicator and storage medium |
| CN116958795A (en) * | 2023-06-30 | 2023-10-27 | 北京房多多信息技术有限公司 | Method and device for identifying flip image, electronic equipment and storage medium |
| CN117315415A (en) * | 2023-07-13 | 2023-12-29 | 东南大学 | Shop sign health performance evaluation method and device |
| CN117315415B (en) * | 2023-07-13 | 2024-06-11 | 东南大学 | Shop sign health performance evaluation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112257709B (en) | 2024-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112257709A (en) | Signboard photo auditing method and device, electronic equipment and readable storage medium | |
| CN109034159B (en) | Image information extraction method and device | |
| CN110348319B (en) | Face anti-counterfeiting method based on face depth information and edge image fusion | |
| CN109034050B (en) | Deep learning-based identification card image text recognition method and device | |
| CN112613501A (en) | Information auditing classification model construction method and information auditing method | |
| CN109711874A (en) | User's portrait generation method, device, computer equipment and storage medium | |
| US10043071B1 (en) | Automated document classification | |
| CN110503099B (en) | Information identification method based on deep learning and related equipment | |
| CN114168732A (en) | Text emotion analysis method and device, computing device and readable medium | |
| CN110659599A (en) | Scanning test paper-based offline handwriting authentication system and using method thereof | |
| CN114821229B (en) | Underwater acoustic data set augmentation method and system based on condition generation countermeasure network | |
| CN112258481A (en) | Portal photo reproduction detection method | |
| CN112508039B (en) | Image detection method and device | |
| Rai et al. | An end-to-end real-time face identification and attendance system using convolutional neural networks | |
| Bintoro et al. | Lampung script recognition using convolutional neural network | |
| Ramesh et al. | Facial recognition as a tool to identify Roman emperors: towards a new methodology | |
| Ige et al. | Towards HDR based facial expression recognition under complex lighting | |
| Li et al. | An improved PCB defect detector based on feature pyramid networks | |
| Ghonge et al. | Real time fake note detection using deep convolutional neural network | |
| CN114329050A (en) | Visual media data deduplication processing method, device, equipment and storage medium | |
| CN114820476A (en) | Identification card identification method based on compliance detection | |
| Rasheed et al. | A Novel Method for Signature Verification Using Deep Learning | |
| Acharjya et al. | A review on forensic science and criminal investigation through a deep learning framework | |
| CN111832622A (en) | Method and system for identifying ugly pictures of specific figures | |
| CN111753108A (en) | Presentation generation method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |