CN115775317A - Certificate information identification matching method and system based on big data - Google Patents

Abstract

The application provides a certificate information identification matching method and system based on big data, and relates to the field of image identification. A big data based certificate information identification matching method comprises the following steps: extracting an effective area including certificate information by using image acquisition equipment to manufacture a sample data set; building a character recognition model and a face recognition model; storing picture information in an operator database by adopting a big data architecture improved HDFS; adopting a Faiss similarity searching tool to match text content and face image feature vectors; and acquiring a certificate sample test picture, respectively inputting the certificate sample test picture into a character recognition model and a face recognition model for recognition, performing big data matching through a Faiss similarity search tool, and outputting related information of the certificate. The multi-dimensional judgment can be carried out through the matching of the character content and the face characteristics, the identification accuracy is improved, and the problem that the information cannot be identified and the identification is wrong due to incomplete single identification mode is avoided.

Description

Certificate information identification matching method and system based on big data

Technical Field

The application relates to the field of image recognition, in particular to a certificate information recognition and matching method and system based on big data.

Background

The big data technology can mine information and knowledge hidden in mass data, and provides basis for social and economic activities, so that the operation efficiency of each field is improved, and the intensification degree of the whole social and economic is greatly improved.

The photographing and scanning certificate identifies the related identity information and is used in the scenes of various business transactions, site entry registration and the like. The traditional mode is for the business personnel to type in identity name or mobile phone number inquiry this user's information, and most business outlets have also been equipped with the license scanning instrument, but the traditional mode of scanning acquisition information need get off the complete shooting of ID card, and is higher to the requirement of card locating place and picture definition, and recognition speed is slow, the low problem that needs to solve is of urgent need to official working efficiency.

The method has the advantages that the requirements for the placement position of the certificate and the definition of the certificate and the picture are high, the recognition speed is low, and the missing content cannot be recognized accurately in the conventional service scene when the scanning equipment recognizes the user certificate. Aiming at the scene, the patent innovatively provides a certificate information identification matching method based on big data, and by applying an image processing identification technology and a big data technology, the requirement on a picture matched with certificate information identification is reduced, the identification accuracy is improved, the information acquisition speed is accelerated, the development of business personnel work is effectively helped, and the user experience degree is improved.

Disclosure of Invention

The purpose of the application is to provide a certificate information identification and matching method based on big data, which can perform multi-dimensional judgment through character content matching and face feature matching, improve the identification accuracy, and avoid the problem that the information is incomplete in a single identification mode, so that the identification cannot be performed and the identification errors cannot be performed.

Another object of the present application is to provide a big data based certificate information recognition matching system, which can operate a big data based certificate information recognition matching method.

The embodiment of the application is realized as follows:

in a first aspect, an embodiment of the application provides a certificate information identification and matching method based on big data, which includes the steps of collecting a picture containing certificate content by using an image collection device, preprocessing the picture, extracting an effective area including certificate information by adopting a semantic segmentation algorithm, and making a sample data set; building a character recognition model, inputting a sample image data set into the model for recognition, outputting a recognized result, and converting the recognized result into characters; building a face recognition model, inputting a sample image data set into the model for recognition, and outputting a recognized result to obtain a face feature vector; storing picture information in an operator database by adopting a big data architecture improved HDFS, wherein the picture information comprises a picture MD5 value, character contents in a picture, a face characteristic vector in the picture and user information; adopting a Faiss similarity search tool to perform character content matching and face image feature vector matching, classifying and judging matching results, and outputting the matching results; and acquiring a certificate sample test picture, respectively inputting the certificate sample test picture into a character recognition model and a face recognition model for recognition, performing big data matching through a Faiss similarity searching tool, and outputting related information of the certificate.

In some embodiments of the present application, the collecting, by using an image collecting device, a picture including certificate content, preprocessing the picture, extracting an effective area including certificate information by using a semantic segmentation algorithm, and making a sample data set includes:

the method comprises the steps of shooting a certificate picture required to be issued when a user transacts a business by utilizing image acquisition equipment of a business network, classifying the sample picture into two types of pictures, namely, a picture with a character part missing and a picture part missing, wherein the sample picture is a sample picture with a position which is not correctly placed and causes the loss of part of the content of the certificate.

Establishing a certificate image semantic segmentation model to extract partial areas containing certificate contents, firstly extracting low-level features, in order to increase the input of feature information, proposing a Block1 layer and a Block2 layer in an Xconvergence model, simultaneously extracting two feature maps with the image sizes of 256x256x128 and 129x129x256 as input information of a decoder, then respectively introducing a space attention module and a channel attention module, processing the feature maps obtained in the model, increasing the utilization rate of important features, effectively filtering background information and improving the accuracy of feature extraction; and a FocalLoss loss function is adopted to replace a cross entropy loss function, so that the loss of characteristic information is reduced, the accuracy of distinguishing target categories is increased, the image segmentation effect is improved, the certificate body part in the sample image is identified and segmented, and the partial region image of the certificate containing the certificate content is obtained.

Processing the extracted partial area image containing the certificate content as an information sample picture, acquiring a white background picture with the size of 300x300 pixels, placing the information sample picture on the upper layer of the white background picture, adjusting the center point of the information picture sample to be superposed with the center point of the white background picture, adjusting the size of the information sample picture to be scaled in equal proportion, completely placing the information sample picture in the white background picture, and only touching one vertex on the boundary of the white background picture to obtain the white background picture containing the information sample picture, thereby preparing a sample image data set.

In some embodiments of the present application, the building a character recognition model, inputting the sample image dataset into the model for recognition, and outputting a recognition result, and the converting into characters includes: and (3) building an SSD algorithm basic framework, wherein a basic network adopts VGG-16, and the network adopts 6 different feature maps to detect targets with different scales, so that a multi-scale feature map prediction structure is realized. When the input is carried out, the image is scaled, namely the RGB image of 300 × 3, the main network adopts Conv5_3 of VGG-16 and the previous partial structure, after the character matrix passes through the network, the character matrix is output to be 19 × 512, then the character matrix is sequentially converted into the character graphs of 19 × 1024, 10 × 512,5 × 5 256,3 × 256,1 × 256 through 5 layers of convolution layers, and finally the detection result is obtained according to a non-maximum inhibition optimization algorithm.

And optimizing an SSD algorithm basic framework, and adding an RFB module into an SSD network, specifically, merging an RFB structure before conv7_ fc is connected with a prediction layer, so that deeper information fusion can be obtained, and a receptive field is added to a part of network layers. The positioning capability of the sensitive information text box can be improved, so that the positioning capability is effectively improved; and replacing the VGG16 backbone network in the SSD network by adopting a MobileNet V3 network to reduce parameter calculation, thereby obtaining a lightweight network structure. On the premise of not influencing the precision, the speed of a network training sample is greatly improved, a small network is considered while optimizing the delay, a model is reconstructed from the angle of deep separable convolution, and an optimized character positioning network structure is obtained; and inputting the ICDAR 2017RCTW data set into a character positioning network structure for training to obtain a character positioning model.

And (3) building a YOLOv3 network framework, optimizing the network framework, specifically improving the fourth detection end of the YOLOv3, and after the YOLOv3 algorithm finishes the 3 rd feature scale processing, adopting 2 times of upsampling processing to expand the output feature scale from 52 x 52 to 104 x 104. And then, performing feature fusion on the feature scale output of the front layer 104 x 104 and the 11 th layer output in the Darknet structure by using a Route layer, completing the 4 th feature scale detection to obtain an optimized YOLOv3 network frame, inputting an ICDAR 2017RCTW data set into the optimized YOLOv3 network frame for training to obtain a content identification model.

Inputting pictures in the sample image data set into a character positioning model for character area positioning extraction to obtain character sample pictures; and inputting the character sample picture into the content recognition model for character recognition, and outputting the recognized character content to obtain a final character recognition model.

In some embodiments of the present application, the constructing a face recognition model, inputting a sample image data set into the model for recognition, and outputting a recognition result to obtain a face feature vector includes: and (3) building a deep ID basic network frame, optimizing, additionally changing the third and fourth layer volume base layers into a shared and introduced central loss verification function, and finally extracting 512-dimensional feature vectors by the deep layer. The CNN of the central loss and the softmax loss combined supervised learning greatly improves the face recognition capability of deep learning features. Compared with the coherent and triplet loss, the central loss has the advantages that a complex and ambiguous sample pair construction process is omitted, and only the introduction of the central loss into a feature output layer is required to obtain the human face feature extraction network framework.

The method comprises the steps of obtaining a picture in a sample image data set, intercepting a human face block diagram in the picture, expanding proper pixels, then cutting to carry out human face preprocessing, wherein the preprocessing is that the human face is aligned, and carrying out affine transformation on the picture by taking five reference points of the human face as a standard.

After the face is preprocessed, the preprocessed cut face is input into a face feature extraction network frame to extract face features, and a 512-dimensional face feature vector is obtained.

In some embodiments of the present application, the above-mentioned image information stored in the operator database by using the big data structure improved HDFS includes an image MD5 value, text content in the image, a face feature vector in the image, and user information:

the method comprises the steps of adopting a big data architecture HDFS for storage, optimizing and improving the architecture, selecting partial functions of a Namenode and transferring control authority of a system, namely storing file metadata of a current storage node by using a Datanode memory on the basis that the Namenode passes through all metadata of a hard disk incremental storage system, and obtaining the improved HDFS.

The improved HDFS is used for storing user information collected by an operator, wherein the user information comprises information such as user names, user numbers, identity card numbers, sexes, addresses, mobile phone numbers, face pictures, face picture recognition feature vectors, face picture MD5 values and the like.

Reading files from HDFS storage, and when a read-write request is sent to a system, firstly selecting the most frequently accessed Datanode from a Datanode list cached in a memory of the system, namely a current system hotspot storage node; if the connection with the hotspot Datanode can be established, searching a target file of the current reading operation in the Datanode, and determining a target data block according to the mapping relation from the file in the metadata to the data block; after the target block is determined, verifying whether the modification identifiers in the IDs of all backup blocks are uniform, if the identifiers are uniform, verifying that the data of the current target block is certain to be the latest data, and then starting to read the file from the target block according to the intra-block offset and the file length of the target file in the metadata; if the condition that the modification identifiers of the copy blocks corresponding to the target block are not uniform is verified, the fact that the file writing of the target block is possibly not completed on all the copies of the target block is shown, and therefore the target block is connected with the Namenode to obtain the latest data of the current target block, and the reading operation of the target file is completed.

In some embodiments of the present application, the performing text content matching and facial image feature vector matching by using the Faiss similarity search tool, and classifying and distinguishing matching results includes: the Faiss feature index service pulls the improved HDFS to the corresponding fragment file, loads the feature file, performs segmentation and dirty data cleaning on the feature, and starts to construct the feature index after completing the loading of the feature file. And requesting the index library to obtain a result, sending the feature vector to the index library for similarity result search, querying the vector feature after the index library receives the request, and calling the Faiss index for vector search. And outputting a vector search result, performing character matching and face detection, setting the confidence coefficient of a matching result, and judging the matching result.

In some embodiments of the present application, the obtaining a certificate sample test picture, inputting the certificate sample test picture into a character recognition model and a face recognition model for recognition, performing big data matching by a Faiss similarity search tool, and outputting related information of the certificate includes: acquiring a certificate sample test picture, inputting the certificate sample test picture into a character recognition model for character content extraction, and then inputting the certificate sample test picture into a face recognition model for face feature vector extraction to obtain the character content and the face feature vector of the test picture. Inputting the text content and the face feature vector of the test picture into a Faiss similarity search tool for vector retrieval, outputting characters and face pictures with high confidence degrees with the test picture, and acquiring the user number corresponding to the similar characters and the user number corresponding to the similar face pictures. And comparing the user number corresponding to the similar characters with the user number corresponding to the similar face picture, and selecting the user number corresponding to the similar characters and the user number corresponding to the similar face picture as a final result to obtain the matched user number. And acquiring user information including user name, user number, identity card number, gender, address, mobile phone number and face picture by matching the user number.

In a second aspect, an embodiment of the present application provides a big data based certificate information identification matching system, which includes an acquisition module, configured to acquire, by using an image acquisition device, a picture including certificate content, pre-process the picture, extract an effective region including certificate information by using a semantic segmentation algorithm, and make the effective region into a sample data set;

the character recognition module is used for building a character recognition model, inputting the sample image data set into the model for recognition, outputting a recognized result and converting the recognized result into characters;

the face recognition module is used for building a face recognition model, inputting the sample image data set into the model for recognition, and outputting a recognized result to obtain a face feature vector;

the picture information storage module is used for storing picture information in an operator database by adopting a big data architecture improved HDFS, wherein the picture information comprises a picture MD5 value, character content in the picture, a face characteristic vector in the picture and user information;

the matching module is used for matching character contents and facial image feature vectors by adopting a Faiss similarity searching tool, classifying and judging matching results and outputting the matching results;

and the output module is used for acquiring a certificate sample test picture, respectively inputting the certificate sample test picture into the character recognition model and the face recognition model for recognition, performing big data matching through a Faiss similarity searching tool, and outputting related information of the certificate.

In some embodiments of the present application, the above includes: at least one memory for storing computer instructions; at least one processor in communication with the memory, wherein the at least one processor, when executing the computer instructions, causes the system to: the device comprises an acquisition module, a character recognition module, a face recognition module, a picture information storage module, a matching module and an output module.

In a third aspect, embodiments of the present application provide a computer-readable storage medium on which a computer program is stored, where the computer program, when executed by a processor, implements a method such as any one of the methods for identifying matches based on big data certificate information.

Compared with the prior art, the embodiment of the application has at least the following advantages or beneficial effects:

1. by adopting the HDFS distributed file system, massive user data of an operator are stored, the HDFS can be transversely expanded, and the stored files can support PB-level or higher-level data storage. By improving the structure of the HDFS, the problem of overlarge Namenode single-point pressure caused by overlarge metadata data due to the massive file quantity is solved.

2. The semantic segmentation algorithm is structurally improved, a space attention module and a channel attention module are respectively introduced to process a feature map acquired from a network model, the utilization rate of important features is increased, background information is effectively filtered, and the accuracy of feature extraction is improved; and finally, a Focal local Loss function is adopted to replace a cross entropy Loss function, so that the Loss of characteristic information is reduced, the accuracy rate of distinguishing target categories is increased, and the image segmentation effect is improved.

3. The character recognition model optimizes the SSD algorithm basic framework, adds the RFB module into the SSD network, can obtain deeper information fusion, adds a receptive field to a part of network layers, and can improve the positioning capability of a sensitive information text box, thereby effectively improving the positioning capability; and replacing the VGG16 backbone network in the SSD network by adopting a MobileNet V3 network to reduce parameter calculation, thereby obtaining a lightweight network structure. On the premise of not influencing the precision, the speed of a network training sample is greatly improved, a small network is considered while the delay is optimized, a model is reconstructed from the perspective of deep separable convolution, and an optimized character positioning network structure is obtained.

4. The character recognition model makes full use of shallow fine-grained characteristics in the network calculation process by improving the YOLOv3 algorithm, and achieves the detection effect of multi-scale character targets.

5. And combining the character content matching with the face feature vector matching result, judging whether the certificate picture shot in from multiple dimensions is matched with the user information in the operator big data, and acquiring the user related data. When characters in a photographed certificate picture or a face image are lost due to incorrect placement position of a certificate, identification information cannot be completely provided, and the situations of identification error, ineffective identification and the like are caused. The multi-dimensional judgment is carried out through the matching of the character content and the face features, the identification accuracy is improved, and the problem that the information cannot be identified and the identification is wrong due to incomplete single identification mode is avoided.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 is a schematic diagram illustrating steps of a big data certificate information identification matching method according to an embodiment of the present application;

fig. 2 is a schematic diagram of a big data certificate information identification matching system module according to an embodiment of the present application;

fig. 3 is an electronic device according to an embodiment of the present disclosure.

Icon: 10-an acquisition module; 20-a character recognition module; 30-a face recognition module; 40-a picture information storage module; 50-a matching module; 60-an output module; 101-a memory; 102-a processor; 103-a communication interface.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

It is to be noted that the term "comprises," "comprising," or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the individual features of the embodiments can be combined with one another without conflict.

Example 1

Referring to fig. 1, fig. 1 is a schematic diagram illustrating steps of a big data certificate information identification matching method according to an embodiment of the present application, and the steps are as follows:

s100, shooting and collecting a picture containing certificate content by using image collection equipment, preprocessing the picture, extracting an effective area containing certificate information by adopting a semantic segmentation algorithm, and making a sample data set.

1) The method comprises the steps of shooting a certificate picture required to be issued when a user transacts a business by utilizing image acquisition equipment of a business network, classifying the sample picture into two types of pictures, namely, a picture with a character part missing and a picture part missing, wherein the sample picture is a sample picture with a position which is not correctly placed and causes the loss of part of the content of the certificate.

2) Establishing a certificate image semantic segmentation model to extract partial areas containing certificate contents, firstly extracting low-level features, in order to increase the input of feature information, proposing a Block1 layer and a Block2 layer in an Xmeeting model, simultaneously extracting two feature maps with the image sizes of 256x256x128 and 129x129x256 as input information of a decoder, then respectively introducing a space attention module and a channel attention module, processing the feature maps obtained from the model, increasing the utilization rate of important features, effectively filtering background information, and improving the accuracy of feature extraction; and (3) replacing a cross entropy loss function with a FocalLoss loss function, reducing the loss of characteristic information, increasing the accuracy of distinguishing target categories to improve the image segmentation effect, and identifying and segmenting the certificate body part in the sample image to obtain a partial region image of the certificate containing the certificate content.

3) Processing the extracted partial area image containing the certificate content as an information sample picture, acquiring a white background picture with the size of 300x300 pixels, placing the information sample picture on the upper layer of the white background picture, adjusting the center point of the information picture sample to be superposed with the center point of the white background picture, adjusting the size of the information sample picture to be scaled in equal proportion, completely placing the information sample picture in the white background picture, and only touching one vertex on the boundary of the white background picture to obtain the white background picture containing the information sample picture, thereby preparing a sample image data set.

And S110, building a character recognition model, inputting a sample image data set into the model for recognition, outputting a recognized result, and converting the recognized result into characters.

1) And (3) building an SSD algorithm basic framework, wherein a basic network adopts VGG-16, and the network adopts 6 different feature maps to detect targets with different scales, so that a multi-scale feature map prediction structure is realized. When the input is carried out, the image is scaled, namely the RGB image of 300 × 3, the main network adopts Conv5_3 of VGG-16 and the previous partial structure, after the character matrix passes through the network, the character matrix is output to be 19 × 512, then the character matrix is sequentially converted into the character graphs of 19 × 1024, 10 × 512,5 × 5 256,3 × 256,1 × 256 through 5 layers of convolution layers, and finally the detection result is obtained according to a non-maximum inhibition optimization algorithm.

2) And optimizing an SSD algorithm basic framework, and adding an RFB module into an SSD network, specifically, merging an RFB structure before conv7_ fc is connected with a prediction layer, so that deeper information fusion can be obtained, and a receptive field is added to a part of network layers. The positioning capability of the sensitive information text box can be improved, so that the positioning capability is effectively improved; and replacing the VGG16 backbone network in the SSD network by adopting a MobileNet V3 network to reduce parameter calculation, thereby obtaining a lightweight network structure. On the premise of not influencing the precision, the speed of a network training sample is greatly improved, a small network is considered while optimizing the delay, a model is reconstructed from the angle of deep separable convolution, and an optimized character positioning network structure is obtained; and inputting the ICDAR 2017RCTW data set into a character positioning network structure for training to obtain a character positioning model.

3) And (3) building a YOLOv3 network framework, optimizing the network framework, specifically improving the fourth detection end of the YOLOv3, and after the YOLOv3 algorithm finishes the 3 rd feature scale processing, adopting 2 times of upsampling processing to expand the output feature scale from 52 x 52 to 104 x 104. And then, performing feature fusion on the feature scale output of the front layer 104 x 104 and the 11 th layer output in the Darknet structure by using a Route layer, completing the 4 th feature scale detection to obtain an optimized YOLOv3 network frame, inputting an ICDAR 2017RCTW data set into the optimized YOLOv3 network frame for training to obtain a content identification model.

4) Inputting pictures in the sample image data set into a character positioning model for character area positioning extraction to obtain character sample pictures; and inputting the character sample picture into the content recognition model for character recognition, and outputting the recognized character content to obtain a final character recognition model.

And S120, building a face recognition model, inputting the sample image data set into the model for recognition, and outputting a recognized result to obtain a face feature vector.

1) And (3) building a deep ID basic network frame, optimizing, additionally changing the third and fourth layer volume base layers into a shared and introduced central loss verification function, and finally extracting 512-dimensional feature vectors by the deep layer. The CNN of the central loss and the softmax loss combined supervised learning greatly improves the face recognition capability of deep learning features. Compared with the coherent and triplet loss, the central loss has the advantages that a complex and ambiguous sample pair construction process is omitted, and only the introduction of the central loss into a feature output layer is required to obtain the human face feature extraction network framework.

2) The method comprises the steps of obtaining a picture in a sample image data set, intercepting a human face block diagram in the picture, expanding proper pixels, then cutting to carry out human face preprocessing, wherein the preprocessing is that the human face is aligned, and carrying out affine transformation on the picture by taking five reference points of the human face as a standard.

3) After the face is preprocessed, the preprocessed cut face is input into a face feature extraction network frame to extract face features, and a 512-dimensional face feature vector is obtained.

And S130, storing picture information in an operator database by adopting a big data architecture improved HDFS, wherein the picture information comprises a picture MD5 value, text contents in a picture, a face characteristic vector in the picture and user information.

1) The method comprises the steps of storing by adopting a big data architecture HDFS, optimizing and improving the architecture, selecting partial functions of a Namenode and control authority of a system to be transferred, namely, storing file metadata of a current storage node by using a dataode memory on the basis that the Namenode stores all metadata of the system through a hard disk increment, and obtaining the improved HDFS.

2) The improved HDFS is used for storing user information collected by an operator, wherein the user information comprises information such as user names, user numbers, identity card numbers, sexes, addresses, mobile phone numbers, face pictures, face picture recognition feature vectors, face picture MD5 values and the like.

3) Reading a file from an HDFS (Hadoop distributed File System) storage, and when a read-write request is initiated to a system, firstly selecting a most frequently accessed Datanode, namely a current system hotspot storage node, from a Datanode list cached in a memory of the system; if the connection with the hotspot dataode can be established, searching a target file of the current reading operation in the dataode, and determining a target data block according to the mapping relation from the file in the metadata to the data block; after the target block is determined, verifying whether the modification identifiers in the IDs of all backup blocks are uniform, if the identifiers are uniform, verifying that the data of the current target block is certain to be the latest data, and then starting to read the file from the target block according to the intra-block offset and the file length of the target file in the metadata; if the condition that the modification identifiers of the copy blocks corresponding to the target block are not uniform is verified, the fact that the file writing of the target block is possibly not completed on all the copies of the target block is shown, and therefore the target block is connected with the Namenode to obtain the latest data of the current target block, and the reading operation of the target file is completed.

And S140, adopting a Faiss similarity searching tool to perform character content matching and face image feature vector matching, classifying and judging matching results, and outputting the matching results.

1) The Faiss feature index service pulls the improved HDFS to the corresponding fragment file, loads the feature file, performs segmentation and dirty data cleaning on the feature, and starts to construct the feature index after completing the loading of the feature file.

2) And requesting the index library to obtain a result, sending the feature vector to the index library for similarity result search, querying the vector feature after the index library receives the request, and calling the Faiss index for vector search.

3) And outputting a vector search result, performing character matching and face detection, setting the confidence coefficient of a matching result, and judging the matching result.

S150, acquiring a certificate sample test picture, inputting the certificate sample test picture into a character recognition model and a face recognition model for recognition, performing big data matching through a Faiss similarity search tool, and outputting related information of the certificate.

1) Acquiring a certificate sample test picture, inputting the certificate sample test picture into a character recognition model for character content extraction, and then inputting the certificate sample test picture into a face recognition model for face feature vector extraction to obtain the character content and the face feature vector of the test picture.

2) Inputting the text content and the face feature vector of the test picture into a Faiss similarity search tool for vector retrieval, outputting characters and face pictures with high confidence degrees with the test picture, and acquiring the user number corresponding to the similar characters and the user number corresponding to the similar face pictures.

3) And comparing the user number corresponding to the similar characters with the user number corresponding to the similar face picture, and selecting the user number corresponding to the similar characters and the user number corresponding to the similar face picture as a final result to obtain the matched user number.

4) And acquiring user information including user name, user number, identity card number, gender, address, mobile phone number and face picture by matching the user number.

Example 2

Referring to fig. 2, fig. 2 is a schematic diagram of a big data certificate information identification matching system module according to an embodiment of the present application, which is shown as follows:

the acquisition module 10 is used for acquiring a picture containing certificate content by using image acquisition equipment, preprocessing the picture, extracting an effective area comprising certificate information by adopting a semantic segmentation algorithm, and making a sample data set;

the character recognition module 20 is used for building a character recognition model, inputting the sample image data set into the model for recognition, outputting a recognized result and converting the recognized result into characters;

the face recognition module 30 is used for building a face recognition model, inputting the sample image data set into the model for recognition, and outputting a recognized result to obtain a face feature vector;

the picture information storage module 40 is used for storing picture information in an operator database by adopting a big data architecture improved HDFS, wherein the picture information comprises a picture MD5 value, text contents in a picture, face characteristic vectors in the picture and user information;

the matching module 50 is used for matching the text content and the face image feature vector by adopting a Faiss similarity searching tool, classifying and judging the matching result and outputting the matching result;

and the output module 60 is used for acquiring a certificate sample test picture, respectively inputting the certificate sample test picture into the character recognition model and the face recognition model for recognition, performing big data matching through a Faiss similarity searching tool, and outputting related information of the certificate.

As shown in fig. 3, an embodiment of the present application provides an electronic device, which includes a memory 101 for storing one or more programs; a processor 102. The one or more programs, when executed by the processor 102, implement the method of any of the first aspects as described above.

Also included is a communication interface 103, and the memory 101, processor 102 and communication interface 103 are electrically connected to each other, directly or indirectly, to enable transfer or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 101 may be used to store software programs and modules, and the processor 102 executes the software programs and modules stored in the memory 101 to thereby execute various functional applications and data processing. The communication interface 103 may be used for communicating signaling or data with other node devices.

The Memory 101 may be, but is not limited to, a Random Access Memory 101 (RAM), a Read Only Memory 101 (ROM), a Programmable Read Only Memory 101 (PROM), an Erasable Read Only Memory 101 (EPROM), an electrically Erasable Read Only Memory 101 (EEPROM), and the like.

The processor 102 may be an integrated circuit chip having signal processing capabilities. The Processor 102 may be a general-purpose Processor 102, including a Central Processing Unit (CPU) 102, a Network Processor 102 (NP), and the like; but may also be a Digital Signal processor 102 (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware components.

In the embodiments provided in the present application, it should be understood that the disclosed method and system may be implemented in other ways. The method and system embodiments described above are merely illustrative, for example, the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

In another aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by the processor 102, implements the method according to any one of the first aspect described above. The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory 101 (ROM), a Random Access Memory 101 (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In summary, the identification and matching method and system based on big data certificate information provided by the embodiment of the application store massive user data of an operator by adopting an HDFS distributed file system, the HDFS can be expanded transversely, and the stored files can support PB-level or higher-level data storage. By improving the structure of the HDFS, the problem of overlarge Namenode single-point pressure caused by overlarge metadata data due to the massive file quantity is solved. The semantic segmentation algorithm is structurally improved, a space attention module and a channel attention module are respectively introduced to process a feature map acquired from a network model, the utilization rate of important features is increased, background information is effectively filtered, and the accuracy of feature extraction is improved; and finally, a Focal local Loss function is adopted to replace a cross entropy Loss function, so that the Loss of characteristic information is reduced, the accuracy rate of distinguishing target categories is increased, and the image segmentation effect is improved. The character recognition model optimizes the SSD algorithm basic framework, adds the RFB module into the SSD network, can obtain deeper information fusion, adds a receptive field to a part of network layers, and can improve the positioning capability of a sensitive information text box, thereby effectively improving the positioning capability; and replacing the VGG16 backbone network in the SSD network by adopting a MobileNet V3 network to reduce parameter calculation, thereby obtaining a lightweight network structure. On the premise of not influencing the precision, the speed of the network training sample is greatly improved, the small network is considered while the delay is optimized, the model is reconstructed from the angle of deep separable convolution, and the optimized character positioning network structure is obtained. The character recognition model makes full use of shallow fine-grained characteristics in the network calculation process through improvement of a YOLOv3 algorithm, and achieves the detection effect of multi-scale character targets. And combining the character content matching with the face feature vector matching result, judging whether the certificate picture shot in from multiple dimensions is matched with the user information in the operator big data, and acquiring the user related data. When characters or face images in the photographed certificate picture are lost due to incorrect placement of the certificate, recognition information cannot be completely provided, and the situations of recognition error, ineffective recognition and the like are caused. The multi-dimensional judgment is carried out through the matching of the character content and the face features, the identification accuracy is improved, and the problem that the information cannot be identified and the identification is wrong due to incomplete single identification mode is avoided.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.

Claims (10)

1. A big data based certificate information identification and matching method is characterized by comprising the following steps:

acquiring a picture containing certificate content by using image acquisition equipment, preprocessing the picture, extracting an effective area comprising certificate information by adopting a semantic segmentation algorithm, and making a sample data set;

building a character recognition model, inputting a sample image data set into the model for recognition, outputting a recognized result, and converting the recognized result into characters;

building a face recognition model, inputting a sample image data set into the model for recognition, and outputting a recognized result to obtain a face feature vector;

storing picture information in an operator database by adopting a big data architecture improved HDFS, wherein the picture information comprises a picture MD5 value, character contents in a picture, a face characteristic vector in the picture and user information;

adopting a Faiss similarity search tool to perform character content matching and face image feature vector matching, classifying and judging matching results, and outputting the matching results;

and acquiring a certificate sample test picture, respectively inputting the certificate sample test picture into a character recognition model and a face recognition model for recognition, performing big data matching through a Faiss similarity search tool, and outputting related information of the certificate.

2. The method for identifying and matching certificate information based on big data as claimed in claim 1, wherein said using image collecting device, collecting the picture containing the certificate content, preprocessing the picture, extracting the effective area including the certificate information by semantic segmentation algorithm, making the sample data set includes:

the method comprises the steps that an image acquisition device of a service network point is utilized to shoot a certificate picture which needs to be issued when a user transacts services, the obtained picture is a sample picture which is not placed correctly and causes the loss of part of the content of the certificate, and the sample picture is classified into two types of pictures, namely, a character part loss and an image part loss;

establishing a certificate image semantic segmentation model to extract partial areas containing certificate contents, then respectively introducing a space attention module and a channel attention module, and processing a feature map acquired from the model; identifying and segmenting the certificate body part in the sample image by adopting a FocalLoss loss function to replace a cross entropy loss function to obtain a part area image of the certificate containing certificate content;

processing the extracted partial area image containing the certificate content as an information sample picture to obtain a white background picture, placing the information sample picture on the upper layer of the white background picture, adjusting the center point of the information picture sample to be superposed with the center point of the white background picture, adjusting the size of the information sample picture to be scaled in equal proportion, completely placing the information sample picture in the white background picture, and only one vertex touches the boundary of the white background picture to obtain the white background picture containing the information sample picture, thereby preparing a sample image data set.

3. The big-data-based certificate information identification and matching method as claimed in claim 1, wherein the building of the character identification model, the inputting of the sample image data set into the model for identification, the outputting of the identified result, and the converting into characters comprises:

building an SSD algorithm basic framework, wherein the basic network adopts VGG-16, zooming images is carried out when inputting, the main network adopts Conv5_3 and the previous partial structure of the VGG-16, after the images pass through the network, the output size of a feature matrix is 19 × 512, then feature graphs of 19 × 19 1024, 10 × 512,5 × 256,3 × 256,1 × 256 are sequentially converted through 5 layers of convolution layers, and finally, a detection result is obtained according to a non-maximum inhibition optimization algorithm;

optimizing an SSD algorithm basic framework, adding an RFB module into an SSD network, replacing a VGG16 backbone network in the SSD network by adopting a MobileNet V3 network to obtain a lightweight network structure, reconstructing a model from the perspective of deep separable convolution to obtain an optimized character positioning network structure, and inputting an ICDAR 2017RCTW data set into the character positioning network structure for training to obtain a character positioning model;

building a YOLOv3 network frame, optimizing the network frame, specifically, improving a fourth detection end of the YOLOv3, after the YOLOv3 algorithm completes the 3 rd time characteristic scale processing, adopting 2 times of upsampling processing to expand the output characteristic scale from 52 x 52 to 104 x 104, then utilizing a Route layer to perform characteristic fusion on the output of the front layer 104 x 104 characteristic scale and the output of the 11 th layer in a Darknet structure, completing the 4 th time characteristic scale detection to obtain the optimized YOLOv3 network frame, inputting an ICDAR 2017RCTW data set into the optimized YOLOv3 network frame for training to obtain a content identification model;

inputting pictures in the sample image data set into a character positioning model for character area positioning extraction to obtain character sample pictures; and inputting the character sample picture into the content recognition model for character recognition, and outputting the recognized character content to obtain a final character recognition model.

4. The big-data-based certificate information identification matching method as claimed in claim 1, wherein the building of the face identification model, inputting the sample image dataset into the model for identification, and outputting the identified result to obtain the face feature vector comprises:

building a DeepID basic network frame and optimizing, additionally changing the third and fourth layer of volume base layers into a sharing and introducing central loss verification function, and extracting 512-dimensional feature vectors by the most depepid layer; CNN of the center loss and the softmax loss joint supervised learning is introduced into the feature output layer to obtain a human face feature extraction network framework;

acquiring a picture in a sample image data set, intercepting a face block diagram in the picture, expanding proper pixels, then cutting to perform face preprocessing, wherein the preprocessing is face alignment, and affine transformation is performed on the picture by taking five reference points of the face as a standard;

after the face is preprocessed, the preprocessed cut face is input into a face feature extraction network framework to extract face features, and face feature vectors are obtained.

5. The big-data-architecture-based certificate information identification and matching method as claimed in claim 1, wherein the storing of the picture information in the operator database by using the big-data-architecture-improved HDFS, the picture information including the picture MD5 value, the text content in the picture, the face feature vector in the picture, and the user information includes:

storing by adopting a big data architecture HDFS, optimizing and improving the architecture, selectively transferring partial functions of a Namenode and control authority of a system, and storing file metadata of a current storage node by using a Datanode memory to obtain an improved HDFS;

storing user information collected by an operator by using an improved HDFS;

reading a file from an HDFS (Hadoop distributed File System) storage, and when a read-write request is initiated to a system, firstly selecting the most frequently accessed dataode from a dataode list cached in a memory of the system; if the connection with the hotspot Datanode can be established, searching a target file of the current reading operation in the Datanode, and determining a target data block according to the mapping relation from the file in the metadata to the data block; after the target block is determined, verifying whether the modification identifiers in the IDs of all backup blocks are uniform, if the identifiers are uniform, verifying that the data of the current target block is certain to be the latest data, and then starting to read the file from the target block according to the intra-block offset and the file length of the target file in the metadata; if the condition that the modification identifiers of the copy blocks corresponding to the target block are not uniform is verified, the fact that the file writing of the target block is possibly not completed on all the copies of the target block is shown, and therefore the target block is connected with the Namenode to obtain the latest data of the current target block, and the reading operation of the target file is completed.

6. The big-data-based certificate information identification and matching method as claimed in claim 1, wherein said adopting the Faiss similarity search tool to perform text content matching, face image feature vector matching, classifying and discriminating the matching result, and outputting the matching result comprises:

the Faiss feature index service pulls the improved HDFS to a corresponding fragment file, loads a feature file, performs segmentation and dirty data cleaning on features, and performs feature index construction after the feature file is loaded;

requesting an index base to obtain a result, sending the feature vector to the index base for similarity result search, querying the vector feature after the index base receives the request, and calling a Faiss index for vector search;

and outputting a vector search result, performing character matching and face detection, setting the confidence coefficient of a matching result, and judging the matching result.

7. The big-data-based certificate information identification and matching method as claimed in claim 1, wherein the obtaining of the certificate sample test picture, inputting into the character recognition model and the face recognition model for recognition, respectively, performing big data matching through a Faiss similarity search tool, and outputting the related information of the certificate comprises:

acquiring a certificate sample test picture, inputting the certificate sample test picture into a character recognition model for character content extraction, and then inputting the certificate sample test picture into a face recognition model for face feature vector extraction to obtain the character content and the face feature vector of the test picture;

inputting the text content and the face feature vector of the test picture into a Faiss similarity search tool for vector retrieval, outputting characters and face pictures with high confidence degrees with the test picture, and acquiring a user number corresponding to the similar characters and a user number corresponding to the similar face pictures;

comparing the user number corresponding to the similar characters with the user number corresponding to the similar face picture, and selecting the user number corresponding to the similar characters as the same as the user number corresponding to the similar face picture as a final result to obtain a matched user number;

and acquiring user information including user name, user number, identity card number, gender, address, mobile phone number and face picture by matching the user number.

8. A big data based certificate information identification matching system is characterized by comprising:

the acquisition module is used for acquiring a picture containing certificate content by using image acquisition equipment, preprocessing the picture, extracting an effective area comprising certificate information by adopting a semantic segmentation algorithm and manufacturing a sample data set;

the character recognition module is used for building a character recognition model, inputting the sample image data set into the model for recognition, outputting a recognized result and converting the recognized result into characters;

the face recognition module is used for building a face recognition model, inputting the sample image data set into the model for recognition, and outputting a recognized result to obtain a face feature vector;

the picture information storage module is used for storing picture information in an operator database by adopting a big data architecture improved HDFS, and the picture information comprises a picture MD5 value, text contents in a picture, face characteristic vectors in the picture and user information;

the matching module is used for matching character content and facial image feature vectors by adopting a Faiss similarity searching tool, classifying and judging matching results and outputting the matching results;

and the output module is used for acquiring a certificate sample test picture, respectively inputting the certificate sample test picture into the character recognition model and the face recognition model for recognition, performing big data matching through a Faiss similarity searching tool, and outputting the related information of the certificate.

9. The big-data certificate information-based identification matching system as claimed in claim 8, comprising:

at least one memory for storing computer instructions;

at least one processor in communication with the memory, wherein the at least one processor, when executing the computer instructions, causes the system to perform: the system comprises an acquisition module, a character recognition module, a face recognition module, a picture information storage module, a matching module and an output module.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.

Images