CN110490193A - Single Text RegionDetection method and ticket contents recognition methods - Google Patents
Single Text RegionDetection method and ticket contents recognition methods Download PDFInfo
- Publication number
- CN110490193A CN110490193A CN201910668919.0A CN201910668919A CN110490193A CN 110490193 A CN110490193 A CN 110490193A CN 201910668919 A CN201910668919 A CN 201910668919A CN 110490193 A CN110490193 A CN 110490193A
- Authority
- CN
- China
- Prior art keywords
- picture
- bill
- identified
- field area
- field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000001514 detection method Methods 0.000 claims description 52
- 238000012549 training Methods 0.000 claims description 33
- 238000012360 testing method Methods 0.000 claims description 12
- 238000013135 deep learning Methods 0.000 claims description 11
- 238000007689 inspection Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 4
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000012937 correction Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 230000009471 action Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003754 machining Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Character Input (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to intelligence to do account technical field, propose single Text RegionDetection method and ticket contents recognition methods, including obtaining field area picture to be identified, and the single character zone in field area picture to be identified is labeled, obtains single character zone picture；Various sizes of field area picture to be identified is zoomed into fixed size；It is operated by convolution, pondization, obtains first layer characteristic pattern；Field area characteristic pattern is extracted by VGG-Net16 network；Initial detecting frame is set, and is sent into softmax layers, is chosen by output probability score and proposes window；To window progress pondization operation is proposed, it will propose that window is normalized to fixed size, the feature vector of unified dimensional；Feature vector is sent into full articulamentum, frame is calculated and returns, obtain frame offset.Through the above technical solutions, solving the problems, such as that the recognition accuracy of ticket contents in the prior art is low.
Description
Technical field
The invention belongs to intelligence to do account technical field, be related to single Text RegionDetection method and ticket contents identification side
Method.
Background technique
In property tax field, accounting before doing account needs that various types of bills are scanned or are shot, and will take
Bill picture in important word content identify, such as the amount of money, date and Business Name of making out an invoice etc..Due to scanner or
Various image documentation equipments can be by many background information intakes unrelated with bill wherein, simultaneously because bill when shooting bill picture
Many kinds of, the extraneous factors such as dump is unclear, photographed scene is complicated influences, field contents to be identified will appear it is fuzzy or
Person's deformation, these all can cause the recognition accuracy to ticket contents low.
Summary of the invention
The present invention proposes single Text RegionDetection method and ticket contents recognition methods, solves bill in the prior art
The low problem of the recognition accuracy of content.
The technical scheme of the present invention is realized as follows: including
S10: field area picture to be identified is obtained, and to the single character zone in the field area picture to be identified
It is labeled, obtains single character zone picture；
S11: zooming to fixed size for various sizes of field area picture to be identified, obtain uniform sizes picture, note
The height of the uniform sizes picture is H pixel, and width is W pixel, the size of the uniform sizes picture for H ×
W pixel；
S12: obtained uniform sizes picture is done into convolution, pondization operation, obtains first layer characteristic pattern；
S13: the obtained first layer characteristic pattern is extracted into field area characteristic pattern by VGG-Net16 network；
S14: each pixel of the obtained field area characteristic pattern is respectively provided with the various sizes of initial inspection of M kind
It surveys frame and corresponding 4 offsets, 4 offsets includes the centre coordinate of the initial detecting frame, the initial detecting
The width of the length of frame and the initial detecting frame, it is each described first by H × W × M described softmax layers of initial detecting frames feeding
Beginning detection block obtains two probability scores；
S15: the initial detecting frame for belonging to prospect is filtered out according to probability score；
S16: initial detecting frame obtained in step S15 is carried out according to probability score, by non-maxima suppression method
Sequence, the proposal for choosing top n result as single character zone export, and complete the extraction for proposing window；
S17: the obtained proposal window is mapped on the field area characteristic pattern, by interest pond layer to institute
It states and proposes that window carries out pondization operation, by the different size of spy for proposing window and being normalized to fixed size, unified dimensional
Levy vector；
S18: sending described eigenvector into full articulamentum, calculates frame using loss function Smooth L1Loss and returns,
The frame offset for exporting single character zone, completes single Text RegionDetection.
Further, judge that each initial detecting frame belongs to prospect or background according to probability score in step S15
Specific standards are as follows: when the probability score of the probability score and single character zone picture of some initial detecting frame
When IOU >=0.8, judge the initial detecting frame for prospect.
Further, the value range of M is 8~10 in step S14, and the value range of N is 280~320 in step S16.
The invention also provides a kind of ticket contents recognition methods, including
S21: bill pictures are obtained；
S22: all document field pictures that bill picture is concentrated by the picture annotation tool in deep learning field
The mark of document field is carried out, while its field area to be identified and single character zone are marked to each document field, and protect
Field area information-recording to be identified is deposited, is concentrated in the bill shooting picture marked, randomly selects 80% picture file structure
At training sample set, using the picture file of residue 20% as test sample collection；
S23: according to bill type, training samples number is counted, the bill for training samples number less than 20 carries out structure
Expansion is made, the training sample set of equal number is obtained；
S24: it by first 4 layers of deep learning network VGG-Net16 as basic network layer, and is formed in conjunction with pyramid network
The network structure of document field detection model, the bill picture that training sample is concentrated is as the defeated of document field detection model
Enter, using the document field data information of mark as the output of document field detection model, training is iterated, until bill area
Output accuracy rate of the domain detection model in test sample collection is greater than previously given threshold value, obtains trained document field inspection
Survey model；
S25: it by first 4 layers of deep learning network VGG-Net16 as basic network layer, and is formed in conjunction with pyramid network
The network structure of field area detection model to be identified, the document field mark picture that training sample is concentrated is as word to be identified
The input of section region detection model, using the field area data information to be identified of mark as field area detection model to be identified
Output, training is iterated, until output accuracy rate of the field area detection model to be identified in test sample collection is greater than
Previously given threshold value obtains trained field area detection model to be identified；
S26: the step of according to S11~S17, detecting the single character zone in field area picture to be identified,
Obtain single character zone image；
S27: using VGG-Net16 as network structure, using single character zone image as input, by field area to be identified
Domain information-recording carries out the training of information-recording identification model in region to be identified as output, records and believes until region to be identified
It ceases output accuracy rate of the identification model in test sample collection and is greater than previously given threshold value, obtain trained region to be identified
Information-recording identification model；
S28: successively load trained document field detection model file, detection model file in field area to be identified,
Information-recording identification model file in region to be identified, and start the web interface service of document field segmentation, with Base64 coding
Form returns to the information of every bill record, completes the identification of ticket contents.
Further, the method that training sample expands in step S23 includes image mixing method and figure layer mixed method, described
Figure mixed method specifically: sample bill picture and another bill background are overlapped according to the ratio of 6:4, formed new
Picture, the content of existing sample bill picture in new picture, and contain another bill background；
The figure layer mixed method specifically:
S231: sample bill picture and bill background picture are opened using photo-editing software；
S232: the constituency replaced in advance in selection bill background picture is replicated the constituency to the figure layer of sample bill picture, is incited somebody to action
The constituency is denoted as constituency one；
S233: the size in adjustment constituency one is to be adapted to sample bill picture, Load Selection one, then contract selection one 3~5
A pixel deletes the corresponding constituency of sample bill figure layer,
S234: the figure layer where figure layer and constituency one where simultaneous selection sample bill is ordered using automatic mixing figure layer
It enables, obtains the picture after the mixing of panorama picture formation figure layer, complete the expansion of sample bill.
Further, step S21 includes
S211: connection scanner reads the image information of bill；
S212: handling the image information of bill, including picture compression, picture enhance, go background process and picture
Correction for direction.
It the working principle of the invention and has the beneficial effect that
1, of the invention that window is proposed by extraction field area characteristic pattern, extraction, will propose that window is normalized to fix greatly
Small feature vector and the detection for being finally completed single character zone, are advantageously implemented the identification to character content.For example, bill
On the amount of money be 23.4 yuan, existing identification method is that all texts of whole bill are identified, due to texts various in bill
Size, font, the difference of printing effect of word, whole bill accuracy rate of Direct Recognition is relatively low, using single in the present invention
Text RegionDetection method can carry out the area of the region detection of character " 2 ", the region detection of character " 3 ", character " " first
Domain detection, the region detection of character " 4 " and character " member " region detection, then each character machining region is carried out respectively
Text region, such specific aim is stronger, and recognition accuracy is high.
Wherein, step S11 is used to various sizes of field area picture to be identified zooming to fixed size, and use is existing
The resize method of Opencv can accomplish that S12~S13 is for extracting field area characteristic pattern, if constructing in S14 step
Dry initial detecting frame, then passes through S15~S16, the immediate N number of initial inspection of single character zone selected and actually marked
Frame is surveyed, step S17~S18 comprehensively considers the N number of initial detecting frame selected in step S16, obtains final single character zone.
2, IOU indicates friendship and is a concept in object detection field than (Intersection-over-Union), this
In we concern field area to be identified, belong to foreground part, by the comparison of IOU, choose belong to foreground part just
Beginning detection block.
3, as shown in Figure 1, for document field mark, field area to be identified mark and single character zone mark signal
Figure, wherein document field mark is using a rectangle frame, the image of the interior only bill of rectangle frame, each word to be identified
Section region and single character zone are also marked by a rectangle frame respectively.
Ticket contents recognition methods of the present invention is based on deep learning theory, successively carries out document field from bill picture concentration
Detection, field area to be identified is detected and single Text RegionDetection, after the completion of single Text RegionDetection, just for single word
Contents in symbol region are identified, the accuracy rate of character recognition can be greatly improved, to improve entire ticket contents
The accuracy rate of identification.
Construction expansion is carried out to the training sample of negligible amounts in the present invention, guarantees the data substantially one of each type bill
Sample, the accuracy rate that study comes out in this way will be very high, be not in certain type of bill feature less than the phenomenon that, favorably
Various bills are accurately identified in realizing.
4, image mixing method can easily be realized by graphics editing software, such as Photoshop in the present invention, complete
At the expansion of rare sample；The realization bill of the scripting language batch of Photoshop software also can be used in figure layer mixed method
Text replacement in picture achievees the purpose that expand rare sample.The training sample extending method used in the present invention can not only
Enough effective expansions for realizing rare sample, and it is easy to operate, practical.
5, the present invention is after obtaining bill images information by scanner, internal molar paste, shooting deformation and shooting field
The bill of scape complexity is pre-processed, and keeps billing information readily identified, and then improves the accuracy rate of ticket contents identification.
Detailed description of the invention
The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
Fig. 1 is that document field mark in the present invention, field area to be identified mark and single character zone mark schematic diagram；
Fig. 2 is the single Text RegionDetection flow chart of the present invention；
Fig. 3 is ticket contents identification process figure in the present invention；
In figure: 1- bill pictures, 2- document field, the field area to be identified 3-, the single character zone of 4-.
Specific embodiment
Below in conjunction with the embodiment of the present invention, technical scheme in the embodiment of the invention is clearly and completely described,
Obviously, described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Based in the present invention
Embodiment, every other embodiment obtained by those of ordinary skill in the art without making creative efforts, all
Belong to the scope of protection of the invention.
As shown in Figure 1-Figure 3, including
S10: field area picture to be identified is obtained, and to the single character zone in the field area picture to be identified
It is labeled, obtains single character zone picture；
S11: zooming to fixed size for various sizes of field area picture to be identified, obtain uniform sizes picture, note
The height of the uniform sizes picture is H pixel, and width is W pixel, i.e., the size of the described uniform sizes picture is H
× W pixel；
S12: obtained uniform sizes picture is done into convolution, pondization operation, obtains first layer characteristic pattern；
S13: the obtained first layer characteristic pattern is extracted into field area characteristic pattern by VGG-Net16 network；
S14: each pixel of the obtained field area characteristic pattern is respectively provided with 9 kinds of various sizes of initial inspections
It surveys frame and corresponding 4 offsets, 4 offsets includes the centre coordinate of the initial detecting frame, the initial detecting
The width of the length of frame and the initial detecting frame, it is each described first by described softmax layers of initial detecting frame feeding of H × W × 9
Beginning detection block obtains two probability scores；
S15: the initial detecting frame for belonging to prospect is filtered out according to probability score；
S16: initial detecting frame obtained in step S15 is carried out according to probability score, by non-maxima suppression method
Sequence, the proposal for choosing top n result as single character zone export, and complete the extraction for proposing window；
S17: the obtained proposal window is mapped on the field area characteristic pattern, by interest pond layer to institute
It states and proposes that window carries out pondization operation, by the different size of spy for proposing window and being normalized to fixed size, unified dimensional
Levy vector；
S18: sending described eigenvector into full articulamentum, calculates frame using loss function Smooth L1Loss and returns,
The frame offset for exporting single character zone, completes single Text RegionDetection.
The present invention is by extracting field area characteristic pattern, extraction proposal window, proposing that window is normalized to fixed size
Feature vector and be finally completed the detection of single character zone, be advantageously implemented the identification to character content.For example, on bill
The amount of money be 23.4 yuan, existing identification method is that all texts of whole bill are identified, due to texts various in bill
Size, font, the difference of printing effect, whole bill accuracy rate of Direct Recognition is relatively low, using the single word in the present invention
Method for detecting area is accorded with, the region of the region detection of character " 2 ", the region detection of character " 3 ", character " " can be carried out first
It detects, the region detection of the region detection of character " 4 " and character " member ", text is then carried out respectively to each character machining region
Word identification, such specific aim is stronger, and recognition accuracy is high.
Wherein, step S11 is used to various sizes of field area picture to be identified zooming to fixed size, and use is existing
The resize method of Opencv can accomplish that S12~S13 is for extracting field area characteristic pattern, if constructing in S14 step
Dry initial detecting frame, then passes through S15~S16, the immediate N number of initial inspection of single character zone selected and actually marked
Frame is surveyed, step S17~S18 comprehensively considers the N number of initial detecting frame selected in step S16, obtains final single character zone.
Further, judge that each initial detecting frame belongs to prospect or background according to probability score in step S15
Specific standards are as follows: when the probability score of the probability score and single character zone picture of some initial detecting frame
When IOU >=0.8, judge the initial detecting frame for prospect.
IOU expression hands over and is a concept in object detection field than (Intersection-over-Union), here
We concern field area to be identified, belong to foreground part, by the comparison of IOU, choose and belong to the initial of foreground part
Detection block.
Further, the value range of N is 280~320 in step S16.
The invention also provides a kind of ticket contents recognition methods, including
S21: bill pictures are obtained；
S22: all document field pictures that bill picture is concentrated by the picture annotation tool in deep learning field
The mark of document field is carried out, while its field area to be identified and single character zone are marked to each document field, and protect
Field area information-recording to be identified is deposited, is concentrated in the bill shooting picture marked, randomly selects 80% picture file structure
At training sample set, using the picture file of residue 20% as test sample collection；
S23: according to bill type, training samples number is counted, the bill for training samples number less than 20 carries out structure
Expansion is made, the training sample set of equal number is obtained；
S24: it by first 4 layers of deep learning network VGG-Net16 as basic network layer, and is formed in conjunction with pyramid network
The network structure of document field detection model, the bill picture that training sample is concentrated is as the defeated of document field detection model
Enter, using the document field data information of mark as the output of document field detection model, training is iterated, until bill area
Output accuracy rate of the domain detection model in test sample collection is greater than previously given threshold value, obtains trained document field inspection
Survey model；
S25: it by first 4 layers of deep learning network VGG-Net16 as basic network layer, and is formed in conjunction with pyramid network
The network structure of field area detection model to be identified, the document field mark picture that training sample is concentrated is as word to be identified
The input of section region detection model, using the field area data information to be identified of mark as field area detection model to be identified
Output, training is iterated, until output accuracy rate of the field area detection model to be identified in test sample collection is greater than
Previously given threshold value obtains trained field area detection model to be identified；
S26: the step of according to S11~S17, detecting the single character zone in field area picture to be identified,
Obtain single character zone image；
S27: using VGG-Net16 as network structure, using single character zone image as input, by field area to be identified
Domain information-recording carries out the training of information-recording identification model in region to be identified as output, records and believes until region to be identified
It ceases output accuracy rate of the identification model in test sample collection and is greater than previously given threshold value, obtain trained region to be identified
Information-recording identification model；
S28: successively load trained document field detection model file, detection model file in field area to be identified,
Information-recording identification model file in region to be identified, and start the web interface service of document field segmentation, with Base64 coding
Form returns to the information of every bill record, completes the identification of ticket contents.
As shown in Figure 1, schematic diagram is marked for document field mark, field area to be identified mark and single character zone,
Wherein document field mark is using a rectangle frame, the image of the interior only bill of rectangle frame, each field to be identified
Region and single character zone are also marked by a rectangle frame respectively.
Ticket contents recognition methods of the present invention is based on deep learning theory, successively carries out document field from bill picture concentration
Detection, field area to be identified is detected and single Text RegionDetection, after the completion of single Text RegionDetection, just for single word
Contents in symbol region are identified, the accuracy rate of character recognition can be greatly improved, to improve entire ticket contents
The accuracy rate of identification.
Construction expansion is carried out to the training sample of negligible amounts in the present invention, guarantees the data substantially one of each type bill
Sample, the accuracy rate that study comes out in this way will be very high, be not in certain type of bill feature less than the phenomenon that, favorably
Various bills are accurately identified in realizing.
Further, the method that training sample expands in step S23 includes image mixing method and figure layer mixed method, described
Figure mixed method specifically: sample bill picture and another bill background are overlapped according to the ratio of 6:4, formed new
Picture, the content of existing sample bill picture in new picture, and contain another bill background；
The figure layer mixed method specifically:
S231: sample bill picture and bill background picture are opened using photo-editing software；
S232: the constituency replaced in advance in selection bill background picture is replicated the constituency to the figure layer of sample bill picture, is incited somebody to action
The constituency is denoted as constituency one；
S233: the size in adjustment constituency one is to be adapted to sample bill picture, Load Selection one, then contract selection one 3~5
A pixel deletes the corresponding constituency of sample bill figure layer,
S234: the figure layer where figure layer and constituency one where simultaneous selection sample bill is ordered using automatic mixing figure layer
It enables, obtains the picture after the mixing of panorama picture formation figure layer, complete the expansion of sample bill.
Image mixing method can easily be realized by graphics editing software, such as Photoshop in the present invention, be completed
The expansion of rare sample；The realization bill of the scripting language batch of Photoshop software also can be used in figure layer mixed method
Text replacement in piece achievees the purpose that expand rare sample.The training sample extending method used in the present invention can not only
Realize effective expansion of rare sample, and easy to operate, practical.
Further, step S21 includes
S211: connection scanner reads the image information of bill；
S212: handling the image information of bill, including picture compression, picture enhance, go background process and picture
Correction for direction.
The present invention is after obtaining bill images information by scanner, internal molar paste, shooting deformation and photographed scene
Complicated bill is pre-processed, and keeps billing information readily identified, and then improves the accuracy rate of ticket contents identification.
The above is merely preferred embodiments of the present invention, be not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (6)
1. single Text RegionDetection method, for identifying single character zone from bill picture, which is characterized in that including
S10: field area picture to be identified is obtained, and the single character zone in the field area picture to be identified is carried out
Mark, obtains single character zone picture；
S11: zooming to fixed size for various sizes of field area picture to be identified, obtains uniform sizes picture, described in note
The height of uniform sizes picture is H pixel, and width is W pixel, and the size of the uniform sizes picture is H × W
Pixel；
S12: obtained uniform sizes picture is done into convolution, pondization operation, obtains first layer characteristic pattern；
S13: the obtained first layer characteristic pattern is extracted into field area characteristic pattern by VGG-Net16 network；
S14: each pixel of the obtained field area characteristic pattern is respectively provided with the various sizes of initial detecting frame of M kind
With corresponding 4 offsets, 4 offsets include the centre coordinate of the initial detecting frame, the initial detecting frame
H × W × M initial detecting the frames are sent into softmax layers, each initial inspection by long and the initial detecting frame width
It surveys frame and obtains two probability scores；
S15: the initial detecting frame for belonging to prospect is filtered out according to probability score；
S16: initial detecting frame obtained in step S15 is arranged according to probability score, by non-maxima suppression method
Sequence, the proposal for choosing top n result as single character zone export, and complete the extraction for proposing window；
S17: the obtained proposal window is mapped on the field area characteristic pattern, is mentioned by interest pond layer to described
Discuss window carry out pondization operate, by the different size of proposal window be normalized to fixed size, the feature of unified dimensional to
Amount；
S18: sending described eigenvector into full articulamentum, calculates frame using loss function Smooth L1Loss and returns, output
The frame offset of single character zone, completes single Text RegionDetection.
2. single Text RegionDetection method according to claim 1, which is characterized in that obtained in step S15 according to probability
Divide and judge that each initial detecting frame belongs to the specific standards of prospect or background are as follows: when some initial detecting frame
Probability score and the single character zone picture probability score IOU >=0.8 when, judge that the initial detecting frame is
Prospect.
3. single Text RegionDetection method according to claim 1, which is characterized in that the value range of M in step S14
It is 8~10, the value range of N is 280~320 in step S16.
4. a kind of ticket contents recognition methods comprising single Text RegionDetection method described in claims 1 to 3, feature
It is, including
S21: bill pictures are obtained；
S22: all document field pictures that bill picture is concentrated are carried out by the picture annotation tool in deep learning field
The mark of document field, while its field area to be identified and single character zone marked to each document field, and save to
It identifies field area information-recording, is concentrated in the bill shooting picture marked, the picture file for randomly selecting 80% constitutes instruction
Practice sample set, using the picture file of residue 20% as test sample collection；
S23: according to bill type, training samples number is counted, the bill type to training samples number less than 20 constructs
Expand, obtains the training sample set of equal number；
S24: by first 4 layers of deep learning network VGG-Net16 as basic network layer, and pyramid network is combined to form bill
The network structure of region detection model, the bill picture that training sample is concentrated, will as the input of document field detection model
Output of the document field data information of mark as document field detection model, is iterated training, until document field is examined
It surveys output accuracy rate of the model in test sample collection and is greater than previously given threshold value, obtain trained document field detection mould
Type；
S25: it by first 4 layers of deep learning network VGG-Net16 as basic network layer, and is formed in conjunction with pyramid network wait know
The network structure of malapropism section region detection model, the document field mark picture that training sample is concentrated is as field area to be identified
The input of domain detection model, using the field area data information to be identified of mark as the defeated of field area detection model to be identified
Out, it is iterated training, until output accuracy rate of the field area detection model to be identified in test sample collection is greater than in advance
Given threshold value obtains trained field area detection model to be identified；
S26: the step of according to S11~S17, the single character zone in field area picture to be identified is detected, is obtained
Single character zone image；
S27: using VGG-Net16 as network structure, using single character zone image as input, field area to be identified is remembered
Information carrying breath carries out the training of information-recording identification model in region to be identified as output, until region information-recording to be identified is known
Output accuracy rate of the other model in test sample collection is greater than previously given threshold value, obtains trained region to be identified and records
Information identification model；
S28: trained document field detection model file, detection model file in field area to be identified are successively loaded, wait know
Other region information-recording identification model file, and start the web interface service of document field segmentation, the form encoded with Base64
The information for returning to the record of every bill, completes the identification of ticket contents.
5. ticket contents recognition methods according to claim 3, which is characterized in that training sample expands in step S23
Method includes image mixing method and figure layer mixed method, the figure mixed method specifically: by sample bill picture and separately
One bill background is overlapped according to the ratio of 6:4, forms new picture, existing sample bill picture is interior in new picture
Hold, and contains another bill background；
The figure layer mixed method specifically:
S231: sample bill picture and bill background picture are opened using photo-editing software；
S232: the constituency replaced in advance in selection bill background picture, replicate the constituency to sample bill picture figure layer, by the choosing
Area is denoted as constituency one；
S233: the size in adjustment constituency one is to be adapted to sample bill picture, Load Selection one, then one 3~5 pictures of contract selection
Element deletes the corresponding constituency of sample bill figure layer,
S234: the figure layer where figure layer and constituency one where simultaneous selection sample bill is obtained using automatic mixing layers command
Picture to after the mixing of panorama picture formation figure layer, completes the expansion of sample bill.
6. ticket contents recognition methods according to claim 3, which is characterized in that step S21 includes
S211: connection scanner reads the image information of bill；
S212: handling the image information of bill, including picture compression, picture enhance, go background process and picture direction
Correction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910668919.0A CN110490193B (en) | 2019-07-24 | 2019-07-24 | Single character area detection method and bill content identification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910668919.0A CN110490193B (en) | 2019-07-24 | 2019-07-24 | Single character area detection method and bill content identification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110490193A true CN110490193A (en) | 2019-11-22 |
| CN110490193B CN110490193B (en) | 2022-11-08 |
Family
ID=68548038
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910668919.0A Active CN110490193B (en) | 2019-07-24 | 2019-07-24 | Single character area detection method and bill content identification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110490193B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027443A (en) * | 2019-12-04 | 2020-04-17 | 华南理工大学 | Bill text detection method based on multitask deep learning |
| CN111507352A (en) * | 2020-04-16 | 2020-08-07 | 腾讯科技（深圳）有限公司 | Image processing method and device, computer equipment and storage medium |
| CN112308036A (en) * | 2020-11-25 | 2021-02-02 | 杭州睿胜软件有限公司 | Bill identification method and device and readable storage medium |
| CN112733726A (en) * | 2021-01-12 | 2021-04-30 | 海尔数字科技(青岛)有限公司 | Bill sample capacity expansion method and device, electronic equipment and storage medium |
| CN112925837A (en) * | 2019-12-06 | 2021-06-08 | 上海高德威智能交通***有限公司 | Text structuring method and device |
| CN113468906A (en) * | 2021-07-12 | 2021-10-01 | 深圳思谋信息科技有限公司 | Graphic code extraction model construction method, recognition method, device, equipment and medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650721A (en) * | 2016-12-28 | 2017-05-10 | 吴晓军 | Industrial character identification method based on convolution neural network |
| CN107766809A (en) * | 2017-10-09 | 2018-03-06 | 平安科技（深圳）有限公司 | Electronic installation, billing information recognition methods and computer-readable recording medium |
| CN107798299A (en) * | 2017-10-09 | 2018-03-13 | 平安科技（深圳）有限公司 | Billing information recognition methods, electronic installation and readable storage medium storing program for executing |
| KR101858099B1 (en) * | 2017-02-03 | 2018-06-27 | 인천대학교 산학협력단 | Method and apparatus for detecting vehicle plates |
| CN108446621A (en) * | 2018-03-14 | 2018-08-24 | 平安科技（深圳）有限公司 | Bank slip recognition method, server and computer readable storage medium |
| CN108596066A (en) * | 2018-04-13 | 2018-09-28 | 武汉大学 | A kind of character identifying method based on convolutional neural networks |
| CN109284750A (en) * | 2018-08-14 | 2019-01-29 | 北京市商汤科技开发有限公司 | Bank slip recognition method and device, electronic equipment and storage medium |
| CN109784342A (en) * | 2019-01-24 | 2019-05-21 | 厦门商集网络科技有限责任公司 | A kind of OCR recognition methods and terminal based on deep learning model |
-
2019
- 2019-07-24 CN CN201910668919.0A patent/CN110490193B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650721A (en) * | 2016-12-28 | 2017-05-10 | 吴晓军 | Industrial character identification method based on convolution neural network |
| KR101858099B1 (en) * | 2017-02-03 | 2018-06-27 | 인천대학교 산학협력단 | Method and apparatus for detecting vehicle plates |
| CN107766809A (en) * | 2017-10-09 | 2018-03-06 | 平安科技（深圳）有限公司 | Electronic installation, billing information recognition methods and computer-readable recording medium |
| CN107798299A (en) * | 2017-10-09 | 2018-03-13 | 平安科技（深圳）有限公司 | Billing information recognition methods, electronic installation and readable storage medium storing program for executing |
| CN108446621A (en) * | 2018-03-14 | 2018-08-24 | 平安科技（深圳）有限公司 | Bank slip recognition method, server and computer readable storage medium |
| CN108596066A (en) * | 2018-04-13 | 2018-09-28 | 武汉大学 | A kind of character identifying method based on convolutional neural networks |
| CN109284750A (en) * | 2018-08-14 | 2019-01-29 | 北京市商汤科技开发有限公司 | Bank slip recognition method and device, electronic equipment and storage medium |
| CN109784342A (en) * | 2019-01-24 | 2019-05-21 | 厦门商集网络科技有限责任公司 | A kind of OCR recognition methods and terminal based on deep learning model |
Non-Patent Citations (4)
| Title |
|---|
| TOMAS BJÖRKLUND ET AL.: "Robust license plate recognition using neural networks trained on synthetic images", 《PATTERN RECOGNITION》 * |
| 王家伟: "基于链接线的自然场景文字检测技术的研究", 《中国优秀硕士学位论文全文数据库信息科辑》 * |
| 肖学锋: "移动端目标检测***的设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 董峻妃: "基于卷积神经网络的车牌区域检测和车牌字符识别研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027443A (en) * | 2019-12-04 | 2020-04-17 | 华南理工大学 | Bill text detection method based on multitask deep learning |
| CN111027443B (en) * | 2019-12-04 | 2023-04-07 | 华南理工大学 | Bill text detection method based on multitask deep learning |
| CN112925837A (en) * | 2019-12-06 | 2021-06-08 | 上海高德威智能交通***有限公司 | Text structuring method and device |
| CN111507352A (en) * | 2020-04-16 | 2020-08-07 | 腾讯科技（深圳）有限公司 | Image processing method and device, computer equipment and storage medium |
| CN111507352B (en) * | 2020-04-16 | 2021-09-28 | 腾讯科技（深圳）有限公司 | Image processing method and device, computer equipment and storage medium |
| CN112308036A (en) * | 2020-11-25 | 2021-02-02 | 杭州睿胜软件有限公司 | Bill identification method and device and readable storage medium |
| CN112733726A (en) * | 2021-01-12 | 2021-04-30 | 海尔数字科技(青岛)有限公司 | Bill sample capacity expansion method and device, electronic equipment and storage medium |
| CN113468906A (en) * | 2021-07-12 | 2021-10-01 | 深圳思谋信息科技有限公司 | Graphic code extraction model construction method, recognition method, device, equipment and medium |
| CN113468906B (en) * | 2021-07-12 | 2024-03-26 | 深圳思谋信息科技有限公司 | Graphic code extraction model construction method, identification device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110490193B (en) | 2022-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110490193A (en) | Single Text RegionDetection method and ticket contents recognition methods | |
| CN109948510B (en) | Document image instance segmentation method and device | |
| CN110399798B (en) | Discrete picture file information extraction system and method based on deep learning | |
| CN107423732A (en) | Vehicle VIN recognition methods based on Android platform | |
| CN109800761A (en) | Method and terminal based on deep learning model creation paper document structural data | |
| CN110414927A (en) | A kind of bill processing automatically generates the method and device of voucher | |
| CN114092700B (en) | Ancient character recognition method based on target detection and knowledge graph | |
| Prusty et al. | Indiscapes: Instance segmentation networks for layout parsing of historical indic manuscripts | |
| CN108364037A (en) | Method, system and the equipment of Handwritten Chinese Character Recognition | |
| CN106778717A (en) | A kind of test and appraisal table recognition methods based on image recognition and k nearest neighbor | |
| CN110781877A (en) | Image recognition method, device and storage medium | |
| CN111814576A (en) | Shopping receipt picture identification method based on deep learning | |
| CN112001200A (en) | Identification code identification method, device, equipment, storage medium and system | |
| CN113592735A (en) | Text page image restoration method and system, electronic equipment and computer readable medium | |
| US9129197B1 (en) | Highlight-based bill processing | |
| CN111626145A (en) | Simple and effective incomplete form identification and page-crossing splicing method | |
| Khan et al. | Car Number Plate Recognition (CNPR) system using multiple template matching | |
| CN110766001B (en) | Bank card number positioning and end-to-end identification method based on CNN and RNN | |
| CN117351505A (en) | Information code identification method, device, equipment and storage medium | |
| CN109544134B (en) | Convenient payment service method and system | |
| CN111881880A (en) | Bill text recognition method based on novel network | |
| CN113065559B (en) | Image comparison method and device, electronic equipment and storage medium | |
| CN111680691B (en) | Text detection method, text detection device, electronic equipment and computer readable storage medium | |
| CN110991265B (en) | Layout extraction method for train ticket image | |
| Khan | A novel segmentation dataset for signatures on bank checks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240409 Address after: 710100 11a-1-5, Chang'an innovation and entrepreneurship center, Wenyuan Middle Road, Guodu street, Chang'an District, Xi'an City, Shaanxi Province Patentee after: Shaanxi taoding Information Technology Co.,Ltd. Country or region after: China Address before: 710000 Room 102, block a, Chang'an cultural center, Wenyuan South Road, Guodu Street office, Chang'an District, Xi'an City, Shaanxi Province Patentee before: Xi'an Network Computing Data Technology Co.,Ltd. Country or region before: China |
|
| TR01 | Transfer of patent right |