CN114120320A - Image multi-target information identification method, system and medium - Google Patents
Image multi-target information identification method, system and medium Download PDFInfo
- Publication number
- CN114120320A CN114120320A CN202111258319.0A CN202111258319A CN114120320A CN 114120320 A CN114120320 A CN 114120320A CN 202111258319 A CN202111258319 A CN 202111258319A CN 114120320 A CN114120320 A CN 114120320A
- Authority
- CN
- China
- Prior art keywords
- image
- target
- area
- circumscribed rectangle
- minimum circumscribed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 238000012937 correction Methods 0.000 claims abstract description 24
- 238000012216 screening Methods 0.000 claims abstract description 24
- 230000009466 transformation Effects 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 39
- 230000000877 morphologic effect Effects 0.000 claims description 12
- 238000003708 edge detection Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 7
- 238000005260 corrosion Methods 0.000 claims description 4
- 230000007797 corrosion Effects 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 238000005520 cutting process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Character Input (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a system and a medium for identifying image multi-target information, wherein the method comprises the following steps: acquiring a gray image, drawing a minimum circumscribed rectangle of each region, screening the minimum circumscribed rectangle which accords with a preset length-width ratio and an area ratio, and taking a corresponding region as a target region; for each target area, respectively carrying out angle correction on the image by taking the center of the image as the center, and intercepting a first rectangular area taking the target area as the center in the corrected image to be used as an image to be processed; respectively detecting edge intersection points of each image to be processed, and carrying out perspective transformation on the image to be processed according to the edge intersection points to obtain a second rectangular area serving as an image to be identified; and performing character recognition on each image to be recognized to obtain an information recognition result of each target area. The method can accurately identify the target areas in one image, and can effectively correct the image of each target area, thereby improving the accuracy of subsequent information identification.
Description
Technical Field
The invention relates to the field of image recognition, in particular to a method, a system and a medium for recognizing multi-target information of an image.
Background
With the development of industrial internet and the continuous maturity of image recognition processing technology, the automatic digital information character recognition based on the image processing method gradually becomes an important link of industrial automation. For example, in the remote automatic meter reading technology, a dial image is obtained through a camera device, and information on the dial is identified through image processing. The meter can be a water meter or an electric meter in a character wheel form, and can also be a pointer-testing pressure meter and the like. For another example, the license plate recognition technology recognizes license plate information by processing a monitoring picture.
The general steps of image recognition include gray scale processing, binarization processing, recognition area detection, correction and interception, segmentation of each character, recognition of each character, and the like. In order to accurately recognize character information, detection, correction and interception of a recognition area are very critical, and accurate detection, correction and interception of a recognition target are the premise of accurate recognition of the character information.
At present, in a scheme for identifying image content, information identification is generally performed only on a single target in an image, for example, patent CN110414517A discloses an identification card text identification algorithm for identifying the content of a single identification card picture, but in an actual situation, there are often more than one targets needing to obtain information, for example, in a large monitoring or production system, there are generally multiple display screens, if identification is performed on information of a single screen picture in sequence according to the prior art, it is obviously time-consuming, and because there is a difference in installation position of each display screen, the angle deviations of the display screens in an image are different, and the accuracy of character identification during information identification may be low.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: aiming at the technical problems in the prior art, the invention provides an image multi-target information identification method, system and medium, which can accurately identify a plurality of target areas in one image, effectively correct the image of each target area and improve the accuracy of subsequent information identification.
In order to solve the technical problems, the technical scheme provided by the invention is as follows:
an image multi-target information identification method comprises the following steps:
s1) acquiring the preprocessed image, carrying out region division on the preprocessed image, drawing a minimum circumscribed rectangle of each region, screening the minimum circumscribed rectangle which accords with a preset length-width ratio and area ratio, and taking a region corresponding to the screened minimum circumscribed rectangle as a target region;
s2) for each target region, performing angle correction on the image respectively with the center of the target region, and intercepting the image of a first rectangular region centered on the target region from the corrected image as an image to be processed of the target region;
s3) respectively detecting edge intersection points in the to-be-processed image of each target area aiming at the to-be-processed image of each target area, and carrying out perspective transformation on the to-be-processed image according to the edge intersection points to obtain an image of a second rectangular area as the to-be-identified image of the target area;
s4) performing character recognition on each image to be recognized, and obtaining an information recognition result for each target area.
Further, the step of drawing the minimum bounding rectangle of each region in the image in step S1) specifically includes:
s111) carrying out Gaussian filtering and binarization processing on the image to obtain a first binarized image;
s112) performing morphological corrosion expansion processing on the first binary image to obtain a first morphological processing image;
s113) carrying out edge detection on the first morphological processing image to obtain edge data of each area;
s114) drawing the minimum bounding rectangle of each area according to the edge data.
Further, the step of screening the minimum circumscribed rectangle meeting the preset aspect ratio and area ratio in the step S1) specifically includes:
s121) selecting a current minimum circumscribed rectangle, calculating the length and the width of the current minimum circumscribed rectangle according to the number of pixel points in the length direction and the width direction of the current minimum circumscribed rectangle, and calculating the area of the current minimum circumscribed rectangle according to the length and the width;
s122) if the error between the aspect ratio of the current minimum circumscribed rectangle and the preset aspect ratio is smaller than the preset error, and the area occupation ratio of the current minimum circumscribed rectangle in the image is larger than or equal to a preset threshold value S, taking the region corresponding to the current minimum circumscribed rectangle as a target region;
s123) returning to the step S121) until all the minimum bounding rectangles in the image are traversed.
Further, step S123) is followed by a step of adjusting the size of the threshold S, specifically: and adjusting the value of the threshold value s to be the area ratio of the minimum circumscribed rectangle corresponding to the actual target area with the minimum area in the current image, and screening and reserving the minimum circumscribed rectangle with the area ratio larger than or equal to the adjusted threshold value s.
Further, step 2) is preceded by a step of verifying the target area, which specifically includes: and intercepting images surrounded by the minimum external rectangles corresponding to all the target areas, respectively carrying out character recognition on each intercepted image, and if no recognition information exists or recognition errors occur, rejecting the corresponding target areas.
Further, step S1) further includes: establishing a coordinate system by taking the horizontal direction as a transverse axis and the vertical direction as a longitudinal axis, acquiring the coordinates of the central point of the minimum circumscribed rectangle, and calculating the inclination direction and the inclination angle of the minimum circumscribed rectangle relative to the transverse axis according to the projection of the long edge of the minimum circumscribed rectangle and the long edge on the transverse axis; the step S2) of performing angle rectification on the image with the center of the target region specifically includes: and rotating the image in the opposite direction of the inclination direction according to the rotation center and the rotation angle value by taking the center point of the minimum circumscribed rectangle corresponding to the target area as a rotation center and the value of the inclination angle of the minimum circumscribed rectangle corresponding to the target area as a rotation angle value, so that the inclination angle of the minimum circumscribed rectangle corresponding to the target area relative to the horizontal axis is 0.
Further, the step S2) of cutting out the image of the first rectangular region centered on the target region from the corrected image specifically includes: and acquiring a central point of the minimum circumscribed rectangle corresponding to the target area, respectively calculating the coordinate value of the interception starting point and the length and width of the first rectangular area according to the coordinate value of the central point and a preset expansion value, and then intercepting the image of the first rectangular area from the interception starting point according to the length and width of the first rectangular area.
Further, respectively calculating the coordinate value of the interception start point and the length and width of the first rectangular region according to the coordinate value of the central point and the preset expansion value specifically includes: calculating the coordinate value of the vertex of the first rectangular region according to the coordinate value of the central point of the minimum external rectangle and a preset expansion value, taking the vertex of the first rectangular region as an intercepting starting point, wherein the coordinate value function expression of the vertex of the first rectangular region is as follows:
in the above formula, xe,yeRespectively the abscissa and ordinate, x, of the vertex of the first rectangular areac,ycRespectively an abscissa and an ordinate of a central point of the minimum circumscribed rectangle, W is a length value of the minimum circumscribed rectangle, H is a width value of the minimum circumscribed rectangle, col/10 is a preset length direction expansion value, col is the number of pixel points in the image length direction, row/10 is a preset width direction expansion value, and row is an image width direction expansion valueThe number of the pixel points;
calculating the length and the width of the first rectangular area according to the size of the minimum bounding rectangle and a preset expansion value, wherein the functional expression of the length and the width of the first rectangular area is as follows:
in the above formula, We,HeThe length and the width of the vertex of the first rectangular region are respectively, col/10 is a preset length direction expansion value, col is the number of pixels in the length direction of the image, row/10 is a preset width direction expansion value, and row is the number of pixels in the width direction of the image.
The invention also provides an image multi-target information identification system, which comprises:
the target screening unit is used for acquiring the preprocessed image, performing region division on the preprocessed image, drawing a minimum circumscribed rectangle of each region, screening the minimum circumscribed rectangle which accords with a preset length-width ratio and an area ratio, and taking a region corresponding to the screened minimum circumscribed rectangle as a target region;
the image first-time correction unit is used for respectively carrying out angle correction on the image by taking the center of the target area as the center of each target area, and intercepting the image of a first rectangular area taking the target area as the center in the corrected image to be used as the image to be processed of the target area;
the image second correction unit is used for respectively detecting edge intersection points in the images to be processed of each target area aiming at the images to be processed of the target areas, and performing perspective transformation on the images to be processed according to the edge intersection points to obtain images of a second rectangular area as the images to be identified of the target areas;
and the information identification unit is used for carrying out character identification on each image to be identified to obtain the information identification result of each target area.
The present invention also provides a computer-readable storage medium storing a computer program programmed or configured to execute any one of the image multi-target information recognition methods.
Compared with the prior art, the invention has the advantages that:
the invention draws the minimum circumscribed rectangle aiming at each area in the image, screens out the area corresponding to the minimum circumscribed rectangle meeting the conditions for information identification, thereby realizing the identification of the contents in a plurality of targets in one image, greatly improving the working efficiency, and sequentially carrying out the angle correction and perspective transformation of the image for each screened target area, thereby improving the identification precision of the information in each target area.
Drawings
FIG. 1 is an overall flow chart of an embodiment of the present invention.
Fig. 2 is a flowchart of step S1 in the embodiment of the present invention.
FIG. 3 is a pre-processed image according to an embodiment of the present invention.
FIG. 4 is a first binarized image according to an embodiment of the present invention.
Fig. 5 is a first morphologically processed image in an embodiment of the present invention.
Fig. 6 is edge data of each region in an image in an embodiment of the present invention.
Fig. 7 is a minimum bounding rectangle for each region in an image in an embodiment of the invention.
Fig. 8 is a schematic diagram of calculating a parameter related to a minimum bounding rectangle according to an embodiment of the present invention.
FIG. 9 is a diagram illustrating a screening result of the minimum bounding rectangle and the corresponding region according to an embodiment of the present invention.
Fig. 10 is a flowchart of step S2 in the embodiment of the present invention.
Fig. 11 shows the angle correction results of the display screen 1 (left screen) and the display screen 2 (right screen) according to the embodiment of the present invention.
FIG. 12 is a diagram illustrating a comparison between a first rectangular area and a minimum bounding rectangle according to an embodiment of the present invention.
Fig. 13 is a diagram of images to be processed of the display screen 1 (left screen) and the display screen 2 (right screen) in the embodiment of the present invention.
Fig. 14 is a flowchart of step S3 in the embodiment of the present invention.
Fig. 15 is an edge intersection of the display screen 1 (left screen) and the display screen 2 (right screen) in the embodiment of the present invention.
Fig. 16 is images to be recognized of the display screen 1 (left screen) and the display screen 2 (right screen) in the embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the drawings and specific preferred embodiments of the description, without thereby limiting the scope of protection of the invention.
Example one
In this embodiment, an image multi-target information recognition method is designed based on an OpenCV image processing tool, which can perform accurate detection, correction and image capture on multiple target areas located in the same image, and create favorable conditions for information recognition, as shown in fig. 1, includes the following steps:
s1) acquiring the preprocessed image, carrying out region division on the preprocessed image, drawing a minimum circumscribed rectangle of each region, screening the minimum circumscribed rectangle which accords with a preset length-width ratio and area ratio, and taking a region corresponding to the screened minimum circumscribed rectangle as a target region;
s2) for each target region, performing angle correction on the image respectively with the center of the target region, and intercepting the image of a first rectangular region centered on the target region from the corrected image as an image to be processed of the target region;
s3) respectively detecting edge intersection points in the to-be-processed image of each target area aiming at the to-be-processed image of each target area, and carrying out perspective transformation on the to-be-processed image according to the edge intersection points to obtain an image of a second rectangular area as the to-be-identified image of the target area;
s4) performing character recognition on each image to be recognized, and obtaining an information recognition result for each target area.
Through the steps, the method of the embodiment determines the target area based on the aspect ratio and the area ratio of the minimum circumscribed rectangle, and performs angle-corrected interception and perspective-transformed interception on the image based on the target area in sequence, so that accurate results can be obtained through information identification.
The key point of this embodiment is how to accurately screen out a target region from an image, where the object identified by the information in this embodiment is mainly characters on a display screen in the image, and generally, a screen region of the display screen on the image has a luminance difference compared to a frame region, so in step S1) of this embodiment, first, based on the luminance difference, the image after preprocessing is subjected to region division, and a minimum bounding rectangle of each region is drawn, as shown in fig. 2, the steps specifically include:
s111) carrying out Gaussian filtering and binarization processing on the image to obtain a first binarized image, as shown in FIG. 3, the image after preprocessing in the embodiment is a gray scale image obtained by carrying out gray scale conversion on a color image, wherein the gray scale image comprises a left display screen 1 and a right display screen 2, the number of the display screens capable of identifying information is not limited to the two display screens in the embodiment, and the information identification can be carried out on more display screens in the same image under the condition of meeting definition, as shown in FIG. 4, the gray scale image is subjected to Gaussian filtering and binarization processing to obtain a first binarized image, the first binarized image is compared with the gray scale image, the difference between the screen areas of the display screen 1 and the display screen 2 is more obvious compared with the frame area, and the Gaussian filtering and binarization processing are both conventional means in the image processing, not described herein too much;
s112) performing morphological corrosion expansion processing on the first binarized image to obtain a first morphologically processed image (c), as shown in FIG. 5, the first morphologically processed image (c) filters noise compared with the first binarized image (c), so that the boundary between the screen area and the frame area of the display screen 1 and the display screen 2 is clearer, and the morphological corrosion expansion processing is a conventional means in image processing and is not described herein;
s113) performing edge detection on the first morphological processing image (c) to obtain edge data of each area, as shown in fig. 6, in this embodiment, an OpenCV image processing tool is used to perform edge detection to obtain an edge detection image (c), as can be seen from the edge detection image (c), besides the edges of the screen areas of the display screen 1 and the display screen 2, there are many interferences, for example, the edges of the areas such as windows, ceiling lamps, walls, etc. in the background are also in the edge detection image (c), using the OpenCV image processing tool to perform edge detection is a conventional means in image processing, which is not described herein;
s114) drawing the minimum bounding rectangle of each region according to the edge data, as shown in fig. 7, in this embodiment, an OpenCV image processing tool is used to draw the minimum bounding rectangle of each region in the grayscale map (i) based on the edge data of the previous step, to obtain an edge contour image (v), where the edge contour image (v) includes the minimum bounding rectangles of the display screen 1 and the display screen 2, and also includes the minimum bounding rectangles of the window, the ceiling lamp, the wall, and the like as interferences, and the OpenCV image processing tool is used to draw the minimum bounding rectangle as a conventional means in image processing, which is not described herein.
By observing the edge profile image (c), we find that the area framed by the minimum circumscribed rectangles of the display screen 1 and the display screen 2 is substantially the screen area of the display screen 1 and the display screen 2, and because there is a unified standard for the aspect ratio of the display device screen at present, the aspect ratios of the minimum circumscribed rectangles of the display screen 1 and the display screen 2 are substantially the same, and there is a significant difference from the aspect ratios of the minimum circumscribed rectangles of other windows, ceiling lights, walls, etc. as the main observation object, in addition, the occupation ratio of the display screen 1 and the display screen 2 in the gray scale map (r) is also larger than part of other interferences, so the observation result provides a thinking for screening the screen areas of the display screen 1 and the display screen 2 as the target area from the edge profile image (c), as shown in fig. 2, for all the minimum circumscribed rectangles in the edge profile image (c) in the step S1) of the present embodiment, screening the minimum circumscribed rectangle conforming to the preset aspect ratio and the preset area ratio to finally obtain a screening result image of the minimum circumscribed rectangle only including the display screen 1 and the display screen 2 as shown in fig. 9, wherein the step of screening the minimum circumscribed rectangle conforming to the preset aspect ratio and the preset area ratio specifically comprises the following steps:
s121) selecting a current minimum circumscribed rectangle, calculating a length and a width of the current minimum circumscribed rectangle according to the number of pixels in the length direction and the width direction of the current minimum circumscribed rectangle, and calculating an area of the current minimum circumscribed rectangle according to the length and the width, as shown in fig. 8, in this embodiment, an XOY plane coordinate system is established with a horizontal direction as an X axis and a vertical direction as a Y axis, and a center point coordinate p including the minimum circumscribed rectangle can be obtained by an OpenCV image processing toolc(xc,yc) The angle of inclination of the minimum circumscribed rectangle with the X-axis, the length W of the minimum circumscribed rectangleiSum width value HiWherein the length value W of the minimum bounding rectangleiSum width value HiThat is, the number of pixels in the length direction and the width direction of the minimum circumscribed rectangle is calculated, and the length value W is calculatediSum width value HiThe product of (A) and (B) yields the area of the minimum bounding rectanglei；
S122) if the error between the aspect ratio of the current minimum circumscribed rectangle and the preset aspect ratio is smaller than the preset error, and the area ratio of the current minimum circumscribed rectangle in the first gray scale map is greater than or equal to the preset threshold, the region corresponding to the current minimum circumscribed rectangle is the target region, in this embodiment, it is determined whether the current minimum circumscribed rectangle satisfies the function expression of the preset aspect ratio and the area ratio as follows:
in the above formula, i is the serial number of the current minimum circumscribed rectangle, RiIs the length-width ratio of the current minimum bounding rectangle, R is the preset length-width ratio, alpha is the preset error, SiThe area ratio of the current minimum circumscribed rectangle in the gray level graph is shown, s is a preset threshold value, and P isiIf the minimum circumscribed rectangle is 1, the current minimum circumscribed rectangle meets the preset length-width ratio and area ratio, and the area corresponding to the current minimum circumscribed rectangle is the display screen 1 or the display screen 2A screen area;
in this embodiment, the expression of the area-to-area ratio function of the current minimum bounding rectangle in the gray scale map (i) is as follows:
in the above formula, i is the serial number of the current minimum circumscribed rectangle, areaiIs the area of the current minimum bounding rectangle, HiIs the width value of the current minimum bounding rectangle, WiThe length value of the current minimum circumscribed rectangle is obtained, row is the number of pixel points in the width direction of the gray-scale map I, and col is the number of pixel points in the length direction of the gray-scale map I;
in this embodiment, in view of the fact that the purpose of the present solution is to obtain information of display screens in an image, which is subject to the current image resolution, when the display screen in the image is too small, the displayed information is too fuzzy to be accurately identified, so a threshold s of an area ratio is set, where a preset value of the threshold s is 0.1, and even if there are 9 display screens in a maximum number of support in one image at the same time, the threshold s may be set to other values according to the maximum number of all display screens expected in one image;
in this embodiment, the aspect ratio function expression of the current minimum bounding rectangle is as follows:
Ri＝Hi/Wi (3)
in the above formula, i is the serial number of the current minimum circumscribed rectangle, HiIs the width value of the current minimum bounding rectangle, WiFor the length value of the current minimum bounding rectangle, the preset length-width ratio R in this embodiment is 16/9, because all current display devices adopt this ratio, and meanwhile, the value of the error α is set according to the actual situation, for example, the value of the error α can be properly increased in the case of image distortion, and is generally set to be 0.1;
s123) returns to step S121) until all the minimum bounding rectangles in the edge contour image (S) are traversed.
As shown in fig. 9, the minimum circumscribed rectangles of the display screen 1 and the display screen 2 in the filtering result image (c) are accurately selected, but the angle deviation problem still exists for the display screen areas of the display screen 1 and the display screen 2, and the minimum circumscribed rectangles cannot accurately frame and select the display screen areas of the display screen 1 and the display screen 2, so as shown in fig. 1 and fig. 10, in the present embodiment, the image angle is corrected according to each target area, the first rectangle area image with the target area as the center is intercepted, according to the content described in the foregoing, in step S1), the horizontal direction is taken as the X axis, the vertical direction is taken as the Y axis to establish the XOY plane coordinate system, and the central point coordinate p of the minimum circumscribed rectangle is obtained by the OpenCV image processing toolc(xc,yc) And as shown in fig. 8, the tilt direction and tilt angle of the minimum circumscribed rectangle with respect to the X axis can be calculated according to the projection of the long side and the long side of the minimum circumscribed rectangle on the X axis, as shown in fig. 9, the minimum circumscribed rectangle of the display screen 1 is tilted to the left with respect to the X axis at a tilt angle of a1, and the minimum circumscribed rectangle of the display screen 2 is tilted to the right with respect to the X axis at a tilt angle of a2 in the screening result image; then, in step S2) of this embodiment, performing angle rectification on the image with the center of the target area specifically includes: taking the central point of the minimum circumscribed rectangle corresponding to the target area as a rotation center, taking the value of the inclination angle of the minimum circumscribed rectangle corresponding to the target area as a rotation angle value, and rotating the image according to the rotation center and the rotation angle value in the opposite direction of the inclination direction to make the inclination angle of the minimum circumscribed rectangle corresponding to the target area relative to the X axis be 0, in this embodiment, for the display screen 1 in the screening result image (c), because the corresponding minimum circumscribed rectangle is inclined to the left, the screening result image (c) or the gray scale diagram (i) is rotated clockwise according to the value of the rotation angle, namely the inclination angle a1, with the rotation center, and the finally obtained angle correction result image (c) is shown as the left picture in fig. 11; for the display screen 2 in the screening result image, because the corresponding minimum circumscribed rectangle is inclined to the right, the rotation center is used for aligning the screening result image or the gray level diagram according to the rotation angleThat is, the value of the inclination angle a2 is rotated counterclockwise, and the resulting angle correction result image is shown in the right picture of fig. 11.
After the angle correction is performed, in order to more accurately acquire and identify information of the target region, it is necessary to first cut a rectangular region including a peripheral extension range from the target region as a center in the angle correction result image (S) to prepare for subsequent image processing, as shown in fig. 12, the step S2) of the present embodiment specifically includes: obtaining a central point of a minimum external rectangle corresponding to the target area, respectively calculating coordinate values of an interception initial point and the length and width of the first rectangular area according to the coordinate values of the central point and a preset expansion value, namely calculating coordinate values of a vertex of the first rectangular area according to the coordinate values of the central point of the minimum external rectangle and the preset expansion value, taking the vertex of the first rectangular area as the interception initial point, wherein the coordinate value function expression of the vertex of the first rectangular area is as follows:
in the above formula, xe,yeRespectively the abscissa and ordinate, x, of the vertex of the first rectangular areac,ycRespectively an abscissa and an ordinate of a central point of the minimum circumscribed rectangle, W is a length value of the minimum circumscribed rectangle, H is a width value of the minimum circumscribed rectangle, col/10 is a preset length direction expansion value, col is the number of pixel points in the image length direction, row/10 is a preset width direction expansion value, and row is the number of pixel points in the image width direction;
calculating the length and the width of the first rectangular area according to the size of the minimum bounding rectangle and a preset expansion value, wherein the functional expression of the length and the width of the first rectangular area is as follows:
in the above formula, We,HeRespectively the length and the width of the vertex of the first rectangular region, wherein col/10 is a preset length direction expansion value, col is the number of pixels in the length direction of the image, row/10 is a preset width direction expansion value, and row is the number of pixels in the width direction of the image;
then, an image of the first rectangular region is cut out in accordance with the length and width of the first rectangular region from the cut-out start point.
The first rectangular region image obtained by cutting the angle correction result image of the display screen 1 and the display screen 2 according to the above steps is taken as an image to be processed (viii), and the images to be processed (viii) of the display screen 1 and the display screen 2 are respectively shown as the left picture and the right picture of fig. 13.
After obtaining the to-be-processed image viii of the display screen 1 and the display screen 2, the to-be-processed image viii may be further subjected to perspective transformation, so as to more accurately intercept the image for information recognition, as shown in fig. 14, in step S3), for each target region, an edge intersection in the to-be-processed image of the target region is detected, which specifically includes the following steps:
s31) carrying out Gaussian filtering and binarization processing on the image to be processed to obtain a second binarized image;
s32) performing morphological erosion expansion processing on the second binary image to obtain a second morphological processing image;
s33) carrying out edge detection on the second morphological processing image to obtain edge data of the target area;
s34) carrying out Hough line detection on the edge data to obtain an edge line of the target area;
s35) to obtain the edge intersection of the target region.
In the above steps, steps S31 to S33 are the same as steps S111 to S113, and are not repeated herein, step S34 and step S35 are also conventional operations in the image processing process, and the processing processes of steps S31 to S35 are mostly applied to the identification of characters of certificates such as id cards and driver licenses. The edge intersection images nintendo of the display screen 1 and the display screen 2 obtained through the processing procedures of steps S31 to S35 are shown in the left and right pictures of fig. 15, respectively.
After the intersection point of the edges of display screen 1 and display screen 2 is obtained, perspective transformation can be performed by using an OpenCV image processing tool, so as to obtain an image of a second rectangular region serving as an image to be identified, where images to be identified in display screen 1 and display screen 2 are shown in left and right pictures in fig. 16. The information identification is performed on the images to be identified on display screen 1 and display screen 2, respectively, to obtain the text contents displayed on display screen 1 and display screen 2.
The embodiment further provides an image multi-target information recognition system, which includes:
the target screening unit is used for acquiring the preprocessed image, performing region division on the preprocessed image, drawing a minimum circumscribed rectangle of each region, screening the minimum circumscribed rectangle which accords with a preset length-width ratio and an area ratio, and taking a region corresponding to the screened minimum circumscribed rectangle as a target region;
the image first-time correction unit is used for respectively carrying out angle correction on the image by taking the center of the target area as the center of each target area, and intercepting the image of a first rectangular area taking the target area as the center in the corrected image to be used as the image to be processed of the target area;
the image second correction unit is used for respectively detecting edge intersection points in the images to be processed of each target area aiming at the images to be processed of the target areas, and performing perspective transformation on the images to be processed according to the edge intersection points to obtain images of a second rectangular area as the images to be identified of the target areas;
and the information identification unit is used for carrying out character identification on each image to be identified to obtain the information identification result of each target area.
The present embodiment also proposes a computer-readable storage medium storing a computer program programmed or configured to execute the image multi-target information recognition method.
Example two
The embodiment is substantially the same as the first embodiment, except that, in steps S121) to S123) of the first embodiment, the preset threshold S and the aspect ratio R are both fixed values, and there may still be interference that is not excluded, for example, the preset threshold S is much smaller than an area ratio corresponding to each display screen in the image, so that a minimum circumscribed rectangle of a partial region of the non-display screen region is also retained, and therefore the embodiment further includes a step of adjusting the size of the threshold S after step S123), specifically: the value of the threshold value s is adjusted to be the area ratio of the display screen with the smallest area in the current image, and the smallest circumscribed rectangle with the area ratio larger than or equal to the adjusted threshold value s is screened and reserved.
EXAMPLE III
The present embodiment is substantially the same as the first embodiment, except that, in steps S121) to S123) of the first embodiment, the preset threshold S and the aspect ratio R are both fixed values, and there may still be unextracted interference, for example, a minimum circumscribed rectangle having the same size and the same aspect ratio as the minimum circumscribed rectangle of the display screen 1 or the display screen 2 exists in the non-display screen region, so that the minimum circumscribed rectangle of the partial region of the non-display screen region is also retained, and step 2) of the present embodiment further includes a step of verifying the target region, which specifically includes: and intercepting images surrounded by the minimum external rectangles corresponding to all the target areas, respectively carrying out character recognition on each intercepted image, and if no recognition information exists or recognition errors occur, rejecting the corresponding target areas. In view of the fact that the largest difference between the display screen region and the non-display screen region is that display information exists in the display screen, and the text recognition result of the display screen region without angle adjustment and perspective transformation is only affected to the possible accuracy degree, in this embodiment, text recognition is performed on images surrounded by all the minimum circumscribed rectangles, and whether the image surrounded by the minimum circumscribed rectangle is the display screen is judged according to whether the recognition result exists, so that the accuracy of target region recognition is improved.
The foregoing is considered as illustrative of the preferred embodiments of the invention and is not to be construed as limiting the invention in any way. Although the present invention has been described with reference to the preferred embodiments, it is not intended to be limited thereto. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical spirit of the present invention should fall within the protection scope of the technical scheme of the present invention, unless the technical spirit of the present invention departs from the content of the technical scheme of the present invention.
Claims (10)
1. The image multi-target information identification method is characterized by comprising the following steps of:
s1) acquiring the preprocessed image, carrying out region division on the preprocessed image, drawing a minimum circumscribed rectangle of each region, screening the minimum circumscribed rectangle which accords with a preset length-width ratio and area ratio, and taking a region corresponding to the screened minimum circumscribed rectangle as a target region;
s2) for each target region, performing angle correction on the image respectively with the center of the target region, and intercepting the image of a first rectangular region centered on the target region from the corrected image as an image to be processed of the target region;
s3) respectively detecting edge intersection points in the to-be-processed image of each target area aiming at the to-be-processed image of each target area, and carrying out perspective transformation on the to-be-processed image according to the edge intersection points to obtain an image of a second rectangular area as the to-be-identified image of the target area;
s4) performing character recognition on each image to be recognized, and obtaining an information recognition result for each target area.
2. The image multi-target information recognition method according to claim 1, wherein the step of drawing the minimum bounding rectangle of each region in the image in step S1) specifically comprises:
s111) carrying out Gaussian filtering and binarization processing on the image to obtain a first binarized image;
s112) performing morphological corrosion expansion processing on the first binary image to obtain a first morphological processing image;
s113) carrying out edge detection on the first morphological processing image to obtain edge data of each area;
s114) drawing the minimum bounding rectangle of each area according to the edge data.
3. The image multi-target information recognition method according to claim 1, wherein the step of screening the minimum bounding rectangle meeting the preset aspect ratio and area ratio in step S1) specifically comprises:
s121) selecting a current minimum circumscribed rectangle, calculating the length and the width of the current minimum circumscribed rectangle according to the number of pixel points in the length direction and the width direction of the current minimum circumscribed rectangle, and calculating the area of the current minimum circumscribed rectangle according to the length and the width;
s122) if the error between the aspect ratio of the current minimum circumscribed rectangle and the preset aspect ratio is smaller than the preset error, and the area occupation ratio of the current minimum circumscribed rectangle in the image is larger than or equal to a preset threshold value S, taking the region corresponding to the current minimum circumscribed rectangle as a target region;
s123) returning to the step S121) until all the minimum bounding rectangles in the image are traversed.
4. The image multi-target information identification method according to claim 3, characterized in that step S123) is followed by a step of adjusting the size of the threshold S, specifically: and adjusting the value of the threshold value s to be the area ratio of the minimum circumscribed rectangle corresponding to the actual target area with the minimum area in the current image, and screening and reserving the minimum circumscribed rectangle with the area ratio larger than or equal to the adjusted threshold value s.
5. The image multi-target information identification method according to claim 1, characterized in that step 2) is preceded by a step of verifying the target area, specifically comprising: and intercepting images surrounded by the minimum external rectangles corresponding to all the target areas, respectively carrying out character recognition on each intercepted image, and if no recognition information exists or recognition errors occur, rejecting the corresponding target areas.
6. The image multi-target information recognition method according to claim 1, wherein the step S1) further includes: establishing a coordinate system by taking the horizontal direction as a transverse axis and the vertical direction as a longitudinal axis, acquiring the coordinates of the central point of the minimum circumscribed rectangle, and calculating the inclination direction and the inclination angle of the minimum circumscribed rectangle relative to the transverse axis according to the projection of the long edge of the minimum circumscribed rectangle and the long edge on the transverse axis; the step S2) of performing angle rectification on the image with the center of the target region specifically includes: and rotating the image in the opposite direction of the inclination direction according to the rotation center and the rotation angle value by taking the center point of the minimum circumscribed rectangle corresponding to the target area as a rotation center and the value of the inclination angle of the minimum circumscribed rectangle corresponding to the target area as a rotation angle value, so that the inclination angle of the minimum circumscribed rectangle corresponding to the target area relative to the horizontal axis is 0.
7. The image multi-target information recognition method according to claim 1, wherein the step S2) of intercepting the image of the first rectangular region centered on the target region from the corrected image specifically includes: and acquiring a central point of the minimum circumscribed rectangle corresponding to the target area, respectively calculating the coordinate value of the interception starting point and the length and width of the first rectangular area according to the coordinate value of the central point and a preset expansion value, and then intercepting the image of the first rectangular area from the interception starting point according to the length and width of the first rectangular area.
8. The image multi-target information recognition method according to claim 7, wherein the calculating the coordinate value of the interception start point and the length and width of the first rectangular region according to the coordinate value of the center point and the preset expansion value respectively comprises: calculating the coordinate value of the vertex of the first rectangular region according to the coordinate value of the central point of the minimum external rectangle and a preset expansion value, taking the vertex of the first rectangular region as an intercepting starting point, wherein the coordinate value function expression of the vertex of the first rectangular region is as follows:
in the above formula, xe,yeRespectively the abscissa and ordinate, x, of the vertex of the first rectangular areac,ycRespectively an abscissa and an ordinate of a central point of the minimum circumscribed rectangle, W is a length value of the minimum circumscribed rectangle, H is a width value of the minimum circumscribed rectangle, col/10 is a preset length direction expansion value, col is the number of pixel points in the image length direction, row/10 is a preset width direction expansion value, and row is the number of pixel points in the image width direction;
calculating the length and the width of the first rectangular area according to the size of the minimum bounding rectangle and a preset expansion value, wherein the functional expression of the length and the width of the first rectangular area is as follows:
in the above formula, We,HeThe length and the width of the vertex of the first rectangular region are respectively, col/10 is a preset length direction expansion value, col is the number of pixels in the length direction of the image, row/10 is a preset width direction expansion value, and row is the number of pixels in the width direction of the image.
9. An image multi-target information recognition system, comprising:
the target screening unit is used for acquiring the preprocessed image, performing region division on the preprocessed image, drawing a minimum circumscribed rectangle of each region, screening the minimum circumscribed rectangle which accords with a preset length-width ratio and an area ratio, and taking a region corresponding to the screened minimum circumscribed rectangle as a target region;
the image first-time correction unit is used for respectively carrying out angle correction on the image by taking the center of the target area as the center of each target area, and intercepting the image of a first rectangular area taking the target area as the center in the corrected image to be used as the image to be processed of the target area;
the image second correction unit is used for respectively detecting edge intersection points in the images to be processed of each target area aiming at the images to be processed of the target areas, and performing perspective transformation on the images to be processed according to the edge intersection points to obtain images of a second rectangular area as the images to be identified of the target areas;
and the information identification unit is used for carrying out character identification on each image to be identified to obtain the information identification result of each target area.
10. A computer-readable storage medium storing a computer program programmed or configured to execute the image multi-target information recognition method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111258319.0A CN114120320A (en) | 2021-10-27 | 2021-10-27 | Image multi-target information identification method, system and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111258319.0A CN114120320A (en) | 2021-10-27 | 2021-10-27 | Image multi-target information identification method, system and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114120320A true CN114120320A (en) | 2022-03-01 |
Family
ID=80377271
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111258319.0A Pending CN114120320A (en) | 2021-10-27 | 2021-10-27 | Image multi-target information identification method, system and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114120320A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116468611A (en) * | 2023-06-09 | 2023-07-21 | 北京五八信息技术有限公司 | Image stitching method, device, equipment and storage medium |
-
2021
- 2021-10-27 CN CN202111258319.0A patent/CN114120320A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116468611A (en) * | 2023-06-09 | 2023-07-21 | 北京五八信息技术有限公司 | Image stitching method, device, equipment and storage medium |
| CN116468611B (en) * | 2023-06-09 | 2023-09-05 | 北京五八信息技术有限公司 | Image stitching method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111474184B (en) | AOI character defect detection method and device based on industrial machine vision | |
| CN110119741B (en) | Card image information identification method with background | |
| CN106960208B (en) | Method and system for automatically segmenting and identifying instrument liquid crystal number | |
| CN106803244B (en) | Defect identification method and system | |
| CN111968144B (en) | Image edge point acquisition method and device | |
| CN111179243A (en) | Small-size chip crack detection method and system based on computer vision | |
| CN109426814B (en) | Method, system and equipment for positioning and identifying specific plate of invoice picture | |
| CN111046872B (en) | Optical character recognition method | |
| CN109034145B (en) | Bank card number identification method based on OpenCV | |
| JP2000011089A (en) | Binarizing method for optical character recognition system | |
| CN111354047B (en) | Computer vision-based camera module positioning method and system | |
| CN112560538A (en) | Method for quickly positioning damaged QR (quick response) code according to image redundant information | |
| CN111861979A (en) | Positioning method, positioning equipment and computer readable storage medium | |
| CN112419207A (en) | Image correction method, device and system | |
| CN114092468A (en) | Standard target counting method based on machine vision | |
| CN114820631A (en) | Fabric defect detection method capable of resisting texture interference | |
| CN114120320A (en) | Image multi-target information identification method, system and medium | |
| CN112419225B (en) | SOP type chip detection method and system based on pin segmentation | |
| CN116563298B (en) | Cross line center sub-pixel detection method based on Gaussian fitting | |
| CN113591507A (en) | Robust two-dimensional code DataMatrix positioning method and system | |
| CN110610163B (en) | Table extraction method and system based on ellipse fitting in natural scene | |
| CN108898584B (en) | Image analysis-based full-automatic veneered capacitor welding polarity discrimination method | |
| CN113284158B (en) | Image edge extraction method and system based on structural constraint clustering | |
| CN114511522A (en) | Automatic reagent judgment method based on fluorescence immunoassay and colloidal gold method | |
| CN114937003A (en) | Multi-type defect detection system and method for glass panel |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |