CN114519788A

CN114519788A - Image processing method, image processing device, electronic equipment and computer readable storage medium

Info

Publication number: CN114519788A
Application number: CN202011311671.1A
Authority: CN
Inventors: 杨爱东; 龚福才
Original assignee: Asiainfo Technologies China Inc
Current assignee: Asiainfo Technologies China Inc
Priority date: 2020-11-20
Filing date: 2020-11-20
Publication date: 2022-05-20

Abstract

The embodiment of the application provides an image processing method and device, electronic equipment and a computer readable storage medium, and relates to the field of image processing. The method comprises the following steps: the method comprises the steps of carrying out graying and binarization processing on an initial image containing a seal to obtain a corresponding binary image, classifying each pixel point of the binary image based on an image segmentation model to obtain a corresponding classification result, and mapping the classification result to a corresponding pixel value to obtain an image not containing the seal. According to the embodiment of the application, each pixel point is classified according to the image characteristics, the type, the color and the shielding condition of the seal are insensitive, and the seal in the image can be removed more accurately.

Description

Image processing method, image processing device, electronic equipment and computer readable storage medium

Technical Field

The present application relates to the field of image processing, and in particular, to an image processing method, an image processing apparatus, an electronic device, and a computer-readable storage medium.

Background

At present, people often use an OCR technology to process bills, the OCR technology can locate character areas of bill pictures and identify character contents, useful information in the bills is extracted, and the bill processing speed is improved. However, the bills are often stamped with stamps on the key information, which causes difficulty in positioning text areas and recognizing text, thereby resulting in failure in extracting bill information.

Therefore, the stamp needs to be removed before OCR recognition is performed. The traditional image processing technology is to filter the seal by a separation method of a color channel, the color of the seal in an actual scene is often similar to the color of the bill characters, and the seal is not only influenced by the environment but also is not simply presented with one color, so that the seal can be removed inaccurately by using the traditional image processing method, for example, the seal can not be removed cleanly or non-seal information can be removed.

Disclosure of Invention

The present application aims to solve at least one of the above-mentioned technical drawbacks, and in particular, to solve the technical drawback that non-stamp information is easily removed by mistake.

In a first aspect, a method for image processing is provided, including:

acquiring an initial image to be processed containing a seal, and acquiring a corresponding binary image based on the initial image; classifying each pixel point of the binary image based on the image segmentation model to obtain a corresponding classification result;

and mapping the classification result into a corresponding pixel value to obtain an image without the seal.

In an optional embodiment of the first aspect, acquiring the corresponding binary image based on the initial image comprises:

carrying out graying processing on the initial image to obtain a corresponding grayscale image;

and carrying out binarization processing on the gray level image to obtain a binary image corresponding to the initial image.

In an optional embodiment of the first aspect, classifying each pixel point of the binary image to obtain a corresponding classification result includes:

compressing the binary image to obtain a feature code;

and decoding the characteristic codes to obtain a classification result of each pixel point.

In an alternative embodiment of the first aspect, the classification result is in the form of a matrix; the elements of the matrix correspond to the classification result of each pixel point; mapping the classification result to a corresponding pixel value, including:

mapping a first class element representing a first classification result in the matrix into a first preset pixel value;

and for each second-class element representing the second classification result in the matrix, determining a corresponding target pixel of the second-class element in the initial image, and mapping the second-class element to the pixel value of the target pixel.

In an optional embodiment of the first aspect, before classifying each pixel point of the binary image based on the image segmentation model, the method further includes:

acquiring a plurality of groups of sample images; each group of sample images comprises an original sample image without a stamp and a corresponding sample image with a stamp;

respectively acquiring multiple groups of corresponding binary sample images based on the multiple groups of sample images;

and training the initial image segmentation model according to the multiple groups of binary sample images to obtain the image segmentation model.

In an alternative embodiment of the first aspect, acquiring a plurality of sets of sample images comprises:

and synthesizing the seal into an original sample image aiming at each group of sample images to obtain the sample image with the seal.

In an optional embodiment of the first aspect, after acquiring the plurality of sets of sample images, the method further comprises:

carrying out data enhancement operation on the sample images with the stamps in the multiple groups of sample images; wherein the data enhancement operation includes at least one of adding a background and adding noise.

In a second aspect, there is provided an apparatus for image processing, the apparatus comprising:

the image preprocessing module is used for acquiring an initial image to be processed and containing a seal and acquiring a corresponding binary image based on the initial image;

the image segmentation module is used for classifying each pixel point of the binary image based on the image segmentation model to obtain a corresponding classification result;

and the image post-processing module is used for mapping the classification result into a corresponding pixel value to obtain an image without the seal.

In an optional embodiment of the second aspect, when the image preprocessing module acquires a corresponding binary image based on the initial image, the image preprocessing module is specifically configured to:

In an optional embodiment of the second aspect, when the image segmentation module classifies each pixel point of the binary image based on the image segmentation model to obtain a corresponding classification result, the image segmentation module is specifically configured to:

compressing the binary image to obtain a feature code;

In an alternative embodiment of the second aspect, the classification result is in the form of a matrix; the elements of the matrix correspond to the classification result of each pixel point; when the image post-processing module maps the classification result to the corresponding pixel value, the image post-processing module is specifically configured to:

In an optional embodiment of the second aspect, the system further includes a model training module, specifically configured to:

In an optional embodiment of the second aspect, the model training module, when acquiring a plurality of sets of sample images, is specifically configured to:

In an optional embodiment of the second aspect, further comprising a data enhancement module configured to:

carrying out data enhancement on sample images with stamps in the multiple groups of sample images; wherein the data enhancement operation includes at least one of adding a background and adding noise.

The present invention also provides an electronic device, including:

the image processing device comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the image processing method of any embodiment is realized.

The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the image processing method of any of the above embodiments.

According to the image processing method, the corresponding binary image is obtained by carrying out graying and binarization processing on the initial image containing the seal, then each pixel point of the binary image is classified based on the image segmentation model to obtain the corresponding classification result, and finally the classification result is mapped to the corresponding pixel value to obtain the image not containing the seal. According to the invention, each pixel point can be divided according to the whole information and the local information of the image without depending on the division of a color channel, so that even aiming at the image with the seal color similar to the bill character color, the pixel points of non-seal information can be retained to the maximum extent in the division, the seal removing effect is better, the interference of the seal on the bill information extraction process is reduced or even eliminated, and the bill processing accuracy is improved.

Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.

Fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application;

fig. 2 is a schematic flowchart of an image processing method according to an embodiment of the present application;

fig. 3 is a schematic diagram illustrating a classification result mapped to a corresponding pixel value in an image processing method according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of an image processing method according to an example of the present application;

fig. 5 is a schematic structural diagram of an image segmentation model provided in an example of the present application;

fig. 6 is a flowchart illustrating an image processing method according to an example provided in an embodiment of the present application;

fig. 7 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of an electronic device for image processing according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present invention.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.

With the development of image processing technology, more convenient methods are provided in the aspect of bill processing application. For example, when enterprise tax accounting and internal reimbursement need to process a large amount of tax invoices, traffic tickets and receipts, OCR technology can be used for replacing manual arrangement of financial staff, processing speed of the tickets is increased, and the condition that a large amount of labor is easy to make mistakes is avoided. OCR (Optical Character Recognition) refers to a process in which an electronic device (e.g., a scanner or a digital camera) checks a Character printed on a paper, determines its shape by detecting a dark or light pattern, and then translates the shape into a computer Character by a Character Recognition method, and is applied to a bill processing direction, i.e., to locate a text area of a bill picture and recognize text contents, so that a financial staff can extract useful information of the bill by scanning the bill.

However, OCR technology is not plain in the way it is used in document processing because documents are often stamped with a wide variety of stamps. When the seal is covered on the key information of the bill, great trouble is caused to the positioning of the character area and the character recognition in the OCR technology, and the error and even failure of the information extraction of the bill are caused.

In order to reduce the interference of the stamp on the subsequent image processing means, the stamp needs to be removed. There are two common seal eliminating methods, the first method is based on traditional image processing technology, and filters out red color by using color channel separation method, so as to achieve the purpose of eliminating the seal. However, in practical applications, the shapes and sizes of the stamps are different, and uneven colors may appear in different environments, so that the situation that the stamps are not completely removed or non-stamp information is also removed often occurs by using the method. Moreover, the method for removing the seal based on the traditional image processing technology cannot be self-adaptive, the binarization threshold value needs to be manually adjusted according to the characteristics of the image, and the efficiency is low. The second method is to use a generation countermeasure network (GAN) to migrate the note with the seal to the style without the seal, thereby achieving the purpose of removing the seal. A picture generated using GAN may have a pixel value different from that of the original image, and thus a generated image completely different from the original image may be output, and the result may be unstable.

The present application provides an image method, an apparatus, an electronic device and a computer-readable storage medium, which aim to solve the above technical problems in the prior art.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

The image processing method provided in the embodiment of the application can be applied to a server and can also be applied to a terminal.

Those skilled in the art will understand that the "terminal" used herein may be a Mobile phone, a tablet computer, a PDA (Personal Digital Assistant), an MID (Mobile Internet Device), etc.; a "server" may be implemented as a stand-alone server or as a server cluster comprised of multiple servers.

The embodiment of the present application provides an image processing method, which may be applied to a server or a terminal, and as shown in fig. 1, the method may include:

step S100, acquiring an initial image to be processed containing a seal, and acquiring a corresponding binary image based on the initial image.

In the embodiment of the present application, the image to be processed may be a stamped image selected from a local image library, an image obtained by an electronic device such as a scanner or a digital camera, or a current image captured in real time by a camera.

The Binary Image (Binary Image) refers to that each pixel in the Image has only two possible values or gray scale states, that is, the gray scale value of any pixel in the Image is 0 or 255, which respectively represents black and white. Acquiring a corresponding binary image based on an initial image to be processed means that when the initial image is not a binary image, performing a certain image processing flow on the initial image and outputting the image as a corresponding binary image converted from the image. The method used in the image processing flow is not limited herein.

And S200, classifying each pixel point of the binary image based on the image segmentation model to obtain a corresponding classification result.

In the embodiment of the present application, the classification result may include a classification result of each pixel, or may include only a classification result of a local pixel, which may be in a matrix form, or in a vector form, or in a text set form, and a specific classification result form is not limited herein.

The image segmentation model can be composed of two parts, namely an encoder and a decoder, the encoder and the decoder can mainly use a full convolution neural network, and the learning mode can be supervised learning. The convolutional neural network has the characteristic learning capacity, high-order features can be extracted from input information, and the classification result of each pixel point can be extracted from the binary image.

The convolutional neural network used by the image segmentation model may be freely collocated and improved based on several common basic networks, or may be improved based on a relatively mature network model, which is not limited herein.

The training method of the image segmentation model will be further described later.

And step S300, mapping the classification result to a corresponding pixel value to obtain an image without the seal.

The mapping in this application refers to querying a corresponding relationship, that is, mapping the classification result to a corresponding pixel value, which may be a pixel value corresponding to the classification result.

Specifically, the classification operation may traverse all pixel points of the binary image, so the classification result may be a classification result of each pixel point; after the pixel value corresponding to the classification result is determined, the pixel value of each pixel point of the binary image may be set as the pixel value corresponding to the classification result.

In addition, the classification operation can be performed only on local pixel points of the image, only the classification result of the pixel points within the preset range is obtained, or only the pixel points meeting the preset condition are classified, which is not limited here.

In this embodiment of the present application, performing two classifications on each pixel point of an image by using a deep learning method may include: and judging whether each pixel point of the image belongs to the pixel point corresponding to the seal. The classification operation is implemented through an image segmentation model, for example, the classification operation is implemented through a neural network, the neural network divides each pixel point according to the overall characteristics and the local characteristics of the whole image, and the defect that the traditional image technology is difficult to automatically adjust the binarization threshold value according to the characteristics of the image is effectively overcome.

Furthermore, each pixel point is classified according to the image characteristics, so that the method is insensitive to the style, color and shielding condition of the seal and has high accuracy.

Furthermore, after each pixel point is classified to obtain a classification result, the classification result is mapped to a corresponding pixel value, the corresponding pixel value is determined by the initial image, the image does not need to be regenerated, and the output result is stable.

In step S100, obtaining a corresponding binary image based on the initial image may include:

(1) and carrying out graying processing on the initial image to obtain a corresponding grayscale image.

Where a grayscale image is an image with only one sample color per pixel, such images are typically displayed in the computer world as grayscales from darkest black to brightest white. The gray image is different from the black and white image, and the black and white image only has two colors of black and white in the field of computer image; grayscale images also have many levels of color depth between black and white. The graying process is a process of changing a color image containing brightness and color into a grayscale image containing only brightness information.

A color image is composed of three channels of red, green and blue, and the graying treatment is to make each pixel point in the initial image satisfy the equality of the red variable value, the green variable value and the blue variable value, i.e. the converted gray value is used to uniformly replace the R value, the G value and the B value in (R, G, B).

The method for converting (R, G, B) to grayscale is not limited herein, and floating point algorithms, B,

Integer method, shift method and average value method, if the initial image is a gray image, it is not necessary to carry on this step, carry on the binary system to the initial image directly.

(2) And carrying out binarization processing on the gray level image to obtain a binary image corresponding to the initial image.

The binarization is to make the gray value of each pixel point of the image be 0 or 255, that is, the whole image has only black and white effects. The range of the gradation value in the image after the gradation processing is 0 to 255, and the range of the gradation value in the image after the binarization is 0 or 255.

The binarization method is not limited, and the following methods are only examples and are easy to understand:

a. after graying the RGB color image, performing binarization on each pixel point by taking a preset numerical value as a threshold value, setting the pixel value to be 0 when the pixel value is smaller than the threshold value, and setting the pixel value to be 255 when the pixel value is larger than the threshold value; the method does not consider the pixel distribution condition and the pixel value characteristic of the image;

b. calculating an average value K of pixels, scanning each pixel value of the image, and setting the pixel value to be 255 when the pixel value is larger than K and setting the pixel value to be 0 when the pixel value is smaller than or equal to K;

c. finding a binary threshold value by using a histogram method, wherein the histogram is an important feature of an image, finding two highest peaks of the image by using a cube map, and then taking the lowest peak valley between the two peaks as the threshold value.

In practical applications, the following situations may occur, and possible treatment methods are listed for the situations respectively: when the initial image is a binary image, directly taking the initial image as the binary image corresponding to the initial image; when the initial image is a gray image, directly carrying out binarization on the gray image to obtain a binary image corresponding to the initial image; when the initial image is a color image, firstly carrying out graying processing on the initial image to obtain a corresponding grayscale image, and then carrying out binarization processing on the grayscale image to obtain a binary image corresponding to the initial image.

In the embodiment of the application, the initial image is firstly subjected to graying processing and then subjected to binarization processing, so that a binary image corresponding to the initial image is obtained and is used as the input of the image segmentation model.

In step S200, classifying each pixel point of the binary image based on the image segmentation model may include:

(1) compressing the binary image to obtain a feature code;

(2) and decoding the characteristic codes to obtain a classification result of each pixel point.

In the embodiment of the present application, the above classification process may be understood as obtaining image features through a feature extraction algorithm, changing contents contained in an image into digitized vector features, and classifying pixels of the image through vectorized image features by using a classifier.

In this embodiment of the present application, the image segmentation model may be composed of an encoder and a decoder, and further, the step of compressing the binary image to obtain the feature code may be completed by the encoder in the image segmentation model, and the step of decoding the feature code to obtain the classification result of each pixel may be completed by the decoder in the image segmentation model.

The encoder can be used for encoding the grayed and binarized picture with the seal into a smaller characteristic diagram. It may be made up of a plurality of convolutional layers and pooling layers, and pooling may be maximized. The encoder module may comprise a plurality of substructures, such as an initiation module and a response module. The decoder may decode the feature map into a classification result of each pixel, for example, different characters such as yes or no may be used to represent different classification results, different numbers such as 0 or 1 may be used to represent different classification results, and different symbols may be used to represent different classification results.

And when the classification result of each pixel point is represented by 0 and 1, outputting a graph without a seal after the gray level and the binarization of the initial image. If the original size is to be restored, the decoder performs an up-sampling operation. The decoder can be viewed as the inverse operation of the encoder, but the substructures of the two need not correspond one-to-one.

In one embodiment, the image segmentation model may be based on an encoder-decoder model architecture. An encoder-decoder model, i.e., a coding-decoding model. Encoding, namely converting an input sequence into a vector with a fixed length; and decoding, namely converting the fixed vector generated before into an output sequence. In specific implementation, both the encoder and the decoder are not fixed, and CNN (Convolutional Neural Networks ), RNN (Recurrent Neural Networks, RNN, cyclic Neural Networks), and the like can be selected and freely combined. For example, CNN may be used for encoding and RNN may be used for decoding.

In one embodiment, the image segmentation model may also be a combination of an Attention model and an encoder-decoder model. The Attention model is an Attention model, and when the Attention model generates output, an Attention range is generated to indicate which parts in an input sequence need to be focused during next output, and then next output is generated according to a focused region, so that the problem that information carried by content input first by an encoder-decoder model is covered by information input later can be solved.

It should be noted that the discussion of the specific structure of the model is only for example, and does not limit the specific network to be selected to implement the function of this step.

The embodiment of the application provides a possible implementation mode, and the classification result is in a matrix form; the elements of the matrix correspond to the classification result of each pixel.

First, in this description, the pixel referred to in the discussion refers to the smallest unit that constitutes a digital image, i.e., a small square with a definite position and color value, and the position and color of the small square determine how the image appears. Each dot matrix image contains a certain number of pixels that determine the size of the image to be presented on the screen. The dot matrix image is also called a bitmap image, and is a main image processing object in the embodiment of the present application, and is composed of a single point of a pixel.

In the embodiment of the present application, the size of the matrix including the classification result output by the image segmentation model may be the same as the size of the pixel point matrix of the initial image.

As an example, to understand the correspondence between the elements and the pixels in the matrix containing the classification result, when the width of the binary image corresponding to an initial image to be processed is 800 pixels and the height is 800 pixels, that is, the image is composed of an 800 × 800 pixel matrix, where the matrix is 800 rows and 800 columns, and the pixel is the minimum unit of the image. Based on the image segmentation model, classifying each pixel point of the binary image, and judging whether the pixel point belongs to the seal, wherein the judgment results are two types, the first classification result is that the pixel point belongs to the seal, the second classification result is that the pixel point does not belong to the seal, and each pixel point in the binary image has a unique classification result. When the binary image is formed by an 800 × 800 pixel matrix, the size of the matrix containing the pixel classification result is also the 800 × 800 matrix, and elements in the matrix correspond to the pixels in the binary image one by one and represent the classification result of each pixel.

As shown in fig. 2, the mapping the classification result to a corresponding pixel value in step S300 may include:

s301, mapping a first class element representing a first classification result in the matrix into a first preset pixel value;

s302, for each second-class element representing the second classification result in the matrix, determining a corresponding target pixel of the second-class element in the initial image, and mapping the second-class element to a pixel value of the target pixel.

Here, the meaning of mapping is explained first, and in the present application, mapping is a relationship that elements between two sets of elements "correspond to" each other, that is, a classification result of one pixel point corresponds to one element in a matrix.

In this embodiment of the application, in step S301, mapping a first class element representing a first classification result in a matrix to a first preset pixel value, which may specifically include:

(1) determining a first preset pixel value corresponding to the first type element;

(2) the first type elements in the matrix are set to a first preset pixel value.

In other embodiments, an output pixel matrix with the same size as the matrix representing the classification result may be generated, pixels in the output pixel matrix with the same position as the first type element in the classification result matrix are determined, and pixel values of the pixels are set as first preset pixel values. The first preset pixel value may be a pixel value set in advance.

In the embodiment of the present application, each element of the second type in the matrix has a corresponding target pixel in the initial image, and the target pixel corresponding to each element of the second type is unique. In an embodiment, since the size of the matrix containing the classification result is the same as the size of the pixel point matrix of the initial image, the position of the second type element in the classification result matrix is determined, the target pixel corresponding to the second type element in the initial image is determined through the position, the second type element in the matrix is set as the pixel value of the target pixel, or an output pixel point matrix with the same size as the matrix representing the classification result is generated, the pixel points in the output pixel point matrix with the same position as the second type element in the classification result matrix are determined, and the pixel values of the pixel points are respectively set as the pixel values of the target pixels corresponding to the pixel points. To summarize more briefly, in one embodiment, this process may be understood simply as setting the pixel values belonging to the stamp to 255 (pure white), with the pixels not belonging to the stamp retaining their original pixel values.

To facilitate understanding of the mapping rule, an embodiment is described herein, when a binary image corresponding to an initial image to be processed has a width of 800 pixels and a height of 800 pixels, that is, the image is composed of a matrix of 800 × 800 pixels, and a pixel is the minimum unit of the image. Based on the image segmentation model, classifying each pixel point of the binary image, and judging whether the pixel point belongs to the seal, wherein the judgment result is two types, the first classification result is that the pixel point belongs to the seal, the second classification result is that the pixel point does not belong to the seal, and each pixel point in the binary image has a unique classification result. When the binary image is formed by an 800 × 800 pixel matrix, the size of the matrix containing the pixel classification result is also the 800 × 800 matrix, and elements in the matrix correspond to the pixels in the binary image one by one and represent the classification result of each pixel. Setting the first type element representing the classification result as belonging to the seal as 1, the second type element representing the classification result as not belonging to the seal as 0, and the first preset pixel value as 0. When mapping is carried out, setting an element of 1 in a matrix containing a classification result as a 0 pixel value; determining the position of each 0 element in the classification result matrix, determining a target pixel corresponding to each 0 element in the initial image through the position, and respectively setting the 0 elements in the matrix containing the classification result as the pixel values of the target pixels corresponding to the 0 elements.

In one embodiment, the initial image may be a binary image, only pixels with a pixel value of 0 (pure black) in the binary image may be classified, the classification result of each pure black pixel may be represented by 0 or 1, and the pixel values of the remaining pure white pixels remain unchanged, and the classification result matrix represents that the corresponding element is directly set to 1. In this case, after the classification result matrix is obtained, only the simplest mapping may be performed. In the classification result matrix, 0 represents that the pixel does not belong to the seal, the pixel value of the pixel is mapped to 0 (pure black), and 1 represents that the pixel belongs to the seal or is a pure white pixel in the initial binary image, so that the pixel value of the pixel is mapped to 255 (pure white), and the effect of removing the seal can be achieved.

As an example, as shown in fig. 3, the initial binary image includes 4 pixels a1, B1, C1, and D1. Assuming that only pure black pixel points are classified, the classification result of the pure black pixel point a1 is that the pure black pixel point belongs to a seal, and the classification result of the pure black pixel point D1 is that the pure black pixel point does not belong to a seal, therefore, in the classification result matrix, the element corresponding to a1 is set to be 1, the element corresponding to a D1 is set to be 0, and the elements corresponding to pure white pixel points in the rest of initial binary images in the matrix are directly set to be 1. Next, mapping elements in the matrix is performed, and mapping 0 element to a pixel value 0 (pure black) and mapping 1 element to a pixel value 255 (pure white), where the effect is shown in the last step of the diagram, where a ', B', C ', and D' are processed pixel points corresponding to pixel points a1, B1, C1, and D1 in the initial binary image, and it can be seen that the classification result is that the a1 pixel point belonging to the seal is removed.

In order to deepen understanding of the above image processing method, an example of the image processing method of the present invention is set forth below, and as shown in fig. 4, in one example, the image processing method of the present application may include the steps of:

1) firstly, acquiring an initial bill image with a seal, and preprocessing the initial bill image, wherein the preprocessing can comprise graying and binaryzation operations, so as to obtain a corresponding binary image;

2) taking the binary image as the input of an image segmentation model; the image segmentation model can be a trained deep learning model, and can classify each pixel point of the binary image to obtain a corresponding classification result; in this example, the training result is a matrix composed of 0 and 1, representing the classification result of each pixel point;

3) and mapping the classification result output by the image segmentation model into a corresponding pixel value to obtain an image without a seal.

In this example, the mapping method may be that element 0 in the matrix represents that the corresponding pixel point does not belong to the stamp pixel, and the pixel point retains its original pixel value; element 1 in the matrix represents that the corresponding pixel belongs to the seal, and the pixel value of the pixel is 255 (pure white). The image processing method can be used in the application of removing the seal, can also be extended and extended to be used in the application of removing the bill background and the seal together, and can be realized by changing the corresponding training method of the model.

The structure of the image segmentation model in fig. 4 will be further explained with reference to an example.

As shown in fig. 5, in one example, the image segmentation model may be comprised of an encoder and a decoder. The encoder mainly performs downsampling operation, compresses an image to obtain feature codes, and transmits the feature codes to the decoder; the decoder mainly performs an upsampling operation to restore the feature codes into a result matrix with the same size as the original pixel point matrix, the matrix includes the classification result corresponding to each pixel point, and the classification result can be represented by 0 or 1. From the input and output point of view, the input of the image segmentation model may be a binary image, and the output may be a matrix consisting of 0 and 1.

In the embodiment of the application, after each pixel point is classified to obtain a classification result, the classification result is mapped to a corresponding pixel value, and the corresponding pixel value is determined by an initial image. Specifically, in the mapping rule, the pixel value of each non-stamp pixel in the initial image is kept as much as possible, which is equivalent to that only one division is made on the pixels of the original image, the classification algorithm is adopted instead of the generation algorithm, and the result is stable and controllable.

A possible implementation manner is provided in the embodiment of the present application, before step S100 or step S200, the implementation manner may further include:

(1) acquiring a plurality of groups of sample images; each group of sample images comprises an original sample image without a stamp and a corresponding sample image with a stamp;

(2) and respectively acquiring multiple groups of corresponding binary sample images based on the multiple groups of sample images.

The above two steps are for creating a training data set for training the image segmentation model. The original sample image without the stamp may be a portion cut from the ticket without the stamp or may be an image taken from a non-ticket. The sample image with the stamp may be obtained based on the original sample image without the stamp, both constituting a set of sample images. And respectively carrying out certain processing on the multiple groups of sample images to obtain multiple groups of corresponding binary sample images.

In one embodiment, the two images in a set of sample images may be of the same type, for example, both may be color images, grayscale images, or binary images. Specifically, when the group of sample images are color images, the group of sample images may be grayed first and then binarized to obtain binary sample images corresponding to the group of sample images; when the group of sample images are gray level images, binarization processing can be performed on the group of sample images to obtain binary sample images corresponding to the group of sample images; if the group of sample images are binary images, no processing may be required.

(3) And training the initial image segmentation model according to the multiple groups of binary sample images to obtain the image segmentation model.

The initial image segmentation model may be trained by using the multiple sets of binary sample images obtained in the previous step, so as to obtain an image segmentation model. In the training process, the input of the model can be a two-value sample image with a seal, and the output can be a two-value sample image without a seal, so that supervised learning is carried out. The specific training flow may be implemented based on the framework TensorFlow. The tensrflow is a symbolic mathematical system based on dataflow programming (dataflow programming), and is widely applied to programming realization of various machine learning (machine learning) algorithms.

The embodiment of the present application provides a possible implementation manner, and when acquiring multiple sets of sample images, the implementation manner further includes: and synthesizing the seal into an original sample image aiming at each group of sample images to obtain the sample image with the seal. In one embodiment, the sample image with the stamp may be composed of the original sample image without the stamp and the stamp, with both images being equal in size. At this time, the training data of the model may not be manually labeled but synthesized, the synthesis means that the two images have the same size, and the pixel points may correspond one to one.

A possible implementation manner is provided in the embodiment of the present application, after obtaining multiple sets of sample images, the implementation manner further includes: carrying out data enhancement on sample images with stamps in the multiple groups of sample images; wherein the data enhancement operation includes at least one of adding a background and adding noise.

Data enhancement, namely, training data is transformed, and the transformation means can be background addition, noise addition and the like. In the embodiment of the application, the data enhancement can be respectively carried out on the sample images with the seals in the multiple groups of sample images, so that the model is more stable, and the phenomenon of network overfitting is reduced.

In one embodiment, the cropping operation may also be performed on two images together in a set of sample images.

In order to better understand the above image processing method, as shown in fig. 6, an example of the image processing method of the present invention is explained in detail as follows:

in one example, the present application provides an image processing method comprising the steps of:

step S401, obtaining a plurality of groups of sample images, wherein each group of sample images comprises an original sample image without a seal and a corresponding sample image with the seal;

step S402, carrying out data enhancement operation on sample images with stamps in a plurality of groups of sample images, wherein the data enhancement operation comprises at least one of background addition and noise addition;

step S403, acquiring multiple groups of corresponding binary sample images based on the multiple groups of sample images respectively;

step S404, training an initial image segmentation model according to a plurality of groups of binary sample images to obtain an image segmentation model;

step S405, acquiring an initial image to be processed containing a seal, and acquiring a corresponding binary image based on the initial image;

step S406, classifying each pixel point of the binary image based on the image segmentation model, and judging whether the classification result of the pixel point is a first classification result or a second classification result; if the classification result is the first classification result, step S407 is executed; if the classification result is the second classification result, executing step S408;

step S407, mapping a first type element representing the first classification result in the matrix to a first preset pixel value;

step S408, aiming at each second-class element representing the second classification result in the matrix, determining a corresponding target pixel of the second-class element in the initial image;

step S409, mapping the second type element to the pixel value of the target pixel;

and step S410, obtaining an image without a seal based on the first pixel value obtained by mapping and the pixel value of the target pixel.

In the above example, the image processing method performs graying and binarization processing on the initial image including the stamp to obtain a corresponding binary image, classifies each pixel point of the binary image based on the image segmentation model to obtain a corresponding classification result, and finally maps the classification result to a corresponding pixel value to obtain an image not including the stamp. According to the invention, each pixel point can be divided according to the whole information and the local information of the image without depending on the division of a color channel, so that even aiming at the image with the seal color similar to the bill character color, the pixel points of non-seal information can be retained to the maximum extent in the division, the seal removing effect is better, the interference of the seal on the bill information extraction process is reduced or even eliminated, and the bill processing accuracy is improved.

An embodiment of the present application provides an image processing apparatus, and as shown in fig. 7, the image processing apparatus 500 may include: an image pre-processing module 5001, an image segmentation module 5002, and an image post-processing module 5003. Wherein the content of the first and second substances,

the image preprocessing module 5001 is configured to obtain an initial image to be processed, which includes a stamp, and obtain a corresponding binary image based on the initial image;

the image segmentation module 5002 is configured to classify each pixel point of the binary image to obtain a corresponding classification result;

the image post-processing module 5003 is configured to map the classification result to a corresponding preset pixel value, so as to obtain an image that does not include a stamp.

In this embodiment of the present application, when obtaining a corresponding binary image based on an initial image, the image preprocessing module 5001 is specifically configured to:

In one embodiment, the image segmentation module 5002 is specifically configured to, when classifying each pixel point of the binary image based on the image segmentation model to obtain a corresponding classification result:

compressing the binary image to obtain a feature code;

In one embodiment, the classification results are in the form of a matrix; the elements of the matrix correspond to the classification result of each pixel point; when the image post-processing module 5003 maps the classification result to the corresponding pixel value, it is specifically configured to:

and for each second type element representing the second classification result in the matrix, determining a corresponding target pixel of the second type element in the initial image, and mapping the second type element to the pixel value of the target pixel.

In one embodiment, the image processing apparatus 500 further includes a model training module, specifically configured to:

In one embodiment, when the model training module acquires a plurality of sets of sample images, the model training module is specifically configured to:

In one embodiment, the image processing apparatus 500 further comprises:

the data enhancement module is used for enhancing the data of the sample images with the stamps in the plurality of groups of sample images; wherein the data enhancement operation includes at least one of adding a background and adding noise.

An embodiment of the present application provides an electronic device, including: a memory and a processor; at least one program stored in the memory for execution by the processor, which when executed by the processor, implements: each pixel point is classified according to the image characteristics, so that the method is insensitive to the style, color and shielding condition of the seal and has higher accuracy. After each pixel point is classified to obtain a classification result, the classification result is mapped to a corresponding pixel value, the corresponding pixel value is determined by the initial image, and the output result is stable.

In an alternative embodiment, an electronic device is provided, as shown in fig. 8, the electronic device 4000 shown in fig. 8 comprising: a processor 4001 and a memory 4003. Processor 4001 is coupled to memory 4003, such as via bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004. In addition, the transceiver 4004 is not limited to one in practical applications, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.

The Processor 4001 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 4001 may also be a combination that performs a computational function, including, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

Bus 4002 may include a path that carries information between the aforementioned components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 8, but this is not intended to represent only one bus or type of bus.

The Memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these.

The memory 4003 is used for storing application codes for executing the scheme of the present application, and the execution is controlled by the processor 4001. Processor 4001 is configured to execute application code stored in memory 4003 to implement what is shown in the foregoing method embodiments.

The electronic devices include, but are not limited to, mobile terminals such as mobile phones, notebook computers, PADs, etc., and fixed terminals such as digital TVs, desktop computers, etc.

The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a computer, enables the computer to execute the corresponding content in the foregoing method embodiments.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

The foregoing is only a partial embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A method of image processing, comprising:

acquiring an initial image to be processed containing a seal, and acquiring a corresponding binary image based on the initial image;

classifying each pixel point of the binary image based on an image segmentation model to obtain a corresponding classification result;

and mapping the classification result to a corresponding pixel value to obtain an image without the seal.

2. The image processing method according to claim 1, wherein the obtaining of the corresponding binary image based on the initial image comprises:

3. The image processing method according to claim 1, wherein the classifying each pixel point of the binary image based on the image segmentation model to obtain a corresponding classification result comprises:

compressing the binary image to obtain a feature code;

4. The image processing method according to claim 1, wherein the classification result is in the form of a matrix; the elements of the matrix correspond to the classification result of each pixel point; mapping the classification result to a corresponding pixel value, comprising:

mapping a first class element representing a first classification result in the matrix to a first preset pixel value;

5. The image processing method according to claim 1, wherein before classifying each pixel point of the binary image based on the image segmentation model, the method further comprises:

and training an initial image segmentation model according to the multiple groups of binary sample images to obtain the image segmentation model.

6. The image processing method of claim 5, wherein the obtaining a plurality of sets of sample images comprises:

and synthesizing the seal into the original sample image aiming at each group of sample images to obtain the sample image with the seal.

7. The image processing method according to claim 5, wherein after the acquiring the plurality of sets of sample images, further comprising:

8. An apparatus for image processing, comprising:

the image segmentation module is used for classifying each pixel point of the binary image based on an image segmentation model to obtain a corresponding classification result;

and the image post-processing module is used for mapping the classification result into a corresponding preset pixel value to obtain an image without a seal.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the image processing method according to any of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, implements the image processing method of any one of claims 1 to 7.