CN113129410B - Sketch image conversion method and related product - Google Patents
Sketch image conversion method and related product Download PDFInfo
- Publication number
- CN113129410B CN113129410B CN201911419866.5A CN201911419866A CN113129410B CN 113129410 B CN113129410 B CN 113129410B CN 201911419866 A CN201911419866 A CN 201911419866A CN 113129410 B CN113129410 B CN 113129410B
- Authority
- CN
- China
- Prior art keywords
- sketch
- image
- sketch image
- image block
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000000638 solvent extraction Methods 0.000 claims abstract description 7
- 239000013598 vector Substances 0.000 claims description 20
- 230000000903 blocking effect Effects 0.000 claims description 14
- 230000015654 memory Effects 0.000 claims description 12
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 210000004709 eyebrow Anatomy 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000000605 extraction Methods 0.000 description 4
- 210000001747 pupil Anatomy 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002146 bilateral effect Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 210000000744 eyelid Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the application provides a sketch image conversion method and related products, wherein the method comprises the following steps: partitioning the sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2; obtaining the background brightness of each sketch image block in the N sketch image blocks, and determining a target matching threshold value of each sketch image block according to the background brightness of each sketch image block; matching each sketch image block according to a target matching threshold value of each sketch image block to obtain a target face image block of each sketch image block; and combining the target face image blocks of each sketch image block to obtain the face image of the sketch image. The embodiment of the application is beneficial to improving the conversion precision of sketch images.
Description
Technical Field
The application relates to the technical field of image recognition, in particular to a sketch image conversion method and related products.
Background
The automatic portrait synthesis technology has attracted attention in recent years, and can be applied to various fields such as judicial or digital codes. For example, in the judicial field, searching a photo database of police officers for criminal suspects using sketch portraits is a very important application. In general, a sketch portrait is dictated by a related person and then drawn by a professional painter, or synthesized by an image synthesis technique according to the description of the related person. Therefore, in order to truly restore the real scene of the drawn person, environmental information (such as environmental brightness) of the drawn person is generally added to the sketch image.
Therefore, when the drawn person is in a dark environment, the drawn face sketch image is darker in brightness, the drawn sketch image is low in definition, and when the sketch image is converted into the face image, rich face features are difficult to extract, so that the conversion accuracy of the sketch image is low.
Disclosure of Invention
The embodiment of the application provides a sketch image conversion method and related products. Through the blocking processing, the target matching threshold value of each sketch image block is used for converting the sketch image, and further conversion accuracy is improved.
In a first aspect, an embodiment of the present application provides a sketch image conversion method, including:
partitioning the sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
obtaining the background brightness of each sketch image block in the N sketch image blocks, and determining a target matching threshold value of each sketch image block according to the background brightness of each sketch image block;
matching each sketch image block according to a target matching threshold value of each sketch image block to obtain a target face image block of each sketch image block;
and combining the target face image blocks of each sketch image block to obtain the face image of the sketch image.
In a second aspect, an embodiment of the present application provides a sketch image conversion device, including:
The block dividing unit is used for carrying out block dividing processing on the sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
the determining unit is used for obtaining the background brightness of each sketch image block in the N sketch image blocks and determining a target matching threshold value of each sketch image block according to the background brightness of each sketch image block;
The matching unit is used for matching each sketch image block according to the target matching threshold value of each sketch image block to obtain a target face image block of each sketch image block;
And the combining unit is used for combining the target face image blocks of each sketch image block to obtain the face image of the sketch image.
In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method according to the first aspect.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program that causes a computer to perform the method according to the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program, the computer being operable to cause a computer to perform the method according to the first aspect.
The embodiment of the application has the following beneficial effects:
it can be seen that in the embodiment of the application, the sketch image is subjected to block processing, the target matching threshold value of each sketch image block is obtained according to the background brightness of each sketch image block, and each sketch image is matched by using the target matching threshold value of each sketch image block, so that the target face image block corresponding to each sketch image block can be accurately determined, and the conversion efficiency and the conversion accuracy of the sketch image are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of sketch image conversion according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of another sketch image conversion according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of another sketch image conversion according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a sketch image conversion device according to an embodiment of the present application;
fig. 5 is a functional unit composition block diagram of a sketch image conversion device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terms "first," "second," "third," and "fourth" and the like in the description and in the claims and drawings are used for distinguishing between different objects and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, result, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The sketch image conversion device in the application can comprise a smart Phone (such as an Android Mobile Phone, an iOS Mobile Phone, a Windows Phone Mobile Phone and the like), a tablet personal computer, a palm computer, a notebook computer, a Mobile internet device MID (Mobile INTERNET DEVICES, abbreviated as MID) or a wearable device and the like. The sketch image conversion device described above is merely exemplary and not exhaustive, including but not limited to the sketch image conversion device described above. In practical applications, the method can further comprise: intelligent vehicle terminals, computer devices, etc.
Referring to fig. 1, fig. 1 is a sketch image conversion method according to an embodiment of the present application. The method is applied to a sketch image conversion device. The method includes, but is not limited to, the steps of:
101: the sketch image conversion device performs block processing on the sketch image to obtain N sketch image blocks.
Wherein N is an integer greater than or equal to 2.
The sketch image is a sketch image containing a human face. Specifically, the sketch image may be segmented in a bilateral symmetry manner to obtain N/2 sketch image block pairs, each sketch image block pair including sketch image blocks that are bilateral symmetry to each other.
102: And the sketch image conversion device acquires the background brightness of each sketch image block in the N sketch image blocks, and determines the target matching threshold value of each sketch image block according to the background brightness of each sketch image block.
The background brightness is the brightness value of each sketch image block.
And determining a target matching threshold of each sketch image block according to the mapping relation between the background brightness and the matching threshold.
103: And the sketch image conversion device matches each sketch image block according to the target matching threshold value of each sketch image block to obtain the target face image block of each sketch image block.
Specifically, extracting the characteristics of each sketch image block to obtain the characteristic vector of each sketch image block; matching the feature vector of each sketch image block with each face sketch image block template to obtain a matching value corresponding to each face sketch image block template; and taking the face sketch image block template with the matching value larger than the target matching value of the sketch image block as a target face image block of the sketch image block.
The feature extraction of each sketch image block can be performed through a neural network, and the neural network can be one or a combination of a plurality of convolutional neural networks, cyclic neural networks and long-term and short-term memory networks.
Optionally, each sketch image block may also be matched by:
Extracting features of feature points of a sketch image block A to obtain a first feature vector, wherein the sketch image block A is any sketch image block in the N sketch image blocks;
extracting features of the outline of the sketch image block A to obtain a second feature vector;
splicing the first feature vector and the second feature vector to obtain a target feature vector of the sketch image block A;
Matching the target feature vector of the sketch image block A with the target feature vector of each face template to obtain a matching value corresponding to each face template, wherein the target feature vector of each face template is also obtained by splicing the feature vector corresponding to the feature point on each face template and the feature vector corresponding to the outline.
104: And combining the target face image blocks of each sketch image block by the sketch image conversion device to obtain the face image of the sketch image.
And finally, combining the target face image blocks of each sketch image block to obtain a face image corresponding to the sketch image. If the number of the target face image blocks of a sketch image block is a plurality of, the target face image block with the largest matching value in the plurality of target face image blocks can be used as a final target face image block; and combining the final target face image block of the sketch image block with the target face image blocks corresponding to other sketch image blocks to obtain a face image corresponding to the sketch image.
It can be seen that in the embodiment of the application, the sketch image is subjected to block processing, the target matching threshold value of each sketch image block is obtained according to the background brightness of each sketch image block, and each sketch image is matched by using the target matching threshold value of each sketch image block, so that the target face image block corresponding to each sketch image block can be accurately determined, and the conversion efficiency and the conversion accuracy of the sketch image are improved.
In one possible implementation manner, when the target face image block of a certain sketch image block is plural, for example, when the target face image block of the sketch image block a is plural (the sketch image block a is any sketch image block of the N sketch image blocks), the face image of the sketch image may be further obtained by:
Combining N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain a plurality of face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
determining a coordination proportion of each face image in the plurality of face images;
and taking the face image with the largest coordination proportion among the plurality of face images as the face image of the sketch image.
When the number of the target face image blocks corresponding to the plurality of sketch blocks is a plurality of the N sketch blocks, the plurality of target face image blocks corresponding to each sketch block are required to be combined to obtain a plurality of face images, then a coordination proportion of each face image is calculated, and the face images of the sketch images are selected according to the coordination proportion.
Optionally, a neural network may be used to perform feature extraction on each of the face images to obtain a feature vector of each face image; classifying each face image according to the feature vector of each face image and each face image library, namely performing one-to-one comparison between each face image and each face template in the face image library to obtain a matching value corresponding to each face template, and taking the maximum matching value as a target matching value of the face image; and taking the target matching value of each face image as the coordination proportion of each face image. Namely, the matching of each face image and the template is equivalent, and the face images with real identities can be matched, so that the coordination ratio of the face images is higher.
Optionally, the distance between any two preset feature points on a face image block a on each face image may be obtained to obtain a plurality of first distances, where the face image block a is a target face image block corresponding to the sketch image block a, the preset feature points correspond to a face area to which the face image block a belongs, generally, 68 preset feature points (e.g., a left pupil, a right pupil, a left eyebrow, and a right eyebrow) may be divided on the face image, and then all preset feature points on the face image block a are determined according to the face area to which the face image block a belongs, and then the distances between any two preset feature points in all preset feature points on the face image block a are obtained to obtain a plurality of first distances; obtaining distances between any two preset feature points on a face image block B on each face image to obtain a plurality of second distances, wherein the face image block B is a target face image block corresponding to a sketch image block B, and the sketch image block B is an image block symmetrical to the sketch image block A in the N sketch image blocks; and obtaining the difference values between the corresponding first distance and the second distance to obtain a plurality of difference values, namely obtaining the difference values between the first distance and the second distance corresponding to the preset characteristic point (the preset characteristic point is a symmetrical characteristic point). For example, the first distance is the distance from the left pupil to the left eyebrow, and the corresponding second distance is the distance from the right pupil to the right eyebrow; and taking the standard deviation of the plurality of differences as the coordination proportion of each face image. Of course, the variance of the plurality of differences may be used as the coordination ratio of each face image.
In one possible embodiment, before performing the blocking processing on the sketch image to obtain N sketch image blocks, the method further includes:
Acquiring voice data, carrying out semantic recognition on the voice data to obtain semantic information of the voice data (semantic information extraction can be carried out by using a GRU network), carrying out keyword extraction on the semantic information to obtain a plurality of keywords, and determining sketch descriptors corresponding to each keyword in the plurality of keywords according to the mapping relation between the keywords and the sketch descriptors; and obtaining the sketch image according to the plurality of sketch descriptors, namely combining the plurality of sketch descriptors to obtain the sketch image.
Among other things, sketch descriptors include, but are not limited to: left eye, right eye, left ear, right ear, double eyelid, single eyelid, eye, nose, mouth, left eyebrow, right eyebrow, ear.
In the embodiment, the sketch image can be automatically generated through the voice data, so that the generation rate of the sketch image is improved, and further the intelligentization of converting the sketch image into the face image is realized.
In one possible embodiment, before performing the blocking processing on the sketch image to obtain N sketch image blocks, the method further includes:
Acquiring a hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image; specifically, the high-frequency direction decomposition is performed on each pixel point in the sketch image to obtain a first component of each pixel point in the horizontal direction, a second component of each pixel point in the vertical direction and a third component of each pixel point in the diagonal direction, wherein the high-frequency direction decomposition is performed on each pixel point to obtain a Hessian (Hessian) matrix of each pixel point, namely, a second partial differential of each pixel point in the horizontal direction, a second partial differential of each pixel point in the vertical direction and a mixed partial differential of each pixel point in the diagonal direction, wherein the second partial differential of each pixel point in the horizontal direction is used as the first component of each pixel point, the second partial differential of each pixel point in the vertical direction is used as the second component, and the mixed partial differential of each pixel point in the diagonal direction is used as the third component, and the process of partial differential is the prior art and is not described. Then, correspondingly forming a first component image, a second component image and a third component image of all pixel points in the sketch image in the horizontal direction, the second component image and the third component image in the diagonal direction;
and superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image.
Further, after the sketch image is enhanced, the enhanced sketch image can be segmented to obtain N sketch image blocks.
In the embodiment of the application, the sketch image is enhanced, so that the enhanced image is clearer, the face features are more obvious, and the conversion accuracy of the sketch image is further improved.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating another sketch image conversion method according to an embodiment of the present application. The method is applied to a sketch image conversion device. The method includes, but is not limited to, the steps of:
201: the sketch image conversion device acquires voice data.
202: And the sketch image conversion device performs semantic recognition on the voice data to obtain semantic information of the voice data.
203: And extracting keywords from the semantic information by the sketch image conversion device to obtain a plurality of keywords.
204: And determining the sketch descriptor corresponding to each keyword in the keywords according to the mapping relation between the keywords and the sketch descriptors by the sketch image conversion device to obtain a plurality of sketch descriptors.
205: The sketch image conversion device obtains a sketch image according to the plurality of sketch descriptors.
206: The sketch image conversion device performs block processing on the sketch image to obtain N sketch image blocks.
207: And the sketch image conversion device acquires the background brightness of each sketch image block in the N sketch image blocks, and determines the target matching threshold value of each sketch image block according to the background brightness of each sketch image block.
208: And the sketch image conversion device matches each sketch image block according to the target matching threshold value of each sketch image block to obtain the target face image block of each sketch image block.
209: And combining the target face image blocks of each sketch image block by the sketch image conversion device to obtain the face image of the sketch image.
It should be noted that, the specific implementation of each step of the method shown in fig. 2 may be referred to the specific implementation of the method shown in fig. 1, which is not described herein.
It can be seen that in the embodiment of the application, the sketch image is subjected to block processing, the target matching threshold value of each sketch image block is obtained according to the background brightness of each sketch image block, and each sketch image is matched by using the target matching threshold value of each sketch image block, so that the target face image block corresponding to each sketch image block can be accurately determined, and the conversion efficiency and the conversion accuracy of the sketch image are improved; moreover, the sketch image can be automatically generated through voice data, so that the intelligentization of sketch image conversion is realized, and the conversion efficiency of the sketch image is further improved.
Referring to fig. 3, fig. 3 illustrates another embodiment of the present application, in which the method is applied to a sketch image conversion device. The method includes, but is not limited to, the steps of:
301: the sketch image conversion device acquires voice data.
302: And the sketch image conversion device performs semantic recognition on the voice data to obtain semantic information of the voice data.
303: And extracting keywords from the semantic information by the sketch image conversion device to obtain a plurality of keywords.
304: And determining the sketch descriptor corresponding to each keyword in the keywords according to the mapping relation between the keywords and the sketch descriptors by the sketch image conversion device to obtain a plurality of sketch descriptors.
305: The sketch image conversion device obtains a sketch image according to the plurality of sketch descriptors.
306: The sketch image conversion device acquires a hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image.
307: And the sketch image conversion device is used for superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image.
308: The sketch image conversion device performs block processing on the sketch image to obtain N sketch image blocks.
309: And the sketch image conversion device acquires the background brightness of each sketch image block in the N sketch image blocks, and determines the target matching threshold value of each sketch image block according to the background brightness of each sketch image block.
310: And the sketch image conversion device matches each sketch image block according to the target matching threshold value of each sketch image block to obtain the target face image block of each sketch image block.
311: And combining the target face image blocks of each sketch image block by the sketch image conversion device to obtain the face image of the sketch image.
It should be noted that, the specific implementation of each step of the method shown in fig. 3 may be referred to the specific implementation of the method shown in fig. 1, which is not described herein.
It can be seen that in the embodiment of the application, the sketch image is subjected to block processing, the target matching threshold value of each sketch image block is obtained according to the background brightness of each sketch image block, and each sketch image is matched by using the target matching threshold value of each sketch image block, so that the target face image block corresponding to each sketch image block can be accurately determined, and the conversion efficiency and the conversion accuracy of the sketch image are improved; moreover, the sketch image can be automatically generated through voice data, so that the intelligentization of sketch image conversion is realized, and the conversion efficiency of the sketch image is further improved; before the blocking processing, the sketch image is enhanced so as to enhance the face characteristics in the sketch image, and further improve the conversion precision.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a sketch image conversion device 400 according to an embodiment of the present application, as shown in fig. 4, the sketch image conversion device 400 includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the following steps:
partitioning the sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
obtaining the background brightness of each sketch image block in the N sketch image blocks, and determining a target matching threshold value of each sketch image block according to the background brightness of each sketch image block;
matching each sketch image block according to a target matching threshold value of each sketch image block to obtain a target face image block of each sketch image block;
and combining the target face image blocks of each sketch image block to obtain the face image of the sketch image.
When the number of the target face image blocks of the sketch image block a is multiple, the sketch image block a is any one sketch image block of the N sketch image blocks, and in the aspect of combining the face sketch image blocks of each sketch image block to obtain the face image of the sketch image, the program is specifically configured to execute the following instructions:
Combining a plurality of target face blocks of the sketch image block A with N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain a plurality of face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
Determining a coordination ratio of each face image in the plurality of face images;
And taking the face image with the largest coordination proportion in the face images as the face image of the sketch image.
In one possible implementation, in determining the coordinated proportion of each face image in the plurality of face images, the above-mentioned program is specifically configured to execute instructions for:
extracting the characteristics of each face image in the plurality of face images by using a neural network to obtain the characteristic vector of each face image;
Classifying each face image according to the feature vector person of each face image and the face image library to obtain a target matching value corresponding to each face image, and taking the target matching value of each face image as the coordination proportion of each face image.
In one possible implementation, in determining the coordinated proportion of each face image in the plurality of face images, the above-mentioned program is specifically configured to execute instructions for:
Obtaining distances between any two preset feature points on a face image block a on each face image to obtain a plurality of first distances, wherein the face image block a is a target face image block corresponding to the sketch image block A;
Obtaining distances between any two preset feature points on a face image block B on each face image to obtain a plurality of second distances, wherein the face image block B is a target face image block corresponding to a sketch image block B, and the sketch image block B is an image block symmetrical to the sketch image block A in the N sketch image blocks;
obtaining a plurality of differences between the corresponding first distances and the second distances;
and taking the standard deviation of the plurality of differences as the coordination proportion of each face image.
In one possible embodiment, before the sketch image is subjected to the blocking processing to obtain N sketch image blocks, the above program is further configured to execute instructions for:
Acquiring voice data;
Carrying out semantic recognition on the voice data to obtain semantic information of the voice data;
Extracting keywords from the semantic information to obtain a plurality of keywords;
determining sketch descriptors corresponding to each keyword in the keywords according to the mapping relation between the keywords and the sketch descriptors to obtain a plurality of sketch descriptors;
and obtaining the sketch image according to the plurality of sketch descriptors.
In one possible embodiment, before the sketch image is subjected to the blocking processing to obtain N sketch image blocks, the above program is further configured to execute instructions for:
Acquiring a hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image;
Superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image;
in the aspect of performing block processing on the sketch image to obtain N sketch image blocks, the program is specifically used for executing the following instructions:
and carrying out blocking processing on the enhanced sketch image to obtain N sketch image blocks.
Referring to fig. 5, fig. 5 shows a block diagram of one possible functional unit composition of the sketch image converting apparatus 500 related to the above embodiment, and the sketch image converting apparatus 500 includes: a partitioning unit 510, a determining unit 520, a matching unit 530, a combining unit 540, wherein:
The partitioning unit 510 is configured to perform partitioning processing on the sketch image to obtain N sketch image blocks, where N is an integer greater than or equal to 2;
A determining unit 520, configured to obtain a background brightness of each sketch image block in the N sketch image blocks, and determine a target matching threshold of each sketch image block according to the background brightness of each sketch image block;
A matching unit 530, configured to match each sketch image block according to a target matching threshold of each sketch image block, so as to obtain a target face image block of each sketch image block;
And a combining unit 540, configured to combine the target face image blocks of each sketch image block to obtain a face image of the sketch image.
In one possible implementation manner, when the number of the target face image blocks of the sketch image block a is multiple, the sketch image block a is any sketch image block of the N sketch image blocks, and in combining the face sketch image blocks of each sketch image block, the combining unit 540 is specifically configured to:
Combining a plurality of target face blocks of the sketch image block A with N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain a plurality of face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
Determining a coordination ratio of each face image in the plurality of face images;
And taking the face image with the largest coordination proportion in the face images as the face image of the sketch image.
In a possible implementation manner, the combining unit 540 is specifically configured to, in determining a coordinated proportion of each face image of the plurality of face images:
extracting the characteristics of each face image in the plurality of face images by using a neural network to obtain the characteristic vector of each face image;
Classifying each face image according to the feature vector person of each face image and the face image library to obtain a target matching value corresponding to each face image, and taking the target matching value of each face image as the coordination proportion of each face image.
In a possible implementation manner, the combining unit 540 is specifically configured to, in determining a coordinated proportion of each face image of the plurality of face images:
Obtaining distances between any two preset feature points on a face image block a on each face image to obtain a plurality of first distances, wherein the face image block a is a target face image block corresponding to the sketch image block A;
Obtaining distances between any two preset feature points on a face image block B on each face image to obtain a plurality of second distances, wherein the face image block B is a target face image block corresponding to a sketch image block B, and the sketch image block B is an image block symmetrical to the sketch image block A in the N sketch image blocks;
obtaining a plurality of differences between the corresponding first distances and the second distances;
and taking the standard deviation of the plurality of differences as the coordination proportion of each face image.
In one possible embodiment, the sketch image converting apparatus 500 further includes a recognition unit 550, where, before performing the block processing on the sketch image to obtain N sketch image blocks, the recognition unit 550 is configured to:
Acquiring voice data;
Carrying out semantic recognition on the voice data to obtain semantic information of the voice data;
Extracting keywords from the semantic information to obtain a plurality of keywords;
determining sketch descriptors corresponding to each keyword in the keywords according to the mapping relation between the keywords and the sketch descriptors to obtain a plurality of sketch descriptors;
and obtaining the sketch image according to the plurality of sketch descriptors.
In one possible embodiment, before performing the blocking process on the sketch image, the sketch image converting apparatus 500 further includes an enhancing unit 560, where the enhancing unit 560 is configured to:
Acquiring a hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image;
Superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image;
in terms of performing a blocking process on the sketch image to obtain N sketch image blocks, the blocking unit 510 is specifically configured to:
and carrying out blocking processing on the enhanced sketch image to obtain N sketch image blocks.
The embodiment of the present application also provides a computer storage medium storing a computer program that is executed by a processor to implement part or all of the steps of any one of the sketch image conversion methods described in the above method embodiments.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps of any one of the sketch image conversion methods described in the method embodiments above.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are alternative embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, such as the division of the units, merely a logical function division, and there may be additional manners of dividing the actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units described above may be implemented either in hardware or in software program modules.
The integrated units, if implemented in the form of software program modules, may be stored in a computer-readable memory for sale or use as a stand-alone product. Based on this understanding, the technical solution of the present application may be embodied essentially or partly in the form of a software product, or all or part of the technical solution, which is stored in a memory, and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned memory includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and the program may be stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
The foregoing has outlined rather broadly the more detailed description of embodiments of the application, wherein the principles and embodiments of the application are explained in detail using specific examples, the above examples being provided solely to facilitate the understanding of the method and core concepts of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (6)
1. A sketch image conversion method, characterized by comprising:
partitioning the sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
Obtaining the background brightness of each sketch image block in the N sketch image blocks, and determining a target matching threshold value of each sketch image block according to the background brightness of each sketch image block;
matching each sketch image block according to a target matching threshold value of each sketch image block to obtain a target face image block of each sketch image block;
Combining the target face image blocks of each sketch image block to obtain a face image of the sketch image, wherein when the target face image blocks of the sketch image block A are a plurality of, the sketch image block A is any sketch image block in the N sketch image blocks, and the method comprises the following steps:
Combining a plurality of target face blocks of the sketch image block A with N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain a plurality of face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
Extracting features of each face image in the plurality of face images by using a neural network to obtain feature vectors of each face image; classifying each face image according to the feature vector person of each face image and a face image library to obtain a target matching value corresponding to each face image, and taking the target matching value of each face image as the coordination proportion of each face image;
Or alternatively
Obtaining distances between any two preset feature points on a face image block a on each face image to obtain a plurality of first distances, wherein the face image block a is a target face image block corresponding to the sketch image block A on each face image; obtaining distances between any two preset feature points on a face image block B on each face image to obtain a plurality of second distances, wherein the face image block B is a target face image block corresponding to a sketch image block B on each face image, and the sketch image block B is a sketch image block symmetrical to the sketch image block A in the N sketch image blocks; obtaining a plurality of differences between the corresponding first distances and the second distances; taking the standard deviation of the plurality of differences as the coordination proportion of each face image;
And taking the face image with the largest coordination proportion in the face images as the face image of the sketch image.
2. The method of claim 1, wherein prior to performing the blocking process on the sketch image to obtain N sketch image blocks, the method further comprises:
Acquiring voice data;
Carrying out semantic recognition on the voice data to obtain semantic information of the voice data;
Extracting keywords from the semantic information to obtain a plurality of keywords;
determining sketch descriptors corresponding to each keyword in the keywords according to the mapping relation between the keywords and the sketch descriptors to obtain a plurality of sketch descriptors;
and obtaining the sketch image according to the plurality of sketch descriptors.
3. The method according to claim 1 or 2, wherein before the sketch image is subjected to the blocking processing to obtain N sketch image blocks, the method further comprises:
Acquiring a hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image;
Superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image;
The block processing is carried out on the sketch image to obtain N sketch image blocks, which comprises the following steps:
and carrying out blocking processing on the enhanced sketch image to obtain N sketch image blocks.
4. A sketch image conversion device, characterized in that the device is adapted to implement the method of any of claims 1-3, the device comprising:
The block dividing unit is used for carrying out block dividing processing on the sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
the determining unit is used for obtaining the background brightness of each sketch image block in the N sketch image blocks and determining a target matching threshold value of each sketch image block according to the background brightness of each sketch image block;
The matching unit is used for matching each sketch image block according to the target matching threshold value of each sketch image block to obtain a target face image block of each sketch image block;
And the combining unit is used for combining the target face image blocks of each sketch image block to obtain the face image of the sketch image.
5. An electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured for execution by the processor, the programs comprising instructions for performing the steps of the method of any of claims 1-3.
6. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which is executed by a processor to implement the method of any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911419866.5A CN113129410B (en) | 2019-12-31 | 2019-12-31 | Sketch image conversion method and related product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911419866.5A CN113129410B (en) | 2019-12-31 | 2019-12-31 | Sketch image conversion method and related product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113129410A CN113129410A (en) | 2021-07-16 |
| CN113129410B true CN113129410B (en) | 2024-06-07 |
Family
ID=76769430
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911419866.5A Active CN113129410B (en) | 2019-12-31 | 2019-12-31 | Sketch image conversion method and related product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113129410B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109785439A (en) * | 2018-12-27 | 2019-05-21 | 深圳云天励飞技术有限公司 | Human face sketch image generating method and Related product |
| CN109858355A (en) * | 2018-12-27 | 2019-06-07 | 深圳云天励飞技术有限公司 | Image processing method and Related product |
| CN110023989A (en) * | 2017-03-29 | 2019-07-16 | 华为技术有限公司 | A kind of generation method and device of sketch image |
| WO2019223102A1 (en) * | 2018-05-22 | 2019-11-28 | 平安科技（深圳）有限公司 | Method and apparatus for checking validity of identity, terminal device and medium |
-
2019
- 2019-12-31 CN CN201911419866.5A patent/CN113129410B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110023989A (en) * | 2017-03-29 | 2019-07-16 | 华为技术有限公司 | A kind of generation method and device of sketch image |
| WO2019223102A1 (en) * | 2018-05-22 | 2019-11-28 | 平安科技（深圳）有限公司 | Method and apparatus for checking validity of identity, terminal device and medium |
| CN109785439A (en) * | 2018-12-27 | 2019-05-21 | 深圳云天励飞技术有限公司 | Human face sketch image generating method and Related product |
| CN109858355A (en) * | 2018-12-27 | 2019-06-07 | 深圳云天励飞技术有限公司 | Image processing method and Related product |
Non-Patent Citations (1)
| Title |
|---|
| 结合改进Surf特征的素描人脸识别;曹林;李猛;张华;;计算机工程与应用(第21期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113129410A (en) | 2021-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112052789B (en) | Face recognition method and device, electronic equipment and storage medium | |
| Hossain et al. | An emotion recognition system for mobile applications | |
| AU2014368997B2 (en) | System and method for identifying faces in unconstrained media | |
| Lee et al. | Intra-class variation reduction using training expression images for sparse representation based facial expression recognition | |
| CN111164601A (en) | Emotion recognition method, intelligent device and computer readable storage medium | |
| CN110503076B (en) | Video classification method, device, equipment and medium based on artificial intelligence | |
| CN110188829B (en) | Neural network training method, target recognition method and related products | |
| US12001479B2 (en) | Video processing method, video searching method, terminal device, and computer-readable storage medium | |
| CN111108508B (en) | Face emotion recognition method, intelligent device and computer readable storage medium | |
| US20210117687A1 (en) | Image processing method, image processing device, and storage medium | |
| CN110956691A (en) | Three-dimensional face reconstruction method, device, equipment and storage medium | |
| CN113362263A (en) | Method, apparatus, medium, and program product for changing the image of a virtual idol | |
| CN113705290A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| US20230143452A1 (en) | Method and apparatus for generating image, electronic device and storage medium | |
| JP2021515321A (en) | Media processing methods, related equipment and computer programs | |
| CN114239717A (en) | Model training method, image processing method and device, electronic device and medium | |
| Mohseni et al. | Recognizing induced emotions with only one feature: A novel color histogram-based system | |
| CN115861515A (en) | Three-dimensional face reconstruction method, computer program product and electronic device | |
| CN114972010A (en) | Image processing method, image processing apparatus, computer device, storage medium, and program product | |
| CN112070744B (en) | Face recognition method, system, device and readable storage medium | |
| CN111444373B (en) | Image retrieval method, device, medium and system thereof | |
| CN113129410B (en) | Sketch image conversion method and related product | |
| CN111814811A (en) | Image information extraction method, training method and device, medium and electronic equipment | |
| Abavisani et al. | A robust sparse representation based face recognition system for smartphones | |
| CN112950641B (en) | Image processing method and device, computer readable storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |