CN113129208B - Sketch-based face image generation method and related products - Google Patents

Sketch-based face image generation method and related products Download PDF

Info

Publication number
CN113129208B
CN113129208B CN201911413846.7A CN201911413846A CN113129208B CN 113129208 B CN113129208 B CN 113129208B CN 201911413846 A CN201911413846 A CN 201911413846A CN 113129208 B CN113129208 B CN 113129208B
Authority
CN
China
Prior art keywords
sketch
image
sub
target
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911413846.7A
Other languages
Chinese (zh)
Other versions
CN113129208A (en
Inventor
程冰
杨武杰
徐磊
陈耀沃
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guang Dongshenggonganting
Shenzhen Intellifusion Technologies Co Ltd
Original Assignee
Guang Dongshenggonganting
Shenzhen Intellifusion Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guang Dongshenggonganting, Shenzhen Intellifusion Technologies Co Ltd filed Critical Guang Dongshenggonganting
Priority to CN201911413846.7A priority Critical patent/CN113129208B/en
Publication of CN113129208A publication Critical patent/CN113129208A/en
Application granted granted Critical
Publication of CN113129208B publication Critical patent/CN113129208B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/04Context-preserving transformations, e.g. by using an importance map
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the application discloses a face image generation method based on sketching and related products, wherein the method comprises the following steps: receiving a target sketch image; extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images; matching calculation is carried out on the sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the sketch sub-images are obtained; and synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image. Trial experiments of the embodiment of the application have the advantage of high user experience.

Description

Sketch-based face image generation method and related products
Technical Field
The application relates to the technical field of electronics, in particular to a sketch-based face image generation method and related products.
Background
Along with the rapid development of image processing and the wide application of intelligent equipment, image processing by the intelligent equipment gradually becomes a popular technology, at present, the technology for restoring the portrait of a criminal suspect according to sketch is gradually widely applied in the judicial field, at present, when a sketch image is converted into a face image, the sketch image is usually compared with a face template in the existing database, so that the face image corresponding to the sketch image is obtained, but the direct comparison process is complex and tedious, so that the conversion efficiency of the sketch image and the face image is slower.
Disclosure of Invention
The embodiment of the application provides a sketch-based face image generation method and related products, wherein a sketch image is divided into a plurality of sketch sub-images, and the face sub-images are determined to be synthesized into a face image according to the plurality of sketch sub-images, so that the conversion efficiency of the sketch image and the face image is improved.
In a first aspect, an embodiment of the present application provides a sketch-based face image generating method, which is applied to an electronic device, and the method includes:
receiving a target sketch image;
extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
matching calculation is carried out on the sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the sketch sub-images are obtained;
and synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image.
In a second aspect, an embodiment of the present application provides a face image generating device based on sketching, where the device includes:
the receiving unit is used for receiving the target sketch image;
the extraction unit is used for extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
The matching unit is used for carrying out matching calculation on the sketch sub-images and a preset face template set to obtain a plurality of face sub-images corresponding to the sketch sub-images;
and the synthesis unit is used for synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image.
In a third aspect, an embodiment of the present application provides an electronic device, including a controller, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the controller, the programs including instructions for performing steps in any of the methods of the first aspect of the embodiments of the present application.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, where the computer program causes a computer to perform some or all of the steps as described in any of the methods of the first aspect of the embodiments of the present application.
In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product comprises a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in any of the methods of the first aspect of embodiments of the present application. The computer program product may be a software installation package.
It can be seen that in the embodiment of the application, the electronic device receives a target sketch image; extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images; matching calculation is carried out on the sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the sketch sub-images are obtained; and synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image. Therefore, the sketch image is divided based on the sketch image feature points of the sketch image, so that the accuracy of the sketch image division is improved, the synthesis of a plurality of face sub-images is determined according to the plurality of sketch sub-images and the face template set, and the conversion efficiency of the sketch image and the face image conversion is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a face image generating method based on sketching according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of another sketch-based face image generation method according to an embodiment of the present application;
FIG. 3 is a flow chart of another sketch-based face image generation method according to an embodiment of the present application;
FIG. 4 is a flowchart of another sketch-based face image generation method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 6 is a functional unit composition block diagram of a sketch-based face image generating device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "first," "second," "third," and "fourth" and the like in the description and in the claims and drawings are used for distinguishing between different objects and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, result, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
The electronic devices may include various handheld devices, vehicle mounted devices, wearable devices (e.g., smart watches, smart bracelets, pedometers, etc.), computing devices or other processing devices communicatively coupled to wireless modems, as well as various forms of User Equipment (UE), mobile Stations (MSs), terminal devices (terminal devices), etc. For convenience of description, the above-mentioned devices are collectively referred to as electronic devices.
Referring to fig. 1, fig. 1 is a flow chart of a sketch-based face image generating method provided in an embodiment of the present application, and the sketch-based face image generating method is applied to an electronic device, and includes:
Step 101, receiving a target sketch image;
optionally, a data transmission request sent by a preset server or terminal device is received, where the data transmission request is used to request the electronic device to establish a data transmission channel with the preset server or terminal device, and the electronic device establishes a data transmission channel according to the data transmission request to connect with the preset server or terminal device, and receives a target scan image transmitted by the preset server or terminal device. In a possible example, the electronic device starts a remote communication module, where the remote communication module is configured to send a connection request to a preset server or a terminal device, where the connection request is configured to request the preset server or the terminal device to perform a communication connection with the electronic device, receive a connection response returned by the preset server or the terminal device, and establish a data transmission channel according to the connection response to connect with the preset server or the terminal device, where the remote communication module may include: wireless fidelity Wi-Fi module, bluetooth module, cellular data network, etc., without limitation.
The above-mentioned manner of enabling the remote communication module by the terminal may be various, for example, in an alternative embodiment, it may be determined whether to simultaneously enable the remote communication module by a specific button. Of course, in another alternative embodiment, the remote communication module may be activated by satisfying a set trigger condition, which may be a specific operation to determine whether to activate the remote communication module, including but not limited to, a specific gesture, or biometric verification, including but not limited to: face recognition verification, fingerprint recognition verification, vein recognition verification, and the like. The embodiments of the present application are not limited to the above-described scheme for activating the remote communication module.
102, extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
optionally, extracting the sketch image feature point set corresponding to the target sketch image includes: and acquiring a preset feature extraction algorithm, and executing the feature extraction algorithm on the target sketch image to obtain the sketch image feature point set.
Step 103, matching calculation is carried out on the sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the sketch sub-images are obtained;
optionally, a preset matching model is obtained, the multiple sketch sub-images and the face template set are used as input of the matching model, multiple face templates matched with the multiple sketch sub-images are obtained, and the multiple face sub-images are determined according to the multiple face templates.
Step 104, synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image.
Optionally, synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image includes: and acquiring a target sketch image and a plurality of sketch sub-images, calculating a plurality of connection matrixes corresponding to the plurality of sketch sub-images according to the target sketch image, and synthesizing the plurality of face sub-images according to the plurality of connection matrixes to obtain the face image.
In a possible example, the extracting the sketch image feature point set corresponding to the target sketch image includes: acquiring a preset image format, wherein the image format comprises at least one of the following: width, height, bit depth, resolution; performing format preprocessing operation on the target sketch image according to the image format to obtain a preprocessed sketch image; and taking the preprocessed sketch image as input of a preset convolutional neural network to obtain the sketch image feature point set.
Wherein the image format may further include: horizontal resolution, vertical resolution, image size, etc., are not limited herein.
Optionally, performing the format preprocessing operation on the target sketch image according to the image format may include: updating the target image format corresponding to the target sketch image according to the image format, namely, assuming that the image format comprises: the method comprises the steps of obtaining a target image format corresponding to a target sketch image, wherein the width 219, the height 191 and the resolution are 219×191, and adjusting the width to 219, the height to 191 and the resolution to 219×191 in the target image format.
Optionally, after the pretreatment sketch image is obtained, detecting the pretreatment sketch image, judging whether a target face image contained in the pretreatment sketch image is blocked, if the target face image is blocked, dividing the pretreatment sketch image into two parts, wherein the first part is a first sketch image which is not blocked in the pretreatment sketch image, the second part is a second sketch image which is blocked in the pretreatment sketch image, and taking the first sketch image as the input of the preset convolutional neural network to obtain the sketch image feature point set; and if the preprocessed sketch image is not blocked, taking the preprocessed sketch image as the input of the preset convolutional neural network to obtain the sketch image feature point set.
In a possible example, the dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images includes: extracting a plurality of part feature point sets corresponding to a plurality of preset parts from the sketch image feature point set, wherein the plurality of parts comprise: eyes, nose, mouth, eyebrows, face, head, and ears; determining a plurality of position coordinate point sets corresponding to the preset positions in the target sketch image based on the plurality of position feature point sets; clipping the target sketch area image based on the plurality of position coordinate point sets to obtain a plurality of sketch sub-images, and marking the plurality of sketch sub-images according to the plurality of preset positions.
In a possible example, detecting the preprocessed sketch image, judging whether a target face image contained in the preprocessed sketch image is blocked, if the target face image is blocked, dividing the preprocessed sketch image into two parts, wherein the first part is a first sketch image which is not blocked in the preprocessed sketch image, and the second part is a second sketch image which is blocked in the preprocessed sketch image; executing a feature extraction algorithm on the first sketch image to obtain a feature point set of the sketch image; if the preprocessed sketch image is not blocked, executing a feature extraction algorithm aiming at the preprocessed sketch image to obtain a feature point set of the sketch image; wherein the feature extraction operation includes: the electronic device obtains a preset feature extraction frame and a search step length, determines a plurality of feature extraction areas in the preprocessed sketch image according to the search step length and the feature extraction frame, and executes a feature extraction algorithm for the plurality of feature extraction areas to obtain a sketch image feature point set, wherein the feature extraction algorithm can comprise: haar-Like feature extraction algorithms, HOG feature extraction algorithms, and the Like, without limitation; acquiring a preset classifier and acquiring a preset training sample, wherein the training sample comprises: the method comprises the steps of training a classifier according to a training sample to obtain a trained classifier, taking a sketch image feature point set as input of the trained classifier to obtain 7 position feature point sets corresponding to the sketch image feature point set, wherein any one of the N sketch feature point sets corresponds to 7 sketch position feature point sets, and the 7 positions respectively comprise: eyes, nose, mouth, eyebrows, face, head, and ears.
Optionally, the preprocessing sketch image is obtained, an edge detection algorithm is performed on the preprocessing sketch image, a boundary of a target sketch face contained in the preprocessing sketch image is determined, and the target sketch face is positioned in the preprocessing sketch image according to the boundary.
Further, a preset structure detection model is obtained, the structure detection model is used for detecting the pretreatment sketch image, a face structure corresponding to a target sketch face included in the pretreatment sketch image is determined, a plurality of preset parts included in the pretreatment sketch image are positioned according to the face structure, position coordinates corresponding to the plurality of preset parts are obtained, a preset cutting frame is obtained, and the target sketch image is cut according to the plurality of position coordinates and the cutting frame, so that a plurality of sketch sub-images are obtained.
In a possible example, the matching calculation of the plurality of sketch sub-images and a preset face template set to obtain a plurality of face sub-images corresponding to the plurality of sketch sub-images includes: determining any sketch sub-image in the plurality of sketch sub-images as a target sketch sub-image; acquiring a target position label corresponding to the target sketch sub-image; acquiring a plurality of target part templates corresponding to the target part labels in the face template set; calculating a plurality of similarities corresponding to the target sketch sub-image and the target part templates, and determining a plurality of weights corresponding to the similarities according to a preset mapping relation between the similarities and the weights; taking the target position templates and the weights as input of a preset face generation model to obtain a face sub-image corresponding to the target sketch sub-image; repeating the steps to obtain a plurality of face sub-images corresponding to the sketch sub-images.
In the specific implementation process, a target sketch sub-image is determined from a plurality of sketch sub-images, wherein the target sketch sub-image can be any sketch sub-image in the plurality of sketch sub-images, and a target part label corresponding to the target sketch sub-image is determined, namely if the target sketch sub-image is an eye image, the target part label corresponding to the target sketch sub-image is determined to be an eye part; the face template set comprises a plurality of part template sets, namely the face template set comprises: an eye template set, a nose template set, a mouth template set, a eyebrow template set, a face template set, a head template set and an ear template set, when the target part label is 'eyes', extracting the eye template set from the face template set; calculating a plurality of similarities between the target sketch sub-image and a plurality of eye templates contained in the eye template set, and determining a plurality of weights corresponding to the plurality of similarities according to a mapping relation between the preset similarities and the weights, wherein the mapping relation between the preset similarities and the weights can comprise:
similarity (x) i ) Weight (w) i )
<10% 0.1
10%<x i <25% 0.3
25%<x i <50% 0.5
50%<x i <75% 0.7
75%<x i <100% 0.9
Wherein n is the total number of the eye templates contained in the eye template set, that is, the eye template set is assumed to contain 5 eye templates, and the similarity between the target sketch sub-image and the 5 eye templates is calculated as follows: 9%, 37%, 95%, 55% and 60%, then the 5 weights are respectively: 0.1, 0.5, 0.9, 0.7 and 0.7; and acquiring a preset face generation model, taking the 5 weights and the eye template set as inputs of the face generation model to obtain a face sub-image corresponding to the target sketch sub-image, wherein the face sub-image is an eye image, and repeating the steps to obtain a plurality of face sub-images.
In a possible example, the synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image includes: and acquiring a preset synthetic model, and taking the face sub-images, the sketch sub-images and the target sketch image as inputs of the synthetic model to obtain the face image.
Optionally, sending an image composition request to a preset server, where the image composition request includes: the image synthesis request is used for requesting the preset server to synthesize the face sub-images, and receiving an image synthesis response returned by the preset server, wherein the image synthesis response may include: and extracting the face image from the image synthesis response according to the face image synthesized by the face sub-images.
It can be seen that in the embodiment of the application, the electronic device receives a target sketch image; extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images; matching calculation is carried out on the sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the sketch sub-images are obtained; and synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image. Therefore, the sketch image is divided based on the sketch image feature points of the sketch image, so that the accuracy of the sketch image division is improved, the synthesis of a plurality of face sub-images is determined according to the plurality of sketch sub-images and the face template set, the conversion efficiency of the sketch image and the face image is improved, the similarity of the sketch image and the face image is improved, and the user experience is improved.
Referring to fig. 2, fig. 2 is a flow chart of another sketch-based face image generating method according to an embodiment of the present application, which is applied to an electronic device, and the sketch-based face image generating method includes:
step 201, receiving a target sketch image;
step 202, acquiring a preset image format, wherein the image format comprises at least one of the following: width, height, bit depth, resolution;
step 203, performing a format preprocessing operation on the target sketch image according to the image format to obtain a preprocessed sketch image;
step 204, taking the preprocessed sketch image as input of a preset convolutional neural network to obtain the sketch image feature point set;
step 205, dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
step 206, performing matching calculation on the sketch sub-images and a preset face template set to obtain face sub-images corresponding to the sketch sub-images;
step 207, synthesizing the face sub-images to obtain a face image corresponding to the target sketch image.
The specific description of the steps 201 to 207 may refer to the corresponding steps of the sketch-based face image generating method described in fig. 1, and are not repeated herein.
It can be seen that in the embodiment of the application, the electronic device receives a target sketch image; acquiring a preset image format, wherein the image format comprises at least one of the following: width, height, bit depth, resolution; performing format preprocessing operation on the target sketch image according to the image format to obtain a preprocessed sketch image; taking the preprocessed sketch image as input of a preset convolutional neural network to obtain a feature point set of the sketch image; dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images; matching calculation is carried out on the plurality of sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the plurality of sketch sub-images are obtained; and synthesizing the face sub-images to obtain the face image corresponding to the target sketch image. Therefore, after the format preprocessing operation is performed on the target sketch image, the sketch feature points are extracted through the convolutional neural network, so that the feature point extraction rate can be improved, the target sketch image is divided into a plurality of sketch sub-images according to the sketch image feature point set to perform matching calculation, the matching rate can be improved, the conversion rate of the sketch image and the face image is further improved, the similarity of the sketch image and the face image is favorably improved, and the user experience is improved.
Referring to fig. 3, fig. 3 is a flow chart of another sketch-based face image generating method according to an embodiment of the present application, which is applied to an electronic device, and the sketch-based face image generating method includes:
step 301, receiving a target sketch image;
step 302, extracting a sketch image feature point set corresponding to the target sketch image, and extracting a plurality of position feature point sets corresponding to a plurality of preset positions from the sketch image feature point set, wherein the plurality of positions comprise: eyes, nose, mouth, eyebrows, face, head, and ears;
step 303, determining 7 position coordinate point sets corresponding to the preset positions in the target sketch image based on the plurality of position feature point sets;
step 304, clipping the target sketch area image based on the plurality of position coordinate point sets to obtain a plurality of sketch sub-images, and marking the plurality of sketch sub-images according to the plurality of preset positions;
step 305, performing matching calculation on the multiple sketch sub-images and a preset face template set to obtain multiple face sub-images corresponding to the multiple sketch sub-images;
step 306, synthesizing the face sub-images to obtain a face image corresponding to the target sketch image.
The specific description of the steps 301 to 306 may refer to the corresponding steps of the sketch-based face image generating method described in fig. 1, and are not repeated herein.
It can be seen that in the embodiment of the application, the electronic device receives a target sketch image; extracting a sketch image feature point set corresponding to the target sketch image, and extracting a plurality of position feature point sets corresponding to a plurality of preset positions from the sketch image feature point set, wherein the plurality of positions comprise: eyes, nose, mouth, eyebrows, face, head, and ears; determining 7 position coordinate point sets corresponding to the preset positions in the target sketch image based on the position feature point sets; clipping the target sketch area image based on the plurality of position coordinate point sets to obtain a plurality of sketch sub-images, and marking the plurality of sketch sub-images according to the plurality of preset positions; matching calculation is carried out on the plurality of sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the plurality of sketch sub-images are obtained; and synthesizing the face sub-images to obtain the face image corresponding to the target sketch image. Therefore, the division of the target sketch image is realized by extracting the part feature point set of the preset part from the image feature point set, the accuracy and the division rate of the image division are improved, the matching calculation is performed according to the sketch sub-images and the face template set, the matching rate can be improved, the conversion rate of the sketch image and the face image is further improved, and the user experience is improved.
Referring to fig. 4, fig. 4 is a flow chart of another sketch-based face image generating method according to an embodiment of the present application, which is applied to an electronic device, and the sketch-based face image generating method includes:
step 401, receiving a target sketch image;
step 402, extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
step 403, determining any sketch sub-image in the plurality of sketch sub-images as a target sketch sub-image;
step 404, obtaining a target part label corresponding to the target sketch sub-image;
step 405, acquiring a plurality of target position templates corresponding to the target position tags in the face template set;
step 406, calculating a plurality of similarities corresponding to the target sketch sub-image and the target part templates, and determining a plurality of weights corresponding to the plurality of similarities according to a mapping relation between preset similarities and weights;
step 407, taking the target part templates and the weights as input of a preset face generation model to obtain a face sub-image corresponding to the target sketch sub-image;
Step 408, repeating the above steps to obtain a plurality of face sub-images corresponding to the plurality of sketch sub-images;
and 409, synthesizing the face sub-images to obtain a face image corresponding to the target sketch image.
The specific description of the steps 401 to 409 may refer to the corresponding steps of the sketch-based face image generating method described in fig. 1, and are not repeated herein.
It can be seen that in the embodiment of the application, the electronic device receives a target sketch image; extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images; determining any one sketch sub-image in the plurality of sketch sub-images as a target sketch sub-image; obtaining a target position label corresponding to the target sketch sub-image; acquiring a plurality of target part templates corresponding to the target part labels in the face template set; calculating a plurality of similarities corresponding to the target sketch sub-image and the target part templates, and determining a plurality of weights corresponding to the similarities according to a preset mapping relation between the similarities and the weights; taking the target position templates and the weights as input of a preset face generation model to obtain a face sub-image corresponding to the target sketch sub-image; repeating the steps to obtain a plurality of face sub-images corresponding to the sketch sub-images; and synthesizing the face sub-images to obtain the face image corresponding to the target sketch image. Therefore, the target part template and the target sketch sub-image can be determined to be matched from the face template set based on the part label, the matching times are reduced, the matching rate is improved, the conversion rate of the sketch image and the face image is further improved, and the user experience is improved.
Referring to fig. 5, fig. 5 is a schematic structural diagram of an electronic device 500 according to an embodiment of the present application, as shown in the drawing, the electronic device 500 includes: an application processor 510, a memory 520, a communication interface 530, and one or more programs 521, wherein the one or more programs 521 are stored in the memory 520 and configured to be executed by the application processor 510, the one or more programs 521 comprising instructions for performing the steps of: the embodiment of the application also provides a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, where the computer program causes a computer to execute part or all of the steps of any one of the methods described in the embodiments of the method, where the computer includes an electronic device.
Receiving a target sketch image;
extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
matching calculation is carried out on the sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the sketch sub-images are obtained;
And synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image.
It can be seen that in the embodiment of the application, the electronic device receives a target sketch image; extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images; matching calculation is carried out on the sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the sketch sub-images are obtained; and synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image. Therefore, the sketch image is divided based on the sketch image feature points of the sketch image, so that the accuracy of the sketch image division is improved, the synthesis of a plurality of face sub-images is determined according to the plurality of sketch sub-images and the face template set, the conversion efficiency of the sketch image and the face image is improved, the similarity of the sketch image and the face image is improved, and the user experience is improved.
In a possible example, the extracting the sketch image feature point set corresponding to the target sketch image is specifically used for executing the following operations: acquiring a preset image format, wherein the image format comprises at least one of the following: width, height, bit depth, resolution; performing format preprocessing operation on the target sketch image according to the image format to obtain a preprocessed sketch image; and taking the preprocessed sketch image as input of a preset convolutional neural network to obtain the sketch image feature point set.
In a possible example, the dividing the target sketch image based on the sketch image feature point set obtains a plurality of sketch sub-images, and the instructions in the program are specifically configured to perform the following operations: extracting a plurality of part feature point sets corresponding to a plurality of preset parts from the sketch image feature point set, wherein the plurality of parts comprise: eyes, nose, mouth, eyebrows, face, head, and ears; determining 7 position coordinate point sets corresponding to the preset positions in the target sketch image based on the position feature point sets; clipping the target sketch area image based on the plurality of position coordinate point sets to obtain a plurality of sketch sub-images, and marking the plurality of sketch sub-images according to the plurality of preset positions.
In a possible example, the matching calculation is performed on the plurality of sketch sub-images and a preset face template set to obtain a plurality of face sub-images corresponding to the plurality of sketch sub-images, and the instructions in the program are specifically configured to perform the following operations: determining any sketch sub-image in the plurality of sketch sub-images as a target sketch sub-image; acquiring a target position label corresponding to the target sketch sub-image; acquiring a plurality of target part templates corresponding to the target part labels in the face template set; calculating a plurality of similarities corresponding to the target sketch sub-image and the target part templates, and determining a plurality of weights corresponding to the similarities according to a preset mapping relation between the similarities and the weights; taking the target position templates and the weights as input of a preset face generation model to obtain a face sub-image corresponding to the target sketch sub-image; repeating the steps to obtain a plurality of face sub-images corresponding to the sketch sub-images.
In a possible example, the synthesizing the face sub-images obtains a face image corresponding to the target sketch image, and the instructions in the program are specifically configured to perform the following operations: and acquiring a preset synthetic model, and taking the face sub-images, the sketch sub-images and the target sketch image as inputs of the synthetic model to obtain the face image.
The foregoing description of the embodiments of the present application has been presented primarily in terms of a method-side implementation. It will be appreciated that the electronic device, in order to achieve the above-described functions, includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the application may divide the functional units of the electronic device according to the above method example, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated in one control unit. The integrated units may be implemented in hardware or in software functional units. It should be noted that, in the embodiment of the present application, the division of the units is schematic, which is merely a logic function division, and other division manners may be implemented in actual practice.
Fig. 6 is a functional block diagram of a sketch-based face-image generating apparatus 600 according to an embodiment of the present application, the sketch-based face-image generating apparatus 600 being applied to an electronic device, the sketch-based face-image generating apparatus 600 including a receiving unit 601, an extracting unit 602, a matching unit 603, and a synthesizing unit 604, wherein:
a receiving unit 601, configured to receive a target sketch image;
an extracting unit 602, configured to extract a sketch image feature point set corresponding to the target sketch image, and divide the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
A matching unit 603, configured to perform matching calculation on the multiple sketch sub-images and a preset face template set, so as to obtain multiple face sub-images corresponding to the multiple sketch sub-images;
and a synthesizing unit 604, configured to synthesize the plurality of face sub-images to obtain a face image corresponding to the target sketch image.
It can be seen that in the embodiment of the application, the electronic device receives a target sketch image; extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images; matching calculation is carried out on the sketch sub-images and a preset face template set, so that a plurality of face sub-images corresponding to the sketch sub-images are obtained; and synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image. Therefore, the sketch image is divided based on the sketch image feature points of the sketch image, so that the accuracy of the sketch image division is improved, the synthesis of a plurality of face sub-images is determined according to the plurality of sketch sub-images and the face template set, the conversion efficiency of the sketch image and the face image is improved, the similarity of the sketch image and the face image is improved, and the user experience is improved.
In a possible example, the extracting unit 602 is specifically configured to extract a sketch image feature point set corresponding to the target sketch image: acquiring a preset image format, wherein the image format comprises at least one of the following: width, height, bit depth, resolution; performing format preprocessing operation on the target sketch image according to the image format to obtain a preprocessed sketch image; and taking the preprocessed sketch image as input of a preset convolutional neural network to obtain the sketch image feature point set.
In a possible example, the dividing the target sketch image based on the sketch image feature point set obtains a plurality of sketch sub-images, and the extracting unit 602 is specifically configured to: extracting a plurality of part feature point sets corresponding to a plurality of preset parts from the sketch image feature point set, wherein the plurality of parts comprise: eyes, nose, mouth, eyebrows, face, head, and ears; determining 7 position coordinate point sets corresponding to the preset positions in the target sketch image based on the position feature point sets; clipping the target sketch area image based on the plurality of position coordinate point sets to obtain a plurality of sketch sub-images, and marking the plurality of sketch sub-images according to the plurality of preset positions.
In a possible example, the matching calculation is performed on the plurality of sketch sub-images and a preset face template set to obtain a plurality of face sub-images corresponding to the plurality of sketch sub-images, and the matching unit 603 is specifically configured to: determining any sketch sub-image in the plurality of sketch sub-images as a target sketch sub-image; acquiring a target position label corresponding to the target sketch sub-image; acquiring a plurality of target part templates corresponding to the target part labels in the face template set; calculating a plurality of similarities corresponding to the target sketch sub-image and the target part templates, and determining a plurality of weights corresponding to the similarities according to a preset mapping relation between the similarities and the weights; taking the target position templates and the weights as input of a preset face generation model to obtain a face sub-image corresponding to the target sketch sub-image; repeating the steps to obtain a plurality of face sub-images corresponding to the sketch sub-images.
In a possible example, the synthesizing unit 604 is specifically configured to synthesize the face sub-images to obtain a face image corresponding to the target sketch image: and acquiring a preset synthetic model, and taking the face sub-images, the sketch sub-images and the target sketch image as inputs of the synthetic model to obtain the face image.
The embodiment of the application also provides a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, where the computer program causes a computer to execute part or all of the steps of any one of the methods described in the embodiments of the method, where the computer includes an electronic device.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any one of the methods described in the method embodiments above. The computer program product may be a software installation package, said computer comprising an electronic device.
It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of action combinations, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required in the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, such as the above-described division of units, merely a division of logic functions, and there may be additional manners of dividing in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, including several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the above-mentioned method of the various embodiments of the present application. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and the program may be stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
The foregoing has outlined rather broadly the more detailed description of embodiments of the present application, wherein specific examples are provided herein to illustrate the principles and embodiments of the present application, the above examples being provided solely to assist in the understanding of the methods of the present application and the core ideas thereof; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims (9)

1. A sketch-based face image generation method, the method comprising:
receiving a target sketch image;
extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
Matching calculation is carried out on the sketch sub-images and a preset face template set to obtain face sub-images corresponding to the sketch sub-images, and the matching calculation comprises the following steps: determining any sketch sub-image in the plurality of sketch sub-images as a target sketch sub-image; acquiring a target position label corresponding to the target sketch sub-image; acquiring a plurality of target part templates corresponding to the target part labels in the face template set; calculating a plurality of similarities corresponding to the target sketch sub-image and the target part templates, and determining a plurality of weights corresponding to the similarities according to a preset mapping relation between the similarities and the weights; taking the target position templates and the weights as input of a preset face generation model to obtain a face sub-image corresponding to the target sketch sub-image; repeating the steps to obtain a plurality of face sub-images corresponding to the sketch sub-images;
synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image;
the dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images comprises the following steps: extracting a plurality of position feature point sets corresponding to a plurality of preset positions from the sketch image feature point set; determining a plurality of position coordinate point sets corresponding to the preset positions in the target sketch image based on the plurality of position feature point sets; clipping the target sketch area image based on the plurality of position coordinate point sets to obtain a plurality of sketch sub-images, and marking the plurality of sketch sub-images according to the plurality of preset positions.
2. The method of claim 1, wherein the extracting the set of sketch image feature points corresponding to the target sketch image comprises:
acquiring a preset image format, wherein the image format comprises at least one of the following: width, height, bit depth, resolution;
performing format preprocessing operation on the target sketch image according to the image format to obtain a preprocessed sketch image;
and taking the preprocessed sketch image as input of a preset convolutional neural network to obtain the sketch image feature point set.
3. The method of claim 2, wherein the plurality of sites comprises: eyes, nose, mouth, eyebrows, face, head, and ears.
4. The method according to claim 1, wherein the synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image includes:
and acquiring a preset synthetic model, and taking the face sub-images, the sketch sub-images and the target sketch image as inputs of the synthetic model to obtain the face image.
5. A sketch-based face image generation device, the device comprising:
The receiving unit is used for receiving the target sketch image;
the extraction unit is used for extracting a sketch image feature point set corresponding to the target sketch image, and dividing the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images;
the matching unit is configured to perform matching calculation on the multiple sketch sub-images and a preset face template set, to obtain multiple face sub-images corresponding to the multiple sketch sub-images, where the matching unit includes: determining any sketch sub-image in the plurality of sketch sub-images as a target sketch sub-image; acquiring a target position label corresponding to the target sketch sub-image; acquiring a plurality of target part templates corresponding to the target part labels in the face template set; calculating a plurality of similarities corresponding to the target sketch sub-image and the target part templates, and determining a plurality of weights corresponding to the similarities according to a preset mapping relation between the similarities and the weights; taking the target position templates and the weights as input of a preset face generation model to obtain a face sub-image corresponding to the target sketch sub-image; repeating the steps to obtain a plurality of face sub-images corresponding to the sketch sub-images;
The synthesizing unit is used for synthesizing the plurality of face sub-images to obtain a face image corresponding to the target sketch image;
the extraction unit is specifically configured to divide the target sketch image based on the sketch image feature point set to obtain a plurality of sketch sub-images: extracting a plurality of position feature point sets corresponding to a plurality of preset positions from the sketch image feature point set; determining a plurality of position coordinate point sets corresponding to the preset positions in the target sketch image based on the plurality of position feature point sets; clipping the target sketch area image based on the plurality of position coordinate point sets to obtain a plurality of sketch sub-images, and marking the plurality of sketch sub-images according to the plurality of preset positions.
6. The apparatus according to claim 5, wherein, in the aspect of extracting the sketch image feature point set corresponding to the target sketch image, the extracting unit is specifically configured to:
acquiring a preset image format, wherein the image format comprises at least one of the following: width, height, bit depth, resolution;
performing format preprocessing operation on the target sketch image according to the image format to obtain a preprocessed sketch image;
And taking the preprocessed sketch image as input of a preset convolutional neural network to obtain the sketch image feature point set.
7. The apparatus of claim 6, wherein the plurality of sites comprises: eyes, nose, mouth, eyebrows, face, head, and ears.
8. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-4.
9. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which is executed by a processor to implement the method of any one of claims 1-4.
CN201911413846.7A 2019-12-31 2019-12-31 Sketch-based face image generation method and related products Active CN113129208B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911413846.7A CN113129208B (en) 2019-12-31 2019-12-31 Sketch-based face image generation method and related products

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911413846.7A CN113129208B (en) 2019-12-31 2019-12-31 Sketch-based face image generation method and related products

Publications (2)

Publication Number Publication Date
CN113129208A CN113129208A (en) 2021-07-16
CN113129208B true CN113129208B (en) 2024-03-26

Family

ID=76770410

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911413846.7A Active CN113129208B (en) 2019-12-31 2019-12-31 Sketch-based face image generation method and related products

Country Status (1)

Country Link
CN (1) CN113129208B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011060289A (en) * 2009-09-08 2011-03-24 Xiaogang Wang Face image synthesis method and system
CN107967667A (en) * 2017-12-21 2018-04-27 广东欧珀移动通信有限公司 Generation method, device, terminal device and the storage medium of sketch
CN109359541A (en) * 2018-09-17 2019-02-19 南京邮电大学 A kind of sketch face identification method based on depth migration study
WO2019128508A1 (en) * 2017-12-28 2019-07-04 Oppo广东移动通信有限公司 Method and apparatus for processing image, storage medium, and electronic device
CN110222588A (en) * 2019-05-15 2019-09-10 合肥进毅智能技术有限公司 A kind of human face sketch image aging synthetic method, device and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011060289A (en) * 2009-09-08 2011-03-24 Xiaogang Wang Face image synthesis method and system
CN107967667A (en) * 2017-12-21 2018-04-27 广东欧珀移动通信有限公司 Generation method, device, terminal device and the storage medium of sketch
WO2019128508A1 (en) * 2017-12-28 2019-07-04 Oppo广东移动通信有限公司 Method and apparatus for processing image, storage medium, and electronic device
CN109359541A (en) * 2018-09-17 2019-02-19 南京邮电大学 A kind of sketch face identification method based on depth migration study
CN110222588A (en) * 2019-05-15 2019-09-10 合肥进毅智能技术有限公司 A kind of human face sketch image aging synthetic method, device and storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于特征融合的人脸素描自动生成***研究;黄双萍;王卫星;吴伟斌;薛昆南;冯绮颖;;计算机应用与软件(第04期);全文 *
结合改进Surf特征的素描人脸识别;曹林;李猛;张华;;计算机工程与应用(第21期);全文 *

Also Published As

Publication number Publication date
CN113129208A (en) 2021-07-16

Similar Documents

Publication Publication Date Title
CN108090450B (en) Face recognition method and device
US9443155B2 (en) Systems and methods for real human face recognition
EP3229171A1 (en) Method and device for determining identity identifier of human face in human face image, and terminal
CN110176226A (en) A kind of speech recognition and speech recognition modeling training method and device
WO2019033525A1 (en) Au feature recognition method, device and storage medium
CN104361276A (en) Multi-mode biometric authentication method and multi-mode biometric authentication system
CN107016346A (en) gait identification method and system
CN106899968A (en) A kind of active noncontact identity identifying method based on WiFi channel condition informations
CN111626371A (en) Image classification method, device and equipment and readable storage medium
EP3685288B1 (en) Apparatus, method and computer program product for biometric recognition
CN107911643B (en) Method and device for showing scene special effect in video communication
CN102254188A (en) Palmprint recognizing method and device
CN111144566B (en) Training method for neural network weight parameters, feature classification method and corresponding device
CN101674363A (en) Mobile equipment and talking method
CN112036261A (en) Gesture recognition method and device, storage medium and electronic device
WO2017088434A1 (en) Human face model matrix training method and apparatus, and storage medium
CN113254491A (en) Information recommendation method and device, computer equipment and storage medium
CN111126254A (en) Image recognition method, device, equipment and storage medium
CN111160157B (en) Text extraction method based on DWG drawing and related products
CN103870812A (en) Method and system for acquiring palmprint image
CN104881647B (en) Information processing method, information processing system and information processing unit
CN113129208B (en) Sketch-based face image generation method and related products
CN107944429B (en) Face recognition method and device and mobile terminal used by same
CN108288023B (en) Face recognition method and device
KR20110032429A (en) Apparatus and method for recognzing gesture in mobile device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant