CN116543211A - Image attribute editing method, device, electronic equipment and storage medium - Google Patents
Image attribute editing method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116543211A CN116543211A CN202310505648.3A CN202310505648A CN116543211A CN 116543211 A CN116543211 A CN 116543211A CN 202310505648 A CN202310505648 A CN 202310505648A CN 116543211 A CN116543211 A CN 116543211A
- Authority
- CN
- China
- Prior art keywords
- vector
- editing
- image
- attribute
- edited
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000003860 storage Methods 0.000 title claims abstract description 29
- 239000013598 vector Substances 0.000 claims abstract description 252
- 230000000875 corresponding effect Effects 0.000 claims description 61
- 238000012549 training Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012706 support-vector machine Methods 0.000 claims description 11
- 238000009877 rendering Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 22
- 238000012545 processing Methods 0.000 description 22
- 230000008569 process Effects 0.000 description 13
- 238000004891 communication Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 8
- 239000011521 glass Substances 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
- G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Processing Or Creating Images (AREA)
Abstract
The disclosure relates to an image attribute editing method, an image attribute editing device, electronic equipment and a storage medium, wherein editing attributes and an image of an object to be processed including a target object are determined, and the image of the object to be processed is input into a trained encoder to obtain a corresponding hidden vector to be edited. And determining attribute editing vectors corresponding to the editing attributes in an editing vector set, wherein the editing vector set comprises at least one editing vector with corresponding attributes, and each editing vector is determined by a sample hidden vector extracted from a real image through an encoder. And determining an edited hidden vector according to the attribute editing vector and the hidden vector to be edited, and inputting the edited hidden vector into a trained generator to obtain a target editing image comprising the target object edited by the edited attribute. According to the method and the device, the editing vector set is determined based on the hidden vector of the real image, so that the editing vector of the editing attribute is consistent with the attribute space of the hidden vector, and the accuracy of editing the image attribute is improved.
Description
Technical Field
The disclosure relates to the field of computer technology, and in particular, to an image attribute editing method, an image attribute editing device, electronic equipment and a storage medium.
Background
In recent years, the generation of countermeasure networks has achieved rapid development, particularly in the field of image generation, with impressive results. And generating a result output by the generator by utilizing the mutual game learning of the generator and the discriminator of the countermeasure network so as to continuously improve the generation of data which is approximately consistent with the training data distribution from the hidden vector. In the application scene of image editing, the real image can be inverted to the hidden space based on the generation of the countermeasure network, then the hidden vector is edited, and the image with the wanted editing effect can be generated by the generator. Only the accurate editing vector is found in the editing process, so that the accurate editing effect can be achieved, namely, only the attribute to be edited is changed, and other attributes are not changed.
Disclosure of Invention
In view of this, the present disclosure proposes an image property editing method, apparatus, electronic device, and storage medium, aiming at improving the accuracy of image editing with a generation countermeasure network.
According to a first aspect of the present disclosure, there is provided an image property editing method, the method comprising:
determining editing attributes and an object image to be processed, wherein the object image to be processed comprises a target object;
inputting the image of the object to be processed into a trained encoder to obtain a corresponding hidden vector to be edited;
determining a property editing vector corresponding to the editing property in an editing vector set, wherein the editing vector set comprises at least one editing vector with a corresponding property, and each editing vector is determined by a sample hidden vector extracted from a real image through an encoder;
determining an edited hidden vector according to the attribute editing vector and the hidden vector to be edited;
and inputting the edited hidden vector into a trained generator to obtain a target editing image comprising the target object edited by the editing attribute.
In one possible implementation, the method further includes:
an edit vector set is determined from the set of real images.
In one possible implementation, the determining the set of editing vectors according to the set of real images includes:
inputting each real image in the real image set into the encoder to obtain a corresponding sample hidden vector;
predicting each real image based on the object attribute multi-classifier obtained by pre-training to obtain corresponding attribute classification probability;
determining a vector attribute set according to the corresponding relation between each sample hidden vector and the attribute classification probability;
and determining an editing vector set according to the vector attribute set.
In one possible implementation, the determining the set of editing vectors according to the set of vector attributes includes:
training a support vector machine according to the vector attribute set, and determining at least one editing vector and corresponding attributes according to the support vector machine obtained by training;
and determining an edit vector set according to the edit vector and the corresponding attribute.
In a possible implementation manner, the determining an edited hidden vector according to the attribute editing vector and the hidden vector to be edited includes:
and calculating the sum of the attribute editing vector and the hidden vector to be edited to obtain an edited hidden vector.
In one possible implementation, the generator generates at least two target editing images, one of which includes a target object at the same angle as the target object in the image to be processed, and the other of which includes a target object at a different angle from the target object in the image to be processed.
According to a second aspect of the present disclosure, an image property editing apparatus, the apparatus comprising:
the information determining module is used for determining editing attributes and an object image to be processed, wherein the object image to be processed comprises a target object;
the first vector determining module is used for inputting the image of the object to be processed into a trained encoder to obtain a corresponding hidden vector to be edited;
the second vector determining module is used for determining attribute editing vectors corresponding to the editing attributes in an editing vector set, wherein the editing vector set comprises at least one editing vector with corresponding attributes, and each editing vector is determined by a sample hidden vector extracted from a real image through an encoder;
the vector editing module is used for determining an edited hidden vector according to the attribute editing vector and the hidden vector to be edited;
and the image rendering module is used for inputting the edited hidden vectors into a trained generator to obtain a target editing image comprising the target object edited by the editing attribute.
In one possible implementation, the apparatus further includes:
and the set determining module is used for determining an edit vector set according to the real image set.
In one possible implementation, the set determining module is further configured to:
inputting each real image in the real image set into the encoder to obtain a corresponding sample hidden vector;
predicting each real image based on the object attribute multi-classifier obtained by pre-training to obtain corresponding attribute classification probability;
determining a vector attribute set according to the corresponding relation between each sample hidden vector and the attribute classification probability;
and determining an editing vector set according to the vector attribute set.
In one possible implementation, the set determining module is further configured to:
training a support vector machine according to the vector attribute set, and determining at least one editing vector and corresponding attributes according to the support vector machine obtained by training;
and determining an edit vector set according to the edit vector and the corresponding attribute.
In one possible implementation, the vector editing module is further configured to:
and calculating the sum of the attribute editing vector and the hidden vector to be edited to obtain an edited hidden vector.
In one possible implementation, the generator generates at least two target editing images, one of which includes a target object at the same angle as the target object in the image to be processed, and the other of which includes a target object at a different angle from the target object in the image to be processed.
According to a third aspect of the present disclosure, there is provided an electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the instructions stored by the memory.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, implement the above-described method.
According to a fifth aspect of the present disclosure, there is provided a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, performs the above method.
In the embodiment of the disclosure, an editing attribute and an object image to be processed including a target object are determined, and the object image to be processed is input into a trained encoder to obtain a corresponding hidden vector to be edited. And determining attribute editing vectors corresponding to the editing attributes in an editing vector set, wherein the editing vector set comprises at least one editing vector with corresponding attributes, and each editing vector is determined by a sample hidden vector extracted from a real image through an encoder. And determining an edited hidden vector according to the attribute editing vector and the hidden vector to be edited, and inputting the edited hidden vector into a trained generator to obtain a target editing image comprising the target object edited by the edited attribute. According to the method and the device, the edit vector set is determined through the hidden vector of the real image, so that the edit vector of the edit attribute is consistent with the attribute space of the hidden vector, and the accuracy of editing the image attribute is improved.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 illustrates a flow chart of an image property editing method according to an embodiment of the present disclosure;
FIG. 2 shows a schematic diagram of an image property editing process, according to an embodiment of the present disclosure;
FIG. 3 illustrates a schematic diagram of an edit vector effect in accordance with an embodiment of the present disclosure;
FIG. 4 illustrates a schematic diagram of an image property editing effect, according to an embodiment of the present disclosure;
FIG. 5 illustrates a schematic diagram of an image property editing apparatus according to an embodiment of the present disclosure;
FIG. 6 shows a schematic diagram of an electronic device according to an embodiment of the disclosure;
fig. 7 shows a schematic diagram of another electronic device according to an embodiment of the disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
The image attribute editing method of the embodiment of the disclosure may be executed by an electronic device such as a terminal device or a server, that is, the terminal device or the server may be used as the first client and/or the second client. The terminal device may be any fixed or mobile terminal such as a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, etc. The server may be a single server or a server cluster composed of a plurality of servers. Any electronic device may implement the image property editing methods of embodiments of the present disclosure by way of a processor invoking computer readable instructions stored in a memory.
Fig. 1 illustrates a flowchart of an image property editing method according to an embodiment of the present disclosure. As shown in fig. 1, an image property editing method of an embodiment of the present disclosure may include the following steps S10 to S50.
Step S10, determining editing attributes and an object image to be processed.
In one possible implementation, the electronic device determines an object image to be processed that needs to be edited, and characterizes editing properties of the type of editing that needs to be performed on the object image to be processed. The object image to be processed comprises a target object, and can be acquired by acquiring an image of a user. Optionally, the object image to be processed may be determined by means of an image acquisition device connected to the electronic device for object acquisition of the target object, or directly by means of the electronic device for receiving the object image of the target object after object acquisition by other devices. The editing attribute may be any attribute that needs to perform editing processing on a target object in the image of the object to be processed, and may include wearing glasses, increasing the age, changing the expression, and the like, for example. Meanwhile, the manner of determining the editing attribute by the electronic device may also be to receive the editing attribute transmitted by other devices, or receive the editing attribute input by the user through man-machine interaction with the user.
Optionally, for determining the image of the object to be processed, the electronic device may further extract the area where the target object is located by image processing means such as object recognition and image clipping after receiving or collecting the image of the target object, so as to obtain the preprocessed image of the object to be processed. The target object may be whole or part of an animal such as a human, cat, dog, etc.
And step S20, inputting the image of the object to be processed into a trained encoder to obtain a corresponding hidden vector to be edited.
In one possible implementation manner, after determining the image of the object to be processed, the electronic device may directly input the image of the object to be processed into the trained encoder, and extract the features therein to obtain the corresponding hidden vector to be edited. The hidden vector to be edited is used for representing characteristic information of a target object in the image of the object to be processed. Alternatively, the encoder may be trained on training images of two different types, a real image and a generated image, and comprising sample objects, and the training process may be trained on the intersection of the real image and the generated image. The training process can be completed through the electronic equipment or directly used by the electronic equipment after being completed through other equipment.
And step S30, determining a property editing vector corresponding to the editing property in the editing vector set.
In one possible implementation, after determining that the editing attribute of the editing type needs to be performed on the target object, the electronic device may determine an attribute editing vector corresponding to the editing attribute in the editing vector set. The editing vector set comprises at least one editing vector with corresponding attribute, and each editing vector is determined by a sample hidden vector extracted from a real image through an encoder. The edit vector set may be predetermined by the electronic device before executing the image attribute editing method, that is, the edit vector set sent by other devices is received or the electronic device is determined by the sample hidden vector corresponding to the real image. Alternatively, the set of edit vectors is also determined during execution of the image property editing method, i.e. the image property editing method further comprises the process of determining the set of edit vectors from the set of real images.
Optionally, the process of determining the edit vector set by the electronic device according to the real image set may be to input each real image in the real image set into the encoder, so as to obtain a corresponding sample hidden vector. And predicting each real image based on the object attribute multi-classifier obtained by pre-training to obtain the corresponding attribute classification probability. And then determining a vector attribute set according to the corresponding relation between each sample hidden vector and the attribute classification probability. And finally, determining an editing vector set according to the vector attribute set, wherein the process of determining the editing vector set according to the vector attribute set can be to train a support vector machine according to the vector attribute set, and determining at least one editing vector and corresponding attribute according to the support vector machine obtained by training. And determining an edit vector set according to the edit vector and the corresponding attribute.
That is, in the embodiment of the present disclosure, the electronic device may take, as an inversion manifold, a plurality of sample hidden vectors obtained by inverting each real image in the real image set by using an encoder, and then input each real image into a pre-trained object attribute multi-classifier, so as to predict the real image by using the object attribute multi-classifier, thereby obtaining attribute classification probability corresponding to each real image, that is, probability that the real image belongs to each attribute type. Further, according to the corresponding relation between the real image and the sample hidden vector in the inversion manifold and the corresponding relation between the real image and the attribute classification probability, the corresponding relation between the sample hidden vector and the attribute classification probability can be obtained, and the vector attribute set representing the corresponding relation is determined. Further, a support vector machine is trained by using the vector attribute set as a training set, and hyperplane distinguishing binary attributes are searched, wherein each attribute corresponds to a hyperplane. And finally, determining each attribute editing direction as an editing vector corresponding to the attribute of the hyperplane according to the normal vector of the hyperplane, and determining an editing vector set according to the editing vector and the corresponding attribute. In the process of editing the image in the attribute, the corresponding hidden vector to be edited of the image moves along the corresponding editing direction, so that corresponding attribute change can be realized.
And S40, determining an edited hidden vector according to the attribute editing vector and the hidden vector to be edited.
In one possible implementation manner, after determining the attribute editing vector and the hidden vector to be edited, the electronic device may determine an edited hidden vector according to the attribute editing vector and the hidden vector to be edited, where the edited hidden vector is a vector obtained by editing the hidden vector to be edited by the attribute editing vector, so that the hidden vector to be edited moves along the direction of the attribute editing vector. Alternatively, the edited hidden vector may be obtained directly by calculating the sum of the attribute editing vector and the hidden vector to be edited.
And S50, inputting the edited hidden vectors into a trained generator to obtain a target editing image comprising the target object edited by the editing attribute.
In one possible implementation manner, after the electronic device edits the hidden vector to be edited according to the attribute editing vector to obtain an edited hidden vector, inputting the edited hidden vector into a generator obtained by training, that is, generating a target editing image through the generator, wherein the target editing image comprises a target object edited by the edited attribute, and the generator can generate a generator in an impedance network. For example, in the case where the editing attribute is wearing glasses, the target object in the target editing image is in a state of wearing glasses in the case where the target object in the target image to be processed is not wearing glasses. And under the condition that the editing attribute is increasing smile and the target object expression in the image of the object to be processed is smile, the target object in the target editing image is smile expression.
Further, the target editing image generated by the generator may be at least two images in which the angles of the target objects are different, that is, edited target object images of different angles are simultaneously output. In at least two target editing images, one included target object has the same angle as the target object in the image to be processed, and the other included target objects have different angles from the target object in the image to be processed.
Fig. 2 shows a schematic diagram of an image property editing process according to an embodiment of the present disclosure. As shown in fig. 2, in the embodiment of the disclosure, the encoder extracts the sample hidden vectors of the real image to obtain an inversion manifold, and determines the edit vector set. Under the condition of editing the image of the object to be processed, extracting hidden vectors to be edited of the image of the object to be processed through an encoder, extracting attribute editing vectors from an editing vector set through editing attributes, adjusting the hidden vectors to be edited according to the attribute editing vectors to obtain corresponding edited hidden vectors, and generating at least one object editing image of the object to be edited by the editing attributes based on the edited hidden vectors through a generator obtained through training.
Fig. 3 shows a schematic diagram of an edit vector effect according to an embodiment of the present disclosure. As shown in fig. 3, in the original hidden space W composed of hidden vectors obtained based on sampling, respectively origin Determining an edit vector set and inverting manifold W composed of sample hidden vectors obtained based on real image inversion inversion In the case of determining the set of edit vectors, the diagonal points represent sampled hidden vectors in the original hidden space, the entire region represents a two-dimensional linear space, hotter clusters correspond to higher inversion manifold probability densities, the red arrows represent the edit direction from the original hidden space from glasses-free to glasses-free, and the blue arrows represent the inverse manifold-free from real imagesThe glasses are not worn to the direction of wearing the glasses. From this, it can be seen that the set of editing vectors determined in the original hidden space has a certain distortion compared with the editing vectors in the set of editing vectors determined in the inverted manifold, and can be expressed as d (Δw) =Δw inversion -Δw origin . Wherein Deltaw inversion Is the edit vector, Δw, found in the inverted manifold to indicate the edit direction origin Is the edit vector found in the original hidden space to indicate the edit direction. Although the image generated by the generator sample can be well edited using the edit vector in the original hidden space, the use to edit the real image can lead to inaccuracy. The edit vector determined using the inverted manifold of the real image determination enables more accurate editing of the real image.
Fig. 4 shows a schematic diagram of an image property editing effect according to an embodiment of the present disclosure. As shown in fig. 4, in the case where at least one editing process such as wearing glasses, adding smiles, increasing ages, and the like is performed on the subject image, it is apparent that the editing vector determined by the inversion manifold is better than the editing result obtained by editing the editing vector determined by the original hidden space.
Based on the technical characteristics, the image attribute editing method of the embodiment of the disclosure determines the editing vector set based on the hidden vector of the real image, so that the editing vector of the editing attribute is consistent with the space to which the hidden vector attribute belongs, errors caused by space difference in the image editing process are reduced, other attributes which do not need to be edited are prevented from being changed in the editing process, and the accuracy of image attribute editing is improved.
Fig. 5 shows a schematic diagram of an image property editing apparatus according to an embodiment of the present disclosure. As shown in fig. 5, an image property editing apparatus of an embodiment of the present disclosure may include:
an information determining module 50, configured to determine an editing attribute and an object image to be processed, where the object image to be processed includes a target object;
the first vector determining module 51 is configured to input the image of the object to be processed into a trained encoder to obtain a corresponding hidden vector to be edited;
a second vector determining module 52, configured to determine an attribute editing vector corresponding to the editing attribute from an editing vector set, where the editing vector set includes at least one editing vector with a corresponding attribute, and each editing vector is determined by a sample hidden vector extracted from a real image by an encoder;
a vector editing module 53, configured to determine an edited hidden vector according to the attribute editing vector and the hidden vector to be edited;
the image rendering module 54 is configured to input the edited hidden vector into a trained generator to obtain a target edited image including the target object edited by the editing attribute.
In one possible implementation, the apparatus further includes:
and the set determining module is used for determining an edit vector set according to the real image set.
In one possible implementation, the set determining module is further configured to:
inputting each real image in the real image set into the encoder to obtain a corresponding sample hidden vector;
predicting each real image based on the object attribute multi-classifier obtained by pre-training to obtain corresponding attribute classification probability;
determining a vector attribute set according to the corresponding relation between each sample hidden vector and the attribute classification probability;
and determining an editing vector set according to the vector attribute set.
In one possible implementation, the set determining module is further configured to:
training a support vector machine according to the vector attribute set, and determining at least one editing vector and corresponding attributes according to the support vector machine obtained by training;
and determining an edit vector set according to the edit vector and the corresponding attribute.
In one possible implementation, the vector editing module 53 is further configured to:
and calculating the sum of the attribute editing vector and the hidden vector to be edited to obtain an edited hidden vector.
In one possible implementation, the generator generates at least two target editing images, one of which includes a target object at the same angle as the target object in the image to be processed, and the other of which includes a target object at a different angle from the target object in the image to be processed.
In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
The disclosed embodiments also provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method. The computer readable storage medium may be a volatile or nonvolatile computer readable storage medium.
The embodiment of the disclosure also provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the instructions stored by the memory.
Embodiments of the present disclosure also provide a computer program product comprising computer readable code, or a non-transitory computer readable storage medium carrying computer readable code, which when run in a processor of an electronic device, performs the above method.
Fig. 6 shows a schematic diagram of an electronic device 800 according to an embodiment of the disclosure. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.
Referring to fig. 6, an electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.
The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.
The multimedia component 808 includes a screen between the electronic device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operational mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.
The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the electronic device 800. For example, the sensor assembly 814 may detect an on/off state of the electronic device 800, a relative positioning of the components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of a user's contact with the electronic device 800, an orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices, either wired or wireless. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi,2G, or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 804 including computer program instructions executable by processor 820 of electronic device 800 to perform the above-described methods.
Fig. 7 shows a schematic diagram of another electronic device 1900 according to an embodiment of the disclosure. For example, electronic device 1900 may be provided as a server or terminal device. Referring to FIG. 7, electronic device 1900 includes a processing component 1922 that further includes one or more processors and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by processing component 1922. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, processing component 1922 is configured to execute instructions to perform the methods described above.
The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 1932, including computer program instructions executable by processing component 1922 of electronic device 1900 to perform the methods described above.
The present disclosure may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for performing the operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (9)
1. A method of editing an image property, the method comprising:
determining editing attributes and an object image to be processed, wherein the object image to be processed comprises a target object;
inputting the image of the object to be processed into a trained encoder to obtain a corresponding hidden vector to be edited;
determining a property editing vector corresponding to the editing property in an editing vector set, wherein the editing vector set comprises at least one editing vector with a corresponding property, and each editing vector is determined by a sample hidden vector extracted from a real image through an encoder;
determining an edited hidden vector according to the attribute editing vector and the hidden vector to be edited;
and inputting the edited hidden vector into a trained generator to obtain a target editing image comprising the target object edited by the editing attribute.
2. The method according to claim 1, wherein the method further comprises:
an edit vector set is determined from the set of real images.
3. The method of claim 2, wherein the determining the set of edit vectors from the set of real images comprises:
inputting each real image in the real image set into the encoder to obtain a corresponding sample hidden vector;
predicting each real image based on the object attribute multi-classifier obtained by pre-training to obtain corresponding attribute classification probability;
determining a vector attribute set according to the corresponding relation between each sample hidden vector and the attribute classification probability;
and determining an editing vector set according to the vector attribute set.
4. A method according to claim 3, wherein said determining an edit vector set from said vector property set comprises:
training a support vector machine according to the vector attribute set, and determining at least one editing vector and corresponding attributes according to the support vector machine obtained by training;
and determining an edit vector set according to the edit vector and the corresponding attribute.
5. The method according to any one of claims 1-4, wherein said determining an edited hidden vector from said property editing vector and said hidden vector to be edited comprises:
and calculating the sum of the attribute editing vector and the hidden vector to be edited to obtain an edited hidden vector.
6. The method of any one of claims 1-5, wherein the generator generates at least two target edit images, one of which includes a target object at the same angle as the target object in the image to be processed, and the other of which includes a target object at a different angle from the target object in the image to be processed.
7. An image property editing apparatus, the apparatus comprising:
the information determining module is used for determining editing attributes and an object image to be processed, wherein the object image to be processed comprises a target object;
the first vector determining module is used for inputting the image of the object to be processed into a trained encoder to obtain a corresponding hidden vector to be edited;
the second vector determining module is used for determining attribute editing vectors corresponding to the editing attributes in an editing vector set, wherein the editing vector set comprises at least one editing vector with corresponding attributes, and each editing vector is determined by a sample hidden vector extracted from a real image through an encoder;
the vector editing module is used for determining an edited hidden vector according to the attribute editing vector and the hidden vector to be edited;
and the image rendering module is used for inputting the edited hidden vectors into a trained generator to obtain a target editing image comprising the target object edited by the editing attribute.
8. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of any one of claims 1 to 6 when executing the instructions stored by the memory.
9. A non-transitory computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 6.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2023103528352 | 2023-04-04 | ||
| CN202310352835 | 2023-04-04 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116543211A true CN116543211A (en) | 2023-08-04 |
Family
ID=87443036
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310505648.3A Pending CN116543211A (en) | 2023-04-04 | 2023-05-06 | Image attribute editing method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116543211A (en) |
-
2023
- 2023-05-06 CN CN202310505648.3A patent/CN116543211A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113538519B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN111310616B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN110889469B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN112001321B (en) | Network training method, pedestrian re-identification method, device, electronic equipment and storage medium | |
| CN111553864B (en) | Image restoration method and device, electronic equipment and storage medium | |
| CN107944447B (en) | Image classification method and device | |
| CN108038102B (en) | Method and device for recommending expression image, terminal and storage medium | |
| CN110569777B (en) | Image processing method and device, electronic device and storage medium | |
| CN111242303B (en) | Network training method and device, and image processing method and device | |
| CN110858924B (en) | Video background music generation method and device and storage medium | |
| CN110781813B (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN111259967B (en) | Image classification and neural network training method, device, equipment and storage medium | |
| CN114240882A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN111523346B (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN113065591B (en) | Target detection method and device, electronic equipment and storage medium | |
| CN110941727B (en) | Resource recommendation method and device, electronic equipment and storage medium | |
| CN112001364A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN111652107B (en) | Object counting method and device, electronic equipment and storage medium | |
| CN110633715B (en) | Image processing method, network training method and device and electronic equipment | |
| CN111523599B (en) | Target detection method and device, electronic equipment and storage medium | |
| CN109447258B (en) | Neural network model optimization method and device, electronic device and storage medium | |
| CN110955800A (en) | Video retrieval method and device | |
| CN111311588B (en) | Repositioning method and device, electronic equipment and storage medium | |
| CN110070046B (en) | Face image recognition method and device, electronic equipment and storage medium | |
| CN115512116B (en) | Image segmentation model optimization method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |