CN107516290A - Image switching network acquisition methods, device, computing device and storage medium - Google Patents
Image switching network acquisition methods, device, computing device and storage medium Download PDFInfo
- Publication number
- CN107516290A CN107516290A CN201710574583.2A CN201710574583A CN107516290A CN 107516290 A CN107516290 A CN 107516290A CN 201710574583 A CN201710574583 A CN 201710574583A CN 107516290 A CN107516290 A CN 107516290A
- Authority
- CN
- China
- Prior art keywords
- image
- network
- style
- sample
- switching network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 112
- 238000012545 processing Methods 0.000 claims abstract description 72
- 230000004927 fusion Effects 0.000 claims abstract description 54
- 238000002156 mixing Methods 0.000 claims abstract description 33
- 238000004891 communication Methods 0.000 claims description 41
- 238000013507 mapping Methods 0.000 claims description 12
- 238000005267 amalgamation Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 description 61
- 230000006870 function Effects 0.000 description 33
- 230000008569 process Effects 0.000 description 24
- 238000012546 transfer Methods 0.000 description 18
- 238000000605 extraction Methods 0.000 description 12
- 230000000644 propagated effect Effects 0.000 description 11
- 230000000007 visual effect Effects 0.000 description 11
- 239000000284 extract Substances 0.000 description 9
- 230000007935 neutral effect Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 238000006243 chemical reaction Methods 0.000 description 5
- 230000008447 perception Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 239000013256 coordination polymer Substances 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 235000012364 Peperomia pellucida Nutrition 0.000 description 1
- 240000007711 Peperomia pellucida Species 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001035 drying Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of image switching network acquisition methods, device, computing device and computer-readable storage medium, wherein, image switching network acquisition methods are based on trained first network and performed, and this method includes:Obtain the first image and the second image;First image and the second image are separately input into first network, and according to default blending weight, is weighted in the ranking operation layer of first network, obtains the second network corresponding with the style after the fusion of the first image and the second image.Technical scheme provided by the invention can be quickly obtained image switching network corresponding with the style after the fusion of two style images using trained first network, the efficiency for obtaining image switching network is effectively improved, optimizes image switching network processing mode.
Description
Technical field
The present invention relates to technical field of image processing, and in particular to a kind of image switching network acquisition methods, device, calculating
Equipment and computer-readable storage medium.
Background technology
Using image stylization treatment technology, the style in style image can be transferred on the image of daily shooting,
Image is enabled to obtain more preferable visual effect.In the prior art, it is to be directly inputted into a given style image
In one neutral net (neural network), followed by substantial amounts of content images as sample image, by repeatedly changing
Generation training obtains image switching network corresponding with given style image, is then realized using the image switching network in input
Hold the style conversion of image.This will may require that the very long training time, cause the efficiency for obtaining image switching network low, separately
Outside, it also is difficult to obtain image switching network corresponding with the style after the fusion of two style images using prior art.
The content of the invention
In view of the above problems, it is proposed that the present invention so as to provide one kind overcome above mentioned problem or at least in part solve on
State image switching network acquisition methods, device, computing device and the computer-readable storage medium of problem.
According to an aspect of the invention, there is provided a kind of image switching network acquisition methods, this method is based on by instructing
Experienced first network and perform, this method includes:
Obtain the first image and the second image;
First image and the second image are separately input into first network, and according to default blending weight, in the first net
The ranking operation layer of network is weighted, and obtains the second net corresponding with the style after the fusion of the first image and the second image
Network.
Further, the first image and the second image are separately input into first network, and according to default blending weight,
It is weighted, obtains corresponding with the style after the fusion of the first image and the second image in the ranking operation layer of first network
The second network further comprise:
First image is inputted into first network, a propagated forward computing is carried out in first network, determines first
Ranking operation layer data corresponding to image;
Second image is inputted into first network, a propagated forward computing is carried out in first network, determines second
Ranking operation layer data corresponding to image;
According to default blending weight, to the ranking operation number of plies corresponding to the first image in the ranking operation layer of first network
According to the second image corresponding to ranking operation layer data be weighted, obtain with after the merging of the first image and the second image
Style corresponding to the second network.
Further, ranking operation layer is bottleneck layer;Bottleneck layer is the vector with minimum in the convolutional layer of first network
The level of dimension.
Further, ranking operation layer data is vector data.
Further, first network training sample image used includes:Multiple first samples of style image library storage
Multiple second sample images of image and content images library storage.
Further, the training process of first network is completed by successive ignition;During an iteration, from style figure
As extracting a first sample image in storehouse, at least one second sample image is extracted from content images storehouse, utilizes one the
One sample image and at least one second sample image realize the training of first network.
Further, during successive ignition, one first sample image of fixed extraction, alternatively extract at least one
Second sample image;After the second sample image extraction in content images storehouse, next first sample image is replaced, then
Alternatively extract at least one second sample image.
Further, the training process of first network is completed by successive ignition;Wherein an iteration process includes:
Using the 3rd network corresponding with the style of first sample image, the 3rd sample corresponding with the second sample image is generated
This image;
According to the style loss between the 3rd sample image and first sample image and the 3rd sample image and the second sample
Content loss between this image, first network loss function is obtained, first network is realized using first network loss function
Training.
Further, the training step of first network includes:
A first sample image is extracted from style image storehouse, at least one second sample is extracted from content images storehouse
Image;
First sample image is inputted into first network, obtains the 3rd net corresponding with the style of first sample image
Network;
Using the 3rd network corresponding with the style of first sample image, given birth to respectively at least one second sample image
Into corresponding 3rd sample image;
According to the style loss and at least one 3rd between at least one 3rd sample image and first sample image
Content loss between sample image and corresponding second sample image, obtains first network loss function, according to first network
Loss function updates the weight parameter of first network;
Iteration performs the training step of first network, until meeting predetermined convergence condition.
Further, predetermined convergence condition includes:Iterations reaches default iterations;And/or first network loss
The output valve of function is less than predetermined threshold value;And/or the visual effect parameter of the 3rd sample image corresponding with the second sample image
Reach default visual effect parameter.
Further, first sample image is inputted into first network, obtained corresponding with the style of first sample image
The 3rd network further comprise:
Style textural characteristics are extracted from first sample image;
Style textural characteristics are inputted into first network, obtain the 3rd network corresponding with style textural characteristics.
Further, first network is the metanetwork for being trained to obtain to neutral net, and the second network is that image is changed
Network.
Further, the 3rd network is the switching network obtained in the training process of first network.
Further, this method is performed by terminal.
According to another aspect of the present invention, there is provided a kind of image stylization method for amalgamation processing, this method include:
Using the second network obtained by above-mentioned image switching network acquisition methods, sector-style is entered to the 3rd pending image
Format processing, obtain the 4th image corresponding with the 3rd image.
According to another aspect of the present invention, there is provided a kind of image stylization intensity adjustment method, this method include:
Using the second network obtained by above-mentioned image switching network acquisition methods, in the first image and the second image
Any one image carries out stylized processing, obtains corresponding 5th image.
According to another aspect of the present invention, there is provided a kind of image switching network acquisition device, the device are based on by instructing
Experienced first network and run, the device includes:
Acquisition module, suitable for obtaining the first image and the second image;
Mapping block, suitable for the first image and the second image are separately input into first network, and according to default fusion
Weights, it is weighted, obtains and the wind after the merging of the first image and the second image in the ranking operation layer of first network
Second network corresponding to lattice.
Further, mapping block is further adapted for:
First image is inputted into first network, a propagated forward computing is carried out in first network, determines first
Ranking operation layer data corresponding to image;
Second image is inputted into first network, a propagated forward computing is carried out in first network, determines second
Ranking operation layer data corresponding to image;
According to default blending weight, to the ranking operation number of plies corresponding to the first image in the ranking operation layer of first network
According to the second image corresponding to ranking operation layer data be weighted, obtain with after the merging of the first image and the second image
Style corresponding to the second network.
Further, ranking operation layer is bottleneck layer;Bottleneck layer is the vector with minimum in the convolutional layer of first network
The level of dimension.
Further, ranking operation layer data is vector data.
Further, first network training sample image used includes:Multiple first samples of style image library storage
Multiple second sample images of image and content images library storage.
Further, the device also includes:First network training module;The training process of first network passes through successive ignition
Complete;
First network training module is suitable to:During an iteration, a first sample is extracted from style image storehouse
Image, at least one second sample image is extracted from content images storehouse, utilize a first sample image and at least one
Two sample images realize the training of first network.
Further, first network training module is further adapted for:
One first sample image of fixed extraction, alternatively extracts at least one second sample image;When content images storehouse
In the second sample image extraction after, replace next first sample image, alternatively still extract at least one second sample
This image.
Further, the device also includes:First network training module;The training process of first network passes through successive ignition
Complete;
First network training module is suitable to:During an iteration, using corresponding with the style of first sample image
3rd network, generate the 3rd sample image corresponding with the second sample image;
According to the style loss between the 3rd sample image and first sample image and the 3rd sample image and the second sample
Content loss between this image, first network loss function is obtained, first network is realized using first network loss function
Training.
Further, the device also includes:First network training module;
First network training module includes:
Extraction unit, suitable for from style image storehouse extract a first sample image, from content images storehouse extraction to
Few second sample image;
Generation unit, suitable for first sample image is inputted into first network, obtain the style with first sample image
Corresponding 3rd network;
Processing unit, suitable for utilizing the 3rd network corresponding with the style of first sample image, respectively at least one
3rd sample image corresponding to the generation of second sample image;
Updating block, suitable for according between at least one 3rd sample image and first sample image style loss and
Content loss between at least one 3rd sample image and corresponding second sample image, obtains first network loss function,
The weight parameter of first network is updated according to first network loss function;
First network training module iteration is run, until meeting predetermined convergence condition.
Further, predetermined convergence condition includes:Iterations reaches default iterations;And/or first network loss
The output valve of function is less than predetermined threshold value;And/or the visual effect parameter of the 3rd sample image corresponding with the second sample image
Reach default visual effect parameter.
Further, generation unit is further adapted for:
Style textural characteristics are extracted from first sample image;
Style textural characteristics are inputted into first network, obtain the 3rd network corresponding with style textural characteristics.
Further, first network is the metanetwork for being trained to obtain to neutral net, and the second network is that image is changed
Network.
Further, the 3rd network is the switching network obtained in the training process of first network.
According to another aspect of the present invention, there is provided a kind of terminal, including above-mentioned image switching network acquisition device.
According to another aspect of the present invention, there is provided a kind of image stylization fusion treatment device, the device include:
First processing module, suitable for using the second network obtained by above-mentioned image switching network acquisition device, treating place
3rd image of reason carries out stylized processing, obtains the 4th image corresponding with the 3rd image.
According to another aspect of the present invention, there is provided a kind of image stylization intensity adjustment device, the device include:
Second processing module, suitable for the second network obtained by the above-mentioned image switching network acquisition device of utilization, to first
Any one image in image and the second image carries out stylized processing, obtains corresponding 5th image.
According to another aspect of the present invention, there is provided a kind of computing device, including:Processor, memory, communication interface and
Communication bus, processor, memory and communication interface complete mutual communication by communication bus;
Memory is used to deposit an at least executable instruction, and executable instruction makes for example above-mentioned image transition net of computing device
Operated corresponding to network acquisition methods.
According to another aspect of the present invention, there is provided a kind of computer-readable storage medium, be stored with least one in storage medium
Executable instruction, executable instruction make computing device be operated as corresponding to above-mentioned image switching network acquisition methods.
According to another aspect of the present invention, there is provided a kind of computing device, including:Processor, memory, communication interface and
Communication bus, processor, memory and communication interface complete mutual communication by communication bus;
Memory is used to deposit an at least executable instruction, and executable instruction makes for example above-mentioned image stylization of computing device
Operated corresponding to method for amalgamation processing.
According to another aspect of the present invention, there is provided a kind of computer-readable storage medium, be stored with least one in storage medium
Executable instruction, executable instruction make computing device be operated as corresponding to above-mentioned image stylization method for amalgamation processing.
According to another aspect of the invention, there is provided a kind of computing device, including:Processor, memory, communication interface and
Communication bus, processor, memory and communication interface complete mutual communication by communication bus;
Memory is used to deposit an at least executable instruction, and executable instruction makes for example above-mentioned image stylization of computing device
Operated corresponding to intensity adjustment method.
In accordance with a further aspect of the present invention, there is provided a kind of computer-readable storage medium, be stored with least one in storage medium
Executable instruction, executable instruction make computing device be operated as corresponding to above-mentioned image stylization intensity adjustment method.
According to technical scheme provided by the invention, the first image and the second image are obtained, then by the first image and second
Image is separately input into first network, and according to default blending weight, fortune is weighted in the ranking operation layer of first network
Calculate, obtain the second network corresponding with the style after the fusion of the first image and the second image.Technical scheme provided by the invention
Image corresponding with the style after the fusion of two style images can be quickly obtained using trained first network to turn
Switching network, the efficiency for obtaining image switching network is effectively improved, optimizes image switching network processing mode.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,
And can be practiced according to the content of specification, and in order to allow above and other objects of the present invention, feature and advantage can
Become apparent, below especially exemplified by the embodiment of the present invention.
Brief description of the drawings
By reading the detailed description of hereafter preferred embodiment, it is various other the advantages of and benefit it is common for this area
Technical staff will be clear understanding.Accompanying drawing is only used for showing the purpose of preferred embodiment, and is not considered as to the present invention
Limitation.And in whole accompanying drawing, identical part is denoted by the same reference numerals.In the accompanying drawings:
Fig. 1 shows the schematic flow sheet of image switching network acquisition methods according to an embodiment of the invention;
Fig. 2 shows the schematic flow sheet of network training method according to an embodiment of the invention;
Fig. 3 shows that the image switching network corresponding with the style of given style image obtained using the present invention is internal
Hold the result example set figure that image carries out stylized processing;
Fig. 4 shows the schematic flow sheet of image switching network acquisition methods in accordance with another embodiment of the present invention;
Fig. 5 a show the schematic flow sheet of image stylization method for amalgamation processing according to an embodiment of the invention;
Fig. 5 b show corresponding with the style after the fusion of the first image and the second image the obtained using the present invention
Two networks carry out the result example set figure of stylized processing to the 3rd image;
Fig. 6 a show the schematic flow sheet of image stylization intensity adjustment method according to an embodiment of the invention;
Fig. 6 b show corresponding with the style after the fusion of the first image and the second image the obtained using the present invention
Two networks carry out the result example set figure of stylized processing to the second image;
Fig. 7 shows the structured flowchart of image switching network acquisition device according to an embodiment of the invention;
Fig. 8 shows image switching network acquisition device and image stylization fusion in accordance with another embodiment of the present invention
The connection block diagram of processing unit;
Fig. 9 shows image switching network acquisition device and image stylization intensity in accordance with another embodiment of the present invention
The connection block diagram of adjusting means;
Figure 10 shows a kind of structural representation of computing device according to embodiments of the present invention.
Embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although the disclosure is shown in accompanying drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
Limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
Completely it is communicated to those skilled in the art.
Fig. 1 shows the schematic flow sheet of image switching network acquisition methods according to an embodiment of the invention, the party
Method is performed by terminal, and this method is based on trained first network and performed, as shown in figure 1, this method comprises the following steps:
Step S100, obtain the first image and the second image.
Wherein, the first image and the second image are two style images for having different-style, for the ease of distinguishing, at this
This two style images are referred to as the first image and the second image in invention.Specifically, the first image and the second image can
To be the style image with any style, however it is not limited to have the style image of some specific styles.
Step S101, the first image and the second image are separately input into first network, and according to default blending weight,
It is weighted, obtains corresponding with the style after the fusion of the first image and the second image in the ranking operation layer of first network
The second network.
In the specific embodiment of the invention, first network is that obtained metanetwork (meta is trained to neutral net
Network), the second network is image switching network.Wherein, first network is trained that specifically, first network is trained
Sample image used includes:Multiple first sample images of style image library storage and multiple the second of content images library storage
Sample image.Wherein, first sample image is style sample image, and the second sample image is content sample image.By training
First network can be perfectly suitable for any style image and arbitrary content image, then the first image is inputted to the first net
After in network, it becomes possible to which rapidly mapping obtains image switching network corresponding with the style of first image, similarly, by the second figure
After inputting into first network, it also can rapidly map to obtain image transition net corresponding with the style of second image
Network, then in order to obtain image switching network corresponding with the style after the fusion of the first image and the second image, in step
Acquired the first image and the second image can be separately input into S101 in first network, and according to default blending weight,
It is weighted in the ranking operation layer of first network, then ranking operation result is substituting in first network, it becomes possible to
It is quickly obtained the second network corresponding with the style after the fusion of the first image and the second image.
Those skilled in the art can set default blending weight according to being actually needed, and not limit herein.Have in the present invention
In body embodiment, default blending weight may include to preset to preset corresponding to blending weight and the second image corresponding to the first image to melt
Close weights.It is assumed that it is 0.8 that blending weight is preset corresponding to the first image, it is 0.2 that blending weight is preset corresponding to the second image,
Style so after the fusion of the first image and the second image has the style and the second of 20% of 80% the first image for fusion
The style of image.
Specifically, the training process of first network is completed by successive ignition.Alternatively, during an iteration, from
A first sample image is extracted in style image storehouse, at least one second sample image is extracted from content images storehouse, is utilized
One first sample image and at least one second sample image realize the training of first network.
Alternatively, an iteration process includes:Using the 3rd network corresponding with the style of first sample image, generation with
3rd sample image corresponding to second sample image;According between the 3rd sample image and first sample image style lose with
And the 3rd content loss between sample image and the second sample image, first network loss function is obtained, utilizes first network
Loss function realizes the training of first network.
Wherein, the 3rd network is the switching network obtained in the training process of first network.Specifically, the second network and
3rd network is image switching network in itself, but difference between the two is:Second network is in actual applications
Obtained image switching network corresponding with the style after the fusion of two style images, and the 3rd network is in first network
The image switching network corresponding with the style of a style image obtained in training process.
The image switching network acquisition methods provided according to embodiments of the present invention, the first image and the second image are obtained, is connect
And the first image and the second image are separately input into first network, and according to default blending weight, in adding for first network
Power operation layer is weighted, and obtains the second network corresponding with the style after the fusion of the first image and the second image.This
Invent the technical scheme that provides using trained first network can be quickly obtained with after the merging of two style images
Style corresponding to image switching network, be effectively improved obtain image switching network efficiency, optimize image transition net
Network processing mode.
Fig. 2 shows the schematic flow sheet of network training method according to an embodiment of the invention, as shown in Fig. 2 the
The training step of one network comprises the following steps:
Step S200, a first sample image is extracted from style image storehouse, at least one is extracted from content images storehouse
Individual second sample image.
In specific training process, 100,000 first sample images of style image library storage, content images library storage
100000 the second sample images, wherein, first sample image is style image, and the second sample image is content images.In step
In S200, a first sample image is extracted from style image storehouse, at least one second sample is extracted from content images storehouse
Image.The quantity that those skilled in the art can set the second sample image according to being actually needed, is not limited herein.
Step S201, first sample image is inputted into first network, obtained corresponding with the style of first sample image
The 3rd network.
In one particular embodiment of the present invention, first network is that obtained metanetwork is trained to neutral net.
For example, neutral net can be VGG-16 convolutional neural networks (convolutional neural network).Specifically, in step
In rapid S201, style textural characteristics are extracted from first sample image, then input the style textural characteristics extracted to the
In one network, propagated forward (forward propagation) computing is carried out in first network, is obtained and style textural characteristics
Corresponding 3rd network.
Step S202, using the 3rd network corresponding with the style of first sample image, respectively at least one second
3rd sample image corresponding to sample image generation.
After the 3rd network corresponding with the style of first sample image has been obtained, so that it may utilize and first sample image
Style corresponding to the 3rd network, respectively at least one second sample image generation corresponding to the 3rd sample image, the 3rd
Sample image is Style Transfer image corresponding with the second sample image, and Style Transfer image has and first sample image one
The style of cause.When being extracted 8 the second sample images in step s 200, then in step S202, respectively for 8 second
3rd sample image corresponding to sample image generation, i.e., generate the 3rd sample corresponding to one for each second sample image
This image.
Step S203, according to the style loss between at least one 3rd sample image and first sample image and at least
Content loss between one the 3rd sample image and corresponding second sample image, obtains first network loss function, according to
First network loss function updates the weight parameter of first network.
Wherein, the particular content that those skilled in the art can set first network loss function according to being actually needed, herein
Do not limit.In a specific embodiment, first network loss function can be:
Wherein, IcFor the second sample image, IsFor first sample image, I is the 3rd sample image, and CP is in perceiving
The other perception function of tolerance, SP are the perception function for perceiving style difference,For the 3rd sample graph
Picture and the content loss between corresponding second sample image,For the 3rd sample image and first sample
Between image style loss, θ be first network weight parameter, λcWeight, λ are lost for preset contentsLost for default style
Weight.According to above-mentioned first network loss function, backpropagation (back propagation) computing is carried out, passes through operation result
Update the weight parameter θ of first network.
In a specific training process, first network is that obtained metanetwork is trained to neutral net, the 3rd
Network is the switching network obtained in the training process of first network.Utilize stochastic gradient descent (stochastic
Gradient descent) Algorithm for Training first network.Specifically training process includes:
1. the iterations k and the second sample image I of a first sample image are setcNumber m.For example, it will can change
Generation number k is arranged to 20, by the second sample image IcNumber m be arranged to 8, represent in the training process of metanetwork, for
One first sample image needs iteration 20 times, and each iteration needs to extract 8 the second sample image I from content images storehousec。
2. one first sample image I of fixed extraction from style image storehouses。
3. by first sample image IsInput to first network N (;In θ), first network N (;Feedovered in θ)
(feed-forward propagation) computing is propagated, is obtained and first sample image IsStyle corresponding to the 3rd network w.
Wherein, the 3rd network w and first network N (;Mapping equation θ) is:w←N(Is;θ).
4. input m the second sample image Ic.Wherein, m the second sample image IcIt can useRepresent.
5. the 3rd network w is utilized, respectively for each the second sample image Ic3rd sample image I corresponding to generation.
6. the weight parameter θ of first network is updated according to first network loss function.
Wherein, first network loss function is specially:
In first network loss function, λcWeight, λ are lost for preset contentsWeight is lost for default style.
Step S204, iteration perform the training step of first network, until meeting predetermined convergence condition.
Wherein, those skilled in the art can set predetermined convergence condition according to being actually needed, and not limit herein.For example,
Predetermined convergence condition may include:Iterations reaches default iterations;And/or the output valve of first network loss function is small
In predetermined threshold value;And/or the visual effect parameter of the 3rd sample image corresponding with the second sample image reaches default vision effect
Fruit parameter.Specifically, can be by judging whether iterations reaches default iterations to judge whether to meet predetermined convergence
Condition, whether predetermined threshold value can also be less than to judge whether to meet predetermined convergence according to the output valve of first network loss function
Condition, whether can also reach default by judging the visual effect parameter of the 3rd sample image corresponding with the second sample image
Visual effect parameter judges whether to meet predetermined convergence condition.In step S204, iteration performs the training step of first network
Suddenly, until meeting predetermined convergence condition, so as to obtain trained first network.
It is worth noting that, for the stability of first network during training for promotion, the present invention is in successive ignition process
In, one first sample image of fixed extraction, alternatively extract at least one second sample image;When in content images storehouse
After two sample images extract, next first sample image is replaced, alternatively still extracts at least one second sample image.
By way of first fixing first sample image and constantly replacing the second sample image, it can efficiently train to obtain
Suitable for the first sample image and the first network of any second sample image, next first sample image is then replaced again
And the second sample image is constantly replaced, it is applied to above-mentioned two first sample image and any second sample graph so as to train to obtain
The first network of picture.Said process is repeated up to the second sample in the first sample image and content images storehouse in style image storehouse
This image is extracted and finished, it becomes possible to which training obtains being applied to the first of any first sample image and any second sample image
Network, the first network of any style image and arbitrary content image is obtained being applied to equivalent to training, so as to effectively contract
Subtract the time needed for training first network, improve the training effectiveness of first network.
Because trained obtained first network can be perfectly suitable for any style image and arbitrary content image, that
It just can rapidly map to obtain the corresponding image switching network of style of the style image with giving using first network, and
It is not directly using neural metwork training to obtain image switching network, therefore compared with prior art, greatly improves
Obtain the speed of image switching network.In addition, using first network, according to default blending weight, transported in the weighting of first network
Calculate layer to be weighted, can also be quickly obtained image transition net corresponding with the style after the fusion of two style images
Network.
Fig. 3 shows that the image switching network corresponding with the style of given style image obtained using the present invention is internal
Hold the result example set figure that image carries out stylized processing, as shown in figure 3, the image in the first row in Fig. 3 is style image,
Image in the first row is content images, and remaining image in Fig. 3 is the Style Transfer image that is obtained after stylization processing.Its
In, the image of the second row secondary series is to utilize image switching network pair corresponding with the style of the style image of the second row first row
The content images of the first row secondary series carry out the Style Transfer image obtained by stylization processing;The tertial image of second row is
Using the corresponding image switching network of the style of the style image with the second row first row to the tertial content images of the first row
The Style Transfer image obtained by stylization processing is carried out, the rest may be inferred.As shown in figure 3, Style Transfer image has been provided with pair
The style for the style image answered.
It is provided by the invention below by comparative illustration is carried out with two kinds of image stylizations processing method of the prior art
Advantage possessed by method.Wherein, table 1 shows the ratio of this method and two kinds of images stylization processing method of the prior art
Relatively result.
Table 1
As shown in table 1, Gai Tisi et al. have submitted paper in 2015《A kind of neural algorithm of artistic style》, the paper
Proposed in method can not obtain image switching network, but any style can be applied to, need to take 9.52s can just obtain pair
The Style Transfer image answered.
Johnson et al. has delivered paper in 2016 in European Computer visual conference《Real-time style conversion and oversubscription
The perception loss of resolution》, the method proposed in the paper need to take 4h and just obtain corresponding image switching network, and can only fit
For a kind of style, but it need to only take 0.015s and obtain corresponding Style Transfer image.
And method provided by the invention can be applied not only to any style compared with above two method, and only need
Time-consuming 0.022s obtains corresponding image switching network, and need to only take 0.015s and obtain corresponding Style Transfer image, effectively
Ground improves the speed for obtaining image switching network and the efficiency for obtaining Style Transfer image.
Fig. 4 shows the schematic flow sheet of image switching network acquisition methods in accordance with another embodiment of the present invention, should
Method is performed by terminal, and this method is based on trained first network and performed, as shown in figure 4, this method includes following step
Suddenly:
Step S400, obtain the first image and the second image.
Wherein, the first image and the second image are two style images for having different-style, the first image and the second figure
As that can be the style image with any style, however it is not limited to have the style image of some specific styles.First image and
Second image can be the style image that style image in website or other users are shared, and not limit herein.
Step S401, the first image is inputted into first network, a propagated forward computing is carried out in first network,
Determine ranking operation layer data corresponding to the first image.
Because first network is trained, the first network can be perfectly suitable for any style image and any
Content images,, only need to be without being trained again for first image so after the first image is inputted into first network
A propagated forward computing is carried out in first network, it becomes possible to quickly determine adding in ranking operation layer corresponding to the first image
Weigh computing layer data.Wherein, ranking operation layer data can be vector data.
Specifically, ranking operation layer can be bottleneck layer, then ranking operation layer data is bottleneck layer data.Bottleneck layer
To have the level of minimum vector dimension in the convolutional layer of first network.The transmission function of bottleneck layer be probably it is linear,
It is probably nonlinear.Bottleneck layer plays vital effect in first network, and bottleneck layer avoids that and is easy to reality
The equal mapping relations of existing man-to-man output and input, it makes first network encode and press the style image of input
Contracting, and decoded and decompressed to produce the estimate of the style image after bottleneck layer.Therefore bottleneck layer is made an uproar with filtering
The effect of sound, the bottleneck layer data in bottleneck layer contain the essential information of the style image.So in step S401, by
One image is inputted to first network, and ranking operation layer data corresponding to identified first image contains the base of the first image
This information.
Step S402, the second image is inputted into first network, a propagated forward computing is carried out in first network,
Determine ranking operation layer data corresponding to the second image.
Because trained first network can be perfectly suitable for any style image and arbitrary content image, so
After second image is inputted into first network, without being trained again for second image, it need to only enter in first network
Propagated forward computing of row, it becomes possible to quickly determine the ranking operation number of plies in ranking operation layer corresponding to the second image
According to, wherein, the ranking operation layer data contains the essential information of the second image.
Step S403, according to default blending weight, to adding corresponding to the first image in the ranking operation layer of first network
Ranking operation layer data corresponding to power computing layer data and the second image is weighted, and obtains and the first image and the second figure
As fusion after style corresponding to the second network.
Specifically, because ranking operation layer data corresponding to the first image contains the essential information of the first image, second
Ranking operation layer data contains the essential information of the second image corresponding to image, then in step S403, is melted according to default
Weights are closed, to corresponding to ranking operation layer data corresponding to the first image and the second image in the ranking operation layer of first network
Ranking operation layer data is weighted, and then ranking operation result is substituting in first network, it becomes possible to is obtained and the
Second network corresponding to style after the fusion of one image and the second image, it is continuous so as to also demonstrate the space of network manifold
Property.
In the specific embodiment of the invention, default blending weight may include to preset blending weight and the corresponding to the first image
Blending weight is preset corresponding to two images.Specifically, can be first by corresponding to the first image in the ranking operation layer of first network
Ranking operation layer data, which is multiplied by, presets blending weight corresponding to the first image, ranking operation layer data corresponding to the second image is multiplied
To preset blending weight corresponding to the second image, then the result after multiplication is weighted again, so as to obtain and first
Second network corresponding to style after the fusion of image and the second image.It is assumed that default blending weight is corresponding to the first image
0.8, default blending weight corresponding to the second image is 0.2, then the style after the fusion of the first image and the second image is fusion
There are the style of 80% the first image and the style of 20% the second image.
The image switching network acquisition methods provided according to embodiments of the present invention, can using trained first network
Ranking operation layer data corresponding to two style images is quickly determined, according to the ranking operation number of plies corresponding to two style images
According to image switching network corresponding with the style after the fusion of two style images can be readily obtained, further increase and obtain
Obtain the efficiency of image switching network.
Present invention also offers a kind of image stylization method for amalgamation processing, this method includes:Using provided by the invention
The second network obtained by image switching network acquisition methods, stylized processing is carried out to pending the 3rd image, obtain with
4th image corresponding to 3rd image.
Fig. 5 a show the schematic flow sheet of image stylization method for amalgamation processing according to an embodiment of the invention, such as
Shown in Fig. 5 a, this method comprises the following steps:
Step S500, obtain the 3rd pending image.
When user is wanted another image procossing in addition to the first image and the second image into both having the first image
Style again with the second image style image when, the image can be obtained in step S500.In order to above first
Image and the second image make a distinction, and in the present invention think user to be processed another in addition to the first image and the second image
One image is referred to as the 3rd pending image.
Step S501, using corresponding second network of style after the fusion with the first image and the second image, treat place
3rd image of reason carries out stylized processing, obtains the 4th image corresponding with the 3rd image.
Image conversion provided by the present invention has been described in detail in above-mentioned image switching network acquisition methods embodiment
How Network Capture method obtains the second network corresponding with the style after the fusion of the first image and the second image, herein no longer
Repeat.
After the 3rd pending image is obtained, using with the style pair after the merging of the first image and the second image
The second network answered, stylized processing is carried out to the 3rd pending image, the 4th resulting image after stylization is handled
Style Transfer image as corresponding with the 3rd image, after the Style Transfer image has the fusion of the first image and the second image
Style.For example, when the second network have for fusion 80% the first image style and 20% the second image style it is right
During the image switching network answered, then the 4th image carried out using the image switching network obtained by stylization processing is had
The style of the style of 80% the first image and 20% the second image.
Fig. 5 b show corresponding with the style after the fusion of the first image and the second image the obtained using the present invention
Two networks carry out the result example set figure of stylized processing to the 3rd image, wherein, in the group figure shown in Fig. 5 b, at first
Image shown in the upper right corner of figure is the first image, is the second image in the image shown in the upper right corner of last figure, and group
Other images in figure are to utilize gained after the stylized processing of the second network progress obtained based on different default blending weights
Multiple 4th images arrived, multiple 4th images have the style of the first image and the style of the second image of different accountings.By
Fig. 5 b can intuitively find out, in this group of figure, the style of the first image accounts for possessed by the 4th shown from left to right image
Than gradually decreasing, and the accounting of the style of its second image gradually increases.
The image stylization method for amalgamation processing provided according to embodiments of the present invention, realizes the fusion to image style,
Stylization easily and quickly can be carried out to image using the corresponding image switching network of style after the fusion with two images
Processing, the Style Transfer image with the style after fusion is obtained, improve the efficiency of image stylization processing, optimize image
Stylized processing mode.
Present invention also offers a kind of image stylization intensity adjustment method, this method includes:Using provided by the invention
The second network obtained by image switching network acquisition methods, sector-style is entered to any one image in the first image and the second image
Format processing, obtain corresponding to the 5th image.
Fig. 6 a show the schematic flow sheet of image stylization intensity adjustment method according to an embodiment of the invention, such as
Shown in Fig. 6 a, this method comprises the following steps:
Step S600, obtain the second image.
Assuming that user wants the second image procossing into the style with the first image and meeting some stylized intensity
During image, second image can be obtained in step S600.
Step S601, using corresponding second network of style after the fusion with the first image and the second image, to second
Image carries out stylized processing, obtains corresponding 5th image.
Image conversion provided by the present invention has been described in detail in above-mentioned image switching network acquisition methods embodiment
How Network Capture method obtains the second network corresponding with the style after the fusion of the first image and the second image, herein no longer
Repeat.
After the second image is obtained, the style corresponding second after the fusion with the first image and the second image is utilized
Network, stylized processing is carried out to the second image, after stylization is handled obtained by the 5th image be and the second image pair
The Style Transfer image answered, the Style Transfer image have had the style of the first image and have met default stylized intensity.
Wherein, the specific size of stylized intensity using image switching network acquisition methods with generating the mistake of the second network
The default blending weight being based in journey is relevant.Assuming that user want the second image procossing into the style with the first image and
When stylized intensity is 60% image, in step s 601, using above-mentioned image switching network acquisition methods in the first figure
Preset as corresponding to blending weight be 0.6 and second preset corresponding to image in the case that blending weight is 0.4 obtained by the
Two networks carry out stylized processing to the second image, and second network is the style and 40% for the first image that fusion has 60%
Image switching network corresponding to the style of second image, then the 5th image corresponding to the second resulting image has 60%
The style of the first image and the style of 40% the second image, be also equivalent to the 5th image and maintain original second image
40% style, i.e. it is 60% that the 5th image, which is provided with the style of the first image and stylized intensity, it is achieved thereby that to figure
As the regulation of stylized intensity.
Fig. 6 b show corresponding with the style after the fusion of the first image and the second image the obtained using the present invention
Two networks carry out the result example set figure of stylized processing to the second image, wherein, in the group figure shown in Fig. 6 b, at first
Image shown in the upper right corner of figure is the second image, is the first image in the image shown in the upper right corner of last figure, and group
Other images in figure are to utilize gained after the stylized processing of the second network progress obtained based on different default blending weights
Multiple 5th images arrived, multiple 5th images have different stylized intensity.Can intuitively it be found out by Fig. 6 b, in the group figure
In, the accounting of the style of the first image gradually increases possessed by the 5th shown from left to right image, i.e. its stylized intensity
Gradually increase.
The image stylization intensity adjustment method provided according to embodiments of the present invention, is realized to image stylization intensity
Regulation, can be easily and quickly to the two figures using the corresponding image switching network of style after the fusion with two images
Any one image as in carries out stylized processing, obtains the Style Transfer image with corresponding stylized intensity, improves
The efficiency of image stylization processing, optimizes image stylization processing mode.
Fig. 7 shows the structured flowchart of image switching network acquisition device according to an embodiment of the invention, the device
Run based on trained first network, as shown in fig. 7, the device includes:Acquisition module 711 and mapping block 712.
Acquisition module 711 is suitable to:Obtain the first image and the second image.
Wherein, the first image and the second image are two style images for having different-style, the first image and the second figure
As that can be the style image with any style, however it is not limited to have the style image of some specific styles.
Mapping block 712 is suitable to:First image and the second image are separately input into first network, and melted according to default
Close weights, be weighted in the ranking operation layer of first network, obtain with after the merging of the first image and the second image
Second network corresponding to style.
Specifically, first network training sample image used includes:Multiple first sample figures of style image library storage
Multiple second sample images of picture and content images library storage.Mapping block 712 is by the first image acquired in acquisition module 711
It is separately input into first network with the second image, and according to default blending weight, is carried out in the ranking operation layer of first network
Ranking operation, then ranking operation result is substituting in first network, it becomes possible to be quickly obtained and the first image and second
Second network corresponding to style after the fusion of image.
The image switching network acquisition device provided according to embodiments of the present invention, acquisition module obtain the first image and second
First image and the second image are separately input into first network by image, mapping block, and according to default blending weight,
The ranking operation layer of one network is weighted, and obtains corresponding with the style after the fusion of the first image and the second image
Two networks.Technical scheme provided by the invention can be quickly obtained and two style images using trained first network
Fusion after style corresponding to image switching network, be effectively improved obtain image switching network efficiency, optimize figure
As switching network processing mode.
Fig. 8 shows image switching network acquisition device and image stylization fusion in accordance with another embodiment of the present invention
The connection block diagram of processing unit, wherein, image switching network acquisition device is based on trained first network and run,
As shown in figure 8, image switching network acquisition device 810 includes:Acquisition module 811, first network training module 812 and mapping mould
Block 813, image stylization fusion treatment device 820 include:First processing module 821.
Acquisition module 811 in image switching network acquisition device 810 is suitable to:Obtain the first image and the second image.
First network training module 812 is suitable to:First network is trained.
Wherein, the training process of first network is completed by successive ignition.First network training module 812 is suitable to:One
In secondary iterative process, a first sample image is extracted from style image storehouse, at least one the is extracted from content images storehouse
Two sample images, the training of first network is realized using a first sample image and at least one second sample image.
Alternatively, first network training module 812 is suitable to:During an iteration, using with first sample image
3rd network corresponding to style, generate the 3rd sample image corresponding with the second sample image;According to the 3rd sample image and
Style loss between one sample image and the content loss between the 3rd sample image and the second sample image, obtain first
Network losses function, the training of first network is realized using first network loss function.
In a specific embodiment, first network training module 812 may include:Extraction unit 8121, generation unit
8122nd, processing unit 8123 and updating block 8124.
Specifically, extraction unit 8121 is suitable to:A first sample image is extracted from style image storehouse, from content images
At least one second sample image is extracted in storehouse.
Generation unit 8122 is suitable to:First sample image is inputted into first network, obtained and first sample image
3rd network corresponding to style.
Wherein, the 3rd network is the switching network obtained in the training process of first network.Generation unit 8122 enters one
Step is suitable to:Style textural characteristics are extracted from first sample image;Style textural characteristics are inputted into first network, obtain with
3rd network corresponding to style textural characteristics.
Processing unit 8123 is suitable to:Using the 3rd network corresponding with the style of first sample image, respectively at least
3rd sample image corresponding to one the second sample image generation.
Updating block 8124 is suitable to:Lost according to the style between at least one 3rd sample image and first sample image
And the content loss between at least one 3rd sample image and corresponding second sample image, obtain first network loss letter
Number, the weight parameter of first network is updated according to first network loss function.Wherein, those skilled in the art can be according to actual need
The particular content of first network loss function is set, not limited herein.In a specific embodiment, first network loses
Function can be:
Wherein, IcFor the second sample image, IsFor first sample image, I is the 3rd sample image, and CP is in perceiving
The other perception function of tolerance, SP are the perception function for perceiving style difference,For the 3rd sample graph
Picture and the content loss between corresponding second sample image,For the 3rd sample image and first sample
Between image style loss, θ be neutral net weight parameter, λcWeight, λ are lost for preset contentsLost for default style
Weight.
The iteration of first network training module 812 is run, until meeting predetermined convergence condition.Specifically, predetermined convergence condition
Including:Iterations reaches default iterations;And/or the output valve of first network loss function is less than predetermined threshold value;With/
Or, the visual effect parameter of the 3rd sample image corresponding with the second sample image reaches default visual effect parameter.
First network training module 812 is further adapted for:One first sample image of fixed extraction, is alternatively extracted at least
One the second sample image;After the second sample image extraction in content images storehouse, next first sample figure is replaced
Picture, alternatively still extract at least one second sample image.By the above-mentioned means, it can efficiently train to obtain suitable for any
The first network of style image and arbitrary content image, so as to effectively reduce the time needed for training first network, improve
The training effectiveness of first network.
Mapping block 813 is suitable to:First image is inputted into first network, a forward direction is carried out in first network and is passed
Computing is broadcast, determines ranking operation layer data corresponding to the first image;Second image is inputted into first network, in first network
Propagated forward computing of middle progress, determines ranking operation layer data corresponding to the second image;According to default blending weight,
To ranking operation layer corresponding to ranking operation layer data corresponding to the first image and the second image in the ranking operation layer of one network
Data are weighted, and obtain the second network corresponding with the style after the fusion of the first image and the second image.
In one particular embodiment of the present invention, first network is that obtained metanetwork is trained to neutral net,
Second network is image switching network.Specifically, ranking operation layer is bottleneck layer, and bottleneck layer is to have in the convolutional layer of first network
There is the level of the vector dimension of minimum.Ranking operation layer data can be vector data.
First processing module 821 in image stylization fusion treatment device 820 is suitable to:Obtained using image switching network
The second network obtained by device 810, stylized processing is carried out to the 3rd pending image, obtained corresponding with the 3rd image
4th image.
Technical scheme provided by the invention can quickly determine two style images using trained first network
Corresponding ranking operation layer data, it can be readily obtained and two according to ranking operation layer data corresponding to two style images
Image switching network corresponding to style after the fusion of style image, further increase the efficiency for obtaining image switching network;
In addition, easily and quickly image can be carried out using the corresponding image switching network of style after the fusion with two images
Stylization processing, obtains the Style Transfer image with the style after fusion, realizes the fusion to image style, improve figure
As the efficiency of stylization processing, image stylization processing mode is optimized.
Fig. 9 shows image switching network acquisition device and image stylization intensity in accordance with another embodiment of the present invention
The connection block diagram of adjusting means, as shown in figure 9, the image switching network acquisition device in the present embodiment is shown in Fig. 8
Image switching network acquisition device 810, here is omitted.Image stylization intensity adjustment device 920 includes:Second processing
Module 921.
Second processing module 921 is suitable to:Using the second network obtained by image switching network acquisition device 810, to
Any one image in one image and the second image carries out stylized processing, obtains corresponding 5th image.
According to technical scheme provided by the invention, realize the regulation to image stylization intensity, using with two images
Fusion after style corresponding to image switching network can easily and quickly in the two images any one image carry out
Stylization processing, the Style Transfer image with corresponding stylized intensity is obtained, improves the efficiency of image stylization processing,
Optimize image stylization processing mode.
Present invention also offers a kind of terminal, the terminal includes above-mentioned image switching network acquisition device.Wherein, terminal
Can be mobile phone, PAD, computer, picture pick-up device etc..
Present invention also offers a kind of nonvolatile computer storage media, computer-readable storage medium is stored with least one can
Execute instruction, the executable instruction can perform the image switching network acquisition methods in above-mentioned any means embodiment.Wherein, count
Calculation machine storage medium can be storage card of the storage card of mobile phone, PAD storage card, the disk of computer, picture pick-up device etc..
Figure 10 shows a kind of structural representation of computing device according to embodiments of the present invention, the specific embodiment of the invention
The specific implementation to computing device does not limit.Wherein, computing device can be mobile phone, PAD, computer, picture pick-up device, server
Deng.
As shown in Figure 10, the computing device can include:Processor (processor) 1002, communication interface
(Communications Interface) 1004, memory (memory) 1006 and communication bus 1008.
Wherein:
Processor 1002, communication interface 1004 and memory 1006 complete mutual lead to by communication bus 1008
Letter.
Communication interface 1004, for being communicated with the network element of miscellaneous equipment such as client or other servers etc..
Processor 1002, for configuration processor 1010, it can specifically perform above-mentioned image switching network acquisition methods and implement
Correlation step in example.
Specifically, program 1010 can include program code, and the program code includes computer-managed instruction.
Processor 1002 is probably central processor CPU, or specific integrated circuit ASIC (Application
Specific Integrated Circuit), or it is arranged to implement the integrated electricity of one or more of the embodiment of the present invention
Road.The one or more processors that computing device includes, can be same type of processor, such as one or more CPU;Also may be used
To be different types of processor, such as one or more CPU and one or more ASIC.
Memory 1006, for depositing program 1010.Memory 1006 may include high-speed RAM memory, it is also possible to also
Including nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.
Program 1010 specifically can be used for so that processor 1002 performs the image conversion in above-mentioned any means embodiment
Network Capture method.The specific implementation of each step may refer in above-mentioned image switching network acquisition embodiment in program 1010
Corresponding description, will not be described here in corresponding steps and unit.It is apparent to those skilled in the art that to retouch
The convenience stated the equipment of foregoing description and the specific work process of module, may be referred in preceding method embodiment with succinctly
Corresponding process describes, and will not be repeated here.
Present invention also offers a kind of nonvolatile computer storage media, computer-readable storage medium is stored with least one can
Execute instruction, the executable instruction can perform the image stylization method for amalgamation processing in above-mentioned any means embodiment.Wherein,
Computer-readable storage medium can be storage card of the storage card of mobile phone, PAD storage card, the disk of computer, picture pick-up device etc..
Present invention also offers a kind of computing device, including:Processor, memory, communication interface and communication bus, processing
Device, memory and communication interface complete mutual communication by communication bus;Memory is used to deposit at least one executable finger
Order, executable instruction make to operate corresponding to the above-mentioned image stylization method for amalgamation processing of computing device.Wherein, computing device
Can be mobile phone, PAD, computer, picture pick-up device etc..The structure of computing device shown in the structural representation and Figure 10 of the computing device
Schematic diagram is identical, and here is omitted.
Present invention also offers a kind of nonvolatile computer storage media, computer-readable storage medium is stored with least one can
Execute instruction, the executable instruction can perform the image stylization intensity adjustment method in above-mentioned any means embodiment.Wherein,
Computer-readable storage medium can be storage card of the storage card of mobile phone, PAD storage card, the disk of computer, picture pick-up device etc..
Present invention also offers a kind of computing device, including:Processor, memory, communication interface and communication bus, processing
Device, memory and communication interface complete mutual communication by communication bus;Memory is used to deposit at least one executable finger
Order, executable instruction make to operate corresponding to the above-mentioned image stylization intensity adjustment method of computing device.Wherein, computing device
Can be mobile phone, PAD, computer, picture pick-up device etc..The structure of computing device shown in the structural representation and Figure 10 of the computing device
Schematic diagram is identical, and here is omitted.
Algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment provided herein.
Various general-purpose systems can also be used together with teaching based on this.As described above, required by constructing this kind of system
Structure be obvious.In addition, the present invention is not also directed to any certain programmed language.It should be understood that it can utilize various
Programming language realizes the content of invention described herein, and the description done above to language-specific is to disclose this hair
Bright preferred forms.
In the specification that this place provides, numerous specific details are set forth.It is to be appreciated, however, that the implementation of the present invention
Example can be put into practice in the case of these no details.In some instances, known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify the disclosure and help to understand one or more of each inventive aspect,
Above in the description to the exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention:I.e. required guarantor
The application claims of shield features more more than the feature being expressly recited in each claim.It is more precisely, such as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following embodiment are expressly incorporated in the embodiment, wherein each claim is in itself
Separate embodiments all as the present invention.
Those skilled in the art, which are appreciated that, to be carried out adaptively to the module in the equipment in embodiment
Change and they are arranged in one or more equipment different from the embodiment.Can be the module or list in embodiment
Member or component be combined into a module or unit or component, and can be divided into addition multiple submodule or subelement or
Sub-component.In addition at least some in such feature and/or process or unit exclude each other, it can use any
Combination is disclosed to all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so to appoint
Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification (including adjoint power
Profit requires, summary and accompanying drawing) disclosed in each feature can be by providing the alternative features of identical, equivalent or similar purpose come generation
Replace.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments
In included some features rather than further feature, but the combination of the feature of different embodiments means in of the invention
Within the scope of and form different embodiments.For example, in the following claims, embodiment claimed is appointed
One of meaning mode can use in any combination.
The all parts embodiment of the present invention can be realized with hardware, or to be run on one or more processor
Software module realize, or realized with combinations thereof.It will be understood by those of skill in the art that it can use in practice
Microprocessor or digital signal processor (DSP) are come one of some or all parts in realizing according to embodiments of the present invention
A little or repertoire.The present invention is also implemented as setting for performing some or all of method as described herein
Standby or program of device (for example, computer program and computer program product).Such program for realizing the present invention can deposit
Storage on a computer-readable medium, or can have the form of one or more signal.Such signal can be from because of spy
Download and obtain on net website, either provide on carrier signal or provided in the form of any other.
It should be noted that the present invention will be described rather than limits the invention for above-described embodiment, and ability
Field technique personnel can design alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference symbol between bracket should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not
Element or step listed in the claims.Word "a" or "an" before element does not exclude the presence of multiple such
Element.The present invention can be by means of including the hardware of some different elements and being come by means of properly programmed computer real
It is existing.In if the unit claim of equipment for drying is listed, several in these devices can be by same hardware branch
To embody.The use of word first, second, and third does not indicate that any order.These words can be explained and run after fame
Claim.
Claims (10)
1. a kind of image switching network acquisition methods, methods described is based on trained first network and performed, methods described
Including:
Obtain the first image and the second image;
Described first image and second image are separately input into the first network, and according to default blending weight,
It is weighted in the ranking operation layer of the first network, obtains merging with described first image and second image
Second network corresponding to style afterwards.
2. a kind of image stylization method for amalgamation processing, methods described include:
Using the second network obtained by the method described in claim 1, stylized processing is carried out to the 3rd pending image,
Obtain the 4th image corresponding with the 3rd image.
3. a kind of image stylization intensity adjustment method, methods described include:
Using the second network obtained by the method described in claim 1, in described first image and second image
Any one image carries out stylized processing, obtains corresponding 5th image.
4. a kind of image switching network acquisition device, described device is based on trained first network and run, described device
Including:
Acquisition module, suitable for obtaining the first image and the second image;
Mapping block, suitable for described first image and second image are separately input into the first network, and according to
Default blending weight, is weighted in the ranking operation layer of the first network, obtains and described first image and described
Second network corresponding to style after the fusion of second image.
5. a kind of terminal, including the image switching network acquisition device described in claim 4.
6. a kind of image stylization fusion treatment device, described device include:
First processing module, suitable for the second network obtained by the device described in utilization claim 4, to the 3rd pending figure
As carrying out stylized processing, the 4th image corresponding with the 3rd image is obtained.
7. a kind of image stylization intensity adjustment device, described device include:
Second processing module, suitable for using the second network obtained by the device described in claim 4, to described first image and
Any one image in second image carries out stylized processing, obtains corresponding 5th image.
8. a kind of computing device, including:Processor, memory, communication interface and communication bus, the processor, the storage
Device and the communication interface complete mutual communication by the communication bus;
The memory is used to deposit an at least executable instruction, and the executable instruction makes the computing device such as right will
Ask and operated corresponding to the image switching network acquisition methods described in 1.
9. a kind of computer-readable storage medium, an at least executable instruction, the executable instruction are stored with the storage medium
Make to operate corresponding to computing device image switching network acquisition methods as claimed in claim 1.
10. a kind of computing device, including:Processor, memory, communication interface and communication bus, the processor, the storage
Device and the communication interface complete mutual communication by the communication bus;
The memory is used to deposit an at least executable instruction, and the executable instruction makes the computing device such as right will
Ask and operated corresponding to the image stylization method for amalgamation processing described in 2.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710574583.2A CN107516290B (en) | 2017-07-14 | 2017-07-14 | Image conversion network acquisition method and device, computing equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710574583.2A CN107516290B (en) | 2017-07-14 | 2017-07-14 | Image conversion network acquisition method and device, computing equipment and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107516290A true CN107516290A (en) | 2017-12-26 |
CN107516290B CN107516290B (en) | 2021-03-19 |
Family
ID=60721877
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710574583.2A Active CN107516290B (en) | 2017-07-14 | 2017-07-14 | Image conversion network acquisition method and device, computing equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107516290B (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108733439A (en) * | 2018-03-26 | 2018-11-02 | 西安万像电子科技有限公司 | Image processing method and device |
CN109035318A (en) * | 2018-06-14 | 2018-12-18 | 西安电子科技大学 | A kind of conversion method of image style |
CN109146825A (en) * | 2018-10-12 | 2019-01-04 | 深圳美图创新科技有限公司 | Photography style conversion method, device and readable storage medium storing program for executing |
WO2019144855A1 (en) * | 2018-01-26 | 2019-08-01 | 腾讯科技(深圳)有限公司 | Image processing method, storage medium, and computer device |
CN111080527A (en) * | 2019-12-20 | 2020-04-28 | 北京金山云网络技术有限公司 | Image super-resolution method and device, electronic equipment and storage medium |
CN111784566A (en) * | 2020-07-01 | 2020-10-16 | 北京字节跳动网络技术有限公司 | Image processing method, migration model training method, device, medium and equipment |
CN113111791A (en) * | 2021-04-16 | 2021-07-13 | 深圳市格灵人工智能与机器人研究院有限公司 | Image filter conversion network training method and computer readable storage medium |
CN113850712A (en) * | 2021-09-03 | 2021-12-28 | 北京达佳互联信息技术有限公司 | Training method of image style conversion model, and image style conversion method and device |
WO2022193077A1 (en) * | 2021-03-15 | 2022-09-22 | Shanghai United Imaging Healthcare Co., Ltd. | Systems and methods for image segmentation |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102682420A (en) * | 2012-03-31 | 2012-09-19 | 北京百舜华年文化传播有限公司 | Method and device for converting real character image to cartoon-style image |
CN103198469A (en) * | 2011-10-31 | 2013-07-10 | 卡西欧计算机株式会社 | Image processing apparatus and image processing method |
CN106778928A (en) * | 2016-12-21 | 2017-05-31 | 广州华多网络科技有限公司 | Image processing method and device |
-
2017
- 2017-07-14 CN CN201710574583.2A patent/CN107516290B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103198469A (en) * | 2011-10-31 | 2013-07-10 | 卡西欧计算机株式会社 | Image processing apparatus and image processing method |
CN102682420A (en) * | 2012-03-31 | 2012-09-19 | 北京百舜华年文化传播有限公司 | Method and device for converting real character image to cartoon-style image |
CN106778928A (en) * | 2016-12-21 | 2017-05-31 | 广州华多网络科技有限公司 | Image processing method and device |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11276207B2 (en) | 2018-01-26 | 2022-03-15 | Tencent Technology (Shenzhen) Company Limited | Image processing method, storage medium, and computer device |
WO2019144855A1 (en) * | 2018-01-26 | 2019-08-01 | 腾讯科技(深圳)有限公司 | Image processing method, storage medium, and computer device |
CN108733439A (en) * | 2018-03-26 | 2018-11-02 | 西安万像电子科技有限公司 | Image processing method and device |
CN109035318A (en) * | 2018-06-14 | 2018-12-18 | 西安电子科技大学 | A kind of conversion method of image style |
CN109035318B (en) * | 2018-06-14 | 2021-11-30 | 西安电子科技大学 | Image style conversion method |
CN109146825B (en) * | 2018-10-12 | 2020-11-27 | 深圳美图创新科技有限公司 | Photography style conversion method, device and readable storage medium |
CN109146825A (en) * | 2018-10-12 | 2019-01-04 | 深圳美图创新科技有限公司 | Photography style conversion method, device and readable storage medium storing program for executing |
CN111080527A (en) * | 2019-12-20 | 2020-04-28 | 北京金山云网络技术有限公司 | Image super-resolution method and device, electronic equipment and storage medium |
CN111080527B (en) * | 2019-12-20 | 2023-12-05 | 北京金山云网络技术有限公司 | Image super-resolution method and device, electronic equipment and storage medium |
CN111784566A (en) * | 2020-07-01 | 2020-10-16 | 北京字节跳动网络技术有限公司 | Image processing method, migration model training method, device, medium and equipment |
CN111784566B (en) * | 2020-07-01 | 2022-02-08 | 北京字节跳动网络技术有限公司 | Image processing method, migration model training method, device, medium and equipment |
WO2022193077A1 (en) * | 2021-03-15 | 2022-09-22 | Shanghai United Imaging Healthcare Co., Ltd. | Systems and methods for image segmentation |
CN113111791A (en) * | 2021-04-16 | 2021-07-13 | 深圳市格灵人工智能与机器人研究院有限公司 | Image filter conversion network training method and computer readable storage medium |
CN113111791B (en) * | 2021-04-16 | 2024-04-09 | 深圳市格灵人工智能与机器人研究院有限公司 | Image filter conversion network training method and computer readable storage medium |
CN113850712A (en) * | 2021-09-03 | 2021-12-28 | 北京达佳互联信息技术有限公司 | Training method of image style conversion model, and image style conversion method and device |
Also Published As
Publication number | Publication date |
---|---|
CN107516290B (en) | 2021-03-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107516290A (en) | Image switching network acquisition methods, device, computing device and storage medium | |
CN107392842A (en) | Image stylization processing method, device, computing device and computer-readable storage medium | |
CN107277391A (en) | Image switching network processing method, server, computing device and storage medium | |
US11704547B2 (en) | Transposing neural network matrices in hardware | |
US20200334536A1 (en) | Performing kernel striding in hardware | |
EP3380992B1 (en) | Generating images using neural networks | |
CN106471526A (en) | Process image using deep neural network | |
CN107392316A (en) | Network training method, device, computing device and computer-readable storage medium | |
CN107766936A (en) | Artificial neural networks, artificial neuron and the control method of artificial neuron | |
Wang et al. | From weighted potential game to weighted harmonic game | |
CN109214543B (en) | Data processing method and device | |
CN109903100A (en) | A kind of customer churn prediction technique, device and readable storage medium storing program for executing | |
CN106373112A (en) | Image processing method, image processing device and electronic equipment | |
CN109145107A (en) | Subject distillation method, apparatus, medium and equipment based on convolutional neural networks | |
CN110009644B (en) | Method and device for segmenting line pixels of feature map | |
CN106169961A (en) | The network parameter processing method and processing device of neutral net based on artificial intelligence | |
CN109117475A (en) | A kind of method and relevant device of text rewriting | |
CN112966729B (en) | Data processing method and device, computer equipment and storage medium | |
CN114462582A (en) | Data processing method, device and equipment based on convolutional neural network model | |
CN109697511B (en) | Data reasoning method and device and computer equipment | |
CN112559864B (en) | Bilinear graph network recommendation method and system based on knowledge graph enhancement | |
CN113297310A (en) | Method for selecting block chain fragmentation verifier in Internet of things | |
CN109308194B (en) | Method and apparatus for storing data | |
CN111460108A (en) | Intelligent response method and device | |
CN108769236A (en) | Using recommendation method, electronic device and computer readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |