CN110033416A - A kind of car networking image recovery method of the more granularities of combination - Google Patents
A kind of car networking image recovery method of the more granularities of combination Download PDFInfo
- Publication number
- CN110033416A CN110033416A CN201910274602.9A CN201910274602A CN110033416A CN 110033416 A CN110033416 A CN 110033416A CN 201910274602 A CN201910274602 A CN 201910274602A CN 110033416 A CN110033416 A CN 110033416A
- Authority
- CN
- China
- Prior art keywords
- image
- generator
- missing
- car networking
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 235000019580 granularity Nutrition 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000006855 networking Effects 0.000 title claims abstract description 32
- 238000011084 recovery Methods 0.000 title claims abstract description 23
- 238000012549 training Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000010276 construction Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 6
- 241001269238 Data Species 0.000 claims description 3
- 238000012217 deletion Methods 0.000 claims description 3
- 230000037430 deletion Effects 0.000 claims description 3
- 238000003475 lamination Methods 0.000 claims description 3
- 241000208340 Araliaceae Species 0.000 claims 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims 1
- 235000003140 Panax quinquefolius Nutrition 0.000 claims 1
- 235000008434 ginseng Nutrition 0.000 claims 1
- 230000009977 dual effect Effects 0.000 abstract description 9
- 238000013527 convolutional neural network Methods 0.000 abstract description 7
- 230000000694 effects Effects 0.000 abstract description 7
- 230000002708 enhancing effect Effects 0.000 abstract description 3
- 238000003062 neural network model Methods 0.000 abstract description 2
- 238000005457 optimization Methods 0.000 abstract description 2
- 238000013135 deep learning Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000007796 conventional method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 235000019587 texture Nutrition 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20192—Edge enhancement; Edge preservation
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to image restoration field, the car networking image recovery method of specially a kind of more granularities of combination carries out enhancing processing to car networking image including the use of multiple dimensioned MSR algorithm, carries out pretreatment to missing image using algorithm of region growing and obtains structural information；According to missing image and its structural information, restoration disposal is carried out using the deep neural network model with coder-decoder structure；Integrality in terms of judging completion resultant content as discriminator using convolutional neural networks；Completion result clarity is judged using Pixel-CNN model as pixel discriminator；Dual training optimization is carried out to generator and two discriminators；When generator training to it is optimal when, model training terminates, by generate result and original missing image splicing be used as final restoration result.The present invention accelerates trained convergence rate, improves recovery effect, can carry out restoring and removing shelter to missing image.
Description
Technical field
The invention belongs to image restoration field more particularly to a kind of car networking image recovery methods of the more granularities of combination.
Background technique
Nowadays, automatic Pilot technology reaches its maturity, by camera in all directions acquisition vehicle periphery information, however by
The complexity and variability of natural environment, the image information that machine is got often generate missing in reality, such as strong
Loss of learning caused by sunlight reflects, image information caused by barrier lack.Since traditional restored method is in face of missing
Cannot obtain good effect when the more situation of information, and production model can be obtained when generating image it is excellent as a result,
A large amount of scholars start that production model is combined to study image restoration technology.
Currently, the method for which can be roughly divided into two types image restoration: the first is the image restoration skill based on conventional method
Art, it is main to be realized using the conventional methods such as textures synthesis or patch search；Second method is the figure based on deep learning
As recovery technique, restoration disposal is mainly carried out to missing image by deep neural network.
In current image recovery method, conventional method can sufficiently obtain the image information at absent region edge, smoothly
Ground carries out completion to missing image, but can not effectively handle the larger situation in absent region；The restored method of deep learning
It will be appreciated that image is whole semantic, restore the absent region for providing complicated semanteme, but can not smoothly with original area
In conjunction with.Therefore it needs to comprehensively consider and understands that the whole semantic marginal information with absent region of image carries out restoration disposal, to reach
Effective restoration result.If on the basis of extracting more granular informations of missing image, then passing through the method pair of deep learning
Image carries out restoration disposal, then can improve model for the understandability of image, obtain better recovery effect.
Summary of the invention
Based on problem of the existing technology, in order to improve image completion model to the understandability of missing image, promoted
The performance of model, the present invention propose a kind of car networking image recovery method of more granularities of combination, comprising:
S1, image enhancement processing is carried out to car networking image using multiple dimensioned MSR algorithm, improves the visual effect of image.
S2, broken parts in enhanced image or shelter part be labeled as to absent region, and by missing image by
RGB color is converted to the HSV space comprising tone, saturation degree and lightness characteristic；
S3, the tone granularity of missing image, saturation degree granularity and lightness granularity are pre-processed respectively, it is raw using region
Long algorithm obtains the structural information of three granularities of missing image；
S4, building have the generator of coder-decoder structure, by missing image and the structural information of three of them granularity
As the input of generator after being spliced, convolution, expansion convolution and deconvolution are carried out to input data and operated；
S5, pre-training is carried out to generator, meets the image meaning of one's words until generator generates, but be the absence of the figure of detailed information
Picture；
S6, content construction discriminator generate result to generator in terms of generating content and identify；
S7, building pixel discriminator, generate result to generator in terms of clarity and identify；
S8, generator and two discriminators are built into generation confrontation network model, and it are trained, Optimized model
Parameter, until generator generates and the consistent complete image of true picture；
S9, absent region corresponding part will be generated in image as completion as a result, splice with missing image, be combined into
One width car networking image；Splicing result is smoothed, treated, and car networking image is final completion result.
Further, image enhancement processing is carried out to car networking image using multiple dimensioned MSR algorithm, in order to guarantee sufficiently to examine
Consider the feature of high, medium and low three scales of image, choosing scale is 3, and the value of the weight w under each scale is both configured toPass through
Enhancing processing is carried out to image pixel on three scales, so as to improve the visual effect of image.
It further, is h, the missing image of w, from three tone of image, saturation degree and lightness grains for length and width dimensions
Degree carries out feature extraction respectively, obtains structural information of the original image in three granularities, facilitates the progress of subsequent completion work.
Therefore it needs the RGB color by original missing image to be converted to hsv color space, utilizes formulaBy R, G, B tri- values normalization, and calculate Δ=max (R', G', B')-min (R',
G',B').The lightness V that image is calculated according to formula V=max (R', G', B'), according to formulaCalculate the saturation degree S of image；According to formula
Calculate the tone H of image.
Further, structural information of the image in three granularities, i.e. three length and width rulers are obtained using algorithm of region growing
Very little is respectively h, the score matrix of w, and the pixel in matrix in image is divided into three classes, and absent region is similar to absent region edge
Pixel and other pixels, absent region be scored at 0, pixel similar with absent region edge is scored at 1, and residual pixel obtains
It is divided into 0.5.
Further, there is encoder-decoding using 11 convolutional layers, 4 expansion convolutional layers and 2 warp lamination constructions
The generator of device structure.Three score matrixes and original HSV image are spliced, having a size of h × w × 6 is constituted
The input as generator is measured, is exported as the HSV image of the size of h × w × 3.
It further, is evaluation index to generator using the L2 loss of great amount of images and complete image and generation image
It is trained, updates the parameter in generator, specifically trained first-loss function is as follows:
L(xl,c,xr)=| | G (xl|c)⊙xr||2
Wherein xrIt is complete image, xlIt is missing from image, c is more Granularity Structure information, and G () is that generator generates figure
Picture；, G (xl| c) indicate missing image xlThe image that generator generates when with more Granularity Structure information c as input；| | | | table
Show two norms；⊙ indicates inner product.Successive ignition training by mass data, until generator can be according to missing image and more
Granularity Structure information generates image similar in a width and complete image.
Further, discriminator, the HSV having a size of h × w × 3 are constructed using 3 convolutional layers and 2 full articulamentums
Input of the image as discriminator exports as a numerical value in section [0,1], indicate input picture in terms of content with
The consistent probability of complete image.
Further, input of the HSV image as pixel discriminator having a size of h × w × 3, passes through symmetrical fill method
By the size expansion of input picture to 95 × 95 × 3, and it is divided into h × w having a size of 32 × 32 × 3 tensor, this h ×
W tensor exports after the Pixel-CNN model with 3 convolutional layers, 2 pond layers and 2 full articulamentum structures as h
This h × w numerical value is stitched together and then by a full articulamentum, output one by the numerical value in × w section [0,1]
A numerical value in section [0,1], indicate input picture in terms of clarity with the consistent probability of complete image.
Further, it is constructed using generator, discriminator and pixel discriminator and generates confrontation network model, and defined
Loss function:
E [R]=E [log (Dc(xr)∧Dp(xr))+log(1-(Dc(G(xl|c))∧Dp(G(xl|c))))]；
Wherein E () indicates the mean value of all training datas, and L () indicates complete image and generates the L2 loss of image,
xrIt is complete image, xlIt is missing from image, c is more Granularity Structure information, and α is hyper parameter, minG、It indicates to minimize G
(), and maximize Dc() and DpThe sum of ()；Dc() is the identification result of discriminator, Dp() is that pixel identifies
The identification result of device, G () are that generator generates image, G (xl| c) indicate missing image xlWith more Granularity Structure information c conducts
The image that generator generates when input；∧ is indicated and operation.Dual training is carried out using mass data, is being schemed so that generating result
As more approaching complete image in terms of content and clarity.
Further, image will be generated to be split, isolates the corresponding deletion sites of missing image, and with original missing
Image is spliced, while being smoothed to splicing edge using mean filter, and finally formed complete image is most
Whole completion result.
Beneficial effects of the present invention:
The present invention is sufficiently extracted more granular informations of original missing image, by sufficiently obtaining image in tone, saturation
Structural information on degree and three granularities of lightness helps model to fully understand image, semantic, and existing work is few to be inputted
The acquisition of more granular informations is carried out before depth model；Two discriminators have been used when construction generates confrontation network model, point
Generation result is constrained not in terms of content and clarity two, improves the quality for generating result；Expansion in generator
It opens convolutional layer and increases the receptive field of model, sufficiently extracted the characteristic information of input data, improved the effect of recovery；?
It before dual training, first using conventional training method, restrains model as early as possible, then carries out the confrontation for generating confrontation network again
Mode is trained, and avoids the case where model collapses during dual training；By utilizing traditional images restored method and depth
Degree study combines, and improves recovery effect, can effectively answer the missing image got under automatic Pilot situation
Original simultaneously removes shelter.
Detailed description of the invention
Fig. 1 is the flow diagram that the present invention uses；
Fig. 2 is the generator structural schematic diagram that the present invention uses；
Fig. 3 is the discriminator structural schematic diagram that the present invention uses；
Fig. 4 is the Pixel-CNN structural schematic diagram that the present invention uses；
Fig. 5 is the dual training process schematic that the present invention uses.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to of the invention real
The technical solution applied in example is clearly and completely described, it is clear that described embodiment is only that present invention a part is implemented
Example, instead of all the embodiments.
Present invention is mainly used for carrying out restoration disposal to image during automatic Pilot, including completion is made because of environment or equipment
At incomplete image and the removal of object is handled, including by image damage part or shelter part labeled as lacking
Lose region, respectively from the tone of missing image, saturation degree, in three granularities of lightness using algorithm of region growing to missing image into
Row pretreatment, obtains the structural information of missing image；According to missing image and its structural information, using with encoder-decoding
The deep neural network model of device structure carries out restoration disposal；Completion knot is judged using convolutional neural networks as discriminator
Integrality in terms of fruit content；Whether completion result clarity aspect is judged using Pixel-CNN model as pixel discriminator
It is consistent with original image；A large amount of dual training Optimization restoration results are carried out to generator and two discriminators；When generator training is to most
When excellent, model training terminates, and carries out splicing by that will generate result and original missing image and is used as final restoration result.
Specifically, the present invention proposes a kind of car networking image recovery method of more granularities of combination, such as Fig. 1, comprising:
S1, image enhancement processing is carried out to car networking image using multiple dimensioned MSR algorithm, improves the visual effect of image.
S2, broken parts in enhanced image or shelter part be labeled as to absent region, and by missing image by
RGB color is converted to the HSV space comprising tone, saturation degree and lightness characteristic；
S3, the tone granularity of missing image, saturation degree granularity and lightness granularity are pre-processed respectively, it is raw using region
Long algorithm obtains the structural information of three granularities of missing image；
S4, building have the generator of coder-decoder structure, by missing image and the structural information of three of them granularity
As the input of generator after being spliced, convolution, expansion convolution and deconvolution are carried out to input data and operated；
S5, pre-training is carried out to generator, meets the image meaning of one's words until generator generates, but be the absence of the figure of detailed information
Picture；
S6, content construction discriminator generate result to generator in terms of generating content and identify；
S7, building pixel discriminator, generate result to generator in terms of clarity and identify；
S8, generator and two discriminators are built into generation confrontation network model, and it are trained, Optimized model
Parameter, until generator generates and the consistent complete image of true picture；
S9, absent region corresponding part will be generated in image as completion as a result, splice with missing image, be combined into
One width car networking image；Splicing result is smoothed, treated, and car networking image is final completion result.
Image enhancement processing is carried out to car networking image first with multiple dimensioned MSR algorithm in the present embodiment, in order to protect
Card fully considers the feature of high, medium and low three scales of image, and choosing scale is 3, and the value of the weight w under each scale is all arranged
ForBy carrying out enhancing processing to image pixel on three scales, so as to improve the visual effect of image.
In the present embodiment, by image absent region or shelter be labeled, indicate to need to restore pixel out
Position is further h, the missing image of w, from the tone of image, three granularities of saturation degree and lightness point for length and width dimensions
Not carry out feature extraction, obtain structural information of the original image in three granularities, facilitate the progress of subsequent completion work.Therefore
It needs the RGB color by original missing image to be converted to hsv color space, utilizes formulaBy R, G, B tri- values normalization, and calculate Δ=max (R', G', B')-min (R',
G',B').The lightness V that image is calculated according to formula V=max (R', G', B'), according to formulaCalculate the saturation degree S of image；According to formulaCalculate the tone H of image.
After color space conversion, the present embodiment, which obtains structure of the image in three granularities using algorithm of region growing, to be believed
Breath, i.e. three length and width dimensions are respectively h, the score matrix of w, and the pixel in matrix in image is divided into three classes, absent region, and are lacked
It loses the similar pixel of edges of regions and other pixels, absent region is scored at 0, pixel score similar with absent region edge
It is 1, residual pixel is scored at 0.5.
The present embodiment utilizes 11 convolutional layers, and 4 expansion convolutional layers and 2 warp lamination constructions have encoder-decoding
The generator of device structure, such as Fig. 2, wherein for convolution kernel having a size of 5 or 3, step value is 2.By three score matrixes and original HSV
Image is spliced, and is constituted input of the tensor having a size of h × w × 6 as generator, is exported as the size of h × w × 3
HSV image.
Training process early period of the present embodiment is lost with the L2 for generating image using great amount of images and complete image to comment
Valence index is trained generator, updates the parameter in generator, and specifically trained loss function is as follows:
L(xl,c,xr)=| | G (xl|c)⊙xr||2
Wherein L (xl,c,xr) indicate complete image and generate the first-loss of image；xrIt is to indicate complete image, collection is complete
Car networking image；xlIt is missing from image, that is, the car networking image lacked, c is more Granularity Structure information, and G () is generator
Generate image, G (xl| c) indicate missing image xlThe image that generator generates when with more Granularity Structure information c as input；||·
| | indicate two norms；⊙ indicates inner product.Successive ignition training by mass data, until generator can be according to missing image
Image similar in a width and complete image is generated with more Granularity Structure information.
Then, the present embodiment constructs discriminator, such as Fig. 3, convolution kernel ruler using 3 convolutional layers and 2 full articulamentums
Very little is 3, and step value is 2.Input of the HSV image as discriminator having a size of h × w × 3, export for section [0,
1] numerical value in, indicate input picture in terms of content with the consistent probability of complete image.
Meanwhile the present embodiment construct pixel discriminator, will the HSV image having a size of h × w × 3 as pixel discriminator
Input, by symmetrical fill method by the size expansion of input picture to 95 × 95 × 3, and be divided into h × w having a size of
32 × 32 × 3 tensor, this h × w tensor pass through with 3 convolutional layers, 2 pond layers and 2 full articulamentum structures
After Pixel-CNN model, such as Fig. 4, exports as the numerical value in h × w section [0,1], this h × w numerical value is stitched together
And then pass through a full articulamentum, a numerical value in section [0,1] is exported, indicates input picture in clarity
Aspect and the consistent probability of complete image.
The present embodiment generates confrontation network model using generator, discriminator and pixel discriminator construction, such as Fig. 5 institute
Show, and define loss function:
E [R]=E [log (Dc(xr)∧Dp(xr))+log(1-(Dc(G(xl|c))∧Dp(G(xl|c))))]；
Wherein, E [R] indicates the mean value of all training datas, L (xl,c,xr) indicate the first of complete image and generation image
Loss；xrIt is complete image, xlIt is missing from image, c is more Granularity Structure information, and α is hyper parameter, minG、It indicates
It minimizes G (), and maximizes Dc(i) and DpThe sum of ()；Dc() is the identification result of discriminator, Dp() is picture
The identification result of plain discriminator, G () are that generator generates image, G (xl| c) indicate missing image xlWith more Granularity Structure information
The image that generator generates when c is as input；∧ is indicated and operation.The dual training in later period is carried out using mass data, so that
It generates result and more approaches complete image in terms of picture material and clarity.
Last the present embodiment is split image is generated, and isolates the corresponding deletion sites of missing image, and with it is original
Missing image is spliced, while being smoothed to splicing edge using mean filter, and finally formed complete image is
For final completion result.
The few acquisitions that more granular informations are carried out before inputting depth model of current existing image recovery method；?
Construction generate confrontation network model when used two discriminators, respectively from content and clarity two in terms of to generation result into
Row constraint, improves the quality for generating result；Expansion convolutional layer in generator increases the receptive field of model, sufficiently extracts
The characteristic information of input data, improves the effect of recovery；Before dual training, first using conventional training method, make
Model is restrained as early as possible, and the confrontation mode for then carrying out generating confrontation network again is trained, and avoids mould during dual training
The case where type collapses；By combining using traditional images restored method with deep learning, recovery effect is improved, it can be effective
Ground restore and remove shelter to the missing image got under automatic Pilot situation.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of above-described embodiment is can
It is completed with instructing relevant hardware by program, which can be stored in a computer readable storage medium, storage
Medium may include: ROM, RAM, disk or CD etc..
Embodiment provided above has carried out further detailed description, institute to the object, technical solutions and advantages of the present invention
It should be understood that embodiment provided above is only the preferred embodiment of the present invention, be not intended to limit the invention, it is all
Any modification, equivalent substitution, improvement and etc. made for the present invention, should be included in the present invention within the spirit and principles in the present invention
Protection scope within.
Claims (8)
1. a kind of car networking image recovery method of the more granularities of combination, which is characterized in that the described method comprises the following steps:
S1, image enhancement processing is carried out to car networking image using multiple dimensioned MSR algorithm；
S2, broken parts in enhanced car networking image or shelter part be labeled as to absent region, and by missing image
The HSV space comprising tone, saturation degree and lightness characteristic is converted to by RGB color；
S3, the tone granularity of missing image, saturation degree granularity and lightness granularity are pre-processed respectively, is calculated using region growing
The structural information of method acquisition three granularities of missing image；
S4, building have the generator of coder-decoder structure, and the structural information of missing image and three of them granularity is carried out
As the input of generator after splicing, convolution, expansion convolution and deconvolution are carried out to input data and operated；
S5, pre-training is carried out to generator, until generator generates the image for meeting the image meaning of one's words and lacking detailed information；
S6, content construction discriminator generate result to generator in terms of generating content and identify；
S7, building pixel discriminator, generate result to generator in terms of clarity and identify；
S8, generator and two discriminators are built into generation confrontation network model, and it is trained, Optimized model ginseng
Number, until generator generates and the consistent complete image of true picture；
S9, absent region corresponding part will be generated in image as completion as a result, splice with missing image, and will be combined into a width
Car networking image；Splicing result is smoothed, treated, and car networking image is final completion result.
2. a kind of car networking image recovery method of the more granularities of combination according to claim 1, which is characterized in that the step
Rapid S3 obtains structural information of the image in three granularities including the use of algorithm of region growing, i.e., three length and width dimensions are respectively h,
The score matrix of w；Pixel in each score matrix is divided into absent region pixel, pixel similar with absent region edge with
And other pixel three classes, absent region pixel are scored at 0, pixel similar with absent region edge is scored at 1, other pixels obtain
It is divided into 0.5.
3. a kind of car networking image recovery method of the more granularities of combination according to claim 1, which is characterized in that the step
Including the use of 11 convolutional layers, 4 expansion convolutional layers and 2 warp laminations are constructed with coder-decoder structure rapid S4
Generator；The HSV image of three score matrixes and original missing image is spliced, constitutes one having a size of h × w × 6
Input of the tensor as generator exports as the HSV image of the size of h × w × 3, wherein h, w are the car networking image of missing
Length and width dimensions.
4. a kind of car networking image recovery method of the more granularities of combination according to claim 1, which is characterized in that the step
Rapid S5 is respectively trained generator with the first-loss for generating image including the use of missing image, complete image, more newborn
Parameter in growing up to be a useful person, specifically trained first-loss function is as follows:
L(xl,c,xr)=| | G (xl|c)⊙xr||2；
Wherein, L (xl,c,xr) indicate complete image and generate the first-loss of image；xrIt is to indicate complete image, collection is complete
Car networking image；xlIt is missing from image, that is, the car networking image lacked, c is more Granularity Structure information, and G () is raw for generator
At image, G (xl| c) indicate missing image xlThe image that generator generates when with more Granularity Structure information c as input；||·||
Indicate two norms；⊙ indicates inner product.
5. a kind of car networking image recovery method of the more granularities of combination according to claim 1, which is characterized in that the step
Rapid S6 constructs discriminator including the use of 3 convolutional layers and 2 full articulamentums, will be having a size of the missing image of h × w × 3
Input of the HSV image as discriminator exports as a numerical value in section [0,1], and numerical value indicates that input picture exists
Content aspect and the consistent probability of complete image.
6. a kind of car networking image recovery method of the more granularities of combination according to claim 1, which is characterized in that the step
Rapid S7 include will input of the HSV image having a size of h × w × 3 as pixel discriminator, figure will be inputted by symmetrical fill method
The size expansion of picture is divided into h × w having a size of 32 × 32 × 3 tensor, this h × w tensor to 95 × 95 × 3
After the Pixel-CNN model with 3 convolutional layers, 2 pond layers and 2 full articulamentum structures, export as h × w area
Between numerical value in [0,1], this h × w numerical value is stitched together and then by a full articulamentum, exports an area Ge
Between a numerical value in [0,1], the numerical value indicate input picture in terms of clarity with the consistent probability of complete image.
7. a kind of car networking image recovery method of the more granularities of combination according to claim 1, which is characterized in that the step
Rapid S8 is constructed including the use of generator, discriminator and pixel discriminator generates confrontation network model, and defines the second loss
Function, until the second loss function minimizes, wherein the second loss function indicates are as follows:
E [R]=E [log (Dc(xr)∧Dp(xr))+log(1-(Dc(G(xl|c))∧Dp(G(xl|c))))]；
Wherein, E [R] indicates the mean value of all training datas, L (xl,c,xr) indicate complete image and generate the first damage of image
It loses；xrIt is complete image, xlIt is missing from image, c is more Granularity Structure information, and α is hyper parameter, minG、It indicates most
Smallization G (), and maximize Dc() and DpThe sum of ()；Dc() is the identification result of discriminator, Dp() is pixel
The identification result of discriminator, G () are that generator generates image, G (xl| c) indicate missing image xlWith more Granularity Structure information c
The image that generator generates when as input；∧ is indicated and operation.
8. a kind of car networking image recovery method of the more granularities of combination according to claim 1, which is characterized in that the step
Rapid S9 includes that will generate image to be split, and isolates the corresponding deletion sites of missing image, and carry out with original missing image
Splicing, while splicing edge is smoothed using mean filter, finally formed complete image is final completion
As a result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910274602.9A CN110033416B (en) | 2019-04-08 | 2019-04-08 | Multi-granularity combined Internet of vehicles image restoration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910274602.9A CN110033416B (en) | 2019-04-08 | 2019-04-08 | Multi-granularity combined Internet of vehicles image restoration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110033416A true CN110033416A (en) | 2019-07-19 |
| CN110033416B CN110033416B (en) | 2020-11-10 |
Family
ID=67237640
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910274602.9A Active CN110033416B (en) | 2019-04-08 | 2019-04-08 | Multi-granularity combined Internet of vehicles image restoration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110033416B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110599435A (en) * | 2019-09-04 | 2019-12-20 | 精英数智科技股份有限公司 | Image synthesis method, device, equipment and storage medium |
| CN110825079A (en) * | 2019-10-15 | 2020-02-21 | 珠海格力电器股份有限公司 | Map construction method and device |
| CN111047543A (en) * | 2019-12-31 | 2020-04-21 | 腾讯科技（深圳）有限公司 | Image enhancement method, device and storage medium |
| CN111080609A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Brake shoe bolt loss detection method based on deep learning |
| CN111311507A (en) * | 2020-01-21 | 2020-06-19 | 山西大学 | Ultra-low light imaging method based on multi-granularity cooperative network |
| WO2022088089A1 (en) * | 2020-10-30 | 2022-05-05 | 京东方科技集团股份有限公司 | Image processing method, image processing apparatus, electronic device, and readable storage medium |
| CN114897722A (en) * | 2022-04-29 | 2022-08-12 | 中国科学院西安光学精密机械研究所 | Self-coding network and wavefront image restoration method based on self-coding network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281588A (en) * | 2013-07-03 | 2015-01-14 | 广州盖特软件有限公司 | Multi-granularity-based cloth image retrieval method |
| CN104598912A (en) * | 2015-01-23 | 2015-05-06 | 湖南科技大学 | Traffic light detection and recognition method based CPU and GPU cooperative computing |
| CN108573222A (en) * | 2018-03-28 | 2018-09-25 | 中山大学 | The pedestrian image occlusion detection method for generating network is fought based on cycle |
| CN109345604A (en) * | 2018-08-01 | 2019-02-15 | 深圳大学 | Image processing method, computer equipment and storage medium |
| CN109685072A (en) * | 2018-12-22 | 2019-04-26 | 北京工业大学 | A kind of compound degraded image high quality method for reconstructing based on generation confrontation network |
-
2019
- 2019-04-08 CN CN201910274602.9A patent/CN110033416B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281588A (en) * | 2013-07-03 | 2015-01-14 | 广州盖特软件有限公司 | Multi-granularity-based cloth image retrieval method |
| CN104598912A (en) * | 2015-01-23 | 2015-05-06 | 湖南科技大学 | Traffic light detection and recognition method based CPU and GPU cooperative computing |
| CN108573222A (en) * | 2018-03-28 | 2018-09-25 | 中山大学 | The pedestrian image occlusion detection method for generating network is fought based on cycle |
| CN109345604A (en) * | 2018-08-01 | 2019-02-15 | 深圳大学 | Image processing method, computer equipment and storage medium |
| CN109685072A (en) * | 2018-12-22 | 2019-04-26 | 北京工业大学 | A kind of compound degraded image high quality method for reconstructing based on generation confrontation network |
Non-Patent Citations (2)
| Title |
|---|
| UGUR DEMIR: ""Patch-Based Image Inpainting with Generative Adversarial Networks"", 《CVPR 2018》 * |
| 杜秋平: ""基于图像云模型语义标注的条件生成对抗网络"", 《模式识别与人工智能》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110599435A (en) * | 2019-09-04 | 2019-12-20 | 精英数智科技股份有限公司 | Image synthesis method, device, equipment and storage medium |
| CN110825079A (en) * | 2019-10-15 | 2020-02-21 | 珠海格力电器股份有限公司 | Map construction method and device |
| CN111080609A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Brake shoe bolt loss detection method based on deep learning |
| CN111080609B (en) * | 2019-12-12 | 2020-12-15 | 哈尔滨市科佳通用机电股份有限公司 | Brake shoe bolt loss detection method based on deep learning |
| CN111047543A (en) * | 2019-12-31 | 2020-04-21 | 腾讯科技（深圳）有限公司 | Image enhancement method, device and storage medium |
| CN111311507A (en) * | 2020-01-21 | 2020-06-19 | 山西大学 | Ultra-low light imaging method based on multi-granularity cooperative network |
| WO2022088089A1 (en) * | 2020-10-30 | 2022-05-05 | 京东方科技集团股份有限公司 | Image processing method, image processing apparatus, electronic device, and readable storage medium |
| CN114897722A (en) * | 2022-04-29 | 2022-08-12 | 中国科学院西安光学精密机械研究所 | Self-coding network and wavefront image restoration method based on self-coding network |
| CN114897722B (en) * | 2022-04-29 | 2023-04-18 | 中国科学院西安光学精密机械研究所 | Wavefront image restoration method based on self-coding network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110033416B (en) | 2020-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110033416A (en) | A kind of car networking image recovery method of the more granularities of combination | |
| CN112818862B (en) | Face tampering detection method and system based on multi-source clues and mixed attention | |
| JP2022548712A (en) | Image Haze Removal Method by Adversarial Generation Network Fusing Feature Pyramids | |
| CN109410239A (en) | A kind of text image super resolution ratio reconstruction method generating confrontation network based on condition | |
| CN110288537A (en) | Facial image complementing method based on the depth production confrontation network from attention | |
| CN107423700A (en) | The method and device of testimony verification | |
| CN111242238B (en) | RGB-D image saliency target acquisition method | |
| CN106845529A (en) | Image feature recognition methods based on many visual field convolutional neural networks | |
| CN107122796A (en) | A kind of remote sensing image sorting technique based on multiple-limb network integration model | |
| CN112967178B (en) | Image conversion method, device, equipment and storage medium | |
| CN113989129A (en) | Image restoration method based on gating and context attention mechanism | |
| CN108921942B (en) | Method and device for 2D (two-dimensional) conversion of image into 3D (three-dimensional) | |
| CN104298974A (en) | Human body behavior recognition method based on depth video sequence | |
| CN113034506B (en) | Remote sensing image semantic segmentation method and device, computer equipment and storage medium | |
| CN110895795A (en) | Improved semantic image inpainting model method | |
| CN109359527A (en) | Hair zones extracting method and system neural network based | |
| CN105894519A (en) | Robustness image segmentation algorithm based on low rank recovery | |
| CN116434012A (en) | Lightweight cotton boll detection method and system based on edge perception | |
| CN114219701A (en) | Dunhuang fresco artistic style conversion method, system, computer equipment and storage medium | |
| CN111539434B (en) | Infrared weak and small target detection method based on similarity | |
| CN117314808A (en) | Infrared and visible light image fusion method combining transducer and CNN (carbon fiber network) double encoders | |
| CN117495935A (en) | Depth completion method based on cross feature extraction window and dynamic feature fusion window | |
| CN115578624A (en) | Agricultural disease and pest model construction method, detection method and device | |
| CN110796716A (en) | Image coloring method based on multiple residual error networks and regularized transfer learning | |
| CN115880477A (en) | Apple detection positioning method and system based on deep convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |