CN114897722A - Self-coding network and wavefront image restoration method based on self-coding network - Google Patents
Self-coding network and wavefront image restoration method based on self-coding network Download PDFInfo
- Publication number
- CN114897722A CN114897722A CN202210478804.7A CN202210478804A CN114897722A CN 114897722 A CN114897722 A CN 114897722A CN 202210478804 A CN202210478804 A CN 202210478804A CN 114897722 A CN114897722 A CN 114897722A
- Authority
- CN
- China
- Prior art keywords
- image
- module
- adder
- shielding
- wavefront
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000012549 training Methods 0.000 claims abstract description 37
- 238000010606 normalization Methods 0.000 claims description 66
- 230000004913 activation Effects 0.000 claims description 34
- 238000012545 processing Methods 0.000 claims description 24
- 238000000605 extraction Methods 0.000 claims description 7
- 238000000638 solvent extraction Methods 0.000 claims description 7
- 208000006440 Open Bite Diseases 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 3
- 238000005259 measurement Methods 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000009827 uniform distribution Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a self-coding network and a wavefront image restoration method based on the self-coding network, which are used for solving the technical problem that wavefront information cannot be completely reconstructed from a wavefront original image due to the fact that an image with a shielding part in the wavefront original image is lost in the measurement of the traditional method. The self-coding network comprises an encoder and a decoder, wherein the encoder is used for training non-occluded image blocks, and the decoder is used for training occluded image blocks. The restoration method provided by the invention comprises the steps of collecting a wave front original image with a shelter for multiple times by using a wave front sensor, and constructing the obtained data to form an image data set; performing network model training on the encoder and the encoder by using an image data set, and transmitting the obtained encoder network parameters to the decoder to obtain decoder network parameters; and inputting the wave front original image with the shielding object into a decoder to obtain a restored non-shielding wave front original image, and completing image restoration.
Description
Technical Field
The invention relates to the field of precision measuring instruments, in particular to a self-coding network and a wavefront image restoration method based on the self-coding network.
Background
In the measurement process of the wavefront sensor, an image with wavefront information is reconstructed from a wavefront original image, and the method is widely applied to the fields of atmospheric measurement, telescopic measurement, remote communication, plasma measurement and the like.
However, in the conventional method, the situation that the image of the shielding part is weak or missing often occurs during measurement, so that the missing part exists in the original wavefront image, and the original reconstruction method is invalid, so that the wavefront information cannot be well reconstructed from the original wavefront image, which usually limits the application range of the wavefront sensor and is a problem difficult to overcome.
Disclosure of Invention
The invention aims to solve the technical problem that the wavefront information cannot be well reconstructed from a wavefront original image due to the fact that an image with a shielding part in the wavefront original image is lost during measurement in the traditional method, and provides a self-coding network and a wavefront image restoration method based on the self-coding network.
In order to solve the technical problems, the technical solution provided by the invention is as follows:
a self-coding network comprising an encoder and a decoder, characterized in that,
the encoder comprises a first normalization module, a first multi-head attention module, a first adder, a second normalization module, a first linearization module, a first nonlinear activation function module and a second adder which are connected in sequence;
the other output end of the first adder is connected with the other input end of the second adder;
the decoder comprises a third normalization module, a second multi-head attention module, a third adder, a fourth normalization module, a second linearization module, a second nonlinear activation function module, a fourth adder, a third linearization module and a regression module which are connected in sequence;
the other output end of the third adder is connected with the other input end of the fourth adder;
the first normalization module and the third normalization module, the first multi-head attention module and the second multi-head attention module, the first adder and the third adder, the second normalization module and the fourth normalization module, the first linearization module and the second linearization module, the first nonlinear activation function module and the second nonlinear activation function module, and the second adder and the fourth adder are arranged in the same structure and share parameters.
The invention also provides a wave-front image restoration method of the self-coding network, which is characterized by comprising the following steps:
1) constructing an image dataset
Acquiring a wavefront original image with a shelter for multiple times by using a wavefront sensor, dividing each acquired image into a plurality of non-sheltered image blocks and sheltered image blocks, and constructing an image data set;
2) network model training for encoders and decoders using image datasets
Sequentially inputting the image blocks without shielding into an encoder to perform network model training to obtain encoder network parameters;
transmitting the encoder network parameters to a decoder, and using the shielded image blocks to sequentially input the shielded image blocks into the decoder to perform network model training to obtain decoder network parameters; completing the training of the self-coding network model;
3) and inputting the non-shielding image block and the shielding image block of the wavefront original image with the shielding object into a trained decoder in sequence to obtain a restored non-shielding wavefront original image, and completing image restoration.
Further, in step 1), the wavefront sensor is a shack-hartmann wavefront sensor.
Further, in step 1), the constructing the image data set comprises:
each image obtained by collection is partitioned into 16-by-16 pixels, and the partitioned parts are numbered in sequence;
and marking the non-occlusion image in each image after the partitioning as a non-occlusion image block for the training of an encoder network model, and marking the occlusion image in each image after the partitioning as an occlusion image block for the image prediction of the occlusion image block.
Further, in step 2), sequentially inputting the non-blocking image blocks into the encoder for network model training specifically comprises:
the method comprises the steps of respectively sending the non-shielding image block signals at a first position in a plurality of non-shielding image blocks into a first normalization module and a first adder, sending output signals of the input signals after normalization processing by the first normalization module into a first multi-head attention module, carrying out target region extraction processing on the input signals by the first multi-head attention module, adding the output signals of the target region extraction processing with the signals of the non-shielding image blocks at the first position in the plurality of non-shielding image blocks sent into the first adder, respectively sending the output signals of the first adder into a second normalization module and a second adder, sending the output signals of the input signals after normalization processing by the second normalization module into a first linearization module, sending the input signals after linearization processing by the first linearization module into a first nonlinear activation function module, and carrying out nonlinear transformation processing on the input signals by the first nonlinear activation function module, the output signal of the first adder is added with the signal sent by the first adder to the second adder, and the output of the second adder is the coding information of the non-shielding image block at the first position in the plurality of non-shielding image blocks; and analogizing in sequence, sequentially inputting other non-shielding image blocks into the encoder for training according to the serial number sequence, and obtaining the coding information of the corresponding non-shielding image blocks until all the non-shielding image blocks are input completely, and finishing the training of the network model of the encoder.
Further, in step 2), the method for transmitting the encoder network parameters to the decoder and using the occluded image blocks to sequentially input into the decoder for network model training specifically comprises the following steps:
respectively transmitting the network parameters of the first normalization module, the first multi-head attention module, the first adder, the second normalization module, the first linearization module, the first nonlinear activation function module and the second adder in the trained network parameters of the encoder to a third normalization module, a second multi-head attention module, a third adder, a fourth normalization module, a second linearization module, a second nonlinear activation function module and a fourth adder of a decoder;
the signal of the image block with shielding at the first position in the image block with shielding is respectively sent to a third normalization module and a third adder, the third normalization module normalizes the input signal and sends the output signal to a second multi-head attention module, the second multi-head attention module extracts the target area of the input signal, the output signal is added with the signal of the image block with shielding at the first position in the image block with shielding and sent to the third adder, the output signal of the third adder is respectively sent to a fourth normalization module and a fourth adder, the fourth normalization module normalizes the input signal and sends the normalized signal to a second linearization module, the second linearization module linearizes the input signal and sends the linearized signal to a second nonlinear activation function module, and the second nonlinear activation function module nonlinearly transforms the input signal, the output signal of the fourth adder is added with the signal sent to the fourth adder by the third adder, the output signal of the fourth adder is sent to the third linearization module, the third linearization module carries out linearization processing on the input signal and then sends the input signal to the regression module, and the regression module carries out regression prediction processing on the input signal and then outputs a predicted image of the image block with the shielding positioned at the first position in the plurality of image blocks with the shielding; and analogizing in sequence, inputting other image blocks with shielding into the decoder in sequence according to the serial number sequence to perform network model training, and obtaining corresponding predicted images of the image blocks with shielding until all the image blocks with shielding are input, and finishing the network model training of the decoder.
Further, in step 2), the first nonlinear activation function module and the second nonlinear activation function module both use a Relu function.
Further, the step 3) is specifically as follows: inputting the non-shielded image blocks and shielded image blocks of the wavefront original image with the shielding objects into a decoder of a trained self-coding network together according to the number sequence to obtain corresponding predicted image blocks; and splicing all the prediction image blocks according to the original numbering sequence to obtain a complete non-shielding wavefront restoration image, and finishing image restoration.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the self-coding network provided by the invention, the encoder and the decoder have good performance in feature extraction and expression, the algorithm precision is high, meanwhile, the efficient parallelization design can be realized, and the operation speed is greatly improved.
2. Compared with the traditional method, the wavefront image restoration method based on the self-coding network provided by the invention has the advantages that the missing part can be predicted only by depending on the characteristic information of the original image without additional prior information and additional measuring equipment, the operation is simple, and the efficiency and the accuracy of the wavefront image restoration are improved.
3. According to the wave-front image restoration method based on the self-coding network, the wave-front sensor adopts the shack-Hartmann wave-front sensor, so that the wave-front image restoration method has the advantages of high light energy utilization rate, high detection speed and stable performance, and further improves the accuracy of wave-front image restoration.
4. The invention provides a wave-front image restoration method based on a self-coding network, wherein a constructed image data set divides a plurality of collected images into blocks and numbers, marks the blocks according to non-blocked image blocks and blocked image blocks, provides a basis for network model training of an encoder and a decoder by using the image data set subsequently, can enable the decoder to better predict the blocked image blocks, and completes wave-front image restoration.
5. The invention provides a wave-front image restoration method based on a self-coding network, which uses an image data set to carry out network model training on an encoder, can encode an image block without shielding and extracts main component information.
6. The invention provides a wavefront image restoration method based on a self-coding network, which is characterized in that the network parameters of an encoder are transmitted to a decoder, an image data set is used for carrying out network model training on the decoder, image prediction can be carried out on an image block with shielding, and a whole non-shielding point row image is obtained.
Drawings
FIG. 1 is a schematic diagram of an encoder according to an embodiment of the present invention;
FIG. 2 is a block diagram of a decoder according to an embodiment of the present invention;
FIG. 3 is a flowchart of an embodiment of a wavefront image restoration method based on a self-coding network according to the present invention;
FIG. 4 is a schematic diagram of a wavefront image restoration process according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the operation of an encoder in an embodiment of the method of the present invention;
fig. 6 is a schematic diagram of the operation of a decoder in the method embodiment of the present invention.
Detailed Description
To further clarify the advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings.
A self-encoding network includes an encoder and a decoder. As shown in fig. 1, the encoder includes a first normalization module, a first multi-head attention module, a first adder, a second normalization module, a first linearization module, a first nonlinear activation function module, and a second adder, which are connected in sequence; the other output end of the first adder is connected with the other input end of the second adder. As shown in fig. 2, the decoder includes a third normalization module, a second multi-head attention module, a third adder, a fourth normalization module, a second linearization module, a second nonlinear activation function module, a fourth adder, a third linearization module, and a regression module, which are connected in sequence; the other output end of the third adder is connected with the other input end of the fourth adder. In this embodiment, the first nonlinear function module and the second nonlinear function module use Relu functions, and the regression module uses softmax functions. The first normalization module, the first multi-head attention module, the first adder, the second normalization module, the first linearization module, the first nonlinear activation function module and the second adder of the encoder are respectively in the same structure arrangement and parameter sharing with the third normalization module, the second multi-head attention module, the third adder, the fourth normalization module, the second linearization module, the second nonlinear activation function module and the fourth adder of the decoder.
As shown in fig. 3 and 4, a wavefront image restoration method based on a self-coding network specifically includes the following steps:
1) constructing an image dataset
The method comprises the steps of using a shack-Hartmann wavefront sensor to collect wavefront original image data with a shelter for multiple times, and constructing a plurality of collected image data to form an image data set. In this embodiment, a shack-hartmann wavefront sensor of french HASO is used as the shack-hartmann wavefront sensor.
The constructing of the image dataset specifically comprises: partitioning each image obtained by acquisition by 16-by-16 pixels, and numbering the partitioned parts according to the sequence of 1,2,3 and …; and marking the non-occlusion image in each image after the partitioning as a non-occlusion image block for the training of an encoder network model, and marking the occlusion image in each image after the partitioning as an occlusion image block for the image prediction of the occlusion image block.
2) Network model training for encoders and decoders using image datasets
As shown in fig. 5, the network model training for the encoder using the image dataset specifically includes: the method comprises the steps of respectively sending the non-shielding image block signals at a first position in a plurality of non-shielding image blocks into a first normalization module and a first adder, sending output signals of the input signals after normalization processing by the first normalization module into a first multi-head attention module, carrying out target region extraction processing on the input signals by the first multi-head attention module, adding the output signals of the target region extraction processing with the signals of the non-shielding image blocks at the first position in the plurality of non-shielding image blocks sent into the first adder, respectively sending the output signals of the first adder into a second normalization module and a second adder, sending the output signals of the input signals after normalization processing by the second normalization module into a first linearization module, sending the input signals after linearization processing by the first linearization module into a first nonlinear activation function module, and carrying out nonlinear transformation processing on the input signals by the first nonlinear activation function module, the output signal of the first adder is added with the signal sent by the first adder to the second adder, and the output of the second adder is the coding information of the non-shielding image block at the first position in the plurality of non-shielding image blocks; and analogizing in sequence, sequentially inputting other non-shielding image blocks into the encoder for training according to the serial number sequence to obtain the corresponding encoding information of the non-shielding image blocks, and completing the training of the network model of the encoder until all the non-shielding image blocks are input to obtain the network parameters of the encoder.
As shown in fig. 6, the network model training for the decoder using the image data set specifically includes: respectively transmitting the network parameters of the first normalization module, the first multi-head attention module, the first adder, the second normalization module, the first linearization module, the first nonlinear activation function module and the second adder in the trained network parameters of the encoder to a third normalization module, a second multi-head attention module, a third adder, a fourth normalization module, a second linearization module, a second nonlinear activation function module and a fourth adder of a decoder;
the signal of the image block with shielding at the first position in the image block with shielding is respectively sent to a third normalization module and a third adder, the third normalization module normalizes the input signal and sends the output signal to a second multi-head attention module, the second multi-head attention module extracts the target area of the input signal, the output signal is added with the signal of the image block with shielding at the first position in the image block with shielding and sent to the third adder, the output signal of the third adder is respectively sent to a fourth normalization module and a fourth adder, the fourth normalization module normalizes the input signal and sends the normalized signal to a second linearization module, the second linearization module linearizes the input signal and sends the linearized signal to a second nonlinear activation function module, and the second nonlinear activation function module nonlinearly transforms the input signal, the output signal of the fourth adder is added with the signal sent to the fourth adder by the third adder, the output signal of the fourth adder is sent to the third linearization module, the third linearization module carries out linearization processing on the input signal and then sends the input signal to the regression module, and the regression module carries out regression prediction processing on the input signal and then outputs a predicted image of the image block with the shielding positioned at the first position in the plurality of image blocks with the shielding; and analogizing in sequence, inputting other image blocks with shielding into the decoder in sequence according to the serial number sequence to perform network model training, and obtaining corresponding predicted images of the image blocks with shielding until all the image blocks with shielding are input, and finishing the network model training of the decoder to obtain the network parameters of the decoder.
During training, the image blocks with shielding and the image blocks without shielding need to be input according to the serial number sequence, and the image blocks cannot be disordered.
3) Inputting the non-shielding image blocks and the shielding image blocks in the wavefront original image with the shielding objects into a trained decoder according to the serial number sequence to obtain corresponding prediction image blocks;
and splicing all the prediction image blocks according to the original numbering sequence to obtain a complete non-shielding wavefront restoration image, and finishing image restoration.
Fig. 4 is a schematic diagram of a wavefront image restoration process in this embodiment, where the input unobstructed image blocks and obstructed image blocks in the decoder and the encoder are both schematic and do not represent the number of the unobstructed image blocks and obstructed image blocks.
The invention can not only predict and restore the wavefront image with the shelter, but also predict and restore the non-wavefront image with the shelter in any uniform distribution, wherein the uniform distribution means that the distribution of the whole area of the original image is consistent, and the situation that a part of the original image is suddenly provided with a target and a part of the original image is not provided with the target does not exist.
The above description is only for the purpose of illustrating the technical solutions of the present invention and not for the purpose of limiting the same, and it will be apparent to those skilled in the art that modifications may be made to the specific technical solutions described in the above embodiments or equivalent substitutions for some technical features, and these modifications or substitutions may not make the essence of the corresponding technical solutions depart from the scope of the technical solutions protected by the present invention.
Claims (8)
1. A self-encoding network comprising an encoder and a decoder, characterized in that:
the encoder comprises a first normalization module, a first multi-head attention module, a first adder, a second normalization module, a first linearization module, a first nonlinear activation function module and a second adder which are connected in sequence;
the other output end of the first adder is connected with the other input end of the second adder;
the decoder comprises a third normalization module, a second multi-head attention module, a third adder, a fourth normalization module, a second linearization module, a second nonlinear activation function module, a fourth adder, a third linearization module and a regression module which are connected in sequence;
the other output end of the third adder is connected with the other input end of the fourth adder;
the first normalization module and the third normalization module, the first multi-head attention module and the second multi-head attention module, the first adder and the third adder, the second normalization module and the fourth normalization module, the first linearization module and the second linearization module, the first nonlinear activation function module and the second nonlinear activation function module, and the second adder and the fourth adder are arranged in the same structure and share parameters.
2. A wavefront image restoration method based on the self-coding network of claim 1, comprising the steps of:
1) constructing an image dataset
Acquiring a wavefront original image with a shelter for multiple times by using a wavefront sensor, dividing each acquired image into a plurality of non-sheltered image blocks and sheltered image blocks, and constructing an image data set;
2) network model training for encoders and decoders using image datasets
Sequentially inputting the image blocks without shielding into an encoder to perform network model training to obtain encoder network parameters;
transmitting the encoder network parameters to a decoder, and using the shielded image blocks to sequentially input the shielded image blocks into the decoder to perform network model training to obtain decoder network parameters; completing the training of the self-coding network model;
3) and inputting the non-shielding image block and the shielding image block of the wavefront original image with the shielding object into a trained decoder in sequence to obtain a restored non-shielding wavefront original image, and completing image restoration.
3. The method for restoring a wavefront image based on a self-coding network as claimed in claim 2, wherein: in the step 1), the wavefront sensor adopts a shack-Hartmann wavefront sensor.
4. The wavefront image restoration method based on the self-coding network as claimed in claim 2 or 3, wherein: in step 1), the constructing an image data set includes:
each image obtained by collection is partitioned into 16-by-16 pixels, and the partitioned parts are numbered in sequence;
and marking the non-occlusion image in each image after the partitioning as a non-occlusion image block for the training of an encoder network model, and marking the occlusion image in each image after the partitioning as an occlusion image block for the image prediction of the occlusion image block.
5. The method for restoring a wavefront image based on a self-coding network as claimed in claim 4, wherein: in the step 2), sequentially inputting the non-shielding image blocks into the encoder to perform network model training specifically comprises the following steps:
the method comprises the steps of respectively sending the non-shielding image block signals at a first position in a plurality of non-shielding image blocks into a first normalization module and a first adder, sending output signals of the input signals after normalization processing by the first normalization module into a first multi-head attention module, carrying out target region extraction processing on the input signals by the first multi-head attention module, adding the output signals of the target region extraction processing with the signals of the non-shielding image blocks at the first position in the plurality of non-shielding image blocks sent into the first adder, respectively sending the output signals of the first adder into a second normalization module and a second adder, sending the output signals of the input signals after normalization processing by the second normalization module into a first linearization module, sending the input signals after linearization processing by the first linearization module into a first nonlinear activation function module, and carrying out nonlinear transformation processing on the input signals by the first nonlinear activation function module, the output signal of the first adder is added with the signal sent by the first adder to the second adder, and the output of the second adder is the coding information of the non-shielding image block at the first position in the plurality of non-shielding image blocks; and analogizing in sequence, sequentially inputting other non-shielding image blocks into the encoder for training according to the serial number sequence, and obtaining the coding information of the corresponding non-shielding image blocks until all the non-shielding image blocks are input completely, and finishing the training of the network model of the encoder.
6. The method for restoring a wavefront image based on a self-coding network as claimed in claim 5, wherein: in step 2), the network parameters of the encoder are transmitted to the decoder, and the network model training is carried out by inputting the shielded image blocks into the decoder in sequence, specifically:
respectively transmitting the network parameters of the first normalization module, the first multi-head attention module, the first adder, the second normalization module, the first linearization module, the first nonlinear activation function module and the second adder in the trained network parameters of the encoder to a third normalization module, a second multi-head attention module, a third adder, a fourth normalization module, a second linearization module, a second nonlinear activation function module and a fourth adder of a decoder;
the signal of the image block with shielding at the first position in the image block with shielding is respectively sent to a third normalization module and a third adder, the third normalization module normalizes the input signal and sends the output signal to a second multi-head attention module, the second multi-head attention module extracts the target area of the input signal, the output signal is added with the signal of the image block with shielding at the first position in the image block with shielding and sent to the third adder, the output signal of the third adder is respectively sent to a fourth normalization module and a fourth adder, the fourth normalization module normalizes the input signal and sends the normalized signal to a second linearization module, the second linearization module linearizes the input signal and sends the linearized signal to a second nonlinear activation function module, and the second nonlinear activation function module nonlinearly transforms the input signal, the output signal of the fourth adder is added with the signal sent to the fourth adder by the third adder, the output signal of the fourth adder is sent to the third linearization module, the third linearization module carries out linearization processing on the input signal and then sends the input signal to the regression module, and the regression module carries out regression prediction processing on the input signal and then outputs a predicted image of the image block with the shielding positioned at the first position in the plurality of image blocks with the shielding; and analogizing in sequence, inputting other image blocks with shielding into the decoder in sequence according to the serial number sequence to perform network model training, and obtaining corresponding predicted images of the image blocks with shielding until all the image blocks with shielding are input, and finishing the network model training of the decoder.
7. The method for restoring a wavefront image based on a self-coding network as claimed in claim 6, wherein: in step 2), the first nonlinear activation function module and the second nonlinear activation function module both adopt Relu functions.
8. The wavefront image restoration method based on the self-coding network as claimed in claim 7, wherein the step 3) is specifically as follows: inputting the non-shielding image block and the shielding image block of the wavefront original image with the shielding object into a decoder of a trained self-coding network according to the number sequence to obtain a corresponding prediction image block; and splicing all the prediction image blocks according to the original numbering sequence to obtain a complete non-shielding wavefront restoration image, and finishing image restoration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210478804.7A CN114897722B (en) | 2022-04-29 | 2022-04-29 | Wavefront image restoration method based on self-coding network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210478804.7A CN114897722B (en) | 2022-04-29 | 2022-04-29 | Wavefront image restoration method based on self-coding network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114897722A true CN114897722A (en) | 2022-08-12 |
| CN114897722B CN114897722B (en) | 2023-04-18 |
Family
ID=82719993
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210478804.7A Active CN114897722B (en) | 2022-04-29 | 2022-04-29 | Wavefront image restoration method based on self-coding network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114897722B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105205453A (en) * | 2015-08-28 | 2015-12-30 | 中国科学院自动化研究所 | Depth-auto-encoder-based human eye detection and positioning method |
| CN108520503A (en) * | 2018-04-13 | 2018-09-11 | 湘潭大学 | A method of based on self-encoding encoder and generating confrontation network restoration face Incomplete image |
| CN109543745A (en) * | 2018-11-20 | 2019-03-29 | 江南大学 | Feature learning method and image-recognizing method based on condition confrontation autoencoder network |
| CN110033416A (en) * | 2019-04-08 | 2019-07-19 | 重庆邮电大学 | A kind of car networking image recovery method of the more granularities of combination |
| CN113538273A (en) * | 2021-07-13 | 2021-10-22 | 荣耀终端有限公司 | Image processing method and image processing apparatus |
| CN114207673A (en) * | 2021-12-20 | 2022-03-18 | 商汤国际私人有限公司 | Sequence identification method and device, electronic equipment and storage medium |
| WO2022078041A1 (en) * | 2020-10-16 | 2022-04-21 | 上海哔哩哔哩科技有限公司 | Occlusion detection model training method and facial image beautification method |
-
2022
- 2022-04-29 CN CN202210478804.7A patent/CN114897722B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105205453A (en) * | 2015-08-28 | 2015-12-30 | 中国科学院自动化研究所 | Depth-auto-encoder-based human eye detection and positioning method |
| CN108520503A (en) * | 2018-04-13 | 2018-09-11 | 湘潭大学 | A method of based on self-encoding encoder and generating confrontation network restoration face Incomplete image |
| CN109543745A (en) * | 2018-11-20 | 2019-03-29 | 江南大学 | Feature learning method and image-recognizing method based on condition confrontation autoencoder network |
| CN110033416A (en) * | 2019-04-08 | 2019-07-19 | 重庆邮电大学 | A kind of car networking image recovery method of the more granularities of combination |
| WO2022078041A1 (en) * | 2020-10-16 | 2022-04-21 | 上海哔哩哔哩科技有限公司 | Occlusion detection model training method and facial image beautification method |
| CN113538273A (en) * | 2021-07-13 | 2021-10-22 | 荣耀终端有限公司 | Image processing method and image processing apparatus |
| CN114207673A (en) * | 2021-12-20 | 2022-03-18 | 商汤国际私人有限公司 | Sequence identification method and device, electronic equipment and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| B. POU: "Denoising wavefront sensor images with deep neural networks" * |
| 李庆武;马云鹏;周亚琴;邢俊;: "基于无监督栈式降噪自编码网络的显著性检测算法" * |
| 李英杰: "基于光场图像的前景遮挡去除方法研究" * |
| 陈琦;潘伟民;: "基于自编码器的图像去噪设计与实现" * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114897722B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108765296B (en) | Image super-resolution reconstruction method based on recursive residual attention network | |
| US10699431B2 (en) | Method and apparatus for generating image generative model | |
| CN110782395A (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN111079764A (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| CN111160298A (en) | Robot and pose estimation method and device thereof | |
| CN109447891A (en) | A kind of high quality imaging method of the spectrum imaging system based on convolutional neural networks | |
| CN110533591B (en) | Super-resolution image reconstruction method based on codec structure | |
| CN116245726A (en) | Compressed sensing polarization super-resolution imaging method based on deep learning framework | |
| CN113015022A (en) | Behavior recognition method and device, terminal equipment and computer readable storage medium | |
| Li et al. | Drcr net: Dense residual channel re-calibration network with non-local purification for spectral super resolution | |
| CN105072359A (en) | CMOS (Complementary Metal Oxide Semiconductor) image sensor imaging system and method based on compressed sensing | |
| CN112580567A (en) | Model obtaining method, model obtaining device and intelligent equipment | |
| CN114066987A (en) | Camera pose estimation method, device, equipment and storage medium | |
| CN115293986A (en) | Multi-temporal remote sensing image cloud region reconstruction method | |
| CN115601281A (en) | Remote sensing image space-time fusion method and system based on deep learning and electronic equipment | |
| CN109672885B (en) | Video image coding and decoding method for intelligent monitoring of mine | |
| CN114897722B (en) | Wavefront image restoration method based on self-coding network | |
| CN112598590B (en) | Optical remote sensing time series image reconstruction method and system based on deep learning | |
| CN111023994B (en) | Grating three-dimensional scanning method and system based on multiple measurement | |
| Zhang et al. | A PV generation data reconstruction method based on improved super-resolution generative adversarial network | |
| CN110633630B (en) | Behavior identification method and device and terminal equipment | |
| CN116740567A (en) | Soil moisture content detection method and system for Paulownia seedling cultivation | |
| CN109615584B (en) | SAR image sequence MAP super-resolution reconstruction method based on homography constraint | |
| CN116503418A (en) | Crop three-dimensional target detection method under complex scene | |
| CN110689513B (en) | Color image fusion method and device and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |