CN113852434B - LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system - Google Patents
LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system Download PDFInfo
- Publication number
- CN113852434B CN113852434B CN202111113281.8A CN202111113281A CN113852434B CN 113852434 B CN113852434 B CN 113852434B CN 202111113281 A CN202111113281 A CN 202111113281A CN 113852434 B CN113852434 B CN 113852434B
- Authority
- CN
- China
- Prior art keywords
- layer
- output
- lstm
- block
- input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
- H04B17/391—Modelling the propagation channel
- H04B17/3912—Simulation models, e.g. distribution of spectral power density or received signal strength indicator [RSSI] for a given geographic region
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/06—Testing, supervising or monitoring using simulated traffic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Electromagnetism (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention discloses an LSTM and ResNet aided deep learning end-to-end intelligent communication method, which uses a long-term and short-term memory unit in an end-to-end intelligent communication system to perform joint coding and decoding and modulation and demodulation processing on a grouping bit sequence in an actual communication system so as to improve the error rate performance of the system; meanwhile, the invention provides an application of a residual network structure in the end-to-end intelligent communication system to effectively improve the convergence rate of the neural network and avoid the problems of gradient disappearance and gradient explosion which possibly occur.
Description
Technical Field
The invention relates to the technical field of communication, in particular to an LSTM and ResNet assisted deep learning end-to-end intelligent communication method and system.
Background
As a powerful machine learning algorithm, the deep learning technique can efficiently learn linear or nonlinear mapping required by a user, and has been widely used in various fields such as image classification, speech recognition, language translation, and the like. On the other hand, the future communication system is also proposed in the global first 6G peak held in 3 months in 2019, and needs to be enhanced and deeply fused with artificial intelligence and machine learning so as to further improve the performance of the communication system and realize intelligent communication.
Currently existing studies of deep learning in combination with communication systems are generally limited to optimizing one module in the communication system, such as demodulation, decoding, etc., using deep learning. The idea that the different modules are respectively optimized is likely to not obtain a global optimal value, so that the upper performance limit of the system can be limited. Meanwhile, the traditional optimization algorithm and the modularized optimization algorithm based on deep learning are difficult to obtain satisfactory performance under the condition that the communication environment is unknown, and the practicability of the high-speed communication system such as 5G in a complex environment is greatly limited. Finally, in recent years, the communication technology is updated rapidly, and the construction cost of the communication infrastructure and the compatibility and upgradeability of equipment are also tested, so that the advantage that the intelligent communication system can switch the neural network structure and adjust the parameters according to the requirement at low cost is more highlighted.
In order to exert the advantage of deep learning while solving the above-mentioned problems, an end-to-end intelligent communication system based on deep learning is proposed. The system regards the whole transmission system as a black box model, and realizes the joint optimization of each component part of a receiving and transmitting end through a neural network, thereby providing a globally optimal end-to-end intelligent communication system comprising the realization of coding, modulating, demodulating and decoding processes. Compared with the traditional communication system and the machine learning auxiliary communication system optimized by the split modules, the end-to-end intelligent communication system can effectively adapt to the unknown communication environment and the nonlinearity imposed by communication equipment or channels. Meanwhile, due to the use of the deep learning technology, the communication system can quickly and cheaply optimize the communication algorithm and parameters.
However, the existing end-to-end intelligent communication system still has the defects, wherein the most important is that the improvement for communication application is lacking in the structure of the used neural network, so that the system still has a large improvement space in performance.
The publication date is 2019, 11, 15 and the Chinese patent with publication number CN110460402A discloses an end-to-end communication system establishment method based on deep learning. The method of the invention is divided into two stages. Firstly, a self-encoder neural network is established, a channel layer is complicated, and a random number simulation is used as a training set to perform preliminary training on the network so as to obtain a coding mode with adaptability to channel interference. The USRP is used for collecting a large amount of communication data under the actual channels and is used as a training set to train the decoding layer independently, so that the decoding layer has better performance for the communication under the actual conditions. Also, this patent, while using deep learning, lacks improvement for communication applications in the neural network architecture used.
Disclosure of Invention
The primary purpose of the invention is to provide an LSTM and ResNet assisted deep learning end-to-end intelligent communication method, which effectively utilizes user data and improves the communication performance of the system.
It is a further object of the present invention to provide an LSTM and ResNet assisted deep learning end-to-end intelligent communication system
In order to solve the technical problems, the technical scheme of the invention is as follows:
an LSTM and ResNet assisted deep learning end-to-end intelligent communication method comprises the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data generated randomly in the step S1 is output to the noise layer after being processed by the sending block;
s4: the noise layer scrambles and denoises the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after processing opposite to the transmitting block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the data to be transmitted by the user in the step S1;
s7: setting the transmitting block after training on the communication transmitting end, taking binary data bits to be transmitted as input vectors of the transmitting block, transmitting output vectors of the transmitting block as transmitting signals into a wireless channel, setting the receiving block after training on the communication receiving end, taking the receiving block as input vectors of the receiving block after the receiving end receives signals, and obtaining estimated user transmitting data after processing.
Preferably, in step S2, a channel simulation environment is set, specifically:
when the actual channel environment is known, simulation is performed directly according to the probability distribution of the known actual channel environment;
if the actual channel environment is unknown, the simulation is performed by adaptively fitting against the generation network and learning the probability distribution of the unknown channel.
Preferably, in step S3, the transmitting block of the deep neural network includes an LSTM layer, a full connection layer, and a regularization layer, where an input of the LSTM layer is data to be transmitted by a user, an output of the LSTM layer is overlapped with an input of the LSTM layer and then used as an input of the full connection layer, to form a res nets structure, an output of the full connection is an input of the regularization layer, and an output of the regularization layer is an input of a noise layer.
Preferably, the user data to be transmitted is firstly converted into one-hot vector and then input to the LSTM layer of the transmission block.
Preferably, the LSTM layer passes through an external input vector x at time t t And an output vector h at time t-1 t-1 Calculating to obtain an output vector h at the time t t :
f t =σ(W f ·[h t-1 ,x t ]+b f )
i t =σ(W i ·[h t-1 ,x t ]+b i )
o t =σ(W o ·[h t-1 ,x t ]+b o )
h t =o t ⊙tanh(c t )
In the formula, [ h ] t-1 ,x t ]Represents x t ,h t-1 Is a concatenation vector of W f 、b f 、W i 、b i 、W o 、b o As a trainable parameter, σ (. Cndot.) represents a sigmoid activation function, tan h (. Cndot.) represents a tan h activation function, and as such, indicates multiplication of corresponding elements; c t-1 A memory cell vector at time t-1; f (f) t 、i t 、o t Respectively a forgetting gate, an input gate and an output gate vector, f t Responsible for controlling c t-1 Which elements need to be reserved and which elements need to be weakened, i t Deciding alternative update vector +.>Which elements need to be added to the cell vector at the current time, f t 、i t 、/>The three co-operate to generate a new memory cell vector c t The method comprises the steps of carrying out a first treatment on the surface of the Next, o t Determining tanh (c) t ) Which of them are output to obtain an output vector h at time t t 。
Preferably, the receiving block of the deep neural network includes two full connection layers and one LSTM unit with a res net structure, wherein:
the input of one full-connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is overlapped with the output of one full-connection layer to be used as the input of the other full-connection layer, the output of the other full-connection layer is a probability vector obtained by mapping that data to be sent by a user are firstly converted into one-hot vectors, the serial number of the largest element in the probability vector is judged as user sending information, and estimated user data are obtained according to the serial number.
Preferably, step S6 trains the deep neural network according to the difference between the estimated data obtained in step S5 and the binary data randomly generated in step S1, specifically:
calculating a loss function value between estimated user data output by the deep neural network and data to be transmitted by a user according to a preset loss function, quantitatively measuring the difference between actual output and expected output, and calculating the bias of the loss value to each trainable parameter in the deep neural network through a back propagation algorithm, so that the parameters of the deep neural network are updated by a preset optimization algorithm, and the difference between the estimated user data and the data to be transmitted by the user of the deep neural network is reduced.
Preferably, in step S7, if the actual channel changes slowly, retraining with low time cost and low computation cost is performed by a method including migration learning, and the synchronization of deep neural network training and deployment application is implemented by a means including parallel training.
An LSTM and resnet assisted deep learning end-to-end intelligent communication system, wherein the system uses the LSTM and resnet assisted deep learning end-to-end intelligent communication method, the system comprising:
the generation module is used for randomly generating binary data serving as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generating module are output to the noise layer after being processed by the sending block;
the noise module is used for scrambling and noise-adding the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sending the signal to the receiving block;
a receiving processing module, in which the receiving block obtains estimated user data after processing opposite to the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
the deployment module sets the transmitting block after training on the communication transmitting end, takes binary data bits to be transmitted as input vectors of the transmitting block, transmits output vectors of the transmitting block as transmitting signals to the wireless channel, sets the receiving block after training on the communication receiving end, and obtains estimated user transmitting data after processing after the receiving block receives the signals as input vectors of the receiving block.
Preferably, the transmitting block of the deep neural network includes an LSTM layer, a full connection layer, and a regularization layer, where an input of the LSTM layer is data to be transmitted by a user, an output of the LSTM layer is overlapped with an input of the LSTM layer and then is used as an input of the full connection layer, a resnet structure is formed, an output of the full connection is an input of the regularization layer, and an output of the regularization layer is an input of a noise layer.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the invention applies a long short-term memory (LSTM) unit to carry out joint coding and decoding and modulation and demodulation processing on the grouping bit sequence in the actual communication system, and effectively utilizes the characteristics of user data to improve the error rate (symbol error rate, SER) performance of the system; meanwhile, the invention provides that a residual error network (residual networks, resNet) structure is applied to an end-to-end intelligent communication system to effectively improve the convergence speed of the neural network and avoid the problems of gradient disappearance and gradient explosion which possibly occur.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of a deep neural network according to the present invention.
Fig. 3 is a schematic structural diagram of the LSTM layer.
FIG. 4 is a schematic diagram of the ResNet structure.
Fig. 5 is a diagram illustrating SER performance comparison between a communication system using the method of the present invention and conventional hamming code, BPSK modulation under AWGN channel.
Fig. 6 is a diagram illustrating SER performance comparison between a rayleigh fast fading channel and a conventional hamming code, BPSK modulated communication system using the method of the present invention.
Fig. 7 is a diagram for comparing SER performance with conventional hamming code using the method of the present invention under different channel estimation accuracy conditions.
Fig. 8 is a diagram showing the comparison of SER performance of a communication system in a deep neural network with and without the addition of a res net structure in an AWGN channel according to the present invention.
Fig. 9 is a schematic diagram of a system structure according to the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the present patent;
for the purpose of better illustrating the embodiments, certain elements of the drawings may be omitted, enlarged or reduced and do not represent the actual product dimensions;
it will be appreciated by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical scheme of the invention is further described below with reference to the accompanying drawings and examples.
Example 1
The embodiment provides an LSTM and ResNet assisted deep learning end-to-end intelligent communication method, as shown in FIG. 1, comprising the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block as shown in fig. 2, and binary data generated randomly in the step S1 is output to the noise layer after being processed by the sending block;
s4: the noise layer scrambles and denoises the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after processing opposite to the transmitting block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the data to be transmitted by the user in the step S1;
s7: setting the transmitting block after training on the communication transmitting end, taking binary data bits to be transmitted as input vectors of the transmitting block, transmitting output vectors of the transmitting block as transmitting signals into a wireless channel, setting the receiving block after training on the communication receiving end, taking the receiving block as input vectors of the receiving block after the receiving end receives signals, and obtaining estimated user transmitting data after processing.
The steps S1 to S6 are training phases of the present invention, and are used for optimizing trainable parameters in the deep neural network, and the step S7 is a deployment application phase of the present invention, in which the deep neural network can directly read the optimized parameters, so as to implement encoding and modulation of bit data to be transmitted by a user at a transmitting end and demodulation and decoding of a received signal at a receiving end in real time.
In step S2, a channel simulation environment is set, specifically:
when the actual channel environment is known, such as a gaussian channel, a rayleigh fading channel and the like, scrambling and noise are directly carried out on an output signal s (t) of a neural network sending block according to the probability distribution of the known actual channel environment to obtain a receiving signal r (t), namely r (t) =i (s (t)), wherein I (·) represents a mapping function from an input signal to an output signal, which is set according to the actual known channel environment in the training process;
if the actual channel environment is unknown, the simulation is performed by adaptively fitting against the generation network and learning the probability distribution of the unknown channel.
The sending block of the deep neural network in step S3 is shown in fig. 3, and includes an LSTM layer, a full connection layer, and a regularization layer, where an input of the LSTM layer is data to be sent by a user, an output of the LSTM layer is overlapped with an input of the LSTM layer and then is used as an input of the full connection layer, a ResNets structure is formed, an output of the full connection is an input of the regularization layer, and an output of the regularization layer is an input of a noise layer.
The user data to be sent is converted into one-hot vector, and then is input into LSTM layer of the sending block, where the vector has only one bit element of 1 and the rest of all are 0, for example, assuming that 2-bit data is used as a group to process user data, d represents 2-bit information, which has 4 possibilities, if the sending end sends 3 rd information, one-hot vector l d =[0,0,1,0]。
The LSTM layer passes through the external input vector x at the time t t And an output vector h at time t-1 t-1 Calculating to obtain an output vector h at the time t t :
f t =σ(W f ·[h t-1 ,x t ]+b f )
i t =σ(W i ·[h t-1 ,x t ]+b i )
o t =σ(W o ·[h t-1 ,x t ]+b o )
h t =o t ⊙tanh(c t )
In the formula, [ h ] t-1 ,x t ]Represents x t ,h t-1 Is a concatenation vector of W f 、b f 、W i 、b i 、W o 、b o As a trainable parameter, σ (. Cndot.) represents a sigmoid activation function, tan h (. Cndot.) represents a tan h activation function, and as such, indicates multiplication of corresponding elements; c t-1 Memory cell at time t-1A metavector; f (f) t 、i t 、o t Respectively a forgetting gate, an input gate and an output gate vector, f t Responsible for controlling c t-1 Which elements need to be reserved and which elements need to be weakened, i t Deciding alternative update vector +.>Which elements need to be added to the cell vector at the current time, f t 、i t 、/>The three co-operate to generate a new memory cell vector c t The method comprises the steps of carrying out a first treatment on the surface of the Next, o t Determining tanh (c) t ) Which of them are output to obtain an output vector h at time t t ;
The LSTM layer is used for capturing dynamic characteristics of input vectors, so that correlation of user data in each time period can be effectively utilized to perform encoding, decoding and modulation demodulation, and reliability of information transmission is improved; even in the special case that the user data in each time period are completely independent, the LSTM layer assisted neural network can still realize joint code modulation operation on the user data in a plurality of time periods. Therefore, the overall optimization advantage is exerted, and meanwhile, the constraint length of the codes is increased under the condition of maintaining the code rate unchanged, so that the reliability of the system is effectively improved.
The full-connection layer is responsible for further carrying out nonlinear operation on the output of the LSTM layer, so that the depth of the neural network is deepened, and the learning capacity of the neural network is improved. And a regularization layer in the transmitting block, which is used for carrying out normalization operation on the coded and modulated signals so as to maintain the power of the transmitted signals at a certain value.
The receiving block of the deep neural network is responsible for learning and performing the inverse operation of the transmitting block to recover the user data bits as correctly as possible from the interfered received signal, comprising two fully connected layers and an LSTM unit with a res net structure, wherein:
the input of one full-connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is overlapped with the output of one full-connection layer to be used as the input of the other full-connection layer, the output of the other full-connection layer is a probability vector obtained by mapping that data to be sent by a user are firstly converted into one-hot vectors, the serial number of the largest element in the probability vector is judged as user sending information, and estimated user data are obtained according to the serial number.
As shown in fig. 4, in the resuts structure, assuming that the input vector is x, the mapping that is desired to be implemented is G (x), and the actual output of the target layer is F (x). Then, before introducing the ResNet structure, it is necessary to train so that F (x) =G (x); after the ResNet structure is introduced, the training object is changed to be H (x) =F (x) +x=G (x), that is, F (x) =G (x) -x. By introducing the structure, the deep neural network can realize residual error learning on the appointed layer, so that the degradation of the neural network is avoided, the possibility of gradient disappearance problem in the training process is reduced, and the convergence efficiency of the neural network is improved.
Step S6, training the deep neural network according to the difference between the estimated data obtained in the step S5 and the binary data randomly generated in the step S1, wherein the training method specifically comprises the following steps:
calculating a loss function value between estimated user data output by the deep neural network and data to be transmitted by a user according to a preset loss function, quantitatively measuring the difference between actual output and expected output, and calculating the bias of the loss value to each trainable parameter in the deep neural network through a back propagation algorithm, so that the parameters of the deep neural network are updated by a preset optimization algorithm, and the difference between the estimated user data and the data to be transmitted by the user of the deep neural network is reduced.
In the step S6, attention is paid to the deep neural network in the training phase, and the transmitting block and the receiving block are training-optimized together. In the deployment application stage after training, the transmitting block and the receiving block of the neural network are deployed at two positions of the transmitting end and the receiving end respectively so as to realize coding, modulation and demodulation and decoding processing of user bit data respectively.
In step S7, if the actual channel changes slowly, retraining with low time cost and low calculation cost is performed by a method including migration learning, and synchronous performance of deep neural network training and deployment application is achieved by a method including parallel training.
In a specific embodiment, the wireless channel on which the signal propagates is assumed to be an AWGN channel, while 4 bits of user information are required to be transmitted over 7 channels. Under this condition, the LSTM and ResNet aided deep learning end-to-end intelligent communication system provided by the invention is compared with the traditional Hamming code coding and BPSK modulation communication system in terms of SER performance, and the result is shown in FIG. 5. Where Q in the legend represents the number of groups of 4 bits of user information that are processed by the LSTM unit in combination at a time. According to fig. 5, the SER performance of the communication system using the method of the present invention is superior to that of the conventional communication system for hard decision decoding under AWGN channel, and the effectiveness of the system of the present invention is demonstrated similarly to the conventional communication system for maximum likelihood decoding. In addition, compared with a neural network using LSTM units without ResNet structures, the neural network using LSTM units with ResNet structures can obtain better SER performance, which shows that the ResNet structures adopted by the invention can effectively improve the reliability of the end-to-end intelligent communication system.
Under the condition that other conditions are unchanged, the wireless channel for signal propagation is set as a Rayleigh fast fading channel, and under the condition, the LSTM and ResNet assisted deep learning end-to-end intelligent communication system provided by the invention is compared with the conventional communication system with Hamming code coding and BPSK modulation to carry out SER performance, and the result is shown in figure 6. According to the result, under the condition of more complex communication environment, compared with the traditional communication system, the end-to-end intelligent communication system can obtain better SER performance, and the performance gain is improved along with the increase of Q. This further demonstrates that the system of the present invention has better reliability and reflects that LSTM units can indeed improve SER performance of information transmission by jointly processing multiple sets of user data bits.
Radio channel setting for signal propagation under otherwise unchanged conditionsFor Rayleigh fast fading channels, SER performance of the system of the present invention and a conventional communication system under different channel estimation accuracy conditions was tested to reflect robustness of the system of the present invention. The results are shown in FIG. 7. Assuming that the actual channel coefficient is h and the channel estimation accuracy rate is ρ, estimating the channel coefficientCan be expressed as:
where ε is a complex Gaussian variable with a mean of 0 and a variance of 1 and ρ is a constant between 0 and 1. In the training phase, ρ is fixed to 1, indicating that the channel estimation during training is completely accurate; in the test stage, SER performance of each system under different rho conditions is tested to reflect the robustness of the system when the training conditions differ from the test conditions. As can be seen from fig. 7, when the channel estimation is not perfect and the test condition deviates from the training condition, the system of the present invention can still obtain SER performance better than the conventional communication system when q=4, 8, which proves that the system has good robustness.
And when other conditions are unchanged, the SER performance of the neural network with and without the ResNet structure is added after the number of each training round is measured, so that the convergence efficiency of the system is reflected. According to the illustration in fig. 8, in the training process, compared with the neural network without the ResNet structure, the deep neural network adopted by the system of the invention can more quickly improve the SER performance before reaching the saturation state, and can obtain better SER performance when the training reaches the saturation state. Therefore, the neural network adopted by the system has higher convergence efficiency and lower training time cost.
Example 2
The embodiment provides an LSTM and res-assisted deep learning end-to-end intelligent communication system, which is characterized in that the system uses the LSTM and res-assisted deep learning end-to-end intelligent communication method described in embodiment 1, as shown in fig. 9, and the system includes:
the generation module is used for randomly generating binary data serving as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generating module are output to the noise layer after being processed by the sending block;
the noise module is used for scrambling and noise-adding the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sending the signal to the receiving block;
a receiving processing module, in which the receiving block obtains estimated user data after processing opposite to the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
the deployment module sets the transmitting block after training on the communication transmitting end, takes binary data bits to be transmitted as input vectors of the transmitting block, transmits output vectors of the transmitting block as transmitting signals to the wireless channel, sets the receiving block after training on the communication receiving end, and obtains estimated user transmitting data after processing after the receiving block receives the signals as input vectors of the receiving block.
The transmitting block of the deep neural network comprises an LSTM layer, a full-connection layer and a regularization layer, wherein the input of the LSTM layer is data to be transmitted by a user, the output of the LSTM layer and the input of the LSTM layer are overlapped to be used as the input of the full-connection layer, a ResNet structure is formed, the full-connection output is the input of the regularization layer, and the output of the regularization layer is the input of the noise layer.
The same or similar reference numerals correspond to the same or similar components;
the terms describing the positional relationship in the drawings are merely illustrative, and are not to be construed as limiting the present patent;
it is to be understood that the above examples of the present invention are provided by way of illustration only and not by way of limitation of the embodiments of the present invention. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the invention are desired to be protected by the following claims.
Claims (6)
1. An LSTM and ResNet assisted deep learning end-to-end intelligent communication method is characterized by comprising the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data generated randomly in the step S1 is output to the noise layer after being processed by the sending block;
s4: the noise layer scrambles and denoises the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after processing opposite to the transmitting block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the data to be transmitted by the user in the step S1;
s7: setting the transmitting block after training on a communication transmitting end, taking binary data bits to be transmitted as input vectors of the transmitting block, transmitting output vectors of the transmitting block as transmitting signals into a wireless channel, setting the receiving block after training on a communication receiving end, taking the receiving block as the input vectors of the receiving block after the receiving end receives signals, and obtaining estimated user transmitting data after processing;
the transmitting block of the deep neural network in the step S3 comprises an LSTM layer, a full-connection layer and a regularization layer, wherein the input of the LSTM layer is data to be transmitted by a user, the output of the LSTM layer and the input of the LSTM layer are overlapped to be used as the input of the full-connection layer to form a ResNet structure, the full-connection output is the input of the regularization layer, and the output of the regularization layer is the input of a noise layer;
the user data to be sent is firstly converted into one-hot vectors and then is input into an LSTM layer of a sending block;
the receiving block of the deep neural network comprises two full connection layers and an LSTM unit with a ResNet structure, wherein:
wherein the input of one full-connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is overlapped with the output of one full-connection layer to be used as the input of the other full-connection layer, the output of the other full-connection layer is a probability vector obtained by mapping that data to be sent by a user is firstly converted into a one-hot vector, the serial number of the largest element in the probability vector is judged as user sending information, and estimated user data is obtained according to the serial number;
step S6, training the deep neural network according to the difference between the estimated data obtained in the step S5 and the binary data randomly generated in the step S1, wherein the training method specifically comprises the following steps:
calculating a loss function value between estimated user data output by the deep neural network and data to be transmitted by a user according to a preset loss function, quantitatively measuring the difference between actual output and expected output, and calculating the bias of the loss value to each trainable parameter in the deep neural network through a back propagation algorithm, so that the parameters of the deep neural network are updated by a preset optimization algorithm, and the difference between the estimated user data and the data to be transmitted by the user of the deep neural network is reduced.
2. The LSTM and ResNets-assisted deep learning end-to-end intelligent communication method according to claim 1, wherein the channel simulation environment is set in step S2, specifically:
when the actual channel environment is known, simulation is performed directly according to the probability distribution of the known actual channel environment;
if the actual channel environment is unknown, the simulation is performed by adaptively fitting against the generation network and learning the probability distribution of the unknown channel.
3. The LSTM and ResNets assisted deep learning end-to-end intelligent communication method of claim 2, wherein said LSTM layer is configured to pass an external input vector x at time t t And an output vector h at time t-1 t-1 Calculating to obtain an output vector h at the time t t :
f t =σ(W f ·[h t-1 ,x t ]+b f )
i t =σ(W i ·[h t-1 ,x t ]+b i )
o t =σ(W o ·[h t-1 ,x t ]+b o )
h t =o t ⊙tanh(c t )
In the formula, [ h ] t-1 ,x t ]Represents x t ,h t-1 Is a concatenation vector of W f 、b f 、W i 、b i 、W o 、b o For trainable parameters, σ (·) represents a sigmoid activation function, tan h (·) represents a tan h activation function, and pair represents multiplication of corresponding elements; c t-1 A memory cell vector at time t-1; f (f) t 、i t 、o t Respectively a forgetting gate, an input gate and an output gate vector, f t Responsible for controlling c t-1 Which elements need to be reserved and which elements need to be weakened, i t Deciding alternative update vector +.>Which elements need to be added to the cell vector at the current time, f t 、i t 、/>The three co-operate to generate a new memory cell vector c t The method comprises the steps of carrying out a first treatment on the surface of the Next, o t Determining tanh (c) t ) Which of them are output to obtain an output vector h at time t t 。
4. The LSTM and ResNets-assisted deep learning end-to-end intelligent communication method according to claim 1, wherein in step S7, if the actual channel changes slowly, retraining with low time cost and low computation cost is performed by a method including migration learning, and synchronization between deep neural network training and deployment application is achieved by a method including parallel training.
5. An LSTM and resnet assisted deep learning end-to-end intelligent communications system using the LSTM and resnet assisted deep learning end-to-end intelligent communications method of any of claims 1 to 4, the system comprising:
the generation module is used for randomly generating binary data serving as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generating module are output to the noise layer after being processed by the sending block;
the noise module is used for scrambling and noise-adding the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sending the signal to the receiving block;
a receiving processing module, in which the receiving block obtains estimated user data after processing opposite to the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
the deployment module sets the transmitting block after training on the communication transmitting end, takes binary data bits to be transmitted as input vectors of the transmitting block, transmits output vectors of the transmitting block as transmitting signals to the wireless channel, sets the receiving block after training on the communication receiving end, and obtains estimated user transmitting data after processing after the receiving block receives the signals as input vectors of the receiving block.
6. The LSTM and ResNets assisted deep learning end-to-end intelligent communication system of claim 5, wherein the transmission block of the deep neural network includes an LSTM layer, a fully connected layer, and a regularized layer, wherein an input of the LSTM layer is data to be transmitted by a user, an output of the LSTM layer is overlapped with an input of the LSTM layer and is used as an input of the fully connected layer, a ResNets structure is formed, an output of the fully connected layer is an input of the regularized layer, and an output of the regularized layer is an input of a noise layer.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111113281.8A CN113852434B (en) | 2021-09-18 | 2021-09-18 | LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111113281.8A CN113852434B (en) | 2021-09-18 | 2021-09-18 | LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113852434A CN113852434A (en) | 2021-12-28 |
CN113852434B true CN113852434B (en) | 2023-07-25 |
Family
ID=78979321
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111113281.8A Active CN113852434B (en) | 2021-09-18 | 2021-09-18 | LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113852434B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230267337A1 (en) * | 2022-02-24 | 2023-08-24 | Protopia AI, Inc. | Conditional noise layers for generating adversarial examples |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109767759A (en) * | 2019-02-14 | 2019-05-17 | 重庆邮电大学 | End-to-end speech recognition methods based on modified CLDNN structure |
CN110460402A (en) * | 2019-07-15 | 2019-11-15 | 哈尔滨工程大学 | A kind of end-to-end communication system method for building up based on deep learning |
-
2021
- 2021-09-18 CN CN202111113281.8A patent/CN113852434B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109767759A (en) * | 2019-02-14 | 2019-05-17 | 重庆邮电大学 | End-to-end speech recognition methods based on modified CLDNN structure |
CN110460402A (en) * | 2019-07-15 | 2019-11-15 | 哈尔滨工程大学 | A kind of end-to-end communication system method for building up based on deep learning |
Non-Patent Citations (1)
Title |
---|
基于ResNet-BLSTM的端到端语音识别;胡章芳;徐轩;付亚芹;夏志广;马苏东;;计算机工程与应用(第18期);全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN113852434A (en) | 2021-12-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Park et al. | Learning to demodulate from few pilots via offline and online meta-learning | |
WO2018233932A1 (en) | Data transmission network configuration | |
Hanna et al. | Signal processing-based deep learning for blind symbol decoding and modulation classification | |
Nguyen et al. | Application of deep learning to sphere decoding for large MIMO systems | |
CN113381828B (en) | Sparse code multiple access random channel modeling method based on condition generation countermeasure network | |
Lu et al. | Deep autoencoder learning for relay-assisted cooperative communication systems | |
CN114337911A (en) | Communication method based on neural network and related device | |
Liu et al. | Toward intelligent wireless communications: Deep learning-based physical layer technologies | |
Morocho-Cayamcela et al. | Learning to communicate with autoencoders: Rethinking wireless systems with deep learning | |
CN113852434B (en) | LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system | |
Thakkar et al. | Deep learning and channel estimation | |
CN113890799B (en) | Underwater acoustic communication channel estimation and signal detection method based on domain countermeasure network | |
Ye et al. | Bilinear convolutional auto-encoder based pilot-free end-to-end communication systems | |
Shen et al. | Blind recognition of channel codes via deep learning | |
CN117176517A (en) | Method and system for implementing channel estimation for MIMO-OFDM visible light communication by adopting genetic algorithm and deep learning | |
Dridi et al. | Blind detection of severely blurred 1d barcode | |
Huang et al. | Extrinsic neural network equalizer for channels with high inter-symbol-interference | |
Liu et al. | MIMO signal multiplexing and detection based on compressive sensing and deep learning | |
Lu et al. | Attention-empowered residual autoencoder for end-to-end communication systems | |
CN117255996A (en) | Signal processing method, communication device and communication system | |
Njoku et al. | BLER performance evaluation of an enhanced channel autoencoder | |
Chaudhari et al. | A resnet based end-to-end wireless communication system under rayleigh fading and bursty noise channels | |
KR100888649B1 (en) | Decoder for Detecting Transmitted Signal at MIMO system and Method thereof | |
Hares et al. | Recurrent neural networks for pilot-aided wireless communications | |
Muthulakshmi et al. | Signal demodulation without channel equalizer using machine learning techniques |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |