CN114818488A - Ultrafast pulse laser reverse implementation method and system based on reinforcement learning - Google Patents
Ultrafast pulse laser reverse implementation method and system based on reinforcement learning Download PDFInfo
- Publication number
- CN114818488A CN114818488A CN202210415895.XA CN202210415895A CN114818488A CN 114818488 A CN114818488 A CN 114818488A CN 202210415895 A CN202210415895 A CN 202210415895A CN 114818488 A CN114818488 A CN 114818488A
- Authority
- CN
- China
- Prior art keywords
- pulse
- laser
- reinforcement learning
- ultrafast
- action
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
- G06F2119/02—Reliability analysis or reliability optimisation; Failure analysis, e.g. worst case scenario performance, failure mode and effects analysis [FMEA]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Lasers (AREA)
Abstract
The invention provides a method and a system for realizing the reverse direction of an ultrafast pulse laser based on reinforcement learning, which comprises the following steps: step 1: acquiring a seed source through a mode locking technology, and determining a pulse form and a light spot mode through the seed source; step 2: amplifying the power of the pulse by adopting an optical fiber amplification technology, and determining the final energy and output characteristics of the pulse; and step 3: the generation process of the ultrafast laser is modularized and integrated, and the propagation process of pulses in the optical fiber is simulated through a distributed Fourier algorithm; and 4, step 4: inputting the simulation data into a long-short term memory network model LSTM according to a preset window size value, and performing step-by-step training; and 5: and inputting the trained result into the reinforcement learning DDPG to obtain the optimal parameters of the laser, so that the laser can rapidly output the specified target pulse state. According to the invention, LSTM neural network modeling is used for replacing the traditional algorithm, so that the simulation time consumption is shortened, and the simulation efficiency is improved.
Description
Technical Field
The invention relates to the technical field of ultrafast pulse lasers, in particular to a method and a system for realizing the reversal of an ultrafast pulse laser based on reinforcement learning.
Background
The pulse laser is a laser which works once every certain time when the pulse width of a single laser is less than 0.25 second, has larger output power and is suitable for laser marking, cutting, distance measurement and the like.
Patent document CN113922199A (application number: CN202111141060.1) discloses a return-resistant main oscillation power amplification pulse laser, which sequentially comprises a seed semiconductor laser, a return light processor, a first-stage amplifier, a first-stage return light processor, a second-stage amplifier, a second-stage return light processor, …, an n-stage amplifier, and an n-stage return light processor; the return light processor is designed to ensure the forward transmission of the signal light, prevent the reverse transmission of nonlinear laser, ASE light and return light outside the optical path, and monitor the return light.
In the prior art, the transmission process of the ultrafast laser is simulated mainly by an iterative distributed Fourier conventional algorithm, the algorithm has high complexity and low efficiency, and the requirement of providing feedback information for the rapid reverse design of the ultrafast laser is difficult to meet; on the other hand, at present, no technology exists for reversely designing the ultrafast laser by using an accurate simulation modeling result, and the design of the ultrafast laser almost completely depends on experience and experimental trials.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method and a system for realizing the reversal of an ultrafast pulse laser based on reinforcement learning.
The ultrafast pulse laser reverse implementation method based on reinforcement learning provided by the invention comprises the following steps:
step 1: acquiring a seed source through a mode locking technology, and determining a pulse form and a light spot mode through the seed source;
step 2: amplifying the power of the pulse by adopting an optical fiber amplification technology, and determining the final energy and output characteristics of the pulse;
and step 3: the generation process of the ultrafast laser is modularized and integrated, and the propagation process of pulses in the optical fiber is simulated through a distributed Fourier algorithm;
and 4, step 4: inputting the simulation data into a long-short term memory network model LSTM according to a preset window size value, and performing step-by-step training;
and 5: and inputting the trained result into the reinforcement learning DDPG to obtain the optimal parameters of the laser, so that the laser can rapidly output the specified target pulse state.
Preferably, the action of obtaining the maximum value is searched through an action estimation network, an action reality network, a state reality network and a state estimation network, and the characteristics of the output pulse of the seed source in the laser, the dispersion coefficient of the broadening grating and the gain coefficient and the length of the gain fiber are output.
Preferably, the pulse output predicted by the model LSTM is used as the input of the reinforcement learning DDPG, different laser parameters are selected as the action, the Q value is estimated through reinforcement learning, the Q value is the sum of the value for evaluating the current action and the reward value for predicting the future action, then the mean square error of the Q value under the ideal pulse condition and the Q value under the current state and action is calculated, and the action which enables the mean square error between the simulation output pulse and the target pulse to be minimum is searched to be used as the optimal laser parameters.
Preferably, in the simulation, the transmission distance of the optical pulse in the optical fiber is set to be h, firstly, only the pulse is subjected to the action of the nonlinear effect, and the dispersion and the loss are zero; the nonlinear effect is then set to zero, taking into account only the effects of dispersion and loss.
Preferably, the transmitted pulse amplitude is expressed as:
wherein A (z, T) represents the pulse amplitude in the z direction over a period T; d represents chromatic dispersion; n represents non-linearity; linear operator
The calculation in the frequency domain is:
wherein, F -1 Representing the inverse fourier transform, ω is the angular frequency,
is the complex amplitude.
The ultrafast pulse laser reverse implementation system based on reinforcement learning provided by the invention comprises:
module M1: acquiring a seed source through a mode locking technology, and determining a pulse form and a light spot mode through the seed source;
module M2: amplifying the power of the pulse by adopting an optical fiber amplification technology, and determining the final energy and output characteristics of the pulse;
module M3: the generation process of the ultrafast laser is modularized and integrated, and the propagation process of pulses in the optical fiber is simulated through a distributed Fourier algorithm;
module M4: inputting the simulation data into a long-short term memory network model LSTM according to a preset window size value, and performing step-by-step training;
module M5: and inputting the trained result into the reinforcement learning DDPG to obtain the optimal parameters of the laser, so that the laser can rapidly output the specified target pulse state.
Preferably, the action of obtaining the maximum value is searched through an action estimation network, an action reality network, a state reality network and a state estimation network, and the characteristics of the output pulse of the seed source in the laser, the dispersion coefficient of the broadening grating and the gain coefficient and the length of the gain fiber are output.
Preferably, the pulse output predicted by the model LSTM is used as the input of the reinforcement learning DDPG, different laser parameters are selected as the action, the Q value is estimated through reinforcement learning, the Q value is the sum of the value for evaluating the current action and the reward value for predicting the future action, then the mean square error of the Q value under the ideal pulse condition and the Q value under the current state and action is calculated, and the action which enables the mean square error between the simulation output pulse and the target pulse to be minimum is searched to be used as the optimal laser parameters.
Preferably, in the simulation, the transmission distance of the optical pulse in the optical fiber is set to be h, firstly, only the pulse is subjected to the action of the nonlinear effect, and the dispersion and the loss are zero; the nonlinear effect is then set to zero, taking into account only the effects of dispersion and loss.
Preferably, the transmitted pulse amplitude is expressed as:
wherein A (z, T) represents the pulse amplitude in the z direction over a period T; d represents chromatic dispersion; n represents non-linearity; linear operator
The calculation in the frequency domain is:
wherein, F -1 Representing the inverse fourier transform, ω is the angular frequency,
is the complex amplitude.
Compared with the prior art, the invention has the following beneficial effects:
(1) the invention provides a fast reverse design method of an ultrafast pulse laser based on reinforcement learning, which comprises the steps of firstly replacing a traditional distributed Fourier algorithm with low efficiency by a long-short term memory network model LSTM, improving simulation efficiency, then combining with neural network fast modeling according to output requirements, carrying out reverse design on the ultrafast laser through reinforcement learning, deriving laser parameters meeting the output requirements, and providing an intelligent reference method for building a real ultrafast pulse laser system;
(2) the ultra-fast pulse laser is modeled by artificial intelligence algorithm modeling instead of the traditional distributed Fourier algorithm, so that the operation complexity is reduced, and the simulation efficiency is improved; carrying out reverse design on the ultrafast pulse laser through reinforcement learning;
(3) according to the invention, LSTM neural network modeling is used for replacing the traditional algorithm, so that the simulation time is shortened, and the simulation efficiency is improved; the ultrafast pulse laser is intelligently and reversely designed by adopting reinforcement learning, and the defect that the design can only be carried out by experience and a large number of experiments in the past is overcome.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a flow chart of an industrial production process of an ultrafast pulse laser;
FIG. 2 is a flow chart of ultrafast pulse amplification;
FIG. 3 is a schematic diagram of an ultrafast pulse laser reverse design based on reinforcement learning;
FIG. 4 is a schematic diagram of DDPG.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
Example (b):
the output of the ultrafast pulse laser has an ultrashort time width (which can be as narrow as femtosecond magnitude), an ultrahigh peak power and an ultrawide spectrum range, and the industrial production process can be roughly divided into three parts: seed source, amplifier, integration as shown in figure 1. The ultrafast laser seed source is generally generated by a mode locking technology, and determines an ultrashort pulse form, a light spot mode and the like; the amplifier often adopts the fiber amplification technology, which determines the final energy and output characteristics of the ultrashort pulse; the integration modularizes and integrates the ultrafast laser generation process, and is convenient for industrial production and use.
In the numerical simulation process of the amplification part, a distributed Fourier algorithm is often adopted to solve the nonlinear Schrodinger equation satisfied in the pulse propagation process, and a Longger Kutta algorithm is utilized to solve the rate equation satisfied in the gain process. Taking the chirped pulse amplification process as an example, fig. 2 shows a flow of ultrafast pulse amplification.
The iterative distributed Fourier conventional algorithm can accurately model the ultrafast pulse laser. However, the traditional iterative distributed fourier algorithm is complex and has long simulation time. On the other hand, the design of the ultrafast pulse laser almost depends on experiments completely, if pulse output with specified characteristics is obtained, parameters of each module of the laser need to be fine-tuned through repeated experiments, the workload is huge, and the requirement on experience of experimenters is high.
The reverse design process of the ultrafast pulse laser based on reinforcement learning provided by the invention is shown in fig. 3, firstly, a large amount of simulation data is generated through a traditional algorithm, and the process mainly involves two parts: 1) simulating the propagation process of the pulse in the optical fiber by using a distributed Fourier algorithm; 2) and solving a rate equation satisfied by the pulse in the gain fiber by utilizing a fourth-order Runge Kutta algorithm. And then, inputting the simulation data into a neural network LSTM according to the set window size value, and performing step-by-step training by adopting a long-short term memory network model (LSTM). Compared with the traditional distributed Fourier algorithm, the well-trained LSTM neural network can greatly shorten the simulation time and improve the simulation efficiency.
The result after the neural network training is input into the reinforcement learning DDPG, the action with the maximum value is searched for through the action estimation network, the action reality network, the state reality network and the state estimation network, so that ideal parameters (such as seed source output pulse characteristics, dispersion coefficients of a broadening grating, gain coefficients and length of a gain optical fiber and the like) of each module in the laser are output, the LSTM neural network modeling is combined, the time complexity of simulation is reduced, and the change process of a pulse light field, a time domain and a frequency domain of the laser under different laser parameters is calculated in a shorter time. And feeding back the calculation result to a reinforcement learning algorithm DPPG, so that the laser can rapidly output a specified target pulse state, the rapid reverse design of the ultrafast laser is realized, and guidance and reference are provided for the establishment of a real ultrafast laser system.
The simulation process is as follows:
setting the transmission distance of the optical pulse in the optical fiber as h, firstly, only allowing the pulse to undergo the action of nonlinear effect, and setting the dispersion and loss as zero; then setting the nonlinear effect to be zero, and only considering the effects of dispersion and loss;
the transmitted pulse amplitude is expressed as:
wherein D represents dispersion, N represents nonlinearity, and linear operator
It can be calculated in the frequency domain:
wherein, F -1 Denotes the inverse fourier transform, w is the angular frequency and a is the complex amplitude.
Solving a rate equation satisfied by the pulse in the gain fiber by using a fourth-order Rungestota algorithm:
wherein h is the solving step length, k 1 、k 2 、k 3 、k 4 Are coefficients.
Inputting the simulation data into a long-short term memory network model LSTM according to a preset window size value, and performing step-by-step training; assuming a step size of 10, data numbers 0 to 9 in the dataset are selected as input for LSTM and then data number 10 is predicted, then data number eleventh is predicted with 1 to 10, and so on.
As shown in fig. 4, combining with the LSTM neural network modeling, the time complexity of the simulation is reduced, and the change processes of the pulsed light field, time domain and frequency domain of the laser under different laser parameters are calculated in a shorter time; the calculation result is fed back to a reinforcement learning algorithm DDPG, the current state is input into an Actor network to select an action A, the state and the action are input into a Critic network together to obtain a current Q value, meanwhile, the obtained action and the state are input into the Critic target network together when the target state is input into the Actor target network to obtain a target Q value, and the error between the target Q value and the Critic target network is calculated with the aim of minimizing the error, so that the laser can quickly output a specified target pulse state, the quick reverse design of the ultrafast laser is realized, and guidance and reference are provided for the establishment of a real ultrafast laser system.
Those skilled in the art will appreciate that, in addition to implementing the systems, apparatus, and various modules thereof provided by the present invention in purely computer readable program code, the same procedures can be implemented entirely by logically programming method steps such that the systems, apparatus, and various modules thereof are provided in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (10)
1. An ultrafast pulse laser reverse implementation method based on reinforcement learning is characterized by comprising the following steps:
step 1: acquiring a seed source through a mode locking technology, and determining a pulse form and a light spot mode through the seed source;
step 2: amplifying the power of the pulse by adopting an optical fiber amplification technology, and determining the final energy and output characteristics of the pulse;
and step 3: the generation process of the ultrafast laser is modularized and integrated, and the propagation process of pulses in the optical fiber is simulated through a distributed Fourier algorithm;
and 4, step 4: inputting the simulation data into a long-short term memory network model LSTM according to a preset window size value, and performing step-by-step training;
and 5: and inputting the trained result into the reinforcement learning DDPG to obtain the optimal parameters of the laser, so that the laser can rapidly output the specified target pulse state.
2. The reinforcement learning-based ultrafast pulse laser reverse implementation method of claim 1, wherein the action of obtaining the maximum value is found through an action estimation network, an action reality network, a state reality network, and a state estimation network, and the seed source output pulse characteristics, the dispersion coefficient of the broadened grating, and the gain coefficient and length of the gain fiber in the output laser are obtained.
3. The reinforcement learning-based ultrafast pulse laser reverse implementation method of claim 1, wherein the pulse output predicted by model LSTM is used as the input of reinforcement learning DDPG, different laser parameters are selected as actions, Q value is estimated through reinforcement learning, Q value is the sum of the value of evaluating the current action and the reward value of predicting future actions, then the mean square error between Q value under ideal pulse condition and Q value under current state and action is calculated, and the action that minimizes the mean square error between the simulated output pulse and the target pulse is searched as the optimal laser parameter.
4. The ultrafast pulse laser backward realization method based on reinforcement learning as claimed in claim 1, wherein in the simulation, the distance of optical pulse transmission in the fiber is set as h, only the pulse is first allowed to experience the effect of nonlinear effect, and the dispersion and loss are zero; the nonlinear effect is then set to zero, taking into account only the effects of dispersion and loss.
5. The ultrafast pulse laser backward realization method based on reinforcement learning of claim 4, wherein the transmitted pulse amplitude is expressed as:
wherein A (z, T) represents the pulse amplitude in the z direction over a period T; d represents chromatic dispersion; n represents non-linearity; linear operator
The calculation in the frequency domain is:
wherein, F -1 Representing the inverse fourier transform, ω is the angular frequency,
is the complex amplitude.
6. An ultrafast pulsed laser reverse implementation system based on reinforcement learning, comprising:
module M1: acquiring a seed source through a mode locking technology, and determining a pulse form and a light spot mode through the seed source;
module M2: amplifying the power of the pulse by adopting an optical fiber amplification technology, and determining the final energy and output characteristics of the pulse;
module M3: the generation process of the ultrafast laser is modularized and integrated, and the propagation process of pulses in the optical fiber is simulated through a distributed Fourier algorithm;
module M4: inputting the simulation data into a long-short term memory network model LSTM according to a preset window size value, and performing step-by-step training;
module M5: and inputting the trained result into the reinforcement learning DDPG to obtain the optimal parameters of the laser, so that the laser can rapidly output the specified target pulse state.
7. The reinforcement learning-based ultrafast pulse laser backward realization system of claim 6, wherein the action of obtaining the maximum value is found through action estimation network, action reality network, state estimation network, and the seed source output pulse characteristics in the output laser, the dispersion coefficient of the broadening grating, the gain coefficient and the length of the gain fiber.
8. The reinforcement learning-based ultrafast pulse laser backward realization system of claim 6, wherein the pulse output predicted by model LSTM is used as the input of reinforcement learning DDPG, different laser parameters are selected as the actions, Q value is estimated by reinforcement learning, Q value is the sum of the value of evaluating the current action and the reward value of predicting the future action, then the mean square error between Q value under ideal pulse condition and Q value under current state and action is calculated, and the action that minimizes the mean square error between the simulated output pulse and the target pulse is searched as the optimal laser parameter.
9. The reinforcement learning-based ultrafast pulse laser backward realization system of claim 6, wherein in the simulation, the distance of optical pulse transmission in the fiber is set to h, only the pulse is first allowed to experience the effect of nonlinear effect, and the dispersion and loss are zero; the nonlinear effect is then set to zero, taking into account only the effects of dispersion and loss.
10. The reinforcement learning-based ultrafast pulsed laser reverse realization system of claim 9, wherein the transmitted pulse amplitude is expressed as:
wherein A (z, T) represents the pulse amplitude in the z direction over a period T; d represents chromatic dispersion; n represents non-linearity; linear operator
The calculation in the frequency domain is:
wherein, F -1 Representing the inverse fourier transform, ω is the angular frequency,
is the complex amplitude.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210415895.XA CN114818488A (en) | 2022-04-20 | 2022-04-20 | Ultrafast pulse laser reverse implementation method and system based on reinforcement learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210415895.XA CN114818488A (en) | 2022-04-20 | 2022-04-20 | Ultrafast pulse laser reverse implementation method and system based on reinforcement learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114818488A true CN114818488A (en) | 2022-07-29 |
Family
ID=82506116
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210415895.XA Pending CN114818488A (en) | 2022-04-20 | 2022-04-20 | Ultrafast pulse laser reverse implementation method and system based on reinforcement learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114818488A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118137277A (en) * | 2024-05-06 | 2024-06-04 | 南京信息工程大学 | Rapid automatic mode locking method, system and equipment based on deep learning |
-
2022
- 2022-04-20 CN CN202210415895.XA patent/CN114818488A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118137277A (en) * | 2024-05-06 | 2024-06-04 | 南京信息工程大学 | Rapid automatic mode locking method, system and equipment based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Darbon et al. | Overcoming the curse of dimensionality for some Hamilton–Jacobi partial differential equations via neural network architectures | |
| Vörös | Modeling and identification of systems with backlash | |
| Owens et al. | Iterative learning control—An optimization paradigm | |
| Wu et al. | Output feedback control of Markovian jump repeated scalar nonlinear systems | |
| Li et al. | Finite energy Lyapunov function candidate for fractional order general nonlinear systems | |
| Solak et al. | Observer-based control of a class of chaotic systems | |
| Aguila-Camacho et al. | Boundedness of the solutions for certain classes of fractional differential equations with application to adaptive systems | |
| Blanchet et al. | On exact sampling of stochastic perpetuities | |
| Paquot et al. | Reservoir computing: a photonic neural network for information processing | |
| CN114818488A (en) | Ultrafast pulse laser reverse implementation method and system based on reinforcement learning | |
| Emelianova et al. | Stability of nonlinear discrete repetitive processes with Markovian switching | |
| WO2013190147A1 (en) | Method for automated assistance to design nonlinear analog circuit with transient solver | |
| Wu et al. | Numerical solutions of wavelet neural networks for fractional differential equations | |
| Altun | FPAA implementations of fractional-order chaotic systems | |
| Song et al. | Stability and l2-gain analysis for a class of discrete-time non-linear Markovian jump systems with actuator saturation and incomplete knowledge of transition probabilities | |
| Waheeb et al. | Forecasting the behavior of gas furnace multivariate time series using ridge polynomial based neural network models | |
| Yang et al. | Input-to-state stable neural ordinary differential equations with applications to transient modeling of circuits | |
| Vander Wal et al. | Transfer learning as a method to reproduce high-fidelity non-local thermodynamic equilibrium opacities in simulations | |
| Dhang et al. | MReC: A multilayer photonic reservoir computing architecture | |
| Abolpour et al. | An equivalent condition for stability analysis of LTI systems with bounded time-invariant delay | |
| Maday et al. | A parallel in time approach for quantum control: the parareal algorithm | |
| Faudzi et al. | Generalized predictive controller using Bat algorithm for double acting pneumatic cylinder | |
| Zhang et al. | A control scheme for bilateral teleoperation systems based on time-varying communication delay identification | |
| Sulikowski et al. | Iterative learning control for a discretized sub-class of spatially interconnected systems | |
| Elloumi et al. | Improved Discrete Techniques of Time‐Delay and Order Estimation for Large‐Scale Interconnected Nonlinear Systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |