CN115830428A - Precipitation approach prediction image generation method and system, electronic device and storage medium - Google Patents
Precipitation approach prediction image generation method and system, electronic device and storage medium Download PDFInfo
- Publication number
- CN115830428A CN115830428A CN202211559910.4A CN202211559910A CN115830428A CN 115830428 A CN115830428 A CN 115830428A CN 202211559910 A CN202211559910 A CN 202211559910A CN 115830428 A CN115830428 A CN 115830428A
- Authority
- CN
- China
- Prior art keywords
- information
- images
- predicted
- coding
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a precipitation approach prediction image generation method, a precipitation approach prediction image generation system, an electronic device and a storage medium, and relates to the technical field of weather prediction, wherein the precipitation approach prediction image generation method comprises the following steps: acquiring m frames of images to be predicted, which need rainfall approaching prediction; the method comprises the steps of performing downsampling on all images to be predicted to obtain image characteristics, and performing time position coding on all images to be predicted to obtain first coding information; coding all image characteristics and the first coding information to obtain second coding information; coding n time points of m frames of time T = [ m +1, m +2, … m + n ] of the image to be predicted to obtain third coding information; decoding the second coding information and the third coding information to obtain n decoded images, wherein the number of the decoded images is the same as that of the time points; and upsampling the decoded n images to obtain n frames of precipitation adjacent predicted images.
Description
Technical Field
The invention relates to the technical field of weather prediction, in particular to a precipitation approach prediction image generation method, a precipitation approach prediction image generation system, an electronic device and a storage medium.
Background
With the development of deep Learning technology, weather prediction is increasingly accepted by meteorological personnel by using a deep Learning manner, and In the aspect of precipitation approach prediction, a typical precipitation approach prediction model is a single-frame input single-frame output, and a required radar image sequence is generated In a recursive manner, for example (PredRNN: current Neural Networks for Predictive tracking using LSTMs and Memory In Memory: A Predictive Neural Networks for tracking high-Order-organic Non-statistical spatial Dynamics). The recursive generation is to continue to predict the radar picture at the next time by using the predicted radar picture, so that the error caused by each prediction gradually increases in an accumulated manner as time goes up. This inevitable accumulation of errors directly results in a dramatic decrease in the prediction effectiveness of the single frame input single frame output model over time.
In order to solve the disadvantages caused by the mode of inputting a single frame and outputting a single frame, research is conducted on a prediction model for inputting multiple frames and outputting multiple frames to predict the approach of precipitation, but the existing prediction models for inputting multiple frames and outputting multiple frames are not very effective, and the reason is that the models do not fully exploit the advantages of MIMO (Multi-In-Multi-Out, multiple-input multiple-output), such as that a Multi-attention module In a transform model utilized In Vit (An image is word 16x16ds. Therefore, it is difficult to achieve prediction requirements by performing prediction in this manner. Yet another model based on convolutional neural networks (SimVP: simpler Yet viewer Video prediction.) wants to predict only by convolutional neural networks. Since the convolution operation is only to perform feature extraction on the local part of the radar picture, the global feature information of the radar picture cannot be well grasped, and thus the effect of the prediction mode is inevitably poor.
Therefore, the existing rainfall approach prediction method is poor in prediction effect, and a rainfall approach prediction method with a better prediction effect is urgently needed.
Disclosure of Invention
The invention mainly aims to provide a precipitation approach prediction image generation method, a precipitation approach prediction image generation system, an electronic device and a storage medium, and aims to solve the technical problem that the existing precipitation approach prediction method is poor in prediction effect.
To achieve the above object, a first aspect of the present invention provides a method for generating a precipitation proximity prediction image, including: acquiring m frames of images to be predicted, which need rainfall approach prediction; down-sampling all the images to be predicted to obtain image characteristics, and performing time position coding on all the images to be predicted to obtain first coding information; coding all image characteristics and the first coding information to obtain second coding information; coding n time points of m frames of the image to be predicted, wherein the time T = [ m +1, m +2, … m + n ], so as to obtain third coding information; decoding the second coding information and the third coding information to obtain n decoded images, wherein the number of the decoded images is the same as that of the time points; and upsampling the decoded n images to obtain n frames of precipitation adjacent predicted images.
Further, the encoding all the image features and the first encoding information includes: extracting attention of the image features and the time position codes by using a pre-constructed two-dimensional multi-head attention mechanism to obtain first extraction information; normalizing the first extraction information, the image characteristics and the time position codes to obtain first normalized data; performing space-time correlation on the first normalized data in a pre-constructed local space-time correlation network to obtain time correlation information and space correlation information among different images to be predicted; and normalizing the time correlation information, the space correlation information and the first normalized data to obtain second coding information.
Further, the decoding the second encoded information and the third encoded information to obtain n decoded images includes: extracting attention of the third coded information by using a pre-constructed two-dimensional multi-head attention mechanism to obtain second extracted information; normalizing the second extraction information and the third coding information to obtain second normalized data; performing attention extraction on the second normalized data and the first coded information by using a pre-constructed two-dimensional multi-head attention mechanism to obtain third extracted information; normalizing the third extraction information and the second normalized data to obtain third normalized data; performing space-time correlation on the third normalized data in a pre-constructed local space-time correlation network to obtain time correlation information and space correlation information among predicted images; and normalizing the time correlation information and the space correlation information among the predicted images and the third normalized data to obtain n decoded images.
Further, the construction method of the two-dimensional multi-head attention mechanism comprises the following steps: the method comprises the steps that a multi-head attention mechanism of a transform model is constructed in advance, and a generating mode of query, key values and value items of the multi-head attention mechanism is replaced by a mode of generating the query, the key values and the value items by convolution; and replacing the linear operation of the attention weighting of the query, the key value and the value item of the multi-head attention mechanism by convolution operation to obtain the two-dimensional multi-head attention mechanism.
Further, the construction method of the two-dimensional multi-head attention mechanism further comprises the following steps: deleting a predictive occlusion mechanism in the multi-head attention mechanism.
Further, the method for constructing the local spatio-temporal correlation network comprises the following steps: the method comprises the steps of constructing a precursor network of a transform model in advance, replacing the precursor network with a three-dimensional convolutional neural network to obtain a local space-time correlation network, and performing convolution operation in a time field and a space field to keep the space correlation and the time correlation between pictures.
Further, the method further comprises: coding and integrating all image characteristics and time position codes into a coder, and replacing the coder in a pre-constructed transform model; and decoding the second coding information and the third coding information to obtain n decoded images, wherein the number of the decoded images is the same as that of the time points, and the n decoded images are integrated into a decoder and replace the decoder in a pre-constructed transform model.
A second aspect of the present invention provides a precipitation proximity prediction image generation system, including: the image acquisition module is used for acquiring m frames of images to be predicted, which need rainfall approach prediction; the image processing module is used for carrying out down-sampling on all the images to be predicted to obtain image characteristics and carrying out time position coding on all the images to be predicted to obtain first coding information; the encoder module is used for encoding all image characteristics and the first encoding information to obtain second encoding information; the time point coding module is used for coding n time points of time T = [ m +1, m +2, … m + n ] of the m frames of the image to be predicted to obtain third coding information; a decoder module, configured to decode the second encoded information and the third encoded information to obtain n decoded images, where the number of the decoded images is the same as the number of the time points; and the upsampling module is used for upsampling the decoded n images to obtain n frames of precipitation adjacent predicted images.
A third aspect of the present invention provides an electronic apparatus comprising: the rainfall adjacency prediction image generation method comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein when the processor executes the computer program, the rainfall adjacency prediction image generation method is realized by any one of the rainfall adjacency prediction image generation methods.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the precipitation proximity prediction image generation method according to any one of the above.
The invention provides a precipitation approach prediction image generation method, a precipitation approach prediction image generation system, an electronic device and a storage medium, and has the beneficial effects that: the method has the advantages that the defects of single-frame input and single-frame output are overcome by inputting m frames of images and outputting multi-frame input and multi-frame output of n frames of images, and the rapid reduction of the prediction effect along with the increase of time can not be caused; in addition, because the image features obtained by down-sampling have spatial features, the method and the system can better use the global feature information of the image to be predicted by combining the time features of n time points of m frames of images to be predicted, and overcome the defect that the existing multi-frame input multi-frame output can only carry out feature extraction on the local part of a radar picture and cannot well grasp the global feature information of the radar picture, so the method, the system, the electronic device and the storage medium for generating the rainfall approach predicted image have better prediction effect of rainfall approach prediction.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is also possible for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a flowchart of a method for generating a precipitation proximity prediction image according to an embodiment of the present invention;
FIG. 2 is a flow chart of a functional implementation of a method for generating a precipitation approach prediction image according to an embodiment of the present invention;
fig. 3 is a block diagram of a precipitation proximity predicted image generation system according to an embodiment of the present invention;
fig. 4 is a block diagram illustrating a structure of an electronic device according to an embodiment of the invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a method for generating a precipitation proximity prediction image includes:
s101, acquiring m frames of images to be predicted, which need rainfall approach prediction;
s102, performing down-sampling on all the images to be predicted to obtain image characteristics, and performing time position coding on all the images to be predicted to obtain first coding information;
s103, coding all image characteristics and the first coding information to obtain second coding information;
s104, coding n time points of time T = [ m +1, m +2, … m + n ] of m frames of images to be predicted to obtain third coding information;
s105, decoding the second coding information and the third coding information to obtain n decoded images, wherein the number of the decoded images is the same as that of the time points;
and S106, upsampling the decoded n images to obtain n frames of precipitation adjacent predicted images.
According to the method for generating the rainfall approach prediction image, the defect of single-frame input and single-frame output is overcome by inputting m-frame images and outputting multi-frame input and multi-frame output of n-frame images, and the prediction effect cannot be rapidly reduced along with the increase of time; in addition, because the image features obtained by down-sampling have spatial features, the method and the system can better use the global feature information of the image to be predicted by combining the time features of n time points of m frames of images to be predicted, and overcome the defect that the existing multi-frame input multi-frame output can only carry out feature extraction on the local part of a radar picture and cannot well grasp the global feature information of the radar picture, so the method, the system, the electronic device and the storage medium for generating the rainfall approach predicted image have better prediction effect of rainfall approach prediction.
In an embodiment, the execution of steps S101 to S106 may also use a structural framework of a transform model to optimize the transform model to realize the output of the precipitation adjacent prediction image, in this embodiment, the transform model is constructed first, and then all image features and time position codes are encoded and integrated into an encoder, and the encoder in the pre-constructed transform model is replaced; and decoding the second coding information and the third coding information to obtain n decoded images, integrating the n decoded images into a decoder with the same number of time points, and replacing the decoder in a pre-constructed transform model.
In one embodiment, encoding all of the image features and the first encoding information comprises:
extracting attention of image features and time position codes by using a pre-constructed two-dimensional multi-head attention mechanism to obtain first extraction information;
normalizing the first extraction information, the image characteristics and the time position codes to obtain first normalized data;
performing space-time correlation on the first normalized data in a pre-constructed local space-time correlation network to obtain time correlation information and space correlation information among different images to be predicted;
and normalizing the time correlation information and the space correlation information with the first normalized data to obtain second coding information.
In this embodiment, a two-dimensional multi-head attention mechanism, a local spatio-temporal correlation network, two normalization modules may form an encoder, which is an optimized version of a transform model encoder, in order to realize multiple inputs by taking advantage of the transform model.
In one embodiment, decoding the second encoded information and the third encoded information to obtain n decoded images comprises:
extracting attention of the third coded information by using a pre-constructed two-dimensional multi-head attention mechanism to obtain second extracted information;
normalizing the second extraction information and the third coding information to obtain second normalized data;
performing attention extraction on the second normalized data and the first coded information by using a pre-constructed two-dimensional multi-head attention mechanism to obtain third extracted information;
normalizing the third extraction information and the second normalized data to obtain third normalized data;
performing space-time correlation on the third normalized data in a pre-constructed local space-time correlation network to obtain time correlation information and space correlation information among predicted images;
and normalizing the time correlation information, the space correlation information and the third normalized data among the predicted images to obtain n decoded images.
In this embodiment, two-dimensional multi-head attention mechanism, a local spatio-temporal correlation network, and three normalization modules can constitute a decoder, which is a novel multi-output decoder designed to replace a transform model decoder in order to realize multi-output by using a transform structure.
Since the transform module has unique arrangement invariance, the present embodiment encodes n time points at time T = [ m +1,m +2, … m + n ] (m is the length of the input picture sequence, and n is the length of the output picture sequence), and the obtained code is input into the decoder, so that the predicted pictures at n time points from m +1 to m + n can be obtained at one time. The method is simple and rapid, and is well integrated with the 2 improved designs, so that the model has a good prediction effect.
In one embodiment, the two-dimensional multi-head attention mechanism construction method comprises the following steps:
a multi-head attention machine system of a transform model is constructed in advance, and a generation mode of query, key values and value items of the multi-head attention machine system is replaced by a mode of generating the query, the key values and the value items by convolution;
and replacing the linear operation of the attention weighting of the query, the key value and the value item of the multi-head attention mechanism by the convolution operation to obtain the two-dimensional multi-head attention mechanism.
In this embodiment, a multi-head attention Mechanism (MHA) in a conventional transform is changed, a manner of generating Q (Query), K (Key, key Value), and V (Value item) is changed, specifically, Q, K, and V are obtained by a convolution manner, and then all linear operations of three for performing attention weighting are replaced by convolution operations. Experiments show that the prediction effect of the mode is obviously improved.
In one embodiment, the method for constructing the two-dimensional multi-head attention mechanism further comprises: the predictive occlusion mechanism in the multi-head attention mechanism is deleted.
In this embodiment, the occlusion mechanism (mask) in the multi-head attention mechanism in the conventional transform is used to predict occlusion, but since the time point is added in this embodiment and is predicted from Embedding (Embedding) of the time point, deleting the mask will obtain better prediction effect because the occlusion mechanism does not need to be predicted.
In one embodiment, the method for constructing the local space-time correlation network comprises the following steps:
the method comprises the steps of constructing a precursor network of a transform model in advance, replacing the precursor network with a three-dimensional convolutional neural network to obtain a local space-time correlation network, and performing convolution operation in the time field and the space field to keep the space correlation and the time correlation between pictures.
In this embodiment, since the FFN (forward Network) in the conventional transform model only has a full connection layer, and many correlations are lost, a three-dimensional convolutional neural Network is used for model prediction. By performing convolution operation in the time domain and the space domain at the same time, the model can not only keep the spatial correlation between the picture sequences, but also keep the temporal correlation between different pictures. Experiments prove that the stronger space-time correlation brings better prediction effect.
After inputting M frames of images to be detected, down-sampling the images to be detected to obtain picture characteristics, encoding the pictures in time positions, inputting the picture characteristics and the time position codes into an encoder, performing two-dimensional multi-head attention extraction and normalization, then inputting the picture characteristics and the time position codes into a local space-time correlation module, performing normalization, outputting M pieces of image encoding information to a decoder, encoding N time points at time T = [ M +1, M +2, … M + N ], inputting the time point codes into the decoder, performing two-dimensional multi-head attention extraction and normalization on the time point codes by the decoder, performing two-dimensional multi-head attention extraction on the normalized data and the image encoding information of M, performing normalization, inputting the normalized data into the local space-time correlation module, performing normalization again, outputting N pieces of decoded data by the decoder, performing up-sampling, and outputting N frames of precipitation adjacent predicted images.
Referring to fig. 3, an embodiment of the present application further provides a system for generating a precipitation proximity prediction image, including: the device comprises an image acquisition module 1, an image processing module 2, an encoder module 3, a time point encoding module 4, a decoder module 5 and an up-sampling module 6; the image acquisition module 1 is used for acquiring m frames of images to be predicted which need rainfall approach prediction; the image processing module 2 is used for performing down-sampling on all the images to be predicted to obtain image characteristics, and performing time position coding on all the images to be predicted to obtain first coding information; the encoder module 3 is configured to encode all image features and the first encoding information to obtain second encoding information; the time point coding module 4 is configured to code n time points of time T = [ m +1, m +2, … m + n ] of m frames of images to be predicted, so as to obtain third coding information; the decoder module 5 is configured to decode the second encoded information and the third encoded information to obtain n decoded images, where the number of the decoded images is the same as the number of time points; the up-sampling module 6 is configured to up-sample the decoded n images to obtain n frames of precipitation adjacent predicted images.
According to the rainfall approach prediction image generation system provided by the embodiment, the defect of single-frame input and single-frame output is overcome by inputting m-frame images and outputting a multi-frame input and multi-frame output mode of n-frame images, and the prediction effect cannot be rapidly reduced along with the increase of time; in addition, because the image features obtained by down-sampling have spatial features, the method and the system can better use the global feature information of the image to be predicted by combining the time features of n time points of m frames of images to be predicted, and overcome the defect that the existing multi-frame input multi-frame output can only carry out feature extraction on the local part of a radar picture and cannot well grasp the global feature information of the radar picture, so the method, the system, the electronic device and the storage medium for generating the rainfall approach predicted image have better prediction effect of rainfall approach prediction.
In one embodiment, the encoder module 3 comprises: the device comprises a first two-dimensional multi-head attention unit, a first normalization unit, a first local space-time correlation unit and a second normalization unit; the first two-dimensional multi-head attention unit is used for extracting attention of image features and time position codes by using a pre-constructed two-dimensional multi-head attention mechanism to obtain first extraction information; the first normalization unit is used for performing normalization processing on the first extraction information, the image characteristics and the time position codes to obtain first normalization data; the first local space-time correlation unit is used for performing space-time correlation on the first normalized data in a pre-constructed local space-time correlation network to obtain time correlation information and space correlation information among different images to be predicted; the second normalization unit is used for normalizing the time correlation information and the space correlation information with the first normalization data to obtain second coding information.
In one embodiment, the decoder module 5 comprises: the device comprises a first two-dimensional multi-head attention unit, a first normalization unit, a second two-dimensional multi-head attention unit, a third two-dimensional multi-head attention unit, a fourth normalization unit, a second local space-time correlation unit and a fifth normalization unit; the second two-dimensional multi-head attention unit is used for extracting attention of the third coded information by using a pre-constructed two-dimensional multi-head attention mechanism to obtain second extracted information; the third normalization unit is used for performing normalization processing on the second extraction information and the third coding information to obtain second normalization data; the third two-dimensional multi-head attention unit is used for extracting attention of the second normalized data and the first coded information by using a pre-constructed two-dimensional multi-head attention mechanism to obtain third extracted information; the fourth normalization unit is used for performing normalization processing on the third extraction information and the second normalization data to obtain third normalization data; the second local space-time correlation unit is used for performing space-time correlation on the third normalized data in a pre-constructed local space-time correlation network to obtain time correlation information and space correlation information among predicted images; and the fifth normalization unit is used for performing normalization processing on the time correlation information and the space correlation information among the predicted images and the third normalization data to obtain n decoded images.
In one embodiment, the system for generating a precipitation proximity prediction image further comprises a building module for building a two-dimensional multi-headed attention unit in the above embodiment, the building module comprising: the multi-head attention mechanism building unit is used for building a multi-head attention mechanism of the transform model in advance; and the replacing unit is used for replacing the generation mode of the query, the key value and the value item of the multi-head attention machine system with the generation mode of the query, the key value and the value item by convolution, and replacing the linear operation of the query, the key value and the value item of the multi-head attention machine system for carrying out attention weighting with the convolution operation to obtain the two-dimensional multi-head attention machine system unit.
In one embodiment, the building module further comprises a deletion unit for deleting a predictive occlusion mechanism in the multi-head attention mechanism.
In an embodiment, the building module further includes a local spatio-temporal correlation network building unit, configured to pre-build a precursor network of the transform model, and replace the precursor network with a three-dimensional convolutional neural network to obtain a local spatio-temporal correlation network, which is used to perform convolution operations in the time domain and the space domain to preserve spatial correlation and temporal correlation between pictures.
In one embodiment, the system for generating a precipitation imminent prediction image further comprises a replacement module for replacing the encoder module with an encoder in a pre-constructed transform model; and replacing the decoder module with the decoder in the pre-constructed transform model.
An embodiment of the present application provides an electronic device, referring to fig. 4, the electronic device includes: a memory 601, a processor 602, and a computer program stored in the memory 601 and executable on the processor 602, the processor 602 implementing the method for generating a precipitation-imminent prediction image described above when executing the computer program.
Further, the electronic device further includes: at least one input device 603, and at least one output device 604.
The memory 601, the processor 602, the input device 603, and the output device 604 are connected by a bus 605.
The input device 603 may be a camera, a touch panel, a physical button, a mouse, or the like. The output device 604 may be embodied as a display screen.
The Memory 601 may be a high-speed Random Access Memory (RAM) Memory, or a non-volatile Memory (non-volatile Memory), such as a disk Memory. The memory 601 is used for storing a set of executable program code, and the processor 602 is coupled to the memory 601.
Further, an embodiment of the present application also provides a computer-readable storage medium, which may be disposed in the electronic device in the foregoing embodiments, and the computer-readable storage medium may be the memory 601 in the foregoing. The computer-readable storage medium has stored thereon a computer program which, when executed by the processor 602, implements the precipitation imminent prediction image generation method described in the foregoing embodiments.
Further, the computer-readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory 601 (ROM), a RAM, a magnetic disk, or an optical disk, and various media that can store program codes.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present invention is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no acts or modules are necessarily required of the invention.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the above description, for the method, the system, the electronic device and the storage medium for generating a precipitation proximity prediction image provided by the present invention, for those skilled in the art, there may be variations in the specific implementation and the application scope according to the idea of the embodiment of the present invention, and in summary, the content of the present specification should not be construed as limiting the present invention.
Claims (10)
1. A method for generating a precipitation proximity prediction image, comprising:
acquiring m frames of images to be predicted, which need rainfall approach prediction;
down-sampling all the images to be predicted to obtain image characteristics, and performing time position coding on all the images to be predicted to obtain first coding information;
coding all image characteristics and the first coding information to obtain second coding information;
coding n time points of time T = [ m +1, m +2, … m + n ] of m frames of the image to be predicted to obtain third coding information;
decoding the second coding information and the third coding information to obtain n decoded images, wherein the number of the decoded images is the same as that of the time points;
and upsampling the decoded n images to obtain n frames of precipitation adjacent predicted images.
2. The method of generating a precipitation-imminent prediction image according to claim 1,
the encoding all the image features and the first encoding information includes:
extracting attention of the image features and the time position codes by using a pre-constructed two-dimensional multi-head attention mechanism to obtain first extraction information;
normalizing the first extraction information, the image characteristics and the time position codes to obtain first normalized data;
performing space-time correlation on the first normalized data in a pre-constructed local space-time correlation network to obtain time correlation information and space correlation information among different images to be predicted;
and normalizing the time correlation information, the space correlation information and the first normalized data to obtain second coding information.
3. The method of generating a precipitation-imminent prediction image according to claim 2,
the decoding the second encoded information and the third encoded information to obtain n decoded images includes:
extracting attention of the third coded information by using a pre-constructed two-dimensional multi-head attention mechanism to obtain second extracted information;
normalizing the second extraction information and the third coding information to obtain second normalized data;
performing attention extraction on the second normalized data and the first coded information by using a pre-constructed two-dimensional multi-head attention mechanism to obtain third extracted information;
normalizing the third extraction information and the second normalized data to obtain third normalized data;
performing space-time correlation on the third normalized data in a pre-constructed local space-time correlation network to obtain time correlation information and space correlation information among predicted images;
and normalizing the time correlation information and the space correlation information among the predicted images and the third normalized data to obtain n decoded images.
4. The method for generating a precipitation-adjacency prediction image according to any one of claims 2 and 3, wherein the method for constructing the two-dimensional multi-head attention mechanism comprises:
the method comprises the steps that a multi-head attention mechanism of a transform model is constructed in advance, and a generating mode of query, key values and value items of the multi-head attention mechanism is replaced by a mode of generating the query, the key values and the value items by convolution;
and replacing the linear operation of the attention weighting of the query, the key value and the value item of the multi-head attention mechanism by convolution operation to obtain the two-dimensional multi-head attention mechanism.
5. The method of generating a precipitation-imminent prediction image according to claim 4,
the construction method of the two-dimensional multi-head attention mechanism further comprises the following steps:
deleting a predictive occlusion mechanism in the multi-head attention mechanism.
6. The method for generating a precipitation-imminent-prediction image according to any one of claims 2 and 3, wherein the method for constructing the local spatiotemporal correlation network comprises:
the method comprises the steps of constructing a precursor network of a transform model in advance, replacing the precursor network with a three-dimensional convolutional neural network to obtain a local space-time correlation network, and performing convolution operation in the time field and the space field to keep the space correlation and the time correlation between pictures.
7. The method of generating a precipitation imminent prediction image according to claim 1,
the method further comprises the following steps:
coding and integrating all image characteristics and time position codes into a coder, and replacing the coder in a pre-constructed transform model;
and decoding the second coding information and the third coding information to obtain n decoded images, wherein the number of the images obtained by decoding is the same as that of the time points, and the images are integrated into a decoder to replace the decoder in a pre-constructed transform model.
8. A system for generating a precipitation proximity prediction image, comprising:
the image acquisition module is used for acquiring m frames of images to be predicted, which need rainfall approaching prediction;
the image processing module is used for carrying out down-sampling on all the images to be predicted to obtain image characteristics and carrying out time position coding on all the images to be predicted to obtain first coding information;
the encoder module is used for encoding all image characteristics and the first encoding information to obtain second encoding information;
the time point coding module is used for coding n time points of time T = [ m +1, m +2, … m + n ] of the m frames of the image to be predicted to obtain third coding information;
the decoder module is used for decoding the second coding information and the third coding information to obtain n decoded images, and the number of the decoded images is the same as that of the time points;
and the up-sampling module is used for up-sampling the decoded n images to obtain n frames of precipitation adjacent predicted images.
9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211559910.4A CN115830428A (en) | 2022-12-06 | 2022-12-06 | Precipitation approach prediction image generation method and system, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211559910.4A CN115830428A (en) | 2022-12-06 | 2022-12-06 | Precipitation approach prediction image generation method and system, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115830428A true CN115830428A (en) | 2023-03-21 |
Family
ID=85545272
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211559910.4A Pending CN115830428A (en) | 2022-12-06 | 2022-12-06 | Precipitation approach prediction image generation method and system, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115830428A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116469013A (en) * | 2023-06-20 | 2023-07-21 | 云途信息科技(杭州)有限公司 | Road ponding prediction method, device, computer equipment and storage medium |
-
2022
- 2022-12-06 CN CN202211559910.4A patent/CN115830428A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116469013A (en) * | 2023-06-20 | 2023-07-21 | 云途信息科技(杭州)有限公司 | Road ponding prediction method, device, computer equipment and storage medium |
| CN116469013B (en) * | 2023-06-20 | 2023-09-08 | 云途信息科技(杭州)有限公司 | Road ponding prediction method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110059772B (en) | Remote sensing image semantic segmentation method based on multi-scale decoding network | |
| CN111026915B (en) | Video classification method, video classification device, storage medium and electronic equipment | |
| CN104885470B (en) | It is divided for the prediction of next-generation video and the content-adaptive of coding | |
| CN111263161B (en) | Video compression processing method and device, storage medium and electronic equipment | |
| CN101622874A (en) | Object archival systems and methods | |
| Tang et al. | Joint graph attention and asymmetric convolutional neural network for deep image compression | |
| Xia et al. | An emerging coding paradigm VCM: A scalable coding approach beyond feature and signal | |
| CN115830428A (en) | Precipitation approach prediction image generation method and system, electronic device and storage medium | |
| CN113556496B (en) | Video resolution improving method and device, storage medium and electronic equipment | |
| CN115861131A (en) | Training method and device based on image generation video and model and electronic equipment | |
| CN112584158B (en) | Video quality enhancement method and system | |
| WO2023077707A1 (en) | Video encoding method, model training method, device, and storage medium | |
| CN116797975A (en) | Video segmentation method, device, computer equipment and storage medium | |
| CN113658073B (en) | Image denoising processing method and device, storage medium and electronic equipment | |
| KR20220043912A (en) | Method and Apparatus for Coding Feature Map Based on Deep Learning in Multitasking System for Machine Vision | |
| CN112991274B (en) | Crowd counting method and device, computer equipment and storage medium | |
| CN101288314A (en) | Dynamic image encoding device, dynamic image decoding device, dynamic image encoding method, dynamic image decoding method, dynamic image encoding program, and dynamic image decoding program | |
| CN104104961A (en) | Video encoding method, decoding method and apparatus | |
| CN115953842A (en) | Video motion recognition method and device, electronic equipment and readable medium | |
| CN114554220B (en) | Fixed scene video overrun compression and decoding method based on abstract features | |
| CN113177483B (en) | Video object segmentation method, device, equipment and storage medium | |
| EP4049453A1 (en) | Video encoding and video decoding | |
| KR20230129061A (en) | Video encoding based on feature extraction and image synthesis | |
| CN113627341B (en) | Video sample comparison method, system, equipment and storage medium | |
| KR100606357B1 (en) | Method of decomposing and composing multimedia data based on discrete wavelet transformation and system for decomposing and composing the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |