CN111159310A - Extensible image generation space-time fusion algorithm with information gain strategy - Google Patents

Extensible image generation space-time fusion algorithm with information gain strategy Download PDF

Info

Publication number
CN111159310A
CN111159310A CN201911280551.7A CN201911280551A CN111159310A CN 111159310 A CN111159310 A CN 111159310A CN 201911280551 A CN201911280551 A CN 201911280551A CN 111159310 A CN111159310 A CN 111159310A
Authority
CN
China
Prior art keywords
image
information
time
low
resolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911280551.7A
Other languages
Chinese (zh)
Other versions
CN111159310B (en
Inventor
王力哲
冯如意
陈佳
韩伟
陈小岛
阎继宁
宋维静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Geosciences
Original Assignee
China University of Geosciences
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Geosciences filed Critical China University of Geosciences
Priority to CN201911280551.7A priority Critical patent/CN111159310B/en
Publication of CN111159310A publication Critical patent/CN111159310A/en
Application granted granted Critical
Publication of CN111159310B publication Critical patent/CN111159310B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/283Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/26Visual data mining; Browsing structured data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/951Indexing; Web crawling techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/955Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an extensible image generation space-time fusion algorithm with an information gain strategy, which comprises the following steps: by usingCycle‑GANSimulating the time sequence of the image, obtaining a plurality of available data sets, and generating a multi-level iterative image; selecting the multi-level iteration image with the image similar to the image in prediction according to the reference information; obtaining gain information of the multi-stage iteration image, and sending the gain information to a prediction moment with spatial informationkA low resolution image of (a); and obtaining image information selected by wavelet transformation, and predicting spatial information in space-time fusion.Cycle‑GANThe idea of simulating the time sequence process and the method of counterstudy are helpful to the time sequenceThe high-resolution image makes reasonable prediction and the generated image is beneficial to generating the image containing more gain information which is not contained in the low-resolution image, thereby providing help for introducing new gain information for space-time fusion.

Description

Extensible image generation space-time fusion algorithm with information gain strategy
Technical Field
The invention relates to the technical field of image generation space-time fusion algorithms, in particular to an extensible image generation space-time fusion algorithm with an information gain strategy.
Background
Due to the cost, technical limitation and different tasks, most of the current satellite sensors obtain single remote sensing images with higher resolution. Which cannot meet the requirements of practical applications. The time-space fusion can utilize the complementarity of information among different remote sensing data to the maximum extent, so that the remote sensing data with different spatial resolutions and time resolutions can finally obtain a fusion image with higher time and spatial resolutions by a fusion technical means. However, in practical applications, because of a great difference in spatial resolution between high-resolution images and low-resolution images, for example, one MODIS pixel corresponds to hundreds of pixels of the Landsat image, which means that the MODIS image contains less than 1% of spatial information of the Landsat image in the same scene, and the conventional methods such as interpolation cannot embody enough spatial information, but too little known information is not enough to represent the correspondence between the high-resolution images and the low-resolution images, which results in lower fusion accuracy.
Disclosure of Invention
Aiming at the technical problems in the related art, the invention provides an extensible image generation space-time fusion algorithm with an information gain strategy, which can overcome the defects in the prior art.
In order to achieve the technical purpose, the technical scheme of the invention is realized as follows:
an extensible image generation spatio-temporal fusion algorithm with an information gain strategy, the method comprising the steps of:
s1: acquiring a plurality of available data sets by adopting a Cycle-GAN analog image time sequence to generate a multi-level iterative image;
s2: selecting the multi-level iteration image with the image similar to the image in prediction according to the reference information;
s3: acquiring gain information of the multistage iteration image, and sending the gain information to a low-resolution image with spatial information at a prediction time k;
s4: and obtaining image information selected by wavelet transformation, and predicting spatial information in space-time fusion.
Further, the step S1 includes the following steps:
s11: setting iteration times, and acquiring image information of two moments k-1 and k +1 before and after a predicted moment k;
s12: setting images at the moments k-1 and k +1 as training samples of Cycle-GAN, and selecting one moment as a generated sample to generate an image;
s13: and iteratively obtaining a plurality of remote sensing images.
Further, the step S2 is a step of:
s21: calculating and predicting a low-resolution image at the moment k;
s22: calculating and predicting mutual information of low-resolution images at the time k and low-resolution images at the time k-1
Figure BDA0002316630010000029
S23: calculating and predicting mutual information of the low-resolution image at the time k and the low-resolution image at the time k +1
Figure BDA0002316630010000028
S24: calculating and predicting a high-resolution image at the time k;
s25: calculating and predicting mutual information of high-resolution image at time k and high-resolution image at time k-1
Figure BDA0002316630010000021
S26: calculating and predicting mutual information of the high-resolution image at the moment k and the high-resolution image at the moment k +1
Figure BDA0002316630010000022
S27: obtaining mutual information in range
Figure BDA0002316630010000023
And
Figure BDA0002316630010000024
where λ represents the correspondence of mutual information between high and low resolutions, and may be an empirical value;
s28: acquiring the image with the maximum information entropy in the selected images, and setting the image as the selected generated image
Figure BDA0002316630010000025
Further, the step S3 includes the following steps:
s31: acquiring an image
Figure BDA0002316630010000026
Information and low resolution image information, for images
Figure BDA0002316630010000027
Performing wavelet transform two-level decomposition on the low-resolution image;
s32: obtaining and reconstructing images
Figure BDA0002316630010000031
Second-order decomposition information and low-resolution image second-order decomposition information.
The invention has the beneficial effects that: by using the method, on one hand, a new thought is provided for space-time fusion for predicting the spatial information, and a new solution is provided for the problem that the corresponding relation is difficult to process due to overlarge information difference between high and low spatial resolutions;
on the other hand, the idea of the Cycle-GAN simulation time sequence process and the method for counterstudy are beneficial to making reasonable prediction on the time sequence high-resolution images and generating images, the generated images contain more gain information which is not contained in the low-resolution images, and help is provided for introducing new gain information for space-time fusion.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flow chart of steps of an extensible image generation spatiotemporal fusion algorithm with information gain strategy according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
As shown in FIG. 1, the scalable image generation spatiotemporal fusion algorithm with information gain strategy according to the embodiment of the present invention comprises the following steps:
step S1, acquiring a plurality of available data sets by adopting the time sequence of the Cycle-GAN simulation image, and generating a multi-stage iterative image;
step S2, selecting the multi-stage iteration image with the image similar to the image in the prediction process according to the reference information;
step S3, obtaining gain information of the multi-stage iteration image, and sending the gain information to a low-resolution image with spatial information at a prediction time k;
and step S4, acquiring image information selected by wavelet transformation, and predicting spatial information in space-time fusion.
Step S1 includes the following steps:
step S11, setting iteration times, and acquiring image information of two moments k-1 and k +1 before and after a predicted moment k;
step S12, setting the images at the time k-1 and k +1 as Cycle-GAN training samples, and selecting one of the time as a generated sample generation image;
and step S13, iteratively acquiring a plurality of remote sensing images.
Step S2 includes the following steps:
step S21, calculating and predicting a low-resolution image at time k;
step S22, calculating and predicting mutual information between the low resolution image at time k and the low resolution image at time k-1
Figure BDA0002316630010000048
Step S23, mutual information between the low resolution image at time k and the low resolution image at time k +1 is calculated and predicted
Figure BDA0002316630010000049
Step S24, calculating and predicting a high-resolution image at time k;
step S25, calculating and predicting mutual information between the high-resolution image at time k and the high-resolution image at time k-1
Figure BDA0002316630010000041
Step S26, calculating and predicting mutual information between the high-resolution image at time k and the high-resolution image at time k +1
Figure BDA0002316630010000042
Step S27, obtaining mutual information in range
Figure BDA0002316630010000043
And
Figure BDA0002316630010000044
where λ represents the correspondence of mutual information between high and low resolutions, and may be an empirical value;
acquiring the image with the maximum information entropy in the selected images, and setting the image as the selected generated image
Figure BDA0002316630010000045
Step S3 includes the following steps:
step S31, acquiring image
Figure BDA0002316630010000046
Information and low resolution image information, for images
Figure BDA0002316630010000047
Performing wavelet transform two-level decomposition on the low-resolution image;
step S32, acquiring and recombining the image
Figure BDA0002316630010000051
Second-order decomposition information and low-resolution image second-order decomposition information.
In order to facilitate understanding of the above-described technical aspects of the present invention, the above-described technical aspects of the present invention will be described in detail below in terms of specific usage.
The overall architecture process of the multi-source remote sensing image data multi-dimensional organization management system based on cloud storage comprises the following modules and processes:
1) the multi-center remote sensing data retrieval module comprises a plurality of parts such as a distributed data center ftp service, a sub-center crawler, metadata mapping and a solr created index. Metadata is integrated into a main center through a sub-center crawler, and a metadata index is created through solr to provide a query retrieval service for multi-center remote sensing metadata for a user.
2) The data is subscribed by a multi-dimensional subscription module, a user subscribes data required by the user through a multi-center remote sensing data retrieval module, the file is transmitted to the shared cloud storage through ftp service provided by the sub-center, the reliability of data storage is guaranteed, and then the view of the user is defined by selecting a view generation rule. Therefore, multi-dimensional organization of the multi-center remote sensing data is realized.
3) The data view sharing module is used for enabling a user to generate a view and simultaneously store the view structure of the user in the user virtual directory database, and then calling the user virtual directory database to share the virtual directory structure of the user to other users, so that movement of real data is avoided, and multi-dimensional organization and management of other multiple users on remote sensing data are facilitated.
When a specific user organizes and manages data in cloud storage, on the basis of OpenStack Swift object storage, the user constructs a sharing code for view data, and shares a virtual directory structure to other users. The remote sensing data resource can be accessed by a user through the HTTP protocol through the Restful API of Swift.
The cloud storage multidimensional data organization module is mainly divided into a front-end visualization layer and provides data view generation service, view sharing service and view data downloading service for users. The intermediate service logic module is mainly responsible for processing basic functions of data import and export cloud storage and generating functions of a data view logic structure, and the bottom layer is mainly provided with persistent storage support by OpenStack object storage and Mysql deployed by a cluster.
When the method is used specifically, a user searches the archived data of each sub-center, selects data subscription, simultaneously leads the subscribed data from the sub-centers into the main center for cloud storage, can select view construction rules to generate a custom view, persists the view to the Mysql database, and can share the data view and download links with other users through the view sharing function and the data downloading function of the cloud storage to finish data preparation.
According to the introduction, the multi-source remote sensing image data multi-dimensional organization method based on cloud storage is designed and completed, and the method can be suitable for remote sensing application needing multi-dimensional representation and analysis of large-scale remote sensing data. The method utilizes a unified management flow, user management based on a cloud platform, automatic multi-dimensional data view construction and virtual data view sharing and downloading based on cloud storage, greatly improves the data storage efficiency and simplifies the data preparation flow required by subsequent data analysis.
In summary, by means of the above technical solution of the present invention, through the use of the method, on one hand, a new idea is provided for the prediction of spatial information by space-time fusion, and a new solution is provided for the problem of processing difficulty of correspondence caused by too large information difference between high and low spatial resolutions; on the other hand, the idea of the Cycle-GAN simulation time sequence process and the method for counterstudy are beneficial to making reasonable prediction on the time sequence high-resolution images and generating images, the generated images contain more gain information which is not contained in the low-resolution images, and help is provided for introducing new gain information for space-time fusion.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (4)

1. An extensible image generation spatio-temporal fusion algorithm with an information gain strategy, characterized by comprising the following steps:
s1: acquiring a plurality of available data sets by adopting a Cycle-GAN analog image time sequence to generate a multi-level iterative image;
s2: selecting the multi-level iteration image with the image similar to the image in prediction according to the reference information;
s3: acquiring gain information of the multistage iteration image, and sending the gain information to a low-resolution image with spatial information at a prediction time k;
s4: and obtaining image information selected by wavelet transformation, and predicting spatial information in space-time fusion.
2. The scalable, information-gain-strategic image-generation spatio-temporal fusion algorithm of claim 1, wherein said step S1 comprises the steps of:
s11: setting iteration times, and acquiring image information of two moments k-1 and k +1 before and after a predicted moment k;
s12: setting images at the moments k-1 and k +1 as training samples of Cycle-GAN, and selecting one moment as a generated sample to generate an image;
s13: and iteratively obtaining a plurality of remote sensing images.
3. The scalable, information-gain-strategic image-generation spatio-temporal fusion algorithm of claim 1, wherein said step S2 comprises the steps of:
s21: calculating and predicting a low-resolution image at the moment k;
s22: calculating and predicting mutual information of low-resolution images at the time k and low-resolution images at the time k-1
Figure FDA0002316629000000011
S23: calculating and predicting mutual information of the low-resolution image at the time k and the low-resolution image at the time k +1
Figure FDA0002316629000000012
S24: calculating and predicting a high-resolution image at the time k;
s25: calculating and predicting mutual information of high-resolution image at time k and high-resolution image at time k-1
Figure FDA0002316629000000013
S26: calculating and predicting mutual information of the high-resolution image at the moment k and the high-resolution image at the moment k +1
Figure FDA0002316629000000021
S27: obtaining mutual interactionInformation is in range
Figure FDA0002316629000000022
And
Figure FDA0002316629000000023
where λ represents the correspondence of mutual information between high and low resolutions, and may be an empirical value;
s28: acquiring the image with the maximum information entropy in the selected images, and setting the image as a selected generated image Fk GAN
4. The scalable, information-gain-strategic image-generation spatio-temporal fusion algorithm of claim 1, wherein said step S3 comprises the steps of:
s31: acquiring an image Fk GANInformation and low resolution image information, for image Fk GANPerforming wavelet transform two-level decomposition on the low-resolution image;
s32: obtaining and reconstructing an image Fk GANSecond-order decomposition information and low-resolution image second-order decomposition information.
CN201911280551.7A 2019-12-13 2019-12-13 Extensible image generation space-time fusion method with information gain strategy Active CN111159310B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911280551.7A CN111159310B (en) 2019-12-13 2019-12-13 Extensible image generation space-time fusion method with information gain strategy

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911280551.7A CN111159310B (en) 2019-12-13 2019-12-13 Extensible image generation space-time fusion method with information gain strategy

Publications (2)

Publication Number Publication Date
CN111159310A true CN111159310A (en) 2020-05-15
CN111159310B CN111159310B (en) 2023-09-29

Family

ID=70557063

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911280551.7A Active CN111159310B (en) 2019-12-13 2019-12-13 Extensible image generation space-time fusion method with information gain strategy

Country Status (1)

Country Link
CN (1) CN111159310B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090226114A1 (en) * 2008-03-07 2009-09-10 Korea Aerospace Research Institute Satellite image fusion method and system
CN102915529A (en) * 2012-10-15 2013-02-06 黄波 Integrated fusion technique and system based on remote sensing of time, space, spectrum and angle
CN105184076A (en) * 2015-09-02 2015-12-23 安徽大学 Space-time integrated fusion method for remote sensing earth surface temperature data
WO2017219263A1 (en) * 2016-06-22 2017-12-28 中国科学院自动化研究所 Image super-resolution enhancement method based on bidirectional recursion convolution neural network
US20180137603A1 (en) * 2016-11-07 2018-05-17 Umbo Cv Inc. Method and system for providing high resolution image through super-resolution reconstruction
US20190004533A1 (en) * 2017-07-03 2019-01-03 Baidu Usa Llc High resolution 3d point clouds generation from downsampled low resolution lidar 3d point clouds and camera images
EP3564903A1 (en) * 2018-05-01 2019-11-06 Koninklijke Philips N.V. Lower to higher resolution image fusion
CN110532897A (en) * 2019-08-07 2019-12-03 北京科技大学 The method and apparatus of components image recognition

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090226114A1 (en) * 2008-03-07 2009-09-10 Korea Aerospace Research Institute Satellite image fusion method and system
CN102915529A (en) * 2012-10-15 2013-02-06 黄波 Integrated fusion technique and system based on remote sensing of time, space, spectrum and angle
CN105184076A (en) * 2015-09-02 2015-12-23 安徽大学 Space-time integrated fusion method for remote sensing earth surface temperature data
WO2017219263A1 (en) * 2016-06-22 2017-12-28 中国科学院自动化研究所 Image super-resolution enhancement method based on bidirectional recursion convolution neural network
US20180137603A1 (en) * 2016-11-07 2018-05-17 Umbo Cv Inc. Method and system for providing high resolution image through super-resolution reconstruction
US20190004533A1 (en) * 2017-07-03 2019-01-03 Baidu Usa Llc High resolution 3d point clouds generation from downsampled low resolution lidar 3d point clouds and camera images
EP3564903A1 (en) * 2018-05-01 2019-11-06 Koninklijke Philips N.V. Lower to higher resolution image fusion
CN110532897A (en) * 2019-08-07 2019-12-03 北京科技大学 The method and apparatus of components image recognition

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CHEN B等: "Comparison of Spatiotemporal Fusion Models:A Review", vol. 7, no. 2, pages 1798 - 1835 *
黄波;赵涌泉;: "多源卫星遥感影像时空融合研究的现状及展望", no. 10 *

Also Published As

Publication number Publication date
CN111159310B (en) 2023-09-29

Similar Documents

Publication Publication Date Title
US20230252724A1 (en) Point cloud data hierarchy
CN106909644B (en) A kind of multistage tissue and indexing means towards mass remote sensing image
Planthaber et al. EarthDB: scalable analysis of MODIS data using SciDB
US10169455B2 (en) Systems and methods for addressing a media database using distance associative hashing
WO2019069304A1 (en) System and method for compact and efficient sparse neural networks
CN102509022B (en) Method for quickly constructing raster database facing to Virtual Earth
CN105786942A (en) Geographic information storage system based on cloud platform
WO2015141463A1 (en) Method for processing input low-resolution (lr) image to output high-resolution (hr) image
US20160299910A1 (en) Method and system for querying and visualizing satellite data
US11860674B1 (en) Query system
CN109657080B (en) Distributed processing method/system and medium for high-resolution satellite remote sensing data
CN107480720B (en) Human body posture model training method and device
CN110569972A (en) search space construction method and device of hyper network and electronic equipment
Guo et al. A spatially adaptive decomposition approach for parallel vector data visualization of polylines and polygons
Pokorný et al. Big data movement: a challenge in data processing
US20220156291A1 (en) Geospatial data analytics and visualization platform
US12019710B2 (en) Managing and streaming a plurality of large-scale datasets
CN111159310A (en) Extensible image generation space-time fusion algorithm with information gain strategy
CN110110107A (en) A kind of Methods on Multi-Sensors RS Image various dimensions method for organizing based on cloud storage
Mitra et al. Glance: A generative approach to interactive visualization of voluminous satellite imagery
CN115662346B (en) Demura compensation value compression method and system
US20200195964A1 (en) Electronic circuit and electronic device performing motion estimation through hierarchical search
Albrecht et al. Pairs (Re) loaded: system design & benchmarking for scalable geospatial applications
Happ et al. Towards distributed region growing image segmentation based on MapReduce
Alkathiri et al. Kluster: Application of k-means clustering to multidimensional GEO-spatial data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant