CN114610938A - Remote sensing image retrieval method and device, electronic equipment and computer readable medium - Google Patents

Remote sensing image retrieval method and device, electronic equipment and computer readable medium Download PDF

Info

Publication number
CN114610938A
CN114610938A CN202210226180.XA CN202210226180A CN114610938A CN 114610938 A CN114610938 A CN 114610938A CN 202210226180 A CN202210226180 A CN 202210226180A CN 114610938 A CN114610938 A CN 114610938A
Authority
CN
China
Prior art keywords
image
remote sensing
scale
features
processed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210226180.XA
Other languages
Chinese (zh)
Inventor
武昊
张俊
侯东阳
蔡彩
王思远
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NATIONAL GEOMATICS CENTER OF CHINA
Original Assignee
NATIONAL GEOMATICS CENTER OF CHINA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NATIONAL GEOMATICS CENTER OF CHINA filed Critical NATIONAL GEOMATICS CENTER OF CHINA
Priority to CN202210226180.XA priority Critical patent/CN114610938A/en
Publication of CN114610938A publication Critical patent/CN114610938A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/55—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Library & Information Science (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the disclosure discloses a remote sensing image retrieval method, a remote sensing image retrieval device, electronic equipment and a computer readable medium. The specific implementation mode of the method comprises the following steps: receiving a remote sensing image to be processed; importing the remote sensing image to be processed into a pre-trained multi-scale image recognition model to obtain target image features, wherein the multi-scale image recognition model is used for acquiring image features of the remote sensing image under a preset scale and fusing the image features under the preset scale into the target image features; and matching a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics. The method and the device improve the accuracy and effectiveness of obtaining the target image corresponding to the remote sensing image to be processed.

Description

Remote sensing image retrieval method and device, electronic equipment and computer readable medium
Technical Field
The embodiment of the disclosure relates to the technical field of remote sensing, in particular to a remote sensing image retrieval method, a remote sensing image retrieval device, electronic equipment and a computer readable medium.
Background
With the development of remote sensing observation technology, the number of earth observation satellites is continuously increased, and the resolution and the number of remote sensing images are greatly improved. According to the requirements of users, interested targets or scenes are efficiently and accurately searched from the multi-source heterogeneous remote sensing big data, so that the massive data is effectively managed, and the method is the basis of subsequent resource sharing and interpretation application. The remote sensing image retrieval based on the content can retrieve interesting remote sensing images from massive remote sensing images according to the visual characteristics of the images, and becomes a research hotspot at present.
Generally, remote sensing image retrieval mainly comprises two parts of feature extraction and image retrieval. The characteristic extraction is mainly used for extracting representative characteristics of the remote sensing image, and the image retrieval is used for measuring the similarity between the query image and the target image and querying the most similar image from the large-scale image library. Because the traditional manually selected features such as color, texture and shape cannot effectively express the semantic information of the image, and a semantic gap exists between feature description and high-level semantics, a deep learning method represented by a convolutional neural network in recent years has strong feature learning capability, can extract abstract and high-level semantic features from an original image, and quickly becomes a mainstream method for solving the problem of remote sensing image retrieval.
However, compared with a natural image with a definite main body and a single background, one of the difficulties in extracting the features of the remote sensing image is that the contents of the remote sensing image are complex. For example, multiple types of land cover data may be contained in the remote sensing image, resulting in multiple types of ground object objects in the image, and the size (which may be the size of the image, for example) of the image is greatly different. In addition, due to differences in imaging conditions such as sensors, shooting angles, shooting weather, and the like, scenes belonging to the same semantic category may have large visual differences in different images. Therefore, the effective feature extraction method is a key problem for improving the remote sensing image retrieval effect, so that the image features can capture context scene information from a global angle and can also mine multi-scale information of ground object targets from a local angle.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure provide a remote sensing image retrieval method, apparatus, electronic device and computer readable medium to solve the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a method for retrieving a remote sensing image, including: receiving a remote sensing image to be processed; importing the remote sensing image to be processed into a pre-trained multi-scale image recognition model to obtain target image features, wherein the multi-scale image recognition model is used for acquiring image features of the remote sensing image under a preset scale and fusing the image features under the preset scale into the target image features; and matching a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics.
Optionally, the multi-scale image recognition model includes a multi-scale convolutional layer, and the multi-scale convolutional layer includes at least one convolutional kernel; and importing the remote sensing image to be processed into a pre-trained multi-scale image recognition model to obtain target image characteristics, wherein the method comprises the following steps: extracting features of different scales of the remote sensing image to be processed by using convolution kernels with different expansion rates in the multi-scale convolution layer; and fusing the features of different scales into target image features corresponding to the remote sensing image to be processed.
Optionally, the multi-scale image recognition model includes a softening pool, and the softening pool is used for acquiring image details; and in the multi-scale convolution layer, extracting features of the remote sensing image to be processed in different scales by using convolution kernels with different expansion rates, wherein the features comprise: acquiring multi-scale image characteristics corresponding to the remote sensing image to be processed through the corresponding convolution kernel; and enabling the multi-scale image features to retain detail features in the pooling process through the softening pool.
Optionally, the multi-scale image recognition model is obtained by training through the following steps: and training through a training sample image set to obtain the multi-scale image recognition model.
Optionally, matching a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics includes: and matching the target image from the image library according to at least one sub-image feature contained in the target image feature.
Optionally, the matching a target image from the image library according to at least one sub-image feature included in the target image feature includes: for each sub-image feature of the at least one sub-image feature, determining an initial image set corresponding to the sub-image feature from the image library, and matching a target image from the initial image set.
Optionally, the determining an initial image set corresponding to the sub-image feature from the image library includes: determining at least one candidate image based on the image feature; at least one initial image is determined from the at least one candidate image.
In a second aspect, some embodiments of the present disclosure provide a remote sensing image retrieval apparatus, including: a receiving unit configured to receive a remote sensing image to be processed; the target image feature acquisition unit is configured to import the remote sensing image to be processed into a pre-trained multi-scale image recognition model to obtain a target image feature, wherein the multi-scale image recognition model is used for acquiring image features of the remote sensing image at a preset scale and fusing the image features at the preset scale into the target image feature; and the target image retrieval unit is configured to match a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics.
Optionally, the multi-scale image recognition model includes a multi-scale convolutional layer, and the multi-scale convolutional layer includes at least one convolutional kernel; and the target image feature acquisition unit includes: the characteristic extraction subunit is configured to extract the characteristics of the remote sensing image to be processed in different scales by using convolution kernels with different expansion rates in the multi-scale convolution layer; and the target image feature fusion subunit is configured to fuse the features of different scales into target image features corresponding to the remote sensing image to be processed.
Optionally, the multi-scale image recognition model includes a softening pool, and the softening pool is used for acquiring image details; and, the feature extraction subunit includes: the multi-scale image characteristic acquisition module is configured to acquire multi-scale image characteristics corresponding to the remote sensing image to be processed through the corresponding convolution kernels; a detail feature pooling module configured to cause the multi-scale image features to retain detail features during pooling by the softening pool.
Optionally, the apparatus further includes a multi-scale image recognition model training unit configured to train a multi-scale image recognition model, where the multi-scale image recognition model training unit includes: a multi-scale image recognition model training subunit configured to obtain the multi-scale image recognition model through training of a training sample image set.
Optionally, the target image retrieving unit includes: and the target image searching subunit is configured to match a target image from the image library according to at least one sub-image feature contained in the target image feature.
Optionally, the target image retrieving subunit includes: and the target image retrieval module is configured to determine an initial image set corresponding to the sub-image features from the image library for each sub-image feature in the at least one sub-image feature, and match a target image from the initial image set.
Optionally, the target image retrieval module includes: a candidate image determination sub-module configured to determine at least one candidate image based on the image feature; an initial image determination sub-module configured to determine at least one initial image from the at least one candidate image.
In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method described in any of the implementations of the first aspect.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium on which a computer program is stored, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect.
The above embodiments of the present disclosure have the following beneficial effects: by the remote sensing image retrieval method of some embodiments of the disclosure, the target image is obtained, and the accuracy of image retrieval is improved. Specifically, the reason why the accuracy of image retrieval is not high is that: the contents contained in the remote sensing image are complex, and the contents of the remote sensing image are not easy to accurately identify. Based on the above, the remote sensing image retrieval method of some embodiments of the present disclosure can identify the image features of the remote sensing image to be processed in multiple scales through the multi-scale image identification model. And image characteristics of the remote sensing image to be processed under multiple scales are obtained, so that a target image corresponding to the remote sensing image to be processed can be accurately matched from the image library, and the effectiveness of obtaining the target image is improved.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.
FIG. 1 is a flow diagram of some embodiments of a remote sensing image retrieval method according to the present disclosure;
FIG. 2 is a flow diagram of further embodiments of a remote sensing image retrieval method according to the present disclosure;
FIG. 3 is a flow diagram of further embodiments of remote sensing image retrieval methods according to the present disclosure;
FIG. 4 is a schematic structural diagram of some embodiments of a remote sensing image retrieval device according to the present disclosure;
FIG. 5 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings. The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Referring to fig. 1, fig. 1 illustrates a flow 100 of some embodiments of a remote sensing image retrieval method according to the present disclosure. The remote sensing image retrieval method comprises the following steps:
step 101, receiving a remote sensing image to be processed.
In some embodiments, the subject of execution of the remote sensing image retrieval method (which may be, for example, remote sensing image server 100) may receive the remote sensing image to be processed by a wired connection or a wireless connection.
And 102, importing the remote sensing image to be processed into a multi-scale image recognition model trained in advance to obtain target image characteristics.
In some embodiments, after receiving the remote sensing image to be processed, the executing subject may import the remote sensing image to be processed into a pre-trained multi-scale image recognition model. The multi-scale image recognition model is used for acquiring image features of the remote sensing image under a preset scale and fusing the image features under the preset scale into target image features. Therefore, the target image features output by the multi-scale image recognition model comprise the image features of the remote sensing image to be processed under multiple scales.
In some optional implementations of some embodiments, the multi-scale image recognition model may be trained by: and training through a training sample image set to obtain the multi-scale image recognition model.
The execution subject can input the training sample images in the training sample image set into artificial neural networks such as a residual error network and a deep learning model, and comprehensively train to obtain the multi-scale image recognition model.
In some optional implementations of some embodiments, in order to improve the effectiveness of the multi-scale image recognition model in obtaining image features, it is necessary to set constraints in the training process of the multi-scale image recognition model. The constraints may include: 1. reducing intra-class differences in features of the same class of images using center loss; 2. cross entropy loss constraint image classification is used. The above constraints are represented by the following equations:
Figure BDA0003539250490000061
Figure BDA0003539250490000062
L＝(1-λ)Lcenter+λLsoftmax (3)
wherein L iscenterLoss at the center; x is the number ofiIs the image characteristic of the training sample; y isiThe real category of the ith training sample;
Figure BDA0003539250490000071
m is the total number of the training samples for the class center corresponding to each training sample; | | non-woven hair2Is a 2-norm formula; l issoftmaxIs the cross entropy loss; k is the prediction category of the input image; k is the total number of image categories; q is yiQ (k ═ y) ofi|xi)＝1，q(k≠yi|xi)＝0；p(k|xi) Is input as xiPredicting the probability of the class k; lambda is an optimization threshold; l is the total loss.
And 103, matching a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics.
In some embodiments, after obtaining the target image features, the executing subject may match the target image from a preset image library. Because the target image features comprise image features of the remote sensing image to be processed under multiple scales, images in the image library may correspond to different scales. Therefore, through the target image characteristics, images under different scales can be matched from the image library, and an accurate target image can be determined. Therefore, the accuracy and effectiveness of obtaining the target image corresponding to the remote sensing image to be processed are improved.
According to the remote sensing image retrieval method disclosed by some embodiments of the disclosure, the target image is obtained, and the accuracy of image retrieval is improved. Specifically, the reason why the accuracy of image retrieval is not high is that: the contents contained in the remote sensing image are complex, and the contents of the remote sensing image are not easy to accurately identify. Based on the above, the remote sensing image retrieval method of some embodiments of the present disclosure can identify the image features of the remote sensing image to be processed in multiple scales through the multi-scale image identification model. And image characteristics of the remote sensing image to be processed under multiple scales are obtained, so that a target image corresponding to the remote sensing image to be processed can be accurately matched from the image library, and the effectiveness of obtaining the target image is improved.
With continued reference to fig. 2, fig. 2 illustrates a flow 200 of some embodiments of a method of remote sensing image retrieval according to the present disclosure. The remote sensing image retrieval method comprises the following steps:
step 201, receiving a remote sensing image to be processed.
The content of step 201 is the same as that of step 101, and is not described in detail here.
And 202, extracting the features of the remote sensing image to be processed in different scales by using convolution kernels with different expansion rates in the multi-scale convolution layer.
The multi-scale convolution layer comprises convolution kernels with a plurality of expansion rates, and the convolution kernels with different expansion rates can extract image features with different scales. Therefore, the characteristics of the remote sensing image to be processed with different scales can be obtained.
In some optional implementations of some embodiments, the above extracting, in the multi-scale convolution layer, features of different scales of the remote sensing image to be processed by using convolution kernels with different expansion rates may include the following steps:
firstly, obtaining multi-scale image characteristics of the remote sensing image to be processed through a plurality of convolution kernels with different expansion rates. The scale is the minimum distinguishable unit for distinguishing the remote sensing image target, and the multi-scale image features comprise the ground object target information under different distinguishable units, so that the target with a large space range can be represented, and the target with a small ground object can also be represented. For example, the main content of the remote sensing image is a river, and arable land, buildings, trees and the like can be located near the river, and the remote sensing image can also include the outlines of the river, the arable land, the buildings, the trees and the like. Accordingly, the multi-scale image features may be river-related image features acquired through convolution kernels of different sizes.
According to the description, different convolution kernels can obtain the image characteristics of the corresponding scales, and correspondingly, a plurality of convolution kernels can obtain the multi-scale image characteristics of the remote sensing image to be processed.
And secondly, enabling the multi-scale image features to retain detailed features in the pooling process through the softening pool. The details of the image reflect the gray level change condition of the image, and the soft pooling can reduce the loss of the details of the image and more retain the information of the outline, the details and the like of the image. Corresponding to the multi-scale image features, the detail features can be image features of contours of rivers, cultivated lands, buildings and trees, which are acquired by convolution kernels of different sizes.
The multi-scale image recognition model comprises a softening pool, and the softening pool is used for acquiring image details. The same remote sensing image to be processed may show different detail characteristics at different scales. The execution main body can process the multi-scale image features through the softening pool to obtain the detail features of the remote sensing image to be processed.
And 203, fusing the features of different scales into target image features corresponding to the remote sensing image to be processed.
After the features of the remote sensing image to be processed in different scales are obtained, the execution main body can fuse the features of the different scales to obtain the target image features of the remote sensing image to be processed.
And 204, matching a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics.
The content of step 204 is the same as that of step 103, and is not described in detail here.
With continued reference to fig. 3, fig. 3 illustrates a flow 300 of some embodiments of a method of remote sensing image retrieval according to the present disclosure. The remote sensing image retrieval method comprises the following steps:
step 301, receiving a remote sensing image to be processed.
And 302, importing the remote sensing image to be processed into a multi-scale image recognition model trained in advance to obtain target image characteristics.
The contents of step 301 and step 302 are the same as those of step 101 and step 102, and are not described in detail here.
Step 303, matching the target image from the image library according to at least one sub-image feature contained in the target image feature.
The sub-image features may include scale information for characterizing a scale of the remote sensing image to be processed when the sub-image features are obtained. The executing subject can match the target image from the image library according to different sub-image characteristics. For example, when the same image can be matched from the image library through the sub-image features at a certain scale, the image can be set as a target image corresponding to the remote sensing image to be processed.
In some optional implementations of some embodiments, the matching the target image from the image library according to at least one sub-image feature included in the target image feature may include the following steps: and for each sub-image feature contained in the at least one sub-image feature, determining an initial image set corresponding to the sub-image feature from the image library, and matching a target image from the initial image set.
In some optional implementations of some embodiments, the determining the initial image set corresponding to the sub-image feature from the image library may include:
in a first step, at least one candidate image is determined based on the sub-image features.
In order to accurately identify the target image from the image library, the execution subject may match at least one candidate image from the image library by finding a characteristic distance and the like.
And secondly, determining at least one initial image from the at least one candidate image.
As can be seen from the above description, the multiple candidate images may be multiple different cells using the same design, or different buildings using the same design. To this end, the executing entity may further determine at least one initial image from the at least one candidate image. Here, the determination method may be: the execution subject may compare the plurality of candidate images with each other based on image features of the plurality of candidate images, and determine a plurality of candidate images having a similarity greater than a set threshold (e.g., may be a similarity exceeding 90%) as the initial image. Thus, the accuracy of finally acquiring the target image can be further improved through the initial image.
According to the scheme, the test is carried out based on the residual error network ResNet50 model, and compared with other existing networks, the obtained comparison data are as follows:
Figure BDA0003539250490000101
TABLE 1 search accuracy of different methods of UCMD dataset (higher mAP search better)
The result shows that the Average detection Precision (mAP) of the method is superior to that of the existing methods. Compared with a reference method, the method has the advantage that the average detection precision is greatly improved by 10.33%. The method obtains better Retrieval rank (ANMRR) of Average Normalized adjustment and Average detection precision, is superior to other existing methods, and explains the effectiveness of the remote sensing image Retrieval task.
With further reference to fig. 4, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a remote sensing image retrieval apparatus, which correspond to those of the method embodiments shown in fig. 1, and which may be applied in various electronic devices.
As shown in fig. 4, the remote sensing image retrieval apparatus 400 of some embodiments includes: a receiving unit 401, a target image feature acquisition unit 402, and a target image retrieval unit 403. Wherein, the receiving unit 401 is configured to receive a remote sensing image to be processed; a target image feature obtaining unit 402, configured to import the remote sensing image to be processed into a pre-trained multi-scale image recognition model to obtain a target image feature, where the multi-scale image recognition model is used to obtain an image feature of the remote sensing image at a preset scale, and fuse the image feature at the preset scale into the target image feature; and a target image retrieval unit 403, configured to match a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics.
In an optional implementation of some embodiments, the multi-scale image recognition model comprises a multi-scale convolutional layer comprising at least one convolution kernel; and, the target image feature acquisition unit 402 may include: a feature extraction subunit (not shown in the figure) and a target image feature fusion subunit (not shown in the figure). The feature extraction subunit is configured to extract features of different scales of the remote sensing image to be processed by using convolution kernels with different expansion rates in the multi-scale convolution layer; and the target image feature fusion subunit is configured to fuse the features of different scales into target image features corresponding to the remote sensing image to be processed.
In an optional implementation of some embodiments, the multi-scale image recognition model comprises a softening pool for obtaining image details; and, the feature extraction subunit may include: a multi-scale image feature acquisition module (not shown) and a detail feature pooling module (not shown). The multi-scale image feature acquisition module is configured to acquire multi-scale image features corresponding to the remote sensing image to be processed through corresponding convolution kernels; a detail feature pooling module configured to cause the multi-scale image features to retain detail features during pooling by the softening pool.
In an optional implementation of some embodiments, the apparatus 400 further includes a multi-scale image recognition model training unit (not shown in the figures) configured to train a multi-scale image recognition model, the multi-scale image recognition model training unit including: a multi-scale image recognition model training subunit (not shown in the figure) configured to obtain the multi-scale image recognition model through training of a training sample image set.
In an optional implementation manner of some embodiments, the target image retrieval unit 403 may include: and a target image searching subunit (not shown in the figure) configured to match a target image from the image library according to at least one sub-image feature contained in the target image feature.
In an optional implementation of some embodiments, the target image retrieval subunit may include: a target image retrieval module (not shown) configured to determine, for each of the at least one sub-image feature, an initial image set corresponding to the sub-image feature from the image library, and to match a target image from the initial image set.
In an optional implementation of some embodiments, the target image retrieval module may include: a candidate image determination sub-module (not shown) and an initial image determination sub-module (not shown). Wherein the candidate image determination sub-module is configured to determine at least one candidate image based on the image feature; an initial image determination sub-module configured to determine at least one initial image from the at least one candidate image.
It will be understood that the units described in the apparatus 400 correspond to the various steps in the method described with reference to fig. 1. Thus, the operations, features and advantages described above with respect to the method are also applicable to the apparatus 400 and the units included therein, and are not described herein again.
As shown in fig. 5, electronic device 500 may include a processing means (e.g., central processing unit, graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Generally, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and a communication device 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 illustrates an electronic device 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 5 may represent one device or may represent multiple devices as desired.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network via the communication means 509, or installed from the storage means 508, or installed from the ROM 502. The computer program, when executed by the processing device 501, performs the above-described functions defined in the methods of some embodiments of the present disclosure.
It should be noted that the computer readable medium described above in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a remote sensing image to be processed; importing the remote sensing image to be processed into a pre-trained multi-scale image recognition model to obtain target image features, wherein the multi-scale image recognition model is used for acquiring image features of the remote sensing image under a preset scale and fusing the image features under the preset scale into the target image features; and matching a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. The described units may also be provided in a processor, and may be described as: a processor includes a receiving unit, a target image feature acquisition unit, and a target image retrieval unit. The names of these units do not in some cases constitute a limitation on the unit itself, and for example, the target image retrieval unit may also be described as a "unit for acquiring a target image".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims (10)

1. A remote sensing image retrieval method comprises the following steps:
receiving a remote sensing image to be processed;
importing the remote sensing image to be processed into a pre-trained multi-scale image recognition model to obtain target image features, wherein the multi-scale image recognition model is used for acquiring image features of the remote sensing image under a preset scale and fusing the image features under the preset scale into the target image features;
and matching a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics.
2. The method of claim 1, wherein the multi-scale image recognition model comprises a multi-scale convolutional layer containing at least one convolution kernel; and
the method for importing the remote sensing image to be processed into a multi-scale image recognition model trained in advance to obtain target image characteristics comprises the following steps:
extracting features of different scales of the remote sensing image to be processed by using convolution kernels with different expansion rates in the multi-scale convolution layer;
and fusing the features of different scales into target image features corresponding to the remote sensing image to be processed.
3. The method of claim 2, wherein the multi-scale image recognition model comprises a softening pool for acquiring image details; and
in the multi-scale convolution layer, extracting the features of the remote sensing image to be processed in different scales by using convolution kernels with different expansion rates comprises the following steps:
acquiring multi-scale image characteristics corresponding to the remote sensing image to be processed through the corresponding convolution kernel;
and enabling the multi-scale image features to retain detail features in the pooling process through the softening pool.
4. The method of claim 1, wherein the multi-scale image recognition model is trained by:
and training through a training sample image set to obtain the multi-scale image recognition model.
5. The method according to claim 1, wherein the matching out of the target image corresponding to the remote sensing image to be processed from a preset image library based on the target image feature comprises:
and matching the target image from the image library according to at least one sub-image feature contained in the target image feature.
6. The method according to claim 5, wherein the matching of the target image from the image library according to the at least one sub-image feature included in the target image feature comprises:
for each sub-image feature of the at least one sub-image feature, an initial image set corresponding to the sub-image feature is determined from the image library, and a target image is matched from the initial image set.
7. The method of claim 6, wherein said determining an initial image set corresponding to the sub-image feature from the image library comprises:
determining at least one candidate image based on the image feature;
at least one initial image is determined from the at least one candidate image.
8. A remote sensing image retrieval apparatus comprising:
a receiving unit configured to receive a remote sensing image to be processed;
the target image feature acquisition unit is configured to import the remote sensing image to be processed into a pre-trained multi-scale image recognition model to obtain target image features, wherein the multi-scale image recognition model is used for acquiring image features of the remote sensing image at a preset scale and fusing the image features at the preset scale into the target image features;
and the target image retrieval unit is configured to match a target image corresponding to the remote sensing image to be processed from a preset image library based on the target image characteristics.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1 to 7.
CN202210226180.XA 2022-03-09 2022-03-09 Remote sensing image retrieval method and device, electronic equipment and computer readable medium Pending CN114610938A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210226180.XA CN114610938A (en) 2022-03-09 2022-03-09 Remote sensing image retrieval method and device, electronic equipment and computer readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210226180.XA CN114610938A (en) 2022-03-09 2022-03-09 Remote sensing image retrieval method and device, electronic equipment and computer readable medium

Publications (1)

Publication Number Publication Date
CN114610938A true CN114610938A (en) 2022-06-10

Family

ID=81860883

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210226180.XA Pending CN114610938A (en) 2022-03-09 2022-03-09 Remote sensing image retrieval method and device, electronic equipment and computer readable medium

Country Status (1)

Country Link
CN (1) CN114610938A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115984709A (en) * 2022-12-20 2023-04-18 中国科学院空天信息创新研究院 Content identification method for rapid large-scale remote sensing image

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115984709A (en) * 2022-12-20 2023-04-18 中国科学院空天信息创新研究院 Content identification method for rapid large-scale remote sensing image
CN115984709B (en) * 2022-12-20 2023-07-04 中国科学院空天信息创新研究院 Content identification method for rapid large-scale remote sensing image

Similar Documents

Publication Publication Date Title
CN111860573B (en) Model training method, image category detection method and device and electronic equipment
CN109801293B (en) Remote sensing image segmentation method and device, storage medium and server
US12020473B2 (en) Pedestrian re-identification method, device, electronic device and computer-readable storage medium
US9466013B2 (en) Computer vision as a service
CN110046600B (en) Method and apparatus for human detection
US9292766B2 (en) Techniques for ground-level photo geolocation using digital elevation
US9846808B2 (en) Image integration search based on human visual pathway model
JP5563494B2 (en) Corresponding reference image search device and method, content superimposing device, system and method, and computer program
CN111680176A (en) Remote sensing image retrieval method and system based on attention and bidirectional feature fusion
CN106462572A (en) Techniques for distributed optical character recognition and distributed machine language translation
CN106415605A (en) Techniques for distributed optical character recognition and distributed machine language translation
CN113688310B (en) Content recommendation method, device, equipment and storage medium
CN113449070A (en) Multimodal data retrieval method, device, medium and electronic equipment
CN111832579B (en) Map interest point data processing method and device, electronic equipment and readable medium
CN110941978A (en) Face clustering method and device for unidentified personnel and storage medium
CN111339340A (en) Training method of image description model, image searching method and device
CN112598673A (en) Panorama segmentation method, device, electronic equipment and computer readable medium
JP2022541832A (en) Method and apparatus for retrieving images
CN113610034B (en) Method and device for identifying character entities in video, storage medium and electronic equipment
CN114610938A (en) Remote sensing image retrieval method and device, electronic equipment and computer readable medium
CN114153959A (en) Key value matching method and device, readable medium and electronic equipment
CN111797266B (en) Image processing method and apparatus, storage medium, and electronic device
CN112465737A (en) Image processing model training method, image processing method and image processing device
CN114742707B (en) Multi-source remote sensing image splicing method and device, electronic equipment and readable medium
CN110765304A (en) Image processing method, image processing device, electronic equipment and computer readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination