CN109543561A - Saliency of taking photo by plane method for detecting area and device - Google Patents

Saliency of taking photo by plane method for detecting area and device Download PDF

Info

Publication number
CN109543561A
CN109543561A CN201811290665.5A CN201811290665A CN109543561A CN 109543561 A CN109543561 A CN 109543561A CN 201811290665 A CN201811290665 A CN 201811290665A CN 109543561 A CN109543561 A CN 109543561A
Authority
CN
China
Prior art keywords
notable
graphs
convolutional neural
neural networks
space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811290665.5A
Other languages
Chinese (zh)
Other versions
CN109543561B (en
Inventor
李甲
付奎
沈鸿泽
赵沁平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN201811290665.5A priority Critical patent/CN109543561B/en
Publication of CN109543561A publication Critical patent/CN109543561A/en
Application granted granted Critical
Publication of CN109543561B publication Critical patent/CN109543561B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the present invention provides one kind and takes photo by plane saliency method for detecting area and device.It include: the significant set of graphs of sample for obtaining sample data set and respectively corresponding M ground level classics conspicuousness model；M single channel convolutional neural networks are initialized, the M convolutional neural networks model using sample data set and sample M single channel convolutional neural networks of significant set of graphs training, after obtaining training；The road N convolutional neural networks model using the road N number of single channel convolutional neural networks model initialization N convolutional neural networks, using the sample data set training road N convolutional neural networks, after obtaining training；The significant set of graphs in airspace of data to be tested collection is obtained according to the road the N convolutional neural networks after training；The significant set of graphs of time domain of data to be tested collection is obtained according to DCT method；According to the significant set of graphs in airspace and the significant set of graphs of time domain, the space-time remarkable set of graphs of data to be tested collection is obtained.Obtain the significant set of graphs of salient region concentration and video of clearly taking photo by plane.

Description

Saliency of taking photo by plane method for detecting area and device
Technical field
It takes photo by plane saliency method for detecting area the present embodiments relate to computer vision field more particularly to one kind And device.
Background technique
Salient region detection in video is the basic problem of computer vision field and information retrieval field, has weight Want meaning, wherein salient region refers to the region that can directly cause visual attention location in video.Currently, for ordinary video Salient region detection technique in (such as holding mobile phone in the video that ground shooting obtains), researcher have been proposed significantly Property method for detecting area, achieves good achievement.
However, having due to taking photo by plane video relative to ordinary video for the detection method of salient region in video of taking photo by plane The characteristics of having oneself: first, video camera does high-speed motion with unmanned plane or other equipment together, the consecutive image in video of taking photo by plane Between there is the transformation such as translation, rotation, i.e., change on space-time between consecutive image obvious；Second, video usual situation of taking photo by plane Lower and ordinary video has very big difference, usually has high visual angle, the features such as Small object and jogging speed.Therefore, if directly will For in ordinary video salient region design detection method be applied to take photo by plane video salient region detection in, significantly Property region detection is not accurate enough.
Summary of the invention
The embodiment of the present invention provides one kind and takes photo by plane saliency method for detecting area and device, to solve in the prior art The problem of salient region in clear and accurate video of taking photo by plane can not be obtained.
It takes photo by plane saliency method for detecting area in a first aspect, the embodiment of the present invention provides one kind, comprising:
By M ground level classics conspicuousness model, obtains sample data set and correspond to each ground level classics significantly Property model the significant set of graphs of sample, the sample data set is to take photo by plane in video to extract from the sample that salient region has marked Continuous sequence of video images；Wherein, M is more than or equal to 2；
M single channel convolutional neural networks are initialized, and utilize the sample data set and the M significant set of graphs of sample The M single channel convolutional neural networks are respectively trained, M convolutional neural networks model after obtaining training；Wherein, the M Single channel convolutional neural networks model structure is identical；
Using at the beginning of N number of single channel convolutional neural networks model in M single channel convolutional neural networks model after the training The road beginningization N convolutional neural networks, and the N using the sample data set training road N convolutional neural networks, after obtaining training Road convolutional neural networks model；Wherein, N is greater than or equal to 2 and is less than or equal to M；
According to the road the N convolutional neural networks model after the training, the significant set of graphs in airspace of data to be tested collection is obtained, The data to be tested collection is from the continuous sequence of video images for currently needing to extract in the video of taking photo by plane detected；
According to the method based on discrete cosine transform, the significant set of graphs of time domain of the data to be tested collection is obtained；
According to the significant set of graphs in the airspace and the significant set of graphs of the time domain, the space-time of the data to be tested collection is obtained Significant set of graphs.
Optionally, N number of single channel convolutional Neural in the M single channel convolutional neural networks model using after the training Network model initializes before the convolutional neural networks of the road N, further includes:
N number of single channel volume is selected from M single channel convolutional neural networks model after the training according to optimization formula Product neural network model；
Wherein, optimize formula are as follows: α*=arg max Ωr+λdΩd, wherein α*For the vector of 1*M, wherein element αiIt is equal to When 1, illustrate the single channel convolutional neural networks model for selecting number for i；ΩrIndicate N number of single channel convolution mind of selection Representativeness through network model；ΩdIndicate the diversity of N number of single channel convolutional neural networks model of selection；λdIndicate balance Parameter, for balancing representative and diversity.
Optionally, described according to the significant set of graphs in the airspace and the significant set of graphs of the time domain, it obtains described to be detected The space-time remarkable set of graphs of data set, comprising:
The time domain obtained in the airspace notable figure and the significant set of graphs of the time domain in the significant set of graphs in the airspace is significant The intersection notable figure of figure；
Select the time domain in the airspace notable figure or the significant set of graphs of the time domain in the significant set of graphs in the airspace significant Figure is as compensation notable figure；
According to the intersection notable figure, the compensation notable figure, weighted value, space-time remarkable figure, the space-time remarkable are obtained Figure is the salient region result figure that the data to be tested concentrate a wherein image；Wherein, the weighted value is for indicating Weighted value of the intersection notable figure in the space-time remarkable figure, power of the intersection notable figure in the space-time remarkable figure The sum of the weighted value of weight values and the compensation notable figure in the space-time remarkable figure is equal to preset value；
The notable figure that the data to be tested concentrate every image is obtained, the space-time remarkable of the data to be tested collection is obtained Set of graphs.
Optionally, in the airspace notable figure and the significant set of graphs of the time domain obtained in the significant set of graphs in airspace Time domain notable figure intersection notable figure, comprising:
Space is obtained to time consistency score and time to Space Consistency score；
It is aobvious that the intersection is obtained to Space Consistency score to time consistency score and the time according to the space Write figure.
Optionally, in the airspace notable figure or the significant set of graphs of the time domain in the selection significant set of graphs in airspace Time domain notable figure as compensation notable figure, comprising:
The compactness of the airspace notable figure and the time domain notable figure is obtained respectively；
Judge whether the compactness of the airspace notable figure is less than or equal to the compactness of the time domain notable figure；
If so, selecting the compensation notable figure for the airspace notable figure；
If it is not, then selecting the compensation notable figure for the time domain notable figure.
Optionally, described according to the intersection notable figure, the compensation notable figure, weighted value, obtain space-time remarkable figure it Before, further includes:
Obtain the compactness of the intersection notable figure；
Judge whether the compactness of the intersection notable figure is less than the compactness and time domain notable figure of airspace notable figure Compactness minimum value and preset threshold product；
If so, determining the space to time consistency score and the time to the minimum in Space Consistency score Value is weighted value of the intersection notable figure in the space-time remarkable figure；
If not, it is determined that weighted value of the intersection notable figure in the notable figure is 0；
It is integrated into the weighted value in the space-time remarkable set of graphs according to the preset value and the intersection notable figure, is obtained The compensation notable figure is integrated into the weighted value in the space-time remarkable set of graphs.
Second aspect, the embodiment of the present invention provide one kind and take photo by plane saliency regional detection device, comprising:
First obtains module, for obtaining sample data set and corresponding to each institute by M ground level classics conspicuousness model The significant set of graphs of sample of ground level classics conspicuousness model is stated, the sample data set is the sample marked from salient region Originally it takes photo by plane the continuous sequence of video images extracted in video；Wherein, M is more than or equal to 2；
Training module, for initializing M single channel convolutional neural networks, and it is described using the sample data set and M The M single channel convolutional neural networks are respectively trained in the significant set of graphs of sample, M convolutional neural networks mould after obtaining training Type；Wherein, the M single channel convolutional neural networks model structure is identical；
The training module is also used to utilize N number of single channel in M single channel convolutional neural networks model after the training The road convolutional neural networks model initialization N convolutional neural networks, and utilize the sample data set training road N convolutional Neural Network, the road the N convolutional neural networks model after obtaining training；Wherein, N is greater than or equal to 2 and is less than or equal to M；
Described first obtains module, is also used to be obtained to be detected according to the road the N convolutional neural networks model after the training The significant set of graphs in the airspace of data set, the data to be tested collection are continuous from currently needing to extract in the video of taking photo by plane detected Sequence of video images；
Described first obtains module, is also used to obtain the number to be detected according to the method based on discrete cosine transform According to the significant set of graphs of the time domain of collection；
Fusion Module, for according to the significant set of graphs in the airspace and the significant set of graphs of the time domain, obtaining described take photo by plane The space-time remarkable set of graphs of video.
Optionally, described device further include:
Optimization module, for being selected from M single channel convolutional neural networks model after the training according to optimization formula N number of single channel convolutional neural networks model；
Wherein, optimize formula are as follows: α*=arg max Ωr+λdΩd, wherein α*For the vector of 1*M, wherein element αiIt is equal to When 1, illustrate the single channel convolutional neural networks model for selecting number for i；ΩrIndicate N number of single channel convolution mind of selection Representativeness through network model；ΩdIndicate the diversity of N number of single channel convolutional neural networks model of selection；λdIndicate balance Parameter, for balancing representative and diversity
Optionally, the Fusion Module, comprising:
Second obtains module, for obtaining airspace notable figure and the time domain notable figure in the significant set of graphs in the airspace The intersection notable figure of time domain notable figure in set；
Processing module, for selecting airspace notable figure or the significant set of graphs of the time domain in the significant set of graphs in the airspace In time domain notable figure as compensation notable figure；
Described second obtains module, is also used to be obtained according to the intersection notable figure, the compensation notable figure, weighted value Space-time remarkable figure, the space-time remarkable figure are the salient region result figure that the data to be tested concentrate a wherein image； Wherein, for the weighted value for indicating weighted value of the intersection notable figure in the space-time remarkable figure, the intersection is significant Figure is in the weighted value and the sum of the weighted value of the compensation notable figure in the space-time remarkable figure etc. in the space-time remarkable figure In preset value；
Described second obtains module, is also used to obtain the notable figure that the data to be tested concentrate every image, obtains institute State the space-time remarkable set of graphs of data to be tested collection.
Optionally, described second module is obtained, is specifically used for:
Space is obtained to time consistency score and time to Space Consistency score；
It is aobvious that the intersection is obtained to Space Consistency score to time consistency score and the time according to the space Write figure.
Optionally, the processing module, is specifically used for:
The compactness of the airspace notable figure and the time domain notable figure is obtained respectively；
Judge whether the compactness of the airspace notable figure is less than or equal to the compactness of the time domain notable figure；
If so, selecting the compensation notable figure for the airspace notable figure；
If it is not, then selecting the compensation notable figure for the time domain notable figure.
Optionally, described second module is obtained according to the intersection notable figure, the compensation notable figure, weighted value, obtain Before space-time remarkable figure, it is also used to obtain the compactness of the intersection notable figure；
The processing module, is also used to judge whether the compactness of the airspace notable figure is less than or equal to the time domain The compactness of notable figure；If so, determining the space to time consistency score and the time to Space Consistency point Minimum value in number is weighted value of the intersection notable figure in the space-time remarkable figure；If not, it is determined that the intersection is aobvious Writing weighted value of the figure in the space-time remarkable figure is 0；
Described second obtains module, and it is aobvious to be also used to be integrated into the space-time according to the preset value and the intersection notable figure The weighted value in set of graphs is write, the compensation notable figure is obtained and is integrated into the weighted value in the space-time remarkable set of graphs.
The third aspect, the embodiment of the present invention provide one kind and take photo by plane saliency regional detection device, the device include: to A few processor and memory；
The memory stores computer executed instructions；At least one described processor executes the meter of the memory storage Calculation machine executes instruction, to execute the described in any item methods of inventive embodiments first aspect sheet.
Fourth aspect, the embodiment of the present invention provide a kind of computer readable storage medium, the computer readable storage medium In be stored with program instruction, described program instruction realizes that inventive embodiments first aspect is described in any item when being executed by processor Method.
5th aspect, the embodiment of the present application provide a kind of program product, and described program product includes computer program, described Computer program is stored in readable storage medium storing program for executing, and at least one processor of saliency of taking photo by plane regional detection device can be with The computer program is read from the readable storage medium storing program for executing, at least one described processor executes the computer program and makes Saliency of taking photo by plane regional detection device implements the inventive embodiments of any one of the present application embodiment first aspect offer Method.
The embodiment of the present invention provides one kind and takes photo by plane saliency method for detecting area and device, and this method by obtaining first Sample significant set of graphs of the sample data set respectively in M ground level classics conspicuousness model is taken, the significant atlas of M sample is utilized Cooperation is desired output M single channel convolutional neural networks of training, and the single channel convolutional neural networks model after alloing training learns The feature of saliency is obtained into ground level classics conspicuousness model.Then N number of single channel convolutional Neural after training is utilized Network model initializes the road N convolutional neural networks, and the feature that saliency is obtained in ground level classics conspicuousness model is answered The road N convolutional neural networks model is used, the significant set of graphs manually marked is used to train the road N convolutional Neural net as desired output Network model.Then, the significant set of graphs in airspace for then by the road N convolutional neural networks model obtaining data to be tested collection, passes through DCT The significant set of graphs of time domain of method acquisition data to be tested collection.Finally, by every airspace notable figure in the significant set of graphs in airspace and The salient region of every notable figure is merged in the significant set of graphs of time domain, to make every in the space-time remarkable set of graphs obtained The salient region of space-time remarkable figure is concentrated and clear
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair Bright some embodiments for those of ordinary skill in the art without any creative labor, can be with It obtains other drawings based on these drawings.
Fig. 1 is the flow chart for the saliency method for detecting area of taking photo by plane that one embodiment of the invention provides；
Fig. 2 is the structure chart for the single channel convolutional neural networks that one embodiment of the invention provides；
Fig. 3 is the road the N convolutional neural networks structure chart that one embodiment of the invention provides；
Fig. 4 is the structural schematic diagram for the saliency regional detection device of taking photo by plane that one embodiment of the invention provides；
Fig. 5 be another embodiment of the present invention provides saliency regional detection device of taking photo by plane structural schematic diagram.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Fig. 1 is the flow chart for the saliency method for detecting area of taking photo by plane that one embodiment of the invention provides.Such as Fig. 1 institute Show, the method for the present embodiment may include:
S101, pass through M ground level classics conspicuousness model, obtain sample data set and correspond to each ground level classics The significant set of graphs of the sample of conspicuousness model.
In the present embodiment, sample data set be take photo by plane from the sample that salient region has marked extract in video it is continuous Sequence of video images.
Specifically, randomly selecting M ground level classics conspicuousness model, i.e. M from ground level classics conspicuousness model basin Sample data concentration is input in any one ground level classics conspicuousness model by={ 1,2,3 ..., M }, to obtain M The significant set of graphs of sample corresponding with ground level classics conspicuousness model.Wherein, it is obtained by ground level classics conspicuousness model The method of the significant set of graphs of the sample of sample data set can refer to the prior art, and details are not described herein again.Wherein, M is more than or equal to 2.
M S102, initialization single channel convolutional neural networks, and it is significant using the sample data set and the M samples The M single channel convolutional neural networks are respectively trained in set of graphs, M convolutional neural networks model after obtaining training.
In the present embodiment, the M single channel convolutional neural networks structure is identical.
Fig. 2 is the structure chart for the single channel convolutional neural networks that one embodiment of the invention provides.As shown in Fig. 2, any one Single channel convolutional neural networks include lower-level modules 11, single channel module 12 and Fusion Features module 13.Wherein, any one single channel The initial method of convolutional neural networks are as follows: the network parameter of the convolutional layer in lower-level modules 11 for example can be by using Preceding two layers of initialization of VGG16 convolutional neural networks.Optionally, in initialization, keeping learning rate is 0, to ensure each point The consistency for the low-level features that road is extracted.The network parameter of single channel module 12 and Fusion Features module 13 can for example use Xavier method initializes.
It should be noted that showing single channel convolutional neural networks in Fig. 2 simply to illustrate that single channel convolutional Neural net One of structure of network, structure as shown in Figure 2 can only be used by being not limited to single channel convolutional neural networks.
After M single channel convolutional neural networks are initialized using the above method, sample data set and any one sample are utilized This significant set of graphs trains any one single channel convolutional neural networks.It is, using the significant set of graphs training one of a sample After a single channel convolutional neural networks, the significant set of graphs of the sample cannot train other single channel convolutional neural networks, also, the list After the completion of the training of road convolutional neural networks, the significant set of graphs of other samples can not be used to train again.Below with one of them Training process for be illustrated:
First using sample data set as the input of single channel convolutional neural networks model, pass through the single channel convolutional neural networks Model obtains reality output, i.e., the current significant set of graphs of sample selects a sample notable figure in the significant set of graphs of M sample Set corresponds to the desired output of the single channel convolutional neural networks model as sample data set.Then, significant using current sample Set of graphs and the significant set of graphs of sample are trained the single channel convolutional neural networks model, the single channel convolution mind after being trained Through network model, wherein the method for training single channel convolutional neural networks can refer to the prior art, such as can be used Adam method and Cross entropy loss function trains single channel convolutional neural networks.
S103, N number of single channel convolutional neural networks mould in M single channel convolutional neural networks model after the training is utilized Type initializes the road N convolutional neural networks, and using the sample data set training road N convolutional neural networks, obtains training The road N convolutional neural networks model afterwards；Wherein, N is greater than or equal to 2 and is less than or equal to M.
In the present embodiment, Fig. 3 is the road the N convolutional neural networks structure chart that one embodiment of the invention provides.As shown in figure 3, N Road convolutional neural networks lower-level modules 21, the road N module 22 and Fusion Features module 23.Wherein, convolutional layer in lower-level modules 21 Network parameter again may be by using before VGG16 convolutional neural networks two layers initialization.Optionally, in initialization, Keeping learning rate is 0, to ensure the consistency of the low-level features of each branch extraction.The road N module 22 includes N branch, wherein Each branch is the single channel module 12 in corresponding single channel convolutional neural networks, i.e., every in each branch in the road N module 22 The network parameter of layer is equal with every layer in single channel module 12 in corresponding single channel convolutional neural networks of network parameter.Fusion Features The network parameter of module 23 can for example be initialized using Xavier method.
It should be noted that showing the road N convolutional neural networks in Fig. 3 simply to illustrate that the road N convolutional neural networks One of structure, structure as shown in Figure 3 can only be used by being not limited to the road N convolutional neural networks.
After the road N convolutional neural networks are initialized using the above method, the training road the N convolutional neural networks model.Wherein, The input data of the road the N convolutional neural networks model is sample data set, and practical output data is denoted as current significant set of graphs. Since sample data set is the continuous sequence of video images extracted in video of taking photo by plane from the sample that salient region has marked, because This, sample data concentrates the salient region of each image it is known that being denoted as the significant set of graphs manually marked, and as Sample data set corresponds to the desired output of the road the N convolutional neural networks model.To using current significant set of graphs and manually The significant set of graphs training of the mark road the N convolutional neural networks model.Wherein, the side of the training road the N convolutional neural networks model Method can refer to the prior art, and Adam method and cross entropy loss function can be used to train the road N convolutional neural networks model.
Optionally, before S103, this method further include: according to optimization formula from M single channel convolution after the training N number of single channel convolutional neural networks model is selected in neural network model；
Optimize formula are as follows: α*=arg max Ωr+λdΩd。
Wherein, α*For the vector of 1*M, wherein element αiWhen equal to 1, illustrate the single channel convolution mind for selecting number as i Through network model；ΩrIndicate the representativeness of N number of single channel convolutional neural networks model of selection, thenAnd ΩdIndicate N number of single channel convolutional neural networks model of selection Diversity, thenWherein SimijRepresent i-th of single channel convolutional neural networks mould The similitude of type and j-th of single channel convolutional neural networks model；λdBalance parameters are indicated, for balancing representative and diversity.
There is diversity and the representative road N number of single channel convolutional neural networks model initialization N by optimization formula selection Convolutional neural networks while reducing calculation amount, can also improve the road N convolution mind in the structure for simplifying the road N convolutional neural networks Performance through network model.
S104, according to the road the N convolutional neural networks model after the training, obtain the airspace notable figure of data to be tested collection Set.
In the present embodiment, the data to be tested collection is from the continuous view for currently needing to extract in the video of taking photo by plane detected Frequency image sequence.Wherein, when obtaining data to be tested collection, the company for the video of taking photo by plane for currently needing to detect is extracted sequentially in time Continuous sequence of video images.
Specifically, to make the continuous of input when data to be tested collection is inputted the road the N convolutional neural networks model after training At least two data to be tested concentrate image be in video continuous frame, obtain data to be tested collection airspace notable figure Set.
S105, method of the basis based on discrete cosine transform, obtain the significant atlas of time domain of the data to be tested collection It closes.
Specifically, by data to be tested collection according to the image input sequence in same S104, using based on discrete cosine transform The method of DCT can obtain the significant set of graphs of time domain of the data to be tested collection.
It should be noted that can first carry out S104 when implementing the embodiment of the present invention and execute S105 again, or It first carries out S105 and executes S104 again, or the S104 and S105 that are performed simultaneously.
S106, according to the significant set of graphs in the airspace and the significant set of graphs of the time domain, obtain the data to be tested collection Space-time remarkable set of graphs.
Specifically, after obtaining the significant set of graphs in airspace and the significant set of graphs of time domain, according to the corresponding relationship of room and time, Time domain notable figure corresponding with every airspace notable figure in the significant set of graphs in airspace can be found in the significant set of graphs of time domain, Obtain Spatial-Temporal notable figure pair.Thus will be in the airspace notable figure and the significant set of graphs of time domain in the significant set of graphs in airspace Time domain notable figure according to room and time corresponding relationship, so that it may obtain Spatial-Temporal notable figure to set.Getting sky After domain-time domain notable figure is to set, the space-time remarkable set of graphs of data to be tested collection is obtained.
For example, Spatial-Temporal notable figure is to for (Sk, Tk), wherein S for the image K that data to be tested are concentratedkFor The airspace notable figure of image K, TkFor the time domain notable figure of image K, k indicates the label of image, can choose airspace notable figure Sk、 Or time domain notable figure TkNotable figure as image K.Selection data to be tested concentrate any one image according to the method described above Space-time remarkable figure, to obtain the space-time remarkable set of graphs of data to be tested collection.
The present embodiment, acquisition sample data set first is respectively in the sample notable figure of M ground level classics conspicuousness model Set, the single channel using the significant set of graphs of M sample as desired output M single channel convolutional neural networks of training, after making training Convolutional neural networks model may learn the feature that saliency is obtained in ground level classics conspicuousness model.Then it utilizes The road N number of single channel convolutional neural networks model initialization N convolutional neural networks after training, will be in ground level classics conspicuousness model The feature for obtaining saliency is applied to the road N convolutional neural networks model, uses the significant set of graphs that manually marks as the phase It hopes and exports the training road N convolutional neural networks model.Then, then by the road N convolutional neural networks model obtain data to be tested collection The significant set of graphs in airspace, pass through DCT method obtain data to be tested collection the significant set of graphs of time domain.Finally, airspace is significant The salient region of every airspace notable figure and the notable figure in the significant set of graphs of time domain is merged in set of graphs, to make to obtain Space-time remarkable set of graphs in the salient region of every space-time remarkable figure concentrate and clear.
In some embodiments, a kind of possible implementation of above-mentioned S106 can be with are as follows:
The time domain in airspace notable figure and the significant set of graphs of the time domain in S201, the acquisition significant set of graphs in airspace The intersection notable figure of notable figure.
Specifically, below to be illustrated for obtaining the wherein notable figure of an image K of data to be tested concentration.
The Spatial-Temporal notable figure of image K is got to (Sk, Tk) after, obtain airspace notable figure SkWith time domain notable figure Tk In all existing salient region, obtain the intersection notable figure of image K
Optionally, the method for obtaining the intersection notable figure of image K can be with are as follows:
S2011, space is obtained to time consistency score and time to Space Consistency score.
Specifically, space can be for example formula (1) to the calculation formula of time consistency score, the time is consistent to space The calculation formula of property score can be for example formula (2):
Cs2t=e (Sk⊙Tk)/e(Tk) formula (1)
Cs2t=e (Sk⊙Tk)/e(Sk) formula (2)
Wherein, e () represents entropy function, and ⊙ represents Pixel-level dot product.
S2012, according to the space to time consistency score and the time to Space Consistency score, described in acquisition Intersection notable figure.
Specifically, the calculation formula for obtaining intersection notable figure can be for example formula (3):
The time domain in airspace notable figure or the significant set of graphs of the time domain in S202, the selection significant set of graphs in airspace Notable figure is as compensation notable figure.
Specifically, comparing the salient region and the significant set of graphs of time domain in the airspace notable figure in the significant set of graphs in airspace In time domain notable figure in salient region clarity, select salient region clarity high as compensation notable figure.
Optionally, the method for obtaining the compensation notable figure of image K can be with are as follows:
S2021, the compactness for obtaining the airspace notable figure and the time domain notable figure respectively.
Specifically, obtaining the compactness of airspace notable figureWith the compactness of time domain notable figureMethod are as follows: it is first The center of gravity of notable figure is first calculated, the abscissa of center of gravity is indicated with Ox, wherein Ox is the conspicuousness of pixel all in notable figure Value is averaged again after being superimposed with the product of its abscissa.The ordinate of center of gravity is indicated with Oy, wherein Oy is all in notable figure The significance value of pixel is averaged again after being superimposed with the product of its ordinate.Then the significant of the pixel in notable figure is calculated Property value and center of gravity space length weighted sum compactness of the mean value as Saliency maps.
S2022, judge whether the compactness of the airspace notable figure is less than or equal to the compact of the time domain notable figure Degree；If so, executing S2023, S2024 is otherwise executed.
Specifically, the compactness of airspace notable figureConcentration for indicating salient region in the notable figure of airspace is clear Degree, the compactness of time domain notable figureFor indicating the concentration of salient region in time domain notable figure clearly degree, Wherein, compactness is smaller, illustrates that salient region the more is concentrated the more clear in notable figure.
Wherein, compensation notable figure is usedIt indicates, calculation formula can for example be indicated with formula (4):
S2023, select the compensation notable figure for the airspace notable figure.
S2024, select the compensation notable figure for the time domain notable figure.
S203, according to the intersection notable figure, the compensation notable figure, weighted value, obtain space-time remarkable figure, the space-time Notable figure is the salient region result figure that the data to be tested concentrate a wherein image.
Wherein, the weighted value is for indicating weight of the intersection notable figure in the space-time remarkable figure, the friendship Collect notable figure in the weighted value of weighted value and the compensation notable figure in the space-time remarkable figure in the space-time remarkable figure The sum of be equal to preset value.
Specifically, a weighted value is set, by adjusting the big of weighted value after intersection notable figure, compensation notable figure determine The feature of the intersection notable figure and compensation notable figure that include in small adjustable space-time remarkable figure, i.e., by adjusting weighted value optimization The concentration clarity of salient region in space-time remarkable figure makes to obtain better space-time remarkable figure.Wherein, space-time remarkable is obtained FigureCalculation formula can be for example formula (5):
Wherein, space-time remarkable figure is obtained by formula (5)When, what weighted value λ was indicated is to indicate the intersection notable figure Weight in the space-time remarkable figure, from formula (5) it is found that weight of the intersection notable figure in space-time remarkable figure and compensation are aobvious It writes the sum of the weight of figure in space-time remarkable figure and is equal to 1, is i.e. preset value can be 1.
Optionally, the method for obtaining weighted value λ may is that
S2031, the compactness for obtaining the intersection notable figure.
Specifically, the method according to S2022 calculates intersection notable figureCompactnessDetails are not described herein again.
S2032, judge whether the compactness of the intersection notable figure is less than the compactness and time domain of airspace notable figure The minimum value of the compactness of notable figure and the product of preset threshold；If so, executing S2033, otherwise, S2034 is executed.
Specifically, given threshold ω, judges intersection notable figureIt is sufficiently compact.For example, threshold value ω is 2.1, whenWhen, illustrate intersection notable figureIt is sufficiently compact；Otherwise, intersection notable figureIt is not compact.
S2033, determine the space to time consistency score and the time to the minimum in Space Consistency score Value is weighted value of the intersection notable figure in the space-time remarkable figure.
S2034, determine that weighted value of the intersection notable figure in the space-time remarkable figure is 0.
Specifically, the calculation formula of weighted value λ can be for example formula (7):
S2035, the weight being integrated into according to the preset value and the intersection notable figure in the space-time remarkable set of graphs Value obtains the compensation notable figure and is integrated into the weighted value in the space-time remarkable set of graphs.
Specifically, from formula (7) it is found that intersection notable figure is obtained after the weight λ in the space-time remarkable figure, according to pre- If value obtains weight of the compensation notable figure in the space-time remarkable figure, i.e. 1- λ, wherein preset value 1.
S204, the space-time remarkable figure that the data to be tested concentrate every image is obtained, obtains the data to be tested collection Space-time remarkable set of graphs.
Obtain data to be tested according to the method for S201-S203 and concentrate the space-time remarkable figure of every image, thus obtain to The space-time remarkable set of graphs of detection data collection.
In the present embodiment, by comparing the compactness of intersection notable figureThe compactness of time domain notable figureWith And the compactness of airspace notable figureIt determines intersection notable figure and compensates notable figure weight shared in space-time remarkable figure, So as to adjust the quality of the salient region of optimization space-time remarkable figure, to obtain, salient region is concentrated and clearly space-time is aobvious Write figure, the space-time remarkable set of graphs of the acquisition data to be tested collection of final high quality.
The method of each embodiment shown in above-mentioned can be executed by following device.
Fig. 4 is the structural schematic diagram for the saliency regional detection device of taking photo by plane that one embodiment of the invention provides.Such as Fig. 4 Shown, the device of the present embodiment may include: the first acquisition module 41, training module 42 and Fusion Module 43.Optionally, described Device further include: optimization module 44.
First obtains module 41, for it is corresponding each to obtain sample data set by M ground level classics conspicuousness model The significant set of graphs of sample of the ground level classics conspicuousness model, the sample data set have been marked from salient region Sample is taken photo by plane the continuous sequence of video images extracted in video；Wherein, M is more than or equal to 2；
Training module 42 for initializing M single channel convolutional neural networks, and utilizes the sample data set and M institute It states the significant set of graphs of sample and the M single channel convolutional neural networks is respectively trained, M convolutional neural networks mould after obtaining training Type；Wherein, the M single channel convolutional neural networks model structure is identical；
The training module 42 is also used to utilize N number of list in M single channel convolutional neural networks model after the training The road road convolutional neural networks model initialization N convolutional neural networks, and utilize the sample data set training road N convolution mind The road N convolutional neural networks model through network, after obtaining training；Wherein, N is greater than or equal to 2 and is less than or equal to M；
Described first obtains module 41, is also used to be obtained to be checked according to the road the N convolutional neural networks model after the training The significant set of graphs in the airspace of measured data collection, the data to be tested collection are from the company for currently needing to extract in the video of taking photo by plane detected Continuous sequence of video images；
Described first obtains module 41, is also used to be obtained described to be detected according to the method based on discrete cosine transform The significant set of graphs of the time domain of data set；
Fusion Module 43, for obtaining the boat according to the significant set of graphs in the airspace and the significant set of graphs of the time domain The space-time remarkable set of graphs to shoot the video.
The optimization module 44, for according to optimization formula from M single channel convolutional neural networks model after the training Middle selection N number of single channel convolutional neural networks model；
Wherein, optimize formula are as follows: α*=arg max Ωr+λdΩd, wherein α*For the vector of 1*M, wherein element αiIt is equal to When 1, illustrate the single channel convolutional neural networks model for selecting number for i；ΩrIndicate N number of single channel convolution mind of selection Representativeness through network model；ΩdIndicate the diversity of N number of single channel convolutional neural networks model of selection；λdIndicate balance Parameter, for balancing representative and diversity.
Optionally, the Fusion Module 43, comprising:
Second obtains module 431, aobvious for obtaining airspace notable figure in the significant set of graphs in the airspace and the time domain Write the intersection notable figure of the time domain notable figure in set of graphs；
Processing module 432, for selecting airspace notable figure or the time domain notable figure in the significant set of graphs in the airspace Time domain notable figure in set is as compensation notable figure；
Described second obtains module 431, is also used to be obtained according to the intersection notable figure, the compensation notable figure, weighted value Space-time remarkable figure is obtained, the space-time remarkable figure is the salient region result that the data to be tested concentrate a wherein image Figure；Wherein, for the weighted value for indicating weighted value of the intersection notable figure in the space-time remarkable figure, the intersection is aobvious Figure is write in the sum of the weighted value of weighted value and the compensation notable figure in the space-time remarkable figure in the space-time remarkable figure Equal to preset value；
Described second obtains module 431, is also used to obtain the space-time remarkable figure that the data to be tested concentrate every image, Obtain the space-time remarkable set of graphs of the data to be tested collection.
Optionally, described second module 431 is obtained, is specifically used for:
Space is obtained to time consistency score and time to Space Consistency score；
It is aobvious that the intersection is obtained to Space Consistency score to time consistency score and the time according to the space Write figure.
Optionally, the processing module 432, is specifically used for:
The compactness of the airspace notable figure and the time domain notable figure is obtained respectively；
Judge whether the compactness of the airspace notable figure is less than or equal to the compactness of the time domain notable figure；
If so, selecting the compensation notable figure for the airspace notable figure；
If it is not, then selecting the compensation notable figure for the time domain notable figure.
Optionally, described second module 431 is obtained according to the intersection notable figure, the compensation notable figure, weighted value, obtain Before obtaining space-time remarkable figure, it is also used to obtain the compactness of the intersection notable figure；
It is described to be also used to judge whether the compactness of the airspace notable figure is less than or equal to for the processing module 432 The compactness of time domain notable figure；If so, determining that the space is consistent to space with the time to time consistency score Property score in minimum value be weighted value of the intersection notable figure in the space-time remarkable figure；If not, it is determined that the friendship Collecting weighted value of the notable figure in the space-time remarkable figure is 0；
Described second obtains module 431, when being also used to be integrated into described according to the preset value and the intersection notable figure Weighted value in empty significant set of graphs, obtains the compensation notable figure and is integrated into the weighted value in the space-time remarkable set of graphs.
It is real to can be used for executing above-mentioned each method for the above-described saliency regional detection device of taking photo by plane of the present embodiment The technical solution in example is applied, it is similar that the realization principle and technical effect are similar, and wherein the function of modules can be implemented with reference to method It is described accordingly in example, details are not described herein again.
Fig. 5 be another embodiment of the present invention provides saliency regional detection device of taking photo by plane structural schematic diagram.Such as Shown in Fig. 5, which can be the chip of the network equipment or the network equipment, and described device can To include: at least one processor 51 and memory 52.Fig. 5 shows the saliency of taking photo by plane taken a processor as an example Regional detection device, wherein
Memory 52, for storing program.Specifically, program may include program code, and said program code includes meter Calculation machine operational order.Memory 52 may include high speed RAM memory, it is also possible to further include nonvolatile memory (non- Volatile memory), a for example, at least magnetic disk storage.
Processor 51, the computer executed instructions stored for executing the memory 52, to realize in above-described embodiment Saliency method for detecting area of taking photo by plane, it is similar that the realization principle and technical effect are similar, and details are not described herein.
Wherein, processor 51 may be graphics processor (Graphics Processing Unit, abbreviation GPU) or one A central processing unit (Central Processing Unit, referred to as CPU) or specific integrated circuit (Application Specific Integrated Circuit, referred to as ASIC), or be arranged to implement the application One or more integrated circuits of embodiment.
Optionally, in specific implementation, if memory 52 and the independent realization of processor 51,
Memory 52 and processor 51 can be connected with each other by bus and complete mutual communication.The bus can be with It is industry standard architecture (Industry Standard Architecture, referred to as ISA) bus, external equipment interconnection (Peripheral Component, referred to as PCI) bus or extended industry-standard architecture (Extended Industry Standard Architecture, referred to as EISA) bus etc..The bus can be divided into address bus, data/address bus, control Bus processed etc., it is not intended that an only bus or a type of bus.
Optionally, it in specific implementation, realizes, stores on one chip if memory 52 and processor 51 integrate Device 52 and processor 51 can be completed by internal interface it is identical between communication.
It is real to can be used for executing above-mentioned each method for the above-described saliency regional detection device of taking photo by plane of the present embodiment The technical solution in example is applied, it is similar that the realization principle and technical effect are similar, and details are not described herein again.
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above-mentioned each method embodiment can lead to The relevant hardware of program instruction is crossed to complete.Program above-mentioned can be stored in a computer readable storage medium.The journey When being executed, execution includes the steps that above-mentioned each method embodiment to sequence；And storage medium above-mentioned include: ROM, RAM, magnetic disk or The various media that can store program code such as person's CD.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution The range of scheme.

Claims (10)

  1. The saliency method for detecting area 1. one kind is taken photo by plane characterized by comprising
    By M ground level classics conspicuousness model, obtains sample data set and correspond to each ground level classics conspicuousness mould The significant set of graphs of the sample of type, the sample data set are the company extracted in video that takes photo by plane from the sample that salient region has marked Continuous sequence of video images；Wherein, M is more than or equal to 2；
    M single channel convolutional neural networks are initialized, and are distinguished using the sample data set and the M significant set of graphs of sample The training M single channel convolutional neural networks, M convolutional neural networks model after obtaining training；
    Utilize N number of single channel convolutional neural networks model initialization N in M single channel convolutional neural networks model after the training Road convolutional neural networks, and the road the N volume using the sample data set training road N convolutional neural networks, after obtaining training Product neural network model；Wherein, N is greater than or equal to 2 and is less than or equal to M；
    According to the road the N convolutional neural networks model after the training, the significant set of graphs in airspace of data to be tested collection is obtained, it is described Data to be tested collection is from the continuous sequence of video images for currently needing to extract in the video of taking photo by plane detected；
    According to the method based on discrete cosine transform, the significant set of graphs of time domain of the data to be tested collection is obtained；
    According to the significant set of graphs in the airspace and the significant set of graphs of the time domain, the space-time remarkable of the data to be tested collection is obtained Set of graphs.
  2. 2. the method according to claim 1, wherein the M single channel convolutional Neural using after the training Before the road N number of single channel convolutional neural networks model initialization N convolutional neural networks in network model, further includes:
    N number of single channel convolution mind is selected from M single channel convolutional neural networks model after the training according to optimization formula Through network model；
    Wherein, optimize formula are as follows: α*=arg max Ωr+λdΩd, wherein α*For the vector of 1*M, wherein element αiWhen equal to 1, Illustrate the single channel convolutional neural networks model for selecting number for i；ΩrIndicate N number of single channel convolutional Neural net of selection The representativeness of network model；ΩdIndicate the diversity of N number of single channel convolutional neural networks model of selection；λdIndicate balance ginseng Number, for balancing representative and diversity.
  3. 3. method according to claim 1 or 2, which is characterized in that described according to the significant set of graphs in the airspace and described The significant set of graphs of time domain obtains the space-time remarkable set of graphs of the data to be tested collection, comprising:
    Obtain the airspace notable figure in the significant set of graphs in the airspace and the time domain notable figure in the significant set of graphs of the time domain Intersection notable figure；
    The time domain notable figure in the airspace notable figure or the significant set of graphs of the time domain in the significant set of graphs in the airspace is selected to make To compensate notable figure；
    According to the intersection notable figure, the compensation notable figure, weighted value, space-time remarkable figure is obtained, the space-time remarkable figure is The data to be tested concentrate the salient region result figure of a wherein image；Wherein, the weighted value is for indicating described Weighted value of the intersection notable figure in the space-time remarkable figure, weighted value of the intersection notable figure in the space-time remarkable figure It is equal to preset value with weighted value the sum of of the compensation notable figure in the space-time remarkable figure；
    The notable figure that the data to be tested concentrate every image is obtained, the space-time remarkable atlas of the data to be tested collection is obtained It closes.
  4. 4. according to the method described in claim 3, it is characterized in that, the airspace obtained in the significant set of graphs in airspace is aobvious Write the intersection notable figure of the time domain notable figure in figure and the significant set of graphs of the time domain, comprising:
    Space is obtained to time consistency score and time to Space Consistency score；
    It is significant that the intersection is obtained to Space Consistency score to time consistency score and the time according to the space Figure.
  5. 5. according to the method described in claim 4, it is characterized in that, the airspace in the selection significant set of graphs in airspace is aobvious The time domain notable figure in figure or the significant set of graphs of the time domain is write as compensation notable figure, comprising:
    The compactness of the airspace notable figure and the time domain notable figure is obtained respectively；
    Judge whether the compactness of the airspace notable figure is less than or equal to the compactness of the time domain notable figure；
    If so, selecting the compensation notable figure for the airspace notable figure；
    If it is not, then selecting the compensation notable figure for the time domain notable figure.
  6. 6. according to the method described in claim 5, it is characterized in that, it is described according to the intersection notable figure, the compensation it is significant Scheme, weighted value, before acquisition space-time remarkable figure, further includes:
    Obtain the compactness of the intersection notable figure；
    Judge the compactness of the intersection notable figure whether be less than airspace notable figure compactness and time domain notable figure it is tight Gather the minimum value of degree and the product of preset threshold；
    If so, determining that the space is to the minimum value in Space Consistency score to time consistency score and the time Weighted value of the intersection notable figure in the space-time remarkable figure；
    If not, it is determined that weighted value of the intersection notable figure in the space-time remarkable figure is 0；
    The weighted value in the space-time remarkable set of graphs is integrated into according to the preset value and the intersection notable figure, described in acquisition Compensation notable figure is integrated into the weighted value in the space-time remarkable set of graphs.
  7. The saliency regional detection device 7. one kind is taken photo by plane characterized by comprising
    First obtains module, for by M ground level classics conspicuousness model, acquisition sample data set correspond to it is each describedly The significant set of graphs of sample of the classical conspicuousness model of face grade, the sample data set are the sample boat marked from salient region Shoot the video the continuous sequence of video images of middle extraction；Wherein, M is more than or equal to 2；
    Training module for initializing M single channel convolutional neural networks, and utilizes the sample data set and the M samples The M single channel convolutional neural networks are respectively trained in significant set of graphs, M convolutional neural networks model after obtaining training；Its In, the M single channel convolutional neural networks structure is identical；
    The training module is also used to utilize N number of single channel convolution in M single channel convolutional neural networks model after the training Neural network model initializes the road N convolutional neural networks, and utilizes the sample data set training road the N convolutional Neural net Network, the road the N convolutional neural networks model after obtaining training；Wherein, N is greater than or equal to 2 and is less than or equal to M；
    Described first obtains module, is also used to obtain data to be tested according to the road the N convolutional neural networks model after the training The significant set of graphs in the airspace of collection, the data to be tested collection are from the continuous view for currently needing to extract in the video of taking photo by plane detected Frequency image sequence；
    Described first obtains module, is also used to obtain the data to be tested collection according to the method based on discrete cosine transform The significant set of graphs of time domain；
    Fusion Module, for according to the significant set of graphs in the airspace and the significant set of graphs of the time domain, obtaining the video of taking photo by plane Space-time remarkable set of graphs.
  8. 8. device according to claim 7, which is characterized in that the Fusion Module, comprising:
    Second obtains module, for obtaining airspace notable figure and the significant set of graphs of the time domain in the significant set of graphs in the airspace In time domain notable figure intersection notable figure；
    Processing module, for selecting in the airspace notable figure or the significant set of graphs of the time domain in the significant set of graphs in the airspace Time domain notable figure is as compensation notable figure；
    Described second obtains module, is also used to obtain space-time according to the intersection notable figure, the compensation notable figure, weighted value Notable figure, the space-time remarkable figure are the salient region result figure that the data to be tested concentrate a wherein image；Wherein, The weighted value is for indicating weighted value of the intersection notable figure in the space-time remarkable figure, and the intersection notable figure is in institute The sum of the weighted value of weighted value and the compensation notable figure in the space-time remarkable figure in space-time remarkable figure is stated equal to default Value；
    Described second obtains module, is also used to obtain the notable figure that the data to be tested concentrate every image, obtain it is described to The space-time remarkable set of graphs of detection data collection.
  9. The saliency regional detection device 9. one kind is taken photo by plane characterized by comprising memory and processor, memory are used In storage program instruction, it is as claimed in any one of claims 1 to 6 that processor is used to call the program instruction in memory to execute Saliency of taking photo by plane method for detecting area.
  10. 10. a kind of readable storage medium storing program for executing, which is characterized in that be stored with computer program on the readable storage medium storing program for executing；The meter Calculation machine program is performed, and realizes saliency method for detecting area as claimed in any one of claims 1 to 6 of taking photo by plane.
CN201811290665.5A 2018-10-31 2018-10-31 Method and device for detecting salient region of aerial video Active CN109543561B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811290665.5A CN109543561B (en) 2018-10-31 2018-10-31 Method and device for detecting salient region of aerial video

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811290665.5A CN109543561B (en) 2018-10-31 2018-10-31 Method and device for detecting salient region of aerial video

Publications (2)

Publication Number Publication Date
CN109543561A true CN109543561A (en) 2019-03-29
CN109543561B CN109543561B (en) 2020-09-18

Family

ID=65845786

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811290665.5A Active CN109543561B (en) 2018-10-31 2018-10-31 Method and device for detecting salient region of aerial video

Country Status (1)

Country Link
CN (1) CN109543561B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104408446A (en) * 2014-12-19 2015-03-11 中国人民解放军国防科学技术大学 Image saliency based method for detecting automatic landing site of UAV (unmanned aerial vehicle)
CN104463890A (en) * 2014-12-19 2015-03-25 北京工业大学 Stereoscopic image significance region detection method
CN105930868A (en) * 2016-04-20 2016-09-07 北京航空航天大学 Low-resolution airport target detection method based on hierarchical reinforcement learning
CN106251355A (en) * 2016-08-03 2016-12-21 江苏大学 A kind of detection method merging visible images and corresponding night vision infrared image
CN106529419A (en) * 2016-10-20 2017-03-22 北京航空航天大学 Automatic detection method for significant stack type polymerization object in video
CN106683046A (en) * 2016-10-27 2017-05-17 山东省科学院情报研究所 Real-time image splicing method for police unmanned aerial vehicle investigation and evidence obtaining
CN106909902A (en) * 2017-03-01 2017-06-30 北京航空航天大学 A kind of remote sensing target detection method based on the notable model of improved stratification
CN106952269A (en) * 2017-02-24 2017-07-14 北京航空航天大学 The reversible video foreground object sequence detection dividing method of neighbour and system
CN108021886A (en) * 2017-12-04 2018-05-11 西南交通大学 A kind of unmanned plane repeats texture image part remarkable characteristic matching process
US20180189573A1 (en) * 2013-05-23 2018-07-05 Sri International Real-time detection, tracking and occlusion reasoning

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180189573A1 (en) * 2013-05-23 2018-07-05 Sri International Real-time detection, tracking and occlusion reasoning
CN104408446A (en) * 2014-12-19 2015-03-11 中国人民解放军国防科学技术大学 Image saliency based method for detecting automatic landing site of UAV (unmanned aerial vehicle)
CN104463890A (en) * 2014-12-19 2015-03-25 北京工业大学 Stereoscopic image significance region detection method
CN105930868A (en) * 2016-04-20 2016-09-07 北京航空航天大学 Low-resolution airport target detection method based on hierarchical reinforcement learning
CN106251355A (en) * 2016-08-03 2016-12-21 江苏大学 A kind of detection method merging visible images and corresponding night vision infrared image
CN106529419A (en) * 2016-10-20 2017-03-22 北京航空航天大学 Automatic detection method for significant stack type polymerization object in video
CN106683046A (en) * 2016-10-27 2017-05-17 山东省科学院情报研究所 Real-time image splicing method for police unmanned aerial vehicle investigation and evidence obtaining
CN106952269A (en) * 2017-02-24 2017-07-14 北京航空航天大学 The reversible video foreground object sequence detection dividing method of neighbour and system
CN106909902A (en) * 2017-03-01 2017-06-30 北京航空航天大学 A kind of remote sensing target detection method based on the notable model of improved stratification
CN108021886A (en) * 2017-12-04 2018-05-11 西南交通大学 A kind of unmanned plane repeats texture image part remarkable characteristic matching process

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
BAK, CAGDAS, ET AL.: "Spatio-temporal saliency networks for dynamic saliency prediction", 《 IEEE TRANSACTIONS ON MULTIMEDIA》 *
ZHANG, L., LI, ET.AL: "Global and local saliency analysis for the extraction of residential areas in high-spatial-resolution remote sensing image", 《IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING》 *
刘君玲: "基于视觉显著性的目标检测方法与应用研究", 《中国优秀博士论文全文数据库 信息科技辑》 *
李大伟: "固定翼无人机地面车辆目标自动检测与跟踪", 《中国优秀博士论文全文数据库 信息科技辑》 *

Also Published As

Publication number Publication date
CN109543561B (en) 2020-09-18

Similar Documents

Publication Publication Date Title
Tang et al. Improving image classification with location context
CN113505792B (en) Multi-scale semantic segmentation method and model for unbalanced remote sensing image
CN109003297B (en) Monocular depth estimation method, device, terminal and storage medium
CN108710863A (en) Unmanned plane Scene Semantics dividing method based on deep learning and system
CN108345875A (en) Wheeled region detection model training method, detection method and device
CN107609485A (en) The recognition methods of traffic sign, storage medium, processing equipment
CN108256404A (en) Pedestrian detection method and device
CN109919110B (en) Video attention area detection method, device and equipment
CN111784623A (en) Image processing method, image processing device, computer equipment and storage medium
CN111523449A (en) Crowd counting method and system based on pyramid attention network
CN111160351B (en) Fast high-resolution image segmentation method based on block recommendation network
CN106909886A (en) A kind of high accuracy method for traffic sign detection and system based on deep learning
CN112330684B (en) Object segmentation method and device, computer equipment and storage medium
CN111310518B (en) Picture feature extraction method, target re-identification method, device and electronic equipment
CN109410190B (en) Tower pole reverse-breaking detection model training method based on high-resolution remote sensing satellite image
CN110414593B (en) Image processing method and device, processor, electronic device and storage medium
CN111814754A (en) Single-frame image pedestrian detection method and device for night scene
CN113724128A (en) Method for expanding training sample
CN104463240A (en) Method and device for controlling list interface
CN110244728A (en) Determine the method, apparatus, equipment and storage medium of unmanned control strategy
CN107003736A (en) For the method and apparatus for the status data for obtaining instruction user state
CN115496891A (en) Wheat lodging degree grading method and device
CN115661767A (en) Image front vehicle target identification method based on convolutional neural network
CN117953299A (en) Land utilization classification method based on multi-scale remote sensing images
CN109359727A (en) Structure determination methodology, device, equipment and the readable medium of neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant