CN102855630A - Method for judging image memorability based on saliency entropy and object bank feature - Google Patents
Method for judging image memorability based on saliency entropy and object bank feature Download PDFInfo
- Publication number
- CN102855630A CN102855630A CN2012102986764A CN201210298676A CN102855630A CN 102855630 A CN102855630 A CN 102855630A CN 2012102986764 A CN2012102986764 A CN 2012102986764A CN 201210298676 A CN201210298676 A CN 201210298676A CN 102855630 A CN102855630 A CN 102855630A
- Authority
- CN
- China
- Prior art keywords
- image
- saliency
- memorability
- entropy
- width
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to a method for judging image memorability based on saliency entropy and object bank feature. A research proves that the image memorability can be influenced by an object contained in an image and the dispersing degree of the image to human visual attraction, so that the object bank feature is used for expressing the object contained in the image and the visual saliency entropy of the image is used for expressing the dispersing degree of the image to the human visual attraction. The two models have excellent expression for the image memorability, so that the two features are combined for expressing a piece of image, and a memorability value of the image with unknown memorability value is forecast through a training supporting regression vector machine model. The method provided by the invention belongs to the field of computer image processing. According to the technical scheme, the image memorability can be judged; the method can be applied to the industries such as advertising industry and news editing; a suitable image can be selected by a practitioner; and the method has a high commercial value.
Description
Technical field
The present invention relates to a kind of iconic memory sex determination method based on saliency entropy and object bank feature, can be applied to all kinds of visible images, the Memorability numerical value of process decision chart picture.
Background technology
The Memorability of image is the new research direction of digital image processing field, and it has a lot of application.Can select easily the image remembered by people as the front cover of magazine such as editor, the image that the advertising man can select easily to be remembered is as propagating poster etc.Therefore, when given piece image, if general-purpose computers determine automatically that can it be remembered by people will be highly significant.
Because the Memorability research of image is a newer research direction, so in the research of this problem, method is not a lot.The certain methods that exists is at present at first extracted the global characteristics (such as SIFT, GIST, HOG etc.) of image, by making up sorter and training pattern, then differentiates the Memorability of a given image.Yet because the Memorability of image is a very complicated problem, be difficult to represent this characteristic with the global characteristics of image, so the judgement effect of the method for existence is relatively poor at present.Therefore need to carry out deep research to the Memorability of image, propose the method for new iconic memory sex determination.
Summary of the invention
The technical matters that solves
In order to solve the deficiencies in the prior art part, the present invention proposes a kind of iconic memory sex determination method based on saliency entropy and object bank feature.
Thought of the present invention is: by studies show that the object and the image that comprise in the image degree of scatter that human vision attracts all can be exerted an influence to the Memorability of image, so object to comprising in the image, we represent with object bank feature, to the degree of scatter that image attracts human vision, we represent with the vision saliency entropy of image.Because these two models all have good expression to the Memorability of image, therefore with two kinds of feature combinations, can obtain better iconic memory sex determination effect.
Technical scheme
A kind of decision method of the iconic memory based on saliency entropy and object bank feature is characterized in that:
Step 1: the saliency entropy that extracts object bank feature and the computed image of each width of cloth input picture;
Extract the object bank feature of each width of cloth input picture, concrete step is as follows:
Step a: utilize Li-Jia Li at the object bank routine package of issue in 2010, to each width of cloth input picture, utilize the down-sampling technology to obtain 12 scalogram pictures of input picture, and 208 object templates in these 12 scalogram pictures and the object bank routine package are carried out convolutional calculation, make every width of cloth input picture obtain 208 * 12 width of cloth response images;
Step b: utilize two interpolation methods, with the response image interpolation of input picture corresponding to 12 yardsticks of each template, obtain the image of same size; Then to each pixel of the image of same size, calculate its maximal value on the scalogram picture of 12 same scale that obtain after the interpolation, consist of a peak response image; Then ask the pixel average of peak response image, obtain the character representation of one 208 dimension of every width of cloth input picture, be the object bank feature of input picture;
The saliency entropy of computed image: at first extract the saliency image of input picture, then the saliency image binaryzation that obtains is obtained bianry image; Calculate bianry image not connected region number with and corresponding area, the saliency entropy of computed image then, computation model is:
Wherein, m is the number of connected region not in the bianry image; S
jIt is each not area of connected region; S is the area of bianry image; P
jIt is the ratio of connected region area and the area of whole image-region not; H represents the saliency entropy of bianry image;
Step 2 model training: with the Memorability numerical value of given training image as training sample, utilize the method for step 1, extract the object bank feature of image in the training sample and the feature of saliency entropy, then consist of the proper vector of one 209 dimension, utilize the Memorability numerical value of image in the training sample as label, training obtains a support vector regression model;
Step 3: for the image an of width of cloth Memorability numerical value the unknown, by the feature of step 1 its object bank feature of extraction and saliency entropy, the support vector regression model that then utilizes step 2 to obtain is judged the Memorability numerical value that obtains this width of cloth image.
Beneficial effect
The iconic memory sex determination method based on vision saliency entropy and object bank feature that the present invention proposes, object bank feature by extracting image and to the calculating of the saliency entropy of image can better be studied processing to this problem of iconic memory.Because this model of saliency entropy is relevant with people's vision noticing mechanism and memory mechanism, the object that comprises in the image simultaneously also can exert an influence to the Memorability of image, object bank feature then is a kind of reliable expression to the object that comprises in the image, so this method has obtained and judges preferably effect.Because two kinds of methods that the present invention adopts can be good at the lower-order questions of presentation video, therefore can obtain better result than the method for the global characteristics of existing extraction image.
Description of drawings
Fig. 1: the basic flow sheet of the inventive method
Fig. 2: the feature of saliency entropy is obtained figure in the inventive method
Fig. 3: the feature of object bank is obtained figure in the inventive method
Fig. 4: use this method process decision chart as the example of Memorability numerical value
(a) training image for example with and corresponding Memorability numerical value (need the great amount of images training, just select one here as an example).
(b) test pattern with and the Memorability numerical value judged with the inventive method.
Embodiment
Now in conjunction with the embodiments, the invention will be further described for accompanying drawing:
The hardware environment that is used for implementing is: Pentium-43G computing machine, 1GB internal memory, 128M video card.The software environment of operation is: Matlab7.0 and Windows XP, Ubuntu12.04.We have realized the method that the present invention proposes with Matlab software.
Implementation of the present invention is as follows:
1, experimental data: we select 2222 width of cloth images that Phillip announced in 2011 and the corresponding Memorability numerical value of every width of cloth image as experimental data.Get at random 1111 width of cloth images as training data, other 1111 width of cloth images are as test data.The data of Phillip are seen paper: Phillip I, Jianxiong X, Antonio T, et al.What makes an image memorable[C] .CVPR, 2011,145-152.
2, feature extraction: to 2222 width of cloth experimental image extract its object bank feature with and saliency entropy feature
(1) obtaining of object bank feature: utilize Li-Jia Li at the object bank routine package of issue in 2010, to each width of cloth input picture, utilize the down-sampling technology to obtain 12 scalogram pictures of input picture, and 208 object templates in these 12 scalogram pictures and the object bank program are carried out convolutional calculation, make every width of cloth input picture obtain 208 * 12 width of cloth response images; Utilize two interpolation methods, with the response image interpolation of input picture corresponding to 12 yardsticks of each template, obtain the image of same size; Then to each pixel of the image of same size, calculate its maximal value on the scalogram picture of 12 same scale that obtain after the interpolation, consist of a peak response image; Then ask the pixel average of peak response image, obtain the character representation of one 208 dimension of every width of cloth input picture, be the object bank feature of input picture;
The object bank program that described Li-Jia Li announced in 2010 is seen paper: Li-Jia L, Hao S, Eric X, et al.Object bank:A high-level image representation for scene classification and semantic feature sparsification[C] .NIPS, 2010.
(2) obtaining of saliency entropy: the method that we utilize Tilke to propose in this example is obtained the saliency(saliency of input picture) image.Then the saliency image binaryzation to obtaining is got threshold value and is
Obtain the bianry image of saliency image corresponding to input picture, to bianry image, the saliency entropy of formula calculating input image below utilizing
Wherein, m is the number of connected region not in the bianry image, S
jBe each not area of connected region, S is the area of bianry image, P
jThat the connected region area does not account for the ratio of the area in whole zone, i.e. probability.H represents the entropy of bianry image.
The method that described Tilke proposes is seen paper: Tilke J, Krista E, Fredo D, et al.Learning to Predict Where Humans Look[C] .ICCV, 2009,2106-2113.
3, model training: take in 1111 width of cloth training images of training with and corresponding Memorability numerical value, utilize the method in step 1 and the step 2, extract the object bank feature of training sample image, and the feature of saliency entropy, then consist of the proper vector of one 209 dimension, the Memorability numerical value of image is as label, train a support vector regression model, here realize with the libSVM software package, parameter is set to '-s3-p0.01-t0-c100 ', the Memorability numerical value that is used for image is judged.
4, the Memorability numerical value of image is judged: for the image an of width of cloth Memorability numerical value the unknown, extract the feature of its object bank feature and saliency entropy, then the support vector regression model that utilizes model training to obtain can be judged the Memorability numerical value that obtains this width of cloth image.
5, the calculating of relative coefficient: in order to verify the effect of the inventive method, the relative coefficient of computational discrimination value and actual value.To 2222 width of cloth experimental image, get at random 1111 width of cloth as training data, 1111 width of cloth extract the feature based on saliency entropy and object bank of each width of cloth image as test data.With training data training support vector regression model, then judge the Memorability numerical value of test data, then calculate related coefficient, the computing formula of related coefficient is as follows:
X wherein
iThe Memorability true value of expression test pattern, Y
iThe Memorability decision content of expression test pattern,
The mean value that represents the Memorability true value of 1111 width of cloth test patterns,
The mean value that represents the Memorability decision content of 1111 width of cloth test patterns, N=1111 represent to have 1111 width of cloth test patterns, and ρ represents related coefficient.Robustness in order to ensure experiment repeats above-mentioned experimentation 25 times, calculates the mean value of related coefficient, can obtain the decision content of the inventive method experiment and the average correlation coefficient of actual value.Table 1 has shown the average correlation coefficient of distinct methods, and average correlation coefficient is larger, shows decision content more near actual value, and that namely judges is more accurate.
The comparison of table 1 relative coefficient
Claims (2)
1. decision method based on the iconic memory of saliency entropy and object bank feature is characterized in that:
Step 1: the saliency entropy that extracts object bank feature and the computed image of each width of cloth input picture;
Extract the object bank feature of each width of cloth input picture, concrete step is as follows:
Step a: utilize Li-Jia Li at the object bank routine package of issue in 2010, to each width of cloth input picture, utilize the down-sampling technology to obtain 12 scalogram pictures of input picture, and 208 object templates in these 12 scalogram pictures and the object bank routine package are carried out convolutional calculation, make every width of cloth input picture obtain 208 * 12 width of cloth response images;
Step b: utilize two interpolation methods, with the response image interpolation of input picture corresponding to 12 yardsticks of each template, obtain the image of same size; Then to each pixel of the image of same size, calculate its maximal value on the scalogram picture of 12 same scale that obtain after the interpolation, consist of a peak response image; Then ask the pixel average of peak response image, obtain the character representation of one 208 dimension of every width of cloth input picture, be the object bank feature of input picture;
The saliency entropy of computed image: at first extract the saliency image of input picture, then the saliency image binaryzation that obtains is obtained bianry image; Calculate bianry image not connected region number with and corresponding area, the saliency entropy of computed image then, computation model is:
Wherein, m is the number of connected region not in the bianry image; S
jIt is each not area of connected region; S is the area of bianry image; P
jIt is the ratio of connected region area and the area of whole image-region not; H represents the saliency entropy of bianry image;
Step 2 model training: with the Memorability numerical value of given training image as training sample, utilize the method for step 1, extract the object bank feature of image in the training sample and the feature of saliency entropy, then consist of the proper vector of one 209 dimension, utilize the Memorability numerical value of image in the training sample as label, training obtains a support vector regression model;
Step 3: for the image an of width of cloth Memorability numerical value the unknown, by the feature of step 1 its object bank feature of extraction and saliency entropy, the support vector regression model that then utilizes step 2 to obtain is judged the Memorability numerical value that obtains this width of cloth image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2012102986764A CN102855630A (en) | 2012-08-21 | 2012-08-21 | Method for judging image memorability based on saliency entropy and object bank feature |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2012102986764A CN102855630A (en) | 2012-08-21 | 2012-08-21 | Method for judging image memorability based on saliency entropy and object bank feature |
Publications (1)
Publication Number | Publication Date |
---|---|
CN102855630A true CN102855630A (en) | 2013-01-02 |
Family
ID=47402192
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2012102986764A Pending CN102855630A (en) | 2012-08-21 | 2012-08-21 | Method for judging image memorability based on saliency entropy and object bank feature |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN102855630A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103440496A (en) * | 2013-08-01 | 2013-12-11 | 西北工业大学 | Video memorability discrimination method based on functional magnetic resonance imaging |
CN107341505A (en) * | 2017-06-07 | 2017-11-10 | 同济大学 | A kind of scene classification method based on saliency Yu Object Bank |
RU2708197C1 (en) * | 2018-12-21 | 2019-12-04 | Акционерное общество "Нейротренд" | Method of measuring memorability of a multimedia message |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050069206A1 (en) * | 2003-09-30 | 2005-03-31 | Yu-Fei Ma | Contrast-based image attention analysis framework |
CN101984464A (en) * | 2010-10-22 | 2011-03-09 | 北京工业大学 | Method for detecting degree of visual saliency of image in different regions |
-
2012
- 2012-08-21 CN CN2012102986764A patent/CN102855630A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050069206A1 (en) * | 2003-09-30 | 2005-03-31 | Yu-Fei Ma | Contrast-based image attention analysis framework |
CN101984464A (en) * | 2010-10-22 | 2011-03-09 | 北京工业大学 | Method for detecting degree of visual saliency of image in different regions |
Non-Patent Citations (4)
Title |
---|
LI-JIA LI ET AL.: "Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification", 《NEURAL INFORMATION PROCESSING SYSTEM CONFERENCE 2010》, 31 December 2010 (2010-12-31), pages 1 - 9 * |
PHILLIP ISOLA ET AL.: "What makes an image memorable?", 《2011 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION(CVPR)》, 25 June 2011 (2011-06-25), pages 145 - 152, XP032038223, DOI: doi:10.1109/CVPR.2011.5995721 * |
TILKE JUDD ET AL.: "Learning to Predict Where Humans Look", 《2009 IEEE 12TH INTERNATIONAL CONFERENCE ON COMPUTER VISION》, 2 October 2009 (2009-10-02), pages 2106 - 2113 * |
陈长远等: "基于视觉显著熵与Object Bank特征的图像记忆性模型", 《计算机应用》, vol. 33, no. 11, 1 November 2013 (2013-11-01) * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103440496A (en) * | 2013-08-01 | 2013-12-11 | 西北工业大学 | Video memorability discrimination method based on functional magnetic resonance imaging |
CN103440496B (en) * | 2013-08-01 | 2016-07-13 | 西北工业大学 | A kind of video memorability method of discrimination based on functional mri |
CN107341505A (en) * | 2017-06-07 | 2017-11-10 | 同济大学 | A kind of scene classification method based on saliency Yu Object Bank |
CN107341505B (en) * | 2017-06-07 | 2020-07-28 | 同济大学 | Scene classification method based on image significance and Object Bank |
RU2708197C1 (en) * | 2018-12-21 | 2019-12-04 | Акционерное общество "Нейротренд" | Method of measuring memorability of a multimedia message |
WO2020130870A1 (en) | 2018-12-21 | 2020-06-25 | Акционерное общество "Нейротренд" | Method for measuring the memorability of a multimedia message |
CN111787860A (en) * | 2018-12-21 | 2020-10-16 | 尼罗特兰德股份公司 | Measuring method for storing multimedia message |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107563385B (en) | License plate character recognition method based on depth convolution production confrontation network | |
Yang et al. | Face recognition using adaptive local ternary patterns method | |
CN106156766A (en) | The generation method and device of line of text grader | |
CN112528997B (en) | Tibetan-Chinese bilingual scene text detection method based on text center region amplification | |
CN105608456A (en) | Multi-directional text detection method based on full convolution network | |
CN102938054A (en) | Method for recognizing compressed-domain sensitive images based on visual attention models | |
CN104298981A (en) | Face microexpression recognition method | |
CN102129568B (en) | Method for detecting image-based spam email by utilizing improved gauss hybrid model classifier | |
CN101930549A (en) | Second generation curvelet transform-based static human detection method | |
CN106021603A (en) | Garment image retrieval method based on segmentation and feature matching | |
CN103186776B (en) | Based on the human body detecting method of multiple features and depth information | |
CN103853724A (en) | Multimedia data sorting method and device | |
CN103617413A (en) | Method for identifying object in image | |
CN104573711A (en) | Object and scene image understanding method based on text-object-scene relations | |
CN104239852A (en) | Infrared pedestrian detecting method based on motion platform | |
CN106529377A (en) | Age estimating method, age estimating device and age estimating system based on image | |
CN104239859A (en) | Face recognition method based on structuralized factor analysis | |
CN112949408A (en) | Real-time identification method and system for target fish passing through fish channel | |
CN105893952A (en) | Hand-written signature identifying method based on PCA method | |
CN104992166A (en) | Robust measurement based handwriting recognition method and system | |
CN102855630A (en) | Method for judging image memorability based on saliency entropy and object bank feature | |
CN101520839A (en) | Human body detection method based on second-generation strip wave conversion | |
CN103186777B (en) | Based on the human body detecting method of Non-negative Matrix Factorization | |
CN102262731B (en) | Character recognizing method based on sparse coding | |
CN101819637B (en) | Method for detecting image-based spam by utilizing image local invariant feature |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20130102 |