WO2017174982A1 - Procédé de mise en correspondance d'une image de croquis et d'une image de visage - Google Patents

Procédé de mise en correspondance d'une image de croquis et d'une image de visage Download PDF

Info

Publication number
WO2017174982A1
WO2017174982A1 PCT/GB2017/050951 GB2017050951W WO2017174982A1 WO 2017174982 A1 WO2017174982 A1 WO 2017174982A1 GB 2017050951 W GB2017050951 W GB 2017050951W WO 2017174982 A1 WO2017174982 A1 WO 2017174982A1
Authority
WO
WIPO (PCT)
Prior art keywords
sketch
sketches
database
forensic
matching
Prior art date
Application number
PCT/GB2017/050951
Other languages
English (en)
Inventor
Yi-zhe SONG
Timothy HOSPEDALES
Original Assignee
Queen Mary University Of London
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Queen Mary University Of London filed Critical Queen Mary University Of London
Publication of WO2017174982A1 publication Critical patent/WO2017174982A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Definitions

  • forensic sketch databases are few and small in size.
  • the main sketch/photo databases are 159 pairs identified by [reference 12], and 190 pairs in the IIIT-D database [reference 2].
  • a realistic evaluation of sketch-based face matching should also include a large pool of mugshots to match against, in addition to the true photo corresponding to each sketch.
  • Reference 12 which trained a matching model on viewed sketches and then tested matching 159 forensic sketches against corresponding photos and a 10,030 mugshot database.
  • Regression models are widely used in cross-domain face recognition [reference 17].
  • regression models may provide facial sketch ->photo synthesis [reference 22] to support matching, for example via support vector regression (SVR)
  • a method of training a machine- learning algorithm using a training database comprising a plurality of records, each record comprising data representing:
  • the plurality of sketches of one record are made by the same artist.
  • the machine learning algorithm is a Gaussian Process Regression algorithm and desirably a multi-task learning algorithm.
  • photographic images comprises performing nearest neighbour matching between the reconstructed sketch and the database of photographic images.
  • Figure 2 illustrates facial regions
  • Figure 3 is a chart comparing root mean square error (RMSE) averaged across full face for learned reconstruction and original sketches
  • Figure 5 shows qualitative results of MTL-GPR model
  • Figure 6 shows CMC curves for matching Good (49) / All (195) forensic sketches against corresponding photos and 10,030 FSMD database mugshots; and
  • Figure 7 depicts a matching system according to an embodiment of the invention. DESCRIPTION OF EMBODIMENTS
  • GPR Gaussian Process Regression
  • the above procedure describes finding the single best match.
  • a ranked list of a predetermined number of the most likely putative matches can be generated for manual inspection.
  • a sketch matching system is shown schematically in Figure 7.
  • An input sketch 10 which is to be matched against mugshots stored in image database 40, is processed in reconstruction unit 20, which contains a trained ML algorithm as described above, to generate a reconstructed sketch for matching purposes.
  • the reconstructed sketch may be expressed in the form of a histogram of gradients.
  • Matching unit 30 uses the reconstructed sketch to find one or more matching photographs in the image database 40. Metadata 11 relating to the sketch and/or the assumed subject of the sketch, can also be taken into account in the matching process.
  • a ranked list 50 of potential matches is output.
  • the present invention addresses two problems: improving facial sketches whose quality is impacted by a large delay between seeing the face and making the sketch; and improving practical forensic sketch recognition.
  • Embodiments of the present invention are able to improve facial sketches drawn after a time-delay, and this translates into the significantly improved performance on the important task of forensic sketch matching.
  • embodiments address the cross-modal and communication gaps only implicitly via MTL sharing.
  • a richer framework more explicitly modelling the contributing factors can also be used.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

L'efficacité de mise en correspondance d'un croquis médico-légal de visage automatisé est améliorée par l'apprentissage des exemples d'oubli du visage au fil du temps. La reconnaissance de croquis médico-légal de visage est une capacité essentielle pour l'application de la loi, mais constitue encore un problème non résolu. Ladite reconnaissance est extrêmement compliquée du fait que trois facteurs distincts contribuent aux écarts de domaines entre les croquis médico-légaux et les photos : l'écart de modalités entre photos et croquis, qui fait l'objet de nombreuses études, et les lacunes moins documentées qui sont dues (i) au processus d'oubli des témoins oculaires et (ii) à leur incapacité à élucider leur mémoire. Une base de données de croquis médico-légaux, créés à différents intervalles de temps, est utilisée pour former un modèle destiné à inverser le processus d'oubli. De manière surprenante, ceci permet à un modèle de se rappeler systématiquement de détails oubliés du visage. Ce modèle est appliqué pour améliorer considérablement la reconnaissance d'un croquis médico-légal dans la pratique.
PCT/GB2017/050951 2016-04-06 2017-04-05 Procédé de mise en correspondance d'une image de croquis et d'une image de visage WO2017174982A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB201605837 2016-04-06
GB1605837.2 2016-04-06

Publications (1)

Publication Number Publication Date
WO2017174982A1 true WO2017174982A1 (fr) 2017-10-12

Family

ID=58632528

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2017/050951 WO2017174982A1 (fr) 2016-04-06 2017-04-05 Procédé de mise en correspondance d'une image de croquis et d'une image de visage

Country Status (1)

Country Link
WO (1) WO2017174982A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109920021A (zh) * 2019-03-07 2019-06-21 华东理工大学 一种基于正则化宽度学习网络的人脸素描合成方法
WO2020037937A1 (fr) * 2018-08-20 2020-02-27 深圳壹账通智能科技有限公司 Procédé et appareil de reconnaissance faciale, terminal, et support de stockage lisible par ordinateur
CN111274883A (zh) * 2020-01-10 2020-06-12 杭州电子科技大学 基于多尺度hog特征和深层特征的合成素描人脸识别方法
US10817752B2 (en) 2018-05-31 2020-10-27 Toyota Research Institute, Inc. Virtually boosted training
CN112733664A (zh) * 2020-12-31 2021-04-30 北京华安信联通信技术有限公司 一种照片分类方法
JP2021530045A (ja) * 2019-03-22 2021-11-04 ベイジン センスタイム テクノロジー デベロップメント カンパニー， リミテッド 顔認識方法及び装置

Non-Patent Citations (30)

* Cited by examiner, † Cited by third party
Title
"Network and Parallel Computing", vol. 9004, 1 January 2015, SPRINGER INTERNATIONAL PUBLISHING, Cham, ISBN: 978-3-642-38997-9, ISSN: 0302-9743, article SHUXIN OUYANG ET AL: "Cross-Modal Face Matching: Beyond Viewed Sketches", pages: 210 - 225, XP055384858, 032548, DOI: 10.1007/978-3-319-16808-1_15 *
A. KUMAR; H. D. III: "Learning task grouping and overlap in multi-task learning", ICML, 2012
A. SHARMA; D. W. JACOBS: "Bypassing synthesis pls for face recognition with pose, low-resolution and sketch", CVPR, 2011
A. W. YOUNG; D. HAY; K. H. MCWEENY; B. M. FLUDE; A. W. ELLIS: "Matching familiar and unfamiliar faces on internal and external features", PERCEPTION, 1985
B. F. KLARE; A. K. JAIN: "Heterogeneous face recognition using kernel prototype similarities", TPAMI, 2013
B. F. KLARE; Z. LI; A. K. JAIN: "Matching forensic sketches to mug shot photos", TPAMI, 2011
BRITISH JOURNAL OF PSYCHOLOGY, 2007
C. E. RASMUSSEN; C. K. I. WILLIAMS, GAUSSIAN PROCESSES FOR MACHINE LEARNING. IN GAUSSIAN PROCESSES FOR MACHINE LEARNING, 2006
C. FROWD, INTRODUCTION TO APPLIED PSYCHOLOGY, CHAPTER EYEWITNESSES AND THE USE AND APPLICATION OF COGNITIVE THEORY, 2011
C. FROWD; W. ERICKSON; J. LAMPINEN; F. SKELTON; A. MCINTYRE; P. HANCOCK: "A decade of evolving composite techniques: Regression-and meta-analysis.", JOURNAL OF FORENSIC PRACTICE, 2015
C. VONDRICK; A. KHOSLA; T. MALISIEWICZ; A. TORRALBA: "Hoggles: Visualizing object detection features", ICCV, 2013
CHARLIE FROWD ET AL: "A Decade of Evolving Composite Techniques: Regression-and Meta-Analysis", 12 April 2015 (2015-04-12), XP055385330, Retrieved from the Internet <URL:https://dspace.stir.ac.uk/bitstream/1893/21802/1/Frowd A decade of evolving composite techniques_ regression- and meta-analysis.pdf> [retrieved on 20170627] *
E. V. BONILLA; K. M. A. CHAI; C. K. I. WILLIAMS: "Multi-task gaussian process prediction", NIPS, 2008
G. HU; Y. YANG; D. YI; J. KITTLER; W. CHRISTMAS; S. Z. LI; T. M. HOSPEDALES: "When face recognition meets with deep learning: an evaluation of convolutional neural networks for face recognition", ICCV WORKSHOPS CHALEARN LOOKING AT PEOPLE, 2015
GONG YUNCHAO ET AL: "A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics", INTERNATIONAL JOURNAL OF COMPUTER VISION, KLUWER ACADEMIC PUBLISHERS, NORWELL, US, vol. 106, no. 2, 2 October 2013 (2013-10-02), pages 210 - 233, XP035362706, ISSN: 0920-5691, [retrieved on 20131002], DOI: 10.1007/S11263-013-0658-4 *
H. GALOOGAHI; T. SIM: "Inter-modality face sketch recognition", ICME, 2012
H. S. BHATT; S. BHARADWAJ; R. SINGH; M. VATSA: "Memetically optimized MCWLD for matching sketches with digital face images", TIFS, 2012
J. CHOI; A. SHARMA; D. W. JACOBS; L. S. DAVIS: "Data insufficiency in sketch versus photo face recognition", CVPR, 2012
J. ZHANG; N. WANG; X. GAO; D. TAO; X. LI: "Face sketch-photo synthesis based on support vector regression", ICIP, 2011
N. KUMAR; A. C. BERG; P. N. BELHUMEUR; S. K. NAYAR: "Attribute and simile classifiers for face verification", ICCV, 2009
N. WANG; D. TAO; X. GAO; X. LI; J. LI: "A comprehensive survey to face hallucination", IJCV, 2014
P. LUO; X. WANG; X. TANG: "A deep sum-product architecture for robust facial attributes analysis", ICCV, 2013
PIYUSH RAI ET AL: "Multi-label Prediction via Sparse Infinite CCA", NIPS 2009, 1 January 2009 (2009-01-01), XP055385101, Retrieved from the Internet <URL:http://www.cs.utah.edu/~piyush/recent/mlp-iccaNIPS09.pdf> [retrieved on 20170626] *
R. G. U. JR; N. DA VICTORIA LOBO: "A framework for recognizing a facial image from a police sketch", CVPR, 1996
S. OUYANG; T. HOSPEDALES; Y.-Z. SONG; X. LI; C. C. LOY; X. WANG: "A Survey on Heterogeneous Face Recognition: Sketch, Infra-red, 3D and Low-resolution", IMAGE AND VISION COMPUTING, 2016
S. OUYANG; T. M. HOSPEDALES; Y.-Z. SONG; X. LI: "Cross-modal face matching: Beyond viewed sketches", ACCV, 2014
SUN SHILIANG ED - ZENG ZHIGANG ET AL: "A survey of multi-view machine learning", NEURAL COMPUTING AND APPLICATIONS, SPRINGER LONDON, LONDON, vol. 23, no. 7, 17 February 2013 (2013-02-17), pages 2031 - 2038, XP035353972, ISSN: 0941-0643, [retrieved on 20130217], DOI: 10.1007/S00521-013-1362-6 *
X. WANG; X. TANG: "Face photo-sketch synthesis and recognition", TPAMI, 2009
Y. YANG; T. M. HOSPEDALES: "A unified perspective on multi-domain and multi-task learning", ICLR, 2015
Z. LEI, M; PIETIKAINEN; S. Z. LI: "Learning discriminant face descriptor", TPAMI, 2014

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10817752B2 (en) 2018-05-31 2020-10-27 Toyota Research Institute, Inc. Virtually boosted training
WO2020037937A1 (fr) * 2018-08-20 2020-02-27 深圳壹账通智能科技有限公司 Procédé et appareil de reconnaissance faciale, terminal, et support de stockage lisible par ordinateur
CN109920021A (zh) * 2019-03-07 2019-06-21 华东理工大学 一种基于正则化宽度学习网络的人脸素描合成方法
CN109920021B (zh) * 2019-03-07 2023-05-23 华东理工大学 一种基于正则化宽度学习网络的人脸素描合成方法
JP2021530045A (ja) * 2019-03-22 2021-11-04 ベイジン センスタイム テクノロジー デベロップメント カンパニー， リミテッド 顔認識方法及び装置
JP7038867B2 (ja) 2019-03-22 2022-03-18 ベイジン センスタイム テクノロジー デベロップメント カンパニー， リミテッド 顔認識方法及び装置
CN111274883A (zh) * 2020-01-10 2020-06-12 杭州电子科技大学 基于多尺度hog特征和深层特征的合成素描人脸识别方法
CN112733664A (zh) * 2020-12-31 2021-04-30 北京华安信联通信技术有限公司 一种照片分类方法
CN112733664B (zh) * 2020-12-31 2024-04-16 北京华安信联通信技术有限公司 一种照片分类方法

Similar Documents

Publication Publication Date Title
Ouyang et al. Forgetmenot: Memory-aware forensic facial sketch matching
Rothe et al. Deep expectation of real and apparent age from a single image without facial landmarks
WO2017174982A1 (fr) Procédé de mise en correspondance d&#39;une image de croquis et d&#39;une image de visage
Zhang et al. Detection of co-salient objects by looking deep and wide
Satta et al. Fast person re-identification based on dissimilarity representations
CN110503076B (zh) 基于人工智能的视频分类方法、装置、设备和介质
Gao et al. Joint metric learning-based class-specific representation for image set classification
Sapkota et al. Large scale unconstrained open set face database
Martinel et al. Person re-identification via unsupervised transfer of learned visual representations
Munir et al. Multi branch siamese network for person re-identification
An Pedestrian Re‐Recognition Algorithm Based on Optimization Deep Learning‐Sequence Memory Model
Lin et al. Lightweight face anti-spoofing network for telehealth applications
Qin et al. Label enhancement-based multiscale transformer for palm-vein recognition
Liu et al. A new patch selection method based on parsing and saliency detection for person re-identification
Sokolova et al. Methods of gait recognition in video
Goud et al. Smart attendance notification system using SMTP with face recognition
Qu et al. Source-free Style-diversity Adversarial Domain Adaptation with Privacy-preservation for person re-identification
Jalal et al. Suspect face retrieval using visual and linguistic information
Padilha et al. Unraveling the Notre-Dame Cathedral fire in space and time: an X-coherence approach
Moreira et al. Image provenance analysis
Bahrum et al. A systematic review of face sketch recognition system
Acharjya et al. A review on forensic science and criminal investigation through a deep learning framework
Sendik et al. What's in a Face? Metric Learning for Face Characterization
Pellegrini et al. Detecting morphing attacks via continual incremental training
Wang et al. Locality constrained joint dynamic sparse representation for local matching based face recognition

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17719305

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17719305

Country of ref document: EP

Kind code of ref document: A1