CN104361332A - Human face eye region positioning method for fatigue driving detection - Google Patents

Human face eye region positioning method for fatigue driving detection Download PDF

Info

Publication number
CN104361332A
CN104361332A CN201410739606.7A CN201410739606A CN104361332A CN 104361332 A CN104361332 A CN 104361332A CN 201410739606 A CN201410739606 A CN 201410739606A CN 104361332 A CN104361332 A CN 104361332A
Authority
CN
China
Prior art keywords
face
value
characteristic area
pixel
gray
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201410739606.7A
Other languages
Chinese (zh)
Other versions
CN104361332B (en
Inventor
唐云建
胡晓力
莫斌
余名
董宁
韩鹏
孙怀义
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Academy of Science and Technology
Original Assignee
Chongqing Academy of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Academy of Science and Technology filed Critical Chongqing Academy of Science and Technology
Priority to CN201410739606.7A priority Critical patent/CN104361332B/en
Publication of CN104361332A publication Critical patent/CN104361332A/en
Application granted granted Critical
Publication of CN104361332B publication Critical patent/CN104361332B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/197—Matching; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a human face eye region positioning method for fatigue driving detection. Under the condition that the head of a driver is relatively static and moves slowly, a human face detection process which needs a lot of calculating resources consumed by a cascade classifier can be omitted, and a face matching template is matched and a human face eye region is positioned quickly according to matching parameters of adjacent frame video images directly, while under the condition that the head of the driver relatively quickly moves, although the positioning efficiency is affected to a certain extent by reusing the cascade classifier for detecting the human face image region, no substantial influence is generated for the fatigue driving alarm function. Moreover, the relative position relationship of each face characteristic region in the human face image region is further determined and verified by virtue of the face matching template and the accuracy of a positioning result is ensured. By adopting a pixel gray level-based matching mode in matching, the data processing amount is small, the executing efficiency is relatively high and the human face eye region positioning speed is enhanced simultaneously when the relatively high accuracy is ensured.

Description

A kind of face eye areas localization method detected for fatigue driving
Technical field
The present invention relates to and belong to image procossing and mode identification technology, be specifically related to a kind of face eye areas localization method detected for fatigue driving.
Background technology
Fatigue driving has become one of traffic hazard principal element, and fatigue driving detector, as the detection when fatigue driving state appears in driver and warning instrument, has started comparatively to be widely used.Fatigue-driving detection technology is the core technology of fatigue driving detector.At present, fatigue-driving detection technology mainly comprises based on physiology signal (comprising brain electricity, electrocardio, skin potential etc.) detection, Vehicular status signal (speed, acceleration, side displacement etc.) detects, driver's operation behavior (direction, throttle and brake etc. control situation) detects and driver's facial image detection (close one's eyes, blink, yawn, head is dynamic).Wherein, eye activity feature detection has the advantage that accuracy is good, reliability is high and untouchable, is the preferred option that fatigue driving detects.And the quick position of eyes is basic conditions of eye activity feature detection.Particularly in fatigue driving detector product, because car speed is high, require that the response speed detecting, send warning is also corresponding higher, therefore, under the finite data process resources supplIes of fatigue driving detector, how improving the location efficiency of eyes, accelerate localization process speed, is one of key problem in technology of fatigue-driving detection technology.
Patent CN104123549A discloses a kind of research localization method for fatigue driving Real-Time Monitoring, the method is based on YCbCr color space complexion model, and according to adjacent frame difference head range of movement, thus reduce Face detection calculated amount, and then carry out human eye area identification.The method that this patent proposes has larger limitation: on the one hand, complexion model is only suitable for daytime, because the aberration caused in night infrared light filling situation can make complexion model complete failure; On the other hand, in camera shooting driver picture, not only drive head and there is motion conditions, in vehicle operation, there is motion conditions in object and other passenger of Che Nei (particularly arranging passenger afterwards) outside window, therefore there is by facing frame difference head range of movement the limitation that multiple situation causes metrical error.
Patent CN104091147 discloses a kind of infrared eyes location and research state recognition methods.The method is in the imaging of 850nm infrared light supply, and use Adaboost algorithm training based on the eyes cascade sort detecting device of Haar feature, for detecting the eye image containing eyebrow, finally utilize the Eye states recognition merged based on HG-LBP model, Classification and Identification goes out human eye area.Method described in this patent is mainly used in solving how accurately detect the position of different head posture state human face eye areas and the problem of state, but because its identification range is based on whole image-region, lack the location of human face region to limit identification range of search, and it is comparatively complicated based on the identifying processing method of Haar feature, data processing amount is large, therefore there is the problem not high to eyes location efficiency.
Patent CN103279752A discloses a kind of eye locating method based on improving Adaboost method and Face geometric eigenvector.The method is in the training of traditional sorter and detection method basis, human eye is searched by face-human eye secondary, and the candidate's human eye treating examination is screened by some geometric error modeling features (such as the horizontal range of eye profile difference in size, eye profile and face perpendicular bisector, the horizontal line angle etc. of candidate's eyes), realize face eye areas location.The method needs to depend on more eye textural characteristics, and there is multiple candidate's human eye and have examination suggestion, and its disposal route is still comparatively complicated, and data processing amount is large, therefore still not high to the efficiency of eyes location.
The above-mentioned face eye areas localization method of the prior art enumerated, main is all on the basis of being located the human face region in image, utilization is judged the image texture characteristic that face eye in image is main and is identified, realize the location to face eye areas, to ensure the accuracy of locating.But, because the textural characteristics of face eye is comparatively tiny and complicated, and its textural characteristics easily changes along with the change of face facial expression, therefore utilize the textural characteristics of face eye to realize face eye areas location, imaging device is not only needed to have higher image quality, and identifying processing equipment also needs to carry out a large amount of data processings to determine these eye textural characteristics to the pixel data in image, its processing procedure is comparatively complicated, data processing amount is also larger, so again under finite data process resources supplIes, not high to eyes location efficiency, the slower problem of locating speed is difficult to avoid.And if by reduce in fixation and recognition process the eye textural characteristics quantity of institute's foundation improve recognition efficiency and recognition speed, can cause occurring eye recognition mistake because lines texture complicated in the image of driver area is more again, cause the situation that face eye areas positioning error is larger.
Summary of the invention
For above shortcomings in prior art, the object of the present invention is to provide a kind of face eye areas localization method detected for fatigue driving, the method gets rid of unnecessary detection factors in the process to face eye areas localization process, adopt default facial match template to realize detection and positioning simultaneously, the efficiency to face eye areas localization process can be improved, obtain the face eye areas positioning result in video image more quickly, and ensure to possess higher Position location accuracy simultaneously.
For achieving the above object, the technological means that the present invention adopts is:
For the face eye areas localization method that fatigue driving detects, by the facial match template preset in computer equipment, the video image frame by frame got computer equipment carries out the localization process of face eye areas; Described facial match template is Rect_T (X t, Y t, W t, H t), X t, Y trepresent pixel abscissa value and the pixel ordinate value of template upper left position when facial matching template is positioned respectively, W t, H trepresent pixel wide value and the pixels tall value of facial matching template initial setting respectively; And in facial match template preset be equipped with 9 characteristic areas, be respectively left eyebrow characteristic area, right eyebrow characteristic area, left eye characteristic area, right eye characteristic area, bridge of the nose characteristic area, left face characteristic area, characteristic area, left nostril, characteristic area, right nostril and right face characteristic area; Wherein:
Left eyebrow characteristic area is Rect_A (Δ X a, Δ Y a, W a, H a), Δ X a, Δ Y arepresent that the upper left position of left eyebrow characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W a, H arepresent pixel wide value and the pixels tall value of left eyebrow characteristic area initial setting respectively;
Right eyebrow characteristic area is Rect_B (Δ X b, Δ Y b, W b, H b), Δ X b, Δ Y brepresent that the upper left position of right eyebrow characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W b, H brepresent pixel wide value and the pixels tall value of right eyebrow characteristic area initial setting respectively;
Left eye characteristic area is Rect_C (Δ X c, Δ Y c, W c, H c), Δ X c, Δ Y crepresent that the upper left position of left eye characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W c, H crepresent pixel wide value and the pixels tall value of left eye characteristic area initial setting respectively;
Right eye characteristic area is Rect_D (Δ X d, Δ Y d, W d, H d), Δ X d, Δ Y drepresent that the upper left position of right eye characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W d, H drepresent pixel wide value and the pixels tall value of right eye characteristic area initial setting respectively;
Bridge of the nose characteristic area is Rect_E (Δ X e, Δ Y e, W e, H e), Δ X e, Δ Y erepresent that the upper left position of bridge of the nose characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W e, H erepresent pixel wide value and the pixels tall value of bridge of the nose characteristic area initial setting respectively;
Left face characteristic area is Rect_F (Δ X f, Δ Y f, W f, H f), Δ X f, Δ Y frepresent that the upper left position of left face characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W f, H frepresent pixel wide value and the pixels tall value of left face characteristic area initial setting respectively;
Characteristic area, left nostril is Rect_G (Δ X g, Δ Y g, W g, H g), Δ X g, Δ Y grepresent that the upper left position of characteristic area, left nostril in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W g, H grepresent pixel wide value and the pixels tall value of characteristic area, left nostril initial setting respectively;
Characteristic area, right nostril is Rect_H (Δ X h, Δ Y h, W h, H h), Δ X h, Δ Y hrepresent that the upper left position of characteristic area, right nostril in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W h, H hrepresent pixel wide value and the pixels tall value of characteristic area, right nostril initial setting respectively;
Right face characteristic area is Rect_I (Δ X i, Δ Y i, W i, H i), Δ X i, Δ Y irepresent that the upper left position of right face characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W i, H irepresent pixel wide value and the pixels tall value of right face characteristic area initial setting respectively;
The method comprises the steps:
1) frame video image is read;
2) judge this last frame video image whether successful match obtain the positioning result of face eye areas; If not, continue to perform step 3; If so, then redirect performs step 6;
3) adopt cascade classifier to carry out Face datection to current frame video image, judge facial image region whether detected in current frame video image; If so, then the pixel abscissa value X of facial image region upper left position that arrives of buffer memory cascade detection of classifier face, pixel ordinate value Y faceand the pixel wide value W in facial image region facewith pixels tall value H face, and continue to perform step 4; Otherwise redirect performs step 11;
4) the pixel wide value W in the facial image region detected in current frame video image according to cascade classifier facewith pixels tall value H face, to the width of facial matching template and each characteristic area thereof with highly carry out proportional zoom, the facial match template after bi-directional scaling is Rect_T (X t, Y t, α *w t, β *h t), thus determine that facial match template is relative to the width scaling α in facial image region in current frame image and height scaling β, and in addition buffer memory; Wherein, α=W face/ W t, β=H face/ H t;
5) according to the pixel abscissa value X of the facial image region upper left position of buffer memory face, pixel ordinate value Y faceand the pixel wide value W in facial image region facewith pixels tall value H face, determine the sensing range Rect_Search (X, Y, W, H) current frame video image being carried out to eye areas localization process:
Rect_Search(X,Y,W,H)=Rect(X Face, Y Face, W Face, H Face)；
Wherein, X, Y represent pixel abscissa value and the pixel ordinate value of sensing range upper left position in current frame video image respectively, and W, H represent pixel wide value and the pixels tall value of sensing range in current frame video image respectively; Then step 7 is performed;
6) the pixel abscissa value X of the facial image region upper left position of buffer memory is utilized face, pixel ordinate value Y faceand the optimum matching side-play amount P of this last frame video image pre(Δ X pre, Δ Y pre), determine the sensing range Rect_Search (X, Y, W, H) current frame video image being carried out to eye areas localization process:
Rect_Search(X,Y,W,H)
=Rect(X Face+ΔX pre-α *W T*γ,Y Face+ΔY pre-β *H T*γ,W T+2*α *W T*γ,H T+2*β *H T*γ)；
Wherein, X, Y represent pixel abscissa value and the pixel ordinate value of sensing range upper left position in current frame video image respectively, and W, H represent pixel wide value and the pixels tall value of sensing range in current frame video image respectively; γ is presetting neighborhood indictor, and 0< γ <1; Then step 7 is performed;
7) in the sensing range Rect_Search (X, Y, W, H) of current frame video image, with presetting detection step-length, the facial match template Rect_T (X after bi-directional scaling is adopted t, Y t, α *w t, β *h t) travel through whole sensing range, and according to the characteristic area after each bi-directional scaling in facial match template, calculate the gray scale values of facial match template each characteristic area that each position is corresponding in the sensing range of current frame video image respectively; Wherein:
The gray scale values of left eyebrow characteristic area gray_leve(range_eyebrow, A) represents the left eyebrow characteristic area Rect_A (α after calculating bi-directional scaling *Δ X a, β *Δ Y a, α *w a, β *h a) middle ratio pixel shared by of gray-scale value within presetting supercilium signature grey scale scope range_eyebrow;
The gray scale values of right eyebrow characteristic area gray_leve(range _eyebrow, B) represent the right eyebrow characteristic area Rect_B (α after calculating bi-directional scaling *Δ X b, β *Δ Y b, α *w b, β *h b) middle ratio pixel shared by of gray-scale value within presetting supercilium signature grey scale scope range_eyebrow;
The gray scale values of left eye characteristic area gray_leve(range_eye, C) represents the left eye characteristic area Rect_C (α after calculating bi-directional scaling *Δ X c, β *Δ Y c, α *w c, β *h c) middle ratio pixel shared by of gray-scale value within presetting eye feature tonal range range_eye;
The gray scale values of right eye characteristic area gray_leve(range_eye, D) represents the right eye characteristic area Rect_D (α after calculating bi-directional scaling *Δ X d, β *Δ Y d, α *w d, β *h d) middle ratio pixel shared by of gray-scale value within presetting eye feature tonal range range_eye;
The gray scale values of bridge of the nose characteristic area gray_leve(range_nosebridge, E) represents the bridge of the nose characteristic area Rect_E (α after calculating bi-directional scaling *Δ X e, β *Δ Y e, α *w e, β *h e) middle ratio pixel shared by of gray-scale value within presetting nose bridge signature grey scale scope range_nosebridge;
The gray scale values of left face characteristic area gray_leve(range_face, F) represents the left face characteristic area Rect_F (α after calculating bi-directional scaling *Δ X f, β *Δ Y f, α *w f, β *h f) middle ratio pixel shared by of gray-scale value within presetting face feature tonal range range_face;
The gray scale values of characteristic area, left nostril gray_leve(range_nostril, G) represents characteristic area, the left nostril Rect_G (α after calculating bi-directional scaling *Δ X g, β *Δ Y g, α *w g, β *h g) middle ratio pixel shared by of gray-scale value within presetting nostril portion signature grey scale scope range_nostril;
The gray scale values of characteristic area, right nostril gray_leve(range_nostril, H) represents characteristic area, the right nostril Rect_H (α after calculating bi-directional scaling *Δ X h, β *Δ Y h, α *w h, β *h h) middle ratio pixel shared by of gray-scale value within presetting nostril portion signature grey scale scope range_nostril;
The gray scale values of right face characteristic area gray_leve(range_face, I) represents the right face characteristic area Rect_I (α after calculating bi-directional scaling *Δ X i, β *Δ Y i, α *w i, β *h i) middle ratio pixel shared by of gray-scale value within presetting face feature tonal range range_face;
Wherein, α, β represent width scaling and the height scaling of buffer memory respectively;
8) for the gray scale values of facial match template each characteristic area that each position is corresponding in the sensing range of current frame video image, if the gray scale values that there is any one characteristic area in facial match template is less than presetting gray scales thresholding gray_leve th, then judge that facial match template is in this location matches failure; If the gray scale values of each characteristic area is all more than or equal to presetting gray scales thresholding in facial match template gray_leve th, then judge that facial match template is in this location matches success, and calculate the matching value ε of facial match template corresponding to relevant position:
ε=[ gray_leve(range_eyebrow,A)*λ eyebrow + gray_leve(range _eyebrow,B)*λ eyebrow + gray_leve(range_eye,C)*λ eye + gray_leve(range_eye,D)*λ eyebrow + gray_leve(range_nosebridge,E)*λ nosebridge + gray_leve(range_face,F)*λ face + gray_leve(range_nostril,G)*λ nostril + gray_leve(range_nostril,H)*λ nostri + gray_leve(range_face,I)*λ face ]；
Obtain facial match template each matching value that the match is successful corresponding to position in the sensing range of current frame video image thus; Wherein, λ eyebrow , λ eye , λ nosebridge , λ nostril , λ face represent presetting supercilium coupling weighting coefficient, eye coupling weighting coefficient, nose bridge coupling weighting coefficient, nostril portion coupling weighting coefficient and face's coupling weighting coefficient respectively;
9) add up facial match template each matching value that the match is successful corresponding to position in the sensing range of current frame video image, judge maximum matching value ε wherein maxwhether be greater than presetting coupling threshold value ε th; If so, then by this maximum matching value ε maxthe template upper left position of corresponding facial match template matches successful location is relative to the pixel coordinate side-play amount P of sensing range upper left position cur(Δ X cur, Δ Y cur) as the optimum matching side-play amount in addition buffer memory of current frame video image, and continue to perform step 10; Otherwise judge that it fails to match to current frame video image detection, redirect performs step 11;
10) according to the width scaling α of buffer memory and the optimum matching side-play amount P of height scaling β and current frame video image cur(Δ X cur, Δ Y cur), face left eye region Rect_LE (X in current frame image is determined in location lE, Y lE, W lE, H lE) and face right eye region Rect_RE (X rE, Y rE, W rE, H rE), and exported as the positioning result of face eye areas in current frame image, then perform step 11;
Wherein, X lE, Y lErepresent the pixel abscissa value of locating the face left eye region upper left position determined and pixel ordinate value respectively, W lE, H lErepresent the pixel wide value of locating the face left eye region determined and pixels tall value respectively; X rE, Y rErepresent the pixel abscissa value of locating the face right eye region upper left position determined and pixel ordinate value respectively, W rE, H rErepresent the pixel wide value of locating the face right eye region determined and pixels tall value respectively, and have:
X LE= X+ΔX cur+α *ΔX C，Y LE= Y+ΔY cur+β *ΔY C；
W LE=α *W C，H LE=β *H C；
X RE= X+ΔX cur+α *ΔX D，Y RE= Y+ΔY cur+β *ΔY D；
W RE=α *W D，H RE=β *H D；
11) read next frame video image, return and perform step 2.
In the above-mentioned face eye areas localization method for fatigue driving detection, as a kind of preferred version, in described step 4, the value of neighborhood indictor γ is 0.1.
In the above-mentioned face eye areas localization method for fatigue driving detection, as a kind of preferred version, in described step 5, the value of supercilium signature grey scale scope range_eyebrow is 0 ~ 60; The value of eye feature tonal range range_eye is 0 ~ 50; The value of nose bridge signature grey scale scope range_nosebridge is 150 ~ 255; The value of face feature tonal range range_face is 0 ~ 40; The value of nostril portion signature grey scale scope range_nostril is 150 ~ 255.
In the above-mentioned face eye areas localization method for fatigue driving detection, as a kind of preferred version, in described step 6, gray scales thresholding gray_leve thvalue be 80%.
In the above-mentioned face eye areas localization method for fatigue driving detection, as a kind of preferred version, in described step 6, supercilium coupling weighting coefficient λ eyebrow value be 0.1; Eye coupling weighting coefficient λ eye value be 0.15; Nose bridge coupling weighting coefficient λ nosebridge value be 0.1; Nostril portion coupling weighting coefficient λ nostril value be 0.1; Face coupling weighting coefficient λ face value be 0.1.
In the above-mentioned face eye areas localization method for fatigue driving detection, as a kind of preferred version, in described step 7, coupling threshold value ε thvalue be 0.85.
Compared to prior art, the present invention has following beneficial effect:
1, when the motion of driver head's geo-stationary is slower, facial image region in video image also has comparatively fixing, be used in the face eye areas localization method of fatigue driving detection in the present invention, mechanism is used for reference owing to have employed consecutive frame images match parameter, therefore, it is possible to skip the cascade classifier Face datection process needing a large amount of consumption calculations resource, directly carry out the matching treatment of facial matching template and the localization process to face eye areas according to the matching parameter of consecutive frame video image, and in the matching treatment of facial match template, adopt the matching way based on pixel grey scale grade, data processing amount is very little, execution efficiency is higher, ensure that the matching treatment to facial matching template and the localization process to face eye areas can be carried out fast.
2, under driver head's movement velocity faster situation, larger change will be there is in the facial image regional location in video image, the present invention is used in the face eye areas localization method of fatigue driving detection, under the condition of the value of neighborhood indictor γ less (0< γ <1), facial match template will easily in sensing range, it fails to match, thus need again to adopt cascade classifier to detect facial image region, carry out face eye areas localization process again, to a certain degree can affect location efficiency, but driver head's movement velocity comparatively fast also show driver there is not fatigue driving state, therefore at this moment face eye areas location is slightly slow also can not produce materially affect to fatigue driving warning function.
3, the face eye areas localization method that the present invention detects for fatigue driving mates the relative position relation determining each facial characteristics region in facial image region in video image respectively by a characteristic area, 9 in facial match template, each characteristic area is utilized mutually to verify coupling accuracy, and then realize the location to face eye areas by this relative position relation, ensure that positioning result possesses higher accuracy.
Accompanying drawing explanation
Fig. 1 is the schematic diagram of the face eye areas localization method septum reset matching template that the present invention detects for fatigue driving.
Fig. 2 is the FB(flow block) of the face eye areas localization method that the present invention detects for fatigue driving.
Embodiment
The invention provides a kind of face eye areas localization method detected for fatigue driving, the method can be applied in and perform in the computer equipment of fatigue driving detection the quick position realized face ocular after carrying out video capture to pilothouse, and ensure to possess higher Position location accuracy simultaneously, detected as to human eye active characteristics the basic condition identifying fatigue state.
By being found concrete condition analyze of fatigue detecting, in normal driving process, driver head frequently rotates, represent that this driver is in observation road conditions and vehicle condition, and there will be when driver is in fatigue driving state dull, be i.e. the very little situation of head movement amplitude.Therefore, the situation that head movement amplitude is excessive, does not belong to the situation needing to monitor fatigue state.Again according to cab environment and imaging device installation site, under the condition that driver head's motion amplitude is very little, be arranged on imaging device on control instruments platform and the characteristic area such as eyebrow, eyes, the bridge of the nose, nostril, face of the clear face to driver and face can carry out blur-free imaging, thus in the video image that can photograph at imaging device, obtain the facial characteristics area image such as driver's face mask and eyebrow, eyes, the bridge of the nose, nostril, face comparatively clearly.Due to compared with the detail textures of face eye, scope and the area in these facial characteristics regions are larger, also can be identified preferably under image quality and data processing complex degree require lower condition, if considered based on the relative position relation between the zoness of different such as eyebrow, eyes, the bridge of the nose, nostril, face, realize the location to face eye areas, so just can avoid the problem that treatment scheme is complicated, data processing amount is large brought to carry out eye recognition according to comparatively tiny, complicated eye textural characteristics.Based on this analytical mathematics, in face eye areas localization method of the present invention, by the facial match template preset in computer equipment, and preset in facial match template and be equipped with left eyebrow characteristic area, right eyebrow characteristic area, left eye characteristic area, right eye characteristic area, bridge of the nose characteristic area, left face characteristic area, characteristic area, left nostril, characteristic area, right nostril and these 9 characteristic areas, right face characteristic area, 9 characteristic areas by this facial match template mate the relative position relation determining each facial characteristics region in facial image region in video image respectively, each characteristic area is utilized mutually to verify coupling accuracy, and then realize the location to face eye areas by this relative position relation, reach raising location efficiency, accelerate the object of locating speed.
In face eye areas localization method of the present invention, as shown in Figure 1, used facial match template is Rect_T (X t, Y t, W t, H t), X t, Y trepresent pixel abscissa value and the pixel ordinate value of template upper left position when facial matching template is positioned respectively, W t, H trepresent pixel wide value and the pixels tall value of facial matching template initial setting respectively; Can see thus, facial match template is Rect_T (X t, Y t, W t, H t) be actually one with pixel coordinate (X t, Y t) be respectively W for the upper left corner, wide and height t, H tthe rectangular area of individual pixel.Meanwhile, 9 characteristic areas in facial match template are as follows respectively:
Left eyebrow characteristic area is Rect_A (Δ X a, Δ Y a, W a, H a), Δ X a, Δ Y arepresent that the upper left position of left eyebrow characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W a, H arepresent pixel wide value and the pixels tall value of left eyebrow characteristic area initial setting respectively;
Right eyebrow characteristic area is Rect_B (Δ X b, Δ Y b, W b, H b), Δ X b, Δ Y brepresent that the upper left position of right eyebrow characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W b, H brepresent pixel wide value and the pixels tall value of right eyebrow characteristic area initial setting respectively;
Left eye characteristic area is Rect_C (Δ X c, Δ Y c, W c, H c), Δ X c, Δ Y crepresent that the upper left position of left eye characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W c, H crepresent pixel wide value and the pixels tall value of left eye characteristic area initial setting respectively;
Right eye characteristic area is Rect_D (Δ X d, Δ Y d, W d, H d), Δ X d, Δ Y drepresent that the upper left position of right eye characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W d, H drepresent pixel wide value and the pixels tall value of right eye characteristic area initial setting respectively;
Bridge of the nose characteristic area is Rect_E (Δ X e, Δ Y e, W e, H e), Δ X e, Δ Y erepresent that the upper left position of bridge of the nose characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W e, H erepresent pixel wide value and the pixels tall value of bridge of the nose characteristic area initial setting respectively;
Left face characteristic area is Rect_F (Δ X f, Δ Y f, W f, H f), Δ X f, Δ Y frepresent that the upper left position of left face characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W f, H frepresent pixel wide value and the pixels tall value of left face characteristic area initial setting respectively;
Characteristic area, left nostril is Rect_G (Δ X g, Δ Y g, W g, H g), Δ X g, Δ Y grepresent that the upper left position of characteristic area, left nostril in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W g, H grepresent pixel wide value and the pixels tall value of characteristic area, left nostril initial setting respectively;
Characteristic area, right nostril is Rect_H (Δ X h, Δ Y h, W h, H h), Δ X h, Δ Y hrepresent that the upper left position of characteristic area, right nostril in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W h, H hrepresent pixel wide value and the pixels tall value of characteristic area, right nostril initial setting respectively;
Right face characteristic area is Rect_I (Δ X i, Δ Y i, W i, H i), Δ X i, Δ Y irepresent that the upper left position of right face characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W i, H irepresent pixel wide value and the pixels tall value of right face characteristic area initial setting respectively.
Certainly, if having needs in a particular application, other characteristic area can also be set in facial match template, such as left/right ear characteristic area, mouth feature district, chin characteristic area etc.
The present invention is used for the idiographic flow of the face eye areas localization method that fatigue driving detects as shown in Figure 2, comprises the steps:
1) frame video image is read.
2) judge this last frame video image whether successful match obtain the positioning result of face eye areas; If not, continue to perform step 3; If so, then redirect performs step 6;
In face eye areas localization method of the present invention, have employed consecutive frame images match parameter and use for reference mechanism; If this last frame video image fails, coupling obtains the positioning result of face eye areas, then continue to perform the facial image region that step 3,4,5 detects current frame video image, and then determines the surveyed area of current frame video image; If this last frame video image successful match obtains the positioning result of face eye areas, so the pixel abscissa value X of the facial image region upper left position that cascade classifier before this detects will be cached with in computer equipment face, pixel ordinate value Y face, pixel wide value W facewith pixels tall value H faceand the optimum matching side-play amount P of this last frame video image pre(Δ X pre, Δ Y pre), then directly jump to step 6 and utilize the data cached surveyed area determining current frame video image, thus reduce unnecessary Face datection link.
3) adopt cascade classifier to carry out Face datection to current frame video image, judge facial image region whether detected in current frame video image; If so, then the pixel abscissa value X of facial image region upper left position that arrives of buffer memory cascade detection of classifier face, pixel ordinate value Y faceand the pixel wide value W in facial image region facewith pixels tall value H face, and continue to perform step 4; Otherwise redirect performs step 11.
Face eye areas localization method of the present invention, also be implement on the basis of locating based on human face region, in video image analysis, adopt cascade classifier to detect facial image region has been the prior art of comparative maturity, all talentedly in several sections of technical literatures mentioned in the introduction use this technology, no longer add to repeat at this.
4) the pixel wide value W in the facial image region detected in current frame video image according to cascade classifier facewith pixels tall value H face, to the width of facial matching template and each characteristic area thereof with highly carry out proportional zoom, the facial match template after bi-directional scaling is Rect_T (X t, Y t, α *w t, β *h t), thus determine that facial match template is relative to the width scaling α in facial image region in current frame image and height scaling β, and in addition buffer memory; Wherein, α=W face/ W t, β=H face/ H t.
The width of facial image opposite zone portion's matching template that this step detects according to cascade classifier and each characteristic area thereof and highly carry out proportional zoom, thus determine that facial match template is relative to the width scaling α in facial image region in current frame image and height scaling β, and to this width scaling α and height scaling β in addition buffer memory, in the follow-up localization process to current frame video image and the localization process to subsequent video images frame, can by the proportionate relationship of facial image region facial characteristics in each characteristic area in the ratio data determination facial match template of this buffer memory and video image.
5) according to the pixel abscissa value X of the facial image region upper left position of buffer memory face, pixel ordinate value Y faceand the pixel wide value W in facial image region facewith pixels tall value H face, determine the sensing range Rect_Search (X, Y, W, H) current frame video image being carried out to eye areas localization process:
Rect_Search(X,Y,W,H)=Rect(X Face, Y Face, W Face, H Face)；
Wherein, X, Y represent pixel abscissa value and the pixel ordinate value of sensing range upper left position in current frame video image respectively, and W, H represent pixel wide value and the pixels tall value of sensing range in current frame video image respectively; Then step 7 is performed.
6) the pixel abscissa value X of the facial image region upper left position of buffer memory is utilized face, pixel ordinate value Y faceand the optimum matching side-play amount P of this last frame video image pre(Δ X pre, Δ Y pre), determine the sensing range Rect_Search (X, Y, W, H) current frame video image being carried out to eye areas localization process:
Rect_Search(X,Y,W,H)
=Rect(X Face+ΔX pre-α *W T*γ,Y Face+ΔY pre-β *H T*γ,W T+2*α *W T*γ,H T+2*β *H T*γ)；
Wherein, X, Y represent pixel abscissa value and the pixel ordinate value of sensing range upper left position in current frame video image respectively, and W, H represent pixel wide value and the pixels tall value of sensing range in current frame video image respectively; γ is presetting neighborhood indictor, and 0< γ <1; Then step 7 is performed.
Step 5 and step 6 are being determined to carry out in the process of the sensing range of eye areas localization process to current frame video image, main using the position, facial image region that cascade classifier detects as the position reference of sensing range, take into account the continuity considered in video image between consecutive frame simultaneously.If this last frame video image fails, coupling obtains the positioning result of face eye areas, then detected the facial image region of current frame video image by step 3,4,5, and then determine the surveyed area of current frame video image; If this last frame video image successful match obtains the positioning result of face eye areas, so then directly utilize the data cached surveyed area determining current frame video image by step 6.
In addition, in step 6, due in the continuous sex situation considering consecutive frame image, sensing range is subject to the size of this last frame video image optimum matching side-play amount adjustment, is the size of getting according to neighborhood indictor γ and fixed; According to the difference of practical situations, also may there is difference in the size of getting of neighborhood indictor γ, needs according to actual conditions and determine.
7) in the sensing range Rect_Search (X, Y, W, H) of current frame video image, with presetting detection step-length, the facial match template Rect_T (X after bi-directional scaling is adopted t, Y t, α *w t, β *h t) travel through whole sensing range, and according to the characteristic area after each bi-directional scaling in facial match template, calculate the gray scale values of facial match template each characteristic area that each position is corresponding in the sensing range of current frame video image respectively; Wherein:
The gray scale values of left eyebrow characteristic area gray_leve(range_eyebrow, A) represents the left eyebrow characteristic area Rect_A (α after calculating bi-directional scaling *Δ X a, β *Δ Y a, α *w a, β *h a) middle ratio pixel shared by of gray-scale value within presetting supercilium signature grey scale scope range_eyebrow;
The gray scale values of right eyebrow characteristic area gray_leve(range _eyebrow, B) represent the right eyebrow characteristic area Rect_B (α after calculating bi-directional scaling *Δ X b, β *Δ Y b, α *w b, β *h b) middle ratio pixel shared by of gray-scale value within presetting supercilium signature grey scale scope range_eyebrow;
The gray scale values of left eye characteristic area gray_leve(range_eye, C) represents the left eye characteristic area Rect_C (α after calculating bi-directional scaling *Δ X c, β *Δ Y c, α *w c, β *h c) middle ratio pixel shared by of gray-scale value within presetting eye feature tonal range range_eye;
The gray scale values of right eye characteristic area gray_leve(range_eye, D) represents the right eye characteristic area Rect_D (α after calculating bi-directional scaling *Δ X d, β *Δ Y d, α *w d, β *h d) middle ratio pixel shared by of gray-scale value within presetting eye feature tonal range range_eye;
The gray scale values of bridge of the nose characteristic area gray_leve(range_nosebridge, E) represents the bridge of the nose characteristic area Rect_E (α after calculating bi-directional scaling *Δ X e, β *Δ Y e, α *w e, β *h e) middle ratio pixel shared by of gray-scale value within presetting nose bridge signature grey scale scope range_nosebridge;
The gray scale values of left face characteristic area gray_leve(range_face, F) represents the left face characteristic area Rect_F (α after calculating bi-directional scaling *Δ X f, β *Δ Y f, α *w f, β *h f) middle ratio pixel shared by of gray-scale value within presetting face feature tonal range range_face;
The gray scale values of characteristic area, left nostril gray_leve(range_nostril, G) represents characteristic area, the left nostril Rect_G (α after calculating bi-directional scaling *Δ X g, β *Δ Y g, α *w g, β *h g) middle ratio pixel shared by of gray-scale value within presetting nostril portion signature grey scale scope range_nostril;
The gray scale values of characteristic area, right nostril gray_leve(range_nostril, H) represents characteristic area, the right nostril Rect_H (α after calculating bi-directional scaling *Δ X h, β *Δ Y h, α *w h, β *h h) middle ratio pixel shared by of gray-scale value within presetting nostril portion signature grey scale scope range_nostril;
The gray scale values of right face characteristic area gray_leve(range_face, I) represents the right face characteristic area Rect_I (α after calculating bi-directional scaling *Δ X i, β *Δ Y i, α *w i, β *h i) middle ratio pixel shared by of gray-scale value within presetting face feature tonal range range_face;
Wherein, α, β represent width scaling and the height scaling of buffer memory respectively.
The main thought of face eye areas localization method of the present invention, mate by each characteristic area in facial match template the relative position relation determining each facial characteristics region in facial image region in video image respectively, each characteristic area is utilized mutually to verify to guarantee to mate accuracy, and the location realized face eye areas, reach the object improving location efficiency, accelerate locating speed.Coupling determines that the processing mode in each facial characteristics region in facial image region in video image has a lot, such as by being trained the mode of coupling to be determined with the image in each facial characteristics region gathered, or respectively by each facial characteristics region analysis of texture and determine.But how effectively can either realize the match cognization to each facial characteristics region, can reduce again match complexity better, reduce data processing amount, be the problem that face eye areas localization method of the present invention is further considered simultaneously.Therefore, present invention employs the mode of the gray scale values based on each characteristic area, realize determining the coupling in each facial characteristics region in facial image region, because the gradation of image distribution situation in each facial characteristics region has and comparatively significantly distinguishes in facial image region, the computing simultaneously adding up grey value profile ratio is very simple, and processing speed is faster.Based on such consideration, with presetting detection step-length in this step, the facial match template after bi-directional scaling is adopted to travel through whole sensing range, and root calculates the gray scale values of facial match template each characteristic area that each position is corresponding in the sensing range of current frame video image respectively, determine the data processing basis in each facial characteristics region as subsequent match.
In this step, the concrete value of supercilium signature grey scale scope range_eyebrow, eye feature tonal range range_eye, nose bridge signature grey scale scope range_nosebridge, face feature tonal range range_face and nostril portion signature grey scale scope range_nostril, needs to determine according to practical situations; Because, fatigue driving detecting system adopts different imaging devices, in its screen image, the gray-scale value situation in each facial characteristics region of facial image may there are differences, and therefore regional gray feature scope also needs according to actual conditions, is determined by data statistics and experiment experience.
8) for the gray scale values of facial match template each characteristic area that each position is corresponding in the sensing range of current frame video image, if the gray scale values that there is any one characteristic area in facial match template is less than presetting gray scales thresholding gray_leve th, then judge that facial match template is in this location matches failure; If the gray scale values of each characteristic area is all more than or equal to presetting gray scales thresholding in facial match template gray_leve th, then judge that facial match template is in this location matches success, and calculate the matching value ε of facial match template corresponding to relevant position:
ε=[ gray_leve(range_eyebrow,A)*λ eyebrow + gray_leve(range _eyebrow,B)*λ eyebrow + gray_leve(range_eye,C)*λ eye + gray_leve(range_eye,D)*λ eyebrow + gray_leve(range_nosebridge,E)*λ nosebridge + gray_leve(range_face,F)*λ face + gray_leve(range_nostril,G)*λ nostril + gray_leve(range_nostril,H)*λ nostri + gray_leve(range_face,I)*λ face ]；
Obtain facial match template each matching value that the match is successful corresponding to position in the sensing range of current frame video image thus; Wherein, λ eyebrow , λ eye , λ nosebridge , λ nostril , λ face represent presetting supercilium coupling weighting coefficient, eye coupling weighting coefficient, nose bridge coupling weighting coefficient, nostril portion coupling weighting coefficient and face's coupling weighting coefficient respectively.
In this step, whether the match is successful in each position in sensing range to judge facial match template, judges according to the situation that in facial match template, the gray scale values in existing characteristics district is less than gray scales thresholding; Because if the gray scale values that there is any one characteristic area in facial match template is too small, be exactly probably that the actual face characteristic area position corresponding to video image, this position, characteristic area is misfitted and caused, occur that this situation about misfitting is likely because the situations such as driver head's inclination, deflection cause, if the position, two characteristic areas in these situations in direct basis facial match template positions the situation probably occurring that deviation is larger to face eye areas, therefore judge that it fails to match; Only in facial match template, the gray scale values of each characteristic area is all more than or equal to presetting gray scales thresholding, namely, when each position, characteristic area matches with each actual face characteristic area position in video image respectively in facial match template, just judge that the match is successful.Thus, each characteristic area that just make use of in facial match template is verified mutually, to guarantee the accuracy of mating.
And have employed gray scale values to mate each facial characteristics regional location due to the inventive method, abandon the identification to regional texture feature, although match complexity can be effectively reduced and reduce data processing amount, but guarantee that coupling is enough accurate, higher gray scale values is just needed to judge requirement, so as the gray scales thresholding of determinating reference gray_leve thvalue need enough large.Gray scales thresholding under normal circumstances gray_leve thvalue at least should be 80%; Certainly, according to the difference of practical situations, gray scales thresholding also can adopt other value.
In addition, the matching value ε of facial match template corresponding to relevant position is mainly used in embodying the matching degree of facial match template in relevant position, and matching value ε is larger, then show that matching degree is higher, matched position is more accurate; In matching value ε calculating formula, each coupling weighting coefficient then to represent in facial match template each characteristic area for the contribution rate of matching degree, and therefore value of each coupling weighting coefficient is also determined the size of matching degree contribution rate according to each characteristic area.Usually each coupling weighting coefficient can be determined by following value, i.e. supercilium coupling weighting coefficient λ eyebrow value be 0.1, eye coupling weighting coefficient λ eye value be 0.15, nose bridge coupling weighting coefficient λ nosebridge value be 0.1, nostril portion coupling weighting coefficient λ nostril value be 0.1, face coupling weighting coefficient λ face value be 0.1; Value thus, in matching value ε calculating formula, namely the summation of the coupling weighting coefficient that 9 characteristic areas are corresponding equals 1.Certainly, when embody rule, the value size of each coupling weighting coefficient also can be determined according to practical situations.
9) add up facial match template each matching value that the match is successful corresponding to position in the sensing range of current frame video image, judge maximum matching value ε wherein maxwhether be greater than presetting coupling threshold value ε th; If so, then by this maximum matching value ε maxthe template upper left position of corresponding facial match template matches successful location is relative to the pixel coordinate side-play amount P of sensing range upper left position cur(Δ X cur, Δ Y cur) as the optimum matching side-play amount in addition buffer memory of current frame video image, and continue to perform step 10; Otherwise judge that it fails to match to current frame video image detection, redirect performs step 11.
This step is totally passed judgment on to the degree of agreement of each corresponding actual face characteristic area position in video image each position, characteristic area in facial matching template.If the maximum matching value ε of facial match template each the match is successful position in the sensing range of current frame video image maxall can not be greater than coupling threshold value ε ththen show that facial match template coupling degree of agreement of all matched positions in the sensing range of current frame video image is all difficult to reach satisfied requirement, this situation is likely because current frame video image is overall comparatively fuzzy or there is gray scale values in Partial Feature district coupling and meet the demands and there is the situation of coincidence, locate for avoiding face eye areas and occur unnecessary mistake, it fails to match in the inventive method, these situations to be all judged to be current frame video image detection, got rid of.The degree size of this eliminating is according to coupling threshold value ε thvalue size and fixed.If the coupling weighting coefficient summation that 9 characteristic areas are corresponding in matching value ε calculating formula equals 1, so under normal circumstances, coupling threshold value ε thvalue may be selected to be 0.85.Certainly, threshold value ε is mated in a particular application thcan get larger value, but unsuitable value is excessive, otherwise there will be that to be matched to power to video image too low and lose the situation that actual face eye areas position application is worth.
10) according to the width scaling α of buffer memory and the optimum matching side-play amount P of height scaling β and current frame video image cur(Δ X cur, Δ Y cur), face left eye region Rect_LE (X in current frame image is determined in location lE, Y lE, W lE, H lE) and face right eye region Rect_RE (X rE, Y rE, W rE, H rE), and exported as the positioning result of face eye areas in current frame image, then perform step 11;
Wherein, X lE, Y lErepresent the pixel abscissa value of locating the face left eye region upper left position determined and pixel ordinate value respectively, W lE, H lErepresent the pixel wide value of locating the face left eye region determined and pixels tall value respectively; X rE, Y rErepresent the pixel abscissa value of locating the face right eye region upper left position determined and pixel ordinate value respectively, W rE, H rErepresent the pixel wide value of locating the face right eye region determined and pixels tall value respectively, and have:
X LE= X+ΔX cur+α *ΔX C，Y LE= Y+ΔY cur+β *ΔY C；
W LE=α *W C，H LE=β *H C；
X RE= X+ΔX cur+α *ΔX D，Y RE= Y+ΔY cur+β *ΔY D；
W RE=α *W D，H RE=β *H D。
The positioning result that this step obtains, with face left eye region positioning result Rect_LE (X lE, Y lE, W lE, H lE) be example, X lE=X+ Δ X cur+ α *Δ X c, Y lE=Y+ Δ Y cur+ β *Δ Y c, namely show, the upper left position pixel coordinate " X of face left eye region positioning result lE, Y lE", be on the basis of the sensing range top left co-ordinate " X, Y " of current frame video image, first offset " Δ X cur, Δ Y cur" (be equivalent in the matching process, according to the optimum matching side-play amount P of current frame video image cur(Δ X cur, Δ Y cur), the upper left corner of facial match template first from " X, Y " offset orientation to " X+ Δ X cur, Y+ Δ Y cur"), and then offset " α *Δ X c, β *Δ Y c" (be equivalent in the matching process, according to left eye characteristic area in the facial match template after bi-directional scaling relative to the side-play amount " α of template upper left position *Δ X c, β *Δ Y c", from face template upper left position " the X+ Δ X of optimum matching cur, Y+ Δ Y cur" be displaced to the left eye region upper left corner " the X+ Δ X of optimum matching cur+ α *Δ X c, Y+ Δ Y cur+ β *Δ Y c"), so just obtain the face left eye region upper left position of optimum matching; Meanwhile, according to facial match template relative to the width scaling α in facial image region in current frame image and height scaling β, the width of the face left eye region of adjustment best match position and height, even W lE=α *w c, make H lE=β *h c, just obtain face left eye region positioning result Rect_LE (X thus lE, Y lE, W lE, H lE).Right eye in like manner.
11) read next frame video image, return and perform step 2.
Jump to next frame video image by step 11 and perform fixation and recognition, just realize the localization process of video image frame by frame being carried out to face eye areas.It should be noted that, in above-mentioned steps, " the optimum matching side-play amount P of current frame video image of institute's buffer memory in a certain frame process cur(Δ X cur, Δ Y cur) ", for next frame video image, be " the optimum matching side-play amount P of this last frame video image pre(Δ X pre, Δ Y pre) ", this point should easy understand.
Can be seen by above-mentioned application flow, when the motion of driver head's geo-stationary is slower, facial image region in video image also has comparatively fixing, due to the present invention be used for fatigue driving detect face eye areas localization method in have employed consecutive frame images match parameter use for reference mechanism, therefore, it is possible to skip the cascade classifier Face datection process needing a large amount of consumption calculations resource, directly carry out the matching treatment of facial matching template and the localization process to face eye areas according to the matching parameter of consecutive frame video image, and in the matching treatment of facial match template, adopt the matching way based on pixel grey scale grade, data processing amount is very little, execution efficiency is higher, ensure that the matching treatment to facial matching template and the localization process to face eye areas can be carried out fast.And under driver head's movement velocity faster situation, larger change will be there is in the facial image regional location in video image, under the condition of the value of neighborhood indictor γ less (0< γ <1), facial match template will easily in sensing range, it fails to match, thus need again to adopt cascade classifier to detect facial image region, carry out face eye areas localization process again, to a certain degree can affect location efficiency, but driver head's movement velocity comparatively fast also show driver there is not fatigue driving state, therefore at this moment face eye areas location is slightly slow also can not produce materially affect to fatigue driving warning function.Simultaneously, the inventive method also mates the relative position relation determining each facial characteristics region in facial image region in video image respectively by a characteristic area, 9 in facial match template, each characteristic area is utilized mutually to verify coupling accuracy, and then realize the location to face eye areas by this relative position relation, ensure that positioning result possesses higher accuracy.
In general, the face eye areas localization method that the present invention is used for fatigue driving detection eliminates unnecessary detection factors, adopt default facial match template to realize detection and positioning simultaneously, the efficiency to face eye areas localization process can be improved, obtain the face eye areas positioning result in video image more rapidly, and ensure to possess higher Position location accuracy simultaneously.
In order to embody the technique effect of the face eye areas localization method that the present invention detects for fatigue driving better, below by experiment, the inventive method is further illustrated.
contrast experiment:
This contrast experiment adopts two kinds of other face eye areas localization methods and face eye areas localization method of the present invention to be contrasted.Except the inventive method, the two kinds of face eye areas localization methods participating in contrast are respectively:
Method I: infrared eyes location and research state recognition methods disclosed in patent CN104091147.
Method II: the patent CN103279752A disclosed eye locating method based on improving Adaboost method and Face geometric eigenvector.
The inventive method then performs face eye areas localization process according to aforesaid step 1 ~ 11, and in concrete processing procedure, the value of neighborhood indictor γ is 0.1; The value of supercilium signature grey scale scope range_eyebrow is 0 ~ 60, the value of eye feature tonal range range_eye is 0 ~ 50, the value of nose bridge signature grey scale scope range_nosebridge is 150 ~ 255, the value of face feature tonal range range_face is 0 ~ 40, and the value of nostril portion signature grey scale scope range_nostril is 150 ~ 255; Gray scales thresholding gray_leve thvalue be 80%; Supercilium coupling weighting coefficient λ eyebrow value be 0.1, eye coupling weighting coefficient λ eye value be 0.15, nose bridge coupling weighting coefficient λ nosebridge value be 0.1, nostril portion coupling weighting coefficient λ nostril value be 0.1, face coupling weighting coefficient λ face value be 0.1; Coupling threshold value ε thvalue be 0.85.
In this contrast experiment, transfer to computing machine after adopting camera collection facial video image, adopt method I, method II and the inventive method to carry out face eye areas localization process respectively by computing machine.The video image pixel size of camera collection is 640*480; Computer processor is Intel (R) Core (TM) i5-2520M CPU 2.5GHz, saves as 4GB RAM in process.Experimentation adopts 5 sections to detect video altogether, every section of video image is all more than 500 frames, employing method I respectively, method II and the inventive method carry out face eye areas localization process to each two field picture that 5 sections are detected video, add up three kinds of methods detect video single frames average positioning time for each section, and for the positioning result of each frame, the eye areas center of location is less than 5% of eye areas scope with the deviation detecting actual persons eye pupil hole site in video and is judged to be accurate positioning, 5% judgement location that deviation is greater than eye areas scope is inaccurate, add up three kinds of methods detect video locating accuracy for each section.Final statistics is as shown in table 1.
Table 1
Can be seen by above-mentioned contrast experiment, the present invention is used for the face eye areas localization method that fatigue driving detects, substantially suitable with method I and method II in the accuracy rate that face eye areas is located, but on single frames average positioning time, the inventive method is obviously better than two kinds of methods of prior art, there is higher face eye areas localization process efficiency, the face eye areas positioning result in video image can be obtained more fast.
What finally illustrate is, above embodiment is only in order to illustrate technical scheme of the present invention and unrestricted, although with reference to embodiment to invention has been detailed description, those of ordinary skill in the art is to be understood that, can modify to technical scheme of the present invention or equivalent replacement, and not departing from aim and the scope of technical solution of the present invention, it all should be encompassed in the middle of right of the present invention.

Claims (6)

1. for the face eye areas localization method that fatigue driving detects, it is characterized in that, by the facial match template preset in computer equipment, the video image frame by frame got computer equipment carries out the localization process of face eye areas; Described facial match template is Rect_T (X t, Y t, W t, H t), X t, Y trepresent pixel abscissa value and the pixel ordinate value of template upper left position when facial matching template is positioned respectively, W t, H trepresent pixel wide value and the pixels tall value of facial matching template initial setting respectively; And in facial match template preset be equipped with 9 characteristic areas, be respectively left eyebrow characteristic area, right eyebrow characteristic area, left eye characteristic area, right eye characteristic area, bridge of the nose characteristic area, left face characteristic area, characteristic area, left nostril, characteristic area, right nostril and right face characteristic area; Wherein:
Left eyebrow characteristic area is Rect_A (Δ X a, Δ Y a, W a, H a), Δ X a, Δ Y arepresent that the upper left position of left eyebrow characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W a, H arepresent pixel wide value and the pixels tall value of left eyebrow characteristic area initial setting respectively;
Right eyebrow characteristic area is Rect_B (Δ X b, Δ Y b, W b, H b), Δ X b, Δ Y brepresent that the upper left position of right eyebrow characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W b, H brepresent pixel wide value and the pixels tall value of right eyebrow characteristic area initial setting respectively;
Left eye characteristic area is Rect_C (Δ X c, Δ Y c, W c, H c), Δ X c, Δ Y crepresent that the upper left position of left eye characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W c, H crepresent pixel wide value and the pixels tall value of left eye characteristic area initial setting respectively;
Right eye characteristic area is Rect_D (Δ X d, Δ Y d, W d, H d), Δ X d, Δ Y drepresent that the upper left position of right eye characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W d, H drepresent pixel wide value and the pixels tall value of right eye characteristic area initial setting respectively;
Bridge of the nose characteristic area is Rect_E (Δ X e, Δ Y e, W e, H e), Δ X e, Δ Y erepresent that the upper left position of bridge of the nose characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W e, H erepresent pixel wide value and the pixels tall value of bridge of the nose characteristic area initial setting respectively;
Left face characteristic area is Rect_F (Δ X f, Δ Y f, W f, H f), Δ X f, Δ Y frepresent that the upper left position of left face characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W f, H frepresent pixel wide value and the pixels tall value of left face characteristic area initial setting respectively;
Characteristic area, left nostril is Rect_G (Δ X g, Δ Y g, W g, H g), Δ X g, Δ Y grepresent that the upper left position of characteristic area, left nostril in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W g, H grepresent pixel wide value and the pixels tall value of characteristic area, left nostril initial setting respectively;
Characteristic area, right nostril is Rect_H (Δ X h, Δ Y h, W h, H h), Δ X h, Δ Y hrepresent that the upper left position of characteristic area, right nostril in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W h, H hrepresent pixel wide value and the pixels tall value of characteristic area, right nostril initial setting respectively;
Right face characteristic area is Rect_I (Δ X i, Δ Y i, W i, H i), Δ X i, Δ Y irepresent that the upper left position of right face characteristic area in facial matching template is relative to the pixel horizontal ordinate side-play amount of template upper left position and pixel ordinate side-play amount respectively, W i, H irepresent pixel wide value and the pixels tall value of right face characteristic area initial setting respectively;
The method comprises the steps:
1) frame video image is read;
2) judge this last frame video image whether successful match obtain the positioning result of face eye areas; If not, continue to perform step 3; If so, then redirect performs step 6;
3) adopt cascade classifier to carry out Face datection to current frame video image, judge facial image region whether detected in current frame video image; If so, then the pixel abscissa value X of facial image region upper left position that arrives of buffer memory cascade detection of classifier face, pixel ordinate value Y faceand the pixel wide value W in facial image region facewith pixels tall value H face, and continue to perform step 4; Otherwise redirect performs step 11;
4) the pixel wide value W in the facial image region detected in current frame video image according to cascade classifier facewith pixels tall value H face, to the width of facial matching template and each characteristic area thereof with highly carry out proportional zoom, the facial match template after bi-directional scaling is Rect_T (X t, Y t, α *w t, β *h t), thus determine that facial match template is relative to the width scaling α in facial image region in current frame image and height scaling β, and in addition buffer memory; Wherein, α=W face/ W t, β=H face/ H t;
5) according to the pixel abscissa value X of the facial image region upper left position of buffer memory face, pixel ordinate value Y faceand the pixel wide value W in facial image region facewith pixels tall value H face, determine the sensing range Rect_Search (X, Y, W, H) current frame video image being carried out to eye areas localization process:
Rect_Search(X,Y,W,H)=Rect(X Face, Y Face, W Face, H Face)；
Wherein, X, Y represent pixel abscissa value and the pixel ordinate value of sensing range upper left position in current frame video image respectively, and W, H represent pixel wide value and the pixels tall value of sensing range in current frame video image respectively; Then step 7 is performed;
6) the pixel abscissa value X of the facial image region upper left position of buffer memory is utilized face, pixel ordinate value Y faceand the optimum matching side-play amount P of this last frame video image pre(Δ X pre, Δ Y pre), determine the sensing range Rect_Search (X, Y, W, H) current frame video image being carried out to eye areas localization process:
Rect_Search(X,Y,W,H)
=Rect(X Face+ΔX pre-α *W T*γ,Y Face+ΔY pre-β *H T*γ,W T+2*α *W T*γ,H T+2*β *H T*γ)；
Wherein, X, Y represent pixel abscissa value and the pixel ordinate value of sensing range upper left position in current frame video image respectively, and W, H represent pixel wide value and the pixels tall value of sensing range in current frame video image respectively; γ is presetting neighborhood indictor, and 0< γ <1; Then step 7 is performed;
7) in the sensing range Rect_Search (X, Y, W, H) of current frame video image, with presetting detection step-length, the facial match template Rect_T (X after bi-directional scaling is adopted t, Y t, α *w t, β *h t) travel through whole sensing range, and according to the characteristic area after each bi-directional scaling in facial match template, calculate the gray scale values of facial match template each characteristic area that each position is corresponding in the sensing range of current frame video image respectively; Wherein:
The gray scale values of left eyebrow characteristic area gray_leve(range_eyebrow, A) represents the left eyebrow characteristic area Rect_A (α after calculating bi-directional scaling *Δ X a, β *Δ Y a, α *w a, β *h a) middle ratio pixel shared by of gray-scale value within presetting supercilium signature grey scale scope range_eyebrow;
The gray scale values of right eyebrow characteristic area gray_leve(range _eyebrow, B) represent the right eyebrow characteristic area Rect_B (α after calculating bi-directional scaling *Δ X b, β *Δ Y b, α *w b, β *h b) middle ratio pixel shared by of gray-scale value within presetting supercilium signature grey scale scope range_eyebrow;
The gray scale values of left eye characteristic area gray_leve(range_eye, C) represents the left eye characteristic area Rect_C (α after calculating bi-directional scaling *Δ X c, β *Δ Y c, α *w c, β *h c) middle ratio pixel shared by of gray-scale value within presetting eye feature tonal range range_eye;
The gray scale values of right eye characteristic area gray_leve(range_eye, D) represents the right eye characteristic area Rect_D (α after calculating bi-directional scaling *Δ X d, β *Δ Y d, α *w d, β *h d) middle ratio pixel shared by of gray-scale value within presetting eye feature tonal range range_eye;
The gray scale values of bridge of the nose characteristic area gray_leve(range_nosebridge, E) represents the bridge of the nose characteristic area Rect_E (α after calculating bi-directional scaling *Δ X e, β *Δ Y e, α *w e, β *h e) middle ratio pixel shared by of gray-scale value within presetting nose bridge signature grey scale scope range_nosebridge;
The gray scale values of left face characteristic area gray_leve(range_face, F) represents the left face characteristic area Rect_F (α after calculating bi-directional scaling *Δ X f, β *Δ Y f, α *w f, β *h f) middle ratio pixel shared by of gray-scale value within presetting face feature tonal range range_face;
The gray scale values of characteristic area, left nostril gray_leve(range_nostril, G) represents characteristic area, the left nostril Rect_G (α after calculating bi-directional scaling *Δ X g, β *Δ Y g, α *w g, β *h g) middle ratio pixel shared by of gray-scale value within presetting nostril portion signature grey scale scope range_nostril;
The gray scale values of characteristic area, right nostril gray_leve(range_nostril, H) represents characteristic area, the right nostril Rect_H (α after calculating bi-directional scaling *Δ X h, β *Δ Y h, α *w h, β *h h) middle ratio pixel shared by of gray-scale value within presetting nostril portion signature grey scale scope range_nostril;
The gray scale values of right face characteristic area gray_leve(range_face, I) represents the right face characteristic area Rect_I (α after calculating bi-directional scaling *Δ X i, β *Δ Y i, α *w i, β *h i) middle ratio pixel shared by of gray-scale value within presetting face feature tonal range range_face;
Wherein, α, β represent width scaling and the height scaling of buffer memory respectively;
8) for the gray scale values of facial match template each characteristic area that each position is corresponding in the sensing range of current frame video image, if the gray scale values that there is any one characteristic area in facial match template is less than presetting gray scales thresholding gray_leve th, then judge that facial match template is in this location matches failure; If the gray scale values of each characteristic area is all more than or equal to presetting gray scales thresholding in facial match template gray_leve th, then judge that facial match template is in this location matches success, and calculate the matching value ε of facial match template corresponding to relevant position:
ε=[ gray_leve(range_eyebrow,A)*λ eyebrow + gray_leve(range _eyebrow,B)*λ eyebrow + gray_leve(range_eye,C)*λ eye + gray_leve(range_eye,D)*λ eyebrow + gray_leve(range_nosebridge,E)*λ nosebridge + gray_leve(range_face,F)*λ face + gray_leve(range_nostril,G)*λ nostril + gray_leve(range_nostril,H)*λ nostri + gray_leve(range_face,I)*λ face ]；
Obtain facial match template each matching value that the match is successful corresponding to position in the sensing range of current frame video image thus; Wherein, λ eyebrow , λ eye , λ nosebridge , λ nostril , λ face represent presetting supercilium coupling weighting coefficient, eye coupling weighting coefficient, nose bridge coupling weighting coefficient, nostril portion coupling weighting coefficient and face's coupling weighting coefficient respectively;
9) add up facial match template each matching value that the match is successful corresponding to position in the sensing range of current frame video image, judge maximum matching value ε wherein maxwhether be greater than presetting coupling threshold value ε th; If so, then by this maximum matching value ε maxthe template upper left position of corresponding facial match template matches successful location is relative to the pixel coordinate side-play amount P of sensing range upper left position cur(Δ X cur, Δ Y cur) as the optimum matching side-play amount in addition buffer memory of current frame video image, and continue to perform step 10; Otherwise judge that it fails to match to current frame video image detection, redirect performs step 11;
10) according to the width scaling α of buffer memory and the optimum matching side-play amount P of height scaling β and current frame video image cur(Δ X cur, Δ Y cur), face left eye region Rect_LE (X in current frame image is determined in location lE, Y lE, W lE, H lE) and face right eye region Rect_RE (X rE, Y rE, W rE, H rE), and exported as the positioning result of face eye areas in current frame image, then perform step 11;
Wherein, X lE, Y lErepresent the pixel abscissa value of locating the face left eye region upper left position determined and pixel ordinate value respectively, W lE, H lErepresent the pixel wide value of locating the face left eye region determined and pixels tall value respectively; X rE, Y rErepresent the pixel abscissa value of locating the face right eye region upper left position determined and pixel ordinate value respectively, W rE, H rErepresent the pixel wide value of locating the face right eye region determined and pixels tall value respectively, and have:
X LE= X+ΔX cur+α *ΔX C，Y LE= Y+ΔY cur+β *ΔY C；
W LE=α *W C，H LE=β *H C；
X RE= X+ΔX cur+α *ΔX D，Y RE= Y+ΔY cur+β *ΔY D；
W RE=α *W D，H RE=β *H D；
11) read next frame video image, return and perform step 2.
2., according to claim 1 for the face eye areas localization method that fatigue driving detects, it is characterized in that, in described step 4, the value of neighborhood indictor γ is 0.1.
3., according to claim 1 for the face eye areas localization method that fatigue driving detects, it is characterized in that, in described step 5, the value of supercilium signature grey scale scope range_eyebrow is 0 ~ 60; The value of eye feature tonal range range_eye is 0 ~ 50; The value of nose bridge signature grey scale scope range_nosebridge is 150 ~ 255; The value of face feature tonal range range_face is 0 ~ 40; The value of nostril portion signature grey scale scope range_nostril is 150 ~ 255.
4., according to claim 1 for the face eye areas localization method that fatigue driving detects, it is characterized in that, in described step 6, gray scales thresholding gray_leve thvalue be 80%.
5., according to claim 1 for the face eye areas localization method that fatigue driving detects, it is characterized in that, in described step 6, supercilium coupling weighting coefficient λ eyebrow value be 0.1; Eye coupling weighting coefficient λ eye value be 0.15; Nose bridge coupling weighting coefficient λ nosebridge value be 0.1; Nostril portion coupling weighting coefficient λ nostril value be 0.1; Face coupling weighting coefficient λ face value be 0.1.
6., according to claim 1 for the face eye areas localization method that fatigue driving detects, it is characterized in that, in described step 7, coupling threshold value ε thvalue be 0.85.
CN201410739606.7A 2014-12-08 2014-12-08 A kind of face eye areas localization method for fatigue driving detection Expired - Fee Related CN104361332B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410739606.7A CN104361332B (en) 2014-12-08 2014-12-08 A kind of face eye areas localization method for fatigue driving detection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410739606.7A CN104361332B (en) 2014-12-08 2014-12-08 A kind of face eye areas localization method for fatigue driving detection

Publications (2)

Publication Number Publication Date
CN104361332A true CN104361332A (en) 2015-02-18
CN104361332B CN104361332B (en) 2017-06-16

Family

ID=52528590

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410739606.7A Expired - Fee Related CN104361332B (en) 2014-12-08 2014-12-08 A kind of face eye areas localization method for fatigue driving detection

Country Status (1)

Country Link
CN (1) CN104361332B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529496A (en) * 2016-11-24 2017-03-22 广西大学 Locomotive driver real-time video fatigue detection method
WO2017096753A1 (en) * 2015-12-11 2017-06-15 腾讯科技（深圳）有限公司 Facial key point tracking method, terminal, and nonvolatile computer readable storage medium
CN108009495A (en) * 2017-11-30 2018-05-08 西安科锐盛创新科技有限公司 Fatigue driving method for early warning
CN108345847A (en) * 2018-01-30 2018-07-31 石数字技术成都有限公司 A kind of facial image label data generation system and method
CN108932461A (en) * 2017-05-27 2018-12-04 杭州海康威视数字技术股份有限公司 A kind of fatigue detection method and device
CN109376637A (en) * 2018-10-15 2019-02-22 齐鲁工业大学 Passenger number statistical system based on video monitoring image processing
CN110143202A (en) * 2019-04-09 2019-08-20 南京交通职业技术学院 A kind of dangerous driving identification and method for early warning and system
CN110930340A (en) * 2019-10-11 2020-03-27 成都华为技术有限公司 Image processing method and device
CN111753659A (en) * 2020-05-20 2020-10-09 广州虹科电子科技有限公司 Fatigue driving detection method, equipment, medium and device based on face registration points
CN112488032A (en) * 2020-12-11 2021-03-12 重庆邮电大学 Human eye positioning method based on nose-eye structure constraint
WO2021082635A1 (en) * 2019-10-29 2021-05-06 深圳云天励飞技术股份有限公司 Region of interest detection method and apparatus, readable storage medium and terminal device
CN113111883A (en) * 2021-03-23 2021-07-13 浙江大华技术股份有限公司 License plate detection method, electronic equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102436715A (en) * 2011-11-25 2012-05-02 大连海创高科信息技术有限公司 Detection method for fatigue driving
US20120314046A1 (en) * 2011-06-07 2012-12-13 Hon Hai Precision Industry Co., Ltd Tiredness state detecting system and method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120314046A1 (en) * 2011-06-07 2012-12-13 Hon Hai Precision Industry Co., Ltd Tiredness state detecting system and method
CN102436715A (en) * 2011-11-25 2012-05-02 大连海创高科信息技术有限公司 Detection method for fatigue driving

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
WEN-BING HORNG 等: "Driver Fatigue Detection Based on Eye Tracking and Dynamic’Template Matching", 《PROCEEDINGS OF THE 2004 IEEE INTERNATIONAL CONFERCNCC ON NCLVORKINP, SENSING & CONTROL》 *
张小欢 等: "司机疲劳驾驶检测中人脸图像的快速人眼定位", 《科技资讯》 *

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11062123B2 (en) 2015-12-11 2021-07-13 Tencent Technology (Shenzhen) Company Limited Method, terminal, and storage medium for tracking facial critical area
WO2017096753A1 (en) * 2015-12-11 2017-06-15 腾讯科技（深圳）有限公司 Facial key point tracking method, terminal, and nonvolatile computer readable storage medium
CN106874826A (en) * 2015-12-11 2017-06-20 腾讯科技（深圳）有限公司 Face key point-tracking method and device
US10452893B2 (en) 2015-12-11 2019-10-22 Tencent Technology (Shenzhen) Company Limited Method, terminal, and storage medium for tracking facial critical area
CN106529496B (en) * 2016-11-24 2019-04-12 广西大学 A kind of method of engine drivers in locomotive depot real-time video fatigue detecting
CN106529496A (en) * 2016-11-24 2017-03-22 广西大学 Locomotive driver real-time video fatigue detection method
CN108932461A (en) * 2017-05-27 2018-12-04 杭州海康威视数字技术股份有限公司 A kind of fatigue detection method and device
CN108009495A (en) * 2017-11-30 2018-05-08 西安科锐盛创新科技有限公司 Fatigue driving method for early warning
CN108345847A (en) * 2018-01-30 2018-07-31 石数字技术成都有限公司 A kind of facial image label data generation system and method
CN109376637A (en) * 2018-10-15 2019-02-22 齐鲁工业大学 Passenger number statistical system based on video monitoring image processing
CN110143202A (en) * 2019-04-09 2019-08-20 南京交通职业技术学院 A kind of dangerous driving identification and method for early warning and system
CN110930340A (en) * 2019-10-11 2020-03-27 成都华为技术有限公司 Image processing method and device
CN110930340B (en) * 2019-10-11 2023-09-29 成都华为技术有限公司 Image processing method and device
WO2021082635A1 (en) * 2019-10-29 2021-05-06 深圳云天励飞技术股份有限公司 Region of interest detection method and apparatus, readable storage medium and terminal device
CN111753659A (en) * 2020-05-20 2020-10-09 广州虹科电子科技有限公司 Fatigue driving detection method, equipment, medium and device based on face registration points
CN112488032A (en) * 2020-12-11 2021-03-12 重庆邮电大学 Human eye positioning method based on nose-eye structure constraint
CN112488032B (en) * 2020-12-11 2022-05-20 重庆邮电大学 Human eye positioning method based on nose and eye structure constraint
CN113111883A (en) * 2021-03-23 2021-07-13 浙江大华技术股份有限公司 License plate detection method, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN104361332B (en) 2017-06-16

Similar Documents

Publication Publication Date Title
CN104361332A (en) Human face eye region positioning method for fatigue driving detection
EP3539054B1 (en) Neural network image processing apparatus
CN109670441B (en) Method, system, terminal and computer readable storage medium for realizing wearing recognition of safety helmet
CN104616438B (en) A kind of motion detection method of yawning for fatigue driving detection
CN102289660B (en) Method for detecting illegal driving behavior based on hand gesture tracking
Junaedi et al. Driver drowsiness detection based on face feature and PERCLOS
JP6398979B2 (en) Video processing apparatus, video processing method, and video processing program
CN105286802B (en) Driver Fatigue Detection based on video information
US20140204193A1 (en) Driver gaze detection system
CN109840565A (en) A kind of blink detection method based on eye contour feature point aspect ratio
Yang et al. All in one network for driver attention monitoring
CN109334563A (en) A kind of anticollision method for early warning based on road ahead pedestrian and bicyclist
CN110728241A (en) Driver fatigue detection method based on deep learning multi-feature fusion
JP2013513155A (en) Cost-effective and robust system and method for eye tracking and driver awareness
Liu et al. A practical driver fatigue detection algorithm based on eye state
WO2018078857A1 (en) Line-of-sight estimation device, line-of-sight estimation method, and program recording medium
CN104123549A (en) Eye positioning method for real-time monitoring of fatigue driving
CN103729646B (en) Eye image validity detection method
CN113989788A (en) Fatigue detection method based on deep learning and multi-index fusion
CN101320477B (en) Human body tracing method and equipment thereof
Panicker et al. Open-eye detection using iris–sclera pattern analysis for driver drowsiness detection
CN116012822B (en) Fatigue driving identification method and device and electronic equipment
Singh et al. Driver fatigue detection using machine vision approach
CN114792437A (en) Method and system for analyzing safe driving behavior based on facial features
CN108205652A (en) A kind of recognition methods of action of having a meal and device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20170616

Termination date: 20171208

CF01 Termination of patent right due to non-payment of annual fee