CN112329516A - Method, device and medium for detecting wearing of mask of driver based on key point positioning and image classification - Google Patents
Method, device and medium for detecting wearing of mask of driver based on key point positioning and image classification Download PDFInfo
- Publication number
- CN112329516A CN112329516A CN202010966583.9A CN202010966583A CN112329516A CN 112329516 A CN112329516 A CN 112329516A CN 202010966583 A CN202010966583 A CN 202010966583A CN 112329516 A CN112329516 A CN 112329516A
- Authority
- CN
- China
- Prior art keywords
- mask
- image
- face
- key point
- detecting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for detecting the wearing of a mask of a driver based on key point positioning and image classification, which comprises the following steps: face detection, key point positioning and mask identification. The present invention relates to an electronic device and a storage medium for performing the above method. The mask detection method is based on the original face detection module of the conventional vehicle-mounted terminal equipment, and mask detection is carried out by adding key point positioning and template matching judgment. Because the key point positioning only needs 5-point detection, the existing public data set can be adopted; the mask detection is realized by adding key point positioning and template matching judgment, so that the mask detection and the face detection functional module are decoupled, the expansion on most existing architectures is facilitated, and the feasibility is high; the accurate position of the oral cavity part can be obtained after the key point is positioned, so that the mask is concentrated in characteristics, the mask can be identified and judged by adopting a simple template matching algorithm, the calculation speed is high, only a template needs to be added for the type of the newly added mask, and a data acquisition training model does not need to be repeated.
Description
Technical Field
The invention relates to the technical field of active safety defensive driving, in particular to a method, equipment and a medium for detecting wearing of a driver mask based on key point positioning and image classification.
Background
In the prior art, an existing face detection module is mainly modified based on a CNN neural network, and mask wearing detection is realized by increasing target detection categories. The method needs a large amount of mask wearing face data because the neural network needs to be retrained and the masks are various; for most vehicle-mounted terminal equipment with the existing active security defense system, the original framework needs to be modified, and the development cost is high; the difficulty of model training convergence is increased due to the increase of the number of detection categories, and finally the reasoning speed and precision of the model are reduced; meanwhile, the wearing mask detection and the face detection are coupled together, so that the butt joint application of a face detection model and other modules is limited, and the design is unreasonable under the condition that the vehicle-mounted equipment has limited computing resources.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide a method for detecting the wearing of a driver mask based on key point positioning and image classification, and solves the problems of large modification on the original architecture, large requirement on training data, high coupling degree with a face detection module and the like in the conventional vehicle-mounted terminal driver mask detection scheme in the market.
The invention provides a method for detecting the wearing of a mask of a driver based on key point positioning and image classification, which comprises the following steps:
detecting a human face in the acquired driver image, and cutting a human face position area to obtain a human face area image;
positioning key points, namely positioning the coordinates of the key points of the face in the face region image;
and identifying the mask, namely correcting the current face image based on a standard link through the face key point coordinates to obtain key points of the lip angles at two sides, generating an area to be identified through the key points of the lip angles at two sides, performing feature matching on the area to be identified and each mask and mouth shape picture in a mask template library, and returning an identification result.
Further, in the face detection step, the face in the acquired driver image is detected through a mobile-ssd neural network.
Further, the detecting the human face in the collected driver image through the mobile-ssd neural network comprises the following steps:
image preprocessing, namely scaling and normalizing the acquired driver image;
and model calculation, namely performing model calculation on the preprocessed image through a mobile-ssd neural network, and outputting the confidence coefficient and the position of the human face target.
Further, in the model calculation step, if a face is detected in the image, the face position area is cut to obtain a face area image, the key point positioning step is skipped to, and if no face is detected in the image, the next frame of image is acquired for processing.
Further, in the key point positioning step, mobile-ldnet is adopted to position the coordinates of key points of the face in the face region image, and a network architecture is designed by adopting a deepwise lightweight neural network structure, and is input into the face image and output as the coordinates of the key points of the face.
Further, in the mask recognition step, the current face image is corrected based on a standard face by adopting perspective transformation.
Further, in the mask identification step, the mask template library is established by selecting masks with various material colors, wearing and recording face pictures one by one, intercepting face mask region images in the pictures, and establishing a mask image template library.
Further, in the mask recognition step, the region to be recognized is generated by the key points of the two lip corners, specifically, a rectangular region is drawn by using the central point of the connecting line of the key points of the two lip corners as the center to serve as the region to be recognized.
An electronic device, comprising: a processor;
a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for performing a driver mask wear detection method based on keypoint localization and image classification.
A computer-readable storage medium having stored thereon a computer program for execution by a processor of a method for driver mask fit detection based on keypoint localization and image classification.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a method for detecting the wearing of a mask of a driver based on key point positioning and image classification, which comprises the following steps: detecting a human face in the acquired driver image, and cutting a human face position area to obtain a human face area image; positioning key points, namely positioning the coordinates of the key points of the face in the face region image; mask recognition, namely correcting the current face image based on a standard link through face key point coordinates to obtain two-side lip corner key points, generating a region to be recognized through the two-side lip corner key points, performing feature matching on the region to be recognized and each mask and mouth shape picture in a mask template library, and returning a recognition result. The invention relates to an electronic device and a storage medium for executing a mask wearing detection method for a driver based on key point positioning and image classification. The mask detection method is based on the original face detection module of the conventional vehicle-mounted terminal equipment, and mask detection is carried out by adding key point positioning and template matching judgment. Because the key point positioning only needs 5-point detection, the existing public data set can be adopted; the mask detection is realized by adding key point positioning and template matching judgment, so that the mask detection and the face detection functional module are decoupled, the expansion on most existing architectures is facilitated, and the feasibility is high; the accurate position of the oral cavity part can be obtained after the key point is positioned, so that the mask is concentrated in characteristics, the mask can be identified and judged by adopting a simple template matching algorithm, the calculation speed is high, only a template needs to be added for the type of the newly added mask, and a data acquisition training model does not need to be repeated.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood and to implement them in accordance with the contents of the description, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings. The detailed description of the present invention is given in detail by the following examples and the accompanying drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a flowchart of a method for detecting the wearing of a mask of a driver based on key point positioning and image classification according to the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and the detailed description, and it should be noted that any combination of the embodiments or technical features described below can be used to form a new embodiment without conflict.
The method for detecting the wearing of the mask of the driver based on the key point positioning and the image classification comprises the following steps as shown in figure 1:
establishing a mask template library, selecting masks with various material colors, wearing and recording face-righting pictures one by one, intercepting facial mask area images in the pictures, and establishing a mask image template library;
detecting a human face in the acquired driver image, and cutting a human face position area to obtain a human face area image; in this embodiment, detecting a face in an acquired driver image through a mobile-ssd neural network specifically includes the following steps:
image preprocessing, namely scaling and normalizing the acquired driver image to meet the input requirement of a neural network; in this embodiment, the image is scaled to 300x300 dimensions and divided by 255 for pixel value normalization.
And model calculation, namely performing model calculation on the preprocessed image through a mobile-ssd neural network, outputting the confidence coefficient and the position of a human face target, cutting a human face position area to obtain a human face area image if the human face is detected in the image, skipping to a key point positioning step, and acquiring the next frame of image for processing if the human face is not detected in the image.
Positioning key points, namely positioning the coordinates of the key points of the face in the face region image; preferably, in the key point positioning step, the mobile-ldnet is adopted to position the coordinates of key points of the face in the face region image, the network architecture is designed by adopting a deepwise lightweight neural network structure, the coordinates of key points of the face are input into the face image, and the coordinates of key points of the face are output. In this embodiment, the face region image is scaled to 96 × 96 width and height images, and is divided by 255 to perform pixel value normalization; adopting a mobile-ldnet neural network to carry out reasoning; and acquiring coordinates of 5 face key points of the inference structure.
Mask recognition, namely correcting the current face image based on a standard link through face key point coordinates to obtain two-side lip corner key points, drawing a rectangular area as a to-be-recognized area by taking the central point of the connecting line of the two-side lip corner key points as the center, performing feature matching on the to-be-recognized area and each mask and mouth shape pictures in a mask template library, and returning a recognition result. In this embodiment, the current face image is corrected based on a standard face by adopting perspective transformation.
It should be understood that the face detection step and the key point positioning step are both realized by adopting a neural network, and the network structure is not fixed and can be replaced by other network structures; the mask identification step adopts template matching for judgment, and the characteristic form based on the template matching is not fixed and can be replaced by other characteristics.
An electronic device, comprising: a processor;
a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for performing a driver mask wear detection method based on the keypoint localization and the image classification.
A computer-readable storage medium having stored thereon a computer program for execution by a processor of a method for detecting driver mask wear based on keypoint localization and image classification.
The invention provides a method for detecting the wearing of a mask of a driver based on key point positioning and image classification, which comprises the following steps: detecting a human face in the acquired driver image, and cutting a human face position area to obtain a human face area image; positioning key points, namely positioning the coordinates of the key points of the face in the face region image; mask recognition, namely correcting the current face image based on a standard link through face key point coordinates to obtain two-side lip corner key points, generating a region to be recognized through the two-side lip corner key points, performing feature matching on the region to be recognized and each mask and mouth shape picture in a mask template library, and returning a recognition result. The invention relates to an electronic device and a storage medium for executing a mask wearing detection method for a driver based on key point positioning and image classification. The mask detection method is based on the original face detection module of the conventional vehicle-mounted terminal equipment, and mask detection is carried out by adding key point positioning and template matching judgment. Because the key point positioning only needs 5-point detection, the existing public data set can be adopted; the mask detection is realized by adding key point positioning and template matching judgment, so that the mask detection and the face detection functional module are decoupled, the expansion on most existing architectures is facilitated, and the feasibility is high; the accurate position of the oral cavity part can be obtained after the key point is positioned, so that the mask is concentrated in characteristics, the mask can be identified and judged by adopting a simple template matching algorithm, the calculation speed is high, only a template needs to be added for the type of the newly added mask, and a data acquisition training model does not need to be repeated.
The foregoing is merely a preferred embodiment of the invention and is not intended to limit the invention in any manner; those skilled in the art can readily practice the invention as shown and described in the drawings and detailed description herein; however, those skilled in the art should appreciate that they can readily use the disclosed conception and specific embodiments as a basis for designing or modifying other structures for carrying out the same purposes of the present invention without departing from the scope of the invention as defined by the appended claims; meanwhile, any changes, modifications, and evolutions of the equivalent changes of the above embodiments according to the actual techniques of the present invention are still within the protection scope of the technical solution of the present invention.
Claims (10)
1. The method for detecting the wearing of the mask of the driver based on the key point positioning and the image classification is characterized by comprising the following steps of:
detecting a human face in the acquired driver image, and cutting a human face position area to obtain a human face area image;
positioning key points, namely positioning the coordinates of the key points of the face in the face region image;
and identifying the mask, namely correcting the current face image based on a standard link through the face key point coordinates to obtain key points of the lip angles at two sides, generating an area to be identified through the key points of the lip angles at two sides, performing feature matching on the area to be identified and each mask and mouth shape picture in a mask template library, and returning an identification result.
2. The method for detecting mask wearing of driver based on keypoint localization and image classification as claimed in claim 1, characterized in that: in the face detection step, the face in the acquired driver image is detected through a mobile-ssd neural network.
3. The method for detecting the mask wearing of the driver based on the keypoint localization and the image classification as claimed in claim 2, wherein the detecting the face in the captured image of the driver by the mobile-ssd neural network comprises the following steps:
image preprocessing, namely scaling and normalizing the acquired driver image;
and model calculation, namely performing model calculation on the preprocessed image through a mobile-ssd neural network, and outputting the confidence coefficient and the position of the human face target.
4. The method for detecting mask wearing of driver based on keypoint localization and image classification as claimed in claim 3, characterized in that: in the model calculation step, if a face is detected in the image, the face position area is cut to obtain a face area image, the key point positioning step is skipped to, and if no face is detected in the image, the next frame image is obtained for processing.
5. The method for detecting mask wearing of driver based on keypoint localization and image classification as claimed in claim 1, characterized in that: in the key point positioning step, mobile-ldnet is adopted to position the coordinates of key points of the human face in the human face area image, a network architecture is designed by adopting a deepwise lightweight neural network structure, the coordinates of the key points of the human face are input into the human face image, and the coordinates of the key points of the human face are output.
6. The method for detecting mask wearing of driver based on keypoint localization and image classification as claimed in claim 1, characterized in that: in the step of mask recognition, the current face image is corrected based on a standard face by adopting perspective transformation.
7. The method for detecting mask wearing of driver based on keypoint localization and image classification as claimed in claim 1, characterized in that: in the step of mask identification, the establishment of mask template library wears and records the face-to-face picture one by one for the mask through selecting multiple material colours, intercepts the regional image of face mask in the picture, and establishes mask image template library.
8. The method for detecting mask wearing of driver based on keypoint localization and image classification as claimed in claim 1, characterized in that: in the mask identification step, the area to be identified is generated through the key points of the two lip corners, and specifically, a rectangular area is drawn by taking the central point of the connecting line of the key points of the two lip corners as the center to serve as the area to be identified.
9. An electronic device, characterized by comprising: a processor;
a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising instructions for carrying out the method of any one of claims 1-8.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program is executed by a processor for performing the method according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010966583.9A CN112329516A (en) | 2020-09-15 | 2020-09-15 | Method, device and medium for detecting wearing of mask of driver based on key point positioning and image classification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010966583.9A CN112329516A (en) | 2020-09-15 | 2020-09-15 | Method, device and medium for detecting wearing of mask of driver based on key point positioning and image classification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112329516A true CN112329516A (en) | 2021-02-05 |
Family
ID=74304002
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010966583.9A Pending CN112329516A (en) | 2020-09-15 | 2020-09-15 | Method, device and medium for detecting wearing of mask of driver based on key point positioning and image classification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112329516A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818953A (en) * | 2021-03-12 | 2021-05-18 | 苏州科达科技股份有限公司 | Mask wearing state identification method, device, equipment and readable storage medium |
| CN113420675A (en) * | 2021-06-25 | 2021-09-21 | 浙江大华技术股份有限公司 | Method and device for detecting mask wearing standardization |
| CN113822152A (en) * | 2021-08-09 | 2021-12-21 | 中标慧安信息技术股份有限公司 | Method for monitoring clothing condition of commercial tenant of food in market |
| CN113936310A (en) * | 2021-08-03 | 2022-01-14 | 秒针信息技术有限公司 | Method and device for identifying confrontation sample mask |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659596A (en) * | 2019-09-11 | 2020-01-07 | 高新兴科技集团股份有限公司 | Face key point positioning method under case and management scene, computer storage medium and equipment |
| CN111428559A (en) * | 2020-02-19 | 2020-07-17 | 北京三快在线科技有限公司 | Method and device for detecting wearing condition of mask, electronic equipment and storage medium |
-
2020
- 2020-09-15 CN CN202010966583.9A patent/CN112329516A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659596A (en) * | 2019-09-11 | 2020-01-07 | 高新兴科技集团股份有限公司 | Face key point positioning method under case and management scene, computer storage medium and equipment |
| CN111428559A (en) * | 2020-02-19 | 2020-07-17 | 北京三快在线科技有限公司 | Method and device for detecting wearing condition of mask, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 文昌卫: "基于深度学习的人脸识别***的设计和实现", 《中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112818953A (en) * | 2021-03-12 | 2021-05-18 | 苏州科达科技股份有限公司 | Mask wearing state identification method, device, equipment and readable storage medium |
| CN113420675A (en) * | 2021-06-25 | 2021-09-21 | 浙江大华技术股份有限公司 | Method and device for detecting mask wearing standardization |
| CN113936310A (en) * | 2021-08-03 | 2022-01-14 | 秒针信息技术有限公司 | Method and device for identifying confrontation sample mask |
| CN113822152A (en) * | 2021-08-09 | 2021-12-21 | 中标慧安信息技术股份有限公司 | Method for monitoring clothing condition of commercial tenant of food in market |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112329516A (en) | Method, device and medium for detecting wearing of mask of driver based on key point positioning and image classification | |
| US11830230B2 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| US11043011B2 (en) | Image processing method, apparatus, terminal, and storage medium for fusing images of two objects | |
| US10049262B2 (en) | Method and system for extracting characteristic of three-dimensional face image | |
| CN112541422B (en) | Expression recognition method, device and storage medium with robust illumination and head posture | |
| CN110287790B (en) | Learning state hybrid analysis method oriented to static multi-user scene | |
| US7912253B2 (en) | Object recognition method and apparatus therefor | |
| CN111563452B (en) | Multi-human-body gesture detection and state discrimination method based on instance segmentation | |
| CN110728209A (en) | Gesture recognition method and device, electronic equipment and storage medium | |
| US11194997B1 (en) | Method and system for thermal infrared facial recognition | |
| CN105139000B (en) | A kind of face identification method and device removing glasses trace | |
| JP2019117577A (en) | Program, learning processing method, learning model, data structure, learning device and object recognition device | |
| WO2020029915A1 (en) | Artificial intelligence-based device and method for tongue image splitting in traditional chinese medicine, and storage medium | |
| CN111914748B (en) | Face recognition method, device, electronic equipment and computer readable storage medium | |
| CN116052222A (en) | Cattle face recognition method for naturally collecting cattle face image | |
| CN114241542A (en) | Face recognition method based on image stitching | |
| CN111898571A (en) | Action recognition system and method | |
| CN113963237B (en) | Model training method, mask wearing state detection method, electronic device and storage medium | |
| CN111626241A (en) | Face detection method and device | |
| CN110674675A (en) | Pedestrian face anti-fraud method | |
| CN113095119A (en) | Face recognition system for correcting face cutting frame | |
| CN110826495A (en) | Body left and right limb consistency tracking and distinguishing method and system based on face orientation | |
| CN111523406B (en) | Deflection face correcting method based on generation confrontation network improved structure | |
| CN116311495A (en) | Dual-stream global-local action recognition method, system, equipment and storage medium based on video input | |
| CN111881732B (en) | SVM (support vector machine) -based face quality evaluation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210205 |
|
| RJ01 | Rejection of invention patent application after publication |