CN112801002A - Facial expression recognition method and device based on complex scene and electronic equipment - Google Patents
Facial expression recognition method and device based on complex scene and electronic equipment Download PDFInfo
- Publication number
- CN112801002A CN112801002A CN202110162293.3A CN202110162293A CN112801002A CN 112801002 A CN112801002 A CN 112801002A CN 202110162293 A CN202110162293 A CN 202110162293A CN 112801002 A CN112801002 A CN 112801002A
- Authority
- CN
- China
- Prior art keywords
- recognized
- face
- image
- facial expression
- facial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008921 facial expression Effects 0.000 title claims abstract description 215
- 238000000034 method Methods 0.000 title claims abstract description 59
- 230000001815 facial effect Effects 0.000 claims abstract description 58
- 238000012545 processing Methods 0.000 claims abstract description 29
- 238000003708 edge detection Methods 0.000 claims abstract description 12
- 230000015654 memory Effects 0.000 claims description 20
- 238000001514 detection method Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 description 8
- 239000003086 colorant Substances 0.000 description 3
- 230000002950 deficient Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a device for recognizing facial expressions based on a complex scene and electronic equipment, and particularly relates to the technical field of face recognition. Wherein the method comprises the following steps: acquiring a face image to be recognized; carrying out image graying processing based on the face image to be recognized to obtain the face grayscale image to be recognized; performing edge detection according to the gray level image of the face to be recognized to obtain an edge contour of at least one face to be recognized in the gray level image of the face to be recognized; correspondingly setting at least one facial expression recognition window based on the edge contour of at least one face to be recognized; the facial expressions in the facial images to be recognized are recognized by the aid of the face recognition windows, so that the facial expression recognition can flexibly adapt to complex scenes, and the facial expression recognition windows are arranged to recognize each piece of facial information in the facial images, so that the facial expressions in the facial images can be recognized accurately.
Description
Technical Field
The invention relates to the technical field of face recognition, in particular to a face expression recognition method and device based on a complex scene and electronic equipment.
Background
With the progress of computer software and hardware technologies, face recognition also enters face expression recognition from single facial recognition, but for the facial expression recognition at the present stage, although the face expression recognition can be performed, because in complex scenes with more people, such as schools, airports, stations and the like, facial expression differences between individuals exist between every two faces or each facial image is affected by background conditions such as environments, lighting positions, shielding and the like in the process of face recognition, a plurality of facial expressions in the face images cannot be accurately recognized.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for recognizing a facial expression based on a complex scene, and an electronic device, so as to solve a problem that a plurality of facial expressions in a facial image cannot be accurately recognized.
According to a first aspect, an embodiment of the present invention provides a facial expression recognition method based on a complex scene, including: acquiring a face image to be recognized; the face image to be recognized comprises at least one face to be recognized; carrying out image graying processing on the basis of the face image to be recognized to obtain a face grayscale image to be recognized; performing edge detection according to the face gray level image to be recognized to obtain an edge contour of the at least one face to be recognized in the face gray level image to be recognized; correspondingly setting at least one facial expression recognition window based on the edge contour of the at least one face to be recognized; and performing facial expression recognition on the face in the facial image to be recognized by utilizing the at least one facial expression recognition window.
In the embodiment, a face image to be recognized is obtained, the face image to be recognized is subjected to image graying processing to highlight face edge contours in the face image, then, a face expression recognition window is arranged on each face edge contour detected in the face image, and a plurality of face recognition windows are used for recognizing face expressions in the face image to be recognized, so that the face expression recognition can flexibly adapt to complex scenes, and meanwhile, each piece of face information in the face image can be recognized by arranging the face expression recognition windows, and therefore, a plurality of face expressions in the face image can be recognized accurately.
With reference to the first aspect, in a first implementation manner of the first aspect, the performing facial expression recognition on a face in a facial image to be recognized by using the at least one facial expression recognition window includes:
acquiring a facial expression image; extracting facial expression features based on the facial expression image to obtain facial expression features; sending the facial expression features and the facial expression images into an identification model for iterative training to obtain a facial expression identification model; intercepting a face to be recognized based on the at least one facial expression recognition window to obtain facial expression information to be recognized; and sending the facial expression information to be recognized into a facial expression recognition model for facial expression recognition, and outputting facial expression recognition information.
In the embodiment, a facial expression recognition model is established by obtaining a facial expression image, and then facial expression recognition is performed by obtaining a face to be recognized intercepted by a facial expression recognition window, so that the problem that in the prior art, due to the fact that a plurality of facial expressions in a facial image are recognized at a time, optimal facial expression recognition information cannot be obtained due to the influence of background conditions such as environment, lighting position and shielding in the facial image is solved, and the recognition accuracy of the facial expressions and the execution efficiency of the facial recognition are improved by setting a plurality of facial expression recognition windows and utilizing the mode of the face to be recognized intercepted by the windows.
With reference to the first aspect, in a second implementation manner of the first aspect, the performing image graying processing based on the face image to be recognized to obtain a face grayscale image to be recognized includes:
extracting three primary color information of the face image to be recognized; carrying out gray level calculation based on the three primary color information of the face image to be recognized to obtain the gray level information of the face to be recognized; and setting the face image to be recognized by using the face gray level information to be recognized to obtain a face gray level image to be recognized.
In this embodiment, the image graying processing is performed on the face image to be recognized, so as to obtain a better edge contour of a single face in the face image, thereby further improving the accuracy of facial expression recognition.
With reference to the first aspect, in a third implementation manner of the first aspect, before the performing edge detection according to the gray-scale image of the face to be recognized to obtain an edge contour of the at least one face to be recognized in the gray-scale image of the face to be recognized, the method further includes: and sending the face gray level image to be recognized into a Gaussian filter for image filtering to obtain a filtering image to be recognized.
In this embodiment, the gray level image of the face to be recognized is sent to a gaussian filter for image filtering, so as to obtain a filtered image to be recognized, thereby further improving the accuracy of facial expression recognition.
With reference to the first aspect or the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the performing edge detection according to the gray-scale image of the face to be recognized to obtain an edge contour of the at least one face to be recognized in the gray-scale image of the face to be recognized includes: acquiring human face features in a human face gray level image to be recognized; judging whether the facial features meet the edge contour generation conditions of the face to be recognized or not based on the facial features; if the facial features of the human face meet the edge contour generation condition of the human face to be recognized, obtaining the edge contour of the human face to be recognized; and if the face facial features do not meet the edge contour generation conditions of the face to be recognized, re-acquiring the face facial features in the gray level image of the face to be recognized.
In this embodiment, the accuracy of facial expression recognition is further improved by determining whether the facial features of the face meet the edge contour generation condition of the face to be recognized.
According to a second aspect, an embodiment of the present invention provides a device for recognizing a facial expression based on a complex scene, including: the first acquisition module is used for acquiring a face image to be recognized; the face image to be recognized comprises at least one face to be recognized; the processing module is used for carrying out image graying processing on the basis of the face image to be recognized to obtain a face grayscale image to be recognized; the detection module is used for carrying out edge detection according to the gray level image of the face to be recognized to obtain the edge contour of the at least one face to be recognized in the gray level image of the face to be recognized; the setting module is used for correspondingly generating at least one facial expression recognition window based on the edge contour of the at least one face to be recognized; and the first recognition module is used for carrying out facial expression recognition on the face in the facial image to be recognized by utilizing the at least one facial expression recognition window.
In this embodiment, a first obtaining module is arranged to obtain a face image to be recognized, the obtained face image to be recognized is sent to a processing module, the processing module performs graying processing on the face image to be recognized, the face image to be recognized after the graying processing is sent to a detection module for face contour detection, detected contour information is sent to a setting module, a face expression recognition window is generated through the setting module, then a first recognition module is used for recognizing a face expression in the face expression recognition window, so that the problem that in the prior art, due to the influence of background conditions such as environment, lighting position and shielding in the face image, optimal face expression recognition information cannot be obtained due to the fact that multiple face expressions in the face image are recognized at a single time is solved, and the recognition accuracy of the face expression and the face recognition are improved by arranging the multiple face expression recognition windows and utilizing a face to be recognized intercepted by the windows The efficiency of execution of (c).
With reference to the first aspect, in a first embodiment of the second aspect, the method includes:
the second acquisition module is used for acquiring the facial expression image;
the first extraction module is used for extracting facial expression characteristics based on the facial expression image to obtain facial expression characteristics;
the training module is used for sending the facial expression characteristics and the facial expression image into a recognition model for iterative training to obtain a facial expression recognition model;
the intercepting module is used for intercepting the face to be identified based on the at least one facial expression identifying window to obtain the facial expression information to be identified;
and the second recognition module is used for sending the facial expression information to be recognized into a facial expression recognition model for facial expression recognition and outputting the facial expression recognition information.
With reference to the second aspect, in a second embodiment of the second aspect, the method includes:
the second extraction module is used for extracting the three primary color information of the face image to be recognized;
the calculation module is used for carrying out gray level calculation on the basis of the three primary color information of the face image to be recognized to obtain the gray level information of the face to be recognized;
and the setting module is used for setting the face image to be recognized by utilizing the face gray level information to be recognized to obtain a face gray level image to be recognized.
According to a third aspect, an embodiment of the present invention provides an electronic device, including: the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the method for recognizing facial expressions based on complex scenes in the first aspect or any one of the embodiments of the first aspect.
According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions for causing a computer to execute the method for recognizing facial expressions based on complex scenes in the first aspect or any one of the implementation manners of the first aspect.
Drawings
The features and advantages of the present invention will be more clearly understood by reference to the accompanying drawings, which are illustrative and not to be construed as limiting the invention in any way, and in which:
FIG. 1 is a flow chart of a facial expression recognition method based on a complex scene;
fig. 2 is a flowchart of step S2 in the method for recognizing facial expressions based on complex scenes according to the embodiment of the present invention;
fig. 3 is a flowchart of step S4 in the method for recognizing facial expressions based on complex scenes according to the embodiment of the present invention;
fig. 4 is a flowchart of step S5 in the method for recognizing facial expressions based on complex scenes according to the embodiment of the present invention;
fig. 5 is a block diagram of a structure of facial expression recognition based on a complex scene according to an embodiment of the present invention;
fig. 6 is a block diagram of an electronic device according to an embodiment of the present invention.
Reference numerals
1-a first acquisition module; 2-a processing module; 3-a detection module; 4-setting a module; 5-a first identification module; 61-a processor; 62-memory.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be further noted that the complex scene in the present application may be understood as a situation where at least one piece of face information exists in face information in a face image, and a scene of face recognition cannot be accurately recognized in a situation where a difference in facial expression between individuals exists between each face in the face image or each face image is affected by factors such as environment, light, and the like. In order to accurately recognize a plurality of facial expressions in the facial image. The embodiment provides a facial expression recognition method based on a complex scene, and specifically, as shown in fig. 1, the method is a flowchart of the facial expression recognition method based on the complex scene, and the facial expression recognition method based on the complex scene includes:
s1, acquiring a face image to be recognized; the face image to be recognized comprises at least one face to be recognized.
In this embodiment, the facial image to be recognized may be the facial image to be recognized that is uploaded to a data center, a cloud, or a memory card through a camera or manually collected in advance, and at least 1 person of facial information exists in the facial image to be recognized. Optionally, in order to prevent validity of the face image to be recognized, the face image with the face information in the image needs to be manually or automatically screened out by a computer before the face image to be recognized is acquired.
And S2, carrying out image graying processing based on the face image to be recognized to obtain the face grayscale image to be recognized.
In this embodiment, when the face image to be recognized is obtained, the image graying processing may be performed on the face image to be recognized through any one or more of a component method, a maximum value method, an average value method and a weighted average method, so as to obtain a gray level image of the face to be recognized, so as to extract the face contour of the face image. Preferably, the image graying process may be performed by a weighted average method to obtain a more balanced grayscale image.
And S3, performing edge detection according to the gray level image of the face to be recognized to obtain an edge contour of at least one face to be recognized in the gray level image of the face to be recognized.
In this embodiment, after obtaining the gray level image of the face to be recognized, smooth image data may be obtained by using gaussian filtering, then the gradient amplitude and the gradient direction in the gray level image of the face to be recognized are obtained by using a canny operator according to the obtained image data, then non-edge information is removed for highlighting the edge profile, the non-edge information is removed in a non-maximum suppression manner, and finally, in order to obtain an accurate and effective edge profile of the face to be recognized, a complete edge profile is obtained by closing the edge by using a dual-threshold method, so that an optimal face edge profile can be extracted, and the accuracy of the face recognition is ensured. Optionally, the edge contour extraction of the face gray level image to be recognized may be implemented by using modeling software such as matlab and labview. The specific implementation steps of the edge detection may refer to the existing execution processes, and are not described herein again.
And S4, correspondingly setting at least one facial expression recognition window based on the edge contour of at least one face to be recognized.
In this embodiment, when the edge profile of the face to be recognized is obtained, a sliding window for presetting facial expression recognition needs to be set according to the edge profile information of the face to be recognized and the maximum length and width data of the edge profile, and the sliding window can be automatically adjusted according to the change of the edge profile. Further improving the implementation efficiency. In addition, in the face image with the face information, the number of windows for face expression recognition preset in the face image is equal to the number of faces to be recognized in the face image.
And S5, performing facial expression recognition on the face in the facial image to be recognized by using at least one facial expression recognition window.
In this embodiment, the facial information in a single facial image to be recognized is intercepted through the facial expression recognition window, and the intercepted content is sent to the facial expression recognition model for facial expression recognition, where the intercepted facial information is input information to the facial expression recognition model, and optionally, a user may select one-by-one facial expression recognition or select parallel recognition of the facial information. If parallel recognition is executed, a plurality of facial expression recognition models need to be connected, so that the recognition precision of the facial expressions and the execution efficiency of the facial recognition are further improved.
In the embodiment, a face image to be recognized is obtained, the face image to be recognized is subjected to image graying processing to highlight face edge contours in the face image, then, a face expression recognition window is arranged on each face edge contour detected in the face image, and a plurality of face recognition windows are used for recognizing face expressions in the face image to be recognized, so that the face expression recognition can flexibly adapt to complex scenes, and meanwhile, each piece of face information in the face image can be recognized by arranging the face expression recognition windows, and therefore, a plurality of face expressions in the face image can be recognized accurately.
Optionally, as shown in fig. 2, step S2 in the method for recognizing facial expressions based on complex scenes provided in this embodiment further includes:
and S21, extracting the three primary color information of the face image to be recognized.
In this embodiment, it is necessary to collect the three primary color information of red, green, and blue of the face image to be recognized by using the image software, and record the collected three primary color information.
And S22, performing gray level calculation based on the three primary color information of the face image to be recognized to obtain the gray level information of the face to be recognized.
And S23, setting the face image to be recognized by using the face gray scale information to be recognized to obtain a face gray scale image to be recognized.
In this embodiment, information values of three primary colors are obtained through image software, a gray level information value of a face to be recognized, which needs to be set, is obtained through an average value method, and simultaneously, the information values of the three primary colors are set as the gray level information value of the face to be recognized through the image software, so as to obtain a gray level image. For example: the formula of the average method is:
C＝(R+B+G)/3；
the method comprises the steps of obtaining a gray level information value to be set, obtaining a red information value in a face image to be recognized, obtaining a blue information value in the face image to be recognized, obtaining a green information value in the face image to be recognized, and obtaining a gray level information value in the face image to be recognized.
And when the C value is obtained, setting the three primary colors as the C value one by using image software, and obtaining the gray image of the face image to be recognized.
Optionally, when the gray-scale value is adjusted, the adjustment may also be performed according to the brightness characteristic of the light.
In this embodiment, the image graying processing is performed on the face image to be recognized, so as to obtain a better edge contour of a single face in the face image, thereby further improving the accuracy of facial expression recognition.
Optionally, before step S4 in the method for recognizing facial expression based on complex scene provided in this embodiment, the method further includes: and sending the gray level image of the face to be recognized into a Gaussian filter for image filtering to obtain a filtering image to be recognized by the face.
In this embodiment, a gaussian filter is used to filter a face grayscale image to be recognized, then a gradient amplitude and a gradient direction are calculated through a derivative, then non-maximum suppression is performed according to the amplitude and the gradient direction, and then a dual threshold is used to perform detection to connect edge points of the face grayscale image, so as to obtain an edge of the face image.
Optionally, as shown in fig. 3, step S4 in the method for recognizing facial expressions based on complex scenes provided in this embodiment further includes:
and S41, acquiring the face features in the gray level image of the face to be recognized.
In this embodiment, a gray-scale image of the face to be recognized may be obtained through step S3, and facial features may be extracted from the gray-scale image of the face to be recognized, where the facial features may be facial feature information, such as: ear, nose, mouth, eye information.
And S42, judging whether the face features meet the edge contour generation conditions of the face to be recognized or not based on the face features.
In this embodiment, the condition for generating the edge contour of the face to be recognized is whether face facial features corresponding to face information in the gray-scale image of the face to be recognized are complete, that is, whether facial feature information in the face information is defective or not needs to be recognized and judged, and if the facial feature is defective, the accuracy of subsequent facial expression recognition is affected, so that in order to ensure the accuracy of facial expression recognition, incomplete face information needs to be removed, so as to ensure the accuracy of a final result.
And S43, if the facial features of the human face meet the edge contour generation condition of the human face to be recognized, obtaining the edge contour of the human face to be recognized.
In this embodiment, if the facial features of the human face satisfy the edge contour generation condition of the human face to be recognized, the obtained edge contour of the human face to be recognized is highlighted.
And S44, if the face features do not meet the edge contour generation conditions of the face to be recognized, re-acquiring the face features in the gray level image of the face to be recognized.
Optionally, as shown in fig. 4, step S5 in the method for recognizing facial expression based on complex scene provided in this embodiment further includes:
and S51, acquiring the facial expression image.
In this embodiment, the facial expression pictures may be extracted from an existing facial database, and the facial expression pictures are arranged into a facial expression image set, where the image set is used to train a facial expression recognition model.
And S52, extracting facial expression features based on the facial expression images to obtain the facial expression features.
In this embodiment, the expression features and facial features may be extracted from the facial expression image. For example: the facial expression features may be raised pictures of the corners of the mouth in the face information.
And S53, sending the facial expression features and the facial expression images into a recognition model for iterative training to obtain a facial expression recognition model.
In this embodiment, partial image information may be extracted from the facial expression image, facial expression feature extraction may be performed on the image information, and then the extracted feature data is used as a reference, and the reference feature information and the facial expression image are sent to a recognition model for training, so as to obtain a facial expression recognition model.
S54, intercepting the face to be recognized based on at least one facial expression recognition window to obtain facial expression information to be recognized;
and S55, sending the facial expression information to be recognized into a facial expression recognition model for facial expression recognition, and outputting the facial expression recognition information.
In this embodiment, the obtained facial expression recognition model is matched with a facial expression recognition window marked in a to-be-recognized facial image, wherein the facial expression recognition window intercepts facial expression information first, and then sends the intercepted facial expression information to the facial expression recognition model for facial information recognition, so as to realize rapid and accurate acquisition of facial expression information.
Optionally, the facial expression recognition window is determined according to the quantity information in the facial image, when a face needs to be recognized by multiple facial expressions in a complex scene, multiple facial expression recognition can be performed through the facial expression recognition method based on the complex scene provided by the application, because the method generates a pair of detection recognition windows, namely the facial expression recognition windows, on a single image, each window is equivalent to an independent detection device, when multiple faces on the same image are recognized, the facial expression difference between individuals of each facial face in the same image can be reduced, or each facial image can be influenced by background conditions such as environment, lighting position, shielding and the like in the process of facial recognition, so that the multiple facial expressions in the facial image can be accurately recognized.
In addition, it should be understood that the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Accordingly, referring to fig. 5, an embodiment of the present invention provides a device for recognizing facial expressions based on a complex scene, where the device includes:
the first acquisition module 1 is used for acquiring a face image to be recognized; wherein the face image to be recognized comprises at least one face to be recognized, and the detailed content refers to step S1;
the processing module 2 is configured to perform image graying processing based on the face image to be recognized to obtain a face grayscale image to be recognized, and the detailed content refers to step S2;
the detection module 3 is configured to perform edge detection according to the gray level image of the face to be recognized, to obtain an edge contour of the at least one face to be recognized in the gray level image of the face to be recognized, and the detailed content refers to step S3;
a setting module 4, configured to generate at least one facial expression recognition window based on the edge contour of the at least one face to be recognized, where the detailed content refers to step S4;
the first recognition module 5 is configured to perform facial expression recognition on a face in a facial image to be recognized by using the at least one facial expression recognition window, and the detailed content refers to step S5.
In this embodiment, a first obtaining module is arranged to obtain a face image to be recognized, the obtained face image to be recognized is sent to a processing module, the processing module performs graying processing on the face image to be recognized, the face image to be recognized after the graying processing is sent to a detection module for face contour detection, detected contour information is sent to a setting module, a face expression recognition window is generated through the setting module, then a first recognition module is used for recognizing a face expression in the face expression recognition window, so that the problem that in the prior art, due to the influence of background conditions such as environment, lighting position and shielding in the face image, optimal face expression recognition information cannot be obtained due to the fact that multiple face expressions in the face image are recognized at a single time is solved, and the recognition accuracy of the face expression and the face recognition are improved by arranging the multiple face expression recognition windows and utilizing a face to be recognized intercepted by the windows The efficiency of execution of (c).
Optionally, in this embodiment, the method may further include:
a second obtaining module, configured to obtain the facial expression image, where details are described with reference to step S51.
A first extraction module, configured to extract facial expression features based on the facial expression image to obtain facial expression features, where the detailed content refers to step S52.
And the training module is used for sending the facial expression features and the facial expression images into a recognition model for iterative training to obtain a facial expression recognition model, and the detailed content refers to the step S53.
And an intercepting module, configured to intercept a face to be recognized based on the at least one facial expression recognition window to obtain facial expression information to be recognized, where the detailed content refers to step S54.
And the second identification module is used for sending the facial expression information to be identified to a facial expression identification model for facial expression identification and outputting the facial expression identification information, and the detailed content refers to the step S55.
Optionally, in this embodiment, the method may further include:
and a second extraction module, configured to extract three primary color information of the face image to be recognized, where details are described in reference to step S21.
And a calculating module, configured to perform gray level calculation based on the three primary color information of the face image to be recognized to obtain gray level information of the face to be recognized, where the detailed content refers to step S22.
And a setting module, configured to set the face image to be recognized by using the face gray scale information to be recognized to obtain a face gray scale image to be recognized, where the details refer to step S23.
An embodiment of the present invention further provides an electronic device, as shown in fig. 6, the electronic device may include a processor 61 and a memory 62, where the processor 61 and the memory 62 may be connected by a bus or in another manner, and fig. 6 illustrates the connection by the bus as an example.
The processor 61 may be a Central Processing Unit (CPU). The Processor 61 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 62 is a non-transitory computer-readable storage medium, and can be used to store a non-transitory software program, a non-transitory computer-executable program, and modules, such as program instructions/modules corresponding to the facial expression recognition method based on a complex scene in the embodiment of the present invention (for example, the first obtaining module 1, the processing module 2, the detecting module 3, the setting module 4, and the first recognition module 5 shown in fig. 5). The processor 61 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 62, that is, implements the method for recognizing facial expressions based on complex scenes in the above method embodiments.
The memory 62 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor 61, and the like. Further, the memory 62 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 62 may optionally include memory located remotely from the processor 61, and these remote memories may be connected to the processor 61 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 62 and when executed by the processor 61, perform a method of facial expression recognition based on complex scenes as in the embodiment shown in fig. 1-4.
The details of the electronic device may be understood by referring to the corresponding descriptions and effects in the embodiments shown in fig. 1 to fig. 4, and are not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.
Claims (10)
1. A facial expression recognition method based on a complex scene is characterized by comprising the following steps:
acquiring a face image to be recognized; the face image to be recognized comprises at least one face to be recognized;
carrying out image graying processing on the basis of the face image to be recognized to obtain a face grayscale image to be recognized;
performing edge detection according to the face gray level image to be recognized to obtain an edge contour of the at least one face to be recognized in the face gray level image to be recognized;
correspondingly setting at least one facial expression recognition window based on the edge contour of the at least one face to be recognized;
and performing facial expression recognition on the face in the facial image to be recognized by utilizing the at least one facial expression recognition window.
2. The facial expression recognition method of claim 1, wherein the facial expression recognition of the face in the facial image to be recognized by using the at least one facial expression recognition window comprises:
acquiring a facial expression image;
extracting facial expression features based on the facial expression image to obtain facial expression features;
sending the facial expression features and the facial expression images into an identification model for iterative training to obtain a facial expression identification model;
intercepting a face to be recognized based on the at least one facial expression recognition window to obtain facial expression information to be recognized;
and sending the facial expression information to be recognized into a facial expression recognition model for facial expression recognition, and outputting facial expression recognition information.
3. The method for recognizing facial expressions according to claim 1, wherein the performing image graying processing based on the facial image to be recognized to obtain a gray image of the facial image to be recognized comprises:
extracting three primary color information of the face image to be recognized;
carrying out gray level calculation based on the three primary color information of the face image to be recognized to obtain the gray level information of the face to be recognized;
and setting the face image to be recognized by using the face gray level information to be recognized to obtain a face gray level image to be recognized.
4. The method according to claim 1, before the performing edge detection according to the gray-scale image of the face to be recognized to obtain an edge contour of the at least one face to be recognized in the gray-scale image of the face to be recognized, further comprising: and sending the face gray level image to be recognized into a Gaussian filter for image filtering to obtain a filtering image to be recognized.
5. The method according to claim 1 or 4, wherein the performing edge detection according to the gray level image of the face to be recognized to obtain an edge contour of the at least one face to be recognized in the gray level image of the face to be recognized comprises:
acquiring human face features in a human face gray level image to be recognized;
judging whether the facial features meet the edge contour generation conditions of the face to be recognized or not based on the facial features;
if the facial features of the human face meet the edge contour generation condition of the human face to be recognized, obtaining the edge contour of the human face to be recognized;
and if the face facial features do not meet the edge contour generation conditions of the face to be recognized, re-acquiring the face facial features in the gray level image of the face to be recognized.
6. A facial expression recognition device based on complex scene, characterized by comprising:
the first acquisition module is used for acquiring a face image to be recognized; the face image to be recognized comprises at least one face to be recognized;
the processing module is used for carrying out image graying processing on the basis of the face image to be recognized to obtain a face grayscale image to be recognized;
the detection module is used for carrying out edge detection according to the gray level image of the face to be recognized to obtain the edge contour of the at least one face to be recognized in the gray level image of the face to be recognized;
the setting module is used for correspondingly setting at least one facial expression recognition window based on the edge outline of the at least one face to be recognized;
and the first recognition module is used for carrying out facial expression recognition on the face in the facial image to be recognized by utilizing the at least one facial expression recognition window.
7. The apparatus according to claim 6, comprising:
the second acquisition module is used for acquiring the facial expression image;
the first extraction module is used for extracting facial expression characteristics based on the facial expression image to obtain facial expression characteristics;
the training module is used for sending the facial expression characteristics and the facial expression image into a recognition model for iterative training to obtain a facial expression recognition model;
the intercepting module is used for intercepting the face to be identified based on the at least one facial expression identifying window to obtain the facial expression information to be identified;
and the second recognition module is used for sending the facial expression information to be recognized into a facial expression recognition model for facial expression recognition and outputting the facial expression recognition information.
8. The apparatus according to claim 6, comprising:
the second extraction module is used for extracting the three primary color information of the face image to be recognized;
the calculation module is used for carrying out gray level calculation on the basis of the three primary color information of the face image to be recognized to obtain the gray level information of the face to be recognized;
and the setting module is used for setting the face image to be recognized by utilizing the face gray level information to be recognized to obtain a face gray level image to be recognized.
9. An electronic device, comprising:
a memory and a processor, wherein the memory and the processor are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the method for recognizing facial expressions based on complex scenes according to any one of claims 1 to 5.
10. A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions for causing the computer to execute the method for recognizing facial expressions based on complex scenes according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110162293.3A CN112801002A (en) | 2021-02-05 | 2021-02-05 | Facial expression recognition method and device based on complex scene and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110162293.3A CN112801002A (en) | 2021-02-05 | 2021-02-05 | Facial expression recognition method and device based on complex scene and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112801002A true CN112801002A (en) | 2021-05-14 |
Family
ID=75814438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110162293.3A Pending CN112801002A (en) | 2021-02-05 | 2021-02-05 | Facial expression recognition method and device based on complex scene and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112801002A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449194A (en) * | 2021-07-14 | 2021-09-28 | 青岛科技大学 | Expression recognition music recommendation system based on convolutional neural network |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102842033A (en) * | 2012-08-17 | 2012-12-26 | 苏州两江科技有限公司 | Human expression emotion semantic recognizing method based on face recognition |
| CN109711378A (en) * | 2019-01-02 | 2019-05-03 | 河北工业大学 | Human face expression automatic identifying method |
| CN110705430A (en) * | 2019-09-26 | 2020-01-17 | 江苏科技大学 | Multi-person facial expression recognition method and system based on deep learning |
-
2021
- 2021-02-05 CN CN202110162293.3A patent/CN112801002A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102842033A (en) * | 2012-08-17 | 2012-12-26 | 苏州两江科技有限公司 | Human expression emotion semantic recognizing method based on face recognition |
| CN109711378A (en) * | 2019-01-02 | 2019-05-03 | 河北工业大学 | Human face expression automatic identifying method |
| CN110705430A (en) * | 2019-09-26 | 2020-01-17 | 江苏科技大学 | Multi-person facial expression recognition method and system based on deep learning |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449194A (en) * | 2021-07-14 | 2021-09-28 | 青岛科技大学 | Expression recognition music recommendation system based on convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110276767B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| KR102641115B1 (en) | A method and apparatus of image processing for object detection | |
| US10740912B2 (en) | Detection of humans in images using depth information | |
| CN109389135B (en) | Image screening method and device | |
| Jung | Efficient background subtraction and shadow removal for monochromatic video sequences | |
| KR20180065889A (en) | Method and apparatus for detecting target | |
| WO2022206319A1 (en) | Image processing method and apparatus, and device, storage medium and computer program product | |
| CN109871845B (en) | Certificate image extraction method and terminal equipment | |
| CN109522775B (en) | Face attribute detection method and device and electronic equipment | |
| WO2020244071A1 (en) | Neural network-based gesture recognition method and apparatus, storage medium, and device | |
| CN111597933B (en) | Face recognition method and device | |
| EP4300417A1 (en) | Method and apparatus for evaluating image authenticity, computer device, and storage medium | |
| CN111444555B (en) | Temperature measurement information display method and device and terminal equipment | |
| CN107944395B (en) | Method and system for verifying and authenticating integration based on neural network | |
| CN110245573B (en) | Sign-in method and device based on face recognition and terminal equipment | |
| EP3282387A1 (en) | Fire detection method, fire detection apparatus and electronic equipment | |
| CN110276831B (en) | Method and device for constructing three-dimensional model, equipment and computer-readable storage medium | |
| CN105678245A (en) | Target position identification method based on Haar features | |
| WO2017167381A1 (en) | Method for foreground and background determination in an image | |
| CN110059607B (en) | Living body multiplex detection method, living body multiplex detection device, computer equipment and storage medium | |
| CN112633221A (en) | Face direction detection method and related device | |
| CN106920266B (en) | The Background Generation Method and device of identifying code | |
| Devadethan et al. | Face detection and facial feature extraction based on a fusion of knowledge based method and morphological image processing | |
| CN112801002A (en) | Facial expression recognition method and device based on complex scene and electronic equipment | |
| CN113822927A (en) | Face detection method, device, medium and equipment suitable for weak-quality images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |