CN111126113A - Method and device for processing face image - Google Patents
Method and device for processing face image Download PDFInfo
- Publication number
- CN111126113A CN111126113A CN201811294871.3A CN201811294871A CN111126113A CN 111126113 A CN111126113 A CN 111126113A CN 201811294871 A CN201811294871 A CN 201811294871A CN 111126113 A CN111126113 A CN 111126113A
- Authority
- CN
- China
- Prior art keywords
- face
- frame
- target picture
- image
- face image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a method and a device for processing a face image, wherein for a target image obtained from a video ordered frame, if the target image is a detection frame, on one hand, first image information of the face image with a face number in a list in the target image is added into the list, on the other hand, the face number is added to the face image which appears in the target image and does not have the face number in the list, and the image with the face image before the target image is traced back. The method for backtracking the face image correlates the same person of each frame, prevents missing frames and missing detection and reduces the amount of calculation. The method can process each picture in the video frame or the picture set without repeatedly starting detection, and improves the processing efficiency. Further, storing the image information of each face image in a list as original data information facilitates outputting the information of each face image in different forms according to the list.
Description
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to a method and a device for processing a face image.
Background
The face recognition technology is developed today, the accuracy rate is greatly improved and reaches the commercial level, the face snapshot is an important link, and the existing method for processing videos or pictures by the snapshot equipment is to extract video frames of each frame in a video source and then carry out face detection frame by frame to realize the face recognition. The method mainly uses a deep convolutional neural network method for the 'face detection' part used at present, and has the problems of large calculation amount, long consumed time, high precision, complex calculation and unsuitability for running on equipment with poor performance. In order to reduce the complexity of the operation, a method for detecting every other N frames exists in the prior art, however, the method often has a missing detection phenomenon, and the algorithm overhead is still large, and when a person appears in a non-detection frame, each frame of data cannot be acquired.
In the process of implementing the embodiment of the invention, the inventor finds that the existing method for processing the video needs to repeatedly start detection, and the frames have no correlation processing, so that the frames cannot be confirmed to be the same person, detection is easy to miss, and the calculation amount is large.
Disclosure of Invention
The invention aims to solve the technical problems that the existing method for processing the video needs to repeatedly start detection, the frames are not associated, the same person cannot be confirmed between the frames, the detection is easy to miss, and the calculation amount is large.
In view of the above technical problems, an embodiment of the present invention provides a method for processing a face image, including:
acquiring a target picture from a video ordered frame with a set frame number, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame number of the target picture and a preset frame interval;
if the number of the face images detected from the target picture is not zero and the detected face images comprise new face images without corresponding face numbers in the list, creating new face numbers corresponding to each new face image;
for each new face number, adding first image information of a new face image in the target picture corresponding to the new face number into the list, acquiring second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than those of the target picture, and adding the second image information into the list;
if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
The invention provides a device for processing a face image, which comprises:
the acquisition module is used for acquiring a target picture from a video ordered frame with a set frame number, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame number of the target picture and a preset frame interval;
a number creation module, configured to create a new face number corresponding to each new face image if the number of face images detected from the target picture is not zero and the detected face images include new face images without corresponding face numbers in the list;
the image tracking module is used for numbering each new face, adding first image information of the new face image corresponding to the new face number in the target picture into the list, acquiring second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than the frame number corresponding to the target picture, and adding the second image information into the list;
if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
The embodiment provides an electronic device, including:
at least one processor, at least one memory, a communication interface, and a bus; wherein the content of the first and second substances,
the processor, the memory and the communication interface complete mutual communication through the bus;
the communication interface is used for information transmission between the electronic equipment and communication equipment of other electronic equipment;
the memory stores program instructions executable by the processor, which when called by the processor are capable of performing the methods described above.
The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the method described above.
The embodiment of the invention provides a method and a device for processing a face image, wherein for a target picture obtained from a video ordered frame, if the target picture is a detection frame, on one hand, image information of a face image with a face number in a list in the target picture is added into the list, on the other hand, the face number is added to the face image which appears in the target picture and does not have the face number in the list, and the picture with the face image before the target picture is traced back. The method for backtracking the face image correlates the same person of each frame, prevents missing frames and missing detection and reduces the amount of calculation. The method can process each picture in the video frame or the picture set without repeatedly starting detection, and improves the processing efficiency. Further, storing the image information of each face image in a list as original data information facilitates outputting the information of each face image in different forms according to the list.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a video frame composition in a method for processing a face image for comparison according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of processing images in a video frame in a method of providing facial image processing as a contrast according to another embodiment of the present invention;
FIG. 3 is a flow chart of a method for processing a face image according to another embodiment of the present invention;
fig. 4 is a schematic diagram of processing an image in a video frame in a method for processing a face image according to another embodiment of the present invention;
FIG. 5 is a flowchart illustrating a specific method for processing a face image according to another embodiment of the present invention;
fig. 6 is a block diagram of a face image processing apparatus according to another embodiment of the present invention;
fig. 7 is a block diagram of an electronic device according to another embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Before describing the method for processing a face image provided by this embodiment, a method for processing a face image as a comparison is described, fig. 1 is a schematic view of a video frame composition in the method for processing a face image as a comparison shown in this embodiment, fig. 2 is a schematic view of processing an image in a video frame in the method for processing a face image as a comparison shown in this embodiment, referring to fig. 1 and fig. 2, a detection frame is usually several frames of pictures extracted from a video, in the method for comparison, face recognition is performed on each frame of pictures in a video, the amount of computation is extremely large, and the detection frame cannot be used in an environment with low power consumption and low computation capability, or only a detection frame (a frame marked as D) in a video frame is detected, and a face appearing in a non-detection frame is easily ignored, so that detection omission is caused. In order to reduce the amount of operation in the process of performing face detection on a video, so that the method for processing a face image can also operate in an environment with low power consumption and low computing power, and meanwhile, avoid missing detection of the face image, fig. 3 is a schematic flow diagram of the method for processing the face image provided by this embodiment, and referring to fig. 3, the method includes:
301: acquiring a target picture from a video ordered frame with a set frame number, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame number of the target picture and a preset frame interval;
302: if the number of the face images detected from the target picture is not zero and the detected face images comprise new face images without corresponding face numbers in the list, creating new face numbers corresponding to each new face image;
303: for each new face number, adding first image information of a new face image in the target picture corresponding to the new face number into the list, acquiring second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than those of the target picture, and adding the second image information into the list;
if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
The present embodiment may be executed by a computer or a server, which is not limited in this embodiment specifically, for example, a camera inputs a captured video ordered frame into the computer, and the computer processes an input picture according to the method for processing a face image provided in the present embodiment, and outputs a video ordered frame that is marked by different marking methods, so that a worker can recognize a face individual and research human behavior according to the marks in the video ordered frame.
The frame number is usually set for each picture according to the playing sequence of the pictures in the video ordered frames, starting from 0. The frame interval is a value set artificially, for example, the frame interval N is 8. The face images of the same person correspond to the same face number. The list stores the image information of each face image in each picture, the faces in each frame of picture of the video can be distinguished according to the list, and the action track of the person corresponding to each face image can be tracked by combining each frame of picture of the video. A video ordered frame comprises a video or a set of pictures. Wherein the picture set needs to be a set of pictures obtained consecutively in the time dimension.
It should be noted that, when identifying whether a new face image corresponding to a new face number exists in a picture before a target picture, the identification may be implemented by using a tracking algorithm, for example, using a KCF, DCF, PLD, mediaflow, or GOTURN algorithm to identify whether a certain face image in different frame pictures is a face image corresponding to the same person.
For a face image corresponding to a new face number, the first image information is image information (e.g., position information) of the face image in a target picture, and the second image information is image information of the face image in a picture before the target picture. And after the detection frame detects a new face image, adding the first image information and the second image information of the new face image into the list. The second image information is the image information of each picture of the newly added face image before the target picture, which is obtained by tracing back from the target picture to the picture in which the newly added face image appears for the first time. After the first image information and the second image information are added into the list, all image information of the newly added face image from the first appearance of the newly added face image in the video to the appearance of the target image is included in the list, and therefore the images with the newly added face image are associated. The backtracking method enables the image information of the newly added face image in each frame of picture to be supplemented into the list under the condition that each frame of picture does not need to be detected, and enables the image information of the newly added face image in each frame of picture to be read through the list. Further, the face images appearing in each picture of the video ordered frame can also be distinguished through the list.
In this embodiment, for a target picture obtained from a video sequential frame, if the target picture is a detection frame, on one hand, first image information of a face image in the target picture, where a face number already exists in a list, is added to the list, and on the other hand, a face number is added to a face image that appears in the target picture and does not exist in the list, and a picture of the face image before the target picture is traced back. The method for backtracking the face image correlates the same person of each frame, prevents missing frames and missing detection and reduces the amount of calculation. The method can process each picture in the video frame or the picture set without repeatedly starting detection, and improves the processing efficiency. Further, storing the image information of each face image in a list as original data information facilitates outputting the information of each face image in different forms according to the list.
Further, on the basis of the above embodiment, the method further includes:
and outputting the video ordered frames of the face images corresponding to different face numbers in each frame of picture according to the face numbers and the image information corresponding to the face images in each frame of picture recorded in the list.
For example, all face images corresponding to the same person in the video are used as boundary lines of the face images by using boxes with the same color according to the list.
The embodiment provides a method for processing a face image, which distinguishes people in a video ordered frame in an intuitive mode according to a list, and is convenient to quickly distinguish different people in the video ordered frame.
Further, on the basis of the foregoing embodiments, if the number of face images detected from the target picture is not zero, and the detected face images include new face images without corresponding face numbers in the list, creating a new face number corresponding to each new face image includes:
if the number of the face images detected from the target picture is not zero, judging whether the face images have corresponding face numbers in the list or not for each face image;
if the face image has a corresponding face number in the list, storing the position information of the face image in the target picture, the face number corresponding to the face image, the frame number corresponding to the target picture and the tracker corresponding to the face image in the list in an associated manner;
if the face image does not have a corresponding face number in the list, taking the face image as a new face image, and creating a new face number corresponding to the new face image;
and generating a tracker corresponding to the face image according to the characteristics of the face image, and identifying the face image.
It should be noted that the position information recorded in the list may be composed of coordinates of an upper left point of the face image and a width and a height of the face image, for example, table 1 is the face image information list provided in this embodiment, in table 1, a number is a frame number, a letter is a face number, the face images of the same person correspond to the same face number, x and y are coordinates of the upper left point of the face image, respectively, and h and w are the height and the width of the face image, respectively. For example, in table 1, image information of the face image is represented in formats i to J (Xji, Yji, Wji, Hji), where i represents a frame number of the picture, i is a number, J and J both represent a face number, J and J are letters, Xji represents an abscissa of an upper left point of the face image with the face number J (J and J are the same) in the picture with the frame number i, Yji represents an ordinate of an upper left point of the face image with the face number J in the picture with the frame number i, Wji represents a width of an image area of the face image with the face number J in the picture with the frame number i, and Hji represents a height of an image area of the face image with the face number J in the picture with the frame number i.
The face tracker is an algorithm for recognizing the face image from the picture by using the feature information in the face image, for example, the face tracker recognizes the face image in the picture by using an area matching method or a feature matching method, which is not limited in this embodiment.
Fig. 4 is a schematic diagram of processing an image in a video frame in the method for processing a face image provided in this embodiment, fig. 5 is a schematic diagram of a flow of a specific method for processing a face image, a video frame is denoted by D in fig. 4, a tracking frame is denoted by T, a frame interval N is 8, and pictures with frame numbers of 0, 8, and 16 in the video frame shown in fig. 4 are detection frames. Referring to fig. 5, the method comprises the following steps:
step 1: making the frame number f equal to 0 and the maximum face number max equal to 0;
step 2: taking an f-th video frame;
and step 3: judging whether the frame is a D-type frame (detection frame), executing the D-type frame 4, otherwise executing the D-type frame 13, wherein if the formula f% N is 0, the frame is the D-type frame, otherwise, the frame is the T-type frame, N is the frame interval, and f is the frame number of the video;
and 4, step 4: detecting the f video frame full image, wherein the number of detected face images is x;
and 5: judging whether the number x of the detected face images is 0, if so, executing 19, otherwise, executing 6;
step 6: judging whether the face image is matched with a face image with a face number in the list, if a matching item exists and the matched face number is i, executing 7, and if not, executing 8;
and 7: according to the existing face number i in the list, recording the position information of the face image in the f video frame into the list, and executing 12;
and 8: and detecting a new face image, newly building a face number max which is max +1, and recording the new face number into a list, and enabling the backtracking offset frame n to be 0.
For example, when the 8 th frame of picture is detected according to the above method, new face images are detected, and these face images do not have corresponding face numbers in the first 7 frames of the list, so at the position corresponding to the 8 th frame of picture in the list, face numbers B and C are respectively added to the new face images, and face region subgraphs of B and C (i.e., pictures of the region where the face image is located) are utilized to generate face tracking templates of B and C, which are respectively stored in information of B and C in the list. And storing the face tracking template in the list to facilitate the tracking of the face image by using the face tracking template. For example, a face image with a face number B may be tracked using a face tracking template with a face number B.
It should be noted that, when determining whether the face image has a corresponding face number in the list, the face image is matched with the face image corresponding to each face number in the list, and if matching is possible, the face number of the face image is the face number of the matched face image. Such a method of face image matching (for example, face image matching in step 6) may be implemented by an area matching method or a feature matching method.
The area matching method comprises the steps that a tracker generated by the previous frame is used for tracking the picture of the current frame, and if the area of a tracking frame selection area and the area of a certain face image detected by the current frame, which accounts for the area of a detection frame selection area, exceeds a threshold value, the two face images are successfully matched and correspond to the same face number. Specifically, the area matching method includes: and (3) detecting and tracking the current frame at the same time (by using a tracker generated by the previous detection frame), and if the overlapped area of the tracking frame selection area and the frame selection area of the current detection frame is compared, if the overlapped area is larger than a threshold value, such as 80 percent, matching is carried out, and the length and width difference of the frame selection area is smaller than 80%, matching is carried out.
The feature matching method includes the steps of directly extracting a framing region subgraph according to image information by using an image feature matching algorithm such as surf, sift or hog, matching the framing region subgraph with a detection framing subgraph, setting a feature matching threshold, matching if feature similarity is larger than the threshold, simply adapting an area matching algorithm to a scene with sparse human faces, directly matching a subgraph small region by using an opposite feature matching method, and having small effect influenced by other factors.
The embodiment provides a method for processing a face image, which adds a face number to a new face image when the new face image appears in a detection frame, so as to add image information of the new face image in a list.
Further, on the basis of the foregoing embodiments, for each new face number, adding, to the list, first image information of the new face image in the target picture corresponding to the new face number, and obtaining, from a picture whose frame number is smaller than a frame number corresponding to the target picture, second image information of the new face image corresponding to the new face number, and adding the second image information to the list, includes:
for each new face number, acquiring position information of a new face image corresponding to the new face number in the target picture, taking the position information as first image information, and storing the first image information, the new face number and a frame number corresponding to a tracker generated by the new face image in the list in an associated manner;
and acquiring position information of the face image corresponding to the new face number in each picture, wherein the frame number is smaller than the frame number corresponding to the target picture and the picture comprises the face image corresponding to the new face number, and storing the second image information, the new face number, the tracker corresponding to the new face image and the frame number corresponding to the target picture in the list in an associated manner as second image information.
Further, on the basis of the foregoing embodiments, the obtaining, from each picture whose frame number is smaller than the frame number corresponding to the target picture and which includes the face image corresponding to the new face number, position information of the face image corresponding to the new face number in the picture as second image information, and storing the second image information, the new face number, the tracker corresponding to the new face image, and the frame number corresponding to the target picture in the list in an associated manner includes:
setting the initial backtracking offset frame number to be 0, and circularly executing a first face tracking operation until no face image corresponding to the newly added face number exists in a tracking frame;
wherein the first face tracking operation comprises:
acquiring a frame number corresponding to the picture of the face image with the newly added face number, which is detected to exist at the latest time, as a current frame number, and calculating the sum of the current backtracking offset frame number minus 1 and the current frame number to obtain a tracking frame number;
and acquiring a picture corresponding to the tracking frame number as a tracking frame, judging whether a face image corresponding to the newly added face number exists in the tracking frame, if so, taking the position information of the face image corresponding to the newly added face number in the tracking frame as second image information, and storing the second image information, the newly added face number, a tracker corresponding to the newly added face image and the frame number corresponding to the target picture in the list in an associated manner.
Further, still include:
and storing the pictures with the number equal to the frame spacing before the frame number corresponding to the target picture so as to backtrack the newly added face image in the stored pictures according to the list.
It should be noted that the tracking frame is actually a frame picture whose remainder of the quotient between the frame number and the inter-frame distance is not zero, but because the method provided by this embodiment adds the new face image in each detection frame to the list, when a new face image is traced back, it is certain to search which pictures have the new face image in the tracking frame between the detection frame and the previous detection frame.
As shown in fig. 5, the method for processing a face image further includes:
and step 9: making the backtracking offset frame number n equal to n-1, and taking the f + n video frame and carrying out face tracking;
step 10: tracking the face of the previous n frames, if finding that the max number face appears, executing 11, otherwise, if the frame is traced back and no max number face appears, executing 12;
step 11: recording the list according to the face number max, and repeatedly executing from 9 until the condition 10 exits to 12;
step 12: let x be x-1 and continue execution from 5 for the next face target detected.
For example, when the 8 th frame picture is detected according to the above method, since tracking finds that there is no matching item with the face image in the list, and the 8 th frame picture is a new face image, face numbers B and C are respectively added, the face image with the face number B is traced back, step 9-11 is performed, and it is found that a face image with the face number B exists in all the 4 th, 5 th, 6 th and 7 th frame pictures, the second image information, the added face number and the tracking frame number are stored in the list in an associated manner, that is, 7-B (Xb7, Yb7, Wb7, Hb7), 6-B (Xb6, Yb6, Wb6, Hb6), 5-B (Xb5, Yb5, Wb5, Hb5) and 4-B (Xb4, Yb4, Wb4, Hb4) are added to the list. Backtracking the face image with the face number C (backward start tracking), executing the step 9-11, and adding 7-C (Xc7, Yc7, Wc7 and Hc7) into the list if the face image with the face number C is found to exist in the 7 th frame picture. Among them, the panorama picture of the current frame and trackers (including characteristic attributes of B and C) about B and C generated from the detection region subgraph also need to be stored in the list.
The embodiment provides a method for processing a face image, which perfects the image information of the face image in a list through backtracking and avoids omission.
Further, on the basis of the foregoing embodiments, the acquiring a target picture from a video ordered frame with a frame number set, and if the target picture is determined to be a detection frame according to the frame number of the target picture and a preset frame interval, acquiring the number of face images detected from the target picture, further includes:
if the target picture is judged to be a tracking frame according to the frame number of the target picture and the preset frame interval, acquiring the maximum face number from the list as an initial tracking face number, and circularly executing a second face tracking operation until a face image corresponding to the current tracking face number does not exist in the target picture;
wherein the second face tracking operation comprises:
judging whether a face image corresponding to the current tracking face number exists in the target picture, if so, storing the position information of the face image corresponding to the current tracking face number in the target picture, the current tracking face number and the frame number corresponding to the target picture in the list in an associated manner, otherwise, subtracting 1 from the tracking face number to obtain the current tracking face number;
and if the remainder of the frame number of the target picture and the frame interval quotient is not zero, the target picture is a tracking frame.
Further, still include: and calculating the difference value between the frame number corresponding to the target picture and the frame interval, and eliminating the information of the picture with the frame number smaller than the difference value, which is stored in the list.
As shown in fig. 5, the method for processing a face image further includes:
step 13: tracking frames (tracking frames) for T-type faces, wherein m is max which is the maximum face number;
step 14: reading a face tracking template of number m in the list;
step 15: carrying out face tracking on the current image of the image;
step 16: if the area matched with the face number m is found, executing 17, otherwise executing 19;
and step 17: recording the number m of the existing face into a list;
step 18: making m equal to m-1, and judging whether m numbers exist in the list, if so, continuing to execute from 14, otherwise, executing 19;
step 19: increasing the frame number, and preparing to process the next frame, wherein f is f + 1;
step 20: and deleting all frame record buffers before the detection frame interval N in the list, and continuing to execute from 2.
For example, when the 4 th frame picture is detected according to the above method, only the image information of the face image with the face number a is stored in the list content of the 4 th frame picture in the list. And adding the image information of the face image with the face number B in the picture of the 4 th frame into the list content corresponding to the detection frame with the frame number 8 in the list.
As can also be seen from table 1, the image information of the 1 st frame picture is removed from the image information of the 9 th frame picture stored in the list, and the image information of the 1 st and 2 nd frames pictures is removed from the image information of the 10 th frame picture stored in the list.
As shown in fig. 4, when an occlusion occurs in the 10 th frame, the 14 th frame, and the 15 th frame, it can be seen from the list in table 1 that the image information of the face image with the face number C is not included in the 10 th frame, the 14 th frame, and the 15 th frame due to the occlusion by the occlusion. Therefore, according to the list of table 1, historical tracking can be performed after the target is occluded.
The embodiment provides a method for processing a face image, a tracking frame adopts a simpler face image detection method, only image information of a face image with a face number is added, a list generation flow is simplified, and the detection efficiency is improved. And deleting the image information of the picture before the detection frame is separated by N frames, releasing the buffer space and ensuring the operation performance of the equipment. In addition, can realize the tracking to sheltering from the thing, be difficult for following the lost.
Further, on the basis of the above embodiments, the method further includes:
if the number of the face images detected from the target picture is zero (see step 5 in fig. 5), or if there is no face image corresponding to the last new face number that is not traversed in the tracking frame (see step 12 in fig. 5), or if there is no face image corresponding to the current tracking face number in the target picture (see step 16 in fig. 5), obtaining a next frame picture of the target picture as a new target picture.
The embodiment provides a method for processing a face image, which circularly processes each frame of picture in a video ordered frame until image information corresponding to the face image in each frame of picture is obtained.
The matching of the face images in this embodiment is performed by comparing templates in a tracking algorithm. The implementation of the method can include a list which can store a group of face frame coordinate mark sequences and a personnel number, a first-in first-out queue which can buffer N frames (N is a detection frame interval), a face detector, a group of image trackers, and a picture characteristic comparator (area or characteristics, the characteristics comprise one or a combination of surf, sift, hog or others). Which can effectively reduce the complexity of the operation. All frames of the same person can be confirmed in the video frame sequence even under the condition of temporary shielding, and the capture missing condition caused by frame-separated detection can be effectively reduced. The historical frames are traced back through a tracking algorithm, the list is looked up to continue the personnel appearing in the history, the frame loss condition is prevented, the detection reliability and the efficiency are guaranteed by combining the historical frames and the table, the same person of each frame is associated, the frame loss is prevented, the efficiency is improved, and the same person of each frame of the time dimension can be connected under the premise of certain shielding to facilitate subsequent identification processing.
TABLE 1 face image information List
Fig. 6 is a block diagram of the structure of the apparatus for processing a face image according to the present embodiment, and referring to fig. 6, the apparatus for processing a face image includes an obtaining module 601, a number creating module 602, and an image tracking module 603, wherein,
an obtaining module 601, configured to obtain a target picture from a video ordered frame with a frame number set, and if the target picture is determined to be a detection frame according to the frame number of the target picture and a preset frame interval, obtain the number of face images detected from the target picture;
a number creating module 602, configured to create a new face number corresponding to each new face image if the number of face images detected from the target picture is not zero and the detected face images include new face images without corresponding face numbers in the list;
the image tracking module 603 is configured to add, to each newly added face number, first image information of the newly added face image in the target picture corresponding to the newly added face number to the list, acquire second image information of the newly added face image corresponding to the newly added face number from a picture whose frame number is smaller than a frame number corresponding to the target picture, and add the second image information to the list;
if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
The apparatus for processing a face image provided in this embodiment is suitable for the method for processing a face image provided in the foregoing embodiment, and details are not repeated here.
In this embodiment, for a target picture acquired from a video ordered frame, if the target picture is a detection frame, on one hand, image information of a face image in the target picture, where a face number already exists in a list, is added to the list, on the other hand, a face number is added to a face image that appears in the target picture and does not exist in the list, and a picture of the face image before the target picture is traced back. The method for backtracking the face image correlates the same person of each frame, prevents missing frames and missing detection and reduces the amount of calculation. The method can process each picture in the video frame or the picture set without repeatedly starting detection, and improves the processing efficiency. Further, storing the image information of each face image in a list as original data information facilitates outputting the information of each face image in different forms according to the list.
Fig. 7 is a block diagram showing the structure of the electronic apparatus provided in the present embodiment.
Referring to fig. 7, the electronic device includes: a processor (processor)701, a memory (memory)702, a communication Interface (Communications Interface)703, and a bus 704;
wherein the content of the first and second substances,
the processor 701, the memory 702 and the communication interface 703 complete mutual communication through the bus 704;
the communication interface 703 is used for information transmission between the electronic device and communication devices of other electronic devices;
the processor 701 is configured to call the program instructions in the memory 702 to execute the methods provided by the above-mentioned method embodiments, for example, including: acquiring a target picture from a video ordered frame with a set frame number, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame number of the target picture and a preset frame interval; if the number of the face images detected from the target picture is not zero and the detected face images comprise new face images without corresponding face numbers in the list, creating new face numbers corresponding to each new face image; for each new face number, adding first image information of a new face image in the target picture corresponding to the new face number into the list, acquiring second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than those of the target picture, and adding the second image information into the list; if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the methods provided by the above method embodiments, for example, including: acquiring a target picture from a video ordered frame with a set frame number, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame number of the target picture and a preset frame interval; if the number of the face images detected from the target picture is not zero and the detected face images comprise new face images without corresponding face numbers in the list, creating new face numbers corresponding to each new face image; for each new face number, adding first image information of a new face image in the target picture corresponding to the new face number into the list, acquiring second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than those of the target picture, and adding the second image information into the list; if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
The present embodiments disclose a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the methods provided by the above-described method embodiments, for example, comprising: acquiring a target picture from a video ordered frame with a set frame number, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame number of the target picture and a preset frame interval; if the number of the face images detected from the target picture is not zero and the detected face images comprise new face images without corresponding face numbers in the list, creating new face numbers corresponding to each new face image; for each new face number, adding first image information of a new face image in the target picture corresponding to the new face number into the list, acquiring second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than those of the target picture, and adding the second image information into the list; if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
The above-described embodiments of the electronic device and the like are merely illustrative, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may also be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the embodiments of the present invention, and are not limited thereto; although embodiments of the present invention have been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for processing a face image, comprising:
acquiring a target picture from a video ordered frame with a set frame number, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame number of the target picture and a preset frame interval;
if the number of the face images detected from the target picture is not zero and the detected face images comprise new face images without corresponding face numbers in the list, creating new face numbers corresponding to each new face image;
for each new face number, adding first image information of a new face image in the target picture corresponding to the new face number into the list, acquiring second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than those of the target picture, and adding the second image information into the list;
if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
2. The method of claim 1, further comprising:
and outputting the video ordered frames of the face images corresponding to different face numbers in each frame of picture according to the face numbers and the image information corresponding to the face images in each frame of picture recorded in the list.
3. The method according to claim 1, wherein if the number of face images detected from the target picture is not zero and the detected face images include new face images without corresponding face numbers in the list, creating new face numbers corresponding to each new face image comprises:
if the number of the face images detected from the target picture is not zero, judging whether the face images have corresponding face numbers in the list or not for each face image;
if the face image has a corresponding face number in the list, storing the position information of the face image in the target picture, the face number corresponding to the face image, the frame number corresponding to the target picture and the tracker corresponding to the face image in the list in an associated manner;
if the face image does not have a corresponding face number in the list, taking the face image as a new face image, and creating a new face number corresponding to the new face image;
and generating a tracker corresponding to the face image according to the characteristics of the face image, and identifying the face image.
4. The method according to claim 3, wherein for each new face number, adding first image information of the new face image corresponding to the new face number in the target picture to the list, and obtaining second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than that of the target picture, and adding the second image information to the list, comprises:
for each new face number, acquiring position information of a new face image corresponding to the new face number in the target picture, and storing the first image information, the new face number, a tracker corresponding to the new face image and a frame number corresponding to the target picture in the list in an associated manner;
and acquiring position information of the face image corresponding to the new face number in each picture, wherein the frame number is smaller than the frame number corresponding to the target picture and the picture comprises the face image corresponding to the new face number, and the position information is used as second image information, and the second image information, the new face number and the frame number corresponding to the tracker corresponding to the new face image are stored in the list in an associated manner.
5. The method according to claim 4, wherein the obtaining, from each picture whose frame number is smaller than the frame number corresponding to the target picture and which includes the face image corresponding to the new face number, position information of the face image corresponding to the new face number in the picture as second image information, and storing the second image information, the new face number, and the frame number corresponding to the tracker corresponding to the new face image in the list in an associated manner includes:
setting the initial backtracking offset frame number to be 0, and circularly executing a first face tracking operation until no face image corresponding to the newly added face number exists in a tracking frame;
wherein the first face tracking operation comprises:
acquiring a frame number corresponding to the picture of the face image with the newly added face number, which is detected to exist at the latest time, as a current frame number, and calculating the sum of the current backtracking offset frame number minus 1 and the current frame number to obtain a tracking frame number;
and acquiring a picture corresponding to the tracking frame number as a tracking frame, judging whether a face image corresponding to the newly added face number exists in the tracking frame, if so, taking the position information of the face image corresponding to the newly added face number in the tracking frame as second image information, and storing the second image information, the newly added face number, a tracker corresponding to the newly added face image and the frame number corresponding to the target picture in the list in an associated manner.
6. The method according to claim 5, wherein the obtaining of the target picture from the video ordered frame with the set frame number, and if the target picture is determined to be the detection frame according to the frame number of the target picture and the preset frame interval, obtaining the number of the face images detected from the target picture, further comprises:
if the target picture is judged to be a tracking frame according to the frame number of the target picture and the preset frame interval, acquiring the maximum face number from the list as an initial tracking face number, and circularly executing a second face tracking operation until a face image corresponding to the current tracking face number does not exist in the target picture;
wherein the second face tracking operation comprises:
judging whether a face image corresponding to the current tracking face number exists in the target picture, if so, storing the position information of the face image corresponding to the current tracking face number in the target picture, the current tracking face number and the frame number corresponding to the target picture in the list in an associated manner, otherwise, subtracting 1 from the tracking face number to obtain the current tracking face number;
and if the remainder of the frame number of the target picture and the frame interval quotient is not zero, the target picture is a tracking frame.
7. The method of claim 6, further comprising:
and if the number of the face images detected from the target picture is zero, or when the face image corresponding to the last new face number which is not traversed does not exist in the tracking frame, or when the face image corresponding to the current tracking face number does not exist in the target picture, acquiring a next frame picture of the target picture as a new target picture.
8. An apparatus for processing a face image, comprising:
the acquisition module is used for acquiring a target picture from a video ordered frame with a set frame number, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame number of the target picture and a preset frame interval;
a number creation module, configured to create a new face number corresponding to each new face image if the number of face images detected from the target picture is not zero and the detected face images include new face images without corresponding face numbers in the list;
the image tracking module is used for numbering each new face, adding first image information of the new face image corresponding to the new face number in the target picture into the list, acquiring second image information of the new face image corresponding to the new face number from pictures with frame numbers smaller than the frame number corresponding to the target picture, and adding the second image information into the list;
if the remainder of the quotient between the frame number and the inter-frame distance of the target picture is zero, the target picture is a detection frame; the list stores information related to the face image in each frame picture of the video ordered frame; in the process of playing the video ordered frame, the picture with smaller frame number has earlier playing time.
9. An electronic device, comprising:
at least one processor, at least one memory, a communication interface, and a bus; wherein the content of the first and second substances,
the processor, the memory and the communication interface complete mutual communication through the bus;
the communication interface is used for information transmission between the electronic equipment and communication equipment of other electronic equipment;
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1-7.
10. A non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811294871.3A CN111126113B (en) | 2018-11-01 | 2018-11-01 | Face image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811294871.3A CN111126113B (en) | 2018-11-01 | 2018-11-01 | Face image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111126113A true CN111126113A (en) | 2020-05-08 |
| CN111126113B CN111126113B (en) | 2023-10-10 |
Family
ID=70494783
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811294871.3A Active CN111126113B (en) | 2018-11-01 | 2018-11-01 | Face image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111126113B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705422A (en) * | 2021-08-25 | 2021-11-26 | 山东云缦智能科技有限公司 | Method for acquiring character video clips through human faces |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005339357A (en) * | 2004-05-28 | 2005-12-08 | Nippon Telegr & Teleph Corp <Ntt> | Device, method, and program for image processing |
| CN101620629A (en) * | 2009-06-09 | 2010-01-06 | 中兴通讯股份有限公司 | Method and device for extracting video index and video downloading system |
| CN102214479A (en) * | 2011-04-28 | 2011-10-12 | 深圳市同洲电子股份有限公司 | Hard disk data processing method and video stream pushing server |
| CN105354543A (en) * | 2015-10-29 | 2016-02-24 | 小米科技有限责任公司 | Video processing method and apparatus |
| CN105590097A (en) * | 2015-12-17 | 2016-05-18 | 重庆邮电大学 | Security system and method for recognizing face in real time with cooperation of double cameras on dark condition |
| CN108200432A (en) * | 2018-02-03 | 2018-06-22 | 王灏 | A kind of target following technology based on video compress domain |
| CN108230574A (en) * | 2017-12-20 | 2018-06-29 | 阿里巴巴集团控股有限公司 | Object storage terminal pickup method and device and electronic equipment |
| WO2018133666A1 (en) * | 2017-01-17 | 2018-07-26 | 腾讯科技（深圳）有限公司 | Method and apparatus for tracking video target |
-
2018
- 2018-11-01 CN CN201811294871.3A patent/CN111126113B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005339357A (en) * | 2004-05-28 | 2005-12-08 | Nippon Telegr & Teleph Corp <Ntt> | Device, method, and program for image processing |
| CN101620629A (en) * | 2009-06-09 | 2010-01-06 | 中兴通讯股份有限公司 | Method and device for extracting video index and video downloading system |
| CN102214479A (en) * | 2011-04-28 | 2011-10-12 | 深圳市同洲电子股份有限公司 | Hard disk data processing method and video stream pushing server |
| CN105354543A (en) * | 2015-10-29 | 2016-02-24 | 小米科技有限责任公司 | Video processing method and apparatus |
| CN105590097A (en) * | 2015-12-17 | 2016-05-18 | 重庆邮电大学 | Security system and method for recognizing face in real time with cooperation of double cameras on dark condition |
| WO2018133666A1 (en) * | 2017-01-17 | 2018-07-26 | 腾讯科技（深圳）有限公司 | Method and apparatus for tracking video target |
| CN108230574A (en) * | 2017-12-20 | 2018-06-29 | 阿里巴巴集团控股有限公司 | Object storage terminal pickup method and device and electronic equipment |
| CN108200432A (en) * | 2018-02-03 | 2018-06-22 | 王灏 | A kind of target following technology based on video compress domain |
Non-Patent Citations (2)
| Title |
|---|
| 杨帅等: "基于异步时域事件的高速运动目标的跟踪算法" * |
| 高灿: "跨摄像机的多目标跟踪***" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705422A (en) * | 2021-08-25 | 2021-11-26 | 山东云缦智能科技有限公司 | Method for acquiring character video clips through human faces |
| CN113705422B (en) * | 2021-08-25 | 2024-04-09 | 山东浪潮超高清视频产业有限公司 | Method for obtaining character video clips through human faces |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111126113B (en) | 2023-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110147717B (en) | Human body action recognition method and device | |
| US11275950B2 (en) | Method and apparatus for segmenting video | |
| Yan et al. | Learning the change for automatic image cropping | |
| JP5554984B2 (en) | Pattern recognition method and pattern recognition apparatus | |
| US11321966B2 (en) | Method and apparatus for human behavior recognition, and storage medium | |
| US9542735B2 (en) | Method and device to compose an image by eliminating one or more moving objects | |
| WO2018099268A1 (en) | Method and device for tracking target, and storage medium | |
| CN113691733B (en) | Video jitter detection method and device, electronic equipment and storage medium | |
| CN110610127B (en) | Face recognition method and device, storage medium and electronic equipment | |
| CN112381071A (en) | Behavior analysis method of target in video stream, terminal device and medium | |
| WO2023103329A1 (en) | Data labeling method, apparatus, and system, device, and storage medium | |
| CN114708287A (en) | Shot boundary detection method, device and storage medium | |
| CN112488072A (en) | Method, system and equipment for acquiring face sample set | |
| CN109684953B (en) | Method and device for pig tracking based on target detection and particle filter algorithm | |
| CN112329663B (en) | Micro-expression time detection method and device based on face image sequence | |
| CN111126113A (en) | Method and device for processing face image | |
| CN116761020A (en) | Video processing method, device, equipment and medium | |
| CN112989869B (en) | Optimization method, device, equipment and storage medium of face quality detection model | |
| CN113642472A (en) | Training method and action recognition method of discriminator model | |
| CN113657219A (en) | Video object detection tracking method and device and computing equipment | |
| CN112528079A (en) | System detection method, apparatus, electronic device, storage medium, and program product | |
| CN114268730A (en) | Image storage method and device, computer equipment and storage medium | |
| CN113221920B (en) | Image recognition method, apparatus, device, storage medium, and computer program product | |
| US11798284B2 (en) | System and method for rough handling recognition in a distribution center via a robust visual solution | |
| CN116433939B (en) | Sample image generation method, training method, recognition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |