CN108734049A - Image processing method and device and image processing system - Google Patents
Image processing method and device and image processing system Download PDFInfo
- Publication number
- CN108734049A CN108734049A CN201710238538.XA CN201710238538A CN108734049A CN 108734049 A CN108734049 A CN 108734049A CN 201710238538 A CN201710238538 A CN 201710238538A CN 108734049 A CN108734049 A CN 108734049A
- Authority
- CN
- China
- Prior art keywords
- image
- track
- face
- trajectory set
- key facial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
- G06V40/173—Classification, e.g. identification face re-identification, e.g. recognising unknown faces across different face tracks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of image processing methods and device and image processing system.Described image processing unit includes：Acquiring unit is configured as obtaining at least two tracks from face-image frame, wherein each track has at least one key facial image；Track combination unit is configured as in the case where the key facial image of the track corresponds to same specific people, by the track combination at a trajectory set；Face-image updating unit is configured as updating the key facial image of each trajectory set.According to the present invention, have the trajectory set of the key facial image for executing human body image retrieval (HIR) processing or face recognition (FID) processing will be more accurate, this precision for handling raising HIR processing or FID.
Description
Technical field
The present invention relates to image processing methods and device and image processing system.
Background technology
During video monitoring, in order to obtain the corresponding informance of specific people, such as in specific time period, the specific people exists
The behavior of specific position (such as, airport, supermarket etc.) tracks the spy usually using human body tracking technology from the video captured
The associated picture (such as, face-image) of fix the number of workers, to obtain several image sequences of the specific people as track
(track), thus operator can pass through the behavior of the trajectory analysis specific people.During human bioequivalence processing, in order to identify
Whether specific people belongs in particular system (such as, door control system, payment system etc.) one of the track registered, usually using face
Portion identify (Face Identification, FID) technology come identify the specific people face-image whether with registration track it
One registration facial images match.
In general, the precision of human body tracking processing (for example, face-image tracking is handled) depends on quantity and the people of track
The precision of body identifying processing and face recognition processing.About human body tracking processing, United States Patent (USP) US7 is disclosed in 636,453B2
A kind of exemplary human's identifying processing technology, the human body identifying processing technology identify the face in neighbouring two tracks in short period of time
Portion's image, and include：All facial characteristics in track are merged into a global feature, then measure input face image
Feature and global feature between similarity, to judge whether these facial image features related to the specific people.
But during video monitoring or human body tracking are handled, no matter record/register face-image, input face image
Or inquiry face-image, even the conventional face in these face-images, also often with various postures, such as Fig. 1
In illustrated by face-image, wherein illustrated by face image (for example, face-image F13 shown in Fig. 1) be facial figure
As in a kind of posture type, and illustrated by side face image (for example, face-image F11, F12, F14 shown in Fig. 1 and
F15) it is other posture types.In an example, specific people walks out video monitoring system (such as, camera) with a kind of posture
Observation scope, and with another posture return monitoring system observation scope.If both postures of the specific people are not
Together, then it is difficult to they being identified as same specific people, therefore track will interrupt.If will also be led with various facial expressions
Cause more complicated human body tracking disposition.Therefore, it is necessary to improve the precision of identifying processing and provide more representative face
Portion's image, to obtain better human-computer interaction user experience.
Invention content
Therefore, in view of the record in background technology above, the disclosure aims to solve the problem that the above problem.
According to an aspect of the present invention, a kind of image processing apparatus is provided, described image processing unit includes：It obtains single
Member is configured as obtaining at least two tracks from face-image frame, wherein each track has at least one key facial figure
As (key face image)；Track combination unit is configured as corresponding to same spy in the key facial image of the track
In the case of fix the number of workers, by the track combination at a trajectory set；Face-image updating unit is configured as updating each rail
The key facial image of mark group.
The track of same specific people can be attached to obtain trajectory set by the present invention；Crucial face in the trajectory set
Portion can also shift and be updated from other tracks, so as to provide higher retrieval precision and better human-computer interaction
User experience.
Using the present invention, in the case where face-image is with different postures, come using key facial image and trajectory set
It executes human body tracking processing or human bioequivalence processing will be more accurate, to which human body tracking processing or human bioequivalence processing will be improved
Precision.
According to the description of exemplary embodiment, other features and advantages of the present invention will become clear with reference to the accompanying drawings
Chu.
Description of the drawings
Including in the description and the attached drawing that forms part of this specification illustrates the embodiment of the present invention, and with text
Word description principle for explaining the present invention together.
Fig. 1 schematically shows the example facial in the face-image with different postures.
Fig. 2 is the block diagram for schematically showing the hardware configuration that technology according to the ... of the embodiment of the present invention may be implemented.
Fig. 3 is the block diagram for the configuration for illustrating image processing apparatus according to a first embodiment of the present invention.
Fig. 4 schematically shows the flow chart of image procossing according to a first embodiment of the present invention.
Fig. 5 schematically shows the flow chart of track combination step S402 as shown in Figure 4 according to the present invention.
Fig. 6 schematically shows the key facial image of multiple tracks and each track.
Fig. 7 schematically shows the key facial image in trajectory set and trajectory set.
Fig. 8 is the block diagram for the configuration for illustrating image processing system according to a second embodiment of the present invention.
Fig. 9 is the block diagram for the configuration for illustrating image processing system according to a third embodiment of the present invention.
Figure 10 A schematically show inquiry face-image and similar track.
Figure 10 B schematically show inquiry face-image and similar track group.
Specific implementation mode
Detailed description of the present invention exemplary embodiment below with reference to accompanying drawings.It should be noted that following description is substantial
Only it is illustrative, illustrative, and is in no way intended to limit the present invention and its application or purposes.Unless stated otherwise, no
Then the positioned opposite of component and step, numerical expression and numerical value described in embodiment are not limit the scope of the invention.Separately
Outside, technology known to those skilled in the art, method and apparatus may not be discussed in detail, but in situation appropriate
It should be the part of this specification.
It note that the similar terms in similar reference numeral and alphabet diagram, therefore, once a project is at one
It is defined, then need not discuss to it in subsequent attached drawing in attached drawing.
As described above, in human body tracking technology, track indicates the sequence of the specific people traced into from video data
Row face-image.There is at least one key facial image in each track, wherein key facial graphical representation is selected from track
The one group of representative face image selected, for example, key facial can be face image as described above, side face image or with not
With other face-images of facial expression.
On the one hand, inventor has found, it is however generally that the track obtained from video data by human body tracking technology is mostly
Short track.For example, in real-time tracking, the duration of tracking is very short (for example, being no more than 3 seconds).This is because from figure
The video data obtained in picture capture device (for example, camera) can only capture duration shorter video data, and specific
Personnel may also walk out the observation scope of image capture device with a kind of posture, but return to image capture device with other postures
Observation scope is interrupted so as to cause track.Therefore, track may only include several face-images.
On the other hand, inventor also found, currently, specific people's tracking is for retrieving (Human Image in human body image
Retrieval, HIR) in improve retrieval precision it is highly useful.If a track can generate longer with other track combinations
Track, or if can recognize that one group of track of same specific people and be combined into a trajectory set, this will have
Help more key facial image recognitions with different postures and/or facial expression be same specific people.
Therefore, inventor considers, representative of the extraction key facial image as each track, and extraction key facial figure
As the representative as each trajectory set.While updating key facial image, trajectory set is also updated.One in trajectory set
The previous key facial image of track is updated to newly by key facial image is shifted in other tracks from same trajectory set
Key facial image.In addition, update operation will be iterated, until matching a certain criterion.For example, trajectory set no longer
In the case of variation, the present invention can terminate update operation.
Therefore, inventor is further contemplated that the track by same specific people connects into one than any track in pre-track
All longer trajectory set；And track for identification can be not only the track data obtained from a camera, can also be
The track data obtained from different cameral (for example, network camera).
(hardware configuration)
First by with reference to Fig. 2 description may be implemented hereafter described in technology hardware configuration.
Fig. 2 is the block diagram for schematically showing the hardware configuration 200 that technology according to the ... of the embodiment of the present invention may be implemented.Firmly
Part configuration 200 is for example including central processing unit (CPU) 210, random access memory (RAM) 220, read-only memory (ROM)
230, hard disk 240, input equipment 250, output equipment 260, network interface 270 and system bus 280.In addition, hardware configuration 200
Can by such as camera, personal digital assistant (PDA), mobile phone, tablet computer, laptop, desktop computer or other
Suitable electronic equipment is realized.
In the first realization method, image procossing according to the present invention is used as hardware configuration by hardware or firmware configuration
200 module or component.For example, hereinafter with reference to Fig. 3 detailed description image processing apparatus 300, will hereinafter join
Image processing system 800 according to Fig. 8 detailed descriptions and the image processing system 900 hereinafter with reference to Fig. 9 detailed descriptions
It is used as the module or component of hardware configuration 200.In the second realization method, image procossing according to the present invention is by being stored in
The software configuration executed in ROM 230 or hard disk 240 and by CPU 210.For example, hereinafter with reference to Fig. 4 detailed descriptions
Image processing process 400 will be used as the program being stored in ROM 230 or hard disk 240.
CPU 210 is any suitable programmable control device (such as, processor), and may be implemented within ROM
230 or hard disk 240 (such as, memory) in various application programs execute the various functions being described hereinafter.RAM 220
It is used to program or data that interim storage is loaded from ROM 230 or hard disk 240, and is held wherein used also as CPU 210
The space of the various programs of row (such as, implementing the technology being described in detail hereinafter with reference to Fig. 4) and other available functions.Hard disk
240 storage much informations, such as, operating system (OS), various application programs, control program and prestored by manufacturer or
Pre-defined data.
In one implementation, input equipment 250 allows user to be interacted with hardware configuration 200.In an example
In, user can pass through 250 come input picture of input equipment/video/data.In another example, user can pass through input
Equipment 250 triggers the correspondence image processing of the present invention.In addition, various forms may be used in input equipment 250, and such as, button, key
Disk or touch screen.In another implementation, input equipment 250 is for receiving from such as digital camera, video camera and/or net
The image/video of the special electrical devices output of network camera etc..
In one implementation, output equipment 260 is used for user's display processing result (such as, inquiry face of input
Portion's image).Moreover, output equipment 260 can also use various forms, such as, cathode-ray tube (CRT) or liquid crystal display.
In another implementation, output equipment 260 is for exporting handling result (such as, key facial image) subsequently to be grasped
Make, such as HIR processing, FID processing or human attributes identification (Human Attribute Recognition, HAR) processing etc..
Network interface 270 provides the interface for hardware configuration 200 to be connected to network.For example, hardware configuration 200 can be through
By network interface 270 and other electronic equipments connected via a network into row data communication.Alternatively, can be hardware configuration 200
Wireless interface is provided, to execute wireless data communication.System bus 280 can be provided in CPU 210, RAM 220, ROM
230, between hard disk 240, input equipment 250, output equipment 260 and network interface 270 etc. mutual data transmission data transmission
Path.Although being referred to as bus, system bus 280 is not limited to any specific data transmission technology.
Above-mentioned hardware configuration 200 is merely illustrative, and is in no way intended to limit invention, its application, or uses.
Moreover, for brevity, only showing a hardware configuration in fig. 2.But it is also possible to be matched as needed using multiple hardware
It sets.
(image procossing)
(first embodiment)
Image procossing according to the present invention is described referring next to Fig. 3 to Fig. 7.
Fig. 3 is the block diagram for the configuration for illustrating image processing apparatus 300 according to a first embodiment of the present invention.Wherein, in Fig. 3
Some or all of shown module can be realized by specialized hardware.Flow chart 400 shown in Fig. 4 is image procossing dress shown in Fig. 3
Set 300 corresponding process.
As shown in figure 3, image processing apparatus 300 includes acquiring unit 310, track combination unit 320 and key facial figure
As updating unit 330.
First, input equipment 250 shown in Fig. 2 from special electrical devices (for example, same camera or different cameral) or
User receives face-image (for example, the facial image illustrated in Fig. 1).Then, input equipment 250 will via system bus 280
The face-image received is transferred to acquiring unit 310.
Then, acquiring unit 310 obtains face-image via system bus 280 from input equipment 250.
In addition, acquiring unit 310 executes step S401 shown in Fig. 4 to obtain track from the face-image received.
As shown in figure 4, in obtaining step S401, acquiring unit 310 obtains track, and each track all has key facial image.
In one implementation, acquiring unit 310 detects by human testing and tracking technique and tracks a sequence
Face-image is as track.For example, track 1 and track 2 shown in Fig. 6.Scheme that is, track 1 includes two faces as shown in Figure 6
As (F61 and F62), and track 2 includes four face-images (F63, F64, F65 and F66) as shown in Figure 6.In addition, acquiring unit
The feature of 310 extractable tracks.In one implementation, feature can be the key facial image represented as track.Example
Such as, as shown in Figure 6, face-image F61 and F62 is the key facial image of track 1, and face-image F63, F64 and F66 are
The key facial image of track 2.Wherein, face-image F65 is the redundancy of face-image F64, and in other words, face-image F64 is
The redundancy of face-image F65.That is, face-image F64 and F65 are face image, therefore only select a pass as track 2
Key face-image.
Track and its key facial image can be stored and/or are registered in ROM 230, hard disk 240 or data pool.Herein,
Data pool can be the cache pool for analysis, the registration pond for index or for the record pond of storage.Data pool can save
Track more than 10 minutes and its key facial image.Therefore, image processing apparatus 300 has enough information same with on-line joining process
Two tracks of one specific people.In one implementation, in order to improve tracking velocity, image processing apparatus 300 can be rail
Mark establishes index structure, as each track addition index.Therefore, image processing apparatus 300 is in the case where data pool is larger
The track with index can be retrieved rapidly.
Track combination unit 320 is by the track combination of same specific people at a trajectory set.As described above, in data pool
With several from the track that acquiring unit 310 obtains.Therefore, track combination unit 320 can perform track combination shown in Fig. 4
Step S402, to generate trajectory set.
In one implementation, track combination unit 320 selects some tracks pair from data pool, herein, can be from same
Track is traced into the face-image that one camera or different cameral obtain, then track combination unit 320 identifies track pair.Example
Such as, track combination unit 320 compares the similarity of each track pair, to determine whether two tracks are similar to each other.In other words, rail
Mark assembled unit 320 judges track to whether including the face-image of same specific people.In track to including same particular person
In the case of the face-image of member, track combination unit 320 is by two track combinations at a trajectory set.Trajectory set can also store
Or it is registered in ROM 230, hard disk 240 or data pool.
In another implementation, in the case where the track in being registered to data pool is with index, track combination list
Whether 320 usable index list of member selects track pair, then identify track to similar according to the above method.In addition, trajectory set
It closes unit 320 to be first compared to obtain similar track track, is then based on the similar track and generates track pair.
In another implementation, flow chart 500 shown in Fig. 5 is rail as shown in Figure 4 according to the present invention
The corresponding process of mark combination step S402.
Turning now to Fig. 5, in track to obtaining in step S501, the track pair to be identified is selected using sliding window.It is sliding
Dynamic window is defined by time interval.Sliding window may include the track traced by same camera or different cameral.
In the case where similarity is more than threshold value T1, in identification step S502, track combination unit 320 is by two tracks
Face-image be identified as same specific people.Wherein, the length of the sliding window of the time interval between identification and two tracks
It spends related or related with the time span of two tracks, or has with the quantity of the face-image of two tracks in time interval
It closes.
In an example, threshold value T1 is set as 0.8, for calculate any two face-image similarity it is similar
It is as follows to spend function:
Wherein, FiIt is i-th of face-image in key facial image, FjIt is j-th of face figure in key facial image
Picture, and i ≠ j.
It is genuine in the recognition result by the similarity calculation between two tracks, in combination step S503
In, then the identification of track combination unit 320 is combined into a track corresponding to two tracks of same specific people
Group.
For example, Fig. 6 schematically shows the key facial image of multiple tracks and each track.Track combination unit 320
Select track 1 and track 2 as track pair.In track, two track identifications are corresponding to same particular person by assembled unit 320
In the case of member, track combination unit 320 will connect described two tracks, to generate trajectory set.That is, track combination unit 320
As shown in Figure 7 by two track combinations at a trajectory set.Fig. 7 schematically shows the key facial figure of trajectory set and trajectory set
Picture.Hereafter, the key facial image of trajectory set will be described according to key facial image update unit 330 shown in execution Fig. 3.
On the other hand, indicate that one group of trajectory set for having identified track of same specific people can be considered as a new track,
And it can store or be registered in ROM 230, hard disk 240 or data pool.It is identified as pair in a track and a trajectory set
Should be in the case of same specific people, track combination unit 320 will be in track combination to trajectory set.
Key facial image update unit 330 is according to the key facial image for the track being incorporated into trajectory set, update
The key facial image of trajectory set.Therefore, key facial image update unit 330 can perform key facial figure shown in Fig. 4
As update step S403, to generate trajectory set, as shown in figures 6 and 7.
In figure 6, track 1 includes two face-images (F61 and F62)；And track 2 include four face-images (F63,
F64, F65 and F66).The key facial image of track 1 is F61 and F62；The key facial image of track 2 be F63, F64 and
F66.Similarity between the key facial image F61 and F63 that are calculated is more than threshold value T1, key facial image F61 and F63 because
This is identified as same specific people, and is combined into a trajectory set as shown in Figure 7.Therefore, in the figure 7, trajectory set packet
Include 6 face-images (F61, F62, F63, F64, F65 and F66)；And the key facial image of trajectory set is key facial image
F61, F62, F64 and F66.Herein, face-image F61 is the redundancy of face-image F63, and in other words, face-image F63 is face
The redundancy of image F61.That is, both face-image F61 and F63 be the same side face image, therefore only select one as trajectory set
Key facial image.If not deleting redundancy key facial image, tracking velocity can reduce.However, regardless of whether
Redundancy key facial image is deleted, precision will not change.It means that redundancy key facial image will influence tracking velocity,
Without influencing precision.
In the case where trajectory set changes, which is considered as a track by image processing apparatus 300, is then turned
To track combination unit 320, the trajectory set is recalculated with the similarity of other tracks to obtain longer trajectory set, and
The trajectory set is also considered as a track by image processing process 400, then goes to step S402.
Therefore, this iterative step increases, and the size of the sliding window of time interval increases therewith.Then, using more preferably
Key facial image, which is matched, also can be improved precision.That is, the iteration of trajectory set help to re-recognize it is same in the neighbouring time
The more multi-trace of specific people.
(image processing system)
(second embodiment)
In the case where user browses video and wants to check the image of the personnel interested of camera capture, output equipment
260 can display for a user newer key facial image.But in most cases, the video image or regard that user checks
Frequency frame and non-prime face-image, or the high-quality face-image of track or image library wanted without user.Herein, high-quality face
Portion's image can be such as positive, clear, unobstructed face-image.
To solve the above-mentioned problems, the present invention provides second embodiment as shown in Figure 8, can be user's selection and close friend
Ground shows representative face-image, to improve human-computer interaction user experience.
Fig. 8 is the block diagram for the configuration for illustrating image processing system 800 according to a second embodiment of the present invention.Wherein, in Fig. 8
Some or all of shown module can be realized by specialized hardware.As shown in Figure 8, image processing system 800 includes image procossing
Device 300 and selecting unit 810.
Image processing apparatus 300 updates the key facial image of trajectory set as described above by track combination at trajectory set.
Image processing apparatus 300 can also obtain face-image as the interested query image of user (for example, special from acquiring unit 310
The face-image of fix the number of workers).
Then, selecting unit 810 selects query image and trajectory set, calculates the key facial figure of query image and trajectory set
Similarity as between, to determine whether the two matches each other.In the case where similarity is more than threshold value T1, selecting unit 810
High-quality face-image is selected from the key facial image of trajectory set.In addition, query image and rail may be selected in selecting unit 810
Mark, the similarity being computed as described above between query image and the key facial image of track.
Sometimes, directly relatively low with the matched similarity of query image.Therefore, image processing apparatus 300 can be from different cameral
Or key facial image is shifted in the track in other times interval, is supplied with improving retrieval recall rate and providing more preferably face-image
Browsing.
Output equipment 260 provides a user high-quality key facial image for browsing.It is presented to the user the high-quality key of browsing
Face-image is user-friendly；The high-quality key facial image is clear, positive, unobstructed representative key face
Portion's image.Therefore, user can easily and rapidly judge that whom currently selected personnel are in browsing.Therefore, image processing system
800 can be improved browse efficiency and save the time for user.
In another implementation, selecting unit 810 is from the most like track obtained in several similar tracks
Select a key facial image.For example, selecting unit 810 can select face image from the key facial image of different postures
It is shown to user as optimal key facial image.It is supplied if desired, several key facial images also may be selected in selecting unit 810
Browsing.
In another implementation, when accident occurs for public domain, police office can wonder suspect whom is with
And suspect present position.Monitor video and witness can help police office to find suspect.Police office can questioning witnesses pass
In suspect is in the site of the accident the case where.Witness photograph/video that information may include attribute, mobile phone or the camera capture of suspect
Deng.The attribute of suspect may include time, place, clothes color, age, hair and face etc..
In the case, according to image processing system 800, the suspect obtained from witness can be used in selecting unit 810
Information, to be searched in track or trajectory set.For example, selecting unit 810 can search for the clothing of suspect in track or trajectory set
Take color.Then, selecting unit 810 obtains some smaller and unclear face-images, and some face-images are side face figure
Picture.Therefore, for witness, it is difficult to judge whether obtained face-image is suspect's face-image.
As described above, selecting unit 810 can select an optimal face in the key facial image of track or trajectory set
Image.For example, selecting unit 810 selects face image as optimal key facial figure from the key facial image of different postures
As being shown to user (for example, witness or police office).
Sometimes, high-quality or suitable face-image is not had to be browsed for witness in track；Image processing apparatus 300 can be from same
Some more preferably face-images are shifted in other tracks in one trajectory set.Selecting unit 810, which can get, compares before more preferably face
Portion's image, is presented to witness.Therefore, witness or police office can easily and rapidly judge that currently selected specific people is
Who.It can improve in this way and verify effect and save the time for witness and police office.For example, witness or police office can be by high-quality
Face image easily and rapidly judge whether currently selected people is suspect.
(3rd embodiment)
The present invention provides 3rd embodiment as shown in Figure 9, can be used the key facial image shifted from trajectory set to rail
Mark is ranked up, and more key facial images can help to improve sequence precision.Next, will be with reference to Fig. 9, Figure 10 A
According to the third embodiment of the invention with Figure 10 B descriptions.
Fig. 9 is the block diagram for the configuration for illustrating image processing system 900 according to a third embodiment of the present invention.Wherein, in Fig. 9
Some or all of shown module can be realized by specialized hardware.As shown in Figure 9, image processing system 900 includes image procossing
Device 300 and sequencing unit 910.
Image processing apparatus 300 can obtain face-image as the interested query image (example of user from acquiring unit 310
Such as, the face-image of specific people).
Then, sequencing unit 910 can calculate the similarity between query image and the key facial image of track, with to phase
It is ranked up like track.For example, Figure 10 A schematically show inquiry face-image and similar track.Figure 10 A include four
Exemplary track (track 2, track 3, track 5 and track 6) and a query image F101.Sequencing unit 910 is according to query graph
As F101 and the key facial image of track 2, track 3, track 6 or track 5, similarity value as shown in Table 1 below is calculated.
Table 1
| Similarity | Track 2 | Track 3 | Track 6 | Track 5 |
| Query image | 0.6 | 0.7 | 0.85 | 0.9 |
In other words, as shown in table 1, the similarity between query image F101 and the key facial image of track 2 is
0.6；Similarity between query image F101 and the key facial image of track 3 is 0.7；Query image F101 and track 6
Similarity between key facial image is 0.85；Similarity between query image F101 and the key facial image of track 5
It is 0.9.
Therefore, sequencing unit 910 obtains sorted lists according to similarity value.That is, the key facial of query image and track 5
Image is most like, because track 5 has a positive face of more good key than track 6, and query image and key facial image
Posture is face image.On the other hand, due to lacking the good positive face of key, track 2 and track 6 are not arranged both
Sequence is most like track.
To solve the above-mentioned problems, image processing apparatus 300 is as described above by track combination at trajectory set, and updates track
The key facial image of group.Then, sequencing unit 910 calculates similar between query image and the key facial image of trajectory set
Degree.
For example, Figure 10 B schematically show inquiry face-image and similar trajectory set.Figure 10 B include an inquiry
Image F101 and two exemplary track groups, trajectory set 1 and trajectory set 2.Wherein, in image processing apparatus 300, trajectory set 1 by
Track 2 and track 6 are composed, and trajectory set 2 is composed of track 3 and track 5.Also, described two trajectory sets such as Figure 10 B
It is shown that there is its key facial image.
Sequencing unit 910 calculates as follows according to query image F101 and the key facial image of trajectory set 1 or trajectory set 2
Similarity value shown in table 2.
Table 2
| Similarity | Trajectory set 1 | Trajectory set 2 |
| Query image | 0.92 | 0.88 |
In other words, as shown in table 2, the similarity between query image F101 and the key facial image of trajectory set 1 is
0.92；Similarity between query image F101 and the key facial image of trajectory set 2 is 0.88.
Therefore, sequencing unit 910 obtains sorted lists according to similarity value.That is, the crucial face of query image and trajectory set 1
Portion's image is most like, because trajectory set 1 has the positive face of more good key than trajectory set 2.
As described above, trajectory set 1 is composed of track 2 and track 6, therefore more good key facials are from other rails
Track in mark or other times interval is transferred in trajectory set, can make trajectory set longer in this way and with more good passes
Key face-image, to which search recall rate will be improved.
As described above, by track combination at trajectory set, to generate longer track for HIR processing and FID processing；Track
The key facial image of group is shifted from different tracks and/or different cameral, to provide better retrieval precision and man-machine
Interactive user experience.
Above-mentioned all units are the exemplary and/or preferred module for realizing process described in the disclosure.These lists
Member can be hardware cell (such as, field programmable gate array (FPGA), digital signal processor, application-specific integrated circuit etc.)
And/or software module (for example, computer-readable program).There is no at large describe for realizing the unit of each step above.
However, in the case of there is the step of executing a certain process, there may be the corresponding function moulds for realizing the same process
Block or unit (passing through hardware and/or software realization).All groups of the step of passing through description and unit corresponding to these steps
The technical solution of conjunction is included in disclosure herein, as long as the technical solution that they are constituted is complete, is applicable in.
Can methods and apparatus of the present invention be implemented in various ways.For example, can by software, hardware, firmware or
It is arbitrarily combined to implement methods and apparatus of the present invention.The above-mentioned steps sequence of this method is merely illustrative, also, unless
Otherwise stipulated, otherwise the step of method of the invention is not limited to the sequence of above-mentioned specific descriptions.In addition, in some implementations
In example, the present invention can also be implemented as recording program in the recording medium comprising for realizing side according to the present invention
The machine readable instructions of method.Therefore, the present invention also covers storage and is situated between for realizing the record of program according to the method for the present invention
Matter.
Although some specific embodiments of the present invention, those skilled in the art have been shown in detail by example
Member is it should be understood that examples detailed above is merely illustrative, and is not limited the scope of the invention.Those skilled in the art should manage
Solution, above-described embodiment can be changed without departing from the scope and spirit of the present invention.The scope of the present invention is by appended
Claim limits.
Claims (14)
1. a kind of image processing apparatus, described image processing unit include：
Acquiring unit is configured as obtaining at least two tracks from face-image frame, wherein each track is at least one
Key facial image；
Track combination unit is configured as in the case where the key facial image of the track corresponds to same specific people,
By the track combination at a trajectory set；
Face-image updating unit is configured as updating the key facial image of each trajectory set.
2. the apparatus according to claim 1, wherein in the key of the key facial image and other tracks of the trajectory set
In the case that face-image corresponds to same specific people, the trajectory set is considered as a track and other described trajectory sets
It closes.
3. the apparatus of claim 2, wherein execute the track combination unit and face-image update repeatedly
Unit is until the trajectory set no longer changes.
4. the apparatus according to claim 1, wherein the assembled unit further includes：Track pair is obtained using sliding window.
5. the apparatus according to claim 1, described device further includes：It adds and indexes for each track.
6. device according to any one of claim 1 to 5, wherein described device further includes：
Output unit is configured as the key facial image of output trajectory group.
7. the apparatus according to claim 1, wherein obtain the face-image frame from same camera or different cameral.
8. a kind of image processing method, described image processing method include：
Obtaining step, for obtaining at least two tracks from face-image frame, wherein each track has at least one key
Face-image；
Combination step is used in the case where the key facial image of the track corresponds to same specific people, by the rail
Mark is combined into a trajectory set；
Face-image updates step, the key facial image for updating each trajectory set.
9. according to the method described in claim 8, wherein, in the key of the key facial image and other tracks of the trajectory set
In the case that face-image corresponds to same specific people, the trajectory set is considered as a track and other described trajectory sets
It closes.
10. according to the method described in claim 9, wherein, executing the track combination step and the face-image repeatedly more
New step, until the trajectory set no longer changes.
11. a kind of image processing system, described image processing system include：
Image processing apparatus is configured as：
Trajectory set is obtained using key facial image according to any one of claim 1 to 7；
Obtain query image；
Selecting unit is configured as：
Calculate the similarity between the query image and the key facial image of each trajectory set；
In the case where the key facial image of the query image and the trajectory set has highest similarity, from the key
At least one key facial image is selected in face-image, to be shown to user.
12. system according to claim 11, wherein selected key facial is the key facial image from different postures
The face image of middle selection.
13. a kind of image processing system, described image processing system include：
Image processing apparatus is configured as：
Trajectory set is obtained using key facial image according to any one of claim 1 to 7；
Obtain query image；
Sequencing unit is configured as：
Calculate the similarity between the query image and the key facial image of each trajectory set；
The trajectory set is ranked up according to the similarity.
14. system according to claim 13, wherein the system also includes：
Output unit is configured as exporting the sorted lists of the trajectory set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710238538.XA CN108734049A (en) | 2017-04-13 | 2017-04-13 | Image processing method and device and image processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710238538.XA CN108734049A (en) | 2017-04-13 | 2017-04-13 | Image processing method and device and image processing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108734049A true CN108734049A (en) | 2018-11-02 |
Family
ID=63923654
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710238538.XA Pending CN108734049A (en) | 2017-04-13 | 2017-04-13 | Image processing method and device and image processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108734049A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006146823A (en) * | 2004-11-24 | 2006-06-08 | Nippon Hoso Kyokai <Nhk> | Video object trajectory adding system and video object trajectory adding program |
| US20080080743A1 (en) * | 2006-09-29 | 2008-04-03 | Pittsburgh Pattern Recognition, Inc. | Video retrieval system for human face content |
| US7636453B2 (en) * | 2004-05-28 | 2009-12-22 | Sony United Kingdom Limited | Object detection |
| CN102165464A (en) * | 2008-07-14 | 2011-08-24 | 谷歌公司 | Method and system for automated annotation of persons in video content |
| CN102457680A (en) * | 2010-11-05 | 2012-05-16 | 佳能株式会社 | Image processing apparatus and image processing method |
| CN102542573A (en) * | 2010-10-27 | 2012-07-04 | 索尼公司 | Image processing apparatus, image processing method, and program |
| CN102788580A (en) * | 2012-06-20 | 2012-11-21 | 天津工业大学 | Flight path synthetic method in unmanned aerial vehicle visual navigation |
| CN105100894A (en) * | 2014-08-26 | 2015-11-25 | Tcl集团股份有限公司 | Automatic face annotation method and system |
-
2017
- 2017-04-13 CN CN201710238538.XA patent/CN108734049A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7636453B2 (en) * | 2004-05-28 | 2009-12-22 | Sony United Kingdom Limited | Object detection |
| JP2006146823A (en) * | 2004-11-24 | 2006-06-08 | Nippon Hoso Kyokai <Nhk> | Video object trajectory adding system and video object trajectory adding program |
| US20080080743A1 (en) * | 2006-09-29 | 2008-04-03 | Pittsburgh Pattern Recognition, Inc. | Video retrieval system for human face content |
| CN102165464A (en) * | 2008-07-14 | 2011-08-24 | 谷歌公司 | Method and system for automated annotation of persons in video content |
| CN102542573A (en) * | 2010-10-27 | 2012-07-04 | 索尼公司 | Image processing apparatus, image processing method, and program |
| CN102457680A (en) * | 2010-11-05 | 2012-05-16 | 佳能株式会社 | Image processing apparatus and image processing method |
| CN102788580A (en) * | 2012-06-20 | 2012-11-21 | 天津工业大学 | Flight path synthetic method in unmanned aerial vehicle visual navigation |
| CN105100894A (en) * | 2014-08-26 | 2015-11-25 | Tcl集团股份有限公司 | Automatic face annotation method and system |
Non-Patent Citations (3)
| Title |
|---|
| LIJUAN WANG 等: "Synthesizing Photo-Real Talking Head via Trajectory-Guided Sample Selection", 《2010 ISCA》 * |
| 刘进: "影像合成运动镜头中的轨迹同步技术", 《中国电影电视技术学会》 * |
| 阳珊 等: "基于BLSTM-RNN的语音驱动逼真面部动画合成", 《清华大学学报（自然科学版）》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Doughty et al. | Who's better? who's best? pairwise deep ranking for skill determination | |
| US10217027B2 (en) | Recognition training apparatus, recognition training method, and storage medium | |
| US10534801B2 (en) | Map screen determining method and apparatus | |
| Wells et al. | Eyewitness identification reforms: Are suggestiveness-induced hits and guesses true hits? | |
| CN111310019A (en) | Information recommendation method, information processing method, system and equipment | |
| JP2019507417A (en) | User interface for multivariable search | |
| CN107644213A (en) | Video person extraction method and device | |
| CN106204156A (en) | A kind of advertisement placement method for network forum and device | |
| JP7238902B2 (en) | Information processing device, information processing method, and program | |
| CN106789543A (en) | The method and apparatus that facial expression image sends are realized in session | |
| US20160306870A1 (en) | System and method for capture, classification and dimensioning of micro-expression temporal dynamic data into personal expression-relevant profile | |
| CN108460154A (en) | Searching method, device, computer equipment and storage medium | |
| CN109871492B (en) | Task processing method and device, computer equipment and computer storage medium | |
| CN108241690A (en) | A kind of data processing method and device, a kind of device for data processing | |
| CN113849723A (en) | Search method and search device | |
| KR101955920B1 (en) | Search method and apparatus using property language | |
| Ringel | Multimarket membership mapping | |
| JPH11250106A (en) | Method for automatically retrieving registered trademark through the use of video information of content substrate | |
| JP7435837B2 (en) | Information processing system, information processing device, information processing method, and program | |
| WO2017020719A1 (en) | Method and device for classifying object information in page | |
| CN108734049A (en) | Image processing method and device and image processing system | |
| Hiippala | Corpus-based insights into multimodality and genre in primary school science diagrams | |
| US20220172271A1 (en) | Method, device and system for recommending information, and storage medium | |
| JP3547338B2 (en) | Information retrieval method and device | |
| US20200104945A1 (en) | System and method for curation of notable work and relating it to involved organizations and individuals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181102 |