CN107437019A - The auth method and device of lip reading identification - Google Patents
The auth method and device of lip reading identification Download PDFInfo
- Publication number
- CN107437019A CN107437019A CN201710643852.6A CN201710643852A CN107437019A CN 107437019 A CN107437019 A CN 107437019A CN 201710643852 A CN201710643852 A CN 201710643852A CN 107437019 A CN107437019 A CN 107437019A
- Authority
- CN
- China
- Prior art keywords
- lip
- information
- user
- image
- structure light
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
- G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
- G06V40/175—Static expression
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Collating Specific Patterns (AREA)
Abstract
The invention discloses the auth method and device of a kind of lip reading identification, wherein, method includes：To the user's face projective structure light for carrying out lip reading identification, and according to the multiple structure light images by active user's face modulation of default collection period shooting；Phase information corresponding to lip position pixel in multiple structure light images is demodulated, obtains multiple three-dimensional lip images；The lip characteristic information corresponding to extraction from multiple three-dimensional lip images, and multiple three-dimensional lip images are according to the lip different information of timing variations；Lip characteristic information and lip different information are matched using the sample characteristics information of default lip reading model library, obtain lip reading information corresponding to the sample characteristics information that the match is successful；By lip reading information compared with default checking information, if comparative result is identical, user identity legal authorization corresponding operating is verified.Thus, authentication mode is enriched, improves the degree of accuracy of authentication.
Description
Technical field
The present invention relates to technical field of information processing, more particularly to the auth method and device of a kind of identification of lip reading.
Background technology
With the development of Internet technology, human language identification technology industry, household electrical appliances, communication, automotive electronics, medical treatment,
The multiple fields such as home services industry are widely used.
In correlation technique, the two dimensional image based on human face and lip carries out the identification of human language, i.e., based on the use to acquisition
The extraction of the two dimensional image progress lip outline of family lip, profile when will extract the outline user difference pronunciation of lip enter
Row compares, to identify the pronunciation of user.
However, in real life, the two-dimensional silhouette information of speech of the user to many words is all identical, thus,
The recognition accuracy of above-mentioned identification method is relatively low.
The content of the invention
The present invention provides a kind of auth method and device of lip reading identification, to solve in the prior art, based on lip
When two dimensional image carries out speech recognition, the problem of inaccurate is identified.
The embodiment of the present invention provides a kind of auth method of lip reading identification, including：To the user for carrying out lip reading identification
Facial projective structure light, and according to the multiple structure light images by active user's face modulation of default collection period shooting；
Phase information corresponding to lip position pixel in the multiple structure light image is demodulated, obtains multiple three-dimensional lip images；From institute
Lip characteristic information corresponding to extraction in multiple three-dimensional lip images is stated, and the multiple three-dimensional lip image becomes according to sequential
The lip different information of change；Using the sample characteristics information of default lip reading model library to the lip characteristic information and described
Lip different information is matched, and obtains lip reading information corresponding to the sample characteristics information that the match is successful；By the lip reading information
Compared with default checking information, if comparative result is identical, user identity legal authorization corresponding operating is verified.
Another embodiment of the present invention provides a kind of authentication means of lip reading identification, including：Acquisition module, for entering
The user's face projective structure light of row lip reading identification, and adjusted according to the shooting of default collection period is multiple by active user's face
The structure light image of system；First acquisition module, for demodulating phase corresponding to lip position pixel in the multiple structure light image
Position information, obtains multiple three-dimensional lip images；Extraction module, for lip corresponding to the extraction from the multiple three-dimensional lip image
Shape characteristic information, and the multiple three-dimensional lip image is according to the lip different information of timing variations；Second acquisition module, use
The lip characteristic information and the lip different information are entered in the sample characteristics information of the default lip reading model library of application
Row matching, obtains lip reading information corresponding to the sample characteristics information that the match is successful；Authentication module, for by the lip reading information with
Default checking information is compared, if comparative result is identical, verifies user identity legal authorization corresponding operating.
Further embodiment of this invention provides a kind of terminal device, including memory and processor, is stored in the memory
There is computer-readable instruction, when the instruction is by the computing device so that the computing device first aspect present invention
The auth method of lip reading identification described in embodiment.
A further embodiment of the present invention provides a kind of non-transitorycomputer readable storage medium, is stored thereon with computer journey
Sequence, realize that the identity of the lip reading identification as described in first aspect present invention embodiment is tested when the computer program is executed by processor
Card method.
Technical scheme provided in an embodiment of the present invention can include the following benefits：
To the user's face projective structure light for carrying out lip reading identification, and it is multiple through excessive according to the shooting of default collection period
The structure light image of preceding user's face modulation, demodulates phase information corresponding to lip position pixel in multiple structure light images, obtains
Multiple three-dimensional lip images are taken, the lip characteristic information corresponding to extraction from multiple three-dimensional lip images, and multiple three-dimensional lips
Portion's image is according to the lip different informations of timing variations, using the sample characteristics information of default lip reading model library to lip feature
Information and lip different information are matched, and lip reading information corresponding to the sample characteristics information that the match is successful are obtained, by lip reading
Information is compared with default checking information, if comparative result is identical, verifies user identity legal authorization corresponding operating.By
This, enriches authentication mode, improves the degree of accuracy of authentication.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partly become from the following description
Obtain substantially, or recognized by the practice of the present invention.
Brief description of the drawings
Of the invention above-mentioned and/or additional aspect and advantage will become from the following description of the accompanying drawings of embodiments
Substantially and it is readily appreciated that, wherein：
Fig. 1 is the flow chart of the auth method of lip reading identification according to an embodiment of the invention；
Fig. 2 (a) is the schematic diagram of a scenario one of structural light measurement according to an embodiment of the invention；
Fig. 2 (b) is the schematic diagram of a scenario two of structural light measurement according to an embodiment of the invention；
Fig. 2 (c) is the schematic diagram of a scenario three of structural light measurement according to an embodiment of the invention；
Fig. 2 (d) is the schematic diagram of a scenario four of structural light measurement according to an embodiment of the invention；
Fig. 2 (e) is the schematic diagram of a scenario five of structural light measurement according to an embodiment of the invention；
Fig. 3 (a) is the local diffraction structure schematic diagram of collimation beam splitting element according to an embodiment of the invention；
Fig. 3 (b) is the local diffraction structure schematic diagram of collimation beam splitting element in accordance with another embodiment of the present invention；
Fig. 4 is that the application scenarios of the auth method identified according to the lip reading of one specific embodiment of the present invention are illustrated
Figure；
Fig. 5 is the structured flowchart of the authentication means of lip reading identification according to an embodiment of the invention；
Fig. 6 is the structured flowchart of the authentication means of lip reading identification in accordance with another embodiment of the present invention；
Fig. 7 is the structured flowchart of the authentication means identified according to the lip reading of another embodiment of the invention；And
Fig. 8 is the structural representation of the image processing circuit in terminal device according to an embodiment of the invention.
Embodiment
Embodiments of the invention are described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end
Same or similar label represents same or similar element or the element with same or like function.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and is not considered as limiting the invention.
Below with reference to the accompanying drawings the auth method and device of the lip reading identification of the embodiment of the present invention are described.
Fig. 1 is the flow chart of the auth method of lip reading identification according to an embodiment of the invention.
As shown in figure 1, the auth method of lip reading identification includes：
Step 101, to the user's face projective structure light for carrying out lip reading identification, and it is more according to the shooting of default collection period
The individual structure light image by active user's face modulation.
When carrying out contours extract for being currently based on two dimensional image to identify user language, the not high technology of recognition accuracy
Problem, the present invention propose a kind of mode being identified based on structure light, wherein, the identification method can be used for arbitrarily passing through
In the scene that identification user language is applied, for the ease of description, the present invention concentrates on to be entered applied in authentication scene
Row explanation.
Specifically, in order to improve the degree of accuracy of the authentication to user, three-dimensional lip is carried out to user based on structure light
The collection of the relevant information of portion's image, such as, laser stripe, Gray code, sine streak or, non-homogeneous speckle etc., thus,
Due to structure light can be based on face profile and depth information carry out to pickup user the relevant information of three-dimensional lip image
Collection, taken pictures the mode that the two-dimentional lip image information of collection is identified compared to only according to camera, the degree of accuracy is higher, just
In the degree of accuracy for ensureing subscriber authentication.
More it is apparent from order that obtaining those skilled in the art, the three-dimensional of user how is gathered according to structure light
The relevant information of lip image, illustrate it by taking a kind of widely used optical grating projection technology (fringe projection technology) as an example below
Concrete principle, wherein, optical grating projection technology belongs to sensu lato area-structure light.
When being projected using area-structure light, as shown in Fig. 2 (a), sine streak is produced by computer programming, by this
Sine streak is by projection to measured object, the degree of crook modulated using CCD camera shooting striped by object, demodulation
The curved stripes obtain phase, then phase is converted into the height of the whole audience.Certain wherein crucial point is exactly system
Demarcation, including the calibration of camera of the demarcation of system geometric parameter and CCD camera and projector equipment, are otherwise likely to produce
Error or error coupler.Because its exterior parameter is not demarcated, correct elevation information can not possibly be calculated by phasometer.
Specifically, the first step, programming produce sine streak figure, because subsequently to utilize deforming stripe figure to obtain phase,
For example phase is obtained using four step phase-shifting methods, therefore four width phase difference pi/2 striped is produced here, then by the four spokes line
Timesharing is projected on measured object (mask), is collected such as the figure on Fig. 2 (b) left sides, while to be gathered shown on the right of Fig. 2 (b)
The striped of the plane of reference.
Second step, phase recovery is carried out, calculated by phase modulation by modulation bar graph by four width collected, obtained here
To phase diagram be to block phase diagram because the result that four step Phase-shifting algorithms obtain be by arctan function calculate gained, thus
It is limited between [- pi, pi], that is to say, that whenever its value exceedes the scope, it can restart again.Obtained phase main value
As shown in Fig. 2 (c).
Wherein, it is necessary to which the saltus step that disappears, it is continuous phase that will block phase recovery, such as Fig. 2 (d) institutes under second step
Show, the left side is the continuous phase modulated, and the right is to refer to continuous phase.
3rd step, subtract each other to obtain phase difference by the continuous phase modulated and with reference to continuous phase, the phase difference then characterizes
Elevation information of the measured object with respect to the plane of reference, then phase and high-degree of conversion formula (wherein relevant parameter is by demarcating) are substituted into,
Obtain the threedimensional model of the object under test as shown in Fig. 2 (e).
It should be appreciated that in actual applications, according to the difference of concrete application scene, employed in the embodiment of the present invention
Structure light in addition to above-mentioned grating, can also be other arbitrary graphic patterns.
, wherein it is desired to, it is emphasized that as a kind of possible implementation, the present invention carries out user using pattern light
Facial information collection.
In the present embodiment, the diffraction element of essentially flat board can be used, the diffraction element has particular phases distribution
Embossment diffraction structure, cross section is floats with two or more concavo-convex step embossment structures, or multiple concavo-convex steps
Carve structure, the thickness substantially l microns of substrate, each step it is highly non-uniform, be 0.7 micron one 0.9 microns.Fig. 3 (a) is
The present embodiment collimation beam splitting element local diffraction structure, Fig. 3 (b) be along the A of section A one cross sectional side view, abscissa and
The unit of ordinate is micron.
So as to, multi beam diffraction light is obtained after diffraction is carried out to light beam due to common diffraction element, but per beam diffraction light light
Strong difference is big, also big to the risk of human eye injury, even carries out re-diffraction, the uniformity of obtained light beam to diffraction light
It is relatively low, object is projected in image information processing device using such light beam, drop shadow effect is poor.
Collimation beam splitting element in the present embodiment not only has the function that to collimate uncollimated rays, also has light splitting
Effect, i.e., through speculum reflection non-collimated light after collimate beam splitting element toward different angle be emitted multi-beam collimation light beam,
And the area of section approximately equal of the multi-beam collimation light beam of outgoing, flux of energy approximately equal, and then to spread out using the light beam
Scatterplot light after penetrating carries out image procossing or the effect of projection is more preferable, meanwhile, laser emitting light is dispersed to every light beam, further
The risk of injury human eye is reduced, and due to being pattern light, relative to other uniform structure lights of arrangement, reaches same
During collection effect, the electric energy consumed is lower.
Specifically, to the user's face projective structure light for carrying out lip reading identification, and it is more according to the shooting of default collection period
It is individual by active user face modulation structure light image, wherein, default collection period can with user speak word speed and
The disposal ability of structure light relevant device is relevant, when the word speed of speaking of user is faster, the disposal ability of structure light relevant device more
By force, frequency acquisition corresponding to collection period is about high.
Step 102, phase information corresponding to lip position pixel in multiple structure light images is demodulated, obtains multiple three-dimensional lips
Portion's image.
Specifically, the principle based on structure light is understood, it is corresponding can to demodulate lip position pixel in multiple structure light images
Phase information, multiple three-dimensional lip images are obtained according to phase information.
It should be noted that according to the difference of application scenarios, can be obtained in different ways based on multiple structure light images
Multiple three-dimensional lip images are taken, are exemplified below：
The first example：
Phase information corresponding to deformation position pixel in each structure light image is demodulated, phase information is converted into height believes
Breath, user's face 3-D view corresponding with each structure light image is obtained according to elevation information, because lip is located at nose
Lower section, the elevation information of nose is more than the elevation information of lip, and the elevation information of lip is higher than facial other positions, therefore,
Can this feature based on the elevation information of lip three-dimensional lip image is extracted from user's face 3-D view, certainly, also may be used
To combine outline identification technology, based on user's face 3-D view, the profile of user's lip is identified, is obtained according to the profile more
Individual three-dimensional lip image.
Second of example：
Using relative profile identification technology, the lip position of user is identified, demodulates lip in each structure light image
Phase information corresponding to the pixel of portion position, elevation information is converted into by phase information, is established according to the elevation information local
User's face 3-D view, and then, three-dimensional lip image is extracted from local users face 3-D view.
Step 103, the lip characteristic information corresponding to extraction from multiple three-dimensional lip images, and multiple three-dimensional lip figures
As the lip different information according to timing variations.
Wherein, lip characteristic information can include opening size of the three-dimensional shape of lip, lip etc..
Step 104, using the sample characteristics information of default lip reading model library to lip characteristic information and lip difference
Information is matched, and obtains lip reading information corresponding to the sample characteristics information that the match is successful.
Step 105, by lip reading information compared with default checking information, if comparative result is identical, user is verified
Identity legal authorization corresponding operating.
It is appreciated that lip reading model library is pre-established, according to the sample characteristics information of the lip reading model library to lip feature
Information and lip different information are matched, and in one embodiment of the invention, collecting sample information, sample information includes
The lip video image and corresponding audio-frequency information of different regions user, lip video image is analyzed by image processing model
Lip characteristic value is obtained, analyzing audio-frequency information by speech recognition modeling obtains language message, passes through deep neural network model
Training samples information, establish the lip reading model library for the corresponding relation for including lip characteristic value and language message.
Wherein, in order to further improve the recognition accuracy of lip reading model, dialectal difference, gender differences, year can be combined
Age difference etc. establishes the recognition accuracy of lip reading model, for example, difference is obtained with reference to the audio-frequency information of different regions user
The average word speed in area etc. establishes model, and then, determine to match with active user location according to the average word speed of different regions
Collection period, by the lip characteristic information identified according to the collection period carry out lip reading information matching.
Specifically, the lip characteristic information corresponding to extraction from multiple three-dimensional lip images of acquisition, and multiple three-dimensionals
Lip image according to the lip different informations of timing variations, and then, using the sample characteristics information pair of default lip reading model library
Lip characteristic information and lip different information are matched, and are obtained lip reading corresponding to the sample characteristics information that the match is successful and are believed
Breath, by lip reading information compared with default checking information, if comparative result is identical, verify user identity legal authorization phase
It should operate.
Wherein, checking information can be that user is set according to demands of individuals, wherein, set-up mode is according to application demand
Difference can be different, for example user is recorded in the early stage, system by the recording of user be identified and using recognition result as
Checking information, and for example, can be when user sets lip reading 3D identifications, there is provided give user multiple checking informations to be selected, this is to be selected
Checking information can be that written form can also be speech form etc., and then, the checking information to be selected selected according to user is corresponding
Lip reading information, as checking information.
In order that obtaining those skilled in the art, the auth method identified to the lip reading of the embodiment of the present invention is more clear
Chu, illustrated with reference to specific application scenarios.
In this example, default checking information is " open sesame ", and the application scenarios of authentication information are gate inhibitions.
As shown in figure 4, user A is in opening gate, the relevant device on gate inhibition to user's A face projective structure light, and
Multiple structure light images by active user's face modulation are shot according to default collection period, now user A says " sesame
Open the door ", after default collection period, phase information corresponding to lip position pixel in multiple structure light images is demodulated, is obtained
Multiple three-dimensional lip images.
And then the lip characteristic information corresponding to extraction from multiple three-dimensional lip images of acquisition, and multiple three-dimensional lips
Portion's image is according to the lip different information of timing variations, and by lip characteristic information, and multiple three-dimensional lip images are according to sequential
The lip different information of change is uploaded to database, and the sample characteristics information of the default lip reading model library of database application is to lip
Characteristic information and lip different information are matched, obtain the sample characteristics information that the match is successful corresponding to lip reading information be
" open sesame ", comparative result is identical, then verifies that user identity legal authorization is opened the door.
Based on above description, it is emphasized that, in above-described embodiment the 3-D view of the lip based on user with when
Between change carry out user identity checking, the degree of accuracy is higher, but under some scenes, may be based only on user's face figure
As may recognize that the identity of user is illegal, the structure light image without being modulated further according to user's face obtains multiple three
Tie up lip image.
Thus, in order to improve recognition efficiency, in one embodiment of the invention, an authorization data storehouse is pre-established,
The face-image for the user for not allowing to be operated can be included in the authorization data storehouse, or, it is allowed to carry out associative operation
The face-image of user, before the user's face projective structure light to progress lip reading identification, can also facial knowledge be carried out to user
Not, face feature information is extracted, face feature information is authenticated using default authorization data storehouse, if authentication passes through, than
Face-image such as user allows the facial images match for carrying out the user of associative operation, or, to not allowing to carry out related behaviour
The face-image of the user of work mismatches, and is verified with then prompting user to carry out lip reading identification.
In summary, the auth method of the lip reading identification of the embodiment of the present invention, to the user plane for carrying out lip reading identification
Portion's projective structure light, and according to the multiple structure light images by active user's face modulation of default collection period shooting, solution
Phase information corresponding to lip position pixel in multiple structure light images is adjusted, multiple three-dimensional lip images are obtained, from multiple three-dimensionals
Lip characteristic information corresponding to extraction in lip image, and multiple three-dimensional lip images are believed according to the lip difference of timing variations
Breath, is matched using the sample characteristics information of default lip reading model library to lip characteristic information and lip different information,
Lip reading information corresponding to the sample characteristics information that the match is successful is obtained, by lip reading information compared with default checking information,
If comparative result is identical, user identity legal authorization corresponding operating is verified.Thus, authentication mode is enriched, is improved
The degree of accuracy of authentication.
In order to realize above-described embodiment, the invention also provides a kind of authentication means of lip reading identification, Fig. 5 is basis
The structured flowchart of the authentication means of the lip reading identification of one embodiment of the invention, as shown in figure 5, the device includes collection mould
Block 100, the first acquisition module 200, extraction module 300, the second acquisition module 400 and authentication module 500.
Wherein, acquisition module 100, for the user's face projective structure light for carrying out lip reading identification, and according to default
The multiple structure light images by active user's face modulation of collection period shooting.
First acquisition module 200, for demodulating phase information corresponding to lip position pixel in multiple structure light images, obtain
Take multiple three-dimensional lip images.
In one embodiment of the invention, as shown in fig. 6, on the basis of as shown in Figure 5, first acquisition module
200 include demodulating unit 210, conversion unit 220, acquiring unit 230 and extraction unit 240.
Wherein, demodulating unit 210, for demodulating phase information corresponding to deformation position pixel in each structure light image.
Conversion unit 220, for phase information to be converted into elevation information.
Acquiring unit 230, for obtaining user's face graphics corresponding with each structure light image according to elevation information
Picture.
Extraction unit 240, for extracting three-dimensional lip image from user's face 3-D view.
Extraction module 300, for lip characteristic information corresponding to the extraction from multiple three-dimensional lip images, and multiple three
Tie up lip different information of the lip image according to timing variations.
Second acquisition module 400, the sample characteristics information for the default lip reading model library of application is to lip characteristic information
And lip different information is matched, lip reading information corresponding to the sample characteristics information that the match is successful is obtained；
Authentication module 500, for by lip reading information compared with default checking information, if comparative result is identical,
Verify user identity legal authorization corresponding operating.
In one embodiment of the invention, the authentication as the lip reading that Fig. 7 is another basic embodiment identifies fills
The structured flowchart put, as shown in fig. 7, on the basis of as shown in Figure 5, the device also includes authentication module 600.
Wherein, extraction module 300, it is additionally operable to carry out face recognition to user, extracts face feature information, authentication module
600, it is additionally operable to authenticate face feature information using default authorization data storehouse, if authentication passes through, prompts user to enter
The identification checking of row lip reading.
It should be noted that the explanation of the foregoing auth method to lip reading identification, is also applied for of the invention real
The authentication means of the lip reading identification of example are applied, unpub details in the embodiment of the present invention, will not be repeated here.
The division of modules is only used for for example, in other embodiment in the authentication means of above-mentioned lip reading identification
In, the authentication means that lip reading identifies can be divided into different modules as required, to complete the body of above-mentioned lip reading identification
All or part of function of part checking device.
In summary, the authentication means of the lip reading identification of the embodiment of the present invention, to the user plane for carrying out lip reading identification
Portion's projective structure light, and according to the multiple structure light images by active user's face modulation of default collection period shooting, solution
Phase information corresponding to lip position pixel in multiple structure light images is adjusted, multiple three-dimensional lip images are obtained, from multiple three-dimensionals
Lip characteristic information corresponding to extraction in lip image, and multiple three-dimensional lip images are believed according to the lip difference of timing variations
Breath, is matched using the sample characteristics information of default lip reading model library to lip characteristic information and lip different information,
Lip reading information corresponding to the sample characteristics information that the match is successful is obtained, by lip reading information compared with default checking information,
If comparative result is identical, user identity legal authorization corresponding operating is verified.Thus, authentication mode is enriched, is improved
The degree of accuracy of authentication.
In order to realize above-described embodiment, the invention also provides a kind of terminal device, above-mentioned terminal device includes image
Process circuit, image processing circuit can utilize hardware and/or component software to realize, it may include define ISP (Image Signal
Processing, picture signal processing) pipeline various processing units.Fig. 8 is that terminal according to an embodiment of the invention is set
The structural representation of standby image processing circuit.As shown in figure 8, for purposes of illustration only, only show related to the embodiment of the present invention
The various aspects of image processing techniques.
As shown in figure 8, image processing circuit 110 includes imaging device 1110, ISP processors 1130 and control logic device
1140.Imaging device 1110 may include the camera and structure light with one or more lens 1112, imaging sensor 1114
The projector 1116.Structured light projector 1116 is by structured light projection to measured object.Wherein, the structured light patterns can be laser strip
Line, Gray code, sine streak or, speckle pattern of random alignment etc..Imaging sensor 1114 catches projection to measured object shape
Into structure light image, and structure light image is sent to ISP processors 1130, by ISP processors 1130 to structure light image
It is demodulated the depth information for obtaining measured object.Meanwhile imaging sensor 1114 can also catch the color information of measured object.When
So, the structure light image and color information of measured object can also be caught respectively by two imaging sensors 1114.
Wherein, by taking pattern light as an example, ISP processors 1130 are demodulated to structure light image, are specifically included, from this
The speckle image of measured object is gathered in structure light image, by the speckle image of measured object with reference speckle image according to pre-defined algorithm
View data calculating is carried out, each speckle point for obtaining speckle image on measured object dissipates relative to reference to the reference in speckle image
The displacement of spot.The depth value of each speckle point of speckle image is calculated using trigonometry conversion, and according to the depth
Angle value obtains the depth information of measured object.
It is, of course, also possible to obtain the depth image by the method for binocular vision or based on jet lag TOF method
Information etc., is not limited herein, as long as can obtain or belong to this by the method for the depth information that measured object is calculated
The scope that embodiment includes.
, can quilt after the color information that ISP processors 1130 receive the measured object that imaging sensor 1114 captures
View data corresponding to surveying the color information of thing is handled.ISP processors 1130 are analyzed view data can with acquisition
For the image statistics for the one or more control parameters for determining imaging device 1110.Imaging sensor 1114 may include color
Color filter array (such as Bayer filters), imaging sensor 1114 can obtain is caught with each imaging pixel of imaging sensor 1114
The luminous intensity and wavelength information caught, and the one group of raw image data that can be handled by ISP processors 1130 is provided.
ISP processors 1130 handle raw image data pixel by pixel in various formats.For example, each image pixel can
Bit depth with 8,10,12 or 14 bits, ISP processors 1130 can be carried out at one or more images to raw image data
Reason operation, image statistics of the collection on view data.Wherein, image processing operations can be by identical or different bit depth
Precision is carried out.
ISP processors 1130 can also receive pixel data from video memory 1120.Video memory 1120 can be storage
Independent private memory in the part of device device, storage device or electronic equipment, and may include DMA (Direct
Memory Access, direct memory access (DMA)) feature.
When receiving raw image data, ISP processors 1130 can carry out one or more image processing operations.
After ISP processors 1130 get color information and the depth information of measured object, it can be merged, obtained
3-D view.Wherein, can be extracted by least one of appearance profile extracting method or contour feature extracting method corresponding
The feature of measured object.Such as pass through active shape model method ASM, active appearance models method AAM, PCA PCA, discrete
The methods of cosine transform method DCT, the feature of measured object is extracted, is not limited herein.It will be extracted respectively from depth information again
The feature of measured object and feature progress registration and the Fusion Features processing that measured object is extracted from color information.Herein refer to
Fusion treatment can be the feature that will be extracted in depth information and color information directly combination or by different images
Middle identical feature combines after carrying out weight setting, it is possibility to have other amalgamation modes, finally according to the feature after fusion, generation
3-D view.
The view data of 3-D view can be transmitted to video memory 1120, to carry out other place before shown
Reason.ISP processors 1130 from the reception processing data of video memory 1120, and to the processing data carry out original domain in and
Image real time transfer in RGB and YCbCr color spaces.The view data of 3-D view may be output to display 1160, for
User watches and/or further handled by graphics engine or GPU (Graphics Processing Unit, graphics processor).
In addition, the output of ISP processors 1130 also can be transmitted to video memory 1120, and display 1160 can be from video memory
1120 read view data.In one embodiment, video memory 1120 can be configured as realizing one or more frame bufferings
Device.In addition, the output of ISP processors 1130 can be transmitted to encoder/decoder 1150, so as to encoding/decoding image data.Compile
The view data of code can be saved, and be decompressed before being shown in the equipment of display 1160.Encoder/decoder 1150 can
Realized by CPU or GPU or coprocessor.
The image statistics that ISP processors 1130 determine, which can be transmitted, gives the unit of control logic device 1140.Control logic device
1140 may include the processor and/or microcontroller that perform one or more routines (such as firmware), and one or more routines can root
According to the image statistics of reception, the control parameter of imaging device 1110 is determined.
It is the step of realizing the auth method of lip reading identification with image processing techniques in Fig. 8 below：
Step 101 ', shot to the user's face projective structure light for carrying out lip reading identification, and according to default collection period
Multiple structure light images by active user's face modulation.
Step 102 ', phase information corresponding to lip position pixel in the multiple structure light image is demodulated, is obtained multiple
Three-dimensional lip image.
Step 103 ', the lip characteristic information corresponding to extraction from the multiple three-dimensional lip image, and it is the multiple
Three-dimensional lip image is according to the lip different informations of timing variations.
Step 104 ', using the sample characteristics information of default lip reading model library to the lip characteristic information and described
Lip different information is matched, and obtains lip reading information corresponding to the sample characteristics information that the match is successful.
Step 105 ', by the lip reading information compared with default checking information, if comparative result is identical, verify
User identity legal authorization corresponding operating.
In order to realize above-described embodiment, the present invention also proposes a kind of non-transitorycomputer readable storage medium, deposited thereon
Computer program is contained, lip reading identification as in the foregoing embodiment can be realized when the computer program is executed by processor
Auth method.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description
Point is contained at least one embodiment or example of the present invention.In this manual, to the schematic representation of above-mentioned term not
Identical embodiment or example must be directed to.Moreover, specific features, structure, material or the feature of description can be with office
Combined in an appropriate manner in one or more embodiments or example.In addition, in the case of not conflicting, the skill of this area
Art personnel can be tied the different embodiments or example and the feature of different embodiments or example described in this specification
Close and combine.
In addition, term " first ", " second " are only used for describing purpose, and it is not intended that instruction or hint relative importance
Or the implicit quantity for indicating indicated technical characteristic.Thus, define " first ", the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the invention, " multiple " are meant that at least two, such as two, three
It is individual etc., unless otherwise specifically defined.
Any process or method described otherwise above description in flow chart or herein is construed as, and represents to include
Module, fragment or the portion of the code of the executable instruction of one or more the step of being used to realize custom logic function or process
Point, and the scope of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discuss suitable
Sequence, including according to involved function by it is basic simultaneously in the way of or in the opposite order, carry out perform function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system including the system of processor or other can be held from instruction
The system of row system, device or equipment instruction fetch and execute instruction) use, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium " can any can be included, store, communicate, propagate or pass
Defeated program is for instruction execution system, device or equipment or the dress used with reference to these instruction execution systems, device or equipment
Put.The more specifically example (non-exhaustive list) of computer-readable medium includes following：Electricity with one or more wiring
Connecting portion (electronic installation), portable computer diskette box (magnetic device), random access memory (RAM), read-only storage
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device, and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium, which can even is that, to print the paper of described program thereon or other are suitable
Medium, because can then enter edlin, interpretation or if necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each several part of the present invention can be realized with hardware, software, firmware or combinations thereof.Above-mentioned
In embodiment, software that multiple steps or method can be performed in memory and by suitable instruction execution system with storage
Or firmware is realized.Such as, if realized with hardware with another embodiment, following skill well known in the art can be used
Any one of art or their combination are realized：With the logic gates for realizing logic function to data-signal from
Logic circuit is dissipated, the application specific integrated circuit with suitable combinational logic gate circuit, programmable gate array (PGA), scene can compile
Journey gate array (FPGA) etc..
Those skilled in the art are appreciated that to realize all or part of step that above-described embodiment method carries
Suddenly it is that by program the hardware of correlation can be instructed to complete, described program can be stored in a kind of computer-readable storage medium
In matter, the program upon execution, including one or a combination set of the step of embodiment of the method.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing module, can also
That unit is individually physically present, can also two or more units be integrated in a module.Above-mentioned integrated mould
Block can both be realized in the form of hardware, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized in the form of software function module and as independent production marketing or in use, can also be stored in a computer
In read/write memory medium.
Storage medium mentioned above can be read-only storage, disk or CD etc..Although have been shown and retouch above
Embodiments of the invention are stated, it is to be understood that above-described embodiment is exemplary, it is impossible to be interpreted as the limit to the present invention
System, one of ordinary skill in the art can be changed to above-described embodiment, change, replace and become within the scope of the invention
Type.
Claims (10)
- A kind of 1. auth method of lip reading identification, it is characterised in that including：Used to the user's face projective structure light for carrying out lip reading identification, and according to the shooting of default collection period is multiple by current The structure light image of family face modulation；Phase information corresponding to lip position pixel in the multiple structure light image is demodulated, obtains multiple three-dimensional lip images；The lip characteristic information corresponding to extraction from the multiple three-dimensional lip image, and the multiple three-dimensional lip image root According to the lip different information of timing variations；Using the sample characteristics information of default lip reading model library to the lip characteristic information and the lip different information Matched, obtain lip reading information corresponding to the sample characteristics information that the match is successful；By the lip reading information compared with default checking information, if comparative result is identical, checking user identity is legal Authorize corresponding operating.
- 2. the method as described in claim 1, it is characterised in that described to the user's face projective structure for carrying out lip reading identification Before light, in addition to：Face recognition is carried out to user, extracts face feature information；The face feature information is authenticated using default authorization data storehouse, if authentication passes through, prompts user to carry out Lip reading identification checking.
- 3. the method as described in claim 1, it is characterised in that lip position picture in the multiple structure light image of demodulation Phase information corresponding to element, multiple three-dimensional lip images are obtained, including：Demodulate phase information corresponding to deformation position pixel in each structure light image；The phase information is converted into elevation information；User's face 3-D view corresponding with each structure light image is obtained according to the elevation information；Three-dimensional lip image is extracted from the user's face 3-D view.
- 4. the method as described in claim 1, it is characterised in that believe in the sample characteristics using default lip reading model library Before breath matches to the lip characteristic information and the feature difference information, in addition to：Collecting sample information, the sample information include the lip video image and corresponding audio letter of different regions user Breath；The lip video image is analyzed by image processing model and obtains lip characteristic value；The audio-frequency information is analyzed by speech recognition modeling and obtains language message；The sample information is trained by deep neural network model, establishes pair for including the lip characteristic value and language message The lip reading model library that should be related to.
- 5. method as claimed in claim 4, it is characterised in that also include：The average word speed of different regions is obtained according to the audio-frequency information of the different regions user；The collection period for determining to match with active user location according to the average word speed of the different regions.
- A kind of 6. authentication means of lip reading identification, it is characterised in that including：Acquisition module, for being shot to the user's face projective structure light for carrying out lip reading identification, and according to default collection period Multiple structure light images by active user's face modulation；First acquisition module, for demodulating phase information corresponding to lip position pixel in the multiple structure light image, obtain Multiple three-dimensional lip images；Extraction module, it is and the multiple for lip characteristic information corresponding to the extraction from the multiple three-dimensional lip image Three-dimensional lip image is according to the lip different informations of timing variations；Second acquisition module, sample characteristics information for the default lip reading model library of application to the lip characteristic information and The lip different information is matched, and obtains lip reading information corresponding to the sample characteristics information that the match is successful；Authentication module, for the lip reading information compared with default checking information, if comparative result is identical, to be verified User identity legal authorization corresponding operating.
- 7. device as claimed in claim 6, it is characterised in that also include：The extraction module, it is additionally operable to carry out face recognition to user, extracts face feature information；Authentication module, it is additionally operable to authenticate the face feature information using default authorization data storehouse, if authentication passes through, User is then prompted to carry out lip reading identification checking.
- 8. device as claimed in claim 6, it is characterised in that first acquisition module includes：Demodulating unit, for demodulating phase information corresponding to deformation position pixel in each structure light image；Conversion unit, for the phase information to be converted into elevation information；Acquiring unit, for obtaining user's face 3-D view corresponding with each structure light image according to the elevation information；Extraction unit, for extracting three-dimensional lip image from the user's face 3-D view.
- 9. a kind of terminal device, it is characterised in that including memory and processor, stored in the memory computer-readable Instruction, when the instruction is by the computing device so that lip of the computing device as described in claim any one of 1-5 The auth method of language identification.
- 10. a kind of non-transitorycomputer readable storage medium, is stored thereon with computer program, it is characterised in that the calculating The auth method of the lip reading identification as described in claim any one of 1-5 is realized when machine program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710643852.6A CN107437019A (en) | 2017-07-31 | 2017-07-31 | The auth method and device of lip reading identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710643852.6A CN107437019A (en) | 2017-07-31 | 2017-07-31 | The auth method and device of lip reading identification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107437019A true CN107437019A (en) | 2017-12-05 |
Family
ID=60460966
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710643852.6A Pending CN107437019A (en) | 2017-07-31 | 2017-07-31 | The auth method and device of lip reading identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107437019A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108197572A (en) * | 2018-01-02 | 2018-06-22 | 京东方科技集团股份有限公司 | A kind of lip reading recognition methods and mobile terminal |
| CN108564033A (en) * | 2018-04-12 | 2018-09-21 | Oppo广东移动通信有限公司 | Safe verification method, device based on structure light and terminal device |
| CN109409204A (en) * | 2018-09-07 | 2019-03-01 | 北京市商汤科技开发有限公司 | False-proof detection method and device, electronic equipment, storage medium |
| CN110033291A (en) * | 2018-01-12 | 2019-07-19 | 北京京东金融科技控股有限公司 | Information object method for pushing, device and system |
| WO2019218243A1 (en) * | 2018-05-16 | 2019-11-21 | 深圳大学 | Method and device for constructing deep neural network model |
| CN111046704A (en) * | 2018-10-12 | 2020-04-21 | 杭州海康威视数字技术股份有限公司 | Method and device for storing identity identification information |
| CN111931662A (en) * | 2020-08-12 | 2020-11-13 | 中国工商银行股份有限公司 | Lip reading identification system and method and self-service terminal |
| CN112528766A (en) * | 2020-11-25 | 2021-03-19 | 维沃移动通信有限公司 | Lip language identification method and device and electronic equipment |
| CN112672021A (en) * | 2020-12-25 | 2021-04-16 | 维沃移动通信有限公司 | Language identification method and device and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376250A (en) * | 2014-12-03 | 2015-02-25 | 优化科技（苏州）有限公司 | Real person living body identity verification method based on sound-type image feature |
| CN104680375A (en) * | 2015-02-28 | 2015-06-03 | 优化科技（苏州）有限公司 | Identification verifying system for living human body for electronic payment |
| CN105488524A (en) * | 2015-11-26 | 2016-04-13 | 中山大学 | Wearable device based lip language identification method and system |
-
2017
- 2017-07-31 CN CN201710643852.6A patent/CN107437019A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104376250A (en) * | 2014-12-03 | 2015-02-25 | 优化科技（苏州）有限公司 | Real person living body identity verification method based on sound-type image feature |
| CN104680375A (en) * | 2015-02-28 | 2015-06-03 | 优化科技（苏州）有限公司 | Identification verifying system for living human body for electronic payment |
| CN105488524A (en) * | 2015-11-26 | 2016-04-13 | 中山大学 | Wearable device based lip language identification method and system |
Non-Patent Citations (2)
| Title |
|---|
| 曲芳等: "基于数字彩色结构光投影的唇动三维侧脸", 《光学技术》 * |
| 郑志平等: "非接触三维测量中新的相位-高度分析模型", 《四川大学学报》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108197572B (en) * | 2018-01-02 | 2020-06-12 | 京东方科技集团股份有限公司 | Lip language identification method and mobile terminal |
| WO2019134463A1 (en) * | 2018-01-02 | 2019-07-11 | Boe Technology Group Co., Ltd. | Lip language recognition method and mobile terminal |
| CN108197572A (en) * | 2018-01-02 | 2018-06-22 | 京东方科技集团股份有限公司 | A kind of lip reading recognition methods and mobile terminal |
| CN110033291A (en) * | 2018-01-12 | 2019-07-19 | 北京京东金融科技控股有限公司 | Information object method for pushing, device and system |
| CN108564033A (en) * | 2018-04-12 | 2018-09-21 | Oppo广东移动通信有限公司 | Safe verification method, device based on structure light and terminal device |
| WO2019218243A1 (en) * | 2018-05-16 | 2019-11-21 | 深圳大学 | Method and device for constructing deep neural network model |
| CN109409204A (en) * | 2018-09-07 | 2019-03-01 | 北京市商汤科技开发有限公司 | False-proof detection method and device, electronic equipment, storage medium |
| CN111046704A (en) * | 2018-10-12 | 2020-04-21 | 杭州海康威视数字技术股份有限公司 | Method and device for storing identity identification information |
| CN111046704B (en) * | 2018-10-12 | 2023-05-09 | 杭州海康威视数字技术股份有限公司 | Method and device for storing identity identification information |
| CN111931662A (en) * | 2020-08-12 | 2020-11-13 | 中国工商银行股份有限公司 | Lip reading identification system and method and self-service terminal |
| CN112528766A (en) * | 2020-11-25 | 2021-03-19 | 维沃移动通信有限公司 | Lip language identification method and device and electronic equipment |
| CN112672021A (en) * | 2020-12-25 | 2021-04-16 | 维沃移动通信有限公司 | Language identification method and device and electronic equipment |
| CN112672021B (en) * | 2020-12-25 | 2022-05-17 | 维沃移动通信有限公司 | Language identification method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107437019A (en) | The auth method and device of lip reading identification | |
| CN107368730A (en) | Unlock verification method and device | |
| CN107480613A (en) | Face identification method, device, mobile terminal and computer-readable recording medium | |
| CN107479801A (en) | Displaying method of terminal, device and terminal based on user's expression | |
| CN107563304A (en) | Unlocking terminal equipment method and device, terminal device | |
| CN107682607A (en) | Image acquiring method, device, mobile terminal and storage medium | |
| CN107707839A (en) | Image processing method and device | |
| CN107451561A (en) | Iris recognition light compensation method and device | |
| CN107610077A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107493428A (en) | Filming control method and device | |
| CN107509045A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107895110A (en) | Unlocking method, device and the mobile terminal of terminal device | |
| CN107491744A (en) | Human body personal identification method, device, mobile terminal and storage medium | |
| CN107423716A (en) | Face method for monitoring state and device | |
| CN107623814A (en) | The sensitive information screen method and device of shooting image | |
| CN107507269A (en) | Personalized three-dimensional model generating method, device and terminal device | |
| CN107491675A (en) | information security processing method, device and terminal | |
| CN107592449A (en) | Three-dimension modeling method, apparatus and mobile terminal | |
| CN107707831A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107464280A (en) | The matching process and device of user's 3D modeling | |
| CN107590828A (en) | The virtualization treating method and apparatus of shooting image | |
| CN108052813A (en) | Unlocking method, device and the mobile terminal of terminal device | |
| CN107705356A (en) | Image processing method and device | |
| CN107592491A (en) | Video communication background display methods and device | |
| CN107622496A (en) | Image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171205 |
|
| RJ01 | Rejection of invention patent application after publication |