WO2018137595A1

WO2018137595A1 - Face recognition method

Info

Publication number: WO2018137595A1
Application number: PCT/CN2018/073734
Authority: WO
Inventors: 丁贤根
Original assignee: 丁贤根
Priority date: 2017-01-25
Filing date: 2018-01-23
Publication date: 2018-08-02
Also published as: CN107766785B; CN107766785A

Abstract

A face recognition method, comprising the steps of collecting a face video, an image sequence and audio of a user, steps of recognizing a face movement and an access object identification, and steps of outputting a movement instruction and accessing confidential information according to a confidential relationship, thereby driving instruction and privacy confidentiality of a host system by means of the face movement and implementing living authentication for account codes of the user. With the present invention, inadvertent movements such as blinking may be recognized; and intentional movements such as blinking, frowning, opening one's mouth and nodding may also be recognized. At the same time, a recognition method by semantic instruction is provided, and said movements are used to drive various functions of a host subsystem. In addition, a confidential relationship having a one-to-one correspondence is established between the access object and accessed information, so that the confidential information is confidential to the access object on the basis of the face recognition. An artificial intelligence network algorithm, fuzzy recognition algorithm and genetic algorithm are used to recognize face movements. The present invention is used for confidential social interaction, confidential privacy, virtual reality (VR), augmented reality (AR), mixed reality (MR), helmet operation, financial payment, a game and a mobile phone APP.

Description

一种面部识别方法Facial recognition method

技术领域Technical field

本发明涉及人脸识别技术，尤其是涉及一种面部识别方法，一种基于人脸识别与人工智能技术实现表情识别并实现后续驱动的方法，属于信息处理技术领域。The invention relates to a face recognition technology, in particular to a face recognition method, a method for realizing facial expression recognition and realizing subsequent driving based on face recognition and artificial intelligence technology, and belongs to the technical field of information processing.

背景技术Background technique

随着人脸识别与人工智能的发展，认人这一工作也越来越成熟。百度推出的人工智能人脸识别，在很多方面已经超过人类的水平。但是作为使用人脸的表情动作的识别以及驱动智能化设备，目前的研究及成果尚数鲜见。With the development of face recognition and artificial intelligence, the work of identifying people is becoming more and more mature. Baidu's artificial intelligence face recognition has surpassed human standards in many aspects. However, as the identification of facial expressions and the driving of intelligent devices, the current research and results are rare.

本发明人对于现有技术进行了大量的研究发现：The inventors conducted a large number of research findings on the prior art:

1、基于人脸识别的技术虽然很多，但都是通过人脸识别来确定用户身份。1. Although there are many techniques based on face recognition, they all identify the user's identity through face recognition.

2、中国专利申请201510297334.4披露了一种基于人脸表情动作识别的技术，其核心特征是云服务器端接受用户的表情照片作为账户注册，然后用于账户验证。另一个中国专利申请201210061716.3，也是用于云服务器端接受用户发送的表情再予以验证。2. Chinese patent application 201510297334.4 discloses a technique based on facial expression motion recognition, the core feature of which is that the cloud server accepts the user's expression photo as an account registration and then uses it for account verification. Another Chinese patent application 201210061716.3 is also used for the cloud server to accept the expression sent by the user and then verify.

3、中国专利申请201410301696.1披露了基于眼球运动检测的技术，其核心特征是通过拍摄眼睛前后的图片，来判断眼睛的运动方向，从而实现翻页功能。3. Chinese patent application 201410301696.1 discloses a technique based on eye movement detection, whose core feature is to determine the direction of movement of the eye by photographing the picture before and after the eye, thereby realizing the page turning function.

4、中国专利申请201410227002.4披露了一种眼睛开闭状态检测装置和方法，其核心特征是通过眼睛的照片来判断眼睛的开闭，提出计算方法。4. Chinese Patent Application No. 201410227002.4 discloses an eye opening and closing state detecting device and method, the core feature of which is to judge the opening and closing of the eye through the photograph of the eye, and propose a calculation method.

因此，现有技术中存在如下的不足：没有使用人脸面部动作来操作；没有采用一一对应的保密关系；社交聊天是没有利用人脸动作来增加趣味性。Therefore, the prior art has the following disadvantages: no facial gestures are used for operation; no one-to-one correspondence is used; social chat does not use facial gestures to increase interest.

现有技术之所以存在上述不足，一方面，是因为应用的推动尚没有到来。以智能手机APP为例：现在还是处在替代PC软件的末期，一个全新的、完全脱胎换骨的人工智能时代尚处于萌芽阶段；另外一方面，基于人脸表情动作的识别技术也刚刚引起创新者的注意，这方面的研究成果很少，相关的技术信息也非常少；第三，本领域技术人员现阶段的注意力还是集中在通过人脸来识别人的身份，而不是识别人的意图。The above-mentioned deficiencies of the prior art are, on the one hand, because the application of the promotion has not yet arrived. Take the smart phone APP as an example: it is still at the end of the replacement PC software. A new and completely reborn artificial intelligence era is still in its infancy; on the other hand, recognition technology based on facial expressions has just caused innovators. Note that the research results in this area are few and the related technical information is very small. Thirdly, the attention of the technicians at this stage is still focused on identifying the identity of the person through the face, rather than identifying the intention of the person.

本发明人经过长期的研究认为：人脸的表情动作，例如：眨眼、皱眉、张嘴、点头、摇头灯动作，不仅是人们习惯的表情方式，更是符合人类的心理活动的表达。再进一步，语义识别，更是最直接表达人类意图的最佳方式。如果能够较好地识别它们，在很多场景中，将会改善对于设备的操作。例如智能头盔，可以让驾驶员凭借这项技术，就可以解放手脚；在隐私保密和社交保密场合，这项技术也能够改善对于隐私的保护。After long-term research, the inventor believes that facial expression movements, such as blinking, frowning, opening mouth, nodding, and moving headlights, are not only the expressions that people are used to, but also the expression of human psychological activities. Further, semantic recognition is the best way to express human intentions most directly. If you can identify them better, in many scenarios, the operation of the device will be improved. For example, smart helmets allow drivers to use their technology to liberate their hands and feet; in privacy and social privacy, this technology also improves privacy.

发明内容Summary of the invention

本发明的目的是提供一种面部识别方法，通过对面部视频或图像序列，识别面部动作，以此去驱动宿主***的下一步指令。It is an object of the present invention to provide a face recognition method for recognizing a facial motion by recognizing a facial motion or a sequence of images, thereby driving the next instruction of the host system.

本发明的目的是采用如下技术方案实现的：The object of the invention is achieved by the following technical solutions:

一种面部识别方法，以下简称本方法，它至少包含：A facial recognition method, hereinafter referred to as the method, which at least includes:

采集人脸视频和/或人脸图像序列的步骤；a step of collecting a sequence of face videos and/or face images;

依据所述人脸视频和/或人脸图像序列识别面部动作的步骤；The step of recognizing a facial motion according to the sequence of face video and/or face image;

基于所述面部动作输出驱动指令的步骤。A step of outputting a drive command based on the facial motion.

在前述技术方案的基础上，在本发明的另一些方案中可以采用如下列的一种或者多种局部改进的措施：前述采集人脸视频和/或人脸图像序列的采样频率不小于每秒5帧。Based on the foregoing technical solutions, in some other aspects of the present invention, one or more of the following partial improvement measures may be adopted: the sampling frequency of the foregoing collected face video and/or face image sequence is not less than 5 frames.

对于本方法所涉及到的程序、数据的存储，包括但不限于本地存储、网络存储、云计算存储；对于程序的实现模式，包括单机模式、C/S模式、B/S模式和SaaS模式。For the storage of programs and data involved in the method, including but not limited to local storage, network storage, cloud computing storage; for the implementation mode of the program, including stand-alone mode, C/S mode, B/S mode and SaaS mode.

作为智能手机的APP应用，利用手机正面的摄像机采集用户的人脸视频，并且识别和驱动指令，是本方法的选择之一。As an APP application for a smart phone, capturing the user's face video using the camera on the front of the mobile phone and identifying and driving the command is one of the choices of the method.

所述依据所述人脸视频和/或人脸图像序列识别面部动作的步骤，包括但不限于以下步骤以及它们的组合：The step of recognizing a facial motion according to the sequence of facial video and/or facial image includes, but is not limited to, the following steps and combinations thereof:

过滤无意识动作S14；Filtering the unconscious action S14;

报警疲劳动作S15、报警异常动作S16和/或S17；Alarm fatigue action S15, alarm abnormal action S16 and / or S17;

执行有意识动作S1、S2-S12和/或S13；Performing conscious actions S1, S2-S12 and/or S13;

其中：among them:

S1为左眼眨动动作，周期大于T1并且小于T2；例如驱动“左划屏”；S1 is a left eye swaying action, the period is greater than T1 and less than T2; for example, driving "left screen";

S2为右眼眨动动作，周期大于T1并且小于T2；例如驱动“右划屏”；S2 is a right eye swaying action, the period is greater than T1 and less than T2; for example, driving "right screen";

S3为双眼眨动动作，周期大于T1并且小于T2；例如驱动“关闭显示或换屏保”；S3 is a binocular swaying action, the period is greater than T1 and less than T2; for example, driving "turn off display or change screen saver";

S4为皱眉动作，周期大于T1并且小于T4；例如驱动“增加音量”；S4 is a frowning action, the period is greater than T1 and less than T4; for example driving "increasing the volume";

S5为张嘴动作，周期大于T1并且小于T4；例如驱动“减小音量”；S5 is a mouth opening action, the period is greater than T1 and less than T4; for example driving "reduced volume";

S6为点头动作，周期大于T1并且小于T4；例如驱动“确认”；S6 is a nodding action, the period is greater than T1 and less than T4; for example, driving "confirm";

S7为仰头动作，周期大于T1并且小于T4；例如驱动“上划屏”；S7 is an upward movement, the period is greater than T1 and less than T4; for example, driving "upper screen";

S8为低头动作，周期大于T1并且小于T4；例如驱动“下划屏”；S8 is a bowing action, the period is greater than T1 and less than T4; for example, driving "down screen";

S9为摇头动作，周期大于T1并且小于T4；例如驱动“取消”；S9 is a shaking action, the period is greater than T1 and less than T4; for example, driving "cancel";

S10为左向偏头动作，周期大于T1并且小于T4；例如驱动“后悔”；S10 is a leftward biasing action, the period is greater than T1 and less than T4; for example, driving "repent";

S11为右向偏头动作，周期大于T1并且小于T4；例如驱动“向前”；S11 is a rightward biasing action, the period is greater than T1 and less than T4; for example driving "forward";

S12为拉左脸肌肉动作，周期大于T1并且小于T4；例如驱动“左划屏”；S12 is a motion of pulling the left face muscle, the period is greater than T1 and less than T4; for example, driving "left screen";

S13为拉右脸肌肉动作，周期大于T1并且小于T4；例如驱动“右划屏”；S13 is a motion of pulling the right face muscle, the period is greater than T1 and less than T4; for example, driving "right screen";

S14为双眼眨眼动作，周期小于T0；过滤，不执行任何驱动；S14 is a blinking operation of both eyes, the period is less than T0; filtering, no driving is performed;

S15为呵欠动作，周期大于T3并且小于T4；疲劳报警；S15 is a yawning action, the period is greater than T3 and less than T4; fatigue alarm;

S16为双眼闭眼动作，时间大于T5；异常情况报警；S16 is a closed-eye action of both eyes, the time is greater than T5; abnormal condition alarm;

S17为张嘴动作，时间大于T6；异常情况报警；S17 is a mouth opening action, the time is greater than T6; abnormal condition alarm;

其中包含周期长短关系是：T6≥T5≥T4≥T3≥T2≥T1≥T0；The relationship between the length and the length of the cycle is: T6≥T5≥T4≥T3≥T2≥T1≥T0;

所述T0是在所述人脸视频和/或人脸图像序列中定位及计算眼睛完成一次无意识自然眨眼次动作从开始到结束的时间；根据人的生理反应敏感度，该时间值设定在0.1秒到0.4秒之间。The T0 is a time for locating and calculating an unconscious natural blinking action of the eye from the start to the end in the sequence of the face video and/or the face image; the time value is set according to the physiological response sensitivity of the person. Between 0.1 seconds and 0.4 seconds.

所述T1是在所述人脸视频和/或人脸图像序列中定位及计算眼睛、眉部、嘴部、头部、脸部肌肉完成一次有意识动作从开始到结束的最短时间；该时间应该大于人的生理反应敏感度，并且，作为一个有意识动作，优化的区间应该为0.4秒到1秒。The T1 is a minimum time for locating and calculating the conscious movement of the eye, the eyebrow, the mouth, the head, and the facial muscle from the beginning to the end in the sequence of the face video and/or the face image; the time should be Greater than human physiological response sensitivity, and, as a conscious action, the optimal interval should be 0.4 seconds to 1 second.

所述T2是在所述人脸视频和/或人脸图像序列中定位及计算眼睛完成一次有意识动作从开始到结束的最长时间；根据人的生理及心理反应，确定该时间为0.6秒到1.5秒。The T2 is to locate and calculate the maximum time from the start to the end of the conscious action of the eye in the sequence of the face video and/or the face image; according to the physiological and psychological reaction of the person, the time is determined to be 0.6 seconds. 1.5 seconds.

所述T3是在所述人脸视频和/或人脸图像序列中定位及计算完成一次有意识的眉部、嘴部、头部、脸部肌肉动作从开始到结束的最长时间；根据人的生理及心理反应，确定该时间为0.6秒到1.5秒。The T3 is a maximum time for locating and calculating a conscious eyebrow, mouth, head, and facial muscle action from the beginning to the end in the sequence of face video and/or face images; Physiological and psychological responses were determined to be between 0.6 seconds and 1.5 seconds.

所述T4是在所述人脸视频和/或人脸图像序列中定位及计算头部完成一次打哈欠动作从开始到结束的最短时间；根据人的生理及心理反应，确定该时间为0.7秒到2.5秒。The T4 is a shortest time from the start to the end of the face video and/or face image sequence to locate and calculate the head yawning action; the time is determined to be 0.7 seconds according to the physiological and psychological reaction of the person. In 2.5 seconds.

所述T5、T6分别是在所述人脸视频和/或人脸图像序列中定位及计算完成一次因疲劳闭眼和张嘴打瞌睡动作从开始到结束的最短时间。这个时间值设定的意图是判断人出现异常情况，例如睡眠、过度疲劳而产生的失意状态，因此确定为大于2.5秒。The T5 and T6 are respectively the shortest time in the face video and/or the face image sequence to locate and calculate the time from the start to the end of the fatigue closed eye and the mouth open doze action. The purpose of this time value setting is to judge the person's abnormal situation, such as sleep, excessive fatigue, and the frustrated state, so it is determined to be greater than 2.5 seconds.

在前述技术方案的基础上，还包括但并不限于音频采集和语音识别、语言指令识别步骤；所述语音识别，是对于所述音频采集的信号进行人类语音的识别，排除环境噪音信号；所述语言指令识别，是对于音频信号进行语义识别，以识别出所述用户的语言指令；所述语音识别和语言指令识别，其识别方法包括并不限于：滤波算法、人工智能网络识别算法、模糊识别算法；基于设定，包括并不限于识别：On the basis of the foregoing technical solutions, the method further includes, but is not limited to, an audio collection and speech recognition, a language instruction recognition step; the speech recognition is to identify a human voice for the signal collected by the audio, and exclude an environmental noise signal; The language instruction recognition is to perform semantic recognition on the audio signal to identify the language instruction of the user; the voice recognition and the language instruction recognition, and the recognition method thereof is not limited to: a filtering algorithm, an artificial intelligence network recognition algorithm, and a fuzzy Identification algorithm; based on settings, including and not limited to identification:

V0为说话动作，张嘴周期大于所述T1并且小于所述T3，并且同时伴有语音产生；V0 is a speaking action, the mouth opening period is greater than the T1 and smaller than the T3, and accompanied by voice generation;

V1为“关闭”语言指令，例如关闭文件、关闭显示；V1 is a "close" language command, such as closing a file and closing the display;

V2为“打开”语言指令，例如打开文件、打开显示；V2 is an "open" language command, such as opening a file and opening the display;

V3为“向上”语言指令，例如光标向上移动、屏显内容向上移动；V3 is an "up" language command, such as the cursor moving up and the screen display content moving up;

V4为“向下”语言指令，例如光标向下移动、屏显内容向下移动；V4 is a "down" language command, such as the cursor moving downwards and the screen display content moving downward;

V5为“向左”语言指令，例如光标向左移动、屏显内容向左移动；V5 is a "left" language command, for example, the cursor moves to the left and the screen display content moves to the left;

V6为“向右”语言指令，例如光标向右移动、屏显内容向右移动；V6 is a "rightward" language command, for example, the cursor moves to the right and the screen display content moves to the right;

V7为“退出”语言指令，例如退出文件、退出屏显、退出聊天；V7 is an "exit" language command, such as exiting a file, exiting an on-screen display, and exiting a chat;

V8为“删除”语言指令，例如删除文件、删除屏显、删除聊天记录；V8 is a “delete” language command, such as deleting a file, deleting an on-screen display, and deleting a chat record;

V9为“隐藏”语言指令，例如隐藏文件、隐藏屏显、隐藏聊天记录；V9 is a "hidden" language command, such as hiding files, hiding screens, and hiding chat records;

V10为“后悔”语言指令，例如后悔上一步操作；V10 is a "repentance" language instruction, such as regretting the previous operation;

V11为“向前”语言指令，例如向前(即反后悔)一步操作；V11 is a "forward" language instruction, such as a forward (ie, counter-repent) one-step operation;

V12-Vn为用户自定义语言指令，n为根据用户的意愿设定值；V12-Vn is a user-defined language command, and n is a set value according to the user's wishes;

通过智能化的语义训练和识别，用户可以实现更多的个性化的、带有方言和个人发音特征的指令。Through intelligent semantic training and recognition, users can implement more personalized instructions with dialect and personal pronunciation features.

所述依据所述人脸视频和/或人脸图像序列识别面部动作的步骤包括并不限于：采用人工智能网络算法、模糊识别算法和/或遗传算法，对所述人脸视频和/或人脸图像序列识进行包括并不限于图像尺寸归一化、滤波与特征提取、抗干扰处理、视频稳像、运动估算、深度学习、训练矫正、遗传矫正和/或贝叶斯矫正操作。The step of recognizing the facial motion according to the sequence of the face video and/or the face image includes, without limitation, using an artificial intelligence network algorithm, a fuzzy recognition algorithm, and/or a genetic algorithm, the face video and/or the person Face image sequence recognition includes, but is not limited to, image size normalization, filtering and feature extraction, anti-interference processing, video stabilization, motion estimation, deep learning, training correction, genetic correction, and/or Bayesian correction operations.

所述图像尺寸归一化，包括并不限于把采集到的所述人脸视频和/或人脸图像序列中人脸部分的尺寸，进行放大或者缩小，得到尺寸统一的视频和/或图像，以便于后续进一步的识别步骤。The image size is normalized, including, without limitation, enlarging or reducing the size of the face portion of the collected face video and/or face image sequence to obtain a uniform size video and/or image. In order to facilitate subsequent identification steps.

在前述技术方案的基础上，还包括并不限于：依据所述人脸视频和/或人脸图像序列识别用户身份的步骤；包括并不限于：In addition to the foregoing technical solutions, the method further includes: a step of identifying a user identity according to the sequence of the face video and/or the face image; including but not limited to:

采用人工智能网络算法、模糊识别算法和/或遗传算法，对所述人脸视频和/或人脸图像序列识进行包括并不限于图像尺寸归一化、滤波与特征提取、抗干扰处理、视频稳像、运动估算、深度学习、训练矫正、遗传矫正和/或贝叶斯矫正操作。The artificial intelligence network algorithm, the fuzzy recognition algorithm and/or the genetic algorithm are used to include the face video and/or the face image sequence, and is not limited to image size normalization, filtering and feature extraction, anti-interference processing, video. Image stabilization, motion estimation, deep learning, training correction, genetic correction, and/or Bayesian correction.

在前述技术方案的基础上，还包括并不限于：基于设定，将通过识别出的用户标识为公开对象和/或保密对象。On the basis of the foregoing technical solutions, the method further includes, without limitation, identifying the identified user as a public object and/or a confidential object based on the setting.

还包括在社交通信***的应用中，发送所述面部动作种类和/或语义文字给社交通信***中的对方，并且在对方显示器上用动画复现面部动作的步骤；Also included in the application of the social communication system, the step of transmitting the facial action category and/or semantic text to a counterpart in the social communication system, and animating the facial motion on the other party's display;

在本方法用在隐私保密和社交保密的***设计时，在访问对象和信息和/或社交对象之间，建立一一对应的保密关系；对于不需要保密的信息和/ 或社交对象，任何访问对象都可以访问，并且把这部分的访问对象标识为公开对象；而需要保密的信息和/或社交对象，将其对应的访问对象标识为保密对象；根据保密关系，任何需要保密的信息和/或社交对象，都有对应的保密对象。In the design of the system for privacy and social security, a one-to-one correspondence is established between the access object and the information and/or social objects; for any information and/or social objects that do not require confidentiality, any access Objects can be accessed, and the access objects of this part are identified as public objects; and confidential information and/or social objects are required to identify their corresponding access objects as confidential objects; according to confidentiality information, any information that needs to be kept secret and/or Or social objects, there are corresponding confidential objects.

由于此项设定，作为公开对象，只能够访问所述保密关系中不需要保密的信息和/或社交对象，不能够访问需要保密的信息和/或社交对象；而作为保密对象，不能够访问所述保密关系中对他需要保密的信息和/或社交对象，只能够访问对他不需要保密的信息和/或社交对象。Due to this setting, as a public object, only information and/or social objects that do not need to be kept secret in the confidential relationship can be accessed, and information and/or social objects that require confidentiality cannot be accessed; and as a confidential object, access is not possible. In the confidential relationship, information and/or social objects that he needs to be kept secret can only access information and/or social objects that he does not need to keep secret.

在前述技术方案的基础上，还包括并不限于：基于设定，对于数据的存储、传输和/或显示进行加密和/或解密的步骤；其中：所述加密和/或解密步骤采用的算法包括并不限于：DES、3DES、AES、RC2、RC4、IDEA、RSA、DSA、ECC、BLOWFISH、KPCS、DM5、SHA、SSF33，SSF28，SCB2和/或SM系列的任意组合。The foregoing technical solution further includes, without limitation, a step of encrypting and/or decrypting data storage, transmission, and/or display based on a setting; wherein: the algorithm used in the encrypting and/or decrypting step Included are not limited to any combination of: DES, 3DES, AES, RC2, RC4, IDEA, RSA, DSA, ECC, BLOWFISH, KPCS, DM5, SHA, SSF33, SSF28, SCB2, and/or SM series.

还包括并不限于，基于设定，所述方法还包括网络***以及网络***对于终端客户的账户和/或密码进行活体认证的步骤，以达到账户自动识别、密码自动识别、账户和密码均自动识别的效果；所述活体认证，至少包括：The method further includes, without limitation, based on the setting, the method further includes the steps of the network system and the network system performing live authentication on the account and/or password of the terminal client, so as to automatically identify the account, automatically identify the account, and automatically update the account and the password. The effect of the recognition; the living body certification includes at least:

用户使用所述网络***为自己建立身份信息和动作编码组合并且保存到所述网络***中存在的用户账户数据库中；所述身份信息是通过所述网络***采集用户的所述人脸视频和/或人脸图像序列、识别所述该用户身份和的结果数据，所述动作编码组合是用户设定的若干面部动作结果数据的组合，例如S1+S1+S5；这一步骤是在此前用户正常输入账户和密码后设定完成的；The user uses the network system to establish identity information and action coding combination for itself and saves it to a user account database existing in the network system; the identity information is that the user's face video and/or are collected by the network system. Or a sequence of face images, the result data identifying the identity of the user, the action coding combination is a combination of a plurality of facial action result data set by the user, for example, S1+S1+S5; this step is normal before the user After the account and password are entered, the settings are completed;

所述网络***提供所述活体认证界面，所述用户通过该界面，重现步骤1，进入其账户和/或重新设定该账户的密码。The network system provides the biometric authentication interface through which the user reproduces step 1, enters his account and/or resets the password of the account.

在前述技术方案的基础上，还包括并不限于利用驱动指令对宿主设备下一步操作提供驱动的步骤；具体包括：利用驱动指令提供程序函数代码，以使宿主软件能够根据所述程序函数代码进行判断并执行下一步功能的；和/或，On the basis of the foregoing technical solutions, the method further includes the step of providing a driver for the next operation of the host device by using the driving instruction; and specifically: providing the program function code by using the driving instruction, so that the host software can perform the function according to the program function code. Judge and perform the next function; and/or,

利用驱动指令提供控制接口的信号，以使宿主设备根据所述信号，执行后续的操作。The drive command is used to provide a signal to the control interface to cause the host device to perform subsequent operations in accordance with the signal.

基于上述方法的组合使用，还包括并不限于隐私保密应用、社交保密应用、XR和头盔操作应用、金融支付应用、手机APP应用、电子游戏应用，具体包括并不限于以下步骤：Based on the combined use of the foregoing methods, the method further includes, without limitation, a privacy application, a social security application, an XR and a helmet operation application, a financial payment application, a mobile APP application, and an electronic game application, and specifically includes the following steps:

对于所述隐私保密应用，基于设定，在用户和隐私信息之间建立一一对应的保密关系，通过所述识别用户身份，查找并访问所述保密关系中允许该用户访问的隐私信息，包括使用所述S1-S13以及它们的组合去驱动操作功能；和/或，For the privacy application, based on the setting, a one-to-one correspondence relationship between the user and the private information is established, and the private information that allows the user to access in the confidential relationship is searched for and accessed through the identification of the user identity, including Using the S1-S13 and combinations thereof to drive operational functions; and/or,

对于所述社交保密的步骤包括：基于设定，在用户和社交对象之间建立一一对应的保密关系；把社交对象分成公开对象和保密对象；通过所述识别用户身份，查找并访问所述保密关系中允许该用户访问的保密社交对象；在用户跟保密对象通信时，使用所述S1-S13以及它们的组合去驱动保密和操作功能；所述社交对象包括社交对象个人和群组；所述动作识别的结果还可以发送给社交对象，在社交对象和/或用户自己的界面上采用动画和音效复现；设定眨双眼为关闭或更换保密对象的屏幕显示、眨左眼为切换上一个保密社交对象、眨右眼为切换下一个保密社交对象、点头为确认，摇头为取消；和/或，The step of the social secrecy includes: establishing a one-to-one correspondence relationship between the user and the social object based on the setting; dividing the social object into a public object and a secret object; and searching and accessing the a secret social object that allows the user to access in a secret relationship; use the S1-S13 and combinations thereof to drive privacy and operational functions when the user communicates with the secure object; the social object includes social object individuals and groups; The result of the motion recognition can also be sent to the social object, and the animation and sound effect are reproduced on the social object and/or the user's own interface; setting the screen to close or replace the secret object, and the left eye is switching. a confidential social object, 眨 right eye to switch to the next secret social object, nod to confirm, shake head to cancel; and / or,

对于所述XR和头盔操作应用，基于设定，在所述头盔内测安装摄像头，拍摄面部视频图像；在所述头盔中安装运动传感器，用于替代所述S6-S11，并和S1-S5、S10-S13及其它们的组合，用于控制连接设备的动作、显示XR内容，S15用于疲劳报警，S16、S17用于异常报警，过滤S14和S18；和/或，For the XR and helmet operating applications, based on settings, a camera is installed within the helmet to capture a facial video image; a motion sensor is mounted in the helmet for replacing the S6-S11, and S1-S5 , S10-S13 and combinations thereof, for controlling the action of the connected device, displaying the XR content, S15 for fatigue alarm, S16, S17 for abnormal alarm, filtering S14 and S18; and/or,

对于所述金融支付***，基于设定，采用所述识别用户身份来完成身份验证，使用所述S1-S13以及它们的组合去驱动账户操作功能；和/或，For the financial payment system, based on the setting, using the identified user identity to complete the identity verification, using the S1-S13 and combinations thereof to drive the account operation function; and/or,

对于所述手机APP应用，基于设定，采用所述S1-S13以及它们的组合去驱动APP操作功能；和/或，For the mobile APP application, based on the settings, the S1-S13 and combinations thereof are used to drive the APP operation function; and/or,

对于所述电子游戏应用，所述S1-S13以及它们的组合去驱动指定的动作。For the video game application, the S1-S13 and combinations thereof drive the specified action.

本方法的实施方式包含软件方式、硬件方式和软硬件融合方式；所述软件方式，是指本方法功能的实现，是运行在宿主平台上的软件；所述硬件方式，是指本方法功能的实现，是连接到宿主平台上的一些硬件，包含通用集成电路和/或专用集成电路和/或常规电子元器件，所述专用集成电路是指专门为本***设计的集成电路；所述软硬件融合方式，是指包含通用集成电路和/或专用集成电路和/或常规电子元器件，以及运行在这些电路上的和/或宿主平台上的软件；所述专用集成电路包含在可编程器件上按照本***功能进行编程后所形成的专用集成电路和/或按照本***功能进行设计的SoC芯片和/或上述专用集成电路以及SoC芯片的组合。The implementation method of the method includes a software mode, a hardware mode, and a software and hardware fusion mode; the software mode refers to the implementation of the function of the method, and is software running on a host platform; the hardware mode refers to the function of the method. Implementation is a piece of hardware connected to a host platform, including a general-purpose integrated circuit and/or an application-specific integrated circuit and/or a conventional electronic component, which refers to an integrated circuit designed specifically for the system; By way of fusion, it is meant to include general purpose integrated circuits and/or application specific integrated circuits and/or conventional electronic components, as well as software running on these circuits and/or host platforms; the application specific integrated circuits are included on the programmable devices An ASIC formed after programming according to the functions of the system and/or a SoC chip designed according to the functions of the system and/or a combination of the above-mentioned ASIC and SoC chip.

与现有技术相比，本发明具有以下有益效果：Compared with the prior art, the present invention has the following beneficial effects:

实现人脸面部动作识别，实现人脸表情的识别与判断。The face and face motion recognition is realized, and the recognition and judgment of the facial expression are realized.

实现面部动作驱动，解放了手脚，使得应用更加智能化。Achieve facial motion drive, freeing hands and feet, making the application more intelligent.

实现面向对象的保密，有利于更加细致地保密隐私。Achieving object-oriented confidentiality helps to maintain privacy in more detail.

用人工智能改善用户体验，应用更加灵活和增加趣味性。Use artificial intelligence to improve the user experience, and the application is more flexible and fun.

附图说明DRAWINGS

图1：是本发明主题的示例性说明；Figure 1: is an exemplary illustration of the subject matter of the present invention;

图2：是本发明手机用户的前摄像机摄像示意图；2 is a schematic diagram of the front camera of the mobile phone user of the present invention;

图3：是本发明人脸视频及人脸图像序列中眨眼动作示例性图；3 is an exemplary diagram of blinking motion in a face video and a face image sequence of the present invention;

图4：是本发明面部动作识别的示例性识别流程图；4 is an exemplary identification flow chart of facial motion recognition of the present invention;

图5：是本发明驱动指令执行的中断方式的示例性中断流程图；Figure 5: is an exemplary interrupt flow diagram of the interrupt mode of the drive instruction execution of the present invention;

图6：是本发明访问对象和访问信息之间的保密关系图；Figure 6 is a diagram showing the confidentiality relationship between the access object and the access information of the present invention;

图7：是本发明保密访问的示例性保密流程图；Figure 7: is an exemplary secure flow diagram of the secure access of the present invention;

图8：是本发明执行动作指令的流程图；Figure 8 is a flow chart showing the execution of an action instruction of the present invention;

图9：是本发明执行活体认证的流程图。Figure 9 is a flow chart of the present invention for performing biometric authentication.

具体实施方式detailed description

为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请具体实施例及相应的附图对本申请技术方案进行描述。The technical solutions of the present application will be described in conjunction with the specific embodiments of the present application and the corresponding drawings.

本发明的一类具体实施方式如下：A specific embodiment of the invention is as follows:

具体实施方式一、面部动作驱动和隐私保护的社交***常规的社交***，如微信，在一次认证通过后就再也没有其他认证了。因此，只要是进入了一个微信号，任何人就可以查看所有的聊天信息和跟任何聊天对象聊天。这种设计，虽然使用方便，但是带来的问题是丝毫没有隐私可言，更无保护隐私的办法。另外一方面，作为隐私动作指令或者基于趣味性的考虑，人脸的动作识别和驱动也是一件有价值的设想。 DETAILED DESCRIPTION 1. Social System for Facial Action Driven and Privacy Protection Conventional social systems, such as WeChat, have no other authentication after a pass. Therefore, as long as a micro signal is entered, anyone can view all the chat information and chat with any chat object. This design, although easy to use, has the problem of no privacy at all and no privacy protection. On the other hand, as a privacy action instruction or based on interesting considerations, the recognition and driving of the face is also a valuable idea.

本发明的这个实施例就是试图设计一种采用面部动作驱动来实现操作，采用人脸身份识别来实现隐私保护。This embodiment of the present invention is an attempt to design a method that uses facial motion driving to implement operations, and uses face recognition to implement privacy protection.

本实施方式是本发明面向智能手机的面部动作驱动和隐私保护的社交***(以下简称“社交***”，如微信、QQ、Facebook等)的示例性例子。The present embodiment is an illustrative example of a social system (hereinafter referred to as "social system" such as WeChat, QQ, Facebook, etc.) for the facial motion driving and privacy protection of the smartphone of the present invention.

该社交***是基于目前常用的智能手机上开发的一套社交软件***，或者是目前流行的社交软件的修改版，修改的内容包括以下的基本要求：The social system is based on a set of social software systems developed on the currently used smartphones, or a modified version of the currently popular social software, the revised content includes the following basic requirements:

1、200图2中，201是手机，202是手机上的正面摄像头，203是用户的人头示意，204是用户的眼睛，205是用户的嘴巴，还有其他，例如：眉毛、面部肌肉、头部轮廓。在社交***中，202在需要的时候，实时地在软件后台监视用户头像，要求用户尽可能将201手机放在用户的正面位置，以便于摄像机能够较好地拍摄到用户的面部。1, 200 Figure 2, 201 is the mobile phone, 202 is the front camera on the mobile phone, 203 is the user's head, 204 is the user's eyes, 205 is the user's mouth, and others, such as: eyebrows, facial muscles, head Outline. In the social system, 202 monitors the user's avatar in the software background in real time when needed, and requires the user to place the 201 mobile phone as far as possible on the user's front position, so that the camera can better capture the user's face.

2、社交***包括依照本发明设计的面部动作识别子***，它是采用人工智能特征设计的。2. The social system includes a facial motion recognition subsystem designed in accordance with the present invention, which is designed using artificial intelligence features.

第一，将摄像头对于人脸的视频或图像采样频率设定在每秒5～30帧，为了保证采样的时间精度，建议按照PAL制式的每秒25帧或者按照NTSC制式的每秒30帧采集视频数据。First, set the video or image sampling frequency of the camera to the face to 5 to 30 frames per second. In order to ensure the time precision of sampling, it is recommended to collect 25 frames per second according to the PAL system or 30 frames per second according to the NTSC system. Video data.

第二，在视频或图像处理时，先捕捉人脸区域，用一个矩形框子套住人脸区域，然后做尺寸归一化处理。所述尺寸归一化处理，是由于随着人脸离摄像头的距离远近的不同，所采集的人脸尺寸也有所不同，为了方便后续的图像或图像处理，所以确定一个标准尺寸。当所采集的图像人脸区域大于这个尺寸时，就缩小图像，达到标准尺寸；反之就放大图像，以达到标准尺寸。所述尺寸，这里是按照像素为单位，建议以最小200×200、最大200×300为准。Second, in video or image processing, first capture the face area, cover the face area with a rectangular frame, and then do the size normalization. The size normalization process is because the size of the collected face is different depending on the distance of the face from the camera, and a standard size is determined in order to facilitate subsequent image or image processing. When the captured image face area is larger than this size, the image is reduced to a standard size; otherwise, the image is enlarged to achieve a standard size. The size, here in units of pixels, is recommended to be a minimum of 200 x 200 and a maximum of 200 x 300.

第三，滤波与特征提取，对于上述处理后的视频或图像，对于每一帧图像，均做灰度处理，去掉彩色，仅保留黑白灰度值，再依据现有公知共用的图像特征提取方法，例如LBP(Local Binary Pattener)方法，提取出眉毛、眼睛、嘴巴、颧骨肌肉、头部轮廓的特征。Third, filtering and feature extraction, for the above-mentioned processed video or image, for each frame of image, grayscale processing is performed, color is removed, only black and white grayscale values are retained, and then according to the conventionally known shared image feature extraction method For example, the LBP (Local Binary Pattener) method extracts features of eyebrows, eyes, mouth, sacral muscles, and head contours.

第四，依据图像帧序列的特征变化，识别动作。如300图3所示，从 301到309，上下眼睑的缝隙由大到小再由小到大，可以看出人眼是在做眨眼动作。以此类推，可识别出嘴巴的张合。将从301到309所经历的时间值作为动作周期T。对于眼睛，由于人类的自然眨眼不是一个有意识的行为，这些必须予以过滤或者忽略。由医学数据中可以看出：人类自然眨眼的周期值通常在0.2秒到0.4秒。于是，我们确定T0＝0.4秒。为了区别有意识眨眼和自然眨眼，我们设定所述有意识眨眼周期为T1＝0.8秒，并且以此作为有意识动作的时间下线，包括皱眉、张嘴、点头、仰头、低头、摇头、偏头、颧骨肌肉等动作。由于人对于时间的记忆是模糊的，所以，有意识动作的时间上限T2需要拉开一些距离，这里确定T2＝1.5秒。T3是打哈欠周期的最短时间，这个时间是因人而异，这里建议T3＝1.5秒。T4是打哈欠周期的最长时间，基于同样的理由，这里建议T4＝2.5秒。Fourth, the action is identified according to the feature change of the image frame sequence. As shown in Figure 3, from 301 to 309, the gap between the upper and lower eyelids is from small to small and then from small to large. It can be seen that the human eye is doing blinking. By analogy, the opening of the mouth can be recognized. The time value experienced from 301 to 309 is taken as the action period T. For the eye, since human natural blinking is not a conscious act, these must be filtered or ignored. It can be seen from the medical data that the period of human natural blinking is usually between 0.2 seconds and 0.4 seconds. So, we determine T0 = 0.4 seconds. In order to distinguish between conscious blinking and natural blinking, we set the conscious blink cycle to T1=0.8 seconds, and use this as a timeline for conscious movements, including frowning, opening, nodding, looking up, heading, shaking, head, Tibial muscles and other movements. Since the memory of people's time is vague, the time limit T2 of the conscious action needs to be opened some distance, here T2 = 1.5 seconds. T3 is the shortest time to yawn cycle. This time varies from person to person. It is recommended to T3=1.5 seconds. T4 is the longest period of yawning cycle. For the same reason, T4=2.5 seconds is recommended here.

3、作为优化，所述社交***加入图像的抗干扰处理、视频稳像、运动估算、深度学习、训练矫正、遗传矫正和/或贝叶斯矫正操作。3. As an optimization, the social system incorporates anti-jamming processing, video stabilization, motion estimation, deep learning, training correction, genetic correction, and/or Bayesian correction operations of the image.

4、如400图4是面部识别驱动的流程图。从401开始，首先进入402“终端禁止”，这是因为摄像机在至少采集一帧视频期间，程序不能被中断，以免影响图像采集质量。此后，进入403，“采集一帧视频或者一幅图像”，根据设定，还可以同步采集音频信号404，采集完成后，进入405“开放中断”。406是“预识别面部动作”，之所以定义为“预识别”，这是因为，一个完整的面部动作，是由301到309所确定的过程，其时间将超过一个视频帧的采集周期，例如以上确定的视频NTSC制式的每秒30帧，那么每帧周期为t＝1/30＝0.033秒，基于T0＝0.4秒，可见T0远大于t，从时间上看，完全是够用的。407是“预识别成功”判断，如果成功，则在408记录“预识别编码”，然后进入409“识别成功”判断。如果预识别成功判断为失败，则进入413，然后进入409“识别成功”判断。这里所述“识别成功”，是指一个完整的动作(如眨眼)识别成功。如果识别成功，则进入411“形成识别动作码，并且申请动作中断”，这里是采用所述“中断”方式来完成所述“驱动指令”。然后返回414点，进入下一帧图像的识别。如果409识别不成功，则执行410“准备下一个域识别流程”，并且返回414点，进入下一帧图像的识别。4. Figure 400 is a flow chart of the face recognition drive. Starting from 401, first enter 402 "terminal inhibition", because the camera can not be interrupted during at least one frame of video capture, so as not to affect the image acquisition quality. Thereafter, the process proceeds to 403, “Acquiring a frame of video or an image”, and according to the setting, the audio signal 404 can also be acquired synchronously, and after the acquisition is completed, the 405 “open interrupt” is entered. 406 is a "pre-identification facial action", which is defined as "pre-identification" because a complete facial action is a process determined by 301 to 309, the time of which exceeds the acquisition period of one video frame, for example The above determined video NTSC system has 30 frames per second, then each frame period is t=1/30=0.033 seconds. Based on T0=0.4 seconds, it can be seen that T0 is much larger than t, which is completely sufficient in terms of time. 407 is a "pre-identification success" judgment, and if successful, "pre-identification code" is recorded at 408, and then 409 "identification success" judgment is entered. If the pre-identification success is judged to be a failure, then go to 413, and then go to 409 "Identification Successful" judgment. The term "identification success" as used herein refers to the successful recognition of a complete action (such as blinking). If the recognition is successful, then 411 "Forms an identification action code, and the application action is interrupted", where the "drive command" is completed using the "interrupt" mode. Then return to 414 points to enter the recognition of the next frame of image. If the 409 recognition is unsuccessful, then 410 "Prepare the next domain identification process" is performed, and 414 points are returned to enter the identification of the next frame image.

5、如500图5，是500“驱动指令中断”的程序流程图。501为中断程序开始，502为“取识别动作编码”，这是在411中形成的，503为“执行动作指令”，这是根据所述动作编码来执行的，所述动作编码是***中在设计时确认的，主要内容为每个编码所对应的动作。动作指令执行完成后，进入504,“记录动作执行状况”，然后执行505“结束返回中断”。5, as shown in Figure 5, is a program flow diagram of 500 "drive command interrupt". 501 is the start of the interrupt program, 502 is "take the recognition action code", which is formed in 411, 503 is the "execution action instruction", which is performed according to the action code, which is in the system The main content confirmed at the time of design is the action corresponding to each code. After the execution of the action instruction is completed, the process proceeds to 504, "record the action execution status", and then executes 505 "end return interrupt".

6、如800，图8，是“执行动作指令”的流程图，实际上，800是所述505的分解步骤，而505又是411根据面部动作识别形成的。例如，用户做了一次“仅眨左眼”动作，在411中形成“S1”动作编码，产生中断后，在505中执行800程序，在800中，根据“S1”，执行具体的“左划屏”动作，依次类推，实现从S1到S17的全部动作。此外，还可以设定更多的动作识别以及重新定义所述“动作”和具体执行的“操作”对应关系。6. As shown in FIG. 8, FIG. 8 is a flowchart of "execution action instruction". Actually, 800 is an decomposition step of the 505, and 505 is 411 formed based on facial motion recognition. For example, the user performs a "left eye only" action, and forms an "S1" action code in 411. After the interrupt is generated, the 800 program is executed in 505. In 800, a specific "left stroke" is executed according to "S1". The screen action, and so on, implements all actions from S1 to S17. In addition, it is also possible to set more motion recognition and redefine the "action" and the specific "operation" correspondence.

7、如600图6，建立“信息”和“访问对象”之间的保密关系，其中，所述信息只是***中的各种信息，例如一张照片、一段文字、段视频、一段音频等，也可以是一个聊天群组、一个聊天对象、一段聊天记录等。信息分为602“保密信息”和603“非保密信息”。所述访问对象是指使用手机的用户，通常，手机都是私人物品，应该只有唯一的用户，可是在实际情况中，还是有很大的可能性不止一个用户的，因此，本发明将用户对象分为606“公开对象”和605“保密对象”。所述601“保密关系”是这样建立的，在602“保密信息”和605“保密对象”之间，建立一一对应的601“保密关系”，所述606“公开对象”，是指能够访问全部603“非保密信息”的用户，605“保密对象”是指在601“保密关系”中能够访问那些“允许访问”的605“保密信息”中的部分信息，而不能够访问那些601“保密关系”中能够访问那些“禁止访问”的605“保密信息”中的部分信息。因此，本发明采用识别用户身份的步骤，来完成用户身份的识别，再进一步，由601“保密关系”来确定能不能访问？以及允许访问那些“保密信息”。7. As shown in Figure 6, Figure 6, establishing a confidential relationship between "information" and "access object", wherein the information is only various information in the system, such as a photo, a piece of text, a video segment, a piece of audio, etc. It can also be a chat group, a chat object, a chat history, and so on. The information is divided into 602 "secret information" and 603 "non-confidential information". The access object refers to a user who uses a mobile phone. Generally, the mobile phone is a private item, and there should be only a single user, but in actual situations, there is still a great possibility of more than one user. Therefore, the present invention will be a user object. It is divided into 606 "public objects" and 605 "secure objects". The 601 "secure relationship" is established such that between 602 "secret information" and 605 "secure object", a one-to-one correspondence 601 "secure relationship" is established, and the 606 "public object" refers to accessibility. All 603 "non-confidential information" users, 605 "secure objects" means that some information in the 605 "secure information" of "allow access" can be accessed in the 601 "secure relationship", and those 601 "secure" are not accessible. "Relationship" can access some of the information in the 605 "Confidential Information" of "No Access". Therefore, the present invention adopts the step of recognizing the identity of the user to complete the identification of the user identity, and further, is the 601 "secure relationship" determined to be able to access? And allow access to those "confidential information."

8、如700，图7，是基于上述“保密关系”的方法所设计的所述保密访问的流程图。在进入701“保密访问”后，进行702“***认证”，这一步就是类似微信的输入用户名和密码工作步骤。通过703的判断，如果认证不通过，则返回711重新认证；如果认证通过，则进入704“识别用户身份”，在识别出用户身份后进入705“查询保密关系”，如果查询结果，进入706判断出该用户的访问对象是“公开对象”，则进入708“访问非保密信息”，再进入循环体710点；如果查询结果，进入706判断出该用户的访问对象是“保密对象”，则进入707“访问保密信息中允许访问的信息”，再进入循环体710点。在这个设计里，其特征是：使用一个循环体来循环识别用户，以实时地、连续地确认其身份，一旦手机换到例外一个人的手上，原来的“保密对象”身份改变，变成另外一个保密对象或者非保密对象，则所允许访问的信息立即改变。本发明就是用这个方法实现用户的隐私保护的。8. For example, 700, FIG. 7 is a flow chart of the secure access designed based on the method of the above “secure relationship”. After entering 701 "secure access", perform 702 "system authentication", this step is similar to WeChat input user name and password work steps. If the authentication fails, the process returns to 711 for re-authentication; if the authentication is passed, the process proceeds to 704, “identify the user identity”, and after identifying the user identity, enter 705 “query confidential relationship”, and if the query results, enter 706 to determine. If the access object of the user is "public object", then enter 708 "access non-confidential information", and then enter loop body 710; if the result of the query, enter 706 to determine that the user's access object is "secure object", then enter 707 "Access information allowed in the confidential information", and then enter the loop body 710 points. In this design, it is characterized in that a loop body is used to cyclically identify the user, and the identity is confirmed in real time and continuously. Once the mobile phone is changed to the exception of one person, the original "secure object" identity changes and becomes If another confidential object or non-confidential object, the information allowed to be accessed changes immediately. The present invention uses this method to implement user privacy protection.

9、基于趣味性考虑，把一对聊天用户的彼此的面部动作和语义发给对方，进而在彼此界面上用动画复现面部动作。发送方式可以采用约定的动作编码和语义文字，如S1到S17，V0到Vn，然后在彼此的屏幕上通过这个编码来复现面部动作动画和语义文字，还可以配上相应的音效。此外，这个设计除了趣味性之外，本身就是对于对方的一种动作通知。这一步的实现，在503中完成。9. Based on the interesting considerations, the facial actions and semantics of a pair of chat users are sent to each other, and then the facial actions are reproduced by animation on the interfaces of each other. The sending method can adopt the agreed action coding and semantic characters, such as S1 to S17, V0 to Vn, and then reproduce the facial action animation and semantic text through the code on each other's screen, and can also be matched with the corresponding sound effect. In addition, this design is a kind of action notification for the other party in addition to fun. The implementation of this step is completed in 503.

10、如900，图9，基于安全考虑，***设定所述“活体认证”功能。把所述“识别用户身份”和“识别面部动作”功能用于社交网络***对于用户进行“身份认证”和“账户密码找回”。所述“活体认证”功能不同于此前常规的身份认证是采用各种需要记忆的密码和回答设定的问题来实现，而是活体的、实时的身份认证，不需要用户记忆，黑客也无法模仿和攻击。这是通过以下步骤实现的：10. As 900, Figure 9, the system sets the "live authentication" function based on security considerations. The "identify user identity" and "recognize facial actions" functions are used by the social networking system to perform "identity authentication" and "account password retrieval" for the user. The "live authentication" function is different from the previous conventional identity authentication by using various password and answer settings that need to be remembered, but a live, real-time identity authentication, without user memory, and the hacker cannot imitate. And attack. This is achieved by the following steps:

10.1、用户使用所述社交网络***，为自己建立身份信息和动作编码组合，并且保存到所述网络***中存在的用户账户数据库中；所述身份信息是通过所述网络***采集用户的所述人脸视频和/或人脸图像序列、识别所述该用户身份和的结果数据，为了防止黑客使用该用户的照片进行网络欺骗，所述身份信息是一段视频或者多幅人脸图像，其中包括头部的微动作，如微微摇头、微微点头；所述动作编码组合是用户设定的若干面部动作结果数据的组合，例如S1+S1+S5；这一步骤是在此前用户正常输入账户和密码后设定完成的；10. The user uses the social network system to establish identity information and action coding combination for himself, and saves it in a user account database existing in the network system; the identity information is collected by the network system. a face video and/or a face image sequence, identifying the user's identity and the result data, in order to prevent the hacker from using the user's photo for network spoofing, the identity information is a video or a plurality of face images, including The micro-action of the head, such as slightly shaking the head, slightly nodding; the action coding combination is a combination of several facial action result data set by the user, for example, S1+S1+S5; this step is that the user inputs the account and password normally before. After the setting is completed;

10.2、当用户忘记账户和/或密码时，所述社交网络***提供所述活体认证界面900，所述用户通过该界面，如图9中，首先进入901“活体认证”，然后在902“***认证”输入认证密码，在903中判断，如果密码正确，说明无需进一步认证，转到701，进入正常的访问流程；如果密码不正确，***认证失败，则进入904“失败用户身份”；从904到909；所述社交网络***得到所述身份信息904和动作编码组合数据905后，与所述社交网络***中存储的用户账户数据库比较906、907，吻合，即通过认证、进入账户908和/或重新设定该账户的密码909，转到710，进入正常的访问流程；不吻合就拒绝进入账户，返回910。10.2, when the user forgets the account and/or password, the social network system provides the living body authentication interface 900, and the user passes through the interface, as shown in FIG. 9, first enters 901 "living body authentication", and then at 902 "system Authentication "Enter the authentication password, judge in 903, if the password is correct, it means no further authentication, go to 701, enter the normal access process; if the password is incorrect, the system authentication fails, then enter 904 "Failed User Identity"; from 904 Going to 909; after the social network system obtains the identity information 904 and the action coding combination data 905, it matches with the

user account database

906, 907 stored in the social network system, that is, by authenticating, entering the account 908 and / Or reset the password 909 of the account, go to 710, enter the normal access process; refuse to enter the account if it does not match, return to 910.

11、作为同类的设计，基于同样的发明思路，可以做如下改变：11. As a similar design, based on the same inventive idea, the following changes can be made:

11.1、利用动作组合来完成用户特殊指令，例如用“眨左眼”+“向左偏头”来隐藏当前信息，用“眨右眼”+“向右偏头”来删除当前信息，以此类推。11.1. Use the action combination to complete the user's special instructions. For example, use “眨 left eye” + “leftward head” to hide the current information, and “眨 right eye” + “rightward right” to delete the current information. analogy.

11.2、在407中加入语义识别，并在503中去执行用户定义的指令。11.2. Semantic recognition is added to 407, and user-defined instructions are executed in 503.

11.3、设计成用户可自定义的动作-操作指令对应关系，让用户可以修改800，重新自定义操作指令。11.3. Designed as a user-definable action-operation command correspondence, allowing the user to modify 800 and re-customize the operation instructions.

11.4、用户可以任意组合所述“面部动作识别”和“用户身份识别”，或者只用“面部动作识别”，或者只用“用户身份识别”。11.4. The user can arbitrarily combine the "face motion recognition" and "user identity recognition", or only "face motion recognition", or only "user identification".

11.5、修改412和500，把中断模式改成顺序执行模式，或依此类推把例如视频、语音采集的改成中断模式。11.5. Modify 412 and 500 to change the interrupt mode to the sequential execution mode, or to change the interrupt mode such as video and voice acquisition.

11.6、用户的数据存储采用加密存储，访问采用同样加密原理的解密方式，加解密算法包括并不限于：DES、3DES、AES、RC2、RC4、IDEA、RSA、DSA、ECC、BLOWFISH、KPCS、DM5、SHA、SSF33，SSF28，SCB2和/或SM系列的任意组合。11.6. The user's data storage adopts encrypted storage and accesses the decryption method using the same encryption principle. The encryption and decryption algorithms are not limited to: DES, 3DES, AES, RC2, RC4, IDEA, RSA, DSA, ECC, BLOWFISH, KPCS, DM5. Any combination of SHA, SSF33, SSF28, SCB2 and/or SM series.

11.7、用户软件可以采用本地模式、B/S模式、C/S模式、SaaS模式。11.7. User software can use local mode, B/S mode, C/S mode, and SaaS mode.

11.8、用户的数据可以存储于本地，也可以存储于网络，还可以存储于云数据中心。11.8. The user's data can be stored locally, stored in the network, or stored in the cloud data center.

特别说明Special Note

需要特别说明的是：1、该实施方式是本发明面向智能手机用户的面部动作驱动和隐私保护的社交***的示例性例子，并不能全面展示本发明的全部类容。2、对于流程类似图1(具体参见图1所示的100中所包含的步骤101-103)到图9，其不同点的修改对于业内的中级技术人员来说，属于公知共用的技能，不需要加以创新即可完成，这里不予重复说明。3、上述流程只是关键提示性步骤，在实际***设计时，需要按照常规的***设计和程序设计，按照业内中级技术人员公知共用的程序设计原则去加以细化和完善。4、本实施例中关于面部动作识别、用户身份识别和语义识别的算法，将采用目前公知共用的AI算法。另外，对于访问对象的信号采集，包含但不限于人脸识别、语音识别、指纹识别、虹膜识别等现有技术。It should be particularly noted that: 1. This embodiment is an illustrative example of a social system for facial motion driving and privacy protection of a smartphone user of the present invention, and does not fully demonstrate the full scope of the present invention. 2. For the process similar to FIG. 1 (refer to steps 101-103 included in 100 shown in FIG. 1) to FIG. 9, the modification of the difference is a well-known common skill for the intermediate technicians in the industry, and It needs to be innovated to complete, and will not be repeated here. 3. The above process is only a key suggestive step. In the actual system design, it needs to be refined and improved according to the conventional system design and program design according to the common programming principles well known to the intermediate technicians in the industry. 4. The algorithm for facial motion recognition, user identification, and semantic recognition in this embodiment will employ a commonly known shared AI algorithm. In addition, the signal acquisition for the access object includes, but is not limited to, the prior art such as face recognition, voice recognition, fingerprint recognition, and iris recognition.

具体实施方式二、面部动作驱动和隐私保护的浏览器Embodiment 2: Browser for facial action driving and privacy protection

该实施方式是本发明面向面部动作驱动和隐私保护的浏览器的示例性例子。所述浏览器，是兼具互联网网页和本地信息浏览的软件，宿主设备是带有摄像头和/或话筒的智能化设备，例如智能手机、PDA、PC、笔记本电脑、专用设备等。This embodiment is an illustrative example of a browser for facial action driving and privacy protection of the present invention. The browser is software that combines Internet web pages and local information browsing. The host device is an intelligent device with a camera and/or a microphone, such as a smart phone, a PDA, a PC, a notebook computer, a dedicated device, and the like.

***差异化说明System differentiation description

本实施方式与前述实施方式相比，相同之处不予复述，差异之处在于：Compared with the foregoing embodiment, the present embodiment is not repeated, and the difference lies in:

1、针对前述实施方式的第7条和第8条，在图6中，被访问的信息去掉聊天群组、聊天对象和聊天记录，增加对于互联网网页，将一部分网站和网页作为所述“保密信息”。1. For the seventh and eighth aspects of the foregoing embodiment, in FIG. 6, the accessed information removes the chat group, the chat object, and the chat record, and adds a part of the website and the web page as the "secure" to the Internet webpage. information".

2、针对前述实施方式的第9条，取消该功能。2. For the ninth aspect of the foregoing embodiment, the function is cancelled.

3、增加网页保存功能，根据用户的设定，所保存的网页可以是所述“保密信息”，也可以是“非保密信息”。3. The webpage saving function is added. According to the setting of the user, the saved webpage may be the “confidential information” or “non-confidential information”.

具体实施方式三、面部动作驱动头盔DETAILED DESCRIPTION OF THE INVENTION Third, facial action driven helmet

该实施方式是本发明面向面部动作驱动头盔的示例性例子。This embodiment is an illustrative example of the face motion driven helmet of the present invention.

***差异化说明System differentiation description

本实施方式与前述实施方式一相比，相同之处不予复述，差异之处在于：Compared with the foregoing first embodiment, the present embodiment is not repeated, and the difference lies in:

1、摄像头安装在头盔内部，正对着人脸或者人眼。1. The camera is mounted inside the helmet, facing the face or the human eye.

2、取消人脸的身份识别。2. Cancel the identification of the face.

3、增加瞳孔运动跟踪功能，并且同步用面部动作执行设定的功能。3. Increase the pupil movement tracking function and synchronize the functions performed with the facial motion.

4、增加凝视功能，并且同步用面部动作执行设定的功能。4. Increase the gaze function and synchronize the functions performed with the facial motion.

具体实施例方式、面部动作驱动金融支付Specific embodiments, facial actions drive financial payment

该实施方式是本发明面向面部动作驱动金融支付示例性例子。This embodiment is an illustrative example of the face-action-driven financial payment of the present invention.

***差异化说明System differentiation description

本实施方式与实施方式一相比，相同之处不予复述，差异之处在于：Compared with the first embodiment, the present embodiment is not repeated, and the difference lies in:

采用面部动作及其序列，作为金融账户的验证密码和/或辅助验证密码。The facial action and its sequence are used as the verification password and/or the secondary verification password for the financial account.

取消账户自动识别。Cancel the automatic identification of the account.

总结to sum up

非限制性说明Non-limiting description

上述实施方式只是本发明的几个示例性例子，并不表示对于本发明技术方案的限制。具体是：The above embodiments are merely a few illustrative examples of the present invention and are not intended to limit the technical solutions of the present invention. specifically is:

所述实施方式的框图，只是按照业内的约定俗成的习惯画法，实际上还可以有其它的画法；框图单元的内容并不是图中所限制的内容划分，仅仅是一种逻辑功能划分，实际实现时可以有另外的划分方式，可以相互组合和/或各自拆分和/或拆分后再组合，所述结合和/或拆分可以集成到另一个***，一些特征可以被忽略，或不执行。框图单元之间的连接是逻辑性的，一些连接可以是电性的，也可以是机械的，还可以是无线的，包含但不限于无线电、光线、声音、磁、热等。The block diagram of the embodiment is only a customary drawing method according to the conventions in the industry, and actually there may be other drawing methods; the content of the block diagram unit is not the content division limited in the figure, but only a logical function division, actually The implementation may have additional divisions, which may be combined with each other and/or split and/or split and combined, the integration and/or splitting may be integrated into another system, some features may be ignored, or not carried out. The connections between the block diagram elements are logical, and some connections may be electrical, mechanical, or wireless, including but not limited to radio, light, sound, magnetism, heat, and the like.

所述实施方式的流程图，同样只是按照业内的约定俗成的习惯画法，实际上还可以有其它的画法。流程图单元的内容并不是图中所限制的内容划分，仅仅是一种逻辑功能划分，实际实现时可以有另外的划分方式。流程图单元可以相互组合和/或各自拆分和/或拆分后再组合，所述结合和/或拆分可以集成到另一个流程图，一些单元可以被忽略，或不执行。流程图中一些单元的前后顺序，也可以有另外的排序。流程图中的一些断和跳转，也可以有另外的组合。流程图的实现，既可以是采用软件方式，也可以是采用硬件方式，还可以是它们的组合。The flow chart of the embodiment is also just a customary drawing according to the conventions in the industry, and actually there are other drawing methods. The content of the flow chart unit is not the content division limited in the figure, but only a logical function division, and there may be another division manner in actual implementation. The flow diagram units can be combined with each other and/or split and/or split and combined, the integration and/or split can be integrated into another flow diagram, some of which can be omitted or not executed. The order of some units in the flow chart can also be ordered differently. Some breaks and jumps in the flow chart can also have other combinations. The implementation of the flow chart can be either software or hardware, or a combination thereof.

所述实施方式的原理性图示，同样只是按照业内的约定俗成的习惯画法，只是一种示意图，实际上还可以有其它的画法。图中的线条是直线还是曲线、是虚线还是实线，除非图中或者说明文字中特别声明，否则并不具备特殊含义。The schematic illustration of the embodiment is just a customary drawing according to the conventions in the industry, but is a schematic diagram, and actually there are other painting methods. Whether the line in the diagram is a straight line or a curve, a dashed line or a solid line does not have a special meaning unless it is specifically stated in the figure or in the text.

所述实施方式的实物图，仅仅是示例，可以按照业内通用的解释，选用其它类似实物图。The physical map of the embodiment is merely an example, and other similar physical maps may be selected according to the general explanation in the industry.

所述实施方式的名词，同样只是按照业内的约定俗成的习惯命名，除非特别声明，否则也可以有其它的同意名词取代，并且不影响描述。The nouns of the embodiment are also named according to the customary conventions in the industry. Unless otherwise stated, other consent terms may be substituted and the description is not affected.

所述实施方式的模式，同样只是按照业内的约定俗成的习惯命名，除非特别声明，否则也可以有其它的名词取代，并且不影响描述。The mode of the embodiment is also named according to the customary conventions in the industry. Unless otherwise stated, other nouns may be substituted and the description is not affected.

所述宿主平台是指包含并不限于PC，手机，VR设备，AR设备，MR设备，服务器，或者网络设备或其他可编程设备等。The host platform refers to and includes not limited to a PC, a mobile phone, a VR device, an AR device, an MR device, a server, or a network device or other programmable device.

Claims

一种面部识别方法，包括：A facial recognition method comprising:

采集人脸视频和/或人脸图像序列的步骤；a step of collecting a sequence of face videos and/or face images;

依据所述人脸视频和/或人脸图像序列识别面部动作的步骤；The step of recognizing a facial motion according to the sequence of face video and/or face image;

基于所述面部动作输出驱动指令的步骤。A step of outputting a drive command based on the facial motion.
根据权利要求1所述的方法，其特征在于：The method of claim 1 wherein:

所述依据所述人脸视频和/或人脸图像序列识别面部动作的步骤，包括：The step of recognizing a facial action according to the sequence of the face video and/or the face image includes:

过滤无意识动作S14；和/或，Filtering the unconscious action S14; and/or,

报警疲劳动作S15、报警异常动作S16和/或S17；和/或，Alarm fatigue action S15, alarm abnormal action S16 and/or S17; and/or,

执行有意识动作S1、S2-S12和/或S13；Performing conscious actions S1, S2-S12 and/or S13;

其中：among them:

S1为左眼眨动动作，周期大于T1并且小于T2；S1 is a left eye swaying action, the period is greater than T1 and less than T2;

S2为右眼眨动动作，周期大于T1并且小于T2；S2 is a right eye swaying action, the period is greater than T1 and less than T2;

S3为双眼眨动动作，周期大于T1并且小于T2；S3 is a double eye swaying action, the period is greater than T1 and less than T2;

S4为皱眉动作，周期大于T1并且小于T3；S4 is a frowning action, the period is greater than T1 and less than T3;

S5为张嘴动作，周期大于T1并且小于T3；S5 is a mouth opening action, the period is greater than T1 and less than T3;

S6为点头动作，周期大于T1并且小于T3；S6 is a nodding action, the period is greater than T1 and less than T3;

S7为仰头动作，周期大于T1并且小于T3；S7 is an upward movement, the period is greater than T1 and less than T3;

S8为低头动作，周期大于T1并且小于T3；S8 is a bowing action, the period is greater than T1 and less than T3;

S9为摇头动作，周期大于T1并且小于T3；S9 is a shaking action, the period is greater than T1 and less than T3;

S10为左向偏头动作，周期大于T1并且小于T3；S10 is a leftward biasing action, and the period is greater than T1 and less than T3;

S11为右向偏头动作，周期大于T1并且小于T3；S11 is a rightward biasing action, the period is greater than T1 and less than T3;

S12为拉左脸肌肉动作，周期大于T1并且小于T3；S12 is a pull left face muscle action, the period is greater than T1 and less than T3;

S13为拉右脸肌肉动作，周期大于T1并且小于T3；S13 is the action of pulling the right face muscle, the period is greater than T1 and less than T3;

S14为双眼眨眼动作，周期小于T0；S14 is a blinking operation of both eyes, and the period is less than T0;

S15为呵欠动作，周期大于T4并且小于T5；S15 is a yawning action, the period is greater than T4 and less than T5;

S16为双眼闭眼动作，时间大于T5；S16 is a closed-eye action of both eyes, and the time is greater than T5;

S17为张嘴动作，时间大于T6；S17 is a mouth opening action, the time is greater than T6;

其中周期长短关系是：T6≥T5≥T4≥T3≥T2≥T1≥T0；The relationship between the length of the cycle is: T6 ≥ T5 ≥ T4 ≥ T3 ≥ T2 ≥ T1 ≥ T0;

所述T0是在所述人脸视频和/或人脸图像序列中定位及计算眼睛完成一次无意识自然眨眼动作从开始到结束的时间；The T0 is a time for locating and calculating an unconscious natural blinking action of the eye from the start to the end in the sequence of the face video and/or the face image;

所述T1是在所述人脸视频和/或人脸图像序列中定位及计算眼睛、眉部、嘴部、头部、脸部肌肉完成一次有意识动作从开始到结束的最短时间；The T1 is a minimum time for locating and calculating the conscious action of the eye, the eyebrow, the mouth, the head, and the facial muscle from the beginning to the end in the sequence of the face video and/or the face image;

所述T2是在所述人脸视频和/或人脸图像序列中定位及计算眼睛完成一次有意识动作从开始到结束的最长时间；The T2 is a positioning and calculating, in the sequence of the face video and/or the face image, a maximum time from the start to the end of the conscious action of the eye;

所述T3是在所述人脸视频和/或人脸图像序列中定位及计算完成一次有意识的眉部、嘴部、头部、脸部肌肉动作从开始到结束的最长时间；The T3 is a maximum time for locating and calculating a conscious eyebrow, mouth, head, and facial muscle action from start to finish in the face video and/or face image sequence;

所述T4是在所述人脸视频和/或人脸图像序列中定位及计算头部完成一次打哈欠动作从开始到结束的最短时间；The T4 is a shortest time from the start to the end of locating and calculating the head yawning action in the face video and/or face image sequence;

所述T5、T6分别是在所述人脸视频和/或人脸图像序列中定位及计算完成一次因疲劳闭眼或者张嘴打瞌睡动作从开始到结束的最短时间。The T5 and T6 are respectively the shortest time in the face video and/or the face image sequence to locate and calculate the time from the start to the end of the fatigue closed eye or the mouth snapping action.
根据权利要求2所述的方法，其特征在于：还包括音频采集和语音识别、语言指令识别步骤；The method according to claim 2, further comprising audio capture and speech recognition, language instruction recognition steps;

所述语音识别，是对于所述音频采集的信号进行人类语音的识别，排除环境噪音信号；The speech recognition is to identify a human voice for the signal collected by the audio, and to eliminate an environmental noise signal;

所述语言指令识别，是对于音频信号进行语义识别，以识别出所述用户的语言指令；The language instruction identification is to perform semantic recognition on the audio signal to identify the language instruction of the user;

所述语音识别和语言指令识别的方法包括：滤波算法、人工智能网络识别算法、模糊识别算法；基于设定，包括识别：The method for recognizing speech recognition and language instruction comprises: a filtering algorithm, an artificial intelligence network recognition algorithm, and a fuzzy recognition algorithm; based on settings, including identification:

V0为说话动作，张嘴周期大于所述T1并且小于所述T3，并且同时伴有语音产生；V0 is a speaking action, the mouth opening period is greater than the T1 and smaller than the T3, and accompanied by voice generation;

V1为“关闭”语言指令；V1 is a "closed" language instruction;

V2为“打开”语言指令；V2 is an "open" language instruction;

V3为“向上”语言指令；V3 is an "up" language instruction;

V4为“向下”语言指令；V4 is a "down" language instruction;

V5为“向左”语言指令；V5 is a "left" language instruction;

V6为“向右”语言指令；V6 is a "rightward" language instruction;

V7为“退出”语言指令；V7 is an "exit" language instruction;

V8为“删除”语言指令；V8 is a "delete" language instruction;

V9为“隐藏”语言指令；V9 is a "hidden" language instruction;

V10为“后悔”语言指令；V10 is a "repentance" language instruction;

V11为“向前”语言指令；V11 is a "forward" language instruction;

V12-Vn为用户自定义语言指令，n为根据用户的意愿设定值。V12-Vn is a user-defined language command, and n is a set value according to the user's wishes.
根据权利要求1所述的方法，其特征在于，所述依据所述人脸视频和/或人脸图像序列识别面部动作的步骤包括：The method according to claim 1, wherein the step of recognizing a facial motion according to the sequence of face video and/or face image comprises:

采用人工智能网络算法、模糊识别算法和/或遗传算法，对所述人脸视频和/或人脸图像序列识进行图像尺寸归一化、滤波与特征提取、抗干扰处理、视频稳像、运动估算、深度学习、训练矫正、遗传矫正和/或贝叶斯矫正操作。Image size normalization, filtering and feature extraction, anti-interference processing, video stabilization, and motion are performed on the face video and/or face image sequence identification by using an artificial intelligence network algorithm, a fuzzy recognition algorithm, and/or a genetic algorithm. Estimation, deep learning, training correction, genetic correction and/or Bayesian correction.
根据权利要求1所述的方法，其特征在于，还包括：依据所述人脸视频和/或人脸图像序列识别用户身份的步骤；具体包括：The method according to claim 1, further comprising the step of: identifying a user identity according to the sequence of the face video and/or the face image; specifically comprising:

采用人工智能网络算法、模糊识别算法和/或遗传算法，对所述人脸视频和/或人脸图像序列识进行图像尺寸归一化、滤波与特征提取、抗干扰处理、视频稳像、运动估算、深度学习、训练矫正、遗传矫正和/或贝叶斯矫正操作。Image size normalization, filtering and feature extraction, anti-interference processing, video stabilization, and motion are performed on the face video and/or face image sequence identification by using an artificial intelligence network algorithm, a fuzzy recognition algorithm, and/or a genetic algorithm. Estimation, deep learning, training correction, genetic correction and/or Bayesian correction.
根据权利要求5所述的方法，其特征在于，还包括：The method of claim 5, further comprising:

基于设定，将通过识别出的用户标识为公开对象和/或保密对象；和/或，Based on the settings, the identified user is identified as a public object and/or a confidential object; and/or,

基于设定，在社交通信***的应用中，发送所述面部动作种类和/或语义文字给社交通信***中的对方，并且在对方显示器上用动画和/或复现面部动作的步骤。Based on the settings, in the application of the social communication system, the facial action category and/or semantic text is sent to the counterpart in the social communication system, and the steps of animating and/or reproducing the facial motion are performed on the counterpart display.
根据权利要求1所述的方法，其特征在于，还包括：The method of claim 1 further comprising:

基于设定，对于数据的存储、传输和/或显示进行加密和/或解密的步骤；其中：所述加密和/或解密步骤采用的算法包括：DES、3DES、AES、RC2、RC4、IDEA、RSA、DSA、ECC、BLOWFISH、KPCS、DM5、SHA、SSF33，SSF28，SCB2和/或SM系列的任意组合；和/或，The step of encrypting and/or decrypting data storage, transmission and/or display based on settings; wherein: the encryption and/or decryption steps include: DES, 3DES, AES, RC2, RC4, IDEA, Any combination of RSA, DSA, ECC, BLOWFISH, KPCS, DM5, SHA, SSF33, SSF28, SCB2 and/or SM series; and/or,

基于设定，网络***对于终端客户的账户和/或密码进行活体认证的步骤；所述活体认证包括：The step of performing biometric authentication on the account and/or password of the end client based on the setting; the biometric authentication includes:

客户预先在所述网络***为自己建立的身份信息和动作编码组合信息的步骤；和/或，a step of the customer pre-coding the combined information for the identity information and the action established by the network system for itself; and/or,

所述网络***基于提供的活体认证界面，根据所述客户预先在所述网络***为自己建立的身份信息和动作编码组合信息，接受客户登录账户和/或重新设定所述账户密码的操作。The network system accepts a client login account and/or resets the account password based on the provided biometric authentication interface according to the identity information and action coding combination information that the client pre-establishes for the network system.
根据权利要求2或3所述的方法，其特征在于，还包括利用驱动指令对宿主设备下一步操作提供驱动的步骤，具体包括：The method according to claim 2 or 3, further comprising the step of providing a driver for the next operation of the host device by using the driving instruction, specifically comprising:

利用驱动指令提供程序函数代码，以使宿主软件能够根据所述程序函数代码进行判断并执行下一步功能；和/或，Providing program function code with a driver instruction to enable the host software to determine and perform the next function according to the program function code; and/or,

利用驱动指令提供控制接口的信号，以使宿主设备根据所述信号，执行后续的操作。The drive command is used to provide a signal to the control interface to cause the host device to perform subsequent operations in accordance with the signal.
根据权利要求5或6所述的方法，其特征在于，还包括隐私保密的步骤、社交保密的步骤、XR和头盔操作的步骤、金融支付的步骤、手机APP的步骤和/或电子游戏的步骤；其中：A method according to claim 5 or claim 6, further comprising the steps of privacy privacy, steps of social security, steps of XR and helmet operation, steps of financial payment, steps of a mobile APP, and/or steps of an electronic game ;among them:

所述隐私保密的步骤包括：基于设定，在用户和隐私信息之间建立一一对应的保密关系；通过所述识别用户身份，查找并访问所述保密关系中允许该用户访问的隐私信息；The step of privacy confidentiality includes: establishing a one-to-one correspondence relationship between the user and the private information based on the setting; and searching for and accessing the private information in the confidential relationship that is allowed to be accessed by the user by the identifying the user identity;

所述社交保密的步骤包括：基于设定，在用户和社交对象之间建立一一对应的保密关系；把社交对象分成公开对象和保密对象；通过所述识别用户身份，查找并访问所述保密关系中允许该用户访问的保密社交对象；在用户跟保密对象通信时，使用所述S1-S13以及它们的组合去驱动保密和操作功能；所述社交对象包括社交对象个人和群组；所述动作识别的结果还可以发送给社交对象，在社交对象和/或用户自己的界面上采用动画和音效复现；The step of social secrecy includes: establishing a one-to-one correspondence relationship between the user and the social object based on the setting; dividing the social object into a public object and a secret object; and searching for and accessing the secret by identifying the user identity a secret social object in the relationship that allows access by the user; using the S1-S13 and combinations thereof to drive privacy and operational functions when the user communicates with the secure object; the social object includes social object individuals and groups; The results of the motion recognition can also be sent to the social object, using animation and sound reproduction on the social object and/or the user's own interface;

所述XR和头盔操作的步骤包括：基于设定，在所述头盔内测安装摄像头，拍摄面部视频图像；The step of operating the XR and the helmet includes: installing a camera in the helmet based on the setting, and capturing a facial video image;

所述金融支付的步骤包括：基于设定，采用所述识别用户身份来完成身份验证，使用所述S1-S13以及它们的组合去驱动账户操作功能；The step of financial payment includes: performing identity verification using the identified user identity based on settings, using the S1-S13 and combinations thereof to drive an account operation function;

所述手机APP的步骤包括：基于设定，采用所述S1-S13以及它们的组合去驱动APP操作功能；The step of the mobile phone APP includes: driving the APP operation function by using the S1-S13 and a combination thereof based on the setting;

所述电子游戏的步骤包括：所述S1-S13以及它们的组合去驱动指定的动作。The steps of the electronic game include: the S1-S13 and combinations thereof to drive the specified action.
根据权利要求1-7所述的任一方法，其特征在于：A method according to any of claims 1-7, characterized in that:

所述采集人脸视频和/或人脸图像序列的采样频率大于或者等于每秒5帧。The sampling frequency of the collected face video and/or face image sequence is greater than or equal to 5 frames per second.