WO2018161906A1

WO2018161906A1 - Motion recognition method, device, system and storage medium

Info

Publication number: WO2018161906A1
Application number: PCT/CN2018/078215
Authority: WO
Inventors: 万猛; 荆彦青; 魏学峰; 曹文升; 耿天平
Original assignee: 腾讯科技（深圳）有限公司
Priority date: 2017-03-09
Filing date: 2018-03-07
Publication date: 2018-09-13
Also published as: CN107016347A

Abstract

Provided in an embodiment of the present invention are a motion recognition method, a device, a system, and a storage medium. The motion recognition method comprises: obtaining motion sensing data of a mobile terminal, wherein the motion sensing data comprises acceleration sensing data or gyro sensing data; obtaining first motion characteristic data according to the motion sensing data of the mobile terminal; and comparing the first motion characteristic data to at least one preset second motion characteristic data, so that a known motion is determined from the known motions corresponding to the at least one second motion characteristic data, as the motion currently performed by the mobile terminal.

Description

动作识别方法、装置、***以及存储介质Motion recognition method, device, system and storage medium

本申请要求于2017年03月09日提交中国专利局、申请号为2017030901429420、名称为“一种体感动作识别方法、装置以及***”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。The present application claims priority to Chinese Patent Application No. 2009030901429420, entitled "A Sense of Motion Recognition Method, Apparatus, and System", filed on March 9, 2017, the entire contents of which are incorporated herein by reference. In the application.

技术领域Technical field

本申请涉及互联网技术领域，尤其涉及一种动作识别方法、装置、***以及存储介质。The present application relates to the field of Internet technologies, and in particular, to a motion recognition method, apparatus, system, and storage medium.

背景background

体感游戏(英文：Motion Sensing Game)顾名思义：用身体去感受的电子游戏。突破以往单纯以手柄按键输入的操作方式，体感游戏是一种用户通过肢体动作变化来进行(操作)的新型电子游戏。Motion Sensing Game (English: Motion Sensing Game) As the name suggests: a video game that uses the body to feel. Breaking through the previous operation mode of simply inputting the handle button, the somatosensory game is a new type of electronic game in which the user performs (operation) through the change of the limb movement.

目前的体感游戏方式通常都需要专门的体感游戏机，通过深度摄像头采集画面中用户的肢体动作变化来识别用户的动作。The current somatosensory game mode usually requires a special somatosensory game machine to recognize the user's movements by collecting the user's limb movement changes in the screen through the depth camera.

技术内容Technical content

本发明实施例提供了一种动作识别方法，所述方法包括：The embodiment of the invention provides a motion recognition method, and the method includes:

获取移动终端的运动传感数据，所述运动传感数据包括加速度传感数据或陀螺仪传感数据；Obtaining motion sensing data of the mobile terminal, the motion sensing data including acceleration sensing data or gyro sensing data;

根据所述移动终端的运动传感数据获取第一动作特征数据；Obtaining first action feature data according to the motion sensing data of the mobile terminal;

将所述第一动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。Comparing the first action feature data with the preset at least one second action feature data to determine, from a known action corresponding to the at least one second action feature data, that a known action is currently performed by the mobile terminal action.

相应地，本发明实施例还提供了一种动作识别装置，包括：存储器、处理器；其中，所述存储器中存储有计算机可读指令，所述处理器执行所述存储中的计算机可读指令，用于：Correspondingly, an embodiment of the present invention further provides a motion recognition apparatus, including: a memory, a processor; wherein the memory stores computer readable instructions, and the processor executes the computer readable instructions in the storing For:

相应地，本发明实施例还提供了一种移动终端，所述移动终端包括：存储器、处理器；其中，所述存储器中存储有计算机可读指令，所述处理器执行所述存储中的计算机可读指令，用于：Correspondingly, an embodiment of the present invention further provides a mobile terminal, where the mobile terminal includes: a memory, a processor; wherein the memory stores computer readable instructions, and the processor executes the computer in the storage Readable instructions for:

采集移动终端的运动传感数据，其中所述运动传感数据包括加速度传感数据或陀螺仪传感数据；Collecting motion sensing data of the mobile terminal, wherein the motion sensing data includes acceleration sensing data or gyro sensing data;

将所述运动传感数据发送至动作识别装置，以使所述动作识别装置根据所述移动终端的运动传感数据获取第一动作特征数据，并将所述第一动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。Transmitting the motion sensing data to the motion recognition device, so that the motion recognition device acquires the first motion feature data according to the motion sensing data of the mobile terminal, and the first motion feature data and the preset The at least one second action feature data is compared to determine, from the known actions corresponding to the at least one second action feature data, that a known action is an action currently performed by the mobile terminal.

相应地，本发明实施例还提供了一种动作识别***，包括动作识别装置和至少一个移动终端，其中：Correspondingly, an embodiment of the present invention further provides a motion recognition system, including a motion recognition apparatus and at least one mobile terminal, where:

所述移动终端用于通过内置的传感器采集移动终端的运动传感数据，并将所述运动传感数据发送至所述动作识别装置，其中所述运动传感数据包括加速度传感数据或陀螺仪传感数据；The mobile terminal is configured to collect motion sensing data of the mobile terminal by using a built-in sensor, and send the motion sensing data to the motion recognition device, wherein the motion sensing data includes acceleration sensing data or a gyroscope Sensing data

所述动作识别装置用于接收所述移动终端发送的所述运动传感数据，根据所述移动终端的运动传感数据获取第一动作特征数据；将所述第一动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。The motion recognition device is configured to receive the motion sensing data sent by the mobile terminal, acquire first motion feature data according to the motion sensing data of the mobile terminal, and set the first motion feature data with a preset The at least one second action feature data is compared to determine, from the known actions corresponding to the at least one second action feature data, that a known action is an action currently performed by the mobile terminal.

本申请实施例还提出了一种非易失性计算机可读存储介质，存储有计算机可读指令，可以使至少一个处理器执行以上所述的方法。The embodiment of the present application further provides a non-transitory computer readable storage medium storing computer readable instructions, which may cause at least one processor to perform the method described above.

附图说明DRAWINGS

为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.

图1A是本发明实施例中的一种动作识别***的架构示意图；1A is a schematic structural diagram of a motion recognition system in an embodiment of the present invention;

图1B是在图1A所示动作识别***架构下的动作识别方法的流程示意图；1B is a schematic flow chart of a motion recognition method in the architecture of the motion recognition system shown in FIG. 1A;

图2是在图1A所示动作识别***架构下的动作识别方法的流程示意图；2 is a schematic flow chart of a motion recognition method in the architecture of the motion recognition system shown in FIG. 1A;

图3是本发明实施例中移动终端执行的动作的示意图；3 is a schematic diagram of actions performed by a mobile terminal in an embodiment of the present invention;

图4是本发明实施例中移动终端执行的另一动作的示意图；4 is a schematic diagram of another action performed by a mobile terminal in an embodiment of the present invention;

图5是本发明实施例中移动终端执行的另一动作的示意图；FIG. 5 is a schematic diagram of another action performed by a mobile terminal in an embodiment of the present invention; FIG.

图6是本发明另一实施例中的动作识别***的架构示意图；6 is a schematic structural diagram of a motion recognition system in another embodiment of the present invention;

图7是在图6所示动作识别***架构下的动作识别方法的流程示意图；7 is a schematic flow chart of a motion recognition method in the architecture of the motion recognition system shown in FIG. 6;

图8是本发明实施例中的一种动作识别装置的结构示意图；FIG. 8 is a schematic structural diagram of a motion recognition apparatus according to an embodiment of the present invention; FIG.

图9是本发明实施例中的传感数据获取模块的结构示意图；9 is a schematic structural diagram of a sensing data acquiring module in an embodiment of the present invention;

图10是本发明实施例中的动作识别模块的结构示意图；FIG. 10 is a schematic structural diagram of a motion recognition module according to an embodiment of the present invention; FIG.

图11是本发明实施例中动作识别装置的一个硬件实体结构示意图；11 is a schematic structural diagram of a hardware entity of a motion recognition apparatus according to an embodiment of the present invention;

图12是本发明实施例中的一种移动终端的结构示意图；以及FIG. 12 is a schematic structural diagram of a mobile terminal according to an embodiment of the present invention;

图13是本发明实施例中移动终端的一个硬件实体结构示意图。FIG. 13 is a schematic structural diagram of a hardware entity of a mobile terminal according to an embodiment of the present invention.

实施方式Implementation

下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

图1A是本发明实施例中的一种动作识别***的架构示意图，如图1A所示包括动作识别装置102和至少一个移动终端101，其中：FIG. 1A is a schematic structural diagram of a motion recognition system according to an embodiment of the present invention. As shown in FIG. 1A, a motion recognition apparatus 102 and at least one mobile terminal 101 are included, where:

所述移动终端101用于通过内置的传感器采集移动终端的运动传感数据，并将所述运动传感数据发送至所述动作识别装置102，其中所述运动传感数据包括加速度传感数据和/或陀螺仪传感数据。The mobile terminal 101 is configured to collect motion sensing data of the mobile terminal by using a built-in sensor, and send the motion sensing data to the motion recognition device 102, wherein the motion sensing data includes acceleration sensing data and / or gyroscope sensing data.

本发明实施例中提及的移动终端101可以包括手机、平板电脑、电子阅读器、穿戴式智能设备等，本实施例中的动作识别装置可以实现在个人电脑、平板电脑、电子阅读器、笔记本电脑、车载终端等用户终端中，移动终端101与动作识别装置102之间可以建立无线数据传输通道用以发送所述移动终端101的运动传感数据，所述无线数据传输通道可以例如wifi、蓝牙或移动通信网络(例如2/3/4/5G)。The mobile terminal 101 mentioned in the embodiment of the present invention may include a mobile phone, a tablet computer, an e-reader, a wearable smart device, etc. The action recognition device in this embodiment may be implemented in a personal computer, a tablet computer, an e-reader, a notebook. In a user terminal such as a computer or a vehicle terminal, a wireless data transmission channel may be established between the mobile terminal 101 and the motion recognition device 102 for transmitting motion sensing data of the mobile terminal 101, and the wireless data transmission channel may be, for example, wifi or Bluetooth. Or a mobile communication network (eg 2/3/4/5G).

所述动作识别装置102用于接收所述移动终端101发送的所述运动传感数据，根据所述移动终端101的运动传感数据获取用户动作特征数据；将当前获取的用户动作特征数据与预设的至少一个已知动作特征数据进行比较，从而将所述至少一个已知动作特征数据对应的已知动作中确定一个已知动作为移动终端101当前执行的动作。The motion recognition device 102 is configured to receive the motion sensing data sent by the mobile terminal 101, acquire user action feature data according to the motion sensing data of the mobile terminal 101, and obtain current user action feature data and a preset The at least one known action feature data is compared to determine a known action among the known actions corresponding to the at least one known action feature data as the action currently performed by the mobile terminal 101.

图1B为在图1A所示动作识别***架构下的动作识别方法的流程示意图。如图1B所示，该动作识别方法由动作识别装置102执行，包括以下步骤：FIG. 1B is a schematic flow chart of a motion recognition method in the architecture of the motion recognition system shown in FIG. 1A. As shown in FIG. 1B, the motion recognition method is performed by the motion recognition apparatus 102, and includes the following steps:

步骤101b：获取移动终端的运动传感数据，所述运动传感数据包括加速度传感数据和/或陀螺仪传感数据。 Step 101b: Acquire motion sensing data of the mobile terminal, the motion sensing data including acceleration sensing data and/or gyroscope sensing data.

在本申请一实施例中，该移动终端的运动传感数据包括：该移动终端的检测设备采集的描述该移动终端的动作的数据。该检测设备可以包括：加速度传感器和/或陀螺仪传感器。In an embodiment of the present application, the motion sensing data of the mobile terminal includes data collected by the detecting device of the mobile terminal and describing an action of the mobile terminal. The detecting device may include an acceleration sensor and/or a gyro sensor.

步骤102b：根据所述移动终端的运动传感数据获取第一动作特征数据。其中，上述第一动作特征数据也可以称为用户动作特征数据。 Step 102b: Acquire first action feature data according to motion sensing data of the mobile terminal. The first action feature data may also be referred to as user action feature data.

步骤103b：将所述第一动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。其中，上述第二动作特征数据也可以称为已知动作特征数据。 Step 103b: Compare the first motion feature data with the preset at least one second action feature data to determine a known action as a mobile terminal from a known action corresponding to the at least one second action feature data. The currently executing action. The second action feature data may also be referred to as known action feature data.

下面结合图2，详细介绍在图1A所示动作识别***架构下的动作识别方法流程：The flow of the motion recognition method under the motion recognition system architecture shown in FIG. 1A is described in detail below with reference to FIG. 2:

S201，移动终端通过内置传感器采集移动终端的运动传感数据，其中所述运动传感数据包括加速度传感数据或陀螺仪传感数据。S201. The mobile terminal collects motion sensing data of the mobile terminal by using a built-in sensor, where the motion sensing data includes acceleration sensing data or gyro sensing data.

在本申请一实施例中，该加速度传感数据包括利用加速度传感器采集的数据。该陀螺仪传感数据包括利用陀螺仪传感器采集的数据。In an embodiment of the present application, the acceleration sensing data includes data acquired using an acceleration sensor. The gyro sensing data includes data acquired using a gyro sensor.

具体实现中，所述内置传感器可以包括加速度传感器或陀螺仪，还可以包括距离传感器、方向传感器等，可以获取移动终端相应的运动传感数据。In a specific implementation, the built-in sensor may include an acceleration sensor or a gyroscope, and may further include a distance sensor, a direction sensor, and the like, and may acquire corresponding motion sensing data of the mobile terminal.

在本申请一实施例中，所述采集到的移动终端的运动传感数据可以包括至少一个时间窗内采集到的多组运动传感数据，即移动终端可以以时间窗为单位采集在该时间窗内移动终端的运动传感数据。在本申请一实施例中该时间窗可以为预设的时间长度。例如在宽度为0.5秒的一个时间窗内采集到以下加速度数据：In an embodiment of the present application, the collected motion sensing data of the mobile terminal may include multiple sets of motion sensing data collected in at least one time window, that is, the mobile terminal may collect the time in units of time windows. Motion sensing data of the mobile terminal in the window. In an embodiment of the present application, the time window may be a preset length of time. For example, the following acceleration data is acquired in a time window with a width of 0.5 seconds:

(X1，Y1，Z1)，(X1, Y1, Z1),

(X2，Y2，Z2)，(X2, Y2, Z2),

(X3，Y3，Z3)，(X3, Y3, Z3),

..

(X50，Y50，Z50)，其中Xn，Yn，Zn是传感器在相互垂直的三个指定方向上的加速度值。(X50, Y50, Z50), where Xn, Yn, Zn are the acceleration values of the sensor in three specified directions perpendicular to each other.

S202，移动终端对所述运动传感数据进行滤波去噪处理。S202. The mobile terminal performs filtering and denoising processing on the motion sensing data.

由于加速度传感器或陀螺仪传感器获取到的原始运动传感数据往往存在一些抖动和噪声，因此在本申请一实施例中，移动终端可以首先对原始的传感器数据进行处理，采用低通滤波等方式进行滤波去噪处理，这样能够提高动作识别装置后续的动作相似度的判定精度，同时也能减小网络流量。Since the original motion sensing data acquired by the acceleration sensor or the gyro sensor often has some jitter and noise, in an embodiment of the present application, the mobile terminal may first process the original sensor data and perform low-pass filtering. Filter denoising processing can improve the accuracy of the subsequent motion similarity of the motion recognition device and also reduce network traffic.

需要指出的是，移动终端也可以将传感器获取到的原始运动传感数据发送至动作识别装置。It should be noted that the mobile terminal can also send the original motion sensing data acquired by the sensor to the motion recognition device.

S203，移动终端将所述经过滤波去噪处理的运动传感数据发送至动作识别装置。S203. The mobile terminal sends the motion sensing data subjected to the filtering and denoising processing to the motion recognition device.

具体实现中，移动终端可以通过建立与动作识别装置之间的无线数据传输通道发送所述运动传感数据，从而所述动作识别装置可以通过所述无线数据传输通道接收所述移动终端发送的运动传感数据。具体的，移动终端中可以运行一个进程与动作识别装置建立socket连接，采用TCP(Transmission Control Protocol，传输控制协议)协议进行数据通信。In a specific implementation, the mobile terminal may send the motion sensing data by establishing a wireless data transmission channel with the motion recognition device, so that the motion recognition device may receive the motion sent by the mobile terminal by using the wireless data transmission channel. Sensing data. Specifically, the mobile terminal can run a process to establish a socket connection with the motion recognition device, and uses TCP (Transmission Control Protocol) protocol for data communication.

在本申请一实施例中，建立上述无线数据传输通道的方式可以为：移动终端通过扫描动作识别装置在所在终端屏幕上显示的二维码的方式获取动作识别装置的网络信息，并根据获取到的动作识别装置的网络信息将移动终端的网络发送给动作识别装置从而建立双方的无线数据传输通道。在本申请一实施方式中，移动终端或或动作识别装置可以通过网络信息广播搜索的方式获取另一方的网络信息，还可以为通过中间网络服务器的中转获取另一方的网络信息，从而根据另一方的网络信息向另一方发送自身的网络信息，从而建立双方的无线数据传输通道。In an embodiment of the present application, the method for establishing the wireless data transmission channel may be: the mobile terminal acquires the network information of the motion recognition device by scanning the motion recognition device to display the two-dimensional code on the screen of the terminal, and according to the obtained The network information of the motion recognition device transmits the network of the mobile terminal to the motion recognition device to establish a wireless data transmission channel of both parties. In an embodiment of the present application, the mobile terminal or the motion recognition device may acquire the network information of the other party by means of network information broadcast search, or may acquire the network information of the other party by the intermediate network server, so that the other party may The network information sends its own network information to the other party, thereby establishing a wireless data transmission channel for both parties.

进而在本申请一实施例中，移动终端向动作识别装置发送的运动传感数据可以携带该移动终端的终端标识，用于区别于其他移动终端发送的运动传感数据，这样如图1A所示有不止一个移动终端连接至动作识别装置，动作识别装置就可以根据运动传感数据携带的终端标识分别进行处理。Further, in an embodiment of the present application, the motion sensing data sent by the mobile terminal to the motion recognition device may carry the terminal identifier of the mobile terminal, which is used to distinguish the motion sensing data sent by other mobile terminals, as shown in FIG. 1A. If more than one mobile terminal is connected to the motion recognition device, the motion recognition device can perform processing according to the terminal identifier carried in the motion sensor data.

S204，动作识别装置根据所述移动终端的运动传感数据获取用户动作特征数据。S204. The motion recognition device acquires user motion feature data according to the motion sensing data of the mobile terminal.

在一种实施方式中，动作识别装置根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，获取所述移动终端在各时间窗内的动作特征向量，从而得到动作特征向量集。In one embodiment, the motion recognition device acquires an action feature vector of the mobile terminal in each time window according to the collected plurality of sets of motion sensing data of the mobile terminal in at least one time window, thereby obtaining an action. Feature vector set.

所述动作特征向量可以包括多种用来表征移动终端在该时间窗内的执行的动作的特征，例如可以包括各个传感数据分量的均值、标准差或各传感数据分量之间的相关系数，以前文采集到的加速度数据为例，X轴上加速度分量的均值的计算方式可以为：The action feature vector may include a plurality of features used to characterize the execution of the mobile terminal within the time window, for example, may include a mean, standard deviation, or correlation coefficient between each sensor data component of each sensor data component. For example, the acceleration data collected in the previous example can be calculated by the mean value of the acceleration component on the X-axis:

标准差的计算方式可以为：The standard deviation can be calculated as:

不同传感数据分量之间的相关度的计算方式可以为：The correlation between different sensor data components can be calculated as:

以此类推可以得到10～20种不同的动作特征，这样我们在单个时间窗内采集到的移动终端的运动传感数据根据不同的运动特征就能得到一个n维的特征向量(λ1，λ2，λ3,...,λn)，其中n为使用的运动特征的种类数。根据一个时间窗内的运动传感数据可以得到一个动作特征向量，根据多个时间窗的运动传感数据得到的动作特征向量就能组成一个特征向量集，可以理解为一个特征值的矩阵m*n，m为采样的时间窗个数，n为特征值数。By analogy, 10 to 20 different motion characteristics can be obtained, so that the motion sensing data of the mobile terminal collected by us in a single time window can obtain an n-dimensional feature vector (λ1, λ2 according to different motion characteristics. Λ3,...,λn), where n is the number of types of motion features used. According to the motion sensing data in a time window, an action feature vector can be obtained. The motion feature vector obtained from the motion sensing data of multiple time windows can form a feature vector set, which can be understood as a matrix of feature values m* n, m is the number of time windows sampled, and n is the number of feature values.

在一些实施例中，为了降低后续的比较运算的复杂度，动作识别装置还可以对得到的上述个n维的特征向量(λ1，λ2，λ3，...，λn)进行主成分分析(PCA：Principal Component Analysis)降维处理，从而将特征向量集的维度降到4-6左右，经过降维后的特征向量集仍能保存动作90％以上的特征。In some embodiments, in order to reduce the complexity of subsequent comparison operations, the motion recognition apparatus may perform principal component analysis (PCA) on the obtained n-dimensional feature vectors (λ1, λ2, λ3, ..., λn). :Principal Component Analysis) reduces the dimension of the feature vector set to about 4-6. After the dimension reduction, the feature vector set can still save more than 90% of the features.

在另一种实施方式中，动作识别装置根据采集到的所述移动终端在至少一个时间窗内的至少一组运动传感数据，获取移动终端在各时间窗内的运动轨迹数据。In another embodiment, the motion recognition device acquires motion trajectory data of the mobile terminal in each time window according to the collected at least one set of motion sensing data of the mobile terminal in at least one time window.

即动作识别装置根据采集到的所述移动终端在至少一个时间窗内的至少一组运动传感数据，计算移动终端在时间窗内的相对位置的变化，从而得到移动终端在各时间窗内的运动轨迹数据。例如图3所示移动终端执行的动作，动作识别装置可以根据采集到的所述移动终端在至少一个时间窗内的至少一组运动传感数据，计算得到移动终端的运动轨迹为往复的弧形轨迹。如图4所示移动终端执行的动作，动作识别装置可以根据采集到的所述移动终端在至少一个时间窗内的至少一组运动传感数据，计算得到移动终端的运动轨迹为一个圆形轨迹。如图5所示移动终端执行的动作，动作识别装置可以根据采集到的所述移动终端在至少一个时间窗内的至少一组运动传感数据，计算得到移动终端的运动轨迹为一个Z形轨迹。而在本申请一实施例中，动作识别装置可以使用至少一个运动矢量组成的运动矢量集合代表移动终端在各时间窗内的运动轨迹数据，其中的每个运动矢量可以表征移动终端在各时间窗内的不同采集时间点上的运动轨迹的相对方向。That is, the motion recognition device calculates the change of the relative position of the mobile terminal in the time window according to the collected at least one set of motion sensing data of the mobile terminal in the at least one time window, thereby obtaining the mobile terminal in each time window. Motion track data. For example, the action performed by the mobile terminal shown in FIG. 3, the motion recognition device may calculate, according to the collected at least one set of motion sensing data of the mobile terminal in at least one time window, the motion track of the mobile terminal is a reciprocating arc. Track. As shown in FIG. 4, the motion recognition apparatus may calculate, according to the collected at least one set of motion sensing data of the mobile terminal in at least one time window, a motion trajectory of the mobile terminal as a circular trajectory. . As shown in FIG. 5, the motion recognition apparatus may calculate, according to the collected at least one set of motion sensing data of the mobile terminal in at least one time window, a motion trajectory of the mobile terminal as a Z-shaped trajectory. . In an embodiment of the present application, the motion recognition apparatus may use motion vector sets composed of at least one motion vector to represent motion trajectory data of the mobile terminal in each time window, where each motion vector may represent the mobile terminal in each time window. The relative direction of the motion trajectory at different acquisition time points within.

在本申请一实施例中，所述时间窗的时长可以为预设的定值，例如0.5～1秒，也可以为动作识别装置通知所述移动终端，移动终端按照动作识别装置的要求提供时间窗内移动终端的运动传感数据。而动作识别装置在进行识别过程中基于的运动传感数据中对应的时间窗的数目，可以是动作识别装置与移动终端预先约定的额定数目，例如3-5个，也可以是动作识别装置根据当前需要用户执行的动作对应的时间窗的数目决定的，例如动作识别装置当前提示用户执行一个较为简单的挥手动作(如图3所示)，该挥手动作的标准动作所需的时间窗的数目是2或3，那么动作识别装置就可以基于当前获取到的2-3个时间窗内的移动终端的运动传感数据进行后续的动作识别，而若动作识别装置当前提示用户执行一个较为复杂的组合动作(例如先执行图4所示的画圈动作，再执行一次图5所示的Z型挥动)，这一组动作对应的标准动作所需的时间窗的数目是8-10，那么动作识别装置就可以基于当前获取到的8-10个时间窗内移动终端的运动传感数据进行后续的动作识别。In an embodiment of the present application, the duration of the time window may be a preset value, for example, 0.5 to 1 second, or may be notified to the mobile terminal by the motion recognition apparatus, and the mobile terminal provides time according to the requirements of the motion recognition apparatus. Motion sensing data of the mobile terminal in the window. The number of corresponding time windows in the motion sensing data based on the motion recognition device during the identification process may be a predetermined number of the motion recognition device and the mobile terminal, for example, 3-5, or the motion recognition device may be The number of time windows corresponding to the action currently required by the user is determined. For example, the motion recognition device currently prompts the user to perform a relatively simple wave motion (as shown in FIG. 3), and the number of time windows required for the standard motion of the wave motion If it is 2 or 3, the motion recognition device can perform subsequent motion recognition based on the motion sensing data of the mobile terminal within the currently acquired 2-3 time windows, and if the motion recognition device currently prompts the user to perform a more complicated process. The combined action (for example, first performing the circle motion shown in FIG. 4 and then performing the Z-type swing shown in FIG. 5), the number of time windows required for the standard action corresponding to the set of actions is 8-10, then the action The identification device can perform subsequent motion recognition based on the motion sensing data of the mobile terminal within the currently acquired 8-10 time windows.

S205，动作识别装置将当前获取的用户动作特征数据与预设的至少一个已知动作特征数据进行比较，从而将所述至少一个已知动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。S205. The motion recognition device compares the currently acquired user action feature data with the preset at least one known action feature data, so as to determine a known action among the known actions corresponding to the at least one known action feature data. The action currently performed by the mobile terminal.

具体实现中，动作识别装置可以使用距离度量法或相似性度量法计算当前获取的用户动作特征数据与预设的至少一个已知动作特征数据之间的相似度，进而将相似度最高的已知动作特征数据对应的已知动作作为移动终端当前执行的动作。In a specific implementation, the motion recognition apparatus may calculate a similarity between the currently acquired user motion feature data and the preset at least one known motion feature data by using a distance metric or a similarity metric, and then the highest similarity is known. The known action corresponding to the action feature data is the action currently performed by the mobile terminal.

若所述用户动作特征数据为获取所述移动终端在至少一个时间窗内的动作特征向量组成的动作特征向量集，则所述预设的至少一个已知动作特征数据包括至少一个已知动作的动作特征向量集，动作识别装置将当前获取到的动作特征向量集与预设的至少一个已知动作特征向量集进行比较，从而将与当前获取到的动作特征向量集之间相似度最高的已知动作特征向量集对应的已知动作确定为移动终端当前执行的动作。If the user action feature data is an action feature vector set that is obtained by acquiring an action feature vector of the mobile terminal in at least one time window, the preset at least one known action feature data includes at least one known action The motion feature vector set, the motion recognition device compares the currently acquired motion feature vector set with the preset at least one known motion feature vector set, so as to have the highest similarity with the currently acquired action feature vector set. The known action corresponding to the set of known action feature vectors is determined as the action currently performed by the mobile terminal.

两个特征向量之间的距离可以通过欧式距离算法、明科夫斯基距离算法等算法计算得到，例如：The distance between two eigenvectors can be calculated by algorithms such as Euclidean distance algorithm and Minkowski distance algorithm, for example:

例如设n维的特征向量x＝(x1,x2,...,xn)，y＝(y1,y2,...,yn)，欧式距离的计算如下：For example, let n-dimensional feature vectors x=(x1,x2,...,xn), y=(y1,y2,...,yn), and the Euclidean distance be calculated as follows:

例如设n维的特征向量x＝(x1,x2,...,xn)，y＝(y1,y2,...,yn)，明科夫斯基距离的计算可以如下：For example, let n-dimensional feature vectors x=(x1,x2,...,xn), y=(y1,y2,...,yn), Minkowski distance can be calculated as follows:

计算出每个维度上判定动作和标准动作特征向量的距离并进行累加可以得到两者之间的相似度。Calculating the distance between the decision action and the standard action feature vector in each dimension and accumulating can obtain the similarity between the two.

而两个特征向量之间的相似度可以通过余弦相似度算法求得。例如设n维的特征向量A＝(A1,A2,...,An)，B＝(B1,B2,...,Bn)，两个特征向量之间的余弦相似度算法可以如下:The similarity between the two eigenvectors can be obtained by the cosine similarity algorithm. For example, if the n-dimensional feature vector A = (A1, A2, ..., An), B = (B1, B2, ..., Bn), the cosine similarity algorithm between the two feature vectors can be as follows:

若所述用户动作特征数据为获取的所述移动终端在至少一个时间窗内的运动轨迹数据，则所述预设的至少一个已知动作特征数据包括至少一个已知动作的运动轨迹数据，动作识别装置将当前获取到的运动轨迹数据与预设的至少一个已知动作的运动轨迹数据进行比较，从而将与当前获取到的运动轨迹数据之间相似度最高的已知动作的运动轨迹数据对应的已知动作确定为移动终端当前执行的动作。If the user action feature data is acquired motion track data of the mobile terminal in at least one time window, the preset at least one known action feature data includes motion track data of at least one known action, and the action The identification device compares the currently acquired motion trajectory data with the motion trajectory data of the preset at least one known motion, so as to correspond to the motion trajectory data of the known motion with the highest similarity between the currently acquired motion trajectory data. The known action is determined as the action currently performed by the mobile terminal.

所述运动轨迹数据之间的相似度，具体可以根据两个运动轨迹的图形相似度或形状相似度作为两个运动轨迹数据之间的相似度。而若使用至少一个运动矢量组成的运动矢量集合代表移动终端在各时间窗内的运动轨迹数据，则可以将当前获取到的移动终端的运动矢量集合与预设的至少一个已知动作的运动轨迹数据对应运动矢量集合之间的距离或相似度作为当前获取到的移动终端的用户动作特征数据与已知动作特征数据之间的相似度，具体可以参考上述提及的欧式距离算法、明科夫斯基距离算法或余弦相似度算法。The similarity between the motion trajectory data may specifically be the similarity between the two motion trajectory data according to the graphic similarity or the shape similarity of the two motion trajectories. And if the motion vector set composed of the at least one motion vector represents the motion trajectory data of the mobile terminal in each time window, the motion vector set of the currently acquired mobile terminal and the motion track of the preset at least one known motion may be The data corresponds to the distance or similarity between the motion vector sets as the similarity between the user action feature data of the currently acquired mobile terminal and the known action feature data. For details, refer to the above-mentioned Euclidean distance algorithm, Minkov Skim distance algorithm or cosine similarity algorithm.

而在本申请一实施例中，动作识别装置还可以根据预设的至少一个已知动作特征向量集以及多个训练动作特征向量集训练得到动作特征分类器，进而将当前获取到的动作特征向量集输入所述动作特征分类器，从而将与当前获取到的动作特征向量集之间相似度最高的已知动作特征向量集对应的已知动作确定为移动终端当前执行的动作。所述动作特征分类器可以例如支持向量机(SVM，Support Vector Machine)分类器、神经网络分类器等，通过输入一定数量的各个已知动作对应的训练动作特征向量集，即可以训练得到分类效果达到要求的动作特征分类器。在通过动作特征分类器在所述至少一个已知动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作后，可以再根据上述使用距离度量法或相似性度量法等计算方式获取当前的动作的用户动作特征数据与对应的已知动作的已知动作特征数据之间的相似度，不再需要与其他已知动作的已知动作特征数据分别进行比较，从而极大幅度的减少了动作识别装置的计算量。In an embodiment of the present application, the motion recognition apparatus may further obtain the motion feature classifier according to the preset at least one known motion feature vector set and the plurality of training action feature vector sets, and further obtain the currently acquired motion feature vector. The set inputs the action feature classifier to determine a known action corresponding to the set of known action feature vectors having the highest similarity between the currently acquired action feature vector sets as the action currently performed by the mobile terminal. The action feature classifier can be, for example, a support vector machine (SVM) classifier, a neural network classifier, etc., and can input training effects by inputting a certain number of training action feature vector sets corresponding to each known action. Achieve the required action feature classifier. After determining, by the action feature classifier, in a known action corresponding to the at least one known action feature data, that a known action is an action currently performed by the mobile terminal, the distance measure or the similarity measure may be further used according to the foregoing. The calculation method obtains the similarity between the user action feature data of the current action and the known action feature data of the corresponding known action, and no longer needs to be compared with the known action feature data of other known actions, thereby greatly The amplitude is reduced by the amount of calculation of the motion recognition device.

S206，动作识别装置输出所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度。S206. The motion recognition device outputs an action identifier of the known action corresponding to the action and a similarity between the action and the corresponding known action.

具体的，动作识别装置可以根据所述已知动作对用户通过移动终端输入的动作进行动作反馈，例如结合当前的游戏进程场景进行游戏动作反馈、动作评分或动作记录等，其中可以根据所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度进行反馈。示例性的如体感跳舞游戏，在动作识别装置通过所在终端向用户播放跳舞游戏标准动作的视频时，接收用户通过移动终端执行的动作的用户动作特征数据并识别得到对应的已知动作，若识别得到的已知动作与当前播放的标准动作对应的已知动作不同，则可以向用户提示动作未执行正确的反馈，若识别得到的已知动作与当前播放的标准动作对应的已知动作相同，则可以根据所述动作与对应的已知动作之间的相似度对用户当前执行的动作进行评价或评分，如根据相似度的数值进行评分，相似度90％则评分为90分，相似度60％则评分为60分，或相似度达到相应阈值则给出对应的评价等级，例如相似度90％则为excellent，相似度80％则为good，等等。Specifically, the action recognition device may perform action feedback on the action input by the user through the mobile terminal according to the known action, for example, perform game action feedback, action score or action record, etc. according to the current game progress scene, wherein the action may be performed according to the action. The action identifier of the corresponding known action and the similarity between the action and the corresponding known action are fed back. An exemplary so-called dance game, when the motion recognition device plays a video of a dance game standard action to the user through the terminal, receives the user action feature data of the action performed by the user through the mobile terminal, and identifies the corresponding known action, if recognized If the obtained known action is different from the known action corresponding to the currently played standard action, the user may be prompted that the action does not perform correct feedback, and if the recognized known action is the same as the known action corresponding to the currently played standard action, Then, the action currently performed by the user may be evaluated or scored according to the similarity between the action and the corresponding known action, such as the score according to the similarity value, and the similarity is 90%, the score is 90, and the similarity is 60. % is scored 60 points, or the similarity reaches the corresponding threshold, then the corresponding evaluation level is given. For example, similarity 90% is excellent, similarity 80% is good, and so on.

进而在一些实施例中，动作识别装置还可以将所述移动终端当前执行的动作对应的已知动作的动作标识与所述移动终端的终端标识关联输出。用于区别于根据不同移动终端发送的运动传感数据识别得到的动作，这样如图1A所示有不止一个移动终端连接至动作识别装置，动作识别装置就可以根据运动传感数据携带的终端标识分别进行处理得到不同移动终端执行的动作，这时也就可以将其与对应移动终端的终端标识关联输出。In some embodiments, the action recognition apparatus may further output an action identifier of a known action corresponding to the action currently performed by the mobile terminal in association with a terminal identifier of the mobile terminal. For distinguishing the actions identified by the motion sensing data transmitted by different mobile terminals, such that as shown in FIG. 1A, more than one mobile terminal is connected to the motion recognition device, and the motion recognition device can be based on the terminal identifier carried by the motion sensing data. The processing performed by different mobile terminals is performed separately, and then it can be outputted in association with the terminal identifier of the corresponding mobile terminal.

本实施例中的动作识别装置通过获取移动终端的运动传感数据，将从中提取得到的用户动作特征数据与已知动作特征数据进行比较，从而将相似的已知动作特征数据对应的已知动作确定为移动终端当前执行的动作，从而可以通过移动终端配合动作识别装置识别移动终端的各种动作。The motion recognition device in the embodiment compares the motion motion data of the mobile terminal, and compares the user motion feature data extracted from the motion sensor data with the known motion feature data, so as to perform known actions corresponding to the similar known motion feature data. The action currently performed by the mobile terminal is determined, so that the mobile terminal can recognize various actions of the mobile terminal in conjunction with the motion recognition device.

图6是本发明另一实施例中的动作识别***的架构示意图，如图所示本实施例中包括至少两个移动终端，本发明实施例的动作识别装置运行于其中的一个移动终端3中，而移动终端3自身可以同时作为用户手持输入动作的移动终端和作为识别用户通过移动终端进行的动作的动作识别装置，同时还可以通过获取其他移动终端(例如图6中所示的移动终端4)的运动传感数据并进行识别从而识别用户通过其他移动终端进行的动作，而动作识别装置识别其他移动终端进行的动作的逻辑与前文结合图1A、图1B和图2中所示的实现逻辑类似，本实施例中不再赘述，本实施例仅对动作识别装置识别自身所在移动终端进行的动作的具体过程进行描述，包括如图7所示的流程：FIG. 6 is a schematic structural diagram of a motion recognition system according to another embodiment of the present invention. In the embodiment, at least two mobile terminals are included in the embodiment, and the motion recognition apparatus in the embodiment of the present invention runs in one of the mobile terminals 3 The mobile terminal 3 itself can simultaneously act as a mobile terminal for the user to input the action and a motion recognition device that recognizes the action performed by the user through the mobile terminal, and can also acquire other mobile terminals (for example, the mobile terminal 4 shown in FIG. 6). Motion sensing data is identified and identified to identify actions by the user through other mobile terminals, and the logic of the motion recognition device identifying actions performed by other mobile terminals is combined with the implementation logic shown in Figures 1A, 1B, and 2 in the foregoing. Similarly, in this embodiment, the specific process of the action recognition device identifying the action performed by the mobile terminal itself is described, including the process shown in FIG. 7 :

S701，动作识别装置获取移动终端的运动传感数据，所述运动传感数据包括加速度传感数据或陀螺仪传感数据。S701. The motion recognition device acquires motion sensing data of the mobile terminal, where the motion sensing data includes acceleration sensing data or gyro sensing data.

本实施例中，为该移动终端上运行的动作识别装置获取自身所在的移动终端的运动传感数据，所述运动传感数据为移动终端内置的运动传感器，所述内置传感器可以包括加速度传感器或陀螺仪，还可以包括距离传感器、方向传感器等，可以获取移动终端相应的运动传感数据。所述动作识别装置可以通过调用移动终端的传感器硬件相关API(Application Programming Interface，应用程序编程接口)接口获取到运动传感器采集到的运动传感数据。In this embodiment, the motion recognition data of the mobile terminal where the mobile terminal is located is obtained by the motion recognition device running on the mobile terminal, and the motion sensor data is a motion sensor built in the mobile terminal, and the built-in sensor may include an acceleration sensor or The gyroscope may further include a distance sensor, a direction sensor, etc., and may acquire corresponding motion sensing data of the mobile terminal. The motion recognition device may acquire motion sensor data collected by the motion sensor by calling a sensor hardware related API (Application Programming Interface) interface of the mobile terminal.

在本申请一实施例中，所述采集到的移动终端的运动传感数据可以包括至少一个时间窗内采集到的多组运动传感数据，即移动终端可以以时间窗为单位采集在该时间窗内移动终端的运动传感数据。In an embodiment of the present application, the collected motion sensing data of the mobile terminal may include multiple sets of motion sensing data collected in at least one time window, that is, the mobile terminal may collect the time in units of time windows. Motion sensing data of the mobile terminal in the window.

在本申请一实施例中，由于加速度传感器或陀螺仪传感器获取到的原始运动传感数据往往存在一些抖动和噪声，因此在本申请一实施例中，动作识别装置可以对原始的传感器数据进行处理，如采用低通滤波等方式进行滤波去噪处理，这样能够提高动作识别装置后续的动作相似度的判定精度。In an embodiment of the present application, since the original motion sensing data acquired by the acceleration sensor or the gyro sensor often has some jitter and noise, in an embodiment of the present application, the motion recognition device can process the original sensor data. If the filter denoising process is performed by low-pass filtering, the accuracy of the subsequent motion similarity of the motion recognition device can be improved.

进而在本申请一实施例中，动作识别装置在获取自身所在移动终端的运动传感数据时可以携带该移动终端的终端标识，用于区别于其他移动终端发送的运动传感数据，这样如图6所示有其他移动终端连接至动作识别装置所在的移动终端，动作识别装置就可以根据运动传感数据携带的终端标识分别进行处理自身所在移动终端的运动传感数据和其他移动终端的运动传感数据。In an embodiment of the present application, the motion recognition device may carry the terminal identifier of the mobile terminal when acquiring the motion sensing data of the mobile terminal where the mobile terminal is located, so as to distinguish the motion sensing data sent by the other mobile terminal, such as 6 shows that other mobile terminals are connected to the mobile terminal where the motion recognition device is located, and the motion recognition device can separately process the motion sensing data of the mobile terminal and the motion transmission of other mobile terminals according to the terminal identifier carried by the motion sensing data. Sense data.

S702，动作识别装置根据所述移动终端的运动传感数据获取用户动作特征数据。S702. The motion recognition apparatus acquires user motion feature data according to the motion sensing data of the mobile terminal.

S703，动作识别装置将当前获取的用户动作特征数据与预设的至少一个已知动作特征数据进行比较。S703. The motion recognition device compares the currently acquired user motion feature data with the preset at least one known motion feature data.

S704，动作识别装置将所述至少一个已知动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。S704. The motion recognition device determines, in the known action corresponding to the at least one known action feature data, that the known action is an action currently performed by the mobile terminal.

本实施例中的步骤S702-S704与前文实施例中的S204-S205相同，即本实施例中的动作识别装置根据所述移动终端的运动传感数据识别得到移动终端当前执行的动作的方式与前文实施例相同，本实施例中不再赘述。Steps S702-S704 in this embodiment are the same as S204-S205 in the foregoing embodiment, that is, the motion recognition apparatus in this embodiment identifies the manner in which the mobile terminal currently performs the action according to the motion sensing data of the mobile terminal. The foregoing embodiments are the same, and are not described in detail in this embodiment.

S705，动作识别装置输出所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度。S705. The motion recognition device outputs an action identifier of the known action corresponding to the action and a similarity between the action and the corresponding known action.

具体实现中，动作识别装置可以在所在移动终端输出所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度，还可以通过移动终端与其他终端的通信，将所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度发送至其他终端，例如数字电视或笔记本电脑上为用户当前作出的动作进行反馈。In a specific implementation, the action recognition device may output the action identifier of the known action corresponding to the action and the similarity between the action and the corresponding known action, and may also communicate with other terminals through the mobile terminal. The action identifier of the known action corresponding to the action and the similarity between the action and the corresponding known action are sent to other terminals, such as a digital television or a laptop, for feedback on the action currently made by the user.

本实施例中的动作识别装置通过获取移动终端的运动传感数据，将从中提取得到的用户动作特征数据与已知动作特征数据进行比较，从而将相似的已知动作特征数据对应的已知动作确定为移动终端当前执行的动作，从而可以通过移动终端配合动作识别装置实现各类动作识别反馈和游戏过程，极大的降低了动作识别应用的体验门槛，让更多的用户可以感受动作识别应用带来的方便和体验。The motion recognition device in the embodiment compares the motion motion data of the mobile terminal, and compares the user motion feature data extracted from the motion sensor data with the known motion feature data, so as to perform known actions corresponding to the similar known motion feature data. The action currently performed by the mobile terminal is determined, so that the mobile terminal cooperates with the motion recognition device to implement various motion recognition feedback and game processes, which greatly reduces the experience threshold of the motion recognition application, so that more users can feel the motion recognition application. Convenience and experience.

图8是本发明实施例中的一种动作识别装置的结构示意图，如图所示本发明实施例中的动作识别装置至少可以包括：传感数据获取模块810，用于获取移动终端的运动传感数据，所述运动传感数据包括加速度传感数据或陀螺仪传感数据。FIG. 8 is a schematic structural diagram of a motion recognition apparatus according to an embodiment of the present invention. The motion recognition apparatus in the embodiment of the present invention may include at least a sensor data acquisition module 810, configured to acquire motion transmission of the mobile terminal. Sensation data, the motion sensing data includes acceleration sensing data or gyro sensing data.

具体实现中，所述移动终端的运动传感数据可以为移动终端的内置传感器获取得到的，所述内置传感器可以包括加速度传感器或陀螺仪，还可以包括距离传感器、方向传感器等，可以获取移动终端相应的运动传感数据。In a specific implementation, the motion sensing data of the mobile terminal may be obtained by using a built-in sensor of the mobile terminal, and the built-in sensor may include an acceleration sensor or a gyroscope, and may further include a distance sensor, a direction sensor, etc., and the mobile terminal may be acquired. Corresponding motion sensing data.

在本申请一实施例中，所述采集到的移动终端的运动传感数据可以包括至少一个时间窗内采集到的多组运动传感数据，即移动终端可以以时间窗为单位采集在该时间窗内移动终端的运动传感数据。所述移动终端的运动传感数据可以携带该移动终端的终端标识，用于区别于其他移动终端发送的运动传感数据，这样如图1A所示有不止一个移动终端连接至动作识别装置，动作识别装置就可以根据运动传感数据携带的终端标识分别进行处理。In an embodiment of the present application, the collected motion sensing data of the mobile terminal may include multiple sets of motion sensing data collected in at least one time window, that is, the mobile terminal may collect the time in units of time windows. Motion sensing data of the mobile terminal in the window. The motion sensing data of the mobile terminal may carry the terminal identifier of the mobile terminal, and is used to distinguish the motion sensing data sent by other mobile terminals, so that more than one mobile terminal is connected to the motion recognition device as shown in FIG. 1A. The identification device can perform processing according to the terminal identifier carried in the motion sensing data.

在本申请一实施例中，动作识别装置可以实现在所述移动终端内部，即如图6所示的场景架构中，动作识别装置可以通过调用移动终端的传感器硬件相关API(Application Programming Interface，应用程序编程接口)接口获取到自身所在移动终端的运动传感器采集到的运动传感数据。In an embodiment of the present application, the motion recognition apparatus may be implemented in the mobile terminal, that is, in the scenario architecture shown in FIG. 6, the motion recognition apparatus may invoke a sensor hardware related API (Application Programming Interface) of the mobile terminal. The programming interface) acquires motion sensing data collected by a motion sensor of the mobile terminal on which it is located.

在本申请一实施例中，动作识别装置与移动终端分处不同的用户终端，例如图1A所示的场景架构中，移动终端与动作识别装置之间可以建立无线数据传输通道用以发送所述移动终端的运动传感数据，所述无线数据传输通道可以例如wifi、蓝牙或移动通信网络(例如2/3/4/5G)。In an embodiment of the present application, the action recognition device and the mobile terminal are separated from each other, for example, in the scenario architecture shown in FIG. 1A, a wireless data transmission channel may be established between the mobile terminal and the motion recognition device for transmitting the Motion sensing data of the mobile terminal, which may be, for example, wifi, Bluetooth or a mobile communication network (eg 2/3/4/5G).

在该实施例中，如图9所示传感数据获取模块810可以进一步包括：In this embodiment, the sensing data acquisition module 810 as shown in FIG. 9 may further include:

传输通道建立单元811，用于与所述移动终端建立无线数据传输通道；a transmission channel establishing unit 811, configured to establish a wireless data transmission channel with the mobile terminal;

具体实现中，传输通道建立单元811可以通过建立与移动终端之间的无线数据传输通道，从而通过所述无线数据传输通道接收所述移动终端发送的运动传感数据。具体的，移动终端中可以运行一个进程与动作识别装置建立socket连接，采用TCP协议进行数据通信。In a specific implementation, the transmission channel establishing unit 811 can receive the motion sensing data sent by the mobile terminal through the wireless data transmission channel by establishing a wireless data transmission channel with the mobile terminal. Specifically, the mobile terminal can run a process to establish a socket connection with the motion recognition device, and uses the TCP protocol for data communication.

传感数据接收单元812，用于通过所述无线数据传输通道接收所述移动终端发送的运动传感数据。The sensing data receiving unit 812 is configured to receive the motion sensing data sent by the mobile terminal by using the wireless data transmission channel.

动作特征获取模块820，用于根据所述移动终端的运动传感数据获取用户动作特征数据。The action feature obtaining module 820 is configured to acquire user action feature data according to the motion sensing data of the mobile terminal.

在一种实施方式中，动作特征获取模块820根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，获取所述移动终端在各时间窗内的动作特征向量，从而得到动作特征向量集。In an embodiment, the action feature acquiring module 820 acquires the action feature vector of the mobile terminal in each time window according to the collected plurality of sets of motion sensing data of the mobile terminal in at least one time window, thereby Get the set of action feature vectors.

所述动作特征向量可以包括多种用来表征移动终端在该时间窗内的执行的动作的特征，例如可以包括各个传感数据分量的均值、标准差或各传感数据分量之间的相关系数。The action feature vector may include a plurality of features used to characterize the execution of the mobile terminal within the time window, for example, may include a mean, standard deviation, or correlation coefficient between each sensor data component of each sensor data component. .

在另一种实施方式中，动作特征获取模块820根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，获取移动终端在各时间窗内的运动轨迹数据。In another embodiment, the action feature acquiring module 820 acquires motion trajectory data of the mobile terminal in each time window according to the collected plurality of sets of motion sensing data of the mobile terminal in at least one time window.

即动作特征获取模块820根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，计算移动终端在时间窗内的相对位置的变化，从而得到移动终端在各时间窗内的运动轨迹数据。例如图3所示移动终端执行的动作，动作特征获取模块820可以根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，计算得到移动终端的运动轨迹为来回的弧形轨迹；而如图4所示移动终端执行的动作，动作特征获取模块820可以根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，计算得到移动终端的运动轨迹为一个圆形轨迹；而如图5所示移动终端执行的动作，动作特征获取模块820可以根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，计算得到移动终端的运动轨迹为一个Z形轨迹。而在本申请一实施例中，动作特征获取模块820可以使用至少一个运动矢量组成的运动矢量集合代表移动终端在各时间窗内的运动轨迹数据，其中的每个运动矢量可以表征移动终端在各时间窗内的不同采集时间点上的运动轨迹的相对方向。That is, the action feature acquiring module 820 calculates the change of the relative position of the mobile terminal in the time window according to the collected plurality of sets of motion sensing data of the mobile terminal in the at least one time window, thereby obtaining the mobile terminal in each time window. Motion track data. For example, the action performed by the mobile terminal shown in FIG. 3, the action feature acquiring module 820 may calculate the motion track of the mobile terminal as a round-trip arc according to the collected plurality of sets of motion sensing data of the mobile terminal in at least one time window. The action track obtaining module 820 can calculate the motion track of the mobile terminal according to the collected plurality of sets of motion sensor data of the mobile terminal in at least one time window, as shown in FIG. For a circular trajectory; as shown in FIG. 5, the action feature acquiring module 820 can calculate the mobile terminal according to the collected plurality of sets of motion sensing data of the mobile terminal in at least one time window. The motion trajectory is a zigzag trajectory. In an embodiment of the present application, the action feature acquiring module 820 may use the motion vector set composed of the at least one motion vector to represent the motion track data of the mobile terminal in each time window, where each motion vector may represent the mobile terminal in each The relative direction of the motion trajectory at different acquisition time points within the time window.

在本申请一实施例中，所述时间窗的时长可以为预设的定值，例如0.5～1秒，也可以为动作识别装置通知所述移动终端，移动终端按照动作识别装置的要求提供时间窗内移动终端的运动传感数据。而动作特征获取模块820在进行识别过程中基于的运动传感数据中对应的时间窗的数目，可以是动作识别装置与移动终端预先约定的额定数目，例如3-5个，也可以是动作识别装置根据当前需要用户执行的动作对应的时间窗的数目决定的，例如动作识别装置当前提示用户执行一个较为简单的挥手动作(如图3所示)，该挥手动作的标准动作所需的时间窗数目是2或3，那么动作特征获取模块820就可以基于当前获取到的2-3个时间窗内移动终端的运动传感数据进行后续的动作识别，而若动作识别装置当前提示用户执行一个较为复杂的组合动作(例如先执行图4所示的画圈动作，再执行一次图5所示的Z型挥动)，这一组动作对应的标准动作所需的时间窗的数目是8-10，那么动作特征获取模块820就可以基于当前获取到的8-10个时间窗内移动终端的运动传感数据进行后续的动作识别。In an embodiment of the present application, the duration of the time window may be a preset value, for example, 0.5 to 1 second, or may be notified to the mobile terminal by the motion recognition apparatus, and the mobile terminal provides time according to the requirements of the motion recognition apparatus. Motion sensing data of the mobile terminal in the window. The number of corresponding time windows in the motion sensing data based on the motion feature acquiring module 820 may be a predetermined number of the action recognition device and the mobile terminal, for example, 3-5, or may be motion recognition. The device is determined according to the number of time windows corresponding to the actions currently required by the user, for example, the action recognition device currently prompts the user to perform a relatively simple wave action (as shown in FIG. 3), and the time window required for the standard action of the wave action If the number is 2 or 3, the action feature acquiring module 820 can perform subsequent motion recognition based on the motion sensing data of the mobile terminal within the currently acquired 2-3 time windows, and if the motion recognition device currently prompts the user to perform a comparison. Complex combined actions (for example, performing the circle motion shown in FIG. 4 and then performing the Z-type swing shown in FIG. 5), the number of time windows required for the standard action corresponding to this set of actions is 8-10. Then, the action feature acquiring module 820 can perform subsequent actions based on the motion sensing data of the mobile terminal within the currently acquired 8-10 time windows. Identification.

动作识别模块830，用于将当前获取的用户动作特征数据与预设的至少一个已知动作特征数据进行比较，从而将所述至少一个已知动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。The action recognition module 830 is configured to compare the currently acquired user action feature data with the preset at least one known action feature data, so as to determine a known one of the known actions corresponding to the at least one known action feature data. The action is currently performed as a mobile terminal.

具体实现中，动作识别模块830可以使用距离度量法或相似性度量法计算当前获取的用户动作特征数据与预设的至少一个已知动作特征数据之间的相似度，进而将相似度最高的已知动作特征数据对应的已知动作作为移动终端当前执行的动作。In a specific implementation, the action recognition module 830 may calculate the similarity between the currently acquired user action feature data and the preset at least one known action feature data by using a distance metric or a similarity measure, and then the highest similarity The known action corresponding to the action feature data is taken as the action currently performed by the mobile terminal.

若所述用户动作特征数据为获取所述移动终端在至少一个时间窗内的动作特征向量组成的动作特征向量集，则所述预设的至少一个已知动作特征数据包括至少一个已知动作的动作特征向量集，动作识别模块830将当前获取到的动作特征向量集与预设的至少一个已知动作特征向量集进行比较，从而将与当前获取到的动作特征向量集之间相似度最高的已知动作特征向量集对应的已知动作确定为移动终端当前执行的动作。If the user action feature data is an action feature vector set that is obtained by acquiring an action feature vector of the mobile terminal in at least one time window, the preset at least one known action feature data includes at least one known action The action feature vector set 830 compares the currently acquired action feature vector set with the preset at least one known action feature vector set, so as to have the highest similarity with the currently acquired action feature vector set. The known action corresponding to the set of action feature vectors is determined to be the action currently performed by the mobile terminal.

两个特征向量之间的距离可以通过欧式距离算法、明科夫斯基距离算法等算法计算得到。The distance between two feature vectors can be calculated by an Euclidean distance algorithm, a Minkowski distance algorithm, and the like.

若所述用户动作特征数据为获取所述移动终端在至少一个时间窗内的运动轨迹数据，则所述预设的至少一个已知动作特征数据包括至少一个已知动作的运动轨迹数据，动作识别模块830将当前获取到的运动轨迹数据与预设的至少一个已知动作的运动轨迹数据进行比较，从而将与当前获取到的运动轨迹数据之间相似度最高的已知动作的运动轨迹数据对应的已知动作确定为移动终端当前执行的动作。If the user action feature data is to acquire motion trajectory data of the mobile terminal in at least one time window, the preset at least one known action feature data includes motion track data of at least one known action, and motion recognition The module 830 compares the currently acquired motion trajectory data with the preset motion trajectory data of at least one known motion, so as to correspond to the motion trajectory data of the known motion with the highest similarity between the currently acquired motion trajectory data. The known action is determined as the action currently performed by the mobile terminal.

所述运动轨迹数据之间的相似度，具体可以根据两个运动轨迹的图形相似度或形状相似度作为两个运动轨迹数据之间的相似度。而若使用至少一个运动矢量组成的运动矢量集合代表移动终端在各时间窗内的运动轨迹数据，则动作识别模块830可以将当前获取到的移动终端的运动矢量集合与预设的至少一个已知动作的运动轨迹数据对应运动矢量集合之间的距离或相似度作为当前获取到的移动终端的用户动作特征数据与已知动作特征数据之间的相似度，具体可以参考上述提及的欧式距离算法、明科夫斯基距离算法或余弦相似度算法。The similarity between the motion trajectory data may specifically be the similarity between the two motion trajectory data according to the graphic similarity or the shape similarity of the two motion trajectories. If the motion vector set composed of the at least one motion vector represents the motion trajectory data of the mobile terminal in each time window, the motion recognition module 830 may set the currently acquired motion vector set of the mobile terminal with at least one preset. The motion trajectory data of the motion corresponds to the distance or similarity between the motion vector sets as the similarity between the user motion feature data of the currently acquired mobile terminal and the known motion feature data, and may refer to the above-mentioned Euclidean distance algorithm. , Minkowski distance algorithm or cosine similarity algorithm.

而在本申请一实施例中，动作识别模块830如图10所示进一步还可以包括：In an embodiment of the present application, the action recognition module 830 may further include:

分类器训练单元831，用于根据预设的至少一个已知动作特征向量集以及多个训练动作特征向量集训练得到动作特征分类器。The classifier training unit 831 is configured to train the action feature classifier according to the preset at least one known action feature vector set and the plurality of training action feature vector sets.

所述动作特征分类器可以例如支持向量机(SVM，Support Vector Machine)分类器、神经网络分类器等，分类器训练单元831通过输入一定数量的各个已知动作对应的训练动作特征向量集，即可以训练得到分类效果达到要求的动作特征分类器。The action feature classifier may be, for example, a Support Vector Machine (SVM) classifier, a neural network classifier, etc., and the classifier training unit 831 inputs a certain number of training action feature vector sets corresponding to each known action, that is, It is possible to train an action feature classifier whose classification effect meets the requirements.

动作识别单元832，用于将当前获取到的动作特征向量集输入所述动作特征分类器，从而将与当前获取到的动作特征向量集之间相似度最高的已知动作特征向量集对应的已知动作确定为移动终端当前执行的动作。The action recognition unit 832 is configured to input the currently acquired action feature vector set into the action feature classifier, so as to correspond to the set of known action feature vectors with the highest similarity between the currently acquired action feature vector sets. The known action is determined as the action currently performed by the mobile terminal.

在通过动作特征分类器在所述至少一个已知动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作后，动作识别模块830可以再根据上述使用距离度量法或相似性度量法等计算方式获取当前的动作的用户动作特征数据与对应的已知动作的已知动作特征数据之间的相似度，不再需要与其他已知动作的已知动作特征数据分别进行比较，从而极大幅度的减少了动作识别模块830的计算量。After determining, by the action feature classifier, that a known action is an action currently performed by the mobile terminal in a known action corresponding to the at least one known action feature data, the action recognition module 830 may further use the distance metric or the like according to the foregoing The calculation method such as the sex measurement method obtains the similarity between the user action feature data of the current action and the known action feature data of the corresponding known action, and no longer needs to be compared with the known action feature data of other known actions. Thus, the amount of calculation of the motion recognition module 830 is greatly reduced.

在本申请一实施例中，动作识别装置还可以包括：In an embodiment of the present application, the action recognition apparatus may further include:

动作识别输出模块840，用于输出所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度。The motion recognition output module 840 is configured to output an action identifier of the known action corresponding to the action and a similarity between the action and the corresponding known action.

具体的，动作识别输出模块840可以根据所述已知动作对用户通过移动终端输入的动作进行动作反馈，例如结合当前的游戏进程场景进行游戏动作反馈、动作评分或动作记录等，其中可以根据所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度进行反馈。示例性的如体感跳舞游戏，在动作识别装置通过所在终端向用户播放跳舞游戏标准动作的视频时，接收用户通过移动终端执行的动作的用户动作特征数据并识别得到对应的已知动作，若识别得到的已知动作与当前播放的标准动作对应的已知动作不同，则动作识别输出模块840可以向用户提示动作未执行正确的反馈，若识别得到的已知动作与当前播放的标准动作对应的已知动作相同，则可以根据所述动作与对应的已知动作之间的相似度对用户当前执行的动作进行评价或评分，如根据相似度的数值进行评分，相似度90％则评分为90分，相似度60％则评分为60分，或相似度达到相应阈值则给出对应的评价等级，例如相似度90％则为excellent，相似度80％则为good，等等。Specifically, the action recognition output module 840 can perform action feedback on the action input by the user through the mobile terminal according to the known action, for example, performing game action feedback, action scoring or action recording, etc. according to the current game progress scene, wherein The action identifier of the known action corresponding to the action and the similarity between the action and the corresponding known action are fed back. An exemplary so-called dance game, when the motion recognition device plays a video of a dance game standard action to the user through the terminal, receives the user action feature data of the action performed by the user through the mobile terminal, and identifies the corresponding known action, if recognized The obtained known action is different from the known action corresponding to the currently played standard action, and the action recognition output module 840 can prompt the user that the action does not perform correct feedback, and if the recognized known action corresponds to the currently played standard action Knowing that the actions are the same, the action currently performed by the user may be evaluated or scored according to the similarity between the action and the corresponding known action, such as the score according to the similarity value, and the similarity is 90% and the score is 90. If the similarity is 60%, the score is 60, or the similarity reaches the corresponding threshold, and the corresponding evaluation level is given. For example, the similarity is 90%, the excellence is 80%, and the similarity is 80%.

进而在本申请一实施例中，动作识别输出模块840还可以将所述移动终端当前执行的动作对应的已知动作的动作标识与所述移动终端的终端标识关联输出。用于区别于根据不同移动终端发送的运动传感数据识别得到的动作，这样如图1A所示有不止一个移动终端连接至动作识别装置，动作识别装置就可以根据运动传感数据携带的终端标识分别进行处理得到不同移动终端执行的动作，这时动作识别输出模块840也就可以将其与对应移动终端的终端标识关联输出。Further, in an embodiment of the present application, the action recognition output module 840 may further output an action identifier of a known action corresponding to the action currently performed by the mobile terminal and a terminal identifier of the mobile terminal. For distinguishing the actions identified by the motion sensing data transmitted by different mobile terminals, such that as shown in FIG. 1A, more than one mobile terminal is connected to the motion recognition device, and the motion recognition device can be based on the terminal identifier carried by the motion sensing data. The actions performed by different mobile terminals are respectively processed, and the action recognition output module 840 can also output the association with the terminal identifier of the corresponding mobile terminal.

这里需要指出的是，上述动作识别装置可以实现为PC这种电子设备，还可以为如PAD，平板电脑，手提电脑这种便携电子设备，不限于这里的描述，可以为实现各单元功能而合并为一实体或各单元功能分体设置的电子设备，动作识别装置至少包括用于存储数据的数据库和用于数据处理的处理器，可以包括内置的存储介质或独立设置的存储介质。It should be noted that the above-mentioned motion recognition device can be implemented as an electronic device such as a PC, and can also be a portable electronic device such as a PAD, a tablet computer or a laptop computer, and is not limited to the description herein, and can be combined for realizing the functions of each unit. The electronic device that is separately provided for an entity or each unit function, the action recognition device includes at least a database for storing data and a processor for data processing, and may include a built-in storage medium or a separately set storage medium.

其中，对于用于数据处理的处理器而言，在执行处理时，可以采用微处理器、中央处理器(CPU，Central Processing Unit)、数字信号处理器(DSP，Digital Singnal Processor)或可编程逻辑阵列(FPGA，Field－Programmable Gate Array)实现；对于存储介质来说，包含操作指令，该操作指令可以为计算机可执行代码，通过所述操作指令来实现上述本发明实施例如图2或7所示的动作识别流程中的各个步骤。Wherein, for the processor for data processing, a microprocessor, a central processing unit (CPU), a digital signal processor (DSP, Digital Singnal Processor) or programmable logic may be used when performing processing. An FPGA (Field-Programmable Gate Array) implementation; for a storage medium, includes an operation instruction, which may be computer executable code, by which the implementation of the present invention described above is implemented, as shown in FIG. 2 or FIG. The actions identify the various steps in the process.

动作识别装置作为硬件实体的一个示例如图11所示。所述装置包括处理器1101、存储介质1102以及至少一个外部通信接口1103；所述处理器1101、存储介质1102以及通信接口1103均通过总线1104连接。该处理器1101可以调用该存储介质1102，例如非易失性存储介质中的操作指令用于执行上述图1B、图2、图8、图9和图10所示实施例所执行的操作。An example of the motion recognition apparatus as a hardware entity is shown in FIG. The apparatus includes a processor 1101, a storage medium 1102, and at least one external communication interface 1103; the processor 1101, the storage medium 1102, and the communication interface 1103 are all connected by a bus 1104. The processor 1101 can invoke the storage medium 1102, such as operational instructions in a non-volatile storage medium for performing the operations performed by the embodiments illustrated in Figures IB, 2, 8, 9, and 10.

在本申请一实施例中，动作识别装置中的处理器1101可以调用存储介质1102中的操作指令执行以下流程：In an embodiment of the present application, the processor 1101 in the action recognition apparatus may invoke an operation instruction in the storage medium 1102 to execute the following process:

这里需要指出的是：以上涉及动作识别装置的描述，与前文动作识别方法的描述是类似的，同方法的有益效果描述，不做赘述。对于本申请动作识别装置实施例中未披露的技术细节，请参照本发明方法实施例的描述。It should be pointed out here that the above description of the motion recognition device is similar to the description of the previous motion recognition method, and the beneficial effects of the same method are described without further description. For technical details not disclosed in the embodiment of the motion recognition device of the present application, please refer to the description of the method embodiment of the present invention.

相应的，本发明实施例还提供了一种移动终端，主要实现于如图1A或图6所示的动作识别***架构，例如为图1A中的移动终端101，或图6中的移动终端3，或移动终端4，如图12所示本发明实施例中的移动终端可以包括：运动传感器1210，用于采集移动终端的运动传感数据，其中所述运动传感数据包括加速度传感数据或陀螺仪传感数据。Correspondingly, the embodiment of the present invention further provides a mobile terminal, which is mainly implemented in the motion recognition system architecture as shown in FIG. 1A or FIG. 6, for example, the mobile terminal 101 in FIG. 1A, or the mobile terminal 3 in FIG. Or the mobile terminal 4, as shown in FIG. 12, the mobile terminal in the embodiment of the present invention may include: a motion sensor 1210, configured to collect motion sensing data of the mobile terminal, where the motion sensing data includes acceleration sensing data or Gyro sensing data.

具体实现中，所述运动传感器可以包括加速度传感器或陀螺仪，还可以包括距离传感器、方向传感器等，可以获取移动终端相应的运动传感数据。In a specific implementation, the motion sensor may include an acceleration sensor or a gyroscope, and may further include a distance sensor, a direction sensor, and the like, and may acquire corresponding motion sensing data of the mobile terminal.

通信模块1220，用于将所述运动传感数据发送至动作识别装置，以使所述动作识别装置根据所述移动终端的运动传感数据获取用户动作特征数据，并将当前获取的用户动作特征数据与预设的至少一个已知动作特征数据进行比较，从而将所述至少一个已知动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。The communication module 1220 is configured to send the motion sensing data to the motion recognition device, so that the motion recognition device acquires user motion feature data according to the motion sensing data of the mobile terminal, and the currently acquired user motion feature The data is compared with the preset at least one known motion feature data such that a known action corresponding to the at least one known action feature data determines a known action as an action currently performed by the mobile terminal.

具体实现中，通信模块1220可以通过建立与动作识别装置之间的无线数据传输通道，从而通过所述无线数据传输通道接收所述移动终端发送的运动传感数据。具体的，移动终端中可以运行一个进程与动作识别装置建立socket连接，采用TCP(Transmission Control Protocol，传输控制协议)协议进行数据通信。In a specific implementation, the communication module 1220 can receive the motion sensing data sent by the mobile terminal through the wireless data transmission channel by establishing a wireless data transmission channel with the motion recognition device. Specifically, the mobile terminal can run a process to establish a socket connection with the motion recognition device, and uses TCP (Transmission Control Protocol) protocol for data communication.

在本申请一实施例中，通信模块1220建立上述无线数据传输通道的方式可以为：移动终端通过扫描动作识别装置在所在终端屏幕上显示的二维码的方式获取动作识别装置的网络信息，并根据获取到的动作识别装置的网络信息将移动终端的网络发送给动作识别装置从而建立双方的无线数据传输通道。在本申请一实施方式中，移动终端或或动作识别装置可以通过网络信息广播搜索的方式获取另一方的网络信息，还可以为通过中间网络服务器的中转获取另一方的网络信息，从而根据另一方的网络信息向另一方发送自身的网络信息，从而建立双方的无线数据传输通道。In an embodiment of the present application, the manner in which the communication module 1220 establishes the wireless data transmission channel may be: the mobile terminal acquires the network information of the motion recognition device by scanning the motion recognition device to display the two-dimensional code on the screen of the terminal, and The network of the mobile terminal is transmitted to the motion recognition device according to the acquired network information of the motion recognition device to establish a wireless data transmission channel of both parties. In an embodiment of the present application, the mobile terminal or the motion recognition device may acquire the network information of the other party by means of network information broadcast search, or may acquire the network information of the other party by the intermediate network server, so that the other party may The network information sends its own network information to the other party, thereby establishing a wireless data transmission channel for both parties.

进而在本申请一实施例中，通信模块1220向动作识别装置发送的运动传感数据可以携带该移动终端的终端标识，用于区别于其他移动终端发送的运动传感数据，这样如图1A所示有不止一个移动终端连接至动作识别装置，动作识别装置就可以根据运动传感数据携带的终端标识分别进行处理。Further, in an embodiment of the present application, the motion sensing data sent by the communication module 1220 to the motion recognition device may carry the terminal identifier of the mobile terminal for distinguishing the motion sensing data sent by other mobile terminals, as shown in FIG. 1A. It is shown that more than one mobile terminal is connected to the motion recognition device, and the motion recognition device can perform processing according to the terminal identifier carried by the motion sensor data.

在本申请一实施例中的移动终端还可以包括，The mobile terminal in an embodiment of the present application may further include

去噪模块1230，用于对所述运动传感数据进行滤波去噪处理。The denoising module 1230 is configured to perform filtering and denoising processing on the motion sensing data.

由于加速度传感器或陀螺仪传感器获取到的原始运动传感数据往往存在一些抖动和噪声，因此在本申请一实施例中，可以通过去噪模块1230对原始的传感器数据进行处理，采用低通滤波等方式进行滤波去噪处理，这样能够提高动作识别装置后续的动作相似度的判定精度，同时也能减小网络流量。Since the original motion sensing data acquired by the acceleration sensor or the gyro sensor often has some jitter and noise, in an embodiment of the present application, the original sensor data can be processed by the denoising module 1230, and low-pass filtering is adopted. The method performs filtering and denoising processing, which can improve the determination accuracy of the subsequent motion similarity of the motion recognition device, and can also reduce the network traffic.

所述通信模块1220向所述动作识别装置发送的是经过滤波去噪处理的运动传感数据。The communication module 1220 sends the motion sensing data subjected to the filtering and denoising processing to the motion recognition device.

移动终端作为硬件实体的一个示例如图13所示。所述移动终端包括处理器1301、存储介质1302以及至少一个外部通信接口1303；所述处理器1301、存储介质1302以及通信接口1303均通过总线1304连接。该处理器1301可以调用该存储介质1302，例如非易失性存储介质中的操作指令用于执行上述图12所示实施例所执行的操作。An example of a mobile terminal as a hardware entity is shown in FIG. The mobile terminal includes a processor 1301, a storage medium 1302, and at least one external communication interface 1303. The processor 1301, the storage medium 1302, and the communication interface 1303 are all connected by a bus 1304. The processor 1301 can invoke the storage medium 1302, such as operational instructions in a non-volatile storage medium for performing the operations performed by the embodiment illustrated in FIG. 12 above.

在本申请一实施例中，移动终端中的处理器1301可以调用存储介质1302中的操作指令执行以下流程：In an embodiment of the present application, the processor 1301 in the mobile terminal may invoke an operation instruction in the storage medium 1302 to perform the following process:

移动终端通过内置传感器采集移动终端的运动传感数据，其中所述运动传感数据包括加速度传感数据或陀螺仪传感数据；The mobile terminal collects motion sensing data of the mobile terminal by using a built-in sensor, wherein the motion sensing data includes acceleration sensing data or gyro sensing data;

移动终端对所述运动传感数据进行滤波去噪处理；The mobile terminal performs filtering and denoising processing on the motion sensing data;

移动终端将所述经过滤波去噪处理的运动传感数据发送至动作识别装置。The mobile terminal transmits the motion-sensing data subjected to the filtering and denoising processing to the motion recognition device.

本发明实施例中的移动终端在获取到自身的运动传感数据后，通过与动作识别装置之间的无线数据传输通道发送所述移动终端的运动传感数据，让动作识别装置识别用户使用移动终端执行的动作，从而可以通过移动终端配合动作识别装置实现各类动作识别反馈和游戏过程，极大的降低了动作识别应用的体验门槛，让更多的用户可以感受动作识别应用带来的方便和体验。After acquiring the motion sensing data of the mobile terminal, the mobile terminal in the embodiment of the present invention transmits the motion sensing data of the mobile terminal by using a wireless data transmission channel with the motion recognition device, so that the motion recognition device recognizes that the user uses the mobile device. The action performed by the terminal enables the mobile terminal to cooperate with the motion recognition device to implement various motion recognition feedback and game processes, which greatly reduces the experience threshold of the motion recognition application, so that more users can feel the convenience brought by the motion recognition application. And experience.

在本申请所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个***，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, such as: multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms. of.

上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元，即可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。The units described above as separate components may or may not be physically separated, and the components displayed as the unit may or may not be physical units, that is, may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

另外，在本发明各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated into one unit; The unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to the program instructions. The foregoing program may be stored in a computer readable storage medium, and the program is executed when executed. The foregoing storage device includes the following steps: the foregoing storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. A medium that can store program code.

或者，本申请上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本申请各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。Alternatively, the above-described integrated unit of the present application may be stored in a computer readable storage medium if it is implemented in the form of a software function module and sold or used as a stand-alone product. Based on such understanding, the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions. A computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present application. The foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a ROM, a RAM, a magnetic disk, or an optical disk.

以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。The foregoing is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application. It should be covered by the scope of protection of this application. Therefore, the scope of protection of the present application should be determined by the scope of the claims.

Claims

一种动作识别方法，应用于动作识别装置，所述方法包括：A motion recognition method is applied to a motion recognition apparatus, and the method includes:

获取移动终端的运动传感数据，所述运动传感数据包括加速度传感数据或陀螺仪传感数据；Obtaining motion sensing data of the mobile terminal, the motion sensing data including acceleration sensing data or gyro sensing data;

根据所述移动终端的运动传感数据获取第一动作特征数据；Obtaining first action feature data according to the motion sensing data of the mobile terminal;

将所述第一动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。Comparing the first action feature data with the preset at least one second action feature data to determine, from a known action corresponding to the at least one second action feature data, that a known action is currently performed by the mobile terminal action.
如权利要求1所述的动作识别方法，其中，所述移动终端的运动传感数据包括至少一个时间窗内采集到的多组运动传感数据；The motion recognition method according to claim 1, wherein the motion sensing data of the mobile terminal comprises a plurality of sets of motion sensing data collected in at least one time window;

所述根据所述移动终端的运动传感数据获取第一用户动作特征数据包括：The acquiring the first user action feature data according to the motion sensing data of the mobile terminal includes:

根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，获取所述移动终端在各时间窗内的动作特征向量，从而得到第一动作特征向量集；Obtaining an action feature vector of the mobile terminal in each time window according to the collected plurality of sets of motion sensing data of the mobile terminal in the at least one time window, thereby obtaining a first action feature vector set;

所述将所述第一用户动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作包括：The comparing the first user action feature data with the preset at least one second action feature data to determine a known action as a mobile terminal from a known action corresponding to the at least one second action feature data The currently performed actions include:

将所述第一动作特征向量集与预设的至少一个第二动作特征向量集进行比较，从而将与所述第一动作特征向量集相似度最高的第二动作特征向量集对应的已知动作确定为移动终端当前执行的动作。Comparing the first set of motion feature vectors with a preset set of at least one second motion feature vector, thereby performing a known action corresponding to the second action feature vector set having the highest similarity with the first action feature vector set Determine the action currently performed by the mobile terminal.
如权利要求2所述的动作识别方法，其中，所述将所述第一动作特征向量集与预设的至少一个第二动作特征向量集进行比较，从而将与所述第一动作特征向量集相似度最高的第二动作特征向量集对应的已知动作确定为移动终端当前执行的动作包括：The motion recognition method according to claim 2, wherein said comparing said first motion feature vector set with a preset at least one second motion feature vector set, thereby combining said first motion feature vector set The known action corresponding to the second action feature vector set with the highest similarity is determined as the action currently performed by the mobile terminal includes:

根据预设的至少一个第二动作特征向量集以及多个训练动作特征向量集训练得到动作特征分类器；Obtaining an action feature classifier according to the preset at least one second action feature vector set and the plurality of training action feature vector sets;

将所述第一动作特征向量集输入所述动作特征分类器，从而将与所述第一动作特征向量集之间相似度最高的第二动作特征向量集对应的已知动作确定为移动终端当前执行的动作。Entering the first action feature vector set into the action feature classifier, so as to determine a known action corresponding to the second action feature vector set with the highest similarity between the first action feature vector set as current The action performed.
权利要求1所述的动作识别方法，其中，所述移动终端的运动传感数据包括至少一个时间窗内采集到的多组运动传感数据；The motion recognition method of claim 1, wherein the motion sensing data of the mobile terminal comprises a plurality of sets of motion sensing data collected in at least one time window;

所述根据所述移动终端的运动传感数据获取用户动作特征数据包括：The acquiring the user action feature data according to the motion sensing data of the mobile terminal includes:

根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，获取移动终端在各时间窗内的运动轨迹数据；Obtaining motion trajectory data of the mobile terminal in each time window according to the collected plurality of sets of motion sensing data of the mobile terminal in at least one time window;

所述将所述第一用户动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作包括：The comparing the first user action feature data with the preset at least one second action feature data to determine a known action as a mobile terminal from a known action corresponding to the at least one second action feature data The currently performed actions include:

将当前获取到的第一运动轨迹数据与预设的至少一个已知动作的第二运动轨迹数据进行比较，从而将与所述第一运动轨迹数据相似度最高的已知动作的第二运动轨迹数据对应的已知动作确定为移动终端当前执行的动作。Comparing the currently acquired first motion trajectory data with the preset second motion trajectory data of the at least one known motion, so that the second motion trajectory of the known motion with the highest similarity to the first motion trajectory data is The known action corresponding to the data is determined as the action currently performed by the mobile terminal.
如权利要求2所述的动作识别方法，其中，所述方法还包括：The motion recognition method of claim 2, wherein the method further comprises:

输出所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度。An action identifier of the known action corresponding to the action and a similarity between the action and the corresponding known action are output.
如权利要求5所述的动作识别方法，其中，进一步包括：The motion recognition method of claim 5, further comprising:

获取所述移动终端的终端标识；Obtaining a terminal identifier of the mobile terminal;

所述输出所述动作对应的已知动作以及所述动作与对应的已知动作之间的相似度包括：The similarity between the known action corresponding to outputting the action and the action and the corresponding known action includes:

将所述移动终端当前执行的动作对应的已知动作的动作标识与所述移动终端的终端标识关联输出。The action identifier of the known action corresponding to the action currently performed by the mobile terminal is associated with the terminal identifier of the mobile terminal.
一种动作识别装置，包括：存储器、处理器；其中，所述存储器中存储有计算机可读指令，所述处理器执行所述存储中的计算机可读指令，用于：A motion recognition apparatus includes: a memory, a processor; wherein the memory stores computer readable instructions, and the processor executes the computer readable instructions in the storing, for:

获取移动终端的运动传感数据，所述运动传感数据包括加速度传感数据或陀螺仪传感数据；Obtaining motion sensing data of the mobile terminal, the motion sensing data including acceleration sensing data or gyro sensing data;

根据所述移动终端的运动传感数据获取第一动作特征数据；Obtaining first action feature data according to the motion sensing data of the mobile terminal;

将所述第一动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。Comparing the first action feature data with the preset at least one second action feature data to determine, from a known action corresponding to the at least one second action feature data, that a known action is currently performed by the mobile terminal action.
如权利要求7所述的动作识别装置，其中，所述移动终端的运动传感数据包括至少一个时间窗内采集到的多组运动传感数据；The motion recognition apparatus according to claim 7, wherein the motion sensing data of the mobile terminal comprises a plurality of sets of motion sensing data collected in at least one time window;

所述处理器进一步执行所述计算机可读指令，用于：根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，获取所述移动终端在各时间窗内的动作特征向量，从而得到第一动作特征向量集；The processor further executes the computer readable instructions for acquiring an action of the mobile terminal in each time window according to the collected plurality of sets of motion sensing data of the mobile terminal in at least one time window a feature vector to obtain a first set of motion feature vectors;

所述处理器进一步执行所述计算机可读指令，用于：将所述第一动作特征向量集与预设的至少一个第二动作特征向量集进行比较，从而将与所述第一动作特征向量集相似度最高的第二动作特征向量集对应的已知动作确定为移动终端当前执行的动作。The processor further executes the computer readable instructions for comparing the first set of motion feature vectors with a preset set of at least one second motion feature vector to thereby interact with the first motion feature vector The known action corresponding to the set of the second action feature vector with the highest similarity is determined as the action currently performed by the mobile terminal.
如权利要求8所述的动作识别装置，其中，所述处理器进一步执行所述计算机可读指令，用于：The motion recognition apparatus according to claim 8, wherein said processor further executes said computer readable instructions for:

根据预设的至少一个第二动作特征向量集以及多个训练动作特征向量集训练得到动作特征分类器；Obtaining an action feature classifier according to the preset at least one second action feature vector set and the plurality of training action feature vector sets;

将所述第一动作特征向量集输入所述动作特征分类器，从而将与所述第一动作特征向量集之间相似度最高的第二动作特征向量集对应的已知动作确定为移动终端当前执行的动作。Entering the first action feature vector set into the action feature classifier, so as to determine a known action corresponding to the second action feature vector set with the highest similarity between the first action feature vector set as current The action performed.
权利要求7所述的动作识别装置，其中，所述移动终端的运动传感数据包括至少一个时间窗内采集到的多组运动传感数据；The motion recognition device of claim 7, wherein the motion sensing data of the mobile terminal comprises a plurality of sets of motion sensing data collected in at least one time window;

所述处理器进一步执行所述计算机可读指令，用于：根据采集到的所述移动终端在至少一个时间窗内的多组运动传感数据，获取移动终端在各时间窗内的运动轨迹数据；The processor further executes the computer readable instructions for: acquiring motion trajectory data of the mobile terminal in each time window according to the collected plurality of sets of motion sensing data of the mobile terminal in at least one time window ;

所述处理器进一步执行所述计算机可读指令，用于：将当前获取到的第一运动轨迹数据与预设的至少一个已知动作的第二运动轨迹数据进行比较，从而将与所述第一运动轨迹数据相似度最高的已知动作的第二运动轨迹数据对应的已知动作确定为移动终端当前执行的动作。The processor further executes the computer readable instructions for comparing the currently acquired first motion trajectory data with a preset second motion trajectory data of at least one known motion, thereby The known action corresponding to the second motion trajectory data of the known action with the highest similarity of the motion trajectory data is determined as the action currently performed by the mobile terminal.
如权利要求7所述的动作识别装置，其中，所述处理器进一步执行所述计算机可读指令，用于：The motion recognition apparatus of claim 7, wherein said processor further executes said computer readable instructions for:

与所述移动终端建立无线数据传输通道；Establishing a wireless data transmission channel with the mobile terminal;

通过所述无线数据传输通道接收所述移动终端发送的运动传感数据。Receiving motion sensing data transmitted by the mobile terminal through the wireless data transmission channel.
如权利要求8所述的动作识别装置，其中，所述处理器进一步执行所述计算机可读指令，用于：输出所述动作对应的已知动作的动作标识以及所述动作与对应的已知动作之间的相似度。The motion recognition apparatus according to claim 8, wherein said processor further executes said computer readable instructions for: outputting an action identification of said known action corresponding to said action and said action and corresponding known The similarity between actions.
如权利要求12所述的动作识别装置，其中，所述处理器进一步执行所述计算机可读指令，用于：获取移动终端的运动传感数据的同时获取所述移动终端的终端标识；The action recognition apparatus according to claim 12, wherein the processor further executes the computer readable instructions for: acquiring a motion sensor data of the mobile terminal while acquiring a terminal identifier of the mobile terminal;

所述处理器进一步执行所述计算机可读指令，用于：将所述移动终端当前执行的动作对应的已知动作的动作标识与所述移动终端的终端标识关联输出。The processor further executes the computer readable instructions for: correlating an action identifier of a known action corresponding to an action currently performed by the mobile terminal with a terminal identifier of the mobile terminal.
一种移动终端，所述移动终端包括：存储器、处理器；其中，所述存储器中存储有计算机可读指令，所述处理器执行所述存储中的计算机可读指令，用于：A mobile terminal, the mobile terminal comprising: a memory, a processor; wherein the memory stores computer readable instructions, and the processor executes the computer readable instructions in the storing, for:

采集移动终端的运动传感数据，其中所述运动传感数据包括加速度传感数据或陀螺仪传感数据；Collecting motion sensing data of the mobile terminal, wherein the motion sensing data includes acceleration sensing data or gyro sensing data;

将所述运动传感数据发送至动作识别装置，以使所述动作识别装置根据所述移动终端的运动传感数据获取第一动作特征数据，并将所述第一动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为移动终端当前执行的动作。Transmitting the motion sensing data to the motion recognition device, so that the motion recognition device acquires the first motion feature data according to the motion sensing data of the mobile terminal, and the first motion feature data and the preset The at least one second action feature data is compared to determine, from the known actions corresponding to the at least one second action feature data, that a known action is an action currently performed by the mobile terminal.
如权利要求14所述的移动终端，其中，所述处理器进一步执行所述计算机可读指令，用于：对所述运动传感数据进行滤波去噪处理；The mobile terminal of claim 14, wherein the processor further executes the computer readable instructions for: performing filter denoising processing on the motion sensing data;

向所述动作识别装置发送的是经过滤波去噪处理的运动传感数据。Transmitted to the motion recognition device is motion sensing data subjected to filter denoising processing.
一种动作识别***，包括动作识别装置和至少一个移动终端，其中：A motion recognition system includes a motion recognition device and at least one mobile terminal, wherein:

所述各移动终端用于通过内置的传感器采集移动终端的运动传感数据，并将所述运动传感数据发送至所述动作识别装置，其中所述运动传感数据包括加速度传感数据或陀螺仪传感数据；Each of the mobile terminals is configured to collect motion sensing data of the mobile terminal by using a built-in sensor, and send the motion sensing data to the motion recognition device, where the motion sensing data includes acceleration sensing data or a gyro Instrument sensing data;

所述动作识别装置用于接收所述各移动终端发送的所述运动传感数据，根据所述各移动终端的运动传感数据获取与所述各移动终端的设备标识对应的第一动作特征数据；将所述第一动作特征数据与预设的至少一个第二动作特征数据进行比较，以从所述至少一个第二动作特征数据对应的已知动作中确定一个已知动作为所述各移动终端当前执行的动作。The motion recognition device is configured to receive the motion sensing data sent by each mobile terminal, and acquire first motion feature data corresponding to the device identifiers of the mobile terminals according to the motion sensing data of each mobile terminal. Comparing the first action feature data with the preset at least one second action feature data to determine a known action as the move from a known action corresponding to the at least one second action feature data The action currently performed by the terminal.
如权利要求16所述的动作识别***，其中，The motion recognition system according to claim 16, wherein

所述移动终端，进一步用于对所述运动传感数据进行滤波去噪处理，向所述动作识别装置发送经过滤波去噪处理的运动传感数据。The mobile terminal is further configured to perform filtering and denoising processing on the motion sensing data, and send motion sensing data subjected to filtering and denoising processing to the motion recognition device.
一种非易失性计算机可读存储介质，存储有计算机可读指令，可以使至少一个处理器执行如权利要求1-6任一项所述的方法。A non-transitory computer readable storage medium storing computer readable instructions for causing at least one processor to perform the method of any of claims 1-6.