WO2022037229A1

WO2022037229A1 - Human image positioning methods and display devices

Info

Publication number: WO2022037229A1
Application number: PCT/CN2021/101020
Authority: WO
Inventors: 杨鲁明; 鲍姗娟; 王之奎; 王大勇; 于文钦
Original assignee: 海信视像科技股份有限公司
Priority date: 2020-08-21
Filing date: 2021-06-18
Publication date: 2022-02-24
Also published as: CN116034580A

Abstract

Provided by the present application are display devices and human image positioning methods, where in a method, a human image position is first detected by means of a camera; and whether to enable adjustment of the orientation of the camera is determined by means of comparing the human image position and a preset region in a proofing image; if the current human image position is not within the preset region, controlling the camera to rotate, so as to cause the imaging position of a user to be located in the middle region of an image. The present method performs human detection on a photographed image and determines a specific position by means of image detection, and the camera is driven to perform fine adjustment, so as to achieve accurate positioning.

Description

人像定位方法及显示设备Portrait positioning method and display device

本申请要求在2020年8月21日提交中国专利局、申请号为202010849806.3、申请名称为“一种显示设备人像定位方法”的中国专利申请的优先权，在2020年8月21日提交中国专利局、申请号为202010847711.8、申请名称为“一种人脸特征值创建方法、人物锁定追踪方法及显示设备”的中国专利申请的优先权，和在2020年12月31日提交中国专利局、申请号为202011620179.2、申请名称为一种显示设备人像定位方法”的中国专利申请的优先权，以及在2021年2月4日提交中国专利局、申请号为202110155019.3、申请名称为“一种人脸特征值创建方法、人物锁定追踪方法及显示设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed on August 21, 2020 with the application number 202010849806.3 and the application name is "A method for positioning portraits of display devices", and the Chinese patent application is filed on August 21, 2020 The priority of the Chinese patent application with the application number of 202010847711.8 and the application title of "A method for creating a facial feature value, a method for tracking a person's lock and a display device", and submitted to the Chinese Patent Office on December 31, 2020, the application The priority of the Chinese patent application No. 202011620179.2 and the application title is a method for positioning a portrait of a display device”, and the application number is 202110155019.3 submitted to the Chinese Patent Office on February 4, 2021, and the application title is “A Facial Feature Value Creation Method, Person Lock Tracking Method, and Display Device" of the Chinese Patent Application, the entire contents of which are incorporated herein by reference.

技术领域technical field

本申请涉及人像定位技术领域，尤其涉及人像定位方法及显示设备。The present application relates to the technical field of portrait positioning, and in particular, to a portrait positioning method and a display device.

背景技术Background technique

显示设备，例如智能电视，以及带有智能显示屏幕的产品等，智能显示设备中配置有智能软件***，智能软件***可以实现多种功能和预装多种应用程序，给用户提供丰富内容服务。例如，显示设备上可以设置有摄像头等图像采集设备，通过摄像头获取用户图像，并配合“肢体运动”应用程序，使显示设备能够实时显示用户身体图像。当用户肢体动作发生变化时，应用程序会也会显示变化后的图像，并通过检测程序对肢体动作的形态进行检测、矫正，达到运动指导的效果。Display devices, such as smart TVs and products with smart display screens, etc., are equipped with smart software systems, which can implement various functions and pre-install various applications to provide users with rich content services. For example, an image acquisition device such as a camera can be set on the display device, and the user's image can be obtained through the camera and cooperated with the "limb movement" application, so that the display device can display the user's body image in real time. When the user's body movement changes, the application will also display the changed image, and detect and correct the shape of the body movement through the detection program to achieve the effect of exercise guidance.

带有摄像头的显示设备中配置的“肢体运动”应用程序包括视频通话、AI健身及摄像头人像追踪等。例如，在健身跟练模式下，若存在多人场景，需要利用摄像头的转动对锁定的某一人进行跟踪，并对动作进行打点，生成跟练数据，统计跟练结果。"Body movement" applications deployed in display devices with cameras include video calling, AI fitness, and camera portrait tracking. For example, in the fitness follow-up mode, if there is a multi-person scene, it is necessary to use the rotation of the camera to track a locked person, and to manage the movements, generate follow-up data, and count the follow-up results.

发明内容SUMMARY OF THE INVENTION

本申请提供一些人像定位方法及显示设备。The present application provides some portrait positioning methods and display devices.

第一方面，本申请提供一些显示设备，包括显示器、外部装置接口以及控制器，所述显示设备通过外部装置接口外接有摄像头。其中，所述摄像头可以转动拍摄角度，用于拍摄不同视角下的图像；所述显示器被配置为呈现用户界面；所述控制器被配置为执行以下程序步骤：获取用户输入的用于启动摄像头应用的控制指令；响应于所述控制指令，从所述摄像头拍摄的图像中检测人像位置；按照所述人像位置向所述摄像头发送旋转指令，以使所述人像位置位于预设判断区域内。In a first aspect, the present application provides some display devices, including a display, an external device interface, and a controller, and the display device is externally connected with a camera through the external device interface. Wherein, the camera can rotate the shooting angle for shooting images under different viewing angles; the display is configured to present a user interface; the controller is configured to execute the following program steps: obtaining a user input for starting the camera application The control instruction; in response to the control instruction, detect the portrait position from the image captured by the camera; send a rotation instruction to the camera according to the portrait position, so that the portrait position is located in the preset judgment area.

第二方面，本申请还提供一些显示设备，包括显示器、摄像头以及控制器。其中，所述摄像头可以转动拍摄角度，用于拍摄不同视角下的图像；所述显示器被配置为呈现用户界面；所述控制器被配置为执行以下程序步骤：获取用户输入的用于启动摄像头应用的控制指令；响应于所述控制指令，从所述摄像头拍摄的图像中检测人像位置；按照所述人像位置向所述摄像头发送旋转指令，以使所述人像位置位于预设判断区域内。In a second aspect, the present application also provides some display devices, including a display, a camera, and a controller. Wherein, the camera can rotate the shooting angle for shooting images under different viewing angles; the display is configured to present a user interface; the controller is configured to execute the following program steps: obtaining a user input for starting the camera application The control instruction; in response to the control instruction, detect the portrait position from the image captured by the camera; send a rotation instruction to the camera according to the portrait position, so that the portrait position is located in the preset judgment area.

第三方面，本申请还提供一些人像定位方法，应用于上述显示设备，包括：获取用户输入的用于启动摄像头应用的控制指令；响应于所述控制指令，从所述摄像头拍摄的图像中检测人像位置；按照所述人像位置向所述摄像头发送旋转指令，以使所述人像位置位于预设判断区域内。In a third aspect, the present application also provides some portrait positioning methods, which are applied to the above-mentioned display device, including: acquiring a control instruction input by a user for starting a camera application; and responding to the control instruction, detecting from an image captured by the camera. Portrait position; send a rotation instruction to the camera according to the portrait position, so that the portrait position is within the preset judgment area.

第四方面，本申请提供了一种显示设备，包括：显示器，被配置为呈现用户界面；摄像头，被配置为采集人物在使用摄像头应用时产生的人物图像信息；与所述显示器和所述摄像头连接的控制器，所述控制器被配置为：获取所述摄像头采集的人物图像信息；识别所述人物图像信息，确定锁定追踪人物，以及，创建所述锁定追踪人物的面部特征信息和特定肢体ID信息；将所述锁定追踪人物的特定肢体ID信息和面部特征信息建立关联关系，得到所述锁定追踪人物的人脸特征值。In a fourth aspect, the present application provides a display device, comprising: a display configured to present a user interface; a camera configured to collect character image information generated by a character when using a camera application; A connected controller, the controller is configured to: acquire person image information collected by the camera; identify the person image information, determine a locked and tracked person, and create facial feature information and specific limbs of the locked and tracked person ID information; establish an association relationship between the specific limb ID information of the locked and tracked person and the facial feature information, and obtain the facial feature value of the locked and tracked person.

第五方面，本申请还提供了一种显示设备，包括：显示器，被配置为呈现用户界面；摄像头，被配置为采集人物在使用摄像头应用时产生的人物图像信息；存储器，被配置为存储在确定锁定追踪人物时创建的人脸特征值；与所述显示器、摄像头和所述存储器连接的控制器，所述控制器被配置为：获取锁定追踪人物的人脸特征值和所述摄像头采集的人物图像信息；如果所述锁定追踪人物在所述人物图像信息中消失，则基于所述人物图像信息中的当前人物重新创建当前人脸特征值；如果所述当前人脸特征值与所述人脸特征值的特征相似度满足阈值条件，则将所述锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物；基于所述当前人脸特征值，对所述锁定追踪人物进行锁定追踪。In a fifth aspect, the present application also provides a display device, comprising: a display, configured to present a user interface; a camera, configured to collect character image information generated when a character uses a camera application; a memory, configured to store in Determine the facial feature value created when locking and tracking the character; a controller connected with the display, the camera and the memory, the controller is configured to: obtain the facial feature value of the locked and tracked character and the data collected by the camera. Character image information; if the locked and tracked character disappears in the character image information, then recreate the current face feature value based on the current character in the character image information; if the current face feature value and the person If the feature similarity of the face feature value satisfies the threshold condition, the locked and tracked person is determined as the current person whose feature similarity meets the threshold condition; based on the current face feature value, the locked and tracked character is locked and tracked.

第六方面，本申请还提供了一种人脸特征值创建方法，所述方法包括：获取所述摄像头采集的人物图像信息；识别所述人物图像信息，确定锁定追踪人物，以及，创建所述锁定追踪人物的面部特征信息和特定肢体ID信息；将所述锁定追踪人物的特定肢体ID信息和面部特征信息建立关联关系，得到所述锁定追踪人物的人脸特征值。In a sixth aspect, the present application further provides a method for creating a face feature value, the method comprising: acquiring the person image information collected by the camera; identifying the person image information, determining to lock and track the person, and creating the The facial feature information and the specific body ID information of the locked and tracked person are locked; the specific body ID information and the facial feature information of the locked and tracked person are established in an association relationship to obtain the facial feature value of the locked and tracked person.

第七方面，本申请还提供了一种人物锁定追踪方法，所述方法包括：获取锁定追踪人物的人脸特征值和所述摄像头采集的人物图像信息；如果所述锁定追踪人物在所述人物图像信息中消失，则基于所述人物图像信息中的当前人物重新创建当前人脸特征值；如果所述当前人脸特征值与所述人脸特征值的特征相似度满足阈值条件，则将所述锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物；基于所述当前人脸特征值，对所述锁定追踪人物进行锁定追踪。In a seventh aspect, the present application also provides a method for locking and tracking a person, the method comprising: acquiring a face feature value of a person to be locked and tracked and the image information of the person collected by the camera; disappear in the image information, then recreate the current face feature value based on the current character in the character image information; if the feature similarity between the current face feature value and the face feature value satisfies the threshold condition, then The locked and tracked person is determined as a current person whose feature similarity satisfies a threshold condition; and based on the current face feature value, the locked and tracked person is locked and tracked.

第八方面，本申请还提供了一种存储介质，该计算机存储介质可存储有程序，该程序执行时可实现包括本申请提供的人脸特征值创建方法和人物锁定追踪方法各实施例中的部分或全部步骤。In an eighth aspect, the present application also provides a storage medium, and the computer storage medium can store a program, and when the program is executed, it can realize the method for creating a face feature value and the method for tracking a person in the various embodiments provided by the present application. some or all of the steps.

附图说明Description of drawings

为了更清楚地说明本申请的实施方式，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the embodiments of the present application more clearly, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, without creative efforts Additional drawings can be obtained from these drawings.

图1中示例性示出了根据一些实施例的显示设备与控制装置之间操作场景的示意图；FIG. 1 exemplarily shows a schematic diagram of an operation scene between a display device and a control apparatus according to some embodiments;

图2中示例性示出了根据一些实施例的显示设备200的硬件配置框图；FIG. 2 exemplarily shows a hardware configuration block diagram of a display device 200 according to some embodiments;

图3中示例性示出了根据一些实施例的控制设备100的硬件配置框图；FIG. 3 exemplarily shows a hardware configuration block diagram of the control device 100 according to some embodiments;

图4为本申请一些实施例中显示设备与摄像头布置结构示意图；FIG. 4 is a schematic diagram of the arrangement structure of a display device and a camera in some embodiments of the present application;

图5为本申请一些实施例中摄像头结构示意图；5 is a schematic structural diagram of a camera in some embodiments of the present application;

图6为本申请一些实施例中像素点位置示意图；6 is a schematic diagram of pixel positions in some embodiments of the present application;

图7a为本申请一些实施例中调整前显示设备场景俯视示意图；7a is a schematic top view of a scene of a display device before adjustment in some embodiments of the present application;

图7b为本申请一些实施例中调整后显示设备场景俯视示意图；FIG. 7b is a schematic top view of a scene of a display device after adjustment in some embodiments of the present application;

图7c为本申请一些实施例中调整前显示设备场景侧视示意图；7c is a schematic side view of a scene of a display device before adjustment in some embodiments of the present application;

图7d为本申请一些实施例中调整后显示设备场景俯视示意图；7d is a schematic top view of a scene of a display device after adjustment in some embodiments of the present application;

图8为本申请一些实施例中人像中心与图像中心示意图；8 is a schematic diagram of a portrait center and an image center in some embodiments of the present application;

图9为本申请一些实施例中计算旋转角度过程的几何关系示意图；9 is a schematic diagram of the geometric relationship of the process of calculating the rotation angle in some embodiments of the present application;

图10为本申请一些实施例中调整旋转角度过程示意图；10 is a schematic diagram of a process of adjusting the rotation angle in some embodiments of the present application;

图11为本申请一些实施例中蹲姿状态示意图；11 is a schematic diagram of a squatting state in some embodiments of the present application;

图12为本申请一些实施例中站姿状态示意图；12 is a schematic diagram of a standing posture state in some embodiments of the present application;

图13中示例性示出了根据一些实施例的人脸特征值创建方法的数据流图；Figure 13 exemplarily shows a data flow diagram of a method for creating a facial feature value according to some embodiments;

图14中示例性示出了根据一些实施例的人脸特征值创建方法的流程图；FIG. 14 exemplarily shows a flowchart of a method for creating a facial feature value according to some embodiments;

图15中示例性示出了根据一些实施例的AI健身场景下人物图像信息的显示示意图；FIG. 15 exemplarily shows a schematic diagram of displaying character image information in an AI fitness scene according to some embodiments;

图16中示例性示出了根据一些实施例的视频通话场景下人物图像信息的示意图；FIG. 16 exemplarily shows a schematic diagram of person image information in a video call scenario according to some embodiments;

图17(a)示例性示出了一些实施例中的单人AI健身场景下人物锁定追踪的示意图；Figure 17(a) exemplarily shows a schematic diagram of person locking tracking in a single-person AI fitness scene in some embodiments;

图17(b)中示例性示出了根据一些实施例的多人AI健身场景下的多人图像示意图；FIG. 17(b) exemplarily shows a schematic diagram of a multi-person image in a multi-person AI fitness scene according to some embodiments;

图18中示例性示出了根据一些实施例的单人视频通话场景下的显示示意图；FIG. 18 exemplarily shows a schematic diagram of a display in a single-person video call scenario according to some embodiments;

图19中示例性示出了根据一些实施例的多人视频通话场景下调整摄像头的示意图；FIG. 19 exemplarily shows a schematic diagram of adjusting a camera in a multi-person video call scenario according to some embodiments;

图20中示例性示出了根据一些实施例的人物锁定追踪方法的数据流图；Figure 20 exemplarily shows a data flow diagram of a method for character lock tracking according to some embodiments;

图21中示例性示出了根据一些实施例的人物锁定追踪方法的流程图；Figure 21 exemplarily shows a flowchart of a method for character lock tracking according to some embodiments;

图22中示例性示出了根据一些实施例的存在人物识别丢失的显示示意图。FIG. 22 exemplarily shows a schematic diagram of a display that there is a loss of person identification according to some embodiments.

具体实施方式detailed description

为使本申请的目的和实施方式更加清楚，下面将结合本申请示例性实施例中的附图，对本申请示例性实施方式进行清楚、完整地描述，显然，描述的示例性实施例仅是本申请一部分实施例，而不是全部的实施例。In order to make the purpose and implementation of the present application clearer, the exemplary embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only the Some embodiments are claimed, but not all embodiments.

需要说明的是，本申请中对于术语的简要说明，仅是为了方便理解接下来描述的实施方式，而不是意图限定本申请的实施方式。除非另有说明，这些术语应当按照其普通和通常的含义理解。It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the embodiments described below, rather than intended to limit the embodiments of the present application. Unless otherwise specified, these terms are to be understood according to their ordinary and ordinary meanings.

本申请中说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”等是用于区别类似或同类的对象或实体，而不必然意味着限定特定的顺序或先后次序，除非另外注明。应该理解这样使用的用语在适当情况下可以互换。The terms "first", "second", "third", etc. in the description and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit specific Sequential or sequential, unless otherwise noted. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

术语“包括”和“具有”以及他们的任何变形，意图在于覆盖但不排他的包含，例如，包含了一系列组件的产品或设备不必限于清楚地列出的所有组件，而是可包括没有清楚地列出的或对于这些产品或设备固有的其它组件。The terms "comprising" and "having", and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that incorporates a series of components is not necessarily limited to all components explicitly listed, but may include no explicit other components listed or inherent to these products or devices.

术语“模块”是指任何已知或后来开发的硬件、软件、固件、人工智能、模糊逻辑或硬件或/和软件代码的组合，能够执行与该元件相关的功能。The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code capable of performing the functions associated with that element.

图1为根据实施例中显示设备与控制装置之间操作场景的示意图。如图1所示，用户可通过智能终端300或控制装置100操作显示设备200。FIG. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in FIG. 1 , the user can operate the display device 200 through the smart terminal 300 or the control device 100 .

控制装置100可以是遥控器，遥控器和显示设备的通信包括红外协议通信或蓝牙协议通信，及其他短距离通信方式，通过无线或有线方式来控制显示设备200。用户可以通过遥控器上按键、语音输入、控制面板输入等输入用户指令，来控制显示设备200。The control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-distance communication methods, and the display device 200 is controlled wirelessly or wiredly. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like.

在一些实施例中，也可以使用智能终端300(如移动终端、平板电脑、计算机、笔记本电脑等)以控制显示设备200。例如，使用在智能设备上运行的应用程序控制显示设备200。In some embodiments, a smart terminal 300 (eg, a mobile terminal, a tablet computer, a computer, a notebook computer, etc.) can also be used to control the display device 200 . For example, the display device 200 is controlled using an application running on the smart device.

在一些实施例中，显示设备200还可以采用除了控制装置100和智能设备300之外的方式进行控制，例如，可以通过显示设备200设备内部配置的获取语音指令的模块直接接收用户的语音指令控制，也可以通过显示设备200设备外部设置的语音控制设备来接收用户的语音指令控制。In some embodiments, the display device 200 can also be controlled in a manner other than the control apparatus 100 and the smart device 300. For example, the module for acquiring voice commands configured inside the display device 200 can directly receive the user's voice command for control. , the user's voice command control can also be received through a voice control device provided outside the display device 200 device.

在一些实施例中，显示设备200还与服务器400进行数据通信。可允许显示设备200通过局域网(LAN)、无线局域网(WLAN)和其他网络进行通信连接。服务器400可以向显示设备200提供各种内容和互动。In some embodiments, the display device 200 is also in data communication with the server 400 . The display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200 .

图2示例性示出了根据示例性实施例中控制装置100的配置框图。如图2所示，控制装置100包括控制器110、通信接口130、用户输入/输出接口140、存储器、供电电源。控制装置100可接收用户的输入操作指令，且将操作指令转换为显示设备200可识别和响应的指令，起用用户与显示设备200之间交互中介作用。FIG. 2 exemplarily shows a configuration block diagram of the control apparatus 100 according to an exemplary embodiment. As shown in FIG. 2 , the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply. The control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, and play an intermediary role between the user and the display device 200 .

图3示出了根据示例性实施例中显示设备200的硬件配置框图。FIG. 3 is a block diagram showing a hardware configuration of the display apparatus 200 according to an exemplary embodiment.

显示设备200包括调谐解调器210、通信器220、检测器230、外部装置接口240、控制器250、显示器275、音频输出接口285、存储器、供电电源、用户接口中的至少一些。 Display apparatus 200 includes at least some of tuner 210, communicator 220, detector 230, external device interface 240, controller 250, display 275, audio output interface 285, memory, power supply, and user interface.

显示器275包括用于呈现画面的显示屏组件，以及驱动图像显示的驱动组件，用于接收源自控制器输出的图像信号，进行显示视频内容、图像内容以及菜单操控界面的组件以及用户操控UI界面。The display 275 includes a display screen component for presenting pictures, and a driving component for driving image display, for receiving image signals output from the controller, components for displaying video content, image content, and menu manipulation interfaces, and user manipulation UI interfaces .

显示器275可为液晶显示器、OLED显示器、以及投影显示器，还可以为一些投影装置和投影屏幕。The display 275 can be a liquid crystal display, an OLED display, and a projection display, as well as some projection devices and projection screens.

通信器220是用于根据各种通信协议类型与外部设备或服务器进行通信的组件。例如：通信器可以包括Wifi模块，蓝牙模块，有线以太网模块等其他网络通信协议芯片或近场通信协议芯片，以及红外接收器中的至少一些。显示设备200可以通过通信器220与外部控制设备100或服务器400建立控制信号和数据信号的发送和接收。The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example, the communicator may include a Wifi module, a Bluetooth module, a wired Ethernet module and other network communication protocol chips or near field communication protocol chips, and at least some of the infrared receivers. The display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220 .

用户接口，可用于接收控制装置100(如：红外遥控器等)的控制信号。The user interface can be used to receive control signals from the control device 100 (eg, an infrared remote control, etc.).

检测器230用于采集外部环境或与外部交互的信号。例如，检测器230包括光接收器，用于采集环境光线强度的传感器；或者，检测器230包括图像采集器，如摄像头，可以用于采集外部环境场景、用户的属性或用户交互手势，再或者，检测器230 包括声音采集器，如麦克风等，用于接收外部声音。The detector 230 is used to collect external environment or external interaction signals. For example, the detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes or user interaction gestures, or , the detector 230 includes a sound collector, such as a microphone, for receiving external sound.

外部装置接口240可以包括但不限于如下：高清多媒体接口接口(HDMI)、模拟或数据高清分量输入接口(分量)、复合视频输入接口(CVBS)、USB输入接口(USB)、RGB端口等任一个或多个接口。也可以是上述多个接口形成的复合性的输入/输出接口。The external device interface 240 may include but is not limited to the following: any one of a high-definition multimedia interface interface (HDMI), an analog or data high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. or multiple interfaces. It may also be a composite input/output interface formed by a plurality of the above-mentioned interfaces.

控制器250和调谐解调器210可以位于不同的分体设备中，即调谐解调器210也可在控制器250所在的主体设备的外置设备中，如外置机顶盒等。The controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

控制器250，通过存储在存储器上中各种软件控制程序，来控制显示设备的工作和响应用户的操作。控制器250控制显示设备200的整体操作。例如：响应于接收到用于选择在显示器275上显示UI对象的用户命令，控制器250便可以执行与由用户命令选择的对象有关的操作。The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display apparatus 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user command.

对象可以是可选对象中的任何一个，例如超链接、图标或其他可操作的控件。与所选择的对象有关操作有：显示连接到超链接页面、文档、图像等操作，或者执行与所述图标相对应程序的操作。Objects can be any of the optional objects, such as hyperlinks, icons, or other actionable controls. The operations related to the selected object include: displaying operations connected to hyperlinked pages, documents, images, etc., or executing operations of programs corresponding to the icons.

在一些实施例中，用户可在显示器275上显示的图形用户界面(GUI)输入用户命令，则用户输入接口通过图形用户界面(GUI)接收用户输入命令。或者，用户可通过输入特定的声音或手势进行输入用户命令，则用户输入接口通过传感器识别出声音或手势，来接收用户输入命令。In some embodiments, the user may input user commands on a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input commands through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

“用户界面”可以指应用程序或操作***与用户之间进行交互和信息交换的介质接口，它实现信息的内部形式与用户可以接受形式之间的转换。用户界面常用的表现形式是图形用户界面(Graphic User Interface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的一个图标、窗口、控件等界面元素，其中控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。在一些实施例中，***可以包括内核(Kernel)、命令解析器(shell)、文件***和应用程序。内核、shell和文件***一起组成了基本的操作***结构，它们让用户可以管理文件、运行程序并使用***。上电后，内核启动，激活内核空间，抽象硬件、初始化硬件参数等，运行并维护虚拟内存、调度器、信号及进程间通信(IPC)。内核启动后，再加载Shell和用户应用程序。应用程序在启动后被编译成机器码，形成一个进程。"User interface" can refer to the medium interface for interaction and information exchange between application programs or operating systems and users, which realizes the conversion between the internal form of information and the form acceptable to users. The commonly used form of user interface is Graphical User Interface (GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of the electronic device, wherein the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc. visual interface elements. In some embodiments, a system may include a kernel (Kernel), a command parser (shell), a file system, and applications. Together, the kernel, shell, and file system make up the basic operating system structures that allow users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. An application is compiled into machine code after startup, forming a process.

显示设备是基于Internet应用技术，具备开放式操作***与芯片，拥有开放式应用平台，可实现双向人机交互功能，集影音、娱乐、数据等多种功能于一体的产品，用于满足用户多样化和个性化需求。The display device is based on Internet application technology, has an open operating system and chip, has an open application platform, can realize two-way human-computer interaction function, and integrates various functions such as audio-visual, entertainment, data, etc. customization and individual needs.

例如，智能电视上可以设置有摄像头等图像采集设备，通过摄像头获取用户图像，并配合“肢体运动”应用程序，使智能电视能够实时显示用户身体图像。当用户肢体动作发生变化时，应用程序会也会显示变化后的图像，并通过检测程序对肢体动作的形态进行检测、矫正，达到运动指导的效果。For example, a smart TV can be provided with an image acquisition device such as a camera, and the user's image can be obtained through the camera, and cooperated with the "limb movement" application, so that the smart TV can display the user's body image in real time. When the user's body movement changes, the application will also display the changed image, and detect and correct the shape of the body movement through the detection program to achieve the effect of exercise guidance.

但是，智能电视内置或外接的摄像头设备位置和拍摄角度固定，导致摄像头进行图像采集的可视角度有限，摄像头预览显示区域只能是摄像头朝向的区域图像。根据用户所处的位置变化以及运动过程的肢体动作变化，用户很容易移动至预览显示区域之外，导致摄像头无法采集到用户的完整图像，影响显示效果。并且，为了重新获得完整图像，用户需要中止运动动作，回到合适的位置，因此会降低用户体验。However, the position and shooting angle of the built-in or external camera device of the smart TV are fixed, resulting in a limited viewing angle of the camera for image acquisition, and the preview display area of the camera can only be the image of the area facing the camera. According to the change of the user's position and the changes of body movements during the exercise, the user can easily move out of the preview display area, so that the camera cannot capture the complete image of the user, which affects the display effect. Also, in order to regain a complete image, the user needs to suspend the motion action and return to a proper position, thus degrading the user experience.

本申请一些实施例中，如图4所示，摄像头232作为一种检测器230可以内置或外接显示设备200上，在启动运行后，摄像头232可以检测图像数据。摄像头232可以通过接口部件与控制器250连接，从而将检测的图像数据发送给控制器250进行处理。摄像头所处的角度，如其在竖直方向上的俯仰角度或者其在水平方向上的角度，关乎摄像头的视场范围。In some embodiments of the present application, as shown in FIG. 4 , as a detector 230 , the camera 232 may be built in or externally connected to the display device 200 , and after the operation is started, the camera 232 may detect image data. The camera 232 can be connected with the controller 250 through an interface component, so as to send the detected image data to the controller 250 for processing. The angle at which the camera is located, such as its tilt angle in the vertical direction or its angle in the horizontal direction, is related to the field of view of the camera.

为了检测图像，摄像头232可以包括镜头组件和云台组件。也可以将包括镜头组件和云台组件的摄像头称为云台摄像头。其中，镜头组件可以是基于CCD(Charge Coupled Device，电荷耦合器件)或CMOS(Complementary Metal Oxide Semiconductor，互补金属氧化物半导体)的检测原理，以根据用户图像生成电信号的图像数据。To detect images, the camera 232 may include a lens assembly and a pan/tilt assembly. A camera including a lens assembly and a gimbal assembly can also be referred to as a gimbal camera. The lens assembly may be based on the detection principle of CCD (Charge Coupled Device, charge coupled device) or CMOS (Complementary Metal Oxide Semiconductor, complementary metal oxide semiconductor), so as to generate image data of electrical signals according to user images.

镜头组件设置在云台组件上，云台组件可以带动镜头组件进行转动，以便更改镜头组件的朝向。云台组件可以包括至少两个转动部件，以分别实现带动镜头组件沿数值方向进行左右转动，以及沿水平方向进行上下转动。每个转动部件可以连接电机，以通过电机驱动其自动进行转动。The lens assembly is arranged on the gimbal assembly, and the gimbal assembly can drive the lens assembly to rotate, so as to change the orientation of the lens assembly. The pan/tilt assembly may include at least two rotating parts, so as to drive the lens assembly to rotate left and right in the numerical direction, and rotate up and down in the horizontal direction, respectively. Each rotating part can be connected to a motor so that it can be automatically rotated by the motor.

需要说明的是，本申请中的摄像头可实现采集图像相关的所有功能，在不涉及角度旋转时，与不具有角度调整的摄像头不作区分。此外，凡是能实现本文中所述功能的摄像头均在本申请的范围之类，并不对摄像头的结构和组成构件做特殊的限定。It should be noted that the camera in the present application can realize all functions related to capturing images, and when the angle rotation is not involved, it is not distinguished from the camera without angle adjustment. In addition, any camera that can realize the functions described in this document is within the scope of the present application, and the structure and components of the camera are not particularly limited.

摄像头在水平方向和/或垂直方向可旋转的极限角度，可以根据需求进行设计。示例性的，摄像头在水平方向上的可旋转角度范围可以为0°～120°，其中，0°和120°分别为在水平方向上的两个旋转方向(向左和向右)对应的极限角度；摄像头在垂直方向上的可旋转角度可以为0°～180°，其中，0°和180°分别为在垂直方向上的两个旋转方向(向上和向下)对应的极限角度。The limit angle that the camera can rotate in the horizontal and/or vertical direction can be designed according to the needs. Exemplarily, the rotatable angle of the camera in the horizontal direction may range from 0° to 120°, where 0° and 120° are the corresponding limits of the two rotation directions (leftward and rightward) in the horizontal direction, respectively. Angle; the rotatable angle of the camera in the vertical direction can be from 0° to 180°, where 0° and 180° are respectively the limit angles corresponding to the two rotation directions (up and down) in the vertical direction.

如图5所示，云台组件可以包括呈竖直状态的第一转轴和呈水平状态的第二转轴，第一转轴设置在显示器275的顶部，与显示器275的顶部可转动地连接；第一转轴上还设有固定件，固定件的顶部可转动的连接有所述第二转轴，第二转轴连接镜头组件，以带动镜头组件进行转动。第一转轴和第二转轴上分别连接有电机以及传动部件。电机可以是能够支持自动控制转角的伺服电机、步进电机等。当获取控制指令后，两个电机可以分别进行旋转以驱动第一转轴和第二转轴进行转动，从而调节镜头组件的朝向。As shown in FIG. 5 , the pan/tilt assembly may include a first rotating shaft in a vertical state and a second rotating shaft in a horizontal state. The first rotating shaft is disposed on the top of the display 275 and is rotatably connected to the top of the display 275; The rotating shaft is also provided with a fixing piece, the top of the fixing piece is rotatably connected with the second rotating shaft, and the second rotating shaft is connected with the lens assembly to drive the lens assembly to rotate. The first rotating shaft and the second rotating shaft are respectively connected with a motor and a transmission component. The motor can be a servo motor, a stepping motor, etc. that can support automatic control of the rotation angle. After acquiring the control command, the two motors can be rotated respectively to drive the first rotating shaft and the second rotating shaft to rotate, so as to adjust the orientation of the lens assembly.

随着镜头组件的不同朝向，镜头组件可以对位于不同位置上的用户进行视频拍摄，从而获取用户图像数据。显然，不同的朝向对应于不同区域的图像采集，当用户在相对于显示器275正前方位置偏左时，可以通过云台组件上的第一转轴带动固定件以及镜头组件向左转动，以使拍摄的图像中，用户人像位置位于画面的中心区域；而当用户躯体成像位置偏下时，可以通过云台组件中的第二转轴带动镜头组件向上转动，以抬高拍摄角度，使用户人像位置位于画面的中心区域。With different orientations of the lens assembly, the lens assembly can capture video of users at different positions, so as to obtain user image data. Obviously, different orientations correspond to image capture in different areas. When the user is at a left position relative to the front of the display 275, the first rotating shaft on the pan/tilt assembly can drive the fixing piece and the lens assembly to rotate to the left, so that the In the captured image, the position of the user's portrait is located in the central area of the screen; when the imaging position of the user's body is lower, the lens assembly can be rotated upward through the second rotating shaft in the gimbal assembly to raise the shooting angle and make the user's portrait position located in the center area of the screen.

基于上述摄像头232，本申请的部分实施例中提供一些人像定位方法，可以在显示设备200中设置自动控制程序，从而通过检测用户的位置调整摄像头232中镜头组件的朝向，并且按照一定频率重复对人像位置的检测过程，从而实现对人像位置的追踪。所述人像定位方法可以具体包括以下步骤：Based on the above-mentioned camera 232, some embodiments of the present application provide some portrait positioning methods, and an automatic control program can be set in the display device 200, so as to adjust the orientation of the lens assembly in the camera 232 by detecting the position of the user, and repeat the alignment according to a certain frequency. The detection process of the portrait position, so as to realize the tracking of the portrait position. The portrait positioning method may specifically include the following steps:

检测人像位置。在摄像头232启动运行后，实时拍摄多帧图像，并将拍摄的图像发送给显示设备200的控制器250。其中，摄像头232的启动可以为手动启动或自动启动。手动启动即用户通过遥控器等控制装置100在操作界面中选择摄像头232对应的图标后，完成启动。自动启动可以是用户在执行某些需要调用摄像头232的交互动作后，自动启动。例如，用户在“我的应用”界面中选择“照镜子”应用，由于该应用需要调用摄像头232，因此在启动运行该应用的同时，也启动摄像头232。Detect portrait position. After the camera 232 starts to run, multiple frames of images are captured in real time, and the captured images are sent to the controller 250 of the display device 200 . Wherein, the activation of the camera 232 may be manual activation or automatic activation. Manual startup means that the startup is completed after the user selects the icon corresponding to the camera 232 in the operation interface through the control device 100 such as the remote controller. The automatic start may be automatically started after the user performs some interactive actions that need to call the camera 232 . For example, when the user selects the "Look in the mirror" application in the "My Application" interface, since the application needs to call the camera 232, the camera 232 is also started when the application is started and run.

启动摄像头232后，控制器250一方面可以根据所启动的应用程序进行图像处理，例如控制显示器275显示该图像；另一方面可以通过调用检测程序对校对图像进行分析，从而确定用户所在的位置。其中，人像位置的检测可以通过图像处理程序完成。即通过实时抓取摄像头232拍摄的图像，检测肢体信息。肢体信息可以包含关键点和包裹肢体的外框，通过检测的关键点和肢体框位置在图像中位置信息，确定人像位置。其中，关键点可以是指人体图像中能够代表人体特征的一系列点。例如，眼睛、耳朵、鼻子、脖子、肩部、手肘、手腕、腰部、膝关节以及踝关节等。After the camera 232 is activated, the controller 250 can, on the one hand, perform image processing according to the activated application program, for example, control the display 275 to display the image; Among them, the detection of the portrait position can be completed by an image processing program. That is, body information is detected by capturing images captured by the camera 232 in real time. The limb information can include key points and an outer frame that wraps the limb, and the position of the portrait is determined by the position information of the detected key points and the position of the limb frame in the image. Among them, the key points may refer to a series of points in the human body image that can represent the characteristics of the human body. For example, eyes, ears, nose, neck, shoulders, elbows, wrists, waist, knees, ankles, etc.

关键点的确定可以通过图像识别获得，即可以通过分析画面中特征形状，并与预设的模板进行匹配从而确定关键点对应的图像，并获取图像对应的位置，从而获取各关键点对应的位置。其中，位置可以通过图像中距离边界的像素点数量进行表示。可以根据摄像头232的分辨率和可视角度，以图像的左上角为原点，以向右和向下为正方向构建平面直角坐标系，则图像中的各个像素点均能够通过这一直角坐标系进行表示。The determination of key points can be obtained through image recognition, that is, the image corresponding to the key points can be determined by analyzing the characteristic shape in the picture and matching with the preset template, and the position corresponding to the image can be obtained, so as to obtain the position corresponding to each key point. . Among them, the position can be represented by the number of pixels in the image from the boundary. According to the resolution and viewing angle of the camera 232, a plane rectangular coordinate system can be constructed with the upper left corner of the image as the origin and the right and downward directions as the positive directions, then each pixel in the image can pass through this rectangular coordinate system. to express.

例如，如图6所示，水平方向和垂直方向摄像头可视角度分别为HFOV和VFOV，可视角度可以根据摄像头CameraInfo获取，摄像头预览图像支持1080P，宽度为1920，高度1080像素，则图像中每个像素点的位置都可以为(x，y)，其中x的取值范围为(0，1920)；y的取值范围为(0,1080)。For example, as shown in Figure 6, the viewing angles of the cameras in the horizontal and vertical directions are HFOV and VFOV, respectively. The viewing angles can be obtained from the camera's CameraInfo. The camera preview image supports 1080P, with a width of 1920 and a height of 1080 pixels. The position of each pixel can be (x, y), where the value range of x is (0, 1920); the value range of y is (0, 1080).

通常为了能够准确表达用户所在的位置，关键点的数量可以设置为多个，并且在一次检测过程中需要对多个关键点的全部或部分进行位置提取，从而确定包裹肢体的外框区域。例如，关键点可以包括18个，即2个眼睛点、2个耳朵点、1个鼻子点、1个脖子点、2个肩部点、2个肘部点、2个腕部点、2个腰部点(或臀部点)、2个膝关节点以及2个踝关节点。显然，这些关键点在识别的过程中会根据用户的面向不同需要不同的识别方式。例如，腰部对应的位置在用户面向显示器275时识别为腰部点，而在用户背对显示器275时，识别为臀部点。Usually, in order to accurately express the position of the user, the number of key points can be set to multiple, and in one detection process, all or part of the multiple key points need to be extracted, so as to determine the outer frame area of the wrapped limb. For example, keypoints can include 18, i.e. 2 eye points, 2 ear points, 1 nose point, 1 neck point, 2 shoulder points, 2 elbow points, 2 wrist points, 2 Waist point (or hip point), 2 knee points, and 2 ankle points. Obviously, in the process of identifying these key points, different identification methods are required according to different user orientations. For example, the position corresponding to the waist is identified as the waist point when the user faces the display 275 , and is identified as the hip point when the user faces away from the display 275 .

显然，当用户所处位置发生改变或者姿态发生变化时，部分关键点的位置将发生变化。随着这种变化的出现，摄像头232采集的图像中人体相对位置也将发生变化。例如，当人体向左移动位置时，将使摄像头232采集的图像中人***置偏左，不便于进行图像分析处理和实时显示。Obviously, when the user's position changes or the posture changes, the positions of some key points will change. With this change, the relative position of the human body in the image captured by the camera 232 will also change. For example, when the human body moves to the left, the position of the human body in the image captured by the camera 232 will be shifted to the left, which is inconvenient for image analysis processing and real-time display.

因此，在检测人像位置后，还需要对比人像位置与校对图像中的预设区域，从而确定当前人像位置是否在预设区域中。Therefore, after detecting the portrait position, it is also necessary to compare the portrait position with the preset area in the proofreading image, so as to determine whether the current portrait position is in the preset area.

在一些实施例中，人像位置可以通过肢体框中心位置进行表示，而肢体框中心位置可以通过检测的各关键点位置坐标计算获得。例如，通过获取肢体框水平左右两侧的关键点x轴位置坐标，计算肢体框中心位置，即中心位置x轴坐标x ₀＝(x ₁+x ₂)/2。 In some embodiments, the portrait position can be represented by the center position of the limb frame, and the center position of the limb frame can be obtained by calculating the position coordinates of each detected key point. For example, by obtaining the x-axis position coordinates of the key points on the left and right sides of the limb frame horizontally, the center position of the limb frame is calculated, that is, the center position x-axis coordinate x ₀ =(x ₁ +x ₂ )/2.

由于本申请一些实施例中摄像头232可以包括两个左右方向旋转和上下方向旋转，因此在计算获得中心位置的x轴坐标后，可以先对x轴坐标进行判断，确定中心位置的x轴坐标是否位于整个图像的中心位置。例如，当校对图像为(1920,1080)的1080P 图像时，校对图像的中心点水平坐标为960。Since the camera 232 in some embodiments of the present application may include two rotations in the left-right direction and the up-down direction, after calculating the x-axis coordinate of the center position, the x-axis coordinate can be judged first to determine whether the x-axis coordinate of the center position is at the center of the entire image. For example, when the proofreading image is a 1080P image of (1920, 1080), the horizontal coordinate of the center point of the proofreading image is 960.

在确定人像中心位置和图像中心点后，可以通过对比确定人像位置是否位于预设判断区域中。为了避免频繁调整带来的处理负荷增加，以及允许部分检测误差。根据实际应用条件要求以及摄像头232的水平方向可视角度，可以预设一个允许坐标区间，当人像中心位置位于允许坐标区间内，则确定当前人像位置在预设区域中。例如，最大允许坐标误差为300像素，则允许坐标区间为[660，1260]，当检测获得的用户中心位置坐标在这一区间内时，确定用户则在预设判断区域中，即计算获得的人像中心位置坐标与960位置相差不大；当检测获得的用户中心位置坐标不在这一区间内时，确定当前人像位置不在预设区域中，即计算获得的人像中心位置坐标与960位置相差较大。After the center position of the portrait and the center point of the image are determined, it can be determined by comparison whether the position of the portrait is located in the preset judgment area. In order to avoid the increase in processing load caused by frequent adjustment, and allow partial detection errors. According to actual application conditions and the horizontal viewing angle of the camera 232, an allowable coordinate range can be preset, and when the center position of the portrait is within the allowable coordinate range, it is determined that the current portrait position is within the preset area. For example, if the maximum allowable coordinate error is 300 pixels, the allowable coordinate interval is [660, 1260]. When the coordinates of the user's center position obtained by detection are within this interval, it is determined that the user is in the preset judgment area, that is, the calculated The center position coordinates of the portrait are not much different from the 960 position; when the user center position coordinates obtained by detection are not within this interval, it is determined that the current portrait position is not in the preset area, that is, the calculated portrait center position coordinates are quite different from the 960 position. .

如图7a、图7b、图7c以及图7d所示，在对比人像位置与校对图像中的预设区域后，可以根据对比结果确定是否需要进行人像追踪，如果当前人像位置不在预设区域内，控制摄像头232分别在水平方向(图7a至图7b)和竖直方向(图7c至图7d)旋转，以使用户成像位置位于画面中部区域或者离的画面中心位置最近。如果当前人像位置在预设区域内，则无需控制摄像头232旋转，维持摄像头朝向即可。As shown in Figure 7a, Figure 7b, Figure 7c and Figure 7d, after comparing the portrait position with the preset area in the proofreading image, it can be determined whether portrait tracking is required according to the comparison result. If the current portrait position is not within the preset area, The camera 232 is controlled to rotate in the horizontal direction (FIG. 7a to FIG. 7b) and the vertical direction (FIG. 7c to FIG. 7d) respectively, so that the imaging position of the user is located in the middle area of the screen or is closest to the center of the screen. If the current portrait position is within the preset area, there is no need to control the rotation of the camera 232, and the orientation of the camera can be maintained.

在当前人像位置不在预设区域内时，为了控制摄像头232进行旋转，控制器250可以根据人像位置计算旋转角度量，并根据旋转角度量生成控制指令，以控制摄像头232进行旋转。When the current portrait position is not within the preset area, in order to control the camera 232 to rotate, the controller 250 may calculate the rotation angle according to the portrait position, and generate a control instruction according to the rotation angle to control the camera 232 to rotate.

具体地，在确定当前人像位置不在预设区域内以后，控制器250可以先计算人像区域的中心位置和图像区域的中心点之间的距离；再根据计算的距离，结合摄像头232镜头组件的最大视角以及图像尺寸计算获得旋转角度；最后将计算的旋转角度以控制指令的形式发送给摄像头232，使得摄像头232中电机带动各转轴进行转动，从而调整镜头组件的朝向。Specifically, after determining that the current portrait position is not within the preset area, the controller 250 may first calculate the distance between the center position of the portrait area and the center point of the image area; The angle of view and image size are calculated to obtain the rotation angle; finally, the calculated rotation angle is sent to the camera 232 in the form of a control command, so that the motor in the camera 232 drives each shaft to rotate, thereby adjusting the orientation of the lens assembly.

例如，如图8、图9所示，摄像头232的预览分辨率为1920x1080，图像的水平宽度：imgWidth＝1920；图像水平中心位置坐标x＝960；人像区域中心位置坐标为(x ₀，y ₀)水平中心位置坐标为x ₀；水平视角为hfov；则人像区域和图像区域的中心距离：hd＝x–x ₀，则摄像头232在水平方向上的旋转角度则按照下式可计算获得： For example, as shown in FIG. 8 and FIG. 9 , the preview resolution of the camera 232 is 1920×1080, the horizontal width of the image: imgWidth=1920; the coordinate of the horizontal center position of the image is x=960; the coordinate of the center position of the portrait area is (x ₀ , y ₀ ) The coordinate of the horizontal center position is x ₀ ; the horizontal viewing angle is hfov; then the center distance between the portrait area and the image area: hd=x-x ₀ , then the rotation angle of the camera 232 in the horizontal direction can be calculated according to the following formula:

通过上式，可以计算出摄像头232需要进行调节的角度，控制器250再对人像区域中心位置与图像区域中心点的坐标数值进行比较，确定人像区域中心位置相对于图像区域中心点的方位，从而确定摄像头232的旋转方向。即，如果人像区域中心水平位置比图像中心大，则向右转动摄像头232；反之向左转动摄像头232。本申请一些实施例中，摄像头232可以采用后置摄像头模式，使得屏幕显示图像与摄像头拍摄图像是左右镜像关系，即水平角度旋转是左右相反的。Through the above formula, the angle at which the camera 232 needs to be adjusted can be calculated, and the controller 250 then compares the center position of the portrait area with the coordinate values of the center point of the image area, and determines the orientation of the center position of the portrait area relative to the center point of the image area, thereby The rotation direction of the camera 232 is determined. That is, if the horizontal position of the center of the portrait area is larger than the center of the image, turn the camera 232 to the right; otherwise, turn the camera 232 to the left. In some embodiments of the present application, the camera 232 may adopt a rear camera mode, so that the image displayed on the screen and the image captured by the camera are in a left-right mirror relationship, that is, the horizontal angle rotation is opposite to the left and right.

在确定旋转角度和方向以后，控制器250可以将旋转角度和方向数据进行封装，生成控制指令，并将控制指令发送给摄像头232。摄像头232中的电机可以在接收到控制指令后进行转动，从而通过转轴带动镜头组件转动，调整镜头组件的朝向。After determining the rotation angle and direction, the controller 250 can package the rotation angle and direction data, generate a control command, and send the control command to the camera 232 . The motor in the camera 232 can rotate after receiving the control command, so as to drive the lens assembly to rotate through the rotating shaft, and adjust the orientation of the lens assembly.

需要说明的是，在上述实施例中，是以水平方向坐标为例进行判断、调整，实际应用中还可以通过比较人像区域中心位置与图像区域中心点位置之间的竖直方向差异，对镜头组件也进行同样的调整，具体的调整方法与水平方向的调整方法相同，即在确定当前人像位置不在预设区域内以后，控制器250可以先计算人像区域的中心位置和图像区域的中心点之间的竖直距离；再根据计算的竖直距离，结合摄像头232镜头组件的竖直方向最大视角以及图像尺寸计算获得旋转角度；最后将计算的旋转角度以控制指令的形式发送给摄像头232，使得摄像头232中电机带动第二转轴进行转动，从而调整镜头组件的朝向。It should be noted that, in the above-mentioned embodiments, the horizontal coordinates are used as an example for judgment and adjustment. The components are also adjusted in the same way, and the specific adjustment method is the same as the adjustment method in the horizontal direction, that is, after determining that the current portrait position is not within the preset area, the controller 250 can first calculate the center position of the portrait area and the center point of the image area. Then according to the calculated vertical distance, combined with the maximum vertical angle of view of the camera 232 lens assembly and the image size, the rotation angle is calculated to obtain the rotation angle; finally, the calculated rotation angle is sent to the camera 232 in the form of a control instruction, so that The motor in the camera 232 drives the second shaft to rotate, thereby adjusting the orientation of the lens assembly.

但在实际应用中，由于受到用户姿态的影响，以及不同应用程序中的需求不同，在部分应用场景下使用中心位置作为人像位置判断的方式并不能获得较好的显示、检测、跟踪效果。因此在一些实施例中，控制摄像头232旋转，以使用户成像位置位于画面中部区域还可以按照以下步骤进行。However, in practical applications, due to the influence of the user's posture and the different requirements in different applications, the method of using the center position as the portrait position judgment method in some application scenarios cannot obtain better display, detection, and tracking effects. Therefore, in some embodiments, controlling the rotation of the camera 232 so that the imaging position of the user is located in the middle area of the screen may also be performed according to the following steps.

在校对图像中检测第一识别点。A first identification point is detected in the proofreading image.

其中，第一识别点为识别出关键点中的一个或多个，用于表征用户的部分肢***置。例如，第一识别点可以为2个眼睛点(或2个耳朵点)，用以表示用户的头部位置。通过在校对图像中匹配眼睛图案(或耳朵图案)所对应的区域，检测出当前图像中是否含有第一识别点，即是否含有眼睛点(或耳朵点)。Wherein, the first identification point is to identify one or more key points, which are used to represent the position of a part of the user's limbs. For example, the first identification points may be 2 eye points (or 2 ear points) to represent the position of the user's head. By matching the area corresponding to the eye pattern (or ear pattern) in the proofreading image, it is detected whether the current image contains the first identification point, that is, whether the eye point (or ear point) is contained.

如果所述校对图像中不含有第一识别点，在所述校对图像中检测第二识别点。If the proofreading image does not contain the first identification point, a second identification point is detected in the proofreading image.

第二识别点是与第一识别点间隔一定距离并且能够具有相对位置关系的关键点。例如，第二识别点可以为胸部点，由于在常规使用状态下，胸部点位于眼睛点的下方，并且胸部点与眼睛点之间间隔20-30cm的距离，因此可以通过对胸部点的检测确定需要调整的方向。The second identification point is a key point that is spaced apart from the first identification point by a certain distance and can have a relative positional relationship. For example, the second identification point may be a chest point. Since the chest point is located below the eye point in a normal use state, and the distance between the chest point and the eye point is 20-30 cm, it can be determined by detecting the chest point The direction that needs to be adjusted.

按照第二识别点与所述第一识别点的方位关系生成转动方向。The rotation direction is generated according to the azimuth relationship between the second identification point and the first identification point.

例如，当在校对图像中未检测到第一识别点，即眼睛点；而检测到第二识别点，即胸部点，则确定当前校对图像中，未能够显示完全用户的头部图像，需要将摄像头232向上抬起，以使人像头部进入图像的预设区域中。For example, when the first recognition point, that is, the eye point, is not detected in the proofreading image, and the second recognition point, that is, the chest point, is detected, it is determined that the current proofreading image cannot display the full user's head image, and the The camera 232 is lifted upwards so that the head of the portrait enters the preset area of the image.

显然，在实际应用中，根据第二识别点与第一识别点的相对方位不同，在校对图像中未检测到第一识别点，而检测到第二识别点时，所确定的旋转方向也是不同的。例如，第一识别点为腰部点，第二识别点为胸部点时，当未检测到腰部点而检测到胸部点，则说明拍摄的图像太靠人像的上半部，因此可以通过降低拍摄角度，使人像下半部进入图像的预设区域中。Obviously, in practical applications, according to the difference in the relative orientation of the second identification point and the first identification point, the first identification point is not detected in the proofreading image, but when the second identification point is detected, the determined rotation direction is also different. of. For example, when the first identification point is the waist point and the second identification point is the chest point, if the waist point is not detected but the chest point is detected, it means that the captured image is too close to the upper half of the portrait, so you can reduce the shooting angle by reducing the shooting angle. to bring the lower half of the portrait into the preset area of the image.

按照所述旋转方向以及预设调节步长控制摄像头232转动，以使人像位于图像预设区域中。The camera 232 is controlled to rotate according to the rotation direction and the preset adjustment step, so that the portrait is located in the image preset area.

其中，预设调节步长可以根据摄像头232的拍摄精度设置为一个固定值。例如，在眼部/耳部等关键点(第一识别点)没有检测到，而肩部等关键点(第二识别点)检测到时，可以上抬起摄像头232，使第一识别点位置每次调整100像素点，直到第一识别点处于1/7-1/5位置处。The preset adjustment step size may be set to a fixed value according to the shooting accuracy of the camera 232 . For example, when key points such as eyes/ears (the first identification point) are not detected, but key points such as the shoulder (the second identification point) are detected, the camera 232 can be lifted up to make the position of the first identification point Adjust 100 pixels each time until the first recognition point is at the 1/7-1/5 position.

如果校对图像中含有第一识别点，则获取第一识别点相对于图像区域所在的位置。If the proofreading image contains the first identification point, the position of the first identification point relative to the image area is obtained.

通过对校对图像中画面的识别，如果识别出第一识别点，则可以进一步对第一识别点所在的位置进行提取，从而确定第一识别点相对于在整个图像区域中所处的位置。例如，如图10所示，在获得校对图像后，如果识别出眼睛点，即确定检测到第一识别点，则可以获取眼睛点当前坐标P(x ₁，y ₁)。再将当前坐标中的x轴坐标值和/或y轴坐标值与图像的整体宽度imgWidth和/或高度imgHeight进行对比，从而确定第一识别点相对于图像区域所在的位置。其中，在水平方向和竖直方向两个方向上可以确定第一识别点相对于图像区域在两个方向上所在的位置。即水平方向上，所述第一识别点相对于图像区域所在的位置为x ₁/imgWidth；在竖直方向上，所述第一识别点相对于图像区域所在的位置为y ₁/imgHeight。 By recognizing the screen in the proofreading image, if the first identification point is identified, the position of the first identification point can be further extracted, thereby determining the position of the first identification point relative to the entire image area. For example, as shown in FIG. 10 , after obtaining the proofreading image, if the eye point is identified, that is, it is determined that the first identification point is detected, the current coordinate P(x ₁ , y ₁ ) of the eye point can be obtained. Then compare the x-axis coordinate value and/or y-axis coordinate value in the current coordinates with the overall width imgWidth and/or height imgHeight of the image, thereby determining the position of the first identification point relative to the image area. Wherein, the positions of the first identification point relative to the image area in the two directions may be determined in the horizontal direction and the vertical direction. That is, in the horizontal direction, the position of the first identification point relative to the image area is x ₁ /imgWidth; in the vertical direction, the position of the first identification point relative to the image area is y ₁ /imgHeight.

在获取第一识别点相对于图像区域所在的位置后，还可以对第一识别点对应位置所在区间进行判断，并根据所在的不同区间，确定不同的调整方式。After acquiring the position of the first identification point relative to the image area, it is also possible to judge the interval where the corresponding position of the first identification point is located, and determine different adjustment methods according to different intervals.

例如，如图10所示，通过检测在竖直方向上，第一识别点相对于图像区域所在的位置时，检测到眼睛(或耳部)在图像画面高度的1/5之下，此时，眼睛位置过低，需要将摄像头232下压，以使眼睛位置升高至合适的区域内，在将摄像头232下压的过程中，如果检测到眼睛的点在图像画面的1/6位置处，则停止下压，完成摄像头232的调整。当检测到眼睛(或耳部)位置在图像画面高度的1/7以下、1/5以上，则确定当前第一识别点位置合适，因此摄像头232的高度不需要进行调整，防止抖动造成摄像头频繁变动。For example, as shown in Fig. 10, by detecting the position of the first recognition point relative to the image area in the vertical direction, it is detected that the eyes (or ears) are below 1/5 of the height of the image, then , the eye position is too low, the camera 232 needs to be pressed down to raise the eye position to a suitable area. During the process of pressing down the camera 232, if the point of the detected eye is at 1/6 of the image screen , then stop pressing down to complete the adjustment of the camera 232 . When it is detected that the position of the eyes (or ears) is below 1/7 and above 1/5 of the height of the image, it is determined that the current position of the first recognition point is appropriate, so the height of the camera 232 does not need to be adjusted to prevent the camera from shaking frequently. change.

由于在实际应用中，用户与显示设备200之间的距离不同时，通过摄像头232所拍摄到的图像中，人像所占比例也不同。即用户在距离显示设备200较远位置时，拍摄的人像也较小；在距离显示设备200较近位置时，拍摄到的人像也较大。为此，当控制摄像头232进行转动时，相同的转动步长会在用户与显示设备200距离不同时，对人像位置的移动量也不同。例如，在用户距离显示设备200较近时，摄像头232调整100像素则可以将人像的眼部区域从边缘位置移动至1/7-1/5位置处；而在用户距离显示设备200较远时，摄像头232调整100像素则可以将人像的眼部区域从边缘位置移动至1/5-1/2位置处，即造成过度调整，影响定位效果。In practical applications, when the distance between the user and the display device 200 is different, the proportion of the portrait in the image captured by the camera 232 is also different. That is, when the user is farther away from the display device 200 , the photographed portrait is also smaller; when the user is closer to the display device 200 , the photographed portrait is also larger. For this reason, when the camera 232 is controlled to rotate, the same rotation step will result in different movement amounts of the portrait position when the distance between the user and the display device 200 is different. For example, when the user is closer to the display device 200 , the camera 232 can adjust 100 pixels to move the eye area of the portrait from the edge position to the 1/7-1/5 position; and when the user is far away from the display device 200 , the camera 232 can be adjusted by 100 pixels to move the eye area of the portrait from the edge position to the 1/5-1/2 position, which will cause excessive adjustment and affect the positioning effect.

为此，在一些实施例中，调节步长还可以根据检测到第一识别点或第二识别点的特点确定。即所述方法还包括以下步骤：To this end, in some embodiments, the adjustment step size may also be determined according to the characteristics of the detected first identification point or the second identification point. That is, the method further includes the following steps:

在所述校对图像中检测两个所述第一识别点或两个所述第二识别点之间的距离；detecting the distance between two of the first identification points or two of the second identification points in the proofreading image;

根据两个所述第一识别点或两个所述第二识别点之间的距离匹配预设调节步长；Match a preset adjustment step size according to the distance between the two first identification points or the two second identification points;

封装所述旋转方向以及预设调节步长，以生成旋转指令。The rotation direction and the preset adjustment step are packaged to generate a rotation instruction.

由于人体特点，相同类型的关键点一般至少有两个，因此，可以在校对图像中检测到至少两个第一识别点或者至少两个第二识别点后。例如，第一识别点为眼部关键点时，用户的左右眼分别对应两个关键点，则可以通过计算两个第一识别点之间的距离，进而估算出用户与显示设备200之间的距离。显然，用户与显示设备200之间的距离越远，两个第一识别点之间的距离越近；用户与显示设备200之间的距离越近，计算的两个第一识别点之间的距离则越远。Due to the characteristics of the human body, there are generally at least two key points of the same type. Therefore, at least two first identification points or at least two second identification points can be detected in the proofreading image. For example, when the first recognition point is an eye key point, and the left and right eyes of the user correspond to two key points, the distance between the two first recognition points can be calculated to estimate the distance between the user and the display device 200. distance. Obviously, the farther the distance between the user and the display device 200 is, the closer the distance between the two first identification points; the closer the distance between the user and the display device 200 is, the closer the calculated distance between the two first identification points is. The distance is farther.

在计算两个第一识别点或者两个第二识别点之间的距离后，可以按照第一识别点或第二识别点的类型以及计算出的距离在预先设置的关系表中进行匹配，确定与当前距离对应的预设调节步长，并将确定的旋转方向与预设调节步长一起封装在旋转指令中，从而将旋转指令发送给摄像头232。摄像头232根据接收的旋转指令，可以按照预设调节步长向旋转方向调整拍摄角度。After calculating the distance between the two first identification points or the two second identification points, matching can be carried out in the preset relation table according to the type of the first identification point or the second identification point and the calculated distance, and determine The preset adjustment step size corresponding to the current distance is encapsulated in the rotation instruction together with the determined rotation direction and the preset adjustment step size, so that the rotation instruction is sent to the camera 232 . According to the received rotation instruction, the camera 232 can adjust the shooting angle in the rotation direction according to the preset adjustment step.

在上述实施例中，摄像头232按照所述预设调节步长每调整一次拍摄角度，可以在所拍摄的图像中重新提取一次校对图像，并按照上述实施例中的判断方式，确定人像位置是否位于预设区域内，如果人像位置仍然不在预设区域内，则再生成一次旋转指令，以便控制摄像头232继续转动角度，依次按照这种循环方式，直到人像位置位于预设区域后，停止向摄像头232发送旋转指令。In the above embodiment, every time the camera 232 adjusts the shooting angle according to the preset adjustment step length, it can re-extract a calibration image from the captured image, and according to the judgment method in the above embodiment, determine whether the position of the portrait is in the In the preset area, if the portrait position is still not within the preset area, a rotation instruction is generated again to control the camera 232 to continue to rotate the angle, and in this cycle, until the portrait position is located in the preset area, stop the camera 232. Send a rotation command.

需要说明的是，为了获得摄像头232的追踪效果，在实际应用中，在确定人像位置位于预设区域中之后的使用过程中，也可以按照设定的频率从摄像头232拍摄的图像中提取校对图像，并按照上述实施例中方式对人像位置进行判断。如果由于用户在使用过程中位置的移动，导致人像位置不在预设区域中，则向摄像头232发送旋转指令，以控制摄像头232转向用户区域，达到追踪效果。It should be noted that, in order to obtain the tracking effect of the camera 232, in practical applications, in the process of use after determining that the position of the portrait is located in the preset area, a calibration image can also be extracted from the image captured by the camera 232 according to the set frequency. , and judge the position of the portrait according to the method in the above embodiment. If the position of the portrait is not in the preset area due to the user's position movement during use, a rotation instruction is sent to the camera 232 to control the camera 232 to turn to the user area to achieve a tracking effect.

通过上述实施例提供的人像定位方法，可以改进当前电视固定摄像头拍摄图像区域固定，无法对水平或垂直移动的人体进行追踪定位拍摄的问题。通过图像检测对拍摄图像进行人物检测确定具***置，驱动摄像头进行微调，以此达到精准定位，使摄像头拍摄人物能够在电视屏幕中聚焦显示，所述人像定位方法对于视频通话、健身等场景，人物站立位置不在默认摄像头拍摄中心区域内对于快速准确定位聚焦人物十分有效。The portrait positioning method provided by the above embodiments can improve the problem that the fixed image area captured by the current TV fixed camera cannot be tracked, positioned and photographed for a human body that moves horizontally or vertically. Character detection is performed on the captured image through image detection to determine the specific position, and the camera is driven for fine-tuning, so as to achieve precise positioning, so that the person captured by the camera can be focused and displayed on the TV screen. The portrait positioning method is suitable for scenes such as video calls and fitness. Standing outside the center of the default camera capture is useful for quickly and accurately locating the subject in focus.

由于在不同的检测过程中，所能够识别出的关键点数量和类型也不同，例如在用户处于蹲姿状态时，腰部点、踝关节点等部分关键点将被遮挡，从而无法识别出来，因此针对不同的检测过程所识别的关键点也可以不同。Because in different detection processes, the number and type of key points that can be identified are also different. For example, when the user is in a squatting position, some key points such as waist points and ankle points will be occluded and cannot be identified. Therefore, The keypoints identified for different detection processes can also be different.

为了获得更加准确的人像位置判断，在一些实施例中，如图11、图12所示，识别出多个关键点以后，还可以根据识别出的关键点建立骨骼线示意图形，从而根据骨骼线图形进一步确定人像所在位置。其中，骨骼线可以通过连接多个关键点进行确定。在用户不同的姿态下，骨骼线所呈现的形状也不同。In order to obtain a more accurate portrait position judgment, in some embodiments, as shown in FIG. 11 and FIG. 12 , after multiple key points are identified, a schematic diagram of a skeleton line can be established according to the identified key points, so that according to the skeleton line The graphics further determine where the portrait is located. Among them, the skeleton line can be determined by connecting multiple key points. Under different user poses, the shape of the skeleton line is also different.

需要说明的是，通过绘制的骨骼线还可以对根据骨骼线的运动变化规律来动态调整摄像头的拍摄位置。例如，在判断骨骼线运动状态变化过程为从蹲姿状态变化到站立状态，则可以抬高摄像头232的视角，以使处于站姿状态的人像也能够处于图像中的合适区域内，即从图11过渡到图12所示的效果。在判断骨骼线运动状态变化过程为从站立状态变化到蹲姿状态，这可以降低摄像头232的视角，以使处于蹲姿状态的人像也能够处于图像中的合适区域内，即从图12过渡到图11所示的效果。It should be noted that the drawn skeletal line can also dynamically adjust the shooting position of the camera according to the movement change rule of the skeletal line. For example, when it is judged that the movement state of the skeletal line changes from the squatting state to the standing state, the viewing angle of the camera 232 can be raised so that the portrait in the standing state can also be in a suitable area in the image, that is, from the image 11 Transition to the effect shown in Figure 12. When judging that the change process of the motion state of the skeletal line is from standing state to squatting state, this can reduce the viewing angle of the camera 232, so that the portrait in the squatting state can also be in a suitable area in the image, that is, the transition from FIG. 12 to The effect shown in Figure 11.

基于上述人像定位方法，在本申请的部分实施例中还提供一些显示设备200，所述显示设备200包括显示器275和控制器250。所述显示设备200可以内置或通过外部装置接口240接入摄像头232。其中，所述摄像头232可以转动拍摄角度，用于拍摄不同视角下的图像；所述显示器275被配置为呈现用户界面；所述控制器250被配置为执行以下程序步骤：Based on the above-mentioned portrait positioning method, some display devices 200 are also provided in some embodiments of the present application. The display devices 200 include a display 275 and a controller 250 . The display device 200 can be built in or connected to the camera 232 through the external device interface 240 . Wherein, the camera 232 can rotate the shooting angle for shooting images under different viewing angles; the display 275 is configured to present a user interface; the controller 250 is configured to execute the following program steps:

获取用户输入的用于启动摄像头应用的控制指令；Obtain the control instruction input by the user to start the camera application;

响应于所述控制指令，从所述摄像头拍摄的图像中检测人像位置；In response to the control instruction, detecting the position of the portrait from the image captured by the camera;

按照所述人像位置向所述摄像头发送旋转指令，以使所述人像位置位于预设判断区域内。A rotation instruction is sent to the camera according to the position of the portrait, so that the position of the portrait is located in a preset judgment area.

由以上技术方案可知，本实施例提供的显示设备200，可以通过在控制器250中配置上述人像定位方法所对应的应用程序，实时上述人像定位方法。即通过对比人像位置与校对图像中的预设区域确定是否启动调整摄像头232朝向。如果当前人像位置不在预设区域内，控制摄像头232旋转，以使用户成像位置位于画面中部区域。所述方法通过图像检测对拍摄图像进行人物检测确定具***置，驱动摄像头232进行微调，以此达到精准定位，使摄像头232拍摄人物能够在显示设备200的显示器275中聚焦显示It can be known from the above technical solutions that the display device 200 provided in this embodiment can implement the above-mentioned portrait positioning method in real time by configuring an application program corresponding to the above-mentioned portrait positioning method in the controller 250 . That is, whether to start adjusting the orientation of the camera 232 is determined by comparing the position of the portrait with the preset area in the proofreading image. If the current portrait position is not within the preset area, the camera 232 is controlled to rotate, so that the user's imaging position is located in the middle area of the screen. The method uses image detection to perform person detection on the captured image to determine the specific position, and drives the camera 232 to perform fine-tuning, so as to achieve precise positioning, so that the camera 232 can focus and display the person captured by the camera 232 on the display 275 of the display device 200.

在一些实施例中，在利用显示设备进行视频通话、AI健身及摄像头人像追踪等场景时，由于人物会在显示设备的前方移动位置，可能造成出画现象。如果人物发现自己出画，则会再次走进摄像头的拍摄区域，使得人物图像再次出现。In some embodiments, when the display device is used for video calls, AI fitness, and camera portrait tracking, etc., because the characters move in front of the display device, a painting phenomenon may occur. If the character finds himself out of the painting, he will walk into the camera's shooting area again, making the image of the character appear again.

若在上述场景中，存在多人同时入画的情况，此时，为便于实现人物追踪，通常需要指定某一人进行追踪(摄像头人像追踪场景下)，追踪某一人的跟练结果(AI健身场景下)，或者云台摄像头跟踪某一人转动(视频通话场景下)。以AI健身场景为例，在健身跟练模式下，需要对锁定的某一人进行跟踪，并对动作进行打点，生成跟练数据，统计跟练结果。If there are multiple people entering the painting at the same time in the above scenario, in this case, in order to facilitate the realization of character tracking, it is usually necessary to designate a certain person to track (in the camera portrait tracking scene), and track the follow-up results of a certain person (in the AI fitness scene). ), or the PTZ camera tracks a person's rotation (in a video call scenario). Taking the AI fitness scene as an example, in the fitness follow-up mode, it is necessary to track a locked person, make a point of action, generate follow-up data, and count the follow-up results.

如果在手机场景下，人脸或者肢体离屏幕较近，人脸或肢体在屏幕中占比较大，每帧图像检测到的人物图像相对移动距离较小，一般不会产生丢失(出画)现象。而显示设备不同于手机场景，在人与显示设备交互时，人与显示设备的距离一般较远，人脸或肢体的屏占比较小，前后帧图像内移动距离会较大，比如人在屏幕前快速走动，极易会导致人物的识别丢失，即产生出画现象。In the mobile phone scene, if the face or body is close to the screen, and the face or body occupies a large proportion of the screen, the relative movement distance of the detected character image in each frame of image is small, and the phenomenon of loss (out of picture) will generally not occur. . The display device is different from the mobile phone scene. When a person interacts with the display device, the distance between the person and the display device is generally far, the screen ratio of the face or body is small, and the moving distance between the front and rear frame images will be large. For example, a person is on the screen. Moving quickly in front of you can easily lead to the loss of character recognition, that is, the phenomenon of painting.

由于当前健身功能多基于肢体信息进行肢体跟随，摄像头实现人像跟随功能通常通过识别人脸或肢体等作为识别依据。不管是肢体或者人脸信息，每次识别到均会生成一个ID作为识别的肢体或人脸的标识。Since the current fitness functions are mostly based on body information for body tracking, the camera to realize the portrait following function usually uses the recognition of faces or limbs as the identification basis. Regardless of body or face information, an ID will be generated each time it is identified as the identification of the identified body or face.

但是，当识别丢失后重新识别，即人物出画又重新入画，会重新生成一个新ID信息，造成同一个人的前后ID不一致，这样对于健身或者摄像头追踪特定人物时，产生丢失会造成不可逆，也就无法对同一个人做到追踪的有效性。However, when the identification is lost and then re-identified, that is, the character is drawn and re-entered, a new ID information will be regenerated, resulting in inconsistent IDs before and after the same person. In this way, when a specific person is tracked by a fitness or camera, the loss will cause irreversible, and also It is impossible to track the effectiveness of the same person.

为实现对同一个人的锁定追踪，本申请实施例提供一些显示设备，在人像定位以后，还可实现基于人脸特征值进行人物锁定追踪。用以解决AI图像检测算法在连续图像中人脸或者肢体有较大的移动时会产生丢失，重新再检测出人脸或肢体会重新识别成另外一个ID，造成前后没有一致性，无法保持追踪的问题。也就是说，针对同一个人，不管执行多少次出画入画动作，均会识别为同一个人进行连续追踪，不会出现识别丢失现象，提高追踪有效性。In order to realize locking and tracking of the same person, some display devices are provided in the embodiments of the present application, and after the portrait is positioned, the locking and tracking of the person based on the facial feature value can also be realized. It is used to solve the problem that the AI image detection algorithm will be lost when there is a large movement of the face or limb in the continuous image. If the face or limb is detected again, it will be re-identified as another ID, resulting in no consistency before and after, and unable to maintain tracking. The problem. That is to say, for the same person, no matter how many times the picture-in-picture action is performed, it will be recognized as the same person for continuous tracking, and there will be no identification loss phenomenon, which improves the tracking effectiveness.

该方法的主要过程是在摄像头图像识别过程中，首先通过特定动作触发人物锁定(比如健身通过肢体叉腰，云台摄像头人脸跟踪可以通过摆头等，叉腰可以通过肢体识别功能输出的肢体关键点进行判断)，锁定特定动作对应的人，实现对作出特定动作的人的锁定追踪。The main process of this method is that, in the process of camera image recognition, first trigger the character lock through a specific action (for example, fitness can be performed by akimbo, PTZ camera face tracking can be performed by swinging the head, etc., akimbo can be performed by the key points of the limb output by the limb recognition function. Judgment), lock the person corresponding to the specific action, and realize the locking and tracking of the person who performs the specific action.

本申请一些实施例提供一些显示设备，包括控制器以及与控制器通信连接的显示器和摄像头，摄像头用于采集人物在使用摄像头应用时产生的人物图像信息，显示器用于呈现显示有人物图像信息的用户界面，控制器用于对摄像头采集的人物图像信息进行处理，并显示在用户界面中。控制器在执行基于人脸特征值的图像人物锁定追踪方法时，需先创建人脸特征值，再基于人脸特征值执行人物锁定追踪方法。Some embodiments of the present application provide some display devices, including a controller, a display and a camera connected in communication with the controller, the camera is used to collect character image information generated when a character uses a camera application, and the display is used to present the image information displayed with the character. User interface, the controller is used to process the person image information collected by the camera and display it in the user interface. When the controller executes the image person lock tracking method based on the face feature value, it needs to create the face feature value first, and then execute the person lock tracking method based on the face feature value.

图13中示例性示出了根据一些实施例的人脸特征值创建方法的数据流图；图14中示例性示出了根据一些实施例的人脸特征值创建方法的流程图。在执行人脸特征值创建方法时，参见图13和图14，本申请实施例提供的一种显示设备，其控制器被配置为下述步骤：FIG. 13 exemplarily shows a data flow diagram of a method for creating a face feature value according to some embodiments; FIG. 14 exemplarily shows a flowchart of a method for creating a face feature value according to some embodiments. When executing the method for creating a face feature value, referring to Fig. 13 and Fig. 14 , a display device provided by an embodiment of the present application, the controller of which is configured to the following steps:

S11、获取摄像头采集的人物图像信息。S11. Acquire the person image information collected by the camera.

由于可同时与显示设备进行交互的人物可为一个或多个，因此，在至少一个人物与显示设备进行交互时，例如，至少一个人物在利用显示设备进行视频通话、AI健身或摄像头人像追踪等时，摄像头实时进行图像采集。摄像头采集到的人物图像信息包括至少一个人物的图像信息，从图像信息中可读取到该人物的肢体动作和面部特征信息等。Since there may be one or more characters that can interact with the display device at the same time, when at least one character interacts with the display device, for example, at least one character is using the display device for video calls, AI fitness, or camera portrait tracking, etc. When the camera is in real-time image acquisition. The person image information collected by the camera includes image information of at least one person, and the person's body movements and facial feature information, etc. can be read from the image information.

S12、识别人物图像信息，确定锁定追踪人物，以及，创建锁定追踪人物的面部特征信息和特定肢体ID信息。S12 , identifying the image information of the person, determining the locked and tracked person, and creating facial feature information and specific body ID information of the locked and tracked person.

在存在多个人物与显示设备进行交互时，或者，在初始时刻只有一个人物与显示设备进行交互，但在交互过程中又存在其他人物出现在摄像头的拍摄区域范围内时，这种多人交互场景下导致显示设备无法确定以哪一个人物的指令作为控制指令进行响应，即无法确定以哪一个人物作为特定人物进行追踪，因此，需要在进行交互时确定锁定追踪人物。锁定追踪人物是与显示设备进行交互的其中一个人物，后续交互时仅响应该锁定追踪人物产生的指令。When there are multiple characters interacting with the display device, or when only one character interacts with the display device at the initial moment, but other characters appear within the shooting area of the camera during the interaction, this multi-person interaction In the scenario, the display device cannot determine which character's instruction is used as the control instruction to respond, that is, it cannot determine which character is used as a specific character for tracking. Therefore, it is necessary to determine the locked and tracked character during interaction. The locked and tracked character is one of the characters that interacts with the display device, and the subsequent interaction only responds to the instructions generated by the locked and tracked character.

为实现对同一个人物进行锁定追踪，需在多个人物与显示设备交互时，选定其中一个人物作为锁定追踪人物。若仅有一个人物与显示设备进行交互，则锁定追踪人物即为该人物。在确定锁定追踪人物时，可根据每个人物是否做出特定动作来进行判断，而人物的动作识别可根据该人物的肢体关键点信息来确定。In order to achieve locked tracking of the same person, it is necessary to select one of the characters as the locked tracking character when multiple characters interact with the display device. If only one person interacts with the display device, the locked and tracked person is that person. When determining the locked and tracked characters, the judgment can be made according to whether each character has made a specific action, and the action recognition of the character can be determined according to the key point information of the body of the character.

在一些实施例中，控制器在执行识别人物图像信息，确定锁定追踪人物，被进一步配置为执行下述步骤：In some embodiments, the controller is further configured to perform the following steps when performing identifying the image information of the person and determining to lock the tracking person:

步骤121、提取人物图像信息中人物的肢体关键点，确定人物的肢体关键点信息。Step 121 , extract the key points of the person's limbs in the person image information, and determine the key point information of the person's limbs.

步骤122、对人物的肢体关键点信息进行肢体动作识别。Step 122 , performing body motion recognition on the key point information of the body of the character.

步骤123、如果人物的肢体关键点信息对应的肢体动作与特定动作一致，则将人物确定为锁定追踪人物。Step 123 , if the limb movements corresponding to the limb key point information of the person are consistent with the specific motion, determine the person as a locked and tracked person.

在人物需要以自己为锁定追踪人物时，该人物可通过肢体做出特定动作，该特定动作可预先设置。在确定锁定追踪人物时，通常以触发相应交互功能的人物作为锁定追踪人物，而可以触发相应功能的方式可采用特定的肢体动作的形式。例如，在实现AI健身功能时，特定的肢体动作可为叉腰等；在实现云台摄像头人脸跟踪功能时，特定的肢体动作可为摆头等。为此，将产生上述动作以启动某个功能的人物作为锁定追踪人物。特定动作是指人物可实现显示设备某个功能时所作出的肢体动作。When the character needs to use himself as the lock to track the character, the character can make a specific action through the body, and the specific action can be preset. When determining the locked and tracked character, the character that triggers the corresponding interactive function is usually used as the locked and tracked character, and the way to trigger the corresponding function can be in the form of a specific body movement. For example, when implementing the AI fitness function, the specific body movement can be akimbo, etc.; when implementing the face tracking function of the PTZ camera, the specific body movement can be swaying the head, etc. To this end, the person who generates the above action to activate a certain function is regarded as the locked tracking person. A specific action refers to a physical action made by a character when a certain function of the display device can be realized.

其中，叉腰或摆头动作等可以通过肢体识别功能输出的肢体关键点进行判断。将识别到的相邻肢体关键点连线，可形成线条图形，由识别到的线条图形作为对应人物的肢体动作。Among them, the action of akimbo or swinging head can be judged by the key points of the limb output by the limb recognition function. By connecting the identified key points of adjacent limbs, a line graph can be formed, and the identified line graph can be used as the body movement of the corresponding character.

为确定每个人物的肢体动作，可对摄像头采集到的人物图像信息进行识别，提取人物图像信息中人物的肢体关键点，以确定人物的肢体关键点信息，通过肢体关键点信息产生对应的肢体动作。若当前有多个人物与显示设备进行互动，则摄像头采集到的人物图像信息中包括多个人物的图像信息。In order to determine the body movements of each person, the image information of the person collected by the camera can be identified, and the key points of the person's limbs in the person's image information can be extracted to determine the key point information of the person's limbs, and the corresponding limbs can be generated through the key point information of the limbs. action. If there are currently multiple characters interacting with the display device, the image information of the characters collected by the camera includes image information of the multiple characters.

肢体关键点可选定头部、脸部、身体、肩膀、手臂、膝盖、眼睛、手腕、脚腕等部分，以其中一个部分作为指定肢体部位对人物图像信息进行识别检测。若选定其中一个肢体部位为指定肢体部位，则对指定肢体部位进行打点，根据得到的多个打点部位，得到肢体关键点信息。Limb key points can be selected from parts such as head, face, body, shoulders, arms, knees, eyes, wrists, ankles, etc., and use one of the parts as a designated body part to identify and detect human image information. If one of the limb parts is selected as the designated limb part, the designated limb part is dotted, and the limb key point information is obtained according to the obtained multiple dotted parts.

将肢体关键点信息进行肢体动作识别，即将肢体关键点信息中包括的多个打点部位连成线条图形，即可得到该人物作出的肢体动作。The body movement recognition is performed on the body key point information, that is, by connecting a plurality of dotted parts included in the body key point information into a line graph, the body movement of the character can be obtained.

例如，若以脸部作为指定肢体部位时，则识别检测人物图像信息中脸部的存在数量，若脸部数量为一个，则人物图像信息中仅包括一个人物的图像；若脸部数量为多个，则人物图像信息中包括多个人物的图像。For example, if the face is used as the designated body part, the number of faces in the person image information will be recognized and detected. If the number of faces is one, the person image information will only include the image of one person; if the number of faces is many , the person image information includes images of a plurality of persons.

若某个肢体部位通常成对出现，例如肩膀、手臂、膝盖、眼睛、手腕、脚腕等部分，则将识别到的所有肢体部位的总数量除以2，即为人物图像信息中包括的人物的数量。例如，以肩膀为指定肢体部位，若检测到存在10个肩膀，来自五个人物，则可确定人物图像信息中包括五个人物。If a body part usually appears in pairs, such as shoulders, arms, knees, eyes, wrists, ankles, etc., then divide the total number of all identified body parts by 2, which is the person included in the person image information quantity. For example, taking the shoulder as the designated body part, if it is detected that there are 10 shoulders from five persons, it can be determined that five persons are included in the person image information.

若当前有多个人物与显示设备进行互动，则对人物图像信息进行识别检测时，可以得到多个人物对应的肢体关键点信息，进而得到多个人物对应的肢体动作。If there are currently multiple characters interacting with the display device, when the image information of the characters is identified and detected, the body key point information corresponding to the multiple characters can be obtained, and then the body movements corresponding to the multiple characters can be obtained.

肢体动作是指人物摆动身体的肢体，如胳膊、腿、手掌或头部等，而产生的动作。人物若要与显示设备交互实现某一特定功能，例如，AI健身功能、摄像头人脸跟踪功能或视频通话功能等，需要以特定的肢体动作的方式启动。在摄像头的拍摄区域内，每个人物的肢体动作可能不同，但会存在某个人物的肢体动作可以启动特定功能的动作。Limb movements refer to the movements produced by the characters swinging the limbs of the body, such as arms, legs, palms or heads. If a character wants to interact with the display device to realize a specific function, such as AI fitness function, camera face tracking function or video call function, etc., it needs to be activated by a specific body movement. In the shooting area of the camera, the body movements of each character may be different, but there will be a certain character's body movements that can activate a specific function.

因此，将每个人物的肢体动作与特定动作进行对比，若其中一个人物的肢体动作与某一个特定动作相匹配，则将该人物的肢体动作确定为特定动作，并将产生特定动作的人物作为锁定追踪人物。Therefore, the body movements of each character are compared with specific movements, and if the body movements of one of the characters match a specific movement, the body movements of the character are determined as the specific movements, and the character that produces the specific movements is used as the Lock the tracker.

图15中示例性示出了根据一些实施例的AI健身场景下人物图像信息的显示示意图。在一些实施例中，参见图15，摄像头采集到包括三个人物的人物图像信息，分别对人物图像信息中的三个人物的肢体动作进行识别。如果识别到2#人物的肢体动作为叉腰，则判定叉腰动作与启动AI健身功能的特定动作相匹配，则将2#人物确定为锁定追踪人物。FIG. 15 exemplarily shows a schematic diagram of displaying character image information in an AI fitness scene according to some embodiments. In some embodiments, referring to FIG. 15 , the camera collects person image information including three persons, and recognizes the body movements of the three persons in the person image information respectively. If it is recognized that the body movement of person 2# is akimbo, it is determined that the akimbo movement matches the specific movement that activates the AI fitness function, and person 2# is determined to be the locked and tracked person.

图16中示例性示出了根据一些实施例的视频通话场景下人物图像信息的显示示意图。在一些实施例中，参见图16，摄像头采集到包括三个人物的人物图像信息，分别对人物图像信息中的三个人物的肢体动作进行识别。如果识别到3#人物的肢体动作为摆头，则判定摆头动作与启动视频通话功能的特定动作相匹配，则将3#人物确定为锁定追踪人物。FIG. 16 exemplarily shows a schematic diagram of displaying character image information in a video call scenario according to some embodiments. In some embodiments, referring to FIG. 16 , the camera collects person image information including three persons, and recognizes the body movements of the three persons in the person image information respectively. If it is recognized that the body movement of person 3# is a head swing, it is determined that the head shake action matches the specific action for initiating the video call function, and the person 3# is determined to be the locked and tracked person.

在一些实施例中，为对人物图像信息中的每个人物进行区分，进而对确定出的锁定追踪人物进行标识，以便于能够对锁定追踪人物进行持续的追踪，需要为该锁定追踪人物建立特定肢体ID信息。In some embodiments, in order to distinguish each person in the person image information, and then identify the determined locked and tracked person, so that the locked and tracked person can be continuously tracked, it is necessary to establish a specific locked and tracked person. Limb ID information.

为对每个人物进行区分和标识，在对人物图像信息进行识别检测时，可创建每个人物对应的肢体ID信息，肢体ID信息用于标定不同的与显示设备进行交互的人物。在一些实施例中，肢体ID信息可采用编号的形式实现，因此，可对检测到的多个人物按照一定的顺序进行编号，并为每个人物生成对应的肢体ID信息，即一个人物对应一个编号，一个人物对应一个肢体ID信息。In order to distinguish and identify each character, when identifying and detecting the image information of the character, body ID information corresponding to each character can be created, and the body ID information is used to demarcate different characters interacting with the display device. In some embodiments, the body ID information can be implemented in the form of numbers. Therefore, the detected multiple characters can be numbered in a certain order, and corresponding body ID information can be generated for each character, that is, one character corresponds to one Number, a character corresponds to a body ID information.

例如，如果当前有五个人正与显示设备进行交互，摄像头可同时采集到包括五个人物图像的人物图像信息，分别为该五个人物创建肢体ID信息，即分别为五个人进行编号，分别为1#、2#、3#、4#、5#。For example, if there are currently five people interacting with the display device, the camera can simultaneously collect the image information of the five people, and create body ID information for the five people, that is, number the five people, which are 1#, 2#, 3#, 4#, 5#.

在一些实施例中，为每个人物生成对应的肢体ID信息的过程，可执行在初次控制器对每个人物的肢体关键点检测的过程中，即控制器对人物图像信息进行肢体关键点的初次检测过程中，确定每个人物对应的肢体ID信息。后续控制器再次对人物的肢体关键点进行检测时，则用于与初次生成的肢体ID信息进行对比，以便于对同一个人物进行锁定追踪。In some embodiments, the process of generating corresponding limb ID information for each character can be performed during the initial detection of the limb key points of each character by the controller, that is, the controller performs limb key point identification on the image information of the person. During the initial detection process, the body ID information corresponding to each character is determined. When the subsequent controller detects the key points of the character's limbs again, it is used to compare with the limb ID information generated for the first time, so as to lock and track the same person.

例如，以头部作为指定肢体部位，若人物图像信息中识别到五个肢体关键点信息，则确定人物图像信息中包括五个人的图像，则为该五个人进行编号，每个人物对应的肢体ID信息分别为1#、2#、3#、4#、5#。For example, taking the head as the designated body part, if five key points of limbs are identified in the person image information, it is determined that the person image information includes images of five people, then the five people are numbered, and the limbs corresponding to each person are The ID information is 1#, 2#, 3#, 4#, and 5#, respectively.

编号顺序可基于人物图像信息中按照人物所在位置从左到右的顺序进行编号，或者按照从右到左的顺序进行编号，具体的编号顺序还可用户自定义，本实施例不做具体限定。The numbering sequence may be numbered in the order from left to right according to the location of the person in the image information of the person, or in the order from right to left, and the specific numbering order can be customized by the user, which is not specifically limited in this embodiment.

因此，在与显示设备进行交互的多个人物中确定出锁定追踪人物后，可根据每个人物的肢体ID信息，获取锁定追踪人物对应的肢体ID，将锁定追踪人物对应的肢体ID作为特定肢体ID信息，特定肢体ID信息用于实现对锁定追踪人物进行锁定追踪。Therefore, after the locked and tracked characters are determined among the multiple characters interacting with the display device, the body ID corresponding to the locked and tracked characters can be obtained according to the body ID information of each character, and the body ID corresponding to the locked and tracked characters can be used as the specific body. ID information, the specific body ID information is used to realize the locked tracking of the locked tracking person.

在一些实施例中，为建立人脸特征值，需要对锁定追踪人物的面部提取特征信息。为此，需要先从包括多个人物图像的人物图像信息中截取出锁定追踪人物的面部图像，再提取面部特征信息。In some embodiments, in order to establish a face feature value, feature information needs to be extracted from the face of the locked tracked person. To this end, it is necessary to first cut out the facial image of the locked and tracked person from the person image information including a plurality of person images, and then extract the facial feature information.

在一些实施例中，控制器在执行基于人物图像信息中锁定追踪人物对应的面部图像，提取锁定追踪人物的面部特征信息，被进一步配置为执行下述步骤：在人物图像信息中，截取与特定肢体ID信息对应的锁定追踪人物的面部图像；基于锁定追踪人物的面部图像，提取锁定追踪人物的面部特征信息。In some embodiments, the controller locks the facial image corresponding to the tracking person in the execution based on the person image information, extracts the facial feature information of the locked tracking person, and is further configured to perform the following steps: in the person image information, intercept and specific The facial image of the locked and tracked person corresponding to the body ID information; based on the facial image of the locked and tracked person, the facial feature information of the locked and tracked person is extracted.

锁定追踪人物的特定肢体ID信息生成后，会同步存储到人物图像信息中，且存储的特定肢体ID信息与人物图像信息中锁定追踪人物的图像相对应，因此，在识别人物图像信息时，可基于特定肢体ID信息，查找到锁定追踪人物的图像，进而在该图像中截取出锁定追踪人物的面部图像。After the specific limb ID information of the locked and tracked person is generated, it will be stored in the person image information synchronously, and the stored specific limb ID information corresponds to the image of the locked and tracked person in the person image information. Therefore, when identifying the person image information, you can Based on the specific body ID information, an image of the locked and tracked person is found, and then the facial image of the locked and tracked person is captured from the image.

面部图像可为包括人物五官在内的图像，提取面部图像的52维数字序列，将面部图像中呈现的人物五官转换为数字信息进行表示，因此，由锁定追踪人物的面部图像，提取锁定追踪人物的面部特征信息，该面部特征信息包括用数字化表示的人物五官特征值。The facial image can be an image including the facial features of a person, extract a 52-dimensional digital sequence of the facial image, and convert the facial features of the person presented in the facial image into digital information for representation. The facial feature information includes the facial feature values of the characters represented by digitization.

S13、将锁定追踪人物的特定肢体ID信息和面部特征信息建立关联关系，得到锁定追踪人物的人脸特征值。S13: Establish an association relationship between the specific body ID information of the locked and tracked person and the facial feature information, and obtain the facial feature value of the locked and tracked person.

由于每个人物具有各自的面部特征信息，且不同人物的面部特征信息不同。因此，可由面部特征信息对每个人物进行识别，若识别到相同或相似的面部特征信息，即可将其识别为同一个人物。Since each character has its own facial feature information, and different characters have different facial feature information. Therefore, each person can be identified by the facial feature information, and if the same or similar facial feature information is identified, it can be identified as the same person.

由于在通常情况下，在锁定追踪人物走出画面时，其对应的特定肢体ID信息即丢失，即人物出画后删除对应的特定肢体ID信息。若该人物又重新走进画面，则重新生成对应的肢体ID信息，易出现将同一个人识别为两个人的情况。Because under normal circumstances, when the locked and tracked person walks out of the screen, the corresponding specific limb ID information is lost, that is, the corresponding specific limb ID information is deleted after the person exits the picture. If the character enters the screen again, the corresponding body ID information will be regenerated, and it is easy to identify the same person as two people.

因此，在一些实施例中，将锁定追踪人物的特定肢体ID信息和面部特征信息建立关联关系，由相互关联的特定肢体ID信息和面部特征信息作为锁定追踪人物的人脸特征值，以对锁定追踪人物进行标识，面部特征信息不会因人物出画而被删除，会一直保存在控制器中。Therefore, in some embodiments, an association relationship is established between the specific body ID information and the facial feature information of the locked and tracked person, and the specific body ID information and the facial feature information associated with each other are used as the facial feature value of the locked and tracked person, so that the locked and tracked person is Tracking characters for identification, facial feature information will not be deleted when the characters are drawn, and will always be saved in the controller.

由于摄像头在持续拍摄人物图像信息，在完成人脸特征值创建后，控制器可根据摄像头拍摄的实时人物图像信息对每个人物进行识别，根据特定肢体ID信息和面部特征信息确定锁定追踪人物，并对锁定追踪人物进行追踪。Since the camera continues to capture the image information of the person, after the creation of the facial feature value, the controller can identify each person according to the real-time person image information captured by the camera, and determine the locked and tracked person according to the specific body ID information and facial feature information. And track the locked tracker.

在基于面部特征信息进行识别时，即使同一个人物出画又入画，但由于先后两个图像中的面部特征信息相同，显示设备也不会将其识别为两个人，因此，可以保证对该人物的锁定追踪。其中，先后两个图像是指人物出画又入画，即特定肢体ID信息丢失前后对应的图像。When identifying based on facial feature information, even if the same person appears in the painting and then enters the painting, the display device will not recognize the person as two people because the facial feature information in the two images is the same. Therefore, it can be guaranteed that the person lock tracking. Among them, the two images in succession refer to the figures appearing and re-entering the painting, that is, the corresponding images before and after the loss of the ID information of a specific body.

在一些实施例中，在确定出锁定追踪人物的人脸特征值后，如果锁定追踪人物持续处于人物图像信息中，则基于人脸特征值对锁定追踪人物进行锁定追踪。例如，在AI健身场景下，可持续采集锁定追踪人物的跟练结果并显示在用户界面中；在视频通话场景下，摄像头可根据锁定追踪人物的位置变化调整拍摄角度。In some embodiments, after determining the facial feature value of the locked and tracked person, if the locked and tracked person continues to be in the person image information, the locked and tracked person is locked and tracked based on the facial feature value. For example, in the AI fitness scene, the follow-up results of the locked and tracked person can be continuously collected and displayed in the user interface; in the video call scene, the camera can adjust the shooting angle according to the position change of the locked and tracked person.

在一些实施例中，在AI健身场景下，控制器在执行基于人脸特征值对锁定追踪人物进行锁定追踪，被进一步配置为执行下述步骤：In some embodiments, in an AI fitness scenario, the controller is further configured to perform the following steps when performing locked and tracked characters based on facial feature values:

步骤1311、在摄像头应用为AI健身应用时，确定锁定追踪人物为健身人员。Step 1311: When the camera application is an AI fitness application, determine that the locked and tracked person is a fitness person.

步骤1312、基于健身人员的人脸特征值，持续采集健身人员基于用户界面中呈现的示范视频进行跟练的跟练动作信息。Step 1312 , based on the facial feature values of the fitness personnel, continuously collect follow-up exercise information of the fitness personnel for follow-up exercises based on the demonstration video presented in the user interface.

步骤1313、基于跟练动作信息生成跟练画面，显示在用户界面中，跟练画面显示在示范视频所在画面的一侧。Step 1313: Generate a follow-up exercise picture based on the follow-up exercise action information, display it on the user interface, and display the follow-up exercise picture on the side of the picture where the demonstration video is located.

在实现AI健身场景时，同时利用显示设备配置的AI健身功能的人物可为一个或多个，若人物为一个时，该健身的人物即为锁定追踪人物。显示器的用户界面中呈现示范视频，便于健身人员进行跟练。When implementing an AI fitness scene, there can be one or more characters who use the AI fitness function configured on the display device at the same time. If there is one person, the fitness person is the locked and tracked person. Demonstration videos are presented in the user interface of the monitor for easy follow-up by fitness personnel.

此时，摄像头应用为AI健身应用，AI健身应用调用摄像头始终采集锁定追踪人物的跟练动作，并显示在显示器中的用户界面中。At this time, the camera application is an AI fitness application, and the AI fitness application calls the camera to always capture the follow-up movements of the locked and tracked characters, and display them in the user interface on the display.

图17(a)中示例性示出了根据一些实施例的单人AI健身场景下进行人物锁定追踪的显示示意图。参见图17(a)所示，在健身人物为一个时，锁定追踪人物即为该健身人员。由于摄像头采集的人物图像信息中仅包括该人物的图像信息，因此，可基于该健身人员的人脸特征值，持续采集健身人员跟随示范视频进行跟练时产生的跟练动作信息。根据跟练动作信息生成跟练画面，显示在用户界面中，跟练画面显示在示范视频所在画面的一侧。FIG. 17( a ) exemplarily shows a display schematic diagram of person-locked tracking in a single-person AI fitness scene according to some embodiments. Referring to FIG. 17( a ), when there is one fitness person, the locked and tracked person is the fitness person. Since the image information of the person collected by the camera only includes the image information of the person, the follow-up action information generated when the fitness person follows the demonstration video for follow-up exercise can be continuously collected based on the fitness person's facial feature value. A follow-up exercise picture is generated according to the follow-up exercise action information, displayed on the user interface, and the follow-up exercise picture is displayed on the side of the screen where the demonstration video is located.

图17(b)中示例性示出了根据一些实施例的多人AI健身场景下的多人图像示意图。参见图17(b)所示，在健身人物为多个人时，摄像头采集到包括多个健身人员的当前人物图像信息，以包括三个健身人员为例，分别为1#、2#、3#，基于当前人物图像信息中的每个健身人员重新创建对应的当前人脸特征值。如果其中一个健身人员2#的当前人脸特征值与之前创建的人脸特征值的特征相似度满足阈值条件(阈值可设定为90％)，则将锁定追踪人物确定为特征相似度满足阈值条件对应的当前健身人员2#。FIG. 17(b) exemplarily shows a schematic diagram of a multi-person image in a multi-person AI fitness scenario according to some embodiments. Referring to Fig. 17(b), when there are multiple people in fitness, the camera collects image information of the current person including multiple fitness personnel, taking the example of including three fitness personnel, which are 1#, 2#, and 3# respectively. , and recreate the corresponding current face feature value based on each fitness person in the current person image information. If the feature similarity between the current face feature value of one of the fitness personnel 2# and the previously created face feature value satisfies the threshold condition (the threshold can be set to 90%), the locked tracker is determined as the feature similarity meets the threshold The current fitness person 2# corresponding to the condition.

而后基于锁定追踪人物(健身人员2#)的人脸特征值，持续采集健身人员2#跟随示范视频进行跟练的跟练动作信息。根据跟练动作信息生成跟练画面，显示在用户界面中，跟练画面显示在示范视频所在画面的一侧。此时，用户界面中仅会显示锁定追踪人物的跟练画面，而其他人物的跟练画面不会显示，可参照图17(a)所示内容。Then, based on the facial feature value of the locked and tracked person (fitness person 2#), the follow-up action information of fitness person 2# following the demonstration video for follow-up training is continuously collected. A follow-up exercise picture is generated according to the follow-up exercise action information, displayed on the user interface, and the follow-up exercise picture is displayed on the side of the screen where the demonstration video is located. At this time, only the follow-up training screen of the locked and tracked character will be displayed in the user interface, and the follow-up training screen of other characters will not be displayed, please refer to the content shown in FIG. 17( a ).

可见，在实现AI健身场景时，可根据锁定追踪人物的人脸特征值，持续采集该人物的跟练动作信息；或者，在存在多个人物同时进行AI健身时，可确定出需要进行追踪的锁定追踪人物，并再根据锁定追踪人物的人脸特征值，持续采集该锁定追踪人物的跟练动作信息。在基于人脸特征值进行追踪时，无论该锁定追踪人物是始终处于摄像头的拍摄区域内，还是经过出画又入画的过程，均可以根据人脸特征值在当前人物产生变化后的人物中确定出锁定追踪人物，进行实现锁定追踪，进而可以保证锁定追踪人物因移动导致识别丢失后仍可连续追踪，保证追踪有效性。It can be seen that when implementing AI fitness scenarios, the tracking action information of the person can be continuously collected according to the facial feature value of the tracking person; or, when there are multiple people performing AI fitness at the same time, it can be determined that the tracking action information needs to be tracked. Lock the tracking person, and then continuously collect the follow-up action information of the locked tracking person according to the facial feature value of the locked tracking person. When tracking based on facial feature values, no matter whether the locked and tracked character is always in the shooting area of the camera, or through the process of drawing and re-entering the picture, it can be determined in the characters after the current character changes according to the facial feature values. The locked and tracked person is released, and the locked and tracked person is realized, which can ensure that the locked and tracked person can still be continuously tracked after the identification is lost due to movement, and the tracking effectiveness is ensured.

在一些实施例中，控制器还可根据摄像头采集的锁定追踪人物的跟练动作信息，与相同时刻示范视频中所呈现的示范动作进行对比，生成跟练数据。如果健身人员的跟练动作与示范动作的相似度超过阈值(可设定为60％)时，可得到健身达标的跟练数据；如果健身人员的跟练动作与示范动作的相似度低于阈值时，可得到健身不达标的跟练数据。最后根据健身时长所生成的数个跟练数据统计跟练结果，可在该锁定追踪人物结束健身后，将该跟练结果呈现在用户界面中，便于该锁定追踪人物查看。In some embodiments, the controller may also generate follow-up data by comparing the follow-up action information of the locked and tracked character collected by the camera with the demonstration action presented in the demonstration video at the same time. If the similarity between the fitness personnel's follow-up exercise and the demonstration movement exceeds the threshold (which can be set to 60%), the fitness-compliant follow-up data can be obtained; if the similarity between the fitness personnel's follow-up movement and the demonstration movement is lower than the threshold When the fitness is not up to standard, follow-up data can be obtained. Finally, according to several follow-up data generated by the exercise duration, the follow-up results can be counted. After the locked tracker finishes exercising, the follow-up results can be presented in the user interface, which is convenient for the locked tracker to view.

在一些实施例中，在视频通话场景下，控制器在执行基于人脸特征值对锁定追踪人物进行锁定追踪，被进一步配置为：In some embodiments, in a video call scenario, the controller is further configured to:

步骤1321、在摄像头应用为视频通话应用时，确定锁定追踪人物为视频通话人员。Step 1321 , when the camera application is a video call application, determine that the locked and tracked person is a video call person.

步骤1322、在视频通话人员移动时，基于视频通话人员的人脸特征值，调整摄像头的拍摄角度，以使视频通话人员持续处于摄像头的采集区域内。Step 1322: When the video caller moves, adjust the shooting angle of the camera based on the facial feature value of the video caller, so that the video caller is continuously within the capture area of the camera.

在实现视频通话场景时，同时利用显示设备配置的视频通话功能的人物可为一个或多个，若人物为一个时，该视频通话人员即为锁定追踪人物。此时，摄像头应用为视频通话应用，用户界面中呈现视频通话界面。视频通话应用调用摄像头始终采集锁定追踪人物的位置信息，以在锁定追踪人物出现位置变化时，为保证该人物的图像始终位于视频通话界面中，可实时调整摄像头的拍摄角度。When implementing a video call scene, there may be one or more persons using the video call function configured on the display device at the same time, and if there is one person, the person on the video call is the locked and tracked person. At this time, the camera application is a video call application, and a video call interface is presented in the user interface. The video call application calls the camera to always collect the position information of the locked and tracked person, so that when the position of the locked and tracked person changes, in order to ensure that the image of the person is always in the video call interface, the shooting angle of the camera can be adjusted in real time.

图18中示例性示出了根据一些实施例的单人视频通话场景下的显示示意图。参见图18所示，在利用显示设备进行视频通话的人物为一个时，则锁定追踪人物为该视频通话人员，视频通话的对方画面显示在用户界面中，锁定追踪人物的本端视频画面显示在对方画面的一侧。由于摄像头采集的人物图像信息中仅包括该人物的图像信息，因此，可基于该视频通话人员的人脸特征值，持续采集视频通话人员的图像信息，以基于图像信息识别视频通话人员的位置。若识别到视频通话人员的位置发生变化，则根据位置变化方向和程度调整摄像头的拍摄角度，以使视频通话人员持续处于摄像头的采集区域内。调整摄像头拍摄角度的示意图可参照图19所示内容。FIG. 18 exemplarily shows a schematic diagram of a display in a single-person video call scenario according to some embodiments. Referring to Figure 18, when there is only one person using the display device to make a video call, the locked and tracked person is the person on the video call, the other party's picture of the video call is displayed on the user interface, and the local video picture of the locked and tracked person is displayed on the side of the other party's screen. Since the image information of the person collected by the camera only includes the image information of the person, the image information of the person on the video call can be continuously collected based on the facial feature value of the person on the video call to identify the location of the person on the video call based on the image information. If it is recognized that the position of the person on the video call has changed, the shooting angle of the camera is adjusted according to the direction and degree of the position change, so that the person on the video call continues to be in the capture area of the camera. For a schematic diagram of adjusting the shooting angle of the camera, please refer to the content shown in FIG. 19 .

图19中示例性示出了根据一些实施例的多人视频通话场景下调整摄像头的示意图。参见图19所示，在利用显示设备进行视频通话的人物为多个时，摄像头采集到包括多个视频通话人员的当前人物图像信息，以包括三个视频通话人员为例，本端视频画面中显示1#、2#、3#视频通话人员，基于当前人物图像信息中的每个视频通话人员重新创建对应的当前人脸特征值。如果其中一个视频通话人员1#的当前人脸特征值与之前创建的人脸特征值的特征相似度满足阈值条件(阈值可设定为90％)，则将锁定追踪人物确定为特征相似度满足阈值条件对应的当前视频通话人员1#。FIG. 19 exemplarily shows a schematic diagram of adjusting a camera in a multi-person video call scenario according to some embodiments. Referring to FIG. 19 , when there are multiple people using the display device to make a video call, the camera collects image information of the current person including multiple video callers. Taking three video callers as an example, in the video screen of the local end Display 1#, 2#, 3# video callers, and recreate the corresponding current face feature value based on each video caller in the current person image information. If the feature similarity between the current face feature value of one of the video callers 1# and the previously created face feature value satisfies the threshold condition (the threshold can be set to 90%), the locked tracking person is determined as the feature similarity satisfying Person 1# of the current video call corresponding to the threshold condition.

而后基于该视频通话人员1#的人脸特征值，持续采集视频通话人员1#的图像信息，以基于图像信息识别视频通话人员的位置。若识别到视频通话人员的位置发生变化，则根据位置变化方向和程度调整摄像头的拍摄角度，以使视频通话人员1#持续处于摄像头的采集区域内。Then, based on the facial feature value of the video caller 1#, the image information of the video caller 1# is continuously collected, so as to identify the position of the video caller based on the image information. If it is recognized that the position of the video caller changes, adjust the shooting angle of the camera according to the direction and degree of the position change, so that the video caller 1# is continuously in the capture area of the camera.

例如，参见图19中(a)所示，如果视频通话人员1#位于正中间位置，则摄像头的拍摄角度为正中；参见图19中(b)所示，如果视频通话人员1#向左移动，则向左调整摄像头的拍摄角度(3#人物出画)；参见图19中(c)所示，如果视频通话人员1#向右移动，则向右调整摄像头的拍摄角度(2#人物出画)。而其他视频通话人员的位置变化并不会触发摄像头的拍摄角度的改变，因此，若视频通话人员1#移动的位置较大，则其他视频通话人员可能会显示在视频通话界面的一侧，或出画。For example, as shown in Fig. 19 (a), if the video caller 1# is in the middle position, the shooting angle of the camera is in the center; as shown in Fig. 19 (b), if the video caller 1# moves to the left , then adjust the shooting angle of the camera to the left (3# character is drawn); see (c) in Figure 19, if the video caller 1# moves to the right, then adjust the camera's shooting angle to the right (2# character is out of the picture) Painting). The position change of other video callers will not trigger the change of the camera's shooting angle. Therefore, if video caller 1# moves to a larger position, other video callers may be displayed on the side of the video call interface, or out of the painting.

可见，在实现视频通话场景时，可根据锁定追踪人物的人脸特征值，持续采集该人物的图像信息；或者，在存在多个人物同时进行视频通话时，可确定出需要进行追踪的锁定追踪人物，并再根据锁定追踪人物的人脸特征值，持续采集该锁定追踪人物的图像信息。根据采集的锁定追踪人物的图像信息，识别该锁定追踪人物的位置变化，并在产生位置变化时实时调整摄像头的拍摄角度。在基于人脸特征值进行追踪时，无论该锁定追踪人物是始终处于摄像头的拍摄区域内，还是经过出画又入画的过程，均可以根据人脸特征值在当前人物产生变化后的人物中确定出锁定追踪人物，进行实现锁定追踪，进而可以保证锁定追踪人物因移动导致识别丢失后仍可连续追踪，保证追踪有效性。It can be seen that when a video call scene is implemented, the image information of the person can be continuously collected according to the facial feature value of the locked and tracked person; or, when there are multiple people making a video call at the same time, the locked tracking that needs to be tracked can be determined. person, and then continuously collects image information of the locked and tracked person according to the facial feature value of the locked and tracked person. According to the collected image information of the locked and tracked person, the position change of the locked and tracked person is recognized, and the shooting angle of the camera is adjusted in real time when the position change occurs. When tracking based on facial feature values, no matter whether the locked and tracked character is always in the shooting area of the camera, or through the process of drawing and re-entering the picture, it can be determined in the characters after the current character changes according to the facial feature values. The locked and tracked person is released, and the locked and tracked person is realized, which can ensure that the locked and tracked person can still be continuously tracked after the identification is lost due to movement, and the tracking effectiveness is ensured.

在一些实施例中，使用显示设备的人物可能会出现出画又入画的情况，此时，每个人物的肢体ID信息发生变化，为保证能够始终对锁定追踪人物进行锁定追踪，控制器被进一步配置为执行下述步骤：In some embodiments, the characters using the display device may appear to be drawn and drawn again. At this time, the body ID information of each character changes. In order to ensure that the locked and tracked characters can always be locked and tracked, the controller is further Configured to perform the following steps:

步骤141、在基于人脸特征值对锁定追踪人物进行锁定追踪时，如果锁定追踪人物在人物图像信息中消失，则获取摄像头采集的当前人物图像信息。Step 141: When the locked and tracked person is locked and tracked based on the facial feature value, if the locked and tracked person disappears in the person image information, obtain the current person image information collected by the camera.

步骤142、基于当前人物图像信息中的当前人物重新创建当前人脸特征值。Step 142: Recreate the current face feature value based on the current character in the current character image information.

步骤143、如果当前人脸特征值与人脸特征值的特征相似度满足阈值条件，则将锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物。Step 143: If the feature similarity between the current face feature value and the face feature value satisfies the threshold condition, determine the locked tracked person as the current person whose feature similarity satisfies the threshold condition.

在基于锁定追踪人物的人脸特征值进行锁定追踪时，如果锁定追踪人物始终处于人物图像信息中，则说明摄像头采集到的人物并未发生变化，此时，可实现对锁定追踪人物进行锁定追踪。如果锁定追踪人物在人物图像信息中消失，说明摄像头采集到的人物发生变化，此时，无法实现对锁定追踪人物进行锁定追踪。When the locked tracking is performed based on the facial feature value of the locked tracking person, if the locked tracking person is always in the person image information, it means that the person captured by the camera has not changed. At this time, the locked tracking person can be locked and tracked. . If the locked and tracked person disappears in the person image information, it means that the person captured by the camera has changed. At this time, the locked and tracked person cannot be locked and tracked.

为准确确定在锁定追踪人物消失的情况下，是重新确定一个新的锁定追踪人物，还是检测锁定追踪人物是否重新进入摄像头的采集区域，需要重新对采集到的当前人物图像信息中的人物重新创建当前人脸特征值。In order to accurately determine whether to re-determine a new locked and tracked character when the locked and tracked character disappears, or to detect whether the locked and tracked character has re-entered the capture area of the camera, it is necessary to re-create the character in the collected image information of the current character. The current face feature value.

在一些实施例中，在判断锁定追踪人物在人物图像信息中消失时，控制器被进一步配置为执行下述步骤：In some embodiments, when it is determined that the locked tracking person disappears in the person image information, the controller is further configured to perform the following steps:

步骤1411、在基于人脸特征值对锁定追踪人物进行锁定追踪时，提取人物图像信息中人物的肢体ID信息。Step 1411 , extracting the body ID information of the person in the person image information when the locked and tracked person is locked and tracked based on the facial feature value.

步骤1412、如果提取到的肢体ID信息中不存在人脸特征值中的特定肢体ID信息，则确定锁定追踪人物在人物图像信息中消失。Step 1412: If there is no specific body ID information in the face feature value in the extracted body ID information, determine that the locked and tracked person disappears in the person image information.

在摄像头的采集区域内，每个人物均对应一个肢体ID信息，因此，根据摄像头实时采集到的人物图像信息，确定每个人物的肢体ID信息。同时，在锁定追踪人物的人脸特征值中解析出特定肢体ID信息，将特定肢体ID信息与每个人物的肢体ID信息进行对比。In the collection area of the camera, each person corresponds to a body ID information. Therefore, according to the person image information collected by the camera in real time, the body ID information of each person is determined. At the same time, the specific body ID information is parsed from the facial feature value of the locked and tracked person, and the specific body ID information is compared with the body ID information of each person.

若存在一个人物的肢体ID信息与特点肢体ID信息对比一致，说明提取到的肢体ID信息中存在人脸特征值中的特定肢体ID信息，则确定锁定追踪人物始终在人物图像信息中；若每一个人物的肢体ID信息与特点肢体ID信息对比均不一致，说明提取到的肢体ID信息中不存在人脸特征值中的特定肢体ID信息，则确定锁定追踪人物始终在人物图像信息中消失。If the body ID information of a person is consistent with the characteristic body ID information, it means that there is specific body ID information in the face feature value in the extracted body ID information, then it is determined that the locked and tracked person is always in the person image information; If the body ID information of a person is inconsistent with the characteristic body ID information, it means that there is no specific body ID information in the face feature value in the extracted body ID information, and it is determined that the locked and tracked person always disappears in the person image information.

在锁定追踪人物在人物图像信息中消失时，需要对采集到的当前人物图像信息中的人物重新创建当前人脸特征值，因此，获取摄像头采集的当前人物图像信息。创建人脸特征值的方法可参照前述实施例的内容，此处不再赘述。When the locked and tracked person disappears in the person image information, the current face feature value needs to be recreated for the person in the collected current person image information. Therefore, the current person image information collected by the camera is obtained. For the method of creating a face feature value, reference may be made to the content of the foregoing embodiments, and details are not described herein again.

由于锁定追踪人物出画又入画时，其对应的肢体ID信息发生变化，但其面部特征信息并不会改变。因此，为准确判断当前的人物中是否存在锁定追踪人物，可将每个人物重新创建的当前人脸特征值与锁定追踪人物的人脸特征值进行对比，并计算特征相似度。Since the locked and tracked characters are drawn and drawn, their corresponding body ID information will change, but their facial feature information will not change. Therefore, in order to accurately determine whether there is a locked and tracked character among the current characters, the current face feature value recreated by each character can be compared with the face feature value of the locked and tracked character, and the feature similarity can be calculated.

在一些实施例中，由于人脸特征值包括肢体ID信息和面部特征信息，而肢体ID信息随人物的出画又入画而改变，因此，在对比人脸特征值时，主要是基于面部特征信息进行对比，即计算每个人物的当前面部特征信息与锁定追踪人物的面部特征信息的特征相似度，进行相似度对比。In some embodiments, since the facial feature value includes body ID information and facial feature information, and the body ID information changes as the character is drawn and drawn, the facial feature information is mainly based on the facial feature information when comparing the facial feature values. The comparison is performed, that is, the feature similarity between the current facial feature information of each person and the facial feature information of the locked and tracked person is calculated, and the similarity is compared.

如果当前人脸特征值与人脸特征值的特征相似度满足阈值条件(阈值可设为90％)，说明当前的人物中存在锁定追踪人物，即该锁定追踪人物经过出画又入画的过程，则将锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物。If the feature similarity between the current face feature value and the face feature value satisfies the threshold condition (the threshold value can be set to 90%), it means that there is a locked and tracked character in the current character, that is, the locked and tracked character goes through the process of drawing and entering the painting. Then, the locked and tracked person is determined as the current person whose feature similarity satisfies the threshold condition.

此时，将重新创建的当前人脸特征值替换原人脸特征值，并作为当前人物(锁定追踪人物)进行锁定追踪的人脸特征值。而后基于当前人脸特征值实现对锁定追踪人物的锁定追踪。At this time, the re-created current face feature value is replaced with the original face feature value, and is used as the face feature value of the current person (locked tracking person) to perform locked tracking. Then, the locked tracking of the locked and tracked person is realized based on the current face feature value.

如果每个当前人脸特征值与人脸特征值的特征相似度均不满足阈值条件(阈值可设为90％)，说明当前的人物中不存在锁定追踪人物，即锁定追踪人物消失，不再执行锁定追踪过程。If the feature similarity between each current face feature value and the face feature value does not meet the threshold condition (the threshold can be set to 90%), it means that there is no locked tracker among the current characters, that is, the locked tracker disappears and no longer exists. Perform the lock tracking process.

可见，本申请实施例提供的显示设备，在创建人脸特征值时，识别摄像头采集的人物图像信息，以确定锁定追踪人物，并根据锁定追踪人物的特定肢体ID信息和面部特征信息创建锁定追踪人物的人脸特征值。在进行锁定追踪时，通过人脸特征值可以有效识别丢失前后图像识别的是否为同一个人，进而可以保证锁定追踪人物因移动导致识别丢失后仍可连续追踪，保证追踪有效性。It can be seen that the display device provided by the embodiment of the present application, when creating the facial feature value, recognizes the character image information collected by the camera to determine the locked and tracked character, and creates the locked tracker according to the specific limb ID information and facial feature information of the locked and tracked character. The facial feature value of the person. During locked tracking, the facial feature value can be used to effectively identify whether the same person was identified in the images before and after the loss, thereby ensuring that the locked and tracked person can still be tracked continuously after the identification is lost due to movement, ensuring the tracking effectiveness.

在一些实施例中，在实现某个功能场景下，例如，在AI健身场景或视频通话场景下，完成人脸特征值创建后，显示设备即可根据人脸特征值在同一功能场景下执行人物锁定追踪方法。In some embodiments, in the implementation of a certain functional scenario, for example, in an AI fitness scenario or a video call scenario, after the face feature value is created, the display device can execute the character in the same functional scenario according to the face feature value. Lock tracking method.

图20中示例性示出了根据一些实施例的人物锁定追踪方法的数据流图；图21中示例性示出了根据一些实施例的人物锁定追踪方法的流程图。本申请实施例提供的一种显示设备，包括：。显示器，被配置为呈现用户界面；摄像头，被配置为采集人物在使用摄像头应用时产生的人物图像信息；存储器，被配置为存储在确定锁定追踪人物时创建的人脸特征值；与显示器、摄像头和存储器连接的控制器，参见图20和图21，在执行人物锁定追踪方法时，其配置的控制器被配置为执行下述步骤：Fig. 20 exemplarily shows a data flow diagram of a method for person locking and tracking according to some embodiments; Fig. 21 exemplarily shows a flowchart of a method for character locking and tracking according to some embodiments. A display device provided by an embodiment of the present application includes: . The display is configured to present the user interface; the camera is configured to collect the character image information generated when the character uses the camera application; the memory is configured to store the facial feature value created when the tracking character is determined to be locked; and the display, the camera The controller connected to the memory, see FIG. 20 and FIG. 21 , when executing the character lock tracking method, the configured controller is configured to perform the following steps:

S21、获取锁定追踪人物的人脸特征值和摄像头采集的人物图像信息。S21. Obtain the facial feature value of the locked and tracked person and the person image information collected by the camera.

在一个或多个人物与显示设备进行交互时，摄像头实时拍摄人物图像信息。在初始拍摄时刻，即启动某个特定功能的初始时刻，控制器基于人物图像信息创建人脸特征值。而后控制器基于初始时刻创建的人脸特征值，实时对某个锁定追踪人物进行锁定追踪。When one or more characters interact with the display device, the camera captures image information of the characters in real time. At the initial shooting moment, that is, the initial moment when a certain function is activated, the controller creates a face feature value based on the person image information. Then, based on the facial feature value created at the initial moment, the controller locks and tracks a locked and tracked character in real time.

在至少一个人物与显示设备进行交互时，则摄像头采集到的人物图像信息中包括至少一个人物的图像。When at least one character interacts with the display device, the image information of the character collected by the camera includes an image of at least one character.

S22、如果锁定追踪人物在人物图像信息中消失，则基于人物图像信息中的当前人物重新创建当前人脸特征值。S22. If the locked and tracked person disappears in the person image information, recreate the current face feature value based on the current person in the person image information.

在创建人脸特征值后采集到人物图像信息后，需先判断锁定追踪人物是否一直处于人物图像信息中，或者，从人物图像信息中消失，以便于确定锁定追踪人物是否出现出画又入画的现象。After the face feature value is created and the character image information is collected, it is necessary to first determine whether the locked and tracked character has been in the character image information, or disappeared from the character image information, so as to determine whether the locked and tracked character appears in the painting and then enters the painting. Phenomenon.

在一些实施例中，控制器在确定锁定追踪人物在人物图像信息中消失，被进一步配置为：In some embodiments, the controller is further configured to:

步骤211、提取人物图像信息中人物的肢体ID信息和人脸特征值中的特定肢体ID信息；Step 211, extract the body ID information of the character in the character image information and the specific body ID information in the face feature value;

步骤212、如果肢体ID信息中不存在特定肢体ID信息，则确定锁定追踪人物在人物图像信息中消失。Step 212: If the specific body ID information does not exist in the body ID information, it is determined that the locked and tracked person disappears in the person image information.

控制器在跟随锁定追踪人物时，可基于摄像头实时且连续拍摄的人物图像信息中，识别到每个人物的肢体ID信息。在摄像头的采集区域内，每个人物均对应一个肢体ID信息，因此，根据摄像头实时采集到的人物图像信息，确定每个人物的肢体ID信息。同时，在锁定追踪人物的人脸特征值中解析出特定肢体ID信息。When the controller follows and locks the tracking person, it can identify the body ID information of each person based on the image information of the person captured by the camera in real time and continuously. In the collection area of the camera, each person corresponds to a body ID information. Therefore, according to the person image information collected by the camera in real time, the body ID information of each person is determined. At the same time, the specific body ID information is parsed from the facial feature value of the locked and tracked person.

由于多个人物在与显示设备进行交互时，每个人物可能会产生位置移动，这将导致存在至少一个人物会出现出画又入画(识别丢失)的情况。因此，为判定是否存在出画又入画的情况，即判定是否存在识别丢失情况，需在当前采集的人物图像信息中提取每个人物的肢体ID信息。Since multiple characters interact with the display device, each character may move in position, which will lead to a situation in which at least one character will appear in and out of the drawing (recognition loss). Therefore, in order to determine whether there is a situation where the picture is drawn and then entered, that is, whether there is a loss of recognition, the body ID information of each person needs to be extracted from the currently collected person image information.

对人物图像信息进行肢体ID信息检测，如果在摄像头的连续拍摄下，没有任何人物走出画面又重新进入画面，则检测到的每个人物的肢体ID信息是不变的；但是，若存在至少一个人物走出过画面又进入画面，则该人物的肢体ID信息丢失，而重新入画后该人物的肢体ID信息将会改变。因此，将最新检测到的每个人物的肢体ID信息与在启动某个特定功能时创建的人脸特征值进行匹配，人脸特征值包括相互对应的特定肢体ID信息和面部特征信息。将特定肢体ID信息与每个人物的肢体ID信息进行对比，以判断是否存在人物出画又入画(识别丢失)。Perform body ID information detection on the image information of the person. If no person leaves the screen and then re-enters the screen under the continuous shooting of the camera, the body ID information of each detected person is unchanged; however, if there is at least one If a character walks out of the screen and enters the screen again, the body ID information of the character will be lost, and the body ID information of the character will be changed after re-entering the painting. Therefore, the newly detected body ID information of each person is matched with the facial feature value created when a specific function is activated, and the facial feature value includes the specific body ID information and facial feature information corresponding to each other. The specific body ID information is compared with the body ID information of each character to determine whether there is a character that is drawn and drawn (identification lost).

若存在一个人物的肢体ID信息与特点肢体ID信息对比一致，说明提取到的肢体ID信息中存在人脸特征值中的特定肢体ID信息，则确定锁定追踪人物始终在人物图像信息中，说明摄像头采集的多个人物均未产生出画又入画情况，即每个人物始终位于摄像头的拍摄区域内，此时，可判定人物图像信息中存在特定肢体ID信息。例如，若特定肢体ID信息对应的编号为2#，而基于当前人物图像信息识别得到的每个人物的肢体ID信息为1#、2#、3#、4#、5#，此时存在2#人物对应的肢体ID信息与特定肢体ID信息相同，则判定人物图像信息中存在特定肢体ID信息。若人物图像信息中存在特定肢体ID信息，说明摄像头拍摄区域内的多个人物并未产生较大的位置变化，即并未走出过画面，未发生识别丢失情况，此时，摄像头可继续拍摄，实现对锁定追踪人物的持续追踪。If the body ID information of a person is consistent with the characteristic body ID information, it means that there is specific body ID information in the face feature value in the extracted body ID information, then it is determined that the locked and tracked person is always in the person image information, indicating that the camera The collected figures do not appear to be drawn and drawn, that is, each figure is always located in the shooting area of the camera. At this time, it can be determined that there is specific body ID information in the person image information. For example, if the number corresponding to the specific body ID information is 2#, and the body ID information of each person identified based on the current person image information is 1#, 2#, 3#, 4#, and 5#, then there are 2 # If the body ID information corresponding to the person is the same as the specific body ID information, it is determined that the specific body ID information exists in the person image information. If there is specific body ID information in the image information of the person, it means that the multiple people in the shooting area of the camera have not changed their positions greatly, that is, they have not walked out of the screen and have not lost their recognition. At this time, the camera can continue to shoot. Realize continuous tracking of locked and tracked characters.

若每一个人物的肢体ID信息与特点肢体ID信息对比均不一致，说明提取到的肢体ID信息中不存在人脸特征值中的特定肢体ID信息，则确定锁定追踪人物始终在人物图像信息中消失，即说明摄像头采集的某个人物产生出画又入画的情况，即该人物并未始终位于摄像头的拍摄区域内，此时，可判定人物图像信息中不存在特定肢体ID信息(识别丢失)。例如，若特定肢体ID信息对应的编号为2#，而基于当前人物图像信息识别得到的每个人物的肢体ID信息为1#、3#、4#、5#、6#。可见，当前的每个肢体ID信息与特定肢体ID信息均不相同，则判定人物图像信息中不存在特定肢体ID信息。If the body ID information of each character is inconsistent with the characteristic body ID information, it means that there is no specific body ID information in the face feature value in the extracted body ID information, then it is determined that the locked and tracked character always disappears in the character image information. , which means that a certain person captured by the camera has drawn and entered the picture, that is, the person is not always located in the shooting area of the camera. At this time, it can be determined that there is no specific body ID information in the person image information (identification is lost). For example, if the number corresponding to the specific body ID information is 2#, and the body ID information of each person identified based on the current person image information is 1#, 3#, 4#, 5#, and 6#. It can be seen that each current body ID information is different from the specific body ID information, and it is determined that the specific body ID information does not exist in the person image information.

由于肢体ID信息以编号的形式实现，因此，若没有人物出画又入画，则每个人物对应的编号不变。若存在至少一个人物出画又入画，则将重新入画的人物重新编号(原始编号丢失)，编号顺序为以前一时刻图像中已存在编号的后续号码进行编号。Since the body ID information is implemented in the form of numbers, if there are no characters in and out of the painting, the number corresponding to each character remains unchanged. If there is at least one character that has been drawn and re-entered, the re-entered character will be renumbered (the original number is lost), and the numbering sequence will be the subsequent number that has existed in the image at the previous moment.

图22中示例性示出了根据一些实施例的存在人物识别丢失的显示示意图。例如，参见图22中(a)，人物图像信息中存在五个人物的肢体ID信息，编号分别为1#、2#、3#、4#、5#。参见图22中(b)和(c)，若2#人物出画又入画(识别丢失)，则再次识别新采集到的人物图像信息时，将重新入画的2#人物识别为6#人物，将2#人物的肢体ID信息丢失，即改变该人物的肢体ID信息，此时，识别到的五个人物的肢体ID信息，编号分别为1#、3#、4#、5#、6#。FIG. 22 exemplarily shows a schematic diagram of a display that there is a loss of person identification according to some embodiments. For example, referring to Fig. 22 (a), there are five persons' body ID information in the person image information, and the numbers are respectively 1#, 2#, 3#, 4#, and 5#. Referring to (b) and (c) in Figure 22, if the 2# figure is drawn out of the painting and then re-entered (the recognition is lost), when re-identifying the newly collected figure image information, the 2# figure re-entered into the painting is identified as the 6# figure, Losing the body ID information of the 2# character, that is, changing the body ID information of the character, at this time, the body ID information of the five recognized characters are numbered 1#, 3#, 4#, 5#, 6# respectively. .

将以最新的每个人物的肢体ID信息，以对当前采集到的人物图像信息中的每个人物进行标识。例如，2#人物的肢体ID信息由原来的2#替换为当前的6#。The latest body ID information of each person will be used to identify each person in the currently collected person image information. For example, the body ID information of person 2# is replaced by the current 6# from the original 2#.

在锁定追踪人物在人物图像信息中消失时，人物图像信息中不存在特定肢体ID信息，说明摄像头拍摄区域内的多个人物中存在一个人物曾经产生过较大位置变化，即走出过画面又重新入画。而由于每个人物的肢体ID信息会随相应人物的出画入画识别结果而变化，也就是说，发生出画入画的人物也可能为锁定追踪人物。此时，为准确判断出与特定肢体ID信息不匹配的人物是否为锁定追踪人物，需要对采集到的当前人物图像信息中的人物重新创建当前人脸特征值。因此，获取每个人物的面部特征信息，并根据当前的每个人物的肢体ID信息，重新创建每个人物对应的当前人脸特征值。创建人脸特征值的方法可参照前述实施例的内容，此处不再赘述。When the locked and tracked person disappears in the person image information, there is no specific body ID information in the person image information, indicating that one of the multiple people in the camera shooting area has had a large position change, that is, walked out of the screen and restarted again. into the painting. Since the body ID information of each character will change with the recognition result of the corresponding character's in-painting and out-painting, that is to say, the characters who are out-painting and in-painting may also be locked and tracked characters. At this time, in order to accurately determine whether the person who does not match the specific body ID information is a locked and tracked person, it is necessary to recreate the current face feature value for the person in the collected current person image information. Therefore, the facial feature information of each character is acquired, and according to the current body ID information of each character, the current facial feature value corresponding to each character is recreated. For the method of creating a face feature value, reference may be made to the content of the foregoing embodiments, and details are not described herein again.

S23、如果当前人脸特征值与人脸特征值的特征相似度满足阈值条件，则将锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物。S23: If the feature similarity between the current face feature value and the face feature value satisfies the threshold condition, determine the locked tracked person as the current person whose feature similarity satisfies the threshold condition.

S24、基于当前人脸特征值，对锁定追踪人物进行锁定追踪。S24. Based on the current face feature value, lock and track the locked and tracked person.

由于锁定追踪人物出画又入画时，其对应的肢体ID信息发生变化，但其面部特征信息并不会改变。因此，为准确判断当前的人物中是否存在锁定追踪人物，可将每个人物重新创建的当前人脸特征值与锁定追踪人物的人脸特征值进行对比，并计算特征相似度。Since the locked and tracked characters are drawn and drawn, their corresponding body ID information will change, but their facial feature information will not change. Therefore, in order to accurately determine whether there is a locked and tracked character among the current characters, the current face feature value recreated for each character can be compared with the face feature value of the locked and tracked character, and the feature similarity can be calculated.

为准确判断出与特定肢体ID信息不匹配的人物是否为锁定追踪人物，需要继续对该人物进行面部特征信息匹配。每个人的面部特征信息单一存在，相互人物之间的面部特征信息不相同，可将面部特征信息作为该人物的标识信息。因此，需对当前采集到的人物图像信息进行面部特征信息提取，若人物图像信息中包括多个人物的图像，则可得到每个人物对应的面部特征信息。In order to accurately determine whether a person who does not match the specific body ID information is a locked and tracked person, it is necessary to continue to perform facial feature information matching on the person. The facial feature information of each person exists in a single manner, and the facial feature information of each person is different, and the facial feature information can be used as the identification information of the person. Therefore, it is necessary to extract facial feature information from the currently collected person image information. If the person image information includes images of multiple persons, the facial feature information corresponding to each person can be obtained.

在人物图像信息中不存在特定肢体ID信息时，为判断新采集的人物图像信息中包括的每个人物的面部特征信息是否为锁定追踪人物的人脸特征值中的面部特征信息时，采用计算当前每个人物的面部特征信息与人脸特征值中的面部特征信息的特征相似度。When there is no specific body ID information in the person image information, in order to determine whether the facial feature information of each person included in the newly collected person image information is the facial feature information in the facial feature value of the locked and tracked person, the calculation method is adopted. The feature similarity between the current facial feature information of each person and the facial feature information in the facial feature value.

由于一个面部特征信息对应一个人物，而不同人物的面部特征信息差异较大，因此，可通过计算特征相似度的方式判断摄像头采集的人物是否发生变化，即判断当前摄像头采集的人物中是否存在锁定追踪人物。Since one facial feature information corresponds to one character, and the facial feature information of different characters is quite different, it is possible to determine whether the characters captured by the camera have changed by calculating the feature similarity, that is, to determine whether there is a lock on the characters captured by the current camera. track people.

将当前每个人物的面部特征信息与人脸特征值中的面部特征信息的特征相似度，作为当前人脸特征值与人脸特征值的特征相似度。每个人物的面部特征信息(当前人脸特征值)对应一个特征相似度，将每个特征相似度与阈值进行对比。在一些实施例中，如果两个面部特征信息来自同一个人物，则特征相似度将大于90％，因此，可设定阈值为0.9。The feature similarity between the current facial feature information of each person and the facial feature information in the face feature value is taken as the feature similarity between the current face feature value and the face feature value. The facial feature information (current facial feature value) of each person corresponds to a feature similarity, and each feature similarity is compared with a threshold. In some embodiments, if the two facial feature information comes from the same person, the feature similarity will be greater than 90%, therefore, a threshold value of 0.9 can be set.

若当前人物图像信息中识别到的每个面部特征信息中，存在一个面部特征信息对应的特征相似度超过阈值(如0.9)，则可判定该特征相似度对应的面部特征信息所属人物可能与锁定追踪人物为同一个人，此时，将当前识别到的肢体ID信息作为该人物的新特定肢体ID信息，并基于新特定肢体ID信息对锁定追踪人物进行持续追踪。If in each facial feature information identified in the current person image information, there is a feature similarity corresponding to the facial feature information that exceeds the threshold (such as 0.9), it can be determined that the facial feature information corresponding to the feature similarity belongs to the person that may be locked. If the tracking person is the same person, at this time, the currently recognized body ID information is used as the new specific body ID information of the person, and the locked tracking person is continuously tracked based on the new specific body ID information.

例如，在初始识别的锁定追踪人物为2#人物，且该人物存在出画又入画(识别丢失)的场景下，获取当前人物图像信息中每个人物产生的特征相似度，以五个人物为例，特征相似度分别为α ₁、α ₃、α ₄、α ₅、α ₆。其中，2#人物对应的特定肢体ID信息在该人物出画时丢失。 For example, in the case where the initially identified locked and tracked character is character 2#, and the character exists in the scene where the character is drawn and then re-entered (the recognition is lost), the feature similarity generated by each character in the current character image information is obtained, and five characters are used as For example, the feature similarities are α ₁ , α ₃ , α ₄ , α ₅ , and α ₆ , respectively. Among them, the specific body ID information corresponding to the 2# character is lost when the character is drawn.

将每个特征相似度与阈值(如0.9)进行对比，若6#人物的特征相似度α ₆大于阈值，则将6#人物确定为锁定追踪人物，即6#人物与2#人物为同一个人，并将6#人物的肢体ID信息更新为锁定追踪人物的新特定肢体ID信息，后续以6#人物的肢体ID信息继续实现对原2#人物(锁定追踪人物)的锁定追踪。 Compare each feature similarity with a threshold (such as 0.9), if the feature similarity α6 of the ₆ # character is greater than the threshold, then determine the 6# character as the locked tracking character, that is, the 6# character and the 2# character are the same person , and update the limb ID information of the 6# person to the new specific limb ID information of the locked and tracked person, and then continue to implement the locked tracking of the original 2# person (the locked and tracked person) with the limb ID information of the 6# person.

在一些实施例中，若当前人物图像信息中识别到的每个面部特征信息中，不存在任何人物对应的面部特征信息产生的特征相似度超过阈值(如0.9)，即每个面部特征信息产生的特征相似度均小于阈值，则说明当前摄像头采集的人物中并不包括锁定追踪人物，此时，可由摄像头继续采集人物图像信息。In some embodiments, if the feature similarity generated by no facial feature information corresponding to any person in each facial feature information identified in the current person image information exceeds a threshold (such as 0.9), that is, each facial feature information generated The feature similarities of all are less than the threshold, which means that the characters currently collected by the camera do not include locked and tracked characters. At this time, the camera can continue to collect the image information of the characters.

步骤2411、在摄像头应用为AI健身应用时，确定锁定追踪人物为健身人员。Step 2411: When the camera application is an AI fitness application, determine that the locked and tracked person is a fitness person.

步骤2412、基于健身人员的人脸特征值，持续采集健身人员基于用户界面中呈现的示范视频进行跟练的跟练动作信息。Step 2412: Based on the facial feature values of the fitness personnel, continuously collect follow-up exercise information of the fitness personnel for follow-up exercises based on the demonstration video presented in the user interface.

步骤2413、基于跟练动作信息生成跟练画面，显示在用户界面中，跟练画面显示在示范视频所在画面的一侧。Step 2413: Generate a follow-up exercise picture based on the follow-up exercise action information, display it in the user interface, and display the follow-up exercise picture on the side of the picture where the demonstration video is located.

在实现AI健身场景时，同时利用显示设备配置的AI健身功能的人物可为一个或多个，若人物为一个时，该健身的人物即为锁定追踪人物。显示器的用户界面中呈现示范视频，便于健身人员进行跟练。When implementing an AI fitness scene, there can be one or more characters who use the AI fitness function configured on the display device at the same time. If there is one person, the fitness person is the locked and tracked person. Demonstration videos are presented in the display's user interface for easy follow-up by fitness personnel.

再次参见图17(a)所示，在健身人物为一个时，锁定追踪人物即为该健身人员。由于摄像头采集的人物图像信息中仅包括该人物的图像信息，因此，可基于该健身人员的人脸特征值，持续采集健身人员跟随示范视频进行跟练时产生的跟练动作信息。根据跟练动作信息生成跟练画面，显示在用户界面中，跟练画面显示在示范视频所在画面的一侧。Referring to Fig. 17(a) again, when there is one fitness person, the locked and tracked person is the fitness person. Since the image information of the person collected by the camera only includes the image information of the person, the follow-up action information generated when the fitness person follows the demonstration video for follow-up exercise can be continuously collected based on the fitness person's facial feature value. A follow-up exercise picture is generated according to the follow-up exercise action information, displayed on the user interface, and the follow-up exercise picture is displayed on the side of the screen where the demonstration video is located.

再次参见图17(b)所示，在健身人物为多个人时，摄像头采集到包括多个健身人员的当前人物图像信息，以包括三个健身人员为例，分别为1#、2#、3#，基于当前人物图像信息中的每个健身人员重新创建对应的当前人脸特征值。如果其中一个健身人员2#的当前人脸特征值与之前创建的人脸特征值的特征相似度满足阈值条件(阈值可设定为90％)，则将锁定追踪人物确定为特征相似度满足阈值条件对应的当前健身人员2#。Referring again to Fig. 17(b), when there are multiple fitness figures, the camera collects the current image information of a plurality of fitness people, and taking the example of including three fitness people, they are 1#, 2#, 3 #, Recreate the corresponding current face feature value based on each fitness person in the current person image information. If the feature similarity between the current face feature value of one of the fitness personnel 2# and the previously created face feature value satisfies the threshold condition (the threshold can be set to 90%), the locked tracker is determined as the feature similarity meets the threshold The current fitness person 2# corresponding to the condition.

而后基于锁定追踪人物(健身人员2#)的人脸特征值，持续采集健身人员2#跟随示范视频进行跟练的跟练动作信息。根据跟练动作信息生成跟练画面，显示在用户界面中，跟练画面显示在示范视频所在画面的一侧。此时，用户界面中仅会显示锁定追踪人物的跟练画面，而其他人物的跟练画面不会显示，可参照图17(a)中所示内容。Then, based on the facial feature value of the locked and tracked person (fitness person 2#), the follow-up action information of fitness person 2# following the demonstration video for follow-up training is continuously collected. A follow-up exercise picture is generated according to the follow-up exercise action information, displayed on the user interface, and the follow-up exercise picture is displayed on the side of the screen where the demonstration video is located. At this time, only the follow-up training screen of the locked and tracked character will be displayed in the user interface, and the follow-up training screen of other characters will not be displayed, please refer to the content shown in FIG. 17( a ).

步骤2421、在摄像头应用为视频通话应用时，确定锁定追踪人物为视频通话人员。Step 2421 , when the camera application is a video call application, determine that the locked and tracked person is the video call person.

步骤2422、在视频通话人员移动时，基于视频通话人员的人脸特征值，调整摄像头的拍摄角度，以使视频通话人员持续处于摄像头的采集区域内。Step 2422: When the video caller moves, adjust the shooting angle of the camera based on the facial feature value of the video caller, so that the video caller is continuously within the capture area of the camera.

再次参见图18所示，在利用显示设备进行视频通话的人物为一个时，则锁定追踪人物为该视频通话人员，视频通话的对方画面显示在用户界面中，锁定追踪人物的本端视频画面显示在对方画面的一侧。由于摄像头采集的人物图像信息中仅包括该人物的图像信息，因此，可基于该视频通话人员的人脸特征值，持续采集视频通话人员的图像信息，以基于图像信息识别视频通话人员的位置。若识别到视频通话人员的位置发生变化，则根据位置变化方向和程度调整摄像头的拍摄角度，以使视频通话人员持续处于摄像头的采集区域内。调整摄像头拍摄角度的示意图可参照图19所示内容。Referring to Fig. 18 again, when there is only one person using the display device to conduct a video call, the locked and tracked person is the person on the video call, the other side of the video call is displayed in the user interface, and the local video screen of the locked and tracked person is displayed. on the other side of the screen. Since the image information of the person collected by the camera only includes the image information of the person, the image information of the person on the video call can be continuously collected based on the facial feature value of the person on the video call to identify the location of the person on the video call based on the image information. If it is recognized that the position of the person on the video call has changed, the shooting angle of the camera is adjusted according to the direction and degree of the position change, so that the person on the video call continues to be in the capture area of the camera. For a schematic diagram of adjusting the shooting angle of the camera, please refer to the content shown in FIG. 19 .

再次参见图19所示，在利用显示设备进行视频通话的人物为多个时，摄像头采集到包括多个视频通话人员的当前人物图像信息，以包括三个视频通话人员为例，本端视频画面中显示1#、2#、3#视频通话人员，基于当前人物图像信息中的每个视频通话人员重新创建对应的当前人脸特征值。如果其中一个视频通话人员1#的当前人脸特征值与之前创建的人脸特征值的特征相似度满足阈值条件(阈值可设定为90％)，则将锁定追踪人物确定为特征相似度满足阈值条件对应的当前视频通话人员1#。Referring to Fig. 19 again, when there are multiple people using the display device to make a video call, the camera collects image information of the current person including multiple video callers. Taking three video callers as an example, the local video screen 1#, 2#, and 3# video callers are displayed in the display, and the corresponding current face feature value is recreated based on each video caller in the current person image information. If the feature similarity between the current face feature value of one of the video callers 1# and the previously created face feature value satisfies the threshold condition (the threshold can be set to 90%), the locked tracking person is determined as the feature similarity satisfying Person 1# of the current video call corresponding to the threshold condition.

控制器对摄像头每采集到的每一张人物图像信息均执行步骤S21至S24的过程，直到检测到存在一个人物产生的特征相似度超过阈值，或者，连续检测的人物图像信息的数量超过设定数量仍然未检测到存在一个人物产生的特征相似度超过阈值时止。在一些实施例中，设定数量可为10张。The controller performs the process of steps S21 to S24 for each piece of person image information collected by the camera until it is detected that there is a person whose feature similarity exceeds the threshold, or the number of continuously detected person image information exceeds the set value. The number still does not detect the presence of a character whose similarity of features exceeds the threshold. In some embodiments, the set number may be 10 sheets.

若后续在检测到存在一个人物产生的特征相似度超过阈值时，则将大于或等于阈值的特征相似度对应的人物确定为锁定追踪人物，以及，将锁定追踪人物的肢体ID信息确定为锁定追踪人物的新特定肢体ID信息，实现对该人物的锁定追踪。If it is subsequently detected that the feature similarity generated by a character exceeds the threshold, the character corresponding to the feature similarity greater than or equal to the threshold is determined as the locked tracking character, and the body ID information of the locked tracking character is determined as the locked tracker The new specific body ID information of the character, realizes the locked tracking of the character.

若连续检测的人物图像信息的数量超过设定数量，例如超过10张，依然没有检测到任一个人物产生的特征相似度超过阈值，则说明当前摄像头采集到的人物均非锁定追踪人物，即锁定追踪人物消失，此时，停止追踪过程。If the number of continuously detected person image information exceeds the set number, for example, more than 10 images, and still no feature similarity generated by any person is detected that exceeds the threshold, it means that the characters captured by the current camera are not locked and tracked, that is, locked. The tracking person disappears, at this point, the tracking process is stopped.

在一些实施例中，在达到检测设定数量时，如果仍未检测到锁定追踪人物，则可在当前摄像头拍摄的人物中再次选定一个人物作为下一次追踪过程的锁定追踪人物。In some embodiments, when the detection set number is reached, if the locked and tracked person is still not detected, one person may be selected again from the characters captured by the current camera as the locked and tracked person in the next tracking process.

在重新确定新的锁定追踪人物时，需根据当前摄像头采集的新人物图像信息重新创建人脸特征值，重新创建人脸特征值的实现过程可参照前述实施例提供的步骤S11至S13的内容，此处不再赘述。When re-determining a new locked and tracked person, it is necessary to re-create the face feature value according to the image information of the new person collected by the current camera. The realization process of re-creating the face feature value can refer to the content of steps S11 to S13 provided in the foregoing embodiment, It will not be repeated here.

基于新创建的人脸特征值，由显示设备再次实现对新确定的锁定追踪人物进行锁定追踪，锁定追踪的实现过程可参照前述实施例提供的步骤S21至S24的内容，此处不再赘述。Based on the newly created facial feature value, the display device implements locking and tracking of the newly determined locked and tracked person again. For the implementation process of locking and tracking, reference may be made to the content of steps S21 to S24 provided in the foregoing embodiments, which will not be repeated here.

由以上技术方案可知，本申请实施例提供的一种显示设备，由控制器根据摄像头采集的人物图像信息进行特定动作识别，确定锁定追踪人物；根据锁定追踪人物对应的特点肢体ID信息和面部特征信息，创建锁定追踪人物的人脸特征值。在进行锁定追踪时，如果锁定追踪人物在所述人物图像信息中消失，则基于人物图像信息中的当前人物重新创建当前人脸特征值，计算当前人脸特征值与人脸特征值的特征相似度，将锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物，并根据当前人脸特征值，对锁定追踪人物进行锁定追踪。可见，本申请实施例提供的显示设备，先创建与锁定追踪人物对应的人脸特征值，通过人脸特征值可以有效识别丢失前后图像识别的是否为同一个人，进而可以保证锁定追踪人物因移动导致识别丢失后仍可连续追踪，保证追踪有效性。As can be seen from the above technical solutions, in the display device provided by the embodiments of the present application, the controller performs specific action recognition according to the person image information collected by the camera, and determines the locked and tracked person; according to the characteristics corresponding to the locked and tracked person, body ID information and facial features information to create facial feature values that lock and track people. During locked tracking, if the locked tracking character disappears in the character image information, the current face feature value is recreated based on the current character in the character image information, and the current face feature value is calculated to be similar to the features of the face feature value. The locked and tracked person is determined as the current person whose feature similarity satisfies the threshold condition, and the locked and tracked person is locked and tracked according to the current face feature value. It can be seen that the display device provided in the embodiment of the present application first creates a face feature value corresponding to the locked and tracked person, and through the face feature value, it can be effectively identified whether the image recognition before and after the loss is the same person, and then it can be ensured that the locked and tracked person moves due to movement. After the identification is lost, it can still be tracked continuously to ensure the tracking effectiveness.

具体实现中，本申请还提供一些非易失性计算机存储介质，其中，该计算机存储介质可存储有程序，该程序执行时可包括本申请提供的屏保展示方法和屏保跳转方法的各实施例中的部分或全部步骤。所述的存储介质可为磁碟、光盘、只读存储记忆体(英文：read-only memory，简称：ROM)或随机存储记忆体(英文：random access memory，简称：RAM)等。In a specific implementation, the present application also provides some non-volatile computer storage media, wherein the computer storage medium can store a program, and when the program is executed, it can include the various embodiments of the screen saver display method and the screen saver jump method provided by the present application some or all of the steps in . The storage medium may be a magnetic disk, an optical disk, a read-only memory (English: read-only memory, abbreviated as: ROM) or a random access memory (English: random access memory, abbreviated as: RAM) and the like.

本领域的技术人员可以清楚地了解到本申请实施例中的技术可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本申请实施例中的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例或者实施例的某些部分所述的方法。Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions in the embodiments of the present application can be embodied in the form of software products in essence or in the parts that make contributions to related technologies, and the computer software products can be stored in storage media, such as ROM/RAM, A magnetic disk, an optical disk, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present application.

最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. Scope.

为了方便解释，已经结合具体的实施方式进行了上述说明。但是，上述示例性的讨论不是意图穷尽或者将实施方式限定到上述公开的具体形式。根据上述的教导，可以得到多种修改和变形。上述实施方式的选择和描述是为了更好的解释原理以及实际的应用，从而使得本领域技术人员更好的使用所述实施方式以及适于具体使用考虑的各种不同的变形的实施方式。For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussions are not intended to be exhaustive or to limit implementations to the specific forms disclosed above. Numerous modifications and variations are possible in light of the above teachings. The above embodiments are chosen and described to better explain the principles and practical applications, so as to enable those skilled in the art to better utilize the described embodiments and various modified embodiments suitable for specific use considerations.

Claims

一种显示设备，包括：A display device comprising:

显示器；monitor;

外部装置接口，被配置为连接摄像头，所述摄像头可转动拍摄角度，被配置为拍摄图像；an external device interface configured to connect a camera, the camera can be rotated to a shooting angle and configured to capture images;

控制器，被配置为：Controller, configured as:

获取用户输入的用于启动摄像头应用的控制指令；Obtain the control instruction input by the user to start the camera application;

响应于所述控制指令，从所述摄像头拍摄的图像中检测人像位置；In response to the control instruction, detecting the position of the portrait from the image captured by the camera;

按照所述人像位置向所述摄像头发送旋转指令，以使所述人像位置位于预设判断区域内。A rotation instruction is sent to the camera according to the position of the portrait, so that the position of the portrait is located in a preset judgment area.
根据权利要求1所述的显示设备，从所述摄像头拍摄的图像中检测人像位置的步骤中，所述控制器被进一步配置为：The display device according to claim 1, in the step of detecting the position of the portrait from the image captured by the camera, the controller is further configured to:

提取校对图像；Extract proofing images;

在所述校对图像中识别特征形状；identifying characteristic shapes in the proofreading image;

将所述特征形状与预设模板进行匹配，定位多个关键点；Matching the feature shape with a preset template to locate multiple key points;

根据所述关键点位置计算中心位置，以获得人像位置。Calculate the center position according to the key point position to obtain the portrait position.
根据权利要求1所述的显示设备，按照所述人像位置向所述摄像头发送旋转指令的步骤中，所述控制器被进一步配置为：The display device according to claim 1, wherein in the step of sending a rotation instruction to the camera according to the portrait position, the controller is further configured to:

对比所述人像位置与预设判断区域；comparing the portrait position with the preset judgment area;

如果所述人像位置位于所述预设判断区域内，控制显示器实时显示所述摄像头拍摄的图像；If the position of the portrait is located in the preset judgment area, control the display to display the image captured by the camera in real time;

如果所述人像位置位于所述预设判断区域外，计算所述人像位置与所述预设判断区域中心的坐标差；If the portrait position is outside the preset judgment area, calculate the coordinate difference between the portrait position and the center of the preset judgment area;

根据所述坐标差生成旋转指令，以及向所述摄像头发送所述旋转指令。A rotation instruction is generated according to the coordinate difference, and the rotation instruction is sent to the camera.
根据权利要求3所述的显示设备，根据所述坐标差生成旋转指令的步骤中，所述控制器被进一步配置为：The display device according to claim 3, in the step of generating a rotation instruction according to the coordinate difference, the controller is further configured to:

对比所述人像位置坐标和所述预设判断区域中心坐标，以获得旋转方向；Comparing the position coordinates of the portrait and the center coordinates of the preset judgment area to obtain the rotation direction;

计算所述人像位置坐标与所述预设判断区域中心坐标之间的距离；Calculate the distance between the position coordinates of the portrait and the center coordinates of the preset judgment area;

根据计算的距离，结合摄像头参数，计算旋转角度，所述摄像头参数包括预览分辨率和最大视角；Calculate the rotation angle according to the calculated distance in combination with camera parameters, where the camera parameters include preview resolution and maximum viewing angle;

封装所述旋转方向和所述旋转角度，以生成所述旋转指令。The rotation direction and the rotation angle are encapsulated to generate the rotation instruction.
根据权利要求1所述的显示设备，从所述摄像头拍摄的图像中检测人像位置的步骤中，所述控制器被进一步配置为：The display device according to claim 1, in the step of detecting the position of the portrait from the image captured by the camera, the controller is further configured to:

在校对图像中检测第一识别点，所述第一识别点为关键点中的一个或多个；detecting a first identification point in the proofreading image, where the first identification point is one or more of the key points;

如果所述校对图像中不含有第一识别点，在所述校对图像中检测第二识别点，所述第二识别点为与所述第一识别点具有预设方位关系的关键点；If the proofreading image does not contain the first identification point, a second identification point is detected in the proofreading image, and the second identification point is a key point having a preset azimuth relationship with the first identification point;

按照第二识别点与所述第一识别点的方位关系生成转动方向。The rotation direction is generated according to the azimuth relationship between the second identification point and the first identification point.
根据权利要求5所述的显示设备，所述第一识别点为用户眼睛点，所述第二识别点为用户肩部点；从所述摄像头拍摄的图像中检测人像位置的步骤中，所述控制器被进一步配置为：The display device according to claim 5, wherein the first identification point is a user's eye point, and the second identification point is a user's shoulder point; in the step of detecting the position of the portrait from the image captured by the camera, the The controller is further configured to:

如果所述校对图像中不含有用户眼睛点，在所述校对图像中检测肩部点；if the user's eye point is not contained in the proofreading image, detecting a shoulder point in the proofreading image;

向所述摄像头发送旋转指令，以控制所述摄像头抬高拍摄角度，使拍摄的图像中含有用户眼睛点，且所述眼睛点处于图像中1/7-1/5高度区间的位置处。A rotation instruction is sent to the camera to control the camera to raise the shooting angle, so that the captured image contains the user's eye point, and the eye point is located at the position of the 1/7-1/5 height interval in the image.
根据权利要求5所述的显示设备，从所述摄像头拍摄的图像中检测人像位置的步骤中，所述控制器被进一步配置为：The display device according to claim 5, in the step of detecting the position of the portrait from the image captured by the camera, the controller is further configured to:

在所述校对图像中检测两个所述第一识别点或两个所述第二识别点之间的距离；detecting the distance between two of the first identification points or two of the second identification points in the proofreading image;

根据两个所述第一识别点或两个所述第二识别点之间的距离匹配预设调节步长；Match a preset adjustment step size according to the distance between the two first identification points or the two second identification points;

封装所述旋转方向以及预设调节步长，以生成旋转指令。The rotation direction and the preset adjustment step are packaged to generate a rotation instruction.
根据权利要求5所述的显示设备，从所述摄像头拍摄的图像中检测人像位置的步骤中，如果所述校对图像中含有用户眼睛点，所述控制器被进一步配置为：The display device according to claim 5, in the step of detecting the position of the portrait from the image captured by the camera, if the proofreading image contains user eye points, the controller is further configured to:

检测用户眼睛点在所述校对图像中的高度区间；detecting the height interval of the user's eye point in the proofreading image;

如果用户眼睛点在所述校对图像高度的1/5以下，向所述摄像头发送旋转指令，以控制所述摄像头降低拍摄高度，至用户眼睛点处于在所述校对图像高度的1/6处停止；If the user's eye point is below 1/5 of the height of the proofreading image, send a rotation instruction to the camera to control the camera to lower the shooting height until the user's eye point is at 1/6 of the height of the proofreading image and stop ;

如果用户眼睛点在所述校对图像高度的1/7-1/5区间内，取消向所述摄像头发送旋转指令，以防止摄像头抖动对检测结果的影响。If the user's eye point is within the range of 1/7-1/5 of the height of the proofreading image, cancel sending a rotation instruction to the camera to prevent camera shake from affecting the detection result.
一种显示设备，包括：A display device comprising:

显示器；monitor;

摄像头，所述摄像头可转动拍摄角度，被配置为拍摄图像；a camera, which is rotatable at a shooting angle and configured to capture images;

控制器，被配置为：Controller, configured as:

获取用户输入的用于启动摄像头应用的控制指令；Obtain the control instruction input by the user to start the camera application;

响应于所述控制指令，从所述摄像头拍摄的图像中检测人像位置；In response to the control instruction, detecting the position of the portrait from the image captured by the camera;

按照所述人像位置向所述摄像头发送旋转指令，以使所述人像位置位于预设判断区域内。A rotation instruction is sent to the camera according to the position of the portrait, so that the position of the portrait is located in a preset judgment area.
一种人像定位方法，应用于显示设备，所述显示设备包括显示器和控制器，所述显示设备内置或通过外部装置接口外接摄像头，所述摄像头可转动拍摄角度，所述人像定位方法包括：A portrait positioning method is applied to a display device, the display device includes a display and a controller, the display device has a built-in or externally connected camera through an external device interface, the camera can rotate a shooting angle, and the portrait positioning method includes:

获取用户输入的用于启动摄像头应用的控制指令；Obtain the control instruction input by the user to start the camera application;

响应于所述控制指令，从所述摄像头拍摄的图像中检测人像位置；In response to the control instruction, detecting the position of the portrait from the image captured by the camera;

按照所述人像位置向所述摄像头发送旋转指令，以使所述人像位置位于预设判断区域内。A rotation instruction is sent to the camera according to the position of the portrait, so that the position of the portrait is located in a preset judgment area.
一种显示设备，包括：A display device comprising:

显示器，被配置为呈现用户界面；a display configured to present a user interface;

摄像头，被配置为采集人物在使用摄像头应用时产生的人物图像信息；a camera, configured to collect the image information of the person generated when the person uses the camera application;

与所述显示器和所述摄像头连接的控制器，所述控制器被配置为：a controller connected to the display and the camera, the controller configured to:

获取所述摄像头采集的人物图像信息；obtaining the person image information collected by the camera;

识别所述人物图像信息，确定锁定追踪人物，以及，创建所述锁定追踪人物的面部特征信息和特定肢体ID信息；Identify the person image information, determine the locked and tracked person, and create the facial feature information and specific body ID information of the locked and tracked person;

将所述锁定追踪人物的特定肢体ID信息和面部特征信息建立关联关系，得到所述锁定追踪人物的人脸特征值。The specific limb ID information and the facial feature information of the locked and tracked person are established in an associative relationship to obtain the facial feature value of the locked and tracked character.
根据权利要求11所述的显示设备，所述控制器在执行所述识别所述人物图像信息，确定锁定追踪人物，被进一步配置为：The display device according to claim 11, wherein the controller is further configured to:

提取所述人物图像信息中人物的肢体关键点，确定人物的肢体关键点信息；Extracting the key points of the person's limbs in the person image information, and determining the key point information of the person's limbs;

对所述人物的肢体关键点信息进行肢体动作识别；Perform body motion recognition on the body key point information of the character;

如果所述人物的肢体关键点信息对应的肢体动作与特定动作一致，则将所述人物确定为锁定追踪人物。If the body movement corresponding to the body key point information of the character is consistent with a specific movement, the character is determined to be a locked and tracked character.
根据权利要求11所述的显示设备，所述控制器被进一步配置为：The display device of claim 11, the controller is further configured to:

如果所述锁定追踪人物持续处于所述人物图像信息中，则基于所述人脸特征值对所述锁定追踪人物进行锁定追踪。If the locked and tracked person continues to be in the person image information, the locked and tracked person is locked and tracked based on the face feature value.
根据权利要求13所述的显示设备，所述控制器在执行所述基于所述人脸特征值对所述锁定追踪人物进行锁定追踪，被进一步配置为：The display device according to claim 13, wherein the controller is further configured to:

在所述摄像头应用为AI健身应用时，确定所述锁定追踪人物为健身人员；When the camera application is an AI fitness application, determine that the locked and tracked person is a fitness person;

基于所述健身人员的人脸特征值，持续采集所述健身人员基于用户界面中呈现的示范视频进行跟练的跟练动作信息；Based on the facial feature value of the fitness personnel, continuously collect the follow-up exercise information of the fitness personnel's follow-up exercise based on the demonstration video presented in the user interface;

基于所述跟练动作信息生成跟练画面，显示在所述用户界面中，所述跟练画面显示在所述示范视频所在画面的一侧。A follow-up exercise picture is generated based on the follow-up exercise action information and displayed on the user interface, and the follow-up exercise picture is displayed on one side of the picture where the demonstration video is located.
根据权利要求13所述的显示设备，所述控制器在执行所述基于所述人脸特征值对所述锁定追踪人物进行锁定追踪，被进一步配置为：The display device according to claim 13, wherein the controller is further configured to:

在所述摄像头应用为视频通话应用时，确定所述锁定追踪人物为视频通话人员；When the camera application is a video call application, determining that the locked and tracked person is a video call person;

在所述视频通话人员移动时，基于所述视频通话人员的人脸特征值，调整所述摄像头的拍摄角度，以使所述视频通话人员持续处于所述摄像头的采集区域内。When the video caller moves, the shooting angle of the camera is adjusted based on the facial feature value of the video caller, so that the video caller is continuously within the capture area of the camera.
根据权利要求11所述的显示设备，所述控制器被进一步配置为：The display device of claim 11, the controller is further configured to:

在基于所述人脸特征值对所述锁定追踪人物进行锁定追踪时，如果所述锁定追踪人物在所述人物图像信息中消失，则获取所述摄像头采集的当前人物图像信息；When the locked and tracked character is locked and tracked based on the facial feature value, if the locked and tracked character disappears in the character image information, obtain the current character image information collected by the camera;

基于所述当前人物图像信息中的当前人物重新创建当前人脸特征值；Recreate the current face feature value based on the current character in the current character image information;

如果所述当前人脸特征值与所述人脸特征值的特征相似度满足阈值条件，则将所述锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物。If the feature similarity between the current face feature value and the face feature value satisfies the threshold condition, the locked tracked person is determined as the current person whose feature similarity satisfies the threshold condition.
根据权利要求16所述的显示设备，所述控制器被进一步配置为：The display device of claim 16, the controller is further configured to:

在基于所述人脸特征值对所述锁定追踪人物进行锁定追踪时，提取所述人物图像信息中人物的肢体ID信息；extracting the body ID information of the character in the character image information when the locked and tracked character is locked and tracked based on the facial feature value;

如果提取到的肢体ID信息中不存在所述人脸特征值中的特定肢体ID信息，则确定所述锁定追踪人物在所述人物图像信息中消失。If the specific body ID information in the face feature value does not exist in the extracted body ID information, it is determined that the locked and tracked person disappears in the person image information.
一种显示设备，包括：A display device comprising:

显示器，被配置为呈现用户界面；a display configured to present a user interface;

摄像头，被配置为采集人物在使用摄像头应用时产生的人物图像信息；a camera, configured to collect the image information of the person generated when the person uses the camera application;

存储器，被配置为存储在确定锁定追踪人物时创建的人脸特征值；a memory configured to store the facial feature values created when the tracking person is determined to be locked;

与所述显示器、摄像头和所述存储器连接的控制器，所述控制器被配置为：a controller connected to the display, the camera and the memory, the controller being configured to:

获取锁定追踪人物的人脸特征值和所述摄像头采集的人物图像信息；Obtain the facial feature value of the locked and tracked person and the person image information collected by the camera;

如果所述锁定追踪人物在所述人物图像信息中消失，则基于所述人物图像信息中的当前人物重新创建当前人脸特征值；If the locked tracking character disappears in the character image information, then recreate the current face feature value based on the current character in the character image information;

如果所述当前人脸特征值与所述人脸特征值的特征相似度满足阈值条件，则将所述锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物；If the feature similarity between the current face feature value and the face feature value satisfies the threshold condition, then the locked and tracked character is determined as the current character whose feature similarity meets the threshold condition;

基于所述当前人脸特征值，对所述锁定追踪人物进行锁定追踪。Based on the current face feature value, the locked tracking person is locked and tracked.
根据权利要求18所述的显示设备，所述控制器在确定所述锁定追踪人物在所述人物图像信息中消失，被进一步配置为：The display device according to claim 18, wherein the controller is further configured to:

提取所述人物图像信息中人物的肢体ID信息和人脸特征值中的特定肢体ID信息；Extract the body ID information of the character in the character image information and the specific body ID information in the face feature value;

如果所述肢体ID信息中不存在所述特定肢体ID信息，则确定所述锁定追踪人物在所述人物图像信息中消失。If the specific body ID information does not exist in the body ID information, it is determined that the locked tracking person disappears in the person image information.
根据权利要求18所述的显示设备，所述控制器在执行所述基于所述当前人脸特征值，对所述锁定追踪人物进行锁定追踪，被进一步配置为：The display device according to claim 18, wherein the controller is further configured to:

在所述摄像头应用为AI健身应用时，确定所述锁定追踪人物为健身人员；When the camera application is an AI fitness application, determine that the locked and tracked person is a fitness person;

基于所述健身人员的人脸特征值，持续采集所述健身人员基于用户界面中呈现的示范视频进行跟练的跟练动作信息；Based on the facial feature value of the fitness personnel, continuously collect the follow-up exercise information of the fitness personnel's follow-up exercise based on the demonstration video presented in the user interface;

基于所述跟练动作信息生成跟练画面，显示在所述用户界面中，所述跟练画面显示在所述示范视频所在画面的一侧。A follow-up exercise picture is generated based on the follow-up exercise action information and displayed on the user interface, and the follow-up exercise picture is displayed on one side of the picture where the demonstration video is located.
根据权利要求18所述的显示设备，所述控制器在执行所述基于所述当前人脸特征值，对所述锁定追踪人物进行锁定追踪，被进一步配置为：The display device according to claim 18, wherein the controller is further configured to:

在所述摄像头应用为视频通话应用时，确定所述锁定追踪人物为视频通话人员；When the camera application is a video call application, determining that the locked and tracked person is a video call person;

在所述视频通话人员移动时，基于所述视频通话人员的人脸特征值，调整所述摄像头的拍摄角度，以使所述视频通话人员持续处于所述摄像头的采集区域内。When the video caller moves, the shooting angle of the camera is adjusted based on the facial feature value of the video caller, so that the video caller is continuously within the capture area of the camera.
一种人脸特征值创建方法，所述方法包括：A method for creating a face feature value, the method comprising:

获取所述摄像头采集的人物图像信息；obtaining the person image information collected by the camera;

识别所述人物图像信息，确定锁定追踪人物，以及，创建所述锁定追踪人物的面部特征信息和特定肢体ID信息；Identify the person image information, determine the locked and tracked person, and create the facial feature information and specific body ID information of the locked and tracked person;

将所述锁定追踪人物的特定肢体ID信息和面部特征信息建立关联关系，得到所述锁定追踪人物的人脸特征值。An association relationship is established between the specific body ID information of the locked and tracked person and the facial feature information to obtain the facial feature value of the locked and tracked person.
一种人物锁定追踪方法，所述方法包括：A person locking and tracking method, the method includes:

获取锁定追踪人物的人脸特征值和所述摄像头采集的人物图像信息；Obtain the facial feature value of the locked and tracked person and the person image information collected by the camera;

如果所述锁定追踪人物在所述人物图像信息中消失，则基于所述人物图像信息中的当前人物重新创建当前人脸特征值；If the locked tracking character disappears in the character image information, re-create the current face feature value based on the current character in the character image information;

如果所述当前人脸特征值与所述人脸特征值的特征相似度满足阈值条件，则将所述锁定追踪人物确定为特征相似度满足阈值条件对应的当前人物；If the feature similarity between the current face feature value and the face feature value satisfies the threshold condition, then the locked and tracked character is determined as the current character whose feature similarity meets the threshold condition;

基于所述当前人脸特征值，对所述锁定追踪人物进行锁定追踪。Based on the current face feature value, the locked tracking person is locked and tracked.