WO2015021746A1

WO2015021746A1 - Processing method, apparatus and system for displaying indicator

Info

Publication number: WO2015021746A1
Application number: PCT/CN2014/070140
Authority: WO
Inventors: 刘兆祥; 胡伟; 张爱东
Original assignee: 华为技术有限公司
Priority date: 2013-08-14
Filing date: 2014-01-06
Publication date: 2015-02-19
Also published as: CN104375626A; CN104375626B

Abstract

Provided are a processing method, apparatus and system for displaying an indicator. The method comprises: obtaining a current scenario image comprising a mobile terminal held by a hand, the hand and an image presentation region; determining, according to the current scenario image, an indication direction in which the mobile terminal points to the image presentation region; determining, according to the indication direction and a range of the image presentation region in the current scenario image, an association position of a region which is between the indication direction and the image presentation region and is located in the image presentation region; and displaying an indicator in the associated position according to a display signal provided by the mobile terminal. By using the method, a user can make an indication in an image presentation region at any time by using any mobile terminal; the method has good generality and is convenient to operate, thereby bringing convenience to work and study of the user.

Description

显示指示标识的处理方法、装置和*** Processing method, device and system for displaying indication mark

本申请要求于 2013 年 8 月 14 日提交中国专利局、申请号为 201310354098.6 , 发明名称为 "显示指示标识的处理方法、装置和*** This application is required to be submitted to the China Patent Office on August 14, 2013, and the application number is 201310354098.6. The invention name is "Processing method, device and system for displaying indication signs".

" 的中国专利申请优先权，上述专利的全部内容通过引用结合在本申请中。技术领域本发明实施例涉及电子信息技术领域，尤其涉及一种显示指示标识的处理方法、装置和***。 The disclosure of the Chinese Patent Application, the entire disclosure of which is incorporated herein by reference.

背景技术在当今的会议场景中，投影仪及投影幕布是最常用的信息展示手段，参会人员在根据投影幕布所展示的信息进行讲解和讨论时，常常通过激光笔在投影幕布上显示激光圓点，对投影内容做以指示。但在一些会议场景中，激光笔往往被忽略携带或者数量有限，造成很多不便，因此，若能让人们日常携带的移动终端，如手机、 PDA ( Personal Digital Assistant, 个人数字助理）等具备类似激光笔的指示功能，将大大方便人们的工作。 BACKGROUND OF THE INVENTION In today's conference scenes, projectors and projection screens are the most commonly used means of information display. When the participants explain and discuss according to the information displayed on the projection screen, the laser circle is often used to display the laser circle on the projection screen. Point, make an indication of the projection content. However, in some conference scenarios, laser pointers are often neglected or limited in number, causing a lot of inconvenience. Therefore, if people can carry mobile terminals such as mobile phones, PDAs (Personal Digital Assistants, etc.) with similar lasers. The pen's indication function will greatly facilitate people's work.

为使上述移动终端具备类似激光笔的指示功能，在现有技术中，通常使移动终端内置一个激光二极管，并配置对应的驱动电路，从而使该激光二极管发出激光，在投影幕布上出现指示效果。 In order to enable the mobile terminal to have an indication function similar to that of a laser pointer, in the prior art, a laser diode is usually built in the mobile terminal, and a corresponding driving circuit is disposed, so that the laser diode emits laser light, and an indication effect appears on the projection screen. .

但实现上述方案时，需对移动终端现有的硬件结构和软件程序进行更改，而当前不同类型的移动终端的硬件结构和软件程序各有不同，需针对不同类型的移动终端设计不同的更改方案，耗费人力物力，且通用性较差。 However, when implementing the above solution, the existing hardware structure and software program of the mobile terminal need to be changed. Currently, the hardware structures and software programs of different types of mobile terminals are different, and different modification schemes need to be designed for different types of mobile terminals. It is labor-intensive and versatile.

发明内容本发明实施例提供一种显示指示标识的处理方法、装置和***，用于通过使用移动终端可对呈像区域做出指示。 SUMMARY OF THE INVENTION Embodiments of the present invention provide a processing method, apparatus, and system for displaying an indication identifier, which can be used to indicate an image area by using a mobile terminal.

第一方面，本发明实施例提供一种显示指示标识的处理方法，包括：获取包括人手握持的移动终端和所述人手及呈像区域的当前场景图像； In a first aspect, an embodiment of the present invention provides a processing method for displaying an indication identifier, including: acquiring a current scene graph including a mobile terminal held by a human hand and the human hand and a rendering area. Like

根据所述当前场景图像，确定所述移动终端指向所述呈像区域的指示方向； Determining, according to the current scene image, an indication direction of the mobile terminal pointing to the image area;

根据所述指示方向与所述呈像区域在所述当前场景图像中的范围，确定所述指示方向与所述呈像区域之间位于所述呈像区域中的关联位置； Determining, according to the indication direction and a range of the image area in the current scene image, an associated position between the indication direction and the image area in the image area;

根据所述移动终端提供的显示信号，在所述关联位置显示指示标识。结合第一方面，在第一实施方式中，所述根据所述当前场景图像，确定所述移动终端的指向所述呈像区域的指示方向，包括： And displaying an indication identifier at the associated location according to the display signal provided by the mobile terminal. With reference to the first aspect, in the first implementation, the determining, by the current scene image, the indication direction of the mobile terminal that is directed to the image area includes:

根据预设的人手识别特征，在所述当前场景图像中确定人手所在位置； Determining a location of a human hand in the current scene image according to a preset human hand recognition feature;

基于所述人手所在位置，采用边缘检测算子确定所述人手握持的所述移动终端的特征平面，所述特征平面为所述移动终端的一个具有中轴线的几何平面图形，或者所述特征平面为所述移动终端所显示的具有中轴线的几何平面图形； Determining, according to the location of the human hand, an edge detection operator, a feature plane of the mobile terminal held by the human hand, the feature plane being a geometric plane graphic having a central axis of the mobile terminal, or the feature a plane is a geometric plane graphic having a central axis displayed by the mobile terminal;

根据所述移动终端的特征平面，确定所述移动终端的特征平面的中轴线作为所述指示方向。 Determining a central axis of a feature plane of the mobile terminal as the indication direction according to a feature plane of the mobile terminal.

结合第一方面或第一方面的第一实施方式，在第二实施方式中，在根据所述移动终端提供的显示信号，在所述关联位置显示指示标识之前，所述显示方法还包括： With the first aspect or the first embodiment of the first aspect, in the second embodiment, before the displaying the indication identifier in the associated location according to the display signal provided by the mobile terminal, the display method further includes:

接收移动终端发送的所述显示信号。 Receiving the display signal sent by the mobile terminal.

结合第一方面第二实施方式，在第三实施方式中，根据所述移动终端的特征平面，确定所述移动终端的特征平面的中轴线作为所述指示方向，包括： With reference to the second embodiment of the first aspect, in the third embodiment, determining the central axis of the feature plane of the mobile terminal as the indication direction according to the feature plane of the mobile terminal includes:

采用直线检测算法，确定所述具有中轴线的几何平面图形的二维区域方程； Using a line detection algorithm to determine a two-dimensional regional equation of the geometric plane graphic having a central axis;

根据所述二维区域方程，获取所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程； Obtaining, according to the two-dimensional region equation, a three-dimensional regional equation of the geometric plane graphic having the central axis in a standard three-dimensional coordinate system;

根据所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程，确定所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 Determining the central axis of the geometric plane graphic having the central axis in the standard three-dimensional equation according to the three-dimensional regional equation of the geometrical planar pattern having the central axis in a standard three-dimensional coordinate system The three-dimensional line equation under the coordinate system serves as the indication direction.

结合第一方面第三实施方式，在第四实施方式中，根据所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程，确定所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向，包括： In conjunction with the third embodiment of the first aspect, in the fourth embodiment, the central axis of the geometrical planar pattern having the central axis is determined according to the three-dimensional regional equation of the geometrical planar pattern having the central axis in a standard three-dimensional coordinate system The three-dimensional line equation in the standard three-dimensional coordinate system as the indication direction includes:

当所述具有中轴线的几何平面图形具有至少两条中轴线；则获取移动终端的内置电子罗盘当前的指示方向； When the geometric plane graphic having the central axis has at least two central axes; obtaining a current indication direction of the built-in electronic compass of the mobile terminal;

在所述具有中轴线的几何平面图形的至少两条中轴线中选择与所述电子罗盘当前的指示方向平行的第一中轴线，并根据所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程，确定所述第一中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 Selecting, in the at least two central axes of the geometrical planar pattern having the central axis, a first central axis parallel to a current indication direction of the electronic compass, and according to the geometrical planar pattern having the central axis in a standard three-dimensional coordinate system The lower three-dimensional region equation determines a three-dimensional line equation of the first central axis in the standard three-dimensional coordinate system as the indication direction.

结合第一方面第三或第四实施方式，在第五实施方式中，根据所述指示方向与所述呈像区域在所述当前场景图像中的范围，确定所述指示方向与所述呈像区域之间位于所述呈像区域中的关联位置，包括： In combination with the third or fourth embodiment of the first aspect, in the fifth embodiment, the indication direction and the image are determined according to the indication direction and a range of the image area in the current scene image The associated locations between the regions in the imaged region include:

根据所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程与所述呈像区域在所述标准三维坐标系中的三维区域方程进行运算，获得在所述标准三维坐标系下的交点坐标，确定所述交点坐标作为所述关联位置。 Calculating a three-dimensional linear equation in the standard three-dimensional coordinate system according to the central axis of the geometrical planar pattern having the central axis and a three-dimensional regional equation of the imaged region in the standard three-dimensional coordinate system, The coordinates of the intersection point in the standard three-dimensional coordinate system are determined as the associated position.

结合第一方面第三实施方式至第五实施方式中的任一种实施方式，在第六实施方式中，在所述根据当前场景图像，确定移动终端的指示方向之前，所述控制方法还包括： With reference to any one of the third to fifth embodiments of the first aspect, in the sixth implementation, before the determining the direction of the indication of the mobile terminal according to the current scene image, the control method further includes :

根据预设的采样检测点，获取定标参数信息； Obtaining calibration parameter information according to a preset sampling detection point;

根据所述定标参数信息确定所述标准三维坐标系； Determining the standard three-dimensional coordinate system according to the calibration parameter information;

根据所述定标参数信息和所述标准三维坐标系确定所述呈像区域在所述标准三维坐标系中的三维区域方程。 A three-dimensional region equation of the imaged region in the standard three-dimensional coordinate system is determined based on the calibration parameter information and the standard three-dimensional coordinate system.

第二方面，本发明实施例提供一种显示指示标识的处理装置，包括：获取模块，用于获取包括人手握持的移动终端和所述人手及呈像区域的当前场景图像； In a second aspect, the embodiment of the present invention provides a processing device for displaying an indication identifier, including: an obtaining module, configured to acquire a current scene image including a mobile terminal held by a human hand and the human hand and a presentation area;

第一确定模块，用于根据所述当前场景图像，确定所述移动终端指向所述呈像区域的指示方向；第二确定模块，用于根据所述指示方向与所述呈像区域在所述当前场景图像中的范围，确定所述指示方向与所述呈像区域之间位于呈像区域中的关联位置； a first determining module, configured to determine, according to the current scene image, an indication direction of the mobile terminal pointing to the image area; a second determining module, configured to determine, according to the indication direction and a range of the image area in the current scene image, an associated position between the indication direction and the image area in the image area;

控制模块，用于根据所述移动终端提供的显示信号，在所述关联位置显示指示标识。 And a control module, configured to display an indication identifier at the associated location according to the display signal provided by the mobile terminal.

结合第二方面，在第一实施方式中，所述第一确定模块，包括：人手识别单元，用于根据预设的人手识别特征，在所述当前场景图像中确定人手所在位置； With reference to the second aspect, in the first embodiment, the first determining module includes: a human hand identifying unit, configured to determine a location of a human hand in the current scene image according to a preset human hand recognition feature;

特征平面确定单元，用于基于所述人手所在位置，采用边缘检测算子确定所述人手握持的所述移动终端的特征平面，所述特征平面为所述移动终端的一个具有中轴线的几何平面图形，或者所述特征平面为所述移动终端所显示的具有中轴线的几何平面图形； a feature plane determining unit, configured to determine, according to the location of the human hand, an edge detection operator to determine a feature plane of the mobile terminal that is held by the human hand, where the feature plane is a geometry of the mobile terminal having a central axis a plane graphic, or the feature plane is a geometric plane graphic having a central axis displayed by the mobile terminal;

指示方向确定单元，用于根据所述移动终端的特征平面，确定所述移动终端的特征平面的中轴线作为所述指示方向。 And an indication direction determining unit, configured to determine, according to a feature plane of the mobile terminal, a central axis of a feature plane of the mobile terminal as the indication direction.

结合第二方面或第二方面第一实施方式，在第二实施方式中，所述处理装置还包括： With reference to the second aspect or the first embodiment of the second aspect, in the second implementation, the processing device further includes:

接收模块，用于接收移动终端发送的所述显示信号。 And a receiving module, configured to receive the display signal sent by the mobile terminal.

结合第二方面第二实施方式，在第三实施方式中，所述指示方向确定单元包括： With reference to the second embodiment of the second aspect, in the third embodiment, the indication direction determining unit includes:

第一方程确定子单元，用于采用直线检测算法，确定所述具有中轴线的几何平面图形的二维区域方程； a first equation determining subunit for determining a two-dimensional region equation of the geometric plane pattern having the central axis by using a line detection algorithm;

第二方程确定子单元，用于根据所述二维区域方程，获取所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程； a second equation determining subunit, configured to obtain a three-dimensional regional equation of the geometric plane graphic having the central axis in a standard three-dimensional coordinate system according to the two-dimensional regional equation;

指示方向确定子单元，用于根据所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程，确定所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 a direction determining subunit for determining a central axis of the geometric plane graphic having the central axis in the standard three-dimensional coordinate system according to the three-dimensional regional equation of the geometrical planar figure having the central axis in a standard three-dimensional coordinate system The three-dimensional line equation is used as the indication direction.

结合第二方面第三实施方式，在第四实施方式中，当所述具有中轴线的几何平面图形具有至少两条中轴线；则所述指示方向确定子单元具体用于获取移动终端的内置电子罗盘当前的指示方向；在所述具有中轴线的几何平面图形的至少两条中轴线中选择与所述电子罗盘当前的指示方向平行的第一中轴线，并根据所述特征平面在标准三维坐标系下的三维区域方程，确定所述第一中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 With reference to the third embodiment of the second aspect, in the fourth embodiment, when the geometric plane pattern having the central axis has at least two central axes; the indication direction determining subunit is specifically configured to acquire the built-in electronic of the mobile terminal The current direction of the compass; the central axis Selecting, from at least two central axes of the geometric plane graphic of the line, a first central axis parallel to a current indication direction of the electronic compass, and determining the first according to a three-dimensional regional equation of the characteristic plane in a standard three-dimensional coordinate system A three-dimensional line equation of a central axis in the standard three-dimensional coordinate system serves as the indication direction.

结合第二方面第三或第四实施方式，在第五实施方式中，所述第二所述标准三维坐标系下的三维直线方程与所述呈像区域在所述标准三维坐标系中的三维区域方程进行运算，获得在所述标准三维坐标系下的交点坐标，确定所述交点坐标作为所述关联位置。 With reference to the third or fourth embodiment of the second aspect, in the fifth embodiment, the three-dimensional straight line equation in the second standard three-dimensional coordinate system and the three-dimensional three-dimensional coordinate system in the standard three-dimensional coordinate system The regional equation is operated to obtain intersection coordinates in the standard three-dimensional coordinate system, and the intersection coordinates are determined as the associated position.

结合第二方面第三至第四实施方式中的任一种实施方式，在第六实施方式中，所述处理装置还包括： With reference to any one of the third to fourth embodiments of the second aspect, in the sixth embodiment, the processing device further includes:

第三确定模块，用于在根据当前场景图像，确定移动终端的指示方向之前，根据预设的采样检测点，获取定标参数信息；根据所述定标参数信息确定所述标准三维坐标系；根据所述定标参数信息和所述标准三维坐标系确定所述呈像区域在所述标准三维坐标系中的三维区域方程。 a third determining module, configured to acquire calibration parameter information according to a preset sampling detection point before determining an indication direction of the mobile terminal according to the current scene image; and determining the standard three-dimensional coordinate system according to the calibration parameter information; A three-dimensional region equation of the imaged region in the standard three-dimensional coordinate system is determined according to the calibration parameter information and the standard three-dimensional coordinate system.

第三方面，本发明实施例提供一种显示指示标识的处理***，包括显示设备、移动终端及控制设备；所述控制设备包括上述第二方面至第二方面第六实施方式中任一种所述的显示指示标识的处理装置；所述显示设备具有用于显示信息的呈像区域；所述移动终端和所述显示设备可分别通过通信接口与控制设备进行数据传输。 According to a third aspect, an embodiment of the present invention provides a processing system for displaying an indication identifier, including a display device, a mobile terminal, and a control device. The control device includes any one of the second aspect to the second aspect. The processing device for displaying the indication identifier; the display device has an image area for displaying information; and the mobile terminal and the display device can perform data transmission with the control device through the communication interface, respectively.

本发明实施例提供的指示标识控制方法、装置和***，通过采集的当前场景图像，确定移动终端的指示方向及该指示方向与信息的呈像区域之间的关联位置，实现通过移动终端在呈像区域上显示的指示标识，便于用户随时使用任意移动终端在信息的呈像区域上做出指示，即该方法通用性较好，且操作便捷，给用户的工作学习带来便利。 The indication indication control method, device and system provided by the embodiment of the present invention determine the indication direction of the mobile terminal and the associated position between the indication direction and the image presentation area of the information through the collected current scene image, and realize The indication mark displayed on the area is convenient for the user to use any mobile terminal to make an indication on the image presentation area of the information at any time, that is, the method has good versatility and convenient operation, and brings convenience to the user's work and study.

附图说明 DRAWINGS

对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。图 1为本发明显示指示标识的处理方法实施例一的流程图；图 2为本发明显示指示标识的处理方法实施例二的流程图；图 3为本发明实施例二中的应用场景示意图； BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are set forth in the description of the claims Other drawings can also be obtained from these drawings on the premise of creative labor. 1 is a flowchart of Embodiment 1 of a method for processing an indication identifier according to the present invention; FIG. 2 is a flowchart of Embodiment 2 of a method for processing an indication identifier according to the present invention; FIG. 3 is a schematic diagram of an application scenario in Embodiment 2 of the present invention;

图 4为本发明实施例二中一种移动终端的示意图； 4 is a schematic diagram of a mobile terminal according to Embodiment 2 of the present invention;

图 5为本发明实施例二中另一种移动终端的示意图； FIG. 5 is a schematic diagram of another mobile terminal according to Embodiment 2 of the present invention; FIG.

图 6为本发明显示指示标识的处理方法实施例三的流程图；图 7为本发明实施例三中应用场景示意图； 6 is a flowchart of a third embodiment of a method for processing an indication identifier according to the present invention; FIG. 7 is a schematic diagram of an application scenario according to Embodiment 3 of the present invention;

图 8为本发明显示指示标识的处理装置实施例一的结构图；图 9为本发明显示指示标识的处理装置实施例二的结构图；图 10为本发明显示指示标识的处理***实施例一的示意图。 FIG. 8 is a structural diagram of a first embodiment of a processing apparatus for displaying an indication identifier according to the present invention; FIG. 9 is a structural diagram of a second embodiment of a processing apparatus for displaying an indication identifier according to the present invention; Schematic diagram.

具体实施方式为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。 The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. The embodiments are a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without the creative work are all within the scope of the present invention.

图 1 为本发明显示指示标识的处理方法实施例一的流程图。本实施例的方法可应用于类似投影显示的情况中。在投影显示时一般设置有投影仪和投影幕布，该投影幕布上显示信息的区域作为呈像区域，而投影仪一般与具备控制功能的控制设备相连，例如所述控制设备常见的是计算机。还需设置一现场图像捕捉设备和移动终端。现场图像捕捉设备例如为摄像机，可以是独立的摄像机，将其采集的图像提供给控制设备，或者也可以是该控制设备集成了图像采集功能。移动终端典型为手机，用户使用该手机做出指示动作，随后控制设备可根据用户使用该手机做出指示动作，在呈像区域中进行指示标识的显示。 FIG. 1 is a flowchart of Embodiment 1 of a method for processing an indication indicator according to the present invention. The method of this embodiment can be applied to a case similar to a projection display. In the projection display, a projector and a projection screen are generally provided. The area on which the information is displayed on the projection screen serves as an image forming area, and the projector is generally connected to a control device having a control function. For example, the control device is usually a computer. A live image capture device and mobile terminal are also required. The live image capturing device is, for example, a camera, which may be an independent camera, and provides images acquired by the camera to the control device, or it may be integrated with the image capturing function. The mobile terminal is typically a mobile phone, and the user uses the mobile phone to make an indication action, and then the control device can perform an indication action according to the user using the mobile phone, and display the indication mark in the image area.

本实施例的方法具体可以由显示指示标识的处理装置来实现，该处理装置可以采用硬件和 /或软件方式来实现，优选设置在上述控制设备内来实现指示标识的显示。如图 1所示，该方法包括： The method of the embodiment may be implemented by a processing device that displays an indication identifier, and the processing device may be implemented by using hardware and/or software, and is preferably disposed in the control device to implement display of the indication identifier. As shown in Figure 1, the method includes:

S 101、获取包括人手握持的移动终端和该人手及呈像区域的当前场景图像。以上述投影显示的情况为例，用户的人手握持移动终端并做出指示动作（例如，用户作为讲解人员需要向参会人员讲解投影幕布上当前显示的机器的一个部件，则会用手握持移动终端指向该部件，即做出指示动作，以期望可在该部件处显示指示标识），此时现场图像捕捉设备受显示指示标识的处理装置的控制，获取到的当前场景图像中包括人手握持的移动终端和该人手，及作为呈像区域的投影幕布。 S101. Acquire a current scene image including the mobile terminal held by the human hand and the human hand and the image area. Taking the case of the above projection display as an example, the user's human hand holds the mobile terminal and makes an instruction action (for example, the user as a commentator needs to explain to the participant a part of the machine currently displayed on the projection screen, and then the hand is gripped. Holding the mobile terminal to point to the component, that is, making an indication action, so that it is expected to display the indication mark at the component), at this time, the live image capturing device is controlled by the processing device that displays the indication flag, and the acquired current scene image includes the human hand. The mobile terminal and the hand held, and the projection screen as the image area.

S 102、根据当前场景图像，确定移动终端指向呈像区域的指示方向；现场图像捕捉设备将当前场景图像提供给显示指示标识的处理装置，该处理装置根据该采集的当前场景图像进行图像处理分析，确定移动终端指向呈像区域的指示方向；在确定上述指示方向时，处理装置可通过图像分析获知移动终端当前所在的位置，并进行特征分析，具体如采用预设的坐标系确定移动终端当前所在的位置，并在该坐标系下进运算处理，确定出表征移动终端的指示方向。举例来说，移动终端为手机，用于显示信息的呈像区域为投影幕布，该投影幕布当前显示了一副机械图片，而讲解人员与该投影幕布之间存在一定距离，讲解人员在对该图进行讲解的过程中，需要配合讲解内容对图中各个零件做出指示，讲解人员经过目测可使用手握持手机并指向当前需讲解的零件所在的方向，则显示指示标识的处理装置根据采集的当前场景图像进行图像处理分析可确定手机当前的指示方向，即确定用户当前通过手机所指示的方向。 S102. Determine, according to the current scene image, an indication direction of the mobile terminal pointing to the imaged area; the live image capturing device provides the current scene image to the processing device that displays the indication identifier, and the processing device performs image processing analysis according to the collected current scene image. Determining the direction in which the mobile terminal points to the imaged area; when determining the direction of the indication, the processing device can obtain the current location of the mobile terminal through image analysis, and perform feature analysis, such as determining the current current state of the mobile terminal by using a preset coordinate system. The location, and the operation processing under the coordinate system determines the direction of the indication characterizing the mobile terminal. For example, the mobile terminal is a mobile phone, and the image forming area for displaying information is a projection screen, and the projection screen currently displays a mechanical picture, and there is a certain distance between the explanation person and the projection screen, and the explanation person is in the In the process of explaining the figure, it is necessary to make an instruction to each part in the figure in conjunction with the explanation content. The person who has been visually observed can use the hand to hold the mobile phone and point to the direction of the part to be explained, and then display the processing device of the indication mark according to the collection. The image processing analysis of the current scene image can determine the current direction of the mobile phone, that is, determine the direction that the user currently indicates through the mobile phone.

S 103、根据指示方向与呈像区域在当前场景图像中的范围，确定指示方向与呈像区域之间位于呈像区域中的关联位置； S103. Determine, according to the indication direction and the range of the image area in the current scene image, the associated position between the indication direction and the image area in the image area;

上述用于显示信息的呈像区域可以为各种可用于显示信息的区域，例如在配置有投影仪及投影幕布的会议室中，上述呈像区域为投影幕布，投影幕布所显示的内容是对计算机显示屏进行面向投影幕布的投影获得的；若在配置有电子屏幕的播放室中，上述呈像区域为该电子屏幕，如 LED ( Light Emitting Diode, 发光二极管）显示屏。可通过预先设定、人工调节或自动识别来使得处理装置能获知呈像区域所在范围，该范围可通过坐标限定。 The image forming area for displaying information may be various areas for displaying information, for example, in a conference room equipped with a projector and a projection screen, the image forming area is a projection screen, and the content displayed on the projection screen is The computer display screen is obtained by projecting projection onto the projection screen; if in the playing room configured with the electronic screen, the image forming area is the electronic screen, such as an LED (Light Emitting Diode) display. The processing device can be made aware of the extent of the imaged area by pre-setting, manual adjustment or automatic identification, which can be defined by coordinates.

上述呈像区域与移动终端在空间位置上具有关联关系，如移动终端与呈像区域之间的距离、角度等关联关系，则根据该关联关系可确定移动终端与呈像区域之间的关联位置；例如，移动终端通常为规则的几何形状，则上述处理装置可以将移动终端的一个侧面（如手机显示屏所在的手机侧面）的中轴线所在的方位作为上述指示方向，该中轴线必然和呈像区域所在的平面存在交点，则该交点即为关联位置。 The image forming area has an associated relationship with the mobile terminal in a spatial position, such as a distance, an angle, and the like between the mobile terminal and the imaged area, and the associated position between the mobile terminal and the imaged area can be determined according to the associated relationship. For example, if the mobile terminal is usually of a regular geometry, the processing device can have one side of the mobile terminal (such as the mobile phone display screen) The orientation of the central axis of the side of the mobile phone is taken as the above-mentioned direction, and the central axis necessarily intersects with the plane where the imaged area is located, and the intersection is the associated position.

S 104、根据移动终端提供的显示信号，在关联位置显示指示标识。指示标识是处理装置根据移动终端提供的显示信号而形成的，例如，人手握持移动终端指向呈像区域，以期望在呈像区域上的某个位置显示指示标识，则处理装置结合移动终端与呈像区域的相对位置，在呈像区域上显示指示标识。例如，信息的呈像区域为投影幕布，讲解人员当前希望在投影幕布的中心点处出现指示标识，如一个红色圓点，则讲解人员通过人手握持的移动终端向投影幕布的中心点的位置做出指示动作，则处理装置通过 S 101〜S 103确定出准确的关联位置后 (由于人眼观察误差，该关联位置可能在投影幕布上与投影幕布的中心点吻合，或者和投影幕布的中心点之间存在一定的距离差），便在该关联位置处显示一个红色圓点，即实现指示标识，若该指示标识与讲解人员期望的中心点之间有距离差，则讲解人员通过人手握持移动终端进行移动调整，从而处理装置再次执行 S 101〜S 104 , 在中心点显示指示标识。 S104. Display an indication identifier at the associated location according to the display signal provided by the mobile terminal. The indication identifier is formed by the processing device according to the display signal provided by the mobile terminal. For example, if the human hand holds the mobile terminal and points to the image-presenting area to display the indication mark at a certain position on the image-forming area, the processing device combines the mobile terminal with the The relative position of the imaged area, and the indicator is displayed on the imaged area. For example, the image presentation area of the information is a projection screen, and the explanation person currently wishes to display an indication mark at a center point of the projection screen, such as a red dot, to explain the position of the mobile terminal held by the human hand to the center point of the projection screen. After the indication action is made, the processing device determines the accurate associated position through S101~S103 (due to the human eye observation error, the associated position may coincide with the center point of the projection screen on the projection screen, or with the center of the projection screen There is a certain distance difference between the points, and a red dot is displayed at the associated position, that is, the indication mark is realized. If there is a distance difference between the indication mark and the center point desired by the explanation person, the explanation person is held by the human hand. The mobile terminal performs movement adjustment, so that the processing device executes S 101 to S 104 again, and displays an indication flag at the center point.

在本实施例中提供的显示指示标识的处理方法，通过采集的当前场景图像并进行图像分析，确定移动终端的指示方向及该指示方向与信息的呈像区域之间的关联位置，实现通过移动终端在呈像区域上显示指示标识，便于用户随时使用任意移动终端在信息的呈像区域上做出指示，即该方法通用性较好，且操作便捷，给用户的工作学习带来便利。 The processing method for displaying the indication identifier provided in the embodiment, by using the collected current scene image and performing image analysis, determining the indication direction of the mobile terminal and the associated position between the indication direction and the image presentation area of the information, and implementing the movement The terminal displays the indication mark on the image area, so that the user can use any mobile terminal to make an indication on the image presentation area of the information at any time, that is, the method has good versatility and convenient operation, and brings convenience to the user's work and study.

图 2 为本发明显示指示标识的处理方法实施例二的流程图。如图 2 所示，本实施例以上述实施例为基础，进一步优化了进行图像处理的过程，该方法包括： FIG. 2 is a flowchart of Embodiment 2 of a method for processing an indication identifier according to the present invention. As shown in FIG. 2, the embodiment further optimizes the process of performing image processing based on the above embodiment, and the method includes:

S201、确定标准三维坐标系。 S201. Determine a standard three-dimensional coordinate system.

图 3为本发明实施例二中的应用场景示意图。如图 3所示，用于显示信息的呈像区域为一个电子显示屏 20 , 现场图像捕捉设备 50 为立体视觉装置，该立体视觉装置采用双摄像机捕捉现场图像。而显示指示标识的处理装置可控制电子显示屏的显示，也可接收立体视觉装置获取的图像并进行图像处理。由于处理装置是根据现场图像捕捉设备采集的场景图像对移动终端 30进行方位分析，因此需确定在现场图像捕捉设备的视角下的标准三维坐标系 - 及处理图像的相关参数，具体的确定方法如下： S la、根据预设的采样检测点，获取定标参数信息； FIG. 3 is a schematic diagram of an application scenario in Embodiment 2 of the present invention. As shown in FIG. 3, the image area for displaying information is an electronic display screen 20, and the live image capturing device 50 is a stereoscopic vision device that captures a live image using a dual camera. The processing device displaying the indication flag can control the display of the electronic display screen, and can also receive the image acquired by the stereo vision device and perform image processing. Since the processing device performs the orientation analysis on the mobile terminal 30 according to the scene image collected by the live image capturing device, it is necessary to determine a standard three-dimensional coordinate system at the viewing angle of the live image capturing device - and the related parameters of the processed image, and the specific determination method is as follows : S la, obtaining calibration parameter information according to a preset sampling detection point;

在实际应用时可使用张正友定标方法获取定标参数信息；具体来说，在场景中预设一个采样检测点，如在场景中放置一个已知固定尺寸的黑白棋盘，并使现场图像捕捉设备捕捉包括该采样检测点的场景图像；通过对现场图像捕捉设备所捕捉的图像进行图像分析处理获取定标参数信息，定标参数信息包括内参数、结构参数；其中内参数包括现场图像捕捉设备采用的焦距、成像的畸变参数等，结构参数包括两个摄像机之间的三维平移矩阵和三维旋 In actual application, Zhang Zhengyou can be used to obtain calibration parameter information; specifically, a sampling detection point is preset in the scene, such as placing a black and white chessboard of known fixed size in the scene, and making the live image capturing device Capturing a scene image including the sampling detection point; obtaining image calibration parameter information by performing image analysis processing on the image captured by the live image capturing device, the calibration parameter information includes an internal parameter and a structural parameter; wherein the internal parameter includes a live image capturing device The focal length, the distortion parameters of the imaging, etc., the structural parameters include the three-dimensional translation matrix and three-dimensional rotation between the two cameras.

矩阵，该平移矩阵为 jMatrix, the translation matrix is j

其中平移矩阵中的元素 ^、及旋转矩阵中元素至的值是将上述捕捉的图像经过图像分析处理后获得的。

The element ^ in the translation matrix and the value of the element in the rotation matrix are obtained by subjecting the captured image to image analysis processing.

S ib, 根据所述定标参数信息确定所述标准三维坐标系； S ib, determining the standard three-dimensional coordinate system according to the calibration parameter information;

具体可使用上述结构参数确定标准三维坐标系。 Specifically, the standard three-dimensional coordinate system can be determined using the above structural parameters.

S ic,根据定标参数信息和所述标准三维坐标系确定呈像区域在标准三维坐标系中的三维区域方程。 S ic determines a three-dimensional region equation of the image region in the standard three-dimensional coordinate system according to the calibration parameter information and the standard three-dimensional coordinate system.

上述呈像区域，如电子显示屏在应用场景中也是位置固定不变的，因此，在确定标准三维坐标系后，便可确定该呈像区域在标准三维坐标系中的三维区域方程。具体的，现场图像捕捉设备拍摄电子显示屏，则处理装置根据上述内参数对拍摄到的图像进行畸变校正，随后对校正后的图像进行图像处理，获取电子显示屏在标准三维坐标系下的三维区域方程。 The above-mentioned image area, such as an electronic display screen, is also fixed in the application scene. Therefore, after determining the standard three-dimensional coordinate system, the three-dimensional area equation of the image area in the standard three-dimensional coordinate system can be determined. Specifically, the on-site image capturing device takes an electronic display screen, and the processing device performs distortion correction on the captured image according to the internal parameter, and then performs image processing on the corrected image to obtain a three-dimensional image of the electronic display screen in a standard three-dimensional coordinate system. Regional equation.

需要说明的是，上述 S201是在应用场景中第一次实施本发明的显示指示标识的处理方法时才会执行，当标准三维坐标系即相关参数确定后，再次实施显示指示标识的处理方法时，直接由 S202开始执行。 It should be noted that the foregoing S201 is performed when the processing method for displaying the indication indicator of the present invention is implemented for the first time in the application scenario. When the standard three-dimensional coordinate system, that is, the relevant parameter is determined, the processing method for displaying the indication identifier is implemented again. , directly from S202 to start execution.

S202、获取包括人手握持的移动终端和人手，及呈像区域的当前场景图像。 S202. Acquire a mobile terminal and a human hand that are held by a human hand, and a current scene image of the imaged area.

S203、根据当前场景图像，确定人手握持的移动终端的特征平面。具体的，显示指示标识的处理装置在采集的当前场景图像中，根据预设的人手识别特征，如人的肤色特征及手部的轮廓识别人手在图像中所在的位置；或者，现场图像捕捉设备集成红外热成像仪的功能，感知了人体温度在图像中人手部位成像，由此确定人手在图像中所在的位置，进而识别人手握持的移动终端。 S203. Determine, according to the current scene image, a feature plane of the mobile terminal held by the human hand. Specifically, the processing device that displays the indication identifier is in the collected current scene image, according to Pre-set human recognition features, such as human skin color features and hand contours, identify where the human hand is located in the image; or, the live image capture device integrates the functions of an infrared thermal imager to sense human body temperature imaging in the image of the human hand , thereby determining the location of the human hand in the image, and thereby identifying the mobile terminal held by the human hand.

基于人手所在位置，可采用边缘检测算子确定人手握持的移动终端的特征平面，具体来说，图 4为本发明实施例二中一种移动终端的示意图。如图 4所示，移动终端为常见的手机，而该手机所占用的空间区域与人手所占用的空间区域有重叠，则可以将人手作为检测基准点，在一定的检测范围内检测手机；例如确定人手所在位置后，在人手的手指区域附近发现图像的阶跃变化，即采用边缘检测算子（如 Canny边缘算子、梯度算子、马尔算子等）检测出手机的轮廓；根据该手机的轮廓，进一步的确定手机的指示方向。 Based on the position of the human hand, the edge detection operator can be used to determine the feature plane of the mobile terminal held by the human hand. Specifically, FIG. 4 is a schematic diagram of a mobile terminal according to the second embodiment of the present invention. As shown in FIG. 4, the mobile terminal is a common mobile phone, and the space area occupied by the mobile phone overlaps with the space area occupied by the human hand, and the human hand can be used as a detection reference point to detect the mobile phone within a certain detection range; for example; After determining the position of the human hand, a step change of the image is found near the finger area of the human hand, that is, an edge detection operator (such as a Canny edge operator, a gradient operator, a Markov operator, etc.) is used to detect the outline of the mobile phone; The contours further determine the direction of the phone.

通常手机为矩形立方体结构，且厚度相对于手机的前侧面或后侧面的长度、宽度来说，尺寸较小，则在图像处理时可将手机看做一个矩形平板，则上述检测出手机的轮廓即为一个近似矩形的图形，也就是说，对手机而言，上述移动终端的特征平面为移动终端的一个具有中轴线的几何平面图形，即手机的一个矩形侧面。 Usually, the mobile phone has a rectangular cube structure, and the thickness is smaller than the length and width of the front side or the rear side of the mobile phone, so that the mobile phone can be regarded as a rectangular flat plate during image processing, and the outline of the mobile phone is detected as described above. That is, an approximate rectangular shape, that is, for the mobile phone, the feature plane of the mobile terminal is a geometric plane graphic of the mobile terminal having a central axis, that is, a rectangular side of the mobile phone.

更灵活的，图 5 为本发明实施例二中另一种移动终端的示意图。如图 5 所示，上述特征平面为移动终端所显示的具有中轴线的几何平面图形；举例来说，讲解人员希望使用手机对会议室内的电子显示屏上的信息做出指示，则可开启手机内一个应用程序，该应用程序可在手机的显示屏上显示一个具有中轴线的几何平面图形，例如一个矩形，则采用边缘检测算子可检测出手机显示屏上显示的矩形图像的轮廓。 More flexible, FIG. 5 is a schematic diagram of another mobile terminal according to Embodiment 2 of the present invention. As shown in FIG. 5, the feature plane is a geometric plane graphic with a central axis displayed by the mobile terminal; for example, if the explanation person wants to use the mobile phone to indicate the information on the electronic display in the conference room, the mobile phone can be turned on. An application that displays a geometric plane graphic with a central axis on the display of the phone, such as a rectangle. The edge detection operator detects the outline of the rectangular image displayed on the phone display.

综上，可以灵活地选择移动终端上具有中轴线的几何图形看做移动终端的特征平面。 In summary, the geometric shape with the central axis on the mobile terminal can be flexibly selected as the feature plane of the mobile terminal.

S204、根据移动终端的特征平面，确定移动终端的特征平面的中轴线作为指示方向。 S204. Determine a center axis of the feature plane of the mobile terminal as the indication direction according to the feature plane of the mobile terminal.

以特征平面为如图 4所示的移动终端的一个具有中轴线的几何平面图形为例，采用直线检测算法，检测出上述手机的轮廓的直线方程 Ll、 L2和 L3 ; 上述直线检测算法可以为随机霍夫（Hough ) 变换及其他基于 hough 变换进行改进得到的直线检测算法；具体以 hough 变换为例，其原理为利用图像空间和 Hough参数空间的点 -线对偶性，把图像空间中的检测问题转换到参数空间，以对图像中的直线进行检测。在此需要说明的是， S202中采集的当前场景图像为一个平面图像，因此此时的 Ll、 L2和 L3是在这个平面图像中的二维直线方程， Ll、 L2和 L3共同表示了该特征平面的二维区域方程；随后在每一条二维直线上选择多个特征点进行立体特征匹配，获取每个特征点在上述标准三维坐标系下的三维坐标，最后根据每一条二维直线上所选择的多个特征点的三维坐标进行最小二乘法拟合运算，获取特征平面在标准三维坐标系下的三维区域方程，即根据二维区域方程，获取特征平面在标准三维坐标系下的三维区域方程。具体来说，本实施例中现场图像捕捉设备采用双摄像机，则场景中同一个点在两个摄像机各自拍摄的两张图像中存在差异；因此在上述每个二维直线上选取 5个特征点（也可根据精度需求设置特征点的数量，具体数量本实施例不作限制），该 5 个特征点的每一个特征点在两个摄像机各自拍摄的两张图像中也是存在差异，采用基于灰度的区域匹配方法分别将两张图像中的 5个特征点——对应，结合上述结构参数中的三维旋转矩阵和三维平移矩阵确定出每条直线上的特征点在标准三维坐标系下的三维坐标，确定该三维坐标（ y , z ) 的具体计算公式如下： Taking the feature plane as a geometric plane figure with a central axis of the mobile terminal as shown in FIG. 4, a straight line detection algorithm is used to detect the line equations L1, L2 and L3 of the contour of the mobile phone; the line detection algorithm may be The Hough transform and other hough transform-based line detection algorithms are improved. The hough transform is taken as an example. The principle is to use the point-line duality of image space and Hough parameter space in the image space. The detection problem is converted to the parameter space to detect the line in the image. It should be noted that the current scene image acquired in S202 is a plane image, so Ll, L2 and L3 at this time are two-dimensional line equations in this plane image, and L1, L2 and L3 jointly represent the feature. The two-dimensional regional equation of the plane; then select multiple feature points on each two-dimensional line for stereo feature matching, obtain the three-dimensional coordinates of each feature point in the above-mentioned standard three-dimensional coordinate system, and finally according to each two-dimensional line The three-dimensional coordinates of selected feature points are subjected to least squares fitting operation to obtain the three-dimensional regional equation of the feature plane in the standard three-dimensional coordinate system, that is, the three-dimensional region of the feature plane in the standard three-dimensional coordinate system is obtained according to the two-dimensional regional equation. equation. Specifically, in the embodiment, the live image capturing device adopts a dual camera, and the same point in the scene has a difference between the two images captured by the two cameras; therefore, five feature points are selected on each of the two-dimensional lines. (The number of feature points can also be set according to the accuracy requirement, and the specific number is not limited in this embodiment.) Each feature point of the five feature points is also different in the two images captured by the two cameras, and is based on gray scale. The region matching method respectively determines the three feature points in the two images—corresponding to the three-dimensional rotation matrix and the three-dimensional translation matrix in the above structural parameters to determine the three-dimensional coordinates of the feature points on each straight line in the standard three-dimensional coordinate system. , determine the specific calculation formula of the three-dimensional coordinates ( y , z ) as follows:

X二 zXJ f , X two zXJ f ,

或者， z₌ 助 _y-

Or, z ₌ help _y -

^Ζ~Υ ^ +r_sY_{l +}f_lr₉)-f_r(r₄X_l +r₅ ）其中，为双目摄像机中左摄像机的有效焦距，为双目摄像机中右摄像机的有效焦距；为特征点在右摄像机中的二维坐标；（A，）为特征点在左摄像机中的二维坐标；举例来说，在上述二维直线方程 L1所表示的二维直线上选择 5个特征点 A、 B、 C、 D、 E为例，特征点 A、 B、 C、 D、 E在一个摄像机所拍摄的图像中位置分别为 Al、 Bl、 Cl、 Dl、 El ; 而在另一个摄像机所拍摄的图像中位置分别为 A2、 B2、 C2、 D2、 E2; 采用基于灰度的区域匹配方法将 Al和 A2, B1和 B2 , C1和 C2, D1和 D2, E1和 E2——对应，并利用上述确定三维坐标（ , z ) 的计算公式分别确定出 A、 B、 C、 D、 E 在三维坐标系下的三维坐标。在确定出每一条二维直线上的各特征点在标准三维坐标系下的三维坐标后，根据同一条二维直线上的多个特征点在三维坐标系下的三维坐标，进行最小二乘法拟合运算，得到该条直线在标准三维坐标系下的三维直线方程；即获得上述 Ll、 L2和 L3在标准三维坐标系下的三维直线方程 L1'、 ²'和 ³' , 从而三维直线方程 1'、 ²'和 ³'可共同表示上述具有中轴线的几何图形在标准三维坐标系下的三维区域方程。 ^Ζ ~Υ ^ +r _s Y _{l +} f _l r ₉ )-f _r (r ₄ X _l +r ₅ ) where is the effective focal length of the left camera in the binocular camera, which is the effective focal length of the right camera in the binocular camera ; is the two-dimensional coordinate of the feature point in the right camera; (A, ) is the two-dimensional coordinate of the feature point in the left camera; for example, selecting five on the two-dimensional line represented by the above two-dimensional line equation L1 Characteristic points A, B, C, D, E are examples, and feature points A, B, The positions of C, D, and E in the images captured by one camera are Al, Bl, Cl, Dl, and El; and the positions of the images captured by the other camera are A2, B2, C2, D2, and E2; The gray-based region matching method associates Al and A2, B1 and B2, C1 and C2, D1 and D2, E1 and E2, and determines the A and B by using the above-mentioned formula for determining the three-dimensional coordinates (, z). , C, D, E 3D coordinates in a three-dimensional coordinate system. After determining the three-dimensional coordinates of each feature point on each two-dimensional line in the standard three-dimensional coordinate system, the least squares method is performed according to the three-dimensional coordinates of the plurality of feature points on the same two-dimensional line in the three-dimensional coordinate system. Combining the operation, the three-dimensional linear equation of the straight line in the standard three-dimensional coordinate system is obtained; that is, the three-dimensional linear equations L1', ² ' and ³ ' of the above L1, L2 and L3 in the standard three-dimensional coordinate system are obtained, so that the three-dimensional linear equation 1 ', ² ' and ³ ' can collectively represent the three-dimensional regional equation of the above-described geometry with a central axis in a standard three-dimensional coordinate system.

根据上述具有中轴线的几何图形在标准三维坐标系下的三维区域方程，即三维直线方程 1'、 ²'和 ³' , 确定了该具有中轴线的几何图形的中轴线在标准三维坐标系下的三维直线方程^ 乍为上述移动终端的指示方向。 According to the three-dimensional regional equations of the above-mentioned geometrical figure with the central axis in the standard three-dimensional coordinate system, that is, the three-dimensional linear equations 1', ² ' and ³ ', the central axis of the geometrical axis with the central axis is determined in the standard three-dimensional coordinate system. The three-dimensional line equation ^ 乍 is the direction indicated by the above mobile terminal.

需要说明的是，在几何学上，对于等腰三角形等图形仅有一条中轴线，而对于矩形来说，却有两条中轴线，因此，当上述具有中轴线的几何图形具有至少两条中轴线时，上述处理装置还需获取移动终端的内置电子罗盘当前的指示方向，从而根据电子罗盘当前的指示方向，在移动终端特征平面的至少两条中轴线中选择与电子罗盘当前的指示方向平行的第一中轴线，并根据所述特征平面在标准三维坐标系下的三维区域方程，确定该第一中轴线在标准三维坐标系下的三维直线方程作为指示方向。 It should be noted that, geometrically, there is only one central axis for a graphic such as an isosceles triangle, and two central axes for a rectangular shape. Therefore, when the above-mentioned geometric shape having a central axis has at least two In the case of the axis, the processing device further needs to obtain the current indication direction of the built-in electronic compass of the mobile terminal, so as to select the direction parallel to the current indication direction of the electronic compass in at least two central axes of the feature plane of the mobile terminal according to the current indication direction of the electronic compass. The first central axis, and determining the three-dimensional linear equation of the first central axis in the standard three-dimensional coordinate system as the indication direction according to the three-dimensional regional equation of the feature plane in the standard three-dimensional coordinate system.

基于图 5 所示的特征平面确定该特征平面的中轴线在标准三维坐标系下的三维直线方程作为指示方向的方法和上述过程一致，在此不再贅述。 The three-dimensional linear equation of the central axis of the feature plane in the standard three-dimensional coordinate system is determined based on the feature plane shown in FIG. 5 as a method for indicating the direction, which is the same as the above process, and will not be described herein.

S205、根据指示方向与信息的呈像区域所在的范围，确定指示方向与呈像区域之间位于呈像区域中的关联位置。 S205. Determine, according to a range in which the indication direction and the image forming area of the information are located, an associated position between the indication direction and the image area in the image area.

由于在 S201中已知呈像区域在标准三维坐标系中的三维区域方程；则根据特征平面的中轴线在标准三维坐标系下的三维直线方程（《或 Ln ) 与呈像区域在标准三维坐标系中的三维区域方程进行运算，获得在所述标准三维坐标系下的交点坐标，确定该交点坐标作为关联位置，即联立特征平面的中轴线在标准三维坐标系下的三维直线方程和呈像区域在标准三维坐标系中的三维区域方程进行求解运算，得到的解的值为交点的三维坐标。 Since the three-dimensional region equation of the image region in the standard three-dimensional coordinate system is known in S201; the three-dimensional line equation ("Ln" and the image region in the standard three-dimensional coordinates according to the central axis of the feature plane in the standard three-dimensional coordinate system) The three-dimensional regional equation in the system is operated to obtain the coordinates of the intersection point in the standard three-dimensional coordinate system, and the coordinate of the intersection point is determined as the associated position, that is, The three-dimensional line equation of the central axis of the simultaneous feature plane in the standard three-dimensional coordinate system and the three-dimensional regional equation of the imaged area in the standard three-dimensional coordinate system are solved, and the obtained solution is the three-dimensional coordinates of the intersection point.

S206、判断是否接收到移动终端发送的显示信号；若是，则执行 S207 , 若否，则停止。 S206. Determine whether a display signal sent by the mobile terminal is received; if yes, execute S207, and if not, stop.

当场景中移动终端较多（如参会人员人手握持一部手机），上述装置可对应每一个移动终端确定移动终端的指示方向与呈像区域之间的关联位置，但若对应每一个移动终端都显示该移动终端的指示标识，则在呈像区域上易出现指示混乱，因此处理装置控制指示标识在关联位置处显示之前，需接收移动终端发送的显示信号；当判断得知接收到移动终端发送的显示信号后，则在该移动终端所对应的关联位置处显示指示标识；若未接收到上述显示消息，则不显示指示标识；但可以理解的，处理装置会保持追踪该移动终端的指示方向，并实时地确定移动终端当前的指示方向与呈像区域之间的关联位置，以便在接收到该移动终端发送的显示信号后，可立即在关联位置显示指示标识。 When there are many mobile terminals in the scene (for example, the participant holds a mobile phone by hand), the above device may determine the associated position between the pointing direction of the mobile terminal and the imaged area corresponding to each mobile terminal, but if corresponding to each mobile terminal If the indication of the mobile terminal is displayed, the indication is apt to appear on the imaged area. Therefore, before the processing device control indication indicator is displayed at the associated location, the display signal sent by the mobile terminal needs to be received; when it is determined that the mobile terminal is received After the display signal is sent, the indication identifier is displayed at the associated location corresponding to the mobile terminal; if the display message is not received, the indication identifier is not displayed; but it can be understood that the processing device keeps tracking the indication of the mobile terminal Direction, and determining the associated position between the current pointing direction of the mobile terminal and the imaged area in real time, so that after receiving the display signal sent by the mobile terminal, the indication mark can be immediately displayed at the associated position.

以会议场景为例，参会人员有 5人，每人携带一部手机，而这 5人中只有一人作为信息讲解人员，则仅需显示基于该讲解人员的手机的指示标识，因此，讲解人员开启手机上装载的适配于本实施例处理方法的应用程序，手机显示屏上出现一个 "显示" 按钮，点击该按钮，触发手机向处理装置发送了信号波，该信号波可理解为手机向处理装置发送的显示信号，灵活的，该信号波也可通过按下手机上的物理按键触发发送；当处理装置控制指示标识显示，即在讲解人员的手机的指示方向与投影幕布之间的关联位置处显示一个红色圓点（红色圓点仅用于说明本实施例的方法，具体的显示形状或颜色，本实施例不做限制），若讲解人员发现该红色圓点不在所希望的位置处，则持续按住 "显示" 按钮并移动手机，而处理装置实时获取当前场景中手机的指示方向与呈像区域之间的关联位置，使红色圓点在当前的关联位置处显示。 Taking the conference scene as an example, there are 5 participants, each carrying a mobile phone, and only one of the 5 people is an information explanation person, and only needs to display the instruction mark based on the mobile phone of the explanation person. Therefore, the explanation personnel Turning on the application loaded on the mobile phone and adapting to the processing method of the embodiment, a "display" button appears on the display of the mobile phone, and clicking the button triggers the mobile phone to send a signal wave to the processing device, which can be understood as a mobile phone The display signal sent by the processing device is flexible, and the signal wave can also be triggered by pressing a physical button on the mobile phone; when the processing device controls the indication indicator display, that is, the relationship between the direction of the instruction of the mobile phone of the person and the projection screen is explained. A red dot is displayed at the position (the red dot is only used to describe the method of the embodiment, and the specific display shape or color, which is not limited in this embodiment), if the explanation person finds that the red dot is not at the desired position. , then keep pressing the "Show" button and move the phone, and the processing device gets the current scene in real time. Indicates the direction of the mobile phone as a region between a position associated with the image, the red dot at the current display position associated.

也可以理解的是，若会议场景中仅有讲解人员持有手机，则处理装置在检测到现场仅有一个手机后（例如处理装置通过一定的通信信号，在会议场景中发出手机可接收的探测波，则处理装置可根据接收的由手机发送的反馈消息确定会议场景中手机的数量；或者，移动终端可通过开启移动终端上装载的应用程序，以控制移动终端上的指示灯进行闪烁，向处理装置表明其该移动终端会做出指向呈像区域的指示动作，可以理解的，其他未开启应用程序以使移动终端上的指示灯进行闪烁的移动终端，则处理装置不会实时地追踪此移动终端的指示方向。 ) , 便默认在手机的指示方向与呈像区域之间的关联位置处显示指示标识，不需执行 S206 中判断是否接收到移动终端发送的指示标识的显示消息，即执行 S205之后可直接执行 S207。 It can also be understood that if only the explanation person holds the mobile phone in the conference scene, the processing device detects that there is only one mobile phone in the scene (for example, the processing device sends a mobile phone receivable detection in the conference scene through a certain communication signal). The processing device may determine the number of mobile phones in the conference scene according to the received feedback message sent by the mobile phone; or the mobile terminal may control the indicator light on the mobile terminal to blink by opening an application loaded on the mobile terminal. Demonstrating to the processing device that the mobile terminal will make an indication action pointing to the image area. It can be understood that other mobile terminals that do not open the application to flash the indicator light on the mobile terminal, the processing device does not track in real time. The direction of indication of this mobile terminal. By default, the indication flag is displayed at the associated position between the pointing direction of the mobile phone and the imaged area, and the display message of the indication flag sent by the mobile terminal is not required to be executed in S206, that is, S207 can be directly executed after executing S205. .

S207、根据移动终端提供的显示信号，在关联位置显示指示标识。根据 S205中确定的交点的三维坐标，控制电子显示屏在对应的位置处显示指示标识。 S207. Display an indication identifier at the associated location according to the display signal provided by the mobile terminal. According to the three-dimensional coordinates of the intersection determined in S205, the electronic display screen is controlled to display the indication mark at the corresponding position.

在本实施例中提供的指示标识控制方法，通过采集的当前场景图像，确定移动终端的指示方向及该指示方向与信息的呈像区域之间的关联位置，实现通过移动终端对在呈像区域上显示指示标识，便于用户随时使用任意移动终端在呈像区域上做出指示，即该方法通用性较好，且操作便捷，给用户的工作学习带来便利。 The indication identifier control method provided in the embodiment determines the indication direction of the mobile terminal and the associated position between the indication direction and the image presentation area of the information through the acquired current scene image, and realizes the pairing in the image presentation area by the mobile terminal. The indicator is displayed on the display, so that the user can use any mobile terminal to make an indication on the image area at any time, that is, the method has good versatility and convenient operation, and brings convenience to the user's work and study.

图 6 为本发明显示指示标识的处理方法实施例三的流程图。如图 6 所示，在上述实施例一的基础上，本实施例还可将图像处理的过程进行另一种优化，该方法包括： FIG. 6 is a flowchart of Embodiment 3 of a method for processing an indication indicator according to the present invention. As shown in FIG. 6, on the basis of the foregoing embodiment 1, the embodiment may further optimize the process of image processing, and the method includes:

S301、确定标准三维坐标系。 S301. Determine a standard three-dimensional coordinate system.

具体的，图 7为本发明实施例三中应用场景示意图。如图 7所示，信息的显示具体通过一个投影幕布 21和一台计算机实现，其中投影幕布上的内容是对一台计算机的计算机屏幕 22进行投影获得的，现场图像捕捉设备 50为立体视觉装置，其中，显示指示标识的处理装置负责对立体视觉装置获取的图像进行处理，而投影幕布的显示上内容的显示由计算机控制，但显示指示标识的处理装置可控制该计算机的显示屏进行显示。 Specifically, FIG. 7 is a schematic diagram of an application scenario in Embodiment 3 of the present invention. As shown in FIG. 7, the display of the information is specifically realized by a projection screen 21 and a computer, wherein the content on the projection screen is obtained by projecting the computer screen 22 of a computer, and the live image capturing device 50 is a stereoscopic visual device. The processing device displaying the indication identifier is responsible for processing the image acquired by the stereoscopic device, and the display of the content on the display screen of the projection screen is controlled by the computer, but the processing device displaying the indication flag can control the display screen of the computer for display.

与 S201类似，需确定在现场图像捕捉设备的视角下的标准三维坐标系及处理图像的相关参数，具体的确定方法如下： Similar to S201, it is necessary to determine the standard three-dimensional coordinate system and the relevant parameters for processing the image from the perspective of the live image capturing device. The specific determination method is as follows:

S3a、根据预设的采样检测点，获取定标参数信息；本实施例中获取定标参数信息的方法与实施例二 S201中的步骤 S la 类似，但在本实施例中，该定标参数信息除了包括内参数、结构参数以外，还包括用于坐标系转换的参数，具体的，根据张正友定标方法，确定基于现场图像捕捉设备视角下的标准三维坐标系和基于投影幕布视角下的相对三维坐标系 Α^^Λ »之间的转换参数，还有计算机显示屏的二维坐标系 -A 和投影幕布的相对三维坐标系之间的转换参数。 S3a, obtaining the calibration parameter information according to the preset sampling detection point; the method for obtaining the calibration parameter information in the embodiment is similar to the step S la in the second embodiment S201, but in the embodiment, the calibration parameter In addition to the internal parameters and structural parameters, the information includes parameters for coordinate system conversion. Specifically, according to Zhang Zhengyou's calibration method, Based on the standard three-dimensional coordinate system from the perspective of the live image capture device and the relative three-dimensional coordinate system based on the projection screen, the conversion parameters between the two-dimensional coordinate system -A and the projection screen The conversion parameters between the relative three-dimensional coordinate systems.

S 3b、根据所述定标参数信息确定标准三维坐标系； S 3b, determining a standard three-dimensional coordinate system according to the calibration parameter information;

本实施例中的步骤 S3b和实施例二 S201 中的步骤 S ib类似，在此不再赘述。 Step S3b in this embodiment is similar to step S ib in the second embodiment S201, and will not be further described herein.

S3c、根据所述定标参数信息和所述标准三维坐标系确定所述呈像区域在所述标准三维坐标系中的三维区域方程。 S3c. Determine a three-dimensional regional equation of the imaged region in the standard three-dimensional coordinate system according to the calibration parameter information and the standard three-dimensional coordinate system.

本实施例中的步骤 S3c和实施例二 S201 中的步骤 S ic类似，在此不再赘述。 Step S3c in this embodiment is similar to step Sic in the second embodiment S201, and will not be further described herein.

S3d、根据上述转换参数确定坐标系之间的转换关系。具体的，确定上述标准三维坐标系和相对三维坐标系之间的转换关 + T S3d determines a conversion relationship between coordinate systems according to the above conversion parameters. Specifically, determining the conversion between the standard three-dimensional coordinate system and the relative three-dimensional coordinate system + T

所示，其中，、和²。表示在标准坐标系下的 x轴、 Y轴和 Z轴，而 ^ps、和²^表示在上述相对坐标系下的 X轴、 Y轴和 Z 轴， R为 3 x 3的旋转矩阵， T为 3 x 1平移矩阵，其中矩阵中元素的赋值根据现场的测量值确定，即由步骤 S3a确定；进一步的，确定计算机屏幕的二维坐标系和投影幕布的相对三维坐标系之间的转换关系 χ _ + γ _ + +

Shown, where, , and ^{2 are} . Indicates the x-axis, the Y-axis, and the Z-axis in the standard coordinate system, and ^ps, and ² ^ represent the X-axis, Y-axis, and Z-axis in the above relative coordinate system, and R is a rotation matrix of 3 x 3, T Is a 3 x 1 translation matrix, wherein the assignment of elements in the matrix is determined according to the measured values in the field, that is, determined by step S3a; further, determining the conversion relationship between the two-dimensional coordinate system of the computer screen and the relative three-dimensional coordinate system of the projection screen χ _ + γ _ + +

A₇X_PS + Α,Υ_Ρ8 +\ 、 , 其中， Α和为计算机屏幕的二维坐标系中的 X轴和 Y轴，转换参数 4至 4的赋值由步骤 S3a中确定。需要说明的是，上述 S301是在应用场景中第一次实施本发明的处理方法时才会执行，当标准三维坐标系的相关参数确定后，再次实施显示指示标识的处理方法时，直接由 S302开始执行。 A ₇ X _PS + Α, Υ _{Ρ 8} + \ , , where Α is the X-axis and Y-axis in the two-dimensional coordinate system of the computer screen, and the assignment of the conversion parameters 4 to 4 is determined in step S3a. It should be noted that, when the processing method of the present invention is implemented for the first time in the application scenario, when the processing parameter of the standard three-dimensional coordinate system is determined, the processing method for displaying the indication identifier is implemented again, directly by S302. Begin execution.

S302、获取包括人手握持的移动终端和人手，及呈像区域的当前场景图像 S302. Acquire a mobile terminal and a human hand including a human hand, and a current field of the image area Scene image

5303、根据当前场景图像，确定移动终端的特征平面。 S303. Determine a feature plane of the mobile terminal according to the current scene image.

5304、根据移动终端的特征平面，确定人手握持的移动终端的特征平面的中轴线作为指示方向。 5304. Determine, according to a feature plane of the mobile terminal, a central axis of a feature plane of the mobile terminal held by the human hand as the indication direction.

S305、根据指示方向与信息的呈像区域所在范围，确定指示方向与呈像区域之间位于呈像区域中的关联位置。 S305. Determine an associated position between the indication direction and the imaged area in the imaged area according to the indication direction and the range of the imaged area of the information.

5306、判断是否接收到移动终端发送的显示信号；若是，则执行 S307 , 若否，则停止。 5306. Determine whether the display signal sent by the mobile terminal is received; if yes, execute S307, and if not, stop.

本实施例中 S302〜S306与实施例二类似，可参考 S202〜S206,在此不再贅述。 S302 to S306 are similar to the second embodiment in this embodiment, and reference may be made to S202 to S206, and details are not described herein.

5307、根据移动终端提供的显示信号，在关联位置显示指示标识。在 S305中确定的关联位置是在标准三维坐标系下确定的，即上述处理装置在标准三维坐标系下确定了移动终端的特征平面的中轴线与投影幕布之间的交点，但由于投影幕布的显示是将计算机显示屏上的内容进行了投影获得的，所以在投影幕布上的关联位置显示指示标识是需要计算机显示屏在对应该关联位置的地方显示指示标识；因此，显示指示标识的处理装置在关联位置显示指示标识时，还需包括下述各步骤： 5307. Display an indication identifier at the associated location according to the display signal provided by the mobile terminal. The associated position determined in S305 is determined in a standard three-dimensional coordinate system, that is, the processing device determines the intersection between the central axis of the feature plane of the mobile terminal and the projection screen in a standard three-dimensional coordinate system, but due to the projection screen The display is obtained by projecting the content on the computer display screen, so displaying the indication mark on the associated position on the projection screen requires the computer display screen to display the indication mark at the position corresponding to the associated position; therefore, the processing device for displaying the indication mark When displaying the indicator in the associated location, you also need to include the following steps:

S4a, 将关联位置进行坐标系的转换。具体的， S305中确定出标准三维坐标系下的关联位置，即移动终端影幕布之间的交点坐标为（ ⁷d , ^zd ) ；根

S4a, the coordinate of the associated position is converted. Specifically, the associated position in the standard three-dimensional coordinate system is determined in S305, that is, the coordinates of the intersection between the screens of the mobile terminal are ( ⁷ d, ^z d );

, , ^Z。i ) 进行转换，获得关联位置在相对坐

, , ^Z. i) to convert and get the associated position in the opposite position

X X

标系下的坐标（ Y, ZCoordinates under the standard system ( Y, Z

) ; 由于计算机显示屏采用二维坐标系，则根据计算机屏幕的二维坐标系和投影幕布的相对三维坐标系之间的转换关系 Y. Since the computer display screen adopts a two-dimensional coordinate system, the conversion relationship between the two-dimensional coordinate system of the computer screen and the relative three-dimensional coordinate system of the projection screen is Y.

) 进行转换，获得（ ^ ) 。 S4b , 向显示设备发送控制显示消息。

) Convert and get ( ^ ). S4b, sending a control display message to the display device.

上述处理装置在向显示设备中的计算机发送控制显示消息时，携带位置指示消息，即坐标（ i , , 举例来说，计算机根据（ i , ) 确定（ Ai , ^ ) 指示的位置为计算机显示屏中心点，则计算机控制计算机显示屏在中心点处显示红色圓点，从而计算机显示屏中心点处的红色圓点经过投影后在投影幕布上的显示位置与 S305 所确定的关联位置吻合，实现指示标识的显示。 When the processing device sends a control display message to the computer in the display device, the location indication message, that is, the coordinates (i, for example, the computer determines, according to (i, ), the position indicated by (Ai, ^) is the computer display screen. At the center point, the computer control computer display shows a red dot at the center point, so that the red dot at the center point of the computer display screen is projected and the display position on the projection screen coincides with the associated position determined by S305, and the indication is realized. The display of the logo.

更灵活的，显示指示标识的处理装置可不执行 S4a, 直接执行 S4b ; 即将移动终端的特征平面的中轴线与投影幕布之间的交点坐标（ ¹ , )直接发送给计算机，随后由计算机根据各个坐标系转换公式将（ ) 转换为 ( ^ ) ，并根据 ( ^ )在计算机显示屏的对应位置处显示指示标识，则该指示标识同样会被投影至投影幕布中由 S304 所确定的关联位置处。 More flexible, the processing device displaying the indication identifier may directly execute S4b without executing S4a; sending the intersection coordinates ( ¹ , ) between the central axis of the feature plane of the mobile terminal and the projection screen directly to the computer, and then according to each coordinate by the computer The conversion formula converts ( ) to ( ^ ) and displays the indication mark at the corresponding position on the computer display screen according to ( ^ ), and the indication indicator is also projected onto the associated position in the projection screen determined by S304.

在本实施例中提供的指示标识控制方法，通过采集的当前场景图像，确定移动终端的指示方向及该指示方向与信息的呈像区域之间的关联位置，实现通过移动终端在呈像区域上显示指示标识，便于用户随时使用任意移动终端在信息的呈像区域上做出指示，即该方法通用性较好，且操作便捷，给用户的工作学习带来便利。 The indication identifier control method provided in the embodiment determines the indication direction of the mobile terminal and the associated position between the indication direction and the image presentation area of the information through the collected current scene image, and realizes the image location on the image area through the mobile terminal. The indication indicator is displayed, so that the user can use any mobile terminal to make an indication on the image presentation area of the information at any time, that is, the method has good versatility and convenient operation, and brings convenience to the user's work and study.

图 8 为本发明显示指示标识的处理装置实施例一的结构图。如图 8 所示，该装置包括： FIG. 8 is a structural diagram of Embodiment 1 of a processing apparatus for displaying an indication mark according to the present invention. As shown in Figure 8, the device includes:

获取模块 10 , 用于获取包括人手握持的移动终端和所述人手及呈像区域的当前场景图像； The obtaining module 10 is configured to acquire a current scene image including the mobile terminal held by the human hand and the human hand and the image area;

第一确定模块 1 1 , 用于根据所述当前场景图像，确定所述移动终端指向所述呈像区域的指示方向； The first determining module 1 1 is configured to determine, according to the current scene image, an indication direction of the mobile terminal pointing to the image area;

第二确定模块 12 , 用于根据所述指示方向与所述呈像区域在所述当前场景图像中的范围，确定所述指示方向与所述呈像区域之间位于呈像区域中的关联位置； a second determining module 12, configured to determine, according to the indication direction and a range of the image area in the current scene image, an associated position between the indication direction and the image area in the image area ;

控制模块 13 , 用于根据所述移动终端提供的显示信号，在所述关联位置显示指示标识。 The control module 13 is configured to display an indication identifier at the associated location according to the display signal provided by the mobile terminal.

在本实施例中提供的显示指示标识的处理装置，通过采集的当前场景图像，确定移动终端的指示方向及该指示方向与信息的呈像区域之间的关联位置，实现通过移动终端在呈像区域上显示指示标识，便于用户随时使用任意移动终端在信息的呈像区域上做出指示，即该装置通用性较好，且操作便捷，给用户的工作学习带来便利。 The processing device for displaying the indication identifier provided in the embodiment determines the indication direction of the mobile terminal and the image orientation area of the indication direction and the information through the collected current scene image. The associated location enables the display of the indicator on the imaged area by the mobile terminal, so that the user can use any mobile terminal to make an indication on the imaged area of the information at any time, that is, the device has good versatility and convenient operation, and is provided to the user. Work and study bring convenience.

图 9 为本发明显示指示标识的处理装置实施例二的结构图。如图 9 所示，上述第一确定模块 1 1具体包括： FIG. 9 is a structural diagram of Embodiment 2 of a processing apparatus for displaying an indication identifier according to the present invention. As shown in FIG. 9, the foregoing first determining module 1 1 specifically includes:

人手识别单元 1 1 1 , 用于根据预设的人手识别特征，在所述当前场景图像中确定人手所在位置； a human hand recognition unit 1 1 1 is configured to determine a location of a human hand in the current scene image according to a preset human hand recognition feature;

特征平面确定单元 1 12 , 用于基于所述人手所在位置，采用边缘检测算子确定所述人手握持的所述移动终端的特征平面，所述特征平面为所述移动终端的一个具有中轴线的几何平面图形，或者所述特征平面为所述移动终端控制显示的具有中轴线的几何平面图形； a feature plane determining unit 1 12, configured to determine, according to the location of the human hand, an edge detection operator to determine a feature plane of the mobile terminal that is held by the human hand, where the feature plane is a central axis of the mobile terminal Geometric plane graphic, or the feature plane is a geometric plane graphic having a central axis displayed by the mobile terminal;

指示方向确定单元 1 13还用于根据所述移动终端的特征平面，确定所述移动终端的特征平面的中轴线作为所述指示方向。 The indication direction determining unit 1 13 is further configured to determine a central axis of a feature plane of the mobile terminal as the indication direction according to a feature plane of the mobile terminal.

进一步的，上述处理装置还包括： Further, the processing device further includes:

接收模块 14 , 用于接收移动终端发送的显示信号。 The receiving module 14 is configured to receive a display signal sent by the mobile terminal.

上述指示方向确定单元 1 13具体包括：第一方程确定子单元 1 13a, 具体用于采用直线检测算法，确定所述具有中轴线的几何平面图形的二维区域方程； The above-mentioned indication direction determining unit 1 13 specifically includes: a first equation determining sub-unit 1 13a, specifically for determining a two-dimensional regional equation of the geometric plane figure having a central axis by using a line detection algorithm;

第二方程确定子单元 1 13b还用于根据所述二维区域方程，获取所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程；形在标准三维坐标系下的三维区域方程，确定所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 The second equation determining subunit 1 13b is further configured to acquire a three-dimensional regional equation of the geometric plane graphic having the central axis in a standard three-dimensional coordinate system according to the two-dimensional regional equation; a three-dimensional region shaped in a standard three-dimensional coordinate system Equation, determining a three-dimensional line equation of the central axis of the geometrical planar pattern having the central axis in the standard three-dimensional coordinate system as the indication direction.

进一步的，当上述具有中轴线的几何平面图形具有至少两条中轴线；则上述指示方向确定子单元 1 13c还用于获取移动终端的内置电子罗盘当前的指示方向； Further, when the geometric plane graphic having the central axis has at least two central axes; the indication direction determining sub-unit 1 13c is further configured to acquire the current indication direction of the built-in electronic compass of the mobile terminal;

在上述具有中轴线的几何平面图形的至少两条中轴线中选择与所述电子罗盘当前的指示方向平行的第一中轴线，并根据所述特征平面在标准三维坐标系下的三维区域方程，确定所述第一中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。进一步的，上述第二确定模块 12具体用于根据所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程与所述呈像区域在所述标准三维坐标系中的三维区域方程进行运算，获得在所述标准三维坐标系下的交点坐标，确定所述交点坐标作为所述关联位置。 Selecting, in the above at least two central axes of the geometrical planar pattern having the central axis, a first central axis parallel to the current indication direction of the electronic compass, and according to the three-dimensional regional equation of the characteristic plane in a standard three-dimensional coordinate system, A three-dimensional line equation of the first central axis in the standard three-dimensional coordinate system is determined as the indication direction. Further, the second determining module 12 is specifically configured to: according to the central axis of the geometric plane graphic having the central axis, the three-dimensional linear equation in the standard three-dimensional coordinate system and the image forming area in the standard three-dimensional coordinate system. The three-dimensional region equation is operated to obtain intersection coordinates in the standard three-dimensional coordinate system, and the intersection coordinates are determined as the associated position.

灵活的，对应于上述方法实施例三，上述处理装置还包括发送模块 Flexible, corresponding to the third embodiment of the foregoing method, the processing device further includes a sending module

15 , 用于向显示设备发送控制显示消息。 15 . , configured to send a control display message to the display device.

第三确定模块 16 , 在根据当前场景图像，确定移动终端的指示方向之前，根据预设的采样检测点，获取定标参数信息；根据所述定标参数信息确定所述标准三维坐标系；根据所述定标参数信息和所述标准三维坐标系确定所述呈像区域在所述标准三维坐标系中的三维区域方程。 The third determining module 16 is configured to obtain calibration parameter information according to the preset sampling detection point before determining the indication direction of the mobile terminal according to the current scene image; and determining the standard three-dimensional coordinate system according to the calibration parameter information; The calibration parameter information and the standard three-dimensional coordinate system determine a three-dimensional region equation of the imaged region in the standard three-dimensional coordinate system.

上述各装置实施例中的各模块对应于上述各方法实施例，在此不再贅述。 The modules in the foregoing embodiments of the present invention correspond to the foregoing method embodiments, and are not described herein again.

在本实施例中提供的显示指示标识的处理装置，通过采集的当前场景图像，确定移动终端的指示方向及该指示方向与信息的呈像区域之间的关联位置，实现通过移动终端在呈像区域上显示指示标识，便于用户随时使用任意移动终端在信息的呈像区域上做出指示，即该装置通用性较好，且操作便捷，给用户的工作学习带来便利。 The processing device for displaying the indication identifier provided in the embodiment determines the indication direction of the mobile terminal and the associated position between the indication direction and the image presentation area of the information through the acquired current scene image, thereby realizing the image representation by the mobile terminal The indication flag is displayed on the area, so that the user can use any mobile terminal to make an indication on the image presentation area of the information at any time, that is, the device has good versatility and convenient operation, and brings convenience to the user's work and study.

进一步的，图 10为本发明显示指示标识的处理***实施例一的示意图。如图 10所示，该***包括显示设备 60、移动终端 30及控制设备 40 , 其中显示设备 60具有用于显示信息的呈像区域，具体可为上述投影幕布 21和计算机 22共同构成；也可为上述一个独立的电子显示屏 20 , 该电子显示屏 20所显示的内容由控制设备 40控制显示；该控制设备 40包括上述显示指示标识的处理装置实施例一和实施例二中所述的显示指示标识的处理装置。另外，控制设备 40可集成上述现场图像捕捉设备 50的功能，也可接收独立的现场图像捕捉设备 50所发送的现场场景图像；当控制设备 40分析现场场景图像后，确定了移动终端 30的指示方向和呈像区域之间的关联位置，则控制设备 40控制显示设备 20在关联位置处显示指示标识。 Further, FIG. 10 is a schematic diagram of Embodiment 1 of a processing system for displaying an indication identifier according to the present invention. As shown in FIG. 10, the system includes a display device 60, a mobile terminal 30, and a control device 40, wherein the display device 60 has an image forming area for displaying information, and specifically, the projection screen 21 and the computer 22 are combined; For the above-mentioned independent electronic display screen 20, the content displayed by the electronic display screen 20 is controlled by the control device 40; the control device 40 includes the display device described in the first embodiment and the second embodiment. A processing device that indicates the identification. In addition, the control device 40 may integrate the functions of the live image capturing device 50 described above, and may also receive the live scene image transmitted by the independent live image capturing device 50; after the control device 40 analyzes the live scene image, the indication of the mobile terminal 30 is determined. The control device 40 controls the display device 20 to display the indication flag at the associated location, with the associated location between the direction and the imaged region.

灵活的，控制设备 40确定关联位置之后，控制显示设备 20在关联位置处显示指示标识之前，需判断是否接收到移动终端 30发送的指示标识的显示消息，若是，则控制显示设备 20在关联位置处显示指示标识；否则，不进行显示；当场景中存在多个移动终端时，通过判断是否接收到某个移动终端 30 发送的指示标识的显示消息进行指示标识的显示控制，避免在信息的呈像区域上出现指示标识显示混乱的情况。 After the control device 40 determines the associated location, before the control display device 20 displays the indication identifier at the associated location, it is determined whether the indicator sent by the mobile terminal 30 is received. Displaying the message, if yes, controlling the display device 20 to display the indication identifier at the associated location; otherwise, not displaying; when there are multiple mobile terminals in the scene, determining whether to receive the indication identifier sent by a certain mobile terminal 30 The display message performs display control of the indication flag to avoid a situation in which the indication flag is displayed confusing on the image area of the information.

在本实施例中提供的指示标识控制***，通过采集的当前场景图像，确定移动终端的指示方向及该指示方向与信息的呈像区域之间的关联位置，实现通过移动终端对该信息的呈像区域的指示标识，便于用户随时使用任意移动终端在信息的呈像区域上做出指示，即该***通用性较好，且操作便捷，给用户的工作学习带来便利。 The indication identifier control system provided in the embodiment determines the indication direction of the mobile terminal and the associated position between the indication direction and the image presentation area of the information by using the collected current scene image, thereby realizing the presentation of the information by the mobile terminal. The indicator of the image area is convenient for the user to use any mobile terminal to make an indication on the image presentation area of the information at any time, that is, the system has good versatility and convenient operation, and brings convenience to the user's work and study.

本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括： ROM、 RAM, 磁碟或者光盘等各种可以存储程序代码的介质。 Those of ordinary skill in the art will appreciate that all or a portion of the steps to implement the various method embodiments described above can be accomplished by hardware associated with the program instructions. The aforementioned program can be stored in a computer readable storage medium. The program, when executed, performs the steps including the above-described method embodiments; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.

最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。 It should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art The technical solutions described in the foregoing embodiments may be modified, or some or all of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the technical solutions of the embodiments of the present invention. range.

Claims

权利要求 Rights request

1、一种显示指示标识的处理方法，其特征在于，包括： 1. A method for displaying indication marks, which is characterized by including:

获取包括人手握持的移动终端和所述人手及呈像区域的当前场景图像； Obtain the current scene image including the mobile terminal held by the human hand and the human hand and the imaging area;

根据所述当前场景图像，确定所述移动终端指向所述呈像区域的指示方向； Determine the direction in which the mobile terminal points to the imaging area according to the current scene image;

根据所述指示方向与所述呈像区域在所述当前场景图像中的范围，确定所述指示方向与所述呈像区域之间位于所述呈像区域中的关联位置； According to the range of the indication direction and the imaging area in the current scene image, determine the associated position in the imaging area between the indication direction and the imaging area;

根据所述移动终端提供的显示信号，在所述关联位置显示指示标识。 According to the display signal provided by the mobile terminal, an indication mark is displayed at the associated position.

2、根据权利要求 1所述的处理方法，其特征在于，所述根据所述当前场景图像，确定所述移动终端的指向所述呈像区域的指示方向，包括：根据预设的人手识别特征，在所述当前场景图像中确定人手所在位置； 2. The processing method according to claim 1, characterized in that, based on the current scene image, determining the pointing direction of the mobile terminal pointing to the imaging area includes: based on preset human hand recognition characteristics , determine the position of the human hand in the current scene image;

基于所述人手所在位置，采用边缘检测算子确定所述人手握持的所述移动终端的特征平面，所述特征平面为所述移动终端的一个具有中轴线的几何平面图形，或者所述特征平面为所述移动终端所显示的具有中轴线的几何平面图形； Based on the position of the human hand, an edge detection operator is used to determine the characteristic plane of the mobile terminal held by the human hand. The characteristic plane is a geometric plane figure with a central axis of the mobile terminal, or the characteristic plane The plane is a geometric plane figure with a central axis displayed by the mobile terminal;

根据所述移动终端的特征平面，确定所述移动终端的特征平面的中轴线作为所述指示方向。 According to the characteristic plane of the mobile terminal, the central axis of the characteristic plane of the mobile terminal is determined as the indication direction.

3、根据权利要求 1或 2所述的处理方法，其特征在于，在根据所述移动终端提供的显示信号，在所述关联位置显示指示标识之前，所述显示方法还包括： 3. The processing method according to claim 1 or 2, characterized in that, before displaying the indication mark at the associated position according to the display signal provided by the mobile terminal, the display method further includes:

接收移动终端发送的所述显示信号。 Receive the display signal sent by the mobile terminal.

4、根据权利要求 2所述的处理方法，其特征在于，根据所述移动终端的特征平面，确定所述移动终端的特征平面的中轴线作为所述指示方向，包括： 4. The processing method according to claim 2, characterized in that, according to the characteristic plane of the mobile terminal, determining the central axis of the characteristic plane of the mobile terminal as the indication direction includes:

采用直线检测算法，确定所述具有中轴线的几何平面图形的二维区域方程； Use a straight line detection algorithm to determine the two-dimensional area equation of the geometric plane figure with a central axis;

根据所述二维区域方程，获取所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程； According to the two-dimensional area equation, the geometric plane figure with the central axis is obtained in Three-dimensional area equation in a quasi-three-dimensional coordinate system;

根据所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程，确定所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 According to the three-dimensional area equation of the geometric plane figure with a central axis in the standard three-dimensional coordinate system, determine the three-dimensional straight line equation of the central axis of the geometric plane figure with the central axis in the standard three-dimensional coordinate system as the indication direction.

5、根据权利要求 4所述的显示方法，其特征在于，根据所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程，确定所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向，包括： 5. The display method according to claim 4, characterized in that, according to the three-dimensional area equation of the geometric plane figure with a central axis in a standard three-dimensional coordinate system, the central axis of the geometric plane figure with a central axis is determined. The three-dimensional straight line equation under the standard three-dimensional coordinate system as the indicated direction includes:

当所述具有中轴线的几何平面图形具有至少两条中轴线；则获取移动终端的内置电子罗盘当前的指示方向； When the geometric plane figure with a central axis has at least two central axes; then obtain the current indicated direction of the built-in electronic compass of the mobile terminal;

在所述具有中轴线的几何平面图形的至少两条中轴线中选择与所述电子罗盘当前的指示方向平行的第一中轴线，并根据所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程，确定所述第一中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 Select the first central axis parallel to the current indication direction of the electronic compass among at least two central axes of the geometric plane figure with a central axis, and calculate the first central axis in a standard three-dimensional coordinate system according to the geometric plane figure with a central axis. The three-dimensional area equation under is determined as the indicated direction as the three-dimensional straight line equation of the first central axis under the standard three-dimensional coordinate system.

6、根据权利要求 4或 5所述的处理方法，其特征在于，根据所述指示方向与所述呈像区域在所述当前场景图像中的范围，确定所述指示方向与所述呈像区域之间位于所述呈像区域中的关联位置，包括： 6. The processing method according to claim 4 or 5, characterized in that, the indication direction and the imaging area are determined according to the range of the indication direction and the imaging area in the current scene image. The associated positions located in the imaging area include:

根据所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程与所述呈像区域在所述标准三维坐标系中的三维区域方程进行运算，获得在所述标准三维坐标系下的交点坐标，确定所述交点坐标作为所述关联位置。 According to the three-dimensional straight line equation of the central axis of the geometric plane figure with the central axis in the standard three-dimensional coordinate system and the three-dimensional area equation of the image area in the standard three-dimensional coordinate system, we obtain in the The intersection coordinates in the standard three-dimensional coordinate system are determined as the associated positions.

7、根据权利要求 4〜6任一项所述的显示方法，其特征在于，在所述根据当前场景图像，确定移动终端的指示方向之前，所述控制方法还包括： 7. The display method according to any one of claims 4 to 6, characterized in that, before determining the indicating direction of the mobile terminal according to the current scene image, the control method further includes:

根据预设的采样检测点，获取定标参数信息； Obtain calibration parameter information according to the preset sampling detection points;

根据所述定标参数信息确定所述标准三维坐标系； Determine the standard three-dimensional coordinate system according to the calibration parameter information;

根据所述定标参数信息和所述标准三维坐标系确定所述呈像区域在所述标准三维坐标系中的三维区域方程。 The three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system is determined according to the calibration parameter information and the standard three-dimensional coordinate system.

8、一种显示指示标识的处理装置，其特征在于，包括： 8. A processing device for displaying indication marks, which is characterized in that it includes:

获取模块，用于获取包括人手握持的移动终端和所述人手及呈像区域的当前场景图像； Acquisition module, used to acquire the mobile terminal including the human hand held by the human hand and the image display area The current scene image of the domain;

第一确定模块，用于根据所述当前场景图像，确定所述移动终端指向所述呈像区域的指示方向； A first determination module, configured to determine the direction in which the mobile terminal points to the imaging area according to the current scene image;

第二确定模块，用于根据所述指示方向与所述呈像区域在所述当前场景图像中的范围，确定所述指示方向与所述呈像区域之间位于呈像区域中的关联位置； A second determination module, configured to determine the associated position between the indication direction and the imaging area in the imaging area based on the range of the indication direction and the imaging area in the current scene image;

控制模块，用于根据所述移动终端提供的显示信号，在所述关联位置显示指示标识。 A control module configured to display an indication mark at the associated position according to the display signal provided by the mobile terminal.

9、根据权利要求 8所述的处理装置，其特征在于，所述第一确定模块，包括： 9. The processing device according to claim 8, characterized in that the first determination module includes:

人手识别单元，用于根据预设的人手识别特征，在所述当前场景图像中确定人手所在位置； A human hand recognition unit, configured to determine the position of the human hand in the current scene image according to the preset human hand recognition characteristics;

特征平面确定单元，用于基于所述人手所在位置，采用边缘检测算子确定所述人手握持的所述移动终端的特征平面，所述特征平面为所述移动终端的一个具有中轴线的几何平面图形，或者所述特征平面为所述移动终端所显示的具有中轴线的几何平面图形； A characteristic plane determination unit, configured to use an edge detection operator to determine the characteristic plane of the mobile terminal held by the human hand based on the position of the human hand, where the characteristic plane is a geometry with a central axis of the mobile terminal. A plane figure, or the characteristic plane is a geometric plane figure with a central axis displayed by the mobile terminal;

指示方向确定单元，用于根据所述移动终端的特征平面，确定所述移动终端的特征平面的中轴线作为所述指示方向。 An indication direction determining unit, configured to determine the central axis of the characteristic plane of the mobile terminal as the indication direction according to the characteristic plane of the mobile terminal.

10、根据权利要求 8或 9所述的处理装置，其特征在于，所述显示装置还包括： 10. The processing device according to claim 8 or 9, characterized in that the display device further includes:

接收模块，用于接收移动终端发送的所述显示信号。 A receiving module, configured to receive the display signal sent by the mobile terminal.

1 1、根据权利要求 9所述的处理装置，其特征在于，所述指示方向确定单元包括： 11. The processing device according to claim 9, characterized in that the indicating direction determining unit includes:

第一方程确定子单元，用于采用直线检测算法，确定所述具有中轴线的几何平面图形的二维区域方程； The first equation determination subunit is used to determine the two-dimensional area equation of the geometric plane figure with a central axis using a straight line detection algorithm;

第二方程确定子单元，用于根据所述二维区域方程，获取所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程； The second equation determination subunit is used to obtain the three-dimensional regional equation of the geometric plane figure with the central axis in the standard three-dimensional coordinate system according to the two-dimensional regional equation;

指示方向确定子单元，用于根据所述具有中轴线的几何平面图形在标准三维坐标系下的三维区域方程，确定所述具有中轴线的几何平面图形的中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 Indicate direction determination subunit, used to determine the central axis of the geometric plane figure with a central axis in the standard three-dimensional coordinate system according to the three-dimensional area equation of the geometric plane figure with a central axis in the standard three-dimensional coordinate system. The three-dimensional straight line equation is used as the indicator square Towards.

12、根据权利要求 1 1所述的处理装置，其特征在于，当所述具有中轴线的几何平面图形具有至少两条中轴线；则所述指示方向确定子单元具体用于获取移动终端的内置电子罗盘当前的指示方向；在所述具有中轴线的几何平面图形的至少两条中轴线中选择与所述电子罗盘当前的指示方向平行的第一中轴线，并根据所述特征平面在标准三维坐标系下的三维区域方程，确定所述第一中轴线在所述标准三维坐标系下的三维直线方程作为所述指示方向。 12. The processing device according to claim 11, characterized in that when the geometric plane figure with a central axis has at least two central axes; then the indication direction determining subunit is specifically used to obtain the built-in information of the mobile terminal. The current indication direction of the electronic compass; Select the first central axis parallel to the current indication direction of the electronic compass from at least two central axes of the geometric plane figure with a central axis, and use the characteristic plane in a standard three-dimensional The three-dimensional area equation under the coordinate system determines the three-dimensional straight line equation of the first central axis under the standard three-dimensional coordinate system as the indicated direction.

13、根据权利要求 1 1或 12所述的处理装置，其特征在于，所述第在所述标准三维坐标系下的三维直线方程与所述呈像区域在所述标准三维坐标系中的三维区域方程进行运算，获得在所述标准三维坐标系下的交点坐标，确定所述交点坐标作为所述关联位置。 13. The processing device according to claim 11 or 12, characterized in that: the third three-dimensional linear equation in the standard three-dimensional coordinate system and the three-dimensional equation of the image region in the standard three-dimensional coordinate system. The regional equation is operated to obtain the intersection coordinates in the standard three-dimensional coordinate system, and the intersection coordinates are determined as the associated positions.

14、根据权利要求 1 1〜13任一项所述的处理装置，其特征在于，处理装置还包括： 14. The processing device according to any one of claims 11 to 13, characterized in that the processing device further includes:

第三确定模块，用于在根据当前场景图像，确定移动终端的指示方向之前，根据预设的采样检测点，获取定标参数信息；根据所述定标参数信息确定所述标准三维坐标系；根据所述定标参数信息和所述标准三维坐标系确定所述呈像区域在所述标准三维坐标系中的三维区域方程。 The third determination module is configured to obtain calibration parameter information based on preset sampling detection points before determining the indicated direction of the mobile terminal based on the current scene image; determine the standard three-dimensional coordinate system based on the calibration parameter information; The three-dimensional area equation of the imaging area in the standard three-dimensional coordinate system is determined according to the calibration parameter information and the standard three-dimensional coordinate system.

15、一种显示指示标识的处理***，其特征在于，包括显示设备、移动终端及控制设备；所述控制设备包括权利要求 7〜14中任一项所述的显示指示标识的处理装置；所述显示设备具有用于显示信息的呈像区域；所述移动终端和所述显示设备可分别通过通信接口与控制设备进行数据传输。 15. A processing system for displaying indication signs, characterized in that it includes a display device, a mobile terminal and a control device; the control device includes the processing device for displaying indication signs according to any one of claims 7 to 14; The display device has an imaging area for displaying information; the mobile terminal and the display device can respectively perform data transmission with the control device through a communication interface.