WO2021147813A1 - Opacity control method and apparatus, ar/mr device, controller, and medium - Google Patents

Opacity control method and apparatus, ar/mr device, controller, and medium Download PDF

Info

Publication number
WO2021147813A1
WO2021147813A1 PCT/CN2021/072456 CN2021072456W WO2021147813A1 WO 2021147813 A1 WO2021147813 A1 WO 2021147813A1 CN 2021072456 W CN2021072456 W CN 2021072456W WO 2021147813 A1 WO2021147813 A1 WO 2021147813A1
Authority
WO
WIPO (PCT)
Prior art keywords
opacity
eye
wearer
parameters
light intensity
Prior art date
Application number
PCT/CN2021/072456
Other languages
French (fr)
Chinese (zh)
Inventor
陈一鸣
管亮
朱海超
Original Assignee
芋头科技（杭州）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 芋头科技（杭州）有限公司 filed Critical 芋头科技（杭州）有限公司
Publication of WO2021147813A1 publication Critical patent/WO2021147813A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features

Definitions

  • the present invention relates to the field of AR/MR technology, and in particular to an opacity control method, device, AR/MR equipment, controller and medium.
  • AR augmented reality
  • MR mixed reality
  • the existing technical solution does not automatically adjust the opacity of the system based on the user's use state, but only allows the system to directly adjust the opacity, but this adjustment method has no direct relationship with the current state of the user , This will greatly affect the user experience.
  • the purpose of the present invention is to provide an opacity control method, device, AR/MR equipment, controller and medium, which can adaptively adjust the opacity of AR/MR equipment based on the opacity regression model according to the most natural state and reaction of the user’s eyes , Improve the user experience.
  • an opacity control method for AR/MR equipment including:
  • the corresponding device parameter is adjusted based on the target value of the device parameter to adjust the opacity of the AR/MR device.
  • the opacity regression model which specifically includes:
  • the light intensity and the corresponding normalized eye state parameters are used as input, and the corresponding device parameters are used as output to perform regression model training to obtain the opacity regression model.
  • the characteristic parameters of the eye state include the opening and closing degree of the glasses and the pupil size.
  • the device parameters include one or more of the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optomechanics, wherein the electrochromic film and the liquid crystal film are installed in AR/MR Inside the front protective cover of the device.
  • an opacity control device for AR/MR equipment including:
  • the parameter acquisition module is configured to acquire the current light intensity and characteristic parameters of the wearer's eye condition
  • the target value output module is configured to input the current light intensity and the characteristic parameters of the wearer's eye state into a pre-trained opacity regression model, and output the target value of the device parameter;
  • the opacity adjustment module is configured to adjust the corresponding device parameter based on the target value of the device parameter to adjust the opacity of the AR/MR device.
  • an opacity regression model training module which specifically includes:
  • the eye state calibration unit is configured to obtain the eye reference state of the experiment wearer
  • the training parameter acquisition unit is configured to use different light intensities as the test environment to acquire the corresponding equipment parameters and the characteristic parameters of the eye condition of the experiment wearer under each light intensity, which makes the experiment wearer in the most comfortable state;
  • An eye feature parameter standardization unit configured to standardize the eye state feature parameters of the experiment wearer based on the eye reference state of the experiment wearer;
  • the regression model training unit is configured to perform regression model training with light intensity and corresponding standardized eye state parameters as input and corresponding device parameters as output to obtain the opacity regression model.
  • the parameter acquisition module includes:
  • An eye tracking sensor for acquiring characteristic parameters of the eye state, the characteristic parameters of the eye state including the opening and closing degree of the glasses and the pupil size;
  • the light intensity sensor is used to obtain the current light intensity.
  • the device parameters include one or more of the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optomechanics, wherein the electrochromic film and the liquid crystal film are installed in AR/MR Inside the front protective cover of the device.
  • an AR/MR device including the opacity control device for AR/MR device.
  • a controller which includes a memory and a processor, the memory stores a computer program, and the program can implement the steps of the method when executed by the processor.
  • a computer-readable storage medium for storing a computer program, which when executed by a computer or a processor, implements the steps of the method.
  • the present invention Compared with the prior art, the present invention has obvious advantages and beneficial effects.
  • the present invention provides an opacity control method, device, AR/MR equipment, controller, and medium that can achieve considerable technological advancement and practicability, and has a wide range of industrial use value, at least Has the following advantages:
  • the invention uses the recognition and judgment of the user's eye state and calls the opacity regression model to adaptively determine the opacity degree of the AR/MR device according to the current environment. Specifically, the degree of discoloration of the electrochromic film and the degree of optical diffusion of the liquid crystal film can be used. And optical power level to control the opacity of AR/MR equipment.
  • the opacity control system based on the eye state eliminates the user's manual operation, and dynamically adjusts the opacity of the AR/MR device according to the wearer's eye state to minimize the impact of visual noise in the external environment on the user , Improve the visibility and distinguishability of the information displayed by the AR/MR device in different external environments, and enhance the user experience.
  • FIG. 1 is a schematic diagram of an opacity control method provided by an embodiment of the present invention
  • Figure 2 (a) is the effect diagram of the electrochromic film through the thin film
  • Figure 2(b) is an effect diagram of a slow-transmitting liquid crystal film through light
  • FIG. 3 is a schematic diagram of the electrochromic film and the liquid crystal film provided in an embodiment of the present invention installed in an AR/MR device;
  • FIG. 4 is a schematic diagram of an opacity control device provided by an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of an opacity control device provided by another embodiment of the present invention.
  • the embodiment of the present invention provides an opacity control method for AR/MR equipment, as shown in FIG. 1, including the following steps:
  • Step S1 Obtain current light intensity and characteristic parameters of the wearer's eye condition
  • Step S2 Input the current light intensity and the characteristic parameters of the wearer's eye condition into the pre-trained opacity regression model, and output the target value of the device parameter;
  • Step S3 Adjust the corresponding device parameter based on the target value of the device parameter to adjust the opacity of the AR/MR device.
  • the method described in the embodiment of the present invention uses the recognition and judgment of the user’s eye state and calls the opacity regression model to adaptively determine the opacity of the AR/MR device according to the current environment, eliminating the need for the user’s manual operation.
  • the opacity of the AR/MR device can be dynamically adjusted by the user’s eye status to minimize the impact of visual noise in the external environment on the user, and to improve the visibility and visibility of the information displayed by the AR/MR device in different external environments. Distinguishability improves user experience.
  • AR equipment includes equipment such as AR glasses
  • MR equipment includes equipment such as MR glasses.
  • an eye tracking sensor may be used to obtain the characteristic parameters of the eye state, and the characteristic parameters of the eye state may include the opening and closing degree of the glasses and the pupil size.
  • haar-like features can be used to locate the eyes, and then the Hough transform can be applied to detect the pupil size.
  • the degree of eye opening and closing is used to indicate the degree of eye opening, and the pupil size is characterized by the diameter of the pupil. It should be noted that the above method for obtaining the characteristic parameters of the eye state is only an example, and there are other methods that can be used to locate the eye, measure the pupil size, and quantify the opening and closing degree of the eye can also be used for this.
  • the light intensity sensor can be used to obtain the current light intensity.
  • the electrochromic film and the light slow-transmitting liquid crystal film allow light to be attenuated and slowed on its surface.
  • the effect of the electrochromic thin film is shown in Figure 2(a), and the effect diagram of the slow-transmitting liquid crystal film is shown in Figure 2(a).
  • the attenuation and diffusion of light isolates the wearer from the external environment, which will enhance the user experience on AR/MR devices.
  • These films are small in size and can be installed in AR or MR devices, allowing users to observe the external environment through them.
  • the electrochromic film and the liquid crystal film can be installed in the front protective cover of the AR/MR device.
  • the device parameters include the degree of discoloration of the electrochromic film and the optical diffusion of the liquid crystal film.
  • the state of the film can be controlled by controlling the pair of electrodes connected to each layer.
  • the opacity control of the AR/MR device in the prior art is still manual, and this manual control method is very troublesome for users.
  • the solution of the embodiment of the present invention automatically adjusts the opacity of the AR/MR device according to the state of the user's eyes, which is very convenient for the user and improves the user experience.
  • the method further includes step S4, training the opacity regression model, which specifically includes:
  • Step S41 Obtain the eye reference state of the experiment wearer.
  • the experiment wearer is the participant used to obtain the training data. It can be a specific one or multiple. If it is a specific one, an opacity regression model matching the specific user can be obtained. If there are multiple experimental wearers, an opacity regression model suitable for the general public can be obtained.
  • Step S42 Use different light intensities as the test environment, and obtain the corresponding device parameters and the characteristic parameters of the eye condition of the test wearer under each light intensity to make the test wearer in the most comfortable state.
  • different lighting conditions can be controlled, the light intensity can be adjusted, and the corresponding test environment can be set.
  • Step S43 Standardize the characteristic parameters of the eye state of the experiment wearer based on the reference state of the eye of the experiment wearer.
  • the reference state of the eyes of the experimental wearer that is, the opening and closing degree and pupil size of the corresponding eyes of the user in the most natural state, are standardized based on this, and the model accuracy is improved.
  • Step S44 Use the light intensity and the corresponding standardized eye state parameters as input, and use the corresponding device parameters as output to perform regression model training to obtain the opacity regression model.
  • Taking the characteristic parameters of the eye condition may include the degree of eyeglass opening and closing and pupil size
  • the device parameters include the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optical machine as an example
  • the user adjusts the discoloration degree of the electrochromic film of the device, the degree of optical diffusion of the liquid crystal film and the optical power level to the most comfortable state for the user. Then, the eye opening and closing degree and pupil size are detected, and the light intensity sensor measures the light intensity.
  • the light intensity, eye opening (after normalization), pupil size (after normalization) and the degree of discoloration of the electrochromic film, the current state of the liquid crystal film's optical slowness and optical power level are recorded into the internal memory as a single data point.
  • the embodiment of the present invention also provides an opacity control device for AR/MR equipment, as shown in FIG. 4, including a parameter acquisition module 1, a target value output module 2 and an opacity adjustment module 3, wherein the parameter acquisition The module 1 is configured to obtain the current light intensity and the wearer’s eye state characteristic parameters; the target value output module 2 is configured to input the current light intensity and the wearer’s eye state characteristic parameters into the pre-trained opacity regression model, and the output device Parameter target value; the opacity adjustment module 3 is configured to adjust the corresponding device parameter based on the device parameter target value to adjust the opacity of the AR/MR device.
  • the parameter acquisition The module 1 is configured to obtain the current light intensity and the wearer’s eye state characteristic parameters
  • the target value output module 2 is configured to input the current light intensity and the wearer’s eye state characteristic parameters into the pre-trained opacity regression model, and the output device Parameter target value
  • the opacity adjustment module 3 is configured to adjust the corresponding device parameter
  • the device described in the embodiment of the present invention adaptively determines the opacity of the AR/MR device according to the current environment by recognizing and judging the state of the user’s eyes and calling the opacity regression model, eliminating the need for the user’s manual operation.
  • the opacity of the AR/MR device can be dynamically adjusted by the user’s eye status to minimize the impact of visual noise in the external environment on the user, and to improve the visibility and visibility of the information displayed by the AR/MR device in different external environments. Distinguishability improves user experience.
  • the parameter acquisition module 1 includes an eye tracking sensor and a light intensity sensor, where the eye tracking sensor is used to acquire eye state characteristic parameters, and the eye state characteristic parameters include glasses opening and closing degree and pupil size;
  • the intensity sensor is used to obtain the current light intensity.
  • the electrochromic film and the light slow-transmitting liquid crystal film allow light to be attenuated and slowed on its surface.
  • the effect of the electrochromic thin film is shown in Figure 2(a), and the effect diagram of the slow-transmitting liquid crystal film is shown in Figure 2(a).
  • the attenuation and diffusion of light isolates the wearer from the external environment, which will enhance the user experience on AR/MR devices.
  • These films are small in size and can be installed in AR or MR devices, allowing users to observe the external environment through them.
  • the electrochromic film and the liquid crystal film can be installed in the front protective cover of the AR/MR device.
  • the device parameters include the degree of discoloration of the electrochromic film and the optical diffusion of the liquid crystal film.
  • the state of the film can be controlled by controlling the pair of electrodes connected to each layer.
  • the opacity control of the AR/MR device in the prior art is still manual, and this manual control method is very troublesome for users.
  • the solution of the embodiment of the present invention automatically adjusts the opacity of the AR/MR device according to the state of the user's eyes, which is very convenient for the user and improves the user experience.
  • the device further includes an opacity regression model training module 4.
  • the regression model training module 4 may be an offline module, which specifically includes an eye state calibration unit, a training parameter acquisition unit, and eye feature parameter standardization. Unit and regression model training unit, where the eye state calibration unit is configured to obtain the reference state of the eye of the experiment wearer. It can be understood that the experiment wearer is the participant used to obtain the training data, which can be a specific one or If it is multiple, if it is a specific one, an opacity regression model matching the specific user can be obtained. If there are multiple experimental wearers, an opacity regression model suitable for the general public can be obtained.
  • the training parameter acquisition unit is configured to use different light intensities as the test environment to acquire the corresponding device parameters and the eye state characteristic parameters of the experimental wearer under each light intensity, which makes the experiment wearer in the most comfortable state, among which, different controllable Adjust the light intensity to set the corresponding test environment.
  • the eye feature parameter standardization unit is configured to standardize the eye state feature parameters of the experimental wearer based on the eye reference state of the experimental wearer, where the eye reference state of the experimental wearer is the user's eye reference state in the most natural state. The opening and closing degree and pupil size are standardized based on this, and the model accuracy is improved.
  • the regression model training unit is configured to perform regression model training with light intensity and corresponding standardized eye state parameters as input and corresponding device parameters as output to obtain the opacity regression model.
  • the regression model training module 4 is performing the model During the training, under different lighting conditions, the user adjusts the discoloration degree of the electrochromic film of the device, the degree of optical diffusion of the liquid crystal film and the optical power level to the most comfortable state for the user. Then, the eye opening and closing degree and pupil size are detected, and the light intensity sensor measures the light intensity.
  • the light intensity, eye opening (after normalization), pupil size (after normalization) and the degree of discoloration of the electrochromic film, the current state of the liquid crystal film's optical slowness and optical power level are recorded into the internal memory as a single data point.
  • the opacity adjustment module 3 adjusts the corresponding AR/MR hardware device parameter based on the target value of the device parameter to adjust the opacity of the AR/MR device.
  • the hardware settings can be updated regularly and can be overwritten through public APIs.
  • the opacity adjustment module 3 obtains three indicators from the target value output module 2 to the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level. Then a corresponding signal is applied to each film layer to simultaneously drive the electrochromic film and the liquid crystal film and the optical power of the opto-mechanical to a desired state.
  • the time interval for updating hardware settings is configurable.
  • the API access provided to developers allows regular updates and allows locking or adjusting the opaque state.
  • API level control can be accessed programmatically through applications, including but not limited to video games, web applications and utilities, or directly accessed by users through applications such as the control panel.
  • the embodiment of the present invention also includes an AR/MR device, including the opacity control device for the AR/MR device described in the embodiment of the present invention.
  • An embodiment of the present invention also provides a controller, which includes a memory and a processor, the memory stores a computer program, and when the program is executed by the processor, the steps of the opacity control method can be implemented.
  • the embodiment of the present invention also provides a computer-readable storage medium for storing a computer program, which, when executed by a computer or a processor, implements the steps of the opacity control method.
  • the opacity degree of the AR/MR device is adaptively determined according to the current environment by recognizing and judging the user's eye state and calling the opacity regression model. Specifically, the degree of discoloration of the electrochromic film can be used, and the liquid crystal film is optically diffused. The opacity of AR/MR equipment is controlled by the degree of radiation and optical power level.
  • the opacity control system based on the eye state eliminates the user's manual operation, and dynamically adjusts the opacity of the AR/MR device according to the wearer's eye state to minimize the impact of visual noise in the external environment on the user , Improve the visibility and distinguishability of the information displayed by the AR/MR device in different external environments, and enhance the user experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Human Computer Interaction (AREA)
  • Eye Examination Apparatus (AREA)
  • Rehabilitation Tools (AREA)

Abstract

Provided are an opacity control method and apparatus, and AR/MR device, a controller, and a medium. The method comprises: acquiring the current light intensity and eye state characteristic parameters of a wearer (S1); inputting the current light intensity and the eye state characteristic parameters of the wearer into a pre-trained opacity regression model, so as to output device parameter target values (S2); and adjusting the corresponding device parameters on the basis of the device parameter target values, so as to adjust the opacity of an AR/MR device (S3). By means of the method, the opacity of an AR/MR device can be self-adaptively adjusted on the basis of the opacity regression model according to the most natural state and reaction of a user's eyes, such that the user experience is improved.

Description

不透明度控制方法、装置以及AR/MR设备、控制器和介质Opacity control method, device, AR/MR equipment, controller and medium 技术领域Technical field
本发明涉及AR/MR技术领域，尤其涉及一种不透明度控制方法、装置以及AR/MR设备、控制器和介质。The present invention relates to the field of AR/MR technology, and in particular to an opacity control method, device, AR/MR equipment, controller and medium.
背景技术Background technique
近年来，增强现实(AR)和混合现实(MR)是新兴作为技术，成为多个行业的发展重点。这类技术以用户可佩戴为特点，并且以一副眼镜或帽子等类似的形势呈现。AR或MR设备能够在用户的视力范围内覆盖数字图像，提供一定程度的实用性或娱乐性。In recent years, augmented reality (AR) and mixed reality (MR) are emerging technologies that have become the development focus of multiple industries. This type of technology is characterized by being wearable by the user and presented in a similar situation such as a pair of glasses or a hat. AR or MR devices can cover digital images within the user's vision, providing a certain degree of practicality or entertainment.
当外部存在视觉干扰或者用户处于明亮的环境中，AR或MR设备看到的外部视觉噪声会降低用户体验或阻碍一些操作。若通过向AR或MR设备添加不透明度控制，可以在不同的外部环境中提高设备显示的信息的可见性和可辨别性。但是现有的技术方案中并没有基于用户的使用状态来自动的调整***的不透明度，而是仅仅允许***直接来调节不透明度，但这种调节的方式和用户当前的状态并无直接的关系，这会极大的影响用户的体验。When there is external visual interference or the user is in a bright environment, the external visual noise seen by the AR or MR device will reduce the user experience or hinder some operations. By adding opacity control to AR or MR devices, the visibility and distinguishability of the information displayed by the device can be improved in different external environments. However, the existing technical solution does not automatically adjust the opacity of the system based on the user's use state, but only allows the system to directly adjust the opacity, but this adjustment method has no direct relationship with the current state of the user , This will greatly affect the user experience.
发明内容Summary of the invention
本发明目的在于，提供一种不透明度控制方法、装置以及AR/MR设备、控制器和介质，根据用户眼睛最自然的状态和反应，基于不透明度回归模型来自适应调整AR/MR设备的不透明度，提升了用户体验。The purpose of the present invention is to provide an opacity control method, device, AR/MR equipment, controller and medium, which can adaptively adjust the opacity of AR/MR equipment based on the opacity regression model according to the most natural state and reaction of the user’s eyes , Improve the user experience.
为了解决上述技术问题，根据本发明一方面，提供了一种不透明度控制方法，用于AR/MR设备，包括：In order to solve the above technical problems, according to one aspect of the present invention, an opacity control method for AR/MR equipment is provided, including:
获取当前光强度和佩戴者的眼睛状态特征参数；Obtain the current light intensity and the characteristic parameters of the wearer's eye condition;
将所述当前光强度和佩戴者的眼睛状态特征参数输入预先训练的不透明度回归模型中，输出设备参数目标值；Input the current light intensity and the characteristic parameters of the wearer's eye state into the pre-trained opacity regression model, and output the target value of the device parameter;
基于所述设备参数目标值调整对应的设备参数，以调整AR/MR设备的不透明度。The corresponding device parameter is adjusted based on the target value of the device parameter to adjust the opacity of the AR/MR device.
进一步的，还包括，训练所述不透明度回归模型，具体包括：Further, it also includes training the opacity regression model, which specifically includes:
获取实验佩戴者的眼睛参考状态；Obtain the eye reference state of the experiment wearer;
以不同的光强度作为测试环境，获取每一光强度下，使得实验佩戴者处于最舒适状态下对应的设备参数以及实验佩戴者的眼睛状态特征参数；Use different light intensities as the test environment to obtain the corresponding equipment parameters and the characteristic parameters of the eye condition of the experiment wearer under each light intensity, which makes the experiment wearer in the most comfortable state;
基于所述实验佩戴者的眼睛参考状态将实验佩戴者的眼睛状态特征参数标准化；Standardize the characteristic parameters of the eye condition of the experiment wearer based on the eye reference condition of the experiment wearer;
将光强度、对应的标准化后的眼睛状态参数作为输入、将对应的设备参数作为输出进行回归模型训练，得到所述不透明度回归模型。The light intensity and the corresponding normalized eye state parameters are used as input, and the corresponding device parameters are used as output to perform regression model training to obtain the opacity regression model.
进一步的，所述眼睛状态特征参数包括眼镜开合度和瞳孔大小。Further, the characteristic parameters of the eye state include the opening and closing degree of the glasses and the pupil size.
进一步的，所述设备参数包括电致变色薄膜变色程度、液晶薄膜光学漫射程度和光机光学功率水平中的一种或多种，其中，所述电致变色薄膜和液晶薄膜安装在AR/MR设备的前保护盖内。Further, the device parameters include one or more of the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optomechanics, wherein the electrochromic film and the liquid crystal film are installed in AR/MR Inside the front protective cover of the device.
根据本发明另一方面，提供了一种的不透明度控制装置，用于AR/MR设备，包括：According to another aspect of the present invention, there is provided an opacity control device for AR/MR equipment, including:
参数获取模块，配置为获取当前光强度和佩戴者的眼睛状态特征参数；The parameter acquisition module is configured to acquire the current light intensity and characteristic parameters of the wearer's eye condition;
目标值输出模块，配置为将所述当前光强度和佩戴者的眼睛状态特征参数输入预先训练的不透明度回归模型中，输出设备参数目标值；The target value output module is configured to input the current light intensity and the characteristic parameters of the wearer's eye state into a pre-trained opacity regression model, and output the target value of the device parameter;
不透明度调整模块，配置为基于所述设备参数目标值调整对应的设备参数，以调整AR/MR设备的不透明度。The opacity adjustment module is configured to adjust the corresponding device parameter based on the target value of the device parameter to adjust the opacity of the AR/MR device.
进一步的，还包括不透明度回归模型训练模块，具体包括：Furthermore, it also includes an opacity regression model training module, which specifically includes:
眼睛状态校准单元，配置为获取实验佩戴者的眼睛参考状态；The eye state calibration unit is configured to obtain the eye reference state of the experiment wearer;
训练参数获取单元，配置为以不同的光强度作为测试环境，获取每一光强度下，使得实验佩戴者处于最舒适状态下对应的设备参数以及实验佩戴者的眼睛状态特征参数；The training parameter acquisition unit is configured to use different light intensities as the test environment to acquire the corresponding equipment parameters and the characteristic parameters of the eye condition of the experiment wearer under each light intensity, which makes the experiment wearer in the most comfortable state;
眼睛特征参数标准化单元，配置为基于所述实验佩戴者的眼睛参考状态将实验佩戴者的眼睛状态特征参数标准化；An eye feature parameter standardization unit configured to standardize the eye state feature parameters of the experiment wearer based on the eye reference state of the experiment wearer;
回归模型训练单元，配置为将光强度、对应的标准化后的眼睛状态参数作为输入、将对应的设备参数作为输出进行回归模型训练，得到所述不透明度回归模型。The regression model training unit is configured to perform regression model training with light intensity and corresponding standardized eye state parameters as input and corresponding device parameters as output to obtain the opacity regression model.
进一步的，所述参数获取模块包括：Further, the parameter acquisition module includes:
眼部追踪传感器，用于获取眼睛状态特征参数，所述眼睛状态特征参数包括眼镜开合度和瞳孔大小；An eye tracking sensor for acquiring characteristic parameters of the eye state, the characteristic parameters of the eye state including the opening and closing degree of the glasses and the pupil size;
光强传感器，用于获取当前光强度。The light intensity sensor is used to obtain the current light intensity.
进一步的，所述设备参数包括电致变色薄膜变色程度、液晶薄膜光学漫射程度和光机光学功率水平中的一种或多种，其中，所述电致变色薄膜和液晶薄膜安装在AR/MR设备的前保护盖内。Further, the device parameters include one or more of the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optomechanics, wherein the electrochromic film and the liquid crystal film are installed in AR/MR Inside the front protective cover of the device.
根据本发明又一方面，提供一种AR/MR设备，包括所述用于AR/MR设备的不透明度控制装置。According to another aspect of the present invention, an AR/MR device is provided, including the opacity control device for AR/MR device.
根据本发明又一方面，提供一种控制器，其包括存储器与处理器，所述存储器存储有计算机程序，所述程序在被所述处理器执行时能够实现所述方法的步骤。According to another aspect of the present invention, a controller is provided, which includes a memory and a processor, the memory stores a computer program, and the program can implement the steps of the method when executed by the processor.
根据本发明又一方面，提供一种计算机可读存储介质，用于存储计算 机程序，所述程序在由一计算机或处理器执行时实现所述方法的步骤。According to another aspect of the present invention, a computer-readable storage medium is provided for storing a computer program, which when executed by a computer or a processor, implements the steps of the method.
本发明与现有技术相比具有明显的优点和有益效果。借由上述技术方案，本发明提供了一种不透明度控制方法、装置以及AR/MR设备、控制器和介质可达到相当的技术进步性及实用性，并具有产业上的广泛利用价值，其至少具有下列优点：Compared with the prior art, the present invention has obvious advantages and beneficial effects. With the above technical solutions, the present invention provides an opacity control method, device, AR/MR equipment, controller, and medium that can achieve considerable technological advancement and practicability, and has a wide range of industrial use value, at least Has the following advantages:
本发明通过对用户眼睛状态的识别和判断，并调用不透明度回归模型来根据当前环境自适应的决定AR/MR设备的不透明程度，具体可通过电致变色薄膜变色程度，液晶薄膜光学漫射程度和光学功率水平等来控制AR/MR设备的不透明度。通过基于眼睛状态的不透明度控制***省去了佩戴用户人为的操作，根据佩戴者的眼睛状态来动态调整AR/MR设备的不透明程度，以让外界环境的视觉噪声对用户的影响水平降到最低，在不同的外部环境中提高AR/MR设备显示的信息的可见性和可辨别性，提升了用户体验。The invention uses the recognition and judgment of the user's eye state and calls the opacity regression model to adaptively determine the opacity degree of the AR/MR device according to the current environment. Specifically, the degree of discoloration of the electrochromic film and the degree of optical diffusion of the liquid crystal film can be used. And optical power level to control the opacity of AR/MR equipment. The opacity control system based on the eye state eliminates the user's manual operation, and dynamically adjusts the opacity of the AR/MR device according to the wearer's eye state to minimize the impact of visual noise in the external environment on the user , Improve the visibility and distinguishability of the information displayed by the AR/MR device in different external environments, and enhance the user experience.
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其他目的、特征和优点能够更明显易懂，以下特举较佳实施例，并配合附图，详细说明如下。The above description is only an overview of the technical solution of the present invention. In order to understand the technical means of the present invention more clearly, it can be implemented in accordance with the content of the specification, and in order to make the above and other objectives, features and advantages of the present invention more obvious and understandable. In the following, the preferred embodiments are cited in conjunction with the drawings, and the detailed description is as follows.
附图说明Description of the drawings
图1为本发明一实施例提供不透明度控制方法示意图；FIG. 1 is a schematic diagram of an opacity control method provided by an embodiment of the present invention;
图2(a)为透过电致变色薄膜薄膜的效果图；Figure 2 (a) is the effect diagram of the electrochromic film through the thin film;
图2(b)为透过光慢射液晶薄膜的效果图；Figure 2(b) is an effect diagram of a slow-transmitting liquid crystal film through light;
图3为本发明一实施例提供的电致变色薄膜和液晶薄膜安装在AR/MR设备示意图；3 is a schematic diagram of the electrochromic film and the liquid crystal film provided in an embodiment of the present invention installed in an AR/MR device;
图4为本发明一实施例提供的不透明度控制装置示意图；4 is a schematic diagram of an opacity control device provided by an embodiment of the present invention;
图5为本发明另一实施例提供的不透明度控制装置示意图。FIG. 5 is a schematic diagram of an opacity control device provided by another embodiment of the present invention.
【符号说明】【Symbol Description】
1：参数获取模块    2：目标值输出模块1: Parameter acquisition module 2: Target value output module
3：透明度调整模块  4：不透明度回归模型训练模块3: Transparency adjustment module 4: Opacity regression model training module
具体实施方式Detailed ways
为更进一步阐述本发明为达成预定发明目的所采取的技术手段及功效，以下结合附图及较佳实施例，对依据本发明提出的一种不透明度控制方法、装置以及AR/MR设备、控制器和介质的具体实施方式及其功效，详细说明如后。In order to further explain the technical means and effects of the present invention to achieve the intended purpose of the invention, in conjunction with the accompanying drawings and preferred embodiments, a method and device for opacity control and AR/MR equipment and control according to the present invention will be described below. The specific implementation of the device and the medium and their effects are described in detail later.
本发明实施例提供了一种不透明度控制方法，用于AR/MR设备，如图1所示，包括以下步骤：The embodiment of the present invention provides an opacity control method for AR/MR equipment, as shown in FIG. 1, including the following steps:
步骤S1、获取当前光强度和佩戴者的眼睛状态特征参数；Step S1: Obtain current light intensity and characteristic parameters of the wearer's eye condition;
步骤S2、将所述当前光强度和佩戴者的眼睛状态特征参数输入预先训练的不透明度回归模型中，输出设备参数目标值；Step S2: Input the current light intensity and the characteristic parameters of the wearer's eye condition into the pre-trained opacity regression model, and output the target value of the device parameter;
步骤S3、基于所述设备参数目标值调整对应的设备参数，以调整AR/MR设备的不透明度。Step S3: Adjust the corresponding device parameter based on the target value of the device parameter to adjust the opacity of the AR/MR device.
本发明实施例所述方法通过对用户眼睛状态的识别和判断，并调用不透明度回归模型来根据当前环境自适应的决定AR/MR设备的不透明程度，省去了佩戴用户人为的操作，根据佩戴者的眼睛状态来动态调整AR/MR设备的不透明程度，以让外界环境的视觉噪声对用户的影响水平降到最低，在不同的外部环境中提高AR/MR设备显示的信息的可见性和可辨别性，提升了用户体验。The method described in the embodiment of the present invention uses the recognition and judgment of the user’s eye state and calls the opacity regression model to adaptively determine the opacity of the AR/MR device according to the current environment, eliminating the need for the user’s manual operation. The opacity of the AR/MR device can be dynamically adjusted by the user’s eye status to minimize the impact of visual noise in the external environment on the user, and to improve the visibility and visibility of the information displayed by the AR/MR device in different external environments. Distinguishability improves user experience.
AR设备包括AR眼镜等设备，MR设备包括MR眼镜等设备。AR equipment includes equipment such as AR glasses, and MR equipment includes equipment such as MR glasses.
作为一种示例，步骤S1中，可采用眼部追踪传感器获取眼睛状态特征参数，所述眼睛状态特征参数可包括眼镜开合度和瞳孔大小。具体可采用haar-like特征定位眼睛，然后应用霍夫变换来检测瞳孔大小。眼睛开合度用来表示眼睛张开的程度，瞳孔大小通过瞳孔的直径来表征。需要说明的是，上述获取眼睛状态特征参数的方法仅为一种示例，还有其他可用于定位眼睛、测量瞳孔大小、并量化眼睛开合度的方法也可用于此。可采用光强传感器获取当前光强度。As an example, in step S1, an eye tracking sensor may be used to obtain the characteristic parameters of the eye state, and the characteristic parameters of the eye state may include the opening and closing degree of the glasses and the pupil size. Specifically, haar-like features can be used to locate the eyes, and then the Hough transform can be applied to detect the pupil size. The degree of eye opening and closing is used to indicate the degree of eye opening, and the pupil size is characterized by the diameter of the pupil. It should be noted that the above method for obtaining the characteristic parameters of the eye state is only an example, and there are other methods that can be used to locate the eye, measure the pupil size, and quantify the opening and closing degree of the eye can also be used for this. The light intensity sensor can be used to obtain the current light intensity.
电致变色薄膜和光慢射液晶薄膜允许光在其表面进行衰减和慢射，透过电致变色薄膜薄膜的效果如图2(a)所示，透过光慢射液晶薄膜的效果图如图2(b)所示。通过对光的衰减和漫射将佩戴者与其外部环境隔离开来，这将提升在AR/MR设备上的用户体验。这些薄膜体积小可以安装在AR或MR设备内，使得用户透过它们观察外部环境。如图3所示示例，可将所述电致变色薄膜和液晶薄膜安装在AR/MR设备的前保护盖内，对应的，所述设备参数包括电致变色薄膜变色程度、液晶薄膜光学漫射程度和光机光学功率水平中的一种或多种。在图3中，可通过控制连接到每层的电极对控制薄膜状态。现有技术中AR/MR设备的不透明度控制还都是手动方式，这种手动控制的方式对用户来说是十分麻烦的。本发明实施例的方案根据用户眼睛的状态来自动调整AR/MR设备的不透明度，对用户来说是十分方便的，提升了户用体验。The electrochromic film and the light slow-transmitting liquid crystal film allow light to be attenuated and slowed on its surface. The effect of the electrochromic thin film is shown in Figure 2(a), and the effect diagram of the slow-transmitting liquid crystal film is shown in Figure 2(a). As shown in 2(b). The attenuation and diffusion of light isolates the wearer from the external environment, which will enhance the user experience on AR/MR devices. These films are small in size and can be installed in AR or MR devices, allowing users to observe the external environment through them. As shown in the example shown in Figure 3, the electrochromic film and the liquid crystal film can be installed in the front protective cover of the AR/MR device. Correspondingly, the device parameters include the degree of discoloration of the electrochromic film and the optical diffusion of the liquid crystal film. One or more of the degree and the optical power level of the optical machine. In Figure 3, the state of the film can be controlled by controlling the pair of electrodes connected to each layer. The opacity control of the AR/MR device in the prior art is still manual, and this manual control method is very troublesome for users. The solution of the embodiment of the present invention automatically adjusts the opacity of the AR/MR device according to the state of the user's eyes, which is very convenient for the user and improves the user experience.
作为一种示例，所述方法还包括步骤S4、训练所述不透明度回归模型，具体包括：As an example, the method further includes step S4, training the opacity regression model, which specifically includes:
步骤S41、获取实验佩戴者的眼睛参考状态。Step S41: Obtain the eye reference state of the experiment wearer.
可以理解的是，实验佩戴者即用于获取训练数据的参与者，可以为特定的一个，也可为多个，若为特定的一个，可得到与该特定用户匹配的不 透明度回归模型。若实验佩戴者为多个，可得到适用于大众的不透明度回归模型。It is understandable that the experiment wearer is the participant used to obtain the training data. It can be a specific one or multiple. If it is a specific one, an opacity regression model matching the specific user can be obtained. If there are multiple experimental wearers, an opacity regression model suitable for the general public can be obtained.
步骤S42、以不同的光强度作为测试环境，获取每一光强度下，使得实验佩戴者处于最舒适状态下对应的设备参数以及实验佩戴者的眼睛状态特征参数。Step S42: Use different light intensities as the test environment, and obtain the corresponding device parameters and the characteristic parameters of the eye condition of the test wearer under each light intensity to make the test wearer in the most comfortable state.
其中，可控制不同的照明条件，调节光强度，从而设置对应的测试环境。Among them, different lighting conditions can be controlled, the light intensity can be adjusted, and the corresponding test environment can be set.
步骤S43、基于所述实验佩戴者的眼睛参考状态将实验佩戴者的眼睛状态特征参数标准化。Step S43: Standardize the characteristic parameters of the eye state of the experiment wearer based on the reference state of the eye of the experiment wearer.
其中，实验佩戴者的眼睛参考状态，即用户的最自然状态时对应的眼睛的开合度和瞳孔尺寸，基于此进行标准化，提升了型准确度。Among them, the reference state of the eyes of the experimental wearer, that is, the opening and closing degree and pupil size of the corresponding eyes of the user in the most natural state, are standardized based on this, and the model accuracy is improved.
步骤S44、将光强度、对应的标准化后的眼睛状态参数作为输入、将对应的设备参数作为输出进行回归模型训练，得到所述不透明度回归模型。Step S44: Use the light intensity and the corresponding standardized eye state parameters as input, and use the corresponding device parameters as output to perform regression model training to obtain the opacity regression model.
以所述眼睛状态特征参数可包括眼镜开合度和瞳孔大小，所述设备参数包括电致变色薄膜变色程度、液晶薄膜光学漫射程度和光机光学功率水平为例，执行所述步骤S4过程中，在不同的照明条件下，用户将设备的电致变色薄膜变色程度，液晶薄膜光学漫射程度和光学功率水平调节到该用户的最舒适状态。然后，检测眼睛开合度和瞳孔大小，并且光强度传感器测量光强度。将光强度，眼睛开合度(标准化后)，瞳孔大小(标准化后)和电致变色薄膜变色程度，液晶薄膜光学慢射程度和光学功率水平的当前状态作为单个数据点记录到内部存储器。对收集到的不同用户的数据进行回归模型训练。这种模型将每个数据点的光强度，眼睛开合度(标准化后)和瞳孔大小(标准化后)作为输入，并且将每个数据点的电致变色薄膜变色程度，液晶薄膜光学漫射程度和光学功率水平作为输出，得到不透明度回归模型。Taking the characteristic parameters of the eye condition may include the degree of eyeglass opening and closing and pupil size, and the device parameters include the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optical machine as an example, in the process of performing step S4, Under different lighting conditions, the user adjusts the discoloration degree of the electrochromic film of the device, the degree of optical diffusion of the liquid crystal film and the optical power level to the most comfortable state for the user. Then, the eye opening and closing degree and pupil size are detected, and the light intensity sensor measures the light intensity. The light intensity, eye opening (after normalization), pupil size (after normalization) and the degree of discoloration of the electrochromic film, the current state of the liquid crystal film's optical slowness and optical power level are recorded into the internal memory as a single data point. Perform regression model training on the collected data of different users. This model takes the light intensity of each data point, the degree of eye opening (after normalization) and pupil size (after normalization) as input, and the degree of discoloration of the electrochromic film at each data point, the degree of optical diffusion of the liquid crystal film, and the The optical power level is taken as the output, and the opacity regression model is obtained.
本发明实施例还提供了一种的不透明度控制装置，用于AR/MR设备，如图4所示，包括参数获取模块1、目标值输出模块2和不透明度调整模块3，其中，参数获取模块1配置为获取当前光强度和佩戴者的眼睛状态特征参数；目标值输出模块2配置为将所述当前光强度和佩戴者的眼睛状态特征参数输入预先训练的不透明度回归模型中，输出设备参数目标值；不透明度调整模块3配置为基于所述设备参数目标值调整对应的设备参数，以调整AR/MR设备的不透明度。本发明实施例所述装置通过对用户眼睛状态的识别和判断，并调用不透明度回归模型来根据当前环境自适应的决定AR/MR设备的不透明程度，省去了佩戴用户人为的操作，根据佩戴者的眼睛状态来动态调整AR/MR设备的不透明程度，以让外界环境的视觉噪声对用户的影响水平降到最低，在不同的外部环境中提高AR/MR设备显示的信 息的可见性和可辨别性，提升了用户体验。The embodiment of the present invention also provides an opacity control device for AR/MR equipment, as shown in FIG. 4, including a parameter acquisition module 1, a target value output module 2 and an opacity adjustment module 3, wherein the parameter acquisition The module 1 is configured to obtain the current light intensity and the wearer’s eye state characteristic parameters; the target value output module 2 is configured to input the current light intensity and the wearer’s eye state characteristic parameters into the pre-trained opacity regression model, and the output device Parameter target value; the opacity adjustment module 3 is configured to adjust the corresponding device parameter based on the device parameter target value to adjust the opacity of the AR/MR device. The device described in the embodiment of the present invention adaptively determines the opacity of the AR/MR device according to the current environment by recognizing and judging the state of the user’s eyes and calling the opacity regression model, eliminating the need for the user’s manual operation. The opacity of the AR/MR device can be dynamically adjusted by the user’s eye status to minimize the impact of visual noise in the external environment on the user, and to improve the visibility and visibility of the information displayed by the AR/MR device in different external environments. Distinguishability improves user experience.
作为一种示例，所述参数获取模块1包括眼部追踪传感器和光强传感器，其中，眼部追踪传感器用于获取眼睛状态特征参数，所述眼睛状态特征参数包括眼镜开合度和瞳孔大小；光强传感器用于获取当前光强度。As an example, the parameter acquisition module 1 includes an eye tracking sensor and a light intensity sensor, where the eye tracking sensor is used to acquire eye state characteristic parameters, and the eye state characteristic parameters include glasses opening and closing degree and pupil size; The intensity sensor is used to obtain the current light intensity.
电致变色薄膜和光慢射液晶薄膜允许光在其表面进行衰减和慢射，透过电致变色薄膜薄膜的效果如图2(a)所示，透过光慢射液晶薄膜的效果图如图2(b)所示。通过对光的衰减和漫射将佩戴者与其外部环境隔离开来，这将提升在AR/MR设备上的用户体验。这些薄膜体积小可以安装在AR或MR设备内，使得用户透过它们观察外部环境。如图3所示示例，可将所述电致变色薄膜和液晶薄膜安装在AR/MR设备的前保护盖内，对应的，所述设备参数包括电致变色薄膜变色程度、液晶薄膜光学漫射程度和光机光学功率水平中的一种或多种。在图3中，可通过控制连接到每层的电极对控制薄膜状态。现有技术中AR/MR设备的不透明度控制还都是手动方式，这种手动控制的方式对用户来说是十分麻烦的。本发明实施例的方案根据用户眼睛的状态来自动调整AR/MR设备的不透明度，对用户来说是十分方便的，提升了户用体验。The electrochromic film and the light slow-transmitting liquid crystal film allow light to be attenuated and slowed on its surface. The effect of the electrochromic thin film is shown in Figure 2(a), and the effect diagram of the slow-transmitting liquid crystal film is shown in Figure 2(a). As shown in 2(b). The attenuation and diffusion of light isolates the wearer from the external environment, which will enhance the user experience on AR/MR devices. These films are small in size and can be installed in AR or MR devices, allowing users to observe the external environment through them. As shown in the example shown in Figure 3, the electrochromic film and the liquid crystal film can be installed in the front protective cover of the AR/MR device. Correspondingly, the device parameters include the degree of discoloration of the electrochromic film and the optical diffusion of the liquid crystal film. One or more of the degree and the optical power level of the optical machine. In Figure 3, the state of the film can be controlled by controlling the pair of electrodes connected to each layer. The opacity control of the AR/MR device in the prior art is still manual, and this manual control method is very troublesome for users. The solution of the embodiment of the present invention automatically adjusts the opacity of the AR/MR device according to the state of the user's eyes, which is very convenient for the user and improves the user experience.
作为一种示例，如图5所示，所述装置还包括不透明度回归模型训练模块4，回归模型训练模块4可为离线模块，具体包括眼睛状态校准单元、训练参数获取单元、眼睛特征参数标准化单元和回归模型训练单元，其中，眼睛状态校准单元配置为获取实验佩戴者的眼睛参考状态，可以理解的是，实验佩戴者即用于获取训练数据的参与者，可以为特定的一个，也可为多个，若为特定的一个，可得到与该特定用户匹配的不透明度回归模型。若实验佩戴者为多个，可得到适用于大众的不透明度回归模型。训练参数获取单元配置为以不同的光强度作为测试环境，获取每一光强度下，使得实验佩戴者处于最舒适状态下对应的设备参数以及实验佩戴者的眼睛状态特征参数，其中，可控制不同的照明条件，调节光强度，从而设置对应的测试环境。眼睛特征参数标准化单元，配置为基于所述实验佩戴者的眼睛参考状态将实验佩戴者的眼睛状态特征参数标准化，其中，实验佩戴者的眼睛参考状态，即用户的最自然状态时对应的眼睛的开合度和瞳孔尺寸，基于此进行标准化，提升了型准确度。回归模型训练单元，配置为将光强度、对应的标准化后的眼睛状态参数作为输入、将对应的设备参数作为输出进行回归模型训练，得到所述不透明度回归模型。As an example, as shown in FIG. 5, the device further includes an opacity regression model training module 4. The regression model training module 4 may be an offline module, which specifically includes an eye state calibration unit, a training parameter acquisition unit, and eye feature parameter standardization. Unit and regression model training unit, where the eye state calibration unit is configured to obtain the reference state of the eye of the experiment wearer. It can be understood that the experiment wearer is the participant used to obtain the training data, which can be a specific one or If it is multiple, if it is a specific one, an opacity regression model matching the specific user can be obtained. If there are multiple experimental wearers, an opacity regression model suitable for the general public can be obtained. The training parameter acquisition unit is configured to use different light intensities as the test environment to acquire the corresponding device parameters and the eye state characteristic parameters of the experimental wearer under each light intensity, which makes the experiment wearer in the most comfortable state, among which, different controllable Adjust the light intensity to set the corresponding test environment. The eye feature parameter standardization unit is configured to standardize the eye state feature parameters of the experimental wearer based on the eye reference state of the experimental wearer, where the eye reference state of the experimental wearer is the user's eye reference state in the most natural state. The opening and closing degree and pupil size are standardized based on this, and the model accuracy is improved. The regression model training unit is configured to perform regression model training with light intensity and corresponding standardized eye state parameters as input and corresponding device parameters as output to obtain the opacity regression model.
以所述眼睛状态特征参数可包括眼镜开合度和瞳孔大小，所述设备参数包括电致变色薄膜变色程度、液晶薄膜光学漫射程度和光机光学功率水平为例，回归模型训练模块4在进行模型训练过程中，在不同的照明条件下，用户将设备的电致变色薄膜变色程度，液晶薄膜光学漫射程度和光学 功率水平调节到该用户的最舒适状态。然后，检测眼睛开合度和瞳孔大小，并且光强度传感器测量光强度。将光强度，眼睛开合度(标准化后)，瞳孔大小(标准化后)和电致变色薄膜变色程度，液晶薄膜光学慢射程度和光学功率水平的当前状态作为单个数据点记录到内部存储器。对收集到的不同用户的数据进行回归模型训练。这种模型将每个数据点的光强度，眼睛开合度(标准化后)和瞳孔大小(标准化后)作为输入，并且将每个数据点的电致变色薄膜变色程度，液晶薄膜光学漫射程度和光学功率水平作为输出，得到不透明度回归模型。Taking the eye condition feature parameters including the opening and closing degree of glasses and pupil size, and the device parameters including the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optical machine as an example, the regression model training module 4 is performing the model During the training, under different lighting conditions, the user adjusts the discoloration degree of the electrochromic film of the device, the degree of optical diffusion of the liquid crystal film and the optical power level to the most comfortable state for the user. Then, the eye opening and closing degree and pupil size are detected, and the light intensity sensor measures the light intensity. The light intensity, eye opening (after normalization), pupil size (after normalization) and the degree of discoloration of the electrochromic film, the current state of the liquid crystal film's optical slowness and optical power level are recorded into the internal memory as a single data point. Perform regression model training on the collected data of different users. This model takes the light intensity of each data point, the degree of eye opening (after normalization) and pupil size (after normalization) as input, and the degree of discoloration of the electrochromic film at each data point, the degree of optical diffusion of the liquid crystal film, and the The optical power level is taken as the output, and the opacity regression model is obtained.
不透明度调整模块3基于所述设备参数目标值调整对应的AR/MR的硬件设备参数，以调整AR/MR设备的不透明度。硬件设置可定期更新，可以通过公开的API进行覆盖。作为一种示例，不透明度调整模块3从目标值输出模块2获取到电致变色薄膜变色程度，液晶薄膜光学漫射程度和光学功率水平等三个指标。然后将对应的信号施加到每个膜层以同时驱动电致变色薄膜和液晶薄膜和光机光学功率到期望的状态。更新硬件设置的时间间隔是可配置的。提供给开发人员的API访问允许定时更新，允许锁定或调整不透明状态。API级别控制可以通过应用程序以编程方式访问，包括但不限于视频游戏，网络应用程序和实用程序，或者由用户通过诸如控制面板的应用程序直接访问。The opacity adjustment module 3 adjusts the corresponding AR/MR hardware device parameter based on the target value of the device parameter to adjust the opacity of the AR/MR device. The hardware settings can be updated regularly and can be overwritten through public APIs. As an example, the opacity adjustment module 3 obtains three indicators from the target value output module 2 to the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level. Then a corresponding signal is applied to each film layer to simultaneously drive the electrochromic film and the liquid crystal film and the optical power of the opto-mechanical to a desired state. The time interval for updating hardware settings is configurable. The API access provided to developers allows regular updates and allows locking or adjusting the opaque state. API level control can be accessed programmatically through applications, including but not limited to video games, web applications and utilities, or directly accessed by users through applications such as the control panel.
本发明实施例还包括一种AR/MR设备，包括本发明实施例所述的用于AR/MR设备的不透明度控制装置。The embodiment of the present invention also includes an AR/MR device, including the opacity control device for the AR/MR device described in the embodiment of the present invention.
本发明实施例还提供一种控制器，其包括存储器与处理器，所述存储器存储有计算机程序，所述程序在被所述处理器执行时能够实现所述不透明度控制方法的步骤。An embodiment of the present invention also provides a controller, which includes a memory and a processor, the memory stores a computer program, and when the program is executed by the processor, the steps of the opacity control method can be implemented.
本发明实施例还提供一种计算机可读存储介质，用于存储计算机程序，所述程序在由一计算机或处理器执行时实现所述不透明度控制方法的步骤。The embodiment of the present invention also provides a computer-readable storage medium for storing a computer program, which, when executed by a computer or a processor, implements the steps of the opacity control method.
本发明实施例通过对用户眼睛状态的识别和判断，并调用不透明度回归模型来根据当前环境自适应的决定AR/MR设备的不透明程度，具体可通过电致变色薄膜变色程度，液晶薄膜光学漫射程度和光学功率水平等来控制AR/MR设备的不透明度。通过基于眼睛状态的不透明度控制***省去了佩戴用户人为的操作，根据佩戴者的眼睛状态来动态调整AR/MR设备的不透明程度，以让外界环境的视觉噪声对用户的影响水平降到最低，在不同的外部环境中提高AR/MR设备显示的信息的可见性和可辨别性，提升了用户体验。In the embodiment of the present invention, the opacity degree of the AR/MR device is adaptively determined according to the current environment by recognizing and judging the user's eye state and calling the opacity regression model. Specifically, the degree of discoloration of the electrochromic film can be used, and the liquid crystal film is optically diffused. The opacity of AR/MR equipment is controlled by the degree of radiation and optical power level. The opacity control system based on the eye state eliminates the user's manual operation, and dynamically adjusts the opacity of the AR/MR device according to the wearer's eye state to minimize the impact of visual noise in the external environment on the user , Improve the visibility and distinguishability of the information displayed by the AR/MR device in different external environments, and enhance the user experience.
以上所述，仅是本发明的较佳实施例而已，并非对本发明作任何形式上的限制，虽然本发明已以较佳实施例揭露如上，然而并非用以限定本发明，任何熟悉本专业的技术人员，在不脱离本发明技术方案范围内，当可 利用上述揭示的技术内容作出些许更动或修饰为等同变化的等效实施例，但凡是未脱离本发明技术方案的内容，依据本发明的技术实质对以上实施例所作的任何简单修改、等同变化与修饰，均仍属于本发明技术方案的范围内。The above are only the preferred embodiments of the present invention and do not limit the present invention in any form. Although the present invention has been disclosed as the preferred embodiments, it is not intended to limit the present invention. Anyone familiar with the profession Those skilled in the art, without departing from the scope of the technical solution of the present invention, can use the technical content disclosed above to make slight changes or modification into equivalent embodiments with equivalent changes, but any content that does not deviate from the technical solution of the present invention, according to the present invention Any simple modifications, equivalent changes and modifications made to the above embodiments are still within the scope of the technical solutions of the present invention.

Claims (11)

  1. 一种不透明度控制方法，用于AR/MR设备，其特征在于，包括：An opacity control method for AR/MR equipment, characterized in that it includes:
    获取当前光强度和佩戴者的眼睛状态特征参数；Obtain the current light intensity and the characteristic parameters of the wearer's eye condition;
    将所述当前光强度和佩戴者的眼睛状态特征参数输入预先训练的不透明度回归模型中，输出设备参数目标值；Input the current light intensity and the characteristic parameters of the wearer's eye state into the pre-trained opacity regression model, and output the target value of the device parameter;
    基于所述设备参数目标值调整对应的设备参数，以调整AR/MR设备的不透明度。The corresponding device parameter is adjusted based on the target value of the device parameter to adjust the opacity of the AR/MR device.
  2. 根据权利要求1所述的不透明度控制方法，其特征在于，The opacity control method according to claim 1, wherein:
    还包括，训练所述不透明度回归模型，具体包括：It also includes training the opacity regression model, which specifically includes:
    获取实验佩戴者的眼睛参考状态；Obtain the eye reference state of the experiment wearer;
    以不同的光强度作为测试环境，获取每一光强度下，使得实验佩戴者处于最舒适状态下对应的设备参数以及实验佩戴者的眼睛状态特征参数；Use different light intensities as the test environment to obtain the corresponding equipment parameters and the characteristic parameters of the eye condition of the experiment wearer under each light intensity, which makes the experiment wearer in the most comfortable state;
    基于所述实验佩戴者的眼睛参考状态将实验佩戴者的眼睛状态特征参数标准化；Standardize the characteristic parameters of the eye condition of the experiment wearer based on the eye reference condition of the experiment wearer;
    将光强度、对应的标准化后的眼睛状态参数作为输入、将对应的设备参数作为输出进行回归模型训练，得到所述不透明度回归模型。The light intensity and the corresponding standardized eye state parameters are used as input, and the corresponding device parameters are used as output to perform regression model training to obtain the opacity regression model.
  3. 根据权利要求1或2所述的不透明度控制方法，其特征在于，The opacity control method according to claim 1 or 2, wherein:
    所述眼睛状态特征参数包括眼镜开合度和瞳孔大小。The characteristic parameters of the eye state include the opening and closing degree of the glasses and the pupil size.
  4. 根据权利要求3所述的不透明度控制方法，其特征在于，The opacity control method according to claim 3, wherein:
    所述设备参数包括电致变色薄膜变色程度、液晶薄膜光学漫射程度和光机光学功率水平中的一种或多种，其中，所述电致变色薄膜和液晶薄膜安装在AR/MR设备的前保护盖内。The device parameters include one or more of the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optical machine. The electrochromic film and the liquid crystal film are installed in front of the AR/MR device. Inside the protective cover.
  5. 一种的不透明度控制装置，用于AR/MR设备，其特征在于，包括：An opacity control device for AR/MR equipment, characterized in that it comprises:
    参数获取模块，配置为获取当前光强度和佩戴者的眼睛状态特征参数；The parameter acquisition module is configured to acquire the current light intensity and characteristic parameters of the wearer's eye condition;
    目标值输出模块，配置为将所述当前光强度和佩戴者的眼睛状态特征参数输入预先训练的不透明度回归模型中，输出设备参数目标值；The target value output module is configured to input the current light intensity and the characteristic parameters of the wearer's eye state into a pre-trained opacity regression model, and output target values of device parameters;
    不透明度调整模块，配置为基于所述设备参数目标值调整对应的设备参数，以调整AR/MR设备的不透明度。The opacity adjustment module is configured to adjust the corresponding device parameter based on the target value of the device parameter to adjust the opacity of the AR/MR device.
  6. 根据权利要求5所述的不透明度控制装置，其特征在于，The opacity control device according to claim 5, wherein:
    还包括不透明度回归模型训练模块，具体包括：It also includes the opacity regression model training module, including:
    眼睛状态校准单元，配置为获取实验佩戴者的眼睛参考状态；The eye state calibration unit is configured to obtain the eye reference state of the experiment wearer;
    训练参数获取单元，配置为以不同的光强度作为测试环境，获取每一光强度下，使得实验佩戴者处于最舒适状态下对应的设备参数以及实验佩戴者的眼睛状态特征参数；The training parameter acquisition unit is configured to use different light intensities as the test environment to acquire the corresponding equipment parameters and the characteristic parameters of the eye condition of the experimental wearer under each light intensity, which makes the experiment wearer in the most comfortable state;
    眼睛特征参数标准化单元，配置为基于所述实验佩戴者的眼睛参考状 态将实验佩戴者的眼睛状态特征参数标准化；An eye feature parameter standardization unit, configured to standardize the eye state feature parameters of the experiment wearer based on the reference state of the eye of the experiment wearer;
    回归模型训练单元，配置为将光强度、对应的标准化后的眼睛状态参数作为输入、将对应的设备参数作为输出进行回归模型训练，得到所述不透明度回归模型。The regression model training unit is configured to perform regression model training with light intensity and corresponding standardized eye state parameters as input and corresponding device parameters as output to obtain the opacity regression model.
  7. 根据权利要求5或6所述的不透明度控制装置，其特征在于，The opacity control device according to claim 5 or 6, wherein:
    所述参数获取模块包括：The parameter acquisition module includes:
    眼部追踪传感器，用于获取眼睛状态特征参数，所述眼睛状态特征参数包括眼镜开合度和瞳孔大小；An eye tracking sensor for acquiring characteristic parameters of the eye state, the characteristic parameters of the eye state including the opening and closing degree of the glasses and the pupil size;
    光强传感器，用于获取当前光强度。The light intensity sensor is used to obtain the current light intensity.
  8. 根据权利要求7所述的不透明度控制装置，其特征在于，The opacity control device according to claim 7, wherein:
    所述设备参数包括电致变色薄膜变色程度、液晶薄膜光学漫射程度和光机光学功率水平中的一种或多种，其中，所述电致变色薄膜和液晶薄膜安装在AR/MR设备的前保护盖内。The device parameters include one or more of the degree of discoloration of the electrochromic film, the degree of optical diffusion of the liquid crystal film, and the optical power level of the optical machine. The electrochromic film and the liquid crystal film are installed in front of the AR/MR device. Inside the protective cover.
  9. 一种AR/MR设备，其特征在于，包括权利要求5-8中任意一项所述用于AR/MR设备的不透明度控制装置。An AR/MR equipment, characterized by comprising the opacity control device for AR/MR equipment according to any one of claims 5-8.
  10. 一种控制器，其包括存储器与处理器，其特征在于，所述存储器存储有计算机程序，所述程序在被所述处理器执行时能够实现权利要求1至4中任意一项权利要求所述的方法的步骤。A controller, comprising a memory and a processor, wherein the memory stores a computer program, and when the program is executed by the processor, the program can implement any one of claims 1 to 4 Steps of the method.
  11. 一种计算机可读存储介质，用于存储计算机程序，其特征在于，所述程序在由一计算机或处理器执行时实现如权利要求1至4中任意一项权利要求所述的方法的步骤。A computer-readable storage medium for storing a computer program, wherein the program implements the steps of the method according to any one of claims 1 to 4 when the program is executed by a computer or a processor.
PCT/CN2021/072456 2020-01-20 2021-01-18 Opacity control method and apparatus, ar/mr device, controller, and medium WO2021147813A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010066325.5A CN111273776A (en) 2020-01-20 2020-01-20 Opacity control method, opacity control device, AR/MR device, controller and medium
CN202010066325.5 2020-01-20

Publications (1)

Publication Number Publication Date
WO2021147813A1 true WO2021147813A1 (en) 2021-07-29

Family

ID=70999007

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/072456 WO2021147813A1 (en) 2020-01-20 2021-01-18 Opacity control method and apparatus, ar/mr device, controller, and medium

Country Status (2)

Country Link
CN (1) CN111273776A (en)
WO (1) WO2021147813A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111273776A (en) * 2020-01-20 2020-06-12 芋头科技（杭州）有限公司 Opacity control method, opacity control device, AR/MR device, controller and medium
CN112130326B (en) * 2020-09-28 2022-06-10 广州引力波科技创新发展有限公司 Method and device for improving eye protection performance of VR glasses
CN112612153B (en) * 2021-02-23 2021-07-23 宁波圻亿科技有限公司 Optical imaging module and wearing device

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018197820A1 (en) * 2017-04-28 2018-11-01 Essilor International Method for determining a light filter applied to a spectacle lens, associated display device and virtual reality helmet
WO2019002416A1 (en) * 2017-06-30 2019-01-03 Essilor International Method for filter selection
CN109633907A (en) * 2019-01-09 2019-04-16 福建天晴数码有限公司 The method of simple eye AR glasses automatic brightness adjustment, storage medium
CN109643469A (en) * 2016-09-30 2019-04-16 英特尔公司 The structured content of augmented reality rendering
US20190290118A1 (en) * 2018-03-26 2019-09-26 Samsung Electronics Co., Ltd. Electronic device for monitoring health of eyes of user and method for operating the same
CN111273776A (en) * 2020-01-20 2020-06-12 芋头科技（杭州）有限公司 Opacity control method, opacity control device, AR/MR device, controller and medium
CN111347845A (en) * 2020-03-17 2020-06-30 北京百度网讯科技有限公司 Electrochromic glass adjusting method and device and electronic equipment

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8752963B2 (en) * 2011-11-04 2014-06-17 Microsoft Corporation See-through display brightness control
WO2014174067A1 (en) * 2013-04-25 2014-10-30 Essilor International (Compagnie Generale D'optique) A method of controlling a head mounted electro-optical device adapted to a wearer

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109643469A (en) * 2016-09-30 2019-04-16 英特尔公司 The structured content of augmented reality rendering
WO2018197820A1 (en) * 2017-04-28 2018-11-01 Essilor International Method for determining a light filter applied to a spectacle lens, associated display device and virtual reality helmet
WO2019002416A1 (en) * 2017-06-30 2019-01-03 Essilor International Method for filter selection
US20190290118A1 (en) * 2018-03-26 2019-09-26 Samsung Electronics Co., Ltd. Electronic device for monitoring health of eyes of user and method for operating the same
CN109633907A (en) * 2019-01-09 2019-04-16 福建天晴数码有限公司 The method of simple eye AR glasses automatic brightness adjustment, storage medium
CN111273776A (en) * 2020-01-20 2020-06-12 芋头科技（杭州）有限公司 Opacity control method, opacity control device, AR/MR device, controller and medium
CN111347845A (en) * 2020-03-17 2020-06-30 北京百度网讯科技有限公司 Electrochromic glass adjusting method and device and electronic equipment

Also Published As

Publication number Publication date
CN111273776A (en) 2020-06-12

Similar Documents

Publication Publication Date Title
WO2021147813A1 (en) Opacity control method and apparatus, ar/mr device, controller, and medium
US10389989B2 (en) Vision defect determination and enhancement using a prediction model
CN111511318B (en) Digital therapeutic correcting glasses
US20190304194A1 (en) Apparatus and method for augmenting sight
WO2021004138A1 (en) Screen display method, terminal device, and storage medium
CN110716639B (en) Electronic device with display operation based on eye movement
CN113362775B (en) Display screen control method and device, electronic equipment and storage medium
KR101803407B1 (en) User equipment and computer program stored in computer-readable medium for controlling the display
US11659987B2 (en) Vision testing via prediction-based setting of initial stimuli characteristics for user interface locations
JP2021530295A (en) A method of performing the astigmatism power test using a computing device having a screen for displaying an image related to the astigmatism power test and a corresponding computing device.
WO2018120928A1 (en) Smart virtual glasses control method based on conditions of eyeballs
KR101727615B1 (en) User device, server and computer program stored in computer-readable medium for determining vision information
CN110785693A (en) Apparatus, method and system for measuring ophthalmic lens design effects
US20220125299A1 (en) Vision testing via prediction-based setting of initial stimuli characteristics for user interface locations
US11776315B2 (en) Appearance based dominant eye selector systems and methods for gaze estimation from passive imagery
Liu et al. IntelliPupil: Pupillometric light modulation for optical see-through head-mounted displays
US11257468B2 (en) User-mountable extended reality (XR) device
WO2017026942A1 (en) Apparatus for display adjustment and method thereof
US20230089522A1 (en) Intelligent extended reality eyewear
KR20240070172A (en) Computer-Readable Recording Medium with Program for Age-related Macular Degeneration Patient and AR Glasses thereof
TW202345739A (en) A multifunctional wearable visual evaluation device for integrating image display and image capture
CN117481592A (en) Visual function checking method based on positive and negative relative adjustment of VR equipment and VR equipment
Langlotz et al. Computational Glasses: Repurposing Augmented Reality Glasses for Vision Assistance Jonathan Sutton

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21744200

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21744200

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21744200

Country of ref document: EP

Kind code of ref document: A1