CN116934584A

CN116934584A - Display method and electronic equipment

Info

Publication number: CN116934584A
Application number: CN202210348297.5A
Authority: CN
Inventors: 王松; 杨旭波; 叶剑南; 吕臻宇; 沈钢
Original assignee: Huawei Device Co Ltd
Current assignee: Huawei Device Co Ltd
Priority date: 2022-04-01
Filing date: 2022-04-01
Publication date: 2023-10-24

Abstract

A display method and an electronic device. The method comprises the following steps: displaying the N frames of images through a display device; the definition of a first object on an ith frame image in the N frames of images is the first definition; the definition of the first object on the j-th frame image is the second definition; the definition of the first object on the kth frame image is the third definition; the first definition is larger than the second definition, the second definition is smaller than the third definition, and i, j and k are positive integers smaller than N, i is smaller than j and smaller than k; the average value of the first definition and the second definition is a fourth definition, and the fourth definition is higher than the first threshold value; the average of the second definition and the third definition is a fifth definition, which is higher than the second threshold. In this way, the definition of the virtual environment seen by the user through the display device is higher, which helps to promote the virtual reality experience.

Description

Display method and electronic equipment

Technical Field

The present application relates to the field of electronic technologies, and in particular, to a display method and an electronic device.

Background

Virtual Reality (VR) technology is a man-machine interaction means created by means of computer and sensor technologies. VR technology integrates a variety of scientific technologies such as computer graphics technology, computer simulation technology, sensor technology, display technology, etc., and can create a virtual environment. The user is immersed in the virtual environment by wearing VR wearable devices (e.g., VR glasses). The virtual environment is presented by continuously refreshing a plurality of three-dimensional images, and the three-dimensional images comprise objects with different depths of field, so that a stereoscopic impression is brought to a user.

In VR technology, the true sense of immersion and the high quality experience are related to the resolution of the image, the higher the resolution the better the experience. However, the improvement of resolution means high requirements on performance and power consumption, and general devices cannot meet the high power consumption requirements, especially for portable, small mobile independent devices (e.g., VR glasses) which are designed to be portable, because of heat dissipation characteristics and limited capacity of batteries, it is more difficult to meet the high power consumption requirements.

Disclosure of Invention

The application aims to provide a display method and electronic equipment, which are used for reducing power consumption and guaranteeing VR experience.

In a first aspect, a display method is provided, which may be applied to an electronic device, which may be a VR device, an AR device, an MR device, or the like. The method comprises the following steps: displaying the N frames of images through a display device; n is a positive integer; the definition of a first object on an ith frame image in the N frames of images is a first definition; the definition of the first object on the j-th frame image in the N frames of images is second definition; the definition of the first object on the kth frame of image in the N frames of images is third definition; the first definition is larger than the second definition, the second definition is smaller than the third definition, and i, j and k are positive integers smaller than N, i is smaller than j and smaller than k; the average value of the first definition and the second definition is a fourth definition, and the fourth definition is higher than a first threshold value; the average of the second definition and the third definition is a fifth definition, which is higher than a second threshold.

In the embodiment of the application, the definition of the first object on different frame images in the image stream displayed by the display device can be switched between high and low. For example, the first object definition on the i-th frame image is high, the first object definition on the j-th frame image is low, and the first object definition on the k-th frame image is high, so that the first object on each frame image does not need to be kept high, and power consumption is saved. In addition, due to the visual retention effect of human eyes, when the human eyes watch the image stream, the brain can fuse different images in the image stream, namely, the human eyes feel the fused images of different frames in the image stream, and the resolution of the fused images is the average value of the definition of the fused images. In the embodiment of the application, since the average value of the definition of the first object on different frame images is higher than the threshold value, the definition of the first object seen when human eyes fuse different frame images is higher than the threshold value. For example, the average value of the sharpness of the first object on the i-th frame image and the j-th frame image is higher than the first threshold, so that the sharpness of the first object seen when the human eye fuses the i-th frame image and the j-th frame image is higher than the first threshold. For another example, the average value of the sharpness of the first object on the jth frame image and the kth frame image is higher than the second threshold, so that the sharpness of the first object seen when the human eye fuses the jth frame image and the kth frame image is higher than the second threshold. Therefore, the definition of the virtual environment seen by the user through the display device is higher, and the visual experience is improved.

In one possible design, the first threshold is a user setting and/or the second threshold is a user setting. That is, the user may define the definition threshold (i.e., the first threshold and the second threshold) of the first object in the virtual environment displayed by the electronic device, so that the definition of the first object in the virtual environment seen by the user through the electronic device is not lower than the threshold, which is helpful for improving the visual experience, and the user may define the definition threshold according to the user's needs, which is helpful for improving the user experience.

In one possible design, the first threshold is an average of the sixth definition and the seventh definition; the sixth definition is the definition of the first object on the rendered original image of the ith frame, and the image of the ith frame is an image after the original image of the ith frame is processed; the sixth definition is less than the first definition; the seventh definition is the definition of the first object on the rendered jth frame original image, the jth frame image is an image after the jth frame original image is processed, and the seventh definition is smaller than or equal to the second definition. That is, after the electronic device renders N frames of artwork, the N frames of artwork may be displayed, and the definition of the virtual environment seen by the user is definition 1. If the electronic equipment renders N frames of original pictures, the N frames of original pictures are processed to improve the definition, when the electronic equipment displays the processed N frames of images, the definition of the virtual environment seen by the user is definition 2, and definition 2 is larger than definition 1, so that the user virtual reality experience is improved.

In one possible design, the second threshold is an average of the eighth definition and the ninth definition; the eighth definition is the definition of the first object on the rendered jth frame original image, and the jth frame image is an image after the jth frame original image is processed; the eighth definition is less than the second definition; the ninth definition is the definition of the first object on the rendered original image of the kth frame, the kth frame image is an image after the processing of the original image of the kth frame, and the ninth definition is smaller than or equal to the third definition. That is, after the electronic device renders N frames of artwork, the N frames of artwork may be displayed, and the definition of the virtual environment seen by the user is definition 1. If the electronic equipment renders N frames of original pictures, the N frames of original pictures are processed to improve the definition, when the electronic equipment displays the processed N frames of images, the definition of the virtual environment seen by the user is definition 2, and definition 2 is larger than definition 1, so that the user virtual reality experience is improved.

In one possible design, the first object is located in an area where the point of non-fixation is located, or the first object is located at a first depth of field, the first depth of field being greater than a preset depth of field. That is, the sharpness of the object in the region where the non-point of regard is located in the different frame images in the image stream, or the object at a distance (object with a larger depth of field) can be switched between high and low, so that the rendering power consumption can be saved. In addition, the average value of the definition of the object or the distant object in the area where the non-fixation point is located on the different frame images is larger than the threshold value (the first threshold value or the second threshold value), so that the definition of the object or the distant object in the area where the non-fixation point is located seen by the human eyes is larger than the threshold value by fusion, and the visual experience is improved.

In one possible design, the sharpness of the second object on the N-frame image is unchanged; the second object is located in the region where the gaze point is located, or the second object is located in a second depth of field, and the second depth of field is smaller than the preset depth of field. That is, when the sharpness of the object in the region where the non-gaze point is located in the image stream is switched between high and low, the sharpness of the object in the region where the gaze point is located may be unchanged; alternatively, when the sharpness of the distant object (object with a large depth of field) is switched between high and low, the sharpness of the near object may be unchanged. For example, the sharpness of the object in the non-fixation area shows a high-definition-low-definition-high-definition change state, and the object in the fixation area is always high-definition or always ultra-high-definition. As another example, the sharpness of the far object may exhibit a high-low-high-definition change state, and the near object may always be high-definition or always be ultra-high-definition. This may reduce rendering power consumption.

In one possible design, the second object has a higher sharpness than the first object. For example, when the sharpness of the object in the region where the non-gaze point is located in the image stream is switched, the sharpness of the object in the region where the gaze point is located may be unchanged; moreover, the sharpness of the object in the region where the gaze point is located is higher than the sharpness of the object in the region where the non-gaze point is located. For example, the object in the region where the gaze point is located in the N-frame image is always ultra-high definition, and the object in the region where the non-gaze point is located exhibits a high definition-low definition-high definition change state. Therefore, the definition of the object or the near object in the gazing area can be ensured to be clear enough, and the influence on the user experience is avoided.

In one possible design, the sharpness of the second object on the i-th frame image is tenth sharpness; the definition of the second object on the j-th frame image is eleventh definition; the definition of the second object on the kth frame image is twelfth definition; the second object is located in the region where the gaze point is located, or the second object is located in a second depth of field, and the second depth of field is smaller than the preset depth of field; an average value of the tenth definition and the eleventh definition is thirteenth definition, and an average value of the eleventh definition and the twelfth definition is fourteenth definition; the thirteenth definition is higher than the fourth definition, and the fourteenth definition is higher than the fifth definition.

That is, when the sharpness of the object (i.e., the first object) in the area where the non-gaze point is located in the image stream is switched between high and low, the sharpness of the object (i.e., the second object) in the area where the gaze point is located in the image stream is also switched between high and low; moreover, the average value of the definition of the object in the region where the gaze point is located on the different frame images is higher than the average value of the definition of the object in the region where the non-gaze point is located, so that the definition of the object in the region where the gaze point is located is higher than the object in the region where the non-gaze point is located in the virtual environment seen by the user. Similarly, when the definition of a far object (i.e. a first object) and the definition of a near object (i.e. a second object) in different frames of images in the image stream are switched, the definition of the near object (i.e. a second object) can be switched; moreover, the average value of the sharpness of the near object on the different frame images is higher than the average value of the sharpness of the far object, so that the near object is higher in sharpness than the far object in the virtual environment seen by the user.

In one possible design, the first object is located in the region where the gaze point is located; or the first object is located at a first depth of field, and the first depth of field is smaller than a preset depth of field. That is, the sharpness of the object in the region where the gaze point is located on a different frame image in the image stream, or the sharpness of the near object (the object with smaller depth of field) can be switched between high and low, so that the rendering power consumption can be saved. In addition, the average value of the definition of the object or the near object in the region where the gaze point is located on the different frame images is larger than the threshold value (the first threshold value or the second threshold value), so that the human eyes fuse the definition of the object or the near object in the region where the gaze point is located on the different frame images to be larger than the threshold value, and the visual experience is improved.

In one possible design, the sharpness of the second object on the N-frame image is unchanged; the second object is located in the area where the non-fixation point is located, or the second object is located in a second depth of field, and the second depth of field is larger than the preset depth of field. That is, when the sharpness of the object in the region where the gaze point is located in the image stream is switched between high and low, the sharpness of the object in the region where the non-gaze point is located may be unchanged; alternatively, the sharpness of the far object may be unchanged when the sharpness of the near object is switched between high and low. For example, the sharpness of an object in a gaze area exhibits a state of change of ultra high definition-ultra high definition, and an object in a non-gaze area is always high definition or always low definition. As another example, the sharpness of near objects may exhibit a changing state of super high definition-super high definition, and far objects may be always high definition or always low definition. In this way, rendering power consumption can be reduced.

In one possible design, the second object has a lower sharpness than the first object. For example, when the sharpness of the object in the region where the gaze point is located on different frame images in the image stream is switched, the sharpness of the object in the region where the non-gaze point is located may be unchanged; moreover, the sharpness of the object in the region where the point of gaze is located is lower than the sharpness of the object in the region where the point of gaze is located. For example, the object in the region where the non-fixation point is located in the N-frame image is always high-definition or low-definition, and the object in the region where the fixation point is located exhibits an ultrahigh-definition-high-ultrahigh-definition change state. Therefore, the definition of the object or the near object in the gazing area can be ensured to be clear enough, and the influence on the user experience is avoided.

In one possible design, the definition of the second object on the ith frame image is tenth definition, the definition of the second object on the jth frame image is eleventh definition, and the definition of the second object on the kth frame image is twelfth definition; the second object is located in the area where the non-fixation point is located, or the second object is located in a second depth of field, and the second depth of field is larger than the preset depth of field; an average value of the tenth definition and the eleventh definition is thirteenth definition, and an average value of the eleventh definition and the twelfth definition is fourteenth definition; the thirteenth definition is lower than the fourth definition, and the fourteenth definition is lower than the fifth definition.

That is, when the sharpness of the object (i.e., the first object) in the region where the gaze point is located in the image stream is switched between high and low, the sharpness of the object (i.e., the second object) in the region where the non-gaze point is located in the image stream may also be switched between high and low; moreover, the average value of the definition of the object in the area where the non-gaze point is located on the different frame image is lower than the average value of the definition of the object in the area where the gaze point is located, so that the definition of the object in the area where the non-gaze point is located is lower than the object in the area where the gaze point is located in the virtual environment seen by the user. Similarly, when the definition of the near object (i.e. the first object) in different frame images in the image stream is switched between high and low, the definition of the far object (i.e. the second object) can also be switched between high and low; moreover, the average value of the sharpness of the far object on the different frame images is lower than the average value of the sharpness of the near object, so that the sharpness of the far object is lower than the near object in the virtual environment seen by the user.

In one possible design, the user gaze point is unchanged during the display of the i-th frame image, the j-th frame image, and the k-th frame image. It can be understood that when the user's gaze point changes, the electronic device may redetermine the area where the user's gaze point is located, and execute the technical solution provided in the present application again, for example, the redefined gaze point switches the sharpness of the object (i.e. the first object) in the area where the user's gaze point is located, and the average value of the sharpness of the first object on different frame images is higher than the threshold, that is, when the user's gaze point changes, the sharpness of the object in the area where the new gaze point is located in the virtual environment seen by the human eye is also higher than the threshold, so that the user's visual experience is better.

In one possible design, a time interval between a display time of the i-th frame image and a display time of the j-th frame image is less than or equal to a user visual dwell time; and/or, a time interval between the display time of the jth frame image and the display time of the kth frame image is smaller than or equal to the user visual stay time. It should be appreciated that when the display time interval of two frames of images is small (e.g., less than the user's visual dwell time), the brain will merge the two frames of images due to the human visual dwell efficiency, so that the human eye sees the environment presented after the two frames of images are merged. Therefore, in the embodiment of the application, when the electronic device displays the image stream, the display time interval of the two frames of images is smaller than the visual stay time of the user, and the average value of the definition of the first object on the two frames of images is higher than the threshold value, so that the definition of the first object in the seen environment is higher when the two frames of images are fused by human eyes.

In one possible design, before displaying the N frames of images by the display device, the method further includes: detecting that a user triggers an operation for starting a definition optimization mode; the user viewing time is greater than at least one of the first duration, the second duration, the user eye blinking or the number of squints is greater than a preset number.

The electronic device has a normal mode and a sharpness optimization mode, for example. In the normal mode, after the image stream is electronically rendered, the image stream is displayed by a display device, and the definition in the virtual environment seen by the user is low. When the electronic equipment starts a definition optimization mode, the electronic equipment performs image processing on the rendered image stream, so that the average value of the definition of different frame images in the image stream is higher than a threshold value, and the definition of the virtual environment seen by a user is improved, and user experience is improved. The electronic device starts the definition optimization mode in various modes, for example, when detecting that a user triggers an operation for starting the definition optimization mode, the electronic device starts the definition optimization mode; or starting a definition optimization mode when the watching time of the user is detected to be longer than the first time length; or starting a definition optimization mode when the eye blink or squint frequency of the user in the second time period is detected to be larger than the preset frequency. In other words, the electronic device may not always be in the sharpness optimization mode, but may be activated when the user needs or when the user needs are detected (e.g., the user's eyes blink or squint times are greater than a preset number of times within the first time period, the second time period, and the like).

In a second aspect, there is also provided a display method, including: when a first application is displayed through first equipment, a display screen of the first equipment displays N frames of first images, wherein N is a positive integer, the N comprises an ith frame of first images and a jth frame of first images, and the definition of a first object on the ith frame of first images is first definition; the definition of the first object on the j-th frame first image is second definition; the average value of the first definition and the second definition is a third definition; when the first application is displayed through a second device, the display screen of the second device displays N frames of second images, wherein the N frames of second images comprise an ith frame of second image and a jth frame of second image, and the definition of a first object on the ith frame of second image is a fourth definition; the definition of the first object on the j-th frame second image is a fifth definition; the average value of the fourth definition and the fifth definition is a sixth definition; wherein the third definition is higher than the sixth definition.

That is, there are two devices, such as two VR glasses, referred to as VR glasses a and VR glasses B for convenience of distinction. The technical scheme provided by the application is used by the VR glasses A, and the technical scheme provided by the application is not used by the VR glasses B. The two VR glasses may display the same application generated image. The same application here may be the same application name, application version, etc. For example, two VR glasses have the same VR driving application thereon. When the user wears VR glasses a and the VR glasses a display images of the application, the definition of the virtual environment seen by the user is high. When the user wears VR glasses B and the VR glasses B also display the image of the application, the virtual environment seen by the user is the same as the virtual environment seen when wearing VR glasses a, but the definition of the virtual environment seen when wearing VR glasses B is lower than the definition of the virtual environment seen when wearing VR glasses a.

In a third aspect, there is also provided an electronic device, comprising:

a processor, a memory, and one or more programs;

wherein the one or more programs are stored in the memory, the one or more programs comprising instructions, which when executed by the processor, cause the electronic device to perform the method steps as provided in the first or second aspects above.

In a fourth aspect, there is also provided a computer readable storage medium for storing a computer program which, when run on a computer, causes the computer to perform the method of the first or second aspect described above.

In a fifth aspect, there is also provided a computer program product comprising a computer program which, when run on a computer, causes the computer to perform the method of the first or second aspect described above.

The advantages of the second aspect to the fifth aspect are described above with reference to the first aspect, and the description thereof is not repeated.

Drawings

Fig. 1 is a schematic diagram of a communication system according to an embodiment of the present application;

Fig. 2 is a schematic diagram of a VR wearable device according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a human eye image flow according to an embodiment of the present application;

fig. 4A to fig. 4C are schematic diagrams illustrating an image viewed by a human eye according to an embodiment of the present application;

FIG. 5 is a diagram showing an effect according to an embodiment of the present application;

fig. 6 to 9 are schematic diagrams of an application scenario provided in an embodiment of the present application;

fig. 10 is a schematic structural diagram of a VR wearable device according to an embodiment of the present application;

FIG. 11 is a schematic diagram of an image processing procedure according to an embodiment of the present application;

FIG. 12 is a schematic diagram of an image processing procedure according to an embodiment of the present application;

fig. 13A to 13B are another schematic diagrams illustrating an image processing procedure according to an embodiment of the present application;

fig. 14A to 14B are yet another schematic diagrams illustrating an image processing procedure according to an embodiment of the present application;

fig. 15 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In the following, some terms in the embodiments of the present application are explained for easy understanding by those skilled in the art.

(1) At least one of the embodiments of the present application includes one or more; wherein, a plurality refers to greater than or equal to two. In addition, it should be understood that in the description of the present application, the words "first," "second," and the like are used merely for distinguishing between the descriptions and not for indicating or implying any relative importance or order. For example, the first object and the second object do not represent the importance of both or the order of both, in order to distinguish the objects.

In the embodiment of the present application, "and/or" is an association relationship describing an association object, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

(2) Virtual Reality (VR) technology is a man-machine interaction means created by means of computer and sensor technologies. VR technology integrates a variety of scientific technologies such as computer graphics technology, computer simulation technology, sensor technology, display technology, etc., and can create a virtual environment. The virtual environment comprises three-dimensional realistic images which are generated by a computer and dynamically played in real time, so that visual perception is brought to a user; moreover, besides visual perception generated by computer graphics technology, there are also perceives such as hearing, touch, force sense, movement, etc., even including smell and taste, etc., also called multi-perception; in addition, the head rotation, eyes, gestures or other human behavior actions of the user can be detected, the computer is used for processing data which are suitable for the actions of the user, the data respond to the actions of the user in real time and are respectively fed back to the five sense organs of the user, and then the virtual environment is formed. For example, a user wearing the VR wearable device may see the VR game interface, through gestures, handles, etc., may interact with the VR game interface as if in a game.

(3) Augmented reality (Augmented Reality, AR) technology refers to overlaying computer-generated virtual objects over a real-world scene, thereby enabling augmentation of the real world. That is, the AR technology needs to acquire a real-world scene and then add a virtual environment on the real world. Thus, VR technology differs from AR technology in that VR technology creates a complete virtual environment, and all users see is a virtual object; while AR technology is the superposition of virtual objects on the real world, i.e. including both real world and virtual objects. For example, a user wears transparent glasses through which a surrounding real environment can be seen, and virtual objects can be displayed on the glasses, so that the user can see both the real objects and the virtual objects.

(4) The Mixed Reality technology (MR) is to introduce real scene information (or referred to as real scene information) into a virtual environment, and bridge an interactive feedback information among the virtual environment, the real world and a user, so as to enhance the sense of Reality of user experience. Specifically, the real object is virtualized (e.g., a camera is used to scan the real object for three-dimensional reconstruction, generating a virtual object), and the virtualized real object is introduced into the virtual environment, so that the user can see the real object in the virtual environment.

It should be noted that, the technical solution provided by the embodiment of the present application may be applied to VR scenes, AR scenes or MR scenes. Of course, other scenarios besides VR, AR and MR may be applicable. Such as naked eye 3D scenes (naked eye 3D display screen, naked eye 3D projection, etc.), theatres (such as 3D movies), VR software in electronic devices, etc. In a word, the technical scheme provided by the application can be applied to any scene needing to display a three-dimensional image.

For ease of understanding, VR scenarios are mainly described below as examples.

For example, please refer to fig. 1, which is a schematic diagram of a VR system according to an embodiment of the present application. VR system includes VR wearing device 100 and image generating device 200. Wherein the image generation device 200 comprises a host (e.g., VR host) or server (e.g., VR server). The VR wearable device 100 connects (wired or wireless) with a VR host or VR server. The VR host or VR server may be a device with greater computing capabilities. For example, the VR host may be a device such as a cell phone, tablet, notebook, etc., and the VR server may be a cloud server, etc. The VR host or VR server is responsible for generating images, etc., and then sends the images to the VR wearable device 100 for display, and the user can see the images by wearing the VR wearable device 100. For example, the VR wearing device 100 may be a head mounted device (Head Mounted Display, HMD), such as glasses, a helmet, or the like. Alternatively, the VR system of fig. 1 may not include the image generating apparatus 200. For example, VR wearable device 100 has image generation functionality locally, without having to acquire an image from image generation device 200 (VR host or VR server) for display. In summary, VR wearable device 100 may display a three-dimensional image through which a stereoscopic virtual environment may be presented to a user due to the different depths of field (see description below) of different objects on the three-dimensional image.

(5) Depth of Field (DOF) the three-dimensional image comprises objects of different image depths. For example, the VR wearing device displays a three-dimensional image, and the user wears the VR wearing device to see a three-dimensional scene (i.e., virtual environment) in which distances between different objects and eyes of the user are different, so that a stereoscopic impression is presented. Thus, image depth can be understood as the distance between the object and the user's eye on a three-dimensional image, the greater the image depth, the farther the user's eye is visually from, appearing as a perspective; the smaller the image depth, the closer the user's eye is visually, and appears as a close-up. The image depth may also be referred to as "depth of field".

In order to clearly illustrate the process of presenting the virtual environment to the user by the VR wearable device, the following briefly describes a human eye vision generation mechanism.

In an actual scene, when a user views an object, human eyes can realize visual perception by acquiring an optical signal in the actual scene and processing the optical signal in the brain. Wherein the light signals in the actual scene may comprise reflected light from different objects and/or light signals directly emitted by the light source. Because the optical signals of the actual scene can carry the related information (such as size, position, color, etc.) of each object in the actual scene, the brain can acquire the information of the objects in the actual scene, namely, the visual perception, by processing the optical signals. It should be understood that the optical signals in the actual scene are continuously generated, the human eyes continuously acquire the optical signals at a certain frequency, and the continuously acquired optical signals are processed by the brain, so that a dynamic visual feeling is formed.

The human eye includes a left eye and a right eye, each of which uses the human eye vision producing mechanism described above to create a dynamic visual perception. The left eye and the right eye have slightly different viewing angles when viewing the same object. The views seen by the left and right eyes are actually different. For example, the left eye may acquire an optical signal of a two-dimensional image (hereinafter simply referred to as left eye image) of a plane in which a focal point of the human eye is located, which is perpendicular to a viewing direction of the left eye. Similarly, the right eye can acquire an optical signal of a two-dimensional image (hereinafter simply referred to as a right-eye image) of a plane in which a focal point of the human eye is located, which is perpendicular to a line of sight of the right eye. The left eye image is slightly different from the right eye image. The brain can acquire the depth of different objects in the actual scene by processing the optical signals of the left eye image and the right eye image, and the stereoscopic vision feeling is acquired. The stereoscopic perception may also be referred to as binocular stereoscopic vision.

The above is a human eye vision generation mechanism by which the VR wearable device presents a virtual environment to the user. For ease of understanding, the VR wearable device is illustrated as VR glasses.

Please refer to fig. 2, which is a schematic diagram of VR glasses. The VR glasses are provided with 2 display screens, for example, a display screen 201 and a display screen 202. Each display screen may display corresponding content to one of the user's eyes (e.g., left or right eye) through a corresponding eyepiece. For example, display 201 corresponds to eyepiece 203 and display 202 corresponds to eyepiece 204. Then, on the display screen 201, a left eye image corresponding to the virtual environment may be displayed. The light rays of the left eye image may be converged at the left eye through the eyepiece 203 so that the left eye sees the left eye image. Similarly, on the display screen 202, a right-eye image corresponding to the virtual environment may be displayed. The light rays of the right eye image may be focused at the right eye through eyepiece 204 so that the right eye image is seen by the right eye. Thus, the brain can make the user see the objects in the virtual environment corresponding to the left eye image and the right eye image by fusing the left eye image and the right eye image. It should be noted that, the display screen 201 and the display screen 202 may be separate display screens, or may be different display areas on the same display screen, which is not limited by the embodiment of the present application.

To bring a dynamic visual experience to the user, the image streams may be displayed on the VR glasses' display 201 and display 202, respectively. The image stream includes N frames of images, N being an integer of 1 or more.

The principle of the VR glasses displaying an image stream will be described below by taking a display screen on the glasses as an example.

In general, after an image generating apparatus (for example, the image generating apparatus 200 in fig. 1) generates N frame images, the N frame images are rendered, and the rendered N frame images are displayed through a certain display screen of VR glasses.

In some embodiments, each of the N frames of images is rendered using a high resolution, and the N frames of images rendered at the high resolution are then displayed through a display screen of VR glasses. For example, please refer to fig. 3, wherein the image stream is displayed on the display screen of the vr glasses. The image stream includes an i-th frame, a j-th frame, a k-th frame, and the like. Where the resolution of each frame is a (a is the higher resolution). For example, each frame is an ultra high definition image or a high definition image, so the definition of each frame image is high.

To illustrate the technical solution of the present application, the talbot-plato law of the human eye is introduced. The talbot-platuo law refers to that when a display screen displays an image stream, the brain fuses different images in the image stream due to the visual retention effect of the human eye, in short, what the human eye perceives is an image fused by different frames of images in the image stream, and the resolution of the fused image is an average of the sharpness of the fused image. Taking fig. 3 as an example, when the display screen displays the ith frame image and the jth frame image, human eyes fuse the ith frame image and the jth frame image, and the resolution of the fused image is an average value of the resolution of the ith frame image and the resolution of the jth frame image. Since the resolutions of the i-th frame image and the j-th frame image are both a, the resolution of the fused image is also a. Thus, the resolution of the fused image perceived by the human eye is a. Similarly, when the display screen displays the jth frame image and the kth frame image, human eyes fuse the jth frame image with the kth frame image, and the resolution of the fused image is the average value of the resolution of the jth frame image and the resolution of the kth frame image, namely A. Thus, the resolution of the fused image perceived by the human eye is also a. That is, when each frame of image in the image stream is ultra-high definition (or high definition), the fused image seen by human eyes is also ultra-high definition (or high definition), and the virtual reality experience is better.

However, it will be appreciated that each frame of image in the image stream remains at a high resolution (e.g., the resolution is a), requiring a large rendering power consumption, and that typical devices cannot meet the high power consumption requirements.

To save power consumption, one implementation is to render different areas on the image using different resolutions. For example, an image is divided into a gazing area and a non-gazing area. High resolution rendering is used for gaze areas and low resolution rendering is used for non-gaze areas. Because the user attention of the non-gazing area is low, the influence on visual experience is not great when the resolution of the non-gazing area is low, and the power consumption can be greatly saved.

Illustratively, as in fig. 4A, the image includes a fixation region and a non-fixation region thereon. The gaze area may be a circular area with a preset length as a radius, and of course, may be a square area or a rectangular area with a preset length as a side length, and the like, with the position of the user gaze point as the center. The non-gazing area is an area other than the gazing area on the image. Continuing with fig. 4A, the resolution of the gaze area is a. The low resolution of the non-fixation region is B. A > B. Thus, the whole image is not required to use high resolution, and the power consumption caused by rendering is reduced.

It should be appreciated that fig. 4A takes one image as an example, and as described above, the image stream, i.e., the N-frame image, is displayed on the display screen of the VR glasses. In some embodiments, each frame of image in the image stream is a high fixation area resolution and a low non-fixation area resolution.

For example, as shown in fig. 4B, the image stream includes an i-th frame, a j-th frame, a k-th frame, and the like. The resolution of the gazing area on the ith frame of image is A, the resolution of the non-gazing area is B, A > B, for example, A is ultrahigh definition, and B is high definition; alternatively, a is high definition and B is low definition. Similarly, the resolution of the gazing region on the j-th frame image is A, and the resolution of the non-gazing region is B. The resolution of the gazing area on the kth frame image is A, and the resolution of the non-gazing area is B. That is, each frame of image in the image stream has high resolution of the gazing region and low resolution of the non-gazing region. In this case, when the ith frame image and the jth frame image are displayed on the display screen of the VR glasses, the human eye fuses the ith frame image and the jth frame image, and the resolution of the fused image is the average of the resolution of the ith frame image and the resolution of the jth frame image, that is, the resolution of the gazing area on the fused image is still a, and the resolution of the non-gazing area is still B, because the resolution of the non-gazing area is (b+b)/2. Similarly, when the jth frame image and the kth frame image are displayed on the display screen of the VR glasses, the human eye fuses the jth frame image and the kth frame image, and the resolution of the fused image is the average value of the resolution of the jth frame image and the resolution of the kth frame image, that is, the resolution of the gazing area on the fused image is still a, and the resolution of the non-gazing area is still B, because the resolution of the non-gazing area is (b+b)/2. Therefore, in fig. 4B, in the visual perception of the human eye, the definition of the non-gazing area is always maintained at B, and since B is low, the definition of the non-gazing area is always low, which results in loss of details of the non-gazing area and reduces the virtual reality experience.

In fig. 4B, the resolution of the gazing area on the different frame images in the image stream is kept unchanged (for example, a is used as an example), and in practice, the resolution of the gazing area on the different frame images may also be changed, for example, the resolution of the gazing area on the ith frame image is a, the resolution of the gazing area on the jth frame image is D, a is greater than D, the resolution of the gazing area on the kth frame image is a, that is, the resolution of the gazing area on the different frame images is switched. In this case, the principle of fusion of the human eyes with the image is similar, for example, the resolution of the gazing region on the image obtained by fusing the i-th frame image and the j-th frame image is an average value of the resolution a of the gazing region on the i-th frame image and the resolution D of the gazing region on the j-th frame image. Therefore, for the sake of description, the description is not repeated for the case that the resolution of the gazing area on the different frame images is changed.

In fig. 4B, the resolution of the non-fixation area in each frame of image in the image stream is the same, and B is the same. In other embodiments, the resolution of the non-fixation areas on different frame images in the image stream may be different, for example, the resolution of the non-fixation areas on different frame images in the image stream may be switched.

For example, as shown in fig. 4C, the image stream includes an i-th frame, a j-th frame, a k-th frame, and the like. The resolution of the gazing area on the ith frame of image is A, the resolution of the non-gazing area is B, A > B, for example, A is ultrahigh definition, and B is high definition; alternatively, a is high definition and B is low definition. The resolution of the gazing area on the j-th frame image is A, the resolution of the non-gazing area is C, A > C, C > B. The resolution of the gazing area on the kth frame image is A, and the resolution of the non-gazing area is B. That is, compared with fig. 4B, the resolution of the non-fixation area on the jth frame image is increased from B to C. In this way, the resolution of the gazing area on the image obtained by fusing the ith frame image and the jth frame image is still A, and the resolution of the non-gazing area is (B+C)/2. Similarly, the resolution of the gazing area on the image obtained by fusing the jth frame image and the kth frame image is A, and the resolution of the non-gazing area is (B+C)/2. Since C > B, (b+c)/2 is higher than B, the definition of the non-gazing area is improved visually by human eyes in fig. 4C as compared with fig. 4B.

As understood from fig. 4B and fig. 4C, the resolution of the non-gazing area of the j-th frame image in the image stream is improved (i.e. from B to C), so that the definition of the non-gazing area is improved in the virtual environment seen by the human eye. Therefore, if the resolution of the non-gazing area in the virtual environment seen by the human eye is to be further improved, the resolution of the non-gazing area on some frame images in the image stream needs to be improved, which obviously causes the problem of increased rendering power consumption.

In order not to increase the rendering power consumption and improve the resolution of the non-gazing area in the virtual environment seen by human eyes, the application provides a method, as shown in fig. 5, of rendering an N-frame image by an image generating device (such as the image generating device 200 in fig. 1); for convenience of distinction, the rendered image is referred to as artwork, including an ith frame artwork, a jth frame artwork, a kth frame artwork, and the like. The resolution of the gazing area on the original picture of the ith frame is A, and the resolution of the non-gazing area is B. The resolution of the gazing area on the original picture of the j frame is A, the resolution of the non-gazing area is C, and C is smaller than A and larger than B. The resolution of the non-fixation area on the original image of the kth frame is B. Therefore, as can be seen from comparing fig. 5 and fig. 4C, the image rendering power consumption is not increased.

With continued reference to fig. 5, the image generating apparatus may process the N frame artwork by using an image post-processing manner to obtain N frame new images, for example, raise the resolution of the non-gazing area on the ith frame artwork from B to B ' to obtain the ith frame new image, and/or raise the resolution of the non-gazing area on the jth frame artwork from C to C ' to obtain the jth frame new image, and/or raise the resolution of the non-gazing area on the kth frame artwork from B to B ' to obtain the kth frame new image. When the display device displays N frames of new images, the resolution of the non-gazing area on the image obtained by fusing the ith frame of new images and the jth frame of new images is (B '+C')/2, and the resolution of the non-gazing area on the image obtained by fusing the jth frame of new images and the kth frame of new images is (B '+C')/2. Thus, when the display device (e.g., VR glasses) displays the image stream after image post-processing, the sharpness of the non-gazing area is enhanced visually by human eyes, but the image rendering power consumption is not significantly increased. The image post-processing process described in fig. 5 will be described later.

Therefore, in the embodiment of the present application, the display device (for example, VR glasses) may display the image stream (including N new frames) in fig. 5, where the image stream includes the ith frame, the jth frame, and the kth frame. Let the definition of the first object (e.g., an object located in a non-fixation area) on the ith frame be the first definition, the definition of the first object on the jth frame be the second definition, and the definition of the first object on the kth frame be the third definition. Wherein the average of the first definition and the second definition is a third definition, the third definition is higher than a first threshold, for example, the third definition is (B '+c')/2, and the first threshold is (b+c)/2. Similarly, the average of the second definition and the third definition is a fourth definition, which is higher than a second threshold, e.g., (B '+c')/2, and the second threshold is (b+c)/2. In this way, the sharpness of the first object in the non-noted area of the virtual environment perceived by the user through the display device (e.g., VR glasses) is enhanced.

In fig. 5, the resolution of the non-gazing area in the image stream is different from the resolution of the gazing area, and the resolution of the non-gazing area in the image stream is raised, for example, the resolution of the non-gazing area in the original image of the ith frame is raised to B ', and the resolution of the non-gazing area in the original image of the jth frame is raised to C', in fact, the resolution of the gazing area in the image may be raised by the image post-processing method provided by the present application. In other embodiments, the image is not divided into regions, i.e. the resolution of the whole image is the same (e.g. fig. 3), in which case the resolution may be improved by using the image post-processing method provided by the present application.

For convenience of description, the following description will take an example of enhancing the resolution of the first object on the image. The first object may be any object on the image. For example, the first object is an object in a non-fixation area on the image. Alternatively, the first object is an object within a gaze area on the image. Alternatively, the first object is a remote object. Alternatively, the first object is a near object.

For convenience of understanding, an application scenario provided by the embodiment of the present application is described below. The application scenario is exemplified by the first object being an object in a non-fixation area on the image.

Fig. 6 is a schematic diagram of an application scenario provided in an embodiment of the present application. As shown in fig. 6, in this application scenario, the user wears VR glasses to perform VR driving.

As shown in fig. 6, VR glasses render an image stream that includes N frames of artwork. Including the ith frame artwork, the jth frame artwork, the kth frame artwork, etc. The original image comprises objects such as a steering wheel, a display, a road, trees positioned on the road, a front vehicle and the like, and virtual driving experience can be brought to a user. Assuming that the tree is located in a non-gazing area, the definition of the tree on the artwork is low, e.g., the tree on the artwork on the ith frame is blurred, the tree on the artwork on the jth frame is clear, and the artwork on the kth frame is blurred. If the VR glasses display the N frames of original pictures, the definition of the tree on the image obtained by fusing the ith frame of original picture and the jth frame of original picture is low, and similarly, the definition of the tree on the image obtained by fusing the jth frame of original picture and the kth frame of original picture is also low. The technical scheme provided by the embodiment of the application can carry out post-processing on the rendered N frames of original pictures to obtain N frames of new pictures, wherein the N frames of new pictures comprise an ith frame of new picture, a jth frame of new picture, a kth frame of new picture and the like. Compared with the original image of the ith frame, the tree on the new image of the ith frame is clearer; compared with the original image of the j frame, the definition of the tree on the new image of the j frame is unchanged or clearer; and compared with the original image of the kth frame, the definition of the tree on the new image of the kth frame is improved. In this way, when the VR glasses display N new images, the definition of the tree on the image obtained by fusing the new image of the ith frame and the new image of the jth frame is improved compared with the definition of the tree on the image obtained by fusing the original image of the ith frame and the original image of the jth frame; similarly, the definition of the tree on the image obtained by fusing the new image of the j frame and the new image of the k frame is improved compared with the definition of the tree on the image obtained by fusing the original image of the j frame and the original image of the j frame.

Alternatively, in fig. 6, taking the example of improving the definition of the first object (i.e. the tree) in the non-noted area on the image, the definition of the object (e.g. the front vehicle) in the noted area may be kept unchanged, or the definition of the object (e.g. the front vehicle) in the noted area may be improved by the scheme provided by the embodiment of the present application. In some embodiments, the sharpness of the object within the gaze area may be higher than the sharpness of the object within the non-gaze area.

It should be noted that, in fig. 6, the first object is an object in a non-gazing area on the image, and in other embodiments, the first object may also be an object in a gazing area on the image, or a far object on the image, or a near object on the image, etc., and the principle is the same as that of fig. 6, and the description is not repeated.

The apparatus to which the present application relates is described below.

For example, please refer to fig. 7, which illustrates a structural schematic diagram of a VR wearing device (e.g., VR glasses) provided by the embodiment of the present application. As shown in fig. 7, VR wearable device 100 may include a processor 110, a memory 120, a sensor module 130 (that may be used to obtain a gesture of a user), a microphone 140, keys 150, an input/output interface 160, a communication module 170, a camera 180, a battery 190, an optical display module 1100, an eye tracking module 1200, and the like.

It is to be understood that the illustrated structure of the embodiments of the present application does not constitute a specific limitation on the VR wearable device 100. In other embodiments of the application, VR wear device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110, which is generally used to control overall operation of the VR wearable device 100, may include one or more processing units, such as: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a video processing unit (video processing unit, VPU) controller, memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments of the application, the processor 110 may be used to control the optical power of the VR wearable device 100. For example, the processor 110 may be configured to control the optical power of the optical display module 1100 to implement the function of adjusting the optical power of the wearable device 100. For example, the processor 110 may adjust the relative positions of the optical devices (such as lenses) in the optical display module 1100, so that the optical power of the optical display module 1100 is adjusted, and further, the position of the corresponding virtual image surface of the optical display module 1100 when imaging the human eye can be adjusted. Thereby achieving the effect of controlling the optical power of the wearing device 100.

In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an integrated circuit (inter-integrated circuit, I2C) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, a serial peripheral interface (serial peripheral interface, SPI) interface, and the like.

In some embodiments, the processor 110 may blur objects at different depths of field to different degrees of sharpness.

The I2C interface is a bi-directional synchronous serial bus comprising a serial data line (SDA) and a serial clock line (derail clock line, SCL). In some embodiments, the processor 110 may contain multiple sets of I2C buses.

The UART interface is a universal serial data bus for asynchronous communications. The bus may be a bi-directional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is typically used to connect the processor 110 with the communication module 170. For example: the processor 110 communicates with a bluetooth module in the communication module 170 through a UART interface to implement a bluetooth function.

The MIPI interface may be used to connect the processor 110 to peripheral devices such as a display screen, camera 180, etc. in the optical display module 1100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or as a data signal. In some embodiments, a GPIO interface may be used to connect the processor 110 with the camera 180, a display screen in the optical display module 1100, the communication module 170, the sensor module 130, the microphone 140, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, an MIPI interface, etc. Optionally, the camera 180 may collect an image including a real object, and the processor 110 may fuse the image collected by the camera with the virtual object, and realistically fuse the resultant image through the optical display module 1100. Optionally, the camera 180 may also capture images including the human eye. The processor 110 performs eye movement tracking through the images.

The USB interface is an interface conforming to the USB standard specification, and can be specifically a Mini USB interface, a Micro USB interface, a USB Type C interface and the like. The USB interface may be used to connect a charger to charge VR wearable device 100, and may also be used to transfer data between VR wearable device 100 and a peripheral device. And can also be used for connecting with a headset, and playing audio through the headset. The interface may also be used to connect other electronic devices, such as cell phones and the like. The USB interface may be USB3.0, which is used for compatible high-speed display interface (DP) signal transmission, and may transmit video and audio high-speed data.

It should be understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is only illustrated schematically, and does not limit the structure of the wearable device 100. In other embodiments of the present application, the wearable device 100 may also use different interfacing manners, or a combination of multiple interfacing manners in the foregoing embodiments.

In addition, VR wearable device 100 may include wireless communication functionality, for example, VR wearable device 100 may receive images from other electronic devices (such as a VR host) for display. The communication module 170 may include a wireless communication module and a mobile communication module. The wireless communication function may be implemented by an antenna (not shown), a mobile communication module (not shown), a modem processor (not shown), a baseband processor (not shown), and the like. The antenna is used for transmitting and receiving electromagnetic wave signals. Multiple antennas may be included in VR wearable device 100, each of which may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module may provide a solution for wireless communication, including second generation (2th generation,2G) network/third generation (3th generation,3G) network/fourth generation (4th generation,4G) network/fifth generation (5 th generation, 5G) network, etc., as applied on VR wearable device 100. The mobile communication module may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module can receive electromagnetic waves by the antenna, filter, amplify and the like the received electromagnetic waves, and transmit the electromagnetic waves to the modem processor for demodulation. The mobile communication module can amplify the signal modulated by the modulation and demodulation processor and convert the signal into electromagnetic waves to radiate through the antenna. In some embodiments, at least some of the functional modules of the mobile communication module may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module may be provided in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to speakers, etc.), or displays images or video through a display screen in the optical display module 1100. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module or other functional module, independent of the processor 110.

The wireless communication module may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (wireless fidelity, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field wireless communication technology (near field communication, NFC), infrared technology (IR), etc., as applied on the VR wearable device 100. The wireless communication module may be one or more devices that integrate at least one communication processing module. The wireless communication module receives electromagnetic waves via an antenna, modulates the electromagnetic wave signals, filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module may also receive a signal to be transmitted from the processor 110, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation through the antenna.

In some embodiments, the antenna and mobile communication module of VR wearable device 100 are coupled such that VR wearable device 100 may communicate with a network and other devices through wireless communication techniques. The wireless communication techniques may include the Global System for Mobile communications (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR techniques, among others. The GNSS may include a global satellite positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a beidou satellite navigation system (beidou navigation satellite system, BDS), a quasi zenith satellite system (quasi-zenith satellite system, QZSS) and/or a satellite based augmentation system (satellite based augmentation systems, SBAS).

VR wearable device 100 implements display functions through a GPU, optical display module 1100, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to the optical display module 1100 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

Memory 120 may be used to store computer-executable program code that includes instructions. The processor 110 executes instructions stored in the memory 120 to perform various functional applications and data processing of the VR wearable device 100. The memory 120 may include a stored program area and a stored data area. The storage program area may store an application program (such as a sound playing function, an image playing function, etc.) required for at least one function of the operating system, etc. The storage data area may store data created during use of the wearable device 100 (e.g., audio data, phonebook, etc.), and so on. In addition, the memory 120 may include a high-speed random access memory, and may also include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like.

VR wearable device 100 may implement audio functionality through an audio module, speaker, microphone 140, headphone interface, and application processor, among others. Such as music playing, recording, etc. The audio module is used for converting digital audio information into analog audio signals for output and also used for converting analog audio input into digital audio signals. The audio module may also be used to encode and decode audio signals. In some embodiments, the audio module may be disposed in the processor 110, or a portion of the functional modules of the audio module may be disposed in the processor 110. Speakers, also known as "horns," are used to convert audio electrical signals into sound signals. The wearable device 100 can listen to music through a speaker or listen to hands-free conversation.

Microphone 140, also known as a "microphone," is used to convert sound signals into electrical signals. The VR wearable device 100 may be provided with at least one microphone 140. In other embodiments, the VR wearable device 100 may be provided with two microphones 140, which may also implement noise reduction in addition to collecting sound signals. In other embodiments, the VR wearable device 100 may also be provided with three, four, or more microphones 140 to enable collection of sound signals, noise reduction, identification of sound sources, directional recording functions, etc.

The earphone interface is used for connecting a wired earphone. The earphone interface may be a USB interface or a 3.5 millimeter (mm) open mobile wearable platform (open mobile terminal platform, OMTP) standard interface, a american cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

In some embodiments, VR wearing device 100 may include one or more keys 150 that may control the VR wearing device to provide a user with access to functionality on VR wearing device 100. The keys 150 may be in the form of buttons, switches, dials, and touch or near touch sensing devices (e.g., touch sensors). Specifically, for example, the user may turn on the optical display module 1100 of the VR wearable device 100 by pressing a button. The keys 150 include a power on key, a volume key, etc. The keys 150 may be mechanical keys. Or may be a touch key. The wearable device 100 may receive key inputs, generating key signal inputs related to user settings of the wearable device 100 and function control.

In some embodiments, VR wearable device 100 may include an input-output interface 160, and input-output interface 160 may connect other means to VR wearable device 100 through suitable components. The components may include, for example, audio/video jacks, data connectors, and the like.

The optical display module 1100 is used for presenting images to a user under the control of the processor 110. The optical display module 1100 may convert the real pixel image display into the virtual image display of the near-eye projection through one or several optical devices such as a reflector, a transmission mirror or an optical waveguide, so as to implement virtual interaction experience, or implement interaction experience combining the virtual and the reality. For example, the optical display module 1100 receives the image data information sent by the processor 110 and presents a corresponding image to the user.

In some embodiments, the VR wearing device 100 may further include an eye-tracking module 1200, the eye-tracking module 1200 being configured to track movement of the human eye, thereby determining the gaze point of the human eye. For example, the pupil position can be located by an image processing technology, and the pupil center coordinates can be obtained, so that the gaze point of the person can be calculated.

For convenience of understanding, the following describes a display method according to an embodiment of the present application, taking VR wearing equipment shown in fig. 7 as an example of VR glasses.

In the embodiment of the application, the VR glasses display an image stream, wherein the image stream comprises an ith frame image, a jth frame image and a kth frame image. The definition of the first object on the ith frame image is the first definition, the definition of the first object on the jth frame image is the second definition, and the definition of the first object on the kth frame image is the third definition. The average value of the first definition and the second definition is a third definition, the third definition is higher than the first threshold value, the average value of the second definition and the third definition is a fourth definition, and the fourth definition is higher than the second threshold value. Like this, can guarantee that the people's eye wears VR glasses and watches the image stream, on the people's eye visual perception, the definition of first object is unlikely to be too low. Illustratively, the display time interval of the i-th frame image and the j-th frame image is less than the user visual dwell time. In some embodiments, j=i+1, i.e., the j-th frame is the next frame to the i-th frame, in other embodiments, j=i+n, n >1.n may be determined based on the user visual dwell time, the image refresh frame rate. Assuming that the user visual dwell time is T and the image refresh frame rate is Z, then a T x Z frame image may be displayed during T time, and then n is less than or equal to T x Z. The visual retention time T may be any value ranging from 0.1s to 3s, or may be set by a user, which is not limited by the embodiment of the present application.

In some embodiments, the first threshold may be a user-defined value and the second threshold may be a user-defined value. Taking the first threshold as an example, the user may set a threshold such that when the VR eye displays the image stream, the sharpness of the first object is not lower than the threshold on the visual perception of the human eye. In this way, the user can customize the definition of the object in the virtual environment, and experience is high.

In other embodiments, the first threshold may be an average of the sharpness of the first object on the original image of the i-th frame and the sharpness of the first object on the original image of the j-th frame. The original image of the ith frame is the original image corresponding to the image of the ith frame (or called a new image of the ith frame), namely, the original image of the ith frame is subjected to image post-processing to obtain the image of the ith frame; the original image of the j-th frame corresponds to the image of the j-th frame (or called a new image of the j-th frame), namely, the image of the j-th frame is obtained by performing image post-processing on the original image of the j-th frame. Taking fig. 5 as an example, assuming that the first object is located in the non-gazing area, the first threshold may be an average value of the resolution of the non-gazing area on the original image of the ith frame and the resolution of the non-gazing area on the original image of the jth frame, that is, (b+c)/2. Similarly, the second threshold may be an average of the sharpness of the first object on the original image of the j-th frame and the sharpness of the first object on the original image of the k-th frame. The original image of the jth frame corresponds to the image of the jth frame (or called a new image of the jth frame), namely, the image of the jth frame is processed after the original image of the jth frame is processed to obtain the image of the jth frame; the original image of the kth frame corresponds to the image of the kth frame (or called a new image of the kth frame), namely the image of the kth frame is obtained by performing image post-processing on the original image of the kth frame. Taking fig. 5 as an example, assuming that the first object is located in the non-gazing area, the second threshold may be an average value of the resolution of the non-gazing area on the original image of the j-th frame and the resolution of the non-gazing area on the original image of the k-th frame, that is, (b+c)/2. In this way, the sharpness of the first object in the virtual environment as seen by the user is improved.

The image post-processing procedure is described below.

Referring to fig. 8, the image generating apparatus renders N frames of original pictures. Including the ith frame artwork, the jth frame artwork, the kth frame artwork, and so on. Let the resolution of the i frame artwork be P, the resolution of the j frame artwork be Q, Q less than P, the resolution of the k frame artwork be P. Then the resolution of the original image of the ith frame is raised to P ', the resolution of the original image of the jth frame is raised to Q ', and the resolution of the original image of the kth frame is raised to P ' by the image post-processing method provided by the application.

For example, taking the ith frame original image and the jth frame original image as examples, please refer to fig. 9, the flow of the image post-processing includes:

(1) And acquiring an ith frame original image which is an image H.

(2) And carrying out blurring processing on the image H to obtain an image L, wherein the resolution of the image L is the same as that of the image H.

One implementation manner, the step (2) includes: the image H is downsampled, and then upsampled to obtain the image L such that the resolution of the image L is the same as the resolution of the image H. For ease of understanding, downsampling and upsampling will be described. Downsampling is also called downsampling, which means that for an image with a resolution of m×n, s times downsampling is performed on the image to obtain an image with a resolution of (M/s) ×n/s), in other words, all pixels in an upper s×s window of the image with a resolution of m×n are changed into one pixel, and the value of the pixel point is the average value of all pixels in the window. It follows that after downsampling, details of the image are lost and the image resolution is reduced. The specific value of s is not limited, and may be set by default or set by a user. Upsampling, also called upsampling, refers to the process of amplifying an image, for example, by interpolating, i.e., inserting new elements between pixels based on the original image pixels using a suitable interpolation algorithm. Among them, there are various interpolation algorithms, such as an edge-based image interpolation algorithm and a region-based image interpolation algorithm, and the present application is not limited thereto.

For example, referring to fig. 10, the image H includes a plurality of pixels (the pixels are represented by black solid dots), and the downsampling process is performed on the image H, for example, each 9 pixels on the image H are processed into 1 pixel, that is, each 9 black solid dots on the image H are processed into one black empty dot, that is, the resolution of the image H is reduced. Then, with continued reference to fig. 10, the downsampled image is then upsampled, that is, the pixels on the image are interpolated by an interpolation method to obtain an image L, so that the number of pixels of the image L is the same as that of the pixels of the image H, that is, the resolution is the same. It should be noted that, in the downsampling process, details of the image are lost, and although the resolution of the image remains unchanged through upsampling processing, details in the image cannot be completely recovered, which may be understood that details of the image L are lost compared with those of the image H.

(3) From the image H and the image L, an image I is calculated, which satisfies i=2h—l.

For example, referring to fig. 10, the image H includes a first pixel, the image L includes a second pixel, and the second pixel corresponds to the first pixel. The image I comprises a third pixel point, and the third pixel point corresponds to the first pixel point and the second pixel point respectively. Moreover, the pixel information of the third pixel point is equal to the pixel information of the first pixel point multiplied by 2 minus the pixel information of the second pixel point. The pixel information includes, for example, color information, and may be color information of 0 to 255, for example, RGB (red), G (green), B (blue). It should be noted that, the pixel point on the image L is information obtained after blurring, this part of information is generally low-frequency information, the pixel point on the image H is information which is not subjected to blurring processing, including high-frequency information, 2 times of the pixel information of the pixel point on the image H, the pixel information of the high-frequency information can be added, if the pixel information of the pixel point on the image L is subtracted, the low-frequency information can be removed, so that the contrast of the image I can be improved.

It should be noted that, taking the image I satisfying i=2h—l as an example above, in some embodiments, the image I satisfying i=mh—l, m may be a value greater than or equal to 1. The value of m can be set by default or user-defined. When the user sets m to take different values, R obtained in step (4) is different, and correspondingly, the image S displayed in step (5) is different, and the post-processing process of the original image (the image H') of the j-th frame is also affected (because R is different). Therefore, when the user-defined m value is different, the processing results of the ith frame original image and the jth frame original image are different, and the definition of the image obtained by fusing the processed ith frame original image and the processed jth frame original image is different. In other words, the sharpness of the image obtained by fusing the i-th frame image and the j-th frame image (i.e., the average value of the sharpness of the i-th frame image and the j-th frame image) varies with the value of m.

(4) An overflow value R of pixel information of a pixel point on the image I is determined.

Since in step (3) the pixel information of the third pixel point on the image I is 2 times the pixel information of the second pixel point and then subtracted by the pixel information of the second pixel point, an overflow value may occur, for example, the color information of the first pixel point is 250 and the color information of the second pixel point is 200, and the color information of the third pixel point is 250×2-200=300, but 300 is not in the range of 0-255, so that the overflow value of the third pixel point is determined to be 300-255=45. Here, taking the overflow value of the third pixel point on the image I as an example, the calculation principles of the overflow values corresponding to other pixel points are the same.

(5) An image S is displayed, the image S satisfying s=i-R.

That is, after the overflow value R is subtracted from the pixel information of the pixel point on the image I, the image S is obtained, and then the image S is displayed. Excessive sharpening of the pixel (e.g., the pixel having color information of 300) can be prevented by the step (5), so that the pixel information of the entire image is smoothed.

The above is the post-processing procedure for the i-th frame original (i.e., the image H), and the post-processing procedure for the j-th frame original (i.e., the image H') is described below. With continued reference to fig. 9, the post-processing of the image H' includes:

(1) And acquiring an original picture of the j frame, which is an image H'. The resolution of image H' is lower than image H.

(2) The image H 'is subjected to an enlargement process to obtain an image L'. The image L' has the same resolution as the image L.

For example, the image H 'may be up-sampled to increase the resolution, resulting in the image L'. For example, referring to fig. 11, the image H ' includes a plurality of pixels (represented by open dots), and the image H ' is up-sampled (i.e., pixel interpolation) to increase the number of pixels and improve the resolution, so as to obtain an image L '. The upsampling process is already described above, and is not repeated here.

(3) From the image L' and the image L that remains, an offset coefficient or a movement parameter W is determined.

Exemplary, W includes a movement parameter from a second pixel point on the image L to a fourth pixel point on the image L', where the fourth pixel point corresponds to the second pixel point, and the movement parameter satisfies the following formula:

wherein s is a custom parameter, and L and R are values stored in the process of processing the ith frame of image.

(4) And obtaining an image I' according to W and the reserved R. The image I 'satisfies L' +wr.

Illustratively, with continued reference to FIG. 11, the image I 'includes a fifth pixel thereon, and the image L' includes a fourth pixel thereon, the fifth pixel corresponding to the fourth pixel. Pixel information of the fifth pixel=pixel information of the fourth pixel+offset W of the fourth pixel with respect to the second pixel multiplied by the overflow value R corresponding to the third pixel.

(5) The image I' is displayed.

The above is the post-processing procedure for the original image of the j-th frame (i.e., the image H').

In the post-processing of the jth frame original image, intermediate results of the post-processing of the ith frame original image, for example, the image L, the overflow value R, or the like, in other words, the overflow portion R in the post-processing of the ith frame original image is supplemented to the jth new image, so that details of the jth frame new image are improved. Thus, when the human eyes fuse the new image of the ith frame and the new image of the jth frame, the definition is improved.

In fig. 8 to 11, the whole image of the i-th frame original image and the j-th frame original image is taken as an example, and in the present application, the first object on the i-th frame original image and the first object on the j-th frame original image may be post-processed, and the implementation principle is the same as that of the whole image. For example, referring to fig. 12, the ith frame artwork includes the first object thereon, and the jth frame artwork includes the first object thereon. Then, the post-processing procedure for the first object on the original picture of the ith frame comprises the following steps: the region where the first object is located on the original image of the i-th frame (for example, the region surrounded by the edge contour of the first object) is determined, and then the processing flow of the first line in fig. 9 is executed on the region, so that the resolution of the first object is improved to P'. The post-processing process for the first object on the original picture of the j-th frame comprises the following steps: the region where the first object is located on the original image of the j-th frame (for example, the region surrounded by the edge contour of the first object) is determined, and then the processing flow of the second line in fig. 9 is executed on the region, so that the resolution of the first object is improved to Q'. The principle of the post-processing process of the first object on the original image of the kth frame is the same, and repeated description is omitted.

As mentioned above, the first object may be any object on the image, which is described in different situations for ease of understanding.

Case 1, the first object is an object within the non-noted region.

For example, referring to fig. 13A, the rendered image stream includes an i-th frame original image, a j-th frame original image, and a k-th frame original image; each frame of original picture comprises a gazing area and a non-gazing area, and the gazing area on each frame of original picture is ultra-high definition. The non-gazing area on the original picture of the ith frame is high definition, the non-gazing area on the original picture of the jth frame is low definition, and the gazing area on the original picture of the kth frame is high definition, namely the non-gazing area on the original picture of different frames is switched between high definition and low definition. In this case, by the technical solution provided by the embodiment of the present application, the definition of the object in the non-noted area in the image stream of fig. 13A may be improved.

For example, referring to fig. 13B, the gazing area on the original image of the ith frame is ultra-high definition, and the non-gazing area is high definition. The non-fixation area is processed by the post-processing method (namely, the flow processing of the first line in fig. 9) provided by the application, so that the resolution is improved, and then the non-fixation area after the post-processing and the fixation area are synthesized into a complete image, so as to obtain a new image of the ith frame. With continued reference to fig. 13B, the gazing area on the jth frame of the original image is ultra-high definition, and the non-gazing area is low definition. The non-gazing area is processed by the post-processing method (namely, the processing flow of the second row in fig. 9) provided by the application, so that the resolution is improved, and then the processed non-gazing area and gazing area are combined into a complete image, so as to obtain a new image of the j frame. The definition of the object in the non-fixation area on the i frame new image and the j frame new image obtained in this way is improved.

In case 2, the first object is an object within the noted area.

For example, referring to fig. 14A, an image stream is rendered, where the image stream includes an ith frame artwork, a jth frame artwork, and a kth frame artwork; each frame of original picture comprises a gazing area and a non-gazing area, and the non-gazing area of each frame of original picture is low-definition. The gazing area on the original picture of the ith frame is low definition, the gazing area on the original picture of the jth frame is high definition, and the gazing area on the original picture of the kth frame is low definition, namely, the gazing areas on the original pictures of different frames are switched between high definition and low definition. In this case, by the technical solution provided by the embodiment of the present application, the definition of the object in the gaze area in the image stream of fig. 14A can be improved.

For example, referring to fig. 14B, the entire image of the original image of the i-th frame is rendered with low-definition, i.e. the resolution of the gazing area and the non-gazing area on the original image of the i-th frame are the same, and are both low-definition. The gazing area on the original picture of the j frame is high definition, and the non-gazing area keeps the original picture of the i frame, namely low definition. For the i-th frame artwork, post-processing (e.g., the processing flow of the second line in fig. 9) may be performed on the gaze area, so that the resolution is improved. For non-fixation areas, no image post-processing may be done. And synthesizing the gazing area after post-treatment and the non-gazing area to obtain a new image of the ith frame. With continued reference to fig. 14B, for the jth frame artwork, the fixation area may be post-processed (i.e., the process flow of the first row in fig. 9) to increase its resolution. The low definition resolution of the ith frame may be followed for non-fixation areas. And synthesizing the gazing area and the non-gazing area after post-processing to obtain a new image of the j frame.

It should be noted that, the case 1 and the case 2 may be used alone or in combination, and the embodiment of the present application is not limited.

In case 3 and case 4, it is necessary to determine which is the far object and which is the near object. In some embodiments, the far object is located at a depth of field greater than the preset depth of field, and the near object is located at a depth of field less than the preset depth of field. The preset depth of field may be a specific depth of field value or a depth of field range, which is not limited in the embodiment of the present application. The preset depth of field is determined in a number of ways, including but not limited to at least one of the following ways.

In one mode, the preset depth of field may be determined according to VR scenes, where the preset depth of field is different according to VR scenes. The VR scene includes, but is not limited to, at least one of VR games, VR movies, VR teaching, and the like.

Taking VR games as an example, VR games include game pieces, and the depth of field may be determined according to the game pieces. For example, if the game is a first-person game, the preset depth of field may be a depth of field of a game character corresponding to the user in the game scene, or a depth of field of a body part on the game character corresponding to the user, or a depth of field of a game device currently held by the game character corresponding to the user. Assuming that the arm of the game piece holds the firearm, the depth of field of the arm or firearm can be determined to be the preset depth of field.

Taking VR viewing as an example, the VR viewing includes a display screen, so that the depth of field of the display screen can be determined as a preset depth of field. Taking VR teaching as an example, the VR teaching includes teaching equipment such as a blackboard, a display screen, a projection, and the like, and the depth of field of the teaching equipment can be determined to be a preset depth of field.

In the second mode, the preset depth of field may be set by a user, for example, the user may set the preset depth of field on VR glasses or an electronic device (such as a mobile phone) connected to the VR glasses. It should be appreciated that various VR applications may be included on the electronic device, and different VR applications may set different preset depths of field. Optionally, the user may set the preset depth of field of the VR application on the electronic device in batch, or may set the depth of field for each VR application separately.

In the third mode, the preset depth of field may also be a default depth of field, where the default depth of field may be understood as a default setting of the VR glasses, or a default setting of an electronic device (such as a mobile phone) connected to the VR glasses, or a default setting of a VR application currently running on the electronic device (such as a mobile phone) connected to the VR glasses, etc., which is not limited by the embodiments of the present application.

In the fourth mode, the preset depth of field can also be according to the depth of field of the subject object in the picture currently displayed by the VR glasses. The subject object may include an object occupying the largest area in the screen, or an object located in a central position in the screen, or a virtual object (such as a UI interface) in the screen, or the like.

The fifth mode, the preset depth of field is the depth of field where the user's gaze point is located. The VR glasses can comprise an eye movement tracking module, the eye movement tracking module can determine the user's gaze point, and the depth of field of the user's gaze point is determined to be a preset depth of field.

The above is a few determination manners of the preset depth of field, and other manners are possible, which are not limited by the embodiments of the present application.

In some embodiments, the technical scheme provided by the application is used by the electronic device when a certain trigger condition is met. For example, taking fig. 8 as an example, there are other images (not shown in fig. 8) before the original image of the i-th frame, and these images may be processed by using the image post-processing method provided in the embodiment of the present application from the i-th frame image without performing image post-processing. That is, the image post-processing method provided by the present application is not used for part of the images in the image stream, and is used when a certain trigger condition is detected to be satisfied.

For example, the trigger conditions include: the user trigger is detected for initiating the sharpness optimization mode. The electronic device has a normal mode and a sharpness optimization mode, for example. In the normal mode, after the image stream is electronically rendered, the image stream is displayed by a display device, and the definition in the virtual environment seen by the user is low. When the electronic equipment starts a definition optimization mode, the electronic equipment performs image processing on the rendered image stream to improve the definition of the rendered image stream, so that the definition of the virtual environment seen by a user is improved, and the user experience is improved.

For another example, the triggering condition further includes: it is detected that the user viewing time is greater than the first duration. That is, when the electronic device determines that the user watching time is long, the technical scheme provided by the application is automatically used, so that the definition of the virtual environment seen by the user is improved.

For another example, the triggering condition further includes: and detecting that the eyes of the user blink or the number of the blinks is larger than a preset number in the second time period. It can be understood that if the definition of the object seen by the user is low, the user may unconsciously blink to try to see the object clearly, so when the electronic device detects that the number of blinks or squints of eyes of the user is greater than the preset number in the second period, the technical scheme provided by the application is automatically used to improve the definition of the virtual environment seen by the user.

While the description of the present application will be presented in conjunction with some embodiments, it is not intended that the features of this application be limited to only this embodiment. Rather, the purpose of the application described in connection with the embodiments is to cover other alternatives or modifications, which may be extended by the claims based on the present description. In order to provide a thorough understanding of the present specification, the following description will contain numerous specific details. The present description may be practiced without these specific details. Furthermore, some specific details are omitted from the description in order to avoid obscuring the description or obscuring the focus of the present description. It should be noted that, without conflict, the embodiments and features of the embodiments in the present specification may be combined with each other.

Based on the same concept, fig. 15 shows an electronic device 1500 provided by the present application. The electronic device 1500 may be VR glasses as previously described. As shown in fig. 15, the electronic device 1500 may include: one or more processors 1501; one or more memories 1502; a communication interface 1503, and one or more computer programs 1504, which may be connected via one or more communication buses 1505. Wherein the one or more computer programs 1504 are stored in the memory 1502 and configured to be executed by the one or more processors 1501, the one or more computer programs 1504 include instructions that can be used to perform the steps associated with VR glasses as in the corresponding embodiments above. The communication interface 1503 is used to enable communication with other devices, such as a transceiver.

In the embodiments of the present application described above, the method provided in the embodiments of the present application is described from the point of view of an electronic device (e.g., VR glasses) as an execution subject. In order to implement the functions in the method provided by the embodiment of the present application, the electronic device may include a hardware structure and/or a software module, where the functions are implemented in the form of a hardware structure, a software module, or a hardware structure plus a software module. Some of the functions described above are performed in a hardware configuration, a software module, or a combination of hardware and software modules, depending on the specific application of the solution and design constraints.

As used in the above embodiments, the term "when …" or "after …" may be interpreted to mean "if …" or "after …" or "in response to determination …" or "in response to detection …" depending on the context. Similarly, the phrase "at the time of determination …" or "if detected (a stated condition or event)" may be interpreted to mean "if determined …" or "in response to determination …" or "at the time of detection (a stated condition or event)" or "in response to detection (a stated condition or event)" depending on the context. In addition, in the above-described embodiments, relational terms such as first and second are used to distinguish one entity from another entity without limiting any actual relationship or order between the entities.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with the present embodiments are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc. The schemes of the above embodiments may be used in combination without conflict.

It is noted that a portion of this patent document contains material which is subject to copyright protection. The copyright owner has reserved copyright rights, except for making copies of patent documents or recorded patent document content of the patent office.

Claims

1. A display method, comprising:

displaying the N frames of images through a display device; n is a positive integer;

the definition of a first object on an ith frame image in the N frames of images is a first definition;

the definition of the first object on the j-th frame image in the N frames of images is second definition;

the definition of the first object on the kth frame of image in the N frames of images is third definition;

the first definition is larger than the second definition, the second definition is smaller than the third definition, and i, j and k are positive integers smaller than N, i is smaller than j and smaller than k;

the average value of the first definition and the second definition is a fourth definition, and the fourth definition is higher than a first threshold value;

the average of the second definition and the third definition is a fifth definition, which is higher than a second threshold.

2. The method according to claim 1, wherein the first threshold is a user setting and/or the second threshold is a user setting.

3. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the first threshold is an average value of the sixth definition and the seventh definition;

the sixth definition is the definition of the first object on the rendered original image of the ith frame, and the image of the ith frame is an image after the original image of the ith frame is processed; the sixth definition is less than the first definition;

the seventh definition is the definition of the first object on the rendered jth frame original image, the jth frame image is an image after the jth frame original image is processed, and the seventh definition is smaller than or equal to the second definition.

4. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the second threshold is an average value of the eighth definition and the ninth definition;

the eighth definition is the definition of the first object on the rendered jth frame original image, and the jth frame image is an image after the jth frame original image is processed; the eighth definition is less than the second definition;

the ninth definition is the definition of the first object on the rendered original image of the kth frame, the kth frame image is an image after the processing of the original image of the kth frame, and the ninth definition is smaller than or equal to the third definition.

5. The method according to any one of claims 1 to 4, wherein,

the first object is located in an area where the point of non-fixation is located, or,

the first object is located at a first depth of field, and the first depth of field is larger than a preset depth of field.

6. The method of claim 5, wherein the step of determining the position of the probe is performed,

the definition of the second object on the N frames of images is unchanged;

the second object is located in the region where the gaze point is located, or the second object is located in a second depth of field, and the second depth of field is smaller than the preset depth of field.

7. The method of claim 6, wherein the second object has a higher sharpness than the first object.

8. The method of claim 5, wherein the step of determining the position of the probe is performed,

the definition of the second object on the ith frame image is tenth definition;

the definition of the second object on the j-th frame image is eleventh definition;

the definition of the second object on the kth frame image is twelfth definition;

the second object is located in the region where the gaze point is located, or the second object is located in a second depth of field, and the second depth of field is smaller than the preset depth of field;

an average value of the tenth definition and the eleventh definition is thirteenth definition, and an average value of the eleventh definition and the twelfth definition is fourteenth definition;

The thirteenth definition is higher than the fourth definition, and the fourteenth definition is higher than the fifth definition.

9. The method according to any one of claims 1 to 4, wherein,

the first object is positioned in the region where the gaze point is positioned; or alternatively, the process may be performed,

the first object is located at a first depth of field, and the first depth of field is smaller than a preset depth of field.

10. The method of claim 9, wherein the step of determining the position of the substrate comprises,

the definition of the second object on the N frames of images is unchanged;

the second object is located in the area where the non-fixation point is located, or the second object is located in a second depth of field, and the second depth of field is larger than the preset depth of field.

11. The method of claim 10, wherein the second object has a sharpness that is lower than the sharpness of the first object.

12. The method of claim 9, wherein the step of determining the position of the substrate comprises,

the sharpness of the second object on the i-th frame image is tenth sharpness,

the sharpness of the second object on the j-th frame image is eleventh sharpness,

the second object is located in the area where the non-fixation point is located, or the second object is located in a second depth of field, and the second depth of field is larger than the preset depth of field;

the thirteenth definition is lower than the fourth definition, and the fourteenth definition is lower than the fifth definition.

13. The method according to any one of claims 5-12, wherein a user gaze point is unchanged during the displaying of the i-th frame image, the j-th frame image, the k-th frame image.

14. The method according to any one of claims 1 to 13, wherein,

the time interval between the display time of the ith frame image and the display time of the jth frame image is smaller than or equal to the visual stay time of a user; and/or the number of the groups of groups,

a time interval between a display time of the j-th frame image and a display time of the k-th frame image is less than or equal to the user visual stay time.

15. The method of any of claims 1-14, further comprising, prior to displaying the N frames of images by the display device:

detecting that a user triggers an operation for starting a definition optimization mode; the user viewing time is greater than at least one of the first duration, the second duration, the user eye blinking or the number of squints is greater than a preset number.

16. An electronic device, comprising:

a processor, a memory, and one or more programs;

wherein the one or more programs are stored in the memory, the one or more programs comprising instructions, which when executed by the processor, cause the electronic device to perform the method steps of any of claims 1-15.

17. A computer readable storage medium for storing a computer program which, when run on a computer, causes the computer to perform the method of any one of claims 1 to 15.

18. A computer program product comprising a computer program which, when run on a computer, causes the computer to perform the method of any of the preceding claims 1-15.