CN109674443B

CN109674443B - Pupil distance measuring method and terminal

Info

Publication number: CN109674443B
Application number: CN201710981332.6A
Authority: CN
Inventors: 王超; 姜帆; 林晓龙; 洪松涛; 张双垒; 赵晓林
Original assignee: Huawei Device Co Ltd
Current assignee: Huawei Device Co Ltd
Priority date: 2017-10-19
Filing date: 2017-10-19
Publication date: 2022-04-26
Anticipated expiration: 2037-10-19
Also published as: CN109674443A

Abstract

The embodiment of the invention discloses a pupil distance measuring method and a terminal, wherein the method comprises the following steps: the method comprises the steps that a terminal shoots a first image, wherein the first image comprises a first pupil and a second pupil of a detected user; the terminal acquires the depth information of the first pupil and/or the second pupil; and the terminal calculates the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image. The embodiment of the invention can simply and conveniently measure the interpupillary distance and has high measurement precision.

Description

Pupil distance measuring method and terminal

Technical Field

The invention relates to the technical field of terminals, in particular to a pupil distance measuring method and a terminal.

Background

Virtual Reality (VR) head-mounted display devices, for example, VR glasses are next-generation human-computer interaction terminal devices, and the matching accuracy of the interpupillary distance of the VR glasses and the interpupillary distance of the user can directly affect the user experience of the wearer. When the interpupillary distance of the VR glasses is not matched with the interpupillary distance of the user, the user may be dizzy or the magnification of the displayed content viewed by the user may be changed. To make VR glasses better fit the wearer, an accurate measurement of the user's interpupillary distance is required.

At present, people can use rulers, meters and the like to directly measure the interpupillary distance of a user, however, the method has low measurement precision and can not meet the requirement of VR glasses on the precision of the interpupillary distance far away. People also can adopt the interpupillary distance appearance to measure user's interpupillary distance, however, the interpupillary distance appearance needs the professional to control, and the user need go to the mechanism or the shop that have the interpupillary distance appearance and measure, and the measurement inefficiency realizes that the degree of difficulty is big.

Disclosure of Invention

The technical problem to be solved by the embodiments of the present invention is to provide a method and a terminal for measuring interpupillary distance, which can simply and conveniently measure interpupillary distance and have high measurement accuracy.

In a first aspect, an embodiment of the present invention provides a method for measuring interpupillary distance, including:

the method comprises the steps that a terminal shoots a first image, wherein the first image comprises a first pupil and a second pupil of a detected user;

the terminal acquires the depth information of the first pupil and/or the second pupil;

and the terminal calculates the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

By executing the method, the terminal calculates the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image by acquiring the first image containing the first pupil and the second pupil of the user to be measured and the depth information of the first pupil and/or the second pupil, so that the interpupillary distance can be simply and conveniently measured, and the measurement precision is high.

In a possible implementation manner, the implementation manner of the terminal acquiring the depth information of the first pupil and/or the second pupil may include:

the terminal acquires the depth information of the first pupil and/or the second pupil through a distance measuring device; or the like, or, alternatively,

and the terminal acquires the depth information of the first pupil and/or the second pupil through at least two cameras.

In a possible implementation manner, one embodiment of the terminal acquiring the depth information of the first pupil and/or the second pupil through the distance measuring device may be:

the terminal transmits a first optical signal to a detected user through the distance measuring device, receives a second optical signal of the first optical signal reflected by the detected user through the distance measuring device, and then calculates the depth information of the first pupil and/or the second pupil according to the time difference or the phase difference between the first optical signal and the second optical signal.

By executing the method, the terminal can measure the interpupillary distance when the measured user is just opposite to the camera, and the calculation is convenient and quick.

the terminal transmits a third optical signal to the first pupil through the distance measuring device, receives a fourth optical signal of the third optical signal reflected by the first pupil through the distance measuring device, and calculates a first distance between the terminal and the first pupil according to a time difference or a phase difference between the third optical signal and the fourth optical signal; calculating the depth information of the first pupil according to the first distance;

the terminal transmits a fifth light signal to the second pupil through the distance measuring device, receives a sixth light signal of the fifth light signal reflected by the second pupil through the distance measuring device, and calculates a second distance between the terminal and the second pupil according to the time difference or the phase difference between the fifth light signal and the sixth light signal; and calculating the depth information of the second pupil according to the second distance.

By executing the method, the terminal acquires the depth information of the two pupils through the distance measuring equipment, so that the accurate measurement of the pupil distance is realized when the user slants the camera.

In another possible implementation manner, the terminal captures a first image, and the at least two cameras acquire the depth information of the first pupil and the second pupil, where an implementation manner may be:

the terminal acquires a first image containing a first pupil and a second pupil of a detected user through a first camera, and acquires a second image containing the first pupil and the second pupil of the detected user through a second camera;

the terminal calculates the depth information of the first pupil according to the parallax of the first pupil in the first image and the second image and calculates the depth information of the second pupil according to the parallax of the second pupil in the first image and the second image.

By executing the method, the terminal acquires the depth information of the two pupils through the two cameras, so that when a user diagonally faces the cameras, the accurate measurement of the pupil distance is realized.

In yet another possible implementation, the actual distance L between the first pupil and the second pupil_relThe calculation formula of (2) is as follows:

wherein h is₁Is the depth information of the first pupil, h₂Is depth information of the second pupil, b₁Is a first pixel distance from a first pupil to a central line in the first image, b₂Is a second pixel distance, L, from the second pupil to the midline in the first image_pix＝b₁+b₂，L_pixThe pixel distance between a first pupil and a second pupil in the first image is defined, and f is the focal length of a camera for acquiring the first image; the central line is a line perpendicular to a connecting line between the central point of the first image and the first pupil imaging point and the second pupil imaging point in the first image.

By executing the method, the terminal calculates the pupil distance based on the depth information of the two pupils, the calculation is more accurate, and the accurate measurement of the pupil distance can be realized when the user slants the camera.

In another possible implementation manner, one embodiment of the terminal calculating the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image may be:

the terminal calculates the image magnification factor corresponding to the depth information according to the corresponding relation between the preset depth value and the image magnification factor; and further, calculating the actual distance between the first pupil and the second pupil according to the image magnification factor corresponding to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

By executing the method, the terminal calculates the interpupillary distance based on the corresponding relation between the preset depth value and the image magnification factor, so that the method is simpler and more convenient and has high measurement precision.

In another possible implementation manner, the terminal calculates an image magnification factor corresponding to the depth information according to a corresponding relationship between the preset depth value and the image magnification factor, and the method further includes:

the terminal shoots a calibration image containing the calibration object at a position with a preset depth value of the calibration object, calculates the pixel size of the calibration object in the calibration image, and further calculates the image magnification factor corresponding to the preset depth value according to the actual size of the calibration object and the pixel size of the calibration object.

wherein L is_pixIs the pixel distance between the first pupil and the second pupil, h is depth information, h₀To preset depth value, A₀For a predetermined depth value h₀Corresponding image magnification, S_relTo calibrate the actual size of the object, S_pixTo be the pixel size of the calibration object,

In a second aspect, an embodiment of the present invention further provides a pupil distance measuring terminal, where the terminal includes:

the system comprises an image acquisition unit, a first image acquisition unit and a second image acquisition unit, wherein the image acquisition unit is used for shooting a first image which comprises a first pupil and a second pupil of a detected user;

a depth acquisition unit configured to acquire depth information of the first pupil and/or the second pupil;

and the processing unit is used for calculating the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

In the embodiment of the invention, the terminal calculates the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image by acquiring the first image containing the first pupil and the second pupil of the user to be measured and the depth information of the first pupil and/or the second pupil, so that the interpupillary distance can be simply and conveniently measured, and the measurement precision is high.

In a possible implementation manner, the depth obtaining unit is specifically configured to:

acquiring depth information of the first pupil and/or the second pupil through a distance measuring device; or the like, or, alternatively,

and acquiring the depth information of the first pupil and/or the second pupil through at least two cameras.

the first optical signal is transmitted to the detected user through the distance measuring device, the second optical signal of the first optical signal reflected by the detected user is received through the distance measuring device, and then the depth information of the first pupil and/or the second pupil is calculated according to the time difference or the phase difference of the first optical signal and the second optical signal.

In another possible implementation manner, the depth obtaining unit is specifically configured to:

emitting a third light signal to the first pupil through a distance measuring device, receiving a fourth light signal of the third light signal reflected by the first pupil through the distance measuring device, and calculating a first distance between the terminal and the first pupil according to a time difference or a phase difference between the third light signal and the fourth light signal; calculating the depth information of the first pupil according to the first distance;

emitting a fifth light signal to the second pupil through a distance measuring device, receiving a sixth light signal of the fifth light signal reflected by the second pupil through the distance measuring device, and calculating a second distance between the terminal and the second pupil according to a time difference or a phase difference between the fifth light signal and the sixth light signal; and calculating the depth information of the second pupil according to the second distance.

In yet another possible implementation form of the method,

the image acquisition unit is specifically configured to: shooting a first image through a first camera and shooting a second image through a second camera; the first image and the second image both comprise a first pupil and a second pupil of a detected user;

the depth acquisition unit is specifically configured to: calculating the depth information of the first pupil according to the parallax of the first pupil in the first image and the second image; and calculating the depth information of the second pupil according to the parallax of the second pupil in the first image and the second image.

wherein h is₁Is the depth information of the first pupil, h₂Is the depth information of the second pupil, b₁Is a first pixel distance, b, from the first pupil to a midline in the first image₂A second pixel distance, L, from the second pupil to the midline in the first image_pix＝b₁+b₂，L_pixThe pixel distance between the first pupil and the second pupil in the first image is defined, and f is the focal length of a camera for acquiring the first image; the central line is a line which is perpendicular to a connecting line between the first image central point and the first pupil imaging point in the first image and the second pupil imaging point.

In another possible implementation manner, the processing unit is specifically configured to:

calculating the image magnification corresponding to the depth information according to the corresponding relation between the preset depth value and the image magnification;

and calculating the actual distance between the first pupil and the second pupil according to the image magnification corresponding to the depth information in the first image and the pixel distance between the first pupil and the second pupil in the first image.

In another possible implementation manner, the terminal further includes a calibration unit, configured to:

shooting a calibration image containing a calibration object at a position with a preset depth value of the calibration object, and calculating the pixel size of the calibration object in the calibration image;

and calculating the image magnification corresponding to the preset depth value according to the actual size of the calibration object and the pixel size of the calibration object.

wherein L is_pixIs the pixel distance between the first pupil and the second pupil, h is the depth information, h₀For the preset depth value, A₀For a predetermined depth value h₀The corresponding magnification of the image is selected,

wherein S is_relIs the actual size of the calibration object, S_pixIs the pixel size of the calibration object.

In a third aspect, an embodiment of the present invention further provides a pupil distance measuring terminal, including a processor, a memory, at least one camera, and a distance measuring device; the processor is connected to the memory, the camera and the distance measuring device through a bus; the camera is used for shooting a first image; the first image comprises a first pupil and a second pupil of the detected user; the distance measuring device is used for acquiring the depth information of the first pupil and/or the second pupil; the processor executes the program in the memory to cause the terminal to perform the method of causing the terminal to perform some or all of the method of the first aspect.

In a fourth aspect, an embodiment of the present invention further provides a pupil distance measuring terminal, where the terminal includes: the system comprises a processor, a memory, a first camera and a second camera; the processor is connected to the memory, the first camera and the second camera through a bus; the first camera is used for shooting a first image, the second camera is used for shooting a second image, the first image and the second image comprise a first pupil and a second pupil of a detected user, and depth information of the first pupil and/or the second pupil is/are obtained according to the first image and the second image; the processor executes the program in the memory to cause the terminal to perform the method of causing the terminal to perform some or all of the method of the first aspect.

In a fifth aspect, an embodiment of the present invention further provides a computer storage medium, where the storage medium is configured to store instructions, and when the instructions are run on a pupil distance measuring terminal, the pupil distance measuring terminal is configured to perform some or all of the methods in the first aspect.

In a sixth aspect, an embodiment of the present invention further provides a computer program, where the computer program is storable in a computer storage medium, and when the computer program runs on a pupil distance measuring terminal, the computer program causes the pupil distance measuring terminal to execute part or all of the method in the first aspect.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments or the background art of the present invention, the drawings required to be used in the embodiments or the background art of the present invention will be described below.

Fig. 1 is a schematic diagram illustrating a principle of measuring a pupil distance by a pupil distance measuring terminal according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating a principle of calculating depth information of a first pupil according to a first distance according to an embodiment of the present invention;

fig. 3 is a schematic diagram of another pupil distance measuring terminal according to another embodiment of the present invention;

fig. 4 is a schematic diagram illustrating a pupil distance calculation method according to an embodiment of the present invention;

fig. 5 is a schematic flowchart of a method for measuring interpupillary distance according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a pupil distance measuring terminal according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of another interpupillary distance measuring terminal according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of another interpupillary distance measuring terminal according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of another interpupillary distance measuring terminal according to an embodiment of the present invention.

Detailed Description

The embodiments of the present invention will be described below with reference to the drawings.

The following describes a depth information acquisition method according to an embodiment of the present invention:

a first acquisition mode of depth information:

referring to fig. 1, fig. 1 is a schematic diagram illustrating a principle of measuring a pupil distance by a pupil distance measuring terminal according to an embodiment of the present invention, where the pupil distance measuring terminal (also referred to as a terminal in the embodiment of the present invention) may include a camera and a distance measuring device, such as a laser distance measuring device (e.g., a laser radar). The terminal can shoot a first image through the camera, the first image comprises a first pupil and a second pupil of the detected user, meanwhile, the terminal can acquire depth information of the first pupil and/or the second pupil through the laser ranging device, and then the actual distance between the first pupil and the second pupil of the detected user is calculated according to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

The depth information of the first pupil is the actual distance from the first pupil to a first plane where the camera is located, and the first plane is parallel to an imaging plane of the camera. Similarly, the depth information of the second pupil is the actual distance from the second pupil to the first plane where the camera is located. The depth information used for calculating the actual distance may be the depth information of the first pupil or the depth information of the second pupil, or may be a depth value calculated by the depth information of the first pupil or the depth information of the second pupil, for example, the depth value is an average value of the depth information of the first pupil and the depth information of the second pupil.

In an embodiment of the invention, the laser ranging device can be fixed on the interpupillary distance measuring terminal, and the laser ranging device and the camera are arranged on the same side. Generally, when the pupil distance is measured, the measured user is facing the camera, the depth information of the first pupil is equal to or slightly different from the depth information of the second pupil, and the distance information measured by the laser ranging device may be used as the depth information of the first pupil or the depth information of the second pupil.

It is understood that the laser ranging apparatus may include a laser transmitter and a laser receiver, the terminal may transmit a first optical signal to the user to be measured through the laser transmitter, receive a second optical signal reflected by the user to be measured through the laser receiver, and calculate distance information between the terminal and the user to be measured according to a time difference or a phase difference between the first optical signal and the second optical signal.

The second acquisition mode of the depth information:

however, when the detected user is not directly facing the camera, the depth information of the first pupil is not equal to the depth information of the second pupil, the distance information measured by the fixed laser ranging device may have a large difference with the depth information of the first pupil and the depth information of the second pupil, and if the distance information measured by the laser ranging device is used as the depth information, the actual distance between the first pupil and the second pupil may not be accurate enough.

In an embodiment of the present invention, the laser distance measuring device may further include a driving device, in addition to the laser transmitter and the laser receiver, where the driving device may receive a rotation instruction sent by the terminal to perform rotation control on the laser distance measuring device or the laser transmitter. Optionally, the laser distance measuring device may implement distance measurement or depth information measurement on a specific point, such as the first pupil, the second pupil, or another position point of the detected user, by combining with the image acquired by the camera, may also scan each point of the detected user to acquire depth information of the detected user, and may also generate a depth image according to the acquired depth information of the detected user, which is not limited in the present invention.

It can be understood that the terminal may transmit the third optical signal to the first pupil through the laser transmitter, receive the fourth optical signal reflected by the first pupil from the third optical signal through the laser receiver, and then calculate the first distance between the terminal and the first pupil according to the time difference or the phase difference between the third optical signal and the fourth optical signal, and calculate the depth information of the first pupil according to the first distance. Similarly, the terminal may transmit a fifth optical signal to the second pupil through the laser transmitter, receive a sixth optical signal reflected by the second pupil from the fifth optical signal through the laser receiver, calculate a second distance between the terminal and the second pupil according to a time difference or a phase difference between the fifth optical signal and the sixth optical signal, and calculate depth information of the second pupil according to the second distance.

Optionally, the manner of calculating, by the terminal, the depth information of the first pupil according to the first distance may be: the terminal can identify a first pupil in the first image, adjust the laser ranging device or the laser transmitter to rotate a first rotation angle according to the position of the first pupil in the first image and the relative position of the laser ranging device and the camera, and calculate the depth information of the first pupil according to the first rotation angle and the first distance.

For example, referring to fig. 2, fig. 2 is a schematic diagram illustrating a principle of calculating depth information of a first pupil according to a first distance. As shown in fig. 2, the camera 201 and the laser distance measuring device 202 are disposed on the same side of the terminal, and a laser emitting point of the laser distance measuring device 202 is on a first plane where a lens of the camera 201 is located, and the laser distance measuring device 202 may be located right below the camera 201, where the first plane is a plane parallel to an imaging plane of the camera. The laser distance measuring device 202 is controlled by the terminal to rotate by a first rotation angle, so that the light 203 emitted by the laser distance measuring device 202 forms an angle θ with the first plane 204, and the depth information d of the first pupil 205 is:

d＝d₁ sinθ

wherein d is₁Is a first distance. The first plane 204 is a plane parallel to the imaging plane 206 of the camera 201.

It should be noted that fig. 2 is only one arrangement manner of the camera and the laser device, and the positions of the camera and the laser distance measuring device according to the embodiment of the present invention may also include other arrangement manners, for example, the camera is located on any side of the terminal, and the laser distance measuring device is arranged on the top of the terminal, so that 360 ° rotation can be achieved, which is not limited by the present invention. The calculation mode of the depth information of the first pupil may further include other calculation modes, which are determined according to the position setting modes of the camera and the laser ranging device.

Similarly, the manner of calculating the depth information of the second pupil by the terminal according to the second distance may be: the terminal can identify a second pupil in the first image, adjust the laser ranging device or the laser transmitter to rotate a second rotation angle according to the position of the second pupil in the first image and the relative position of the laser ranging device and the camera, and calculate the depth information of the second pupil according to the second rotation angle and the second distance.

The third acquisition mode of the depth information:

referring to fig. 3, fig. 3 is a schematic diagram of another pupil distance measuring terminal according to an embodiment of the present invention, where the pupil distance measuring terminal may include at least two cameras, such as a first camera 301 and a second camera 302, the terminal may acquire a first image 305 including a first pupil 303 and a second pupil 304 through the first camera 301, acquire a second image 306 including the first pupil 303 and the second pupil 304 through the second camera 302, and further calculate depth information of the first pupil 301 according to a parallax of the first pupil 301 in the first image 305 and the second image 306; and/or calculating the depth information of the second pupil 302 according to the parallax of the second pupil 302 in the first image 305 and the second image 306.

Where the parallax is the amount of displacement of the position of the measurement point in the first image 305 relative to the position of the measurement point in the second image 306. The measurement point may be the first pupil 303 or the second pupil 304.

For example, as shown in fig. 3, a first camera 301 and a second camera 302 are arranged in parallel, and the distance between the first camera 301 and the second camera 302 is b; the focal length of the first camera 301 is the same as that of the second camera 302, and both are f; then the process is completed.

Depth information h of the first pupil 303₁The calculation formula of (2) is as follows:

h₁＝bf/ΔX₁

wherein, Δ X₁Is the parallax of the first pupil 303 in the first image 305 and the second image 306.

Similarly, the depth information h of the second pupil 304₂The calculation formula of (2) is as follows:

h₂＝bf/ΔX₂

wherein, Δ X₂Is the parallax of the second pupil 304 in the first image 305 and the second image 306.

The following describes a calculation method of the actual distance between the first pupil and the second pupil according to the present invention:

the first pupil distance calculation method:

the terminal can calibrate the corresponding relation between the depth value and the image magnification, and the calibration method comprises the following steps:

taking a calibration object with a known size as a measured object, shooting a calibration image containing the calibration object at a position with a preset depth value of the calibration object by a terminal, calculating the pixel size of the calibration object in the calibration image by the terminal, and further calculating the image magnification factor corresponding to the preset depth value according to the actual size of the calibration object and the pixel size of the calibration object, wherein the preset depth value h is₀Corresponding image magnification A₀As the actual size S of the calibration object_relAnd the pixel size S of the calibration object_pixThe ratio of (a) to (b). Namely:

for a camera with a fixed focal length, the larger the depth information of the measured object is, the smaller the size of the measured object in the shot formed image is, i.e. the larger the image magnification of the measured object is. That is, the depth information of the object to be measured is proportional to the image magnification of the object to be measured acquired by the camera. When the terminal acquires the depth information h of the detected user, the image magnification factor A corresponding to the depth information h is as follows:

furthermore, the depth information h of the tested user and the pixel distance L between the first pupil and the second pupil of the tested user are known_pixThen, the real distance L between the first pupil and the second pupil of the tested user can be calculated_rel：

The second pupil distance calculation method:

the terminal can be used for measuring the depth information h of the user to be measured and the pixel distance L between the first pupil and the second pupil in the first image_pixCalculating the actual distance L between the first pupil and the second pupil by the focal length f of the camera and the like_rel：

Wherein L is_pixThe pixel distance between the first pupil and the second pupil in the first image is shown.

In the first pupil distance calculation method and the second pupil distance calculation method, the depth information h of the first pupil is calculated based on the depth information h of the first pupil₁Depth information h of the second pupil₂The depth information h may be the depth information h of the first pupil when equal or not much different₁Or depth information h of the second pupil₂(ii) a Depth information h at the first pupil₁Depth information h of the second pupil₂When they are not equal, the depth information h may be the depth information h of the first pupil₁Depth information h of the second pupil₂Average value.

The third pupil distance calculation method:

referring to fig. 4, fig. 4 is a schematic diagram illustrating a pupil distance calculation method according to an embodiment of the present invention.

Distance h₁The distance from the first pupil 401 to the first plane 403, i.e. the depth information of the first pupil 401; distance h₂The distance from the second pupil 402 to the first plane 403, i.e. the depth information of the second pupil 402; point O is the lens position of the camera, the first plane 403 is a plane passing through point O and parallel to the imaging plane of the camera, point S1 is the imaging point of the first pupil 401 at the imaging plane 404, and point S2 is the imaging point of the second pupil 402 at the imaging plane 404; distance b₁Is a first pixel distance, distance b, of a first pupil 401 in the first image 405 to the midline 406₂The second pixel distance from the second pupil 402 in the first image 405 to the middle line 406, and the middle line 406 is a line perpendicular to the connection line between the center point C of the first image 405 and the imaging point S1 of the first pupil 401 and the imaging point S2 of the second pupil 402 in the first image.

From the triangle similarity principle, it can be derived that the depth information h at the first pupil₁Depth information h of the second pupil₂When the distances are not equal, the actual distance L between the first pupil and the second pupil_relThe calculation formula of (2) is as follows:

wherein h is₁Is the depth information of the first pupil, h₂Is the depth information of the second pupil, b₁Is a first pixel distance, b, from the first pupil to a midline in the first image₂A second pixel distance, L, from the second pupil to the midline in the first image_pix＝b₁+b₂，L_pixThe pixel distance between the first pupil and the second pupil in the first image is defined, and f is the focal length of a camera for acquiring the first image; the central line is a line passing through the central point of the first image and perpendicular to the connecting line of the first pupil and the second pupil in the first image。

The fourth pupil distance calculation method:

the third acquisition mode corresponding to the depth information is combined with the third interpupillary distance calculation mode to obtain the following interpupillary distance calculation mode, namely h₁＝bf/ΔX₁，h₂＝bf/ΔX₂Is substituted into

It can be derived that:

wherein, b₁、b₂、ΔX₁、ΔX₂The coordinates of the imaging point of the first pupil on the first image and the coordinates of the imaging point of the second pupil on the first image can be calculated.

The following describes a first pupil distance measurement method according to an embodiment of the present invention:

referring to fig. 5, fig. 5 is a schematic flowchart of a pupil distance measuring method according to an embodiment of the present invention,

step S510: the terminal captures a first image.

The first image comprises a first pupil and a second pupil of the detected user.

Step S520: the terminal acquires the depth information of the first pupil and/or the second pupil.

The manner in which the terminal obtains the depth information may refer to any one of the above manners for obtaining the depth information, which is not described in detail herein.

It can be understood that the terminal may identify the first pupil and the second pupil in the first image through an image identification algorithm, acquire the position coordinate of the imaging point of the first pupil on the first image and the position coordinate of the imaging point of the second pupil on the first image, and further calculate the pixel distance between the first pupil and the second pupil in the first image according to the acquired position coordinates of the imaging points.

It should be noted that, the execution sequence of step S510 and step S520 may not be sequential, or may be executed simultaneously, and the present invention is not limited thereto.

Optionally, the terminal acquires depth information of the first pupil and/or the second pupil while taking the first image.

For example, the terminal may acquire the depth information of the first pupil and/or the second pupil through the distance measuring device while taking the first image through the camera. For details, reference may be made to the related description in the first obtaining manner of the depth information, and the description of the present invention is omitted here for brevity.

For another example, the terminal may capture the first image through the camera, identify the position of the first pupil and/or the second pupil included in the first image through an image recognition algorithm, and instruct the distance measuring device to acquire the depth information of the first pupil and/or the second pupil according to the position of the first pupil and/or the second pupil. For details, reference may be made to the related description in the second obtaining manner of the depth information, and the description of the present invention is omitted here for brevity.

For another example, the terminal may capture the first image and the second image simultaneously by the first camera and the second camera. For details, reference may be made to the related description in the third obtaining manner of the depth information, and the description of the present invention is omitted.

Step S530: and the terminal calculates the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

It can be understood that the depth information includes depth information of the first pupil and/or depth information of the second pupil, and the calculation method of the actual distance between the first pupil and the second pupil may refer to the first pupil distance measurement method and the second pupil distance measurement method, which is not described in detail herein. The depth information used for calculating the interpupillary distance may be depth information of any point on a plane where the detected user is located, or may be depth information of the first pupil or depth information of the second pupil, or may be depth information obtained by calculation according to the depth information of the first pupil and the depth information of the second pupil, for example, an average value of the depth information of the first pupil and the depth information of the second pupil.

Optionally, the depth information includes depth information of the first pupil and depth information of the second pupil, and the calculation method of the actual distance between the first pupil and the second pupil may refer to the third pupil distance measurement method and the fourth pupil distance measurement method, which is not described in detail herein.

In an embodiment of the present invention, an implementation manner of acquiring the depth information of the first pupil and/or the second pupil by the terminal may include:

In an embodiment of the present invention, an implementation manner of acquiring the depth information of the first pupil and/or the second pupil by the terminal through the distance measuring device may include: the terminal transmits a first optical signal to a detected user; the terminal receives a second optical signal of the first optical signal reflected by the tested user; the terminal calculates the depth information of the first pupil and/or the second pupil according to the time difference or the phase difference of the first light signal and the second light signal. For details, reference may be made to the related description in the first obtaining manner of the depth information, and the description of the present invention is omitted here for brevity.

In an embodiment of the present invention, the depth information includes depth information of a first pupil and depth information of a second pupil; the terminal acquires the depth information of the first pupil and/or the second pupil through the distance measuring device, and one embodiment of the method may include: the terminal transmits a third optical signal to the first pupil through the distance measuring device, receives a fourth optical signal of the third optical signal reflected by the first pupil through the distance measuring device, calculates a first distance between the terminal and the first pupil according to a time difference or a phase difference between the third optical signal and the fourth optical signal, and calculates depth information of the first pupil according to the first distance; the terminal transmits a fifth light signal to the second pupil through the distance measuring device, receives a sixth light signal of the fifth light signal reflected by the second pupil through the distance measuring device, calculates a second distance between the terminal and the second pupil according to a time difference or a phase difference between the fifth light signal and the sixth light signal, and calculates depth information of the second pupil according to the second distance. For details, reference may be made to the related description in the second obtaining manner of the depth information, and the description of the present invention is omitted here for brevity.

In an embodiment of the present invention, the terminal includes at least two cameras, the depth information includes depth information of the first pupil and/or depth information of the second pupil, and one implementation manner of the terminal capturing the first image and acquiring the depth information of the first pupil and/or the second pupil may be: the terminal acquires a first image containing a first pupil and a second pupil of a detected user through a first camera, and acquires a second image containing the first pupil and the second pupil of the detected user through a second camera; the terminal calculates the depth information of the first pupil according to the parallax of the first pupil in the first image and the second image and calculates the depth information of the second pupil according to the parallax of the second pupil in the first image and the second image. For details, reference may be made to the related description in the third obtaining manner of the depth information, and the description of the present invention is omitted.

In an embodiment of the present invention, the actual distance L between the first pupil and the second pupil_relThe calculation formula of (2) is as follows:

For details, reference may be made to the related description in the third pupil distance calculation mode, which is not repeated herein.

In an embodiment of the present invention, one implementation manner of calculating, by the terminal, an actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image may be: the terminal calculates the image magnification factor corresponding to the depth information according to the corresponding relation between the preset depth value and the image magnification factor, and further calculates the actual distance between the first pupil and the second pupil according to the image magnification factor corresponding to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

Optionally, before step S520, the terminal may further capture a calibration image including the calibration object at a preset depth value from the calibration object, and calculate a pixel size of the calibration object in the calibration image; and calculating the image magnification corresponding to the preset depth value according to the actual size of the calibration object and the pixel size of the calibration object.

Optionally, the actual distance L between the first pupil and the second pupil_relThe calculation formula of (2) is as follows:

wherein L is_pixIs the pixel distance between the first pupil and the second pupil, h is depth information, h₀To preset depth value, A₀For a predetermined depth value h₀The corresponding magnification of the image is selected,

wherein S is_relTo calibrate the actual size of the object, S_pixIs the pixel size of the calibration object.

For details, reference may be made to the related description in the second pupil distance calculation method, which is not repeated herein.

By implementing the embodiment of the invention, the terminal calculates the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image by acquiring the first image containing the first pupil and the second pupil of the user to be measured and the depth information of the first pupil and/or the second pupil, so that the interpupillary distance can be simply and conveniently measured, and the measurement precision is high.

The pupil distance measuring terminal related to the embodiment of the invention is introduced as follows:

referring to fig. 6, fig. 6 is a schematic structural diagram of a pupil distance measuring terminal according to an embodiment of the present invention, where the terminal 60 may include: an image acquisition unit 610, a depth acquisition unit 620, and a processing unit 620. Wherein the detailed description of each unit is as follows

An image acquisition unit 610, configured to capture a first image, where the first image includes a first pupil and a second pupil of a user to be detected;

a depth acquiring unit 620, configured to acquire depth information of the first pupil and/or the second pupil;

a processing unit 630, configured to calculate an actual distance between the first pupil and the second pupil according to the depth information and a pixel distance between the first pupil and the second pupil in the first image.

In a possible implementation manner, the depth obtaining unit 620 is specifically configured to:

In another possible implementation manner, the depth obtaining unit 620 is specifically configured to:

In yet another possible implementation form of the method,

the image obtaining unit 610 is specifically configured to: shooting a first image through a first camera and shooting a second image through a second camera; the first image and the second image both comprise a first pupil and a second pupil of a detected user;

the depth obtaining unit 620 is specifically configured to: calculating the depth information of the first pupil according to the parallax of the first pupil in the first image and the second image; and calculating the depth information of the second pupil according to the parallax of the second pupil in the first image and the second image.

wherein h is₁Is the depth information of the first pupil, h₂Is the depth information of the second pupil, b₁For the first pupil to midline in the first imageA first pixel distance of b₂A second pixel distance, L, from the second pupil to the midline in the first image_pix＝b₁+b₂，L_pixThe pixel distance between the first pupil and the second pupil in the first image is defined, and f is the focal length of a camera for acquiring the first image; the central line is a line which is perpendicular to a connecting line between the first image central point and the first pupil imaging point in the first image and the second pupil imaging point.

In another possible implementation manner, the processing unit 630 is specifically configured to:

In yet another possible implementation manner, the terminal further includes a calibration unit. Referring to fig. 7, fig. 7 is a schematic structural diagram of another pupil distance measuring terminal according to an embodiment of the present invention, where the terminal 70 includes, in addition to an image obtaining unit 610, a depth obtaining unit 620, and a processing unit 630: a calibration unit 640 for:

It should be noted that the implementation of each unit may also correspond to the corresponding description of the method embodiments shown in fig. 1 to 5.

In the terminal described in fig. 6 or fig. 7, the terminal obtains the first image including the first pupil and the second pupil of the user to be measured and the depth information of the first pupil and/or the second pupil, and calculates the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image, so that the interpupillary distance can be simply and conveniently measured, and the measurement accuracy is high.

Referring to fig. 8, fig. 8 is a schematic structural diagram of another pupil distance measuring terminal according to an embodiment of the present invention, where the terminal 80 includes: a processor 810, a memory 820, at least one camera 830, and a ranging device 840; the processor 810 is connected to the memory 820, the camera 830 and the distance measuring device 840 through a bus 850.

The Memory 820 includes, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), or a portable Read Only Memory (CD-ROM), and the Memory 820 is used for related program instructions and data.

The camera 830 is used to take a first image; the first image includes a first pupil and a second pupil of the user under test.

The distance measuring device 840 is used for acquiring depth information of the first pupil and/or the second pupil; ranging device 840 may include, but is not limited to, a laser ranging device, an infrared ranging device, and the like. Laser rangefinder (for example laser radar) can be fixed in on the interpupillary distance measurement terminal, and laser rangefinder sets up with camera 830 homonymy. The terminal may capture a first image including a first pupil and a second pupil of the detected user through the camera 830, and at the same time, the terminal may acquire depth information of the first pupil and/or the second pupil.

The laser ranging device may include, but is not limited to, a laser transmitter and a laser receiver. The laser ranging device can receive a measurement instruction sent by the equipment, control the laser transmitter to transmit a first optical signal to the user to be measured, and receive a second optical signal of the first optical signal reflected by the user to be measured through the laser receiver. The device can calculate the distance information between the device and the user to be tested according to the time difference or the phase difference between the first optical signal and the second optical signal.

Optionally, the laser distance measuring device may further include a driving device besides the laser transmitter and the laser receiver, and the driving device may receive a rotation instruction sent by the terminal to control the rotation of the laser distance measuring device. Alternatively, the laser distance measuring device may combine the first image acquired by the camera 830 to implement distance measurement or depth information measurement on a specific point, such as the first pupil, the second pupil, or another position point of the measured object, or may scan each point of the measured object to acquire depth information of the measured object, or may generate a depth image according to the acquired depth information of the measured object, which is not limited in the present invention.

The processor 810 may be one or more Central Processing Units (CPUs), and in the case that the processor 810 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 810 is configured to call the data and programs stored in the memory 820, and perform:

shooting a first image through a camera 830, wherein the first image comprises a first pupil and a second pupil of a measured object;

acquiring depth information of the first pupil and/or the second pupil through a distance measuring device 840;

and calculating the actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

Optionally, the executing, by the processor 810, the obtaining of the depth information of the first pupil and/or the second pupil by the distance measuring device 840 specifically includes:

the first optical signal is transmitted to the tested user through the distance measuring device 840, the second optical signal of the first optical signal reflected by the tested user is received through the distance measuring device 840, and the first pupil and/or the second pupil depth information is calculated according to the time difference or the phase difference between the first optical signal and the second optical signal.

emitting a third light signal to the first pupil through the distance measuring device 840, receiving a fourth light signal of the third light signal reflected by the first pupil through the distance measuring device 840, and calculating a first distance between the terminal and the first pupil according to a time difference or a phase difference between the third light signal and the fourth light signal; calculating the depth information of the first pupil according to the first distance;

emitting a fifth light signal to the second pupil through the distance measuring device 840, receiving a sixth light signal of the fifth light signal reflected by the second pupil through the distance measuring device 840, and calculating a second distance between the terminal and the second pupil according to a time difference or a phase difference between the fifth light signal and the sixth light signal; and calculating the depth information of the second pupil according to the second distance.

Optionally, the step of calculating, by the processor 810, an actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image specifically includes:

calculating the image magnification factor corresponding to the depth information according to the corresponding relation between the preset depth value and the image magnification factor;

and calculating the actual distance between the first pupil and the second pupil according to the image magnification corresponding to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

Optionally, before the processor 810 performs the step of calculating the image magnification corresponding to the depth information according to the corresponding relationship between the preset depth value and the image magnification, the following steps are further performed:

wherein L is_pixIs the pixel distance between the first pupil and the second pupil, h is depth information, h₀To a predetermined depthValue of A₀For a predetermined depth value h₀The corresponding magnification of the image is selected,

It should be noted that the implementation of each operation may also correspond to the related description of the method embodiments shown in fig. 1 to 5.

In the terminal described in fig. 8, the terminal captures a first image including a first pupil and a second pupil of a user to be measured through the camera, acquires depth information of the first pupil and/or the second pupil through the distance measuring device, and calculates an actual distance between the first pupil and the second pupil according to the depth information and a pixel distance between the first pupil and the second pupil in the first image, so that the interpupillary distance can be simply and conveniently measured, and the measurement precision is high.

Referring to fig. 9, fig. 9 is a schematic structural diagram of another pupil distance measuring terminal according to an embodiment of the present invention, where the terminal includes: a processor 910, a memory 920, at least one camera, such as a first camera 930 and a second camera 940; the processor 910 is connected to the memory 920, the first camera 930, and the second camera 940 through a bus 950. Wherein the content of the first and second substances,

the Memory 920 includes, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), or a portable Read Only Memory (CD-ROM), and the Memory 920 is used for related program instructions and data.

The first camera 930 is configured to capture a first image, and the second camera 940 is configured to capture a second image, where the first image and the second image include a first pupil and a second pupil of the detected user, so as to obtain depth information of the first pupil and/or the second pupil according to the first image and the second image. The first camera 930 and the second camera 940 may be disposed in parallel on the same side of the terminal, and the imaging plane of the first camera 930 and the imaging plane of the second camera 940 are parallel.

The processor 910 may be one or more Central Processing Units (CPUs), and in the case that the processor 910 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 910 is configured to call the data and programs stored in the memory 920, and perform:

taking a first image with a first camera 930 and a second image with a second camera 940, wherein the first image and the second image each comprise a first pupil and a second pupil of the user to be tested;

calculating the depth information of the first pupil according to the parallax of the first pupil in the first image and the second image; and/or calculating the depth information of the second pupil according to the parallax of the second pupil in the first image and the second image;

wherein h is₁Is the depth information of the first pupil, h₂Is depth information of the second pupil, b₁Is a first pixel distance from a first pupil to a central line in the first image, b₂Is a second pixel distance, L, from the second pupil to the midline in the first image_pix＝b₁+b₂，L_pixThe pixel distance between a first pupil and a second pupil in the first image is defined, and f is the focal length of a camera for acquiring the first image; the central line is the central point of the first image and the imaging of the first pupil in the first imageAnd a line perpendicular to a line connecting the points to the second pupil imaging point.

Optionally, the step of calculating, by the processor 910, an actual distance between the first pupil and the second pupil according to the depth information and the pixel distance between the first pupil and the second pupil in the first image specifically includes:

and calculating the actual distance between the first pupil and the second pupil according to the image magnification factor corresponding to the depth information and the pixel distance between the first pupil and the second pupil in the first image.

Optionally, the processor 910 performs the following steps of calculating an image magnification corresponding to the depth information according to the corresponding relationship between the preset depth value and the image magnification:

It should be noted that the implementation of the respective operations may also correspond to the corresponding description of the method embodiments shown in fig. 1 to 5.

In the terminal depicted in fig. 9, the terminal obtains a first image including the first pupil and the second pupil of the user to be tested through the first camera and a second image including the first pupil and the second pupil of the user to be tested through the second camera; calculating the depth information of the first pupil according to the parallax of the first pupil in the first image and the second image, and/or calculating the depth information of the second pupil according to the parallax of the second pupil in the first image and the second image; furthermore, the actual distance between the first pupil and the second pupil is calculated according to the depth information and the pixel distance between the first pupil and the second pupil in the first image, so that the interpupillary distance can be simply and conveniently measured, and the measurement precision is high.

It should be further noted that, in each embodiment of the present invention, the terminal may further include a user interface and/or a communication interface, and the terminal may implement information interaction with the user through the user interface, and the terminal may implement data connection with other devices through the communication interface. Alternatively, the user interface may include, but is not limited to, a display screen, a touch screen, a keyboard, and the like.

It should be further noted that, in each embodiment of the present invention, the terminal may be a mobile terminal or a fixed terminal that includes a camera and a laser ranging device, and may also be a mobile terminal or a fixed terminal that includes two cameras. The mobile terminal may include, but is not limited to, a smart phone, a tablet computer, a personal digital assistant, a Virtual Reality (VR) device, an Augmented Reality (AR) device, and the like; the fixed terminal may be, but is not limited to, a computer or the like.

One of ordinary skill in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by hardware related to instructions of a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the above method embodiments. And the aforementioned storage medium includes: various media capable of storing program codes, such as ROM or RAM, magnetic or optical disks, etc.

Claims

1. A method of measuring interpupillary distance, the method comprising:

the terminal acquires the depth information of the first pupil and the second pupil;

the terminal determines a first pixel distance from the first pupil to a middle line in the first image and a second pixel distance from the second pupil to the middle line in the first image, wherein the middle line is a line perpendicular to a connecting line between a central point of the first image and an imaging point of the first pupil in the first image and an imaging point of the second pupil in the first image; the terminal calculates the actual distance between the first pupil and the second pupil according to the depth information of the first pupil and the second pupil, the first pixel distance and the second pixel distance and the pixel distance between the first pupil and the second pupil in the first image;

an actual distance L between the first pupil and the second pupil_relThe calculation formula of (2) is as follows:

wherein h is₁Is the depth information of the first pupil, h₂Is the depth information of the second pupil, b₁Is the first pixel distance, b₂And f is the second pixel distance, and f is the focal length of the camera for acquiring the first image.

2. The method of claim 1, wherein the acquiring, by the terminal, the depth information of the first pupil and the second pupil specifically comprises:

the terminal acquires the depth information of the first pupil and the second pupil through a distance measuring device; or the like, or, alternatively,

and the terminal acquires the depth information of the first pupil and the second pupil through at least two cameras.

3. The method of claim 2, wherein the terminal acquiring the depth information of the first pupil and the second pupil through a distance measuring device comprises:

emitting a first light signal to the first pupil through the distance measuring device, receiving a second light signal of the first light signal reflected by the first pupil through the distance measuring device, and calculating a first distance between the terminal and the first pupil according to a time difference or a phase difference between the first light signal and the second light signal; calculating the depth information of the first pupil according to the first distance;

emitting a third light signal to the second pupil through the distance measuring device, receiving a fourth light signal of the third light signal reflected by the second pupil through the distance measuring device, and calculating a second distance between the terminal and the second pupil according to a time difference or a phase difference between the third light signal and the fourth light signal; and calculating the depth information of the second pupil according to the second distance.

4. The method of claim 2, wherein the terminal capturing the first image comprises:

the terminal shoots a first image through a first camera and shoots a second image through a second camera, wherein the first image and the second image both comprise a first pupil and a second pupil of a detected user;

acquiring the depth information of the first pupil and the second pupil through at least two cameras comprises:

the terminal calculates the depth information of the first pupil according to the parallax of the first pupil in the first image and the second image; and calculating the depth information of the second pupil according to the parallax of the second pupil in the first image and the second image.

5. A pupil distance measuring terminal, characterized in that the terminal comprises:

a depth acquisition unit configured to acquire depth information of the first pupil and the second pupil;

a processing unit, configured to determine a first pixel distance from the first pupil to a central line in the first image and a second pixel distance from the second pupil to the central line in the first image, where the central line is a line where a central point of the first image is perpendicular to a connection line between the first pupil imaging point and the second pupil imaging point in the first image; calculating the actual distance between the first pupil and the second pupil according to the depth information of the first pupil and the second pupil, the first pixel distance and the second pixel distance, and the pixel distance between the first pupil and the second pupil in the first image;

6. The terminal of claim 5, wherein the depth acquisition unit is specifically configured to:

acquiring depth information of the first pupil and the second pupil through a distance measuring device; or the like, or, alternatively,

and acquiring the depth information of the first pupil and the second pupil through at least two cameras.

7. The terminal according to claim 6, wherein the depth acquisition unit is specifically configured to:

8. The terminal of claim 6,

the image acquisition unit is specifically configured to: shooting a first image through a first camera and shooting a second image through a second camera; the first image and the second image each include a first pupil and a second pupil;

9. A pupil distance measuring terminal, characterized in that the terminal comprises: the system comprises a processor, a memory, at least one camera and a distance measuring device; the processor is connected to the memory, the camera and the distance measuring device through a bus; the camera is used for shooting a first image; the first image comprises a first pupil and a second pupil of the detected user; the distance measuring device is used for acquiring the depth information of the first pupil and/or the second pupil; the processor executes the program in the memory to cause the terminal to execute the pupil distance measuring method of any one of claims 1 to 3.

10. A pupil distance measuring terminal, characterized in that the terminal comprises: the system comprises a processor, a memory, a first camera and a second camera; the processor is connected to the memory, the first camera and the second camera through a bus;

the first camera is used for shooting a first image, the second camera is used for shooting a second image, the first image and the second image comprise a first pupil and a second pupil of a detected user, and depth information of the first pupil and/or the second pupil is/are obtained according to the first image and the second image; the processor executes the program in the memory to cause the terminal to execute the pupil distance measuring method according to any one of claims 1 to 2 and 4.

11. A computer storage medium for storing instructions that, when run on a interpupillary distance measurement terminal, cause the interpupillary distance measurement terminal to perform the method of any of claims 1-4.