WO2022152221A1 - Touch-control feedback system, terminal device, touch-control feedback control method, and storage medium - Google Patents

Touch-control feedback system, terminal device, touch-control feedback control method, and storage medium Download PDF

Info

Publication number
WO2022152221A1
WO2022152221A1 PCT/CN2022/071859 CN2022071859W WO2022152221A1 WO 2022152221 A1 WO2022152221 A1 WO 2022152221A1 CN 2022071859 W CN2022071859 W CN 2022071859W WO 2022152221 A1 WO2022152221 A1 WO 2022152221A1
Authority
WO
WIPO (PCT)
Prior art keywords
touch
ultrasonic
ultrasonic transmitter
array
target
Prior art date
Application number
PCT/CN2022/071859
Other languages
French (fr)
Chinese (zh)
Inventor
黄瑞
刘鸿
范超
韩东成
Original Assignee
安徽省东超科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202110055738.8A external-priority patent/CN112764592B/en
Priority claimed from CN202110057244.3A external-priority patent/CN112764593A/en
Application filed by 安徽省东超科技有限公司 filed Critical 安徽省东超科技有限公司
Publication of WO2022152221A1 publication Critical patent/WO2022152221A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/043—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures

Definitions

  • the present disclosure relates to the technical field of touch feedback, and in particular, to a touch feedback control system, a terminal device, a touch feedback control method, and a storage medium.
  • haptic perception is realized by using vibration, electrostatic force and other principles and technologies with the assistance of related devices (such as screens, gloves, etc.).
  • related devices such as screens, gloves, etc.
  • this method must be in contact with auxiliary equipment.
  • the cross-use of equipment will cause public health and safety problems, and users cannot get rid of the shackles of bulky equipment; on the other hand, after users use the equipment, fingerprints and palms left on the equipment Personal information such as tattoos will cause the risk of personal information security to be leaked.
  • an object of the present disclosure is to provide a touch feedback system, which can not only realize the user's non-contact tactile feedback, but also satisfy the user's interactive experience both visually and tactilely, which is more hygienic and safer.
  • the second purpose of the present disclosure is to provide a terminal device.
  • the third purpose of the present disclosure is to provide a touch feedback control method.
  • the fourth purpose of the present disclosure is to provide a storage medium.
  • the touch feedback system provided by the embodiment of the first aspect of the present disclosure includes an imaging subsystem, which is used for imaging and displaying the human-computer interaction information in the air target area to form an air interaction interface, and when it is detected that the user interacts with
  • the touch feedback control instruction and the touch point coordinate information are sent when the interaction signal of the air interaction interface is performed;
  • the haptic feedback subsystem is connected to the imaging subsystem, and is used for the touch feedback control instruction and the touch control according to the touch feedback.
  • the point coordinate information emits ultrasonic waves and focuses the ultrasonic waves on the touch points of the air interactive interface.
  • the human-computer interaction information is imaged and displayed in the air target area through the imaging subsystem to form an air interaction interface, that is, the imaging subsystem adopts an interactive aerial imaging technology, and the air interaction interface is used as tactile perception.
  • the reference plane is used to guide the user to touch, and when the imaging subsystem detects the interaction signal between the user and the air interface, it sends the touch feedback control command and touch point coordinate information to the haptic feedback subsystem.
  • the system emits ultrasonic waves and focuses the ultrasonic waves on the touch points of the air interface, that is, the haptic feedback subsystem uses ultrasonic waves to radiate pressure between the human body and the touch points.
  • the haptic feedback effect achieves the purpose of non-contact haptic feedback for the user, and in the embodiment of the present disclosure, based on presenting an air interactive interface in the air target area, the touch feedback operation can be triggered by the user touching the air interactive interface, which is visually and tactilely At the same time, it satisfies the user's interactive experience, and the operation mode is also more natural and comfortable. There is no need to set additional restrictions on the user's operation, avoiding the risk of the user touching the device during operation, and avoiding the leakage of personal information due to the user's information being left on the contact surface. problem, more hygienic and safer.
  • the haptic feedback subsystem includes: a first casing provided with a window and a first receiving cavity formed therein; an ultrasonic transmitter array, the ultrasonic transmitter array is arranged in the first accommodating cavity, and includes a plurality of ultrasonic transmitters, and the transmitting end of each ultrasonic transmitter faces the opening window; the controller is arranged in the first accommodating cavity, and is connected with the The ultrasonic transmitter array is connected to control each of the ultrasonic transmitters to emit ultrasonic waves according to the touch feedback control instruction and the coordinate information of the touch points, and focus the ultrasonic waves to the touch points.
  • the controller includes: a data processing module configured to determine the delay phase of each of the ultrasonic transmitters according to the coordinate information of the touch point and the position information of each of the ultrasonic transmitters;
  • a driving module connected with the ultrasonic transmitter array and the data processing module, is used for generating an original driving signal in response to the touch feedback control command, and adjusting the ultrasonic transmitter according to the delay phase of each ultrasonic transmitter
  • the original driving signal is used to focus the ultrasonic waves emitted by the plurality of ultrasonic wave transmitters to the touch point.
  • the driving module when responding to the touch feedback control instruction, the driving module generates a clock signal and a modulation signal according to the touch feedback control instruction, and generates a raw signal according to the clock signal and the modulation signal. drive signal.
  • the driving module performs delay processing on the original driving signal according to the delayed phase of each of the ultrasonic transmitters, so as to obtain a target driving signal of each of the ultrasonic transmitters, and performs delay processing on the original driving signal according to the delayed phase of each of the ultrasonic transmitters.
  • the target drive signal drives each of the ultrasonic transmitters.
  • the data processing module includes: a first storage unit for storing data and programs in a data processing process; a data processing unit, the data processing unit is connected to the first storage unit for The delay phase of each ultrasonic transmitter is determined according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter.
  • the drive module includes: a second storage unit for storing data and files in a drive control process; a control unit, the control unit is connected to the second storage unit, for The touch feedback control instruction generates a clock signal and a modulation signal, and generates an original drive signal according to the clock signal and the modulation signal, and delays the original drive signal according to the delay phase of each ultrasonic transmitter processing; a driving unit, the driving unit is connected to the control unit, and is used for amplifying the delayed original driving signal to obtain the target driving signal of each ultrasonic transmitter, and according to the target driving signal Each of the ultrasonic transmitters is driven.
  • the haptic feedback subsystem further includes: a blocking cover, disposed on the first housing, for blocking the window opening.
  • the imaging subsystem includes: a second casing, the second casing is formed with a display window and a second accommodating cavity is formed inside; an imaging component is disposed in the second accommodating cavity, It is used to image and display the human-computer interaction information in the air target area to form an air interaction interface; a detection module is used to detect the interaction signal between the user and the air interaction interface; the main control module is connected to the imaging component and the detection module. The connection is used for sending touch feedback control instructions and touch point coordinate information in response to the interaction signal.
  • the imaging assembly includes: a display, the display is disposed in the second accommodating cavity, is connected to the main control module, and is used for displaying the human-computer interaction information; an optical assembly, the optical The component is arranged in the second accommodating cavity, and is used for converging and imaging the light carrying the human-computer interaction information on the aerial target area to form an aerial interaction interface; wherein, the display is arranged on the light source of the optical component side, the display window is on the imaging side of the optical assembly.
  • the haptic feedback subsystem is disposed above the imaging subsystem and on the touch side of the air interface; the transmitting end of the ultrasonic transmitter array is disposed in parallel and opposite to the air interface ; The transmitting end of the ultrasonic transmitter array faces the air interactive interface.
  • the haptic feedback subsystem is disposed above the imaging subsystem and on the imaging side of the air interface, the ultrasound transmitter array is perpendicular to the optical assembly, and the ultrasound transmits The transmitter array is flush with the air interaction interface in the horizontal direction, and the transmitting end of the ultrasonic transmitter array faces the air interaction interface.
  • the haptic feedback subsystem is disposed in the second accommodating cavity, and is located on the reflection path of the optical component to the light carrying the human-computer interaction information, and the ultrasonic transmitter array The transmitting end faces the air interaction interface.
  • the haptic feedback subsystem is disposed above the imaging subsystem and on the touch side of the air interface, the ultrasonic transmitter array is disposed in parallel with the optical assembly, and the ultrasonic wave transmits The transmitter array is flush with the air interaction interface in the vertical direction, and the transmitting end of the ultrasonic transmitter array faces the air interaction interface.
  • Embodiments of the second aspect of the present disclosure provide a terminal device, including a device body; and the touch feedback system according to the foregoing embodiments, where the touch feedback system is provided on the device body.
  • the purpose of non-contact tactile feedback for the user can be achieved, and no additional restricting user operation device is required, and the operation method is also more natural and convenient. Satisfy the user's interactive experience both visually and tactilely.
  • a third aspect of the present disclosure provides a touch feedback control method, including: imaging and displaying human-computer interaction information in an aerial target area to form an aerial interactive interface; detecting an interaction signal between a user and the aerial interactive interface, and obtaining touch control point coordinate information and send touch feedback control instructions; control the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch control of the air interactive interface according to the touch feedback control instructions and the touch point coordinate information point.
  • an aerial interactive interface is formed by imaging and displaying human-computer interaction information in an aerial target area, that is, an interactive aerial imaging technology is adopted, and the aerial interactive interface is used as a reference surface for tactile perception, so as to It is used to guide the user to touch, and when the interaction signal between the user and the air interaction interface is detected, obtain the touch feedback control instruction and touch point coordinate information, so as to obtain the touch feedback control instruction and touch point coordinate information according to the acquired touch feedback control instruction and touch point coordinate information, Control the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch points of the air interactive interface, that is, through ultrasonic radiation pressure, to generate tactile feedback effects between the human body and the touch points, to achieve the purpose of non-contact tactile feedback for users, And in the embodiment of the present disclosure, based on the air interactive interface presented in the air target area, the touch feedback operation can be triggered by the user touching the air interactive interface, which satisfies the user's interactive experience both visually and
  • controlling the ultrasonic transmitter array to emit ultrasonic waves and focusing the ultrasonic waves to the touch points of the air interactive interface according to the touch feedback control instruction and the touch point coordinate information includes: The coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter in the ultrasonic transmitter array determine the delay phase of each ultrasonic transmitter; the original drive signal is generated according to the touch feedback control instruction; The original driving signal is adjusted according to the delayed phase of each of the ultrasonic transmitters, so as to focus the ultrasonic waves emitted by the plurality of ultrasonic transmitters to the touch point.
  • generating the original driving signal according to the touch feedback control instruction includes: generating a clock signal and a modulation signal according to the touch feedback control instruction; generating the original driving signal according to the clock signal and the modulation signal drive signal.
  • adjusting the original driving signal according to the delayed phase of each of the ultrasonic transmitters includes: delaying the original driving signal according to the delayed phase of each of the ultrasonic transmitters processing; obtaining a target driving signal of each of the ultrasonic transmitters; and driving each of the ultrasonic transmitters according to the target driving signal.
  • the method further includes: acquiring an ambient temperature; and correcting the delay phase of each of the ultrasonic transmitters according to the ambient temperature.
  • the ultrasonic transmitter array includes n ultrasonic transmitter sub-arrays distributed on the same plane, the air interactive interface is divided into n touch sub-areas, the n ultrasonic transmitter sub-arrays and the n ultrasonic transmitter sub-arrays
  • the number of ultrasonic transmitter arrays is n, and the n ultrasonic transmitter arrays are distributed on the same plane or staggered at a preset angle, and the air interactive interface is divided into n touch sub-areas, The n ultrasonic transmitter arrays are arranged in a one-to-one correspondence with the n touch sub-areas; the ultrasonic transmitter array is controlled to emit ultrasonic waves according to the touch feedback control instruction and the touch point coordinate information and transmit the ultrasonic waves to the ultrasonic wave.
  • Focusing on the touch point includes: determining, according to the coordinate information of the touch point, that the touch point corresponds to a target touch sub-area in the air interactive interface; determining a target ultrasonic transmitter array according to the target touch sub-area; The touch point coordinate information and the position information of the transmitting end of each ultrasonic transmitter in the target ultrasonic transmitter array determine the delay phase of each ultrasonic transmitter in the target ultrasonic transmitter array; according to the The touch feedback control instruction generates a clock signal and a modulation signal, and generates an original drive signal according to the clock signal and the modulation signal; according to the delay phase of each ultrasonic transmitter in the target ultrasonic transmitter array performing delay processing on the original driving signal to obtain a target driving signal for each of the ultrasonic transmitters in the target ultrasonic transmitter array; and driving each ultrasonic wave in the target ultrasonic transmitter array according to the target driving signal launcher.
  • Embodiments of the fourth aspect of the present disclosure provide a storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the touch feedback control method described in the foregoing embodiments is implemented.
  • FIG. 1 is a structural block diagram of a touch feedback system according to an embodiment of the present disclosure
  • FIG. 2 is a schematic structural diagram of a haptic feedback subsystem according to an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of an ultrasonic transmitter array according to one embodiment of the present disclosure.
  • FIG. 4 is a structural block diagram of a data processing module according to an embodiment of the present disclosure.
  • FIG. 5 is a structural block diagram of a driving module according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural diagram of a touch feedback system according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of human-computer interaction according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of an optical assembly according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic diagram of a first optical waveguide array and a second optical waveguide array according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic diagram of a front structure of an optical assembly along the thickness direction according to an embodiment of the present disclosure
  • FIG. 11 is a schematic partial structure diagram of a first optical waveguide array and a second optical waveguide array according to an embodiment of the present disclosure
  • FIG. 12 is a schematic diagram of an optical path of an optical assembly according to an embodiment of the present disclosure.
  • FIG. 13 is a schematic diagram of the arrangement of the ultrasonic transmitter array disposed above the second housing according to an embodiment of the present disclosure
  • FIG. 14 is a schematic diagram of the arrangement of the ultrasonic transmitter array disposed in the second accommodating cavity according to an embodiment of the present disclosure
  • 15 is a schematic diagram of an arrangement of an ultrasonic transmitter array disposed above the second housing according to another embodiment of the present disclosure
  • FIG. 16 is a structural block diagram of a terminal device according to an embodiment of the present disclosure.
  • FIG. 17 is a schematic flowchart of a touch feedback control method according to an embodiment of the present disclosure.
  • FIG. 18 is a schematic flowchart of a touch feedback control method according to an embodiment of the present invention.
  • FIG. 19 is a schematic layout diagram of a monolithic ultrasonic transmitter array according to an embodiment of the present disclosure.
  • FIG. 20 is a schematic layout diagram of a multi-block ultrasonic transmitter array according to an embodiment of the present disclosure.
  • Terminal device 2000 touch feedback system 1000; device body 300; imaging subsystem 100; haptic feedback subsystem 200;
  • an embodiment of the first aspect of the present disclosure proposes a touch feedback system, which can not only realize user non-contact tactile feedback, but also satisfy the user's interactive experience in both visual and tactile senses, which is more hygienic and safer.
  • a touch feedback system 1000 provided by an embodiment of the present disclosure includes an imaging subsystem 100 and a haptic feedback subsystem 200 .
  • the imaging subsystem 100 is used to image and display the human-computer interaction information in the aerial target area to form an aerial interactive interface, and send touch feedback control instructions and touch point coordinate information when detecting the interaction signal between the user and the aerial interactive interface .
  • the haptic feedback subsystem 200 is connected to the imaging subsystem 100, and the haptic feedback subsystem 200 is used for transmitting ultrasonic waves and focusing the ultrasonic waves on the touch points of the air interactive interface according to the touch feedback control instructions and the coordinate information of the touch points.
  • the imaging subsystem 100 adopts the interactive aerial imaging technology to form a floating real image, that is, an aerial interactive interface at a certain position in the air, as a reference plane for the user's tactile perception, and covers the three-dimensional space where the floating real image is located. Space is the air target area.
  • the imaging subsystem 100 gathers the human-computer interaction information into the aerial target area to form an aerial interactive interface, and when detecting the interaction signal between the user and the aerial interactive interface, the imaging subsystem 100 sends touch feedback control instructions and touch Point coordinate information to the haptic feedback subsystem 200 .
  • the touch area of the haptic feedback subsystem 200 is set to cover the three-dimensional space where the air interaction interface is located, that is, the air target area. Specifically, since the size and display position of the air interaction interface are relatively fixed, the haptic feedback subsystem 200 generates a touch feedback plane of the same size and position according to the known air interaction interface. That is, the imaging subsystem 100 provides an air interface to guide the user to touch, and the haptic feedback subsystem 200 provides a touch feedback plane to feedback the user's perception of the touch object.
  • the human-computer interaction information is imaged and displayed in the air target area through the imaging subsystem 100 to form an air interaction interface, and the air interaction interface is used as the reference surface of the user's tactile perception for guiding
  • the user touches and when the imaging subsystem 100 detects the interaction signal between the user and the air interaction interface, it sends a touch feedback control instruction and touch point coordinate information to the haptic feedback subsystem 200 .
  • the haptic feedback subsystem 200 transmits ultrasonic waves and focuses the ultrasonic waves on the touch points according to the received touch feedback control instructions and touch point coordinate information, and generates a haptic feedback effect between the human body and the touch points, so that the human body can truly feel
  • the existence of touch points realizes the purpose of non-contact tactile feedback for users.
  • the touch feedback operation can be triggered by the user touching the air interactive interface, which satisfies the user's interactive experience both visually and tactilely, and the operation mode is also more natural and comfortable. There is no need to set additional restrictions on user operation to avoid the risk of users touching the equipment during operation. At the same time, information security problems caused by the leakage of personal information due to residual fingerprint information of users are avoided.
  • the haptic feedback subsystem 200 includes a first housing 210 , an ultrasonic transmitter array 220 and a controller 250 .
  • the first housing 210 is provided with a window and has a first accommodating cavity 2 formed therein.
  • the first housing 210 can provide protection and support for the haptic feedback subsystem 200 to prevent damage to the haptic feedback subsystem 200 caused by external force shock and vibration.
  • the ultrasonic transmitter array 220 is arranged in the first accommodating cavity 2 to play a protective role.
  • ultrasonic waves can generate sound pressure in a certain space, and the sound pressure distribution is inversely related to the distance. Since the sound pressure generated by the ultrasonic waves emitted by a single ultrasonic transmitter is not enough to provide tactile feedback to the user, when the number of ultrasonic transmitters When a certain number is reached, the ultrasonic waves generated by each ultrasonic transmitter reach a certain point in space at the same time and are focused and superimposed at that point, and the superimposed sound pressure is far greater than the sound pressure generated by a single ultrasonic generator. Vibration is sensed, enabling ultrasonic haptic feedback.
  • the ultrasonic transmitter array 220 includes a plurality of ultrasonic transmitters 30 , and the plurality of ultrasonic transmitters 30 are combined to form an ultrasonic wave in an N*N array or an M*N array.
  • the transmitter array 220 and as shown in FIG. 2, the transmitting end of each ultrasonic transmitter 30 faces the window and is exposed to the window, so that ultrasonic waves can be radiated normally and the loss of radiation energy can be reduced.
  • the ultrasound transmitter array 220 may consist of several low-power, miniaturized ultrasound transmitters 30 for commercial use and integration.
  • the number of ultrasonic transmitters 30 used in the ultrasonic transmitter array 220 can be based on system requirements, such as the size of the touch area, touch distance, etc., and according to the technical indicators of the ultrasonic transmitter 30 itself, such as resonant frequency, sound pressure level, and direction angle. It depends on the actual situation, and here, there is no limit to the number of ultrasonic generators 30 used.
  • the controller 250 is arranged in the first accommodating cavity 2 and connected to the ultrasonic transmitter array 220, and is used to control each ultrasonic transmitter 30 to emit ultrasonic waves according to the touch feedback control instruction and the touch point coordinate information, and focus the ultrasonic waves to the touch point. control point.
  • the focal point position of the ultrasonic transmitter array 220 is set as the area where the air interactive interface is located.
  • the controller 250 controls the ultrasonic transmitter array 220 to emit ultrasonic waves according to the touch feedback control instruction, and controls the ultrasonic waves of each ultrasonic transmitter 30 to focus on the touch point according to the coordinate information of the touch point.
  • the ultrasonic sound pressure at the control point is superimposed to generate a tactile feedback force, so that the human body can truly feel the existence of the touch point and realize the perception of the touch object.
  • the controller 250 includes a data processing module 230 and a driving module 240 .
  • the data processing module 230 is configured to determine the delay phase of the ultrasonic wave transmitted by each ultrasonic transmitter 30 according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter 30 . Specifically, since the spatial distance from the transmitting end of each ultrasonic transmitter 30 on the ultrasonic transmitter array 220 to the touch point is different, in order to make the ultrasonic waves emitted by each ultrasonic transmitter 30 reach the touch point at the same time, so that the The control point generates tactile feedback. After the data processing module 230 completes the touch point coordinate data processing and phase calculation, it delays the phase of the control signal of the ultrasonic transmitter 30 according to the position of each ultrasonic transmitter 30 and the touch point coordinate data. Time control, so that the ultrasonic waves are transmitted to the touch points at the same time, so that multiple ultrasonic waves are superimposed on the touch points at the same time to generate vibration feedback.
  • the data processing module 230 calculates the delay phase of the ultrasonic wave transmitted by the ultrasonic transmitter 30 .
  • the array plane of the ultrasonic transmitter array 220 is taken as the coordinate plane
  • the geometric center of the array plane is taken as the coordinate origin O
  • a set of orthogonal axes on the array plane that are parallel to the array sides and pass through the origin O are taken as the x and y axes
  • z The axis is perpendicular to the array plane and passes through the origin O to establish a space rectangular coordinate system, wherein the array plane is the plane where the transmitting end of the ultrasonic transmitter array 220 is located.
  • the data processing module 230 obtains the coordinates of the geometric center of the transmitting end of the ultrasonic transmitter 30 according to the arrangement of the ultrasonic transmitters 30 on the ultrasonic transmitter array 220, and combines the touch point coordinate information sent by the imaging subsystem 100 to calculate and obtain each The spatial distance from the geometric center of the transmitting end of the ultrasonic transmitter 30 to each touch point, and then the spatial distance difference between the ultrasonic transmitters 30 in the ultrasonic transmitter array 220 is substituted into the sound velocity value c, and finally each ultrasonic transmitter 30 is obtained. delay phase. It should be noted that the coordinates of the touch point acquired by the imaging subsystem 100 and the coordinates of the geometric center of the transmitting end of the ultrasonic transmitter 30 are in the same coordinate system.
  • the driving module 240 is connected with the ultrasonic transmitter array 220 and the data processing module 230, and is used for generating the original driving signal in response to the touch feedback control instruction, and adjusting the original driving signal according to the delay phase of each ultrasonic transmitter 30 to convert the plurality of ultrasonic transmitters 30.
  • the ultrasonic waves emitted by the ultrasonic transmitter 30 are focused to the touch point.
  • the driving module 240 delays the original driving signal according to the delay phase of each ultrasonic transmitter 30
  • the processing is to control the multiple ultrasonic sound pressures to be superimposed on the touch points at the same time by adjusting the emission time of each ultrasonic transmitter 30 to transmit ultrasonic waves, so as to realize haptic feedback.
  • the driving module 240 when responding to the touch feedback control command, the driving module 240 generates a clock signal and a modulation signal according to the touch feedback control command, and generates an original driving signal according to the clock signal and the modulation signal.
  • the imaging subsystem 100 transmits the touch point coordinate information and the touch feedback control instruction in the air interactive interface to the data processing module 230 and the driver respectively.
  • module 240 The data processing module 230 calculates the delayed phase of each ultrasonic transmitter 30 according to the spatial coordinates of the touch point and the position of each ultrasonic transmitter 30 on the ultrasonic transmitter array 220 , and sends the delayed phase to the driving module 240 .
  • the driving module 240 receives the touch feedback control instruction and completes the signal modulation process and clock distribution, that is, generates the clock signal and the modulation signal to generate the original driving signals of several ultrasonic transmitters 30; Delay phase, the original drive signal of each ultrasonic transmitter 30 is delayed and controlled, so that each ultrasonic transmitter 30 emits ultrasonic waves in the corresponding phase, and reaches the touch point at the same time for superposition and convergence to generate Touch feedback allows the human body to truly feel the presence of touch points.
  • the driving module 240 performs delay processing on the original driving signal according to the delayed phase of each ultrasonic transmitter 30 to obtain the target driving signal of each ultrasonic transmitter 30, and drives each ultrasonic transmitter 30 according to the target driving signal
  • the ultrasonic transmitter 30 transmits ultrasonic waves. Specifically, after the driving module 240 receives the delayed phase transmitted by the data processing module 230, the driving module 240 performs delay processing on the original driving signal of each ultrasonic transmitter 30, and then amplifies the delayed signal.
  • each ultrasonic transmitter 30 processing to obtain the target driving signal required by each ultrasonic transmitter 30, and synchronously transmit the target driving signal of each ultrasonic transmitter 30 to the ultrasonic transmitter array 220, so as to drive each ultrasonic transmitter 30 in the corresponding target driving signal
  • the ultrasonic waves are emitted in sequence, so that the ultrasonic waves emitted by each ultrasonic transmitter 30 reach the touch point at the same time, so as to generate haptic feedback at the touch point.
  • the ultrasonic transmitter 30 transmits ultrasonic waves in the form of spherical waves, and the spherical center of the spherical wave is the geometric center of the transmitting end of the ultrasonic transmitter 30 .
  • the sound pressure distribution of ultrasonic waves in a limited space is negatively correlated with distance on the one hand, that is, the farther the distance, the weaker the sound pressure, so that the distance between the touch point and the array surface of the ultrasonic transmitter array 220 is limited; on the other hand It is related to the ultrasonic frequency f.
  • the free space wave number of ultrasonic waves in the air medium That is, the higher the frequency f, the larger the free space wave number k, but the ultrasonic radiation power E ⁇ k 2 , the higher the ultrasonic frequency, the shorter the wavelength, the weaker the ultrasonic penetration, and the greater the energy loss during propagation.
  • the value of the ultrasonic frequency should not be too large, and preferably, the value range of the ultrasonic frequency is less than 60KHz.
  • the ultrasonic waves emitted by the ultrasonic transmitter 30 are directional and cannot be radiated in the form of ideal spherical waves, which are limited by the direction angle of the ultrasonic transmitter 30.
  • the direction angle of the ultrasonic transmitter 30 ranges from 60° to 80°. .
  • the speed of the ultrasonic wave propagating in the air medium also changes due to the influence of the ambient temperature.
  • the actual transmission speed of the ultrasonic wave is where T is the ambient temperature. Therefore, in this embodiment of the present disclosure, a temperature sensor may be provided, and the data processing module 230 may be used for control to measure the ambient temperature in real time, and to correct the actual speed of sound according to the measured temperature value. Therefore, by correcting the speed of sound, the system error can be reduced, the delay accuracy of the control signal can be improved, and the ultrasonic focusing accuracy can be ensured, thereby avoiding abnormal system functions caused by abrupt changes in temperature.
  • the ultrasonic wave can be focused at any point in the air target area by controlling the emission time difference of different ultrasonic transmitters 30.
  • each touch point is changed according to a certain refresh frequency.
  • the difference in the transmission time of the ultrasonic transmitter 30 can meet the requirements of multi-touch interaction.
  • high-speed refresh of the haptic feedback subsystem 200 can be used to realize multi-touch feedback, which can be combined with the aerial interactive interface presented by the imaging subsystem 100 to guide the user to perform touch operations.
  • the embodiment of the present disclosure can also use multiple haptic feedback subsystems 200 to work synchronously, so as to realize user and complex operations by combining and splicing.
  • the human-computer interaction of the interface, especially for the contour perception of 3D display objects, improves the user experience.
  • the data processing module 230 includes a first storage unit 11 , a data processing unit 12 , a first interface unit 13 and a first power supply unit 14 .
  • the first interface unit 13 includes a first serial interface 15 and a first parallel interface 16 .
  • the first serial interface 15 is connected to the imaging subsystem 100 and the data processing unit 12 to realize information transmission between the imaging subsystem 100 and the data processing module 230.
  • the first serial interface 15 may be, for example, a USB interface, or Realize the transmission of touch point coordinate data and system control commands.
  • the first parallel interface 16 is connected with the driving module 240 and the data processing unit 12 to realize the information transmission between the driving module 240 and the data processing module 230. processing technology) integrated Upp parallel port to complete the transmission of delayed phase data and feedback data.
  • the first power supply unit 14 is connected to the first storage unit 11 , the data processing unit 12 and the first interface unit 13 .
  • the first power supply unit 14 is used for power conversion, and converts the input power into the required power for each module in the data processing module 230 .
  • Various stable and reliable power sources are used to supply power to it and ensure its normal operation.
  • the first storage unit 11 is used to store data and programs during data processing. As shown in FIG. 4 , the first storage unit 11 is divided into a data storage unit and a program storage unit.
  • the data storage unit is used to store data in the data processing process
  • the program storage unit is used to store the system boot program and the loading program.
  • the data storage unit adopts DDR2 SDRAM (Double-Data-Rate Two Synchronous Dynamic Random Access Memory, random access memory)
  • the program storage unit adopts NAND FLASH (flash memory).
  • the data processing unit 12 is connected to the first storage unit 11.
  • the data processing unit 12 can use a floating-point DSP chip as the main control, and the main frequency is above 200MHz to ensure system real-time requirements.
  • the data processing unit 12 receives the coordinate data of the touch point, determines the delay phase of each ultrasonic transmitter 30 according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter 30, and transmits the delay phase of each ultrasonic transmitter 30 through the first parallel interface 16.
  • the delayed phase information is transmitted to the driving module 240, so that the driving module 240 performs delay processing on the original driving signal according to the delayed phase of each ultrasonic transmitter 30, and controls each ultrasonic transmitter based on the delayed processed driving signal 30 , so that the ultrasonic waves emitted by each ultrasonic transmitter 30 reach the touch point at the same time and focus and overlap, so as to generate vibration at the touch point, so that the human body can perceive the touch feedback.
  • the driving module 240 includes a second storage unit 17 , a control unit 18 , a driving unit 19 , a second power supply unit 20 and a second interface unit 21 .
  • the second interface unit 21 includes a power input port 22 , a second parallel interface 23 and a drive signal output port 24 .
  • the power input port 22 is connected to the second power supply unit 20
  • the second parallel interface 23 is connected to the control unit 18 and the first parallel interface 16
  • the drive signal output port 24 is connected to the drive unit 19 .
  • the second parallel interface 23, such as the Upp parallel port performs data communication with the control unit 18 to realize the transmission of delayed phase data, control commands and feedback data.
  • the second storage unit 17 is used for storing data and files in the driving control process, for example, for storing the processing data of the control unit 18 and the data in the signal modulation process.
  • the second storage unit 17 can select SRAM (Static Random-Access Memory, static random access memory), and select a dedicated FLASH (solid-state memory) chip at the same time for storing and loading the logic target file of the control unit 18.
  • the control unit 18 is connected to the second storage unit 17, and is used for generating a clock signal and a modulation signal according to the touch feedback control instruction, and generating an original driving signal according to the clock signal and the modulation signal, and according to the delay phase of each ultrasonic transmitter 30 Delay processing of the original drive signal.
  • the control unit 18 can select an FPGA (Field Programmable Gate Array, Field Programmable Gate Array) as the main control chip, so as to directly and synchronously output the drive signal to the drive unit 19 through the configured I/O port to complete the transmission of each ultrasonic wave.
  • the controller 30 transmits ultrasonic waves, and at the same time, the FPGA main control chip has abundant pin resources and the pins can be redefined and configured, which can meet the system design requirements.
  • the driving unit 19 is connected to the control unit 18 for amplifying the delayed original driving signal to obtain a target driving signal for each ultrasonic transmitter 30 , and driving each ultrasonic transmitter 30 according to the target driving signal.
  • the second power supply unit 20 is connected to the second storage unit 17 , the control unit 18 and the driving unit 19 .
  • the second power supply unit 20 is used to provide a stable and reliable power supply for the entire system.
  • An external power supply is input through the power input port 22, and after DC-DC conversion and linear conversion, it is the driving module 240, the control unit 18, and the ultrasonic transmitter array 220. Provide the required power.
  • the control unit 18 since the size of the control unit 18 is smaller than that of the driving unit 19 , the control unit 18 can be fixed on the driving unit 19 through a high-speed inter-board connector.
  • the size of the driving unit 19 is equivalent to the size of the ultrasonic transmitter array 220, and the two can be interconnected through an inter-board connector.
  • the ultrasonic transmitter array 220, the control unit 18 and the drive unit 19 are assembled, and the assembled whole is packaged in the first housing 210, so as to improve the integration and aesthetics of the system.
  • first heat dissipation holes 3 are provided on the first side wall of the first accommodating cavity 2
  • second heat dissipation holes 3 are provided on the second side wall of the second accommodating cavity 2 4.
  • the first side wall is opposite to the second side wall.
  • the haptic feedback subsystem 200 also includes a suction fan and an exhaust fan.
  • the suction fan is arranged on the first side wall, and is used for sucking external air to the first accommodating cavity 2 through the first heat dissipation hole 3 .
  • the exhaust fan is arranged on the second side wall, and is used to exhaust the air in the first accommodating cavity 2 through the second heat dissipation holes 4 . Based on opening the heat dissipation holes on both sides of the first casing 210, and combining the suction fan and the exhaust fan, the heat dissipation effect on the system is further improved.
  • the haptic feedback subsystem 200 further includes a blocking cover 5 , which is disposed on the first housing 210 and is used to block the opening window and prevent the transmitting end of the ultrasonic transmitter 30 from being directly exposed to the air. and play a protective role, while improving the aesthetics of the product.
  • a blocking cover 5 which is disposed on the first housing 210 and is used to block the opening window and prevent the transmitting end of the ultrasonic transmitter 30 from being directly exposed to the air. and play a protective role, while improving the aesthetics of the product.
  • the setting material of the blocking cover 5 is not limited, for example, the blocking cover 5 can be a mesh protective cover or a wave-transmitting material baffle.
  • the data processing module 230 , the driving module 240 and the ultrasonic transmitter array 220 are all assembled through micro-miniature board connectors, and the haptic feedback subsystem 200 adopts a modular and miniaturized design as a whole, so that the After the system is assembled, the overall height and volume can be reduced, the occupied space can be reduced, and the aesthetics and integration of the system can be improved.
  • the imaging subsystem 100 includes a second housing 110 , an imaging assembly 120 , a detection module 130 and a main control module 140 .
  • the second casing 110 is formed with a display window 40 and a second accommodating cavity 50 is formed inside.
  • the imaging component 120 is disposed in the second accommodating cavity 50, and is used for imaging and displaying the human-machine interaction information in the aerial target area 10 to form an aerial interaction interface.
  • the imaging assembly 120 forms a floating real image at a certain position in the air, that is, an aerial interactive interface, and the three-dimensional space covering the floating real image is the aerial target area 10. That is, the imaging assembly 120 can operate in the air without using a physical medium.
  • the target area 10 presents a floating real image, which satisfies the user's visual interactive experience, and the imaging component 120 also does not need to set an additional restriction mechanism to guide the user to operate, thereby reducing the risk of the user contacting the device body.
  • the detection module 130 is used to detect the interaction signal between the user and the air interface. Specifically, as shown in FIG. 7 , the sensing area of the detection module 130 is located on the same plane as the air interaction interface and includes the three-dimensional space where the air interaction interface is located.
  • the detection module 130 detects the user's interactive operation on the aerial interactive interface in the aerial target area 10 in real time, so that when the interactive signal between the user and the aerial interactive interface is detected, the detected interactive signal is fed back to the main control module 140.
  • the main control module 140 In response to the interaction signal, the touch feedback control instruction and the touch point coordinate information are sent to the haptic feedback subsystem 200 to trigger the haptic feedback subsystem 200 to feedback the user's perception of the touch object. Therefore, the user triggers the operation mode of the haptic feedback subsystem 200 to perform haptic feedback by directly interacting with the air interaction interface, which is more natural and comfortable.
  • the detection module 130 may be an optical sensor, and its sensing form may include, but is not limited to, far and near infrared, ultrasonic wave, laser interference, grating, encoder, optical fiber type or CCD (Charge-coupled Device, charge-coupled device) and the like.
  • the detection module 130 can select the best sensing form according to the installation space, viewing angle and use environment, so as to facilitate the user to operate in the air target area 10 with the best posture and improve the user experience.
  • the main control module 140 is connected to the imaging component 120 and the detection module 130, and is used for sending touch feedback control instructions and touch point coordinate information in response to the interactive signal.
  • the main control module 140 and the detection module 130 can be connected in a wired or wireless manner to transmit digital or analog signals, so that the volume of the overall device can be flexibly controlled, and the electrical stability of the touch feedback system 1000 can be enhanced.
  • imaging assembly 120 includes display 25 and optical assembly 26 .
  • the display 25 is disposed in the second accommodating cavity 50 and is connected to the main control module 140 for displaying human-computer interaction information.
  • the optical component 26 is disposed in the second accommodating cavity 50, and is used for converging and imaging (eg, refracting) the light carrying the human-computer interaction information on the air target area 10, so as to form an air interaction interface.
  • the display 25 is disposed on the light source side of the optical assembly 26
  • the display window 40 is disposed on the imaging side of the optical assembly 26 .
  • the main control module 140 controls the display 25 to display human interaction information, and the light of the human interaction information displayed on the display 25 is imaged and displayed in the aerial target area 10 through the optical component 26 to form an aerial interaction interface for guiding the user to touch.
  • the imaging mode of the display 25 may include RGB (red, green, blue) light emitting diodes (Light Emitting Diode, LED), LCD (Liquid Crystal Display, liquid crystal display), LCOS (Liquid Crystal on Silicon, liquid crystal attached Silicon) devices, OLED (Organic Light-Emitting Diode, organic light-emitting diode) arrays, projections, lasers, laser diodes or any other suitable displays or stereoscopic displays, which are not limited.
  • the display 25 can provide a clear, bright and high-contrast dynamic image light source.
  • the main control module 140 controls the display 25 to display the human interaction information, and is refracted by the optical component 26 to present a clear aerial interactive interface at the position of the aerial target area 10, thereby User-friendly operation.
  • the brightness of the display 25 may be set to be not lower than 500 cd/m 2 to reduce the influence of brightness loss in the propagation of the light path.
  • the display brightness of the display 25 can also be adjusted according to the brightness of the ambient light.
  • the viewing angle control processing can be performed on the display image surface of the display 25, so as to reduce the afterimage of the aerial target area 10, improve the picture quality, and at the same time, it can also prevent others from peeping, which is convenient for being widely used in other applications that require privacy information protection. input device.
  • optical assembly 26 in the embodiment of the present disclosure and the principle of realizing imaging will be described below.
  • the optical assembly 26 may adopt a flat lens, and the flat lens is fixed on the second housing 110 .
  • the flat lens includes two transparent substrates 8 and is placed between the two transparent substrates 8
  • the first optical waveguide array 6 and the second optical waveguide array 7 are closely attached on the same plane and are arranged orthogonally.
  • the thicknesses of the first optical waveguide array 6 and the second optical waveguide array 7 are the same, which is convenient for design and production.
  • the flat lens sequentially includes a first transparent substrate 8 , a first optical waveguide array 6 , a second optical waveguide array 7 and a second glass substrate 8 from the display 25 side to the air target area 10 side. .
  • the first transparent substrate 8 and the second transparent substrate 8 both have two optical surfaces, and the transparent substrate 8 has a transmittance of 90%-100% for light with wavelengths between 390 nm and 760 nm.
  • the material of the transparent substrate 8 can be at least one of glass, plastic, polymer and acrylic resin, for protecting the optical waveguide array and filtering out excess light. It should be noted that, if the strength of the first optical waveguide array 6 and the second optical waveguide array 7 after close and orthogonal bonding is sufficient, or the installation environment has a thickness limit, only one transparent substrate 8 may be configured or no transparent substrate is configured at all. Substrate 8.
  • the principle of the optical assembly 26 to achieve aerial imaging is that the first optical waveguide array 6 and the second optical waveguide array 7 are composed of a plurality of reflection units 9 with rectangular cross-sections, and the length of each reflection unit 9 is limited by the peripheral size of the optical waveguide array. Different lengths. As shown in FIG. 9 , the extension direction of the reflection unit 9 in the first optical waveguide array 6 is X, the extension direction of the reflection unit 9 of the second optical waveguide array 7 is Y, and the Z direction is the thickness direction of the optical waveguide array.
  • the extension directions (optical waveguide array directions) of the reflection units 9 in the first optical waveguide array 6 and the second optical waveguide array 7 are perpendicular to each other, that is, viewed from the Z direction (thickness direction), the first optical waveguide array 6 and the second optical waveguide
  • the arrays 7 are arranged orthogonally, so that the two light beams in the orthogonal direction converge at one point, and the object image plane (the light source side and the imaging side) is guaranteed to be symmetrical with respect to the flat lens, resulting in an equivalent negative refraction phenomenon, realizing aerial imaging .
  • the first optical waveguide array 6 or the second optical waveguide array 7 is composed of a plurality of parallel-arranged reflective units 9 that are obliquely arranged with a user viewing angle deflection of 45°.
  • the first optical waveguide array 6 can be composed of reflective units 9 that are aligned at 45° in the lower left direction and have a rectangular cross section
  • the second optical waveguide array 7 can be composed of reflective units 9 that are aligned at 45° in the lower right direction and have a rectangular cross section.
  • composition, the arrangement directions of the reflection units 9 in the two groups of optical waveguide arrays can be interchanged.
  • the extension direction of the reflection unit 9 in the first optical waveguide array 6 is Y
  • the extension direction of the reflection unit 9 in the second optical waveguide array 7 is X
  • the Z direction is the thickness direction of the optical waveguide array.
  • the first optical waveguide array 6 and the second optical waveguide array 7 are arranged orthogonally, so that the two light beams in the orthogonal direction converge at one point, and ensure that the object image plane (light source side and imaging side) is relative to the flat plate
  • the lens is symmetrical, which produces an equivalent negative refraction phenomenon and realizes aerial imaging.
  • the optical waveguide material has an optical refractive index n1, and in some embodiments, n1>1.4, for example, the value of n1 is 1.5, 1.8, 2.0, or the like.
  • the cross section of the reflection unit 9 may be rectangular, and a reflection film 27 is provided on one or both sides along the arrangement direction of the reflection unit 9 .
  • a reflection film 27 is provided on one or both sides along the arrangement direction of the reflection unit 9 in the arrangement direction of the optical waveguide array.
  • both sides of each reflection unit 9 are coated with a reflection film 27 , and the material of the reflection film 27 can be a metal material such as aluminum, silver, etc. that realizes total reflection or Other non-metallic compound materials.
  • the function of the reflective film 27 is to prevent light from entering into adjacent optical waveguide arrays due to lack of total reflection to form stray light from affecting imaging.
  • each reflective unit 9 may also add a dielectric film on the reflective film 27, and the function of the dielectric film is to improve the light reflectivity.
  • Large-scale requirements can be achieved by splicing multiple optical waveguide arrays when displaying on a large screen.
  • the overall shape of the optical waveguide array is set according to the application scenario.
  • the two groups of optical waveguide arrays have a rectangular structure as a whole, the two diagonal reflection units 9 are triangular, and the middle reflection unit 9 is a trapezoidal structure.
  • the lengths of the individual reflection units 9 are not equal, the reflection units 9 located on the diagonal of the rectangle have the longest length, and the reflection units 9 at both ends have the shortest length.
  • the flat lens may further include an anti-reflection component and a viewing angle control component, and the anti-reflection component can improve the overall transmittance of the flat lens and improve the clarity and brightness of the aerial interactive interface imaged in the aerial target area 10 .
  • the viewing angle control component can be used to eliminate the afterimage of the aerial interactive interface imaged in the aerial target area 10, reduce the dizziness of the observer, prevent the observer from peeping into the device from other angles, and improve the overall aesthetics of the device.
  • the anti-reflection component and the viewing angle control component may be combined, or may be independently disposed between the transparent substrate 8 and the waveguide array, between two layers of the waveguide array, or on the outer layer of the transparent substrate 8 .
  • the imaging principle of the flat lens is described below with reference to FIG. 8 , and the specific content is as follows.
  • the orthogonal decomposition of any optical signal is performed using mutually orthogonal double-layer waveguide array structures.
  • the original signal is projected on the first optical waveguide array 6, and a rectangular coordinate system is established with the original signal projection point as the origin and perpendicular to the first optical waveguide array 6 as the x-axis, in which the original signal is decomposed into x-axis.
  • the signal X and the signal Y located on the y-axis are mutually orthogonal signals.
  • the signal X passes through the first optical waveguide array 6, it is totally reflected on the surface of the reflective film 27 at the same reflection angle as the incident angle; at this time, the signal Y remains parallel to the first optical waveguide array 6 and passes through the first optical waveguide array 6.
  • the surface of the second optical waveguide array 7 performs total reflection on the surface of the reflective film 27 at the same reflection angle as the incident angle, and the reflected optical signal composed of the reflected signal Y and the signal X is the same as the original light.
  • the signal is mirror-symmetrical.
  • the light in any direction can achieve mirror symmetry through the flat lens, and the divergent light of any light source will be re-converged into a floating real image at a symmetrical position through the flat lens, that is, the air interactive interface is imaged at the air target area 10, floating in the air.
  • the imaging distance of the real image is the same as the distance from the flat lens to the image source, that is, the display 25, which is equidistant imaging, and the position of the floating real image is in the air, no specific carrier is needed, but the real image is directly presented in the air. Therefore, the image in the space seen by the user is the image emitted by the display 25 .
  • the incident angles after convergent imaging are ⁇ 1 , ⁇ 2 , ⁇ 3 « is L, and the viewing angle ⁇ of the floating real image is 2 times max( ⁇ ).
  • the size of the optical waveguide array is small, the image can only be seen at a certain distance from the imaging side of the optical waveguide array; and if the size of the optical waveguide array becomes larger, a larger imaging distance can be achieved. , thereby increasing the field of view.
  • the angle between the flat lens and the display 25 is set to be in the range of 45° ⁇ 5°, so that the size of the flat lens can be effectively utilized, the image quality can be improved and the effect of afterimages can be reduced.
  • other angles can also be selected while sacrificing part of the imaging quality.
  • the size and position of the flat lens can also be freely adjusted according to the actual display screen, which is not limited.
  • the above mainly describes the imaging principle of a flat lens with a double-layer optical waveguide array structure.
  • the cubic-columnar reflection units 9 are all arranged in an array along the X and Y directions in the one-layer optical waveguide array structure, that is, the two-layer optical waveguide arrays are combined into one layer.
  • the imaging principle is the same as that of the double-layer optical waveguide array structure. It can be used as the structure of flat lens.
  • the thicknesses of the first optical waveguide array 6 and the second optical waveguide array 7 are the same, so that the structural complexity of the first optical waveguide array 6 and the second optical waveguide array 7 can be simplified, and the first optical waveguide array 6 can be reduced.
  • the manufacturing difficulty of the second optical waveguide array 7 improves the production efficiency of the first optical waveguide array 6 and the second optical waveguide array 7 and reduces the production cost of the first optical waveguide array 6 and the second optical waveguide array 7 .
  • the same thickness here is a relative range, not absolutely the same, that is, for the purpose of improving production efficiency, on the premise of not affecting the quality of aerial imaging, there may be a certain thickness difference between the optical waveguide arrays. .
  • a light absorbing layer is provided on the inner wall of the second accommodating cavity 50 . That is, the parts other than the display surface of the display 25 in the second housing 110 are treated with black light absorption, such as spraying light-absorbing paint or pasting a light-absorbing film, so as to eliminate the diffuse reflection of the light by the internal components of the second housing 110, Improve the display effect of the air interface.
  • black light absorption such as spraying light-absorbing paint or pasting a light-absorbing film
  • the ultrasonic transmitter array 220 can adopt the ultrasonic transmitter 30 with strong penetrability. Therefore, the arrangement of the haptic feedback subsystem 200, especially the ultrasonic transmitter array 220, can be determined according to the actual application and system. flexibly arranged according to the requirements, and there is no restriction on this. Several preferred arrangements of the embodiments of the present disclosure are specifically described below.
  • the haptic feedback subsystem 200 is disposed above the imaging subsystem 100 and on the touch side of the air interface.
  • the transmitting end of the ultrasonic transmitter array 220 is arranged in parallel and opposite to the air interface.
  • the transmitting end of the ultrasonic transmitter array 220 faces the air interface.
  • the haptic feedback subsystem 200 is disposed above the imaging subsystem 100 and on the imaging side of the air interface.
  • the ultrasonic transmitter array 220 is perpendicular to the optical component 26, and the ultrasonic transmitters The array 220 is flush with the air interface in the horizontal direction, and the transmitting end of the ultrasonic transmitter array 220 faces the air interface.
  • the haptic feedback subsystem 200 is disposed in the second accommodating cavity 50 and is located on the reflection path of the optical component 26 to the light carrying the human-computer interaction information.
  • the ultrasonic transmitter array 220 The transmitting end faces the air interface.
  • the haptic feedback subsystem 200 is disposed above the imaging subsystem 100 and on the touch side of the air interface.
  • the ultrasonic transmitter array 220 is disposed in parallel with the optical assembly 26 .
  • the array 220 is flush with the air interface in the vertical direction, and the transmitting end of the ultrasonic transmitter array 220 faces the air interface.
  • the imaging subsystem 100 adopts the interactive aerial imaging technology to form an aerial interactive interface at a determined position in the air, that is, the aerial target area 10 does not need any physical medium.
  • a floating real image is presented, and the ultrasonic radiation pressure touch feedback technology is used by the haptic feedback subsystem 200 to adjust the ultrasonic focus area to cover the position of the air interactive interface, and this area constitutes a touch area for guiding the user.
  • the imaging subsystem 100 triggers the haptic feedback subsystem 200 to generate a haptic feedback force by superimposing ultrasonic sound pressure at the touch point, that is, without the aid of a physical device, the haptic touch of the floating touch object can be realized perception. Therefore, the embodiment of the present disclosure deeply integrates the ultrasonic radiation pressure tactile feedback technology with the interactive aerial imaging technology, and through the integration and development of the touch feedback system 1000, not only realizes the user's non-contact tactile feedback, but also provides the user with visual and tactile feedback. Create a new display and interactive experience.
  • the target area 10 in the air provides the user with an actual touch support, and the user can accurately perceive the touch situation through the fingertips , so that on the basis of visual feedback, the efficiency of touch interaction can be improved and users can avoid meaningless repeated touches in the air.
  • the ultrasonic transmitter 30 adopts a low-power miniaturized product, which can also avoid damage to the human body.
  • Embodiments of the second aspect of the present disclosure provide a terminal device.
  • the terminal device 2000 includes a device body 300 and the touch feedback system 1000 provided in the above embodiments.
  • the touch feedback system 1000 is provided on the device body 300 .
  • the terminal device may include an elevator, a ticket machine, a cash machine, a service inquiry machine, etc., as well as other applicable devices with touch interaction functions.
  • the purpose of non-contact haptic feedback for the user can be achieved, and there is no need to set an additional restricting user operation device, and the operation method is also more natural and convenient , and satisfy the user's interactive experience both visually and tactilely.
  • the embodiment of the third aspect of the present disclosure provides a touch feedback control method. As shown in FIG. 17 , the method of the embodiment of the present disclosure includes at least steps S1 to S3 .
  • Step S1 image and display the human-computer interaction information in the air target area to form an air interaction interface.
  • the embodiment of the present disclosure adopts the method of combining the interactive aerial imaging technology and the ultrasonic radiation pressure tactile feedback technology to efficiently complete the aerial tactile feedback interaction.
  • the human-computer interaction information can be displayed at a determined position in the air to form an aerial interactive interface, which visually satisfies the user's interactive experience.
  • the position of the air interface is relatively fixed in the air, so the three-dimensional space where the air interface is located is taken as the air target area. Therefore, the user can directly interact with the air interface without setting additional restriction mechanisms to guide the user to operate, avoiding the risk of the user coming into contact with the device body, reducing the public health and safety issues caused by the cross-use of the user, and also Avoid information security problems caused by the leakage of personal information due to residual fingerprint information of users.
  • step S2 an interaction signal between the user and the air interaction interface is detected, the coordinate information of the touch point is acquired, and a touch feedback control instruction is sent.
  • the air interaction interface is used as a reference plane for tactile perception to guide the user to touch.
  • the coordinate information of the touch point can be determined according to the display objects of the air interactive interface and the distribution of the touch points, and then the coordinate information of the touch point and the touch feedback control command can be sent. , to provide the user with haptic feedback at the touch point.
  • Step S3 controlling the ultrasonic transmitter array to emit ultrasonic waves according to the touch feedback control instruction and the coordinate information of the touch points and focus the ultrasonic waves on the touch points of the air interactive interface.
  • the embodiment of the present disclosure controls the ultrasonic transmitter array to emit ultrasonic waves by using the ultrasonic radiation pressure tactile feedback technology in response to the touch feedback control instruction, and simultaneously controls the ultrasonic wave emitted by the ultrasonic transmitter array in combination with the coordinate information of the touch point. Focusing on the touch points of the air interactive interface at the same time, so that the ultrasonic sound pressure superimposed at the touch points can be sensed by the human body, realizing ultrasonic tactile feedback.
  • an aerial interactive interface is formed by imaging and displaying human-computer interaction information in an aerial target area, that is, an interactive aerial imaging technology is adopted, and the aerial interactive interface is used as a reference surface for tactile perception, so as to It is used to guide the user to touch, and when the interaction signal between the user and the air interactive interface is detected, obtain the touch feedback control instruction and touch point coordinate information, so as to obtain the touch feedback control instruction and touch point coordinate information according to the acquired touch feedback control instruction and touch point coordinate information, Control the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch points, that is, through the ultrasonic radiation pressure, to generate a tactile feedback effect between the human body and the touch points, so that the human body can truly feel the existence of the touch points and realize the user experience.
  • the touch feedback operation can be triggered by the user touching the air interactive interface, which not only satisfies the user's interaction both visually and tactilely Experience, and the operation method is also more natural and comfortable, and there is no need to set additional restrictions on user operation, avoiding the risk of users touching the equipment during operation, and avoiding the problem of personal information leakage caused by user information left on the contact surface, which is more hygienic Safety.
  • a fourth aspect of the present disclosure provides a touch feedback control method, where the touch feedback control method is used in a haptic feedback subsystem, and the haptic feedback subsystem includes an ultrasonic transmitter array.
  • the haptic feedback subsystem is the haptic feedback subsystem 200 in the touch feedback system 1000 shown in FIG. 1 .
  • the touch feedback control method includes at least step S10 and step S20:
  • the touch feedback control method for the haptic feedback subsystem can also avoid the risk of the user touching the device during operation, and also avoid the occurrence of user information left on the contact surface.
  • the problem of personal information leakage is caused by the above, and it is more hygienic and safe.
  • touch feedback system can be adaptively applied to the touch feedback control methods according to the embodiments of the third and fourth aspects of the present invention, which are not repeated here for brevity.
  • the following descriptions are applicable to the touch feedback control methods of the embodiments of the third aspect and the fourth aspect.
  • the ultrasonic transmitter array for controlling the ultrasonic transmitter array to emit ultrasonic waves and transmit the ultrasonic waves according to the touch feedback control instruction and the touch point coordinate information Focusing on the touch point of the air interactive interface, including determining the delay phase of each ultrasonic transmitter according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter in the ultrasonic transmitter array, and determining the delay phase of each ultrasonic transmitter according to the touch feedback
  • the control commands generate raw drive signals.
  • the original driving signal is adjusted according to the delay phase of each ultrasonic transmitter to control the transmission time of each ultrasonic transmitter to transmit ultrasonic waves, and the ultrasonic waves emitted by multiple ultrasonic transmitters are focused to the touch point at the same time, so as to realize Haptic feedback.
  • the ultrasonic transmitter array includes a plurality of ultrasonic transmitters. Therefore, generating the original driving signal according to the touch feedback control instruction includes generating a clock signal and a modulation signal according to the touch feedback control instruction, and generating a clock signal and a modulation signal according to the touch feedback control instruction. The original driving signal is generated with the modulation signal, so as to realize the control of each ultrasonic transmitter in the ultrasonic transmitter array according to the corresponding original driving signal.
  • adjusting the original driving signal according to the delayed phase of each ultrasonic transmitter includes performing a delay processing on the original driving signal according to the delayed phase of each ultrasonic transmitter to obtain each ultrasonic transmitter The target driving signal of the transmitter; each ultrasonic transmitter is driven according to the target driving signal. Therefore, by controlling the phase of a certain number of ultrasonic transmitters, all ultrasonic transmitters can reach the touch point at the same time for superposition, so that the superimposed ultrasonic sound pressure can be truly perceived by the human body, and ultrasonic tactile feedback can be realized.
  • the present disclosure needs to collect the ambient temperature in real time, which specifically includes: The delay phase of each ultrasonic transmitter is corrected to reduce the delay control error and improve the focusing accuracy of the ultrasonic transmitter array.
  • a single-block array sub-regional control or multi-block array synchronous control scheme is respectively adopted, as follows.
  • the present disclosure sets the ultrasonic transmitter array to include n ultrasonic transmitter sub-arrays distributed on the same plane, and the air interaction interface is divided into There are n touch sub-regions, and the n ultrasonic transmitter sub-arrays are arranged in a one-to-one correspondence with the n touch sub-regions.
  • the ultrasonic transmitter array is divided into several sub-arrays of ultrasonic transmitters by using a single-block array sub-area control method, and each sub-array of ultrasonic transmitters is mapped to the corresponding touch sub-area, which is defined by All touch sub-areas form a complex two-dimensional interactive interface.
  • controlling the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves on the touch points according to the touch feedback control instructions and the coordinate information of the touch points including: according to the coordinates of the touch points The information determines that the touch point corresponds to the target touch sub-area in the air interactive interface; the target ultrasonic transmitter sub-array is determined according to the target touch sub-area, and each ultrasonic transmitter in the target ultrasonic transmitter sub-array is determined according to the coordinate information of the touch point and the target ultrasonic transmitter sub-array
  • the position information of the transmitting end of the device determines the delay phase of each ultrasonic transmitter in the ultrasonic transmitter sub-array; generates a clock signal and a modulation signal according to the touch feedback control command, and generates an original drive signal according to the clock signal and modulation signal; according to the target ultrasonic wave
  • the delay phase of each ultrasonic transmitter in the transmitter sub-array performs
  • Each ultrasonic transmitter in the array is divided into multiple areas, by controlling the phase of each ultrasonic transmitter in the target touch sub-area, the number of focusing points and the phase delay time difference can be reduced, the focusing accuracy can be guaranteed, and the system requirements can be met. real-time requirements.
  • the embodiment of the present disclosure divides the air interactive interface into four touch sub-regions, and maps them to the corresponding ultrasonic transmitter sub-arrays, so as to reduce the number of focus points and the phase delay time difference, And use local focusing to achieve focused scanning.
  • the data processing module obtains the coordinates of all touch points in the touch area
  • the touch points are allocated to the corresponding ultrasonic transmitter sub-arrays according to the principle of proximity, and four touch sub-areas are grouped with the ultrasonic transmitter sub-arrays.
  • the focus control process of each group is the same as that of the single-array focus control process, that is, according to the coordinate information of the touch point, the touch sub-area in the air interactive interface corresponding to the touch point is determined, and this area is used as the target touch sub-area, the target touch sub-area.
  • the ultrasonic transmitter sub-array corresponding to the area is the target ultrasonic transmitter sub-array.
  • the data processing module calculates the delayed phase data of all ultrasonic transmitters corresponding to the target touch sub-area, and sends the delayed phase data to the driving module.
  • the drive module completes the signal modulation, clock distribution, delay control and drive amplification of the target touch sub-area according to the received delay phase data and touch feedback control instructions, and outputs the target drive signal synchronously to the corresponding ultrasonic transmitter
  • the sub-array is used to control each ultrasonic transmitter on the sub-array of ultrasonic transmitters to emit ultrasonic waves at a specified time, thereby completing the focusing control of the corresponding touch points and ensuring the focusing accuracy.
  • the present disclosure sets n ultrasonic transmitter arrays, and the n ultrasonic transmitter arrays are distributed on the On the same plane or staggered at a preset angle, the air interaction interface is divided into n touch sub-areas, and the n ultrasonic transmitter arrays are arranged in a one-to-one correspondence with the n touch sub-areas. That is, multiple ultrasonic transmitter arrays are used, the touch area is divided into several touch sub-areas, and the corresponding ultrasonic transmitter arrays are arranged according to the positions of the touch sub-areas, that is, the multi-block array is controlled synchronously.
  • the arrangement of the multiple ultrasonic transmitter arrays may be distributed on the same plane according to actual requirements, or may be staggered according to a certain angle, which is not limited.
  • controlling the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch points according to the touch feedback control instructions and the coordinate information of the touch points including: according to the coordinate information of the touch points Determine the touch point corresponding to the target touch sub-area in the air interactive interface; determine the target ultrasonic transmitter array according to the target touch sub-area; according to the coordinate information of the touch point and the transmitting end of each ultrasonic transmitter in the target ultrasonic transmitter array
  • the position information determines the delay phase of each ultrasonic transmitter in the target ultrasonic transmitter array; generates a clock signal and a modulation signal according to the touch feedback control command, and generates an original drive signal according to the clock signal and modulation signal; according to the target ultrasonic transmitter array
  • the delay phase of each ultrasonic transmitter in the original driving signal is delayed to obtain the target driving signal of each ultrasonic transmitter in the target ultrasonic transmitter array; each ultrasonic wave in the target ultrasonic transmitter array is driven according
  • eight ultrasonic transmitter arrays can be set to correspond to eight haptic feedback subsystems, and the touch points are allocated to the corresponding haptic feedback subsystems according to the principle of proximity to form eight touch sub-areas and haptic feedback.
  • Subsystems are grouped, and the focus control process of each group is the same as the single-array focus control process.
  • the data processing modules corresponding to the eight tactile feedback subsystems obtain the coordinates of the touch points corresponding to the touch sub-areas respectively, and determine the touch sub-areas corresponding to the touch points according to the coordinates of the touch points as the target touch sub-areas.
  • the ultrasonic transmitter array corresponding to the target touch sub-area is the target ultrasonic transmitter array
  • the data processing module corresponding to the target ultrasonic transmitter array calculates the delay phase data of all ultrasonic transmitters in the target ultrasonic transmitter array, and converts the The data is sent to the corresponding driver module.
  • the driving module completes signal modulation, clock distribution, delay control and drive amplification for the target ultrasonic transmitter array according to the received delay phase data of the target ultrasonic transmitter array and touch feedback control instructions, and transmits the target driving signal Give the target ultrasonic transmitter array to control each ultrasonic transmitter on the target ultrasonic transmitter array to emit ultrasonic waves at a specified time to complete the focus control of the corresponding touch point. Therefore, by using a plurality of ultrasonic transmitter arrays to control the focus of the touch object at different angles, the focus accuracy and the real-time performance of the system can be guaranteed.
  • the present disclosure considers the synchronization problem of the control of multiple ultrasonic transmitter arrays, and sets one synchronization control signal in the entire system, so that each touch sub-area and the haptic feedback subsystem can be grouped under the control of the synchronization control signal. It works synchronously to ensure synchronous scanning of touch points in each touch sub-area, provide users with a good touch experience, and avoid rendering distortions such as discontinuous border touches.
  • Embodiments of a fifth aspect of the present disclosure provide a storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the touch feedback control method provided by the foregoing embodiments is implemented.
  • any description of a process or method in a flowchart or otherwise described herein may be understood to represent a representation of executable instructions comprising one or more steps for implementing a custom logical function or process modules, segments or portions of code, and the scope of preferred embodiments of the present disclosure include alternative implementations, which may not be in the order shown or discussed, including in a substantially simultaneous manner or in the reverse order depending on the functionality involved , to perform functions, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.
  • a "computer-readable medium” can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus.
  • computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM).
  • the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.
  • portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
  • various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
  • the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Disclosed are a touch-control feedback system (1000), a terminal device (2000), a touch-control feedback control method, and a storage medium. The touch-control feedback system (1000) comprises: an imaging subsystem (100), which is used for performing imaging display on human-computer interaction information in an air target area, so as to form an air interaction interface, and sending a touch-control feedback control instruction and touch-control point coordinate information when a signal of interaction between a user and the air interaction interface is detected; and a haptic feedback subsystem (200), which is connected to the imaging subsystem (100), and is used for emitting an ultrasonic wave according to the touch-control feedback control instruction and the touch-control point coordinate information, and focusing the ultrasonic wave on a touch-control point of the air interaction interface.

Description

触控反馈***、终端设备、触控反馈控制方法和存储介质Touch feedback system, terminal device, touch feedback control method and storage medium 技术领域technical field
本公开涉及触控反馈技术领域，具体而言，涉及一种触控反馈控制***、一种终端设备以及一种触控反馈控制方法和一种存储介质。The present disclosure relates to the technical field of touch feedback, and in particular, to a touch feedback control system, a terminal device, a touch feedback control method, and a storage medium.
背景技术Background technique
相关技术中，对于接触式触觉反馈技术，通过借助相关设备(如屏幕、手套等)辅助，利用振动、静电力等原理和技术实现触觉感知。但是，该方式必须与辅助设备进行接触，一方面设备的交叉使用会引起公共卫生安全问题，用户也无法摆脱笨重设备的束缚；另一方面，用户使用设备后，在设备上遗留的指纹、掌纹等个人信息，会造成个人信息安全面临泄露的风险。In the related art, for the contact-type haptic feedback technology, haptic perception is realized by using vibration, electrostatic force and other principles and technologies with the assistance of related devices (such as screens, gloves, etc.). However, this method must be in contact with auxiliary equipment. On the one hand, the cross-use of equipment will cause public health and safety problems, and users cannot get rid of the shackles of bulky equipment; on the other hand, after users use the equipment, fingerprints and palms left on the equipment Personal information such as tattoos will cause the risk of personal information security to be leaked.
发明内容SUMMARY OF THE INVENTION
本公开旨在至少解决现有技术中存在的技术问题之一。为此，本公开的一个目的在于提出一种触控反馈***，该***既可以实现用户非接触式触觉反馈，又可以在视觉和触觉上同时满足用户的交互体验，更加卫生安全。The present disclosure aims to solve at least one of the technical problems existing in the prior art. To this end, an object of the present disclosure is to provide a touch feedback system, which can not only realize the user's non-contact tactile feedback, but also satisfy the user's interactive experience both visually and tactilely, which is more hygienic and safer.
本公开的目的之二在于提出一种终端设备。The second purpose of the present disclosure is to provide a terminal device.
本公开的目的之三在于提出一种触控反馈控制方法。The third purpose of the present disclosure is to provide a touch feedback control method.
本公开的目的之四在于提出一种存储介质。The fourth purpose of the present disclosure is to provide a storage medium.
为了解决上述问题，本公开第一方面实施例提供的触控反馈***，包括，成像子***，用于将人机交互信息在空中目标区域成像显示以形成空中交互界面，并在检测到用户与所述空中交互界面的交互信号时发送触控反馈控制指令和触控点坐标信息；触觉反馈子***，与所述成像子***连接，用于根据所述触控反馈控制指令和所述触控点坐标信息发射超声波并将所述超声波聚焦至所述空中交互界面的触控点。In order to solve the above problem, the touch feedback system provided by the embodiment of the first aspect of the present disclosure includes an imaging subsystem, which is used for imaging and displaying the human-computer interaction information in the air target area to form an air interaction interface, and when it is detected that the user interacts with The touch feedback control instruction and the touch point coordinate information are sent when the interaction signal of the air interaction interface is performed; the haptic feedback subsystem is connected to the imaging subsystem, and is used for the touch feedback control instruction and the touch control according to the touch feedback. The point coordinate information emits ultrasonic waves and focuses the ultrasonic waves on the touch points of the air interactive interface.
根据本公开实施例的触控反馈***，通过成像子***将人机交互信息在空中目标区域成像显示以形成空中交互界面，即成像子***采用可交互空中成像技术，将空中交互界面作为触觉感知的基准面，以用于引导用户触控，并在成像子***检测到用户与空中交互界面的交互信号时，发送触控反馈控制指令和触控点坐标信息至触觉反馈子***，触觉反馈子***根据接收的触控反馈控制指令和触控点坐标信息，发射超声波并将超声波聚焦至空中交互界面的触控点，即触觉反馈子***以超声波辐射压力，在人体与触控点之间产生触觉反馈效果，实现用户非接触式触觉反馈的目的，以及本公开实施例中基于在空中目标区域呈现空中交互界面，通过用户触摸空中交互界面即可触发触控反馈的操作，在视觉和触觉上同时满足用户的交互体验，且操作方式也更加自然舒适，无需设置额外的限制用户操作装置，避免用户操作时接触设备的风险，也避免出现因用户信息遗留在接触面上而造成个人信息泄露的问题，更加卫生安全。According to the touch feedback system of the embodiment of the present disclosure, the human-computer interaction information is imaged and displayed in the air target area through the imaging subsystem to form an air interaction interface, that is, the imaging subsystem adopts an interactive aerial imaging technology, and the air interaction interface is used as tactile perception. The reference plane is used to guide the user to touch, and when the imaging subsystem detects the interaction signal between the user and the air interface, it sends the touch feedback control command and touch point coordinate information to the haptic feedback subsystem. According to the received touch feedback control instructions and touch point coordinate information, the system emits ultrasonic waves and focuses the ultrasonic waves on the touch points of the air interface, that is, the haptic feedback subsystem uses ultrasonic waves to radiate pressure between the human body and the touch points. The haptic feedback effect achieves the purpose of non-contact haptic feedback for the user, and in the embodiment of the present disclosure, based on presenting an air interactive interface in the air target area, the touch feedback operation can be triggered by the user touching the air interactive interface, which is visually and tactilely At the same time, it satisfies the user's interactive experience, and the operation mode is also more natural and comfortable. There is no need to set additional restrictions on the user's operation, avoiding the risk of the user touching the device during operation, and avoiding the leakage of personal information due to the user's information being left on the contact surface. problem, more hygienic and safer.
在一些实施例中，所述触觉反馈子***包括：第一壳体，所述第一壳体设置有开窗且在内部形成有第一容纳腔；超声波发射器阵列，所述超声波发射器阵列设置于所述第一容纳腔中，包括多个超声波发射器，每个所述超声波发射器的发射端均朝向所述开窗；控制器，设置于所述第一容纳腔中，与所述超声波发射器阵列连接，用于根据所述触控反馈控制指令和所述触控点坐标信息控制每个所述超声波发射器发射超声波，并将所述超声波聚焦至所述触控点。In some embodiments, the haptic feedback subsystem includes: a first casing provided with a window and a first receiving cavity formed therein; an ultrasonic transmitter array, the ultrasonic transmitter array is arranged in the first accommodating cavity, and includes a plurality of ultrasonic transmitters, and the transmitting end of each ultrasonic transmitter faces the opening window; the controller is arranged in the first accommodating cavity, and is connected with the The ultrasonic transmitter array is connected to control each of the ultrasonic transmitters to emit ultrasonic waves according to the touch feedback control instruction and the coordinate information of the touch points, and focus the ultrasonic waves to the touch points.
在一些实施例中，所述控制器包括：数据处理模块，用于根据所述触控点坐标信息和每个所述超声波发射器的位置信息确定每个所述超声波发射器的延时相位；驱动模块，与所述超声波发射器阵列和所述数据处理模块连接，用于响应于所述触控反馈控制指令生成原始驱动信号，并根据每个所述超声波发射器的延时相位调整所述原始驱动信号以将多个所述超声波发射器发射的超声波聚焦至所述触控点。In some embodiments, the controller includes: a data processing module configured to determine the delay phase of each of the ultrasonic transmitters according to the coordinate information of the touch point and the position information of each of the ultrasonic transmitters; A driving module, connected with the ultrasonic transmitter array and the data processing module, is used for generating an original driving signal in response to the touch feedback control command, and adjusting the ultrasonic transmitter according to the delay phase of each ultrasonic transmitter The original driving signal is used to focus the ultrasonic waves emitted by the plurality of ultrasonic wave transmitters to the touch point.
在一些实施例中，所述驱动模块在响应于所述触控反馈控制指令时，根据所述触控反馈控制指令产生时钟信号和调制信号，并根据所述时钟信号和所述调制信号生成原始驱动信号。In some embodiments, when responding to the touch feedback control instruction, the driving module generates a clock signal and a modulation signal according to the touch feedback control instruction, and generates a raw signal according to the clock signal and the modulation signal. drive signal.
在一些实施例中，所述驱动模块根据每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理，以获得每个所述超声波发射器的目标驱动信号，并根据所述目标驱动信号驱动每个所述超声波发射器。In some embodiments, the driving module performs delay processing on the original driving signal according to the delayed phase of each of the ultrasonic transmitters, so as to obtain a target driving signal of each of the ultrasonic transmitters, and performs delay processing on the original driving signal according to the delayed phase of each of the ultrasonic transmitters. The target drive signal drives each of the ultrasonic transmitters.
在一些实施例中，所述数据处理模块包括：第一存储单元，用于存储数据处理过程中的数据和程序；数据处理单元，所述数据处理单元与所述第一存储单元连接，用于根据所述触控点坐标信息和每个所述超声波发射器的发射端的位置信息确定每个所述超声波发射器的延时相位。In some embodiments, the data processing module includes: a first storage unit for storing data and programs in a data processing process; a data processing unit, the data processing unit is connected to the first storage unit for The delay phase of each ultrasonic transmitter is determined according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter.
在一些实施例中，所述驱动模块包括：第二存储单元，用于存储驱动控制过程中的数据和文件；控制单元，所述控制单元与所述第二存储单元连接，用于根据所述触控反馈控制指令产生时钟信号和调制信号，并根据所述时钟信号和所述调制信号生成原始驱动信号，并根据每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理；驱动单元，所述驱动单元与所述控制单元连接，用于对延时后的原始驱动信号进行放大处理，获得每个所述超声波发射器的目标驱动信号，并根据所述目标驱动信号驱动每个所述超声波发射器。In some embodiments, the drive module includes: a second storage unit for storing data and files in a drive control process; a control unit, the control unit is connected to the second storage unit, for The touch feedback control instruction generates a clock signal and a modulation signal, and generates an original drive signal according to the clock signal and the modulation signal, and delays the original drive signal according to the delay phase of each ultrasonic transmitter processing; a driving unit, the driving unit is connected to the control unit, and is used for amplifying the delayed original driving signal to obtain the target driving signal of each ultrasonic transmitter, and according to the target driving signal Each of the ultrasonic transmitters is driven.
在一些实施例中，所述触觉反馈子***还包括：挡盖，设置在所述第一壳体上，用于遮挡所述开窗。In some embodiments, the haptic feedback subsystem further includes: a blocking cover, disposed on the first housing, for blocking the window opening.
在一些实施例中，所述成像子***包括：第二壳体，所述第二壳体形成有显示窗口且在内部形成有第二容纳腔；成像组件，设置于所述第二容纳腔，用于将人机交互信息在空中目标区域成像显示以形成空中交互界面；检测模块，用于检测用户与所述空中交互界面的交互信号；主控模块，与所述成像组件和所述检测模块连接，用于响应于所述交互信号发送触控反馈控制指令和触控点坐标信息。In some embodiments, the imaging subsystem includes: a second casing, the second casing is formed with a display window and a second accommodating cavity is formed inside; an imaging component is disposed in the second accommodating cavity, It is used to image and display the human-computer interaction information in the air target area to form an air interaction interface; a detection module is used to detect the interaction signal between the user and the air interaction interface; the main control module is connected to the imaging component and the detection module. The connection is used for sending touch feedback control instructions and touch point coordinate information in response to the interaction signal.
在一些实施例中，所述成像组件包括：显示器，所述显示器设置于所述第二容纳腔，与所述主控模块连接，用于显示所述人机交互信息；光学组件，所述光学组件设置于所述第二容纳腔，用于将携带所述人机交互信息的光线汇聚成像在所述空中目标区域，以形成空中交互界面；其中，所述显示器设置于所述光学组件的光源侧，所述显示窗口在所述光学组件的成像侧。In some embodiments, the imaging assembly includes: a display, the display is disposed in the second accommodating cavity, is connected to the main control module, and is used for displaying the human-computer interaction information; an optical assembly, the optical The component is arranged in the second accommodating cavity, and is used for converging and imaging the light carrying the human-computer interaction information on the aerial target area to form an aerial interaction interface; wherein, the display is arranged on the light source of the optical component side, the display window is on the imaging side of the optical assembly.
在一些实施例中，所述触觉反馈子***设置于所述成像子***的上方且位于所述空中交互界面的触摸侧；所述超声波发射器阵列的发射端与所述空中交互界面平行相对设置；所述超声波发射器阵列的发射端朝向所述空中交互界面。In some embodiments, the haptic feedback subsystem is disposed above the imaging subsystem and on the touch side of the air interface; the transmitting end of the ultrasonic transmitter array is disposed in parallel and opposite to the air interface ; The transmitting end of the ultrasonic transmitter array faces the air interactive interface.
在一些实施例中，所述触觉反馈子***设置于所述成像子***的上方且位于所述空中交互界面的成像侧，所述超声波发射器阵列与所述光学组件垂直，且所述超声波发射器阵列与所述空中交互界面在水平方向平齐，所述超声波发射器阵列的发射端朝向所述空中交互界面。In some embodiments, the haptic feedback subsystem is disposed above the imaging subsystem and on the imaging side of the air interface, the ultrasound transmitter array is perpendicular to the optical assembly, and the ultrasound transmits The transmitter array is flush with the air interaction interface in the horizontal direction, and the transmitting end of the ultrasonic transmitter array faces the air interaction interface.
在一些实施例中，所述触觉反馈子***设置于所述第二容纳腔中，且位于所述光学组件对携带所述人机交互信息的光线的反射路径上，所述超声波发射器阵列的发射端朝向所述空中交互界面。In some embodiments, the haptic feedback subsystem is disposed in the second accommodating cavity, and is located on the reflection path of the optical component to the light carrying the human-computer interaction information, and the ultrasonic transmitter array The transmitting end faces the air interaction interface.
在一些实施例中，所述触觉反馈子***设置于所述成像子***的上方且位于所述空中交互界面的触摸侧，所述超声波发射器阵列与所述光学组件平行设置，所述超声波发射器阵列与所述空中交互界面在垂直方向平齐，所述超声波发射器阵列的发射端朝向所述空中交互界面。In some embodiments, the haptic feedback subsystem is disposed above the imaging subsystem and on the touch side of the air interface, the ultrasonic transmitter array is disposed in parallel with the optical assembly, and the ultrasonic wave transmits The transmitter array is flush with the air interaction interface in the vertical direction, and the transmitting end of the ultrasonic transmitter array faces the air interaction interface.
本公开第二方面实施例提供一种终端设备，包括，设备本体；上述实施例所述的触控反馈***，所述触控反馈***设置在所述设备本体上。Embodiments of the second aspect of the present disclosure provide a terminal device, including a device body; and the touch feedback system according to the foregoing embodiments, where the touch feedback system is provided on the device body.
根据本公开实施例的终端设备，通过采用上述实施例提供的触控反馈***，可以实现用户非接触式触觉反馈的目的，且无需设置额外的限制用户操作装置，操作方式也更加自然方便，并在视觉和触觉上同时满足用户的交互体验。According to the terminal device according to the embodiments of the present disclosure, by using the touch feedback system provided by the above embodiments, the purpose of non-contact tactile feedback for the user can be achieved, and no additional restricting user operation device is required, and the operation method is also more natural and convenient. Satisfy the user's interactive experience both visually and tactilely.
本公开第三方面实施例提供一种触控反馈控制方法，包括：将人机交互信息在空中目标区域成像显示以形成空中交互界面；检测到用户与所述空中交互界面的交互信号，获取触控点坐标信息并发送触控反馈控制指令；根据所述触控反馈控制指令和所述触控点坐标信息控制超声波发射器阵列发射超声波并将所述超声波聚焦至所述空中交互界面的触控点。A third aspect of the present disclosure provides a touch feedback control method, including: imaging and displaying human-computer interaction information in an aerial target area to form an aerial interactive interface; detecting an interaction signal between a user and the aerial interactive interface, and obtaining touch control point coordinate information and send touch feedback control instructions; control the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch control of the air interactive interface according to the touch feedback control instructions and the touch point coordinate information point.
根据本公开实施例的触控反馈控制方法，通过将人机交互信息在空中目标区域成像显示以形成空中交互界面，即采用可交互空中成像技术，将空中交互界面作为触觉感知的基准面，以用于引导用户触控，并在检测到用户与空中交互界面的交互信号时，获取触控反馈控制指令和触控点坐标信息，以根据获取的触控反馈控制指令和触控点坐标信息，控制超声波发射器阵列发射超声波并将超声波聚焦至空中交互界面的触控点，即通过超声波辐射压力，以在人体与触控点之间产生触觉反馈效果，实现用户非接触式触觉反馈的目的，以及本公开实施例中基于在空中目标区域呈现空中交互界面，通过用户触摸空中交互界面即可触发触控反馈的操作，在视觉和触觉上同时满足用户的交互体验，且操作方式也更加自然舒适，无需设置额外的限制用户操作装置，避免用户操作时接触设备的风险，也避免出现因用户信息遗留在接触面上而造成个 人信息泄露的问题，更加卫生安全。According to the touch feedback control method of the embodiment of the present disclosure, an aerial interactive interface is formed by imaging and displaying human-computer interaction information in an aerial target area, that is, an interactive aerial imaging technology is adopted, and the aerial interactive interface is used as a reference surface for tactile perception, so as to It is used to guide the user to touch, and when the interaction signal between the user and the air interaction interface is detected, obtain the touch feedback control instruction and touch point coordinate information, so as to obtain the touch feedback control instruction and touch point coordinate information according to the acquired touch feedback control instruction and touch point coordinate information, Control the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch points of the air interactive interface, that is, through ultrasonic radiation pressure, to generate tactile feedback effects between the human body and the touch points, to achieve the purpose of non-contact tactile feedback for users, And in the embodiment of the present disclosure, based on the air interactive interface presented in the air target area, the touch feedback operation can be triggered by the user touching the air interactive interface, which satisfies the user's interactive experience both visually and tactilely, and the operation mode is also more natural and comfortable. , there is no need to set additional restrictions on user operation, to avoid the risk of users touching the equipment during operation, and to avoid the problem of personal information leakage caused by user information left on the contact surface, which is more hygienic and safer.
在一些实施例中，根据所述触控反馈控制指令和所述触控点坐标信息控制超声波发射器阵列发射超声波并将所述超声波聚焦至所述空中交互界面的触控点，包括：根据所述触控点坐标信息和所述超声波发射器阵列中每个超声波发射器的发射端的位置信息确定每个所述超声波发射器的延时相位；根据所述触控反馈控制指令生成原始驱动信号；根据每个所述超声波发射器的延时相位对所述原始驱动信号进行调整，以将多个所述超声波发射器发射的超声波聚焦至所述触控点。In some embodiments, controlling the ultrasonic transmitter array to emit ultrasonic waves and focusing the ultrasonic waves to the touch points of the air interactive interface according to the touch feedback control instruction and the touch point coordinate information includes: The coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter in the ultrasonic transmitter array determine the delay phase of each ultrasonic transmitter; the original drive signal is generated according to the touch feedback control instruction; The original driving signal is adjusted according to the delayed phase of each of the ultrasonic transmitters, so as to focus the ultrasonic waves emitted by the plurality of ultrasonic transmitters to the touch point.
在一些实施例中，根据所述触控反馈控制指令生成原始驱动信号，包括：根据所述触控反馈控制指令产生时钟信号和调制信号；根据所述时钟信号和所述调制信号生成所述原始驱动信号。In some embodiments, generating the original driving signal according to the touch feedback control instruction includes: generating a clock signal and a modulation signal according to the touch feedback control instruction; generating the original driving signal according to the clock signal and the modulation signal drive signal.
在一些实施例中，根据每个所述超声波发射器的延时相位对所述原始驱动信号进行调整，包括：根据每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理；获得每个所述超声波发射器的目标驱动信号；根据所述目标驱动信号驱动每个所述超声波发射器。In some embodiments, adjusting the original driving signal according to the delayed phase of each of the ultrasonic transmitters includes: delaying the original driving signal according to the delayed phase of each of the ultrasonic transmitters processing; obtaining a target driving signal of each of the ultrasonic transmitters; and driving each of the ultrasonic transmitters according to the target driving signal.
在一些实施例中，还包括：获取环境温度；根据所述环境温度对每个所述超声波发射器的延时相位进行校正。In some embodiments, the method further includes: acquiring an ambient temperature; and correcting the delay phase of each of the ultrasonic transmitters according to the ambient temperature.
在一些实施例中，超声波发射器阵列包括n个分布在同一平面的超声波发射器子阵列，所述空中交互界面划分为n个触控子区，n个所述超声波发射器子阵列与n个所述触控子区一一对应配置；根据所述触控反馈控制指令和所述触控点坐标信息控制超声波发射器阵列发射超声波并将所述超声波聚焦至触控点，包括：根据所述触控点坐标信息确定触控点对应所述空中交互界面中的目标触控子区；根据所述目标触控子区确定目标超声波发射器子阵列，根据所述触控点坐标信息和所述目标超声波发射器子阵列中每个超声波发射器的发射端的位置信息确定所述目标超声波发射器子阵列中每个所述超声波发射器的延时相位；根据所述触控反馈控制指令产生时钟信号和调制信号，并根据所述时钟信号和所述调制信号生成原始驱动信号；根据所述目标超声波发射器子阵列中每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理以获得所述超声波发射器子阵列的每个所述超声波发射器的目标驱动信号；根据所述目标驱动信号驱动所述超声波发射器子阵列的每个所述超声波发射器。In some embodiments, the ultrasonic transmitter array includes n ultrasonic transmitter sub-arrays distributed on the same plane, the air interactive interface is divided into n touch sub-areas, the n ultrasonic transmitter sub-arrays and the n ultrasonic transmitter sub-arrays The touch sub-areas are configured in one-to-one correspondence; controlling the ultrasonic transmitter array to emit ultrasonic waves and focusing the ultrasonic waves to the touch points according to the touch feedback control instructions and the touch point coordinate information includes: according to the The touch point coordinate information determines that the touch point corresponds to the target touch sub-area in the air interactive interface; the target ultrasonic transmitter sub-array is determined according to the target touch sub-area, and the touch point coordinate information and the The position information of the transmitting end of each ultrasonic transmitter in the target ultrasonic transmitter sub-array determines the delay phase of each ultrasonic transmitter in the target ultrasonic transmitter sub-array; a clock signal is generated according to the touch feedback control instruction and modulation signal, and generate an original drive signal according to the clock signal and the modulation signal; delay the original drive signal according to the delay phase of each of the ultrasonic transmitters in the target ultrasonic transmitter sub-array processing to obtain a target driving signal of each of the ultrasonic transmitters of the ultrasonic transmitter sub-array; and driving each of the ultrasonic transmitters of the ultrasonic transmitter sub-array according to the target driving signal.
在一些实施例中，所述超声波发射器阵列为n个，n个所述超声波发射器阵列分布在同一平面或者以预设角度错开设置，所述空中交互界面划分为n个触控子区，n个所述超声波发射器阵列与n个所述触控子区一一对应配置；根据所述触控反馈控制指令和所述触控点坐标信息控制超声波发射器阵列发射超声波并将所述超声波聚焦至触控点，包括：根据所述触控点坐标信息确定触控点对应所述空中交互界面中的目标触控子区；根据所述目标触控子区确定目标超声波发射器阵列，根据所述触控点坐标信息和所述目标超声波发射器阵列中每个超声波发射器的发射端的位置信息确定所述目标超声波发射器阵列中每个所述超声波发射器的延时相位；根据所述触控反馈控制指令产生时钟信号和调制信号，并根据所述时钟信号和所述调制信号生成原始驱动信号；根据所述目标超声波发射器阵列中每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理以获得所述目标超声波发射器阵列中每个所述超声波发射器的目标驱动信号；根据所述目标驱动信号驱动所述目标超声波发射器阵列中每个所述超声波发射器。In some embodiments, the number of ultrasonic transmitter arrays is n, and the n ultrasonic transmitter arrays are distributed on the same plane or staggered at a preset angle, and the air interactive interface is divided into n touch sub-areas, The n ultrasonic transmitter arrays are arranged in a one-to-one correspondence with the n touch sub-areas; the ultrasonic transmitter array is controlled to emit ultrasonic waves according to the touch feedback control instruction and the touch point coordinate information and transmit the ultrasonic waves to the ultrasonic wave. Focusing on the touch point includes: determining, according to the coordinate information of the touch point, that the touch point corresponds to a target touch sub-area in the air interactive interface; determining a target ultrasonic transmitter array according to the target touch sub-area; The touch point coordinate information and the position information of the transmitting end of each ultrasonic transmitter in the target ultrasonic transmitter array determine the delay phase of each ultrasonic transmitter in the target ultrasonic transmitter array; according to the The touch feedback control instruction generates a clock signal and a modulation signal, and generates an original drive signal according to the clock signal and the modulation signal; according to the delay phase of each ultrasonic transmitter in the target ultrasonic transmitter array performing delay processing on the original driving signal to obtain a target driving signal for each of the ultrasonic transmitters in the target ultrasonic transmitter array; and driving each ultrasonic wave in the target ultrasonic transmitter array according to the target driving signal launcher.
本公开第四方面实施例提供一种存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现上述实施例所述的触控反馈控制方法。Embodiments of the fourth aspect of the present disclosure provide a storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the touch feedback control method described in the foregoing embodiments is implemented.
本公开的附加方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本公开的实践了解到。Additional aspects and advantages of the present disclosure will be set forth, in part, from the following description, and in part will become apparent from the following description, or may be learned by practice of the present disclosure.
附图说明Description of drawings
本公开的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：The above and/or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:
图1是根据本公开一个实施例的触控反馈***的结构框图；FIG. 1 is a structural block diagram of a touch feedback system according to an embodiment of the present disclosure;
图2是根据本公开一个实施例的触觉反馈子***的结构示意图；FIG. 2 is a schematic structural diagram of a haptic feedback subsystem according to an embodiment of the present disclosure;
图3是根据本公开一个实施例的超声波发射器阵列的示意图；3 is a schematic diagram of an ultrasonic transmitter array according to one embodiment of the present disclosure;
图4是根据本公开一个实施例的数据处理模块的结构框图；4 is a structural block diagram of a data processing module according to an embodiment of the present disclosure;
图5是根据本公开一个实施例的驱动模块的结构框图；5 is a structural block diagram of a driving module according to an embodiment of the present disclosure;
图6是根据本公开一个实施例的触控反馈***的结构示意图；FIG. 6 is a schematic structural diagram of a touch feedback system according to an embodiment of the present disclosure;
图7是根据本公开一个实施例的人机交互的结构示意图；7 is a schematic structural diagram of human-computer interaction according to an embodiment of the present disclosure;
图8是根据本公开一个实施例的光学组件的结构示意图；8 is a schematic structural diagram of an optical assembly according to an embodiment of the present disclosure;
图9是根据本公开一个实施例的第一光波导阵列和第二光波导阵列的示意图；9 is a schematic diagram of a first optical waveguide array and a second optical waveguide array according to an embodiment of the present disclosure;
图10是根据本公开一个实施例的光学组件沿厚度方向的正面结构示意图；10 is a schematic diagram of a front structure of an optical assembly along the thickness direction according to an embodiment of the present disclosure;
图11是根据本公开一个实施例的第一光波导阵列和第二光波导阵列的局部结构示意图；FIG. 11 is a schematic partial structure diagram of a first optical waveguide array and a second optical waveguide array according to an embodiment of the present disclosure;
图12是根据本公开一个实施例的光学组件的光路示意图；12 is a schematic diagram of an optical path of an optical assembly according to an embodiment of the present disclosure;
图13是根据本公开一个实施例的超声波发射器阵列设置于第二壳体的上方的布置示意图；13 is a schematic diagram of the arrangement of the ultrasonic transmitter array disposed above the second housing according to an embodiment of the present disclosure;
图14是根据本公开一个实施例的超声波发射器阵列设置于第二容纳腔的布置示意图；14 is a schematic diagram of the arrangement of the ultrasonic transmitter array disposed in the second accommodating cavity according to an embodiment of the present disclosure;
图15是根据本公开另一个实施例的超声波发射器阵列设置于第二壳体的上方的布置示意图；15 is a schematic diagram of an arrangement of an ultrasonic transmitter array disposed above the second housing according to another embodiment of the present disclosure;
图16是根据本公开一个实施例的终端设备的结构框图；16 is a structural block diagram of a terminal device according to an embodiment of the present disclosure;
图17是根据本公开一个实施例的触控反馈控制方法的流程示意图；17 is a schematic flowchart of a touch feedback control method according to an embodiment of the present disclosure;
图18是根据本发明一个实施例的触控反馈控制方法的流程示意图；18 is a schematic flowchart of a touch feedback control method according to an embodiment of the present invention;
图19是根据本公开一个实施例的单块超声波发射器阵列的布局示意图；19 is a schematic layout diagram of a monolithic ultrasonic transmitter array according to an embodiment of the present disclosure;
图20是根据本公开一个实施例的多块超声波发射器阵列的布局示意图。FIG. 20 is a schematic layout diagram of a multi-block ultrasonic transmitter array according to an embodiment of the present disclosure.
附图标记：Reference number:
终端设备2000；触控反馈***1000；设备本体300；成像子***100；触觉反馈子***200； Terminal device 2000; touch feedback system 1000; device body 300; imaging subsystem 100; haptic feedback subsystem 200;
第一壳体210；超声波发射器阵列220；控制器250；数据处理模块230；驱动模块240；The first housing 210; the ultrasonic transmitter array 220; the controller 250; the data processing module 230; the driving module 240;
第一容纳腔2；第一散热孔3；第二散热孔4；挡盖5；超声波发射器30；第一存储单元11；数据处理单元12；第一接口单元13；第一电源单元14；第一串行接口15；第一并行接口16；第二存储单元17；控制单元18；驱动单元19；第二电源单元20；第二接口单元21；电源输入口22；第二并行接口23；驱动信号输出口24；第二壳体110；成像组件120；检测模块130；主控模块140；显示器25；光学组件26；空中目标区域10；显示窗口40；第二容纳腔50；第一光波导阵列6；第二光波导阵列7；透明基板8；反射单元9；反射膜27；胶粘剂28。first accommodating cavity 2; first heat dissipation hole 3; second heat dissipation hole 4; cover 5; ultrasonic transmitter 30; first storage unit 11; data processing unit 12; first interface unit 13; first power supply unit 14; The first serial interface 15; the first parallel interface 16; the second storage unit 17; the control unit 18; the drive unit 19; the second power supply unit 20; the second interface unit 21; the power input port 22; the second parallel interface 23; The driving signal output port 24; the second housing 110; the imaging assembly 120; the detection module 130; the main control module 140; the display 25; the optical assembly 26; Waveguide array 6 ; second optical waveguide array 7 ; transparent substrate 8 ; reflection unit 9 ; reflection film 27 ;
具体实施方式Detailed ways
下面详细描述本公开的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本公开，而不能理解为对本公开的限制。Embodiments of the present disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present disclosure and should not be construed as a limitation of the present disclosure.
为了解决上述问题，本公开第一方面实施例提出一种触控反馈***，该***既可以实现用户非接触式触觉反馈，又可以在视觉和触觉上同时满足用户的交互体验，更加卫生安全。In order to solve the above problems, an embodiment of the first aspect of the present disclosure proposes a touch feedback system, which can not only realize user non-contact tactile feedback, but also satisfy the user's interactive experience in both visual and tactile senses, which is more hygienic and safer.
下面参考图1描述本公开实施例提出的触控反馈***。The following describes the touch feedback system proposed by the embodiment of the present disclosure with reference to FIG. 1 .
如图1所示，本公开实施例提供的触控反馈***1000包括成像子***100和触觉反馈子***200。As shown in FIG. 1 , a touch feedback system 1000 provided by an embodiment of the present disclosure includes an imaging subsystem 100 and a haptic feedback subsystem 200 .
其中，成像子***100用于将人机交互信息在空中目标区域成像显示以形成空中交互界面，并在检测到用户与空中交互界面的交互信号时发送触控反馈控制指令和触控点坐标信息。触觉反馈子***200与成像子***100连接，触觉反馈子***200用于根据触控反馈控制指令和触控点坐标信息发射超声波并将超声波聚焦至空中交互界面的触控点。The imaging subsystem 100 is used to image and display the human-computer interaction information in the aerial target area to form an aerial interactive interface, and send touch feedback control instructions and touch point coordinate information when detecting the interaction signal between the user and the aerial interactive interface . The haptic feedback subsystem 200 is connected to the imaging subsystem 100, and the haptic feedback subsystem 200 is used for transmitting ultrasonic waves and focusing the ultrasonic waves on the touch points of the air interactive interface according to the touch feedback control instructions and the coordinate information of the touch points.
在本公开实施例中，成像子***100采用可交互空中成像技术，在空中的确定位置处形成浮空实像即空中交互界面，以作为用户触觉感知的基准面，而覆盖浮空实像所在的三维空间即为空中目标区域。通过成像子***100将人机交互信息汇聚成像在空中目标区域，以形成空中交互界面，并在检测到用户与空中交互界面的交互信号时，成像子***100发送触控反馈控制指令和触控点坐标信息至触觉反馈子***200。In the embodiment of the present disclosure, the imaging subsystem 100 adopts the interactive aerial imaging technology to form a floating real image, that is, an aerial interactive interface at a certain position in the air, as a reference plane for the user's tactile perception, and covers the three-dimensional space where the floating real image is located. Space is the air target area. The imaging subsystem 100 gathers the human-computer interaction information into the aerial target area to form an aerial interactive interface, and when detecting the interaction signal between the user and the aerial interactive interface, the imaging subsystem 100 sends touch feedback control instructions and touch Point coordinate information to the haptic feedback subsystem 200 .
触觉反馈子***200的触控区域设置为覆盖空中交互界面所在三维空间，即空中目标区域。具体地，基于空中交互界面的大小和显示位置是相对固定的，触觉反馈子***200根据已知的空中交互界面，产生一个与之等大等位置的触控反馈平面。即由成像子***100提供空中交互界面，以引导用户触摸，以及由触觉反馈子***200提供触控反馈平面，以反馈用户对触摸对象的感知。The touch area of the haptic feedback subsystem 200 is set to cover the three-dimensional space where the air interaction interface is located, that is, the air target area. Specifically, since the size and display position of the air interaction interface are relatively fixed, the haptic feedback subsystem 200 generates a touch feedback plane of the same size and position according to the known air interaction interface. That is, the imaging subsystem 100 provides an air interface to guide the user to touch, and the haptic feedback subsystem 200 provides a touch feedback plane to feedback the user's perception of the touch object.
根据本公开实施例的触控反馈***1000，通过成像子***100将人机交互信息在空中目标 区域成像显示以形成空中交互界面，将空中交互界面作为用户触觉感知的基准面，以用于引导用户触控，并在成像子***100检测到用户与空中交互界面的交互信号时，发送触控反馈控制指令和触控点坐标信息至触觉反馈子***200。触觉反馈子***200根据接收的触控反馈控制指令和触控点坐标信息，发射超声波并将超声波聚焦至触控点，在人体与触控点之间产生触觉反馈效果，使得人体可以真实感受到触控点的存在，实现用户非接触式触觉反馈的目的。本公开实施例中基于在空中目标区域呈现空中交互界面，通过用户触摸空中交互界面即可触发触控反馈的操作，在视觉和触觉上同时满足用户的交互体验，且操作方式也更加自然舒适，无需设置额外的限制用户操作装置，避免用户操作时接触设备的风险。同时避免出现因用户指纹信息残留，使得个人信息泄露而造成的信息安全问题。According to the touch feedback system 1000 of the embodiment of the present disclosure, the human-computer interaction information is imaged and displayed in the air target area through the imaging subsystem 100 to form an air interaction interface, and the air interaction interface is used as the reference surface of the user's tactile perception for guiding The user touches, and when the imaging subsystem 100 detects the interaction signal between the user and the air interaction interface, it sends a touch feedback control instruction and touch point coordinate information to the haptic feedback subsystem 200 . The haptic feedback subsystem 200 transmits ultrasonic waves and focuses the ultrasonic waves on the touch points according to the received touch feedback control instructions and touch point coordinate information, and generates a haptic feedback effect between the human body and the touch points, so that the human body can truly feel The existence of touch points realizes the purpose of non-contact tactile feedback for users. In the embodiment of the present disclosure, based on presenting an air interactive interface in the air target area, the touch feedback operation can be triggered by the user touching the air interactive interface, which satisfies the user's interactive experience both visually and tactilely, and the operation mode is also more natural and comfortable. There is no need to set additional restrictions on user operation to avoid the risk of users touching the equipment during operation. At the same time, information security problems caused by the leakage of personal information due to residual fingerprint information of users are avoided.
在一些实施例中，如图2所示，触觉反馈子***200包括第一壳体210、超声波发射器阵列220和控制器250。In some embodiments, as shown in FIG. 2 , the haptic feedback subsystem 200 includes a first housing 210 , an ultrasonic transmitter array 220 and a controller 250 .
第一壳体210上设置有开窗且在内部形成有第一容纳腔2。第一壳体210可以为触觉反馈子***200提供保护和支撑，以防止外力冲击、振动对触觉反馈子***200造成损坏。The first housing 210 is provided with a window and has a first accommodating cavity 2 formed therein. The first housing 210 can provide protection and support for the haptic feedback subsystem 200 to prevent damage to the haptic feedback subsystem 200 caused by external force shock and vibration.
超声波发射器阵列220设置于第一容纳腔2中，以起到保护作用。根据声学理论，超声波可以在一定空间内产生声压，声压分布与距离呈反相关，由于单个超声波发射器发出的超声波所产生的声压不足以对用户提供触觉反馈，当超声波发射器的数量达到一定数量时，每个超声波发射器所产生的超声波同时到达空间中某一点并在该点进行聚焦叠加，所叠加的声压远远大于单个超声波发生器产生的声压，在该点人体可感知到振动，从而实现超声波触觉反馈。因此，在本公开实施例中，例如图3所示的结构示意图，超声波发射器阵列220包括多个超声波发射器30，多个超声波发射器30以N*N阵列或M*N阵列组合构成超声波发射器阵列220，且如图2所示，每个超声波发射器30的发射端均朝向开窗，并暴露于开窗处，使得超声波可以正常辐射，减少辐射能量的损失。The ultrasonic transmitter array 220 is arranged in the first accommodating cavity 2 to play a protective role. According to the acoustic theory, ultrasonic waves can generate sound pressure in a certain space, and the sound pressure distribution is inversely related to the distance. Since the sound pressure generated by the ultrasonic waves emitted by a single ultrasonic transmitter is not enough to provide tactile feedback to the user, when the number of ultrasonic transmitters When a certain number is reached, the ultrasonic waves generated by each ultrasonic transmitter reach a certain point in space at the same time and are focused and superimposed at that point, and the superimposed sound pressure is far greater than the sound pressure generated by a single ultrasonic generator. Vibration is sensed, enabling ultrasonic haptic feedback. Therefore, in the embodiment of the present disclosure, such as the schematic structural diagram shown in FIG. 3 , the ultrasonic transmitter array 220 includes a plurality of ultrasonic transmitters 30 , and the plurality of ultrasonic transmitters 30 are combined to form an ultrasonic wave in an N*N array or an M*N array. The transmitter array 220, and as shown in FIG. 2, the transmitting end of each ultrasonic transmitter 30 faces the window and is exposed to the window, so that ultrasonic waves can be radiated normally and the loss of radiation energy can be reduced.
超声波发射器阵列220可以由若干个小功率、小型化的超声波发射器30组成，以便用于商用和集成。The ultrasound transmitter array 220 may consist of several low-power, miniaturized ultrasound transmitters 30 for commercial use and integration.
需要说明的是，超声波发射器阵列220采用超声波发射器30的数量可以根据***需求如触控区域大小、触控距离等以及根据超声波发射器30自身技术指标例如谐振频率、声压级、方向角等实际情况来决定，在此，对于采用超声波发生器30的数量不作限制。It should be noted that the number of ultrasonic transmitters 30 used in the ultrasonic transmitter array 220 can be based on system requirements, such as the size of the touch area, touch distance, etc., and according to the technical indicators of the ultrasonic transmitter 30 itself, such as resonant frequency, sound pressure level, and direction angle. It depends on the actual situation, and here, there is no limit to the number of ultrasonic generators 30 used.
控制器250设置于第一容纳腔2中，与超声波发射器阵列220连接，用于根据触控反馈控制指令和触控点坐标信息控制每个超声波发射器30发射超声波，并将超声波聚焦至触控点。具体地，将超声波发射器阵列220的聚焦点位置设定为空中交互界面所在区域。当用户与空中交互界面交互时，控制器250根据触控反馈控制指令控制超声波发射器阵列220发射超声波且根据触控点坐标信息控制每个超声波发射器30的超声波聚焦至触控点，在触控点处超声波声压叠加而产生触觉反馈力，使得人体真实感受到触控点的存在，实现对触摸对象的感知。The controller 250 is arranged in the first accommodating cavity 2 and connected to the ultrasonic transmitter array 220, and is used to control each ultrasonic transmitter 30 to emit ultrasonic waves according to the touch feedback control instruction and the touch point coordinate information, and focus the ultrasonic waves to the touch point. control point. Specifically, the focal point position of the ultrasonic transmitter array 220 is set as the area where the air interactive interface is located. When the user interacts with the air interface, the controller 250 controls the ultrasonic transmitter array 220 to emit ultrasonic waves according to the touch feedback control instruction, and controls the ultrasonic waves of each ultrasonic transmitter 30 to focus on the touch point according to the coordinate information of the touch point. The ultrasonic sound pressure at the control point is superimposed to generate a tactile feedback force, so that the human body can truly feel the existence of the touch point and realize the perception of the touch object.
在一些实施例中，如图2所示，控制器250包括数据处理模块230以及驱动模块240。In some embodiments, as shown in FIG. 2 , the controller 250 includes a data processing module 230 and a driving module 240 .
数据处理模块230用于根据触控点坐标信息和每个超声波发射器30的发射端的位置信息确定每个超声波发射器30发射超声波的延时相位。具体地，由于超声波发射器阵列220上的每个超声波发射器30发射端到触控点的空间距离不同，为了使得每个超声波发射器30发射的超声波在同一时刻到达触控点，以在触控点产生触觉反馈，数据处理模块230在完成触控点坐标数据处理与相位计算后，根据每个超声波发射器30的位置和触控点坐标数据，对超声波发射器30的控制信号相位进行延时控制，使超声波在同时刻传播至触控点，实现多个超声波同时刻在触控点进行叠加而产生振动反馈。The data processing module 230 is configured to determine the delay phase of the ultrasonic wave transmitted by each ultrasonic transmitter 30 according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter 30 . Specifically, since the spatial distance from the transmitting end of each ultrasonic transmitter 30 on the ultrasonic transmitter array 220 to the touch point is different, in order to make the ultrasonic waves emitted by each ultrasonic transmitter 30 reach the touch point at the same time, so that the The control point generates tactile feedback. After the data processing module 230 completes the touch point coordinate data processing and phase calculation, it delays the phase of the control signal of the ultrasonic transmitter 30 according to the position of each ultrasonic transmitter 30 and the touch point coordinate data. Time control, so that the ultrasonic waves are transmitted to the touch points at the same time, so that multiple ultrasonic waves are superimposed on the touch points at the same time to generate vibration feedback.
在本公开实施例中，由数据处理模块230计算超声波发射器30发射超声波的延时相位。具体地，以超声波发射器阵列220的阵列面为坐标平面，以阵列面几何中心作为坐标原点O，阵列面上与阵列边平行且经过原点O的一组正交轴作为x、y轴，z轴垂直于阵列面且经过原点O，建立空间直角坐标系，其中，阵列面为超声波发射器阵列220的发射端所在平面。数据处理模块230根据超声波发射器30在超声波发射器阵列220上的排布，获取超声波发射器30发射端几何中心的坐标，并结合成像子***100发送的触控点坐标信息，计算获得每个超声波发射器30发射端几何中心至各触控点的空间距离，进而将超声波发射器阵列220中各超声波发射器30之间的空间距离差值，代入声速值c，最终获得各超声波发射器30的延时相位。需要说明的是，成像子***100获取的触控点坐标与超声波发射器30发射端的几何中心的坐标为同一坐标系。In the embodiment of the present disclosure, the data processing module 230 calculates the delay phase of the ultrasonic wave transmitted by the ultrasonic transmitter 30 . Specifically, the array plane of the ultrasonic transmitter array 220 is taken as the coordinate plane, the geometric center of the array plane is taken as the coordinate origin O, a set of orthogonal axes on the array plane that are parallel to the array sides and pass through the origin O are taken as the x and y axes, z The axis is perpendicular to the array plane and passes through the origin O to establish a space rectangular coordinate system, wherein the array plane is the plane where the transmitting end of the ultrasonic transmitter array 220 is located. The data processing module 230 obtains the coordinates of the geometric center of the transmitting end of the ultrasonic transmitter 30 according to the arrangement of the ultrasonic transmitters 30 on the ultrasonic transmitter array 220, and combines the touch point coordinate information sent by the imaging subsystem 100 to calculate and obtain each The spatial distance from the geometric center of the transmitting end of the ultrasonic transmitter 30 to each touch point, and then the spatial distance difference between the ultrasonic transmitters 30 in the ultrasonic transmitter array 220 is substituted into the sound velocity value c, and finally each ultrasonic transmitter 30 is obtained. delay phase. It should be noted that the coordinates of the touch point acquired by the imaging subsystem 100 and the coordinates of the geometric center of the transmitting end of the ultrasonic transmitter 30 are in the same coordinate system.
驱动模块240与超声波发射器阵列220和数据处理模块230连接，用于响应于触控反馈控制指令生成原始驱动信号，并根据每个超声波发射器30的延时相位调整原始驱动信号以将多个超声波发射器30发射的超声波聚焦至触控点。具体地，由于超声波发射器阵列220上每个超声波发射器30发射端到触控点的空间距离不同，因此通过驱动模块240根据每个超声波发射器30的延时相位对原始驱动信号进行延时处理，即通过调节每个超声波发射器30发射超声波的发射时间，以控制多个超声波声压同时刻在触控点进行叠加，实现触觉反馈。The driving module 240 is connected with the ultrasonic transmitter array 220 and the data processing module 230, and is used for generating the original driving signal in response to the touch feedback control instruction, and adjusting the original driving signal according to the delay phase of each ultrasonic transmitter 30 to convert the plurality of ultrasonic transmitters 30. The ultrasonic waves emitted by the ultrasonic transmitter 30 are focused to the touch point. Specifically, since the spatial distance from the transmitting end of each ultrasonic transmitter 30 on the ultrasonic transmitter array 220 to the touch point is different, the driving module 240 delays the original driving signal according to the delay phase of each ultrasonic transmitter 30 The processing is to control the multiple ultrasonic sound pressures to be superimposed on the touch points at the same time by adjusting the emission time of each ultrasonic transmitter 30 to transmit ultrasonic waves, so as to realize haptic feedback.
在一些实施例中，驱动模块240在响应于触控反馈控制指令时，根据触控反馈控制指令产生时钟信号和调制信号，并根据时钟信号和调制信号生成原始驱动信号。In some embodiments, when responding to the touch feedback control command, the driving module 240 generates a clock signal and a modulation signal according to the touch feedback control command, and generates an original driving signal according to the clock signal and the modulation signal.
具体地，在成像子***100检测到用户与空中交互界面的交互信号后，成像子***100将空中交互界面中的触控点坐标信息和触控反馈控制指令分别传送给数据处理模块230和驱动模块240。数据处理模块230根据触控点空间坐标以及超声波发射器阵列220上各超声波发射器30的位置，计算出每个超声波发射器30的延时相位，并将延时相位发送给驱动模块240。驱动模块240接收触控反馈控制指令并完成信号调制过程与时钟分配，即产生时钟信号和调制信号，以生成若干超声波发射器30的原始驱动信号；以及驱动模块240再根据数据处理模块230发送的延时相位，对每个超声波发射器30的原始驱动信号进行延时控制，以使得各个超声波发射器30在对应的相位下发射超声波，并在同一时刻到达触控点处进行叠加汇聚，以产生触控反馈，使得人体可以真实感受到触控点的存在。Specifically, after the imaging subsystem 100 detects the interaction signal between the user and the air interactive interface, the imaging subsystem 100 transmits the touch point coordinate information and the touch feedback control instruction in the air interactive interface to the data processing module 230 and the driver respectively. module 240. The data processing module 230 calculates the delayed phase of each ultrasonic transmitter 30 according to the spatial coordinates of the touch point and the position of each ultrasonic transmitter 30 on the ultrasonic transmitter array 220 , and sends the delayed phase to the driving module 240 . The driving module 240 receives the touch feedback control instruction and completes the signal modulation process and clock distribution, that is, generates the clock signal and the modulation signal to generate the original driving signals of several ultrasonic transmitters 30; Delay phase, the original drive signal of each ultrasonic transmitter 30 is delayed and controlled, so that each ultrasonic transmitter 30 emits ultrasonic waves in the corresponding phase, and reaches the touch point at the same time for superposition and convergence to generate Touch feedback allows the human body to truly feel the presence of touch points.
在一些实施例中，驱动模块240根据每个超声波发射器30的延时相位对原始驱动信号进行延时处理，以获得每个超声波发射器30的目标驱动信号，并根据目标驱动信号驱动每个超声波发射器30发射超声波。具体地，在驱动模块240接收到数据处理模块230传输的延时相位后，驱动模块240对每个超声波发射器30的原始驱动信号进行延时处理，并对延时控制后的信号再进行放大处理，以获得每个超声波发射器30所需的目标驱动信号，并将各个超声波发射器30的目标驱动信号同步传输至超声波发射器阵列220，以驱动各个超声波发射器30在对应的目标驱动信号下依次发射超声波，实现每个超声波发射器30发射的超声波在同一时刻到达触控点，以在触控点产生触觉反馈。In some embodiments, the driving module 240 performs delay processing on the original driving signal according to the delayed phase of each ultrasonic transmitter 30 to obtain the target driving signal of each ultrasonic transmitter 30, and drives each ultrasonic transmitter 30 according to the target driving signal The ultrasonic transmitter 30 transmits ultrasonic waves. Specifically, after the driving module 240 receives the delayed phase transmitted by the data processing module 230, the driving module 240 performs delay processing on the original driving signal of each ultrasonic transmitter 30, and then amplifies the delayed signal. processing to obtain the target driving signal required by each ultrasonic transmitter 30, and synchronously transmit the target driving signal of each ultrasonic transmitter 30 to the ultrasonic transmitter array 220, so as to drive each ultrasonic transmitter 30 in the corresponding target driving signal The ultrasonic waves are emitted in sequence, so that the ultrasonic waves emitted by each ultrasonic transmitter 30 reach the touch point at the same time, so as to generate haptic feedback at the touch point.
本公开实施例中，优选地，超声波发射器30采用类球面波形式向外发射超声波，球面波球心即为超声波发射器30发射端的几何中心。In the embodiment of the present disclosure, preferably, the ultrasonic transmitter 30 transmits ultrasonic waves in the form of spherical waves, and the spherical center of the spherical wave is the geometric center of the transmitting end of the ultrasonic transmitter 30 .
此外，超声波在有限空间内的声压分布，一方面与距离呈负相关，即距离越远声压越弱，使得触控点距超声波发射器阵列220的阵列面的距离受限制；另一方面与超声波频率f有关，根据超声波相关理论，超声波在空气介质中的自由空间波数
Figure PCTCN2022071859-appb-000001
即频率f越高，自由空间波数k越大，但超声波辐射功率E∝k 2，超声波频率越高波长越短，使得超声波穿透性越弱，传播途中能量损失也越大。因此为保证有效触控距离，超声波频率取值不宜过大，优选地，超声波频率取值范围为小于60KHz。另外，超声波发射器30发射的超声波具有方向性，无法以理想球面波的形式进行辐射，受超声波发射器30的方向角的限制，优选地超声波发射器30的方向角范围为60°-80°。在布置超声波发射器阵列220中的超声波发射器30时，需要结合阵列大小、方向角、触控区域大小综合考虑，例如，超声波发射器30取80°方向角时，在距离阵列面300mm处，可呈现与阵列面面积相当的触控区域。 In addition, the sound pressure distribution of ultrasonic waves in a limited space is negatively correlated with distance on the one hand, that is, the farther the distance, the weaker the sound pressure, so that the distance between the touch point and the array surface of the ultrasonic transmitter array 220 is limited; on the other hand It is related to the ultrasonic frequency f. According to the related theory of ultrasonic waves, the free space wave number of ultrasonic waves in the air medium
Figure PCTCN2022071859-appb-000001
That is, the higher the frequency f, the larger the free space wave number k, but the ultrasonic radiation power E∝k 2 , the higher the ultrasonic frequency, the shorter the wavelength, the weaker the ultrasonic penetration, and the greater the energy loss during propagation. Therefore, in order to ensure the effective touch distance, the value of the ultrasonic frequency should not be too large, and preferably, the value range of the ultrasonic frequency is less than 60KHz. In addition, the ultrasonic waves emitted by the ultrasonic transmitter 30 are directional and cannot be radiated in the form of ideal spherical waves, which are limited by the direction angle of the ultrasonic transmitter 30. Preferably, the direction angle of the ultrasonic transmitter 30 ranges from 60° to 80°. . When arranging the ultrasonic transmitters 30 in the ultrasonic transmitter array 220, it is necessary to comprehensively consider the size of the array, the direction angle, and the size of the touch area. A touch area equivalent to the area of the array surface can be presented.
此外，在空气介质中传播的超声波，其速度也因环境温度的影响而改变，在标准大气压下，温度为0℃时，声速为c 0＝331.45m/s，超声波实际传输速度为
Figure PCTCN2022071859-appb-000002
其中，T为环境温度。因此，本公开实施例可以设置温度传感器，通过数据处理模块230进行控制，以实时测量所处的环境温度，并根据实测温度值，对实际声速进行修正。从而，通过修正声速，可以减小***误差，提高控制信号延时精度，进而保证超声波聚焦精度，避免因温度陡变而引起的***功能异常。 In addition, the speed of the ultrasonic wave propagating in the air medium also changes due to the influence of the ambient temperature. Under the standard atmospheric pressure, when the temperature is 0 °C, the sound speed is c 0 =331.45m/s, and the actual transmission speed of the ultrasonic wave is
Figure PCTCN2022071859-appb-000002
where T is the ambient temperature. Therefore, in this embodiment of the present disclosure, a temperature sensor may be provided, and the data processing module 230 may be used for control to measure the ambient temperature in real time, and to correct the actual speed of sound according to the measured temperature value. Therefore, by correcting the speed of sound, the system error can be reduced, the delay accuracy of the control signal can be improved, and the ultrasonic focusing accuracy can be ensured, thereby avoiding abnormal system functions caused by abrupt changes in temperature.
因此，依据以上控制超声波发射器30发射超声波的原理，通过控制不同超声波发射器30的发射时间差可以使超声波在空中目标区域内任意点聚焦，对于多个触控点，按照一定的刷新频率更改各个超声波发射器30的发射时间差，即可实现多点触控交互需求。Therefore, according to the above principle of controlling the ultrasonic transmitter 30 to emit ultrasonic waves, the ultrasonic wave can be focused at any point in the air target area by controlling the emission time difference of different ultrasonic transmitters 30. For multiple touch points, each touch point is changed according to a certain refresh frequency. The difference in the transmission time of the ultrasonic transmitter 30 can meet the requirements of multi-touch interaction.
此外，对于多个触控点，可通过触觉反馈子***200高速刷新来实现多点触控反馈，并与成像子***100呈现的空中交互界面相结合，以引导用户进行触控操作。对于现实对象较大或 3D对象触控的应用场景，因其聚焦点数目增多，本公开实施例也可以采用多个触觉反馈子***200同步工作，以组合和拼接的方式，实现用户与复杂操作界面的人机交互，尤其对于3D显示对象的轮廓感知，提高用户体验。In addition, for multiple touch points, high-speed refresh of the haptic feedback subsystem 200 can be used to realize multi-touch feedback, which can be combined with the aerial interactive interface presented by the imaging subsystem 100 to guide the user to perform touch operations. For application scenarios where real objects are large or 3D objects are touched, due to the increase in the number of focus points, the embodiment of the present disclosure can also use multiple haptic feedback subsystems 200 to work synchronously, so as to realize user and complex operations by combining and splicing. The human-computer interaction of the interface, especially for the contour perception of 3D display objects, improves the user experience.
在一些实施例中，如图4所示，数据处理模块230包括第一存储单元11、数据处理单元12、第一接口单元13以及第一电源单元14。In some embodiments, as shown in FIG. 4 , the data processing module 230 includes a first storage unit 11 , a data processing unit 12 , a first interface unit 13 and a first power supply unit 14 .
其中，第一接口单元13包括第一串行接口15和第一并行接口16。第一串行接口15与成像子***100、数据处理单元12连接，以实现成像子***100与数据处理模块230之间的信息传输，例如，第一串行接口15可以为例如USB接口，可以实现触控点坐标数据及***控制指令的传输。第一并行接口16与驱动模块240、数据处理单元12连接，以实现驱动模块240与数据处理模块230之间的信息传输，例如，第一并行接口16可以采用由DSP(Digital Signal Processing，数字信号处理技术)集成的Upp并口，完成延时相位数据及反馈数据的传送。The first interface unit 13 includes a first serial interface 15 and a first parallel interface 16 . The first serial interface 15 is connected to the imaging subsystem 100 and the data processing unit 12 to realize information transmission between the imaging subsystem 100 and the data processing module 230. For example, the first serial interface 15 may be, for example, a USB interface, or Realize the transmission of touch point coordinate data and system control commands. The first parallel interface 16 is connected with the driving module 240 and the data processing unit 12 to realize the information transmission between the driving module 240 and the data processing module 230. processing technology) integrated Upp parallel port to complete the transmission of delayed phase data and feedback data.
第一电源单元14与第一存储单元11、数据处理单元12和第一接口单元13连接，第一电源单元14用于进行电源转换，将输入电源转换为数据处理模块230中各模块所需的各种稳定、可靠的电源，以为其供电，保证其正常工作。The first power supply unit 14 is connected to the first storage unit 11 , the data processing unit 12 and the first interface unit 13 . The first power supply unit 14 is used for power conversion, and converts the input power into the required power for each module in the data processing module 230 . Various stable and reliable power sources are used to supply power to it and ensure its normal operation.
第一存储单元11用于存储数据处理过程中的数据和程序。如图4所示，第一存储单元11分为数据存储单元和程序存储单元，数据存储单元用于存储数据处理过程中的数据，程序存储单元用于存储***引导程序以及加载程序。优选地，数据存储单元采用DDR2 SDRAM(Double-Data-Rate Two Synchronous Dynamic Random Access Memory，随机存取存储器)，程序存储单元采用NAND FLASH(闪存)。The first storage unit 11 is used to store data and programs during data processing. As shown in FIG. 4 , the first storage unit 11 is divided into a data storage unit and a program storage unit. The data storage unit is used to store data in the data processing process, and the program storage unit is used to store the system boot program and the loading program. Preferably, the data storage unit adopts DDR2 SDRAM (Double-Data-Rate Two Synchronous Dynamic Random Access Memory, random access memory), and the program storage unit adopts NAND FLASH (flash memory).
数据处理单元12与第一存储单元11连接，优选地，数据处理单元12可以采用浮点数DSP芯片作为主控，主频在200MHz以上，以保证***实时性要求。数据处理单元12接收到触控点坐标数据，根据触控点坐标信息和每个超声波发射器30的发射端的位置信息确定每个超声波发射器30的延时相位，并通过第一并行接口16将延时相位信息传输给驱动模块240，以使得驱动模块240根据每个超声波发射器30的延时相位对原始驱动信号进行延时处理，并基于延时处理后的驱动信号控制每个超声波发射器30，以使得每个超声波发射器30发射的超声波同时刻到达触控点并聚焦叠加，以在触控点产生振动，使得人体能够感知触控反馈。The data processing unit 12 is connected to the first storage unit 11. Preferably, the data processing unit 12 can use a floating-point DSP chip as the main control, and the main frequency is above 200MHz to ensure system real-time requirements. The data processing unit 12 receives the coordinate data of the touch point, determines the delay phase of each ultrasonic transmitter 30 according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter 30, and transmits the delay phase of each ultrasonic transmitter 30 through the first parallel interface 16. The delayed phase information is transmitted to the driving module 240, so that the driving module 240 performs delay processing on the original driving signal according to the delayed phase of each ultrasonic transmitter 30, and controls each ultrasonic transmitter based on the delayed processed driving signal 30 , so that the ultrasonic waves emitted by each ultrasonic transmitter 30 reach the touch point at the same time and focus and overlap, so as to generate vibration at the touch point, so that the human body can perceive the touch feedback.
在一些实施例中，如图5所示，驱动模块240包括第二存储单元17、控制单元18、驱动单元19、第二电源单元20以及第二接口单元21。In some embodiments, as shown in FIG. 5 , the driving module 240 includes a second storage unit 17 , a control unit 18 , a driving unit 19 , a second power supply unit 20 and a second interface unit 21 .
其中，第二接口单元21包括电源输入口22、第二并行接口23和驱动信号输出口24。其中，电源输入口22与第二电源单元20连接，第二并行接口23与控制单元18、第一并行接口16连接，驱动信号输出口24与驱动单元19连接。第二并行接口23如Upp并口与控制单元18进行数据通信，实现延时相位数据、控制指令以及反馈数据的传输。The second interface unit 21 includes a power input port 22 , a second parallel interface 23 and a drive signal output port 24 . The power input port 22 is connected to the second power supply unit 20 , the second parallel interface 23 is connected to the control unit 18 and the first parallel interface 16 , and the drive signal output port 24 is connected to the drive unit 19 . The second parallel interface 23, such as the Upp parallel port, performs data communication with the control unit 18 to realize the transmission of delayed phase data, control commands and feedback data.
第二存储单元17用于存储驱动控制过程中的数据和文件，例如，用于存储控制单元18的处理数据以及信号调制过程中的数据。第二存储单元17可以选用SRAM(Static Random-Access Memory，静态随机存取存储器)，同时选用专用FLASH(固态存储器)芯片，以用于存储和加载控制单元18的逻辑目标文件。The second storage unit 17 is used for storing data and files in the driving control process, for example, for storing the processing data of the control unit 18 and the data in the signal modulation process. The second storage unit 17 can select SRAM (Static Random-Access Memory, static random access memory), and select a dedicated FLASH (solid-state memory) chip at the same time for storing and loading the logic target file of the control unit 18.
控制单元18与第二存储单元17连接，用于根据触控反馈控制指令产生时钟信号和调制信号，并根据时钟信号和调制信号生成原始驱动信号，并根据每个超声波发射器30的延时相位对原始驱动信号进行延时处理。优选地，控制单元18可以选用FPGA(Field Programmable Gate Array，现场可编程逻辑门阵列)作为主控芯片，以通过配置的I/O口直接同步输出驱动信号至驱动单元19，完成对各超声波发射器30发射超声波的控制，同时FPGA主控芯片管脚资源丰富且管脚可重定义配置，能够满足***设计需求。The control unit 18 is connected to the second storage unit 17, and is used for generating a clock signal and a modulation signal according to the touch feedback control instruction, and generating an original driving signal according to the clock signal and the modulation signal, and according to the delay phase of each ultrasonic transmitter 30 Delay processing of the original drive signal. Preferably, the control unit 18 can select an FPGA (Field Programmable Gate Array, Field Programmable Gate Array) as the main control chip, so as to directly and synchronously output the drive signal to the drive unit 19 through the configured I/O port to complete the transmission of each ultrasonic wave. The controller 30 transmits ultrasonic waves, and at the same time, the FPGA main control chip has abundant pin resources and the pins can be redefined and configured, which can meet the system design requirements.
驱动单元19与控制单元18连接，用于对延时后的原始驱动信号进行放大处理以获得每个超声波发射器30的目标驱动信号，并根据目标驱动信号驱动每个超声波发射器30。The driving unit 19 is connected to the control unit 18 for amplifying the delayed original driving signal to obtain a target driving signal for each ultrasonic transmitter 30 , and driving each ultrasonic transmitter 30 according to the target driving signal.
第二电源单元20与第二存储单元17、控制单元18和驱动单元19连接。第二电源单元20用于为整个***提供稳定可靠的电源，通过电源输入口22输入外部电源，并经DC-DC转换以及线性转换后，为驱动模块240、控制单元18、超声波发射器阵列220提供所需的电源。The second power supply unit 20 is connected to the second storage unit 17 , the control unit 18 and the driving unit 19 . The second power supply unit 20 is used to provide a stable and reliable power supply for the entire system. An external power supply is input through the power input port 22, and after DC-DC conversion and linear conversion, it is the driving module 240, the control unit 18, and the ultrasonic transmitter array 220. Provide the required power.
在本公开实施例中，由于控制单元18的尺寸相较于驱动单元19的尺寸较小，因此可以通过高速板间连接器将控制单元18固定于驱动单元19上。驱动单元19的尺寸与超声波发射器阵列220的尺寸相当，两者之间可以通过板间连接器实现互联。将超声波发射器阵列220、控制 单元18和驱动单元19进行组装，并将组装后的整体封装于第一壳体210内，提高***的集成性和美观性。In the embodiment of the present disclosure, since the size of the control unit 18 is smaller than that of the driving unit 19 , the control unit 18 can be fixed on the driving unit 19 through a high-speed inter-board connector. The size of the driving unit 19 is equivalent to the size of the ultrasonic transmitter array 220, and the two can be interconnected through an inter-board connector. The ultrasonic transmitter array 220, the control unit 18 and the drive unit 19 are assembled, and the assembled whole is packaged in the first housing 210, so as to improve the integration and aesthetics of the system.
在一些实施例中，如图2所示，在第一容纳腔2的第一侧壁上设置有第一散热孔3，在第二容纳腔2的第二侧壁上设置有第二散热孔4，第一侧壁与第二侧壁相对。通过在第一壳体210两侧分别开通散热孔，可以为***提供散热效果。In some embodiments, as shown in FIG. 2 , first heat dissipation holes 3 are provided on the first side wall of the first accommodating cavity 2 , and second heat dissipation holes 3 are provided on the second side wall of the second accommodating cavity 2 4. The first side wall is opposite to the second side wall. By opening heat dissipation holes on both sides of the first casing 210 respectively, a heat dissipation effect can be provided for the system.
在一些实施例中，触觉反馈子***200还包括吸风扇和排风扇。具体地，如图2所示，吸风扇设置在第一侧壁上，用于通过第一散热孔3吸取外部空气至第一容纳腔2。排风扇设置在第二侧壁上，用于通过第二散热孔4将第一容纳腔2内空气排出。基于在第一壳体210两侧分别开通散热孔，并结合吸风扇和排风扇，进一步提高对***的散热效果。In some embodiments, the haptic feedback subsystem 200 also includes a suction fan and an exhaust fan. Specifically, as shown in FIG. 2 , the suction fan is arranged on the first side wall, and is used for sucking external air to the first accommodating cavity 2 through the first heat dissipation hole 3 . The exhaust fan is arranged on the second side wall, and is used to exhaust the air in the first accommodating cavity 2 through the second heat dissipation holes 4 . Based on opening the heat dissipation holes on both sides of the first casing 210, and combining the suction fan and the exhaust fan, the heat dissipation effect on the system is further improved.
在一些实施例中，如图2所示，触觉反馈子***200还包括挡盖5，设置在第一壳体210上，用于遮挡开窗，防止超声波发射器30的发射端直接暴露于空气中，并起到保护作用，同时提高产品的美观性。In some embodiments, as shown in FIG. 2 , the haptic feedback subsystem 200 further includes a blocking cover 5 , which is disposed on the first housing 210 and is used to block the opening window and prevent the transmitting end of the ultrasonic transmitter 30 from being directly exposed to the air. and play a protective role, while improving the aesthetics of the product.
在一些实施例中，在不阻碍超声波正常辐射的前提下，对挡盖5的设置材料不作限制，例如挡盖5可以为网状保护罩或透波材料挡板。In some embodiments, on the premise that the normal radiation of ultrasonic waves is not hindered, the setting material of the blocking cover 5 is not limited, for example, the blocking cover 5 can be a mesh protective cover or a wave-transmitting material baffle.
此外，本公开实施例中数据处理模块230、驱动模块240和超声波发射器阵列220之间均通过微小型板间连接器进行装配，且触觉反馈子***200整体采用模块化、小型化设计，使得在***组装后可以降低整体高度、缩小体积，减少占用空间，提高***的美观性和集成性。In addition, in the embodiment of the present disclosure, the data processing module 230 , the driving module 240 and the ultrasonic transmitter array 220 are all assembled through micro-miniature board connectors, and the haptic feedback subsystem 200 adopts a modular and miniaturized design as a whole, so that the After the system is assembled, the overall height and volume can be reduced, the occupied space can be reduced, and the aesthetics and integration of the system can be improved.
在一些实施例中，如图6所示，成像子***100包括第二壳体110、成像组件120、检测模块130以及主控模块140。In some embodiments, as shown in FIG. 6 , the imaging subsystem 100 includes a second housing 110 , an imaging assembly 120 , a detection module 130 and a main control module 140 .
其中，第二壳体110形成有显示窗口40且在内部形成有第二容纳腔50。Wherein, the second casing 110 is formed with a display window 40 and a second accommodating cavity 50 is formed inside.
成像组件120设置于第二容纳腔50中，用于将人机交互信息在空中目标区域10成像显示以形成空中交互界面。具体地，成像组件120在空中的确定位置处形成浮空实像即空中交互界面，覆盖浮空实像所在的三维空间即为空中目标区域10，也就是，成像组件120无需借助实体介质即可在空中目标区域10呈现浮空实像，满足用户在视觉上的交互体验，以及成像组件120也无需设置额外的限制机构来引导用户进行操作，减少了用户与设备本体接触的风险。The imaging component 120 is disposed in the second accommodating cavity 50, and is used for imaging and displaying the human-machine interaction information in the aerial target area 10 to form an aerial interaction interface. Specifically, the imaging assembly 120 forms a floating real image at a certain position in the air, that is, an aerial interactive interface, and the three-dimensional space covering the floating real image is the aerial target area 10. That is, the imaging assembly 120 can operate in the air without using a physical medium. The target area 10 presents a floating real image, which satisfies the user's visual interactive experience, and the imaging component 120 also does not need to set an additional restriction mechanism to guide the user to operate, thereby reducing the risk of the user contacting the device body.
检测模块130用于检测用户与空中交互界面的交互信号。具体地，如图7所示，检测模块130的感应区域与空中交互界面位于同一平面且包含空中交互界面所处三维空间。通过检测模块130实时检测空中目标区域10处用户对空中交互界面的交互操作，以在检测到用户与空中交互界面的交互信号时，将检测的交互信号反馈至主控模块140，主控模块140响应于交互信号发送触控反馈控制指令和触控点坐标信息至触觉反馈子***200，以触发触觉反馈子***200反馈用户对触摸对象的感知。因此，用户通过与空中交互界面直接交互，来触发触觉反馈子***200进行触觉反馈的操作方式，更加自然舒适。The detection module 130 is used to detect the interaction signal between the user and the air interface. Specifically, as shown in FIG. 7 , the sensing area of the detection module 130 is located on the same plane as the air interaction interface and includes the three-dimensional space where the air interaction interface is located. The detection module 130 detects the user's interactive operation on the aerial interactive interface in the aerial target area 10 in real time, so that when the interactive signal between the user and the aerial interactive interface is detected, the detected interactive signal is fed back to the main control module 140. The main control module 140 In response to the interaction signal, the touch feedback control instruction and the touch point coordinate information are sent to the haptic feedback subsystem 200 to trigger the haptic feedback subsystem 200 to feedback the user's perception of the touch object. Therefore, the user triggers the operation mode of the haptic feedback subsystem 200 to perform haptic feedback by directly interacting with the air interaction interface, which is more natural and comfortable.
在实施例中，检测模块130可以为光学传感器，其感应形式可以包括但不限于远近红外、超声波、激光干涉、光栅、编码器、光纤式或CCD(Charge-coupled Device，电荷耦合器件)等。In an embodiment, the detection module 130 may be an optical sensor, and its sensing form may include, but is not limited to, far and near infrared, ultrasonic wave, laser interference, grating, encoder, optical fiber type or CCD (Charge-coupled Device, charge-coupled device) and the like.
在实际应用时，检测模块130可以根据安装空间、观看角度和使用环境选择最佳的感应形式，从而方便用户以最佳的姿态在空中目标区域10进行操作，提高用户体验。In practical application, the detection module 130 can select the best sensing form according to the installation space, viewing angle and use environment, so as to facilitate the user to operate in the air target area 10 with the best posture and improve the user experience.
主控模块140与成像组件120和检测模块130连接，用于响应于交互信号发送触控反馈控制指令和触控点坐标信息。其中，主控模块140与检测模块130之间可以采用有线或无线方式连接，以传输数字或模拟信号，从而可以灵活控制整体装置的体积，且可以增强触控反馈***1000的电气稳定性。The main control module 140 is connected to the imaging component 120 and the detection module 130, and is used for sending touch feedback control instructions and touch point coordinate information in response to the interactive signal. The main control module 140 and the detection module 130 can be connected in a wired or wireless manner to transmit digital or analog signals, so that the volume of the overall device can be flexibly controlled, and the electrical stability of the touch feedback system 1000 can be enhanced.
在一些实施例中，如图6所示，成像组件120包括显示器25和光学组件26。In some embodiments, as shown in FIG. 6 , imaging assembly 120 includes display 25 and optical assembly 26 .
具体地，显示器25设置于第二容纳腔50中，与主控模块140连接，用于显示人机交互信息。光学组件26设置于第二容纳腔50中，用于将携带人机交互信息的光线汇聚成像(例如折射)在空中目标区域10，以形成空中交互界面。Specifically, the display 25 is disposed in the second accommodating cavity 50 and is connected to the main control module 140 for displaying human-computer interaction information. The optical component 26 is disposed in the second accommodating cavity 50, and is used for converging and imaging (eg, refracting) the light carrying the human-computer interaction information on the air target area 10, so as to form an air interaction interface.
其中，如图6所示，显示器25设置于光学组件26的光源侧，显示窗口40在光学组件26的成像侧。通过主控模块140控制显示器25显示人体交互信息，显示器25显示的人体交互信息的光线通过光学组件26成像显示在空中目标区域10，形成空中交互界面，以用于引导用户触控。Wherein, as shown in FIG. 6 , the display 25 is disposed on the light source side of the optical assembly 26 , and the display window 40 is disposed on the imaging side of the optical assembly 26 . The main control module 140 controls the display 25 to display human interaction information, and the light of the human interaction information displayed on the display 25 is imaged and displayed in the aerial target area 10 through the optical component 26 to form an aerial interaction interface for guiding the user to touch.
在实施例中，显示器25的成像模式可以包括RGB(红色、绿色、蓝色)发光二极管(Light  Emitting Diode，LED)、LCD(Liquid Crystal Display，液晶显示器)、LCOS(Liquid Crystal on Silicon，液晶附硅)器件、OLED(Organic Light-Emitting Diode，有机发光二极管)阵列、投影、激光、激光二极管或任何其他合适的显示器或立体显示器，对此不作限制。显示器25可以提供清晰、明亮且高对比度的动态图像光源，主控模块140控制显示器25显示人体交互信息，并经光学组件26折射，可以在空中目标区域10位置呈现出清晰的空中交互界面，从而便于用户操作。In an embodiment, the imaging mode of the display 25 may include RGB (red, green, blue) light emitting diodes (Light Emitting Diode, LED), LCD (Liquid Crystal Display, liquid crystal display), LCOS (Liquid Crystal on Silicon, liquid crystal attached Silicon) devices, OLED (Organic Light-Emitting Diode, organic light-emitting diode) arrays, projections, lasers, laser diodes or any other suitable displays or stereoscopic displays, which are not limited. The display 25 can provide a clear, bright and high-contrast dynamic image light source. The main control module 140 controls the display 25 to display the human interaction information, and is refracted by the optical component 26 to present a clear aerial interactive interface at the position of the aerial target area 10, thereby User-friendly operation.
在实施例中，可以设置显示器25的亮度不低于500cd/m 2，以降低光路传播中因亮度损失造成的影响。当然，在实际应用时，也可以根据环境光的亮暗来调整显示器25的显示亮度。 In an embodiment, the brightness of the display 25 may be set to be not lower than 500 cd/m 2 to reduce the influence of brightness loss in the propagation of the light path. Of course, in practical application, the display brightness of the display 25 can also be adjusted according to the brightness of the ambient light.
在实施例中，可以对显示器25的显示图像表面进行可视角控制处理，以减轻空中目标区域10的残影，提高画面质量，同时也可以防止他人窥视，便于广泛应用于其他需要隐私信息保护的输入装置。In the embodiment, the viewing angle control processing can be performed on the display image surface of the display 25, so as to reduce the afterimage of the aerial target area 10, improve the picture quality, and at the same time, it can also prevent others from peeping, which is convenient for being widely used in other applications that require privacy information protection. input device.
下面对本公开实施例的光学组件26的结构及其实现成像的原理进行说明。The structure of the optical assembly 26 in the embodiment of the present disclosure and the principle of realizing imaging will be described below.
在一些实施例中，光学组件26可以采用平板透镜，平板透镜固定于第二壳体110上，如图8所示，平板透镜包括两个透明基板8，以及置于两个透明基板8之间的第一光波导阵列6和第二光波导阵列7。其中，第一光波导阵列6和第二光波导阵列7在同一平面紧密贴合且正交布置。优选地，第一光波导阵列6和第二光波导阵列7的厚度相同，便于设计和生产。In some embodiments, the optical assembly 26 may adopt a flat lens, and the flat lens is fixed on the second housing 110 . As shown in FIG. 8 , the flat lens includes two transparent substrates 8 and is placed between the two transparent substrates 8 The first optical waveguide array 6 and the second optical waveguide array 7. The first optical waveguide array 6 and the second optical waveguide array 7 are closely attached on the same plane and are arranged orthogonally. Preferably, the thicknesses of the first optical waveguide array 6 and the second optical waveguide array 7 are the same, which is convenient for design and production.
具体地，如图8所示，平板透镜从显示器25一侧到空中目标区域10一侧依次包括第一透明基板8、第一光波导阵列6、第二光波导阵列7和第二玻璃基板8。Specifically, as shown in FIG. 8 , the flat lens sequentially includes a first transparent substrate 8 , a first optical waveguide array 6 , a second optical waveguide array 7 and a second glass substrate 8 from the display 25 side to the air target area 10 side. .
其中，第一透明基板8和第二透明基板8均具有两个光学面，透明基板8对波长在390nm至760nm之间的光线具有90％-100％的透射率。透明基板8的材料可以为玻璃、塑料、聚合物和丙烯酸树脂中的至少一个，用于保护光波导阵列及滤除多余光线。需要说明的是，如果第一光波导阵列6和第二光波导阵列7紧密正交贴合后的强度足够，或者安装环境有厚度限制，则也可以只配置一个透明基板8或完全不配置透明基板8。The first transparent substrate 8 and the second transparent substrate 8 both have two optical surfaces, and the transparent substrate 8 has a transmittance of 90%-100% for light with wavelengths between 390 nm and 760 nm. The material of the transparent substrate 8 can be at least one of glass, plastic, polymer and acrylic resin, for protecting the optical waveguide array and filtering out excess light. It should be noted that, if the strength of the first optical waveguide array 6 and the second optical waveguide array 7 after close and orthogonal bonding is sufficient, or the installation environment has a thickness limit, only one transparent substrate 8 may be configured or no transparent substrate is configured at all. Substrate 8.
光学组件26实现空中成像的原理为，第一光波导阵列6和第二光波导阵列7由多个横截面为矩形的反射单元9组成，各反射单元9的长度由光波导阵列***尺寸限制从而长短不一。如图9所示，第一光波导阵列6中反射单元9的延伸方向为X，第二光波导阵列7的反射单元9的延伸方向为Y，Z方向为光波导阵列的厚度方向。第一光波导阵列6和第二光波导阵列7中反射单元9的延伸方向(光波导阵列方向)相互垂直，即从Z方向(厚度方向)看，第一光波导阵列6和第二光波导阵列7之间正交布置，从而使处于正交方向的两个光束会聚于一点，且保证物像面(光源侧和成像侧)相对于平板透镜对称，产生等效负折射现象，实现空中成像。The principle of the optical assembly 26 to achieve aerial imaging is that the first optical waveguide array 6 and the second optical waveguide array 7 are composed of a plurality of reflection units 9 with rectangular cross-sections, and the length of each reflection unit 9 is limited by the peripheral size of the optical waveguide array. Different lengths. As shown in FIG. 9 , the extension direction of the reflection unit 9 in the first optical waveguide array 6 is X, the extension direction of the reflection unit 9 of the second optical waveguide array 7 is Y, and the Z direction is the thickness direction of the optical waveguide array. The extension directions (optical waveguide array directions) of the reflection units 9 in the first optical waveguide array 6 and the second optical waveguide array 7 are perpendicular to each other, that is, viewed from the Z direction (thickness direction), the first optical waveguide array 6 and the second optical waveguide The arrays 7 are arranged orthogonally, so that the two light beams in the orthogonal direction converge at one point, and the object image plane (the light source side and the imaging side) is guaranteed to be symmetrical with respect to the flat lens, resulting in an equivalent negative refraction phenomenon, realizing aerial imaging .
在一些实施例中，如图10所示，第一光波导阵列6或第二光波导阵列7由以用户视角偏转45°斜向布置的多个平行排布的反射单元9组成。具体地，第一光波导阵列6可由呈左下方向45°并排且横截面为矩形的反射单元9组成，第二光波导阵列7可由呈右下方向45°并排且横截面为矩形的反射单元9组成，两组光波导阵列中反射单元9的排列方向可以互换。例如，第一光波导阵列6中反射单元9的延伸方向为Y，第二光波导阵列7的反射单元9的延伸方向为X，Z方向为光波导阵列的厚度方向，从Z方向(厚度方向)看，第一光波导阵列6和第二光波导阵列7之间正交布置，使处于正交方向的两个光束会聚于一点，且保证物像面(光源侧和成像侧)相对于平板透镜对称，产生等效负折射现象，实现空中成像。其中，光波导材料具有光学折射率n1，在一些实施例中，n1>1.4，例如n1取值为1.5、1.8、2.0等。In some embodiments, as shown in FIG. 10 , the first optical waveguide array 6 or the second optical waveguide array 7 is composed of a plurality of parallel-arranged reflective units 9 that are obliquely arranged with a user viewing angle deflection of 45°. Specifically, the first optical waveguide array 6 can be composed of reflective units 9 that are aligned at 45° in the lower left direction and have a rectangular cross section, and the second optical waveguide array 7 can be composed of reflective units 9 that are aligned at 45° in the lower right direction and have a rectangular cross section. composition, the arrangement directions of the reflection units 9 in the two groups of optical waveguide arrays can be interchanged. For example, the extension direction of the reflection unit 9 in the first optical waveguide array 6 is Y, the extension direction of the reflection unit 9 in the second optical waveguide array 7 is X, and the Z direction is the thickness direction of the optical waveguide array. ), the first optical waveguide array 6 and the second optical waveguide array 7 are arranged orthogonally, so that the two light beams in the orthogonal direction converge at one point, and ensure that the object image plane (light source side and imaging side) is relative to the flat plate The lens is symmetrical, which produces an equivalent negative refraction phenomenon and realizes aerial imaging. The optical waveguide material has an optical refractive index n1, and in some embodiments, n1>1.4, for example, the value of n1 is 1.5, 1.8, 2.0, or the like.
如图11所示，对于第一光波导阵列6和第二光波导阵列7，各反射单元9与其相邻的反射单元9之间存在两个交接面，各交接面之间由透光性较好的胶粘剂28接合。优选地，胶粘剂28可以选择光敏胶或热固胶，胶粘剂28的厚度为T1，且满足T1>0.001mm，例如，T1＝0.002mm或者T1＝0.003mm或者T1＝0.0015mm，具体厚度可以依据具体需要设置。平板透镜中相邻的光波导阵列之间以及光波导阵列与透明基板8之间均设置有胶粘剂28，增加牢固性。As shown in FIG. 11 , for the first optical waveguide array 6 and the second optical waveguide array 7 , there are two interface surfaces between each reflection unit 9 and its adjacent reflection unit 9 , and the interface surfaces are separated by a relatively high light transmittance. Good adhesive 28 joins. Preferably, the adhesive 28 can be selected from photosensitive adhesive or thermosetting adhesive, and the thickness of the adhesive 28 is T1 and satisfies T1>0.001mm, for example, T1=0.002mm or T1=0.003mm or T1=0.0015mm, and the specific thickness can be determined according to the specific thickness. Setup is required. Adhesives 28 are provided between adjacent optical waveguide arrays in the flat lens and between the optical waveguide arrays and the transparent substrate 8 to increase firmness.
在一些实施例中，反射单元9的横截面可以为矩形，且沿反射单元9的排布方向的一侧或两侧面设置有反射膜27。具体地，如图11所示，在光波导阵列排布方向上，各反射单元9两侧均镀有反射膜27，该反射膜27的材料可以为实现全反射的铝、银等金属材料或其他非金属化合物材料。反射膜27的作用是防止光线因没有全反射而进入相邻光波导阵列中形成杂光影响成像。或者，各反射单元9也可以在反射膜27上添加介质膜，介质膜的作用是提高光反射率。In some embodiments, the cross section of the reflection unit 9 may be rectangular, and a reflection film 27 is provided on one or both sides along the arrangement direction of the reflection unit 9 . Specifically, as shown in FIG. 11 , in the arrangement direction of the optical waveguide array, both sides of each reflection unit 9 are coated with a reflection film 27 , and the material of the reflection film 27 can be a metal material such as aluminum, silver, etc. that realizes total reflection or Other non-metallic compound materials. The function of the reflective film 27 is to prevent light from entering into adjacent optical waveguide arrays due to lack of total reflection to form stray light from affecting imaging. Alternatively, each reflective unit 9 may also add a dielectric film on the reflective film 27, and the function of the dielectric film is to improve the light reflectivity.
单个反射单元9的横截面宽a和横截面长b，满足0.1mm≤a≤5mm，0.1mm≤b≤5mm，例 如a＝2mm，b＝4mm；或者，a＝3mm，b＝5mm。在大屏幕显示时可以通过拼接多块光波导阵列来实现大尺寸需求。光波导阵列的整体形状根据应用场景需要设置，本实施例中，两组光波导阵列整体呈矩形结构，两对角的反射单元9为三角形，中间的反射单元9为梯形结构。单个反射单元9的长度不等，位于矩形对角线的反射单元9长度最长，两端的反射单元9长度最短。The cross-sectional width a and cross-sectional length b of a single reflection unit 9 satisfy 0.1mm≤a≤5mm, 0.1mm≤b≤5mm, for example, a=2mm, b=4mm; or, a=3mm, b=5mm. Large-scale requirements can be achieved by splicing multiple optical waveguide arrays when displaying on a large screen. The overall shape of the optical waveguide array is set according to the application scenario. In this embodiment, the two groups of optical waveguide arrays have a rectangular structure as a whole, the two diagonal reflection units 9 are triangular, and the middle reflection unit 9 is a trapezoidal structure. The lengths of the individual reflection units 9 are not equal, the reflection units 9 located on the diagonal of the rectangle have the longest length, and the reflection units 9 at both ends have the shortest length.
此外，平板透镜还可以包括增透部件和视角控制部件，增透部件可以提高平板透镜的整体透过率，提高成像于空中目标区域10的空中交互界面的清晰度和明亮度。视角控制部件可以用于消除成像于空中目标区域10的空中交互界面的残像，降低观察者的眩晕感，同时防止观察者从其他角度窥视到装置内部，提升装置整体的美观度。其中，增透部件和视角控制部件可以组合，或者也可以分别独立设置在透明基板8与波导阵列的之间、两层波导阵列之间或透明基板8的外层。In addition, the flat lens may further include an anti-reflection component and a viewing angle control component, and the anti-reflection component can improve the overall transmittance of the flat lens and improve the clarity and brightness of the aerial interactive interface imaged in the aerial target area 10 . The viewing angle control component can be used to eliminate the afterimage of the aerial interactive interface imaged in the aerial target area 10, reduce the dizziness of the observer, prevent the observer from peeping into the device from other angles, and improve the overall aesthetics of the device. The anti-reflection component and the viewing angle control component may be combined, or may be independently disposed between the transparent substrate 8 and the waveguide array, between two layers of the waveguide array, or on the outer layer of the transparent substrate 8 .
下面参考图8对平板透镜的成像原理进行说明，具体内容如下。The imaging principle of the flat lens is described below with reference to FIG. 8 , and the specific content is as follows.
在微米尺度上，使用相互正交的双层波导阵列结构，来对任意光信号进行正交分解。原始信号投射在第一光波导阵列6，以原始信号投射点作为原点、垂直于第一光波导阵列6为x轴建立直角坐标系，在该直角坐标系内原始信号被分解为位于x轴的信号X和位于y轴的信号Y两路相互正交信号。其中，信号X在经过第一光波导阵列6时，按照与入射角相同的反射角在反射膜27表面进行全反射；此时，信号Y保持平行于第一光波导阵列6，穿过第一光波导阵列6后，在第二光波导阵列7表面按照与入射角相同的反射角在反射膜27表面进行全反射，反射后的信号Y与信号X组成的反射后的光信号便与原始光信号成镜面对称。因此任意方向的光线经过此平板透镜均可实现镜面对称，任意光源的发散光经过此平板透镜便会在对称位置重新汇聚成浮空实像，即在空中目标区域10处成像空中交互界面，浮空实像的成像距离与平板透镜到像源即显示器25的距离相同，为等距离成像，且浮空实像的位置在空中，不需要具体载体，而是直接在空气中呈现实像。因此，使用者所看到的空间中的影像即是显示器25发出的图像。On the micrometer scale, the orthogonal decomposition of any optical signal is performed using mutually orthogonal double-layer waveguide array structures. The original signal is projected on the first optical waveguide array 6, and a rectangular coordinate system is established with the original signal projection point as the origin and perpendicular to the first optical waveguide array 6 as the x-axis, in which the original signal is decomposed into x-axis. The signal X and the signal Y located on the y-axis are mutually orthogonal signals. Among them, when the signal X passes through the first optical waveguide array 6, it is totally reflected on the surface of the reflective film 27 at the same reflection angle as the incident angle; at this time, the signal Y remains parallel to the first optical waveguide array 6 and passes through the first optical waveguide array 6. After the optical waveguide array 6, the surface of the second optical waveguide array 7 performs total reflection on the surface of the reflective film 27 at the same reflection angle as the incident angle, and the reflected optical signal composed of the reflected signal Y and the signal X is the same as the original light. The signal is mirror-symmetrical. Therefore, the light in any direction can achieve mirror symmetry through the flat lens, and the divergent light of any light source will be re-converged into a floating real image at a symmetrical position through the flat lens, that is, the air interactive interface is imaged at the air target area 10, floating in the air. The imaging distance of the real image is the same as the distance from the flat lens to the image source, that is, the display 25, which is equidistant imaging, and the position of the floating real image is in the air, no specific carrier is needed, but the real image is directly presented in the air. Therefore, the image in the space seen by the user is the image emitted by the display 25 .
在本公开实施例中，显示器25光源发出的光线在穿过平板透镜时，在平板透镜上发生上述过程。具体地，如图12所示，光线在第一光波导阵列6上的入射角分别为α 1、α 2和α 3，光线在第一光波导阵列6上的反射角为β 1、β 2和β 3，其中α 1＝β 1，α 2＝β 2，α 3＝β 3，经过第一光波导阵列6反射后，在第二光波导阵列7上的入射角分别为γ 1、γ 2和γ 3，在第二光波导阵列7上的反射角分别为δ 1、δ 2和δ 3，其中，γ 1＝δ 1，γ 2＝δ 2，γ 3＝δ 3。 In the embodiment of the present disclosure, when the light emitted by the light source of the display 25 passes through the flat lens, the above process occurs on the flat lens. Specifically, as shown in FIG. 12 , the incident angles of the light on the first optical waveguide array 6 are α 1 , α 2 and α 3 respectively, and the reflection angles of the light on the first optical waveguide array 6 are β 1 , β 2 and β 3 , where α 1 =β 1 , α 2 =β 2 , α 3 =β 3 , after being reflected by the first optical waveguide array 6, the incident angles on the second optical waveguide array 7 are γ 1 , γ respectively 2 and γ 3 , the reflection angles on the second optical waveguide array 7 are δ 1 , δ 2 and δ 3 , respectively, where γ 1 =δ 1 , γ 2 =δ 2 , and γ 3 =δ 3 .
进一步地，汇聚成像后的入射角分别为α 1，α 2，α 3…..α n，显示器25的光源与平板透镜的距离为L，则浮空实像的成像位置与平板透镜的距离也为L，且该浮空实像的可视角度ε为2倍max(α)。 Further, the incident angles after convergent imaging are α 1 , α 2 , α 3 ...... is L, and the viewing angle ε of the floating real image is 2 times max(α).
可以理解的是，若光波导阵列的尺寸较小，则仅在距离光波导阵列成像侧的一定距离才可看到影像；而若光波导阵列的尺寸变大，即可实现更大的成像距离，从而增大视野率。It can be understood that if the size of the optical waveguide array is small, the image can only be seen at a certain distance from the imaging side of the optical waveguide array; and if the size of the optical waveguide array becomes larger, a larger imaging distance can be achieved. , thereby increasing the field of view.
优选地，平板透镜与显示器25的夹角设置为45°±5°的范围，从而可以有效利用平板透镜的尺寸，提高成像质量和降低残像影响。此外，如果对成像位置有其他需求，则也可以在牺牲部分成像质量的情况下选择其他角度，优选地，平板透镜的大小设置为可以显示整个显示器25所呈现的浮空实像的画面。但如果实际使用时仅需要看到显示器25的部分画面，则平板透镜的尺寸也可以根据实际显示画面自由调整大小和位置，对此不作限制。Preferably, the angle between the flat lens and the display 25 is set to be in the range of 45°±5°, so that the size of the flat lens can be effectively utilized, the image quality can be improved and the effect of afterimages can be reduced. In addition, if there are other requirements for the imaging position, other angles can also be selected while sacrificing part of the imaging quality. However, if only a part of the screen of the display 25 needs to be seen in actual use, the size and position of the flat lens can also be freely adjusted according to the actual display screen, which is not limited.
另外，以上主要表述采用双层光波导阵列结构的平板透镜的成像原理，在另一些实施例中，若将四周面均设为附有反射膜27的多个立方柱状反射单元9，且多个立方柱状反射单元9均在一层光波导阵列结构中沿X和Y方向呈阵列排布，即将两层光波导阵列合并成一层，其成像原理与双层光波导阵列结构的成像原理相同，也可以作为平板透镜的结构。In addition, the above mainly describes the imaging principle of a flat lens with a double-layer optical waveguide array structure. The cubic-columnar reflection units 9 are all arranged in an array along the X and Y directions in the one-layer optical waveguide array structure, that is, the two-layer optical waveguide arrays are combined into one layer. The imaging principle is the same as that of the double-layer optical waveguide array structure. It can be used as the structure of flat lens.
在实施例中，第一光波导阵列6与第二光波导阵列7的厚度相同，从而可以简化第一光波导阵列6与第二光波导阵列7结构的复杂度，降低第一光波导阵列6与第二光波导阵列7的制造难度，提升第一光波导阵列6与第二光波导阵列7的生产效率，减少第一光波导阵列6与第二光波导阵列7的生产成本。需要注意的是，此处的厚度相同为一个相对的范围，并非是绝对相同，即以提高生产效率为目的，在不影响空中成像质量的前提下，光波导阵列之间可以存在一定的厚度差。In the embodiment, the thicknesses of the first optical waveguide array 6 and the second optical waveguide array 7 are the same, so that the structural complexity of the first optical waveguide array 6 and the second optical waveguide array 7 can be simplified, and the first optical waveguide array 6 can be reduced. The manufacturing difficulty of the second optical waveguide array 7 improves the production efficiency of the first optical waveguide array 6 and the second optical waveguide array 7 and reduces the production cost of the first optical waveguide array 6 and the second optical waveguide array 7 . It should be noted that the same thickness here is a relative range, not absolutely the same, that is, for the purpose of improving production efficiency, on the premise of not affecting the quality of aerial imaging, there may be a certain thickness difference between the optical waveguide arrays. .
在一些实施例中，第二容纳腔50的内壁上设置有吸光层。也就是，在第二壳体110内除显示器25显示面以外的部分均做黑色吸光处理，如喷涂吸光涂料或张贴吸光膜，以用于消除第二壳体110内部构件对光线的漫反射，提高空中交互界面的显示效果。In some embodiments, a light absorbing layer is provided on the inner wall of the second accommodating cavity 50 . That is, the parts other than the display surface of the display 25 in the second housing 110 are treated with black light absorption, such as spraying light-absorbing paint or pasting a light-absorbing film, so as to eliminate the diffuse reflection of the light by the internal components of the second housing 110, Improve the display effect of the air interface.
本公开实施例中，超声波发射器阵列220可以采用穿透性较强的超声波发射器30，因此，对于触觉反馈子***200尤其是超声波发射器阵列220的布置方式，可以根据实际应用场合以及***的需求进行灵活布置，对此不作限制。以下具体说明本公开实施例优选的几种布置方案。In the embodiment of the present disclosure, the ultrasonic transmitter array 220 can adopt the ultrasonic transmitter 30 with strong penetrability. Therefore, the arrangement of the haptic feedback subsystem 200, especially the ultrasonic transmitter array 220, can be determined according to the actual application and system. flexibly arranged according to the requirements, and there is no restriction on this. Several preferred arrangements of the embodiments of the present disclosure are specifically described below.
在一些实施例中，如图6所示，触觉反馈子***200设置于成像子***100的上方且位于空中交互界面的触摸侧。超声波发射器阵列220的发射端与空中交互界面平行相对设置。超声波发射器阵列220的发射端朝向空中交互界面。通过将超声波发射器阵列220与空中交互界面呈平行布置，可以减小各超声波发射器30与触控点之间的空间距离，提高聚焦精度。In some embodiments, as shown in FIG. 6 , the haptic feedback subsystem 200 is disposed above the imaging subsystem 100 and on the touch side of the air interface. The transmitting end of the ultrasonic transmitter array 220 is arranged in parallel and opposite to the air interface. The transmitting end of the ultrasonic transmitter array 220 faces the air interface. By arranging the ultrasonic transmitter array 220 in parallel with the air interactive interface, the spatial distance between each ultrasonic transmitter 30 and the touch point can be reduced, and the focusing accuracy can be improved.
在另一些实施例中，触觉反馈子***200设置于成像子***100的上方且位于空中交互界面的成像侧，如图13所示，超声波发射器阵列220与光学组件26垂直，且超声波发射器阵列220与空中交互界面在水平方向平齐，超声波发射器阵列220的发射端朝向空中交互界面。通过将超声波发射器阵列220与空中交互界面布置在同一侧，可以减少超声波因受外物阻碍而造成的辐射能量损耗。In other embodiments, the haptic feedback subsystem 200 is disposed above the imaging subsystem 100 and on the imaging side of the air interface. As shown in FIG. 13 , the ultrasonic transmitter array 220 is perpendicular to the optical component 26, and the ultrasonic transmitters The array 220 is flush with the air interface in the horizontal direction, and the transmitting end of the ultrasonic transmitter array 220 faces the air interface. By arranging the ultrasonic transmitter array 220 on the same side as the air interface, the radiation energy loss of ultrasonic waves caused by being obstructed by foreign objects can be reduced.
在另一些实施例中，触觉反馈子***200设置于第二容纳腔50中，且位于光学组件26对携带人机交互信息的光线的反射路径上，如图14所示，超声波发射器阵列220的发射端朝向空中交互界面。通过将超声波发射器阵列220集成于成像子***100中，在整体上提高触控反馈***1000的美观性和集成性。In other embodiments, the haptic feedback subsystem 200 is disposed in the second accommodating cavity 50 and is located on the reflection path of the optical component 26 to the light carrying the human-computer interaction information. As shown in FIG. 14 , the ultrasonic transmitter array 220 The transmitting end faces the air interface. By integrating the ultrasonic transmitter array 220 into the imaging subsystem 100 , the aesthetics and integration of the touch feedback system 1000 are improved as a whole.
在另一些实施例中，触觉反馈子***200设置于成像子***100的上方且位于空中交互界面的触摸侧，如图15所示，超声波发射器阵列220与光学组件26平行设置，超声波发射器阵列220与空中交互界面在垂直方向平齐，超声波发射器阵列220的发射端朝向空中交互界面。通过将超声波发射器阵列220与空中交互界面布置在同一侧，可以减少超声波因受外物阻碍而造成的辐射能量损耗。In other embodiments, the haptic feedback subsystem 200 is disposed above the imaging subsystem 100 and on the touch side of the air interface. As shown in FIG. 15 , the ultrasonic transmitter array 220 is disposed in parallel with the optical assembly 26 . The array 220 is flush with the air interface in the vertical direction, and the transmitting end of the ultrasonic transmitter array 220 faces the air interface. By arranging the ultrasonic transmitter array 220 on the same side as the air interface, the radiation energy loss of ultrasonic waves caused by being obstructed by foreign objects can be reduced.
概括来说，根据本公开实施例的触控反馈***1000，通过成像子***100采用可交互空中成像技术，在空中的确定位置形成空中交互界面，即无需借助任何实体介质在空中目标区域10处呈现浮空实像，以及，通过触觉反馈子***200采用超声波辐射压力触控反馈技术，将超声波聚焦点区域调整为覆盖空中交互界面所处位置，该区域构成指导用户触控区。当用户触摸空中交互界面时，成像子***100触发触觉反馈子***200在触控点处利用超声波声压叠加而产生触觉反馈力，即无需借助实体设备，实现对该浮空触控对象的触觉感知。因此，本公开实施例将超声波辐射压力触觉反馈技术与可交互空中成像技术深度融合，通过触控反馈***1000的集成与开发，既实现用户非接触式触觉反馈，又在视觉和触觉上给用户营造全新的显示与交互体验。以及，基于触觉反馈子***200发射超声波所产生的声压叠加聚焦形成的触控点，使得在空中目标区域10为用户提供了实际的触控依托，用户通过指尖即可准确感知触控情况，从而在视觉反馈的基础上，提高触控交互效率，避免用户在空中无意义的重复点触。以及，本公开实施例中超声波发射器30采用低功率的小型化产品，也可以避免对人体造成损害。In general, according to the touch feedback system 1000 according to the embodiment of the present disclosure, the imaging subsystem 100 adopts the interactive aerial imaging technology to form an aerial interactive interface at a determined position in the air, that is, the aerial target area 10 does not need any physical medium. A floating real image is presented, and the ultrasonic radiation pressure touch feedback technology is used by the haptic feedback subsystem 200 to adjust the ultrasonic focus area to cover the position of the air interactive interface, and this area constitutes a touch area for guiding the user. When the user touches the air interactive interface, the imaging subsystem 100 triggers the haptic feedback subsystem 200 to generate a haptic feedback force by superimposing ultrasonic sound pressure at the touch point, that is, without the aid of a physical device, the haptic touch of the floating touch object can be realized perception. Therefore, the embodiment of the present disclosure deeply integrates the ultrasonic radiation pressure tactile feedback technology with the interactive aerial imaging technology, and through the integration and development of the touch feedback system 1000, not only realizes the user's non-contact tactile feedback, but also provides the user with visual and tactile feedback. Create a new display and interactive experience. And, based on the touch points formed by the superposition of the sound pressure generated by the ultrasonic wave emitted by the haptic feedback subsystem 200, the target area 10 in the air provides the user with an actual touch support, and the user can accurately perceive the touch situation through the fingertips , so that on the basis of visual feedback, the efficiency of touch interaction can be improved and users can avoid meaningless repeated touches in the air. Moreover, in the embodiment of the present disclosure, the ultrasonic transmitter 30 adopts a low-power miniaturized product, which can also avoid damage to the human body.
本公开第二方面实施例提供一种终端设备，如图16所示，终端设备2000包括设备本体300以及上述实施例提供的触控反馈***1000，触控反馈***1000设置在设备本体300上。在实施例中，终端设备可以包括电梯、取票机、取款机、业务查询机器等以及其它可适用的具备触控交互功能的设备。Embodiments of the second aspect of the present disclosure provide a terminal device. As shown in FIG. 16 , the terminal device 2000 includes a device body 300 and the touch feedback system 1000 provided in the above embodiments. The touch feedback system 1000 is provided on the device body 300 . In an embodiment, the terminal device may include an elevator, a ticket machine, a cash machine, a service inquiry machine, etc., as well as other applicable devices with touch interaction functions.
根据本公开实施例的终端设备2000，通过采用上述实施例提供的触控反馈***1000，可以实现用户非接触式触觉反馈的目的，且无需设置额外的限制用户操作装置，操作方式也更加自然方便，并在视觉和触觉上同时满足用户的交互体验。According to the terminal device 2000 according to the embodiment of the present disclosure, by using the touch feedback system 1000 provided by the above-mentioned embodiments, the purpose of non-contact haptic feedback for the user can be achieved, and there is no need to set an additional restricting user operation device, and the operation method is also more natural and convenient , and satisfy the user's interactive experience both visually and tactilely.
本公开第三方面实施例提供一种触控反馈控制方法，如图17所示，本公开实施例的方法至少包括步骤S1-步骤S3。The embodiment of the third aspect of the present disclosure provides a touch feedback control method. As shown in FIG. 17 , the method of the embodiment of the present disclosure includes at least steps S1 to S3 .
步骤S1，将人机交互信息在空中目标区域成像显示以形成空中交互界面。Step S1, image and display the human-computer interaction information in the air target area to form an air interaction interface.
本公开实施例采用以可交互空中成像技术与超声波辐射压力触觉反馈技术相结合的方式，用以高效完成空中触觉反馈交互。The embodiment of the present disclosure adopts the method of combining the interactive aerial imaging technology and the ultrasonic radiation pressure tactile feedback technology to efficiently complete the aerial tactile feedback interaction.
在实施例中，本公开实施例通过采用可交互空中成像技术，可以在空中的确定位置处将人机交互信息进行显示，以形成空中交互界面，在视觉上满足用户的交互体验。空中交互界面位置在空中是相对固定的，因此将空中交互界面所在的三维空间作为空中目标区域。因此，用户可以直接与空中交互界面进行交互，而无需设置额外的限制机构来引导用户进行操作，避免了用户与设备本体接触的风险，减少因用户交叉使用而引发的公共卫生安全问题，同时也避免因 用户指纹信息残留，使得个人信息泄露而造成的信息安全问题。In the embodiment, by adopting the interactive aerial imaging technology, the human-computer interaction information can be displayed at a determined position in the air to form an aerial interactive interface, which visually satisfies the user's interactive experience. The position of the air interface is relatively fixed in the air, so the three-dimensional space where the air interface is located is taken as the air target area. Therefore, the user can directly interact with the air interface without setting additional restriction mechanisms to guide the user to operate, avoiding the risk of the user coming into contact with the device body, reducing the public health and safety issues caused by the cross-use of the user, and also Avoid information security problems caused by the leakage of personal information due to residual fingerprint information of users.
步骤S2，检测到用户与空中交互界面的交互信号，获取触控点坐标信息并发送触控反馈控制指令。In step S2, an interaction signal between the user and the air interaction interface is detected, the coordinate information of the touch point is acquired, and a touch feedback control instruction is sent.
在实施例中，将空中交互界面作为触觉感知的基准面，用于引导用户触控。在检测到用户与空中交互界面的交互信号后，可以根据空中交互界面的显示对象，并结合触控点的分布，确定触控点坐标信息，进而发送触控点坐标信息和触控反馈控制指令，以在触控点处为用户提供触觉反馈。In the embodiment, the air interaction interface is used as a reference plane for tactile perception to guide the user to touch. After detecting the interaction signal between the user and the air interactive interface, the coordinate information of the touch point can be determined according to the display objects of the air interactive interface and the distribution of the touch points, and then the coordinate information of the touch point and the touch feedback control command can be sent. , to provide the user with haptic feedback at the touch point.
步骤S3，根据触控反馈控制指令和触控点坐标信息控制超声波发射器阵列发射超声波并将超声波聚焦至空中交互界面的触控点。Step S3, controlling the ultrasonic transmitter array to emit ultrasonic waves according to the touch feedback control instruction and the coordinate information of the touch points and focus the ultrasonic waves on the touch points of the air interactive interface.
在实施例中，本公开实施例通过采用超声波辐射压力触觉反馈技术，响应于触控反馈控制指令，控制超声波发射器阵列发射超声波，同时结合触控点坐标信息，控制超声波发射器阵列发射的超声波在同时刻聚焦至空中交互界面的触控点，使得在触控点处所叠加后的超声波声压可以被人体感知，实现超声波触觉反馈。In the embodiment, the embodiment of the present disclosure controls the ultrasonic transmitter array to emit ultrasonic waves by using the ultrasonic radiation pressure tactile feedback technology in response to the touch feedback control instruction, and simultaneously controls the ultrasonic wave emitted by the ultrasonic transmitter array in combination with the coordinate information of the touch point. Focusing on the touch points of the air interactive interface at the same time, so that the ultrasonic sound pressure superimposed at the touch points can be sensed by the human body, realizing ultrasonic tactile feedback.
根据本公开实施例的触控反馈控制方法，通过将人机交互信息在空中目标区域成像显示以形成空中交互界面，即采用可交互空中成像技术，将空中交互界面作为触觉感知的基准面，以用于引导用户触控，并在检测到用户与空中交互界面的交互信号时，获取触控反馈控制指令和触控点坐标信息，以根据获取的触控反馈控制指令和触控点坐标信息，控制超声波发射器阵列发射超声波并将超声波聚焦至触控点，即通过超声波辐射压力，以在人体与触控点之间产生触觉反馈效果，使得人体可以真实感受到触控点的存在，实现用户非接触式触觉反馈的目的，以及本公开实施例中基于在空中目标区域呈现空中交互界面，通过用户触摸空中交互界面即可触发触控反馈的操作，既在视觉和触觉上同时满足用户的交互体验，且操作方式也更加自然舒适，又无需设置额外的限制用户操作装置，避免用户操作时接触设备的风险，也避免出现因用户信息遗留在接触面上而造成个人信息泄露的问题，更加卫生安全。According to the touch feedback control method of the embodiment of the present disclosure, an aerial interactive interface is formed by imaging and displaying human-computer interaction information in an aerial target area, that is, an interactive aerial imaging technology is adopted, and the aerial interactive interface is used as a reference surface for tactile perception, so as to It is used to guide the user to touch, and when the interaction signal between the user and the air interactive interface is detected, obtain the touch feedback control instruction and touch point coordinate information, so as to obtain the touch feedback control instruction and touch point coordinate information according to the acquired touch feedback control instruction and touch point coordinate information, Control the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch points, that is, through the ultrasonic radiation pressure, to generate a tactile feedback effect between the human body and the touch points, so that the human body can truly feel the existence of the touch points and realize the user experience. The purpose of the non-contact tactile feedback, and in the embodiment of the present disclosure, based on the presentation of the air interactive interface in the air target area, the touch feedback operation can be triggered by the user touching the air interactive interface, which not only satisfies the user's interaction both visually and tactilely Experience, and the operation method is also more natural and comfortable, and there is no need to set additional restrictions on user operation, avoiding the risk of users touching the equipment during operation, and avoiding the problem of personal information leakage caused by user information left on the contact surface, which is more hygienic Safety.
本公开第四方面实施例提供一种触控反馈控制方法，该触控反馈控制方法用于触觉反馈子***，触觉反馈子***包括超声波发射器阵列。例如，该触觉反馈子***为图1中所示的触控反馈***1000中的触觉反馈子***200。A fourth aspect of the present disclosure provides a touch feedback control method, where the touch feedback control method is used in a haptic feedback subsystem, and the haptic feedback subsystem includes an ultrasonic transmitter array. For example, the haptic feedback subsystem is the haptic feedback subsystem 200 in the touch feedback system 1000 shown in FIG. 1 .
如图18所示，该触控反馈控制方法至少包括步骤S10和步骤S20：As shown in FIG. 18 , the touch feedback control method includes at least step S10 and step S20:
S10、接收到空中交互界面的触控点坐标信息和触控反馈控制指令；S10, receiving the touch point coordinate information and touch feedback control instructions of the air interactive interface;
S20、根据所述触控反馈控制指令和所述触控点坐标信息控制所述超声波发射器阵列发射超声波，并将所述超声波聚焦至所述空中交互界面的触控点。S20. Control the ultrasonic transmitter array to emit ultrasonic waves according to the touch feedback control instruction and the touch point coordinate information, and focus the ultrasonic waves on the touch points of the air interactive interface.
与上文的解释类似地，根据本公开第四方面实施例的用于触觉反馈子***的触控反馈控制方法也可以避免用户操作时接触设备的风险，也避免出现因用户信息遗留在接触面上而造成个人信息泄露的问题，更加卫生安全。Similar to the above explanation, the touch feedback control method for the haptic feedback subsystem according to the embodiment of the fourth aspect of the present disclosure can also avoid the risk of the user touching the device during operation, and also avoid the occurrence of user information left on the contact surface. The problem of personal information leakage is caused by the above, and it is more hygienic and safe.
需要注意的是，上文针对触控反馈***的描述可以适应性地适用于根据本发明第三方面和第四方面实施例的触控反馈控制方法，为了简洁，这里不再赘述。另外，下文的描述适用于第三方面和第四方面实施例的触控反馈控制方法。It should be noted that the above description of the touch feedback system can be adaptively applied to the touch feedback control methods according to the embodiments of the third and fourth aspects of the present invention, which are not repeated here for brevity. In addition, the following descriptions are applicable to the touch feedback control methods of the embodiments of the third aspect and the fourth aspect.
在一些实施例中，由于超声波发射器阵列中每个超声波发射器与触控点的距离不同，因此，对于根据触控反馈控制指令和触控点坐标信息控制超声波发射器阵列发射超声波并将超声波聚焦至空中交互界面的触控点，包括，根据触控点坐标信息和超声波发射器阵列中每个超声波发射器的发射端的位置信息确定每个超声波发射器的延时相位，以及根据触控反馈控制指令生成原始驱动信号。从而根据每个超声波发射器的延时相位对原始驱动信号进行调整，以控制每个超声波发射器发射超声波的发射时间，将多个超声波发射器发射的超声波在同时刻聚焦至触控点，实现触觉反馈。In some embodiments, since the distance between each ultrasonic transmitter in the ultrasonic transmitter array and the touch point is different, for controlling the ultrasonic transmitter array to emit ultrasonic waves and transmit the ultrasonic waves according to the touch feedback control instruction and the touch point coordinate information Focusing on the touch point of the air interactive interface, including determining the delay phase of each ultrasonic transmitter according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter in the ultrasonic transmitter array, and determining the delay phase of each ultrasonic transmitter according to the touch feedback The control commands generate raw drive signals. Therefore, the original driving signal is adjusted according to the delay phase of each ultrasonic transmitter to control the transmission time of each ultrasonic transmitter to transmit ultrasonic waves, and the ultrasonic waves emitted by multiple ultrasonic transmitters are focused to the touch point at the same time, so as to realize Haptic feedback.
在一些实施例中，超声波发射器阵列中包含多个超声波发射器，因此对于根据触控反馈控制指令生成原始驱动信号，包括，根据触控反馈控制指令产生时钟信号和调制信号，并根据时钟信号和调制信号生成原始驱动信号，以根据对应的原始驱动信号实现对超声波发射器阵列中各个超声波发射器的控制。In some embodiments, the ultrasonic transmitter array includes a plurality of ultrasonic transmitters. Therefore, generating the original driving signal according to the touch feedback control instruction includes generating a clock signal and a modulation signal according to the touch feedback control instruction, and generating a clock signal and a modulation signal according to the touch feedback control instruction. The original driving signal is generated with the modulation signal, so as to realize the control of each ultrasonic transmitter in the ultrasonic transmitter array according to the corresponding original driving signal.
在一些实施例中，对于根据每个超声波发射器的延时相位对原始驱动信号进行调整，包括，根据每个超声波发射器的延时相位对原始驱动信号进行延时处理，获得每个超声波发射器的目标驱动信号；根据目标驱动信号驱动每个超声波发射器。从而，通过对一定数量的超声波发射 器相位的控制，实现所有超声波发射器同时刻到达触控点处进行叠加，以使叠加后的超声波声压可以被人体真实感知，实现超声波触觉反馈。In some embodiments, adjusting the original driving signal according to the delayed phase of each ultrasonic transmitter includes performing a delay processing on the original driving signal according to the delayed phase of each ultrasonic transmitter to obtain each ultrasonic transmitter The target driving signal of the transmitter; each ultrasonic transmitter is driven according to the target driving signal. Therefore, by controlling the phase of a certain number of ultrasonic transmitters, all ultrasonic transmitters can reach the touch point at the same time for superposition, so that the superimposed ultrasonic sound pressure can be truly perceived by the human body, and ultrasonic tactile feedback can be realized.
在一些实施例中，由于声速值随环境温度而变化，对延时相位数据的计算影响较大，因此本公开需对环境温度进行实时采集，具体包括，获取环境温度，并根据环境温度对每个超声波发射器的延时相位进行校正，以减小延时控制误差，提高超声波发射器阵列的聚焦精度。In some embodiments, since the sound velocity value changes with the ambient temperature, which has a great influence on the calculation of the delayed phase data, the present disclosure needs to collect the ambient temperature in real time, which specifically includes: The delay phase of each ultrasonic transmitter is corrected to reduce the delay control error and improve the focusing accuracy of the ultrasonic transmitter array.
本公开实施例中对于较复杂的二维交互界面或三维立体交互对象，分别采用单块阵列分区域控制或多块阵列同步控制的方案，具体如下。For a more complex two-dimensional interactive interface or three-dimensional interactive object in the embodiments of the present disclosure, a single-block array sub-regional control or multi-block array synchronous control scheme is respectively adopted, as follows.
在一些实施例中，由于超声波发射器阵列所提供的触控区域面积的限制，复杂交互界面触控区域较大，单一阵列控制时对***刷新频率要求高，且大阵列所有超声波传感器都进行聚焦控制时，相位延时时间增大，无法保证聚焦精度，无法满足***实时性要求，因此，本公开设置超声波发射器阵列包括n个分布在同一平面的超声波发射器子阵列，空中交互界面划分为n个触控子区，n个超声波发射器子阵列与n个触控子区一一对应配置。也就是，采用单块阵列分区域控制的方式，将超声波发射器阵列进行分区，以分成若干个超声波发射器子阵列，且每个超声波发射器子阵列分别映射到对应的触控子区，由所有的触控子区组成复杂的二维交互界面。In some embodiments, due to the limitation of the touch area area provided by the ultrasonic transmitter array, the touch area of the complex interactive interface is large, the system refresh frequency is required to be high when a single array is controlled, and all ultrasonic sensors in the large array are focused During control, the phase delay time increases, the focusing accuracy cannot be guaranteed, and the system real-time requirements cannot be met. Therefore, the present disclosure sets the ultrasonic transmitter array to include n ultrasonic transmitter sub-arrays distributed on the same plane, and the air interaction interface is divided into There are n touch sub-regions, and the n ultrasonic transmitter sub-arrays are arranged in a one-to-one correspondence with the n touch sub-regions. That is, the ultrasonic transmitter array is divided into several sub-arrays of ultrasonic transmitters by using a single-block array sub-area control method, and each sub-array of ultrasonic transmitters is mapped to the corresponding touch sub-area, which is defined by All touch sub-areas form a complex two-dimensional interactive interface.
对于单块阵列分区域控制的方式，本公开实施例中根据触控反馈控制指令和触控点坐标信息控制超声波发射器阵列发射超声波并将超声波聚焦至触控点，包括，根据触控点坐标信息确定触控点对应空中交互界面中的目标触控子区；根据目标触控子区确定目标超声波发射器子阵列，根据触控点坐标信息和目标超声波发射器子阵列中每个超声波发射器的发射端的位置信息确定超声波发射器子阵列中每个超声波发射器的延时相位；根据触控反馈控制指令产生时钟信号和调制信号，并根据时钟信号和调制信号生成原始驱动信号；根据目标超声波发射器子阵列中每个超声波发射器的延时相位对原始驱动信号进行延时处理以获得目标超声波发射器子阵列中每个超声波发射器的目标驱动信号；根据目标驱动信号驱动超声波发射器子阵列中每个超声波发射器。由此，基于超声波发射器阵列分为多个区域，通过控制目标触控子区中的每个超声波发射器的相位，从而可以减少聚焦点数目以及相位延时时间差，保证聚焦精度，满足***的实时性要求。For the method of sub-regional control of a single-block array, in the embodiment of the present disclosure, controlling the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves on the touch points according to the touch feedback control instructions and the coordinate information of the touch points, including: according to the coordinates of the touch points The information determines that the touch point corresponds to the target touch sub-area in the air interactive interface; the target ultrasonic transmitter sub-array is determined according to the target touch sub-area, and each ultrasonic transmitter in the target ultrasonic transmitter sub-array is determined according to the coordinate information of the touch point and the target ultrasonic transmitter sub-array The position information of the transmitting end of the device determines the delay phase of each ultrasonic transmitter in the ultrasonic transmitter sub-array; generates a clock signal and a modulation signal according to the touch feedback control command, and generates an original drive signal according to the clock signal and modulation signal; according to the target ultrasonic wave The delay phase of each ultrasonic transmitter in the transmitter sub-array performs delay processing on the original driving signal to obtain the target driving signal of each ultrasonic transmitter in the target ultrasonic transmitter sub-array; the ultrasonic transmitter sub-array is driven according to the target driving signal. Each ultrasonic transmitter in the array. Therefore, based on the ultrasonic transmitter array is divided into multiple areas, by controlling the phase of each ultrasonic transmitter in the target touch sub-area, the number of focusing points and the phase delay time difference can be reduced, the focusing accuracy can be guaranteed, and the system requirements can be met. real-time requirements.
举例说明，如图19所示，本公开实施例将空中交互界面分割成四个触控子区，并分别映射到对应的超声波发射器子阵列上，以减少聚焦点数目以及相位延时时间差，并采用局部聚焦的方式实现聚焦扫描。具体地，在数据处理模块获取触控区域所有触控点坐标后，按照就近原则将触控点分配给对应的超声波发射器子阵列，组成四个触控子区域与超声波发射器子阵列分组，各分组的聚焦控制过程与单阵列聚焦控制过程原理相同，即根据触控点坐标信息确定触控点对应空中交互界面中的触控子区，该区域作为目标触控子区，目标触控子区对应的超声波发射器子阵列即为目标超声波发射器子阵列。数据处理模块计算目标触控子区所对应的所有超声波发射器的延时相位数据，并将延时相位数据发送至驱动模块。驱动模块根据接收到的延时相位数据以及触控反馈控制指令，完成目标触控子区的信号调制、时钟分配、延时控制以及驱动放大，并将目标驱动信号同步输出至对应的超声波发射器子阵列，以控制超声波发射器子阵列上每个超声波发射器在规定时刻发射超声波，由此完成对应触控点的聚焦控制，保证聚焦精度。For example, as shown in FIG. 19 , the embodiment of the present disclosure divides the air interactive interface into four touch sub-regions, and maps them to the corresponding ultrasonic transmitter sub-arrays, so as to reduce the number of focus points and the phase delay time difference, And use local focusing to achieve focused scanning. Specifically, after the data processing module obtains the coordinates of all touch points in the touch area, the touch points are allocated to the corresponding ultrasonic transmitter sub-arrays according to the principle of proximity, and four touch sub-areas are grouped with the ultrasonic transmitter sub-arrays. The focus control process of each group is the same as that of the single-array focus control process, that is, according to the coordinate information of the touch point, the touch sub-area in the air interactive interface corresponding to the touch point is determined, and this area is used as the target touch sub-area, the target touch sub-area. The ultrasonic transmitter sub-array corresponding to the area is the target ultrasonic transmitter sub-array. The data processing module calculates the delayed phase data of all ultrasonic transmitters corresponding to the target touch sub-area, and sends the delayed phase data to the driving module. The drive module completes the signal modulation, clock distribution, delay control and drive amplification of the target touch sub-area according to the received delay phase data and touch feedback control instructions, and outputs the target drive signal synchronously to the corresponding ultrasonic transmitter The sub-array is used to control each ultrasonic transmitter on the sub-array of ultrasonic transmitters to emit ultrasonic waves at a specified time, thereby completing the focusing control of the corresponding touch points and ensuring the focusing accuracy.
在一些实施例中，对于三维立体交互场景，由于聚焦点分布在空间任意点，而并非在同一个平面上，因此，本公开设置超声波发射器阵列为n个，n个超声波发射器阵列分布在同一平面或者以预设角度错开设置，空中交互界面划分为n个触控子区，n个超声波发射器阵列与n个触控子区一一对应配置。也就是，采用多块超声波发射器阵列，并将触控区域分为若干个触控子区域，按照触控子区域所在的位置布置相应的超声波发射器阵列，即以多块阵列同步控制的方式，实现各超声波发射器在触控点叠加汇聚。其中，对于多块超声波发射器阵列的布置，根据实际需求可以分布在同一平面上，也可以按照一定角度错开布置，对此不作限制。In some embodiments, for a three-dimensional interactive scene, since the focus points are distributed at any point in space, rather than on the same plane, the present disclosure sets n ultrasonic transmitter arrays, and the n ultrasonic transmitter arrays are distributed on the On the same plane or staggered at a preset angle, the air interaction interface is divided into n touch sub-areas, and the n ultrasonic transmitter arrays are arranged in a one-to-one correspondence with the n touch sub-areas. That is, multiple ultrasonic transmitter arrays are used, the touch area is divided into several touch sub-areas, and the corresponding ultrasonic transmitter arrays are arranged according to the positions of the touch sub-areas, that is, the multi-block array is controlled synchronously. , to realize the superposition and convergence of each ultrasonic transmitter at the touch point. Among them, the arrangement of the multiple ultrasonic transmitter arrays may be distributed on the same plane according to actual requirements, or may be staggered according to a certain angle, which is not limited.
对于多块阵列同步控制的方式，本公开实施例中根据触控反馈控制指令和触控点坐标信息控制超声波发射器阵列发射超声波并将超声波聚焦至触控点，包括，根据触控点坐标信息确定触控点对应空中交互界面中的目标触控子区；根据目标触控子区确定目标超声波发射器阵列；根据触控点坐标信息和目标超声波发射器阵列中每个超声波发射器的发射端的位置信息确定目标超声波发射器阵列中每个超声波发射器的延时相位；根据触控反馈控制指令产生时钟信号和调制信号，并根据时钟信号和调制信号生成原始驱动信号；根据目标超声波发射器阵列中每个 超声波发射器的延时相位对原始驱动信号进行延时处理以获得目标超声波发射器阵列中每个超声波发射器的目标驱动信号；根据目标驱动信号驱动目标超声波发射器阵列中每个超声波发射器。基于多块超声波发射器阵列的设置，通过控制目标超声波发射器阵列中的每个超声波发射器的相位，可以保证聚焦精度，满足***的实时性要求。For the synchronous control of multiple arrays, in the embodiment of the present disclosure, controlling the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch points according to the touch feedback control instructions and the coordinate information of the touch points, including: according to the coordinate information of the touch points Determine the touch point corresponding to the target touch sub-area in the air interactive interface; determine the target ultrasonic transmitter array according to the target touch sub-area; according to the coordinate information of the touch point and the transmitting end of each ultrasonic transmitter in the target ultrasonic transmitter array The position information determines the delay phase of each ultrasonic transmitter in the target ultrasonic transmitter array; generates a clock signal and a modulation signal according to the touch feedback control command, and generates an original drive signal according to the clock signal and modulation signal; according to the target ultrasonic transmitter array The delay phase of each ultrasonic transmitter in the original driving signal is delayed to obtain the target driving signal of each ultrasonic transmitter in the target ultrasonic transmitter array; each ultrasonic wave in the target ultrasonic transmitter array is driven according to the target driving signal. launcher. Based on the setting of multiple ultrasonic transmitter arrays, by controlling the phase of each ultrasonic transmitter in the target ultrasonic transmitter array, the focusing accuracy can be ensured and the real-time requirements of the system can be met.
举例说明，如图20所示，可以设置八个超声波发射器阵列对应八个触觉反馈子***，触控点按照就近原则分配给对应的触觉反馈子***，组成八个触控子区与触觉反馈子***分组，各分组的聚焦控制过程与单阵列聚焦控制过程原理相同。具体地，八个触觉反馈子***对应的数据处理模块分别获取对应触控子区的触控点坐标，根据触控点坐标确定该触控点所对应的触控子区即为目标触控子区，目标触控子区对应的超声波发射器阵列即为目标超声波发射器阵列，目标超声波发射器阵列对应的数据处理模块计算目标超声波发射器阵列内所有超声波发射器的延时相位数据，并将数据发送给对应的驱动模块。驱动模块根据接收到的目标超声波发射器阵列的延时相位数据以及触控反馈控制指令，完成对目标超声波发射器阵列的信号调制、时钟分配、延时控制以及驱动放大，并将目标驱动信号传送给目标超声波发射器阵列，以控制目标超声波发射器阵列上每个超声波发射器在规定时刻发射超声波，完成对应触控点的聚焦控制。因此，通过采用多个超声波发射器阵列，以在不同角度对触控对象进行聚焦控制，可以保证聚焦精度和***实时性。For example, as shown in Figure 20, eight ultrasonic transmitter arrays can be set to correspond to eight haptic feedback subsystems, and the touch points are allocated to the corresponding haptic feedback subsystems according to the principle of proximity to form eight touch sub-areas and haptic feedback. Subsystems are grouped, and the focus control process of each group is the same as the single-array focus control process. Specifically, the data processing modules corresponding to the eight tactile feedback subsystems obtain the coordinates of the touch points corresponding to the touch sub-areas respectively, and determine the touch sub-areas corresponding to the touch points according to the coordinates of the touch points as the target touch sub-areas. area, the ultrasonic transmitter array corresponding to the target touch sub-area is the target ultrasonic transmitter array, and the data processing module corresponding to the target ultrasonic transmitter array calculates the delay phase data of all ultrasonic transmitters in the target ultrasonic transmitter array, and converts the The data is sent to the corresponding driver module. The driving module completes signal modulation, clock distribution, delay control and drive amplification for the target ultrasonic transmitter array according to the received delay phase data of the target ultrasonic transmitter array and touch feedback control instructions, and transmits the target driving signal Give the target ultrasonic transmitter array to control each ultrasonic transmitter on the target ultrasonic transmitter array to emit ultrasonic waves at a specified time to complete the focus control of the corresponding touch point. Therefore, by using a plurality of ultrasonic transmitter arrays to control the focus of the touch object at different angles, the focus accuracy and the real-time performance of the system can be guaranteed.
需要注意的是，本公开考虑多个超声波发射器阵列控制的同步问题，在整个***中设置一路同步控制信号，使得各触控子区与触觉反馈子***分组在同步控制信号的控制下可以进行同步工作，从而保证各触控子区域触控点的同步扫描，为用户提供良好的触控感受，避免出现渲染失真如边界触控不连续的问题。It should be noted that the present disclosure considers the synchronization problem of the control of multiple ultrasonic transmitter arrays, and sets one synchronization control signal in the entire system, so that each touch sub-area and the haptic feedback subsystem can be grouped under the control of the synchronization control signal. It works synchronously to ensure synchronous scanning of touch points in each touch sub-area, provide users with a good touch experience, and avoid rendering distortions such as discontinuous border touches.
本公开第五方面实施例提供一种存储介质，其上存储有计算机程序，其中，计算机程序被处理器执行时实现上述实施例提供的触控反馈控制方法。Embodiments of a fifth aspect of the present disclosure provide a storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the touch feedback control method provided by the foregoing embodiments is implemented.
在本说明书的描述中，流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本公开的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本公开的实施例所属技术领域的技术人员所理解。In the description of this specification, any description of a process or method in a flowchart or otherwise described herein may be understood to represent a representation of executable instructions comprising one or more steps for implementing a custom logical function or process modules, segments or portions of code, and the scope of preferred embodiments of the present disclosure include alternative implementations, which may not be in the order shown or discussed, including in a substantially simultaneous manner or in the reverse order depending on the functionality involved , to perform functions, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行***、装置或设备(如基于计算机的***、包括处理器的***或其他可以从指令执行***、装置或设备取指令并执行指令的***)使用，或结合这些指令执行***、装置或设备而使用。就本说明书而言，"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行***、装置或设备或结合这些指令执行***、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.
应当理解，本公开的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行***执行的软件或固件来实现。如，如果用硬件来实现和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.
此外，在本公开各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用 硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
上述提到的存储介质可以是只读存储器，磁盘或光盘等。尽管上面已经示出和描述了本公开的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本公开的限制，本领域的普通技术人员在本公开的范围内可以对上述实施例进行变化、修改、替换和变型。The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present disclosure have been shown and described above, it should be understood that the above-described embodiments are exemplary and should not be construed as limitations of the present disclosure, and those of ordinary skill in the art may interpret the above-described embodiments within the scope of the present disclosure. Embodiments are subject to variations, modifications, substitutions and variations.
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示意性实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本公开的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。In the description of this specification, reference to the terms "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples", etc., is meant to incorporate the embodiments A particular feature, structure, material, or characteristic described by an example or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example.
尽管已经示出和描述了本公开的实施例，本领域的普通技术人员可以理解：在不脱离本公开的原理和宗旨的情况下可以对这些实施例进行多种变化、修改、替换和变型，本公开的范围由权利要求及其等同物限定。Although embodiments of the present disclosure have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and alterations can be made in these embodiments without departing from the principles and spirit of the present disclosure, The scope of the present disclosure is defined by the claims and their equivalents.

Claims (20)

  1. 一种触控反馈***，其特征在于，包括：A touch feedback system, comprising:
    成像子***，用于将人机交互信息在空中目标区域成像显示以形成空中交互界面，并在检测到用户与所述空中交互界面的交互信号时发送触控反馈控制指令和触控点坐标信息；The imaging subsystem is used to image and display the human-computer interaction information in the aerial target area to form an aerial interactive interface, and send touch feedback control instructions and touch point coordinate information when detecting the interaction signal between the user and the aerial interactive interface ;
    触觉反馈子***，与所述成像子***连接，用于根据所述触控反馈控制指令和所述触控点坐标信息发射超声波并将所述超声波聚焦至所述空中交互界面的触控点。A haptic feedback subsystem, connected to the imaging subsystem, is configured to emit ultrasonic waves according to the touch feedback control instructions and the coordinate information of the touch points and focus the ultrasonic waves on the touch points of the air interactive interface.
  2. 根据权利要求1所述的触控反馈***，其特征在于，所述触觉反馈子***包括：The touch feedback system according to claim 1, wherein the haptic feedback subsystem comprises:
    第一壳体，所述第一壳体上设置有开窗且在内部形成有第一容纳腔；a first casing, the first casing is provided with a window and a first accommodating cavity is formed inside;
    超声波发射器阵列，设置于所述第一容纳腔中，包括多个超声波发射器，每个所述超声波发射器的发射端均朝向所述开窗；an array of ultrasonic transmitters, arranged in the first accommodating cavity, including a plurality of ultrasonic transmitters, and the transmitting end of each of the ultrasonic transmitters faces the opening window;
    控制器，设置于所述第一容纳腔中，与所述超声波发射器阵列连接，用于根据所述触控反馈控制指令和所述触控点坐标信息控制每个所述超声波发射器发射超声波，并将所述超声波聚焦至所述触控点。a controller, disposed in the first accommodating cavity, connected to the ultrasonic transmitter array, and configured to control each ultrasonic transmitter to emit ultrasonic waves according to the touch feedback control instruction and the touch point coordinate information , and focus the ultrasonic wave to the touch point.
  3. 根据权利要求2所述的触控反馈***，其特征在于，所述控制器包括：The touch feedback system according to claim 2, wherein the controller comprises:
    数据处理模块，用于根据所述触控点坐标信息和每个所述超声波发射器的位置信息确定每个所述超声波发射器的延时相位；a data processing module, configured to determine the delay phase of each of the ultrasonic transmitters according to the coordinate information of the touch point and the position information of each of the ultrasonic transmitters;
    驱动模块，与所述超声波发射器阵列和所述数据处理模块连接，用于响应于所述触控反馈控制指令生成原始驱动信号，并根据每个所述超声波发射器的延时相位调整所述原始驱动信号以将多个所述超声波发射器发射的超声波聚焦至所述触控点。A driving module, connected with the ultrasonic transmitter array and the data processing module, is used for generating an original driving signal in response to the touch feedback control command, and adjusting the ultrasonic transmitter according to the delay phase of each ultrasonic transmitter The original driving signal is used to focus the ultrasonic waves emitted by the plurality of ultrasonic wave transmitters to the touch point.
  4. 根据权利要求3所述的触控反馈***，其特征在于，所述驱动模块在响应于所述触控反馈控制指令时，根据所述触控反馈控制指令产生时钟信号和调制信号，并根据所述时钟信号和所述调制信号生成所述原始驱动信号。The touch feedback system according to claim 3, wherein when responding to the touch feedback control command, the driving module generates a clock signal and a modulation signal according to the touch feedback control command, and generates a clock signal and a modulation signal according to the touch feedback control command. The clock signal and the modulation signal generate the original drive signal.
  5. 根据权利要求3或4所述的触控反馈***，其特征在于，所述驱动模块根据每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理，以获得每个所述超声波发射器的目标驱动信号，并根据所述目标驱动信号驱动每个所述超声波发射器。The touch feedback system according to claim 3 or 4, wherein the driving module performs delay processing on the original driving signal according to the delayed phase of each ultrasonic transmitter, so as to obtain each The target driving signal of the ultrasonic transmitter is generated, and each of the ultrasonic transmitters is driven according to the target driving signal.
  6. 根据权利要求3-5中任一项所述的触控反馈***，其特征在于，所述数据处理模块包括：The touch feedback system according to any one of claims 3-5, wherein the data processing module comprises:
    第一存储单元，用于存储数据处理过程中的数据和程序；a first storage unit, used to store data and programs in the data processing process;
    数据处理单元，所述数据处理单元与所述第一存储单元连接，用于根据所述触控点坐标信息和每个所述超声波发射器的发射端的位置信息确定每个所述超声波发射器的延时相位。A data processing unit, the data processing unit is connected with the first storage unit, and is used for determining the position of each ultrasonic transmitter according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter. Delay phase.
  7. 根据权利要求3-6中任一项所述的触控反馈***，其特征在于，所述驱动模块包括：The touch feedback system according to any one of claims 3-6, wherein the driving module comprises:
    第二存储单元，用于存储驱动控制过程中的数据和文件；The second storage unit is used to store data and files in the drive control process;
    控制单元，所述控制单元与所述第二存储单元连接，用于根据所述触控反馈控制指令产生时钟信号和调制信号，并根据所述时钟信号和所述调制信号生成原始驱动信号，并根据每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理；a control unit, the control unit is connected to the second storage unit, and is configured to generate a clock signal and a modulation signal according to the touch feedback control instruction, and generate an original drive signal according to the clock signal and the modulation signal, and performing delay processing on the original drive signal according to the delayed phase of each of the ultrasonic transmitters;
    驱动单元，所述驱动单元与所述控制单元连接，用于对延时后的原始驱动信号进行放大处理，以获得每个所述超声波发射器的目标驱动信号，并根据所述目标驱动信号驱动每个所述超声波发射器。a driving unit, which is connected to the control unit, and is used for amplifying the delayed original driving signal to obtain a target driving signal of each of the ultrasonic transmitters, and driving according to the target driving signal each of the ultrasonic transmitters.
  8. 根据权利要求2-7中任一项所述的触控反馈***，其特征在于，所述触觉反馈子***还包括：The touch feedback system according to any one of claims 2-7, wherein the haptic feedback subsystem further comprises:
    挡盖，设置在所述第一壳体上，用于遮挡所述开窗。The blocking cover is arranged on the first casing and is used to block the opening window.
  9. 根据权利要求2-8中任一项所述的触控反馈***，其特征在于，所述成像子***包括：The touch feedback system according to any one of claims 2-8, wherein the imaging subsystem comprises:
    第二壳体，所述第二壳体形成有显示窗口且在内部形成有第二容纳腔；a second casing, the second casing is formed with a display window and a second accommodating cavity is formed inside;
    成像组件，设置于所述第二容纳腔中，用于将人机交互信息在空中目标区域成像显示以形成空中交互界面；an imaging component, disposed in the second accommodating cavity, used for imaging and displaying the human-machine interaction information in the air target area to form an air interaction interface;
    检测模块，用于检测用户与所述空中交互界面的交互信号；a detection module for detecting the interaction signal between the user and the air interaction interface;
    主控模块，与所述成像组件和所述检测模块连接，用于响应于所述交互信号发送触控反馈控制指令和触控点坐标信息。The main control module is connected with the imaging component and the detection module, and is used for sending touch feedback control instructions and touch point coordinate information in response to the interaction signal.
  10. 根据权利要求9所述的触控反馈***，其特征在于，所述成像组件包括：The touch feedback system according to claim 9, wherein the imaging component comprises:
    显示器，所述显示器设置于所述第二容纳腔中，与所述主控模块连接，用于显示所述人 机交互信息；a display, which is arranged in the second accommodating cavity and is connected to the main control module for displaying the human-computer interaction information;
    光学组件，所述光学组件设置于所述第二容纳腔中，用于将携带所述人机交互信息的光线汇聚成像在所述空中目标区域，以形成空中交互界面；an optical component, which is arranged in the second accommodating cavity and is used for converging and imaging the light carrying the human-machine interaction information on the aerial target area to form an aerial interaction interface;
    其中，所述显示器设置于所述光学组件的光源侧，所述显示窗口在所述光学组件的成像侧。Wherein, the display is arranged on the light source side of the optical component, and the display window is on the imaging side of the optical component.
  11. 根据权利要求10所述的触控反馈***，其特征在于，The touch feedback system according to claim 10, wherein:
    所述触觉反馈子***设置于所述成像子***的上方且位于所述空中交互界面的触摸侧，所述超声波发射器阵列的发射端与所述空中交互界面平行相对设置，且所述超声波发射器阵列的发射端朝向所述空中交互界面；或者The haptic feedback subsystem is arranged above the imaging subsystem and is located on the touch side of the air interaction interface, the transmitting end of the ultrasonic transmitter array is arranged in parallel and opposite to the air interaction interface, and the ultrasonic wave transmits the transmitting end of the transmitter array facing the air interface; or
    所述触觉反馈子***设置于所述成像子***的上方且位于所述空中交互界面的成像侧，所述超声波发射器阵列与所述光学组件垂直，且所述超声波发射器阵列与所述空中交互界面在水平方向平齐，所述超声波发射器阵列的发射端朝向所述空中交互界面；或者The haptic feedback subsystem is arranged above the imaging subsystem and is located on the imaging side of the air interface, the ultrasonic transmitter array is perpendicular to the optical component, and the ultrasonic transmitter array is connected to the air The interactive interface is flush in the horizontal direction, and the transmitting end of the ultrasonic transmitter array faces the air interactive interface; or
    所述触觉反馈子***设置于所述第二容纳腔中，且位于所述光学组件对携带所述人机交互信息的光线的反射路径上，所述超声波发射器阵列的发射端朝向所述空中交互界面；或者所述触觉反馈子***设置于所述成像子***的上方且位于所述空中交互界面的触摸侧，所述超声波发射器阵列与所述光学组件平行设置，所述超声波发射器阵列与所述空中交互界面在垂直方向平齐，所述超声波发射器阵列的发射端朝向所述空中交互界面。The tactile feedback subsystem is arranged in the second accommodating cavity, and is located on the reflection path of the optical component to the light carrying the human-machine interaction information, and the transmitting end of the ultrasonic transmitter array faces the air interactive interface; or the haptic feedback subsystem is disposed above the imaging subsystem and on the touch side of the air interactive interface, the ultrasonic transmitter array is disposed in parallel with the optical assembly, and the ultrasonic transmitter array is disposed in parallel with the optical component. It is flush with the air interaction interface in the vertical direction, and the transmitting end of the ultrasonic transmitter array faces the air interaction interface.
  12. 一种终端设备，其特征在于，包括：A terminal device, characterized in that it includes:
    设备本体；equipment body;
    权利要求1-11任一项所述的触控反馈***，所述触控反馈***设置在所述设备本体上。The touch feedback system according to any one of claims 1-11, wherein the touch feedback system is provided on the device body.
  13. 一种触控反馈控制方法，其特征在于，包括：A touch feedback control method, comprising:
    将人机交互信息在空中目标区域成像显示以形成空中交互界面；Image and display the human-computer interaction information in the air target area to form an air interaction interface;
    检测到用户与所述空中交互界面的交互信号，获取触控点坐标信息并发送触控反馈控制指令；Detecting the interaction signal between the user and the air interaction interface, acquiring the coordinate information of the touch point and sending the touch feedback control instruction;
    根据所述触控反馈控制指令和所述触控点坐标信息控制超声波发射器阵列发射超声波并将所述超声波聚焦至所述空中交互界面的触控点。According to the touch feedback control instruction and the touch point coordinate information, the ultrasonic transmitter array is controlled to emit ultrasonic waves and focus the ultrasonic waves on the touch points of the air interactive interface.
  14. 根据权利要求13所述的触控反馈控制方法，其特征在于，根据所述触控反馈控制指令和所述触控点坐标信息控制超声波发射器阵列发射超声波并将所述超声波聚焦至所述空中交互界面的触控点，包括：The touch feedback control method according to claim 13, wherein the ultrasonic transmitter array is controlled to emit ultrasonic waves and focus the ultrasonic waves on the air according to the touch feedback control instructions and the coordinate information of the touch points. Touch points of the interactive interface, including:
    根据所述触控点坐标信息和所述超声波发射器阵列中每个超声波发射器的发射端的位置信息确定每个所述超声波发射器的延时相位；Determine the delay phase of each ultrasonic transmitter according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter in the ultrasonic transmitter array;
    根据所述触控反馈控制指令生成原始驱动信号；generating an original drive signal according to the touch feedback control instruction;
    根据每个所述超声波发射器的延时相位对所述原始驱动信号进行调整，以将多个所述超声波发射器发射的超声波聚焦至所述触控点。The original driving signal is adjusted according to the delayed phase of each of the ultrasonic transmitters, so as to focus the ultrasonic waves emitted by the plurality of ultrasonic transmitters to the touch point.
  15. 根据权利要求14所述的触控反馈控制方法，其特征在于，根据所述触控反馈控制指令生成原始驱动信号，包括：The touch feedback control method according to claim 14, wherein generating an original driving signal according to the touch feedback control instruction comprises:
    根据所述触控反馈控制指令产生时钟信号和调制信号；generating a clock signal and a modulation signal according to the touch feedback control instruction;
    根据所述时钟信号和所述调制信号生成所述原始驱动信号。The original drive signal is generated from the clock signal and the modulation signal.
  16. 根据权利要求15所述的触控反馈控制方法，其特征在于，根据每个所述超声波发射器的延时相位对所述原始驱动信号进行调整，包括：The touch feedback control method according to claim 15, wherein adjusting the original driving signal according to the delay phase of each of the ultrasonic transmitters, comprising:
    根据每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理；performing delay processing on the original drive signal according to the delayed phase of each of the ultrasonic transmitters;
    获得每个所述超声波发射器的目标驱动信号；obtaining a target drive signal for each of the ultrasonic transmitters;
    根据所述目标驱动信号驱动每个所述超声波发射器。Each of the ultrasonic transmitters is driven according to the target driving signal.
  17. 根据权利要求14所述的触控反馈控制方法，其特征在于，还包括：The touch feedback control method according to claim 14, further comprising:
    获取环境温度；Get the ambient temperature;
    根据所述环境温度对每个所述超声波发射器的延时相位进行校正。The delay phase of each of the ultrasonic transmitters is corrected according to the ambient temperature.
  18. 根据权利要求13-17中任一项所述的触控反馈控制方法，其特征在于，超声波发射器阵列包括n个分布在同一平面的超声波发射器子阵列，所述空中交互界面划分为n个触控子区，n个所述超声波发射器子阵列与n个所述触控子区一一对应配置；The touch feedback control method according to any one of claims 13-17, wherein the ultrasonic transmitter array comprises n ultrasonic transmitter sub-arrays distributed on the same plane, and the air interaction interface is divided into n sub-arrays touch sub-areas, the n ultrasonic transmitter sub-arrays are arranged in a one-to-one correspondence with the n touch sub-areas;
    根据所述触控反馈控制指令和所述触控点坐标信息控制超声波发射器阵列发射超声波并将所述超声波聚焦至触控点，包括：Controlling the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch points according to the touch feedback control instructions and the touch point coordinate information, including:
    根据所述触控点坐标信息确定触控点对应所述空中交互界面中的目标触控子区；Determine according to the coordinate information of the touch point that the touch point corresponds to the target touch sub-area in the air interaction interface;
    根据所述目标触控子区确定目标超声波发射器子阵列，根据所述触控点坐标信息和所述目标超声波发射器子阵列中每个超声波发射器的发射端的位置信息确定所述目标超声波发射器子阵列中每个所述超声波发射器的延时相位；The target ultrasonic transmitter sub-array is determined according to the target touch sub-area, and the target ultrasonic transmitter is determined according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter in the target ultrasonic transmitter sub-array. the delay phase of each of the ultrasonic transmitters in the transmitter sub-array;
    根据所述触控反馈控制指令产生时钟信号和调制信号，并根据所述时钟信号和所述调制信号生成原始驱动信号；generating a clock signal and a modulation signal according to the touch feedback control instruction, and generating an original driving signal according to the clock signal and the modulation signal;
    根据所述目标超声波发射器子阵列中每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理以获得所述目标超声波发射器子阵列的每个所述超声波发射器的目标驱动信号；Perform delay processing on the original drive signal according to the delayed phase of each of the ultrasonic transmitters in the target ultrasonic transmitter sub-array to obtain a target drive signal;
    根据所述目标驱动信号驱动所述目标超声波发射器子阵列的每个所述超声波发射器。Each of the ultrasonic transmitters of the target ultrasonic transmitter sub-array is driven according to the target drive signal.
  19. 根据权利要求13-18中任一项所述的触控反馈控制方法，其特征在于，所述超声波发射器阵列为n个，n个所述超声波发射器阵列分布在同一平面或者以预设角度错开设置，所述空中交互界面划分为n个触控子区，n个所述超声波发射器阵列与n个所述触控子区一一对应配置；The touch feedback control method according to any one of claims 13-18, wherein the number of the ultrasonic transmitter arrays is n, and the n ultrasonic transmitter arrays are distributed on the same plane or at a preset angle Staggered setting, the air interactive interface is divided into n touch sub-regions, and the n ultrasonic transmitter arrays are arranged in a one-to-one correspondence with the n touch sub-regions;
    根据所述触控反馈控制指令和所述触控点坐标信息控制超声波发射器阵列发射超声波并将所述超声波聚焦至触控点，包括：Controlling the ultrasonic transmitter array to emit ultrasonic waves and focus the ultrasonic waves to the touch points according to the touch feedback control instructions and the touch point coordinate information, including:
    根据所述触控点坐标信息确定触控点对应所述空中交互界面中的目标触控子区；Determine according to the coordinate information of the touch point that the touch point corresponds to the target touch sub-area in the air interaction interface;
    根据所述目标触控子区确定目标超声波发射器阵列，根据所述触控点坐标信息和所述目标超声波发射器阵列中每个超声波发射器的发射端的位置信息确定所述目标超声波发射器阵列中每个所述超声波发射器的延时相位；The target ultrasonic transmitter array is determined according to the target touch sub-area, and the target ultrasonic transmitter array is determined according to the coordinate information of the touch point and the position information of the transmitting end of each ultrasonic transmitter in the target ultrasonic transmitter array the delay phase of each of the ultrasonic transmitters in the
    根据所述触控反馈控制指令产生时钟信号和调制信号，并根据所述时钟信号和所述调制信号生成原始驱动信号；generating a clock signal and a modulation signal according to the touch feedback control instruction, and generating an original driving signal according to the clock signal and the modulation signal;
    根据所述目标超声波发射器阵列中每个所述超声波发射器的延时相位对所述原始驱动信号进行延时处理以获得所述目标超声波发射器阵列中每个所述超声波发射器的目标驱动信号；The original driving signal is subjected to delay processing according to the delayed phase of each ultrasonic transmitter in the target ultrasonic transmitter array to obtain the target driving of each ultrasonic transmitter in the target ultrasonic transmitter array Signal;
    根据所述目标驱动信号驱动所述目标超声波发射器阵列中每个所述超声波发射器。Each of the ultrasonic transmitters in the target ultrasonic transmitter array is driven according to the target driving signal.
  20. 一种存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求13-19任一项所述的触控反馈控制方法。A storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the touch feedback control method according to any one of claims 13-19 is implemented.
PCT/CN2022/071859 2021-01-15 2022-01-13 Touch-control feedback system, terminal device, touch-control feedback control method, and storage medium WO2022152221A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202110057244.3 2021-01-15
CN202110055738.8A CN112764592B (en) 2021-01-15 2021-01-15 Touch feedback system, terminal device, touch feedback control method and storage medium
CN202110057244.3A CN112764593A (en) 2021-01-15 2021-01-15 Touch feedback control method, storage medium, touch feedback system and terminal device
CN202110055738.8 2021-01-15

Publications (1)

Publication Number Publication Date
WO2022152221A1 true WO2022152221A1 (en) 2022-07-21

Family

ID=82446938

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/071859 WO2022152221A1 (en) 2021-01-15 2022-01-13 Touch-control feedback system, terminal device, touch-control feedback control method, and storage medium

Country Status (1)

Country Link
WO (1) WO2022152221A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103858074A (en) * 2011-08-04 2014-06-11 视力移动技术有限公司 System and method for interfacing with a device via a 3d display
CN111175763A (en) * 2020-02-21 2020-05-19 吉林大学 Local ultrasonic sensor array focusing method based on multi-array synthetic aperture
CN111766949A (en) * 2020-07-16 2020-10-13 腾讯科技（深圳）有限公司 Three-dimensional image display device, display method, electronic device, and storage medium
CN112764593A (en) * 2021-01-15 2021-05-07 安徽省东超科技有限公司 Touch feedback control method, storage medium, touch feedback system and terminal device
CN112764592A (en) * 2021-01-15 2021-05-07 安徽省东超科技有限公司 Touch feedback system, terminal device, touch feedback control method and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103858074A (en) * 2011-08-04 2014-06-11 视力移动技术有限公司 System and method for interfacing with a device via a 3d display
CN111175763A (en) * 2020-02-21 2020-05-19 吉林大学 Local ultrasonic sensor array focusing method based on multi-array synthetic aperture
CN111766949A (en) * 2020-07-16 2020-10-13 腾讯科技（深圳）有限公司 Three-dimensional image display device, display method, electronic device, and storage medium
CN112764593A (en) * 2021-01-15 2021-05-07 安徽省东超科技有限公司 Touch feedback control method, storage medium, touch feedback system and terminal device
CN112764592A (en) * 2021-01-15 2021-05-07 安徽省东超科技有限公司 Touch feedback system, terminal device, touch feedback control method and storage medium

Similar Documents

Publication Publication Date Title
CN112764592B (en) Touch feedback system, terminal device, touch feedback control method and storage medium
US9830019B2 (en) Touch-sensing LCD panel
US8773399B2 (en) Touch display device
JP6131386B2 (en) Display with ultrasonic sensor array on the back
KR20180059720A (en) Display device
TWI425401B (en) Laser-optical position detecting module
CN101840089A (en) Touch control type display device
KR20190053732A (en) Electronic Device which includes Fingerprint Sensor
CN103049142A (en) Acoustic touch apparatus
TWI490753B (en) Touch control device
KR20230170876A (en) Electronic devices with enhanced display areas
WO2022138297A1 (en) Mid-air image display device
WO2022113745A1 (en) Floating-in-space-image display device
WO2022158209A1 (en) Spatial floating image display device
CN101859207B (en) Touch panel and electronic device applying same
WO2022152221A1 (en) Touch-control feedback system, terminal device, touch-control feedback control method, and storage medium
CN113324265A (en) Contactless control type electromagnetic oven and operation method and storage medium
CN214202336U (en) Touch feedback system and terminal equipment
US20130162597A1 (en) Optical touch control module
TW201606602A (en) Light guide plate optical touch device
JP7471432B2 (en) Liquid crystal module, electronic device, and screen interaction system
US11778856B2 (en) Electronic device having emissive display with light recycling
US20210036265A1 (en) Electronic Device Having Display With Internal Light Reflection Suppression
JP2022097901A (en) Space floating video display device
CN112764593A (en) Touch feedback control method, storage medium, touch feedback system and terminal device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22739097

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22739097

Country of ref document: EP

Kind code of ref document: A1