CN110493729B

CN110493729B - Interaction method and device of augmented reality device and storage medium

Info

Publication number: CN110493729B
Application number: CN201910764578.7A
Authority: CN
Inventors: 张复尧; 吕诚
Original assignee: Yutou Technology Hangzhou Co Ltd
Current assignee: Yutou Technology Hangzhou Co Ltd
Priority date: 2019-08-19
Filing date: 2019-08-19
Publication date: 2020-11-06
Anticipated expiration: 2039-08-19
Also published as: CN110493729A

Abstract

The invention relates to an interaction method, equipment and a storage medium of augmented reality equipment, wherein the method comprises the following steps: receiving settings for one or more service modules, including receiving settings for one or more associated devices and/or associated network environments of the service module, and settings for one or more associated programs of the service module; acquiring signals transmitted by devices around augmented reality equipment and/or acquiring signals of a network environment around the augmented reality equipment; and if the equipment which accords with the setting of the associated equipment and/or the network environment which accords with the setting of the associated network environment are detected, starting the corresponding service module, and executing the program corresponding to the service module according to the setting of the associated program. By using the method and the device, the service module aiming at the scene can be automatically opened or switched in the scene meeting the conditions, the equipment intelligence is increased, and better scene enhancement experience and artificial intelligence experience are brought to the user.

Description

Interaction method and device of augmented reality device and storage medium

Technical Field

The invention relates to the technical field of human-computer interaction, in particular to an interaction method and equipment of augmented reality equipment and a storage medium.

Background

The common problem of the current Augmented Reality (AR) glasses is that due to the size of the AR glasses, there are few interactive media devices between the user and the AR glasses, and the operation mode of the AR glasses is limited and the experience is poor.

For example, an interaction mode of gesture recognition based on a camera of AR glasses has a large gesture recognition error, a user needs to lift an arm for a long time to operate, and the arm is easy to fatigue after long-time operation;

based on the interactive mode of voice control AR glasses, semantic understanding errors and keyword recognition are mechanical and tedious, and meanwhile, the interaction mode can be interfered by surrounding noise;

based on the interaction mode of the control equipment such as a wireless somatosensory remote controller and a Bluetooth remote controller, extra equipment carrying burden is added to a user, and power reserve of the control equipment is required to be maintained;

due to the volume of the AR glasses, the touch area is limited, which makes it difficult to support multi-touch operations, and there are false touches and touch recognition errors.

Disclosure of Invention

The invention aims to provide a novel interaction method and equipment of an augmented reality device and a storage medium.

The purpose of the invention is realized by adopting the following technical scheme. The interaction method of the augmented reality equipment provided by the invention comprises the following steps: receiving settings for one or more service modules, including receiving settings for one or more associated devices and/or associated network environments of the service module and settings for one or more associated programs of the service module; acquiring signals transmitted by devices around augmented reality equipment and/or acquiring signals of a network environment around the augmented reality equipment; and if the equipment which accords with the setting of the associated equipment and/or the network environment which accords with the setting of the associated network environment are detected, starting the corresponding service module, and executing the program corresponding to the service module according to the setting of the associated program.

The object of the invention can be further achieved by the following technical measures.

In the interaction method for the augmented reality device, the acquiring of the signal transmitted by the device around the augmented reality device includes obtaining device characteristic information by monitoring the signal of the device around the augmented reality device, where the device characteristic information includes one or more of a device name, a device identifier, a device signal strength, a device relative distance, and a device type; the acquiring of the network environment around the augmented reality device includes acquiring network environment characteristic information by monitoring signals of the network environment around the augmented reality device, where the network environment characteristic information includes information of wireless signals and information of signal transmitters, specifically includes a wireless network name, a wireless network signal strength, MAC addresses of one or more signal transmitters transmitting the wireless network signals, and one or more of the positions of the augmented reality device acquired by using a wireless positioning technology.

The interaction method of the augmented reality device, which receives the setting of one or more associated devices of the service module, includes one or more of the following steps:

acquiring a list of Bluetooth devices paired with the augmented reality device, displaying the list of Bluetooth devices, and setting one or more Bluetooth devices selected from the list of Bluetooth devices as the associated device in response to a user selection operation;

acquiring a list of public equipment in a wireless local area network, displaying the list of the public equipment, and setting one or more public equipment selected from the list of the public equipment as the associated equipment in response to a selection operation of a user;

the method comprises the steps of obtaining a list of Wi-Fi direct equipment paired with the augmented reality equipment, displaying the list of the Wi-Fi direct equipment, responding to a selected operation of a user, and setting one or more selected Wi-Fi direct equipment in the list of the Wi-Fi direct equipment as the associated equipment.

The interaction method of the augmented reality device, wherein the receiving of the setting of one or more associated network environments of the service module includes: and receiving and recording the information of one or more signal transmitters corresponding to the service module and the setting of the information of the wireless signals sent by the signal transmitters.

In the interaction method of the augmented reality device, the receiving the setting of one or more associated devices and/or associated network environments of the service module further includes receiving the setting of start position information and/or start signal strength information of the service module; the acquiring signals transmitted by devices around an augmented reality device and/or acquiring signals of a network environment around an augmented reality device further comprises determining current location information and/or current signal strength information of the augmented reality device based on one or more of the signals transmitted by the devices around the augmented reality device and the signals of the network environment around the augmented reality device; and if the equipment which accords with the setting of the associated equipment is detected and/or the network environment which accords with the setting of the associated network environment is detected, starting the corresponding service module comprises judging whether the current position information and/or the current signal strength information accords with the setting of the starting position information and the starting signal strength information, and starting the service module which accords with the setting.

The receiving of the setting of one or more associated devices and/or associated network environments of the service module includes receiving the setting of priorities of a plurality of associated devices and/or priorities of a plurality of associated network environments; if a device conforming to the setting of the associated device is detected and/or a network environment conforming to the setting of the associated network environment is detected, the corresponding service module is started, including if a plurality of devices conforming to the setting of the associated device are detected and/or a plurality of network environments conforming to the setting of the associated network environment are detected, the associated device with the highest priority or the service module corresponding to the associated network environment is started.

In the interaction method for the augmented reality device, if a device meeting the setting of the associated device is detected and/or a network environment meeting the setting of the associated network environment is detected, the starting of the corresponding service module includes one or more of the following steps:

if the equipment which is based on the long-distance wireless signal and the equipment which is based on the short-distance wireless signal and accords with the setting of the associated equipment/the network environment which accords with the setting of the associated network environment are detected simultaneously, the service module which corresponds to the associated equipment/the associated network environment which is based on the long-distance wireless signal is started preferentially;

if the equipment which is based on the wireless WiFi and the Bluetooth and accords with the setting of the associated equipment is detected at the same time, the service module corresponding to the associated equipment based on the wireless WiFi is started preferentially;

if the equipment according with the setting of the associated equipment and the network environment according with the setting of the associated network environment are detected at the same time, the service module corresponding to the associated network environment is started preferentially.

The interaction method of the augmented reality device further includes: when a first service module is in a starting state, continuing to monitor signals transmitted by devices around the augmented reality device and/or signals of a network environment around the augmented reality device, wherein the first service module is the service module corresponding to a first associated device or a first associated network environment; if a device meeting the setting of a second associated device or a network environment meeting the setting of a second associated network environment is detected, reacting to the second associated device/the second associated network environment if the first service module contains the setting of a rule for the second associated device/the second associated network environment.

In the interaction method of the augmented reality device, the setting of the one or more associated programs of the service module includes an identifier of the associated program corresponding to the service module and one or more to-be-executed behaviors of the associated program, which are selected by a user; the executing the program corresponding to the service module according to the setting of the associated program includes executing the to-be-executed behavior corresponding to the service module.

In the interaction method of the augmented reality device, the receiving of the setting of one or more associated programs of the service module further includes receiving a setting of an execution sequence and/or an execution state of a plurality of associated programs or a plurality of to-be-executed behaviors of one associated program; the execution state comprises an inquiry attribute, and is used for inquiring a user before the associated program or the behavior to be executed with the inquiry attribute is executed, and judging whether to execute the associated program or the behavior to be executed according to the response of the user to the inquiry; the executing the program corresponding to the service module according to the setting of the associated program includes executing the to-be-executed behavior of the associated program corresponding to the service module according to the execution order and/or the setting of the execution state.

In the interaction method of the augmented reality device, the receiving the setting for the one or more service modules further includes: receiving a setting for one or more alternative service modules associated with one of the service modules and a setting for an association priority between a plurality of the alternative service modules; the method further comprises the following steps: displaying the alternative service modules and arranging the alternative service modules according to the associated priorities; and responding to the selection operation of the user, and switching from the current service module to the alternative service module selected by the user.

In the interaction method of the augmented reality device, the action to be executed includes: program open, perform voice query, present augmented reality information, program pause, program resume, program close.

The object of the present invention is also achieved by the following technical means. According to the invention, an apparatus is proposed, comprising: a memory for storing non-transitory computer readable instructions; and a processor for executing the computer readable instructions such that the computer readable instructions, when executed by the processor, implement: receiving settings for one or more service modules, including receiving settings for one or more associated devices and/or associated network environments of the service module and settings for one or more associated programs of the service module; acquiring signals transmitted by devices around augmented reality equipment and/or acquiring signals of a network environment around the augmented reality equipment; and if the equipment which accords with the setting of the associated equipment and/or the network environment which accords with the setting of the associated network environment are detected, starting the corresponding service module, and executing the program corresponding to the service module according to the setting of the associated program.

The foregoing apparatus, when executed by the processor, performs: the acquiring of the signal transmitted by the device around the augmented reality device includes obtaining device feature information by monitoring the signal of the device around the augmented reality device, where the device feature information includes one or more of a device name, a device identifier, a device signal strength, a device relative distance, and a device type; the acquiring of the network environment around the augmented reality device includes acquiring network environment characteristic information by monitoring signals of the network environment around the augmented reality device, where the network environment characteristic information includes information of wireless signals and information of signal transmitters, specifically includes a wireless network name, a wireless network signal strength, MAC addresses of one or more signal transmitters transmitting the wireless network signals, and one or more of the positions of the augmented reality device acquired by using a wireless positioning technology.

The foregoing apparatus, when executed by the processor, performs: the receiving settings for one or more associated devices of the service module comprises one or more of the following steps:

The foregoing apparatus, when executed by the processor, performs: the receiving settings for one or more associated network environments of the service module comprises: and receiving and recording the information of one or more signal transmitters corresponding to the service module and the setting of the information of the wireless signals sent by the signal transmitters.

The foregoing apparatus, when executed by the processor, performs: the receiving settings for one or more associated devices and/or associated network environments of the service module further comprises receiving settings for activation location information and/or activation signal strength information of the service module; the acquiring signals transmitted by devices around an augmented reality device and/or acquiring signals of a network environment around an augmented reality device further comprises determining current location information and/or current signal strength information of the augmented reality device based on one or more of the signals transmitted by the devices around the augmented reality device and the signals of the network environment around the augmented reality device; and if the equipment which accords with the setting of the associated equipment is detected and/or the network environment which accords with the setting of the associated network environment is detected, starting the corresponding service module comprises judging whether the current position information and/or the current signal strength information accords with the setting of the starting position information and the starting signal strength information, and starting the service module which accords with the setting.

The foregoing apparatus, when executed by the processor, performs: the receiving settings for one or more associated devices and/or associated network environments of the service module comprises receiving settings for priorities of a plurality of the associated devices and/or priorities of a plurality of the associated network environments; if a device conforming to the setting of the associated device is detected and/or a network environment conforming to the setting of the associated network environment is detected, the corresponding service module is started, including if a plurality of devices conforming to the setting of the associated device are detected and/or a plurality of network environments conforming to the setting of the associated network environment are detected, the associated device with the highest priority or the service module corresponding to the associated network environment is started.

The foregoing apparatus, when executed by the processor, performs: if a device conforming to the setting of the associated device is detected and/or a network environment conforming to the setting of the associated network environment is detected, the starting of the corresponding service module includes one or more of the following steps:

if the equipment which is based on the long-distance wireless signal and the equipment which is based on the short-distance wireless signal and accords with the setting of the associated equipment/the network environment which accords with the setting of the associated network environment are detected simultaneously, a service module which corresponds to the associated equipment/the associated network environment which is based on the long-distance wireless signal is started preferentially;

The aforementioned apparatus, when executed by the processor, causes the apparatus to further perform the steps of: when a first service module is in a starting state, continuing to monitor signals transmitted by devices around the augmented reality device and/or signals of a network environment around the augmented reality device, wherein the first service module is the service module corresponding to a first associated device or a first associated network environment; if a device meeting the setting of a second associated device or a network environment meeting the setting of a second associated network environment is detected, reacting to the second associated device/the second associated network environment if the first service module contains the setting of a rule for the second associated device/the second associated network environment.

The foregoing apparatus, when executed by the processor, performs: the setting of the one or more associated programs of the service module comprises an identification of the associated program corresponding to the service module and one or more to-be-executed behaviors of the associated program, which are selected by a user; the executing the program corresponding to the service module according to the setting of the associated program includes executing the to-be-executed behavior corresponding to the service module.

The foregoing apparatus, when executed by the processor, performs: the receiving of the setting of the one or more associated programs of the service module further includes receiving a setting of an execution sequence and/or an execution state of a plurality of the associated programs or a plurality of the to-be-executed behaviors of one associated program; the execution state comprises an inquiry attribute, and is used for inquiring a user before the associated program or the behavior to be executed with the inquiry attribute is executed, and judging whether to execute the associated program or the behavior to be executed according to the response of the user to the inquiry; the executing the program corresponding to the service module according to the setting of the associated program includes executing the to-be-executed behavior of the associated program corresponding to the service module according to the execution order and/or the setting of the execution state.

The foregoing apparatus, when executed by the processor, performs: the receiving settings for one or more service modules further comprises: receiving a setting for one or more alternative service modules associated with one of the service modules and a setting for an association priority between a plurality of the alternative service modules; the method further comprises the following steps: displaying the alternative service modules and arranging the alternative service modules according to the associated priorities; and responding to the selection operation of the user, and switching from the current service module to the alternative service module selected by the user.

The foregoing apparatus, when executed by the processor, performs: the behavior to be executed comprises: program open, perform voice query, present augmented reality information, program pause, program resume, program close.

The object of the present invention is also achieved by the following technical means. The invention provides a computer storage medium, which includes computer instructions, when the computer instructions are executed on a device, the device executes the interaction method of any one of the possible augmented reality devices in any one of the above aspects.

The object of the present invention is also achieved by the following technical means. According to the invention, a computer program product is proposed, which, when run on a device, causes the device to perform the interaction method of any one of the above-mentioned aspects of the possible augmented reality device.

Compared with the prior art, the invention has obvious advantages and beneficial effects. By the technical scheme, the interaction method, the interaction equipment and the storage medium of the augmented reality equipment provided by the invention at least have the following advantages and beneficial effects:

(1) the method and the system can reduce the necessary operation of the user on the AR glasses equipment, automatically open or switch the service module aiming at the scene in the scene meeting the conditions, increase the equipment intelligence and bring better scene enhancement experience and artificial intelligence experience for the user;

(2) according to the method and the device, the position of the user is judged by using the peripheral wireless signals, and the corresponding service module is started and switched according to the specific voice name of the user, so that the equipment intelligence can be increased, and better scene enhancement experience and artificial intelligence experience are brought to the user;

(3) the user can customize the service module of the invention, and the intelligence, convenience and humanization of the equipment are improved by setting the elements of the associated equipment, the associated network environment, the associated program behavior, the associated alternative service module and the like of the service module, so that better scene enhancement experience and artificial intelligence experience are brought to the user;

(4) the invention sets the default of the service module to be that the priority of the long-distance wireless signal is greater than that of the short-distance wireless signal, the priority of the wireless WiFi is greater than that of the Bluetooth, and the priority of the associated network environment is greater than that of the associated equipment, thereby conforming to the practical application and bringing better scene enhancement experience and artificial intelligence experience for users.

The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical means of the present invention more clearly understood, the present invention may be implemented in accordance with the content of the description, and in order to make the above and other objects, features, and advantages of the present invention more clearly understandable, the following preferred embodiments are described in detail with reference to the accompanying drawings.

Drawings

Fig. 1 is a flow chart diagram of an interaction method of an augmented reality device according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating a provisioning service module according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a specific example of an interaction method of an augmented reality device according to an embodiment of the present invention;

FIG. 4 is a schematic block diagram of an apparatus according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an apparatus according to an embodiment of the present invention.

Detailed Description

To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined objects, the following detailed description will be given of specific embodiments, structures, features and effects of an augmented reality device interaction method, an augmented reality device and a storage medium according to the present invention with reference to the accompanying drawings and preferred embodiments.

Fig. 1 is a schematic flow chart diagram of an embodiment of an interaction method of an augmented reality device according to the present invention. Fig. 2 is a schematic diagram of a provisioning service module according to an embodiment of the present invention. Fig. 3 is a schematic diagram illustrating a specific example of an interaction method of an augmented reality device according to the present invention. Referring to fig. 1, fig. 2 and fig. 3, the interaction method of the augmented reality device of the present invention mainly includes the following steps, referring to fig. 1, the interaction method of the augmented reality device of the present invention mainly includes the following steps:

in step S11, settings for one or more service modules are received.

The service module provided by the invention is also called a scene service module, is a user-definable module, can be triggered according to the starting condition set by the user, and executes corresponding operation according to the setting of the user. The setting of the service module is also referred to as the service module definition.

Optionally, the setting of the service module is used for performing subsequent processing in real time after the receiving, or the setting of the service module is recorded after the receiving, so that the setting of the service module is read in the subsequent processing.

In some examples, the service module contains one or more elements including: one or more of associated devices, associated network environments, and associated program behavior, which may all be user-defined. In addition, the service module may further include other custom elements according to product requirements and user requirements, for example, further including settings of associated alternative service modules.

As an alternative embodiment, the step S11 specifically includes: settings for one or more associated devices and/or associated network environments of the service module and settings for one or more associated programs of the service module are received.

The setting of the associated device is also referred to as an associated device definition, and is used to define a device corresponding to the service module as the associated device, and generally, the associated device is a wireless device.

The setting of the associated network environment is also referred to as an associated network environment definition, and is used to define a network environment corresponding to the service module as an associated network environment.

The setting of the associated program, also referred to as associated program behavior definition, includes: setting one or more programs corresponding to one service module as associated programs; and setting the behavior to be executed of each associated program of one service module when the service module is started. The associated program may be a foreground application program or a background service program, and thus, the setting of the associated device may also be referred to as application and background service behavior definition.

In some examples, setting the associated device and the associated network environment is used to determine the scene in which the AR device is located, which is in fact the setting of the start condition of the service module.

Step S12, acquiring signals emitted by devices around the augmented reality device, and/or acquiring signals of a network environment around the augmented reality device.

The acquiring of the signal transmitted by the device around the augmented reality device specifically includes monitoring a wireless signal transmitted by the device around the augmented reality device, for example, a 2.4Ghz wireless network signal, a 5.1Ghz wireless network signal, a bluetooth wireless signal, and the like. The acquiring of the signal of the network environment around the augmented reality device specifically includes monitoring a wireless signal emitted by one or more signal emitters of the network environment around the augmented reality device. In some examples, the aforementioned devices are devices that have the ability to transmit certain wireless signals.

Note that in this step, only the device may be monitored; it is also possible to monitor only the network environment; or may monitor both the device and the network environment. Optionally, in some examples where both the device and the network environment are monitored, even if there is only one of the two in the current scenario, the system does not stop detecting the other.

In step S13, if it is detected that a device that meets the setting of the relevant device exists around the augmented reality device and/or it is detected that the network environment around the augmented reality device is a network environment that meets the setting of the relevant network environment, the corresponding service module is started, and the program corresponding to the service module is executed according to the setting of the relevant program.

In some examples, the detecting of the presence of the device around the augmented reality device in this step that conforms to the setting of the associated device includes: a wireless signal emitted by an associated device is detected.

Note that, in this step, the case of detecting that the setting is met may be that the associated device is detected, that the associated network environment is detected, or that the associated device and the associated network environment are detected at the same time.

In some examples, the associated alternative service modules may be prepared after their own priorities according to predefined rules among the plurality of service modules. Execution of a certain service module may cause the priority of the associated other service module to be changed.

Augmented reality devices (also referred to as AR devices) referred to in the present invention include, but are not limited to, AR eyewear devices. AR eyewear devices include, but are not limited to: the integrated AR glasses are internally provided with a related computing module, a wireless network module, a Bluetooth module and the like; the split type AR glasses are provided with an external computing module, a wireless network module, a Bluetooth module and the like; external expansion display AR glasses, this type of AR glasses rely on user's smart mobile phone, intelligent panel computer, notebook computer, personal desktop computer etc. as calculation module, wireless network module, bluetooth module etc..

In some embodiments of the present invention, the acquiring signals transmitted by devices around the augmented reality device in step S12 specifically includes: the device characteristic information is obtained by monitoring signals of devices around the augmented reality device. The device feature information includes one or more of device name, device identifier, device signal strength, device relative distance, device type, and the like. Optionally, only signals of paired associated devices around the augmented reality device are monitored, and device characteristic information of the associated devices is obtained.

In some embodiments of the present invention, the acquiring the signal of the network environment around the augmented reality device in the foregoing step S12 includes: and obtaining network environment characteristic information by monitoring signals of the network environment around the augmented reality equipment. The network environment characteristic information includes information of wireless network signals and information of signal emitters, specifically including one or more of wireless network name, wireless network signal strength, MAC address of one or more signal emitters sending wireless network signals, and location of augmented reality equipment obtained by using wireless positioning technology.

In some embodiments of the present invention, the aforementioned step S12 is continued. Further, when one of the service modules is in the activated state, the monitoring of the signals transmitted by the devices around the augmented reality device and/or the signals of the network environment around the augmented reality device in step S12 is continued.

It is to be noted that in the monitoring process of step S12 or further in the positioning by wireless signals, the augmented reality device does not have to maintain a connection state with the associated device, and does not have to log into the associated network environment, but rather, monitors and positions by borrowing wireless signals from devices, signal transmitters, and the like.

It should be noted that the present invention is not limited to the type of signal, and is generally a wireless network signal, including but not limited to a wireless fidelity signal (or referred to as a Wi-Fi signal), a Bluetooth signal (or referred to as a Bluetooth signal), a ZigBee signal (or referred to as a ZigBee signal), or any other wireless signal.

In some examples, the user is allowed to name individual service modules, such as multiple service modules named "office scenario 1", "office scenario 2", and so forth.

(one) to associated devices and associated network environments.

The setting of the associated device, or referred to as the associated device definition, is used to describe one or more wireless devices with which the service module is associated.

The associated device is a device that transmits wireless signals, including devices that transmit short-range wireless signals such as bluetooth devices. And the user's AR device will interact with the associated device directly via this wireless signal.

The devices mentioned in the present invention include but are not limited to: the system comprises a Bluetooth keyboard, a Bluetooth mouse, Bluetooth sound equipment, a wireless printer, a wireless scanner, wireless intelligent household Internet of things equipment and the like, and can also comprise other wireless equipment which can be used by AR equipment used by a user to detect wireless signals. The user can customize the associated device. When the AR glasses or the related wireless network module of the user detects that the defined related equipment exists around the user, the corresponding service module is automatically activated or switched or the application and background service behaviors of the related service module are automatically executed.

In some embodiments of the invention, the setting of the associated device comprises: and setting and recording the identification of the associated equipment.

In some embodiments of the present invention, the receiving of the setting of the associated device of the one or more service modules in the foregoing step S11 specifically includes one or more of the following steps:

the method comprises the steps of obtaining a list of Bluetooth devices paired with augmented reality devices, displaying the list of the Bluetooth devices to a user, responding to the operation of a selected associated device of the user, recording the identification of the selected device of the user, and setting one or more Bluetooth devices selected from the list of the Bluetooth devices as the associated devices;

acquiring a list of public devices in a wireless local area network (Wi-Fi local area network), such as a printer, a projector and the like, displaying the list of public devices to a user, responding to the operation of the selected associated device of the user, and recording the identification of the selected device of the user so as to set one or more public devices selected from the list of public devices as the associated devices;

the method comprises the steps of obtaining a list of Wi-Fi direct equipment paired with augmented reality equipment, displaying the list of the Wi-Fi direct equipment to a user, responding to the operation of selected associated equipment of the user, recording the identification of the selected equipment of the user, and setting one or more selected Wi-Fi direct equipment in the list of the Wi-Fi direct equipment as associated equipment.

Further, based on the three wireless device adding rules, the associated device definition allows the user to simultaneously add a wireless device, which is detected by an AR device that can be used by the user, such as a bluetooth device, a public device in a wireless local area network, and a Wi-Fi direct device, to the associated device definition as an activation device for activating the service module.

Alternatively, the associated device is set by recording one or more of the id, mac address, name, etc. of the selected device, which can distinguish the identifications of the different devices. In step S13, it is determined whether a device is an associated device based on the identification.

It should be noted that the device equipped with bluetooth 4.0 starts to support intelligent connection, and the devices can actively connect after the devices are paired, and can complete various data transmission. The device in the Wi-Fi local area network can present a device name, a device MAC address and a binding fixed local area network IP address. The user only needs to carry out a Wi-Fi local area network distribution once, and can automatically link to the Wi-Fi local area network when entering the coverage range of the Wi-Fi local area network next time.

The setting of the associated network environment, or referred to as associated network environment definition, is used to describe the wireless network environment associated with the service module.

The associated wireless network environment means: there are typically multiple wireless networks in a physical environment, and depending on the user's settings, when an AR device enters a wireless network named "a," some action is taken. But there is still a wireless network named "B" in the physical environment, for which the system will not take any action if the user does not make any rule definitions.

The wireless networks mentioned in the present invention include but are not limited to: a wireless personal area network (or referred to as WPAN) such as bluetooth, a wireless local area network (or referred to as WLAN) such as Wi-Fi, and the like. The user can customize the network types corresponding to the service module requirements.

In some embodiments of the invention, the setting of the associated network environment comprises: setting and recording the identification of the associated network environment.

It should be noted that the associated network environment definition may include not only recording the network name of the network environment or the MAC address of the router sending the Wi-Fi signal, but also additional value-added services such as a location service provided by applying the network environment. When the AR glasses or the related wireless network module of the user detect the defined network environment, the corresponding service module is automatically activated or switched or the application and background service behaviors of the related service module are automatically executed.

In a physical environment, such as a shopping mall or an office building, a network environment is defined as the integration of information of all wireless signal transmitters in the physical environment and information of wireless signals transmitted by the wireless signal transmitters. The wireless signal transmitter includes one or more of Wi-Fi routing, bluetooth low energy transmitter (or called beacon), etc. The information of the wireless signal transmitter mainly comprises: one or more network names defined by the signal transmitters, and the MAC address and physical location address of each transmitter, etc. The information of the wireless signal mainly includes: the strength value of the wireless signal sent by each transmitter received by the user at a location in the physical environment may also include other information such as the frequency of each wireless signal. Note that one wireless network may be formed of a plurality of signal transmitters, and thus one wireless network may correspond to a plurality of signal transmitters.

In some embodiments of the present invention, the receiving, in step S11, the setting of the associated network environment for the one or more service modules specifically includes: and receiving and recording the information of one or more signal transmitters corresponding to the service module and the setting of the information of the wireless signals sent by the signal transmitters. Wherein the information of the signal transmitter includes one or more of a network name set by the signal transmitter, a network address of the signal transmitter, and a physical location of the signal transmitter. The information of the wireless signals comprises one or more of the strength value of the wireless signals sent by each signal transmitter and the frequency of the wireless signals received by the user at a certain position in the physical environment.

It is noted that logging into a network environment and entering into a network environment are two concepts. Logging into a network environment means network connecting and logging into the network environment, also referred to as joining the network environment. And entering a network environment means physically entering the network environment. For example, a user walks into a company where a plurality of wireless routers are located, which send out wireless signals, and the different signal strengths of their respective devices make up a particular wireless network environment. The user does not need to be specially added into a specific certain network, and the wireless network positioning can collect all signal intensities around you without the fact whether the user is added into the wireless network or not.

In some embodiments of the present invention, the receiving of the setting of the associated device and/or the associated network environment of the service module in the foregoing step S11 further includes: settings for activation location information and/or activation signal strength information for the service module are received. Optionally, the start location information and the start signal strength information are relative location relationships and signal strength relationships between the augmented reality device and the associated device or the associated network environment. The aforementioned step S12 further includes: the augmented reality device is positioned based on one or more of signals transmitted by devices surrounding the augmented reality device, signals of a network environment surrounding the augmented reality device, current location information and/or current signal strength information of the augmented reality device is determined. The current location information and/or the current signal strength information of the augmented reality device is a location relationship/signal strength relationship between the current augmented reality device and surrounding devices and/or network environments. The aforementioned step S13 includes: and judging whether the current position information and/or the current signal strength information accords with the setting of the starting position information and/or the starting signal strength information, and starting the service module which accords with the setting. Specifically, if it is detected that at least one device or network environment meets the identifier of the associated device or the associated network environment, and the current location relationship/the current signal strength between the augmented reality device and the associated device and/or the associated network environment meets the setting of the start location information/the start signal strength information, the service module corresponding to the associated device or the associated network environment meeting the setting of the start location information/the start signal strength information is started.

Optionally, the network environment, the device is utilized to locate. For example, an approximate location of the augmented reality device may be obtained using signals of the network environment, and the location of the augmented reality device may be corrected using signals of the device at the same time to obtain an accurate location of the augmented reality device using multiple positioning results at the same time.

It is to be noted that the present invention is not limited to the type of the location information, and for example, the aforementioned start location information includes one or more of a specific location point, a location range, and distance information such as a distance threshold. The present invention is not limited in the type of signal strength information, for example, the activation signal strength information includes one or more of a signal strength threshold with the device, a signal strength threshold with a signal transmitter of the network environment.

In fact, the positional relationship of the AR glasses (and their accessories) with a single device may be detected by wireless signals transmitted by the device. Alternatively, the received signal strength (or as RSSI value) may be utilized to calculate the distance between the associated device and the augmented reality device. The calculation formula is as follows:

d＝10^((abs(RSSI)-A)/(10*n))，

where d is the resulting distance, RSSI is the received signal strength, abs is a function of the absolute value, a is the signal strength at a distance of 1 meter from the transmitting device to the receiving device, and n is the ambient attenuation factor.

It should be noted that, in the example of determining the distance by using the RSSI according to the present invention, the above formula is not necessarily used, and other formulas may also be used, and the calculated distance result may also be optimized by additional algorithms, such as calculating the average value for multiple times, etc. Also, the present invention does not limit the use of the received signal strength RSSI to determine the distance.

It should be noted that, in some embodiments, the bluetooth device of the present invention is a bluetooth device using bluetooth system Class B, an effective communication distance of the bluetooth device is about 8 to 30 meters, and a typical distance is 10 meters, and a signal coverage radius of the bluetooth device is short, so that signal quality of the bluetooth device generates a significant signal loss within a short distance, and relative location information between a user and an associated bluetooth device can be determined more accurately by determining the quality of a bluetooth signal emitted by the bluetooth device.

Note that all wireless signals detectable by the AR device are available and usable. In some examples only, the enabling location information or enabling signal strength information is used to filter those wireless signals whose strength meets the algorithm requirements when passing through the algorithm layer, and in fact can reflect whether the AR device is close to the associated device or enters the associated network environment based on whether the signal meets the strength requirements. So that a wireless signal meeting the strength requirement will become truly valid for the system. When the AR glasses and the associated equipment reach the starting range, the module is started or the behavior is executed. Therefore, the activation position and the activation signal strength may be referred to as an effective position and an effective signal strength.

In some embodiments of the invention, different service modules are launched depending on the positioning of the augmented reality device. Specifically, the foregoing step S11 further includes: settings for one or more selected locations are received. The aforementioned step S12 includes: determining the position of the augmented reality device by utilizing a wireless positioning technology based on one or more of signals transmitted by devices around the augmented reality device and signals of a network environment around the augmented reality device. The aforementioned step S13 includes: and judging the selected position corresponding to the position of the augmented reality equipment, and starting the service module corresponding to the selected position.

In some embodiments of the present invention, if multiple service modules with different start location information/start signal strength information are associated with the same device/network environment, when the augmented reality device is worn for movement and the location relationship with the associated device/signal strength changes, or when the location within the associated network environment changes/the signal strength with the signal transmitter changes, different service modules are automatically started to perform different actions.

It should be noted that the present invention is not limited to the wireless location technology used. For example, determining a relative location of the augmented reality device based on relative signal strengths of one or more associated devices and the augmented reality device may be by the associated devices; or positioning the augmented reality device based on the wireless signal fingerprint by using the associated wireless network environment; or determining a relative positional relationship of the augmented reality device to a plurality of signal transmitters in an associated network environment; and the positioning can be carried out by integrating equipment and a network environment.

According to the invention, the position of the user is judged by using the peripheral wireless signals, and the corresponding service module is started and switched according to the specific voice name of the user, so that the equipment intelligence can be increased, and better scene enhancement experience and artificial intelligence experience are brought to the user.

After a user adds an associated device, the associated device may be prioritized relative to other associated devices. And when the AR equipment is connected with a plurality of associated equipment simultaneously and all the equipment accords with the starting position information, the service module corresponding to the equipment with high priority is activated preferentially. This definition may be user-default, i.e., a preset default priority may be utilized. When the priority is not defined, the priority of the devices may be assigned according to the priority of the added devices, or the priority may be assigned randomly, and so on. But associated devices that have not defined priorities will be lower in priority than associated devices that have defined priorities.

In some embodiments of the present invention, the receiving of the setting of the associated device and/or the associated network environment of the service module in the foregoing step S11 further includes: settings for priorities of a plurality of associated devices and/or priorities of a plurality of associated network environments are received. The priority of the associated device is not referred to as a device priority, and the priority of the associated network environment is referred to as a network environment priority. The aforementioned step S13 includes: and if a plurality of devices which accord with the setting of the associated device and/or a plurality of network environments which accord with the setting of the associated network environment are detected, starting the associated device with the highest priority or a service module corresponding to the associated network environment.

Further, in some embodiments of the present invention, the aforementioned step S13 includes: if it is detected that the plurality of devices conform to the setting of the associated device, or the plurality of network environments conform to the setting of the associated network environment, or the one or more devices conform to the setting of the associated device and the one or more network environments conform to the setting of the associated network environment, by: comparing one or more of a relative signal strength between the associated device and/or the associated network environment and the augmented reality device, a relative distance between the associated device and/or the associated network environment and the augmented reality device, and a priority of the associated device and/or the associated network environment, and synthesizing a plurality of conditions to determine a service module to be started.

As an alternative example, the modules and applications are switched by comparing the relative signal strengths of a plurality of wireless devices, so as to perform the starting and switching of the service module by means of the peripheral wireless network environment and the peripheral wireless devices.

As an alternative example, when the AR glasses device is connected to a plurality of associated wireless devices at the same time, if the user defines the relative distance of the devices in advance, the corresponding service module will be activated preferentially, and when a plurality of associated wireless devices satisfy respective relative distance settings at the same time, the service module corresponding to the associated device with the highest priority is activated.

In some embodiments where the service module is activated with a default priority, the priority of the long range wireless signal is greater than the priority of the short range wireless signal, the priority of the wireless WiFi is greater than the bluetooth, and the priority of the associated network environment is greater than the priority of the associated device.

In some embodiments of the invention, the aforementioned step S13 includes one or more of the following steps:

if the equipment which is based on the long-distance wireless signal and the equipment which is based on the short-distance wireless signal and accords with the setting of the associated equipment/the network environment which accords with the setting of the associated network environment are detected simultaneously, the service module which corresponds to the associated equipment/the associated network environment which are based on the long-distance wireless signal is started preferentially;

if the equipment which accords with the setting of the associated equipment and the network environment which accords with the setting of the associated network environment are detected at the same time, the service module corresponding to the associated network environment is started preferentially.

Note that the associated network environment definition is not the same as the associated device definition, and the main difference is that: the associated device definition is a 'point-to-point' service relationship, for example, the wireless signal transmitted by a single device is used for detecting the position relationship between the AR glasses (and the accessory devices thereof) and the device, so as to activate the corresponding service; the associated network environment is defined as a "point-to-surface" service relationship, for example, the AR glasses (and the attached devices thereof) use the wireless positioning technology to know the position of the AR glasses in the physical environment by integrating the wireless signals transmitted by the multiple devices, so as to activate the corresponding service according to the current position of the user. For example, in the case of an associated device definition, music is automatically played when the AR glasses detect that they are in the vicinity of one bluetooth sound; in the case of an associated network environment, when the AR glasses detect that they are in one office through wireless location technology, the lights of the office are automatically turned on. Note that in this example, there is no wireless signal transmitter on the lamp, so instead of turning on the lamp by detecting that the AR glasses are close to the lamp, the AR glasses are detected to be in the office.

In some optional embodiments of the present invention, the setting of one service module includes setting of one or more associated devices and setting of one or more associated network environments. Further, the relationship between the associated device that starts the service module and the associated network environment may be set, including setting: the service module is started when the associated device and the associated network environment are detected simultaneously, or the service module can be started when one of the associated device and the associated network environment is detected. In step S13, it is determined whether or not the start condition of the service module is satisfied based on the setting. For example, as shown in scenario D and service module D in fig. 3, in step S11, the settings of the associated network environment and the settings of the associated device of one service module may be defined as: and judging that the user enters a study room according to the wireless indoor positioning, judging that the user approaches the Bluetooth keyboard to a preset distance threshold value according to a signal transmitted by the Bluetooth keyboard, and starting the service module if the preset distance threshold value is met.

In some embodiments of the present invention, in the aforementioned step S11, the setting of one service module associated with one device/network environment further includes a rule for another device/network environment. As an optional example, the setting of one service module may include: when the service module is started, if the selected associated equipment/associated network environment is monitored, the service module is automatically switched to the corresponding other service module. As an optional example, the setting of one service module may include: when the service module is started, if the selected equipment/network environment is monitored, the selected program behavior is triggered.

In some embodiments of the present invention, the interaction method of the augmented reality device of the examples of the present invention further includes the steps of:

when the first service module is in a starting state, continuously monitoring signals transmitted by equipment around the augmented reality equipment and/or continuously monitoring signals of a network environment around the augmented reality equipment; the first service module is a service module corresponding to the first associated equipment or the first associated network environment;

if a device conforming to the settings of the second associated device is detected or a network environment conforming to the settings of the second associated network environment is detected,

reacting to the second associated device/second associated network environment if the first service module contains a setting of a rule for the second associated device/second associated network environment.

For example, if a signal from a bluetooth keyboard is detected and a user has previously defined a rule such as opening a browser or opening a text editor, then the action is initiated when the bluetooth keyboard is detected and the signal strength exceeds a certain threshold.

As a specific example, if a user is in the coverage of a corporate wireless network, the first service module is opened according to the preset of the user when the preset is met, and the priority of the second service module, the third service module, and the like is set to be next to the priority of the first service module, so that the user can conveniently and rapidly switch among the several modules manually. When a user approaches an associated device, such as a bluetooth keyboard, the first service module reacts to the bluetooth keyboard if it contains a rule definition for the bluetooth keyboard, and does not react if not. And if the definition exists in the second service module or the third service module, executing the defined behavior immediately after the user manually switches the modules.

And (II) relating to the association procedure.

The setting of the associated program, or referred to as associated program behavior definition, or application and background service behavior definition, is used to describe: when the service module is started, which foreground application programs and background service programs are added to the service module and executed in what sequence and state. Application and background service behavior definitions can be customized by the user. For example, one or more applications specified by the user are automatically launched, or one or more background services specified by the user are installed, in a sequence specified by the user. When the service module is executed, the execution is carried out in sequence according to the behavior sequence of the application and the background service recorded by the application and background service behavior definition until all behaviors are finished.

Note that the foregoing programs include foreground application programs and/or background service programs, and may be programs inside the augmented reality device, or programs of other devices that can be controlled.

Specifically, in some embodiments of the present invention, the setting of the one or more association programs for the service module in the foregoing step S11 includes: an identification, such as a name, ID, etc., of the associated program corresponding to the service module selected by the user, and one or more behaviors of the associated program to be executed. The behavior to be executed is a behavior executed when the service module is started, and may also be referred to as a behavior or an action. The execution of the program corresponding to the service module according to the setting of the associated program in step S13 includes: and executing one or more to-be-executed behaviors of one or more associated programs corresponding to the service module.

It should be noted that a program may correspond to one or more actions, such as one or more of program open, executing voice query, presenting augmented reality information, program pause, program resume, program close, and so on. Taking the example of playing music with an augmented reality device, the actions to be performed may include: one or more of opening a music player, presenting music information in augmented reality form, performing a voice query, pausing playback based on interaction with a user, continuing playback, and the like. When setting the associated program, a plurality of program behavior definitions may be performed on a program, which includes some basic behaviors such as basic on/off and extended behaviors related to a functional interface provided by a program. E.g., how to turn on a voice, how to turn on a particular function interface, etc. It should be noted that the present invention is not limited to the type of program behavior that can be set, and may be any behavior that can be executed by the corresponding program.

In fact, the setting of the associated program can be regarded as the setting of one or more behaviors for one or more programs, which is also called as associated program behavior definition, or as application and background service behavior definition.

As a specific example of guiding the user to perform the setting of the association program, in some embodiments of the present invention, the receiving of the setting of the one or more association programs for the service module in the foregoing step S11 includes: the method comprises the steps of obtaining a list of programs which can be controlled by the augmented reality device, displaying the program list to a user, and setting one or more programs selected from the program list as associated programs in response to the operation of the selected associated programs of the user.

Further, in some embodiments of the present invention, the receiving of the setting of the one or more association programs for the service module in the step S11 further includes: the method comprises the steps of receiving setting of an execution sequence and/or an execution state of a plurality of associated programs or a plurality of to-be-executed behaviors of one associated program. The execution of the program corresponding to the service module according to the setting of the associated program in step S13 includes: and executing the to-be-executed behaviors of the associated programs corresponding to the service modules according to the setting of the execution sequence and/or the execution state.

The execution sequence is an execution sequence of a plurality of program behaviors associated with the service module, and may also be regarded as a priority of the program behaviors. For example, in one example, when a service module is started, a user experience (abbreviated as "UX") of multiple windows needs to present multiple contents to a user at the same time, but in fact, the presentation process is performed in sequence, and only after the actions are completed in a moment, the user cannot perceive the actions, so that the execution sequence needs to be set.

The execution state is a logic state for further determining the execution sequence of the application and the background service in the service module. In the setting process of the associated program, a user can set one or more execution states for an application program or a background service. Optionally, the execution state includes an inquiry attribute, and is configured to inquire, before the associated program/to-be-executed behavior with the inquiry attribute is executed, whether to execute the associated program/to-be-executed behavior from the user, and determine whether to further execute the associated program/to-be-executed behavior according to a reply to the inquiry from the user. Thus, the user can be provided with functions such as interrupting the execution of the program, skipping the execution of a certain action, and the like, by using the execution state such as inquiring about the attribute.

Note that the service module itself is composed of multiple applications and background service execution behaviors. Opening a service module necessarily entails execution of the corresponding rule. Even if the service module itself does not contain any behavior, once invoked it means that the service module is faithfully executed.

It is noted that while in some examples the associated program of a service module is a program related to the associated device, associated network environment, to which the service module corresponds, it need not be a related program, and indeed the associated program may be any program that the AR device is capable of invoking.

And (III) the associated alternative service module.

The setting of the alternative service module, or referred to as alternative service module definition, is used to describe the association relationship between the service module and the service modules of other scenarios. One service module may be associated with one or more alternative service modules and may define priorities between the alternative service modules. Alternative service modules associated with a service module may be user-customizable. The definition has the effect of establishing the association relationship between different service modules and helping a user to quickly and manually switch from the service module to other associated service modules, wherein the alternative service modules with higher priority are arranged at the selection positions which are more front. If the priority of the alternative service module is not set, the priority of the alternative service module can be determined according to the adding sequence of the alternative service module.

Specifically, the receiving of the setting for one or more service modules in the step S11 further includes: settings for one or more alternative service modules associated with one service module and settings for an association priority between a plurality of alternative service modules are received. Moreover, the method further comprises the following steps: displaying the alternative service modules and arranging the alternative service modules according to the associated priority; responding to the operation of the selected alternative service module of the user, switching from the current service module to the alternative service module selected by the user, so as to help the user to establish quick manual switching between the service modules. Note that the alternative service module may be presented when the service module is started; or a control for switching the service modules can be provided on the interactive interface, and the alternative service modules are displayed when the user operates the control.

For example, a user defines that a service module B and a service module C are associated in a service module a, the priority level of the service module B is higher than that of the service module C, when the user enters the service module a and wants to switch to another service module in an interactive interface, after selecting a control for switching another service module in the interactive interface, the user will see that the first switching option is the service module B and the second is the service module C in the interactive interface. Other undefined service modules may also be present in the manual switching sequence, but these undefined service modules will have a lower priority than the user defined service modules B and C.

In a specific example of the interaction method of the augmented reality device of the present invention shown in fig. 3, four service modules, namely a service module a, a service module B, a service module C, and a service module D, are shown. In fact, each service module corresponds to a scenario. It is noted that these scenes, and the lateral arrows between these scenes, are examples of behavior gestures for the user, including, for example, a series of gestures into a car, out of a car, into a home, and into a study. And fig. 3 shows that when the user performs the series of actions, the corresponding service module is automatically triggered according to the setting of the service module, and the corresponding program behavior is automatically executed. It should be noted that the in-vehicle central control, the mobile phone, the home network and related devices, the indoor wireless, bluetooth keyboard, etc. in fig. 3 all include the signal transceiver unit, which can be used as the related device or the related network environment. "action 1", "action 2", etc. in fig. 3 represent the execution order of the plurality of program actions associated with the service module, and "priority 2", "priority 3", etc. represent the plurality of alternative service modules associated with the current service module and their priorities. Specifically, in the process that the user enters the home, when the home network and the connection of the related devices are detected, the service module C is started, the actions of opening the music player, adjusting the room temperature and opening the e-mail list are sequentially executed according to the execution sequence, and the service module D and the service module B which are optional are displayed in the interactive interface according to the priority sequence for the user to select. When the user enters the study from the home and walks into the study, the user is judged to enter the study and approach the Bluetooth keyboard according to the wireless positioning, and then the service module D is automatically switched to execute corresponding program behaviors.

Based on the above embodiments, the present invention provides a service module implemented based on user customization, which can be triggered by a network environment or by a single device. The service module includes a set of a series of actions under which a user can define a series of action instructions for the device. In fact, the user would define a service module to target the scenario. The scene may be a network environment or a single device currently connected to the scene. Only after defining a service module can the action be defined.

The corresponding service module is automatically activated by monitoring the characteristics of the wireless network environment around the AR glasses. And sequentially executing related application and background services according to the behavior definition of the service module by monitoring Bluetooth wireless signals sent by the paired Bluetooth devices around the AR glasses.

Note that the setting of the service module has certain rules. As an alternative example, the settings of the service module contain the following rules:

in a service module, a user can only declare an associated device definition and default an associated network environment definition, and the service module of the type can be activated only after a device detects and connects to a corresponding associated device; alternatively, in one service module, the user may only declare the associated network environment definition and default the associated device definition, such type of service module may be activated only after the device enters and connects to the corresponding network environment; or, in a service module, the user may declare the associated device definition and the associated network environment definition at the same time, and the service module at this time needs to access the associated wireless network environment and can be activated after detecting and connecting to the associated device at the same time;

in a service module, a user must define application and background service behavior definitions, and the user cannot default the definition, otherwise, the service module has no meaning;

in one service module, the user may default to an alternative service definition.

As a specific example of determining a scene and switching service modules based on peripheral bluetooth devices: the matched Bluetooth sound equipment exists around the AR equipment, the working state of the Bluetooth sound equipment is judged after the equipment is detected and connection is completed, and then the corresponding music auxiliary service is started to carry out a series of scene enhancement, such as displaying current song information, lyrics, a play list and the like. As another specific example of determining a scene and switching service modules based on peripheral bluetooth devices: when the user is in the automobile, the AR glasses detect the vehicle-mounted central control device, and then start a service module aiming at the scene, for example, the user can be enhanced to know various device running states in the current automobile, including the automobile state in an OBD data interface, a song played currently, a destination inquired of the user, navigation display and the like. In the whole process, the user only needs to complete the input action of the target location once in advance.

As a specific example of determining a scene and switching service modules based on a peripheral wireless lan: when a user enters the range of a company wireless local area network, the AR glasses can automatically connect the wireless local area network, and then a service module defined by the user for the network environment is automatically started, wherein the service module can comprise behaviors and other service modules, such as prompt information for displaying how many mails are in a company mailbox, adjacent schedules and the like. As another specific example of determining a scenario and switching service modules based on a peripheral wireless local area network: when a user enters a defined market wireless local area network environment, the AR glasses automatically start a service module defined by the user for the network environment, such as opening the discount recommendation of the current day of a corresponding market, opening a commodity wish list of the user, opening a commodity identification function and the like.

As a specific example of prioritizing service modules based on wireless signal strength: when a user is in the Bluetooth signal range of two devices which are associated with two different service modules, namely the intelligent television and the intelligent sound box, and the two devices work, if the Bluetooth signal quality of the intelligent television is higher than that of the intelligent sound box, the service module defined for the intelligent television is preferentially started, and the service module associated with the intelligent sound box is suspended in a secondary priority mode. The user can send the instruction of switching the service module to the AR glasses device through sliding gestures, a touch pad, voice and the like so as to quickly complete switching.

As another specific example of prioritizing service modules based on wireless signal strength: when paired Bluetooth keyboard and Bluetooth sound appear around the AR glasses simultaneously, if the signal quality of the Bluetooth keyboard is higher, the service module corresponding to the Bluetooth keyboard is started preferentially, and the service module corresponding to the Bluetooth sound is hung or operated in the background. When the user is far away from the Bluetooth keyboard, the AR glasses are preferentially switched to the service module which is the strongest in signal quality in the scene and is targeted by the device closest to the user. For example, when the signal of the bluetooth sound is best and closest to the user, the bluetooth sound restarts the corresponding music service.

It should be noted that, in each embodiment of the present invention, the present invention does not limit the specific presentation effect of the interactive interface, and the specific presentation effect may be adjusted accordingly according to the development requirement and the user requirement.

Optionally, the interaction method of the augmented reality device provided by the present invention may be implemented by using an Application program (APP for short), or software such as a service running in a device background. The augmented reality device or the related device is provided with an application or service for realizing the interaction method of the augmented reality device, so that the device can be used for interaction.

Fig. 4 is a hardware block diagram illustrating an apparatus according to one embodiment of the invention. As shown in fig. 4, the apparatus 100 according to an embodiment of the present invention includes a memory 101 and a processor 102. The components in the device 100 are interconnected by a bus system and/or other form of connection mechanism (not shown). The device 100 of the present invention may be implemented in various forms, including, but not limited to, mobile terminal devices such as augmented reality glasses (or AR glasses, smart glasses) or other augmented reality devices (or AR devices), virtual reality devices (or VR devices), smart watches, smart phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), navigation apparatuses, in-vehicle terminal devices, in-vehicle display terminals, in-vehicle electronic rear view mirrors, and the like, and fixed terminal devices such as digital TVs, desktop computers, and the like.

The memory 101 is used to store non-transitory computer readable instructions. In particular, memory 101 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc.

The processor 102 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the device 100 to perform desired functions. In an embodiment of the present invention, the processor 102 is configured to execute the computer readable instructions stored in the memory 101, so that the device 100 performs all or part of the aforementioned steps of the interaction method of the augmented reality device according to the embodiments of the present invention.

It will be appreciated that the apparatus, in order to carry out the above-described functions, comprises corresponding hardware and/or software modules for performing the respective functions. The present application is capable of being implemented in hardware or a combination of hardware and computer software in conjunction with the exemplary algorithm steps described in connection with the embodiments disclosed herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, with the embodiment described in connection with the particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In this embodiment, the device may be divided into functional modules according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.

The integrated module may be implemented in the form of hardware. It should be noted that the division of the modules in this embodiment is schematic, and is only a logic function division, and there may be another division manner in actual implementation.

In the case of dividing each functional module by corresponding functions, fig. 5 shows a possible composition diagram of the apparatus 200 involved in the above embodiment, and as shown in fig. 5, the apparatus 200 may include: a setting unit 201, an acquisition unit 202, and a processing unit 203.

Among other things, the setup unit 201 may be used to support the device 200 in performing the above-described step S11, etc., and/or other processes for the techniques described herein.

The acquisition unit 202 may be used to support the device 200 in performing the above-described step S12, etc., and/or other processes for the techniques described herein.

Processing unit 203 may be used to enable device 200 to perform steps S13, etc., described above, and/or other processes for the techniques described herein.

It should be noted that all relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional module, and are not described herein again.

The device provided by the embodiment is used for executing the interaction method of the augmented reality device, so that the same effect as the implementation method can be achieved.

In some embodiments, the devices 100, 200 of embodiments of the present invention are augmented reality glasses.

An embodiment of the present invention further provides a computer storage medium, where a computer instruction is stored in the computer storage medium, and when the computer instruction runs on a device, the device executes the relevant method steps to implement the interaction method of the augmented reality device in the above embodiment.

Embodiments of the present invention further provide a computer program product, which when running on a computer, causes the computer to execute the above related steps to implement the interaction method of the augmented reality device in the above embodiments.

In addition, the embodiment of the present invention further provides an apparatus, which may specifically be a chip, a component or a module, and the apparatus may include a processor and a memory connected to each other; when the device runs, the processor can execute the computer execution instructions stored in the memory, so that the chip can execute the interaction method of the augmented reality device in the above method embodiments.

The device, the computer storage medium, the computer program product, or the chip provided by the present invention are all configured to execute the corresponding method provided above, and therefore, the beneficial effects achieved by the device, the computer storage medium, the computer program product, or the chip may refer to the beneficial effects in the corresponding method provided above, and are not described herein again.

Although the present invention has been described with reference to a preferred embodiment, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. An interaction method of an augmented reality device, the method comprising the steps of:

receiving settings for one or more service modules, including receiving settings for one or more associated devices and/or associated network environments of the service module and settings for one or more associated programs of the service module;

acquiring signals transmitted by devices around augmented reality equipment and/or acquiring signals of a network environment around the augmented reality equipment;

if the equipment which accords with the setting of the associated equipment and/or the network environment which accords with the setting of the associated network environment are detected, starting the corresponding service module, and executing the program corresponding to the service module according to the setting of the associated program;

the method further comprises the following steps:

when a first service module is in a starting state, continuing to monitor signals transmitted by devices around the augmented reality device and/or signals of a network environment around the augmented reality device, wherein the first service module is the service module corresponding to a first associated device or a first associated network environment;

reacting to the second associated device/the second associated network environment if the first service module contains a setting of a rule for the second associated device/the second associated network environment.

2. The interaction method of the augmented reality device according to claim 1, wherein:

the acquiring of the signal transmitted by the device around the augmented reality device includes obtaining device feature information by monitoring the signal of the device around the augmented reality device, where the device feature information includes one or more of a device name, a device identifier, a device signal strength, a device relative distance, and a device type;

the acquiring of the network environment around the augmented reality device includes acquiring network environment characteristic information by monitoring signals of the network environment around the augmented reality device, where the network environment characteristic information includes information of wireless signals and information of signal transmitters, specifically includes a wireless network name, a wireless network signal strength, MAC addresses of one or more signal transmitters transmitting the wireless network signals, and one or more of the positions of the augmented reality device acquired by using a wireless positioning technology.

3. The method of claim 1, wherein the receiving settings for one or more associated devices of the service module comprises one or more of the following steps:

4. The method of claim 1, wherein the receiving settings for one or more associated network environments of the service module comprises: and receiving and recording the information of one or more signal transmitters corresponding to the service module and the setting of the information of the wireless signals sent by the signal transmitters.

5. The method of claim 3 or 4, wherein the method comprises:

the receiving settings for one or more associated devices and/or associated network environments of the service module further comprises: receiving a setting of start position information and/or start signal strength information for the service module;

the acquiring signals transmitted by devices around the augmented reality device and/or acquiring signals of a network environment around the augmented reality device further comprises: determining current location information and/or current signal strength information of the augmented reality device based on one or more of signals transmitted by devices surrounding the augmented reality device, signals of a network environment surrounding the augmented reality device;

if the device meeting the setting of the associated device and/or the network environment meeting the setting of the associated network environment is detected, the starting of the corresponding service module comprises the following steps: and judging whether the current position information and/or the current signal strength information accord with the setting of the starting position information and the starting signal strength information, and starting the service module which accords with the setting.

6. The method of interacting with an augmented reality device of claim 1,

the receiving settings for one or more associated devices and/or associated network environments of the service module comprises: receiving a setting of a priority for a plurality of the associated devices and/or a priority for a plurality of the associated network environments;

if the device meeting the setting of the associated device and/or the network environment meeting the setting of the associated network environment is detected, the starting of the corresponding service module comprises the following steps: if a plurality of devices which accord with the setting of the associated device and/or a plurality of network environments which accord with the setting of the associated network environment are detected, the associated device with the highest priority or the service module corresponding to the associated network environment is started.

7. The method according to claim 1, wherein if a device that meets the setting of the associated device is detected and/or a network environment that meets the setting of the associated network environment is detected, starting the corresponding service module comprises one or more of the following steps:

8. The interaction method of the augmented reality device according to claim 1, wherein:

the setting of the one or more associated programs of the service module comprises an identification of the associated program corresponding to the service module and one or more to-be-executed behaviors of the associated program, which are selected by a user;

the executing the program corresponding to the service module according to the setting of the associated program includes executing the to-be-executed behavior corresponding to the service module.

9. The method of claim 8, wherein the method comprises:

the receiving of the setting of the one or more associated programs of the service module further includes receiving a setting of an execution sequence and/or an execution state of a plurality of the associated programs or a plurality of the to-be-executed behaviors of one associated program; the execution state comprises an inquiry attribute, and is used for inquiring a user before the associated program or the behavior to be executed with the inquiry attribute is executed, and judging whether to execute the associated program or the behavior to be executed according to the response of the user to the inquiry;

the executing the program corresponding to the service module according to the setting of the associated program includes executing the to-be-executed behavior of the associated program corresponding to the service module according to the execution order and/or the setting of the execution state.

10. The method of interacting with an augmented reality device of claim 1,

the receiving settings for one or more service modules further comprises: receiving a setting for one or more alternative service modules associated with one of the service modules and a setting for an association priority between a plurality of the alternative service modules;

the method further comprises the following steps: displaying the alternative service modules and arranging the alternative service modules according to the associated priorities; and responding to the selection operation of the user, and switching from the current service module to the alternative service module selected by the user.

11. The interaction method of the augmented reality device according to claim 8, wherein the action to be performed comprises: program open, perform voice query, present augmented reality information, program pause, program resume, program close.

12. An augmented reality device comprising:

a memory for storing non-transitory computer readable instructions; and

a processor for executing the computer readable instructions, such that the computer readable instructions, when executed by the processor, implement the interaction method of the augmented reality device of any one of claims 1 to 11.

13. A computer storage medium comprising computer instructions which, when run on a device, cause the device to perform the method of interaction of an augmented reality device of any one of claims 1 to 11.