CN107358007B - It controls the method, apparatus of smart home system and calculates readable storage medium storing program for executing - Google Patents
It controls the method, apparatus of smart home system and calculates readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN107358007B CN107358007B CN201710692424.2A CN201710692424A CN107358007B CN 107358007 B CN107358007 B CN 107358007B CN 201710692424 A CN201710692424 A CN 201710692424A CN 107358007 B CN107358007 B CN 107358007B
- Authority
- CN
- China
- Prior art keywords
- smart home
- home system
- user
- server
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 97
- 230000009471 action Effects 0.000 claims abstract description 142
- 230000033001 locomotion Effects 0.000 claims abstract description 102
- 238000009877 rendering Methods 0.000 claims abstract description 28
- 230000008569 process Effects 0.000 claims description 28
- 230000001960 triggered effect Effects 0.000 claims description 13
- 230000006399 behavior Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 230000000977 initiatory effect Effects 0.000 claims description 4
- 230000026676 system process Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 9
- 230000003993 interaction Effects 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000008447 perception Effects 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 208000003443 Unconsciousness Diseases 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000004378 air conditioning Methods 0.000 description 2
- 230000000712 assembly Effects 0.000 description 2
- 238000000429 assembly Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 238000007654 immersion Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 239000011800 void material Substances 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000005401 electroluminescence Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000012092 media component Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 230000032696 parturition Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/13—Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B15/00—Systems controlled by a computer
- G05B15/02—Systems controlled by a computer electric
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Automation & Control Theory (AREA)
- Computer Hardware Design (AREA)
- Signal Processing (AREA)
- Structural Engineering (AREA)
- Human Computer Interaction (AREA)
- Architecture (AREA)
- Civil Engineering (AREA)
- Quality & Reliability (AREA)
- Manufacturing & Machinery (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Evolutionary Computation (AREA)
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Present invention discloses a kind of method, apparatus and computer readable storage medium for controlling smart home system.The method includes：Model data is requested to server；Rendering, which is carried out, by model data obtains the associated virtual scene of smart home system；The display of virtual scene is executed by the virtual reality device of access；Control executes user and acquires in the action video data of virtual scene trigger action, obtains action message with trigger the server；According to the movement feedback procedure executed in virtual scene, smart home system is controlled by the movement feedback procedure.The movement feedback procedure that user executes into the projection of virtual scene and virtual scene, allow users to be known by means of virtual reality device smart home system after the manipulation of smart home system is initiated by control situation, and then ensure that the safety and accuracy of smart home system control.
Description
Technical field
The present invention relates to field of computer technology, in particular to a kind of method, apparatus and meter for controlling smart home system
Calculation machine readable storage medium storing program for executing.
Background technique
With the rapid development of Internet application, traditional home equipment is gradually evolved to smart home device, into
And constitute smart home system.Smart home system has network access facility, and user can be by self-contained terminal device
It is connected to the network with smart home system, and then thus terminal device controls smart home system.
Specifically, terminal device jumps the manipulation page into smart home system under the manipulation of user, page is manipulated
Face is configured with several buttons for controlling smart home system, and each button, which can be used in touching, executes smart home system
Corresponding control process.User clicks the i.e. controllable smart home system of specific button in the manipulation page of terminal device and enters phase
The state answered, smart home system are able to operate under the control of terminal device.
But the clicking operation that user triggers button in the manipulation page, it is limited to terminal device and terminal device
Existing many unreliable factors between smart home system, it is not yet in effect to have hair when clicking button, accidentally the touching situations such as button
It is raw.At this point, if user actively checks that the operational situation side of smart home system can know that the manipulation of smart home system is held
Row is as a result, be user's necessarily unaware if not checking the operational situation of smart home system actively.
Therefore, the realization of existing smart home system control is checked due to being necessarily dependent upon the active of user, there is
Lack the technological deficiency of control security and accuracy.
Summary of the invention
In order to solve the realization that smart home system controls in the related technology, there is lack control security and accuracy
The technical issues of, the present invention provides a kind of method, apparatus and computer readable storage medium for controlling smart home system.
A method of control smart home system, the method includes：
The model data of smart home system is manipulated to server request user；
Rendering is carried out by the model data and obtains the associated virtual scene of smart home system, shows the void
Quasi- scene；
The display of the virtual scene is executed by the virtual reality device of access；
Control executes the user and acquires in the action video data of the virtual scene trigger action, and passes through acquisition
The action video data-triggered server obtains action message；
The movement executed in the virtual scene is associated with according between the action message and the smart home system
Feedback procedure, the smart home system are controlled by the movement feedback procedure.
A kind of device controlling smart home system, described device include：
Data demand module, for manipulating the model data of smart home system to server request user；
Scene rendering display module obtains the smart home system association for carrying out rendering by the model data
Virtual scene；
Projection module executes the display of the virtual scene for the virtual reality device by accessing；
Sensing module is acted, executes the user in the action video data of the virtual scene trigger action for controlling
Acquisition, and action message is obtained by the action video data-triggered server of acquisition；
Feedback module is acted, for according to being associated with described in execution between the action message and the smart home system
Movement feedback procedure in virtual scene, the smart home system are controlled by the movement feedback procedure.
A kind of device controlling smart home system, including：
Processor；And
Memory is stored with computer-readable instruction on the memory, and the computer-readable instruction is by the processing
The method of control smart home system as described above is realized when device executes.
A kind of computer readable storage medium, is stored thereon with computer program, and the computer program is held by processor
The method of control smart home system as described above is realized when row.
The technical solution that the embodiment of the present invention provides is include at least the following beneficial effects：
For the control for carrying out smart home system, the model data of manipulation smart home system is requested to server first,
Rendering is carried out by model data and obtains the associated virtual scene of smart home system, then passes through the virtual reality device of access
The display of virtual scene is executed, in order to which so far virtual scene, control execution user are dynamic in virtual scene triggering by user's projection
The action video data of work acquire, and obtain action message by the action video data-triggered server of acquisition, last according to dynamic
Make to be associated with the movement feedback procedure executed in virtual scene between smart home system, smart home system is controlled by this
Feedback procedure is acted, realizes the control of smart home system with this, in the projection of user to virtual scene and virtual scene
The movement feedback procedure for being associated with and executing between movement and smart home system based on user's triggering, allows users to borrow
Help virtual reality device know smart home system after the manipulation of smart home system is initiated by control situation, no longer
What the artificial active that needs to rely on carried out checks process, and then ensure that the safety and accuracy of smart home system control,
Avoid occurring to perceive in time the unstability of smart home system controlled by control situation and smart home system.
It should be understood that the above general description and the following detailed description are merely exemplary, this can not be limited
Invention.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention
Example, and in specification together principle for explaining the present invention.
Fig. 1 is implementation environment schematic diagram according to the present invention shown according to an exemplary embodiment；
Fig. 2 is the application schematic diagram of a set of smart home system shown according to an exemplary embodiment
Fig. 3 is a kind of block diagram of device shown according to an exemplary embodiment；
Fig. 4 is a kind of flow chart of method for controlling smart home system shown according to an exemplary embodiment；
Fig. 5 is according to the flow chart that the details of step 310 is described shown in Fig. 4 corresponding embodiment；
Fig. 6 is according to the flow chart that the details of step 370 is described shown in Fig. 4 corresponding embodiment；
Fig. 7 is according to the flow chart that the details of step 390 is described shown in Fig. 4 corresponding embodiment；
Fig. 8 is a kind of flow chart of the method for the control smart home system shown according to another exemplary embodiment；
Fig. 9 is the flow chart for generating model data in server shown according to an exemplary embodiment in real time；
Figure 10 is a kind of block diagram of device for controlling smart home system shown according to an exemplary embodiment；
Figure 11 is according to the block diagram that the details of data demand module is described shown in Figure 10 corresponding embodiment；
Figure 12 is according to the block diagram that the details of movement sensing module is described shown in Figure 10 corresponding embodiment；
Figure 13 is according to the block diagram that the details of movement feedback module is described shown in Figure 10 corresponding embodiment；
Figure 14 is a kind of block diagram of the device of the control smart home system shown according to another exemplary embodiment
Specific embodiment
Here will the description is performed on the exemplary embodiment in detail, the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistented with the present invention.On the contrary, they be only with it is such as appended
The example of device and method being described in detail in claims, some aspects of the invention are consistent.
Fig. 1 is implementation environment schematic diagram according to the present invention shown according to an exemplary embodiment.In an example
Property embodiment in, implementation environment according to the present invention, as shown in Figure 1, include at least smart home system control terminal 110 with
And cooperation control terminal 110 realizes the server 130 of smart home system control.
The control terminal 110 of smart home system, including terminal device 111, virtual reality device 113 and Image Acquisition are set
Standby 115.
The control terminal 110 of smart home system provides the control entrance of smart home system for user, and user will be controlled herein
End 110 processed carries out the relevant various operations of smart home system control institute, and obtains respective feedback therewith.
Server 130 is used to provide computing capability for control terminal 110, and then control terminal 110 is cooperated to carry out smart home system
The control of system.
Specifically, implementation environment as illustrated in FIG. 1, in this implementation environment in the control terminal 110 of smart home system,
Terminal device 111 can be the portable mobile terminals such as smart phone, tablet computer.Control of the terminal device 111 as control terminal 110
Center processed, and data interaction is carried out with server 130.Virtual reality device 113, then for carrying out the control of smart home system
Three-dimensional visual effect and three-dimensional feedback effects are realized in the realization of neutral body effect, the control of as smart home system.
It is appreciated that virtual reality device 113, includes that virtual reality shows equipment 113, virtual reality device 113 is used
It is connected to the network in being established with terminal device 111, by itself interaction between terminal device 111, realization accesses intelligent family
Occupy the stereoscopic effect of system control and the feedback based on stereoscopic effect.
Therefore, in addition to this, virtual reality device 113 is according to the needs in configuration, can also include it is various for realizing
The controller of various manipulations will allow the triggering of user's control action to be not limited to terminal device 111 as a result,.
Image capture device 115 carries out Image Acquisition, and on the one hand the configuration for virtual scene in smart home system is preparatory
It realizes image data acquiring, on the other hand, also will provide hardware foundation and guarantee for comprehensive accurate perception of user's control action.
Fig. 2 is the application schematic diagram of a set of smart home system shown according to an exemplary embodiment.As shown in Figure 2
, can intuitively it know, smart home system includes lamplight curtain, central air-conditioning, video intercom, safety monitoring, center
Heating, the sound box system for realizing background music and intelligent video, the equipment for realizing central inletting fresh air and other intelligent appliances.
But according to the demand of user side, the equipment that smart home system is covered is not fixed, but flexible configuration obtains
's.
Fig. 3 is a kind of block diagram of device shown according to an exemplary embodiment.For example, device 200 is implemented shown in Fig. 1
Terminal device 111 in environment.
Referring to Fig. 2, device 200 includes at least one component：Processing component 202, memory 204, power supply module 206 are more
Media component 208, audio component 210, sensor module 214 and communication component 216.
The integrated operation of the usual control device 200 of processing component 202, such as with display, telephone call, data communication, phase
Machine operation and the associated operation of record operation etc..Processing component 202 includes at least one processor 218 to execute instruction,
To complete all or part of the steps of following methods.In addition, processing component 202 includes at least one module, it is convenient for processing group
Interaction between part 202 and other assemblies.For example, processing component 202 includes at least multi-media module, to facilitate multimedia component
Interaction between 208 and processing component 202.
Memory 204 is configured as storing various types of data to support the operation in device 200.These data are shown
Example includes the instruction of any application or method for operating on the device 200.Memory 204 can be by any kind of
Volatibility or non-volatile memory device or their combination are realized, such as static random access memory (Static Random
Access Memory, abbreviation SRAM), electrically erasable programmable read-only memory (Electrically Erasable
Programmable Read-Only Memory, abbreviation EEPROM), Erasable Programmable Read Only Memory EPROM (Erasable
Programmable Read Only Memory, abbreviation EPROM), programmable read only memory (Programmable Red-
Only Memory, abbreviation PROM), read-only memory (Read-Only Memory, abbreviation ROM), magnetic memory, flash
Device, disk or CD.One or more modules are also stored in memory 204, which is configured to by this
One or more processors 218 execute, to complete the whole or portion in any shown method of following Fig. 3, Fig. 4, Fig. 5 and Fig. 6
Step by step.
Power supply module 206 provides electric power for the various assemblies of device 200.Power supply module 206 includes power-supply management system, until
A few power supply and other with for device 200 generate, manage, and distribute the associated component of electric power.
Multimedia component 208 includes the screen of one output interface of offer between described device 200 and user.One
In a little embodiments, screen may include liquid crystal display (Liquid Crystal Display, abbreviation LCD) and touch panel.
If screen includes touch panel, screen may be implemented as touch screen, to receive input signal from the user.Touch panel
Including at least one touch sensor to sense the gesture on touch, slide, and touch panel.The touch sensor can not
The boundary of a touch or slide action is only sensed, but also detects duration associated with the touch or slide operation and pressure
Power.Screen further includes display of organic electroluminescence (Organic Light Emitting Display, abbreviation OLED).
Audio component 210 is configured as output and/or input audio signal.For example, audio component 210 includes a Mike
Wind (Microphone, abbreviation MIC), when device 200 is in operation mode, such as call model, logging mode and speech recognition mould
Whens formula etc., microphone is configured as receiving external audio signal.The received audio signal can be further stored in storage
Device 204 is sent via communication component 216.In some embodiments, audio component 210 further includes a loudspeaker, for defeated
Audio signal out.
Sensor module 214 includes at least one sensor, for providing the status assessment of various aspects for device 200.
For example, sensor module 214 can detecte the state that opens/closes of device 200, the relative positioning of component, sensor module
214 can be with the position change of 200 1 components of detection device 200 or device and the temperature change of device 200.In some realities
It applies in example, which further includes Magnetic Sensor, pressure sensor or temperature sensor.
Communication component 216 is configured to facilitate the communication of wired or wireless way between device 200 and other equipment.Device
200 can access the wireless network based on communication standard, such as WiFi (WIreless-Fidelity, Wireless Fidelity).Show at one
In example property embodiment, communication component 216 receives broadcast singal or broadcast from external broadcasting management system via broadcast channel
Relevant information.In one exemplary embodiment, the communication component 216 further includes near-field communication (Near Field
Communication, abbreviation NFC) module, to promote short range communication.For example, radio frequency identification (Radio can be based in NFC module
Frequency Identification, abbreviation RFID) technology, Infrared Data Association (Infrared Data
Association, abbreviation IrDA) technology, ultra wide band (Ultra Wideband, abbreviation UWB) technology, Bluetooth technology and other skills
Art is realized.
In the exemplary embodiment, device 200 is by least one application specific integrated circuit (Application
Specific Integrated Circuit, abbreviation ASIC), it is digital signal processor, digital signal processing appts, programmable
Logical device, field programmable gate array, controller, microcontroller, microprocessor or other electronic components are realized, for executing
Following methods.
Fig. 4 is a kind of flow chart of method for controlling smart home system shown according to an exemplary embodiment.The control
The method of smart home system processed is suitable for the terminal device of implementation environment shown in Fig. 1, and the terminal device is in an exemplary reality
It applies and can be device shown in Fig. 3 in example.As shown in figure 4, the method for the control smart home system, is executed by terminal device,
Include the following steps.
In the step 310, the model data of smart home system is manipulated to server request user.
Wherein, smart home system is deployed in the houses such as house.For example, being deployed in multiple intelligent families of home dwelling
It occupies equipment and just constitutes smart home system, and many smart home devices for being deployed in public place can also constitute intelligent family
Occupy system.It it should be noted that this smart home system can be deployed in any house, but must be the currently requested control of user
Smart home system.
Every suit smart home system has its model data, this model data is stored in server, when needed,
Terminal device can be to this model data of server request.Model data for the manipulation of smart home system for providing virtually
Scene shows this virtual scene in the form of image frame.In exemplary embodiment in the specific implementation, model
Data are three-dimensional modeling datas, configure virtual three dimensional field to the manipulation of smart home system for user by three-dimensional modeling data
Scape.
When needing to control smart home system, user can be asked by self-contained terminal device to server
The corresponding model data of this smart home system is sought, terminal device obtains requested pattern number and the interaction with server
According to.
In a step 330, rendering is carried out by model data and obtains the associated virtual scene of smart home system.
Wherein, virtual scene is the image frame for realizing smart home system control.Model data is substantially that image is drawn
The description data in face, therefore, terminal device can carry out wash with watercolours via model data after obtaining model data to server
Dye to obtain the image frame being consistent with this model data, the i.e. associated virtual scene of smart home system, and then will render institute
The virtual scene of acquisition transfers to virtual reality device to be shown.
Terminal device obtains that virtual reality device will be transferred to after the associated virtual scene of smart home system in rendering
The display of this virtual scene is carried out, at this point, just the picture of achievable smart home system control is provided for user, for intelligence
For the control of house system, intuitively and interactive performance is enabled to be enhanced.
It should remark additionally herein, on the one hand being associated between smart home system and virtual scene refers to and be based on
On the other hand the association of user, smart home system also refer to the virtual of rendering display by virtual scene corresponding to user
Scene is for realizing the control of the currently used smart home system of user.
In step 350, the display of virtual scene is executed by the virtual reality device of access.
Wherein, virtual reality device, which is used to match with terminal device, realizes that virtual reality is shown combined into user.For example, empty
Quasi- real world devices are VR glasses.As previously described, terminal device carries out the display of virtual scene, at this point, in virtual reality
Under the intervention of equipment, the projection of user to virtual scene is executed, to make user be immersed in virtual scene.
Virtual reality device encapsulates user to extraneous vision and the sense of hearing, and guidance user generates one kind in virtual scene
Feeling, virtual scene is the three-dimensional image frame presented for a user.
It should be appreciated that the projection of user to virtual scene is to have dressed the user of virtual reality device to check the virtual of display
Scene and realize, thus obtain submergence sense and Multi-Sensory, for example, Auditory Perception, tactilely-perceptible etc..
In step 370, control executes user and acquires in the action video data of virtual scene trigger action, and by adopting
The action video data-triggered server of collection obtains action message.
Wherein, image capture device has been installed in the house for disposing smart home system, and user is also in this house
In.The image capture device of control access is carried out the acquisition of action video data by terminal device, that is to say, that user is occurred
Movement variation will all be captured by image capture device.
The action video data of acquisition are uploaded to server, carry out under the action of server to this action video data
It calculates in real time and generates action message, and returned to terminal device, terminal device is thus aloowed to perceive the movement of user,
And then the control of the feedback of the movement in virtual scene and smart home system is carried out according to this movement.
By server it is real-time calculate the action message generated be used to indicate user triggered in virtual scene it is dynamic
Make.Further, action message further includes describing user in the animation information of virtual scene institute trigger action, which uses
In the movement that tracking presentation user triggers in virtual scene.That is, animation information is substantially that user action carries out process
With merging for virtual scene.
Due to the generation of action message, on the one hand need to calculate acquired action video data in real time, another party
Face needs to generate corresponding animation, and blends with virtual scene, and the complexity of time performance and calculating is relatively high, needs
Very strong real-time computing, therefore will be realized by means of server.
In step 390, the movement executed in virtual scene is associated with according between action message and smart home system
Feedback procedure, smart home system are controlled by movement feedback procedure.
Wherein, as previously described, action message indicates the movement that user triggers in virtual scene.It is appreciated that by throwing
It is incident upon the user of virtual scene, that is, the user for being immersed in virtual scene is unlikely to be stationary, will do it some conscious
Movement and unconscious movement.For example, conscious movement includes the control action to smart home system, and virtual
Various interactive actions in scene etc..
Being associated between action message and smart home system refers to movement indicated by action message and smart home system
Correlation between system.This correlation is for reflecting whether movement indicated by action message is related to smart home system.Such as
Movement indicated by fruit action message is control action, then is relevant to smart home system；If indicated by action message
Movement be other movements, then it is uncorrelated to smart home system.
Movement feedback procedure in virtual scene be on the one hand by action message and the user action tracking realized was presented
On the other hand journey is then the intelligent family for acting the relevance between smart home system according to indicated by action message and executing
Occupy system manipulation feedback.
Performed smart home system manipulation feedback includes at least the relevant user of movement institute and executes behaviour in virtual scene
Make correct or wrong picture to show.When movement is control action, it is correct in virtual scene execution operation to carry out user
Picture is shown, conversely, being then other movements in movement, for example, with the not related movement of smart home system, it will used
Family is shown in the picture that virtual scene executes operating mistake.
Therefore, the movement feedback procedure in performed virtual scene is the display process of image frame.As user exists
The triggering acted in virtual scene accordingly executes the movement feedback procedure in virtual scene under the auxiliary of server.
Smart home system is controllable by movement feedback procedure, with the execution of movement feedback procedure, is carrying out user
When virtual scene executes and operates the display of correct picture, control signal is generated according to the movement of action message instruction, transmitting should
Control signal is sent to smart home system i.e. and smart home system may make to carry out corresponding operating condition.
So far the control of smart home system is just realized based on virtual scene, on the one hand, due in this virtual scene reality
Existing movement perception, i.e. the acquisition of action video data is to be directed to everything of the user in virtual scene to carry out, because
This, is no longer due to many restrictions of terminal device, for example, screen size, and there is the case where not perceiving, and then ensure that intelligence
The accuracy controlled in energy house system.
On the other hand, it is determined that for user after the movement that virtual scene triggers, the movement carried out in virtual scene is anti-
Feedback, and smart home system is controllable by this movement feedback procedure, so that user was fed back by this movement
Journey and know smart home system by control situation, be able to guarantee the safety of smart home system control.
Fig. 5 is according to the flow chart that the details of step 310 is described shown in Fig. 4 corresponding embodiment.The step
310, as shown in figure 5, at least including the following steps.
In step 311, the identification information logged according to user initiates smart home system manipulation request to server.
Wherein, as previously described, the control of smart home system be by terminal device by access of virtual real world devices and
Image capture device and realize.Corresponding application program is run in terminal device, to execute step of the present invention.With
Family executes login process in application program by its identification information, completes the control that smart home system can be carried out after logging in
System.
The smart home system that identification information is manipulated for uniquely indicating user.The application program that terminal device is run
After completing login process, smart home system manipulation request can be initiated by the identification information logged in.Smart home system
System manipulation request carries the identification information of user's login, needs to control smart home system into a certain operating condition in user
When, smart home system manipulation request is initiated from the application program that terminal device is run to server.
By foregoing description it is recognised that the manipulation of smart home system is carried out in virtual scene, therefore for not
For user in smart home system manipulation state, obtained by the initiation of smart home system manipulation request for real
The model data of existing virtual scene.
In step 313, the model data that server manipulates request in response to smart home system and returns, pattern number are received
According to being pre- to first pass through real scene where smart home system and construct to obtain and correspond in smart home system manipulation request
Identification information.
Wherein, server extracts in smart home system after the smart home system manipulation request for receiving user
The identification information of carrying carries out the lookup of model data according to identification information, to obtain the model that can be returned to terminal device
Data.
Server is with the storage that identification information is that index carries out model data.That is, in advance to deploying intelligent family
The house for occupying system carries out image data acquiring, and then this image data is uploaded to server, in order to which server can be pre-
Model data first is generated for smart home system, and is stored.
Real scene where signified smart home system as disposes the house of smart home system.Based on true field
Scape and the model data building realized, realize the virtual scene being consistent with real scene for user, and anti-in subsequent action
In the execution of feedback process, so that image frame relevant to user action shows the virtual field being also consistent at this with real scene
Scape carries out, and enhances the feeling of immersion and authenticity of smart home system control, greatly so as to improve smart home system
The ease for use of control reduces the threshold for freely manipulating smart home system.
In one exemplary embodiment, the step 330 at least includes the following steps.
Image frame rendering is carried out to model data, the scene for constructing model data is applied to image frame and obtains intelligence
The associated virtual scene of house system.
Wherein, as previously described, model data corresponds to smart home system, model data smart home system thus
System, which provides, realizes the virtual scene that control can be realized to wherein smart home device.This virtual scene is in the form of image frame
It has been shown that, and the content shown, i.e., provided by model data.
It constructs to obtain since model data is in advance based on a scene, the image that model data is realized is drawn
Face content is shown then will be related to this scene.In one exemplary embodiment, this scene can be preconfigured general
Scene, it can suitable for a certain default scene of all smart home systems control, occupied with reducing to calculate pressure and reduce
Server resource but it is also possible to be the real scene as described in Fig. 4 corresponding embodiment, can also be certain that user chooses in advance
One scene etc..
Image frame rendering is one from model of place i.e. model data, to the treatment process of display image.Pass through image
The execution of picture rendering so that the scene for constructing model data is applied in image frame before this, obtain by
The virtual scene that this scene and other elements for realizing smart home system control are formed.
Model data is corresponding smart home system, therefore, the void that corresponding smart home system is obtained with rendering
Quasi- scene relating, i.e. this virtual scene will be for realizing the control of associated smart home system.
Under the action of terminal device, the rendering of virtual scene is completed, so that rendering obtained virtual scene energy
Enough virtual reality devices by accessing realize display, and user is projected in virtual scene under the action of virtual reality device,
For the user for dressing virtual reality device, three-dimensional display effect can be obtained under the intervention of virtual reality device
Fruit achievees the purpose that itself being projected to virtual scene.
Fig. 6 is according to the flow chart that the details of step 370 is described shown in Fig. 4 corresponding embodiment.The step
370, as shown in fig. 6, at least including the following steps.
In step 371, the image capture device for controlling access carries out the acquisition of action video data, upload movement to user
Video data is to server.
Wherein, user side is to realize the control of smart home system, has accessed image capture device.Image capture device can
To be the equipment for capturing outdoor scene such as camera.The user for being projected to virtual scene will trigger certain movement, lead to
Triggered movement is crossed to control smart home system, the sense of user action is realized therefore, it is necessary to incoming image acquisition equipment
Know.
Image capture device persistently carries out the acquisition of action video data, and action video data describe user and are projected
The everything occurred after to virtual scene, various conscious actions and unconscious movement including aforementioned meaning.
With the acquisition of action video data, identification information that collected action video data will be logged in user
Server is uploaded to together, and then user carries out the real-time calculating of action video data to trigger the server thus, according to calculating
As a result action message is generated.
After realizing virtual scene, realizes user therewith by image capture device and adopted in the movement of virtual scene
Collection, since image capture device is to continue to carry out, can be avoided the inaccuracy of movement perception, guaranteed subsequent institute
Realize the accuracy of smart home system control.
In step 373, the real-time calculating and the return of calculated result of action video data are used by server
The action message that family is generated in virtual scene triggering.
Wherein, the real-time calculating process that server executes action video data, the movement that user is triggered for identification,
And then it obtains and indicates the calculated result of user institute trigger action.Therefore, user can be generated by calculated result in virtual field
The action message that scape triggering generates.
In one exemplary embodiment, action message is used to indicate the movement that user is triggered, therefore, can directly by
Calculated result is set to action message, to return to terminal device.
In another exemplary embodiment, action message further includes that user believes in the animation of virtual scene institute trigger action
Breath.Animation information is used to carry out tracking display in the movement that virtual scene triggers to user.For example, being projected to virtual scene
When the movement of user's triggering is the dummy object in crawl virtual environment, by animation information come this process of tracking display, at this moment,
It is realized by touch feedback and visual feedback, specifically, being so that user has the feeling for being held by object, and is regarding
Open country, i.e., in virtual scene, the dummy object being booked can also be moved with the movement set about at once, this process will pass through animation
The broadcasting of the provided image frame of information is realized.
Here, it is recognised that the generation of animation information, the tracking on the one hand needing to be acted according to action video data realization
Identification, the movement change procedure on the other hand then needing to track and identify are blended with virtual scene, are formd visual
Image frame feedback.The generation of animation information needs to obtain very high timeliness, and the accuracy calculated.
Therefore, after this action video data being uploaded to server, just by means of the computing resource of server and reality
When computing capability be user side generate action message, export the image frame that can be interacted in real time to terminal device.
By this process, the interactive performance of smart home system control will be improved, guaranteed by server in real time
In the case where property, so that the user for being projected to virtual scene has very high operate to the dummy object in virtual scene
Degree, so can after manipulation process to the smart home system that user is carried out and manipulation smart home system quilt
Control situation is timely and effectively fed back, and can guarantee the safety of smart home system control.
Fig. 7 is according to the flow chart that the details of step 390 is described shown in Fig. 4 corresponding embodiment.The step
390, as shown in fig. 7, at least including the following steps.
In step 391, judge whether movement indicated by action message is that the manipulation set in smart home system is moved
Make, if YES, thens follow the steps 393, if NO, then follow the steps 395.
Wherein, the realization in advance for various control processes in the smart home system disposed sets control action, often
One control process has the corresponding control action set.For a user, it is only necessary to which triggering control action may make intelligence
The control process that energy house system executes.
As previously described, the user for being projected to virtual scene often triggers various movements in virtual scene, for example, respectively
Plant conscious or unconscious movement.Action message indicates a series of actions, i.e., is formed according to chronological order
Action sequence.Corresponding, action message indicates multiple movements, needs therefrom to identify the manipulation of smart home system
Movement.
In step 393, the correct picture of operation is executed in virtual scene and shows that the movement is for generating smart home system
The control signal of system.
Wherein, if it is determined that movement indicated by action message be the control action set in smart home system, then
Illustrate the user for being projected to virtual scene, correct operation is performed in virtual scene, therefore, on the one hand to the correct of operation
Property fed back, on the other hand also will accordingly according to this movement and generate control signal, control signal generate after be sent to intelligence
House system.
In step 395, shown in the picture that virtual scene executes operating mistake.
Wherein, if it is determined that being not that manipulation indicated by smart home system is dynamic to movement indicated by action message
Make, then illustrates that user does not trigger any movement to manipulate smart home device in virtual scene, only need at this time
Virtual scene is that the picture of user's execution operating mistake is shown, and the movement for not needing response user carries out smart home
The control of system.
By the above exemplary embodiments, the feedback that user acts in virtual scene is realized, to provide for user
Experience good user operation process.
In addition to this, whether correctly anti-in addition to being operated in the case where action message further comprises animation information
Except feedback, for example, operated whether correctly feed back before, execute the described image frame of animation information display, with
Tracking display is realized to the movement that user triggers immediately, it will be understood that the substantial image capture device of shown image frame
The splicing of the action video data that acquire and transmit in real time and original virtual scene corresponding image picture reaches empty with this
The purpose that motion tracking is shown in quasi- scene.
Fig. 8 is a kind of flow chart of the method for the control smart home system shown according to another exemplary embodiment.It should
The method for controlling smart home system, before step 310, as shown in figure 8, further comprising the steps of.
In step 410, the real scene disposed in smart home system is obtained by the image capture device of access
Real scene image data.
Wherein, it for the smart home system of deployment, to realize control of the user to this smart home system, needs
Image data acquiring is carried out to the house for deploying smart home system in advance, i.e., execution smart home system is disposed true
The acquisition of real scene image data corresponding to scene.It is appreciated that the house for deploying smart home system can be house,
Hotel etc..
The image capture device of access is installed on the designated position in house, for example, the position in corner, behind the door, on wall
Deng.
Terminal device obtains real scene image data by the image capture device of access, and then according to real scene figure
As smart home system executes the process being pre-configured to data request service device thus, with by the execution of pre-provisioning procedure so that with
Family is able to carry out the manipulation of this smart home system.
It should be understood that the pre-provisioning procedure of request server execution according to real image data, is substantially to true
Scene image data carries out the process of dynamic modeling and the corresponding model data of final smart home system.
In one exemplary embodiment, image capture device can carry out the video of outdoor scene to place house or panorama is clapped
It takes the photograph, to export the real scene image data for corresponding to panoramic video or panorama sketch, and then the computing capability for passing through server
The augmented reality content that can be interacted, i.e. model data are exported in real time.
In step 430, real scene image data is uploaded to server according to the identification information that user logs in, by true
The upload trigger the server of real scene image data carries out dynamic modeling according to real scene image data and generates model data, mould
Type data are index storage with identification information by server.
Wherein, it should be noted that the identification information that user logs in corresponds to smart home system.Terminal device exists
After obtaining real scene image data by the image capture device accessed, identification information carries out real scene image data thus
Upload, here, being substantially the upload for executing real scene image data for corresponding smart home system, in order to obtain
Obtain the model data corresponding to smart home system.
Server executes dynamic modeling after the real scene image data for receiving upload.Real scene image data
It is that various three-dimensional informations relevant to the disposed house of smart home system are obtained by image capture device.
In exemplary embodiment in the specific implementation, performed dynamic modeling process, includes at least building entity
It is geometry, NURBS (Non-Uniform Rational B-Splines, non-uniform rational B-spline) modeling, polygon modeling, thin
Divide the processes such as curved surface and implicit surface.When needed, dynamic modeling process further includes editor's body surface or material
Matter is (for example, color, fluorescence, diffusion and specular components-roughness and finish, reflection characteristic, transparency or impermeable
Lightness or refractive index), increase the process of texture, bump mapping and other feature.
By the execution of dynamic modeling process, modeling, scene layout and drawing image picture three phases are completed, in turn
Model data is obtained, is finally that index is stored in server with identification information.
In exemplary embodiment in the specific implementation, the model data store that server is realized, is to pass through to configure
Mysql database realizing.
Further, Fig. 9 is to generate three-dimensional modeling data in real time in server shown according to an exemplary embodiment
Flow chart.Server is that real scene image data is obtained and the interaction with terminal device.As shown in figure 9, in image
Under the intervention for acquiring equipment, terminal device realizes real scene image data acquiring, will upload this real scene image data
To server, i.e., such as process that step 510 is realized.
After server receives real scene image data, data grouping process is executed, in this data grouping process
In execution, first should to the user corresponding to terminal device carry out Authority Verification, as described in step 610, server with
Itself data interaction between terminal device, the identification information that user in terminal device is logged in is had also obtained, to this mark
Know information and carry out Authority Verification, after through Authority Verification, step 620 is executed, according to different visual angles to true field
Scape image data packets, to obtain the real image data sorted out according to visual angle.
This view is adapted to obtain such as step 630 to the real image data creation time axis that each visual angle is sorted out
Feel the time shaft of angle, the real image data of multiple visual angles and compatible time shaft can form pattern number as a result,
According to.
It is that this model data is put in storage storage by index with identification information, i.e., as illustrated in step 640.
So far being stored in advance in server of model data corresponding to the smart home system that user is controlled is realized.
Once user initiates to request by terminal device to server, terminal device can pass through the model data real-time rendering figure of storage
Shape, and image frame is returned to via network transmission and to terminal device, and then real under the action of institute's access of virtual real world devices
The display of existing virtual scene, as described in step 650 to step 670.
It is corresponding, it is in virtual scene constructed by user, after perceiving the movement of user, server is held
Row step 690 passes through the realization computing capability of itself and network transmission performance realization movement feedback.
In one exemplary embodiment, between server and terminal device network transmission realization, may rely on and match
The SiBEAM Snap technology set, to guarantee that the realization of augmented reality in smart home system will not be due to network transmission speed not
It matches and the fluency of image quality and picture is caused to be unable to satisfy the case where viewing requires.
The exemplary embodiment realized through the invention, so that the realization of smart home system control is blended in virtually
In reality technology, uninteresting, stiff between user and smart home system and passively status is changed, and pass through tracking user
Head, hand and body perception user movement, and interactivity is enhanced by feedback that tactile, audio and image are shown
Can, user initiates the accuracy of smart home system control and safety is obtained and improved comprehensively.
The exemplary embodiment realized through the invention, user are seen by taking virtual reality device, such as VR glasses
To the figure of server transport, and then it itself will be placed in a virtual scene, is operated such as in this virtual scene by acting
The content of the switches such as washing machine, air-conditioning, electric cooker can guarantee the correctness and safety of operation in this way, and user is to intelligent family
Occupy the more standby enhancing sense of operation of system.
In the exemplary embodiment that the present invention is realized, by the use of virtual reality device, so that user is to intelligence
The control of house system can incorporate virtual reality technology, on the one hand realize user and manipulate and obtained instead to smart home system
The immersion of feedback is experienced, and on the other hand also directly improves validity and accuracy of the smart home system in control, is guaranteed
User's manipulation timely responds to.
Following is apparatus of the present invention embodiment, and the method that can be used for executing the above-mentioned control smart home system of the present invention is real
Apply example.For undisclosed details in apparatus of the present invention embodiment, the method for please referring to present invention control smart home system is real
Apply example.
Figure 10 is a kind of block diagram of device for controlling smart home system shown according to an exemplary embodiment.The control
The device of smart home system, as shown in Figure 10, including but not limited to：Data demand module 810, scene rendering display module
830, projection module 850, movement sensing module 870 and movement feedback module 890.
Data demand module 810, for manipulating the model data of smart home system to server request user.
Scene rendering module 830 obtains the associated virtual field of smart home system for carrying out rendering by model data
Scape.
Display module 850 executes the display of virtual scene for the virtual reality device by access.
Sensing module 870 is acted, is acquired for controlling execution user in the action video data of virtual scene trigger action,
And action message is obtained by the action video data-triggered server of acquisition.
Feedback module 890 is acted, for executing virtual scene according to being associated between action message and smart home system
In movement feedback procedure, smart home system is controlled by movement feedback procedure.
Figure 11 is according to the block diagram that the details of data demand module is described shown in Figure 10 corresponding embodiment.The number
According to request module 810, as shown in figure 11, including but not limited to：Request initiating cell 811 and data receipt unit 813.
Request initiating cell 811, the identification information for being logged according to user initiate smart home system behaviour to server
Control request.
Data receipt unit 813, the pattern number for manipulating request for receiving server in response to smart home system and returning
According to model data is that the pre- smart home system place real scene that first passes through constructs to obtain and correspond to smart home system manipulation
Identification information in request.
In one exemplary embodiment, scene rendering module 830 is further used for carrying out image frame by model data
Rendering, the scene for constructing model data are applied to image frame and obtain the associated virtual scene of smart home system.
Figure 12 is according to the block diagram that the details of movement sensing module is described shown in Figure 10 corresponding embodiment.This is dynamic
Make sensing module 870, as shown in figure 12, including but not limited to：DCU data control unit 871 and action message receiving unit 873.
DCU data control unit 871, the image capture device for controlling access carry out the acquisition of action video data to user,
Action video data are uploaded to server.
Action message receiving unit 873, for passing through server to the real-time calculating of action video data and calculated result
Return obtain user the virtual scene triggering generate action message.
Figure 13 is according to the block diagram that the details of movement feedback module is described shown in Figure 10 corresponding embodiment.This is dynamic
Make feedback module 890, as shown in figure 13, including but not limited to：Judging unit 891, correct operation feedback unit 893 and mistake behaviour
Make feedback unit 895.
Judging unit 891, for judging whether movement indicated by action message is the behaviour set in smart home system
Control movement, if YES, then triggers correct operation feedback unit 893, if NO, then triggers faulty operation feedback unit 895.
Correct operation feedback unit 893 shows for executing the correct picture of operation in institute's virtual scene, acts for giving birth to
At the control signal of smart home system.
Faulty operation feedback unit 895, if the everything for action message instruction is not smart home system
The control action of middle setting is then shown in the picture that virtual scene executes operating mistake.
Figure 14 is a kind of block diagram of the device of the control smart home system shown according to another exemplary embodiment.The control
The device of smart home system processed further includes but is not limited to as shown in figure 14：Data pre-acquired module 910 and scene request are matched
Set module 930.
Data pre-acquired module 910, the real scene for being disposed in smart home system, is adopted by the image of access
Collect equipment and obtains real scene image data；
Scene request configuration module 930, the identification information for being logged according to user upload real scene figure to server
As data, dynamic modeling life is carried out according to real scene image data by the upload trigger the server of real scene image data
At model data, model data is index storage with identification information by server.
Optionally, the present invention also provides a kind of device for controlling smart home system, Fig. 4, Fig. 5 Fig. 6, Fig. 7 and Fig. 8 are executed
The all or part of step of the method for smart home system is controlled shown in any.Described device includes：
Processor；
Memory for storage processor executable instruction；
Wherein, the processor is configured to executing：
The model data of smart home system is manipulated to server request user；
Rendering, which is carried out, by the model data obtains the associated virtual scene of smart home system；
The display of the virtual scene is executed by the virtual reality device of access；
Control executes the user and acquires in the action video data of the virtual scene trigger action, and passes through acquisition
The action video data-triggered server obtains action message；
The movement executed in the virtual scene is associated with according between the action message and the smart home system
Feedback procedure, the smart home system are controlled by the movement feedback procedure.
In the exemplary embodiment, a kind of storage medium is additionally provided, which is computer readable storage medium,
It such as can be the provisional and non-transitorycomputer readable storage medium for including instruction.The storage medium is for example including instruction
Memory 204, above-metioned instruction can executes by the processor 218 of device 200 to complete the side of above-mentioned control smart home system
Method.
It should be understood that the present invention is not limited to the precise structure already described above and shown in the accompanying drawings, and
And various modifications and change can be executed interior without departing from the scope.The scope of the present invention is only limited by the attached claims
System.
Claims (14)
1. a kind of method for controlling smart home system, which is characterized in that the method includes：
The model data of smart home system is manipulated to server request user；
Rendering, which is carried out, by the model data obtains the associated virtual scene of smart home system；
The display of the virtual scene is executed by the virtual reality device of access；
Control executes the user and acquires in the action video data of the virtual scene trigger action, and passes through the described of acquisition
Action video data-triggered server obtains action message；
The movement feedback in the virtual scene is executed according to being associated between the action message and the smart home system
Process, the smart home system are controlled by the movement feedback procedure；
Wherein, being associated between the action message and the smart home system reflect movement indicated by action message with
It is whether related between smart home system；The movement feedback procedure executed includes the user that realizes by the action message
Motion tracking is presented, and whether the smart home system of related execution manipulates between smart home system according to the movement
Feedback；
With the execution of movement feedback procedure, user is being carried out when virtual scene executes and operates the display of correct picture, according to
The movement of action message instruction generates control signal, transmits the control signal to smart home system, knows smart home system
By control situation.
2. the method according to claim 1, wherein described manipulate smart home system to server request user
Model data, including：
The identification information logged according to the user initiates smart home system manipulation request to the server；
The model data that the server responds the smart home system manipulation request and returns is received, the model data is
Real scene where first passing through the smart home system in advance constructs to obtain and correspond to smart home system manipulation request
In identification information.
3. the method according to claim 1, wherein described carried out described in rendering acquisition by the model data
The associated virtual scene of smart home system, including：
Image frame rendering is carried out to the model data, the scene for constructing the model data is applied to described image picture
It obtains and the associated virtual scene of the smart home system.
4. being touched the method according to claim 1, wherein the control executes the user in the virtual scene
Start the action video data made acquisition, and action message obtained by the action video data-triggered server of acquisition,
Including：
The image capture device for controlling access carries out the acquisition of action video data to the user, uploads the action video data
To the server；
The user is obtained to the real-time calculating and the return of calculated result of the action video data by the server to exist
The action message that the virtual scene triggering generates.
5. the method according to claim 1, wherein described according to the action message and the smart home system
Association between system executes the movement feedback procedure in the virtual scene, and it is anti-that the smart home system is controlled by the movement
Feedback process, including：
Judge whether movement indicated by the action message is the control action set in smart home system, if YES,
It then executes the correct picture of operation in the virtual scene to show, the movement is believed for generating the manipulation of smart home system
Number；
If the everything of the action message instruction is not the control action set in smart home system, described
The picture that virtual scene executes operating mistake is shown.
6. the method according to claim 1, wherein described manipulate smart home system to server request user
Model data before, the method also includes：
In the real scene that the smart home system is disposed, the real scene is obtained by the image capture device of access
Image data；
The real scene image data is uploaded to the server according to the identification information that user logs in, passes through the true field
The upload of scape image data triggers the server and carries out the dynamic modeling generation mould according to the real scene image data
Type data, the model data are index storage with the identification information by the server.
7. a kind of device for controlling smart home system, which is characterized in that described device includes：
Data demand module, for manipulating the model data of smart home system to server request user；
Scene rendering module obtains the associated virtual field of smart home system for carrying out rendering by the model data
Scape；
Display module executes the display of the virtual scene for the virtual reality device by accessing；
Sensing module is acted, is adopted for controlling the execution user in the action video data of the virtual scene trigger action
Collection, and action message is obtained by the action video data-triggered server of acquisition；
Feedback module is acted, it is described virtual for being executed according to being associated between the action message and the smart home system
Movement feedback procedure in scene, the smart home system are controlled by the movement feedback procedure；
With the execution of movement feedback procedure, user is being carried out when virtual scene executes and operates the display of correct picture, according to
The movement of action message instruction generates control signal, transmits the control signal to smart home system, knows smart home system
By control situation.
8. device according to claim 7, which is characterized in that the data demand module includes：
Request initiating cell, the identification information for being logged according to the user initiate smart home system behaviour to the server
Control request；
Data receipt unit, the pattern number for responding the smart home system manipulation request for receiving the server and returning
According to the model data is that the pre- smart home system place real scene that first passes through constructs to obtain and corresponds to the intelligence
Identification information in house system manipulation request.
9. device according to claim 7, which is characterized in that the scene rendering module is further used for the model
Data carry out image frame rendering, and the scene for constructing the model data is applied to described image picture and obtains and the intelligence
The associated virtual scene of house system.
10. device according to claim 7, which is characterized in that the movement sensing module includes：
DCU data control unit, the image capture device for controlling access carry out the acquisition of action video data to the user, on
The action video data are passed to the server；
Action message receiving unit, for by the server to the action video data it is real-time calculating and calculated result
Return obtain the action message that the user generates in virtual scene triggering.
11. device according to claim 7, which is characterized in that the movement feedback module includes：
Judging unit, for judging whether movement indicated by the action message is that the manipulation set in smart home system is moved
Make, if YES, then triggers correct operation feedback unit, if NO, then trigger faulty operation feedback unit；
The correct operation feedback unit, which is used to execute the correct picture of operation in the virtual scene, shows that the movement is used for
Generate the control signal of smart home system；
If everything of the faulty operation feedback unit for action message instruction is not smart home system
The control action of middle setting is then shown in the picture that the virtual scene executes operating mistake.
12. device according to claim 7, which is characterized in that described device further includes：
Data pre-acquired module, the real scene for being disposed in the smart home system, passes through the Image Acquisition of access
Equipment obtains the real scene image data；
Scene request configuration module, the identification information for being logged according to user upload the real scene figure to the server
As data, by the upload of the real scene image data trigger the server according to the real scene image data into
Mobile state modeling generates the model data, and the model data is index storage with the identification information by the server.
13. a kind of device for controlling smart home system, which is characterized in that including：
Processor；And
Memory is stored with computer-readable instruction on the memory, and the computer-readable instruction is held by the processor
The method of control smart home system described in any one of claims 1 to 6 is realized when row.
14. a kind of computer readable storage medium, is stored thereon with computer program, the computer program is executed by processor
The method of Shi Shixian control smart home system described in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710692424.2A CN107358007B (en) | 2017-08-14 | 2017-08-14 | It controls the method, apparatus of smart home system and calculates readable storage medium storing program for executing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710692424.2A CN107358007B (en) | 2017-08-14 | 2017-08-14 | It controls the method, apparatus of smart home system and calculates readable storage medium storing program for executing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107358007A CN107358007A (en) | 2017-11-17 |
| CN107358007B true CN107358007B (en) | 2018-11-30 |
Family
ID=60286504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710692424.2A Active CN107358007B (en) | 2017-08-14 | 2017-08-14 | It controls the method, apparatus of smart home system and calculates readable storage medium storing program for executing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107358007B (en) |
Families Citing this family (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108093030B (en) * | 2017-11-29 | 2022-02-22 | 杭州博联智能科技股份有限公司 | Artificial intelligence model deployment method based on cloud server |
| CN108345907B (en) * | 2018-02-09 | 2022-02-11 | 广东美的制冷设备有限公司 | Recognition method, augmented reality device, and storage medium |
| CN110276819A (en) * | 2018-03-14 | 2019-09-24 | 佛山市顺德区美的电热电器制造有限公司 | A kind of displaying control method and device and computer readable storage medium |
| CN108710308A (en) * | 2018-05-29 | 2018-10-26 | 广东汇泰龙科技有限公司 | A kind of method and system of the given pose triggering scene linkage based on intelligent cloud lock |
| CN108919663B (en) * | 2018-07-25 | 2020-08-04 | 珠海格力电器股份有限公司 | Management method of intelligent household equipment and electronic equipment |
| CN108958057A (en) * | 2018-08-01 | 2018-12-07 | 珠海格力电器股份有限公司 | The control method and device of household appliance, storage medium, electronic device |
| CN110852770B (en) * | 2018-08-21 | 2023-05-26 | 阿里巴巴集团控股有限公司 | Data processing method and device, computing device and display device |
| CN109660508A (en) * | 2018-10-23 | 2019-04-19 | 平安壹钱包电子商务有限公司 | Data visualization method, electronic device, computer equipment and storage medium |
| CN111752443A (en) * | 2019-03-28 | 2020-10-09 | 华为技术有限公司 | Method, related device and system for controlling page by display equipment |
| CN110426965A (en) * | 2019-09-17 | 2019-11-08 | 苏州百宝箱科技有限公司 | A kind of smart home long-range control method based on cloud platform |
| CN110794777A (en) * | 2019-10-25 | 2020-02-14 | 浙江工贸职业技术学院 | Intelligent household control equipment and control method |
| CN110928411B (en) * | 2019-11-18 | 2021-03-26 | 珠海格力电器股份有限公司 | AR-based interaction method and device, storage medium and electronic equipment |
| CN111131735B (en) * | 2019-12-31 | 2022-02-22 | 歌尔光学科技有限公司 | Video recording method, video playing method, video recording device, video playing device and computer storage medium |
| CN111538252B (en) * | 2020-05-25 | 2021-09-21 | 厦门大学 | Intelligent home demonstration system applying VR technology |
| CN112460743A (en) * | 2020-11-30 | 2021-03-09 | 珠海格力电器股份有限公司 | Scene rendering method, scene rendering device and environment regulator |
| CN113225549B (en) * | 2021-04-19 | 2022-07-01 | 广州朗国电子科技股份有限公司 | VR intelligence life system |
| CN113808728A (en) * | 2021-09-02 | 2021-12-17 | 武汉联影医疗科技有限公司 | Medical equipment management method and device, computer equipment and storage medium |
| CN113885345B (en) * | 2021-10-29 | 2024-03-19 | 广州市技师学院(广州市高级技工学校、广州市高级职业技术培训学院、广州市农业干部学校) | Interaction method, device and equipment based on intelligent home simulation control system |
| CN114885031A (en) * | 2022-03-21 | 2022-08-09 | 深圳绿米联创科技有限公司 | Interactive processing method and device based on equipment control and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105629744A (en) * | 2015-04-30 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Control method of smart home, control system, terminal and smart home system |
| CN205809568U (en) * | 2016-06-21 | 2016-12-14 | 武汉方格信息技术有限公司 | A kind of combination VR and the intelligent domestic system of robot |
| CN106249607A (en) * | 2016-07-28 | 2016-12-21 | 桂林电子科技大学 | Virtual Intelligent household analogue system and method |
| CN106445156A (en) * | 2016-09-29 | 2017-02-22 | 宇龙计算机通信科技(深圳)有限公司 | Method, device and terminal for intelligent home device control based on virtual reality |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102306020A (en) * | 2011-06-27 | 2012-01-04 | 中国科学院深圳先进技术研究院 | Virtual modeling thing internet and cloud computing combining method and system |
| CN105117017A (en) * | 2015-09-07 | 2015-12-02 | 众景视界（北京）科技有限公司 | Gloves used in interaction control of virtual reality and augmented reality |
| CN106681479A (en) * | 2015-11-05 | 2017-05-17 | 丰唐物联技术（深圳）有限公司 | User interaction method and system based on virtual reality |
| CN105376125B (en) * | 2015-12-08 | 2018-12-18 | 深圳众乐智府科技有限公司 | A kind of smart home system control method and device |
| CN106200983B (en) * | 2016-07-25 | 2019-06-18 | 圣域科技（深圳）有限公司 | A kind of system of combination virtual reality and BIM realization virtual reality scenario architectural design |
-
2017
- 2017-08-14 CN CN201710692424.2A patent/CN107358007B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105629744A (en) * | 2015-04-30 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Control method of smart home, control system, terminal and smart home system |
| CN205809568U (en) * | 2016-06-21 | 2016-12-14 | 武汉方格信息技术有限公司 | A kind of combination VR and the intelligent domestic system of robot |
| CN106249607A (en) * | 2016-07-28 | 2016-12-21 | 桂林电子科技大学 | Virtual Intelligent household analogue system and method |
| CN106445156A (en) * | 2016-09-29 | 2017-02-22 | 宇龙计算机通信科技(深圳)有限公司 | Method, device and terminal for intelligent home device control based on virtual reality |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107358007A (en) | 2017-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107358007B (en) | It controls the method, apparatus of smart home system and calculates readable storage medium storing program for executing | |
| US11206373B2 (en) | Method and system for providing mixed reality service | |
| JP7130057B2 (en) | Hand Keypoint Recognition Model Training Method and Device, Hand Keypoint Recognition Method and Device, and Computer Program | |
| EP3526658B1 (en) | Method and device for obtaining real time status and controlling of transmitting devices | |
| CN107703872B (en) | Terminal control method and device of household appliance and terminal | |
| CN103631768B (en) | Collaboration data editor and processing system | |
| CN104735057B (en) | Share the method and device of equipment control | |
| CN110495819B (en) | Robot control method, robot, terminal, server and control system | |
| CN109920065A (en) | Methods of exhibiting, device, equipment and the storage medium of information | |
| CN106791893A (en) | Net cast method and device | |
| CN108885495A (en) | Electronic equipment and the in the electronic device method of offer information | |
| CN106209800B (en) | Equipment Authority sharing method and apparatus | |
| CN105205479A (en) | Human face value evaluation method, device and terminal device | |
| KR102481486B1 (en) | Method and apparatus for providing audio | |
| WO2015102866A1 (en) | Physical object discovery | |
| US20200184968A1 (en) | Artificial intelligence device | |
| US10038834B2 (en) | Video call method and device | |
| CN104484858B (en) | Character image processing method and processing device | |
| CN106488109A (en) | Mobile device and its control method | |
| US20240078703A1 (en) | Personalized scene image processing method, apparatus and storage medium | |
| CN103067727A (en) | Three-dimensional 3D glasses and three-dimensional 3D display system | |
| WO2022048398A1 (en) | Multimedia data photographing method and terminal | |
| CN107005611A (en) | Attachment arrangement and its method for controlling electronic installation | |
| CN110519134A (en) | Home equipment control method and device, storage medium | |
| CN107371052A (en) | Apparatus control method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |