CN111245688A - Method and system for intelligently controlling electrical equipment based on indoor environment - Google Patents
Method and system for intelligently controlling electrical equipment based on indoor environment Download PDFInfo
- Publication number
- CN111245688A CN111245688A CN201911371121.6A CN201911371121A CN111245688A CN 111245688 A CN111245688 A CN 111245688A CN 201911371121 A CN201911371121 A CN 201911371121A CN 111245688 A CN111245688 A CN 111245688A
- Authority
- CN
- China
- Prior art keywords
- indoor
- module
- equipment
- master control
- nlp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000009471 action Effects 0.000 claims abstract description 22
- 238000013135 deep learning Methods 0.000 claims abstract description 8
- 238000005516 engineering process Methods 0.000 claims abstract description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 30
- 238000004458 analytical method Methods 0.000 claims description 25
- 230000033001 locomotion Effects 0.000 claims description 16
- 238000012549 training Methods 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 10
- 238000003491 array Methods 0.000 claims description 7
- 238000013136 deep learning model Methods 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims 1
- 239000002994 raw material Substances 0.000 claims 1
- 230000000694 effects Effects 0.000 description 9
- 230000036544 posture Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 238000003860 storage Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000000739 chaotic effect Effects 0.000 description 3
- 239000003795 chemical substances by application Substances 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 241000203475 Neopanax arboreus Species 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000005389 magnetism Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007664 blowing Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L12/2816—Controlling appliance services of a home automation network by calling their functionalities
- H04L12/282—Controlling appliance services of a home automation network by calling their functionalities based on user interaction within the home
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/113—Recognition of static hand signs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/66—Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Acoustics & Sound (AREA)
- General Engineering & Computer Science (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- General Health & Medical Sciences (AREA)
- Automation & Control Theory (AREA)
- Selective Calling Equipment (AREA)
Abstract
The invention discloses a method and a system for intelligently controlling electrical equipment based on indoor, which comprises the following steps that equipment interconnection modules 100 are interconnected, and a master control equipment control module 500 and indoor intelligent electrical equipment 101 are connected with the same gateway; positioning the actual position of the indoor personnel by using an indoor auxiliary positioning module 200; capturing gestures or signs of the indoor person with a recognition module 300; judging the action pointing intention of the indoor personnel by combining machine vision and deep learning; the master control device control module 500 collects scene voice data; the NLP speech recognition parsing module 400 parses the speech data and converts the speech data into a corresponding control instruction; the general control device control module 500 receives the control instruction and controls the state of the indoor electric device. The method can control specific electric equipment in indoor complex scenes, has unique target and is more accurate, and the scheme is more intelligent, convenient to operate and rich in science and technology sense by applying various AI technologies in the comprehensive scheme of intelligent home service.
Description
Technical Field
The invention relates to the technical field of intelligent home furnishing, in particular to a method and a system for intelligently controlling electrical equipment based on indoor.
Background
1994-1999 are the first development stage of smart home, the whole industry is still in the stages of concept familiarity and product cognition, no professional smart home manufacturer appears at this time, only Shenzhen has two companies engaged in US X-10 smart home agency sales to engage in import retail business, and products are sold to European and American users living in China. Fifty intelligent home research and development enterprises are established successively in 2000-2005, mainly focusing on Shenzhen, Shanghai, Tianjin, Beijing, Hangzhou, Xiamen and other places. The marketing and technical training system of the smart home is gradually improved, and foreign smart home products basically do not enter the domestic market at this stage. After 2005, due to the wild growth and the malignant competition of the smart home enterprises in the last stage, the smart home industry is greatly affected negatively: the intelligent home system has the advantages that functions of the intelligent home system are exaggerated excessively, the effect cannot be achieved actually, a manufacturer only needs to develop an agent and neglects training and support of the agent, so that the agent is difficult to operate, and the product is unstable, so that the user complaint rate is high. Industry users and media begin to question the actual effect of smart home, the original blowing becomes cautious, and the market sales is also increased and slowed down in several years, and even the sales volume is reduced in some areas. In 2005-2007, more than 20 smart home manufacturing enterprises exited the market, and there are not a few who carry out business turn-over among the various local agencies. Many persistent smart home enterprises have experienced a reduced-scale experience over the years. In this period, foreign smart home brands enter the Chinese market in a hidden layout manner, and the main foreign smart home brands which are active in the market enter the Chinese market at this period, and some living enterprises in China gradually find their own development directions.
In the prior art, indoor equipment is actually and interactively controlled by indoor personnel and master control equipment (such as an AI sound box), but often an object of the electric equipment is operated wrongly or is operated complicatedly, if the similarity of the electric equipment is higher or a plurality of similar equipment are arranged at the same position, the condition of obvious control error occurs, a specific electric equipment cannot be accurately controlled, and a chaotic condition exists.
Disclosure of Invention
This section is for the purpose of summarizing some aspects of embodiments of the invention and to briefly introduce some preferred embodiments. In this section, as well as in the abstract and the title of the invention of this application, simplifications or omissions may be made to avoid obscuring the purpose of the section, the abstract and the title, and such simplifications or omissions are not intended to limit the scope of the invention.
The invention is provided in view of the problems of error and complicated operation of the conventional electric equipment operation objects.
Therefore, the invention provides the method and the system for controlling the electrical equipment based on indoor intelligence, which can control the electrical equipment in indoor complex scenes more accurately and is simple and convenient to operate.
In order to solve the technical problems, the invention provides the following technical scheme: the equipment interconnection module establishes interconnection, and connects the master control equipment control module and the indoor intelligent electrical equipment with the same gateway; positioning the actual position of indoor personnel by using an indoor auxiliary positioning module; capturing gestures or signs of the indoor person with a recognition module; judging the action pointing intention of the indoor personnel by combining machine vision and deep learning; the master control equipment control module collects scene voice data; the NLP voice recognition analysis module analyzes the voice data and converts the voice data into a corresponding control instruction; and the master control equipment control module controls the state of the indoor electric equipment.
As a preferable scheme of the method for intelligently controlling electrical equipment based on indoor described in the present invention, wherein: establishing interconnection comprises the steps of opening the gateway and finding a master control device on the terminal; the general control equipment is configured and networked; establishing an intelligent network of the indoor intelligent electrical equipment which can be interconnected with the master control equipment on the terminal; and the master control equipment monitors and manages the state of the indoor intelligent electrical equipment.
As a preferable scheme of the method for intelligently controlling electrical equipment based on indoor described in the present invention, wherein: providing the auxiliary judgment of the indoor personnel position comprises deploying corresponding indoor intelligent electrical equipment at an indoor proper position; the indoor intelligent electrical equipment and the master control equipment are interconnected with one gateway; position information is determined using indoor assisted positioning techniques.
As a preferable scheme of the method for intelligently controlling electrical equipment based on indoor described in the present invention, wherein: the indoor auxiliary positioning further specifically comprises the step of determining the range of an area to be positioned of the indoor space; dividing the area to be positioned into a plurality of small-range positioning areas; correspondingly installing an infrared sensor and four microphone arrays positioned at different spatial positions in the small-range positioning area respectively; detecting the body surface temperature of the moving object in real time by using the infrared sensor, and determining the only small-range positioning area where the moving object is located; and accurately positioning the moving object by utilizing the corresponding four microphone arrays in the current region.
As a preferable scheme of the method for intelligently controlling electrical equipment based on indoor described in the present invention, wherein: the motion auxiliary judgment is provided by training an algorithm model in advance and deploying the algorithm model in the recognition module; capturing a posture or gesture of the indoor person when the indoor person wants to operate the indoor intelligent electrical equipment by using a camera; and judging the action pointing angle and direction of the indoor personnel according to the machine vision and the deep learning model.
As a preferable scheme of the method for intelligently controlling electrical equipment based on indoor described in the present invention, wherein: the method specifically comprises the steps of marking the collected image data of various postures and gesture states with relevant characteristics; performing algorithm training by using the deep learning gesture recognition algorithm in combination with various marked gesture and posture state picture data to obtain the algorithm model; testing and optimizing the accuracy of the algorithm model; collecting characteristic gestures and posture state data of the moving object by using the camera, and inputting the data into the algorithm model; and the algorithm model judges the angle and the orientation probability of the current gesture or posture to finish the action direction identification.
As a preferable scheme of the method for intelligently controlling electrical equipment based on indoor described in the present invention, wherein: providing voice auxiliary judgment comprises the steps of collecting scene voice data by using the master control equipment control module; the NLP voice recognition analysis module analyzes the voice data and converts the voice data into a corresponding control instruction; the equipment initialization starting module receives the control instruction and configures related parameters; and the master control equipment control module controls the state of the indoor electric equipment.
As a preferable scheme of the system for intelligently controlling electrical equipment indoors according to the present invention, wherein: the equipment interconnection module is connected with the master control equipment control module through a gateway and comprises indoor intelligent electrical equipment, and state information of the indoor intelligent electrical equipment is presented in real time through connection of the terminal network; the indoor auxiliary positioning module receives the position information of the indoor personnel, and the positioning unit judges the movement position information of the personnel in real time through the infrared sensor and tracks the movement position information through the connected indoor intelligent electrical equipment; the recognition module is connected with and penetrates through the indoor auxiliary positioning module, and the camera captures the action of indoor personnel, converts the action into picture data and transmits the picture data into the NLP voice recognition analysis module.
As a preferable scheme of the system for intelligently controlling electrical equipment indoors according to the present invention, wherein: the NLP voice recognition and analysis module comprises an intention recognition unit and an instruction analysis unit, after voice data and the action picture data are obtained, the intention recognition unit finds out relevant intention information and transmits the relevant intention information to the instruction analysis unit, the action instruction of the person is analyzed, and a real intention is obtained through an NLP technology; the master control equipment control module is connected with the NLP voice recognition and analysis module, the master control equipment can supervise and control state information of the indoor intelligent electric equipment and collect scene voice, and after the NLP voice recognition and analysis module analyzes the intention of the personnel action instruction, the master control equipment sends a control command to the equipment initialization starting module; the equipment initialization starting module is connected with the NLP voice recognition analysis module, converts voice recognition into a corresponding control instruction after receiving an operation instruction, configures relevant parameters and executes the relevant operation command.
The invention has the beneficial effects that: the intelligent household intelligent control system can control specific electric equipment in indoor complex scenes, has unique target and is more accurate, and the scheme is more intelligent, convenient to operate and rich in technological sense by applying a plurality of AI technologies in an integrated scheme of intelligent household business.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise. Wherein:
fig. 1 is a schematic overall flow chart of a method for intelligently controlling an electrical device based on indoor provided by the invention;
FIG. 2 is a flow chart of model training of a method for controlling electrical equipment based on indoor intelligence provided by the invention;
fig. 3 is a schematic diagram of module distribution of a system for intelligently controlling electrical equipment indoors according to the present invention;
fig. 4 is another schematic diagram of a module structure of a system for intelligently controlling electrical devices indoors according to the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, specific embodiments accompanied with figures are described in detail below, and it is apparent that the described embodiments are a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making creative efforts based on the embodiments of the present invention, shall fall within the protection scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.
Furthermore, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
The present invention will be described in detail with reference to the drawings, wherein the cross-sectional views illustrating the structure of the device are not enlarged partially in general scale for convenience of illustration, and the drawings are only exemplary and should not be construed as limiting the scope of the present invention. In addition, the three-dimensional dimensions of length, width and depth should be included in the actual fabrication.
Meanwhile, in the description of the present invention, it should be noted that the terms "upper, lower, inner and outer" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of describing the present invention and simplifying the description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation and operate, and thus, cannot be construed as limiting the present invention. Furthermore, the terms first, second, or third are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
The terms "mounted, connected and connected" in the present invention are to be understood broadly, unless otherwise explicitly specified or limited, for example: can be fixedly connected, detachably connected or integrally connected; they may be mechanically, electrically, or directly connected, or indirectly connected through intervening media, or may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Example 1
The deep learning network model is formed by combining a lightweight classification network MobileNet and a target detection network SSD in a convolutional neural network, and the improved SSD algorithm is finely adjusted based on the influence of different sizes of input pictures on the model and the introduction of a porous convolution. To increase the recognition speed, a Deepwise convolution is used to further reduce the network parameters and the computational load. Where MobileNet used a v1 network. Intercepting the first 12 layers of convolutional layers of the v1 network as a basic feature extraction layer of the network, and then adding 6 layers of auxiliary feature extraction networks to form a backbone network of the algorithm. An offset value exists between the label border and the default box, so the offset value is taken as the content of the web learning. And the end-to-end loss function is calculated by combining the classification errors, and the back propagation calculation and updating are performed, so that the recognition speed is greatly improved on the premise of ensuring better gesture recognition precision, a feasible algorithm is provided for real-time gesture recognition, and the use requirements of various embedded platform scenes can be met.
Referring to fig. 1 and 2, for a first embodiment of the present invention, there is provided a method for controlling an electrical device based on indoor intelligence, comprising the steps of:
s1: the device interconnection module 100 establishes interconnection, and connects the master control device control module 500 and the indoor intelligent electrical device 101 to the same gateway. In which it is to be noted that,
opening a gateway, and finding a master control device 501 on the terminal;
configuring and networking the master control equipment 501;
an intelligent network of the indoor intelligent electrical equipment 101 which can be interconnected with the master control equipment 501 is established on the terminal;
the general control device 501 monitors and manages the state of the indoor intelligent electric device 101.
And S2, positioning the actual position of the indoor person by using the indoor auxiliary positioning module 200. It should be noted that, the providing of the auxiliary judgment of the position of the indoor personnel includes:
deploying corresponding indoor intelligent electrical equipment 101 at an indoor appropriate position;
the indoor intelligent electric equipment 101 and the master control equipment 501 are interconnected with one gateway;
position information is determined using indoor assisted positioning techniques.
Further, the positioning judgment specifically includes,
determining the range of an area to be positioned in indoor space;
dividing a region to be positioned into a plurality of small-range positioning regions;
correspondingly installing infrared sensors and four microphone arrays positioned at different spatial positions in a small-range positioning area respectively;
detecting the body surface temperature of the moving object in real time by using an infrared sensor, and determining the only small-range positioning area where the moving object is located;
and precisely positioning the moving object by utilizing the corresponding four microphone arrays in the current area.
S3: gestures or signs of the persons in the room are captured using the recognition module 300. It should be further noted that the providing of the motion assistance determination includes:
training an algorithm model in advance and deploying the algorithm model in the recognition module 300;
capturing a posture or gesture of an indoor person when the indoor person wants to operate the indoor intelligent electrical equipment 101 by using the camera 301;
and judging the action pointing angle and direction of the indoor personnel according to the machine vision and deep learning model.
S4: and judging the action pointing intention of the indoor personnel by combining machine vision and deep learning. Referring to fig. 2, in this step, it should be further explained that the obtaining of the algorithm model specifically includes:
marking the collected various gesture and gesture state picture data with relevant characteristics;
performing algorithm training by using a deep learning gesture recognition algorithm in combination with various marked gesture and posture state picture data to obtain an algorithm model;
testing and optimizing the accuracy of the algorithm model;
the camera 301 is used for collecting the characteristic gestures and posture state data of the moving object and inputting the data into the algorithm model;
and the algorithm model judges the angle and the orientation probability of the current gesture or posture to finish the action direction identification.
S5: the master control device control module 500 collects scene voice data.
S6: the NLP speech recognition parsing module 400 parses the speech data and converts the speech data into a corresponding control command.
S7: the device initialization start module 600 receives the control command and configures the relevant parameters.
S8: the master control device control module 500 controls the state of the indoor electric devices.
Furthermore, the collected pictures comprise complex gesture structures, illumination, backgrounds, environments and other factors, and the accuracy rate of recognition can be influenced by directly training and recognizing, so that a gesture recognition scheme combining a skin color model and a convolutional neural network is adopted. The method comprises the following steps of preprocessing a gesture picture containing a complex background, selecting a proper skin color model, determining the region position of a gesture in an original image, and separating the region from the background; and secondly, extracting and reconstructing the gesture area subjected to marking separation by using algorithms such as morphological operation, a filtering algorithm, a connected domain marking method and the like. After the work of preprocessing the original image is finished, determining a network structure of the constructed convolutional neural network, the number and the size of convolutional kernels, a pooling method and size, an activation function and a classification method. The method comprises the steps that a training model acquires 25 groups of data pictures of gestures, the data sets comprise motion pictures of pointing, stone, cloth, ok, specific center, five-finger stretching and the like, the picture data subjected to manual marking processing are prepared and divided into a training set, a verification set and a test set according to a certain proportion, and then the training set is input into a MobileNet-SSD network for iterative learning training through a preprocessing process until the model training is completed. And repeating the steps for multiple times of experiments to obtain the model. And finally, selecting the model with the best performance effect on the verification set, testing on the test set to obtain related data, and selecting a model with excellent effect to be placed in the recognition module to perform prediction judgment to finish action direction identification. For example, a single-finger or double-finger pointing, five-finger stretching, or the like is considered to have a desire to operate the device, the prediction and judgment accuracy reaches about 95%, and the motion is considered to be confirmed.
Preferably, the present invention adopts a strategy of combining positioning auxiliary judgment, action auxiliary judgment and voice auxiliary judgment, so as to greatly improve the accuracy of the operating device, and when the positioning specific position data and the gesture confirmation action data of the operating device are input to the master control device control module 500, the specific device to be operated is determined by combining voice recognition, for example, the voice contains the indicators such as "this", "that", "this", "close to me", and the like, so as to provide accurate judgment, and the control operation is completed through the master control device control module 500. In the prior art, only voice instruction control equipment is provided, on a platform or an APP, a specific ID or name of the equipment is set, such as 'a ceiling lamp of a kitchen' and 'an air conditioner of a bedroom', the name is set to be unique, for example, a 'lamp of a living room' is turned on, if a plurality of lamps exist, a chaotic condition occurs, and the target is not clear; through experimental tests, the control accuracy of the scheme of the invention is improved by more than 60-70% compared with the control accuracy of the existing scene only with a voice instruction.
Preferably, the method of the invention determines the accuracy of the positioning precision by using a positioning scheme of fusing an infrared sensor and a microphone array, and sets a plurality of reference monitoring points uniformly to cover the space of the area to be positioned, correspondingly defines coordinates, sets an infrared sensor receiving device on the plurality of reference monitoring points to radiate out small-range positioning areas, and sets four microphone arrays in each small-range positioning area to complete the installation. With infrared sensor, microphone array, bluetooth, four groups of independent location experiments of earth magnetism, wherein infrared sensor, bluetooth, earth magnetism are all setting up independent test on a plurality of reference monitoring points, and the microphone array is removing independent test under a plurality of infrared sensor, verifies the location scheme advantage that infrared sensor and microphone array fuse.
The traditional technical scheme is complex in operation, names and types of indoor electrical equipment are unified, and chaotic operation is easy to occur when a plurality of similar equipment are available. Compared with the traditional method, the method has the advantages of high control indoor complex scenes, and the method has the advantages of unique target, higher accuracy, higher intelligence and convenience in operation. In the embodiment, the traditional voice recognition control and the method are respectively added with indoor auxiliary positioning and gesture recognition technologies for real-time measurement and comparison. The following table mainly takes an indoor specific scene as an example, and compares the difference and the accuracy of the control effect of the traditional method and the control effect of the method, and the table specifically shows that:
the test data is 10 groups of samples, and the traditional method only succeeds in operating 4 groups; the method has 10 groups of successful operation, improves the success probability of the operation by 150 percent, has obvious effect and accurate operation; of course, the 10 groups of samples are only artificially defined samples, and generally, the operation success probability is improved to be about 60-70%.
Example 2
Referring to fig. 3 and 4, a second embodiment of the present invention, which is different from the first embodiment, provides a system for controlling electrical devices based on indoor intelligence, the system comprising a device interconnection module 100, an indoor auxiliary positioning module 200, an identification module 300, an NLP speech recognition parsing module 400, a general control device control module 500 and a device initialization starting module 600. The equipment interconnection module 100 is connected with the master control equipment control module 500 through a gateway, and comprises indoor intelligent electrical equipment 101, and state information of the indoor intelligent electrical equipment 101 is presented in real time through connection of a terminal network; the indoor auxiliary positioning module 200 receives indoor personnel position information, and the positioning unit 201 judges personnel activity position information in real time through an infrared sensor and tracks the personnel activity position information through the connected indoor intelligent electrical equipment 101; the recognition module 300 is connected to and penetrates through the indoor auxiliary positioning module 200, and the camera 301 captures the motion of indoor personnel, converts the motion into picture data, and transmits the picture data to the NLP speech recognition analysis module 400.
Further, the NLP speech recognition and analysis module 400 includes an intention recognition unit 401 and an instruction analysis unit 402, and after the speech data and the motion picture data are acquired, the intention recognition unit 401 finds out the relevant intention information and transmits the relevant intention information to the instruction analysis unit 402, so as to analyze the motion instruction of the person, and obtain the real intention through the NLP technology; the master control device control module 500 is connected with the NLP speech recognition analysis module 400, the master control device 501 can supervise and control the state information of the indoor intelligent electrical equipment 101 and collect scene speech, and when the NLP speech recognition analysis module 400 analyzes the intention of a person action instruction, the master control device 501 sends a control command to the device initialization start-up module 600; the device initialization starting module 600 is connected to the NLP speech recognition parsing module 400, and after receiving the operation command, converts the speech recognition into a corresponding control command, configures the relevant parameters, and executes the relevant operation command.
It should be recognized that embodiments of the present invention can be realized and implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, according to the methods and figures described in the detailed description. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.
Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein. A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.
As used in this application, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being: a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of example, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and/or thread of execution and a component can be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the internet with other systems by way of the signal).
It should be noted that the above-mentioned embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, which should be covered by the claims of the present invention.
Claims (9)
1. A method for intelligently controlling electrical equipment based on indoor is characterized in that: comprises the steps of (a) preparing a mixture of a plurality of raw materials,
the equipment interconnection module (100) establishes interconnection, and connects the master control equipment control module (500) and the indoor intelligent electrical equipment (101) with the same gateway;
positioning the actual position of indoor personnel by utilizing an indoor auxiliary positioning module (200);
capturing a gesture or gesture of the indoor person with a recognition module (300);
judging the action pointing intention of the indoor personnel by combining machine vision and deep learning;
the master control equipment control module (500) collects scene voice data;
the NLP voice recognition analysis module (400) analyzes the voice data and converts the voice data into a corresponding control instruction;
and the master control equipment control module (500) receives the control instruction and controls the state of the indoor electric equipment.
2. The method for intelligently controlling electric devices based on indoors as claimed in claim 1, wherein: establishing the interconnection includes establishing the connection between the first and second nodes,
opening the gateway, and finding a master control device (501) on the terminal;
configuring the general control device (501) to be networked;
establishing an intelligent network of the indoor intelligent electrical equipment (101) which can be interconnected with the master control equipment (501) on the terminal;
the master control device (501) monitors and manages the state of the indoor intelligent electrical equipment (101).
3. The method for controlling electric devices based on indoor intelligence of claim 1 or 2, wherein: providing the indoor personnel location assistance determination includes,
deploying the respective indoor smart appliances (101) in an indoor location;
the indoor intelligent electric appliance equipment (101) and the master control equipment (501) are interconnected with one gateway;
position information is determined using indoor assisted positioning techniques.
4. The method for intelligently controlling electric appliances based on indoors as claimed in claim 3, wherein: the indoor auxiliary positioning may further specifically include,
determining the range of an area to be positioned in indoor space;
dividing the area to be positioned into a plurality of small-range positioning areas;
correspondingly installing an infrared sensor and four microphone arrays positioned at different spatial positions in the small-range positioning area respectively;
detecting the body surface temperature of the moving object in real time by using the infrared sensor, and determining the only small-range positioning area where the moving object is located;
and accurately positioning the moving object by utilizing the corresponding four microphone arrays in the current region.
5. The method for intelligently controlling electric devices based on indoors as claimed in claim 1 or 4, wherein: providing the motion assistance determination includes providing the motion assistance determination,
training an algorithm model in advance and deploying in the recognition module (300);
capturing a gesture or gesture of the indoor person when the indoor person wants to operate the indoor intelligent electrical equipment (101) by using a camera (301);
and judging the action pointing angle and direction of the indoor personnel according to the machine vision and the deep learning model.
6. The method for intelligently controlling electric devices based on indoors as claimed in claim 5, wherein: the obtaining of the algorithm model specifically includes that,
marking the collected various gesture and gesture state picture data with relevant characteristics;
performing algorithm training by using the deep learning gesture recognition algorithm in combination with various marked gesture and posture state picture data to obtain the algorithm model;
testing and optimizing the accuracy of the algorithm model;
collecting characteristic gestures and posture state data of the moving object by using the camera (301), and inputting the data into the algorithm model;
and the algorithm model judges the angle and the orientation probability of the current gesture or posture to finish the action direction identification.
7. The method for intelligently controlling electric devices based on indoors as claimed in claim 1, wherein: providing a voice-assisted determination includes providing a voice-assisted determination,
acquiring scene voice data by using the master control equipment control module (500);
the NLP voice recognition analysis module (400) analyzes the voice data and converts the voice data into a corresponding control instruction;
the equipment initialization starting module (600) receives the control instruction and configures related parameters;
and a master control equipment control module (500) controls the state of the indoor electric equipment.
8. The utility model provides a system based on indoor intelligent control electrical equipment which characterized in that: comprises an equipment interconnection module (100), an indoor auxiliary positioning module (200) and an identification module (300),
the equipment interconnection module (100) is connected with the master control equipment control module (500) through a gateway and comprises indoor intelligent electric equipment (101), and state information of the indoor intelligent electric equipment (101) is presented in real time through connection of the terminal network;
the indoor auxiliary positioning module (200) receives the position information of indoor personnel, and the positioning unit (201) judges the movement position information of the personnel in real time through the infrared sensor and tracks the movement position information through the connected indoor intelligent electrical equipment (101);
the recognition module (300) is connected with and penetrates through the indoor auxiliary positioning module (200), and the camera (301) captures the motion of indoor personnel, converts the motion into picture data and transmits the picture data to the NLP voice recognition analysis module (400).
9. The system for intelligently controlling electric devices based on indoors according to claim 8, wherein: also comprises an NLP voice recognition and analysis module (400), a master control equipment control module (500) and an equipment initialization starting module (600),
the NLP voice recognition and analysis module (400) comprises an intention recognition unit (401) and an instruction analysis unit (402), after voice data and the action picture data are obtained, the intention recognition unit (401) finds out relevant intention information and transmits the intention information to the instruction analysis unit (402), the action instructions of the personnel are analyzed, and real intentions are obtained through an NLP technology;
the master control equipment control module (500) is connected with the NLP voice recognition analysis module (400), the master control equipment (501) can supervise and control state information of the indoor intelligent electric equipment (101) and collect scene voice, and after the NLP voice recognition analysis module (400) analyzes the intention of the personnel action instruction, the master control equipment (501) sends a control command to the equipment initialization starting module (600);
the equipment initialization starting module (600) is connected with the NLP voice recognition analysis module (400), and after receiving an operation instruction, converts voice recognition into a corresponding control instruction, configures related parameters, and executes the related operation command.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911371121.6A CN111245688A (en) | 2019-12-26 | 2019-12-26 | Method and system for intelligently controlling electrical equipment based on indoor environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911371121.6A CN111245688A (en) | 2019-12-26 | 2019-12-26 | Method and system for intelligently controlling electrical equipment based on indoor environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111245688A true CN111245688A (en) | 2020-06-05 |
Family
ID=70871809
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911371121.6A Pending CN111245688A (en) | 2019-12-26 | 2019-12-26 | Method and system for intelligently controlling electrical equipment based on indoor environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111245688A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112684711A (en) * | 2020-12-24 | 2021-04-20 | 青岛理工大学 | Interactive identification method for human behavior and intention |
| CN112750437A (en) * | 2021-01-04 | 2021-05-04 | 欧普照明股份有限公司 | Control method, control device and electronic equipment |
| CN113325723A (en) * | 2021-04-27 | 2021-08-31 | 深圳市壹厘米科技有限公司 | Personalized intelligent scene generation control method, device, equipment and storage medium |
| CN113362823A (en) * | 2021-06-08 | 2021-09-07 | 深圳市同行者科技有限公司 | Multi-terminal response method, device, equipment and storage medium of household appliance |
| CN114640568A (en) * | 2022-05-18 | 2022-06-17 | 南京邮电大学 | Network intelligent management and control architecture system based on deep reinforcement learning and operation method |
| CN115390469A (en) * | 2022-08-19 | 2022-11-25 | 青岛海尔科技有限公司 | Control method, system and storage medium for household electrical appliance |
| CN116098536A (en) * | 2021-11-08 | 2023-05-12 | 青岛海尔科技有限公司 | Robot control method and device |
| CN117219071A (en) * | 2023-09-20 | 2023-12-12 | 北京惠朗时代科技有限公司 | Voice interaction service system based on artificial intelligence |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202563884U (en) * | 2011-11-18 | 2012-11-28 | 深圳市派高模业有限公司 | Voice recognition processor and intelligent device |
| CN102824092A (en) * | 2012-08-31 | 2012-12-19 | 华南理工大学 | Intelligent gesture and voice control system of curtain and control method thereof |
| CN202801140U (en) * | 2012-08-31 | 2013-03-20 | 华南理工大学 | Smart gesture and voice control system for curtain |
| CN203178799U (en) * | 2013-04-26 | 2013-09-04 | 河北工业大学 | Remote-control intelligent interactive household mobile system |
| CN107179700A (en) * | 2017-07-03 | 2017-09-19 | 杭州善居科技有限公司 | A kind of intelligent home control system and method based on Alljoyn and machine learning |
| CN108256504A (en) * | 2018-02-11 | 2018-07-06 | 苏州笛卡测试技术有限公司 | A kind of Three-Dimensional Dynamic gesture identification method based on deep learning |
| US20190129688A1 (en) * | 2017-10-31 | 2019-05-02 | Baidu Usa Llc | System and method for controlling colors of smart lights based on user intent using natural language processing |
| CN110244570A (en) * | 2019-05-22 | 2019-09-17 | 安徽三品技术服务有限公司 | Language house keeper's house system based on posture induction |
| US20190371322A1 (en) * | 2018-05-31 | 2019-12-05 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method, apparatus and system for testing intelligent voice device |
| CN110597077A (en) * | 2019-09-25 | 2019-12-20 | 的卢技术有限公司 | Method and system for realizing intelligent scene switching based on indoor positioning |
-
2019
- 2019-12-26 CN CN201911371121.6A patent/CN111245688A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202563884U (en) * | 2011-11-18 | 2012-11-28 | 深圳市派高模业有限公司 | Voice recognition processor and intelligent device |
| CN102824092A (en) * | 2012-08-31 | 2012-12-19 | 华南理工大学 | Intelligent gesture and voice control system of curtain and control method thereof |
| CN202801140U (en) * | 2012-08-31 | 2013-03-20 | 华南理工大学 | Smart gesture and voice control system for curtain |
| CN203178799U (en) * | 2013-04-26 | 2013-09-04 | 河北工业大学 | Remote-control intelligent interactive household mobile system |
| CN107179700A (en) * | 2017-07-03 | 2017-09-19 | 杭州善居科技有限公司 | A kind of intelligent home control system and method based on Alljoyn and machine learning |
| US20190129688A1 (en) * | 2017-10-31 | 2019-05-02 | Baidu Usa Llc | System and method for controlling colors of smart lights based on user intent using natural language processing |
| CN108256504A (en) * | 2018-02-11 | 2018-07-06 | 苏州笛卡测试技术有限公司 | A kind of Three-Dimensional Dynamic gesture identification method based on deep learning |
| US20190371322A1 (en) * | 2018-05-31 | 2019-12-05 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method, apparatus and system for testing intelligent voice device |
| CN110244570A (en) * | 2019-05-22 | 2019-09-17 | 安徽三品技术服务有限公司 | Language house keeper's house system based on posture induction |
| CN110597077A (en) * | 2019-09-25 | 2019-12-20 | 的卢技术有限公司 | Method and system for realizing intelligent scene switching based on indoor positioning |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112684711A (en) * | 2020-12-24 | 2021-04-20 | 青岛理工大学 | Interactive identification method for human behavior and intention |
| CN112684711B (en) * | 2020-12-24 | 2022-10-11 | 青岛理工大学 | Interactive recognition method for human behavior and intention |
| CN112750437A (en) * | 2021-01-04 | 2021-05-04 | 欧普照明股份有限公司 | Control method, control device and electronic equipment |
| CN113325723A (en) * | 2021-04-27 | 2021-08-31 | 深圳市壹厘米科技有限公司 | Personalized intelligent scene generation control method, device, equipment and storage medium |
| CN113362823A (en) * | 2021-06-08 | 2021-09-07 | 深圳市同行者科技有限公司 | Multi-terminal response method, device, equipment and storage medium of household appliance |
| CN116098536A (en) * | 2021-11-08 | 2023-05-12 | 青岛海尔科技有限公司 | Robot control method and device |
| CN116098536B (en) * | 2021-11-08 | 2024-07-19 | 青岛海尔科技有限公司 | Robot control method and device |
| CN114640568A (en) * | 2022-05-18 | 2022-06-17 | 南京邮电大学 | Network intelligent management and control architecture system based on deep reinforcement learning and operation method |
| CN115390469A (en) * | 2022-08-19 | 2022-11-25 | 青岛海尔科技有限公司 | Control method, system and storage medium for household electrical appliance |
| CN117219071A (en) * | 2023-09-20 | 2023-12-12 | 北京惠朗时代科技有限公司 | Voice interaction service system based on artificial intelligence |
| CN117219071B (en) * | 2023-09-20 | 2024-03-15 | 北京惠朗时代科技有限公司 | Voice interaction service system based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111245688A (en) | Method and system for intelligently controlling electrical equipment based on indoor environment | |
| CN111543902B (en) | Floor cleaning method and device, intelligent cleaning equipment and storage medium | |
| CN107528753B (en) | Intelligent household voice control method, intelligent equipment and device with storage function | |
| CN107965828B (en) | Air suction state control method of range hood and range hood | |
| CN107422859B (en) | Gesture-based regulation and control method and device, computer-readable storage medium and air conditioner | |
| CN103375880B (en) | Remote control device and method for air conditioner | |
| CN110535732B (en) | Equipment control method and device, electronic equipment and storage medium | |
| WO2022227775A1 (en) | Method and device for controlling air conditioner, and air conditioner | |
| CN105912142B (en) | A kind of note step and Activity recognition method based on acceleration sensor | |
| CN111128157B (en) | Wake-up-free voice recognition control method for intelligent household appliance, computer readable storage medium and air conditioner | |
| US10769909B1 (en) | Using sensor data to detect events | |
| CN109595757B (en) | Control method and device of air conditioner and air conditioner with control device | |
| JP6713057B2 (en) | Mobile body control device and mobile body control program | |
| CN110426962A (en) | A kind of control method and system of smart home device | |
| CN108474576A (en) | Control device, control method and program | |
| CN109542233A (en) | A kind of lamp control system based on dynamic gesture and recognition of face | |
| CN111413877A (en) | Method and device for controlling household appliance | |
| CN105042789A (en) | Control method and system of intelligent air conditioner | |
| US10791607B1 (en) | Configuring and controlling light emitters | |
| CN109218145A (en) | Display methods, system, equipment and the storage medium of IOT appliance control interface | |
| CN113377193A (en) | Vending machine interaction method and system based on reliable gesture recognition | |
| CN111973076A (en) | Room attribute identification method and device, sweeping robot and storage medium | |
| CN109343481B (en) | Method and device for controlling device | |
| CN114203176A (en) | Control method and device of intelligent equipment, storage medium and electronic device | |
| CN113852657A (en) | Intelligent home local control method and system based on edge calculation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200605 |