CN116022617A - Non-contact intelligent calling and safety control system - Google Patents
Non-contact intelligent calling and safety control system Download PDFInfo
- Publication number
- CN116022617A CN116022617A CN202310323459.4A CN202310323459A CN116022617A CN 116022617 A CN116022617 A CN 116022617A CN 202310323459 A CN202310323459 A CN 202310323459A CN 116022617 A CN116022617 A CN 116022617A
- Authority
- CN
- China
- Prior art keywords
- call
- elevator
- wake
- intelligent
- controller
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B50/00—Energy efficient technologies in elevators, escalators and moving walkways, e.g. energy saving or recuperation technologies
Landscapes
- Indicating And Signalling Devices For Elevators (AREA)
Abstract
The invention relates to a non-contact intelligent call and safety management and control system, which comprises a cloud end, an intelligent scheduling platform, a control software module, external communication equipment, a ladder control intelligent terminal, a Bluetooth beacon, a multi-wake-up word and wake-up mode building module, wherein the ladder control intelligent terminal comprises a controller positioned at the top of an elevator car and a layer selection controller positioned in the elevator car, the controller and the cloud end respectively interact data with the external communication equipment in an http+mqtt mode, the intelligent scheduling platform is used for receiving an external communication equipment request signal, solving an optimal solution for the communication of the controller after logic judgment, the controller sends a signal to the cloud end, and the cloud end transmits information to the external communication equipment; the control software module is used for providing a call request for external communication equipment, can perform the call request in a non-contact way, and the system realizes intelligent call in a multi-master multi-slave ad hoc network communication mode, and covers different types of Internet of things equipment and elevator control systems, so that the universality and compatibility of the call system are improved.
Description
Technical Field
The invention belongs to the field of intelligent calls, and particularly relates to a non-contact intelligent call and safety management and control system.
Background
An elevator is an indispensable vertical public transport means in life of people, along with the development of social economy, the intelligent manufacturing industry is continuously developed and innovated, and the traditional calling mode can not meet the interaction requirements of different crowds and different intelligent devices. Most of the automatic call systems on the market at present utilize 2G, 4G wireless signals, bluetooth and WIFI to establish communication connection, if the network fluctuates, the automatic call system can have the phenomenon of unsmooth communication in an elevator car, so that the reliability of call operation is not high; there are also robot call systems mainly using the Lora wireless communication technology, and certain frequency spectrum interference may occur, so that call failure is caused, and real-time observation of elevator states is also required.
Therefore, there is an urgent need for an intelligent call system for robots and handicapped persons that is highly compatible and intelligent.
In addition, from the safety management analysis, in order to strengthen the safety management of the elevator in the use process, most elevators are introduced into a video monitoring device to monitor the whole operation of equipment, but the video monitoring device still needs full-time monitoring personnel to monitor by man. However, people cannot concentrate on monitoring the safety problem of the elevator at all times, and most monitoring staff cannot know the elevator deeply, and the sudden accident of the elevator cannot be solved timely, so that passengers are injured and killed when the accident occurs, and larger social influence is caused.
Disclosure of Invention
In order to solve the problems, the invention provides a non-contact intelligent call and safety control system, which is used for calling a call request to an elevator under the condition of not contacting a panel, and the system is used for realizing intelligent call in a multi-master multi-slave ad hoc network communication mode through a multi-master multi-slave ad hoc network communication mode, and covers different types of Internet of things equipment and elevator control systems, so that the universality and compatibility of the call system are improved.
The technical scheme of the invention is as follows:
the system comprises a cloud end, an intelligent scheduling platform, a control software module, external communication equipment, a ladder control intelligent terminal, a Bluetooth beacon and a multi-wake-up word and wake-up mode building module, wherein the ladder control intelligent terminal comprises a controller positioned at the top of an elevator car and a layer selection controller in the elevator car, the controller and the cloud end respectively interact data with the external communication equipment in an http+mqtt mode, the intelligent scheduling platform is used for receiving the request signal of the external communication equipment, solving out an optimal solution for the controller to communicate after logic judgment, and the controller sends a signal to the cloud end which transmits information to the external communication equipment; the control software module is used for providing a call request for the external communication equipment; the multi-wake-up word and wake-up mode establishing module comprises: the voice sensing module is arranged in the car and the PC end; the voice sensing module comprises a voice recognition unit and an acoustic signal acquisition unit; the voice recognition unit is used for recognizing voice of the voice signals collected by the voice signal collection unit, recognizing wake-up words and sending the wake-up words to the PC end, the PC end is used for sending the wake-up words to the intelligent scheduling platform, the intelligent scheduling platform is used for carrying out different wake-up operations according to the wake-up words and feeding back the content of the wake-up operations to passengers of the lift car through the PC end, when people enter the lift, the passenger is an excitation point, the voice recognition module is started to start the multiple wake-up words and the wake-up mode building module, the voice recognition unit is used for recognizing voice of the voice signals collected by the voice signal collection unit, recognizing the wake-up words and the voice main frequency amplitude are sent to the PC end, the PC end is used for sending the intelligent scheduling platform, and the intelligent scheduling platform is used for carrying out different wake-up operations according to the wake-up words, wake-up words in low colloquia voice, asking for help and scree; therefore, the method and the device recognize the non-civilized speech and abnormal sound information of the passengers, provide an interactive perception alarm function, calm emotion operation during accidents and record the accident occurrence condition to assist rescue work.
Preferably, the elevator control intelligent terminal is provided with a flat layer sensor, an acceleration sensor, a door opening and closing sensor, a 4G communication module, a Lora communication module and a power module which are arranged at the top of the elevator car, the controller is provided with a state detector, the state detector is used for being connected with the flat layer sensor, the acceleration sensor and the door opening and closing sensor to acquire related data signals, the 4G communication module can communicate with a cloud end, and the Lora communication module can carry out data interaction with the Lora communication module in external communication equipment by receiving and transmitting radio signals.
Preferably, the protocol of the communication between the 4G communication module and the cloud is the mqtt protocol and the 4G communication protocol.
Preferably, the cloud end performs data interaction with a 4G communication module of the external communication equipment through an http(s) and 4G/5G communication protocol, receives a call request of the external communication equipment, and obtains elevator information.
Preferably, the system also uses serial port and Bluetooth as a communication mechanism for information exchange between devices.
Preferably, the bluetooth beacon is a unique identification code and is installed on different elevator floors, and is used for providing the current floor position information for external communication equipment, and the accuracy of the distance detection information is achieved.
Preferably, the process of accessing the 4G communication module into the elevator is specifically as follows:
step 1: the external communication equipment sends a call request;
step 2: the cloud receives a call request, sends the request to an intelligent scheduling platform, and sends a signal to a 4G communication module after the intelligent scheduling platform processes information;
step 3: after receiving the signals, the 4G communication module performs data interaction with the controller, and the controller communicates with a layer selecting controller in the elevator car, wherein the layer selecting controller performs layer selecting;
step 4: the layer selecting controller returns the layer selecting signal to the controller, the controller returns the layer selecting signal to the cloud, the cloud transmits the layer selecting signal to external communication equipment, and the external communication equipment acquires elevator information.
Preferably, the rule of accessing the 4G communication module into the call is specifically as follows:
parameter holding stage: the target calls the elevator, obtains bluetooth beacon data, and requests a call by using the bluetooth beacon data, the authorization information and the call information;
authentication phase: verifying whether the authorization information is valid or not, namely verifying whether the mac address of the target is consistent with the mac address stored in the cloud, refusing access and returning a refusal reason if the mac address of the target is inconsistent with the mac address stored in the cloud; if the mac address of the target is consistent with the mac address stored in the cloud, verifying whether the Bluetooth beacon data is correct, namely whether the distance between the target and the call is within a required range or not, and rejecting access and returning a reject reason if the distance is out of the required range; if the content or the address of the target transmission information is inconsistent with the content of the ladder control intelligent terminal, the call information is invalid, access is refused and a refusing reason is returned; if the verification is successful, entering a ladder control stage;
ladder control stage: the intelligent ladder control terminal processes the call signal, and the target monitors the call state and uses the call.
Preferably, the calling process of the Lora communication module specifically comprises the following steps:
step 1: the Lora communication module of the external communication equipment and the Lora communication module of the controller are directly communicated through radio, and a call request is sent;
step 2: the controller sends a call request to a layer selecting controller in the elevator car, and the layer selecting controller selects a layer;
step 3: the layer selecting controller returns the layer selecting information to the controller, the Lora communication module of the controller sends a radio signal to the external communication equipment, and the external communication equipment receives the layer selecting information.
Preferably, the specific rule of controlling the elevator by accessing the Lora communication module is as follows:
the preparation stage: the target receives the elevator status;
parameter holding stage: the target obtains Bluetooth beacon data, combines the Bluetooth beacon data with authorization information, encrypts the Bluetooth beacon data through a public key and sends the Bluetooth beacon data and the authorization information to the ladder control intelligent terminal;
authentication phase: verifying whether the authorization information is valid or not, namely verifying whether the mac address of the target is consistent with the mac address stored in the cloud, refusing access and returning a refusal reason if the mac address of the target is inconsistent with the mac address stored in the cloud; if the mac address of the target is consistent with the mac address stored in the cloud, verifying whether the Bluetooth beacon data is correct, namely whether the distance between the target and the call is within a required range or not, and rejecting access and returning a reject reason if the distance is out of the required range; if the range is within the range, entering a ladder control stage;
ladder control stage: returning a temporary call key, constructing a call request, encrypting the call request by using the temporary key, processing the call by the intelligent terminal for ladder control, verifying whether the temporary key is effective or not, feeding back rejection information if the temporary key is ineffective, requesting the temporary key again, and returning to a parameter holding stage; and if the temporary key is valid, the elevator control intelligent terminal decrypts and processes the call request, and the target successfully uses the elevator.
Preferably, when the intelligent scheduling platform monitors that the wake-up word is a normal speech, the process is not performed, and if the wake-up word is a low-custom speech wake-up word, a warning prompt is performed; typical voice every second changes below 5Hz, the fluctuation of the distress call is far greater than 5Hz, a first main frequency amplitude which is higher than the daily life is set for identifying the distress call, the main frequency amplitude of the voice is higher than the first main frequency amplitude, and for seeking help and awakening, a frequency change auxiliary identification acoustic signal is adopted to inform maintenance personnel for rescue or alarm; setting a second main frequency amplitude range of the screaming signal, wherein the sound main frequency amplitude is higher than the second main frequency amplitude range, so that the screaming signal wakes up, and the maintenance personnel is informed to rescue or alarm.
Preferably, a vocabulary database is arranged in the intelligent scheduling platform, normal language and low-custom language are stored in the vocabulary database, and the wake-up words are compared with the vocabularies in the vocabulary database to judge whether the vocabularies are the normal language or the low-custom language; normal speech is not processed, and low-custom speech is warned.
Preferably, the intelligent scheduling platform analyzes the main frequency amplitude of the sound to determine the initial sound frame state; setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low thresholds to determine whether first frame data are sound frames or not, and determining whether next frame data are sound frames or not according to the relation between upper frame data and lower frame data; if the average amplitude of the next frame data is more than 3dB greater than that of the previous frame, judging the next frame data as a sound frame; db is db which determines the starting point and ending point of a sound event; determination of a sound event starting point: setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, if the amplitude of the current frame exceeds the amplitude threshold values or the zero crossing rate exceeds the zero crossing rate threshold values, determining the current frame as a starting point of a sound event, or if the average amplitude of the current frame data is suddenly changed by more than 3dB compared with the last frame data, determining the current frame as a starting position of the sound event; determination of end points: setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, wherein if the amplitude of the current frame is lower than the amplitude threshold value or the zero crossing rate is lower than the zero crossing rate threshold value, the amplitude threshold values and the zero crossing rate threshold values are the ending points of sound events, or if the length of a mute section is long enough, the amplitude threshold values and the zero crossing rate threshold values are also the ending points of the sound events; classifying and identifying the abnormal sound training result through the SVM model, and classifying abnormal signals according to the training result; the sound dominant frequency amplitude is obtained by determining the starting point and the ending point of the sound event.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a non-contact intelligent call and safety control system, which is used for carrying out call request on an elevator under the condition of not touching a panel, and the system realizes intelligent call in a multi-master multi-slave ad hoc network communication mode and covers different types of Internet of things equipment and elevator control systems in a form, so that the universality and compatibility of the call system are improved. In addition, the invention is an integrated artificial intelligence and straight ladder safety control technology, and takes AI voice as the 'mouth' of the artificial intelligence; different abnormal sound signals are found to give an alarm, passengers emotion is helped to be calmed in the distress, the monitoring system is further intelligently energized, the abnormal elevator taking sound signals are detected and identified at multiple angles, the capability of replacing management staff to analyze, make a decision and carry out emergency treatment on sudden dangerous situations is achieved, accidents are effectively prevented, and personal safety is guaranteed.
Drawings
FIG. 1 is a schematic diagram of a system framework of the present invention;
FIG. 2 is a flow chart of a 4G access control call;
fig. 3 is a flow chart of a Lora access control elevator;
FIG. 4 is a schematic block diagram of a multi-wake word and wake mode establishment module according to the present invention.
In the figure: 1-ladder control intelligent terminal.
Detailed Description
The invention will now be described in detail with reference to the drawings and to specific embodiments.
Referring to fig. 1-3, a non-contact intelligent call and safety management and control system comprises a cloud end, an intelligent scheduling platform, a control software module, external communication equipment, a ladder control intelligent terminal 1, a bluetooth beacon and a multi-wake-up word and wake-up mode establishing module, wherein the ladder control intelligent terminal comprises a controller positioned at the top of an elevator car and a layer selection controller in the elevator car, the controller and the cloud end respectively interact data with the external communication equipment in an http+mqtt mode, the intelligent scheduling platform is used for receiving an external communication equipment request signal, calculating an optimal solution to communicate with the controller after logic judgment, the controller sends a signal to the cloud end, and the cloud end transmits information to the external communication equipment; the control software module is used for providing a call request for the external communication equipment; the multi-wake-up word and wake-up mode establishing module comprises: the voice sensing module is arranged in the car and the PC end; the voice sensing module comprises a voice recognition unit and an acoustic signal acquisition unit; the voice recognition unit is used for recognizing voice of the voice signals collected by the voice signal collection unit, recognizing wake-up words and sending the wake-up words to the PC end, the PC end is used for sending the wake-up words to the intelligent scheduling platform, the intelligent scheduling platform is used for carrying out different wake-up operations according to the wake-up words and feeding back the content of the wake-up operations to passengers of the lift car through the PC end, when people enter the lift, the passenger is an excitation point, the voice recognition module is started to start the multiple wake-up words and the wake-up mode building module, the voice recognition unit is used for recognizing voice of the voice signals collected by the voice signal collection unit, recognizing the wake-up words and the voice main frequency amplitude are sent to the PC end, the PC end is used for sending the intelligent scheduling platform, and the intelligent scheduling platform is used for carrying out different wake-up operations according to the wake-up words, wake-up words in low colloquia voice, asking for help and scree; therefore, the method and the device recognize the non-civilized speech and abnormal sound information of the passengers, provide an interactive perception alarm function, calm emotion operation during accidents and record the accident occurrence condition to assist rescue work.
In a specific embodiment of the invention, the elevator control intelligent terminal is provided with a leveling sensor, an acceleration sensor, a door opening and closing sensor, a 4G communication module, a Lora communication module and a power module which are positioned at the top of an elevator car, and the controller is provided with a state detector, wherein the state detector is used for being connected with the leveling sensor, the acceleration sensor and the door opening and closing sensor to acquire related data signals, the 4G communication module can communicate with a cloud, and the Lora communication module can perform data interaction with the Lora communication module in external communication equipment by receiving and transmitting radio signals.
In a specific embodiment of the present invention, as shown in fig. 2, the rules for accessing the call by the 4G communication module are specifically:
parameter holding stage: the target calls the elevator, the target is external communication equipment used by a robot or a handicapped person, bluetooth beacon data are obtained, and the target requests a call by using the Bluetooth beacon data, authorization information and call information;
authentication phase: verifying whether the authorization information is valid or not, namely verifying whether the mac address of the target is consistent with the mac address stored in the cloud, refusing access and returning a refusal reason if the mac address of the target is inconsistent with the mac address stored in the cloud; if the mac address of the target is consistent with the mac address stored in the cloud, verifying whether the Bluetooth beacon data is correct, namely whether the distance between the target and the call is within 3 meters or not, and rejecting access and returning a reject reason if the distance is out of range; if the call information is in the range of 3 meters, verifying whether the call information is effective or not, wherein the target is communicated with the ladder control intelligent terminal through a Modbus protocol, and verifying the received content, wherein the original 110 possibly becomes 11 or 01101 if the content or address of the target transmission information is inconsistent with the content of the ladder control intelligent terminal because interference exists in the transmission process or the transmission is too fast to be received, the call information is invalid, access is refused and a refusal reason is returned; if the verification is successful, entering a ladder control stage;
ladder control stage: the intelligent ladder control terminal processes the call signal, and the target monitors the call state and uses the call.
As shown in fig. 3, the specific rule for controlling the elevator by accessing the Lora communication module is as follows:
the preparation stage: the method comprises the steps that a target receives an elevator state, wherein the target is external communication equipment used by a robot or a handicapped person;
parameter holding stage: the target obtains Bluetooth beacon data, combines the Bluetooth beacon data with authorization information, encrypts the Bluetooth beacon data through a public key and sends the Bluetooth beacon data and the authorization information to the ladder control intelligent terminal;
authentication phase: verifying whether the authorization information is valid or not, namely verifying whether the mac address of the target is consistent with the mac address stored in the cloud, refusing access and returning a refusal reason if the mac address of the target is inconsistent with the mac address stored in the cloud; if the mac address of the target is consistent with the mac address stored in the cloud, verifying whether the Bluetooth beacon data is correct, namely whether the distance between the target and the call is within a range of 3 meters or not, and rejecting access and returning a reject reason if the distance exceeds the range of 3 meters; in the range of 3 meters, entering a ladder control stage;
ladder control stage: returning a temporary call key, constructing a call request, encrypting the call request by using the temporary key, processing the call by the intelligent terminal for ladder control, verifying whether the temporary key is effective or not, feeding back rejection information if the temporary key is ineffective, requesting the temporary key again, and returning to a parameter holding stage; and if the temporary key is valid, the elevator control intelligent terminal decrypts and processes the call request, and the target successfully uses the elevator.
When the intelligent elevator control system works, when the 4G signal is poor, the Lora communication module can be used for communication, otherwise, the 4G communication module is used for communication, a robot or a handicapped person calls an elevator, a plurality of controllers work simultaneously, the elevator located at the lowest floor is intelligently selected, the elevator can be automatically controlled to one floor, a counter in the controller is reset, and whether the floor is judged by judging whether the direction is matched with a flat layer sensor or not through an acceleration sensor so as to obtain the number of floors. In one embodiment of the present invention, the protocol for the communication between the 4G communication module and the cloud is the mqtt protocol and the 4G communication protocol.
In another embodiment of the present invention, the cloud performs data interaction with the 4G communication module of the external communication device through http(s) and a 4G/5G communication protocol, and the cloud receives a call request of the external communication device, and the external communication device obtains elevator information.
In one embodiment of the invention, the system also uses serial port and Bluetooth as a communication mechanism for information exchange between devices.
In one embodiment of the invention, the Bluetooth beacon is a unique identification code and is arranged on different elevator floors to provide the current floor position information for external communication equipment, and the problem of floor positioning deviation is effectively solved through the accuracy of the distance detection information.
In one embodiment of the invention, the process of accessing the 4G communication module into the elevator is specifically as follows:
step 1: the external communication equipment sends a call request;
step 2: the cloud receives a call request, sends the request to an intelligent scheduling platform, and sends a signal to a 4G communication module after the intelligent scheduling platform processes information;
step 3: after receiving the signals, the 4G communication module performs data interaction with the controller, and the controller communicates with a layer selecting controller in the elevator car, wherein the layer selecting controller performs layer selecting;
step 4: the layer selecting controller returns the layer selecting signal to the controller, the controller returns the layer selecting signal to the cloud, the cloud transmits the layer selecting signal to external communication equipment, and the external communication equipment acquires elevator information.
In one embodiment of the present invention, the procedure of calling the Lora communication module is specifically:
step 1: the Lora communication module of the external communication equipment and the Lora communication module of the controller are directly communicated through radio, and a call request is sent;
step 2: the controller sends a call request to a layer selecting controller in the elevator car, and the layer selecting controller selects a layer;
step 3: the layer selecting controller returns the layer selecting information to the controller, the Lora communication module of the controller sends a radio signal to the external communication equipment, and the external communication equipment receives the layer selecting information.
In one embodiment of the present invention, the control software module may be a mobile phone App, a computer, or mobile terminal control software.
For example, handicapped people call through App, transmit information to the cloud through 4G, after receiving, communicate with the intelligent scheduling platform, the intelligent scheduling platform processes information, the intelligent terminal of elevator control is mobilized to work, and after verification information passes, elevator calling is achieved.
As shown in fig. 4, the multi-wake-up word and wake-up mode establishing module establishes a system for identification by adopting a mode of special distress signals such as life-saving schematic words, screaming audio identification and low-custom language identification, when a person enters an elevator (or starts to select a floor), the multi-wake-up word is started to identify, the system can be installed in the elevator, and array microphones are used for collecting surround sound signals. If the abnormal sound signal is detected to be connected with the elevator central control original alarm system for assisting rescue.
When the intelligent scheduling platform monitors that the wake-up words are normal words, the process is not performed, and when the wake-up words are low-custom words, warning prompt is performed; typical voice every second changes below 5Hz, the fluctuation of the distress call is far greater than 5Hz, a first main frequency amplitude which is higher than the daily life is set for identifying the distress call, the main frequency amplitude of the voice is higher than the first main frequency amplitude, and for seeking help and awakening, a frequency change auxiliary identification acoustic signal is adopted to inform maintenance personnel for rescue or alarm; setting a second main frequency amplitude range of the screaming signal, wherein the sound main frequency amplitude is higher than the second main frequency amplitude range, so that the screaming signal wakes up, and the maintenance personnel is informed to rescue or alarm.
The recognition of the wake-up word is used for judging that the wake-up word is normally wake-up and is wake-up in a low-custom language; the intelligent scheduling platform is provided with a vocabulary database, wherein normal language and low-custom language are stored in the vocabulary database, and the wake-up words are compared with the vocabulary in the vocabulary database to judge whether the words are normal language or low-custom language; normal speech is not processed, and low-custom speech is warned.
The main frequency amplitude of the sound is used for judging help seeking awakening and screaming awakening; the intelligent scheduling platform analyzes the main frequency amplitude of the sound and determines the initial sound frame state; setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low thresholds to determine whether first frame data are sound frames or not, and determining whether next frame data are sound frames or not according to the relation between upper frame data and lower frame data; if the average amplitude of the next frame data is more than 3dB than that of the previous frame, judging the next frame data as a sound frame; db is db which determines the starting point and ending point of a sound event; determination of a sound event starting point: setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, if the amplitude of the current frame exceeds the amplitude threshold values or the zero crossing rate exceeds the zero crossing rate threshold values, determining the current frame as a starting point of a sound event, or if the average amplitude of the current frame data is suddenly changed by more than 3dB compared with the last frame data, determining the current frame as a starting position of the sound event; determination of end points: setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, wherein if the amplitude of the current frame is lower than the amplitude threshold value or the zero crossing rate is lower than the zero crossing rate threshold value, the amplitude threshold values and the zero crossing rate threshold values are the ending points of sound events, or if the length of a mute section is long enough, the amplitude threshold values and the zero crossing rate threshold values are also the ending points of the sound events; classifying and identifying the abnormal sound training result through the SVM model, and classifying abnormal signals according to the training result; the sound dominant frequency amplitude is obtained by determining the starting point and the ending point of the sound event.
In addition, in the present invention: 1. taking the distress signal as a wake-up mode;
active help seeking, such as rescuing, adopts voice comparison to identify abnormal language, such as rescuing, rescuing me, SOS, bad person and the like, and adopts frequency change to assist in identifying sound signals. Typical speech changes below 5Hz per second, while the squeal wave is much greater than 5Hz. This fluctuation gives the roughness of sound quality, which is not found in any human voice. The computer algorithm is used for carrying out signal processing and analysis on the fundamental tone frequency and the spectrogram so as to identify the relevant abnormal sound signals.
Others ask for help, such as a cheer, and the words are used as a wake-up mode and are identified by setting a main frequency range higher than the daily frequency range.
2. Waking up and alarming by taking the squeaking sound as a wake-up; setting a second main frequency amplitude range of the screaming signal, wherein the sound main frequency amplitude is higher than the second main frequency amplitude range, so that the screaming signal wakes up, and the maintenance personnel is informed to rescue or alarm.
3. The daily dialogue signal and the main frequency amplitude range of the distress signal have larger difference, the main frequency amplitude range of the distress signal is set, and then the main frequency range is screened to identify the distress signal;
an initial sound frame state is determined. And determining whether the first frame data is a sound frame or not by adopting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, and determining whether the next frame data is a sound frame or not according to the relation between the upper frame data and the lower frame data. If the average amplitude of the next frame data is 3dB or more larger than that of the previous frame, it is determined to be a sound frame. db is the decibel that determines the starting point and ending point of a sound event. Determination of a sound event starting point: setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, if the amplitude of the current frame exceeds the amplitude threshold values or the zero crossing rate exceeds the zero crossing rate threshold values, determining the current frame as a starting point of a sound event, and in addition, if the average amplitude of the current frame data is suddenly changed by more than 3dB compared with the last frame data, determining the current frame as the starting position of the sound event. Determination of end points: and setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, and ending the sound event if the amplitude of the current frame is lower than the amplitude threshold value or the zero crossing rate is lower than the zero crossing rate threshold value. If the length of the silence segment is sufficiently long, the sound event also ends. And classifying and identifying the abnormal sound training result through the SVM model, and classifying abnormal signals according to the training result.
4. Wake-up in a low-custom language, and then carry out warning prompt.
When intelligent calling is carried out, a user sends a target floor to be taken by the user to an elevator, the elevator with the shortest waiting time is obtained, a dynamic time measuring and calculating model is adopted, and the shortest time is dynamically estimated:
elevator and external direction layer selection
；
k is the total height of the floor;
acquiring current floor x Current floor Destination floor x Destination floor Number x of passengers Number of people riding on stairs Number of selected floors x Layer selection Elevator speed x Ladder speed The parameters are input as a dynamic time measuring and calculating model;
the floor difference is x Floor difference =x Current floor -x Destination floor
Taking the real-time change data return error of the elevator into consideration, and carrying out offset correction on elevator parameters by using a time window coefficient: through history riding density
Analysis, x Total number of selected layers The number of floors selected for the elevator in the T time period;
X=(x floor difference , x Number of people riding on stairs , x Layer selection , x Ladder speed ,x Density of riding ladder …,x j ) Is a set of all states, v= { V 1 ,v 2 ,v 3 ,v 4 ,v 5 ,…,v i -all observation sets;
the observation value matrix of all state sets X and all observation sets V is as follows:
wherein v is i For the ith observation value, x, in all observation sets j For the j-th state value, x in all state sets ij Matrix entries representing the ith row and jth column of the matrix,
the normalized and transformed observation value matrix is as follows
Wherein the method comprises the steps of
In the middle of
For the variables->
Mean value of the observations of s ij As variable x j Variance of observed values of>
Is the standard deviation; n and p are natural numbers; after normalization transformation, the observation matrix +.>
The mean value of each column is 0, the standard deviation is 1; ->
Is a matrix item of the standardized transformed observation value matrix;
define input weight w=（w Floor difference ，w Destination floor , w Number of people riding on stairs , w Layer selection , w Ladder speed …,w j ) The parameters are used as the input weight of the dynamic time measuring and calculating model, and B is the offset; w (w) j The j-th weight value in the input weight W;
dynamic time measuring and calculating model
W' is the matrix transpose of the input weight W;
for example: 1. acquiring current floor x Current floor (e.g. current floor 1 has a value of 3), destination floor x Destination floor (e.g., target destination floor is 3 on 3 floors, the value is 3), and the number of passengers is x Number of people riding on stairs (e.g., 3 number of people in the elevator is recognized as 3, and 3 value), the number of selected floors is x Layer selection (e.g., 3 for a selected floor in the identified elevator, 3 for a value of 3), elevator speed x Ladder speed (for example, the ladder is in normal operation, the value of the ladder speed is 1.5m/s is 1.5), and the parameters are input as a dynamic time measuring and calculating model;
2.V={v 1 ,v 2 ,v 3 ,v 4 ,v 5 ,…,v i -all observation sets, wherein an observation set is a possible occurrence of an observation state; v 1 ,v 2 ,v 3 ,v 4 ,v 5 ,…,v i The numerical value of (1) is calculated according to the daily observation speed, and the numerical value is set as [0,1]]；
3. Establishing a state set of X and an observation set observation value matrix of V, wherein the numerical value in the observation value matrix is given by the state value X j Under the condition V i Determining the occurrence probability;
4. adjusting an observation value matrix after standardized transformation;
5. defining weights, wherein the weight size is set to be in a range of [0,1] according to the relative importance degree of the factors or indexes which are emphasized by human settings;
6. dynamic time measuring and calculating model
The method comprises the steps of carrying out a first treatment on the surface of the Model results range from 0,1]In the section, if the waiting time of the elevator tends to be 0 and is shortest,trend 1 elevator waiting time is longest; the specific time will use b as the time scaling. Updating the stair riding density parameter after a period>
For a period of time, x' Total number of selected layers Is->
The number of floors selected by the elevator in the time period;
updating an input matrix X in real time, and acquiring a shortest waiting time formula as follows:
；
and b is a waiting time conversion constant, t is the shortest waiting time, the obtained shortest waiting time t is transmitted back to the intelligent scheduling platform, and the waiting time is prompted.
The elevator traffic control mechanism can effectively shorten the time for a user to wait for an elevator, thereby improving the experience of the user.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent structures or equivalent processes or direct or indirect application in other related technical fields are included in the scope of the present invention.
Claims (14)
1. The system is characterized by comprising a cloud, an intelligent scheduling platform, a control software module, external communication equipment, a ladder control intelligent terminal, a Bluetooth beacon and a multi-wake-up word and wake-up mode establishing module, wherein the ladder control intelligent terminal comprises a controller positioned at the top of an elevator car and a layer selection controller in the elevator car, the controller and the cloud respectively interact data with the external communication equipment in the mode of http+mqtt, the intelligent scheduling platform is used for receiving an external communication equipment request signal, solving an optimal solution for the controller to communicate after performing logic judgment, and the controller sends a signal to the cloud which transmits information to the external communication equipment; the control software module is used for providing a call request for the external communication equipment; the multi-wake-up word and wake-up mode establishing module comprises: the voice sensing module is arranged in the car and the PC end; the voice sensing module comprises a voice recognition unit and an acoustic signal acquisition unit; the voice recognition unit is used for recognizing voice of the voice signals collected by the voice signal collection unit, recognizing wake-up words and sending the wake-up words to the PC end, the PC end is used for sending the wake-up words to the intelligent scheduling platform, the intelligent scheduling platform is used for carrying out different wake-up operations according to the wake-up words and feeding back the content of the wake-up operations to passengers of the lift car through the PC end, when people enter the lift, the passenger is an excitation point, the voice recognition module is started to start the multiple wake-up words and the wake-up mode building module, the voice recognition unit is used for recognizing voice of the voice signals collected by the voice signal collection unit, recognizing the wake-up words and the voice main frequency amplitude are sent to the PC end, the PC end is used for sending the intelligent scheduling platform, and the intelligent scheduling platform is used for carrying out different wake-up operations according to the wake-up words, wake-up words in low colloquia voice, asking for help and scree; therefore, the method and the device recognize the non-civilized speech and abnormal sound information of the passengers, provide an interactive perception alarm function, calm emotion operation during accidents and record the accident occurrence condition to assist rescue work.
2. The non-contact intelligent call and safety control system according to claim 1, wherein the elevator control intelligent terminal comprises a flat layer sensor, an acceleration sensor, a door opening and closing sensor, a 4G communication module, a Lora communication module and a power module which are positioned at the top of an elevator car, and the controller comprises a state detector which is used for being connected with the flat layer sensor, the acceleration sensor and the door opening and closing sensor to acquire related data signals, the 4G communication module can communicate with the cloud, and the Lora communication module can perform data interaction with the Lora module in external communication equipment by receiving and transmitting radio signals.
3. The system of claim 1, wherein the protocol for communication between the 4G communication module and the cloud is mqtt protocol and 4G communication protocol.
4. The non-contact intelligent call and safety control system according to claim 2, wherein the cloud terminal performs data interaction with a 4G communication module of the external communication device through an http(s) and 4G/5G communication protocol, and the cloud terminal receives a call request of the external communication device and obtains elevator information.
5. A contactless intelligent call and security management system according to claim 2, wherein the system also acts as a communication mechanism for information exchange from device to device via serial port, bluetooth.
6. The system of claim 1, wherein the bluetooth beacon is a unique identification code and is installed on different elevator floors to provide the current floor location information to the external communication device, and the accuracy of the information is detected by distance.
7. The non-contact intelligent call and safety control system according to claim 2, wherein the process of accessing the elevator by the 4G communication module is specifically as follows:
step 1: the external communication equipment sends a call request;
step 2: the cloud receives a call request, sends the request to an intelligent scheduling platform, and sends a signal to a 4G communication module after the intelligent scheduling platform processes information;
step 3: after receiving the signals, the 4G communication module performs data interaction with the controller, and the controller communicates with a layer selecting controller in the elevator car, wherein the layer selecting controller performs layer selecting;
step 4: the layer selecting controller returns the layer selecting signal to the controller, the controller returns the layer selecting signal to the cloud, the cloud transmits the layer selecting signal to external communication equipment, and the external communication equipment acquires elevator information.
8. The non-contact intelligent call and security management and control system according to claim 1, wherein the rules for the access of the 4G communication module to the call are specifically:
parameter holding stage: the target calls the elevator, obtains bluetooth beacon data, and requests a call by using the bluetooth beacon data, the authorization information and the call information;
authentication phase: verifying whether the authorization information is valid or not, namely verifying whether the mac address of the target is consistent with the mac address stored in the cloud, refusing access and returning a refusal reason if the mac address of the target is inconsistent with the mac address stored in the cloud; if the mac address of the target is consistent with the mac address stored in the cloud, verifying whether the Bluetooth beacon data is correct, namely whether the distance between the target and the call is within a required range or not, and rejecting access and returning a reject reason if the distance is out of the required range; if the content or the address of the target transmission information is inconsistent with the content of the ladder control intelligent terminal, the call information is invalid, access is refused and a refusing reason is returned; if the verification is successful, entering a ladder control stage;
ladder control stage: the intelligent ladder control terminal processes the call signal, and the target monitors the call state and uses the call.
9. The non-contact intelligent call and safety control system according to claim 2, wherein the procedure of the Lora communication module call is specifically as follows:
step 1: the Lora communication module of the external communication equipment and the Lora communication module of the controller are directly communicated through radio, and a call request is sent;
step 2: the controller sends a call request to a layer selecting controller in the elevator car, and the layer selecting controller selects a layer;
step 3: the layer selecting controller returns the layer selecting information to the controller, the Lora communication module of the controller sends a radio signal to the external communication equipment, and the external communication equipment receives the layer selecting information.
10. The non-contact intelligent call and safety control system according to claim 1, wherein the specific rules for controlling the elevator access by the Lora communication module are:
the preparation stage: the target receives the elevator status;
parameter holding stage: the target obtains Bluetooth beacon data, combines the Bluetooth beacon data with authorization information, encrypts the Bluetooth beacon data through a public key and sends the Bluetooth beacon data and the authorization information to the ladder control intelligent terminal;
authentication phase: verifying whether the authorization information is valid or not, namely verifying whether the mac address of the target is consistent with the mac address stored in the cloud, refusing access and returning a refusal reason if the mac address of the target is inconsistent with the mac address stored in the cloud; if the mac address of the target is consistent with the mac address stored in the cloud, verifying whether the Bluetooth beacon data is correct, namely whether the distance between the target and the call is within a required range or not, and rejecting access and returning a reject reason if the distance is out of the required range; if the range is within the range, entering a ladder control stage;
ladder control stage: returning a temporary call key, constructing a call request, encrypting the call request by using the temporary key, processing the call by the intelligent terminal for ladder control, verifying whether the temporary key is effective or not, feeding back rejection information if the temporary key is ineffective, requesting the temporary key again, and returning to a parameter holding stage; and if the temporary key is valid, the elevator control intelligent terminal decrypts and processes the call request, and the target successfully uses the elevator.
11. A contactless intelligent call and security management system according to claim 1, wherein: when the intelligent scheduling platform monitors that the wake-up words are normal words, the process is not performed, and when the wake-up words are low-custom words, warning prompt is performed; typical voice every second changes below 5Hz, the fluctuation of the distress call is far greater than 5Hz, a first main frequency amplitude which is higher than the daily life is set for identifying the distress call, the main frequency amplitude of the voice is higher than the first main frequency amplitude, and for seeking help and awakening, a frequency change auxiliary identification acoustic signal is adopted to inform maintenance personnel for rescue or alarm; setting a second main frequency amplitude range of the screaming signal, wherein the sound main frequency amplitude is higher than the second main frequency amplitude range, so that the screaming signal wakes up, and the maintenance personnel is informed to rescue or alarm.
12. The contactless intelligent call and security management system according to claim 11, wherein: the intelligent scheduling platform is provided with a vocabulary database, wherein normal language and low-custom language are stored in the vocabulary database, and the wake-up words are compared with the vocabulary in the vocabulary database to judge whether the words are normal language or low-custom language; normal speech is not processed, and low-custom speech is warned.
13. The contactless intelligent call and security management system according to claim 12, wherein: the intelligent scheduling platform analyzes the main frequency amplitude of the sound and determines the initial sound frame state; setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low thresholds to determine whether first frame data are sound frames or not, and determining whether next frame data are sound frames or not according to the relation between upper frame data and lower frame data; if the average amplitude of the next frame data is more than 3dB greater than that of the previous frame, judging the next frame data as a sound frame; db is db which determines the starting point and ending point of a sound event; determination of a sound event starting point: setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, if the amplitude of the current frame exceeds the amplitude threshold values or the zero crossing rate exceeds the zero crossing rate threshold values, determining the current frame as a starting point of a sound event, or if the average amplitude of the current frame data is suddenly changed by more than 3dB compared with the last frame data, determining the current frame as a starting position of the sound event; determination of end points: setting amplitude threshold values and zero crossing rate threshold values corresponding to different dB numbers as low threshold values, wherein if the amplitude of the current frame is lower than the amplitude threshold value or the zero crossing rate is lower than the zero crossing rate threshold value, the amplitude threshold values and the zero crossing rate threshold values are the ending points of sound events, or if the length of a mute section is long enough, the amplitude threshold values and the zero crossing rate threshold values are also the ending points of the sound events; classifying and identifying the abnormal sound training result through the SVM model, and classifying abnormal signals according to the training result; the sound dominant frequency amplitude is obtained by determining the starting point and the ending point of the sound event.
14. A contactless intelligent call and security management system according to claim 1, wherein: when intelligent calling is carried out, a user sends a target floor to be taken by the user to an elevator, the elevator with the shortest waiting time is obtained, a dynamic time measuring and calculating model is adopted, and the shortest time is dynamically estimated:
elevator and external direction layer selection
；
k is the total height of the floor;
acquiring current floor x Current floor Destination floor x Destination floor Number x of passengers Number of people riding on stairs Number of selected floors x Layer selection Elevator speed x Ladder speed The parameters are input as a dynamic time measuring and calculating model;
the floor difference is x Floor difference =x Current floor -x Destination floor
Taking the real-time change data return error of the elevator into consideration, and carrying out offset correction on elevator parameters by using a time window coefficient: through history riding density
Analysis, x Total number of selected layers The number of floors selected for the elevator in the T time period;
X=(x floor difference , x Number of people riding on stairs , x Layer selection , x Ladder speed ,x Density of riding ladder …,x j ) Is a set of all states, v= { V 1 ,v 2 ,v 3 ,v 4 ,v 5 ,…,v i -all observation sets;
the observation value matrix of all state sets X and all observation sets V is as follows:
wherein v is i For the ith observation value, x, in all observation sets j For the j-th state value, x in all state sets ij Matrix entries representing the ith row and jth column of the matrix,
the normalized and transformed observation value matrix is as follows
Wherein the method comprises the steps of
In the middle of
For the variables->
Mean value of the observations of s ij As variable x j Variance of observed values of>
Is the standard deviation; n and p are natural numbers; after normalization transformation, the observation matrix +.>
The mean value of each column is 0, the standard deviation is 1; ->
Is a matrix item of the standardized transformed observation value matrix;
define input weight w= (W) Floor difference ，w Destination floor , w Number of people riding on stairs , w Layer selection , w Ladder speed …,w j ) The parameters are used as the input weight of the dynamic time measuring and calculating model, and B is the offset; w (w) j The j-th weight value in the input weight W;
dynamic time measuring and calculating model
W' is the matrix transpose of the input weight W; updating the stair riding density parameter after a period>
For a period of time, x' Total number of selected layers Is->
The number of floors selected by the elevator in the time period;
updating an input matrix X in real time, and acquiring a shortest waiting time formula as follows:
；
and b is a waiting time conversion constant, t is the shortest waiting time, the obtained shortest waiting time t is transmitted back to the intelligent scheduling platform, and the waiting time is prompted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310323459.4A CN116022617B (en) | 2023-03-30 | 2023-03-30 | Non-contact intelligent calling and safety control system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310323459.4A CN116022617B (en) | 2023-03-30 | 2023-03-30 | Non-contact intelligent calling and safety control system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116022617A true CN116022617A (en) | 2023-04-28 |
| CN116022617B CN116022617B (en) | 2023-06-02 |
Family
ID=86072676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310323459.4A Active CN116022617B (en) | 2023-03-30 | 2023-03-30 | Non-contact intelligent calling and safety control system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116022617B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160150323A (en) * | 2015-06-22 | 2016-12-30 | 주식회사 성진전자 | Sensor-based Intelligent Emergency Call Client Device |
| CN205990117U (en) * | 2016-07-22 | 2017-03-01 | 广州广日电梯工业有限公司 | Intelligent security guard apparatus for controlling elevator |
| CN106966242A (en) * | 2017-03-29 | 2017-07-21 | 深圳市旺龙智能科技有限公司 | A kind of control elevator device and method based on high in the clouds |
| CN107758454A (en) * | 2017-11-24 | 2018-03-06 | 安徽奥里奥克科技股份有限公司 | Helper information interaction system based on intelligent terminal |
| CN109665397A (en) * | 2018-12-19 | 2019-04-23 | 上海新时达电气股份有限公司 | Elevator Internet of Things security protection interactive system and method based on recognition of face |
| CN111874765A (en) * | 2020-08-20 | 2020-11-03 | 永大电梯设备（中国）有限公司 | Mobile phone calling landing platform system and method |
| CN114476883A (en) * | 2022-01-25 | 2022-05-13 | 华恺智联电梯科技有限公司 | Non-contact calling landing system and method |
-
2023
- 2023-03-30 CN CN202310323459.4A patent/CN116022617B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20160150323A (en) * | 2015-06-22 | 2016-12-30 | 주식회사 성진전자 | Sensor-based Intelligent Emergency Call Client Device |
| CN205990117U (en) * | 2016-07-22 | 2017-03-01 | 广州广日电梯工业有限公司 | Intelligent security guard apparatus for controlling elevator |
| CN106966242A (en) * | 2017-03-29 | 2017-07-21 | 深圳市旺龙智能科技有限公司 | A kind of control elevator device and method based on high in the clouds |
| CN107758454A (en) * | 2017-11-24 | 2018-03-06 | 安徽奥里奥克科技股份有限公司 | Helper information interaction system based on intelligent terminal |
| CN109665397A (en) * | 2018-12-19 | 2019-04-23 | 上海新时达电气股份有限公司 | Elevator Internet of Things security protection interactive system and method based on recognition of face |
| CN111874765A (en) * | 2020-08-20 | 2020-11-03 | 永大电梯设备（中国）有限公司 | Mobile phone calling landing platform system and method |
| CN114476883A (en) * | 2022-01-25 | 2022-05-13 | 华恺智联电梯科技有限公司 | Non-contact calling landing system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116022617B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110342355B (en) | Intelligent building elevator dispatching platform system and working method thereof | |
| CN109690609B (en) | Passenger assist device, method, and program | |
| CN109584008B (en) | Voice recognition-based network car booking abnormal driving environment monitoring system and method | |
| CN107304017B (en) | Call operation based on wrist wearable intelligent device | |
| CN111362089B (en) | Method and system for identifying entering of electric vehicle into lift car through artificial intelligence | |
| US20150274017A1 (en) | Method and system for in-vehicle function control | |
| US20090050417A1 (en) | Intelligent destination elevator control system | |
| CN107933471A (en) | The method and vehicle-mounted automatic help system of accident active call rescue | |
| US10332385B2 (en) | Location based support request messages responsive to alert recommendation | |
| CN107085937A (en) | A kind of reply system and method for unmanned cab-getter's emergency situation | |
| KR20200086785A (en) | A method and system for preventing drowsiness driving and keeping vehicle safe | |
| CN112071309A (en) | Network appointment car safety monitoring device and system | |
| CN112478963A (en) | Elevator control auxiliary system and elevator control method | |
| CN116022617B (en) | Non-contact intelligent calling and safety control system | |
| CN112897266B (en) | Call calling method and system based on intelligent terminal | |
| CN214929564U (en) | Safety guarantee system for sudden serious illness of school bus driver | |
| KR20150087471A (en) | Apparatus and method for getting on and off management of public transportation | |
| JP2005335893A (en) | Traffic demand predicting device of elevator | |
| CN113141392A (en) | Man-vehicle interconnection interactive system based on LoRaWAN network | |
| CN112607539A (en) | Elevator system | |
| CN210001354U (en) | elevator self-adaptive operation control system | |
| KR20160028542A (en) | an emergency management and crime prevention system for cars and the method thereof | |
| KR101437406B1 (en) | an emergency management and crime prevention system for cars and the method thereof | |
| CN109264523A (en) | A kind of automatic rescue method and system of elevator | |
| KR20160125763A (en) | Security and surveillance system and method for elevator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |