CN110728701A - Control method and device for walking stick with millimeter wave radar and intelligent walking stick - Google Patents
Control method and device for walking stick with millimeter wave radar and intelligent walking stick Download PDFInfo
- Publication number
- CN110728701A CN110728701A CN201910786764.0A CN201910786764A CN110728701A CN 110728701 A CN110728701 A CN 110728701A CN 201910786764 A CN201910786764 A CN 201910786764A CN 110728701 A CN110728701 A CN 110728701A
- Authority
- CN
- China
- Prior art keywords
- information
- target
- target tracking
- obstacle
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- A—HUMAN NECESSITIES
- A45—HAND OR TRAVELLING ARTICLES
- A45B—WALKING STICKS; UMBRELLAS; LADIES' OR LIKE FANS
- A45B3/00—Sticks combined with other objects
- A45B3/08—Sticks combined with other objects with measuring or weighing appliances
-
- A—HUMAN NECESSITIES
- A45—HAND OR TRAVELLING ARTICLES
- A45B—WALKING STICKS; UMBRELLAS; LADIES' OR LIKE FANS
- A45B9/00—Details
- A45B9/02—Handles or heads
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Radar Systems Or Details Thereof (AREA)
- Traffic Control Systems (AREA)
Abstract
The application relates to a control method and device for a walking stick with a millimeter wave radar and an intelligent walking stick, belonging to the technical field of intelligent home, wherein the method comprises the following steps: acquiring point cloud data in a preset range based on the received echo signal of the millimeter wave radar; analyzing the point cloud data to obtain motion information and characteristic information of each detection object within a preset range; determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object; and if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, obstacle avoidance processing is carried out. By the aid of the method and the device, automatic obstacle avoidance can be achieved, and safety of users is improved.
Description
Technical Field
The application relates to the technical field of smart homes, in particular to a control method and device for a walking stick with a millimeter wave radar and an intelligent walking stick.
Background
With the increasing number of aging population, the middle-aged and old people in the society often need to walk with the help of a walking stick to avoid obstacles in the road due to inconvenient movement or blurred vision and the like.
The intelligent walking stick is taken as an auxiliary tool for the walking of the old, and is highly concerned by the society. At present, walking sticks in the market generally have single function, and most of the walking sticks utilize technical modes such as laser radar, infrared ray, vision processing and the like to detect surrounding objects. The walking sticks have low detection sensitivity, are greatly influenced by external illumination and temperature factors, and have high false detection rate.
Disclosure of Invention
In order to solve the technical problems or at least partially solve the technical problems, the application provides a control method and a control device of a walking stick with a millimeter wave radar and an intelligent walking stick.
In a first aspect, the application provides a control method for a walking stick with a millimeter wave radar, and the method comprises the following steps:
acquiring point cloud data in a preset range based on the received echo signal of the millimeter wave radar;
analyzing the point cloud data to obtain motion information and characteristic information of each detection object in the preset range;
determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object;
and if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, obstacle avoidance processing is carried out.
Optionally, the feature information includes one or more of the following: micro-doppler bias, coherence bandwidth, centroid, total bandwidth, frequency, height, and standard deviation.
Optionally, the obtaining of the motion information of each detection object within the preset range by analyzing the point cloud data includes:
tracking and positioning each detection object according to the point cloud data and a Kalman tracking algorithm to obtain motion information of each detection object;
wherein the motion information comprises: the speed of the test object, the distance of the test object relative to the hand wand, and the angle of the test object relative to the direction of movement of the hand wand.
Optionally, the preset condition includes at least one of the following conditions:
the target obstacle is positioned on the current moving route of the target tracking object;
the minimum distance between the target obstacle and the target tracking object is smaller than or equal to a preset distance;
the movement track of the target obstacle and the action route have an intersection point, and the time difference of reaching the intersection point is less than or equal to the preset time.
Optionally, if the operation information of the target tracking object and the motion information of the target obstacle satisfy a preset condition, performing obstacle avoidance processing, including:
if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, determining an obstacle avoidance route according to the motion information of the target obstacle and a preset path optimization algorithm;
and outputting obstacle avoidance reminding information according to the obstacle avoidance route.
Optionally, the method further includes:
identifying whether the target tracking object generates a preset dangerous action or not according to the characteristic information of the target tracking object and a pre-trained action identification model;
and if the target tracking object generates a preset dangerous action, sending alarm information to a preset terminal device.
Optionally, the method further includes:
acquiring current geographical position information through a map module;
and determining the positioning information of the target tracking object according to the geographical position information and the motion information of the target tracking object.
In a second aspect, the present application provides a control device for a cane with millimeter wave radar, the device comprising:
the acquisition module is used for acquiring point cloud data in a preset range based on the received echo signal of the millimeter wave radar;
the analysis module is used for analyzing the point cloud data to obtain motion information and characteristic information of each detection object in the preset range;
the determining module is used for determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object;
and the obstacle avoidance module is used for carrying out obstacle avoidance processing if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions.
In a third aspect, the present application provides an intelligent walking stick, which includes a millimeter wave radar, a data processing device and an obstacle avoidance device:
the millimeter wave radar is used for acquiring point cloud data in a preset range based on the received echo signal;
the data processing device is used for analyzing the point cloud data to obtain motion information and characteristic information of each detection object in the preset range; determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object;
and the obstacle avoidance device is used for carrying out obstacle avoidance processing when the running information of the target tracking object and the motion information of the target obstacle meet preset conditions.
Optionally, the data processing apparatus is specifically configured to:
tracking and positioning each detection object according to the point cloud data and a Kalman tracking algorithm to obtain motion information of each detection object;
wherein the motion information comprises: the speed of the test object, the distance of the test object relative to the hand wand, and the angle of the test object relative to the direction of movement of the hand wand.
Optionally, the obstacle avoidance device is specifically configured to:
when the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, determining an obstacle avoidance route according to the motion information of the target obstacle and a preset path optimization algorithm;
and outputting obstacle avoidance reminding information according to the obstacle avoidance route.
Optionally, the smart cane further comprises a communication device;
the data processing device is further used for identifying whether the target tracking object generates a preset dangerous action or not according to the characteristic information of the target tracking object and a pre-trained action identification model;
and the communication device is used for sending alarm information to preset terminal equipment when the target tracking object generates preset dangerous actions.
Optionally, the intelligent walking stick further comprises a map positioning device;
the map positioning device is used for acquiring current geographic position information;
the data processing device is further configured to determine positioning information of the target tracking object according to the geographic position information and the motion information of the target tracking object.
In a fourth aspect, the present application provides an electronic device, comprising: the system comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
the memory is used for storing a computer program;
the processor is configured to, when executing the computer program, implement the method steps of the first aspect.
In a fifth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method steps of the first aspect described above.
In a sixth aspect, embodiments of the present application further provide a computer program product containing instructions, which when executed on a computer, cause the computer to perform the method steps of the first aspect.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages:
the embodiment of the application provides a control method of a walking stick with a millimeter wave radar, in the method, point cloud data in a preset range can be obtained based on received echo signals of the millimeter wave radar, then the point cloud data are analyzed to obtain motion information and characteristic information of all detection objects in the preset range, then classification results of all detection objects are determined according to the characteristic information of all detection objects and a classification model trained in advance, and target tracking objects and target obstacles are determined according to the classification results of all detection objects. And if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, obstacle avoidance processing is carried out. Based on the scheme, the target tracking object can be tracked and positioned through the millimeter wave radar, and surrounding target obstacles are identified, so that an obstacle avoidance function is realized, the target tracking object is prevented from colliding or rubbing with the target obstacles, the safety of a user is improved, and the detection accuracy is high.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a flow chart of a control method of a walking stick with a millimeter wave radar according to an embodiment of the present application;
FIG. 2 is a flowchart of an example of a control method for a walking stick with a millimeter wave radar according to an embodiment of the present disclosure;
FIG. 3 is a schematic structural diagram of a control device of a walking stick with a millimeter wave radar according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of an intelligent walking stick provided by an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the invention provides a control method of a walking stick with a millimeter wave radar, which can be applied to the walking stick, in particular to a control system of the walking stick. A plurality of millimeter wave radars (namely millimeter wave radar groups) can be arranged on the walking stick, and each millimeter wave radar can be arranged in a plurality of directions of the walking stick so as to realize 360-degree all-around detection of the walking stick. In implementation, the specific setting position of the millimeter wave radar can be determined according to the detection range of the millimeter wave radar and the actual shape of the walking stick, and the embodiment of the application does not limit the setting position.
The following describes in detail a control method of a walking stick with a millimeter wave radar according to an embodiment of the present application with reference to a specific embodiment, and as shown in fig. 1, specific steps are as follows.
In the embodiment of the application, the millimeter wave radar can emit the electromagnetic wave in the millimeter wave frequency band within the preset range, then the millimeter wave radar can collect the echo Signal, perform analog-to-Digital conversion on the collected echo Signal, and then perform Digital Signal Processing (DSP) on the converted Signal to obtain the point cloud data.
And 102, analyzing the point cloud data to obtain motion information and characteristic information of each detection object in a preset range.
In the embodiment of the application, the motion information and the characteristic information of each detection object existing around the walking stick (namely in a preset range) can be obtained by analyzing and calculating the processed signals. The motion information may include: the speed of the sensing object, the distance of the sensing object relative to the cane, and the angle of the sensing object relative to the direction of movement of the cane. The characteristic information may include a combination of one or more of the following: micro-doppler shift, interference bandwidth, centroid, total bandwidth, frequency, height and standard deviation. The process of obtaining motion information and feature information according to point cloud data belongs to the prior art, and is not repeated in the application.
Optionally, the specific processing procedure of analyzing the point cloud data to obtain the motion information of each detection object within the preset range may be as follows: and tracking and positioning each detection object according to the point cloud data and a Kalman tracking algorithm to obtain the motion information of each detection object.
In the embodiment of the application, the memory component of the walking stick can store the Kalman tracking algorithm, and after the point cloud data is detected by the millimeter wave radar, the point cloud data can be processed by the Kalman tracking algorithm, so that the tracking and positioning of each detection object are realized, and the motion information of each detection object is obtained.
And 103, determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object.
Wherein the target tracking object may be a user of the cane, i.e. a user.
In the embodiment of the application, the classification result of each detection object can be determined according to the feature information of each detection object and a pre-trained classification model, and then the target tracking object and the target obstacle can be determined according to the classification result of each detection object. Specifically, the feature information of each detection object may be input to a classification model trained in advance, and a classification result of each detection object may be output, where the classification result may indicate that the detection object is a target tracking object or is not a target tracking object. Thus, the target tracking object can be determined based on the classification result of each detection object. For example, the classification result may be a user or another object, where the classification result is a detection object of the user, i.e., a target tracking object, and the classification result is other objects and is not the target tracking object. Furthermore, the types of other detection objects, such as passerby, television, refrigerator, gate, water pit and the like, can be identified through the classification model.
Optionally, the training process of the classification model may be: training samples are obtained, which may include positive and negative samples. The positive sample comprises characteristic information of the user and a label belonging to the user; the negative examples include characteristic information of other objects, and labels of types of other objects. Then, the initial classification model can be trained according to the training samples and a preset training algorithm to obtain a trained classification model. The classification model may be implemented by a model having a classification function in the prior art, such as an SVM (Support Vector Machine), an LSSVM (least square Support Vector Machine), and the like, which is not limited in the embodiment of the present application.
And 104, if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, carrying out obstacle avoidance processing.
In the embodiment of the application, after the walking stick is started, the current position information is initialized, then, the current position and the motion track of the target tracking object can be determined according to the motion information of the target tracking object (such as the speed of the target tracking object, the distance of the target tracking object relative to the walking stick, and the angle of the target tracking object relative to the moving direction of the walking stick), and then, the action route of the target tracking object is determined according to the current position and the motion track of the target tracking object, so as to realize the tracking and positioning of the target tracking object.
Since other detection objects (i.e., target obstacles) may exist in the movement route of the user, in order to avoid collision or friction between the user and the target obstacles, it may be determined whether the operation information of the user and the movement information of the target obstacles satisfy preset conditions, that is, whether collision or friction between the user and the target obstacles may occur. Specifically, the distance between the user and the target obstacle, the movement route of the user, and the position or movement track of the target obstacle may be determined according to the operation information of the user and the movement information of the target obstacle. Accordingly, the preset condition may include at least one of the following conditions: the target obstacle is positioned on the current moving route of the target tracking object; the minimum distance between the target obstacle and the target tracking object is smaller than or equal to a preset distance; the motion trail of the target obstacle and the action route have an intersection point, and the time difference of reaching the intersection point is less than or equal to the preset time.
And if any one of the preset obstacle avoidance conditions is met, the walking stick can carry out obstacle avoidance processing. Optionally, in an implementation manner, if the walking stick does not have a moving device, when it is determined that the operation information of the target tracking object and the movement information of the target obstacle meet the preset condition, an obstacle avoidance route may be determined according to the movement information of the target obstacle and a preset path optimization algorithm, where the obstacle avoidance route is a moving route for enabling a user to avoid the target obstacle. Then, the walking stick can output obstacle avoidance reminding information according to the obstacle avoidance route. The path optimization algorithm in the prior art can be applied to the embodiment of the present application, and the embodiment of the present application is not limited.
For example, if an electric fan exists on the current moving route of the user, replanning an obstacle avoidance route bypassing the electric fan, and outputting obstacle avoidance reminding information; or, when the electric vehicle comes to the user, the motion track of the electric vehicle and the action route of the user have an intersection point, and the time difference of the electric vehicle reaching the intersection point is less than or equal to the preset time, the user can be reminded to stop walking, and an alarm is given out to prompt the electric vehicle to avoid the user. Optionally, the obstacle avoidance reminding information may further include information on the type and position of the target obstacle. For example, the obstacle avoidance reminding information may be "there is an electric fan 2 meters ahead, please move to the right for 1 meter and continue to move".
In another implementation mode, the walking stick is provided with a movement device, so that the walking stick can drive according to an obstacle avoidance route while outputting obstacle avoidance reminding information so as to lead a user to avoid a target obstacle. In addition, after the walking stick detects that the walking stick avoids the target obstacle, the walking stick can be automatically switched from the running mode to the user control mode, or the walking stick can be switched from the running mode to the user control mode after receiving a running stopping instruction input by a user.
In addition, if the preset obstacle avoidance condition is not satisfied, the processing may not be performed.
Optionally, the intelligent walking stick can also realize accurate positioning of the user, and the specific processing process is as follows: acquiring current geographical position information through a map module; and determining the positioning information of the target tracking object according to the geographical position information and the motion information of the target tracking object.
In this application embodiment, the walking stick can also obtain current geographical position information through the map module, and the map module can be a Global Positioning System (GPS) module. Therefore, the user can be roughly positioned according to the geographical position information, then the position information (namely the relative position information) of the user relative to other detection objects is determined, and the positioning information of the target tracking object is further determined by combining the geographical position information and the relative position information, so that the target tracking object is accurately positioned. In one example, the geographical location information positions that the user is located in a shopping square, and when the walking stick detects a fountain, the location of the user relative to the fountain can be determined according to the location information of the fountain and the motion information of the user, so that accurate positioning is achieved. In another example, the geographical location information locates that the user is located at home, and the cane can detect the user's location information relative to surrounding household appliances, thereby enabling accurate location of the user indoors. In addition, the walking stick can also send the positioning information of the user to the preset terminal equipment, and the terminal equipment can display the positioning information of the user, so that a guardian of the user can acquire the accurate position of the user, and the user is prevented from being lost.
Optionally, the cane in this application embodiment can also realize preventing falling the alarming function, and specific processing procedure can be: identifying whether the target tracking object generates a preset dangerous action or not according to the characteristic information of the target tracking object and a pre-trained action identification model; and if the target tracking object generates a preset dangerous action, sending alarm information to a preset terminal device.
In the embodiment of the application, a pre-trained motion recognition model can be stored in the walking stick. After determining the feature information of the target tracking object, the feature information of the target tracking object may be input to a classification model trained in advance to recognize whether a preset dangerous action, such as a wrestling or slipping, occurs to the target tracking object. If the identified result is that the target tracking object has a preset dangerous action, alarm information may be sent to a preset terminal device, where the alarm information may include event information (e.g., occurrence of slipping), positioning information of the user, time information, and the like. After receiving the alarm information, the terminal equipment can output the alarm information to prompt people to rescue the target tracking object. Therefore, the guardian of the user can be prompted to cause danger in time, so that the guardian can help the user in time. If the result of the recognition is that the target tracking object does not have the preset dangerous action, the processing is not required.
In another implementation mode, the walking stick can further comprise a voice output component such as a loudspeaker, so that when the target tracking object is detected to have a preset dangerous action, alarm information can be output through the voice output component to prompt surrounding people that the target tracking object falls down or slides to, and therefore the surrounding people can timely rescue the target tracking object.
Optionally, the training process of the motion recognition model may be: training samples are obtained, which may include positive and negative samples. The positive sample comprises characteristic information of a user when a preset dangerous action occurs and a label of the dangerous action; the negative examples include characteristic information of the user when no preset dangerous action occurs and a label of the user when no dangerous action occurs. Then, the initial motion recognition model can be trained according to the training samples and a preset training algorithm to obtain the motion recognition model. The action recognition model may be implemented by any model with a recognition function in the prior art, such as an SVM, an LSSVM, and the like, which is not limited in the embodiment of the present application.
Optionally, in this application, the cane may also provide a leisure and entertainment function, and a specific processing procedure may be: and when a preset playing instruction is received, acquiring the multimedia data and playing the multimedia data. In one implementation, a voice output component such as a speaker may be disposed in the stick, and a user may enable the stick to receive a playing instruction by performing a preset operation (for example, clicking a preset button, or performing control through voice, etc.), and then the stick may obtain multimedia data, for example, locally pre-stored multimedia data may be obtained, or the multimedia data may also be obtained through the internet. The multimedia data may be data of any multimedia file such as songs, news, novels, and the like. The cane can play the multimedia data through the voice output component, thereby providing the leisure and entertainment functions for the user.
In the embodiment of the application, the target tracking object can be tracked and positioned through the millimeter wave radar, the target obstacles around the target tracking object can be identified, the dynamic path real-time obstacle avoidance is realized, the collision or friction between a user and the obstacles is avoided, and the safety of the user is guaranteed. Meanwhile, dangerous actions such as wrestling and sliding of the user can be detected, an alarm is given when the dangerous actions of the user are detected, and other people can conveniently and timely stretch out of the rescue aid for rescue. In addition, the map module can be combined to realize accurate positioning and tracking of the user, and the user is prevented from being lost.
The embodiment of the application also provides an example of a control method of the walking stick with the millimeter wave radar, and as shown in fig. 2, the method specifically comprises the following steps.
Step 201, point cloud data in a preset range is obtained based on the received echo signal of the millimeter wave radar.
Step 202, analyzing the point cloud data to obtain motion information and characteristic information of each detection object within a preset range.
Step 203, determining the classification result of each detection object according to the feature information of each detection object and a classification model trained in advance.
Step 204, determining a target tracking object and a target obstacle.
Step 205, whether the running information of the target tracking object and the motion information of the target obstacle meet preset conditions or not is performed.
If so, go to step 206-207; if not, the process is ended.
And step 206, if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, determining an obstacle avoidance route according to the motion information of the target obstacle and a preset path optimization algorithm.
And step 207, outputting obstacle avoidance reminding information according to the obstacle avoidance route.
And step 208, identifying whether the target tracking object generates a preset dangerous action or not according to the characteristic information of the target tracking object and a pre-trained action identification model.
If yes, go to step 209; if not, the process is ended.
And step 209, sending alarm information to preset terminal equipment.
Step 210, obtaining current geographical location information through a map module.
And step 211, determining the positioning information of the target tracking object according to the geographical position information and the motion information of the target tracking object.
Based on the same technical concept, the embodiment of the present application further provides a control device for a walking stick with a millimeter wave radar, as shown in fig. 3, the device includes:
an obtaining module 310, configured to obtain point cloud data within a preset range based on a received echo signal of the millimeter wave radar;
the analysis module 320 is configured to analyze the point cloud data to obtain motion information and feature information of each detection object within the preset range;
a determining module 330, configured to determine a classification result of each detection object according to the feature information of each detection object and a pre-trained classification model, and determine a target tracking object and a target obstacle according to the classification result of each detection object;
and the obstacle avoidance module 340 is configured to perform obstacle avoidance processing if the operation information of the target tracking object and the motion information of the target obstacle meet preset conditions.
Optionally, the feature information includes one or more of the following: micro-doppler bias, coherence bandwidth, centroid, total bandwidth, frequency, height, and standard deviation.
Optionally, the analysis module 320 is specifically configured to:
tracking and positioning each detection object according to the point cloud data and a Kalman tracking algorithm to obtain motion information of each detection object;
wherein the motion information comprises: the speed of the test object, the distance of the test object relative to the hand wand, and the angle of the test object relative to the direction of movement of the hand wand.
Optionally, the preset condition includes at least one of the following conditions:
the target obstacle is positioned on the current moving route of the target tracking object;
the minimum distance between the target obstacle and the target tracking object is smaller than or equal to a preset distance;
the movement track of the target obstacle and the action route have an intersection point, and the time difference of reaching the intersection point is less than or equal to the preset time.
Optionally, the obstacle avoidance module 340 is specifically configured to:
if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, determining an obstacle avoidance route according to the motion information of the target obstacle and a preset path optimization algorithm;
and outputting obstacle avoidance reminding information according to the obstacle avoidance route.
Optionally, the apparatus further comprises:
the identification module is used for identifying whether the target tracking object generates a preset dangerous action or not according to the characteristic information of the target tracking object and a pre-trained action identification model;
and the sending module is used for sending alarm information to preset terminal equipment if the target tracking object generates preset dangerous actions.
Optionally, the apparatus further comprises:
the map module is used for acquiring current geographic position information;
and the positioning module is used for determining the positioning information of the target tracking object according to the geographical position information and the motion information of the target tracking object.
In the embodiment of the application, the target tracking object can be tracked and positioned through the millimeter wave radar, the target obstacles around the target tracking object can be identified, the dynamic path real-time obstacle avoidance is realized, the collision or friction between a user and the obstacles is avoided, and the safety of the user is guaranteed. Meanwhile, dangerous actions such as wrestling and sliding of the user can be detected, an alarm is given when the dangerous actions of the user are detected, and other people can conveniently and timely stretch out of the rescue aid for rescue. In addition, the map module can be combined to realize accurate positioning and tracking of the user, and the user is prevented from being lost.
Based on the same technical concept, the embodiment of the present application further provides an intelligent walking stick, as shown in fig. 4, the intelligent walking stick includes a millimeter wave radar 1, a data processing device 2, and an obstacle avoidance device 3:
the millimeter wave radar 1 is used for acquiring point cloud data in a preset range based on the received echo signal;
the data processing device 2 is configured to analyze the point cloud data to obtain motion information and feature information of each detection object within the preset range; determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object;
and the obstacle avoidance device 3 is configured to perform obstacle avoidance processing when the operation information of the target tracking object and the motion information of the target obstacle satisfy a preset condition.
Optionally, the data processing apparatus 2 is specifically configured to:
tracking and positioning each detection object according to the point cloud data and a Kalman tracking algorithm to obtain motion information of each detection object;
wherein the motion information comprises: the speed of the test object, the distance of the test object relative to the hand wand, and the angle of the test object relative to the direction of movement of the hand wand.
Optionally, the obstacle avoidance device 3 is specifically configured to:
when the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, determining an obstacle avoidance route according to the motion information of the target obstacle and a preset path optimization algorithm;
and outputting obstacle avoidance reminding information according to the obstacle avoidance route.
Optionally, the smart cane further comprises a communication device 4;
the data processing device 2 is further configured to identify whether the target tracking object generates a preset dangerous action according to the feature information of the target tracking object and a pre-trained action identification model;
and the communication device 4 is configured to send alarm information to a preset terminal device when the target tracking object performs a preset dangerous action.
Optionally, the smart cane further comprises a map location device 5;
the map positioning device 5 is used for acquiring current geographic position information;
the data processing device 2 is further configured to determine the positioning information of the target tracking object according to the geographic position information and the motion information of the target tracking object.
In the embodiment of the application, the target tracking object can be tracked and positioned through the millimeter wave radar, the target obstacles around the target tracking object can be identified, the dynamic path real-time obstacle avoidance is realized, the collision or friction between a user and the obstacles is avoided, and the safety of the user is guaranteed. Meanwhile, dangerous actions such as wrestling and sliding of the user can be detected, an alarm is given when the dangerous actions of the user are detected, and other people can conveniently and timely stretch out of the rescue aid for rescue. In addition, the map module can be combined to realize accurate positioning and tracking of the user, and the user is prevented from being lost.
An embodiment of the present application further provides an electronic device, as shown in fig. 5, the electronic device may include: the system comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 are communicated with each other through the communication bus 504.
A memory 503 for storing a computer program;
the processor 501 is configured to implement the above steps when executing the computer program stored in the memory 503.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring point cloud data in a preset range based on the received echo signal of the millimeter wave radar;
analyzing the point cloud data to obtain motion information and characteristic information of each detection object in the preset range;
determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object;
and if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, obstacle avoidance processing is carried out.
Optionally, the feature information includes one or more of the following: micro-doppler bias, coherence bandwidth, centroid, total bandwidth, frequency, height, and standard deviation.
Optionally, the computer program, when executed by the processor, further implements the steps of:
tracking and positioning each detection object according to the point cloud data and a Kalman tracking algorithm to obtain motion information of each detection object;
wherein the motion information comprises: the speed of the test object, the distance of the test object relative to the hand wand, and the angle of the test object relative to the direction of movement of the hand wand.
Optionally, the preset condition includes at least one of the following conditions:
the target obstacle is positioned on the current moving route of the target tracking object;
the minimum distance between the target obstacle and the target tracking object is smaller than or equal to a preset distance;
the movement track of the target obstacle and the action route have an intersection point, and the time difference of reaching the intersection point is less than or equal to the preset time.
Optionally, the computer program, when executed by the processor, further implements the steps of:
if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, determining an obstacle avoidance route according to the motion information of the target obstacle and a preset path optimization algorithm;
and outputting obstacle avoidance reminding information according to the obstacle avoidance route.
Optionally, the computer program, when executed by the processor, further implements the steps of:
identifying whether the target tracking object generates a preset dangerous action or not according to the characteristic information of the target tracking object and a pre-trained action identification model;
and if the target tracking object generates a preset dangerous action, sending alarm information to a preset terminal device.
Optionally, the computer program, when executed by the processor, further implements the steps of:
acquiring current geographical position information through a map module;
and determining the positioning information of the target tracking object according to the geographical position information and the motion information of the target tracking object.
The present application provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the above-mentioned method steps.
Embodiments of the present application also provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the above-mentioned method steps.
It should be noted that, for the above-mentioned apparatus, electronic device, computer-readable storage medium and computer program product embodiments, since they are substantially similar to the method embodiments, the description is simple, and the relevant points can be referred to the partial description of the method embodiments.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (13)
1. A method for controlling a cane with a millimeter wave radar, the method comprising:
acquiring point cloud data in a preset range based on the received echo signal of the millimeter wave radar;
analyzing the point cloud data to obtain motion information and characteristic information of each detection object in the preset range;
determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object;
and if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, obstacle avoidance processing is carried out.
2. The method of claim 1, wherein the feature information comprises a combination of one or more of: micro-doppler bias, coherence bandwidth, centroid, total bandwidth, frequency, height, and standard deviation.
3. The method according to claim 1, wherein the obtaining motion information of each detection object within the preset range by analyzing the point cloud data comprises:
tracking and positioning each detection object according to the point cloud data and a Kalman tracking algorithm to obtain motion information of each detection object;
wherein the motion information comprises: the speed of the test object, the distance of the test object relative to the hand wand, and the angle of the test object relative to the direction of movement of the hand wand.
4. The method according to claim 1, wherein the preset condition comprises at least one of the following conditions:
the target obstacle is positioned on the current moving route of the target tracking object;
the minimum distance between the target obstacle and the target tracking object is smaller than or equal to a preset distance;
the movement track of the target obstacle and the action route have an intersection point, and the time difference of reaching the intersection point is less than or equal to the preset time.
5. The method according to claim 1, wherein if the operation information of the target tracking object and the motion information of the target obstacle satisfy a preset condition, performing obstacle avoidance processing, including:
if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, determining an obstacle avoidance route according to the motion information of the target obstacle and a preset path optimization algorithm;
and outputting obstacle avoidance reminding information according to the obstacle avoidance route.
6. The method of claim 1, further comprising:
identifying whether the target tracking object generates a preset dangerous action or not according to the characteristic information of the target tracking object and a pre-trained action identification model;
and if the target tracking object generates a preset dangerous action, sending alarm information to a preset terminal device.
7. The method of any of claim 1, further comprising:
acquiring current geographical position information through a map module;
and determining the positioning information of the target tracking object according to the geographical position information and the motion information of the target tracking object.
8. A control device for a pole with millimeter wave radar, characterized in that the device comprises:
the acquisition module is used for acquiring point cloud data in a preset range based on the received echo signal of the millimeter wave radar;
the analysis module is used for analyzing the point cloud data to obtain motion information and characteristic information of each detection object in the preset range;
the determining module is used for determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object;
and the obstacle avoidance module is used for carrying out obstacle avoidance processing if the running information of the target tracking object and the motion information of the target obstacle meet preset conditions.
9. The intelligent walking stick is characterized by comprising a millimeter wave radar, a data processing device and an obstacle avoidance device:
the millimeter wave radar is used for acquiring point cloud data in a preset range based on the received echo signal;
the data processing device is used for analyzing the point cloud data to obtain motion information and characteristic information of each detection object in the preset range; determining the classification result of each detection object according to the characteristic information of each detection object and a pre-trained classification model, and determining a target tracking object and a target obstacle according to the classification result of each detection object;
and the obstacle avoidance device is used for carrying out obstacle avoidance processing when the running information of the target tracking object and the motion information of the target obstacle meet preset conditions.
10. The smart cane of claim 9, wherein the data processing device is specifically configured to:
tracking and positioning each detection object according to the point cloud data and a Kalman tracking algorithm to obtain motion information of each detection object;
wherein the motion information comprises: the speed of the test object, the distance of the test object relative to the hand wand, and the angle of the test object relative to the direction of movement of the hand wand.
11. The intelligent cane as claimed in claim 9, wherein the obstacle avoidance device is specifically configured to:
when the running information of the target tracking object and the motion information of the target obstacle meet preset conditions, determining an obstacle avoidance route according to the motion information of the target obstacle and a preset path optimization algorithm;
and outputting obstacle avoidance reminding information according to the obstacle avoidance route.
12. The smart cane of claim 9 further comprising a communication device;
the data processing device is further used for identifying whether the target tracking object generates a preset dangerous action or not according to the characteristic information of the target tracking object and a pre-trained action identification model;
and the communication device is used for sending alarm information to preset terminal equipment when the target tracking object generates preset dangerous actions.
13. The smart cane of any one of claim 9 further comprising a map locating device;
the map positioning device is used for acquiring current geographic position information;
the data processing device is further configured to determine positioning information of the target tracking object according to the geographic position information and the motion information of the target tracking object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910786764.0A CN110728701B (en) | 2019-08-23 | 2019-08-23 | Control method and device for walking stick with millimeter wave radar and intelligent walking stick |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910786764.0A CN110728701B (en) | 2019-08-23 | 2019-08-23 | Control method and device for walking stick with millimeter wave radar and intelligent walking stick |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110728701A true CN110728701A (en) | 2020-01-24 |
| CN110728701B CN110728701B (en) | 2022-07-12 |
Family
ID=69217158
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910786764.0A Active CN110728701B (en) | 2019-08-23 | 2019-08-23 | Control method and device for walking stick with millimeter wave radar and intelligent walking stick |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110728701B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112329754A (en) * | 2021-01-07 | 2021-02-05 | 深圳市速腾聚创科技有限公司 | Obstacle recognition model training method, obstacle recognition method, device and system |
| CN112472432A (en) * | 2020-11-25 | 2021-03-12 | 武汉理工大学 | Automatic walking stick-wheelchair following system and method |
| CN112539531A (en) * | 2020-12-08 | 2021-03-23 | 青岛海尔空调器有限总公司 | Obstacle avoidance control method and device and air conditioner |
| CN113671849A (en) * | 2021-09-03 | 2021-11-19 | 珠海格力电器股份有限公司 | Intelligent household equipment control method and device |
| CN113780399A (en) * | 2021-09-03 | 2021-12-10 | 珠海格力电器股份有限公司 | Object classification method, device, equipment and storage medium |
| CN114047513A (en) * | 2021-11-11 | 2022-02-15 | 珠海格力电器股份有限公司 | Voice prompt method and device and electronic equipment |
| CN114199168A (en) * | 2021-12-16 | 2022-03-18 | 珠海格力电器股份有限公司 | Indoor volume detection method, device, equipment and medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012143952A2 (en) * | 2011-04-20 | 2012-10-26 | B Amutha | A system and apparatus for safe remote on-line tracing, shadowing, surveillance, inter-communication, location, navigation, tagging, rescue, recovery and restitution of humans and stolen/missing chattels, and the method/s thereof |
| CN102973395A (en) * | 2012-11-30 | 2013-03-20 | 中国舰船研究设计中心 | Multifunctional intelligent blind guiding method, processor and multifunctional intelligent blind guiding device |
| US20130131985A1 (en) * | 2011-04-11 | 2013-05-23 | James D. Weiland | Wearable electronic image acquisition and enhancement system and method for image acquisition and visual enhancement |
| CN107422730A (en) * | 2017-06-09 | 2017-12-01 | 武汉市众向科技有限公司 | The AGV transportation systems of view-based access control model guiding and its driving control method |
| CN108845321A (en) * | 2018-04-19 | 2018-11-20 | 北京三快在线科技有限公司 | Recognition methods, device and the unmanned smart machine of target object |
| CN109275089A (en) * | 2018-09-18 | 2019-01-25 | 广东小天才科技有限公司 | A kind of smart phone wrist-watch and method improving personal safety |
| CN109541612A (en) * | 2018-11-09 | 2019-03-29 | 同济大学 | Self aligning system in robot chamber based on single line laser radar identification column |
| CN109581361A (en) * | 2018-11-22 | 2019-04-05 | 九牧厨卫股份有限公司 | A kind of detection method, detection device, terminal and detection system |
-
2019
- 2019-08-23 CN CN201910786764.0A patent/CN110728701B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130131985A1 (en) * | 2011-04-11 | 2013-05-23 | James D. Weiland | Wearable electronic image acquisition and enhancement system and method for image acquisition and visual enhancement |
| WO2012143952A2 (en) * | 2011-04-20 | 2012-10-26 | B Amutha | A system and apparatus for safe remote on-line tracing, shadowing, surveillance, inter-communication, location, navigation, tagging, rescue, recovery and restitution of humans and stolen/missing chattels, and the method/s thereof |
| CN102973395A (en) * | 2012-11-30 | 2013-03-20 | 中国舰船研究设计中心 | Multifunctional intelligent blind guiding method, processor and multifunctional intelligent blind guiding device |
| CN107422730A (en) * | 2017-06-09 | 2017-12-01 | 武汉市众向科技有限公司 | The AGV transportation systems of view-based access control model guiding and its driving control method |
| CN108845321A (en) * | 2018-04-19 | 2018-11-20 | 北京三快在线科技有限公司 | Recognition methods, device and the unmanned smart machine of target object |
| CN109275089A (en) * | 2018-09-18 | 2019-01-25 | 广东小天才科技有限公司 | A kind of smart phone wrist-watch and method improving personal safety |
| CN109541612A (en) * | 2018-11-09 | 2019-03-29 | 同济大学 | Self aligning system in robot chamber based on single line laser radar identification column |
| CN109581361A (en) * | 2018-11-22 | 2019-04-05 | 九牧厨卫股份有限公司 | A kind of detection method, detection device, terminal and detection system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112472432A (en) * | 2020-11-25 | 2021-03-12 | 武汉理工大学 | Automatic walking stick-wheelchair following system and method |
| CN112539531A (en) * | 2020-12-08 | 2021-03-23 | 青岛海尔空调器有限总公司 | Obstacle avoidance control method and device and air conditioner |
| CN112329754A (en) * | 2021-01-07 | 2021-02-05 | 深圳市速腾聚创科技有限公司 | Obstacle recognition model training method, obstacle recognition method, device and system |
| CN112329754B (en) * | 2021-01-07 | 2021-05-14 | 深圳市速腾聚创科技有限公司 | Obstacle recognition model training method, obstacle recognition method, device and system |
| CN113671849A (en) * | 2021-09-03 | 2021-11-19 | 珠海格力电器股份有限公司 | Intelligent household equipment control method and device |
| CN113780399A (en) * | 2021-09-03 | 2021-12-10 | 珠海格力电器股份有限公司 | Object classification method, device, equipment and storage medium |
| CN114047513A (en) * | 2021-11-11 | 2022-02-15 | 珠海格力电器股份有限公司 | Voice prompt method and device and electronic equipment |
| CN114199168A (en) * | 2021-12-16 | 2022-03-18 | 珠海格力电器股份有限公司 | Indoor volume detection method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110728701B (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110728701B (en) | Control method and device for walking stick with millimeter wave radar and intelligent walking stick | |
| US9041592B2 (en) | Radar sensor and method of detecting object using the same | |
| US9075143B2 (en) | Sparse array RF imaging for surveillance applications | |
| US10902263B1 (en) | Image processing system for object identification | |
| EP3712809A1 (en) | Character recognition in air-writing based on network of radars | |
| Barnes et al. | Exploiting 3D semantic scene priors for online traffic light interpretation | |
| CN110674696A (en) | Monitoring method, device, system, monitoring equipment and readable storage medium | |
| CN111174778B (en) | Building entrance determining method and device based on pedestrian dead reckoning | |
| CN112462368B (en) | Obstacle detection method and device, vehicle and storage medium | |
| CN110873863A (en) | Target display method, radar system and electronic equipment | |
| US10969464B2 (en) | Method for registering presence of a stationary object exhibiting motion at a location in a scene monitored by a radar detector | |
| CN110765823A (en) | Target identification method and device | |
| Scheiner et al. | Automated ground truth estimation of vulnerable road users in automotive radar data using GNSS | |
| CN103577795A (en) | Detection equipment and method, detector generation equipment and method and monitoring system | |
| CN111308456A (en) | Target position judging method and device, millimeter wave radar and storage medium | |
| Mehrabian et al. | Sensor fusion for indoor positioning system through improved RSSI and PDR methods | |
| CN115424233A (en) | Target detection method and target detection device based on information fusion | |
| US11263684B2 (en) | Product recommendation method and product recommendation device | |
| CN113253299B (en) | Obstacle detection method, obstacle detection device and storage medium | |
| Van Eeden et al. | A comparative cepstral based analysis of simulated and measured S-band and X-band radar Doppler spectra of human motion | |
| CN110727277A (en) | Control method and device of car washer with millimeter wave radar and intelligent car washer | |
| KR102456151B1 (en) | Sensor fusion system based on radar and camera and method of calculating the location of nearby vehicles | |
| JP7294323B2 (en) | Moving body management device, moving body management system, moving body management method, and computer program | |
| CN113780399A (en) | Object classification method, device, equipment and storage medium | |
| CN111148016B (en) | Mobile equipment positioning method and system in indoor environment and mobile equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |