CN109993308B - Cloud platform-based shared learning system and method, shared platform and method and medium - Google Patents
Cloud platform-based shared learning system and method, shared platform and method and medium Download PDFInfo
- Publication number
- CN109993308B CN109993308B CN201910248301.9A CN201910248301A CN109993308B CN 109993308 B CN109993308 B CN 109993308B CN 201910248301 A CN201910248301 A CN 201910248301A CN 109993308 B CN109993308 B CN 109993308B
- Authority
- CN
- China
- Prior art keywords
- model
- sharing
- private
- cloud
- platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 37
- 230000004927 fusion Effects 0.000 claims abstract description 160
- 230000007613 environmental effect Effects 0.000 claims description 60
- 238000004891 communication Methods 0.000 claims description 42
- 238000004364 calculation method Methods 0.000 claims description 33
- 238000011156 evaluation Methods 0.000 claims description 30
- 230000002787 reinforcement Effects 0.000 claims description 29
- 238000010606 normalization Methods 0.000 claims description 20
- 238000004088 simulation Methods 0.000 claims description 16
- 238000013508 migration Methods 0.000 claims description 15
- 230000005012 migration Effects 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 11
- 238000005728 strengthening Methods 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 description 11
- 238000013526 transfer learning Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 239000003795 chemical substances by application Substances 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000006386 memory function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N7/00—Computing arrangements based on specific mathematical models
- G06N7/02—Computing arrangements based on specific mathematical models using fuzzy logic
- G06N7/06—Simulation on general purpose computers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Mathematical Analysis (AREA)
- Molecular Biology (AREA)
- Fuzzy Systems (AREA)
- Biomedical Technology (AREA)
- Algebra (AREA)
- Artificial Intelligence (AREA)
- Computational Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Automation & Control Theory (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The invention provides a robot sharing learning system based on a cloud platform, which comprises a private model generating terminal and a cloud fusion computing sharing platform, wherein the private model generating terminal is used for uploading a locally generated private model and environment characteristic information collected by generating the private model to the cloud fusion computing sharing platform, the cloud fusion computing sharing platform comprises a model fusion computing module, the model fusion computing module is used for carrying out fusion computation on the uploaded private model, the environment characteristic information uploaded by other robot terminals, the private model generating terminal and/or other private model generating terminals and a sharing model stored on the cloud fusion computing sharing platform, and generating a new sharing model which is used for being downloaded and/or learned by other robot terminals, the private model generating terminal and/or other private model generating terminals.
Description
Technical Field
The invention relates to a robot sharing learning system and method, in particular to a cloud platform-based robot sharing learning system and method for warehouse logistics.
Background
Robot navigation refers to a given robot's target point that the robot can reach without encountering obstacles, while at the same time the path should be made as short as possible. Decision model learning based on reinforcement learning in path planning is currently a more advanced approach. Reinforcement learning is a learning of an Agent (Agent) in a "trial and error" manner, and directs the behavior by interacting with the environment to obtain the maximum prize for the Agent, and is different from supervised learning in connection with learning, which is mainly represented by reinforcement signals, in which reinforcement signals provided by the environment are an evaluation of how well an action is generated (typically a scalar signal), rather than telling the reinforcement learning system RLS (reinforcement LEARNING SYSTEM) how to generate the correct action. Since little information is provided by the external environment, RLS must learn from its own experiences. In this way, the RLS obtains knowledge in the context of the action-assessment, improving the action plan to suit the context. However, this approach still has drawbacks including the setting of the training environment, the long training time, the inability to use previous or other robotically learned experiences, etc.
The robot cloud sharing technology is a technology that solves some problems in the robot field in combination with the cloud computing technology. The problem that experience fusion cannot be carried out due to long training time in training of a robot navigation decision model can be effectively solved by utilizing a robot cloud sharing technology. Chinese patent CN108801269a proposes an indoor cloud robot navigation system, but the system does not propose a method in specific path planning, but only in map positioning, and cannot solve the problems of long training time and experience fusion. At present, the invention of a navigation decision model learning system by strengthening learning through a cloud sharing technology for a robot does not exist.
Disclosure of Invention
The technical problem to be solved by the invention is that the robot navigation is limited by the setting of training environment, the training time is long, and the previous experience or other robot learning experience cannot be utilized.
In order to solve the technical problems, the invention provides a robot sharing learning system based on a cloud platform, which comprises a private model generating terminal and a cloud fusion computing sharing platform, wherein the private model generating terminal is used for uploading a locally generated private model and environment characteristic information collected by generating the private model to the cloud fusion computing sharing platform, the cloud fusion computing sharing platform comprises a model fusion computing module, and the model fusion computing module is used for carrying out fusion computing on the uploaded private model, the other robot terminals, the private model generating terminal and/or the environment characteristic information uploaded by the other private model generating terminal and the sharing model stored on the cloud fusion computing sharing platform to generate a new sharing model, and the new sharing model is used for downloading and/or learning by the other robot terminals, the private model generating terminal and/or the other private model generating terminal.
According to a preferred embodiment of the present invention, the private model generating terminal includes a characteristic collecting module, an environment simulating module, and an reinforcement and transfer learning module, where the characteristic collecting module is used to collect environment characteristic information, the environment simulating module is used to generate an environment model by using the environment characteristic information, the reinforcement and transfer learning module includes a reinforcement learning unit, the reinforcement learning unit is used to perform reinforcement learning after inputting the environment characteristic information on the environment model, and output the private model, and the private model includes the environment model and a navigation policy generated by the private model generating terminal for the environment model.
According to a preferred embodiment of the present invention, the reinforcement and migration learning module further includes a migration computing unit, where the migration computing unit is configured to perform migration computation after inputting the environmental feature information into the downloaded shared model, and output new environmental feature information for reinforcement learning by the reinforcement learning unit to output a new private model, and the private model generating terminal also uploads the new private model to the cloud fusion computing shared platform to perform fusion computation further.
According to the preferred embodiment of the invention, the private model generation terminal adopts an intelligent robot terminal, a computer terminal or other intelligent terminal equipment.
According to the preferred embodiment of the invention, the characteristic collection module adopts a camera and/or a laser radar to collect the environmental characteristic information or directly receives the characteristic information of the simulation structure input by other devices as the environmental characteristic information.
According to the preferred embodiment of the invention, the environment simulation module adopts gazebo simulation software to construct a simulation environment.
According to the preferred embodiment of the invention, the model fusion calculation module comprises a model normalization unit and a fusion calculation unit, wherein the model normalization unit is used for inputting the environmental characteristic information into the private model uploaded by the private model generation terminal and the shared model stored on the cloud fusion calculation sharing platform to obtain an output result, further carrying out normalization processing on the output result, carrying out confidence evaluation on the output result by using information entropy, carrying out weighted summation on the evaluation score based on the confidence evaluation to obtain a label score, and carrying out fusion calculation on the environmental characteristic information uploaded by the private model generation terminal and the label score output by the model normalization unit by using the fusion calculation unit to obtain the new shared model.
According to a preferred embodiment of the present invention, the private model generating terminal further includes a first communication module, a first data receiving module, and a data uploading module, where the cloud fusion computing sharing platform further includes a second communication module, the first communication module is configured to obtain a network address of the cloud fusion computing sharing platform and establish communication with the second communication module of the cloud fusion computing sharing platform, the data uploading module is configured to upload the private model to the cloud fusion computing sharing platform after the communication between the first communication module and the cloud fusion computing sharing platform is established, and the first data receiving module is configured to download the sharing model on the cloud fusion computing sharing platform to the private model generating terminal after the communication between the first communication module and the cloud fusion computing sharing platform is established.
According to a preferred embodiment of the present invention, the cloud fusion computing sharing platform further includes a second data receiving module, a data downloading module, and a model storage module, where the second data receiving module is configured to receive the private model uploaded by the private model generating terminal to the cloud fusion computing sharing platform after communication is established with the private model generating terminal, and the data downloading module is configured to download the sharing model on the cloud fusion computing sharing platform to the private model generating terminal after communication is established with the private model generating terminal, and the model storage module is configured to store the sharing model on the cloud fusion computing sharing platform.
According to the preferred embodiment of the invention, the cloud fusion computing sharing platform comprises a model fusion computing module, wherein the model fusion computing module is used for carrying out fusion computing on the private model and the environmental characteristic information uploaded by the robot terminal and/or other intelligent terminals and the sharing model on the cloud fusion computing sharing platform to generate a new sharing model, and the new sharing model is used for downloading and/or learning of the robot terminal and/or other intelligent terminals and/or other robot terminals.
According to a preferred embodiment of the present invention, the model fusion calculation module includes a model normalization unit and a fusion calculation unit, where the model normalization unit is configured to input the environmental feature information uploaded by the robot terminal and/or the other intelligent terminals to the private model uploaded by the robot terminal and/or the other intelligent terminals and the sharing model stored on the cloud fusion calculation sharing platform to obtain an output result, further normalize the output result, perform confidence evaluation on the output result by using information entropy, and perform weighted summation based on an evaluation score to obtain a tag score, and the fusion calculation module performs fusion calculation by using the environmental feature information uploaded by the robot terminal and/or the other intelligent terminals and the tag score output by the model normalization unit to obtain the new sharing model.
According to a preferred embodiment of the present invention, the cloud fusion computing and sharing platform further includes a data receiving module, a data downloading module, and a model storage module, where the data receiving module is configured to receive the private model uploaded by the robot terminal and/or the other intelligent terminal to the cloud fusion computing and sharing platform after communication with the robot terminal and/or the other intelligent terminal is established, and the data downloading module is configured to download the sharing model on the cloud fusion computing and sharing platform to the robot terminal and/or the other intelligent terminal and/or the other robot terminal after communication with the robot terminal and/or the other intelligent terminal is established, and the model storage module is configured to store the sharing model generated by the cloud fusion computing and sharing platform.
In order to solve the technical problems, the invention provides a robot sharing learning method based on a cloud platform, which comprises the following steps:
step 1: the private model generating terminal collects environment information and generates a local private model;
Step 2: the private model generating terminal uploads the locally generated private model and the collected environmental characteristic information to the cloud fusion computing sharing platform;
Step 3: the cloud fusion computing sharing platform carries out fusion computation on the uploaded private model, and the environmental characteristic information uploaded by other robot terminals, the private model generating terminal and/or other private model generating terminals and the sharing model on the cloud fusion computing sharing platform to generate a new sharing model;
Step 4: and the other robot terminals, the private model generating terminal and/or the other private model generating terminals download the sharing model on the cloud fusion computing sharing platform.
According to a preferred embodiment of the present invention, in step 1, the step of generating the local private model by the private model generating terminal is as follows:
Step 1.1: the private model generating terminal collects environment information or receives characteristic information of simulation structures input by other devices as the environment characteristic information;
Step 1.2: the private model generating terminal builds an environment model by gazebo software based on the environment characteristic information;
Step1.3: and running a reinforcement learning algorithm on the environment model to obtain a navigation strategy, wherein the private model comprises the navigation strategy and the environment model.
According to a preferred embodiment of the present invention, in step 2, the step of uploading, by the private model generating terminal, the locally generated private model and the collected environmental feature information to the cloud fusion computing and sharing platform is as follows:
Step 2.1: the private model generating terminal acquires the communication address of the cloud fusion computing sharing platform and sends a request;
step 2.2: and the private model generating terminal uploads the private model and the environmental characteristic information to the cloud fusion computing sharing platform after receiving the information of the cloud fusion computing sharing platform.
According to a preferred embodiment of the present invention, in step 3, the step of generating a new sharing model by the cloud fusion computing sharing platform includes:
Step 3.1: the cloud fusion computing sharing platform inputs the environmental characteristic information uploaded by the private model generating terminal into the private model and a stored sharing model on the cloud fusion computing sharing platform to obtain an output result, and normalizes the output result;
step 3.2: the cloud fusion computing sharing platform carries out confidence evaluation on the normalization processing result by using information entropy;
Step 3.3: the cloud fusion computing sharing platform performs weighted summation based on the evaluation score of the confidence evaluation to obtain a label score;
step 3.4: and the cloud fusion computing and sharing platform generates environmental characteristic information uploaded by the terminal and the tag score according to the private model, and calculates to obtain a new sharing model.
According to a preferred embodiment of the present invention, in step 4, the other robot terminals, the private model generating terminal and/or the other private model generating terminal download the shared model on the cloud fusion computing shared platform and then perform migration learning to generate a new private model, where the step of performing migration learning by the private model generating terminal is:
Step 4.1: the private model generating terminal downloads the sharing model from the cloud fusion computing sharing platform;
Step 4.2: inputting the newly collected environmental characteristic data into the sharing model, and outputting evaluation values of all directions;
Step 4.3: adding the evaluation value to the environmental characteristic information to serve as a new environmental model;
step 4.4: and performing reinforcement learning in the new environment model, and generating a new private model for further uploading to the cloud fusion computing sharing platform.
In order to solve the technical problems, the invention provides a cloud fusion computing sharing-based method, which comprises the following steps:
Step 1: and carrying out fusion calculation on the received private models generated by the robot terminal and the private model generating terminal and the environment characteristic information uploaded by other robot terminals and/or other private model generating terminals and a sharing model on a cloud fusion calculation sharing platform to generate a new sharing model, wherein the new sharing model is used for downloading and/or learning of the robot terminal, other intelligent terminals and/or other robot terminals.
According to a preferred embodiment of the present invention, step 1, the step of generating a new sharing model is:
Step 1.1: inputting the received environmental characteristic information into the private model and a sharing model on a cloud fusion computing sharing platform to obtain an output result, and carrying out normalization processing on the output result;
step 1.2: performing confidence evaluation on the normalization processing result by using information entropy;
Step 1.3: weighting and summing based on the evaluation score of the confidence evaluation to obtain a label score;
step 1.4: and calculating according to the environmental characteristic information and the label score to obtain the new sharing model.
Compared with the prior art, in the cloud platform-based robot sharing learning and system, the cloud fusion computing sharing platform can accept the private model uploaded by the private model generating terminal, and fusion computing is carried out on the environment characteristic information uploaded by other robot terminals, the private model generating terminal and/or other private model generating terminals and the sharing model on the cloud fusion computing sharing platform to generate a new sharing model; the method and the system enable other robot terminals, the private model generation terminal and/or other private model generation terminals to download the sharing model on the cloud fusion computing sharing platform for local navigation, further transfer learning can be carried out, new private models are generated and uploaded to the cloud fusion computing sharing platform for further downloading and sharing, and the problems that training time is long and experience fusion cannot be carried out in training of a robot navigation decision model can be effectively solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
Fig. 1 is a system diagram of a cloud platform-based robot sharing learning system of the present invention.
Fig. 2 is a flow chart of the cloud platform-based robot sharing learning system of the present invention.
Fig. 3 is a flowchart of a robot sharing learning method based on a cloud platform.
Fig. 4 is a flowchart of a transfer learning method of the robot sharing learning method based on the cloud platform.
Fig. 5 is a flowchart of a fusion calculation of a robot sharing learning method based on a cloud platform.
Fig. 6 is a schematic structural diagram of a preferred embodiment of an electronic device for implementing the cloud platform-based robot sharing learning method in at least one embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
The terms first, second, third and the like in the description and in the claims of the invention and in the above-described figures, are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
As shown in fig. 1, fig. 1 is a system diagram of a robot sharing learning system based on a cloud platform, which implements the present invention. The cloud platform-based robot sharing learning system 100 includes a private model generation terminal 110 and a cloud fusion computing sharing platform 120. The private model generating terminal 110 includes a characteristic collecting module 111, an environment simulation module 112, a reinforcement and migration learning module 113, a first data interpretation module 114, a data uploading module 115, and a first communication module 116, wherein the reinforcement and migration learning module 113 includes a reinforcement learning unit 113a and a migration computing unit 113b. The cloud fusion computing and sharing platform 120 includes a data downloading module 121, a model fusion computing module 122, a model storage module 123, a second data receiving module 124, and a second communication module 125, where the model fusion computing module 122 includes a model normalization unit 122a and a fusion computing unit 122b.
Referring to fig. 2, a flowchart of the cloud platform-based robot sharing learning system of the present invention is shown in fig. 2. The characteristic collection module 111 in the private model generating terminal 110 is configured to collect environmental characteristic information, the environment simulation module 112 is configured to generate an environmental model using the environmental characteristic information, the reinforcement and migration learning module 113 includes a reinforcement learning unit 113a, reinforcement learning is performed after the environmental characteristic information is input on the environmental model, the private model is output, and the private model includes the environmental model and a navigation policy generated by the private model generating terminal 110 for the environmental model. The first communication module 116 of the private model generating terminal 110 is configured to obtain the network address of the cloud converged computing and sharing platform 120, and establish communication with the second communication module 124 of the cloud converged computing and sharing platform 120, and the data uploading module 115 is configured to upload the private model to the cloud converged computing and sharing platform 120 after the communication between the first communication module 116 and the cloud converged computing and sharing platform 120 is established.
The second data receiving module 124 in the cloud fusion computing sharing platform 120 is configured to receive the private model uploaded by the private model generating terminal 110 to the cloud fusion computing sharing platform 120 after communication with the private model generating terminal 110 is established. The model fusion calculation module 122 in the cloud fusion calculation sharing platform 120 is configured to perform fusion calculation on the uploaded private model, in combination with other robot terminals, the private model generating terminal 110, and/or environmental feature information uploaded by other private model generating terminals 110, and the sharing model stored on the cloud fusion calculation sharing platform 120, to generate a new sharing model, where the new sharing model is used for downloading and/or learning by other robot terminals, the private model generating terminal 110, and/or other private model generating terminals 120. The model normalization unit 122a in the cloud fusion computing and sharing platform 120 is configured to input the environmental characteristic information to the private model uploaded by the private model generating terminal 110 and the shared model stored on the cloud fusion computing and sharing platform 120 to obtain an output result, perform normalization processing on the output result, perform confidence evaluation on the output result by using information entropy, and perform weighted summation based on the evaluation score of the confidence evaluation to obtain a label score, and the fusion computing unit 122b performs fusion computation by using the environmental characteristic information uploaded by the private model generating terminal 110 and the label score output by the model normalization unit 122a to obtain the new shared model. After the first data receiving module 114 in the private model generating terminal 110 and the data downloading module 121 of the cloud fusion computing and sharing platform 120 are established in communication, the sharing model in the model storage module 123 on the cloud fusion computing and sharing platform 120 is downloaded to the private model generating terminal 110. The migration computing unit 113b of the reinforcement and migration learning module 113 in the private model generating terminal 110 performs migration computation after inputting the environmental characteristic information into the downloaded shared model, outputs new environmental characteristic information for the reinforcement learning unit 113a to perform reinforcement learning so as to output a new private model, and the private model generating terminal 110 also uploads the new private model to the cloud fusion computing shared platform 120 to further perform fusion computation, so as to generate a new shared model, where the new shared model is used for downloading and/or learning by other robot terminals, the private model generating terminal 110 and/or other private model generating terminals 110. By repeating this step, the model on the cloud becomes more and more powerful.
Specifically, the private model generating terminal 110 may be an intelligent robot terminal, a computer terminal, and/or other intelligent terminal devices. The intelligent robot terminal may be a mobile robot of a warehouse logistics system, the environmental characteristic information may be environmental characteristic information collected in a warehouse logistics environment, but is not limited to the foregoing, and the private model generating terminal 110 may also be other navigation robots applied to multiple environments. The feature collection module 111 uses a camera and/or a laser radar to randomly collect the environmental feature information or directly receives feature information of a simulation structure input by other devices as the environmental feature information. The environment simulation module adopts gazebo simulation software to construct a simulation environment.
As shown in fig. 3, fig. 3 is a flowchart of a learning method adopted by the cloud platform-based robot sharing learning system of the present invention. It will be appreciated that the order of the steps in the flowchart may be changed and certain steps omitted, depending on the requirements.
Step S1: the private model generating terminal collects environment information and generates a local private model;
Step S2: the private model generating terminal uploads the locally generated private model and the collected environmental characteristic information to the cloud fusion computing sharing platform;
Step S3: the cloud fusion computing sharing platform carries out fusion computation on the uploaded private model, and the environmental characteristic information uploaded by other robot terminals, the private model generating terminal and/or other private model generating terminals and the sharing model on the cloud fusion computing sharing platform to generate a new sharing model;
Step S4: and the other robot terminals, the private model generation terminal and/or the other private model generation terminals download the sharing model on the cloud fusion computing sharing platform, and transfer learning is carried out to generate a new private model.
Specifically, the shift learning in this step S4 is further described as follows.
As shown in fig. 4, fig. 4 is a flowchart of a transfer learning method of the robot sharing learning method based on the cloud platform:
Step S41: and the private model generating terminal collects environment information or receives characteristic information of a simulation structure input by other devices as the environment characteristic information, utilizes gazebo software to construct an environment model, runs a reinforcement learning algorithm on the environment model to obtain a navigation strategy, and the private model comprises the navigation strategy and the environment model. And the private model generating terminal uploads the locally generated private model and the collected environmental characteristic information to the cloud fusion computing sharing platform to generate a1 st generation sharing model.
Step S42: the private model generating terminal downloads the sharing model from the cloud fusion computing sharing platform, inputs newly collected environmental characteristic data into the sharing model, outputs the output of the sharing model as additional characteristics into a Q network or transmits all parameters into the Q network, outputs evaluation values of all directions, and adds the evaluation values into the environmental characteristic information to serve as a new environmental model;
Step S43: and performing reinforcement learning in the new environment model, wherein the feature vector of the input layer consists of the original feature vector and the vector output by the sharing model, and outputting a new private model for further uploading to the cloud fusion computing sharing platform to generate a 2 nd generation sharing model. Further, the generation principles of the 3 rd generation sharing model, the 4 th generation sharing model and the n th generation sharing model are basically the same as those of the 2 nd generation sharing model, and will not be described herein.
Further, the fusion calculation in this step S3 is further described below.
As shown in fig. 5, fig. 5 is a flowchart of a fusion calculation method adopted by the cloud platform-based robot sharing learning platform of the present invention. The step S3 may include steps S31, S32, and S33.
Step S31: and the cloud fusion computing sharing platform inputs the environmental characteristic information uploaded by the private model generating terminal into the private model and a stored sharing model on the cloud fusion computing sharing platform to obtain an output result.
Step S32: and carrying out normalization processing on the output result, carrying out self-confidence evaluation on the normalization processing result by using information entropy by the cloud fusion computing and sharing platform, and carrying out weighted summation on the basis of the evaluation score of the self-confidence evaluation by the cloud fusion computing and sharing platform to obtain a label score.
Step S33: and the cloud fusion computing and sharing platform generates environmental characteristic information uploaded by the terminal and the tag score according to the private model, and calculates to obtain a new sharing model.
Compared with the prior art, in the cloud platform-based robot sharing learning and system, the cloud fusion computing sharing platform can accept the private model uploaded by the private model generating terminal, and fusion computing is carried out on the environment characteristic information uploaded by other robot terminals, the private model generating terminal and/or other private model generating terminals and the sharing model on the cloud fusion computing sharing platform to generate a new sharing model; the method and the system enable other robot terminals, the private model generation terminal and/or other private model generation terminals to download the sharing model on the cloud fusion computing sharing platform for local navigation, further transfer learning can be carried out, new private models are generated and uploaded to the cloud fusion computing sharing platform for further downloading and sharing, and the problems that training time is long and experience fusion cannot be carried out in training of a robot navigation decision model can be effectively solved.
As shown in fig. 6, a schematic structural diagram of a computer device 5 for performing the method in the above embodiment is shown. The computer device 5 includes, but is not limited to: at least one memory 51, at least one processor 52, at least one communication device 53, and at least one communication bus. Wherein the communication bus is used to enable connection communication between these components.
The computer device 5 is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and its hardware includes, but is not limited to, a microprocessor, an Application SPECIFIC INTEGRATED Circuit (ASIC), a Programmable gate array (Field-Programmable GATE ARRAY, FPGA), a digital Processor (DIGITAL SIGNAL Processor, DSP), an embedded device, and the like. The computer means 5 may also comprise network devices and/or user devices. Wherein the network device includes, but is not limited to, a single network server, a server group composed of a plurality of network servers, or a Cloud based Cloud Computing (Cloud Computing) composed of a large number of hosts or network servers, wherein Cloud Computing is one of distributed Computing, and is a super virtual computer composed of a group of loosely coupled computer sets.
The computer device 5 may be, but is not limited to, any electronic product that can perform man-machine interaction with a user through a keyboard, a touch pad, or a voice control device, for example, a tablet computer, a smart phone, a Personal digital assistant (Personal DIGITAL ASSISTANT, PDA), a smart wearable device, a camera device, a monitoring device, and other terminals.
The network in which the computer device 5 is located includes, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), and the like.
The communication device 53 may be a wired transmission port or a wireless device, and includes an antenna device for data communication with other devices, for example.
The memory 51 is used for storing program codes. The Memory 51 may be a circuit with a Memory function, such as RAM (Random-Access Memory), FIFO (FIRST IN FIRST Out), etc., which is not in physical form in the integrated circuit. Or the memory may be a physical form of memory, such as a memory bank, TF card (Trans-FLASH CARD), smart media card (SMART MEDIA CARD), secure digital card (secure DIGITAL CARD), flash memory card (FLASH CARD), or the like.
The processor 52 may include one or more microprocessors, digital processors. The processor may call program code stored in the memory to perform the associated function; for example, the modules depicted in fig. 3 are program code stored in a memory and executed by the processor to implement a cloud robot sharing learning method. The processor is also called a central processing Unit (CPU, central Processing Unit), is a very large scale integrated circuit, and is an operation Core (Core) and a Control Unit (Control Unit).
Embodiments of the present invention also provide a computer-readable storage medium having stored thereon computer instructions that, when executed by one or more processors, cause a cloud platform based robot to learn to perform a cloud robot sharing learning method as described in the method embodiments above.
The characteristic means of the present invention described above may be implemented by an integrated circuit and control the function of implementing the cloud robot sharing learning method described in any of the above embodiments.
The functions that can be realized by the cloud robot sharing learning method in any embodiment can be installed in the electronic device through the integrated circuit of the present invention, so that the electronic device can perform the functions that can be realized by the cloud robot sharing learning method in any embodiment, which will not be described in detail herein.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present invention. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present invention.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, such as the division of the units, merely a logical function division, and there may be additional manners of dividing the actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes. The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (13)
1. The utility model provides a shared learning system of robot based on cloud platform, its includes private model generation terminal and cloud fusion calculation sharing platform, its characterized in that: the private model generating terminal is used for uploading the locally generated private model and the environmental characteristic information collected by generating the private model to the cloud fusion computing sharing platform, the cloud fusion computing sharing platform comprises a model fusion computing module, the model fusion computing module is used for carrying out fusion computing on the uploaded private model, combining other robot terminals, the private model generating terminal and/or other private model generating terminals with the environmental characteristic information uploaded by the private model generating terminal and the sharing model stored on the cloud fusion computing sharing platform to generate a new sharing model, and the new sharing model is used for being downloaded and/or learned by other robot terminals, the private model generating terminal and/or other private model generating terminals;
Wherein the model fusion calculation module comprises a model normalization unit and a fusion calculation unit,
The model normalization unit is used for inputting the environmental characteristic information into the private model uploaded by the private model generation terminal and the sharing model obtained by the cloud fusion calculation sharing platform, further normalizing the output result, performing confidence evaluation on the output result by using information entropy, performing weighted summation based on the evaluation score of the confidence evaluation to obtain a label score,
And the fusion calculation unit utilizes the environment characteristic information uploaded by the private model generation terminal and the label score output by the model normalization unit to perform fusion calculation to obtain the new sharing model.
2. The cloud platform based robotic sharing learning system of claim 1, wherein: the private model generating terminal comprises a characteristic collecting module, an environment simulation module and a strengthening and transferring learning module, wherein the characteristic collecting module is used for collecting environment characteristic information, the environment simulation module is used for generating an environment model by utilizing the environment characteristic information, the strengthening and transferring learning module comprises a strengthening learning unit, the strengthening learning unit is used for carrying out strengthening learning after inputting the environment characteristic information on the environment model and outputting the private model, and the private model comprises the environment model and a navigation strategy generated by the private model generating terminal aiming at the environment model.
3. The cloud platform based robotic sharing learning system of claim 2, wherein: the strengthening and transferring learning module further comprises a transferring computing unit, wherein the transferring computing unit is used for performing transferring computation after the environmental characteristic information is input into the downloaded sharing model, outputting new environmental characteristic information for strengthening learning by the strengthening learning unit so as to output the new private model, and the private model generating terminal also uploads the new private model to the cloud fusion computing sharing platform so as to further perform fusion computation.
4. The cloud platform based robotic sharing learning system of claim 1 or 2, wherein: the private model generation terminal adopts an intelligent robot terminal, a computer terminal or other intelligent terminal equipment.
5. The cloud platform based robotic sharing learning system of claim 2, wherein: the characteristic collection module adopts a camera and/or a laser radar to collect the environmental characteristic information or directly receives characteristic information of a simulation structure input by other devices as the environmental characteristic information.
6. The cloud platform based robotic sharing learning system of claim 2, wherein: the environment simulation module adopts gazebo simulation software to construct a simulation environment.
7. The cloud platform based robotic sharing learning system of claim 1, wherein: the private model generation terminal further comprises a first communication module, a first data receiving module and a data uploading module, the cloud fusion calculation sharing platform further comprises a second communication module, the first communication module is used for acquiring the network address of the cloud fusion calculation sharing platform and establishing communication with the second communication module of the cloud fusion calculation sharing platform, the data uploading module is used for uploading the private model to the cloud fusion calculation sharing platform after the first communication module is established in communication with the cloud fusion calculation sharing platform, and the first data receiving module is used for downloading the sharing model on the cloud fusion calculation sharing platform to the private model generation terminal after the communication with the cloud fusion calculation sharing platform is established.
8. The cloud platform based robotic sharing learning system of claim 1, wherein: the cloud fusion computing sharing platform further comprises a second data receiving module, a data downloading module and a model storage module, wherein the second data receiving module is used for receiving the private model uploaded by the private model generating terminal into the cloud fusion computing sharing platform after communication with the private model generating terminal is established, the data downloading module is used for downloading the sharing model on the cloud fusion computing sharing platform to the private model generating terminal after communication with the private model generating terminal is established, and the model storage module is used for storing the sharing model on the cloud fusion computing sharing platform.
9. The robot sharing learning method based on the cloud platform is characterized by comprising the following steps of:
step 1: the private model generating terminal collects environment information and generates a local private model;
Step 2: the private model generating terminal uploads the locally generated private model and the collected environmental characteristic information to the cloud fusion computing sharing platform;
Step 3: the cloud fusion computing sharing platform carries out fusion computation on the uploaded private model, and the environmental characteristic information uploaded by other robot terminals, the private model generating terminal and/or other private model generating terminals and the sharing model on the cloud fusion computing sharing platform to generate a new sharing model;
Step 4: the other robot terminals, the private model generation terminal and/or the other private model generation terminals download the sharing model on the cloud fusion computing sharing platform;
In the step 3, the step of generating a new sharing model by the cloud fusion computing sharing platform includes:
Step 3.1: the cloud fusion computing sharing platform inputs the environmental characteristic information uploaded by the private model generating terminal into the private model and a stored sharing model on the cloud fusion computing sharing platform to obtain an output result, and normalizes the output result;
step 3.2: the cloud fusion computing sharing platform carries out confidence evaluation on the normalization processing result by using information entropy;
Step 3.3: the cloud fusion computing sharing platform performs weighted summation based on the evaluation score of the confidence evaluation to obtain a label score;
step 3.4: and the cloud fusion computing and sharing platform generates environmental characteristic information uploaded by the terminal and the tag score according to the private model, and calculates to obtain a new sharing model.
10. The cloud platform based robot sharing learning method of claim 9, wherein: in the step 1, the step of generating the local private model by the private model generating terminal is as follows:
Step 1.1: the private model generating terminal collects environment information or receives characteristic information of simulation structures input by other devices as the environment characteristic information;
Step 1.2: the private model generating terminal builds an environment model by gazebo software based on the environment characteristic information;
Step1.3: and running a reinforcement learning algorithm on the environment model to obtain a navigation strategy, wherein the private model comprises the navigation strategy and the environment model.
11. The cloud platform based robot sharing learning method of claim 9, wherein: in step 2, the step that the private model generating terminal uploads the locally generated private model and the collected environmental characteristic information to the cloud fusion computing sharing platform is as follows:
Step 2.1: the private model generating terminal acquires the communication address of the cloud fusion computing sharing platform and sends a request;
step 2.2: and the private model generating terminal uploads the private model and the environmental characteristic information to the cloud fusion computing sharing platform after receiving the information of the cloud fusion computing sharing platform.
12. The cloud platform based robot sharing learning method of claim 9, wherein: in the step 4, the other robot terminals, the private model generating terminal and/or the other private model generating terminal download the sharing model on the cloud fusion computing sharing platform and then perform migration learning to generate a new private model, wherein the step of performing migration learning by the private model generating terminal is as follows:
Step 4.1: the private model generating terminal downloads the sharing model from the cloud fusion computing sharing platform;
Step 4.2: inputting the newly collected environmental characteristic data into the sharing model, and outputting evaluation values of all directions;
Step 4.3: adding the evaluation value to the environmental characteristic information to serve as a new environmental model;
step 4.4: and performing reinforcement learning in the new environment model, and generating a new private model for further uploading to the cloud fusion computing sharing platform.
13. A computer readable storage medium storing at least one instruction which when executed by a processor performs the method of any one of claims 9-12.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910248301.9A CN109993308B (en) | 2019-03-29 | 2019-03-29 | Cloud platform-based shared learning system and method, shared platform and method and medium |
| PCT/CN2019/130567 WO2020199690A1 (en) | 2019-03-29 | 2019-12-31 | Cloud platform-based sharing learning system and method, sharing platform and method, and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910248301.9A CN109993308B (en) | 2019-03-29 | 2019-03-29 | Cloud platform-based shared learning system and method, shared platform and method and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109993308A CN109993308A (en) | 2019-07-09 |
| CN109993308B true CN109993308B (en) | 2024-05-24 |
Family
ID=67131830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910248301.9A Active CN109993308B (en) | 2019-03-29 | 2019-03-29 | Cloud platform-based shared learning system and method, shared platform and method and medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109993308B (en) |
| WO (1) | WO2020199690A1 (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109993308B (en) * | 2019-03-29 | 2024-05-24 | 深圳先进技术研究院 | Cloud platform-based shared learning system and method, shared platform and method and medium |
| CN110827167A (en) * | 2019-09-29 | 2020-02-21 | 武汉开目信息技术股份有限公司 | Product design manufacturability knowledge sharing method and device for collaborative manufacturing |
| CN110766169A (en) * | 2019-10-31 | 2020-02-07 | 深圳前海微众银行股份有限公司 | Transfer training optimization method and device for reinforcement learning, terminal and storage medium |
| CN110796267A (en) * | 2019-11-12 | 2020-02-14 | 支付宝(杭州)信息技术有限公司 | Machine learning method and machine learning device for data sharing |
| CN111027713B (en) * | 2019-12-10 | 2022-09-02 | 支付宝(杭州)信息技术有限公司 | Shared machine learning system and method |
| CN111126609B (en) * | 2019-12-20 | 2021-04-23 | 深圳前海微众银行股份有限公司 | Knowledge migration method, device, equipment and medium based on federal learning |
| CN112100145B (en) * | 2020-09-02 | 2023-07-04 | 南京三眼精灵信息技术有限公司 | Digital model sharing learning system and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103926838A (en) * | 2014-04-22 | 2014-07-16 | 山东大学 | Autonomous metal development cloud robot system based on cloud computing |
| CN106027300A (en) * | 2016-05-23 | 2016-10-12 | 深圳市飞仙智能科技有限公司 | System and method for parameter optimization of intelligent robot applying neural network |
| CN108398660A (en) * | 2018-01-08 | 2018-08-14 | 国网江苏省电力有限公司 | A kind of terminal device localization method and system based on Wi-Fi cloud platform systems |
| CN108896739A (en) * | 2018-07-16 | 2018-11-27 | 河南聚合科技有限公司 | It is a kind of can assaying ingredient solar-energy machine people's cloud platform |
| CN109116854A (en) * | 2018-09-16 | 2019-01-01 | 南京大学 | A kind of robot cooperated control method of multiple groups based on intensified learning and control system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10754328B2 (en) * | 2014-09-05 | 2020-08-25 | Accenture Global Solutions Limited | Self-adaptive device intelligence as a service enterprise infrastructure for sensor-rich environments |
| US9713871B2 (en) * | 2015-04-27 | 2017-07-25 | Microsoft Technology Licensing, Llc | Enhanced configuration and control of robots |
| CN107053184A (en) * | 2017-06-22 | 2017-08-18 | 北京信息科技大学 | Multi-Agent Cooperation processing system and method based on private clound |
| CN108921298B (en) * | 2018-06-12 | 2022-04-19 | 中国科学技术大学 | Multi-agent communication and decision-making method for reinforcement learning |
| CN109086550B (en) * | 2018-08-27 | 2019-05-28 | 山东师范大学 | The evacuation emulation method and system of Q study are shared based on multi-Agent |
| CN109993308B (en) * | 2019-03-29 | 2024-05-24 | 深圳先进技术研究院 | Cloud platform-based shared learning system and method, shared platform and method and medium |
-
2019
- 2019-03-29 CN CN201910248301.9A patent/CN109993308B/en active Active
- 2019-12-31 WO PCT/CN2019/130567 patent/WO2020199690A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103926838A (en) * | 2014-04-22 | 2014-07-16 | 山东大学 | Autonomous metal development cloud robot system based on cloud computing |
| CN106027300A (en) * | 2016-05-23 | 2016-10-12 | 深圳市飞仙智能科技有限公司 | System and method for parameter optimization of intelligent robot applying neural network |
| CN108398660A (en) * | 2018-01-08 | 2018-08-14 | 国网江苏省电力有限公司 | A kind of terminal device localization method and system based on Wi-Fi cloud platform systems |
| CN108896739A (en) * | 2018-07-16 | 2018-11-27 | 河南聚合科技有限公司 | It is a kind of can assaying ingredient solar-energy machine people's cloud platform |
| CN109116854A (en) * | 2018-09-16 | 2019-01-01 | 南京大学 | A kind of robot cooperated control method of multiple groups based on intensified learning and control system |
Non-Patent Citations (3)
| Title |
|---|
| "A navigation model for a multi-robot system Based on Client/Server model";Tlijani Hayet等;2016 International Conference on Control, Decision and Information Technologies (CoDIT);全文 * |
| "Client/Server Framework for Providing Context-Aware Services to Network Based Robots";Hyun Kim等;RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication;全文 * |
| 洪臣等."云机器人架构和特征概述".《机器人技术与应用》.2018,(第6期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020199690A1 (en) | 2020-10-08 |
| CN109993308A (en) | 2019-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109993308B (en) | Cloud platform-based shared learning system and method, shared platform and method and medium | |
| WO2022022274A1 (en) | Model training method and apparatus | |
| CN113408743B (en) | Method and device for generating federal model, electronic equipment and storage medium | |
| CN110163368B (en) | Deep learning model training method, device and system based on mixed precision | |
| WO2022068623A1 (en) | Model training method and related device | |
| US20180018555A1 (en) | System and method for building artificial neural network architectures | |
| CN111797992A (en) | Machine learning optimization method and device | |
| WO2023246819A1 (en) | Model training method and related device | |
| CN114997412A (en) | Recommendation method, training method and device | |
| WO2023231961A1 (en) | Multi-agent reinforcement learning method and related device | |
| CN116502692A (en) | Model training method, photovoltaic power generation power prediction method and device | |
| CN111126590A (en) | Artificial neural network operation device and method | |
| CN115238909A (en) | Data value evaluation method based on federal learning and related equipment thereof | |
| CN111966361A (en) | Method, device and equipment for determining model to be deployed and storage medium thereof | |
| WO2021036397A1 (en) | Method and apparatus for generating target neural network model | |
| CN116684330A (en) | Traffic prediction method, device, equipment and storage medium based on artificial intelligence | |
| CN110490317B (en) | Neural network operation device and operation method | |
| CN111160049B (en) | Text translation method, apparatus, machine translation system, and storage medium | |
| CN112765481B (en) | Data processing method, device, computer and readable storage medium | |
| CN113627421A (en) | Image processing method, model training method and related equipment | |
| WO2023197857A1 (en) | Model partitioning method and related device thereof | |
| CN116739154A (en) | Fault prediction method and related equipment thereof | |
| CN116827685A (en) | Dynamic defense strategy method of micro-service system based on deep reinforcement learning | |
| EP4401000A1 (en) | Model training method and related device | |
| CN115565104A (en) | Action prediction method and related equipment thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |