US20210347047A1 - Generating robot trajectories using neural networks - Google Patents
Generating robot trajectories using neural networks Download PDFInfo
- Publication number
- US20210347047A1 US20210347047A1 US16/867,437 US202016867437A US2021347047A1 US 20210347047 A1 US20210347047 A1 US 20210347047A1 US 202016867437 A US202016867437 A US 202016867437A US 2021347047 A1 US2021347047 A1 US 2021347047A1
- Authority
- US
- United States
- Prior art keywords
- trajectory
- network
- current
- output
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1628—Programme controls characterised by the control loop
- B25J9/163—Programme controls characterised by the control loop learning, adaptive, model based, rule based expert control
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
- G05B13/027—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion using neural networks only
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G06N3/0445—
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/33—Director till display
- G05B2219/33025—Recurrent artificial neural network
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/39—Robotics, robotics to robotics hand
- G05B2219/39298—Trajectory learning
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40449—Continuous, smooth robot motion
Definitions
- This specification relates to generating robot trajectories using neural networks.
- Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input.
- Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer.
- Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters.
- a recurrent neural network is a neural network that receives an input sequence and generates an output sequence from the input sequence.
- a recurrent neural network can use some or all of the internal state of the network from a previous time step in computing an output at a current time step.
- a recurrent neural network is a Long Short-Term Memory (LSTM) neural network that includes one or more LSTM memory blocks.
- Each LSTM memory block can include one or more cells that each include an input gate, a forget gate, and an output gate that allow the cell to store previous states for the cell, e.g., for use in generating a current activation or to be provided to other components of the LSTM neural network.
- Robot trajectory planning refers to generating plans for controlling a movement of a robot from an initial pose to a desired final pose, including traversing a plurality of intermediate poses.
- generating robot trajectories typically involves generating a plurality of trajectory points that each correspond to a desired robot pose at a particular time step.
- the neural network system can receive a system input that includes data specifying a robot path and process the system input to generate a system output that specifies a robot trajectory.
- the robot trajectory is typically parameterized by time and defines how a robot can travel through the robot path specified by the system input.
- the neural network system can be efficiently adapted to emulate any desired trajectory behavior.
- the neural network system thus can generate high quality trajectories, e.g., trajectories with desired temporal or spatial precisions, for various types of robots and from different input robot paths. Trajectories generated by the neural network system are generally more stable, e.g., when compared with trajectories generated by closed trajectory generators such as a robot controller simulation (RCS) model which might generate different trajectories for substantially the same input paths.
- RCS robot controller simulation
- the neural network system is more flexible, thus being suitable for deployment in many robotic development pipelines involving a range of hardware or software platforms. Generating trajectories using the neural network system is thus more resource-efficient, because doing so can save the substantial amount of computational resources, wall-clock time, or both that is otherwise required for data communication between two or more different systems (e.g., a robotic development system and a server system hosting the closed trajectory generator) that are typically involved in planning robot trajectories. As such, the neural network system also facilitates rapid robotic cell planning by generating hundreds or thousands of alternative trajectories more quickly than other conventional approaches, including using the closed trajectory generator.
- FIG. 1 shows an example trajectory generation system in relation to an example closed trajectory generator.
- FIG. 2 is a flow diagram of an example process for generating robot trajectories.
- FIG. 3A is an illustration of example network inputs and outputs.
- FIG. 3B is an illustration of example adjustments to network outputs.
- FIG. 1 shows an example trajectory prediction system 100 in relation to an example closed trajectory generator 140 .
- the trajectory prediction system 100 is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below can be implemented.
- the closed trajectory generator 140 is a software module or system that generates a trajectory from an input path.
- closed trajectory generator is a trajectory generator whose behavior the trajectory prediction system 100 is attempting to emulate as closely as possible using machine learning techniques.
- the closed trajectory generator 140 can be closed in the sense that the entity operating the trajectory prediction system 100 does not have access to the source code or other documentation explaining how the trajectories are generated by the closed trajectory generator 140 .
- any other appropriate trajectory generator that is or is not open to source code inspection can also be considered a “closed trajectory generator” when the trajectory prediction system 100 is trained to emulate its behavior.
- the closed trajectory generator 140 can include a trajectory planner, e.g., a robot controller simulation (RCS) model or a B-Spline model.
- a trajectory planner e.g., a robot controller simulation (RCS) model or a B-Spline model.
- the RCS model can implement software that is configured to receive data specifying a given robot path 102 and generate one or more corresponding robot trajectories 142 (which are also referred to in this documentation as “actual trajectories”) defining how the robot should travel through the robot path 102 .
- the closed trajectory generator 140 is used to generate the actual trajectory 142 to be executed by a robot at run-time.
- the closed trajectory generator 140 may prove to be problematic for a number of reasons.
- the closed trajectory generator 140 may be far too slow in terms of wall clock time and generate results that are unstable or nondeterministic.
- the closed trajectory generator 140 typically operates in a form of black box, hindering interpolations or adjustments from being applied to the trajectory planning process.
- the path planning process can be greatly sped up by using the trajectory prediction system 100 instead of the closed trajectory generator 140 .
- the trajectory prediction system 100 can be massively parallelized to generate trajectories for thousands or millions of candidate paths.
- the trajectory prediction system 100 is a machine learning system that receives a system input specifying a robot path 102 and generates, from the robot path 102 , a system output specifying a predicted robot trajectory 132 . Referring to the trajectories generated by the system 100 as predicted trajectories indicates that the system 100 is specifically configured to generate predicted trajectories that imitate the actual trajectories generated by the closed trajectory generator 140 .
- the system input includes data specifying a sequence of path points that each correspond to a particular pose of a robot, i.e., with reference to a predetermined coordinate frame.
- the path points can be defined, for example, in robot configuration space (i.e., joint space) or task space (i.e., Cartesian space).
- the sequence of path points defines a geometric path for moving a robot from an initial pose to a desired final pose.
- the trajectory prediction system 100 can then determine, from the geometric path defined by the system input, the system output that includes a sequence of trajectory points.
- the sequence of trajectory points which are usually time-parameterized, define how the robot can travel through the geometric path.
- the system 100 can process the system input to generate the system output specifying what pose the robot should be in at each of a plurality of time steps.
- a pose of the robot refers to an orientation, a position, or both of the robot with reference to the predetermined coordinate frame.
- poses can generally be defined using multi-dimensional structured data. The exact dimension of the structured data representing a pose is generally dependent on degrees of freedom (DoF) of the robot. For example, if the robot is a fixed-base robot with six revolute joints, then a particular pose of the robot can be defined using a 6-dimensional vector, with each element of the vector representing a respective joint angle, e.g., measured in radians.
- DoF degrees of freedom
- the trajectory prediction system 100 includes a trajectory generation neural network 120 and, in some implementations, a trajectory adjustment engine 130 .
- the trajectory generation neural network 120 may be a feedforward neural network or a recurrent neural network that is configured to receive a sequence of inputs 112 that each include information that is specified by or derived from the system input, and process the inputs 112 in accordance with current parameter values of the network 120 to generate, over multiple time steps, a sequence of network outputs 122 defining an initial predicted robot trajectory 132 , which is also referred to in this document as a “forward trajectory”.
- the trajectory prediction system 100 generates a current input 112 for the network 120 based on (i) the system input that specifies a robot path 102 , (ii) previous inputs in the sequence of inputs 112 , (iii) previous outputs generated by the network 120 , or a combination of (i)-(iii). Generating the sequence of inputs 112 will be described in more detail below with reference to FIG. 2 and FIG. 3A .
- Example recurrent neural networks include long-short term memory (LSTM) networks or gated recurrent unit (GRU) networks. That is, in some cases, the trajectory generation neural network 120 may be a recurrent neural network that includes one or more long-short term memory (LSTM) layers or gated recurrent unit (GRU) layers. Each layer in turn includes one or more memory cells. For example, each LSTM layer can include one or more memory cells that each include an input gate, a forget gate, and an output gate that allow the cell to store previous states for the cell, e.g., for use in generating a current activation or to be provided to other components of the LSTM neural network.
- LSTM long-short term memory
- GRU gated recurrent unit
- the trajectory generation neural network 120 To generate the sequence of network outputs 122 that define a forward trajectory of the robot, at each of the multiple time steps, the trajectory generation neural network 120 generally receives as input (i) a current input 102 for the current time step and (ii) a preceding network output 122 that was generated by the network at the preceding time step, and generates a current output 122 for the current time step.
- the trajectory generation neural network 120 refers to a fully-learned neural network.
- a neural network is said to be “fully-learned” if the neural network has been trained to compute a desired prediction.
- a fully-learned neural network generates an output based solely on being trained on training data rather than on human-programmed decisions.
- the training data for use in training the network 120 can be derived from the actual trajectories that are generated by the closed trajectory generator 140 for multiple given robot paths.
- the given robot path can be any path for which corresponding robot trajectories need to be determined.
- the discrete trajectory points to be used in computing the target output that is associated with each training input can then be obtained by sampling the actual robot trajectories generated by the closed trajectory generator 140 at a fixed frequency, e.g., 10 Hz, 20 Hz, or 30 Hz.
- a training engine can iteratively adjust current parameter values of the network 120 by optimizing an objective function that measures a difference between network outputs and target outputs that are derived from actual trajectories generated by closed trajectory generator 140 , e.g., based on a computed gradient of the objective function and using a gradient descent optimization technique, e.g., an RMSprop or Adam technique.
- an objective function that measures a difference between network outputs and target outputs that are derived from actual trajectories generated by closed trajectory generator 140 , e.g., based on a computed gradient of the objective function and using a gradient descent optimization technique, e.g., an RMSprop or Adam technique.
- the trajectory adjustment engine 130 when included, can then receive the network outputs 122 which collectively define the forward trajectory and generate an adjusted predicted trajectory 132 from the network outputs 122 .
- the adjusted predicted robot trajectory 132 generated by the trajectory adjustment engine 130 is also referred to in this document as a “backward trajectory”.
- the trajectory adjustment engine 130 determines whether to apply an adjustment to the forward trajectory point defined by the network output. The trajectory adjustment engine 130 then determines, from the adjustments to the forward trajectory generated by the neural network 120 for one or more of the sequence of inputs 102 , the backward trajectory for the input path 102 . Determining adjustments to the network outputs 122 will be described in more detail below with reference to FIG. 2 and FIG. 3B .
- FIG. 2 is a flow diagram of an example process 200 for generating robot trajectories.
- the process 300 will be described as being performed by a system of one or more computers located in one or more locations.
- a trajectory generation system e.g., the trajectory generation system 100 of FIG. 1 , appropriately programmed in accordance with this specification, can perform the process 200 .
- the system receives a plurality of path points ( 202 ).
- the plurality of path points can define a robot path for which one or more corresponding trajectories need to be determined.
- the system processes each network input in an input sequence that is derived from the path points using a trajectory generation neural network to generate an output sequence that includes a plurality of network outputs ( 204 ).
- the trajectory generation neural network is configured to auto-regressively generate data specifying robot trajectories over multiple time steps, at each time step the system can instantaneously, i.e., in real-time, generate a current network input for the network based on (i) a received system input that specifies a sequence of path points that collectively define a robot path for which a trajectory needs to be determined, (ii) previous network inputs in the input sequence, (iii) previous network outputs generated by the network, or a combination of one or more of (i)-(iii).
- FIG. 3A is an illustration of example network inputs and outputs.
- a network input specifies a current trajectory point q t 302 , a current reference direction d t 304 for the current trajectory point q t 302 , a future reference direction d′ t 306 for the current trajectory point q t 302 , and “goal” vector g t 308 for the current trajectory point q t 302 .
- the current trajectory point q t is the starting trajectory point from which the system predicts a subsequent movement of a robot.
- the system generally determines the current trajectory point q t from a preceding network output o t ⁇ 1 and a preceding trajectory point q t ⁇ 1 .
- the system instead uses the first path point in the sequence of path points specified by the system input as the current trajectory point.
- the system can obtain the current reference direction
- the system can keep a record of respective distances between the generated trajectory points and the current path point. The system can then proceed to use a subsequent path point in the input sequence as the current path point when the distance begins to increase.
- the system can obtain the future reference direction
- d t ′ p k ⁇ ( qt ) + 1 - p k ( qt ) ⁇ p k ⁇ ( qt ) + 1 - p k ( qt ) ⁇
- the system can obtain the “goal” vector g t 308 based computing a displacement from the current trajectory point q t 302 to the current path point p k(q t ) 314 of the current trajectory point q t 302 .
- Each network output in turn specifies a respective displacement between a current trajectory point and a subsequent trajectory point.
- the system generates the plurality of network outputs over multiple time steps.
- the system provides the trajectory generation neural network with (i) a current network input and (ii) a preceding network output and uses the network to generate a current network output that specifies a displacement between a current trajectory point and a subsequent trajectory point.
- the system can instead provide the network with the current network input and a predetermined placeholder input, i.e., in place of the preceding network output.
- the trajectory generation neural network then processes the current input and the predetermined placeholder input to generate the current network output for the first time step.
- the system uses the trajectory generation neural network to generate a current network output o t 332 which defines a displacement from the current trajectory point q t 302 to the subsequent trajectory point q t+1 352 .
- the system predicts q t+1 352 to be the next trajectory point when generating the robot trajectory from the robot path.
- the system generates a predicted trajectory of the robot ( 206 ) that is derived from the output sequence. For example, because each network output specifies a respective displacement between two adjacent trajectory points, the system can generate the predicted trajectory by computing a concatenation the respective displacements specified by the output sequence.
- the predicted trajectory in this way is also referred to as a forward trajectory of the robot.
- the system can also generate a backward trajectory from the forward trajectory by determining adjustments to one or more of the network outputs included in the sequence.
- the system iteratively determines whether the displacement o t that is specified by the network output is parallel to the current reference direction d t of the current trajectory point q t as specified by the corresponding network input.
- the system determines an adjustment to the displacement based on two adjacent path points of the current trajectory point. In general, the system determines such adjustment to require that, when the displacement of the current trajectory point is parallel to its current reference direction, a robot should travel in a line connecting the preceding path point and the current path point.
- FIG. 3B is an illustration of example adjustments to network outputs.
- the system determines that the displacement o t 384 of the current trajectory point q t 382 is parallel to its current reference direction d t . Accordingly, the system can apply an adjustment to move the displacement to o t * 386 by projecting the displacement o t 384 to a line connecting two adjacent path points of the current trajectory point, i.e., the line connecting the preceding path point p k(q t ) ⁇ 1 of the current trajectory point q t 382 and the current path point p k(q t ) of the current trajectory point q t 382 .
- the system follows a backward iteration process to iteratively determine adjustments to respective displacements specified by preceding network outputs in the output sequence.
- the system in response to a negative determination, e.g., upon determining that the displacement that is specified by the network output is not parallel to the reference direction of the current trajectory point, the system generally moves onto a preceding network output in the output sequence without specifically applying any adjustments to the trajectory point.
- the system can generate the backward trajectory from the adjustments being applied to the output sequence that is generated by the trajectory generation neural network.
- the system can use the backward trajectory instead of or in addition to the forward trajectory for use in planning a movement of the robot to travel through the robot path that is defined by the system input.
- the system can also generate a “smoothed trajectory” by computing a weighted average of the forward trajectory and the backward trajectory.
- the smoothed trajectory when generated, will then be similarly used in planning the movement of the robot. Examples of forward, backward, and smoothed trajectories are shown in FIG. 3B .
- Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.
- Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus.
- the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.
- the computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
- data processing apparatus encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers.
- the apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
- the apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
- a computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
- a computer program may, but need not, correspond to a file in a file system.
- a program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code.
- a computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
- the processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output.
- the processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
- special purpose logic circuitry e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
- Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit.
- a central processing unit will receive instructions and data from a read only memory or a random access memory or both.
- the essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data.
- a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
- mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
- a computer need not have such devices.
- a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.
- PDA personal digital assistant
- GPS Global Positioning System
- USB universal serial bus
- Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks.
- semiconductor memory devices e.g., EPROM, EEPROM, and flash memory devices
- magnetic disks e.g., internal hard disks or removable disks
- magneto optical disks e.g., CD ROM and DVD-ROM disks.
- the processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
- a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer.
- a display device e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor
- keyboard and a pointing device e.g., a mouse or a trackball
- Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
- a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a
- Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components.
- the components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.
- LAN local area network
- WAN wide area network
- the computing system can include clients and servers.
- a client and server are generally remote from each other and typically interact through a communication network.
- the relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
- Embodiment 1 is a method comprising:
- Embodiment 2 is the method of embodiment 1, wherein the predicted trajectory of the robot represents a prediction for an output trajectory of a closed trajectory generator when given the path points.
- Embodiment 3 is the method of any one of embodiments 1-2, wherein each network input specifies (i) a position of a current trajectory point, (ii) a current reference direction of the current trajectory point, (iii) a future reference direction of the current trajectory point, and (iv) a goal vector measuring a displacement between the current trajectory point and a current path point.
- Embodiment 4 is the method of any one of embodiments 1-3, further comprising generating an adjusted predicted trajectory from the predicted trajectory, comprising, for each network output in the output sequence:
- Embodiment 5 is the method of any one of embodiments 1-4, wherein:
- the trajectory generation neural network is a recurrent neural network
- generating the output sequence comprising the plurality of network outputs comprises, at each of a plurality of time steps: processing, using the trajectory generation neural network, a current network input and a preceding network output to generate a current network output.
- Embodiment 6 is the method of any one of embodiments 4-5, wherein determining the adjustment to the displacement comprises: projecting the displacement to a line connecting two adjacent path points of the current trajectory point.
- Embodiment 7 is the method of any one of embodiments 4-6, wherein determining the adjustment to the displacement further comprises: iteratively determining adjustments to respective displacements specified by preceding network outputs in the output sequence.
- Embodiment 8 is the method of any one of embodiments 4-7, further comprising generating a smoothened predicted trajectory by computing a weighted average of the predicted trajectory and the adjusted predicted trajectory.
- Embodiment 9 is the method of any one of embodiments 1-8, wherein each trajectory point or path point is represented by multi-dimensional data having a respective dimension that is dependent on degrees of freedom (DoF) of the robot.
- DoF degrees of freedom
- Embodiment 10 is the method of any one of embodiments 1-9, further comprising training the trajectory generation neural network by optimizing an objective function measuring a difference between network outputs and target outputs that are derived from trajectories generated by Robot Controller Simulation (RCS).
- RCS Robot Controller Simulation
- Embodiment 11 is a system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform the method of any one of embodiments 1 to 10.
- Embodiment 12 is a computer storage medium encoded with a computer program, the program comprising instructions that are operable, when executed by data processing apparatus, to cause the data processing apparatus to perform the method of any one of embodiments 1 to 10.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Mechanical Engineering (AREA)
- Robotics (AREA)
- Theoretical Computer Science (AREA)
- Automation & Control Theory (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Fuzzy Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Manipulator (AREA)
- Feedback Control In General (AREA)
Abstract
Description
- This specification relates to generating robot trajectories using neural networks.
- Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters.
- Some neural networks are recurrent neural networks. A recurrent neural network is a neural network that receives an input sequence and generates an output sequence from the input sequence. In particular, a recurrent neural network can use some or all of the internal state of the network from a previous time step in computing an output at a current time step.
- An example of a recurrent neural network is a Long Short-Term Memory (LSTM) neural network that includes one or more LSTM memory blocks. Each LSTM memory block can include one or more cells that each include an input gate, a forget gate, and an output gate that allow the cell to store previous states for the cell, e.g., for use in generating a current activation or to be provided to other components of the LSTM neural network.
- Robot trajectory planning refers to generating plans for controlling a movement of a robot from an initial pose to a desired final pose, including traversing a plurality of intermediate poses. As such, generating robot trajectories typically involves generating a plurality of trajectory points that each correspond to a desired robot pose at a particular time step.
- This specification describes how a system implemented as computer programs on one or more computers in one or more locations can generate robot trajectories using a neural network system. The neural network system can receive a system input that includes data specifying a robot path and process the system input to generate a system output that specifies a robot trajectory. The robot trajectory is typically parameterized by time and defines how a robot can travel through the robot path specified by the system input.
- Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.
- Because of the adaptive nature of neural networks, the neural network system can be efficiently adapted to emulate any desired trajectory behavior. The neural network system thus can generate high quality trajectories, e.g., trajectories with desired temporal or spatial precisions, for various types of robots and from different input robot paths. Trajectories generated by the neural network system are generally more stable, e.g., when compared with trajectories generated by closed trajectory generators such as a robot controller simulation (RCS) model which might generate different trajectories for substantially the same input paths.
- In addition, unlike closed trajectory generators which typically operate in form of a black box on very few dedicated platforms, the neural network system is more flexible, thus being suitable for deployment in many robotic development pipelines involving a range of hardware or software platforms. Generating trajectories using the neural network system is thus more resource-efficient, because doing so can save the substantial amount of computational resources, wall-clock time, or both that is otherwise required for data communication between two or more different systems (e.g., a robotic development system and a server system hosting the closed trajectory generator) that are typically involved in planning robot trajectories. As such, the neural network system also facilitates rapid robotic cell planning by generating hundreds or thousands of alternative trajectories more quickly than other conventional approaches, including using the closed trajectory generator.
- The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.
-
FIG. 1 shows an example trajectory generation system in relation to an example closed trajectory generator. -
FIG. 2 is a flow diagram of an example process for generating robot trajectories. -
FIG. 3A is an illustration of example network inputs and outputs. -
FIG. 3B is an illustration of example adjustments to network outputs. - Like reference numbers and designations in the various drawings indicate like elements.
-
FIG. 1 shows an example trajectory prediction system 100 in relation to an example closedtrajectory generator 140. The trajectory prediction system 100 is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below can be implemented. - The closed
trajectory generator 140 is a software module or system that generates a trajectory from an input path. In this specification, closed trajectory generator is a trajectory generator whose behavior the trajectory prediction system 100 is attempting to emulate as closely as possible using machine learning techniques. In practice, the closedtrajectory generator 140 can be closed in the sense that the entity operating the trajectory prediction system 100 does not have access to the source code or other documentation explaining how the trajectories are generated by the closedtrajectory generator 140. However, any other appropriate trajectory generator that is or is not open to source code inspection can also be considered a “closed trajectory generator” when the trajectory prediction system 100 is trained to emulate its behavior. - The closed
trajectory generator 140 can include a trajectory planner, e.g., a robot controller simulation (RCS) model or a B-Spline model. As one example, the RCS model can implement software that is configured to receive data specifying a givenrobot path 102 and generate one or more corresponding robot trajectories 142 (which are also referred to in this documentation as “actual trajectories”) defining how the robot should travel through therobot path 102. - In a typical situation, the closed
trajectory generator 140 is used to generate theactual trajectory 142 to be executed by a robot at run-time. However, at path planning time, the closedtrajectory generator 140 may prove to be problematic for a number of reasons. For example, the closedtrajectory generator 140 may be far too slow in terms of wall clock time and generate results that are unstable or nondeterministic. In addition, it may not be possible in practice to parallelize the closedtrajectory generator 140 to generate multiple candidate trajectories in parallel at path planning time. This can be because of software license issues or technical limitations. Thus, the closedtrajectory generator 140 typically operates in a form of black box, hindering interpolations or adjustments from being applied to the trajectory planning process. - The path planning process can be greatly sped up by using the trajectory prediction system 100 instead of the closed
trajectory generator 140. Unlike the closedtrajectory generator 140, the trajectory prediction system 100 can be massively parallelized to generate trajectories for thousands or millions of candidate paths. - The trajectory prediction system 100 is a machine learning system that receives a system input specifying a
robot path 102 and generates, from therobot path 102, a system output specifying a predictedrobot trajectory 132. Referring to the trajectories generated by the system 100 as predicted trajectories indicates that the system 100 is specifically configured to generate predicted trajectories that imitate the actual trajectories generated by the closedtrajectory generator 140. - For example, the system input includes data specifying a sequence of path points that each correspond to a particular pose of a robot, i.e., with reference to a predetermined coordinate frame. The path points can be defined, for example, in robot configuration space (i.e., joint space) or task space (i.e., Cartesian space). Collectively, the sequence of path points defines a geometric path for moving a robot from an initial pose to a desired final pose. The trajectory prediction system 100 can then determine, from the geometric path defined by the system input, the system output that includes a sequence of trajectory points. Collectively, the sequence of trajectory points, which are usually time-parameterized, define how the robot can travel through the geometric path. In other words, the system 100 can process the system input to generate the system output specifying what pose the robot should be in at each of a plurality of time steps.
- A pose of the robot refers to an orientation, a position, or both of the robot with reference to the predetermined coordinate frame. In addition, poses can generally be defined using multi-dimensional structured data. The exact dimension of the structured data representing a pose is generally dependent on degrees of freedom (DoF) of the robot. For example, if the robot is a fixed-base robot with six revolute joints, then a particular pose of the robot can be defined using a 6-dimensional vector, with each element of the vector representing a respective joint angle, e.g., measured in radians.
- In particular, the trajectory prediction system 100 includes a trajectory generation
neural network 120 and, in some implementations, a trajectory adjustment engine 130. The trajectory generationneural network 120 may be a feedforward neural network or a recurrent neural network that is configured to receive a sequence ofinputs 112 that each include information that is specified by or derived from the system input, and process theinputs 112 in accordance with current parameter values of thenetwork 120 to generate, over multiple time steps, a sequence ofnetwork outputs 122 defining an initial predictedrobot trajectory 132, which is also referred to in this document as a “forward trajectory”. - Briefly, at each of the multiple time steps, the trajectory prediction system 100 generates a
current input 112 for thenetwork 120 based on (i) the system input that specifies arobot path 102, (ii) previous inputs in the sequence ofinputs 112, (iii) previous outputs generated by thenetwork 120, or a combination of (i)-(iii). Generating the sequence ofinputs 112 will be described in more detail below with reference toFIG. 2 andFIG. 3A . - Example recurrent neural networks include long-short term memory (LSTM) networks or gated recurrent unit (GRU) networks. That is, in some cases, the trajectory generation
neural network 120 may be a recurrent neural network that includes one or more long-short term memory (LSTM) layers or gated recurrent unit (GRU) layers. Each layer in turn includes one or more memory cells. For example, each LSTM layer can include one or more memory cells that each include an input gate, a forget gate, and an output gate that allow the cell to store previous states for the cell, e.g., for use in generating a current activation or to be provided to other components of the LSTM neural network. - To generate the sequence of
network outputs 122 that define a forward trajectory of the robot, at each of the multiple time steps, the trajectory generationneural network 120 generally receives as input (i) acurrent input 102 for the current time step and (ii) a precedingnetwork output 122 that was generated by the network at the preceding time step, and generates acurrent output 122 for the current time step. - For convenience, the trajectory generation
neural network 120 as used in throughout this document refers to a fully-learned neural network. A neural network is said to be “fully-learned” if the neural network has been trained to compute a desired prediction. In other words, a fully-learned neural network generates an output based solely on being trained on training data rather than on human-programmed decisions. - In some cases, the training data for use in training the
network 120 can be derived from the actual trajectories that are generated by theclosed trajectory generator 140 for multiple given robot paths. The given robot path can be any path for which corresponding robot trajectories need to be determined. The discrete trajectory points to be used in computing the target output that is associated with each training input can then be obtained by sampling the actual robot trajectories generated by theclosed trajectory generator 140 at a fixed frequency, e.g., 10 Hz, 20 Hz, or 30 Hz. To obtain the fully-learned trajectory generationneural network 120, a training engine (not shown in the figure) can iteratively adjust current parameter values of thenetwork 120 by optimizing an objective function that measures a difference between network outputs and target outputs that are derived from actual trajectories generated byclosed trajectory generator 140, e.g., based on a computed gradient of the objective function and using a gradient descent optimization technique, e.g., an RMSprop or Adam technique. - The trajectory adjustment engine 130, when included, can then receive the network outputs 122 which collectively define the forward trajectory and generate an adjusted predicted
trajectory 132 from the network outputs 122. The adjusted predictedrobot trajectory 132 generated by the trajectory adjustment engine 130 is also referred to in this document as a “backward trajectory”. - Briefly, from each
network output 122 generated by the trajectory generationneural network 120, the trajectory adjustment engine 130 determines whether to apply an adjustment to the forward trajectory point defined by the network output. The trajectory adjustment engine 130 then determines, from the adjustments to the forward trajectory generated by theneural network 120 for one or more of the sequence ofinputs 102, the backward trajectory for theinput path 102. Determining adjustments to the network outputs 122 will be described in more detail below with reference toFIG. 2 andFIG. 3B . -
FIG. 2 is a flow diagram of anexample process 200 for generating robot trajectories. For convenience, the process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a trajectory generation system, e.g., the trajectory generation system 100 ofFIG. 1 , appropriately programmed in accordance with this specification, can perform theprocess 200. - The system receives a plurality of path points (202). For example, the plurality of path points can define a robot path for which one or more corresponding trajectories need to be determined.
- The system processes each network input in an input sequence that is derived from the path points using a trajectory generation neural network to generate an output sequence that includes a plurality of network outputs (204). Because the trajectory generation neural network is configured to auto-regressively generate data specifying robot trajectories over multiple time steps, at each time step the system can instantaneously, i.e., in real-time, generate a current network input for the network based on (i) a received system input that specifies a sequence of path points that collectively define a robot path for which a trajectory needs to be determined, (ii) previous network inputs in the input sequence, (iii) previous network outputs generated by the network, or a combination of one or more of (i)-(iii).
-
FIG. 3A is an illustration of example network inputs and outputs. As depicted inFIG. 3A , a network input specifies a currenttrajectory point q t 302, a currentreference direction d t 304 for the currenttrajectory point q t 302, a future reference direction d′t 306 for the currenttrajectory point q t 302, and “goal”vector g t 308 for the currenttrajectory point q t 302. - Specifically, for each network input in the input sequence, the current trajectory point qt is the starting trajectory point from which the system predicts a subsequent movement of a robot. The system generally determines the current trajectory point qt from a preceding network output ot−1 and a preceding trajectory point qt−1. For the very first time step, because there is no preceding network output or preceding trajectory point, the system instead uses the first path point in the sequence of path points specified by the system input as the current trajectory point.
- The system can obtain the current reference direction
-
- based on computing a displacement from the preceding path point pk(q
t )−1 to the current path point pk(qt ) of the current trajectory point qt. In the example ofFIG. 3A , for the currenttrajectory point q t 302, its currentpath point p k(qt ) 314 corresponds to the first path point that will be met starting from the current trajectory point qt, and its precedingpath point p k(qt )−1 312 corresponds to the immediately preceding path point of the currentpath point p k(qt ) 314 in the sequence of path points pk that define the robot path. - To determine which path point in the input sequence should be used as the current path point, the system can keep a record of respective distances between the generated trajectory points and the current path point. The system can then proceed to use a subsequent path point in the input sequence as the current path point when the distance begins to increase.
- The system can obtain the future reference direction
-
- based on computing a displacement from the current path point pk(q
t ) to the subsequent path point pk(qt )+1 of the current trajectory point qt. In the example ofFIG. 3A , for the currenttrajectory point q t 302, its subsequentpath point p k(qt )+1 316 corresponds to the immediately subsequent path point of the currentpath point p k(qt ) 314 in the sequence of path points pk that define the robot path. - The system can obtain the “goal” vector gt=pk(q
t )−qt based on computing a displacement from the current trajectory point qt to the current path point pk(qt ) of the current trajectory point qt. In the example ofFIG. 3A , for the currenttrajectory point q t 302, the system can obtain the “goal”vector g t 308 based computing a displacement from the currenttrajectory point q t 302 to the currentpath point p k(qt ) 314 of the currenttrajectory point q t 302. - Each network output in turn specifies a respective displacement between a current trajectory point and a subsequent trajectory point. As described above, the system generates the plurality of network outputs over multiple time steps.
- In particular, at each time step, the system provides the trajectory generation neural network with (i) a current network input and (ii) a preceding network output and uses the network to generate a current network output that specifies a displacement between a current trajectory point and a subsequent trajectory point. For the very first time step, because there is no preceding network output, the system can instead provide the network with the current network input and a predetermined placeholder input, i.e., in place of the preceding network output. The trajectory generation neural network then processes the current input and the predetermined placeholder input to generate the current network output for the first time step.
- In the example of
FIG. 3A , the system uses the trajectory generation neural network to generate a currentnetwork output o t 332 which defines a displacement from the currenttrajectory point q t 302 to the subsequenttrajectory point q t+1 352. In other words, in this example, the system predictsq t+1 352 to be the next trajectory point when generating the robot trajectory from the robot path. - The system generates a predicted trajectory of the robot (206) that is derived from the output sequence. For example, because each network output specifies a respective displacement between two adjacent trajectory points, the system can generate the predicted trajectory by computing a concatenation the respective displacements specified by the output sequence. The predicted trajectory in this way is also referred to as a forward trajectory of the robot.
- Optionally, in some cases, the system can also generate a backward trajectory from the forward trajectory by determining adjustments to one or more of the network outputs included in the sequence.
- Specifically, starting from the last network output in the output sequence, the system iteratively determines whether the displacement ot that is specified by the network output is parallel to the current reference direction dt of the current trajectory point qt as specified by the corresponding network input.
- In response to a positive determination, i.e., upon determining that the displacement that is specified by the network output is parallel to the reference direction of the current trajectory point, the system determines an adjustment to the displacement based on two adjacent path points of the current trajectory point. In general, the system determines such adjustment to require that, when the displacement of the current trajectory point is parallel to its current reference direction, a robot should travel in a line connecting the preceding path point and the current path point.
-
FIG. 3B is an illustration of example adjustments to network outputs. As shown inFIG. 3B , the system determines that thedisplacement o t 384 of the currenttrajectory point q t 382 is parallel to its current reference direction dt. Accordingly, the system can apply an adjustment to move the displacement to ot* 386 by projecting thedisplacement o t 384 to a line connecting two adjacent path points of the current trajectory point, i.e., the line connecting the preceding path point pk(qt )−1 of the currenttrajectory point q t 382 and the current path point pk(qt ) of the currenttrajectory point q t 382. - From this network output, the system follows a backward iteration process to iteratively determine adjustments to respective displacements specified by preceding network outputs in the output sequence.
- In various cases, in response to a negative determination, e.g., upon determining that the displacement that is specified by the network output is not parallel to the reference direction of the current trajectory point, the system generally moves onto a preceding network output in the output sequence without specifically applying any adjustments to the trajectory point.
- Once this backward iteration process has completed, the system can generate the backward trajectory from the adjustments being applied to the output sequence that is generated by the trajectory generation neural network. In other words, the system can use the backward trajectory instead of or in addition to the forward trajectory for use in planning a movement of the robot to travel through the robot path that is defined by the system input.
- Optionally, the system can also generate a “smoothed trajectory” by computing a weighted average of the forward trajectory and the backward trajectory. The smoothed trajectory, when generated, will then be similarly used in planning the movement of the robot. Examples of forward, backward, and smoothed trajectories are shown in
FIG. 3B . - This specification uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.
- Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.
- Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
- The term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
- A computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
- The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
- Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.
- Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
- To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.
- Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.
- The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
- In addition to the embodiments described above, the following embodiments are also innovative:
-
Embodiment 1 is a method comprising: - receiving a plurality of path points;
- processing each network input of a plurality of network inputs in an input sequence that is derived from the path points using a trajectory generation neural network to generate an output sequence comprising a plurality of network outputs, each network output specifying a respective displacement between two adjacent trajectory points; and
- generating, based on the output sequence, a predicted trajectory of the robot.
-
Embodiment 2 is the method ofembodiment 1, wherein the predicted trajectory of the robot represents a prediction for an output trajectory of a closed trajectory generator when given the path points. - Embodiment 3 is the method of any one of embodiments 1-2, wherein each network input specifies (i) a position of a current trajectory point, (ii) a current reference direction of the current trajectory point, (iii) a future reference direction of the current trajectory point, and (iv) a goal vector measuring a displacement between the current trajectory point and a current path point.
- Embodiment 4 is the method of any one of embodiments 1-3, further comprising generating an adjusted predicted trajectory from the predicted trajectory, comprising, for each network output in the output sequence:
- determining whether the displacement that is specified by the network output is parallel to the reference direction of the current trajectory point; and
- in response to a positive determination: determining, based on two adjacent path points of the current trajectory point, an adjustment to the displacement.
-
Embodiment 5 is the method of any one of embodiments 1-4, wherein: - the trajectory generation neural network is a recurrent neural network; and
- generating the output sequence comprising the plurality of network outputs comprises, at each of a plurality of time steps: processing, using the trajectory generation neural network, a current network input and a preceding network output to generate a current network output.
- Embodiment 6 is the method of any one of embodiments 4-5, wherein determining the adjustment to the displacement comprises: projecting the displacement to a line connecting two adjacent path points of the current trajectory point.
- Embodiment 7 is the method of any one of embodiments 4-6, wherein determining the adjustment to the displacement further comprises: iteratively determining adjustments to respective displacements specified by preceding network outputs in the output sequence.
- Embodiment 8 is the method of any one of embodiments 4-7, further comprising generating a smoothened predicted trajectory by computing a weighted average of the predicted trajectory and the adjusted predicted trajectory.
- Embodiment 9 is the method of any one of embodiments 1-8, wherein each trajectory point or path point is represented by multi-dimensional data having a respective dimension that is dependent on degrees of freedom (DoF) of the robot.
- Embodiment 10 is the method of any one of embodiments 1-9, further comprising training the trajectory generation neural network by optimizing an objective function measuring a difference between network outputs and target outputs that are derived from trajectories generated by Robot Controller Simulation (RCS).
- Embodiment 11 is a system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform the method of any one of
embodiments 1 to 10. - Embodiment 12 is a computer storage medium encoded with a computer program, the program comprising instructions that are operable, when executed by data processing apparatus, to cause the data processing apparatus to perform the method of any one of
embodiments 1 to 10. - While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a sub combination.
- Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
- Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.
Claims (20)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/867,437 US20210347047A1 (en) | 2020-05-05 | 2020-05-05 | Generating robot trajectories using neural networks |
PCT/US2021/030399 WO2021225923A1 (en) | 2020-05-05 | 2021-05-03 | Generating robot trajectories using neural networks |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/867,437 US20210347047A1 (en) | 2020-05-05 | 2020-05-05 | Generating robot trajectories using neural networks |
Publications (1)
Publication Number | Publication Date |
---|---|
US20210347047A1 true US20210347047A1 (en) | 2021-11-11 |
Family
ID=76076486
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/867,437 Abandoned US20210347047A1 (en) | 2020-05-05 | 2020-05-05 | Generating robot trajectories using neural networks |
Country Status (2)
Country | Link |
---|---|
US (1) | US20210347047A1 (en) |
WO (1) | WO2021225923A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220152817A1 (en) * | 2020-11-18 | 2022-05-19 | Dibi (Chongqing) Intelligent Technology Research Institute Co., Ltd. | Neural network adaptive tracking control method for joint robots |
CN116117825A (en) * | 2023-04-04 | 2023-05-16 | 人工智能与数字经济广东省实验室(广州) | FPGA implementation method based on noise-resistant fuzzy recurrent neural network |
WO2024098956A1 (en) * | 2022-11-10 | 2024-05-16 | 中国测绘科学研究院 | Method for fusing social media data and moving track data |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220113724A1 (en) * | 2019-07-03 | 2022-04-14 | Preferred Networks, Inc. | Information processing device, robot system, and information processing method |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110198813B (en) * | 2017-01-31 | 2023-02-28 | 株式会社安川电机 | Robot path generation device and robot system |
US20190184561A1 (en) * | 2017-12-15 | 2019-06-20 | The Regents Of The University Of California | Machine Learning based Fixed-Time Optimal Path Generation |
EP3793783A1 (en) * | 2018-05-18 | 2021-03-24 | Google LLC | System and methods for pixel based model predictive control |
-
2020
- 2020-05-05 US US16/867,437 patent/US20210347047A1/en not_active Abandoned
-
2021
- 2021-05-03 WO PCT/US2021/030399 patent/WO2021225923A1/en active Application Filing
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220113724A1 (en) * | 2019-07-03 | 2022-04-14 | Preferred Networks, Inc. | Information processing device, robot system, and information processing method |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220152817A1 (en) * | 2020-11-18 | 2022-05-19 | Dibi (Chongqing) Intelligent Technology Research Institute Co., Ltd. | Neural network adaptive tracking control method for joint robots |
US11772264B2 (en) * | 2020-11-18 | 2023-10-03 | Dibi (Chongqing) Intelligent Technology Research Institute Co., Ltd. | Neural network adaptive tracking control method for joint robots |
WO2024098956A1 (en) * | 2022-11-10 | 2024-05-16 | 中国测绘科学研究院 | Method for fusing social media data and moving track data |
CN116117825A (en) * | 2023-04-04 | 2023-05-16 | 人工智能与数字经济广东省实验室(广州) | FPGA implementation method based on noise-resistant fuzzy recurrent neural network |
Also Published As
Publication number | Publication date |
---|---|
WO2021225923A1 (en) | 2021-11-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11886997B2 (en) | Training action selection neural networks using apprenticeship | |
US20230330848A1 (en) | Reinforcement and imitation learning for a task | |
US20210347047A1 (en) | Generating robot trajectories using neural networks | |
KR102242516B1 (en) | Train machine learning models on multiple machine learning tasks | |
US20210201156A1 (en) | Sample-efficient reinforcement learning | |
US11403513B2 (en) | Learning motor primitives and training a machine learning system using a linear-feedback-stabilized policy | |
US10860927B2 (en) | Stacked convolutional long short-term memory for model-free reinforcement learning | |
US20210158162A1 (en) | Training reinforcement learning agents to learn farsighted behaviors by predicting in latent space | |
US20210103815A1 (en) | Domain adaptation for robotic control using self-supervised learning | |
US20210357731A1 (en) | Controlling agents using amortized q learning | |
US20230073326A1 (en) | Planning for agent control using learned hidden states | |
US20220366246A1 (en) | Controlling agents using causally correct environment models | |
US20220343157A1 (en) | Robust reinforcement learning for continuous control with model misspecification | |
US20240095495A1 (en) | Attention neural networks with short-term memory units | |
US20220076099A1 (en) | Controlling agents using latent plans | |
US20210349444A1 (en) | Accelerating robotic planning for operating on deformable objects | |
US20210232928A1 (en) | Placement-Aware Accelaration of Parameter Optimization in a Predictive Model | |
US11931908B2 (en) | Detecting robotic calibration accuracy discrepancies | |
US20240126812A1 (en) | Fast exploration and learning of latent graph models | |
US20240256883A1 (en) | Reinforcement learning using quantile credit assignment | |
Fry et al. | Adapting autonomous ocean vehicle software systems to changing environments | |
WO2023144395A1 (en) | Controlling reinforcement learning agents using geometric policy composition | |
WO2024149747A1 (en) | Training reinforcement learning agents to perform multiple tasks across diverse domains | |
WO2022069758A1 (en) | Robust reinforcement learning for constraint satisfaction while accounting for model misspecification |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: X DEVELOPMENT LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BANDARI, MARYAM;CHEN, KUANGYE;REEL/FRAME:052783/0221 Effective date: 20200528 |
|
AS | Assignment |
Owner name: INTRINSIC INNOVATION LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:X DEVELOPMENT LLC;REEL/FRAME:057650/0405 Effective date: 20210701 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: ADVISORY ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |