US20090088884A1 - Manufacturing automation system components compatibility and performance testing with integrated virtual and real environment - Google Patents
Manufacturing automation system components compatibility and performance testing with integrated virtual and real environment Download PDFInfo
- Publication number
- US20090088884A1 US20090088884A1 US12/237,894 US23789408A US2009088884A1 US 20090088884 A1 US20090088884 A1 US 20090088884A1 US 23789408 A US23789408 A US 23789408A US 2009088884 A1 US2009088884 A1 US 2009088884A1
- Authority
- US
- United States
- Prior art keywords
- virtual
- real
- coupling
- component
- world
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 50
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 claims abstract description 100
- 230000008878 coupling Effects 0.000 claims description 112
- 238000010168 coupling process Methods 0.000 claims description 108
- 238000005859 coupling reaction Methods 0.000 claims description 88
- 230000008569 process Effects 0.000 abstract description 69
- 229910003460 diamond Inorganic materials 0.000 description 21
- 239000010432 diamond Substances 0.000 description 21
- 238000010586 diagram Methods 0.000 description 10
- 230000008859 change Effects 0.000 description 8
- 239000012636 effector Substances 0.000 description 8
- 230000009471 action Effects 0.000 description 6
- 238000013459 approach Methods 0.000 description 6
- 238000010200 validation analysis Methods 0.000 description 5
- 238000004088 simulation Methods 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000003066 decision tree Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010399 physical interaction Effects 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/41885—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by modeling, simulation of the manufacturing system
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/31—From computer integrated manufacturing till monitoring
- G05B2219/31078—Several machines and several buffers, storages, conveyors, robots
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/39—Robotics, robotics to robotics hand
- G05B2219/39014—Match virtual world with real world
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/40—Robotics, robotics mapping to robotics vision
- G05B2219/40288—Integrate sensor, actuator units into a virtual manipulator
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Definitions
- This invention relates generally to a system and method for emulating a manufacturing automation system and, more particularly, to a system and method for emulating a manufacturing automation system using coupled virtual and physical sensors/actuators to provide component compatibility testing and system performance prediction.
- One approach is referred to as a traditional field test where testing of new physical equipment using existing systems can be achieved by either setting up a duplicate testing purpose only system or using the existing system in the plant. While there is excessive cost associated with creating and maintaining a duplicate testing purpose only system, testing using the plant requires interrupting current production. Also, the field testing is not feasible when the new physical equipment is unobtainable.
- virtual commissioning includes testing with emulated systems and components.
- virtual commissioning enables reasonable accuracy and greater efficiency of validation allowing for many test scenarios that would not be possible at the installation site without adding significant deployment time and cost.
- the control logic is tested on a virtual model of the system or a station before actually being built, thus greatly reducing the time required for field testing.
- virtual commissioning hinges on the completeness and accuracy of the virtual model of the system and components.
- the modeling fidelity of each automation component such as the PLC, I/O block, sensors/actuators, network, wiring and HMI, has a direct impact on the validity of the virtual commissioning results.
- creating a 100% complete and accurate virtual environment is not likely to be realized in the near future.
- the third potential approach is referred to as a hybrid emulation environment.
- the hybrid emulation environment is comprised of virtual emulated models and real physical system components whose virtual model is not readily available.
- the manufacturing automation system moves the part from one manufacturing station to another manufacturing station while performing certain manufacturing activities, such as welding, to the part.
- certain manufacturing activities such as welding
- the part When a part is moved from a station in the virtual world to another station in the physical world, the part has to be presented to the physical station so that the physical station may function properly.
- the physical part when a part is moved from a physical work-station to a virtual work-station, the physical part has to be properly handled in the real world and its transfer presence to the virtual world has to be effectively communicated to the virtual world.
- One technique for simulating such a process includes emulating the process in the virtual world using algorithms on a computer system.
- a system and method for providing an integrated virtual and real emulation environment for a manufacturing process to provide component compatibility testing and system performance prediction.
- a real manufacturing system includes one or more real components that are controlled by a programmable logic controller.
- the real component can be replaced with a virtual component to determine whether it is compatible in the process, where the programmable controller sends signals to and receive signals from the virtual component as if it were the real component.
- a virtual manufacturing process is provided that includes virtual components where the virtual process is controlled by a programmable logic control as if it were a real process to determine the performance of a real system using the virtual process.
- FIG. 1 is a flow chart diagram showing a process for determining whether a hybrid virtual and real emulation process needs to be developed that is a combination of one or more real machines in a virtual process based on available information;
- FIG. 2 is a plan view showing integration of an emulated and real manufacturing automation system
- FIG. 3 is an illustration of a sensor being directly attached to a controller in a manufacturing process by a network infrastructure
- FIG. 4 is an illustration of a sensor being attached to a machine, either emulated or physical, in a manufacturing process
- FIG. 5 is a plan view of an integrated real and virtual manufacturing automation system including complimentary pairs of physical sensors and actuators;
- FIG. 6 is a plan view of an integrated real and virtual manufacturing automation system showing a direct electrical connection between a virtual domain and a real domain;
- FIG. 7 is a flow diagram showing a transition between the real domain and the virtual domain in a manufacturing process, where sensors are provided between the real and virtual domains;
- FIG. 8 is a flow chart diagram showing a process for determining virtual/physical coupling with and without a part
- FIG. 9 is a flow chart diagram showing a process for selecting sensor/actuator coupling without a part across an integrated virtual and real automation system
- FIG. 10 is a flow chart diagram showing a process for part handling of an integrated virtual and real automation system
- FIG. 11 is a plan view of an integrated real and virtual manufacturing automation system showing physically coupled sensors and actuators for a part flow from the real domain to the virtual domain;
- FIG. 12 is a plan view of an integrated real and virtual manufacturing automation system showing virtually coupled sensors and actuators for a part flow from the real domain to the virtual domain;
- FIG. 13 is a plan view of an integrated real and virtual manufacturing automation system showing physically coupled sensors and actuators for a part flow from the virtual domain to the real domain;
- FIG. 14 is a plan view of an integrated real and virtual manufacturing automation system showing virtually coupled sensors and actuators with a virtual part flow from the virtual domain to the real domain;
- FIG. 15 is a plan view of an integrated real and virtual manufacturing automation system showing physically coupled sensors and actuators with a real part flow from the real domain to the virtual domain;
- FIG. 16 is a plan view of an integrated real and virtual manufacturing automation system showing virtually coupled sensors and actuators with a virtual part flow from the real domain to the virtual domain;
- FIG. 17 is a plan view of an integrated real and virtual manufacturing automation system showing virtually coupled sensors and actuators with a virtual part flow from the virtual domain to the real domain;
- FIG. 18 is a plan view of an integrated real and virtual manufacturing automation system showing a virtual part flow in the real domain.
- the present invention integrates physical and emulated automation components with data and information exchange between the virtual world and the real world.
- the invention provides an effective platform for developing and testing hardware components that are difficult or time consuming to simulate with existing resources and technology.
- the present invention also provides a design for automation systems and component compatibility and performance testing. With this approach, new system validation times can be reduced, and the launch quality of the new system can be improved.
- the present invention provides a hybrid environment that integrates physical and emulated automation components including data and information exchange between the virtual world and the real world.
- the technique can be used for automation system and component compatibility and performance testing.
- a generic process can be developed to decide whether it is effective to use hardware-in-the-loop (HIL) emulation for automation component compatibility and performance testing, and corresponding validation procedures are defined.
- HIL hardware-in-the-loop
- the HIL emulation for automation component compatibility and performance testing provides a number of advantages including control validation for testing control logic on hybrid or simulated processes, hypothetical machine arrangements providing plug and play characteristics that make it possible to test various arrangements easily and the testing of new hardware where the new hardware can be plugged into an existing process simulation and run in real time to test its compatibility and performance and simulations can be plugged into an existing system and run in real time to test feasibility.
- the HIL emulation benefits include shorter system development cycles because the hard-to-model virtual component can be replaced by a real component, earlier testing system and feedback using virtual components along with real system set-up environment, lower cost for various scenario testing as the virtual models can be reused and duplicated seamlessly, and higher confidence on new system performance with usage of real network and standard interfaces.
- the real and virtual integration provides any level of virtual and real combination, seamless part handling and data exchange, plug and play modular characteristics, controller and components are unaware of the presence of other regions, real-time event-driven data handling, scalable and hierarchical, and standardized signal coupling and interface.
- FIG. 1 is a flow chart diagram 10 showing a general overview of a process of the invention for determining whether an emulated process of a manufacturing process will include real machines, according to an embodiment of the present invention.
- the flow chart diagram shows two different scenarios for an integrated real and virtual manufacturing automation system and steps for setting up such a hybrid system. Both set-up scenarios enable an engineer to evaluate system performance or machine compatibility when new equipment is added to or is used to replace existing equipment in an existing manufacturing automation system.
- the system performs a machine compatibility test with the integrated real and virtual manufacturing automation system, which can be done either in the plant or the lab. However, when the test is done in the field, the physical machines will be in the majority of the integrated real and virtual manufacturing automation system.
- This method allows an engineer to test the new system machines with real production manufacturing automation system without concern of fidelity of the virtual models and incurring cost for transporting the real new system machines to the testing facility, or even higher cost for setting up a duplicate system in a prototype shop.
- system performance and machine compatibility testing can be carried out by integrating the new physical system machines with a perspective emulated manufacturing automation system that is a virtual model of an existing system and has been calibrated against the corresponding physical system. Furthermore, this method allows an engineer to conduct the system performance and machine compatibility using the earlier stage prototype of the new system machines before finalizing the system design.
- the process first selects a particular automation system or component that needs to be validated or emulated at box 12 . The process then determines whether complete and accurate emulation models are available for the particular system or component that is to be emulated at decision diamond 14 . If all of the models that are necessary for the emulation are available at the decision diamond 14 , then the process performs a 100% virtual emulation and validation of the system or component at box 16 . If all of the models are not available at the decision diamond 14 , then the process determines whether a complete prototype of those systems or components that cannot be emulated are available at the plant location at decision diamond 18 .
- the process will perform in field debugging and testing processes at the plant at box 20 . If a prototype of the system or component is not available at the plant at the decision diamond 18 , then the process prepares an HIL for testing at box 22 , and then determines if the new system or component is ready for lab testing at decision diamond 24 .
- the emulated system or component may be integrated with an existing physical system and process, starting at box 26 , which results in HIL emulation in the plant. Therefore, the actual physical machine can be provided at its normal location for automation system compatibility and performance testing. In this situation, most of the system exists in the physical world, with some virtual components.
- the process selects a real and virtual world coupling method at the plant location at box 28 to couple the physical system with a virtual component, such as an emulated robot on a computer.
- the process then configures an input/output (I/O) interface at box 30 which is a bridge between the real and virtual world, and chooses a part passing method at box 32 that determines how a real part location sensing to be coupled between a virtual machine and a real machine in the system.
- the process then synchronizes the real machine and the virtual world clocks at box 34 , and tests the component or system for performance and compatibility at box 36 .
- the process performs the HIL emulation in the lab where the emulation is mostly in the virtual world with a few physical machines.
- the process selects the real and virtual world coupling method at box 40 , configures the input/output interface at box 42 , determines how the part location information will be passed between the virtual world and the real world at box 44 , synchronizes the real world clocks and the virtual world clocks at box 46 and tests the component or system for performance and compatibility at box 48 .
- FIG. 2 is a plan view of an exemplary automated manufacturing system 50 including two real manufacturing cells 52 and 56 and a virtual manufacturing cell 54 linked together by a conveyer 58 , where the conveyer 58 delivers a part 74 from cell to cell.
- the system 50 is controlled by a programmable logic controller (PLC) 60 based on the status of various sensors, for example sensors 62 , 64 and 66 , through a network 68 provided throughout the system 50 , where other system status information is also provided on the network 68 .
- PLC programmable logic controller
- a real robot 70 is controlled by a robot controller 72 in the real manufacturing cells 52 and 56 and carries the part 74 from the conveyer 58 to a machine 76 that will perform an operation on the part 74 and return it back to the conveyer 58 .
- An end-effector 78 on the robot 70 may be directly controlled by the robot controller 72 or indirectly controlled by the PLC 60 depending on the system architecture. In the latter case, information, such as end-effector is in position, may have to be sent to the PLC 60 so that the PLC 60 can issue the necessary automation command to activate the end-effector 78 to pick up the part 74 .
- the opening of the end-effector 78 will be directly controlled by the robot controller 72 or the PLC 60 based on system architecture.
- the robot 70 will pick up the part 74 from the corresponding machine 76 and drop off the part 74 at the original location on the conveyer 58 .
- the conveyer 58 will move the part 74 a step forward to the next cell 54 so that the new operation cycle can start.
- the virtual cell 54 in the system 50 includes an emulated or virtual robot 80 including a virtual end-effector sensor 96 instead of a real robot.
- the corresponding sensor information that triggers the real robot operation is transmitted to the robot controller 72 and then the program that carries out the robot simulation and the status change information of the virtual robot 80 when it drops off the part 74 at the designated location is transmitted to the PLC 60 so that the machine 76 at that cell will function properly.
- FIGS. 3 and 4 provide two sensor connecting methods for the virtual end-effector sensor 96 between the virtual robot 80 and the real machine 76 in the cell 54 and the rest of the system.
- a sensor 82 is directly attached to an automation controller, such as a PLC 90 , through a control network 84 .
- a sensor 92 is attached to a local machine 88 , such as a robot.
- the PLC 90 only controls the start of the robot 86 and only when the complete task of the robot 86 is finished, the PLC 90 will be informed that the next task is ready to be performed.
- the sensing signal from the sensor 92 and a state change of actuator, such as an end-effector 94 are not necessarily transmitted to the PLC 90 .
- FIG. 5 is an illustration of an integrated real and virtual manufacturing automation system 100 including complimentary pairs of physical sensors and actuators.
- the system 100 includes a real world domain 102 including a real machine 106 , such as a robot, and a virtual world domain 104 including a virtual world machine 108 , where the robot 106 is a component of a manufacturing automation system.
- the real domain 102 and the virtual domain 104 are electronically isolated. Signals passed between the real world domain 102 and the virtual world domain 104 is facilitated by matching an actuator with a complimentary sensor and a sensor with a complimentary actuator, as will be discussed in more detail below.
- the robot 106 and the virtual machine 108 are controlled by a PLC 110 through an industrial network device and a corresponding input/output (I/O) module 112 .
- An I/O interface card 114 transfers signals between the real world domain 102 and the virtual world domain 104 .
- the real world domain 102 includes a real sensor 118 and a real actuator 116 , where the real sensor 118 provides a signal of the position of the robot 106 and the real actuator 116 actuates the robot 106 to move it from position to position.
- the real world domain 102 also includes a complimentary pair of a sensor 120 and an actuator 122 and a complimentary pair of an actuator 124 and a sensor 126 , where the sensors detect actuation of the actuator in the pair.
- the virtual world domain 104 includes a virtual sensor 128 that senses the position of the machine 108 and a virtual actuator 130 that virtually actuates the machine 108 . Signals are said to be mapped between a pair of coupled sensors when the sensor signal from either a virtual sensor or a real sensor is used to override or by-pass the signal from a real sensor or a virtual sensor, respectively.
- complimentary sensors and actuators can be set forth as follows.
- a complimentary actuator could be a blade mounted on a shaft of a stepper motor. By moving the blade into and out of the photo-eye path, a part passing through the path can be simulated.
- a proximity sensor the presence of a part can be simulated by moving a metallic object into and out of the sensing area of the sensor.
- a complimentary sensor could be a pair of switches at two extreme positions of the actuator.
- a virtual sensor coupled to a synchronized real sensor or vice versa is said to be coupled in parallel.
- a virtual actuator coupled to a synchronized real actuator or vice versa is also said to be coupled in parallel.
- a sensor triggered by an immediate upstream actuator through a virtual or real process is said to be coupled in serial. If a sensor is coupled to an actuator using a virtual process, the coupling is referred to as virtual coupling. If a sensor is coupled to an actuator using a real process, the coupling is referred to as physical coupling.
- Sensor signals from the real sensor 118 of the position of the robot 108 are directly connected to the I/O module 112 and signals from the virtual sensor 128 become the output signals of the interface card 114 and trigger the actuation of the real actuator 122 , whose state of change will be sensed by the real sensor 120 .
- the output signal of the real sensor 120 is connected to the I/O module 112 .
- the virtual sensor 128 is coupled to the real sensor 120 in parallel while the real actuator 122 and the real sensor 120 are coupled in serial.
- an output signal from the I/O module 112 to the robot 106 is connected directly.
- the output signal of the I/O module 112 is connected to the real actuator 124 , whose state change will be sensed by the complimentary sensor 126 .
- the output of the sensor 126 is the input to the interface card 114 and will drive the virtual actuator 130 to alter its state.
- the real actuator 124 is coupled to the virtual actuator 130 in parallel, while the real actuator 124 is coupled to the real sensor in serial.
- the complimentary actuator 126 is needed to capture the signal to relay it to the virtual domain 104 through the interface card 114 .
- the actuator 116 will have to be connected to a complimentary sensor, and then the state change of the actuator 116 will be captured and sent to the virtual world 104 through the interface card 114 .
- FIG. 6 is a plan view of an integrated real and virtual manufacturing automation system 140 providing a direct electrical connection between the real domain 102 and the virtual domain 104 , where like elements to the system 100 are identified by the same reference numeral.
- the complimentary pairs of sensors and actuators between the I/O module 112 and the I/O interface card 114 are replaced with a voltage-translation device 142 so that the real and virtual portions of the system 140 are linked directly together.
- the I/O interface card 114 will facilitate the signal passing between the real and virtual domains of the system 140 and the voltage-translation device 142 will match the difference in the analog signal levels between the virtual cell 104 and the real world machines.
- Sensor signals from the robot 106 are directly coupled to the I/O module 112 , while signals from the virtual sensor 128 become the output signals of the interface card 114 , and using the voltage-translation device 142 , connects directly to the input of the I/O module 112 .
- the output signal from the I/O module 112 to the robot 106 through the actuator 116 are connected directly.
- the output signal of the I/O module 112 is connected directly to the interface card 114 through the voltage-translation device 142 , which converts the output of the I/O module 112 to a voltage acceptable for the interface card 114 .
- FIG. 7 is a plan view of a manufacturing process 150 for a part 152 moving through and between real world cells 154 including real world machines 156 , such as a real robot, and virtual world cells 158 including virtual machines, such as a virtual robot 160 , that is a model of an actual robot.
- real world machines 156 such as a real robot
- virtual world cells 158 including virtual machines, such as a virtual robot 160
- a part presence token 164 will be on to indicate that the real part 152 is available in the process.
- the part presence token 164 will be off to indicate that there is no physical part in the process anymore, and the virtual robot 160 will work on a virtual part.
- the virtual part is provided in the real cells 154 that provides a signal to a coupled real sensor telling the machine 156 that the part 152 is actually in its proper orientation, even though it may not be and the part presence token 164 is off.
- the actual part 152 may be worked on by the machine 156 in the real cell 154 , where the virtual part 162 is inactive.
- Sensors 166 are provided between the real cells 154 and the virtual cells 158 so that the process knows when the part 152 is moved from a real cell 154 to a virtual cell 158 and from a virtual cell 158 to a real cell 154 .
- the part 152 moving from the real world to virtual world and returning to the real world from the virtual world for the given example set-up can only be emulated by the following.
- the actual part remains at the current location and a virtual part is created in the virtual world when the sensor information is received that the part is moved into the virtual world.
- the virtual part in the virtual world is destroyed and at the same time signals are provided that the part is at its designated location in the real world.
- the part presence token 164 works according to the rules discussed above. However, in other situations, the actual part 152 may be worked on by the machine in the real cell 154 , where the virtual part is inactive. It is assumed that sensors are provided between the real cells 154 and the virtual cells 158 so that the process knows when the part 152 is moved from the real world to the virtual world and from the virtual world to the real world. If there is no sensor available at the interface of the real cells 154 and the virtual cells 158 , a sensor needs to be added to the integrated virtual and real manufacturing automation system.
- FIG. 8 is a flow chart diagram 170 of a decision tree for selecting a proper sequence (serial) coupling method for handling the actuator action to the sensor sensing.
- a proper sequence serial
- the actuation of the actuator will eventually lead to the state change of the preceding sensor, directly or indirectly, and through a part or not through a part.
- a motor encoder sensor will immediately sense the motion of the motor.
- the up and down position sensors of a lift only change their state when the lift arrives at and leaves from a corresponding position, not when the lift is in motion between the two corresponding positions.
- Another example of this scenario is that sensors on the two downstream branches of a sorting machine alter their state when the part travels through its corresponding branch as a consequence of the upstream actuator or sorter. Therefore, there is a need to consider the sequence coupling of the actuator action to the sensor sensing. Some of the actuator actions to the sensor sensing involve the part, while others do not. To address those issues, it is necessary that some sensors and actuators be duplicated in the virtual system so that the integrated virtual real system will function as a real system does.
- the first step in the process is to analyze the interactions between sensor and actuator coupling methods at box 172 . From this analysis, the process determines whether the actuator and sensor are directly coupled at decision diamond 174 , such as a lifter with a position sensor, or coupled indirectly, such as a pusher with a photo-eye sensor. If the coupling is direct at the decision diamond 174 , then the process uses non-part coupling at box 176 . The process then determines whether duplicated real actuator and sensor coupling is available at decision diamond 178 . If duplicated real actuator and sensor coupling is available at the decision diamond 178 , then the process uses physical coupling without a part at box 180 .
- duplicated real actuator and sensor coupling is not available at the decision diamond 178 . If duplicated real actuator and sensor coupling is not available at the decision diamond 178 , then the process determines whether duplicated virtual actuator and sensor coupling is available at decision diamond 182 . If duplicated virtual actuator and sensor coupling is available at the decision diamond 182 , then the process provides virtual coupling without a part at box 184 . If duplicated virtual actuator and sensor coupling is not available at the decision diamond 182 , then the process uses other non-hybrid testing techniques at box 186 .
- the same process is used if the actuator does not directly interact with the sensor at the decision diamond 174 , where a part is involved in the coupling at box 188 .
- the process determines whether duplicated real actuator, part and sensor coupling is available at decision diamond 190 , and if so, provides physical coupling with the part at box 192 . If duplicated real actuator, part and sensor coupling is not available at the decision diamond 190 , then the process determines whether duplicated virtual actuator, part and sensor coupling is available at decision diamond 194 , and if so, provides virtual coupling with the part at box 196 . Otherwise, the process goes to the box 186 to provide other non-hybrid testing.
- FIG. 9 is a flow chart diagram 200 showing a decision tree process for selecting the proper sensor/actuator coupling when there is no part involved in an actuator action and a sensor sensing sequence.
- this problem is divided into two groups, namely, coupling purely inside the real or virtual domains and coupling across the boundary of the real and virtual domains at boxes 202 and 204 , respectfully.
- For coupling inside the real or virtual domain at the box 202 there is no special attention required as the actuator action to the sensor sensing sequence coupling happens naturally as the process flow.
- the real domain at box 206 there is direct physical interaction between the sensor and actuator at box 208 .
- the virtual domain at box 210 there is direct virtual interaction between the actuator and sensor at box 212 .
- Coupling from the virtual domain to the real domain is provided at box 220 , which is broken down to physical coupling at box 222 and virtual coupling at box 224 .
- the process can add a mapped real actuator that synchronizes with a virtual actuator to couple a real sensor to trigger a real domain execution, as will be discussed below with reference to FIG. 13 .
- the process can add a mapped virtual sensor coupling with a virtual actuator to overwrite a corresponding real sensor to trigger a real side execution, as will be discussed below with reference to FIG. 14 .
- FIG. 10 is a flow chart diagram 230 showing a decision tree process for selecting a proper coupling method for handling a part transferred across the virtual and real domains in an integrated system as provided at box 232 .
- This part handling problem can be provided into two groups, namely, part handling purely inside the real or virtual domain at box 234 and a part traveling across the boundary between the real and virtual domains at box 236 .
- the decision can be further divided based on the absence or presence of a real part.
- normal processes are used at box 242 .
- the process duplicates the real actuator and real sensor in the virtual domain using information obtained from the virtual actuator and virtual part interaction, and uses the duplicate sensor signal to override the signal from non-functional real sensor due to the absence of a real part at box 246 . If the part is in the virtual domain at box 248 and a virtual part is being used at box 250 , then normal processes are used at box 252 . The situation where a real part travels in the virtual domain does not exist so that it is not applicable at box 256 .
- a part traveling across the boundary between the virtual and real domains can either travel from the real domain to the virtual domain at box 258 or travel from the virtual domain to the real domain at box 260 .
- physical coupling is used at box 264 .
- a virtual coupling method is used with a virtual part at box 268 .
- a virtual coupling method is used with a virtual part at box 272 .
- FIG. 11 is a plan view of an integrated real and virtual manufacturing automation system 280 showing a physically coupled sensor/actuator for a part flow from the real domain 102 to the virtual domain 104 , particularly, a part flow from the robot 106 to the virtual machine 108 , without a part, where like elements to the system 100 are identified by the same reference numeral.
- the robot 106 uses the real actuator 116 and the virtual machine 108 uses the virtual sensor 128 .
- This physical coupling process starts with providing a duplicated real sensor 282 that is coupled in serial to the real actuator 116 through a coupler interface 284 .
- the output signal from the I/O module 112 to the real actuator 116 would normally be sensed by the real sensor 118 and sent to the virtual sensor 128 through the interface card 114 .
- the signal from the duplicate real sensor 282 is instead sent to a switch 286 that sends the sensor signal to the I/O module 112 so that the virtual sensor 128 is by-passed and the signal from the virtual sensor 128 is overridden by the signal from the duplicated sensor 282 .
- FIG. 12 is a plan view of an integrated real and virtual manufacturing automated system 290 showing a virtually coupled sensor and actuator for a part flow from the real domain 102 to the virtual domain 104 , particularly, a part flow from the robot 106 to the virtual machine 108 , without a physical part, where like elements to those in the system 100 are identified by the same reference numeral.
- the robot 106 uses the real actuator 116 and the virtual machine 108 uses the virtual sensor 128 .
- This virtual coupling process starts with providing a duplicated virtual actuator 292 that is coupled to the virtual sensor 130 by a coupling interface 294 .
- the output signal from the I/O module 112 that would normally be intended for the real actuator 116 is switched by a switch 296 to the interface card 114 to be received by the duplicated virtual actuator 292 .
- the virtual actuator 292 subsequently provides the state change for the virtual sensor 128 .
- FIG. 13 is a plan view of an integrated real and virtual manufacturing automation system 300 showing physically coupled sensors and actuators with a part flow from the virtual domain 104 to the real domain 102 without a physical part, where like elements to the system 100 are identified by the same reference numeral.
- the robot 108 uses the real sensor 118 and the virtual machine 108 uses the virtual actuator 130 .
- the physical coupling process starts by adding a duplicated real actuator 302 that is coupled to the real sensor 118 through a coupling interface 304 .
- the output from the I/O module 112 that would normally be sent to the virtual actuator 130 is by-passed by a switch 306 so that the output from the I/O module 112 is sent to the duplicated real actuator 302 .
- FIG. 14 is a plan view of an integrated real and virtual manufacturing automation system 310 showing virtual coupling between a sensor and an actuator for a virtual part flow from the virtual domain 104 to the real domain 102 , where like elements to the system 100 are identified by the same reference numeral.
- the robot 106 uses the real sensor 118 and the virtual machine 108 uses the virtual actuator 130 .
- This virtual coupling process employs a duplicated virtual sensor 312 that is coupled to the virtual actuator 130 through a virtual coupling interface 314 . Signals from the virtual actuator 130 are sensed by the duplicated actuator 312 , and the signals from the duplicate sensor 312 are sent as an input to the I/O module 112 through a switch 316 that by-passes the real sensor 118 .
- FIG. 15 is a plan view of an integrated real and virtual manufacturing automation system 320 showing a physical coupling between a sensor and an actuator for a part flow from the real domain 102 to the virtual domain 104 using a real part, where like elements to the system 100 are identified by the same reference numeral.
- the robot 106 uses the real actuator 116 and the virtual machine 108 uses the virtual sensor 128 .
- the real actuator 116 interacts with a real part 324 .
- the physical coupling process starts with providing a duplicated real sensor 326 that is coupled to the real actuator 116 indirectly through the part 324 and coupling interfaces 328 and 330 . Signals from the duplicated sensor 326 are sent to the interface card 114 that triggers the birth of a virtual part 332 .
- the virtual part 332 is coupled to the virtual sensor 128 through a coupling interface 334 .
- FIG. 16 is a plan view of an integrated real and virtual manufacturing automation system 340 showing virtual coupling between a sensor and an actuator for a part flow from the real domain 102 to the virtual domain 104 using a virtual part, where like elements to the system 100 are identified by the same reference numeral.
- the robot 106 uses the real actuator 116 and the virtual machine 108 uses the virtual sensor 128 .
- the real actuator 116 interacts with a virtual part 342 .
- This embodiment for the virtual coupling process provides a duplicated virtual actuator 344 that is coupled to the virtual sensor 128 indirectly through the virtual part 342 and coupling interfaces 346 and 348 .
- Output signals from the I/O module 112 that were intended for the real actuator 116 and by-passed by a switch 350 so that the signals are sent to the interface card 114 to drive the virtual actuator 344 and trigger the virtual machine 108 to perform some action.
- FIG. 17 is a plan view of an integrated real and virtual manufacturing automation system 360 showing virtual coupling between a sensor and actuator for a virtual part flow from the virtual domain 104 to the real domain 102 , where like elements to the system 100 are identified by the same reference numeral.
- the robot 106 uses the real sensor 118 and the virtual machine 108 uses the virtual actuator 130 .
- the virtual actuator 130 interacts with a virtual part 362 .
- the virtual coupling provides a duplicated virtual sensor 364 that is coupled to the virtual actuator 130 indirectly through the virtual part 362 and coupling interfaces 366 and 368 . Signals from the virtual sensor 364 and sent to the interface card 114 as an output, which is used to replace the signal from the real sensor 118 by a switch 370 .
- FIG. 18 is a plan view of an integrated real and virtual manufacturing automation system 380 showing sensors and actuators in the real domain 102 interacting with a virtual part, where like elements to the system 100 are identified by the same reference numeral.
- a second real world machine such as a robot 382 , is provided in the real domain 102 .
- a real sensor 384 receives signals from the robot 382 and the real actuator 116 controls the actuation of the robot 106 .
- the coupling technique is applicable to the situation where the real part does not exist in the real world.
- a virtual part 388 in the virtual domain 104 is provided.
- a virtual actuator 390 is provided to duplicate the real actuator 116 and a virtual sensor 392 is provided to duplicate the real sensor 384 .
- the virtual actuator 390 is synchronized with the real actuator 116 through the interface card 114 and the virtual sensor 392 overwrites the real sensor 384 through the interface card 114 .
- the coupling is achieved indirectly between the virtual actuator 390 and the virtual sensor 392 through the virtual part 388 and coupling interfaces 394 and 396 to provide the virtual part between the robots 106 and 382 .
Landscapes
- Engineering & Computer Science (AREA)
- Manufacturing & Machinery (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Programmable Controllers (AREA)
- Control By Computers (AREA)
Abstract
Description
- This application claims the benefit of the filing date of Provisional Application No. 60/976,101 titled Methods of Integrating Real and Virtual World Using Coupled Sensor/Actuator, filed Sep. 28, 2007.
- 1. Field of the Invention
- This invention relates generally to a system and method for emulating a manufacturing automation system and, more particularly, to a system and method for emulating a manufacturing automation system using coupled virtual and physical sensors/actuators to provide component compatibility testing and system performance prediction.
- 2. Discussion of the Related Art
- Various approaches are known in the art for testing and debugging manufacturing automation systems and processes. One approach is referred to as a traditional field test where testing of new physical equipment using existing systems can be achieved by either setting up a duplicate testing purpose only system or using the existing system in the plant. While there is excessive cost associated with creating and maintaining a duplicate testing purpose only system, testing using the plant requires interrupting current production. Also, the field testing is not feasible when the new physical equipment is unobtainable.
- Another approach is referred to virtual commissioning that includes testing with emulated systems and components. By using the emulated or simulated system and components, virtual commissioning enables reasonable accuracy and greater efficiency of validation allowing for many test scenarios that would not be possible at the installation site without adding significant deployment time and cost. Specifically, for virtual commissioning of manufacturing automation controls, the control logic is tested on a virtual model of the system or a station before actually being built, thus greatly reducing the time required for field testing. However, virtual commissioning hinges on the completeness and accuracy of the virtual model of the system and components. In particular, the modeling fidelity of each automation component, such as the PLC, I/O block, sensors/actuators, network, wiring and HMI, has a direct impact on the validity of the virtual commissioning results. Furthermore, creating a 100% complete and accurate virtual environment is not likely to be realized in the near future.
- The third potential approach is referred to as a hybrid emulation environment. The hybrid emulation environment is comprised of virtual emulated models and real physical system components whose virtual model is not readily available. During the manufacturing process, the manufacturing automation system moves the part from one manufacturing station to another manufacturing station while performing certain manufacturing activities, such as welding, to the part. When a part is moved from a station in the virtual world to another station in the physical world, the part has to be presented to the physical station so that the physical station may function properly. In the contrast, when a part is moved from a physical work-station to a virtual work-station, the physical part has to be properly handled in the real world and its transfer presence to the virtual world has to be effectively communicated to the virtual world.
- As discussed above, it is known in the art to simulate manufacturing or other processes prior to implementing the process so that engineers and technicians can ensure that the process will operate adequately and efficiently as intended. One technique for simulating such a process includes emulating the process in the virtual world using algorithms on a computer system.
- It is sometimes not possible to provide a realistic virtual model of a particular robot or other machine that the engineer wants to test. There are many reasons as to why a particular machine cannot be accurately modeled, including trade secret information of the particular machine from the manufacturer. In these situations, it is desirable to provide a hybrid environment of the manufacturing cell, where certain portions of the cell are provided in the virtual world and those machines that cannot be virtually modeled are provided as real devices in the process.
- Current virtual emulation approaches require extensive model development that requires costly engineering time and software investment. Emulation is increasingly used for virtual testing of complex automation systems, but it cannot guarantee the complete correctness of a given automation system because of the inaccuracies in a particular model. The development of simulation models requires additional time and engineering expertise. Highly accurate model development is time consuming, expensive and sometimes not possible. Field testing and debugging requires the availability of new equipment and access to deployed automation systems that may result in costly regular production shutdown.
- In accordance with the teachings of the present invention, a system and method are disclosed for providing an integrated virtual and real emulation environment for a manufacturing process to provide component compatibility testing and system performance prediction. In one embodiment, a real manufacturing system includes one or more real components that are controlled by a programmable logic controller. The real component can be replaced with a virtual component to determine whether it is compatible in the process, where the programmable controller sends signals to and receive signals from the virtual component as if it were the real component. For system performance prediction, a virtual manufacturing process is provided that includes virtual components where the virtual process is controlled by a programmable logic control as if it were a real process to determine the performance of a real system using the virtual process.
- Additional features of the present invention will become apparent from the following description and appended claims taken in conjunction with the accompanying drawings.
-
FIG. 1 is a flow chart diagram showing a process for determining whether a hybrid virtual and real emulation process needs to be developed that is a combination of one or more real machines in a virtual process based on available information; -
FIG. 2 is a plan view showing integration of an emulated and real manufacturing automation system; -
FIG. 3 is an illustration of a sensor being directly attached to a controller in a manufacturing process by a network infrastructure; -
FIG. 4 is an illustration of a sensor being attached to a machine, either emulated or physical, in a manufacturing process; -
FIG. 5 is a plan view of an integrated real and virtual manufacturing automation system including complimentary pairs of physical sensors and actuators; -
FIG. 6 is a plan view of an integrated real and virtual manufacturing automation system showing a direct electrical connection between a virtual domain and a real domain; -
FIG. 7 is a flow diagram showing a transition between the real domain and the virtual domain in a manufacturing process, where sensors are provided between the real and virtual domains; -
FIG. 8 is a flow chart diagram showing a process for determining virtual/physical coupling with and without a part; -
FIG. 9 is a flow chart diagram showing a process for selecting sensor/actuator coupling without a part across an integrated virtual and real automation system; -
FIG. 10 is a flow chart diagram showing a process for part handling of an integrated virtual and real automation system; -
FIG. 11 is a plan view of an integrated real and virtual manufacturing automation system showing physically coupled sensors and actuators for a part flow from the real domain to the virtual domain; -
FIG. 12 is a plan view of an integrated real and virtual manufacturing automation system showing virtually coupled sensors and actuators for a part flow from the real domain to the virtual domain; -
FIG. 13 is a plan view of an integrated real and virtual manufacturing automation system showing physically coupled sensors and actuators for a part flow from the virtual domain to the real domain; -
FIG. 14 is a plan view of an integrated real and virtual manufacturing automation system showing virtually coupled sensors and actuators with a virtual part flow from the virtual domain to the real domain; -
FIG. 15 is a plan view of an integrated real and virtual manufacturing automation system showing physically coupled sensors and actuators with a real part flow from the real domain to the virtual domain; -
FIG. 16 is a plan view of an integrated real and virtual manufacturing automation system showing virtually coupled sensors and actuators with a virtual part flow from the real domain to the virtual domain; -
FIG. 17 is a plan view of an integrated real and virtual manufacturing automation system showing virtually coupled sensors and actuators with a virtual part flow from the virtual domain to the real domain; and -
FIG. 18 is a plan view of an integrated real and virtual manufacturing automation system showing a virtual part flow in the real domain. - The following discussion of the embodiments of the invention directed to a system and method for providing hardware-in-the-loop emulation in a manufacturing process to provide component compatibility testing and system performance prediction is merely exemplary in nature and is in no way intended to limit the invention or its application or uses. For example, the present invention has particular application for manufacturing processes in an automotive manufacturing plant. However, as will be appreciated by those skilled in the art, the system and method of the invention may have application for other production facilities.
- As will be discussed in detail below, the present invention integrates physical and emulated automation components with data and information exchange between the virtual world and the real world. The invention provides an effective platform for developing and testing hardware components that are difficult or time consuming to simulate with existing resources and technology. The present invention also provides a design for automation systems and component compatibility and performance testing. With this approach, new system validation times can be reduced, and the launch quality of the new system can be improved. The present invention provides a hybrid environment that integrates physical and emulated automation components including data and information exchange between the virtual world and the real world. The technique can be used for automation system and component compatibility and performance testing. A generic process can be developed to decide whether it is effective to use hardware-in-the-loop (HIL) emulation for automation component compatibility and performance testing, and corresponding validation procedures are defined.
- The HIL emulation for automation component compatibility and performance testing provides a number of advantages including control validation for testing control logic on hybrid or simulated processes, hypothetical machine arrangements providing plug and play characteristics that make it possible to test various arrangements easily and the testing of new hardware where the new hardware can be plugged into an existing process simulation and run in real time to test its compatibility and performance and simulations can be plugged into an existing system and run in real time to test feasibility. The HIL emulation benefits include shorter system development cycles because the hard-to-model virtual component can be replaced by a real component, earlier testing system and feedback using virtual components along with real system set-up environment, lower cost for various scenario testing as the virtual models can be reused and duplicated seamlessly, and higher confidence on new system performance with usage of real network and standard interfaces. The real and virtual integration provides any level of virtual and real combination, seamless part handling and data exchange, plug and play modular characteristics, controller and components are unaware of the presence of other regions, real-time event-driven data handling, scalable and hierarchical, and standardized signal coupling and interface.
-
FIG. 1 is a flow chart diagram 10 showing a general overview of a process of the invention for determining whether an emulated process of a manufacturing process will include real machines, according to an embodiment of the present invention. The flow chart diagram shows two different scenarios for an integrated real and virtual manufacturing automation system and steps for setting up such a hybrid system. Both set-up scenarios enable an engineer to evaluate system performance or machine compatibility when new equipment is added to or is used to replace existing equipment in an existing manufacturing automation system. The system performs a machine compatibility test with the integrated real and virtual manufacturing automation system, which can be done either in the plant or the lab. However, when the test is done in the field, the physical machines will be in the majority of the integrated real and virtual manufacturing automation system. This method allows an engineer to test the new system machines with real production manufacturing automation system without concern of fidelity of the virtual models and incurring cost for transporting the real new system machines to the testing facility, or even higher cost for setting up a duplicate system in a prototype shop. - While in a lab setting and when the virtual model of new system machines is not available, system performance and machine compatibility testing can be carried out by integrating the new physical system machines with a perspective emulated manufacturing automation system that is a virtual model of an existing system and has been calibrated against the corresponding physical system. Furthermore, this method allows an engineer to conduct the system performance and machine compatibility using the earlier stage prototype of the new system machines before finalizing the system design.
- The process first selects a particular automation system or component that needs to be validated or emulated at
box 12. The process then determines whether complete and accurate emulation models are available for the particular system or component that is to be emulated at decision diamond 14. If all of the models that are necessary for the emulation are available at the decision diamond 14, then the process performs a 100% virtual emulation and validation of the system or component atbox 16. If all of the models are not available at the decision diamond 14, then the process determines whether a complete prototype of those systems or components that cannot be emulated are available at the plant location atdecision diamond 18. If the complete system or component is available at the plant at thedecision diamond 18, then the process will perform in field debugging and testing processes at the plant atbox 20. If a prototype of the system or component is not available at the plant at thedecision diamond 18, then the process prepares an HIL for testing atbox 22, and then determines if the new system or component is ready for lab testing atdecision diamond 24. - If the new system or component is not ready for lab testing at the
decision diamond 24, then the emulated system or component may be integrated with an existing physical system and process, starting atbox 26, which results in HIL emulation in the plant. Therefore, the actual physical machine can be provided at its normal location for automation system compatibility and performance testing. In this situation, most of the system exists in the physical world, with some virtual components. The process then selects a real and virtual world coupling method at the plant location atbox 28 to couple the physical system with a virtual component, such as an emulated robot on a computer. The process then configures an input/output (I/O) interface atbox 30 which is a bridge between the real and virtual world, and chooses a part passing method atbox 32 that determines how a real part location sensing to be coupled between a virtual machine and a real machine in the system. The process then synchronizes the real machine and the virtual world clocks atbox 34, and tests the component or system for performance and compatibility atbox 36. - If the new system or component is ready for lab testing at the
decision diamond 24, then the process performs the HIL emulation in the lab where the emulation is mostly in the virtual world with a few physical machines. As above, the process selects the real and virtual world coupling method atbox 40, configures the input/output interface atbox 42, determines how the part location information will be passed between the virtual world and the real world atbox 44, synchronizes the real world clocks and the virtual world clocks atbox 46 and tests the component or system for performance and compatibility atbox 48. -
FIG. 2 is a plan view of an exemplaryautomated manufacturing system 50 including tworeal manufacturing cells virtual manufacturing cell 54 linked together by aconveyer 58, where theconveyer 58 delivers apart 74 from cell to cell. Thesystem 50 is controlled by a programmable logic controller (PLC) 60 based on the status of various sensors, forexample sensors network 68 provided throughout thesystem 50, where other system status information is also provided on thenetwork 68. - A
real robot 70 is controlled by arobot controller 72 in thereal manufacturing cells part 74 from theconveyer 58 to amachine 76 that will perform an operation on thepart 74 and return it back to theconveyer 58. An end-effector 78 on therobot 70 may be directly controlled by therobot controller 72 or indirectly controlled by thePLC 60 depending on the system architecture. In the latter case, information, such as end-effector is in position, may have to be sent to thePLC 60 so that thePLC 60 can issue the necessary automation command to activate the end-effector 78 to pick up thepart 74. Similarly, when the end-effector 78 moves into a position to drop off thepart 74 at one of themachines 76, the opening of the end-effector 78 will be directly controlled by therobot controller 72 or thePLC 60 based on system architecture. When themachine 76 finishes its operation on thepart 74, therobot 70 will pick up thepart 74 from the correspondingmachine 76 and drop off thepart 74 at the original location on theconveyer 58. After thepart 74 has been placed back on theconveyer 58, theconveyer 58 will move the part 74 a step forward to thenext cell 54 so that the new operation cycle can start. - The
virtual cell 54 in thesystem 50 includes an emulated orvirtual robot 80 including a virtual end-effector sensor 96 instead of a real robot. To integrate the emulated orvirtual robot 80 with the real physical system, the corresponding sensor information that triggers the real robot operation is transmitted to therobot controller 72 and then the program that carries out the robot simulation and the status change information of thevirtual robot 80 when it drops off thepart 74 at the designated location is transmitted to thePLC 60 so that themachine 76 at that cell will function properly. -
FIGS. 3 and 4 provide two sensor connecting methods for the virtual end-effector sensor 96 between thevirtual robot 80 and thereal machine 76 in thecell 54 and the rest of the system. InFIG. 3 , a sensor 82 is directly attached to an automation controller, such as aPLC 90, through acontrol network 84. InFIG. 4 , asensor 92 is attached to alocal machine 88, such as a robot. With the system configuration ofFIG. 4 , thePLC 90 only controls the start of therobot 86 and only when the complete task of therobot 86 is finished, thePLC 90 will be informed that the next task is ready to be performed. The sensing signal from thesensor 92 and a state change of actuator, such as an end-effector 94, are not necessarily transmitted to thePLC 90. -
FIG. 5 is an illustration of an integrated real and virtualmanufacturing automation system 100 including complimentary pairs of physical sensors and actuators. Thesystem 100 includes areal world domain 102 including areal machine 106, such as a robot, and avirtual world domain 104 including avirtual world machine 108, where therobot 106 is a component of a manufacturing automation system. In this embodiment, thereal domain 102 and thevirtual domain 104 are electronically isolated. Signals passed between thereal world domain 102 and thevirtual world domain 104 is facilitated by matching an actuator with a complimentary sensor and a sensor with a complimentary actuator, as will be discussed in more detail below. - The
robot 106 and thevirtual machine 108 are controlled by aPLC 110 through an industrial network device and a corresponding input/output (I/O)module 112. An I/O interface card 114 transfers signals between thereal world domain 102 and thevirtual world domain 104. Thereal world domain 102 includes areal sensor 118 and areal actuator 116, where thereal sensor 118 provides a signal of the position of therobot 106 and thereal actuator 116 actuates therobot 106 to move it from position to position. Thereal world domain 102 also includes a complimentary pair of asensor 120 and anactuator 122 and a complimentary pair of an actuator 124 and asensor 126, where the sensors detect actuation of the actuator in the pair. Thevirtual world domain 104 includes avirtual sensor 128 that senses the position of themachine 108 and avirtual actuator 130 that virtually actuates themachine 108. Signals are said to be mapped between a pair of coupled sensors when the sensor signal from either a virtual sensor or a real sensor is used to override or by-pass the signal from a real sensor or a virtual sensor, respectively. - As discussed herein, complimentary sensors and actuators can be set forth as follows. For a photo-eye sensor, a complimentary actuator could be a blade mounted on a shaft of a stepper motor. By moving the blade into and out of the photo-eye path, a part passing through the path can be simulated. For a proximity sensor, the presence of a part can be simulated by moving a metallic object into and out of the sensing area of the sensor. For an actuator, such as a lift or a pusher, a complimentary sensor could be a pair of switches at two extreme positions of the actuator.
- There are two types of sensor and actuator coupling, namely, parallel coupling and serial coupling, sometimes referred to as direct coupling and sequence coupling, respectively. A virtual sensor coupled to a synchronized real sensor or vice versa is said to be coupled in parallel. Similarly, a virtual actuator coupled to a synchronized real actuator or vice versa is also said to be coupled in parallel. A sensor triggered by an immediate upstream actuator through a virtual or real process is said to be coupled in serial. If a sensor is coupled to an actuator using a virtual process, the coupling is referred to as virtual coupling. If a sensor is coupled to an actuator using a real process, the coupling is referred to as physical coupling.
- Sensor signals from the
real sensor 118 of the position of therobot 108 are directly connected to the I/O module 112 and signals from thevirtual sensor 128 become the output signals of theinterface card 114 and trigger the actuation of thereal actuator 122, whose state of change will be sensed by thereal sensor 120. The output signal of thereal sensor 120 is connected to the I/O module 112. In this way, thevirtual sensor 128 is coupled to thereal sensor 120 in parallel while thereal actuator 122 and thereal sensor 120 are coupled in serial. Similarly, an output signal from the I/O module 112 to therobot 106 is connected directly. For thevirtual world domain 104, the output signal of the I/O module 112 is connected to the real actuator 124, whose state change will be sensed by thecomplimentary sensor 126. The output of thesensor 126 is the input to theinterface card 114 and will drive thevirtual actuator 130 to alter its state. In this way, the real actuator 124 is coupled to thevirtual actuator 130 in parallel, while the real actuator 124 is coupled to the real sensor in serial. - In the event that the
real sensor 118 is not directly connected to thePLC 110, thecomplimentary actuator 126 is needed to capture the signal to relay it to thevirtual domain 104 through theinterface card 114. Similarly, theactuator 116 will have to be connected to a complimentary sensor, and then the state change of theactuator 116 will be captured and sent to thevirtual world 104 through theinterface card 114. -
FIG. 6 is a plan view of an integrated real and virtualmanufacturing automation system 140 providing a direct electrical connection between thereal domain 102 and thevirtual domain 104, where like elements to thesystem 100 are identified by the same reference numeral. In thesystem 140, the complimentary pairs of sensors and actuators between the I/O module 112 and the I/O interface card 114 are replaced with a voltage-translation device 142 so that the real and virtual portions of thesystem 140 are linked directly together. The I/O interface card 114 will facilitate the signal passing between the real and virtual domains of thesystem 140 and the voltage-translation device 142 will match the difference in the analog signal levels between thevirtual cell 104 and the real world machines. Sensor signals from therobot 106 are directly coupled to the I/O module 112, while signals from thevirtual sensor 128 become the output signals of theinterface card 114, and using the voltage-translation device 142, connects directly to the input of the I/O module 112. Similarly, the output signal from the I/O module 112 to therobot 106 through theactuator 116 are connected directly. For thevirtual domain 104 of thesystem 140, the output signal of the I/O module 112 is connected directly to theinterface card 114 through the voltage-translation device 142, which converts the output of the I/O module 112 to a voltage acceptable for theinterface card 114. -
FIG. 7 is a plan view of amanufacturing process 150 for apart 152 moving through and betweenreal world cells 154 includingreal world machines 156, such as a real robot, andvirtual world cells 158 including virtual machines, such as avirtual robot 160, that is a model of an actual robot. When the part is in thereal world cell 154, thereal robot 156 can work on thepart 152 as a normal process. Apart presence token 164 will be on to indicate that thereal part 152 is available in the process. When thepart 152 moves into thevirtual world cell 158, thepart presence token 164 will be off to indicate that there is no physical part in the process anymore, and thevirtual robot 160 will work on a virtual part. When a real robot receives a virtual part from the precedingmachine 160, it needs to think that thepart 152 is actually there. Therefore, the virtual part is provided in thereal cells 154 that provides a signal to a coupled real sensor telling themachine 156 that thepart 152 is actually in its proper orientation, even though it may not be and thepart presence token 164 is off. - In other situations, the
actual part 152 may be worked on by themachine 156 in thereal cell 154, where thevirtual part 162 is inactive.Sensors 166 are provided between thereal cells 154 and thevirtual cells 158 so that the process knows when thepart 152 is moved from areal cell 154 to avirtual cell 158 and from avirtual cell 158 to areal cell 154. - Because the
part 152 cannot be physically transported across the boundary of the virtual and real domains, thepart 152 moving from the real world to virtual world and returning to the real world from the virtual world for the given example set-up can only be emulated by the following. When a part is moving from the real world to the virtual world, the actual part remains at the current location and a virtual part is created in the virtual world when the sensor information is received that the part is moved into the virtual world. For the opposite direction, when the part is moving from the virtual world to the real world, the virtual part in the virtual world is destroyed and at the same time signals are provided that the part is at its designated location in the real world. Obviously, this scenario only works under the condition that the part remains at the same location before moving to the virtual world from the real world and after returning to the real world from the virtual world. Therefore, in the case of the middle cell (cell 54 inFIG. 2 ), the real machine (machine 76 inFIG. 2 ) will not see the real part that remains on the conveyer (conveyor 58 inFIG. 2 ) due to the presence of the virtual robot between the real machine and the conveyer. To ensure the proper working order of the system, thepart presence token 164 is used and depending on the condition, thepart presence token 164 will send proper part presence signals from the virtual world or the real sensors to the PLC. This practice of part sensing signal mapping will be discussed further below. - The
part presence token 164 works according to the rules discussed above. However, in other situations, theactual part 152 may be worked on by the machine in thereal cell 154, where the virtual part is inactive. It is assumed that sensors are provided between thereal cells 154 and thevirtual cells 158 so that the process knows when thepart 152 is moved from the real world to the virtual world and from the virtual world to the real world. If there is no sensor available at the interface of thereal cells 154 and thevirtual cells 158, a sensor needs to be added to the integrated virtual and real manufacturing automation system. -
FIG. 8 is a flow chart diagram 170 of a decision tree for selecting a proper sequence (serial) coupling method for handling the actuator action to the sensor sensing. In general, there are no sensors and actuators duplicated in the integrated virtual or real manufacturing automation system, except for the boundary region. The actuation of the actuator will eventually lead to the state change of the preceding sensor, directly or indirectly, and through a part or not through a part. For example, a motor encoder sensor will immediately sense the motion of the motor. However, the up and down position sensors of a lift only change their state when the lift arrives at and leaves from a corresponding position, not when the lift is in motion between the two corresponding positions. Another example of this scenario is that sensors on the two downstream branches of a sorting machine alter their state when the part travels through its corresponding branch as a consequence of the upstream actuator or sorter. Therefore, there is a need to consider the sequence coupling of the actuator action to the sensor sensing. Some of the actuator actions to the sensor sensing involve the part, while others do not. To address those issues, it is necessary that some sensors and actuators be duplicated in the virtual system so that the integrated virtual real system will function as a real system does. - The first step in the process is to analyze the interactions between sensor and actuator coupling methods at
box 172. From this analysis, the process determines whether the actuator and sensor are directly coupled at decision diamond 174, such as a lifter with a position sensor, or coupled indirectly, such as a pusher with a photo-eye sensor. If the coupling is direct at the decision diamond 174, then the process uses non-part coupling atbox 176. The process then determines whether duplicated real actuator and sensor coupling is available atdecision diamond 178. If duplicated real actuator and sensor coupling is available at thedecision diamond 178, then the process uses physical coupling without a part atbox 180. However, if duplicated real actuator and sensor coupling is not available at thedecision diamond 178, then the process determines whether duplicated virtual actuator and sensor coupling is available atdecision diamond 182. If duplicated virtual actuator and sensor coupling is available at thedecision diamond 182, then the process provides virtual coupling without a part atbox 184. If duplicated virtual actuator and sensor coupling is not available at thedecision diamond 182, then the process uses other non-hybrid testing techniques atbox 186. - The same process is used if the actuator does not directly interact with the sensor at the decision diamond 174, where a part is involved in the coupling at
box 188. The process determines whether duplicated real actuator, part and sensor coupling is available atdecision diamond 190, and if so, provides physical coupling with the part atbox 192. If duplicated real actuator, part and sensor coupling is not available at thedecision diamond 190, then the process determines whether duplicated virtual actuator, part and sensor coupling is available atdecision diamond 194, and if so, provides virtual coupling with the part atbox 196. Otherwise, the process goes to thebox 186 to provide other non-hybrid testing. -
FIG. 9 is a flow chart diagram 200 showing a decision tree process for selecting the proper sensor/actuator coupling when there is no part involved in an actuator action and a sensor sensing sequence. In general, this problem is divided into two groups, namely, coupling purely inside the real or virtual domains and coupling across the boundary of the real and virtual domains atboxes box 202, there is no special attention required as the actuator action to the sensor sensing sequence coupling happens naturally as the process flow. Particularly, in the real domain atbox 206, there is direct physical interaction between the sensor and actuator atbox 208. In the virtual domain atbox 210, there is direct virtual interaction between the actuator and sensor atbox 212. - For coupling across the boundary between the real and virtual domains at the
box 204 there are four different scenarios. For coupling from the real domain to the virtual domain atbox 214, physical coupling from the real domain to the virtual domain is provided atbox 216. This process can add a mapped physical sensor to couple with physical actuator and overwrite corresponding virtual sensor to drive the emulation, as will be discussed below with reference toFIG. 11 . Also for coupling from the real domain to the virtual domain, virtual coupling is provided atbox 218. This process can add a mapped virtual actuator that synchronizes with a real actuator to couple with a virtual sensor and drive the emulation, as will be discussed below with reference toFIG. 12 . Coupling from the virtual domain to the real domain is provided atbox 220, which is broken down to physical coupling atbox 222 and virtual coupling atbox 224. For the physical coupling from the virtual domain to the real domain, the process can add a mapped real actuator that synchronizes with a virtual actuator to couple a real sensor to trigger a real domain execution, as will be discussed below with reference toFIG. 13 . For the virtual coupling from the virtual domain to the real domain, the process can add a mapped virtual sensor coupling with a virtual actuator to overwrite a corresponding real sensor to trigger a real side execution, as will be discussed below with reference toFIG. 14 . -
FIG. 10 is a flow chart diagram 230 showing a decision tree process for selecting a proper coupling method for handling a part transferred across the virtual and real domains in an integrated system as provided atbox 232. This part handling problem can be provided into two groups, namely, part handling purely inside the real or virtual domain atbox 234 and a part traveling across the boundary between the real and virtual domains atbox 236. For part handling inside either the real or virtual domain at thebox 234, the decision can be further divided based on the absence or presence of a real part. When the part is present in the real domain atbox 238, and there is a real part atbox 240, then normal processes are used atbox 242. If a virtual part is being used in the real domain atbox 244, the process duplicates the real actuator and real sensor in the virtual domain using information obtained from the virtual actuator and virtual part interaction, and uses the duplicate sensor signal to override the signal from non-functional real sensor due to the absence of a real part atbox 246. If the part is in the virtual domain atbox 248 and a virtual part is being used atbox 250, then normal processes are used atbox 252. The situation where a real part travels in the virtual domain does not exist so that it is not applicable atbox 256. - A part traveling across the boundary between the virtual and real domains can either travel from the real domain to the virtual domain at
box 258 or travel from the virtual domain to the real domain atbox 260. For a real part traveling from the real domain to the virtual domain atbox 262, physical coupling is used atbox 264. For a virtual part traveling from the real domain to the virtual domain atbox 266, a virtual coupling method is used with a virtual part atbox 268. For a virtual part traveling from the virtual domain to the real domain atbox 270, a virtual coupling method is used with a virtual part atbox 272. The situation where a real part travels from the virtual domain to the real domain atbox 274 does not exist so that it is not applicable atbox 276. -
FIG. 11 is a plan view of an integrated real and virtualmanufacturing automation system 280 showing a physically coupled sensor/actuator for a part flow from thereal domain 102 to thevirtual domain 104, particularly, a part flow from therobot 106 to thevirtual machine 108, without a part, where like elements to thesystem 100 are identified by the same reference numeral. Therobot 106 uses thereal actuator 116 and thevirtual machine 108 uses thevirtual sensor 128. This physical coupling process starts with providing a duplicatedreal sensor 282 that is coupled in serial to thereal actuator 116 through acoupler interface 284. The output signal from the I/O module 112 to thereal actuator 116 would normally be sensed by thereal sensor 118 and sent to thevirtual sensor 128 through theinterface card 114. In this embodiment, the signal from the duplicatereal sensor 282 is instead sent to aswitch 286 that sends the sensor signal to the I/O module 112 so that thevirtual sensor 128 is by-passed and the signal from thevirtual sensor 128 is overridden by the signal from the duplicatedsensor 282. -
FIG. 12 is a plan view of an integrated real and virtual manufacturing automatedsystem 290 showing a virtually coupled sensor and actuator for a part flow from thereal domain 102 to thevirtual domain 104, particularly, a part flow from therobot 106 to thevirtual machine 108, without a physical part, where like elements to those in thesystem 100 are identified by the same reference numeral. Therobot 106 uses thereal actuator 116 and thevirtual machine 108 uses thevirtual sensor 128. This virtual coupling process starts with providing a duplicatedvirtual actuator 292 that is coupled to thevirtual sensor 130 by acoupling interface 294. The output signal from the I/O module 112 that would normally be intended for thereal actuator 116 is switched by aswitch 296 to theinterface card 114 to be received by the duplicatedvirtual actuator 292. Thevirtual actuator 292 subsequently provides the state change for thevirtual sensor 128. -
FIG. 13 is a plan view of an integrated real and virtualmanufacturing automation system 300 showing physically coupled sensors and actuators with a part flow from thevirtual domain 104 to thereal domain 102 without a physical part, where like elements to thesystem 100 are identified by the same reference numeral. Therobot 108 uses thereal sensor 118 and thevirtual machine 108 uses thevirtual actuator 130. The physical coupling process starts by adding a duplicatedreal actuator 302 that is coupled to thereal sensor 118 through acoupling interface 304. The output from the I/O module 112 that would normally be sent to thevirtual actuator 130 is by-passed by aswitch 306 so that the output from the I/O module 112 is sent to the duplicatedreal actuator 302. -
FIG. 14 is a plan view of an integrated real and virtualmanufacturing automation system 310 showing virtual coupling between a sensor and an actuator for a virtual part flow from thevirtual domain 104 to thereal domain 102, where like elements to thesystem 100 are identified by the same reference numeral. Therobot 106 uses thereal sensor 118 and thevirtual machine 108 uses thevirtual actuator 130. This virtual coupling process employs a duplicatedvirtual sensor 312 that is coupled to thevirtual actuator 130 through a virtual coupling interface 314. Signals from thevirtual actuator 130 are sensed by the duplicatedactuator 312, and the signals from theduplicate sensor 312 are sent as an input to the I/O module 112 through aswitch 316 that by-passes thereal sensor 118. -
FIG. 15 is a plan view of an integrated real and virtualmanufacturing automation system 320 showing a physical coupling between a sensor and an actuator for a part flow from thereal domain 102 to thevirtual domain 104 using a real part, where like elements to thesystem 100 are identified by the same reference numeral. Therobot 106 uses thereal actuator 116 and thevirtual machine 108 uses thevirtual sensor 128. Thereal actuator 116 interacts with areal part 324. The physical coupling process starts with providing a duplicatedreal sensor 326 that is coupled to thereal actuator 116 indirectly through thepart 324 andcoupling interfaces sensor 326 are sent to theinterface card 114 that triggers the birth of avirtual part 332. Thevirtual part 332 is coupled to thevirtual sensor 128 through acoupling interface 334. -
FIG. 16 is a plan view of an integrated real and virtualmanufacturing automation system 340 showing virtual coupling between a sensor and an actuator for a part flow from thereal domain 102 to thevirtual domain 104 using a virtual part, where like elements to thesystem 100 are identified by the same reference numeral. Therobot 106 uses thereal actuator 116 and thevirtual machine 108 uses thevirtual sensor 128. Thereal actuator 116 interacts with avirtual part 342. This embodiment for the virtual coupling process provides a duplicatedvirtual actuator 344 that is coupled to thevirtual sensor 128 indirectly through thevirtual part 342 andcoupling interfaces O module 112 that were intended for thereal actuator 116 and by-passed by aswitch 350 so that the signals are sent to theinterface card 114 to drive thevirtual actuator 344 and trigger thevirtual machine 108 to perform some action. -
FIG. 17 is a plan view of an integrated real and virtualmanufacturing automation system 360 showing virtual coupling between a sensor and actuator for a virtual part flow from thevirtual domain 104 to thereal domain 102, where like elements to thesystem 100 are identified by the same reference numeral. Therobot 106 uses thereal sensor 118 and thevirtual machine 108 uses thevirtual actuator 130. Thevirtual actuator 130 interacts with avirtual part 362. The virtual coupling provides a duplicatedvirtual sensor 364 that is coupled to thevirtual actuator 130 indirectly through thevirtual part 362 andcoupling interfaces virtual sensor 364 and sent to theinterface card 114 as an output, which is used to replace the signal from thereal sensor 118 by aswitch 370. -
FIG. 18 is a plan view of an integrated real and virtualmanufacturing automation system 380 showing sensors and actuators in thereal domain 102 interacting with a virtual part, where like elements to thesystem 100 are identified by the same reference numeral. A second real world machine, such as arobot 382, is provided in thereal domain 102. Areal sensor 384 receives signals from therobot 382 and thereal actuator 116 controls the actuation of therobot 106. The coupling technique is applicable to the situation where the real part does not exist in the real world. To facilitate the proper sensing of the virtual part by therobot 382, avirtual part 388 in thevirtual domain 104 is provided. Avirtual actuator 390 is provided to duplicate thereal actuator 116 and avirtual sensor 392 is provided to duplicate thereal sensor 384. Thevirtual actuator 390 is synchronized with thereal actuator 116 through theinterface card 114 and thevirtual sensor 392 overwrites thereal sensor 384 through theinterface card 114. The coupling is achieved indirectly between thevirtual actuator 390 and thevirtual sensor 392 through thevirtual part 388 andcoupling interfaces robots - The foregoing discussion discloses and describes merely exemplary embodiments of the present invention. One skilled in the art will readily recognize from such discussion and from the accompanying drawings and claims that various changes, modifications and variations can be made therein without departing from the spirit and scope of the invention as defined in the following claims.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/237,894 US20090088884A1 (en) | 2007-09-28 | 2008-09-25 | Manufacturing automation system components compatibility and performance testing with integrated virtual and real environment |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US97610107P | 2007-09-28 | 2007-09-28 | |
US12/237,894 US20090088884A1 (en) | 2007-09-28 | 2008-09-25 | Manufacturing automation system components compatibility and performance testing with integrated virtual and real environment |
Publications (1)
Publication Number | Publication Date |
---|---|
US20090088884A1 true US20090088884A1 (en) | 2009-04-02 |
Family
ID=40509283
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/237,894 Abandoned US20090088884A1 (en) | 2007-09-28 | 2008-09-25 | Manufacturing automation system components compatibility and performance testing with integrated virtual and real environment |
US12/237,944 Expired - Fee Related US7996113B2 (en) | 2007-09-28 | 2008-09-25 | Methods of integrating real and virtual world using virtual sensor/actuator |
US12/237,923 Abandoned US20090089700A1 (en) | 2007-09-28 | 2008-09-25 | Methods of integrating real and virtual world using physical sensor/actuator |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/237,944 Expired - Fee Related US7996113B2 (en) | 2007-09-28 | 2008-09-25 | Methods of integrating real and virtual world using virtual sensor/actuator |
US12/237,923 Abandoned US20090089700A1 (en) | 2007-09-28 | 2008-09-25 | Methods of integrating real and virtual world using physical sensor/actuator |
Country Status (3)
Country | Link |
---|---|
US (3) | US20090088884A1 (en) |
DE (1) | DE112008002573T5 (en) |
WO (1) | WO2009045865A2 (en) |
Cited By (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090319068A1 (en) * | 2004-02-06 | 2009-12-24 | Sager Robert D | System and method for mass custom manufacturing of dental crowns and crown components |
US20100174395A1 (en) * | 2009-01-06 | 2010-07-08 | Gm Global Technology Operations, Inc. | Integrated testing system and method for validation of a manufacturing automation system |
EP2508954A1 (en) * | 2011-04-06 | 2012-10-10 | ABB Technology AG | System and method for the configuration of a clustered simulation network |
WO2012092476A3 (en) * | 2010-12-29 | 2012-12-27 | Athens Group Holdings, Llc | Method and system for drilling rig testing using virtualized components |
US20130024031A1 (en) * | 2011-07-18 | 2013-01-24 | GM Global Technology Operations LLC | Statistical method to obtain high accuracy in forecasting plant energy use |
EP2755137A1 (en) * | 2013-01-14 | 2014-07-16 | BAE Systems PLC | Avionics data testing |
FR3019920A1 (en) * | 2014-04-14 | 2015-10-16 | Thales Sa | SUBSTITUTION EQUIPMENT FOR AVIONIC DISPLAY EQUIPMENT; ASSOCIATED SYSTEM AND MODULE |
US20150331403A1 (en) * | 2008-12-10 | 2015-11-19 | International Business Machines Corporation | Network driven actuator mapping agent and bus and method of use |
US9519572B2 (en) | 2014-03-31 | 2016-12-13 | International Business Machines Corporation | Creating a software performance testing environment on a virtual machine system |
EP3376325A1 (en) * | 2017-03-16 | 2018-09-19 | Siemens Aktiengesellschaft | Development of control applications in augmented reality environment |
EP3410260A3 (en) * | 2017-05-11 | 2019-02-27 | Günther Battenberg | Method for haptic testing of an object |
CN109696915A (en) * | 2019-01-07 | 2019-04-30 | 上海托华机器人有限公司 | A kind of test method and system |
US10503160B2 (en) | 2016-06-10 | 2019-12-10 | Honeywell International Inc. | Integrated testing mechanism for industrial process control and automation systems |
US11498223B2 (en) * | 2018-05-09 | 2022-11-15 | Sony Interactive Entertainment Inc. | Apparatus control systems and method |
Families Citing this family (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10719070B2 (en) * | 2007-09-25 | 2020-07-21 | Siemens Aktiengesellschaft | System and method for modeling signal flows in automation technology equipment |
KR20090067822A (en) * | 2007-12-21 | 2009-06-25 | 삼성전자주식회사 | System for making mixed world reflecting real states and method for embodying it |
US20100017026A1 (en) * | 2008-07-21 | 2010-01-21 | Honeywell International Inc. | Robotic system with simulation and mission partitions |
US8352871B2 (en) | 2008-12-04 | 2013-01-08 | International Business Machines Corporation | System and method for virtual environment preservation based on automated item reduction |
US9288242B2 (en) | 2009-01-15 | 2016-03-15 | Social Communications Company | Bridging physical and virtual spaces |
KR100981330B1 (en) * | 2009-03-25 | 2010-09-10 | 주식회사 유디엠텍 | Simulation apparatus and method for testing plc code of generating input/output model automatically |
DE102011014299A1 (en) * | 2011-03-17 | 2012-09-20 | Kuka Roboter Gmbh | Method and means for controlling an automation device, in particular a robot |
US8600715B2 (en) | 2011-09-13 | 2013-12-03 | The Procter & Gamble Company | Methods for machine emulation and process response prediction |
US8660830B2 (en) | 2011-09-13 | 2014-02-25 | The Procter & Gamble Company | Machine emulator methods |
US8600714B2 (en) | 2011-09-13 | 2013-12-03 | The Procter & Gamble Company | Systems for machine emulation and process response prediction |
US8660829B2 (en) | 2011-09-13 | 2014-02-25 | The Procter & Gamble Company | Machine emulator machines |
US8670965B2 (en) | 2011-09-13 | 2014-03-11 | The Procter & Gamble Company | Machine emulator products |
US9505130B2 (en) * | 2012-07-13 | 2016-11-29 | General Electric Company | System and method for performing remote welding operations on an apparatus |
US9720393B2 (en) | 2012-08-31 | 2017-08-01 | P.C. Automax Inc. | Automation system and method of manufacturing product using automated equipment |
WO2014138949A1 (en) * | 2013-03-15 | 2014-09-18 | P.C. Automax Inc. | Automation system and method of manufacturing products using automated equipment |
EP2778816B1 (en) * | 2013-03-12 | 2015-10-07 | ABB Technology AG | System and method for testing a distributed control system of an industrial plant |
FR3009548B1 (en) * | 2013-08-07 | 2017-04-28 | Solystic | CONVEYING SYSTEM COMPRISING A TRACKING OF OBJECTS CONVOYED BY IMAGING AND VIRTUAL SENSORS |
US9705737B2 (en) | 2014-07-21 | 2017-07-11 | Cisco Technology, Inc. | Deterministic control loop scheduling |
EP3001313A1 (en) * | 2014-09-23 | 2016-03-30 | dSPACE digital signal processing and control engineering GmbH | Methods for simulating an application program of an electronic control device on a computer |
EP3213157B1 (en) * | 2014-10-30 | 2024-03-20 | Siemens Aktiengesellschaft | Using soft-sensors in a programmable logic controller |
CN104460349B (en) * | 2014-11-28 | 2017-01-18 | 北京经纬恒润科技有限公司 | Real automobile in-the-loop simulation testing method, real-time simulation machine and system |
CA2882968C (en) * | 2015-02-23 | 2023-04-25 | Sulfur Heron Cognitive Systems Inc. | Facilitating generation of autonomous control information |
US10853104B2 (en) * | 2015-02-27 | 2020-12-01 | Plasma Business Intelligence, Inc. | Virtual environment for simulating a real-world environment with a large number of virtual and real connected devices |
US10054912B2 (en) * | 2015-10-19 | 2018-08-21 | General Electric Company | Method and apparatus for auto synchronizing control system data structures with model composition |
US10500726B2 (en) * | 2016-04-25 | 2019-12-10 | Kindred Systems Inc. | Facilitating device control |
EP3507672B1 (en) | 2016-09-05 | 2024-07-03 | IOT.nxt BV | Software-defined device interface system and method |
US10828716B2 (en) | 2017-06-19 | 2020-11-10 | Lincoln Global, Inc. | Systems and methods for real time, long distance, remote welding |
EP3729216B1 (en) * | 2018-01-22 | 2024-04-24 | Siemens Aktiengesellschaft | Skill matching for control of an industrial production machine |
US12031882B2 (en) * | 2019-06-26 | 2024-07-09 | Movensys Corporation | Methods and systems for testing robotic systems in an integrated physical and simulated environment |
CN114787838A (en) | 2019-11-12 | 2022-07-22 | 光明机器公司 | Software defined manufacturing/assembly system |
US11842455B1 (en) | 2022-06-20 | 2023-12-12 | International Business Machines Corporation | Synchronizing physical and virtual environments using quantum entanglement |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040186697A1 (en) * | 2001-06-13 | 2004-09-23 | Kurt Schreier | Method and system for assisting in the planning of manufacturing facilities |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001344472A (en) * | 2000-05-31 | 2001-12-14 | Toshiba Corp | Method and system for, electronic commerce of semiconductor product |
JP4713018B2 (en) * | 2001-06-11 | 2011-06-29 | 大日本印刷株式会社 | Production plan adjustment system and method |
JP4788066B2 (en) * | 2001-06-18 | 2011-10-05 | 大日本印刷株式会社 | Material ordering system |
US7308327B2 (en) * | 2006-05-12 | 2007-12-11 | Ford Motor Company | Method of application protocol monitoring for programmable logic controllers |
-
2008
- 2008-09-25 US US12/237,894 patent/US20090088884A1/en not_active Abandoned
- 2008-09-25 US US12/237,944 patent/US7996113B2/en not_active Expired - Fee Related
- 2008-09-25 US US12/237,923 patent/US20090089700A1/en not_active Abandoned
- 2008-09-26 DE DE112008002573T patent/DE112008002573T5/en not_active Withdrawn
- 2008-09-26 WO PCT/US2008/077767 patent/WO2009045865A2/en active Application Filing
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040186697A1 (en) * | 2001-06-13 | 2004-09-23 | Kurt Schreier | Method and system for assisting in the planning of manufacturing facilities |
Non-Patent Citations (1)
Title |
---|
Fangming Gu, "Hardware-In-Loop for Manufacturing Automation Control: Current Status and Identified Needs", IEEE conference, Sept 22-25, 2007 * |
Cited By (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090319068A1 (en) * | 2004-02-06 | 2009-12-24 | Sager Robert D | System and method for mass custom manufacturing of dental crowns and crown components |
US8751031B2 (en) * | 2004-02-06 | 2014-06-10 | Zircore, Llc | System and method for mass custom manufacturing of dental crowns and crown components |
US10295968B2 (en) * | 2008-12-10 | 2019-05-21 | International Business Machines Corporation | Network driven actuator mapping agent and bus and method of use |
US10712721B2 (en) | 2008-12-10 | 2020-07-14 | International Business Machines Corporation | Network driven actuator mapping agent and bus and method of use |
US20150331403A1 (en) * | 2008-12-10 | 2015-11-19 | International Business Machines Corporation | Network driven actuator mapping agent and bus and method of use |
US20100174395A1 (en) * | 2009-01-06 | 2010-07-08 | Gm Global Technology Operations, Inc. | Integrated testing system and method for validation of a manufacturing automation system |
US7974725B2 (en) * | 2009-01-06 | 2011-07-05 | GM Global Technology Operations LLC | Integrated testing system and method for validation of a manufacturing automation system |
GB2503116A (en) * | 2010-12-29 | 2013-12-18 | Athens Group Holdings Llc | Method and system for drilling rig testing using virtualized components |
US8793114B2 (en) | 2010-12-29 | 2014-07-29 | Athens Group Holdings Llc | Method and system for drilling rig testing using virtualized components |
WO2012092476A3 (en) * | 2010-12-29 | 2012-12-27 | Athens Group Holdings, Llc | Method and system for drilling rig testing using virtualized components |
WO2012136326A1 (en) * | 2011-04-06 | 2012-10-11 | Abb Technology Ag | System and method for the configuration of a clustered simulation network |
EP2508954A1 (en) * | 2011-04-06 | 2012-10-10 | ABB Technology AG | System and method for the configuration of a clustered simulation network |
US8606421B2 (en) * | 2011-07-18 | 2013-12-10 | GM Global Technology Operations LLC | Statistical method to obtain high accuracy in forecasting plant energy use |
US20130024031A1 (en) * | 2011-07-18 | 2013-01-24 | GM Global Technology Operations LLC | Statistical method to obtain high accuracy in forecasting plant energy use |
EP2755137A1 (en) * | 2013-01-14 | 2014-07-16 | BAE Systems PLC | Avionics data testing |
US9519572B2 (en) | 2014-03-31 | 2016-12-13 | International Business Machines Corporation | Creating a software performance testing environment on a virtual machine system |
US9519573B2 (en) | 2014-03-31 | 2016-12-13 | International Business Machines Corporation | Creating a software performance testing environment on a virtual machine system |
US9921940B2 (en) | 2014-03-31 | 2018-03-20 | International Business Machines Corporation | Creating a software performance testing environment on a virtual machine system |
US9921939B2 (en) | 2014-03-31 | 2018-03-20 | International Business Machines Corporation | Creating a software performance testing environment on a virtual machine system |
FR3019920A1 (en) * | 2014-04-14 | 2015-10-16 | Thales Sa | SUBSTITUTION EQUIPMENT FOR AVIONIC DISPLAY EQUIPMENT; ASSOCIATED SYSTEM AND MODULE |
US10503160B2 (en) | 2016-06-10 | 2019-12-10 | Honeywell International Inc. | Integrated testing mechanism for industrial process control and automation systems |
CN108628595A (en) * | 2017-03-16 | 2018-10-09 | 西门子股份公司 | The exploitation of control application program under augmented reality environment |
EP3376325A1 (en) * | 2017-03-16 | 2018-09-19 | Siemens Aktiengesellschaft | Development of control applications in augmented reality environment |
US10782668B2 (en) | 2017-03-16 | 2020-09-22 | Siemens Aktiengesellschaft | Development of control applications in augmented reality environment |
EP3410260A3 (en) * | 2017-05-11 | 2019-02-27 | Günther Battenberg | Method for haptic testing of an object |
US11498223B2 (en) * | 2018-05-09 | 2022-11-15 | Sony Interactive Entertainment Inc. | Apparatus control systems and method |
CN109696915A (en) * | 2019-01-07 | 2019-04-30 | 上海托华机器人有限公司 | A kind of test method and system |
Also Published As
Publication number | Publication date |
---|---|
US7996113B2 (en) | 2011-08-09 |
WO2009045865A2 (en) | 2009-04-09 |
US20090089700A1 (en) | 2009-04-02 |
DE112008002573T5 (en) | 2010-07-15 |
WO2009045865A3 (en) | 2009-05-22 |
US20090088885A1 (en) | 2009-04-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7996113B2 (en) | Methods of integrating real and virtual world using virtual sensor/actuator | |
Mykoniatis et al. | A digital twin emulator of a modular production system using a data-driven hybrid modeling and simulation approach | |
EP3819091A1 (en) | Robotic digital twin control with industrial context simulation | |
Kim et al. | A modular factory testbed for the rapid reconfiguration of manufacturing systems | |
CN113260935A (en) | Method and device for computer-aided simulation of a modular technical system | |
Kowal et al. | Delta modeling for variant-rich and evolving manufacturing systems | |
EP3121666A1 (en) | Modular test platform | |
KR20040007701A (en) | Method and system for assisting in the planning of manufacturing facilities | |
US8170861B2 (en) | Method for distributed hybrid emulation of manufacturing systems | |
Putman et al. | Virtual fusion: a hybrid environment for improved commissioning in manufacturing systems | |
CN104317259B (en) | A kind of method for establishing PLC/DCS platform device logical models | |
Dzinic et al. | Simulation-based verification of PLC programs | |
Brecher et al. | Model-based control of a handling system with SysML | |
US10719070B2 (en) | System and method for modeling signal flows in automation technology equipment | |
Putman et al. | Virtual Fusion: Integrating Virtual Components into a Physical Manufacturing System | |
Neugebauer et al. | Virtual-reality-based simulation of NC programs for milling machines | |
Kim et al. | Adaptive disassembly sequence control by using product and system information | |
Rodriguez-Guerra et al. | A methodology for real-time HiL validation of hydraulic-press controllers based on novel modeling techniques | |
Hill et al. | Virtual Commissioning Implementation of Side Panel Assembly Process for Automotive Shop Floor | |
DUMITRAȘCU et al. | Virtual commissioning of a robotic cell using tecnomatix process simulate | |
Fält et al. | Emulation of a production cell-Developing a virtual commissioning model in a concurrent environment | |
Harrison et al. | Virtual fusion: Hybrid process simulation and emulation-in-the-loop | |
Kim et al. | Development of flexible manufacturing system using virtual manufacturing paradigm | |
Lin et al. | Model-based customisation of intelligent mechatronic systems using SysML | |
Malik | Simulation Based High Fidelity Digital Twins of Manufacturing Systems: An Application Model and Industrial Use Case |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: GM GLOBAL TECHNOLOGY OPERATIONS, INC., MICHIGAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YUAN, CHENGYIN;GU, FANGMING;BILLER, STEPHAN R.;AND OTHERS;REEL/FRAME:021873/0892;SIGNING DATES FROM 20080929 TO 20081001 |
|
AS | Assignment |
Owner name: UNITED STATES DEPARTMENT OF THE TREASURY, DISTRICT Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:022201/0448 Effective date: 20081231 Owner name: UNITED STATES DEPARTMENT OF THE TREASURY,DISTRICT Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:022201/0448 Effective date: 20081231 |
|
AS | Assignment |
Owner name: CITICORP USA, INC. AS AGENT FOR BANK PRIORITY SECU Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:022554/0538 Effective date: 20090409 Owner name: CITICORP USA, INC. AS AGENT FOR HEDGE PRIORITY SEC Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:022554/0538 Effective date: 20090409 |
|
AS | Assignment |
Owner name: GM GLOBAL TECHNOLOGY OPERATIONS, INC., MICHIGAN Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:UNITED STATES DEPARTMENT OF THE TREASURY;REEL/FRAME:023126/0914 Effective date: 20090709 Owner name: GM GLOBAL TECHNOLOGY OPERATIONS, INC., MICHIGAN Free format text: RELEASE BY SECURED PARTY;ASSIGNORS:CITICORP USA, INC. AS AGENT FOR BANK PRIORITY SECURED PARTIES;CITICORP USA, INC. AS AGENT FOR HEDGE PRIORITY SECURED PARTIES;REEL/FRAME:023155/0769 Effective date: 20090814 Owner name: GM GLOBAL TECHNOLOGY OPERATIONS, INC.,MICHIGAN Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:UNITED STATES DEPARTMENT OF THE TREASURY;REEL/FRAME:023126/0914 Effective date: 20090709 Owner name: GM GLOBAL TECHNOLOGY OPERATIONS, INC.,MICHIGAN Free format text: RELEASE BY SECURED PARTY;ASSIGNORS:CITICORP USA, INC. AS AGENT FOR BANK PRIORITY SECURED PARTIES;CITICORP USA, INC. AS AGENT FOR HEDGE PRIORITY SECURED PARTIES;REEL/FRAME:023155/0769 Effective date: 20090814 |
|
AS | Assignment |
Owner name: UNITED STATES DEPARTMENT OF THE TREASURY, DISTRICT Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:023156/0313 Effective date: 20090710 Owner name: UNITED STATES DEPARTMENT OF THE TREASURY,DISTRICT Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:023156/0313 Effective date: 20090710 |
|
AS | Assignment |
Owner name: UAW RETIREE MEDICAL BENEFITS TRUST, MICHIGAN Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:023162/0237 Effective date: 20090710 Owner name: UAW RETIREE MEDICAL BENEFITS TRUST,MICHIGAN Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:023162/0237 Effective date: 20090710 |
|
AS | Assignment |
Owner name: GM GLOBAL TECHNOLOGY OPERATIONS, INC., MICHIGAN Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:UNITED STATES DEPARTMENT OF THE TREASURY;REEL/FRAME:025245/0909 Effective date: 20100420 |
|
AS | Assignment |
Owner name: GM GLOBAL TECHNOLOGY OPERATIONS, INC., MICHIGAN Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:UAW RETIREE MEDICAL BENEFITS TRUST;REEL/FRAME:025315/0046 Effective date: 20101026 |
|
AS | Assignment |
Owner name: WILMINGTON TRUST COMPANY, DELAWARE Free format text: SECURITY AGREEMENT;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:025324/0515 Effective date: 20101027 |
|
AS | Assignment |
Owner name: GM GLOBAL TECHNOLOGY OPERATIONS LLC, MICHIGAN Free format text: CHANGE OF NAME;ASSIGNOR:GM GLOBAL TECHNOLOGY OPERATIONS, INC.;REEL/FRAME:025781/0211 Effective date: 20101202 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |