CN107424207A - A kind of Virtual Maintenance Simulation method and device based on data fusion - Google Patents
A kind of Virtual Maintenance Simulation method and device based on data fusion Download PDFInfo
- Publication number
- CN107424207A CN107424207A CN201710573334.1A CN201710573334A CN107424207A CN 107424207 A CN107424207 A CN 107424207A CN 201710573334 A CN201710573334 A CN 201710573334A CN 107424207 A CN107424207 A CN 107424207A
- Authority
- CN
- China
- Prior art keywords
- data
- immersion
- maintenance
- emulation
- motion capture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a kind of Virtual Maintenance Simulation method and device based on data fusion, is related to data fusion auxiliary virtual emulation field, and methods described includes：Maintenance task is decomposed, obtains maintenance behavior；According to the simulation requirements of the maintenance behavior, the emulation mode of the maintenance behavior is determined；According to the emulation mode of the maintenance behavior, non-immersion emulation data and immersion motion capture data are obtained；Data and the immersion motion capture data, Virtual Maintenance behavior of the generation visual human to the maintenance task are emulated according to the non-immersion.Two kinds of data source Collaborative Controls synthesis Virtual Maintenance behavior of the embodiment of the present invention, the purpose of emulation quick, true to nature can be reached.
Description
Technical field
The present invention relates to data fusion auxiliary virtual emulation field, more particularly to a kind of Virtual Maintenance based on data fusion
Emulation mode and device.
Background technology
1. economy and convenience that Virtual Maintenance Simulation technology improves to product design
Important application of the Virtual Maintenance Simulation technology as virtual reality technology in Product maintenance design field, to production
The raising of product capability of maintenance design plays the role of important.In recent years, the either qualitative factor or quantitative factor of maintainability, it is empty
Intending maintenance simulation technology has relevant art means to provide technical support to the two importances.Virtual Maintenance Simulation technology is put
Dependence of the maintainability analysis to model machine has been taken off, has carried out more wheels, multimode, more processes by means of the digital prototype of product and divides
Analysis, product design costses are greatly reduced, and designer can be by modes such as remote collaborative, interactive operations jointly to production
Product are analyzed, and avoid that traditional approach is numerous and diverse, mode of redundancy, improve the efficiency of product design work.
2. effect of the visual human in Virtual Maintenance Simulation
A part of the visual human as Virtual Maintenance based on certain, it is mainly used in ergonomic's analysis and demonstration standard repair action
Flow.Whole maintenance work process almost needs visual human to be participated in the overall process as maintenance operation main body, motion of virtual human emulation
It is whole maintenance simulation core content.The speed that Virtual Maintenance Simulation is realized, and its automaticity, degree true to nature are largely
Quality depending on motion of virtual human the Realization of Simulation.The emulation of maintenance personal's motion is considered in Virtual Maintenance Simulation, is mainly had
Key frame emulation, Kinematics Simulation, dynamics simulation, based on dynamic catch data simulation and maintenance operation planning simulation.
3. the peculiar technical advantage of immersion and non-immersion Virtual Maintenance Simulation
Immersion Virtual Maintenance based on certain obtains the movable information of operating personnel using capturing movement equipment in real time, for controlling
The motion of manikin in virtual environment；Stereoscopic display is carried out to scene using three-dimensional output equipment, improves the feeling of immersion of emulation；
Selection, operation and motion control that direct interactive meanses are used for object in scene are provided simultaneously, repaired operating personnel
The perceptual knowledge of difficulty or ease.
Non-immersion Virtual Maintenance based on certain application generation human action and the action and motion of motion algorithm generation visual human,
Visual human is driven to complete maintenance operation.Simulation process needs to meet authenticity requirement, that is, must is fulfilled for the constraint of 4 classes：Naturality and
Verisimilitude constraint, additional space constraint, model machine behavior restraint and task restriction.
4. data fusion
Data fusion is the information and data of integrated treatment multisensor or multi-source, obtains accurate believable conclusion.Profit
With computer technology, automatically analyzed under certain criterion, the information of the comprehensive multiple sensors obtained based on the time, complete to need
Decision-making and the information process of estimation task.
The content of the invention
, can be preferably it is an object of the invention to provide a kind of Virtual Maintenance Simulation method and device based on data fusion
Solve the problems, such as that Virtual Maintenance Simulation efficiency is low.
According to an aspect of the invention, there is provided a kind of Virtual Maintenance Simulation method based on data fusion, including：
Maintenance task is decomposed, obtains maintenance behavior；
According to the simulation requirements of the maintenance behavior, the emulation mode of the maintenance behavior is determined；
According to the emulation mode of the maintenance behavior, non-immersion emulation data and immersion motion capture data are obtained；
Data and the immersion motion capture data are emulated according to the non-immersion, visual human is to the maintenance for generation
The Virtual Maintenance behavior of task.
Preferably, the emulation mode according to the maintenance behavior, obtains non-immersion emulation data and immersion moves
Included as Catch the Data Pack：
If the emulation mode of the maintenance behavior is non-immersion emulation mode, calculated using human action and motion generation
Method, generate the non-immersion emulation data based on 3D human skeleton models；
If the emulation mode of the maintenance behavior is immersion emulation mode, obtained using motion capture system based on real
The immersion motion capture data of the human skeleton model of border operating personnel.
Preferably, described to emulate data and the immersion motion capture data according to the non-immersion, generation is virtual
Virtual Maintenance behavior of the people to the maintenance task includes：
Data are emulated to the non-immersion and the immersion motion capture data carries out spatio-temporal synergy processing, when obtaining
The non-immersion emulation data and immersion motion capture data of sky collaboration；
Data and immersion motion capture data are emulated using the non-immersion of the spatio-temporal synergy, visual human is to institute for synthesis
State the Virtual Maintenance behavior of maintenance task.
Preferably, it is described that data and immersion motion capture data progress spatio-temporal synergy are emulated to the non-immersion
Processing, the non-immersion emulation data and immersion motion capture data for obtaining spatio-temporal synergy include：
Using dynamic time warping algorithm, the non-immersion is emulated into data and the immersion motion capture data
Time unifying；
According to human body root node grid bearing, the non-immersion is emulated into data and the immersion motion capture data
Coordinate space alignment.
Preferably, the non-immersion is being emulated at data and immersion motion capture data progress spatio-temporal synergy
Reason, before obtaining the non-immersion emulation data and immersion motion capture data of spatio-temporal synergy, in addition to：
The non-immersion is emulated into human skeleton model corresponding to one of data and the immersion motion capture data
As template, by human skeleton model corresponding to another data and the template alignment.
Preferably, the non-immersion using the spatio-temporal synergy emulates data and immersion motion capture data, closes
Virtual Maintenance behavior into visual human to the maintenance task includes：
, will using non-immersion emulation one of data and the immersion motion capture data as collaboration reference data
Another data fusion is entered in reference data, obtains fused data, to utilize the fused data, synthesizes visual human's Virtual Maintenance
Behavior.
Preferably, after fused data is obtained, in addition to：To the human body root node locus in the fused data
Interpolation processing is carried out with joint attitude angle, obtains the fused data for making maintenance behavior seamless link.
According to another aspect of the present invention, there is provided a kind of Virtual Maintenance Simulation device based on data fusion, including：
Task-decomposing module, for being decomposed to maintenance task, obtain maintenance behavior；
Mode determining module, for the simulation requirements according to the maintenance behavior, determine the emulation side of the maintenance behavior
Formula；
Data acquisition module, for the emulation mode according to the maintenance behavior, obtain non-immersion emulation data and sink
Immersion motion capture data；
Synthesis module is emulated, it is raw for emulating data and the immersion motion capture data according to the non-immersion
Virtual Maintenance behavior into visual human to the maintenance task.
Preferably, the data acquisition module is when the emulation mode of the maintenance behavior is non-immersion emulation mode,
Using human action and motion generation algorithm, the non-immersion emulation data based on 3D human skeleton models are generated, in the dimension
Practice Buddhism or Taoism for emulation mode be immersion emulation mode when, utilize motion capture system obtain the human body based on practical operation personnel
The immersion motion capture data of skeleton model.
Preferably, the emulation synthesis module emulates data and the immersion motion capture data to the non-immersion
Spatio-temporal synergy processing is carried out, obtains the non-immersion emulation data and immersion motion capture data of spatio-temporal synergy, and utilize institute
State the non-immersion emulation data and immersion motion capture data of spatio-temporal synergy, void of the synthesis visual human to the maintenance task
Intend maintenance behavior.
Compared with prior art, the beneficial effects of the present invention are：
The present invention organically combines non-immersion emulation and immersion emulation, realizes based on the fast of human body movement data fusion
Fast Virtual Maintenance Simulation, specifically, based on the human body movement data provided in non-immersion and immersion emulation, quickly to transport
Dynamic emulation is embodied as target, the human body maintenance behavior of analysis oriented for maintenance task simulation, void of the structure suitable for maintenance simulation
Personification simplifies human synovial hierarchical model, using emulation data and gathered data amalgamation mode, passes through pair of two kinds of data sources
Neat seamlessly transitting for processing and maintenance behavior realizes quickly generating for complicated maintenance process emulation.
Brief description of the drawings
Fig. 1 is the Virtual Maintenance Simulation method flow diagram provided in an embodiment of the present invention based on data fusion；
Fig. 2 is the Virtual Maintenance Simulation device block diagram provided in an embodiment of the present invention based on data fusion；
Fig. 3 is the maintenance simulation Method And Principle block diagram based on human emulation data and gathered data fusion；
Fig. 4 a are the human synovial structural model schematic diagrames after simplifying；
Fig. 4 b are the human body limb and articulation structure hierarchical relationship schematic diagram after simplifying；
Fig. 5 is coordinate space entirety alignment operation flow chart；
Fig. 6 is the operational flowchart that two kinds of data source asynchronous controllings realize new motion synthesis；
Fig. 7 is the operational flowchart that two kinds of data source Synchronization Controls realize new motion synthesis；
Fig. 8 is that different dimensional is practiced Buddhism or Taoism as the algorithm flow chart of seamless link.
Embodiment
Below in conjunction with accompanying drawing to a preferred embodiment of the present invention will be described in detail, it will be appreciated that described below is excellent
Select embodiment to be merely to illustrate and explain the present invention, be not intended to limit the present invention.
Fig. 1 is the Virtual Maintenance Simulation method flow diagram provided in an embodiment of the present invention based on data fusion, such as Fig. 1 institutes
Show,
Step 1, maintenance task is decomposed, obtain maintenance behavior.
Step 2, the simulation requirements according to the maintenance behavior, determine the emulation mode of the maintenance behavior.
Step 3, the emulation mode according to the maintenance behavior, obtain non-immersion emulation data and immersion motion capture
Data.
Wherein, if non-immersion emulation mode, then it is based on 3D people using human action and motion generation algorithm, generation
The non-immersion emulation data of body skeleton model, i.e. emulation animation data or emulation data.
Wherein, if immersion emulation mode, then the people based on practical operation personnel is obtained using motion capture system
The immersion motion capture data of body skeleton model, that is, move to catch exercise data or move and catch data or gathered data.
Step 4, data and the immersion motion capture data are emulated according to the non-immersion, visual human is to institute for generation
State the Virtual Maintenance behavior of maintenance task.
Data are emulated to the non-immersion first and the immersion motion capture data carries out spatio-temporal synergy processing, tool
Say body, using dynamic time warping algorithm, the non-immersion is emulated into data and the immersion motion capture data
Time unifying, according to human body root node grid bearing, the non-immersion is emulated into data and the immersion motion capture number
According to coordinate space align, obtain the non-immersion emulation data and immersion motion capture data of spatio-temporal synergy.Then utilize
The non-immersion emulation data and immersion motion capture data of the spatio-temporal synergy, visual human is to the maintenance task for synthesis
Virtual Maintenance behavior, when specifically synthesizing, the non-immersion can be emulated data and the immersion motion capture data it
One, as collaboration reference data, another data fusion is entered in reference data, obtains fused data, to utilize the fusion number
According to synthesis visual human's Virtual Maintenance behavior.
It should be noted that to avoid fused data distortion or beyond restriction range, before spatio-temporal synergy processing is carried out,
The non-immersion can also be emulated to human skeleton model corresponding to one of data and the immersion motion capture data to make
For template, by human skeleton model corresponding to another data and the template alignment.
It should be noted that for realize different dimensional practice Buddhism or Taoism for seamlessly transit, can also be in the fused data
Human body root node locus and joint attitude angle carry out interpolation processing, obtain the fused data for making maintenance behavior seamless link.
The present embodiment, which will emulate data and move, catches data both data source Collaborative Controls synthesis maintenance behavior, can reach
Quickly, the purpose of emulation true to nature, Virtual Maintenance Simulation efficiency is improved.
Fig. 2 is the Virtual Maintenance Simulation device block diagram provided in an embodiment of the present invention based on data fusion, as shown in Fig. 2
Task-decomposing module, for being decomposed to maintenance task, obtain maintenance behavior.
Mode determining module, for the simulation requirements according to the maintenance behavior, determine the emulation side of the maintenance behavior
Formula.
Data acquisition module, for the emulation mode according to the maintenance behavior, obtain non-immersion emulation data and sink
Immersion motion capture data.
Synthesis module is emulated, it is raw for emulating data and the immersion motion capture data according to the non-immersion
Virtual Maintenance behavior into visual human to the maintenance task.
The course of work of described device is as follows：Task-decomposing module is decomposed to maintenance task, obtains maintenance behavior.Side
For formula determining module according to the simulation requirements of the maintenance behavior, the emulation mode for determining the maintenance behavior is non-immersion emulation
Mode or immersion emulation mode.If non-immersion emulation mode, data acquisition module utilizes human action and motion
Generating algorithm, generate the non-immersion emulation data based on 3D human skeleton models；If immersion emulation mode, data obtain
Modulus block obtains the immersion motion capture data of the human skeleton model based on practical operation personnel using motion capture system.
Emulate synthesis module and data and immersion motion capture data progress spatio-temporal synergy processing are emulated to the non-immersion, obtain
Non-immersion to spatio-temporal synergy emulates data and immersion motion capture data, and utilizes the non-immersion of the spatio-temporal synergy
Emulate data and immersion motion capture data, Virtual Maintenance behavior of the synthesis visual human to the maintenance task.
The embodiment of the present invention is imitated based on the human body movement data provided in non-immersion and immersion emulation with quick motion
Really it is embodied as target, the human body maintenance behavior of analysis oriented for maintenance task simulation, visual human of the structure suitable for maintenance simulation
Simplify human synovial hierarchical model, using emulation data and gathered data amalgamation mode, at the alignment by two kinds of data sources
Reason and seamlessly transitting for maintenance behavior realize quickly generating for complicated maintenance process emulation.
The embodiment of the present invention carries out maintenance task decomposition centered on maintenance personal's activity, to the visual human after Task-decomposing
Maintenance behavior carries out simulation requirements, emulation mode analysis.On the basis of simulation analysis, with reference to visual human's H-Anim master patterns,
Establish the simplification virtual human body bone-joint hierarchical model and sports immunology model for being applicable emulation maintenance personal's behavior.
The embodiment of the present invention considers influence of the simulated environment scope of application to maintenance behavior simulation, obtains respectively suitable for non-
The human body maintenance behavioral data of immersion animation simulation and the human body motion capture data suitable for immersion emulation, and to two kinds
The processing such as data source is cut out, adjusted, Data Format Transform, the serial of methods controlled using spatio-temporal synergy, realize two kinds
The fusion of data source, final Fast back-projection algorithm go out the maintenance behavior of emulation personnel needs.
Fig. 3 is the maintenance simulation Method And Principle block diagram based on human emulation data and gathered data fusion, as shown in figure 3,
Including：
Step 1: oriented for maintenance task carries out maintenance behavioural analysis.
Analyzed based on existing maintenance task decomposition method, the maintenance task decomposition model centered on maintenance personal's activity,
Behavior is repaired to the visual human after Task-decomposing and carries out simulation requirements, emulation mode analysis.
Specifically, in the portraying of maintenance task virtual emulation, maintenance personal participates in making its dynamic increasingly complex.It is logical
The conventional maintenance task decomposition model of analysis is crossed, proposes the maintenance task composition model method centered on maintenance personal's activity.Tool
Body acupuncture repairs behavior (i.e. behavior class, pose adjustment class and operation by human hand class) to three classes of behavior layer in maintenance task decomposition model
Simulation requirements analyzed.Meanwhile consider two kinds of non-immersion, immersion simulated environment scope of applications, maintenance behavior is closed
Analyzed into applicable pattern, establish the simplification skeleton-joint hierarchical model for being applied to description maintenance behavior, and be based on
Model describes human motion.
Wherein, Fig. 4 a are the human synovial structural model schematic diagrames after simplifying, and Fig. 4 b are the human body limb and pass after simplifying
Structural hierarchical relation schematic diagram is saved, is visual human's master pattern (i.e. H-Anim according to hierarchical structure shown in Fig. 4 a and Fig. 4 b
Master pattern), obtained from human skeleton composition is fixedly connected according to certain order, the root artis in model is virtual
A center of gravity node being manually set in manikin, is generally selected in human body hipbone center of gravity.The embodiment of the present invention is based on figure
4a or Fig. 4 b, it is determined that needing the data obtained, such as obtain left wrist joint data and left elbow joint data etc..
Step 2: the emulation data (i.e. non-immersion emulation data) and motion capture data (i.e. immersion of human action
Motion capture data or human body motion capture data) processing.
Human motion is described using skeleton-joint hierarchical model, can be obtained respectively suitable for non-immersion
The emulation data of animation simulation and the human body motion capture data suitable for immersion emulation, and two kinds of data sources are cut
The processing such as sanction, adjustment, Data Format Transform.For example, (i.e. non-immersion is imitated for expression and acquisition DELMIA position of human body attitude data
True data), select applicable motion capture system gathered data (i.e. immersion motion capture data) to be analyzed, and to data
It is cut out, adjusts, to utilize motion capture data driving DELMIA visual humans.
Step 3: two kinds of action data sources are subjected to spatio-temporal synergy.
The human body maintenance behavior obtained using two kinds of emulation modes exist coordinate position and direction is inconsistent, sampling rate not
First, sampling period not parity problem, therefore, it is necessary to consider data when carrying out fusion treatment in two kinds of action data sources to acquisition
Space-time characterisation, carry out corresponding collaboration processing.It should confirm to cooperate with reference data first, by the data of another data source
It is integrated into template data, so as to correct the new maintenance behavior of primary template Data Synthesis.Meanwhile because we using people
Body fixed length skeleton model is calculated, and before collaboration processing, should be carried out human skeleton model corresponding to two kinds of data sources pair
Neat processing, not so can cause the data distortion of resultant motion or beyond restriction range.
1) bone registration process：Selection will dress the dynamic real human body skeleton model caught as template, allow in DELMIA
3D human skeleton models and template alignment.
2) time unifying is handled：On time unifying processing, generally use dynamic time warping DTW methods.Transported in human body
Application time alignment schemes in dynamic data, similar posture is found to exercise data sequence using Time alignment by align to path, can
Support is provided for the fusion of subsequent motion sequence.
3) coordinate space registration process：The human body grid bearing of each posture frame of the two or more motion segments of alignment requirements
Alignd on three dimensions, that is, require the human body root node grid bearing pair of each posture frame in the motion segments that needs merge
Together.The posture sequence coordinate of the present embodiment, which integrally aligns, converts the Bonding Problem of mainly two motion segments of solution, with animation appearance
Template motion of the state sequence as alignment, by the use of the spatial variations matrix of joining place human body root node coordinate as corrected parameter, by
Frame corrective action catches the human body root node position of collection posture sequence, finally realizes that two kinds of data source coordinate spaces are integrally right
Together.Fig. 5 is coordinate space entirety alignment operation flow chart, and as shown in Figure 5, step includes：First by motion capture system
Coordinate system is converted into emulation platform DELMIA coordinate system, and integrating action catches collection people under DELMIA world coordinate system
(human body root node track data i.e. in immersion motion capture data dynamic catches root to body root node track ROOTB initial data
Node location data) and the DELMIA animation human body root nodes track ROOTA initial data (people i.e. in non-immersion emulation data
Body root node track data or emulation root node position data) location presentation form, obtain the human body root node of motion capture
Position 4*4 expression matrixs form and DELMIA animation human body root node position 4*4 expression matrix forms.Secondly according to motion capture
Human body root node position 4*4 expression matrixs form and DELMIA animation human body root node position 4*4 expression matrix forms, it is alive
The homogeneous transformation matrices T of 4*4 of changes in coordinates are calculated under boundary's coordinate system.Then join the homogeneous transformation matrices T of 4*4 as amendment
Number, amendment is dynamic frame by frame catches root node position data, obtains the position 4*4 expression matrixs after the human body root node alignment of motion capture
Form, finally realize that two kinds of data source coordinate spaces integrally align.Finally by after the human body root node alignment of the motion capture
Position 4*4 matrixes carry out Data Transform, obtain dynamic catching data source DELMIA person body orientation expression-form 1*12 data groups.
Step 4: carry out maintenance behavior synthesis based on two kinds of data source Collaborative Controls.
Obtain the emulation animation data (i.e. non-immersion emulation data) of spatio-temporal synergy and dynamic seizure data (i.e. motion capture
Exercise data dynamic catches exercise data or immersion emulation data) after, maintenance behavior is synthesized by, there is asynchronous and synchronous collaboration
Two kinds of control modes.By analyzing the control sequential of two kinds of cooperative modes, maintenance behavior is carried out under different Collaborative Control modes
Synthesis.
1) two kinds of asynchronous drivings of data source
Maintenance behavior is subjected to Discrete control in time, can only be driven in certain a period of time by a kind of data source in scene
Virtual Maintenance personnel (i.e. visual human) produce corresponding actions, final overall chain is bonded into required maintenance behavior.Fig. 6 is two
Kind of data source asynchronous controlling realizes the operational flowchart of new motion synthesis, specific building-up process as shown in Figure 6, including：First
Emulation animation data slot is split, obtained for the embedded dynamic insertion entrance posture frame (i.e. ingress frame) for catching data.Its
It is secondary that the dynamic data slot of catching for the data format for having been converted into emulation platform DELMIA is subjected to coordinate with emulation animation data slot
Alignment.Then through ingress frame at, corresponding dynamic will catch data slot insertion emulation animation data, it is real to pass through interpolation after embedded
Smoothing processing is now acted, the final new motion of synthesis.The core of this flow is that the data slot of motion capture collection is flat to emulation
Platform DELMIA is converted.Original emulation animation data slot is split using posture registration process method, obtains to move and catches data insertion
Ingress frame.
2) two kinds of data sources synchronously drive
The difference of Synchronization Control and asynchronous controlling is in maintenance behavior sometime in fragment, by animation data and action
Catch two kinds of data sources of exercise data and drive the different limbs of human body, joint part respectively, coordinated drive visual human completes maintenance
Action.Fig. 7 is the operational flowcharts that two kinds of data source Synchronization Controls realize new motion synthesis, as shown in Figure 7, first with point
Not from emulation animation data slot and it is driven catch the non-limbs data interested extracted in data slot, carry out two kinds of data sources
Time unifying processing, and utilize the driven limbs data interested caught and extracted in data slot, the coordinate of two kinds of data sources of progress
Registration process.Then two kinds of data sources after time and coordinate alignment are merged, can also be realized after fusion by interpolation
Act smoothing processing, the final new motion of synthesis.Such a mode is realized under different data sources driving, human body difference limb action
Fusion, the maintenance behavior needed for quick synthesis of artificial personnel true to nature provide a good method and approach.
Synchronization Control operating process is added to non-limbs data extraction interested and non-sense in emulation animation data slot
Interest limbs data time registration process.Here non-limbs data interested refer to that motion capture data need not be embedded in into work melts
Close the exercise data of human body other body parts of processing, for example, by motion capture to human body right arm turn action and be embedded into void
In the emulation animation of anthropomorphic Manual-disassembling screw, now the right clavicle of human body, right upper arm, the action data in right forearm joint are that sense is emerging
Interesting limbs data, and the data of human body other limbs joints motion are then considered as non-limbs data interested.In above-mentioned fusion process
In, if in the emulation animation of original Manual-disassembling screw operation, only left hand articulations digitorum manus and left hand wrist joint is changing, then may be used
Using the exercise data of the left finesse in non-limbs data interested as paying close attention to object, a left side for two kinds of data sources of registration process
Wrist-sport data.
Step 5: smooth transient method between maintenance behavior.
To realize the emulation of complete maintenance process, it is necessary to which the visual human synthesized in step 4 is walked class, pose adjustment class
Connection is combined with the maintenance behavior of operation by human hand class.Because maintenance behavior differs greatly on locus and joint posture,
Seamlessly transitting between maintenance behavior is realized using visual human's posture keyframe interpolation.
Based on foregoing human motion description, it is seen that change is mainly reflected in human body root section caused by motion of virtual human
On space of points position and joint posture.Root node dimensional orientation determines the whole path of motion of virtual human and in each position
The direction of point, the change of joint attitude angle then derive motion.So visual human's key poses frame (i.e. visual human's posture key frame
Or posture frame) interpolation include human body root node locus interpolation and each posture joint angle (i.e. joint attitude angle) it is slotting
Value.Seamlessly transitting for human space position between maintenance behavior is realized using three dimensional space coordinate linear interpolation method, used for reference simultaneously
The theory and interpolation method of quaternary number realize seamlessly transitting for human body attitude joint angle between maintenance behavior, using the mistake of insertion new frame
Mode is crossed, realizes seamless link between maintenance behavior.Fig. 8 be different dimensional practice Buddhism or Taoism for the algorithm flow chart of seamless link, algorithm it is specific
Flow as shown in figure 8, including：First will posture sequence qB corresponding to posture sequence qA corresponding to maintenance behavior A and maintenance behavior B
Posture alignment is carried out, specifically includes and determines cutout window, most like posture frame is searched in window, cut both ends posture sequence
Row, it is determined that the head and the tail frame for posture frame interpolation.Then judge whether two frame configuration space positions of linking approach, if close
Interpolation processing is then directly carried out, is otherwise handled integrally align laggard row interpolation of root node grid bearing space, interpolation processing tool
Body includes human body root node position spatial linear interpolation and human synovial angle quaternary number sphere interpolation, wherein basis is used for posture frame
The head and the tail frame of interpolation, joint Eulerian angles are determined, and then joint Eulerian angles are converted into human synovial angle quaternary number.Finally will insertion
Joint angle quaternary number be converted to Eulerian angles, finally realize seamlessly transitting between maintenance behavior.
The asynchronous cooperation control method example of example 1. -- certain type guided missile electrical control gear is dismantled
During whole dismounting, Virtual Maintenance personnel are largely repaired behavior simulation and can imitated using non-immersion animation
True mode is completed, but for the more narrow and small operational simulation in certain operations space, to avoid interfering with surrounding devices, it is necessary to
Precise controlling visual human's posture is finely tuned frame by frame.Now seem complicated cumbersome using the animation edit mode of non-immersion, and
The emulation mode that corresponding pose adjustment is completed using immersion motion capture driving visual human is seemed convenient and efficient.
1) the pose adjustment data of Virtual Maintenance personnel are obtained in two steps.
(1) DELMIA emulation animation platforms are utilized, the posture for completing obvious, the more easy-to-dismount cable plug dismounting of exposure is adjusted
Full line is animation simulation, obtains the people of the visual human's pose adjustment animation frame made manually under DELMIA platforms and matlab emulation
Change corresponding to body bone posture, root node；
(2) it is more hidden for position, not easy-to-dismount cable plug, using immersion motion capture mode, utilize
Phase space realize that the motion capture data obtained under immersion simulated environment corresponds to visual human's bone under CAVE environment
With the change of root node track；
2) motion capture data under immersive environment needs to be spliced to after DELMIA platform animation datas, realizes two kinds
The asynchronous driving visual human of data source.
3) after data smoothing is handled, visual human's maintenance is synthesized under DELMIA environment, under two kinds of data source asynchronous controllings
Behavior.
2. synchronous synergetic control method of example -- the elastic regulation of automobile clutch
The maintenance process of regulation clutch pedal height usually requires maintenance personal and lain prone before operator seat seat, and the right hand makes
Loosened the clutch the fixing nut of bracing wire to the left with open-end wrench, and left hand presses down on pedal.It is related to maintenance personal right-hand man
Synchronous coordination operates, and in the process, the intermittent pressing pedal action of left hand is relatively easy, and the cycle is relatively fixed, should can basis
Simulation developer needs, self-defined number of cycles, so being realized by the way of non-immersion animation simulation；The right hand along with
When the left hand palm presses clutch pedal, operation spanner turns the hold-doun nut of bracing wire, and the operating space of this action is relatively narrow and small,
Intense adjustment is needed, the motion capture mode of immersion can be used to gather exercise data.
1) non-immersion animation simulation mode is based on, visual human's maintenance activity preparatory stage is completed under DELMIA platforms and is lived
Dynamic emulation, such as run to car door, opening car door, adjustment posture hand and touch Seeding location.
2) Virtual Maintenance personnel's bimanualness data are obtained.
(1) animation mode is used, makes the cyclical action that left hand presses down on clutch pedal；
(2) immersion motion capture mode is used, both hands maintenance operation is captured under CAVE environment using Phase space
Data.
3) emulate in need by motion capture to right hand operation data be loaded on the virtual human body in emulation animation, will
Two kinds of data carry out space-time alignment.
4) original animation simulation is corrected under DELMIA environment, realizes the visual human synthesized under two kinds of data source Synchronization Controls
Hand adjusts the maintenance behavior of clutch cable.
That is, research of the embodiment of the present invention is based on human body movement data fusion Fast back-projection algorithm maintenance personal's behavior, it is real
Existing rapid-maintenance emulation.First, carry out the maintenance behavioural analysis of oriented for maintenance task, establish and be adapted to the virtual of description maintenance behavior
People simplifies skeleton-joint hierarchical model and motion model；Secondly, carry out and catch data based on dynamic (i.e. immersion is acted and caught
Catch data) the maintenance behavior fast synthesis method that merge with emulation animation data (i.e. non-immersion emulation data) studies, studies
Expression, acquisition, cutting, spatio-temporal synergy processing and the control method of two kinds of data sources；Then, for human body root node locus
Change with joint attitude angle, study the smooth transient method between maintenance behavior, and the feasibility of verification method.
In summary, the present invention has following technique effect：
1. the embodiment of the present invention organically combines immersion virtual emulation and non-immersion virtual emulation, gather two kinds of emulation
The peculiar advantage of mode, realize that visual human repairs the Fast back-projection algorithm of behavior, and then realize maintenance simulation rapidly and efficiently；
2. the embodiment of the present invention is realizes that rapid-maintenance emulation provides a kind of specific, feasible approach, by right
DELMIA interface secondary development, it can be needed to realize some individual requirements of maintenance simulation according to developer；
3. the embodiment of the present invention reuses immersion simulated environment data message, for later deeper into development immersion virtual ring
Data message analysis, which reuses, under border provides idea and method guidance；
4. the embodiment of the present invention enriches motion of virtual human Study of synthesis method, by proposing to melt based on human body movement data
The method that behavior is repaired in symphysis into visual human, further enrich and improve motion of virtual human synthetic method collection.
Although the present invention is described in detail above, the invention is not restricted to this, those skilled in the art of the present technique
Various modifications can be carried out according to the principle of the present invention.Therefore, all modifications made according to the principle of the invention, all should be understood to
Fall into protection scope of the present invention.
Claims (10)
- A kind of 1. Virtual Maintenance Simulation method based on data fusion, it is characterised in that including：Maintenance task is decomposed, obtains maintenance behavior；According to the simulation requirements of the maintenance behavior, the emulation mode of the maintenance behavior is determined；According to the emulation mode of the maintenance behavior, non-immersion emulation data and immersion motion capture data are obtained；Data and the immersion motion capture data are emulated according to the non-immersion, visual human is to the maintenance task for generation Virtual Maintenance behavior.
- 2. according to the method for claim 1, it is characterised in that the emulation mode according to the maintenance behavior, obtain Non-immersion, which emulates data and immersion motion capture data, to be included：If the emulation mode of the maintenance behavior is non-immersion emulation mode, using human action and motion generation algorithm, Generate the non-immersion emulation data based on 3D human skeleton models；If the emulation mode of the maintenance behavior is immersion emulation mode, obtained using motion capture system based on actual behaviour Make the immersion motion capture data of the human skeleton model of personnel.
- 3. according to the method for claim 2, it is characterised in that described according to non-immersion emulation data and described heavy Immersion motion capture data, Virtual Maintenance behavior of the generation visual human to the maintenance task include：Data are emulated to the non-immersion and the immersion motion capture data carries out spatio-temporal synergy processing, obtain space-time association Same non-immersion emulation data and immersion motion capture data；Data and immersion motion capture data are emulated using the non-immersion of the spatio-temporal synergy, visual human is to the dimension for synthesis Repair the Virtual Maintenance behavior of task.
- 4. according to the method for claim 3, it is characterised in that described that data are emulated to the non-immersion and described are immersed Formula motion capture data carries out spatio-temporal synergy processing, obtains non-immersion emulation data and the immersion motion capture of spatio-temporal synergy Data include：Using dynamic time warping algorithm, by the time of non-immersion emulation data and the immersion motion capture data Alignment；According to human body root node grid bearing, by the seat of non-immersion emulation data and the immersion motion capture data Mark spatial alignment.
- 5. the method according to claim 3 or 4, it is characterised in that data and described heavy are being emulated to the non-immersion Immersion motion capture data carries out spatio-temporal synergy processing, and the non-immersion emulation data and immersion action for obtaining spatio-temporal synergy are caught Before catching data, in addition to：Using the non-immersion emulate one of data and the immersion motion capture data corresponding to human skeleton model as Template, by human skeleton model corresponding to another data and the template alignment.
- 6. according to the method for claim 3, it is characterised in that the non-immersion using the spatio-temporal synergy emulates number Include according to immersion motion capture data, Virtual Maintenance behavior of the synthesis visual human to the maintenance task：, will be another using non-immersion emulation one of data and the immersion motion capture data as collaboration reference data Data fusion is entered in reference data, obtains fused data, to utilize the fused data, synthesizes visual human's Virtual Maintenance row For.
- 7. according to the method for claim 6, it is characterised in that after fused data is obtained, in addition to：To the fusion Human body root node locus and joint attitude angle in data carry out interpolation processing, obtain making melting for maintenance behavior seamless link Close data.
- A kind of 8. Virtual Maintenance Simulation device based on data fusion, it is characterised in that including：Task-decomposing module, for being decomposed to maintenance task, obtain maintenance behavior；Mode determining module, for the simulation requirements according to the maintenance behavior, determine the emulation mode of the maintenance behavior；Data acquisition module, for the emulation mode according to the maintenance behavior, obtain non-immersion emulation data and immersion Motion capture data；Synthesis module is emulated, for emulating data and the immersion motion capture data according to the non-immersion, generation is empty The anthropomorphic Virtual Maintenance behavior to the maintenance task.
- 9. device according to claim 8, it is characterised in that emulation of the data acquisition module in the maintenance behavior When mode is non-immersion emulation mode, using human action and motion generation algorithm, generate based on 3D human skeleton models Non-immersion emulates data, when the emulation mode of the maintenance behavior is immersion emulation mode, utilizes motion capture system Obtain the immersion motion capture data of the human skeleton model based on practical operation personnel.
- 10. device according to claim 9, it is characterised in that the emulation synthesis module emulates to the non-immersion Data and the immersion motion capture data carry out spatio-temporal synergy processing, obtain spatio-temporal synergy non-immersion emulation data and Immersion motion capture data, and data and immersion motion capture data are emulated using the non-immersion of the spatio-temporal synergy, Synthesize Virtual Maintenance behavior of the visual human to the maintenance task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710573334.1A CN107424207A (en) | 2017-07-10 | 2017-07-10 | A kind of Virtual Maintenance Simulation method and device based on data fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710573334.1A CN107424207A (en) | 2017-07-10 | 2017-07-10 | A kind of Virtual Maintenance Simulation method and device based on data fusion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107424207A true CN107424207A (en) | 2017-12-01 |
Family
ID=60427874
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710573334.1A Pending CN107424207A (en) | 2017-07-10 | 2017-07-10 | A kind of Virtual Maintenance Simulation method and device based on data fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107424207A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109470263A (en) * | 2018-09-30 | 2019-03-15 | 北京诺亦腾科技有限公司 | Motion capture method, electronic equipment and computer storage medium |
| CN112206525A (en) * | 2020-09-30 | 2021-01-12 | 深圳市瑞立视多媒体科技有限公司 | Information processing method and device for hand-twisting virtual article in UE4 engine |
| CN113450438A (en) * | 2020-03-24 | 2021-09-28 | 深圳市灼华互娱科技有限公司 | Virtual character driving method and device based on motion capture and computer equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123007A (en) * | 2014-07-29 | 2014-10-29 | 电子科技大学 | Multidimensional weighted 3D recognition method for dynamic gestures |
| CN105930767A (en) * | 2016-04-06 | 2016-09-07 | 南京华捷艾米软件科技有限公司 | Human body skeleton-based action recognition method |
| CN106469467A (en) * | 2016-09-05 | 2017-03-01 | 北京航空航天大学 | A kind of Virtual Maintenance behavior synthetic method based on bodily fuctions position and system |
-
2017
- 2017-07-10 CN CN201710573334.1A patent/CN107424207A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123007A (en) * | 2014-07-29 | 2014-10-29 | 电子科技大学 | Multidimensional weighted 3D recognition method for dynamic gestures |
| CN105930767A (en) * | 2016-04-06 | 2016-09-07 | 南京华捷艾米软件科技有限公司 | Human body skeleton-based action recognition method |
| CN106469467A (en) * | 2016-09-05 | 2017-03-01 | 北京航空航天大学 | A kind of Virtual Maintenance behavior synthetic method based on bodily fuctions position and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109470263A (en) * | 2018-09-30 | 2019-03-15 | 北京诺亦腾科技有限公司 | Motion capture method, electronic equipment and computer storage medium |
| CN113450438A (en) * | 2020-03-24 | 2021-09-28 | 深圳市灼华互娱科技有限公司 | Virtual character driving method and device based on motion capture and computer equipment |
| CN112206525A (en) * | 2020-09-30 | 2021-01-12 | 深圳市瑞立视多媒体科技有限公司 | Information processing method and device for hand-twisting virtual article in UE4 engine |
| CN112206525B (en) * | 2020-09-30 | 2022-04-29 | 深圳市瑞立视多媒体科技有限公司 | Information processing method and device for hand-twisting virtual article in UE4 engine |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101098834B1 (en) | Apparatus and method for generating motion based on dynamics | |
| CN104573230B (en) | Towards the visual human's job task simulation analysis system and method for spacecraft maintenance | |
| Zhao | A survey on virtual reality | |
| CN110599573B (en) | Method for realizing real-time human face interactive animation based on monocular camera | |
| CN106200983A (en) | A kind of combined with virtual reality and BIM realize the system of virtual reality scenario architectural design | |
| CN108287483A (en) | A kind of immersion Virtual Maintenance Simulation method and system towards Product maintenance verification | |
| CN107833503B (en) | Distribution core job augmented reality simulation training system | |
| CN107424207A (en) | A kind of Virtual Maintenance Simulation method and device based on data fusion | |
| Nunnari et al. | Animating azee descriptions using off-the-shelf ik solvers | |
| Qinping | A survey on virtual reality | |
| Cui et al. | Digital media animation control technology based on maya | |
| Wang et al. | Assembly design and evaluation based on bare-hand interaction in an augmented reality environment | |
| Huang et al. | Planning motions and placements for virtual demonstrators | |
| CN110853131A (en) | Virtual video data generation method for behavior recognition | |
| Schmitz et al. | A simulation framework for human-robot interaction | |
| Rossmann et al. | The virtual testbed: Latest virtual reality technologies for space robotic applications | |
| CN100361074C (en) | Visualized three-dimensional graphic control programming method | |
| CN107738256A (en) | A kind of teach-by-doing apery teaching robot's programing system | |
| Wan et al. | A method of motion-based immersive design system for vehicle occupant package | |
| Hempe et al. | A semantics-based, active render framework to realize complex eRobotics applications with realistic virtual testing environments | |
| Guo et al. | Mixing control of animating virtual human for maintenance simulation | |
| CN113126568B (en) | Industrial robot operation and demonstration system based on augmented reality technology | |
| CN110782533B (en) | System for virtual role interaction control in virtual previewing | |
| Gaisbauer et al. | Introducing a Modular Concept for Exchanging Character Animation Approaches. | |
| Yaoyao et al. | Study on the method of intelligent maintenance simulation based on the fusion of human motion data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171201 |
|
| RJ01 | Rejection of invention patent application after publication |