CN113128716A

CN113128716A - Operation guidance interaction method and system

Info

Publication number: CN113128716A
Application number: CN202110450554.1A
Authority: CN
Inventors: 赵青; 李晓兴; 单桂华; 田东
Original assignee: Computer Network Information Center of CAS
Current assignee: Computer Network Information Center of CAS
Priority date: 2021-04-25
Filing date: 2021-04-25
Publication date: 2021-07-16

Abstract

The invention discloses an operation guidance interaction method and system. Wherein, the method comprises the following steps: establishing a virtual model corresponding to a railway construction scene; displaying the virtual model through VR equipment, and receiving operation interaction data of operation interaction between the VR equipment and the virtual model; and generating a real-time interactive video of the VR equipment in the virtual model according to the operation interactive data, and sending the real-time interactive video to a display screen for displaying. The invention solves the technical problems that the operation guidance mode in the related technology is not real enough, and the guided person has real feeling, which results in lower guidance efficiency.

Description

Operation guidance interaction method and system

Technical Field

The invention relates to the field of operation guidance, in particular to an operation guidance interaction method and system.

Background

The high-quality rapid development of the modern railway of China puts forward higher requirements on ensuring the benign operation of a railway system and reducing the occurrence probability of safety accidents, and further improves the accident prevention, emergency treatment and rescue capabilities of railway practitioners on the basis of the profession and the normalization urgently.

The traditional railway disease treatment teaching is site construction teaching, and the flow is theoretical teaching, construction preparation, corresponding treatment operation aiming at different diseases, cleaning and removing yard. The teaching process is complex and is also influenced by the actual conditions such as weather, railway conditions and the like.

In the related art, the railway operation guidance usually only stays on the text image or the video in the traditional teaching mode, and is difficult for operators to feel and experience due to too large area. Therefore, the traditional railway operation guidance is caused, and the efficiency of guiding the operation personnel is low.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the invention provides an operation guidance interaction method and system, which at least solve the technical problems that an operation guidance mode in the related technology is not real enough, and a guided person has real feeling, so that the guidance efficiency is low.

According to an aspect of an embodiment of the present invention, there is provided a method for interacting work guidance, including: establishing a virtual model corresponding to a railway construction scene; displaying the virtual model through VR equipment, and receiving operation interaction data of operation interaction between the VR equipment and the virtual model; and generating a real-time interactive video of the VR equipment in the virtual model according to the operation interactive data, and sending the real-time interactive video to a display screen for displaying.

Optionally, the establishing of the virtual model corresponding to the railway construction scene includes: scaling in equal proportion according to the real sizes of the buildings, the equipment and the tools in the railway construction scene to obtain the virtual sizes of the virtual buildings, the virtual equipment and the virtual tools in the virtual model; and establishing the virtual model through a physical simulation engine according to the virtual size.

Optionally, after the virtual model is established by the physical simulation engine according to the virtual size, the method further includes: determining pose information of the VR device in the virtual model according to the operation interaction data of the VR device; rendering the virtual model through a three-dimensional scene rendering engine according to the pose information, and determining the display effect of the virtual model for displaying the VR equipment.

Optionally, displaying the virtual model through a VR device, and receiving job interaction data of the VR device and the virtual model for job interaction includes: sending the virtual model to a display device of the VR device for displaying; and receiving operation information of operation equipment of the VR equipment, and determining job interaction data of the operation equipment in the virtual model.

Optionally, generating a real-time interactive video of the VR device in the virtual model according to the job interaction data, and sending the real-time interactive video to a display screen for displaying includes: performing simulation processing on the virtual model according to the operation interaction data to obtain a real-time interaction video of the VR equipment in the virtual model; and sending the interactive video to the display screen for displaying.

Optionally, after sending the interactive video to the display screen for displaying, the method further includes: receiving control operation data of the display screen; and performing control operation on the virtual model according to the control operation data, wherein the control operation comprises at least one of prompting fault information in the virtual model, modifying the display process of the virtual model, and suspending or terminating the display of the virtual model.

According to another aspect of the embodiments of the present invention, there is also provided a work guidance interaction system, including: the system comprises a server, VR equipment and a display, wherein the server comprises a virtual model building module, and the virtual model building module is used for building a virtual model of a railway construction scene; the VR equipment is in data connection with the server, and is used for displaying the virtual model and performing operation interaction with the virtual model; the display is in data connection with the server and used for displaying a real-time interactive video of the VR device in the virtual model.

Optionally, the VR device includes: the display device is used for displaying the virtual model; the operation equipment is used for performing operation interaction operation in the virtual model; and the transmission and synchronization equipment is used for transmitting the operation interactive operation of the operation equipment to the server and synchronizing the operation interactive operation and the virtual model in real time.

Optionally, the server further includes: the simulation processing module is used for adjusting the virtual model displayed by the display equipment according to the operation interactive operation of the operating equipment; and the rendering output module is used for integrating and rendering the operation interactive operation of the VR equipment and the virtual model to obtain an interactive video and outputting the interactive video to the display.

Optionally, the display includes: the rendering and displaying device is used for rendering and displaying the interactive video of the rendering and outputting module; and the control operation module is used for controlling the virtual model in the interaction.

Optionally, the method further includes: and the real object display cabinet is used for displaying the virtual object in the virtual model.

According to another aspect of the embodiments of the present invention, there is also provided a processor, configured to execute a program, where the program executes to perform the job guidance interaction method described in any one of the above.

According to another aspect of the embodiments of the present invention, there is also provided a computer storage medium, where the computer storage medium includes a stored program, and when the program runs, the apparatus where the computer storage medium is located is controlled to execute any one of the above-mentioned job guidance interaction methods.

In the embodiment of the invention, a virtual model corresponding to a railway construction scene is established; displaying the virtual model through VR equipment, and receiving operation interaction data of operation interaction between the VR equipment and the virtual model; according to the operation interactive data, a real-time interactive video of the VR equipment in the virtual model is generated, the real-time interactive video is sent to the display screen to be displayed, operation guidance is conducted on the operation personnel through the VR equipment, operation guidance learning of the operation personnel in the presence is achieved, the impression of the operation personnel is deepened, and the purpose of feeling the actual situation of an operation scene is achieved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1 is a flow diagram of a method of job guidance interaction according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a work guidance interaction system according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a hardware device architecture of a system according to an embodiment of the invention;

FIG. 4 is a schematic diagram of functional modules of a system according to an embodiment of the invention;

FIG. 5 is a schematic diagram of a work guidance interaction system, according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In accordance with an embodiment of the present invention, there is provided a method embodiment of a method of work direction interaction, it being noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than presented herein.

Fig. 1 is a flowchart of a method for job guidance interaction according to an embodiment of the present invention, as shown in fig. 1, the method including the steps of:

step S102, establishing a virtual model corresponding to a railway construction scene;

step S104, displaying the virtual model through VR equipment, and receiving operation interaction data of operation interaction between the VR equipment and the virtual model;

and S106, generating a real-time interactive video of the VR equipment in the virtual model according to the operation interactive data, and sending the real-time interactive video to a display screen for displaying.

Through the steps, a virtual model corresponding to a railway construction scene is established; displaying the virtual model through VR equipment, and receiving operation interaction data of operation interaction between the VR equipment and the virtual model; according to the operation interactive data, a real-time interactive video of the VR equipment in the virtual model is generated, the real-time interactive video is sent to the display screen to be displayed, operation guidance is conducted on the operation personnel through the VR equipment, operation guidance learning of the operation personnel in the presence is achieved, the impression of the operation personnel is deepened, and the purpose of feeling the actual situation of an operation scene is achieved.

The railway construction scene can be various scenes in the railway construction process, and can be a railway engineering construction scene, which is called an engineering scene for short and comprises scenes such as railway laying, bridge construction, railway maintenance and the like; the system can also be used in railway power supply construction scenes, such as power line laying, maintenance and inspection, and communication line laying, maintenance and inspection; the method can also be used in railway electric service construction scenes, such as the installation and maintenance of ground signal devices, the installation and maintenance of locomotive signal devices, the control and maintenance of railway turnouts and the like; the method can also be used in railway vehicle service construction scenes, such as vehicle inspection, vehicle equipment inspection, vehicle scheduling and the like; the method can also be used in railway logistics construction scenes, such as railway material management, warehouse management, material inspection and the like.

The corresponding virtual model may include a building model, a background and a placed object model in a specific scene, and may also include a target object model of the scene, for example, a railway laying scene, which requires targets such as sleepers, road beds and rails, and the target objects are used for operating according to a preset sequence and manner to achieve the operation purpose of the scene. That is, the work guidance for the operator can be realized by the target object. Additionally, in some embodiments, the target may also include a grandfather's tool, such as a rail laying locomotive, a wrench, an electric hammer, and the like.

The VR device is also a Virtual Reality device, Virtual Reality. The VR device may be a wearable device, e.g., VR glasses, VR handle, etc. The operation personnel can experience through the immersion type through VR equipment, establish the image sound of the above-mentioned virtual model of impression of body position, even temperature, information such as sense of touch, can also carry out the operation through VR equipment, simulate real operation process in the virtual model, in order to realize reinforcing the operation personnel and carry out the impression of operation guidance, and then reached and made the operation personnel can be personally on the scene and carried out the operation and guide the study, the impression of operation personnel has been deepened, experience the actual conditions's of experience operation scene more really, thereby realized making the operation personnel study more deeply, improve the technological effect to the instruction efficiency of operation personnel and sexual guidance, and then solved the mode of operation guidance in the correlation technique, it is not real enough, it has real impression of cutting into the body to be difficult to make by the person who is guided, lead to guiding the lower technical problem of efficiency.

Specifically, the virtual model is displayed through the VR device, and job interaction data of job interaction between the VR device and the virtual model is received. Therefore, interaction between the virtual model and the operator is formed, and the aim of performing operation guidance training on the operator is fulfilled.

Above-mentioned display screen can show the real-time interaction video of VR equipment in virtual model, also is the real-time operation interaction video of operating personnel in virtual model, shows through third party's visual angle, can supply the third party to look over the operation process of operating personnel in virtual model.

Can also be provided with controlling means on the above-mentioned display screen, indicate and guide the operation interaction of operation personnel in virtual model through VR equipment, it is more convenient to use, has further improved the accuracy of carrying out the operation guidance to the operation personnel to further strengthen operation personnel's instruction impression and guide efficiency.

Optionally, the establishing of the virtual model corresponding to the railway construction scene includes: scaling in equal proportion according to the real sizes of buildings, equipment and tools in a railway construction scene to obtain the virtual sizes of virtual buildings, virtual equipment and virtual tools in a virtual model; and establishing a virtual model through a physical simulation engine according to the virtual size.

The operation guidance process can be that the actual scene is displayed, the operator only observes and checks through the VR device to experience the feeling of the actual scene, and can also be that the operation is carried out in the actual scene, and the operator checks the actual scene through the VR device and operates through the VR device.

When the virtual model is established, the real sizes of buildings, equipment, tools and the like in a railway construction scene are scaled in an equal proportion, so that the virtual model is closer to a real construction scene, the real feeling of operators is further improved, and the guidance efficiency of the operators is improved.

And establishing a virtual model through a physical simulation engine according to the virtual size obtained by scaling the real size of the real building, the real equipment and the real tool in equal proportion. The above physical simulation engine is prior art.

Optionally, after the virtual model is established by the physical simulation engine according to the virtual size, the method further includes: determining pose information of the VR equipment in the virtual model according to the operation interaction data of the VR equipment; rendering the virtual model through a three-dimensional scene rendering engine according to the pose information, and determining the display effect of the virtual model for displaying the VR equipment.

According to the operation interaction data, the pose information of the VR device in the virtual model, namely the position of the operator in the virtual model, can be determined, and the three-dimensional scene is rendered, so that the display effect of the virtual model on the VR device is determined, the display effect comprises the display visual angle of the virtual model, the display visual angle comprises the shape change and the color change of the virtual model, and the real action feeling of each model in the virtual model on the VR device is determined, and the efficiency and the effect of operation guidance on the operator are enhanced.

Optionally, displaying the virtual model through the VR device, and receiving job interaction data of job interaction between the VR device and the virtual model includes: sending the virtual model to display equipment of the VR equipment for displaying; and receiving operation information of the operation equipment of the VR equipment, and determining job interaction data of the operation equipment in the virtual model.

The VR device may include a display device for displaying scenes and contents of the virtual model, and the operation device is used for acquiring and processing spatial gestures operated by an operator through the VR operation device, and transmitting and synchronizing the spatial gestures interacted with the operation through the synchronization device and synchronizing the scenes and the contents of the virtual model in real time.

The operation information may be the spatial gesture, and may further include a moving position, a moving speed, and the like of the user.

Optionally, according to the job interaction data, a real-time interaction video of the VR device in the virtual model is generated and sent to a display screen for displaying, including: performing simulation processing on the virtual model according to the operation interaction data to obtain a real-time interaction video of the VR equipment in the virtual model; and sending the interactive video to a display screen for displaying.

According to the operation interaction data, a real-time interaction video of the VR equipment in the virtual model is generated by combining the virtual model and is sent to the display screen to be displayed, so that the visual angle of the VR equipment can be displayed through a third party, and the VR equipment can be used for the third party to check the operation process of an operator in the virtual model.

Optionally, after sending the interactive video to the display screen for displaying, the method further includes: receiving control operation data of a display screen; and performing control operation on the virtual model according to the control operation data, wherein the control operation comprises at least one of prompting fault information in the virtual model, modifying the display process of the virtual model, and suspending or terminating the display of the virtual model.

During prompting and guiding, a virtual prompt box can be generated, the virtual prompt box is suspended in the center of the visual angle of the VR equipment, prompting is carried out through characters or images, and fault information in a virtual model can be prompted in a mode of highlighting a flashing fault component. The display process of the virtual model can be modified through control, namely, the change display process relative to the VR equipment is modified when the virtual model is dynamically changed. The display of the virtual model can be paused or terminated, that is, the working process of the operator in the virtual model is paused or terminated, so that the working instruction of the operator is paused or terminated.

It should be noted that the present application also provides an alternative implementation, and the details of the implementation are described below.

The virtual reality technology based on VR in the embodiment has the advantages of high immersion, real and accurate simulation, safety and practicality, no risk, low investment, customization, expandability and the like, can exert the unique advantages of experience education, combines the customization requirement of the operation management of the general railways, develops visual experience teaching on key contents such as railway safety protection, railway traffic accident emergency treatment, railway traffic accident rescue, railway safety monitoring and the like, realizes the full coverage of main content main points of daily operation, improves the training strength and the observation learning efficiency, learns the normal operation standard, strengthens the safety key risk control, promotes personnel to be familiar with emergency treatment programs, enhances the safety responsibility consciousness of staff, cultivates the good habit of complying with the management standard and the operation standard, constructs a high-tech vocational skill training system consistent with the general railways, the service quality of the practitioner is comprehensively improved, the method has stronger long-term competitive advantage and is beneficial to stabilizing the green, efficient and digital status of the world first-class heavy haul railway of the general plastic railway.

The railway safety experience type education system based on the virtual reality technology is developed and applied, under the technical innovation mode of deep fusion for production and research, the key technical innovation and the industrialized application are developed in the important fields of railway construction, operation, safety and the like by combining a new generation of information technology, artificial intelligence and actual production requirements of enterprises.

Our country has a considerable distance to research on virtual reality technology and some developed countries abroad, and the application cases in railway and medical military are far behind that of developed countries such as the united states, and the virtual reality technology is initially used for simulation training of astronauts and flight drivers by united states military. With the continuous development of science and technology and society, the virtual reality technology is gradually changed to provide and use for various industries. By now, successful cases in a plurality of fields including aviation VR training systems, VR medical operation training systems and the like are realized, along with the development of computer system engineering and computer graphics technology, people in all the world of China pay more and more attention to virtual reality, but no good successful case exists for VR training systems related to railway engineering at present, so that the application prospect is in a blue sea state at present.

The core technology mainly comprises an environmental physical simulation technology. The core content of the virtual reality technology when the virtual environment is built is a difficult problem of technology development how to make the virtual environment closer to reality as much as possible, and the purpose of the physical simulation technology is to restore a physical model of the real world through a computer physical simulation algorithm and create the most 'real' virtual environment. Is suitable for the development of humanized natural human-computer interaction. Although the immersion can be enhanced by the aid of equipment such as a virtual reality helmet and the like, the interaction between a person and a virtual environment is lack of the immersion and feedback, the interaction mode uses the most natural visual sense, tactile sense, auditory sense and natural body sensing action, and the interaction effect of virtual reality can be effectively improved.

The railway safety experience type education system is based on a virtual reality technology and is matched with an autonomously defined physical simulation engine, a three-dimensional scene rendering engine, a panoramic video playing engine and the like to carry out customized immersion type scene construction on the working processes of work, power supply, electric service, vehicle service, logistics and the like, and staff can carry out interactive operation in a comfortable mode under the natural and convenient scene guidance, not only can personally experience various dangerous occurrence scenes in a high-reduction-degree mode, but also can know various factors causing accidents in the experience process and carry out cognition and learning on the dangerous occurrence scenes. Meanwhile, the system also supports a free experience mode, simulation operation is carried out in person in a VR environment, knowledge mastered by the system is flexibly used, and dangerous events are avoided.

The key technology comprises the following steps: (1) a three-dimensional rendering method based on virtual reality; (2) a physical simulation algorithm based on a virtual scene; (3) a positioning algorithm in the virtual reality room; (4) capturing recognition based on the handle position.

The technical scheme and the application product of the railway safety experience type education system based on the virtual reality technology. The development of a virtual reality railway staff safety education simulation method verification system is completed, a novel immersion type guidance case in a display form is realized in the railway staff safety practical education, a real-time guidance feedback function and a crisis experience function are customized through technologies such as a physical simulation algorithm and spatial positioning, a brand-new guidance education mode is provided for the relevant education field of railway staff, the education efficiency is accelerated, the staff learning enthusiasm is improved, and the work risk is greatly reduced.

The innovation points of the embodiment are as follows: (1) virtual experience, exploring the possibility of fusion of railway engineering training and virtual reality technology; (2) virtual interaction, simulating disaster accidents through physical simulation; (3) virtual learning, namely deepening the memory of a student and improving the learning efficiency through experience and real operation; (4) virtual extension, which has the extension possibility, can interface more modules and simulation contents in the future.

FIG. 2 is a schematic diagram of a work guidance interaction system according to an embodiment of the present invention, as shown in FIG. 2, the system including the following:

a physical simulation engine: the virtual environment establishment is the core content of the virtual reality technology, and the physical simulation engine aims to restore a physical model of the real world through a computer physical simulation algorithm, form an object simulation engine through developing a physical simulation algorithm, restore the variables of gravity, speed, acceleration, friction and the like of an object in the virtual environment, support a user-defined interface and automatically define key physical parameters, so that a desired physical model is established, and simulation guarantee is provided for the establishment of different types of virtual environments.

A three-dimensional scene rendering engine: the three-dimensional scene rendering engine is a dependence engine which is the most basic and the most bottom layer of a virtual reality railway safety experience type education system, provides a visualization algorithm in a virtual reality environment for grid data, model data and the like, provides basic visualization algorithms such as time sequence organization, grid scatter point drawing, surface drawing and the like, and completes contents such as space mapping, color mapping and the like.

Panoramic video playing engine.

A virtual interaction engine.

And (3) integrally designing software and hardware.

The three-dimensional rendering method based on virtual reality, the physical simulation algorithm based on virtual scenes, the indoor positioning algorithm based on virtual reality and the method based on handle positioning, capturing and identifying can be realized.

Along with the rapid development of railways in China, higher requirements are put forward for technical literacy of railway practitioners, and the characteristics of structures and common diseases need to be comprehensively familiar from the aspects of professionalism and normalization, and identification, detection and treatment are completed, so that the safety operation consciousness is enhanced, and the potential safety hazard is reduced.

The construction of education training base utilizes the advanced teaching training system that collects simulation practice teaching system and software platform integration technical solution as an organic whole to advanced technology, the security is good, the function is perfect, the sightseeing show of fidelity and sightseeing experience system two parts content are the main parts, daily work and the training of professional skill of practitioner are organically linked, through building a high-precision removable reduced scale model and virtual training experimental environment that have the uniformity with real railway operation system in essence, lifelike, accurate, easy-to-use, easy maintenance, it is stable, reliable, the expansibility is strong, the practicality is high, realize the full coverage of daily operation main content main points, improve training dynamics and sightseeing learning efficiency, promote personnel to be familiar with the incident procedure, shorten training time. A high-tech professional skill training system consistent with the railway transportation business is constructed, the business quality of the practitioner is comprehensively improved, and the long-term competitive advantage is stronger.

This embodiment has the following advantages:

1. reliability;

the main equipment in the system adopts mature technology and technology, all links from the server, the rendering workstation to the virtual reality helmet and the spliced large screen run normally, the requirements on functions, performance and the like are met, and the mature reliability of the technology is ensured. Meanwhile, in consideration of the reliability, stability and safety of the system, a certain resource margin is reserved during computer type selection and hardware configuration so as to ensure that all parts can work stably under the highest operation load of the system, and the training education system can be used continuously for a long time.

2. Ease of use;

the system interface is simple in operation, easy to use and flexible, and the style is uniform and easy to learn. The system has complete help documents such as a use instruction, a guide book and the like, and is easy to use software. All operating systems adopt the Chinese Windows 10 system and the versions, all interactive systems provide Chinese graphical interfaces, accord with the operating mode of a conventional window system, and can skillfully master the operation of the whole system through short-term training.

3. Easy maintenance;

software and hardware equipment in the system have good maintainability, can be assembled, disassembled and adjusted in a modular mode, and is convenient for daily maintenance. At the same time, the system must have a lower maintenance cost.

4. The system is scalable and extensible;

the system needs to adopt a modular design, the system can meet the requirement of long-term use, and the reserved interface meets the requirements of later-stage upgrading and expansion.

5. Technical maturity and advancement;

the system is designed from the overall structure to the adoption of key technology and follows the principle of advanced practicality, the overall stability of the system is high, and the comprehensive reliability of all links from the server, the rendering workstation to the virtual reality helmet and the spliced large screen to normal operation and the core system can reach a higher level.

6. Authenticity;

the structural model is reduced by adopting full-simulation railway equipment according to a certain proportion, so that the internal structure can be truly displayed, and the observation and study are more intuitive.

The system of the present embodiment mainly includes the following four major components:

the method comprises the steps of displaying a bridge, tunnel and culvert structure model, treating common diseases and common equipment, strengthening and detecting equipment, performing VR (virtual reality) standard operation and performing safety education experience.

(1) And (3) displaying the bridge, tunnel and culvert structure model:

and selecting a representative ground feature, a landform and a structure, displaying bridge, tunnel and culvert related equipment in a bridge operation state in the system, uniformly scaling each model according to the actual size and the structure of a design drawing in proportion, and ensuring that the model is completely real in detail, attractive in whole and convenient to maintain. The model can be disassembled and assembled manually, so that the student can conveniently disassemble and learn at the same time, or the structure can be conveniently known by the student through the main structure in the local display structure.

(2) Common diseases of equipment, a method for remedying the diseases and a reinforcing material are displayed;

the corresponding contents are displayed by adopting a character and photo display board or a large screen, and the real objects are placed in the display cabinet for display by adopting a mode of combining images and descriptions, so that the students can browse and study conveniently.

(3) Equipment strengthening and detection standardization work;

because the increase of the service life of railway bridges, tunnels and culverts and the increasing of load in the use process, the bridge, tunnels and culverts equipment easily generates damage or diseases of different degrees, and in order to guarantee the safety of normal use and railway transportation of the equipment, the standard detection needs to be carried out on the bridge, tunnels and culverts structure in the daily process, the scheme of reinforcing or rebuilding is determined according to the detection result, and the equipment of aging, fatigue and damage of the structure is subjected to the standard operation strengthening and reinforcing.

3D model animation or video is adopted to display various reinforced construction or detection processes in a three-dimensional mode, so that students can comprehensively know the construction process, the consciousness of standard operation is deepened, and necessary conditions are provided for railway safety operation. The video or animation display method can set up an independent operation table through a large screen, and can manually carry out operation control and selective playing on the displayed content.

(4) VR standardized work and safety education experience;

the virtual reality technology based on VR has the advantages of high immersion, real and accurate simulation, safety, practicality, no risk, low investment, customization, expandability and the like, can give play to the unique advantages of experience type education, combines the customization requirement of the operation management of the general railway, develops visual experience teaching on key contents such as railway safety protection, railway traffic accident emergency treatment, railway traffic accident rescue, railway safety monitoring and the like, realizes the full coverage of main content key points of daily operation, improves the training force and the observation learning efficiency, learns the normal operation standard, strengthens the safety key risk control, promotes personnel emergency event processing procedures, enhances the safety consciousness of staff, cultivates the good habit of complying with the management standard and the operation standard, constructs a high professional skill training system consistent with the railway transportation service, and comprehensively improves the service quality of the staff, has stronger long-term competitive advantage. Under the technical innovation mode of deep fusion for production and research, the key technical innovation and the industrialized application are developed in the important fields of railway construction, operation, safety and the like by combining a new generation of information technology, artificial intelligence and the actual production requirements of enterprises.

The railway safety experience type education system is based on a virtual reality technology and is matched with an autonomously defined physical simulation engine, a three-dimensional scene rendering engine, a panoramic video playing engine and the like to carry out customized immersion type scene construction on the working processes of work, power supply, electric service, vehicle service, logistics and the like, and staff can carry out interactive operation in a comfortable mode under the natural and convenient scene guidance, not only can personally experience various dangerous occurrence scenes in a high-reduction-degree mode, but also can know various factors causing accidents in the experience process and carry out cognition and learning on the dangerous occurrence scenes. Meanwhile, the system also supports a free experience mode, simulation operation is carried out in person in a VR environment, knowledge mastered by the system is flexibly used, and dangerous events are avoided. The technical scheme and the application product of the railway safety experience type education system based on the virtual reality technology. The development of a virtual reality railway staff safety education simulation method verification system is completed, a novel immersion type guidance case in a display form is realized in the railway staff safety practical education, a real-time guidance feedback function and a crisis experience function are customized through technologies such as a physical simulation algorithm and spatial positioning, a brand-new guidance education mode is provided for the relevant education field of railway staff, the education efficiency is accelerated, the staff learning enthusiasm is improved, and the work risk is greatly reduced.

The system of this embodiment also includes a VR standardized work guidance subsystem.

The VR standardized operation guidance subsystem is based on a virtual reality technology, combines an interaction algorithm, a cooperation algorithm and the like which are automatically defined, and is designed according to a story script according to standardized operation data in a project and oriented to the running environment of VR glasses, a bridge and tunnel virtual model to be operated is established and fused into a real space in a virtual-real mode, an interaction method based on gestures is provided, and a method and content are provided for VR display and interaction. The VR standardized operation guidance subsystem supports the cooperative interaction of a plurality of glasses terminals, shares interaction parameters and scene parameter information on the plurality of glasses terminals in real time, and realizes information synchronization among interactive virtual reality demonstration tool software on the plurality of glasses terminals.

The VR standardized operation guidance subsystem provides a function of transmitting the pictures of the glasses terminal to the monitoring end in real time, and a teacher can remotely monitor the operation pictures of a student in real time to comprehensively master the process.

The VR standardized operation guidance subsystem is composed of two integrated VR devices, a wireless router, a computer, a display, a split VR glasses, two operating handles and two positioning base stations, students can enter a virtual scene in a simulation mode to experience correct operation processes, or virtual models and real models to be operated are fused into a real space, and the virtual models are checked under the guidance of the system through walking, gesture capture interaction and other modes.

VR incident simulation experience system.

The VR safety accident simulation experience subsystem is based on a virtual reality technology, is matched with an autonomously defined physical simulation engine, a three-dimensional scene rendering engine, a panoramic video playing engine and the like, and carries out immersive scene construction on the operation processes of work, power supply, electricity, vehicle, logistics and the like, and an experiencer can operate in a comfortable interactive mode under natural interactive guidance, not only can personally experience various dangerous scenes in a high-reduction form, but also can know various factors causing accidents in the experience process and learn the factors. Meanwhile, the system also supports a free experience mode, simulation operation is carried out in person in a VR environment, knowledge mastered by the system is flexibly used, and dangerous events are avoided.

The VR safety accident simulation experience subsystem is mainly composed of a work accident experience module, a power supply accident experience module, a power accident experience module, a vehicle accident experience module, a logistics accident experience module and a free experience module, and an experiencer can autonomously select to switch a scene to be experienced under voice guidance.

The VR safety accident simulation experience subsystem is composed of a computer, a display, a pair of split VR glasses, two operating handles and two positioning base stations. The students can enter the virtual scene in a simulation mode to experience and learn various accidents possibly occurring in the operation process, so that the purpose of safety education is achieved.

Fig. 3 is a schematic diagram of a hardware device architecture of a system according to an embodiment of the present invention, and as shown in fig. 3, the upper diagram is a system architecture diagram, and system hardware includes two integrated VR devices, a display, a split VR glasses, and two operating handles. The trainees can enter a virtual scene in a simulation mode to experience a correct operation process, or the virtual and real models to be operated are fused into a real space, and the virtual models are checked under the guidance of a system through walking, gesture capturing interaction and the like.

Fig. 4 is a schematic diagram of functional modules of a system according to an embodiment of the present invention, and as shown in fig. 4, there are two modules in a display, including a console operation module and a large screen high definition rendering module.

The VR equipment is internally provided with three modules, the display equipment is responsible for displaying scenes and contents, the operation equipment is responsible for acquiring and processing spatial gestures of a user, and the transmission and synchronization equipment is responsible for transmitting user interaction gestures and synchronizing real-time scenes and contents.

The workstation is divided into three modules, each function is as follows:

the model preprocessing module is responsible for analyzing and organizing the three-dimensional model and completes the analyzing and organizing capability of the simulation data, the chartlet texture and other data related to the three-dimensional model. And the method supports the input of not less than three common model grid data formats, the input of not less than three format texture data, the self-defined path index and the dynamic loading of data.

And the simulation processing module completes the simulation processing capacity of the simulation scene. And a physical simulation algorithm is supported, and rendering of particles, two-dimensional planes and three-dimensional models is supported.

The rendering output module completes the output display supporting capacity of various display terminals and provides the back display capacity of the traditional display. The integrated virtual-real integrated picture output and interaction of the integrated virtual-real terminal are supported, the picture can be rendered in the real environment of a user in a three-dimensional mode, the immersive picture output and interaction of the split virtual-real terminal are supported, and the picture of the terminal is supported to be transmitted back to display equipment such as a display and a television.

When the method is implemented, the VR standardized operation guidance system is based on a virtual reality technology, combines an interaction algorithm, a cooperation algorithm and the like which are automatically defined, faces to the running environment of VR glasses according to standardized operation data in a project, is designed according to a story script, establishes a bridge-tunnel virtual model to be operated, fuses virtual and real into a real space, provides an interaction method based on gestures, and provides a method and content for VR display and interaction.

The VR standardized operation guidance system generally adopts a mode of combining integrated VR equipment with split VR equipment, comprises a bridge manufacturing replacement operation module and a bridge manufacturing defect inspection operation module, and can provide different interaction forms and picture presentation forms according to different hardware environments.

The VR standardized operation guidance system supports the cooperative interaction of a plurality of glasses terminals, shares interaction parameters and scene parameter information on the plurality of glasses terminals in real time, and realizes information synchronization among interactive virtual reality demonstration tool software on the plurality of glasses terminals.

The VR standardized operation guidance system provides a function of transmitting the pictures of the glasses terminal to the monitoring end in real time, and a teacher can remotely monitor the operation pictures of a student in real time to comprehensively master the process.

The implementation mode provides a brand-new interaction mode and a brand-new display mode. And supports multi-person collaborative analysis. The virtual model is scaled, the details are completely real, the whole is beautiful, and the maintenance is convenient. The novel display form is a sink-dip type and a virtual-real fused guide case. A brand-new education guiding mode is formed, the education efficiency is accelerated, the learning enthusiasm of workers is improved, and the work risk is reduced.

Fig. 5 is a schematic diagram of a work guidance interaction system according to an embodiment of the present invention, and as shown in fig. 5, according to another aspect of the embodiment of the present invention, there is also provided a work guidance interaction system including: a server 52, a VR device 54 and a display 56, the system of which is described in detail below.

The server 52 comprises a virtual model building module, and the virtual model building module is used for building a virtual model of a railway construction scene; the VR device 54 is in data connection with the server 52, and the VR device 54 is used for displaying the virtual model and performing operation interaction with the virtual model; the display 56 is in data connection with the server 52 for presenting real-time interactive video of the VR device 54 in the virtual model.

By the system, a virtual model corresponding to a railway construction scene is established; displaying the virtual model through VR equipment, and receiving operation interaction data of operation interaction between the VR equipment and the virtual model; according to the operation interactive data, a real-time interactive video of the VR equipment in the virtual model is generated, the real-time interactive video is sent to the display screen to be displayed, operation guidance is conducted on the operation personnel through the VR equipment, operation guidance learning of the operation personnel in the presence is achieved, the impression of the operation personnel is deepened, and the purpose of feeling the actual situation of an operation scene is achieved.

Optionally, the VR device includes: the virtual model display system comprises display equipment, operating equipment and transmission and synchronization equipment, wherein the display equipment is used for displaying the virtual model; the operation equipment is used for performing operation interactive operation in the virtual model; and the transmission and synchronization equipment is used for transmitting the job interactive operation of the operation equipment to the server and synchronizing the job interactive operation and the virtual model in real time.

The VR device can comprise a display device, an operation device and a transmission and synchronization device, wherein the display device is used for displaying scenes and contents of the virtual model, the operation device is used for acquiring and processing space gestures operated by an operator through the VR operation device, and the transmission and synchronization device is used for transmitting space gestures of operation interaction and synchronizing the scenes and the contents of the virtual model in real time.

Optionally, the server further includes: the simulation processing module is used for adjusting the virtual model displayed by the display equipment according to the operation interactive operation of the operation equipment; and the rendering output module is used for integrating and rendering the operation interactive operation of the VR equipment and the virtual model to obtain an interactive video and outputting the interactive video to the display.

When the virtual model is established, the real sizes of buildings, equipment, tools and the like in a railway construction scene are scaled in an equal proportion through the simulation processing module, so that the virtual model is closer to a real construction scene, the real feeling of operators is further improved, and the guidance efficiency of the operators is improved.

Optionally, the display comprises: the rendering and displaying device is used for rendering and displaying the interactive video of the rendering and outputting module; and the control operation module is used for controlling the virtual model in the interaction.

Optionally, the method further includes: and the physical display cabinet is used for displaying the virtual objects in the virtual model.

Through the material object display cabinet, the operator can correspond the virtual model in the virtual model with the real material object, can carry out the backstepping on the operation interaction process in the virtual model, and correspond with food, deepen the impression, further improved the instruction efficiency of carrying out the operation guidance to the operator.

According to another aspect of the embodiments of the present invention, there is also provided a processor, configured to execute a program, where the program executes to perform the job guidance interaction method of any one of the above.

According to another aspect of the embodiments of the present invention, there is also provided a computer storage medium, where the computer storage medium includes a stored program, and when the program runs, the apparatus where the computer storage medium is located is controlled to execute the job guidance interaction method of any one of the above.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A method of work direction interaction, comprising:

establishing a virtual model corresponding to a railway construction scene;

displaying the virtual model through VR equipment, and receiving operation interaction data of operation interaction between the VR equipment and the virtual model;

and generating a real-time interactive video of the VR equipment in the virtual model according to the operation interactive data, and sending the real-time interactive video to a display screen for displaying.

2. The method of claim 1, wherein establishing the virtual model corresponding to the railway construction scenario comprises:

scaling in equal proportion according to the real sizes of the buildings, the equipment and the tools in the railway construction scene to obtain the virtual sizes of the virtual buildings, the virtual equipment and the virtual tools in the virtual model;

and establishing the virtual model through a physical simulation engine according to the virtual size.

3. The method of claim 2, wherein after building the virtual model by a physics simulation engine according to the virtual dimensions, further comprising:

determining pose information of the VR device in the virtual model according to the operation interaction data of the VR device;

rendering the virtual model through a three-dimensional scene rendering engine according to the pose information, and determining the display effect of the virtual model for displaying the VR equipment.

4. The method of claim 1, wherein exposing the virtual model via a VR device and receiving job interaction data for job interactions by the VR device with the virtual model comprises:

sending the virtual model to a display device of the VR device for displaying;

and receiving operation information of operation equipment of the VR equipment, and determining job interaction data of the operation equipment in the virtual model.

5. The method of claim 1, wherein generating a real-time interactive video of the VR device in the virtual model from the job interaction data and sending the real-time interactive video to a display screen for display comprises:

performing simulation processing on the virtual model according to the operation interaction data to obtain a real-time interaction video of the VR equipment in the virtual model;

and sending the interactive video to the display screen for displaying.

6. The method of claim 5, wherein after sending the interactive video to the display screen for display, further comprising:

receiving control operation data of the display screen;

and performing control operation on the virtual model according to the control operation data, wherein the control operation comprises at least one of prompting fault information in the virtual model, modifying the display process of the virtual model, and suspending or terminating the display of the virtual model.

7. A work guidance interaction system, comprising: the server, the VR device and the display,

the server comprises a virtual model building module, wherein the virtual model building module is used for building a virtual model of a railway construction scene;

the VR equipment is in data connection with the server, and is used for displaying the virtual model and performing operation interaction with the virtual model;

the display is in data connection with the server and used for displaying a real-time interactive video of the VR device in the virtual model.

8. The work guidance interaction system of claim 7, wherein the VR device comprises: a presentation device, an operating device, and a transmission and synchronization device,

the display equipment is used for displaying the virtual model;

the operation equipment is used for performing operation interaction operation in the virtual model;

and the transmission and synchronization equipment is used for transmitting the operation interactive operation of the operation equipment to the server and synchronizing the operation interactive operation and the virtual model in real time.

9. The work guidance interaction system of claim 8, wherein the server further comprises: a simulation processing module, a rendering output module,

the simulation processing module is used for adjusting the virtual model displayed by the display equipment according to the operation interactive operation of the operation equipment;

and the rendering output module is used for integrating and rendering the operation interactive operation of the VR equipment and the virtual model to obtain an interactive video and outputting the interactive video to the display.

10. The work guidance interaction system of claim 9, wherein the display comprises: a rendering and displaying device and a control operation module,

the rendering and displaying equipment is used for rendering and displaying the interactive video of the rendering output module;

and the control operation module is used for controlling the virtual model in the interaction.

11. The work guidance interaction system according to any one of claims 7 to 10, further comprising: a real object display cabinet, a display cabinet,

the physical display cabinet is used for displaying the virtual objects in the virtual model.

12. A processor, characterized in that the processor is configured to run a program, wherein the program when running performs the job guidance interaction method according to any one of claims 1 to 6.

13. A computer storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the computer storage medium is located to perform the method of any one of claims 1 to 6.