CN108010079A - Status information remote monitoring system and method based on projection fusion and image recognition - Google Patents
Status information remote monitoring system and method based on projection fusion and image recognition Download PDFInfo
- Publication number
- CN108010079A CN108010079A CN201710978943.5A CN201710978943A CN108010079A CN 108010079 A CN108010079 A CN 108010079A CN 201710978943 A CN201710978943 A CN 201710978943A CN 108010079 A CN108010079 A CN 108010079A
- Authority
- CN
- China
- Prior art keywords
- information
- equipment
- model
- status information
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/02—Electrically-operated educational appliances with visual presentation of the material to be studied, e.g. using film strip
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Educational Technology (AREA)
- Educational Administration (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention relates to a kind of status information monitoring system based on projection fusion and image recognition to include equipment body, sensor model, more than one status information monitoring unit；Sensor model equipment using when be placed on projection imaging and information gathering screen, for marking equipment to be observed；Shown using high-performance computer, ultrashort out-of-focus projection's equipment, projection imaging and information gathering screen, video fusion device and image high speed identification equipment progress scene, model information collection and fusion treatment are shown, the status information monitoring unit is connected by Ethernet with equipment body, for remote parameter configuration and status information interaction and monitors threedimensional model scene real time status information.Real-time cloth column-shaped state and work state information can be carried out distributed post, solve problem of detached island of information, real-time status is shown with a kind of three dimensional constitution directly perceived, image.
Description
Technical field
The present invention relates to computer man-machine interacting technical field, more particularly to it is a kind of based on projection fusion and image recognition
Status information monitoring system and control method.
Background technology
In recent years, with the hair of the technologies such as computer, optics, graphics, computer vision, virtual reality and augmented reality
Exhibition, various human-computer interaction technology and man-machine interactive equipment occur, based on the fusion of ultrashort out-of-focus projection and image recognition with
The universal interaction equipment of track technology be with reference to the ultrashort out-of-focus projection's technology of laser, image recognition tracking technique, virtual reality technology with
And a kind of of augmented reality ensures equipment and technology scheme.
Original same type interactive device mainly uses the year-on-year zoom model of physics, for aiding in related personnel, uses
The traditional means such as voice, video, visual safeguard the information such as equipment or device location, working condition, still use manually
Occupation mode, establishes the work situation information of entirety on interactive device, but does not possess the automatic identification ability of each zoom model,
Lacking intelligent automation means, different operative scenarios needs to build different physical models, and single device is unable to generalization, and
Interactive device is generally used in single scene, can not be in the multidisciplinary cooperating of different places.
The content of the invention
In view of above-mentioned analysis, the present invention is intended to provide a kind of status information based on projection fusion and image recognition monitors
System and control method, can not be in difference to overcome the scheme that can not provide a kind of general, configurable scene in the prior art
Place is automatic, global work situation in real time, is intuitively presented, and can not meet human factor engineering feature and provide a kind of new human-computer interaction
The problems such as mode.
The purpose of the present invention is mainly achieved through the following technical solutions：
In the one side based on the embodiment of the present invention, there is provided a kind of state based on projection fusion and image recognition is believed
Cease monitoring system, including equipment body, sensor model, more than one status information monitoring unit；The equipment body includes height
Project Computer, projection imaging and information gathering screen, video fusion device, ultrashort out-of-focus projection's equipment and image high speed identification equipment；
Sensor model equipment using when be placed on projection imaging and information gathering screen, for marking equipment to be observed；Using high-performance
Computer, ultrashort out-of-focus projection's equipment, projection imaging and information gathering screen, video fusion device and image high speed identification equipment carry out field
Scape is shown, model information gathers and fusion treatment is shown, the status information monitoring unit passes through Ethernet and equipment body
Connection, for remote parameter configuration and status information interaction and monitors threedimensional model scene real time status information.
In another embodiment based on present system, it is aobvious that each status information monitoring unit includes host, three-dimensional
The high-performance computer of display screen and manipulation unit, host and equipment body carries out data interaction, and three-dimensional display screen, which is used to show, to be seen
The three-dimensional simulation scene of equipment live and to be observed is surveyed, manipulation unit is used for remote parameter configuration and status information interaction.
In another embodiment based on present system, the manipulation stage body for installing each component devices is further included, is
Equipment provider's installation frame.
In another embodiment based on present system, when carrying out scene display, by the high-performance computer by field
Scape image outpuT carries out video dispatch ＆ display, by ultrashort by video fusion device to video fusion device according to required resolution ratio
Image projection to projection imaging and information gathering screen are carried out imaging and shown by out-of-focus projection's equipment；
When carrying out model information collection, two images above high speed identification equipments gather at the same time is placed on projection imaging and letter
Sensor model information on breath collection screen, and the information collected is transmitted to high-performance computer, merged by computer
Processing, and image recognition and tracking are carried out, the position of sensor model is calculated in real time；
When being shown for fusion treatment, high-performance computer is according to the virtual three-dimensional of the information architecture sensor model collected
Model, is merged with three-dimensional simulation scene, and the three-dimensional of status information monitoring unit is output this to by high-performance computer
Display screen carries out three-dimensional scenic and shows.
In another embodiment based on present system, further include light and adjust module, light adjusts module and is used to grasp
It is uniform to control light path inside platform, ensures accuracy of the internal image high speed identification equipment to image information collecting and identification.
In another embodiment based on present system, sensor model is the mould for marking a certain equipment or equipment
Type, has special identifier；During work, sensor model is placed on projection imaging and information gathering screen, drags sensor model to control
The simultaneous display of the corresponding threedimensional model of equipment to be observed under three-dimensional simulation scene processed.
In the other side based on the embodiment of the present invention, there is provided a kind of control method, including：
Step S1：System starts, and the video display information that ultrashort out-of-focus projection's equipment is provided according to high-performance computer, is throwing
The global working status two-dimensional scene of generation in shadow imaging and information gathering screen；
Step S2：Unique mark of the high-performance computer according to entrained by the sensor model being placed in two-dimensional scene, is obtained
Take the entity type and status information, progress status information Dynamic Maintenance of equipment to be observed；
Step S3：High-performance computer establishes three-dimensional simulation scene, is established according to the sensor model information collected and waits to see
The corresponding threedimensional model of measurement equipment is simultaneously merged with three-dimensional simulation scene, and high-performance computer will include threedimensional model and three-dimensional simulation
The model information of scene is published to Ethernet；
Step S4：The model information that high-performance computer is issued in status information monitoring unit subscription step S3, according to connecing
Mouth agreement parses information, and in Three-dimensional Display screen display three-dimensional simulation scene and threedimensional model；
Step S5：Status information monitoring unit carries out real-time update according to subscription information content change to model information；
Step S6：Operating personnel manipulate unit by operating, and remotely the state of sensor model are adjusted, final effect
It is shown on local three-dimensional display screen.
In another embodiment based on the method for the present invention, further include operating personnel pass through it is to be observed under three-dimensional simulation scene
The real-time status of equipment is operated according to need of work, is specially：
Step S01：Ensure and deduce and impart knowledge to students, using the historical data information of record, reappear global work situation, and convection current
Cheng Jinhang reappears and playback；
Step S02：Scene walkthrough, by controlling sensor model, is placed on the sensor model interaction area for perceiving desktop,
Under 3D visual angles, scene walkthrough is carried out, fully understands working condition.
Beneficial effect of the present invention：
First, global work situation information can be concentrated in together, uses interactive table mode in a manner of one succinct
Show, reduce the interaction cost between personnel, improve overall operating efficiency, reduce the possibility of error；
2nd, using real-time natural interactive mode, real work situation is reflected in the form of " What You See Is What You Get ", regarding
The information heard in frequency or audio is changed into the actual physics model seen and virtual guarantee situation, can be understood more intuitively
Global work situation, is easy to use personnel to grasp global information, makes a policy；
3rd, interactive device generalization, can according to different application demands, by configuration interactive equipment carry out display scene and
The replacement of base map, an equipment adapt to different applications by disposing different scenes；
4th, related procedure deduction and reproduction can be carried out, user of service quickly pushes away related prediction scheme available for auxiliary
Drill, easy to pinpoint the problems, also quickly user of service can be aided in be familiar with operational process；
5th, simple and convenient, no complex operations, left-hand seat is easy, meets human body operation and interaction habits；
6th, real-time cloth column-shaped state and work state information can be carried out distributed post, solves information island and ask
Topic, real-time status is shown with a kind of three dimensional constitution directly perceived, image, it is proposed that a kind of distributed information monitoring side
Formula, has expanded the usage scenario of system, has improved information sharing degree, improves equipment utilization benefit.
Other features and advantages of the present invention will illustrate in the following description, also, partial become from specification
Obtain it is clear that or being understood by implementing the present invention.The purpose of the present invention and other advantages can be by the explanations write
Specifically noted structure is realized and obtained in book, claims and attached drawing.
Brief description of the drawings
Attached drawing is only used for showing the purpose of specific embodiment, and is not considered as limitation of the present invention, in whole attached drawing
In, identical reference symbol represents identical component.
Fig. 1 is a kind of structure diagram of the status information monitoring system based on projection fusion and image recognition；
Fig. 2 is the optoelectronic device light path front view of one embodiment of the invention；
Fig. 3 is the optoelectronic device light path side view of one embodiment of the invention；
Fig. 4 is the operation chart of sensor model.
Embodiment
The preferred embodiment of the present invention is specifically described below in conjunction with the accompanying drawings, wherein, attached drawing forms the application part, and
It is used to explain the principle of the present invention together with embodiments of the present invention.
A specific embodiment according to the present invention, discloses a kind of status information based on projection fusion and image recognition
Monitoring system, as shown in Figure 1, including equipment body, sensor model, more than one status information monitoring unit；The equipment master
Body, which includes high-performance computer, projection imaging and information gathering screen, video fusion device, ultrashort out-of-focus projection's equipment and image, to be known at a high speed
Other equipment；Sensor model equipment using when be placed on projection imaging and information gathering screen, for marking equipment to be observed；It is described
System is using high-performance computer, ultrashort out-of-focus projection's equipment, projection imaging and information gathering screen, video fusion device and image high speed
Identification equipment is shown for scene, model information collection and fusion treatment show, the status information monitoring unit by with
Too net is connected with equipment body, for remote parameter configuration and status information interaction and the real-time shape of remote monitoring threedimensional model scene
State information.
Status information monitoring unit includes host and three-dimensional display screen and manipulation unit, status information monitoring unit by with
Too net is connected with equipment body, and the high-performance computer of host and equipment body carries out data interaction, three-dimensional display screen and manipulation
Unit is connected by connecting firmware with host, and three-dimensional display screen is used to show observation site and the three-dimensional simulation field of equipment to be observed
Scape, the use of status information monitoring unit is distributed in one particular embodiment of the present invention, can be deployed in not in practical application
Same field of employment, synchronizing information transmission is carried out by message subscribing-issue mechanism, and is carried out in status information monitoring unit
Scene rebuilding.For more places, long-range display threedimensional model scene real time status information, the institute of equipment to be observed are stateful at the same time
Information change situation can be shown on the display of status information monitoring unit, easy to remotely monitor global state, easy to more
Department's cooperating, and grasp the real-time status of equipment to be observed in time and prevent the generation of manual operation accident；
Unit is manipulated, for parameter configuration and status information interaction and ensures situation information gathering.For the ease of operator
Member's operation, manipulation unit hold the mobile terminals such as tablet using portable, are connected by wireless network with host, realize remote
Manipulation, it is convenient and efficient.The switch that is set by manipulating on unit at the same time, the state of changeable three-dimensional scenic, such as scene daytime, night
Between status display；Headlamp switch is shown；Daylight illumination is shown, for simulating true environment.In projection fusion and image recognition
Interactive system basis on, using Internet advertising distribution technology, by the real-time cloth column-shaped state of interactive system and ensure situation
Information is published to multiple status information monitoring units, and each status information monitoring unit can carry out distributed deployment to different places,
Using subscription-issue mechanism, interactive system Run-time scenario is rebuild using three dimensional pattern, to the instantaneous operating conditions of interactive system
It is monitored.Status information monitoring unit can show interactive system cloth column information, the status information of cloth column unit and equipment shape
State work state information；Manipulate unit and information exchange is carried out by wireless communication module and interactive system host, it is empty from equipment
Between limit, by personnel carry carry out various functions operate, improve the interaction capabilities and ease of use of equipment, improve equipment
Utilization benefit.
Status information release unit is further included, status information release unit is connected by Ethernet with equipment body, automatically
Gather the global information of observation site and equipment to be observed, such as the lamp of the malfunction of equipment, state of opening the light and observation site
The information such as light state, and the status information collected is distributed to equipment body automatically and is used to simulate global scene.It is automatic to receive
With transmission status information, the state of relevant device is safeguarded automatically, the workload to reduce the personnel to operate, improves efficiency and accuracy.
The manipulation stage body for installing each component devices is further included, is equipment provider's installation frame；The present invention's
In one specific embodiment, high intensity metal plate or aluminium alloy can be used by manipulating the stage body cabinet material of stage body, and fitment material uses
Stainless steel, high-quality steel or high-strength aluminum alloy material, the processing of stage body outer wall plastic-spraying, conductive adhesive tape sealing, inside is filtered
Design.The size of manipulation stage body is that length × width × height is 2000mm × 1000mm × 1000mm, interaction area size be not less than
1780mm×540mm.By using ultrashort out-of-focus projection and video fusion technology, by Projection Display be limited in a 2000mm ×
Within the interactive space of the class manipulation stage body of 1000mm × 1000mm sizes, make device miniaturization, improve equipment applicability；
Further include light and adjust module, it is uniform by light path inside light adjusting module light regulating operation bench, protect
Demonstrate,prove accuracy of the interior video cameras to image information collecting and identification.
Present device main body major function shows for scene display function, model information acquisition function and fusion treatment
Function, core control equipment are high-performance computer.
Scene display function mainly by high-performance computer by scene graph as being output to video fusion device, by video fusion device
Video dispatch ＆ display is carried out according to required resolution ratio, by projector equipment by image projection to projection imaging and information gathering screen
Imaging is carried out to show, high-performance computer provides video information for ultrashort out-of-focus projection's equipment, using image high speed identification equipment and
Ultrashort out-of-focus projection's equipment, projection imaging and information gathering screen realize the interaction area of virtual reality, interaction area display two dimension
Top view；
Model information acquisition function is mainly that the perception with exclusive mark is placed in projection imaging and information gathering screen
Model, two images above high speed identification equipment collection video image informations gather the identification information on sensor model at the same time, and
By the information collected be transmitted to video fusion device carry out video fusion, after be transmitted to high-performance computer；High-performance computer
Data analysis and process is carried out to the information collected, calculates status information of the sensor model under two-dimensional scene, state letter
Breath includes the position and direction of sensor model；
Fusion treatment display function is that the sensor model status information collected under two-dimensional scene is converted into three-dimensional letter
The virtual three-dimensional model that sensor model is reconstructed under three-dimensional simulation scene is ceased, wherein each sensor model has an exclusive mark
Know, the relevant parameter of equipment to be observed is actually represented with the exclusive mark binding, such as type, rotor folded state are and unique
The virtual three-dimensional model of equipment to be observed establish one-to-one mapping, realize reconstruction of the two dimensional model to threedimensional model, and with three
Dimension simulated scenario is merged, and realizes the display effect of actual situation combination and good interaction, most three-dimensional simulation scene and sense at last
Perception model three-dimensional state information is issued on three-dimensional display screen；During principle realization, to effectively improve the knowledge of sensor model
Not rate, supplements equipment inside light path by light adjusting module inside equipment, ensures the efficient of sensor model mark
Identification and tracking.
In one particular embodiment of the present invention, high-performance computer uses high-performance processor, can carry out video
Image processing in real time and calculating, memory are not less than 16GB, configure independent video memory capacity and are not less than 8GB, resolution ratio supports aspect
Than resolution ratio is not less than 2560 × 1600, and supports custom resolution, backward compatible；Configuration ethernet interface, high speed USB
Interface is connected for peripheral hardware.
Projection imaging and information gathering screen are used to show projection image, while for the information gathering of sensor model；At this
In one specific embodiment of invention, using the coated toughened glass of thickness 5mm as base material, special projection film is pasted at rear portion, real
The display of existing projection image, glass entirety ITO coating film treatments.
Video fusion device, when ultrashort out-of-focus projection's equipment is more than two, for the figure for exporting high-performance computer
Signal output to ultrashort out-of-focus projection's equipment after the processing of video fusion device carries out tiled display；When image high speed identification equipment is two
When more than platform, image co-registration and images steganalysis are carried out to the image of image high speed identification equipment subregion collection.In the present invention
A specific embodiment in, video input resolution ratio support 2560 × 1600.
Ultrashort out-of-focus projection's equipment sets more than two, and the high-definition image for computer to be exported carries out ultrashort out-of-focus projection simultaneously
Tiled display；In one particular embodiment of the present invention, using zoom lens more ultrashort than 0.248 are projected, using laser projection
Ensure projection image uniformity, brightness is not less than 3000 lumens, and contrast is not less than 10000:1.
Image high speed identification equipment is used for the collection of image；Light adjusting module is uniform for light path inside operation bench, protects
Demonstrate,prove accuracy of the interior video cameras to image information collecting and identification.
In one particular embodiment of the present invention, Fig. 2-3 is in the interactive device based on projection fusion and image recognition
Portion's projector equipment and optoelectronic device light path schematic diagram.Wherein ultrashort out-of-focus projection's equipment and image high speed identification equipment are because being projected
The limitation of light path and shooting collection light path, is respectively adopted ultrashort out-of-focus projection's equipment 2 and image high speed identification equipment 3, solid line
Represent view field, broken line representation electron-light cameraing region, view field is overlapping to melt projection image by video fusion device
Display is closed, 3 road photoelectric images are merged by video fusion algorithm in video overlay region, and based on the image pair after fusion
Sensor model mark is identified and tracks calculating.In addition, the adjusting mechanism design of each component is considered as during structure design, for life
Micro-positioning regulators during production use.
Sensor model, for mark it is a certain equipment or equipment model, there is special identifier, easy to optoelectronic device identification with
Tracking, and can be quoted sensor model, with perception interactive first person, on related three-dimensional picture display device with
Three dimensional form reduction operation bench two dimension shows situation information；During work, sensor model is placed in projection imaging and information gathering screen
On, dragging sensor model controls the simultaneous display of the three-dimensional correlation model, the mode of operation of sensor model includes placing, from
By pulling, rotating, moving horizontally and vertically moving；In addition the state of sensor model can be adjusted by manipulating unit,
Simulation of real scenes, can also be by carrying out data interaction, by the true of object being observed with the object being observed in real scene
Real status information is arranged to the status information that sensor model marks equipment or model.
Fig. 4 is the operation chart of sensor model, is merged based on ultrashort out-of-focus projection general with image recognition tracking technique
The operation of interactive device projects into mainly in projection imaging and information gathering screen region and manipulation unit as operation input equipment
As and information gathering screen region models sensor model with natural interaction language, each functional information is embodied.User can
The sensor model being placed in direct dragging on sensing region, controls the content shown in virtual two-dimensional/three-dimensional scenic, and right
Virtual scene, object in virtual scene even visual human are arbitrarily put, are controlled, including lighting effects in scene,
Sunshine and scheme of colour, can also control in real time.Emergency training, in real time monitoring commander etc. can also be carried out in virtual scene
Function.Control to relevant device state in scene, can by click on sensor model in projection imaging and information gathering screen it is right
The sensor model label that should be generated is changed related status information or is realized by manipulating unit input state information.Control
Selection can be set by clicking on the sensor model label on sensing region or selecting correspondent entity to carry out state on manipulation unit
Put.
Control based on the above-mentioned status information monitoring system based on the fusion of ultrashort out-of-focus projection and image recognition tracking technique
Method, comprises the following steps：
Step S1：System starts, and high-performance computer provides video display information to ultrashort out-of-focus projection's equipment, is projecting into
The global working status two-dimensional scene of generation on picture and information gathering screen；
Specifically, high-performance computer by scene graph as being output to video fusion device, by video fusion device according to required
Resolution ratio carries out video dispatch ＆ display, and image projection to projection imaging and information gathering screen are carried out imaging by projector equipment shows
Show, high-performance computer provides video information for ultrashort out-of-focus projection's equipment, utilizes image high speed identification equipment and ultrashort out-of-focus projection
Equipment, projection imaging and information gathering screen realize the interaction area of virtual reality, which shows two-dimensional top-down view；
Step S2：Sensor model is put in two-dimensional scene, high-performance computer is obtained according to sensor model unique mark
The entity type and status information of equipment to be observed, carry out status information Dynamic Maintenance；
Specially operating personnel place the sensor model with unique mark in projection imaging and information gathering screen, set
Image high speed identification equipment inside operation bench carries out information gathering to the mark of sensor model bottom；Identification camera will at a high speed
The information collected is transferred to high-performance computer and carries out data analysis and process, is calculated corresponding to sensor model and waits to see
The entity type and status information of measurement equipment, status information mainly include the position and direction of sensor model；
Operating personnel are by clicking on sensor model label or setting sensor model to correspond to be observed set by manipulating unit
Standby entity type and status information, and carry out status information Dynamic Maintenance.User of service can according to the actual conditions of monitoring,
Calculated according to the change information obtained by putting label into row position, shape is carried out by sensing region correlation interactive operation
State is safeguarded.
In another specific embodiment of the present invention, the entity type and status information of equipment to be observed are believed by state
Cease release unit to obtain, status information release unit is connected by Ethernet with equipment body, collection observation site and to be observed
The global information of equipment, such as the information such as light status of the malfunction of equipment, state of opening the light and observation site, and will collection
To status information be distributed to equipment body be used for simulate global scene.
Step S3：High-performance computer establishes three-dimensional simulation scene, is established according to the sensor model information collected and waits to see
The corresponding threedimensional model of measurement equipment is simultaneously merged with three-dimensional simulation scene, and high-performance computer will include threedimensional model and three-dimensional simulation
The model information of scene is published to Ethernet；
Specifically, the sensor model status information collected under two-dimensional scene is converted into three-dimensional letter by high-performance computer
Cease using the virtual three-dimensional model of dimensional Modeling Technology reconstruct sensor model under three-dimensional simulation scene, wherein each sensor model
With a unique mark, the relevant parameter for actually representing equipment to be observed is bound with the unique mark, as type, rotor fold
State etc., and one-to-one mapping is established with the virtual three-dimensional model of unique equipment to be observed, realize two dimensional model to three-dimensional mould
The reconstruction of type, and being merged with three-dimensional simulation scene, realizes display effect and good interaction that actual situation combines, most at last three
Simulated scenario and the issue of sensor model three-dimensional state information are tieed up on three-dimensional display screen；During principle realization, effectively to carry
The discrimination of high sensor model, supplements equipment inside light path by light adjusting module inside equipment, ensures to perceive
The efficient identification of model identification and tracking.
Step S4：The model information that high-performance computer is issued in status information monitoring unit subscription step S3, according to connecing
Mouth agreement parses information, and in Three-dimensional Display screen display three-dimensional simulation scene and threedimensional model；
Three-dimensional display screen is used for the three-dimensional simulation scene for showing observation site and equipment to be observed, in the tool of the present invention
Multiple status information monitoring units are set to be used for more places long-range display threedimensional model at the same time at the same time in more places in body embodiment
Scene real time status information, all state information change situations of equipment to be observed can be in the display of status information monitoring unit
Shown on device, easy to remotely monitor global state, easy to multidisciplinary cooperating, and grasp the real-time of equipment to be observed in time
State and the generation for preventing manual operation accident；
Step S5：Status information monitoring unit carries out real-time update according to subscription information content change to model information；
Step S6：Operating personnel manipulate unit by operating, and remotely the state of sensor model are adjusted, final effect
It is shown on local three-dimensional display screen.
The switch that is set by manipulating on unit at the same time, the state of changeable three-dimensional scenic, such as scene daytime, night status
Display；Headlamp switch is shown；Daylight illumination is shown, for simulating true environment.
Further include step：
Operating personnel are operated by the real-time status of equipment to be observed under three-dimensional simulation scene according to need of work；Behaviour
Making content can carry out according to need of work, can specifically include：
Step S01：Ensure and deduce and impart knowledge to students.Using the historical data information of record, global work situation, and convection current are reappeared
Cheng Jinhang reappears and playback.
Step S02：The sensor model, by controlling a certain first person sensor model, is placed on perception table by scene walkthrough
The interaction area in face, you can under 3D visual angles, obtain the visual angle situation of current location, can be carried out by moving the sensor model
Scene walkthrough, fully understands working condition.
Beneficial effect：
First, global work situation information can be concentrated in together, uses interactive table mode in a manner of one succinct
Show, reduce the interaction cost between personnel, improve overall operating efficiency, reduce the possibility of error；
2nd, using real-time natural interactive mode, real work situation is reflected in the form of " What You See Is What You Get ", regarding
The information heard in frequency or audio is changed into the actual physics model seen and virtual guarantee situation, can be understood more intuitively
Global work situation, is easy to use personnel to grasp global information, makes a policy；
3rd, interactive device generalization, can according to different application demands, by configuration interactive equipment carry out display scene and
The replacement of base map, an equipment adapt to different applications by disposing different scenes；
4th, related procedure deduction and reproduction can be carried out, user of service quickly pushes away related prediction scheme available for auxiliary
Drill, easy to pinpoint the problems, also quickly user of service can be aided in be familiar with operational process；
5th, simple and convenient, no complex operations, left-hand seat is easy, meets human body operation and interaction habits；
6th, the real-time cloth column-shaped state and work state information of interactive system can be carried out distributed post, solved
Problem of detached island of information, the real-time status of interactive system is shown with a kind of three dimensional constitution directly perceived, image, it is proposed that a kind of
Distributed information monitoring mode, has expanded the usage scenario of interactive system, has improved information sharing degree, and improving equipment makes
Use benefit.
It will be understood by those skilled in the art that realizing all or part of flow of above-described embodiment method, meter can be passed through
Calculation machine program instructs relevant hardware to complete, and the program can be stored in computer-readable recording medium.Wherein, institute
Computer-readable recording medium is stated as disk, CD, read-only memory or random access memory etc..
The foregoing is only a preferred embodiment of the present invention, but protection scope of the present invention be not limited thereto,
Any one skilled in the art the invention discloses technical scope in, the change or replacement that can readily occur in,
It should be covered by the protection scope of the present invention.
Claims (8)
1. a kind of status information monitoring system based on projection fusion and image recognition, it is characterised in that including equipment body, sense
Perception model, more than one status information monitoring unit；The equipment body is adopted including high-performance computer, projection imaging and information
Collect screen, video fusion device, ultrashort out-of-focus projection's equipment and image high speed identification equipment；Sensor model equipment using when be placed in projection
In imaging and information gathering screen, for marking equipment to be observed；Using high-performance computer, ultrashort out-of-focus projection's equipment, project into
Picture and information gathering screen, video fusion device and image high speed identification equipment progress scene is shown, model information gathers and fusion
Processing display, the status information monitoring unit are connected by Ethernet with equipment body, for remote parameter configuration and state
Information exchange simultaneously monitors threedimensional model scene real time status information.
2. the system as claimed in claim 1, it is characterised in that it is aobvious that each status information monitoring unit includes host, three-dimensional
The high-performance computer of display screen and manipulation unit, host and equipment body carries out data interaction, and three-dimensional display screen, which is used to show, to be seen
The three-dimensional simulation scene of equipment live and to be observed is surveyed, manipulation unit is used for remote parameter configuration and status information interaction.
3. system as claimed in claim 2, it is characterised in that further include the manipulation stage body for installing each component devices, be
Equipment provider's installation frame.
4. system as claimed in claim 3, it is characterised in that when carrying out scene display, by the high-performance computer by field
Scape image outpuT carries out video dispatch ＆ display, by ultrashort by video fusion device to video fusion device according to required resolution ratio
Image projection to projection imaging and information gathering screen are carried out imaging and shown by out-of-focus projection's equipment；
When carrying out model information collection, collection is placed on projection imaging to two images above high speed identification equipments at the same time and information is adopted
Sensor model information on collection screen, and the information collected is transmitted to high-performance computer, fusion treatment is carried out by computer,
And image recognition and tracking are carried out, the position of sensor model is calculated in real time；
When being shown for fusion treatment, high-performance computer is according to the virtual three-dimensional mould of the information architecture sensor model collected
Type, is merged with three-dimensional simulation scene, and the three-dimensional for outputting this to by high-performance computer status information monitoring unit is shown
Display screen carries out three-dimensional scenic and shows.
5. system as claimed in claim 4, it is characterised in that further include light and adjust module, light adjusts module and is used to grasp
It is uniform to control light path inside platform, ensures accuracy of the internal image high speed identification equipment to image information collecting and identification.
6. system as claimed in claim 5, it is characterised in that the sensor model is for marking a certain equipment or equipment
Model, has special identifier；During work, sensor model is placed on projection imaging and information gathering screen, drags sensor model
Control the simultaneous display of the corresponding threedimensional model of equipment to be observed under three-dimensional simulation scene.
A kind of 7. control method based on one of the claim 1-6 systems, it is characterised in that including：
Step S1：System starts, and the video display information that ultrashort out-of-focus projection's equipment is provided according to high-performance computer, is projecting into
The global working status two-dimensional scene of generation on picture and information gathering screen；
Step S2：Unique mark of the high-performance computer according to entrained by the sensor model being placed in two-dimensional scene, acquisition are treated
The entity type and status information of scope, carry out status information Dynamic Maintenance；
Step S3：High-performance computer establishes three-dimensional simulation scene, and to be observed set is established according to the sensor model information collected
Standby corresponding threedimensional model is simultaneously merged with three-dimensional simulation scene, and high-performance computer will include threedimensional model and three-dimensional simulation scene
Model information be published to Ethernet；
Step S4：The model information that high-performance computer is issued in status information monitoring unit subscription step S3, is assisted according to interface
View parses information, and in Three-dimensional Display screen display three-dimensional simulation scene and threedimensional model；
Step S5：Status information monitoring unit carries out real-time update according to subscription information content change to model information；
Step S6：Operating personnel manipulate unit by operating, and remotely the state of sensor model are adjusted, final effect is shown
On local three-dimensional display screen.
8. the method for claim 7, it is characterised in that further include operating personnel pass through it is to be observed under three-dimensional simulation scene
The real-time status of equipment is operated according to need of work, is specially：
Step S01：Ensure and deduce and teaching, using the historical data information of record, reappear global work situation, and to flow into
Row reproduction and playback；
Step S02：Scene walkthrough, by controlling sensor model, is placed on the sensor model interaction area for perceiving desktop, in 3D
Under visual angle, scene walkthrough is carried out, fully understands working condition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710978943.5A CN108010079B (en) | 2017-10-19 | 2017-10-19 | State information remote monitoring system and method based on projection fusion and image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710978943.5A CN108010079B (en) | 2017-10-19 | 2017-10-19 | State information remote monitoring system and method based on projection fusion and image recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108010079A true CN108010079A (en) | 2018-05-08 |
| CN108010079B CN108010079B (en) | 2021-11-02 |
Family
ID=62051007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710978943.5A Active CN108010079B (en) | 2017-10-19 | 2017-10-19 | State information remote monitoring system and method based on projection fusion and image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108010079B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111586351A (en) * | 2020-04-20 | 2020-08-25 | 上海市保安服务(集团)有限公司 | Visual monitoring system and method for fusion of three-dimensional videos of venue |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005009051A1 (en) * | 2003-07-16 | 2005-01-27 | Aurora Digital Advertising Inc. | Three dimensional display method, system and apparatus |
| WO2009076303A2 (en) * | 2007-12-11 | 2009-06-18 | Bbn Technologies, Corp. | Methods and systems for marking and viewing stereo pairs of images |
| CN101510074A (en) * | 2009-02-27 | 2009-08-19 | 河北大学 | High present sensation intelligent perception interactive motor system and implementing method |
| CN101646067A (en) * | 2009-05-26 | 2010-02-10 | 华中师范大学 | Digital full-space intelligent monitoring system and method |
| CN103034755A (en) * | 2012-11-29 | 2013-04-10 | 北京科东电力控制***有限责任公司 | Visual inspection method based on virtual reality technology for transformer substation |
| CN103632581A (en) * | 2013-12-17 | 2014-03-12 | 国家电网公司 | Debugging, operation and maintenance simulation method of electric energy collection terminals |
| CN103824318A (en) * | 2014-02-13 | 2014-05-28 | 西安交通大学 | Multi-camera-array depth perception method |
| CN105303615A (en) * | 2015-11-06 | 2016-02-03 | 中国民航大学 | Combination method of two-dimensional stitching and three-dimensional surface reconstruction of image |
| CN105549725A (en) * | 2016-02-03 | 2016-05-04 | 深圳市中视典数字科技有限公司 | Three-dimensional scene interaction display device and method |
| CN107014375A (en) * | 2017-02-22 | 2017-08-04 | 上海谦尊升网络科技有限公司 | The indoor locating system and method for ultralow deployment |
-
2017
- 2017-10-19 CN CN201710978943.5A patent/CN108010079B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005009051A1 (en) * | 2003-07-16 | 2005-01-27 | Aurora Digital Advertising Inc. | Three dimensional display method, system and apparatus |
| WO2009076303A2 (en) * | 2007-12-11 | 2009-06-18 | Bbn Technologies, Corp. | Methods and systems for marking and viewing stereo pairs of images |
| CN101510074A (en) * | 2009-02-27 | 2009-08-19 | 河北大学 | High present sensation intelligent perception interactive motor system and implementing method |
| CN101646067A (en) * | 2009-05-26 | 2010-02-10 | 华中师范大学 | Digital full-space intelligent monitoring system and method |
| CN103034755A (en) * | 2012-11-29 | 2013-04-10 | 北京科东电力控制***有限责任公司 | Visual inspection method based on virtual reality technology for transformer substation |
| CN103632581A (en) * | 2013-12-17 | 2014-03-12 | 国家电网公司 | Debugging, operation and maintenance simulation method of electric energy collection terminals |
| CN103824318A (en) * | 2014-02-13 | 2014-05-28 | 西安交通大学 | Multi-camera-array depth perception method |
| CN105303615A (en) * | 2015-11-06 | 2016-02-03 | 中国民航大学 | Combination method of two-dimensional stitching and three-dimensional surface reconstruction of image |
| CN105549725A (en) * | 2016-02-03 | 2016-05-04 | 深圳市中视典数字科技有限公司 | Three-dimensional scene interaction display device and method |
| CN107014375A (en) * | 2017-02-22 | 2017-08-04 | 上海谦尊升网络科技有限公司 | The indoor locating system and method for ultralow deployment |
Non-Patent Citations (2)
| Title |
|---|
| JASON GENG: "A volumetric 3D display based on a DLP projection engine", 《DISPLAYS》 * |
| 吴廷: "基于立体感知的三维全景技术研究", 《中国优秀博硕士学位论文全文数据库（硕士）信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111586351A (en) * | 2020-04-20 | 2020-08-25 | 上海市保安服务(集团)有限公司 | Visual monitoring system and method for fusion of three-dimensional videos of venue |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108010079B (en) | 2021-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106598229A (en) | Virtual reality scene generation method and equipment, and virtual reality system | |
| CN104219584B (en) | Panoramic video exchange method and system based on augmented reality | |
| CN104268939B (en) | Transformer substation virtual-reality management system based on three-dimensional panoramic view and implementation method of transformer substation virtual-reality management system based on three-dimensional panoramic view | |
| EP2629267A2 (en) | Real-time compositing of live recording-based and computer graphics-based media streams | |
| CN109859326B (en) | Cross-platform two-dimensional and three-dimensional graph linkage display system and method for transformer substation | |
| CN105264478A (en) | Hologram anchoring and dynamic positioning | |
| CN105629653A (en) | Interactive holographic projection method on the basis of three-dimensional model | |
| US11164377B2 (en) | Motion-controlled portals in virtual reality | |
| CN105639818A (en) | Intelligent safety helmet based on augmented reality, space scanning and gesture recognition technologies | |
| CN107863049A (en) | A kind of finishing outpost display system based on line holographic projections technology | |
| TWI644235B (en) | Wearable instant interaction system | |
| CN108280873A (en) | Model space position capture and hot spot automatically generate processing system | |
| CN104238730A (en) | Smart grid visualization platform and demonstration control method | |
| CN204945944U (en) | A kind of holographic interaction image system | |
| CN103745627A (en) | Three-dimension visual simulation method and three-dimension visual simulation system for driving simulation | |
| CN106971658A (en) | A kind of mixed reality goods electronic sand map system and analogy method | |
| CN106791629A (en) | A kind of building based on AR virtual reality technologies builds design system | |
| CN106791630A (en) | A kind of building based on AR virtual reality technologies builds supervisory systems | |
| CN113744585A (en) | Fire accident emergency treatment drilling system and method | |
| CN209118529U (en) | Wisdom education Environment-Ecosystem | |
| CN108010079A (en) | Status information remote monitoring system and method based on projection fusion and image recognition | |
| CN114861262A (en) | Visualized large-scale building operation platform system based on Unity3D | |
| Broll | Augmented reality | |
| CN108008813A (en) | A kind of interactive system and control method based on projection fusion and image recognition | |
| CN111047713B (en) | Augmented reality interaction system based on multi-vision positioning and operation method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |