CN105326567B - Picture for being orientated sensitive display device observes application and method - Google Patents

Abstract

Picture for being orientated sensitive display device observes application and method.A kind of system and method provide three-dimensional image volume data with sensitive display device is orientated, and as a result, by the inclination of the display device, rise and decline, this is simply navigable as volume.It does so, is to provide picture related with the angles and positions of the display device on screen, so that the impression that user obtains is, the device itself can be used as entering the window as volume, especially when the device is placed on or near the source as data of such as patient.

Description

Picture for being orientated sensitive display device observes application and method

The application is the Shens PCT that Chinese Patent Application No. is 201180061055.8, the applying date is on 03 14th, 2011 Please PCT/US2011/028412, it is entitled " for be orientated sensitive display device picture observe application and method " invention The divisional application of patent application.

Technical field

The present invention provides a kind of use and has device, observes the intuitive application of complicated three-dimensional image volume, these have Device includes accelerometer Giroscope or other places and/or orientation sensitive technology, such as smart phone,Such as This etc..It does so, is to provide the excellent of these Direct-View Devices offer to such as doctor, surveyor, engineer and so on user Point.

Invention content

It is contemplated that the application can be used together with unconfined various object to be imaged.Non-limitative example, Include medical treatment patient, the object made, the geological structures of animal, such as oil well, volcano and fault line, computer reproduce It is assumed that structure, etc..In entire the application, all explicitly point out is used as example, will all refer to the patient of medical treatment.This it is not intended that The patient for implying that the present invention preferably can be applied to medical treatment, and should not be explained in this way.

One aspect of the present invention, provides for the use of the orientation sensitive of the display device and the application of positioning, as long as The positioning application can be applied to the specific device, automatically to change display according to the orientation of the device.It does so, is to create Such illusion, that is, the device itself can actually be seen in the source into the 3D volumes of (see into) such as patient.It is shown Picture be still aligned with actual patient, it is unrelated with the orientation of the device.Therefore, if the device is held vertically on patient Side, then the display is the top view for the patient being aligned in an x-y plane, makes the display in this way it appear that the device is X-ray As machine.If the device is rotated and is maintained at beside patient in the vertical direction, the side view of patient is shown.It is best It is that the display continuously changes with the change in orientation of the device, holding is aligned with patient.

Another aspect of the present invention is planning ability, and the device can be effectively used for identifying and marking mesh as a result, Mark such as damages the region of (lesion) or other concerns, is for future reference.

Another aspect of the present invention provides a kind of program, allows user to plan and reaches navigating by water for selected target (navigable) paths vias.This can navigation path access planning characteristic, may be desirable to rely on natural spatial information, the information Be loaded into three-dimensional image data, or be segmented into part as data pretreatment, be used to help to be automatically provided and reach the mesh Target logic, minimum crude an dimpetuous (least obtrusive) paths vias.

Yet another aspect of the present invention, is to provide corresponding internal virtual as option.User can select virtual on demand View, to help to visualize.Preferably, the part as paths vias feature is flown over (fly-through) option and is also carried For, and during flying over, user has unique capability by with the interaction of its display, learning the paths vias in real time.

Therefore, the present invention provides a kind of method showing picture on being orientated sensitive display device, including：To described device Three-dimensional image volume data in relation to object is provided；Record place and the orientation of (register) described device and the object； In the display device, display uses the picture that the data corresponding with the place of described device and orientation generate；It generates New picture, the new picture correspond to the variation in the place and orientation relative to the object described device；Wherein, shown The corresponding described device that gives the user between the picture of generation and the object is just being seen into the impression in the object.

Place and the orientation for recording described device and the object may include that at least one label is placed on the object On body, and track position of the described device relative at least one label.

Position of the described device relative at least one label is tracked, may include using optical tracking mechanism to institute State label positioning.

Position of the described device relative at least one label is tracked, may include using magnetic positioning system.

Place and the orientation for recording described device and the object may include fixed using the whole world for being put into described device Position system, the global positioning system provide the place of the object and described device.

The preferred method of the present invention can also include giving the user the option for locking current picture, make described device in this way Move the variation that will not lead to the picture again, until needed so far.

The preferred method of the present invention can also include providing option to the user, for planning through the object The path needed.

The preferred method of the present invention can also include providing and flying over feature, as a result, when selected, the road of the needs The animation of diameter indicates, shown from point of observation along the path.

The present invention also provides the methods for providing the three-dimensional image volume data in relation to object, including：It is sensitive aobvious using being orientated Showing device accesses the three-dimensional image volume data；Record place and the orientation of described device and the object；It is filled in the display It sets, the picture that display is generated using the data corresponding with the place of described device and orientation；It continuously generates newly Picture, these new pictures correspond to relative to the variation on the place and orientation of the object described device；Wherein, shown The corresponding described device that gives the user between the picture of generation and the object is just being seen into the impression in the object.

Place and the orientation for recording described device and the object may include that at least one label is placed on the object On body, and track position of the described device relative at least one label.

Position of the described device relative at least one label is tracked, may include using optical tracking mechanism to institute State label positioning.

Position of the described device relative at least one label is tracked, may include using magnetic positioning system.

Place and the orientation for recording described device and the object may include fixed using the whole world for being put into described device Position system, the global positioning system provide the place of the object and described device.

Preferred method can also include giving the user the option for locking current picture, make moving again for described device in this way The dynamic variation that will not lead to the picture, until needed so far.

Preferred method can also include providing option to the user, be used to plan the needs for passing through the object Path.

Preferred method can also include providing and flying over feature, and as a result, when selected, the path of the needs is moved It draws and indicates, it is shown from point of observation along the path.

The present invention also provides a kind of systems used in providing interior of articles picture, including：It is orientated sensitive display device； Three-dimensional image volume data in relation to the object, the data can be accessed by described device；And the meter that can be executed by described device Calculation machine program, the program are received from the orientation sensors of described device and are inputted, and the place relative to the object and orientation Record data between the place and orientation of described device, and on the display of the display device, generate and correspond to relatively The inside picture in the place of the display device described in the object and the object of orientation, so that from the user's point of view, seeming He or she is pried through by the display device into the object.

The preferred embodiment of the system of the present invention can also include computer program, whenever the movement of the display device When being detected, which just updates the place and the orientation corresponded to relative to display device described in the object The object the internal picture.

The preferred embodiment of the system of the present invention, can also include computer program, which provides locking institute The option for stating the picture of object makes the movement of described device not generate the picture of variation in this way.

The preferred embodiment of the system of the present invention can also include slider feature, when displaced, the slider feature Described device is caused to generate the variation of picture, which corresponds to the movement of the hypothesis along the axis perpendicular to the display screen plane.

The preferred embodiment of the system of the present invention can also include to fly over feature, as a result, when selected, described in generation The animation in the path needed indicates, shown from point of observation along the path.

Description of the drawings

Fig. 1 is the embodiment of the screenshotss of the application of the present invention；

Fig. 2 is the simplification figure for the three-D volumes for having interior geometry, and for description, the present invention uses；

Fig. 3 draws the device of the application using the present invention, is maintained in the orientation relative to the volume of Fig. 2；

Fig. 4 is the embodiment such as the screenshotss of the application of the present invention seen on the device of Fig. 3；

Fig. 5 draws the device of the application using the present invention, is maintained in the orientation relative to the volume of Fig. 2；

Fig. 6 is the embodiment for the screenshotss applied such as the present invention seen on the device of Fig. 5；

Fig. 7 draws the device of the application using the present invention, is maintained in the orientation relative to the volume of Fig. 2；

Fig. 8 is the embodiment such as the screenshotss of the application of the present invention seen on the device of Fig. 7；

Fig. 9-11 shows the embodiment such as the screenshotss generated by the control of change slider seen on the device of Fig. 7； With

Figure 12 shows the embodiment for being used for the screenshotss of the application of the present invention of paths vias planning.

Specific implementation mode

The present invention provides a kind of software application, and for being used together with display device, it is quick which adopts position Sense and/or orientation sensitive technology, such as illustrate, accelerometer or gyroscope.The non-limitative example of this device, includingDevice, such asWith Chinese mugwort windIt should be appreciated that screenshotss described herein are only unrestricted Property example, universal for explaining the present invention, and although some specific features being discussed herein, can be made by claim For the part of the present invention, but their modes for being depicted in screenshotss, it is not intended to it limits.

Referring now to the drawings, and referring initially to Fig. 1, the screenshotss of device, the application of the device operation present invention are drawn on figure 20 embodiment.The screenshotss include as 30 and user's control 40,50,60,70 and 80.

As 30 be the two-dimensional slice of the three-D volumes as data, this such as illustrates as data imaging technique, CT, MRI, fluorescopy obtain.As is well known such as this field, multiple bidimensional parallel sweeps, such as illustrate, CT scan, can be by It obtains and is assembled into data volume.Once by collecting, which can be used to create bidimensional picture slice, these are not right as being sliced It should be scanned in specific, but still in the content of the geometrically exact picture volume.Therefore, user can use the volume, A certain slice is observed in virtual any plane that the volume includes.

In order to clearly explain that the application of the present invention, the extremely simplified model of three-D volumes are depicted in fig. 2.Volume 10 include imaging boundary 12, shown in dotted line.These lines 12 are represented by the boundary of imaging volume.It includes three points to be in the picture The tubular reticulum 14 of branch 15,16 and 17.There is circular cross section in branch 15, and there is lozenge shape cross-section in branch 16, and there is triangle in branch 17 Cross section.Cross sections of these variations are only clearly to provide.Although such three-dimensional image volume is known, the present invention's Using such imaging technique, is combined with the sensitive display technology of orientation above-mentioned, create virtual X-ray device.This is new The importance of newness leap, which is simply strong user's control on surface, to be confirmed.

The shape of control 40 is similar to padlock, and can switch between locking and unlocked position.It is as shown in the figure when being in When unlocked position, constantly change with the inclination of device 20 as 30.Some embodiments described below also provide and work as the device The picture changed when laterally or vertically being moved.For clarity, the orientation according to the device for the application for starting the present invention is sensitive Property, as the inclination with device 20 changes glibly and continuously.It not only switches between quadrature position.

When control 40 is in the locked position, as 30 remains stationaries.When the thing and needs of picture display concern maintain to be somebody's turn to do As and when mobile device 20, as 30 remains stationaries are useful, be convenient for that this is discussed with certain people in addition to original user in this way Picture.

Control 50 is sliding control, it allows user along in the axis of the current plane of device 20, changes as 30. In one embodiment, when inclined, as automatically, but the Change of types of the orientation sensors used according to the device, work as device 20 when being raised or decline along the normal axis of the device, can not change.Sliding control 50 allows this variation.If device 20 Such as comprising gyroscope technology, this feature can automatically occur with the device along moving for normal axis.However, sliding control 50 Still it can be needed.For example, even if one embodiment allows when controlling 40 in locked position, sliding is used. Allow user that device 20 is put on the table in this way, and roll " upper and lower " and pass through the object, it is not necessary to the mobile device.

Control 60 is boundary mark (landmark)/way point (waypoint) marker characteristic.It allows user to as 20 add Label, to identify a certain feature.It is used together with the crosshair 90 being positioned among display.Preferably, the display Device is touch screen.In this way, by simply sliding finger on the display, crosshair 90 can be relative to being moved as 30. In one embodiment, crosshair 90 can be placed on display Anywhere.In another embodiment, as can be Crosshair " under " moved around crosshair, which is maintained at the center of display.No matter which mode, once crosshair 90 are placed on the place as paying close attention on 30, can be with press control 60, so that boundary mark is placed on the place.

Control 70 is boundary mark/course point deletion feature.The label in labeled place before it simply deletes or eliminates. In one embodiment, when depressing first time, it is worked by deleting last place, and when depressing for second, it deletes Except the previous place in last place, and so on.In another embodiment, crosshair 90 is placed in existing boundary mark, and Button 70 is touched, and the boundary mark of any order is thus allowed to be deleted.

Control 80 is to fly over feature.Once leading to the paths vias of target by planning (as described below), the control is touched System 80 makes to change the dummy model to three-D volumes as 30.Therefore the display shows true perspective view, which provides a user From logic starting point to target, chamber (lumen) (for example, in the case of vascular or bronchial imaging volume) is flown through Experience.This feature will provide physicians with desired view, and seemingly it is the same to pass through endoscope viewing by the doctor.It is being preferably implemented In example, the figure of bird flies to the starting point as 30 from button itself, flies to flutter downwards and behind, view is converted into virtual representation. As such, user accurately sees where the trip starts.

It should be appreciated that the control being discussed herein is enumerated as just example, and they can according to the use of the application and Change.For example, it is envisaged that it is useful that this, which is applied in terms of fire resistance property, for providing the virtual of building to fireman , real-time three-dimensional map, to provide the bright of the place in the environment full of smog ignored by them to them Display.In this applications, various different communications control can be provided, to allow the touch of fireman's button to outside User sends communication, rather than has to rely on audio communication equipment.This application will also be obtained from second device is linked to To benefit, the user of observer and interior of building outside building is enable to see identical picture 30.

Fig. 3-8 is provided when user is from volume 10 shown in each different angle observation chart 2, can be seen on the display 20 What example.Position of the display 20 relative to volume 10 is drawn referring initially to Fig. 3 and 4, Fig. 3.It can be seen that display 20 It is maintained at 10 top of volume, along horizontal alignment.Fig. 4 draws the obtained picture 30 from the orientation for be maintained at Fig. 3.It can see It arrives, as 30 be the level cross-sectionn by tubular reticulum 14.

In Figure 5, display is tilted slightly along the longitudinal axis.The picture 30 that Fig. 6 is drawn.

In the figure 7, display is further tilted, therefore it is vertically oriented now.The picture that Fig. 8 is drawn.

Fig. 9-11 shows the use of slider control 50.The relative orientation of display 20 shown in Fig. 7, Fig. 9 is still used to draw The picture 30 obtained when the slider is in leftmost position.It does so, is obtaining being placed on closest to display as 30 Position in.Figure 10 draws the slider for being moved to the right, and Figure 11 draws the slider for being moved to even more the right.

As previously mentioned, the modification of slider 50, be relative position between device 20 and volume (object) 10 it is monitored or Counted in some way and embodiment.In other words, device 20 is recorded together with volume or object 10.Realize the one of record Kind mode is with the label in one or more sensors or volume 10.Device 20 can use Optical devices or magnetic location system System, tracks its position relative to sensor or label.In addition, in the case of larger volume, such as geographical volume or build Object is built, global positioning system can be used.Device 20 makes device 20 in this way using accurate gyroscope technology and possible Once being recorded with volume 10, the gyroscope ability of device 20 continues the relative position of tracking and 10 associated device 20 of volume. It is envisaged that periodically record will be needed again, to prevent the accumulation of composition error.

The present invention also includes paths vias planning characteristic.Referring now to Figure 12, patient's lung drawing hypothesis on figure, simplifying Picture 30 display 20.Doctor identification of damage 110 and relative to crosshair 90 move the picture, to make crosshair 90 at this The center of damage.Preferably, doctor makes device 20 tilt by several different angles, to ensure that crosshair 90 is substantially damaging The center of the volume of wound 110.

Then, doctor touches locking control 40 and makes to fix as 30.Using the crosshair 90 for being in preferred place, doctor touches Additional way point/boundary mark control 60.If doctor does not like the position of the crosshair 90, deletes control 50 and can be touched It touches.

In next step, doctor marks second point 120 in flight access.In one embodiment, this is by using touch Screen makes as sliding, until what is completed on the point 120 that crosshair is positioned in needs.Because locking 40 is worked, if device 20 It is changed with the relative position of volume (being not drawn into fig. 12), seems indeclinable.However, the picture can be relative to crosshair 90 are moved.In another embodiment, crosshair 90 can be reapposed over any position on display 20.Once with The second position 120 meets, and doctor then touches additional control 60 again, and not only the second position 120 is labeled, and algorithm is certainly Paths vias arrival process input point is from first to last illuminated dynamicly, is tracheae 130 in this example.People are it can be seen that this is same Paths vias is planned, can be used for other application, such as mine rescues, fire-fighting, etc..

One aspect of the present invention, provide it is a kind of as volume access system, it use scanning technique, such as optical reader with Wireless database access capability together, for the device appropriate uploaded to as volume for particular cases or patient.For example, Doctor make a round of visits (making rounds) single display device 20 can be carried.When reaching patient, doctor can use the device 20, the label with patient or in case history (patient ' s record) is scanned, the picture of region-of-interest in patient body is uploaded Volume.This step ensures correctly as volume is loaded into the device.

It is also foreseen that device 20 can be used together with certain Real Time Imaging Technology, available such as in medical probe Electromagnetic position and orientation sensitive system.By recording the device 20 and volume 10 and/or the electromagnetic system (not shown), probe In real time as that can be superimposed on picture 30.

It is also foreseen that, laser pointer system can by medical treatment or other setting, merge with the device, accurately to open up Show the plane of the picture 30 relative to volume 10.For example, even if when be recorded as 30 and volume 10, but it requires certain supposition Amount, with judge the plane enter volume 10 have it is mostly deep.This is because in the application of such as medical applications, display 10 is not It can be inserted into volume 10.However, top plate and/or the laser beam of wall installation, can be used to illuminate the straight line on patient, it is accurate Really show the plane as 30.

A kind of application of the present invention of special interest, is medical diagnosis applications, 3D imaging datas are loaded into the dress as a result, Set 20.Using the doctor of the application of the invention, it then can at an unprecedented rate be evaluated using the device 20 and come from each difference The picture data of angle.The application can be on the scene in object (patient) or be implemented when being absent from the scene.Patient is evaluated in doctor not at the scene These as in the case of, will there is no required record between device 20 and patient.

Now, when carrying out diagnostic analysis, as data are evaluated by primary sliding every time.Doctor watches by memory Difference between sliding in succession makes the region of concern seem and becomes such in three-dimensional so as to these slidings of collecting theoretically Image.The process very elapsed time, and doctor part is then required extremely attentively.It can be from each using the doctor of the present invention Different angle moves around the device, not only " climbs over (flip through) " thousands of slidings backward forward, but also by changing Become the angle of device.The new picture generated by the device never occurs previously as sliding.Therefore, make the dynamic of CT scan State, mobility assessment be possibly realized, this is unprecedented.

The present invention is being described herein with only example.It will be appreciated, however, that the possible real world of the present invention Application, it is almost inexhaustible, and provide description be not intended to limit anyway.It is only to the scope of the present invention Limitation, the claims that will be described below illustrate.

Related application

This application claims the U.S. Provisional Application serial number No.61/409 submitted on November 2nd, 2010,495, it is entitled Image Viewing Application For Orientationally Sensitive Display Devices' is preferential Power, is cited in full text this application, for reference herein.

Claims (20)

1. a kind of system used in providing interior of articles picture, the system include：

It is orientated sensitive display device comprising orientation sensors and display and the display device is configured as：

Access the three-dimensional image volume data in relation to the object；

It receives and inputs from the orientation sensors, and

On the display, the display device is based on relative to three-dimensional image volume corresponding with the three-dimensional image volume data Place and be orientated the slice map for showing the object；

Display target marks in the slice map；

It detects and slidably inputs on the display of the display device；

It is relative to the place of the three-dimensional image volume and the change of orientation and defeated based on the sliding based on the display device Enter, generate new slice map, wherein the new slice map be by according to the orientation in the place come by cutting three-dimensional image volume The two dimensional image reconfigured from three-dimensional image volume；With

The flat of the shown new slice map relative to the three-dimensional image volume is shown on the display of the display device The instruction in face,

Wherein, when based on it is described slidably input to generate new slice map when, new slice map is new at this according to target label Slice map on place and the direction that slidably inputs generate.

2. system according to claim 1, wherein when the display device moves, user is based on the display device Relative to the three-dimensional image volume place and be orientated from multiple angles dimensionally, in real time and continuously from the object.

3. system according to claim 1, wherein the slice map of the display includes being marked with multiple labels by user The path planned in advance at least part.

4. system according to claim 3, wherein the path planned in advance is by from defeated in three-dimensional image volume data Access point is tagged to the target identified by user.

5. system according to claim 4, wherein the path responses planned in advance are in the placement of the multiple label And it is automatically illuminated.

6. system according to claim 1, wherein by placing on the object and passing through at least one label Track position of the display device relative at least one label, by the place of the display device and be orientated with it is described Three-dimensional image volume records together.

7. system according to claim 1 further includes slider, when displaced, which causes the display to fill It sets the orientation based on the display and generates another slice map,

Wherein, which is along the axis vertical relative to the orientation of three-dimensional image volume with the display device Depth obtain from the three-dimensional image volume data, and the depth corresponds to the movement of slider.

8. system according to claim 1 further includes flying over feature, when selected, the path planned in advance is shown Animation from point of observation along the path indicates.

9. a kind of method showing picture on the display for being orientated sensitive display device, this method include：

Three-dimensional image volume data in relation to object is provided to the display device；

On the display of the display device, based on the display device relative to corresponding with the three-dimensional image volume data The place of three-dimensional image volume and the slice map for being orientated the object that display is generated from the three-dimensional image volume data, wherein should Slice map is by cutting three-dimensional image volume by the X-Y scheme that is reconfigured from three-dimensional image volume according in the orientation in the place Picture；

Display target marks in the slice map；

It detects and slidably inputs on the display of the display device；

Based on the display device relative to the place of the three-dimensional image volume and the change of orientation and based on slidably input come Generate new slice map；And

Show the instruction of the plane of the shown new slice map relative to the three-dimensional image volume,

Wherein, when based on it is described slidably input to generate new slice map when, new slice map is new at this according to target label Slice map on place and the direction that slidably inputs generate.

10. according to the method described in claim 9, further including allowing user to be based on described aobvious when the display device moves Showing device relative to the three-dimensional image volume place and be orientated from multiple angles dimensionally, in real time and continuously from described in Object.

11. according to the method described in claim 9, further including being marked come pre- in the three-dimensional image volume data internal labeling with multiple At least part in the path first planned.

12. further including according to the method for claim 11, being identified from the input point in three-dimensional image volume data to by user Target label described in the path planned in advance.

13. further including according to the method for claim 12, automatically illuminating institute in response to the placement of the multiple label State the path planned in advance.

14. according to the method described in claim 9, further include place by the display device in real space and be orientated with The three-dimensional image volume records together.

15. according to the method for claim 14, further include by by least one label placement on the object and Position of the display device relative at least one label is tracked, to record place and the orientation of the display device.

16. further including according to the method for claim 15, that track the display device by optical tracking mechanism opposite In the position of at least one label.

17. further including according to the method for claim 15, that track the display device by magnetic positioning system opposite In the position of at least one label.

18. according to the method described in claim 9, further including locking current slice map in situ so that filled in the display When setting further mobile, slice map is kept at present.

19. according to the method described in claim 9, further including showing the path planned in advance when flying over feature and being selected Animation from point of observation along the path indicates.

20. according to the method described in claim 9, further including taking relative to the three-dimensional image volume based on the display device To causing the display device to generate another slice map by mobile slider；

Wherein, which is along the axis vertical relative to the orientation of three-dimensional image volume with the display device Depth obtain from the three-dimensional image volume data, which corresponds to the movement of slider.