EP1634438A1 - Method and system for remote and adaptive visualization of graphical image data - Google Patents
Method and system for remote and adaptive visualization of graphical image dataInfo
- Publication number
- EP1634438A1 EP1634438A1 EP03720290A EP03720290A EP1634438A1 EP 1634438 A1 EP1634438 A1 EP 1634438A1 EP 03720290 A EP03720290 A EP 03720290A EP 03720290 A EP03720290 A EP 03720290A EP 1634438 A1 EP1634438 A1 EP 1634438A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- data
- screen image
- image
- connection
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
- 238000000034 method Methods 0.000 title claims abstract description 89
- 238000012800 visualization Methods 0.000 title claims abstract description 18
- 230000003044 adaptive effect Effects 0.000 title abstract description 5
- 230000006835 compression Effects 0.000 claims abstract description 86
- 238000007906 compression Methods 0.000 claims abstract description 86
- 238000007405 data analysis Methods 0.000 claims abstract description 6
- 230000001143 conditioned effect Effects 0.000 claims description 6
- 238000009877 rendering Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 238000013079 data visualisation Methods 0.000 claims description 3
- 230000009471 action Effects 0.000 description 8
- 239000003086 colorant Substances 0.000 description 7
- 238000002591 computed tomography Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000002604 ultrasonography Methods 0.000 description 4
- 238000004883 computer application Methods 0.000 description 3
- 230000006837 decompression Effects 0.000 description 3
- 238000002595 magnetic resonance imaging Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 101100437998 Arabidopsis thaliana BZIP2 gene Proteins 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 2
- 238000013497 data interchange Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 229940079593 drug Drugs 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000002600 positron emission tomography Methods 0.000 description 2
- 238000002601 radiography Methods 0.000 description 2
- 229910052710 silicon Inorganic materials 0.000 description 2
- 239000010703 silicon Substances 0.000 description 2
- 238000002603 single-photon emission computed tomography Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 206010028980 Neoplasm Diseases 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008276 biophysical mechanism Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- TVMXDCGIABBOFY-UHFFFAOYSA-N octane Chemical compound CCCCCCCC TVMXDCGIABBOFY-UHFFFAOYSA-N 0.000 description 1
- 238000010561 standard procedure Methods 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/4143—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a Personal Computer [PC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N1/327—Initiating, continuing or ending a single-mode communication; Handshaking therefor
- H04N1/32765—Initiating a communication
- H04N1/32771—Initiating a communication in response to a request, e.g. for a particular document
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N1/327—Initiating, continuing or ending a single-mode communication; Handshaking therefor
- H04N1/32765—Initiating a communication
- H04N1/32771—Initiating a communication in response to a request, e.g. for a particular document
- H04N1/32776—Initiating a communication in response to a request, e.g. for a particular document using an interactive, user-operated device, e.g. a computer terminal, mobile telephone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N1/333—Mode signalling or mode changing; Handshaking therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N1/333—Mode signalling or mode changing; Handshaking therefor
- H04N1/33353—Mode signalling or mode changing; Handshaking therefor according to the available bandwidth used for a single communication, e.g. the number of ISDN channels used
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N1/333—Mode signalling or mode changing; Handshaking therefor
- H04N1/33376—Mode signalling or mode changing; Handshaking therefor according to characteristics or state of one of the communicating parties, e.g. available memory capacity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2347—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving video stream encryption
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4405—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video stream decryption
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/61—Network physical structure; Signal processing
- H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
- H04N21/6125—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via Internet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
- H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/16—Analogue secrecy systems; Analogue subscription systems
- H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
- H04N7/17309—Transmission or handling of upstream communications
- H04N7/17318—Direct or substantially direct transmission and handling of requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N2201/333—Mode signalling or mode changing; Handshaking therefor
- H04N2201/33307—Mode signalling or mode changing; Handshaking therefor of a particular mode
- H04N2201/33342—Mode signalling or mode changing; Handshaking therefor of a particular mode of transmission mode
- H04N2201/33357—Compression mode
Definitions
- the present invention relates to a method and system for remote visualization and data analysis of graphical data, in particular the invention relates to remote visualization and data analysis of graphical medical data.
- 3D-scanners such as: Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US), Positron Emission Tomography (PET), and Single Photon Emission Computed Tomography (SPECT), as well as 2D-scanners, such as: Computed Radiography (CR) and Digital Radiography (DR) are available.
- CT Computed Tomography
- MRI Magnetic Resonance Imaging
- US Ultrasound
- PET Positron Emission Tomography
- SPECT Single Photon Emission Computed Tomography
- 2D-scanners such as: Computed Radiography (CR) and Digital Radiography (DR) are available.
- CR Computed Radiography
- DR Digital Radiography
- the CT scanner detects X-ray absorption in a specific volume element of the patient who is scanned
- the MRI scanner uses magnetic fields to detect the presence of water in a specific volume element of the patient who is scanned.
- Both these scanners provide slices of the body, which can be assembled to form a complete 3D image of the scanned section of the patient.
- a common factor of most medical scanners is that the acquired data sets, especially with the 3D-scanners, are quite large, consisting of several hundreds of megabytes for each patient. Such large data sets require significant computing power in order to visualize the data, and especially to process and manipulate the data. Furthermore, transmitting such image data across common networks presents challenges regarding security and traffic congestion.
- the image data generated with medical image scanners are generally managed and stored via electronic database systems under the broad category of Picture Archiving and Communications Systems (PACS systems) which implement the Digital Imaging and Communications in Medicine standard (DICOM standard).
- PPS systems Picture Archiving and Communications Systems
- DICOM standard Digital Imaging and Communications in Medicine standard
- the scanner is connected to a central server computer, or a cluster of server computers, which stores the patient data sets.
- the data may then be accessed from a single or a few dedicated visualization workstations.
- Such workstations are expensive and can therefore normally only be accessed in dedicated diagnostic suites, and not in clinicians offices, hospital wards or operating theaters.
- Another type of less expensive system exists in which a general client-server architecture is used.
- the central server computer may be accessed from a variety of different client types, e.g. a thin client.
- client types e.g. a thin client.
- a visualization program is run on the central server, and the output of the program is via a network connection routed to a remote display of the client.
- a client-server system is the OpenGL VizserverTM system provided by Silicon Graphics, Inc. (http://www.SQi.com/software/vizserver/).
- the system enables clients such as Silicon Graphics® Octane®, and PC based workstations to access the rendering capabilities of an SGI® Onyx® server.
- special software is required to be installed at the client side. This not only limits the type of client, which may be used to access the server, but also adds additional maintenance requirements, as the VizserverTM client software must be installed locally on each client workstation. Further more, the VizserverTM server software does not attempt to re-use information from previously sent frames. It is therefore only feasible to run such a system if a dedicated high-speed data network is available. This is often not the case for many hospitals; furthermore installation of such a network is an expensive task.
- a system for adaptively transporting video over networks wherein the available bandwidth varies with time comprises a video/audio encoder/decoder that functions to compress, code, decode and decompress video streams that are transmitted over the network connection.
- the system adjusts the compression ratio to accommodate a plurality of bandwidths.
- Bandwidth adjustability is provided by offering a trade-off between video resolution, frame rate and individual frame quality.
- the raw video source is split into frames where each frame comprises a multitude of levels of data representing varying degrees of quality.
- a video client receives a number of levels for each frame depending upon the bandwidth, the higher the level received for each frame, the higher the quality of the frame.
- the invention provides a method for transferring graphical data from a first device to an at least second device in a computer-network system, the method comprises the steps of:
- the graphical data may be any type of graphical data but is preferably medical image data, e.g. data acquired in connection with a medical scanning of a patient.
- the graphical data is stored on a first device that may be a central computer, or a central cluster of computers.
- the first device may comprise any type of computer, or cluster of computers, with the necessary aggregate storage capacity to store large data sets which, e.g., arise from scanning of a large number of patients at a hospital.
- the first device should furthermore be equipped with the necessary computing power to be able to handle the demanding tasks of analyzing and manipulating large 3D data sets, such as a 3D image of a human head, a chest, etc.
- the at least second device can be any type of computer machine equipped with a screen for graphical visualization.
- the term visualization should be interpreted to include both 2D visualization and 3D visualization.
- the at least second device may, e.g., be a thin client, a wireless handheld device such as a personal digital assistant (PDA), a personal computer (PC), a tablet PC, a laptop computer or a workstation.
- PDA personal digital assistant
- PC personal computer
- the at least second device machine may merely act as a graphical terminal of the first device.
- the at least second device may be capable of receiving request actions from a user and transferring the requests to the first device, as well as receiving and showing screen images generated by the first device.
- the screen of the at least second device can in many respects be looked upon as a screen connected to the first device.
- An action is requested, e.g. by the user of the at least second device, or by a program call.
- the action may, e.g., result in that a list of possible choices may be shown on the screen of the at least second device, or the action may, e.g., result in that an image related patient data may be shown on the screen of the at least second device.
- the request may be based upon user instructions received from user interaction events such as keystrokes, mouse movements, mouse clicks, etc.
- the first device Upon receiving a request, the first device interprets the request in terms of a request for a specific screen image.
- the first device obtains the relevant patient data from a storage medium to which it is connected.
- the storage medium may be any type of storage medium, such as a hard disk.
- a screen image is generated as a result of the request.
- the present bandwidth of the connection is estimated, and based on the estimated available bandwidth and the type of the request, the screen image is compressed using a corresponding compression method.
- the first device forwards the compressed screen image to the at least second device.
- the first device may, however, also without receiving a request from the at least second device generate a non-requested screen image.
- the non-requested screen image may be based upon relevant patient data, or the non-requested screen image may be unrelated to patient data or any request made by the user.
- the non-requested screen image may be generated due to instructions present at the first device.
- the generation of the screen image may further be conditioned upon a type of the at least second device. If, e.g., the at least second device is a PDA it may be redundant to generate a high-resolution image, since the PDA's available today are limited in their resolution. Therefore the same images generated to a PDA and a thin client, may be generated with lower screen resolution in the case of the PDA than in the case of the thin client.
- the at least second device is a PDA it may be redundant to generate a high-resolution image, since the PDA's available today are limited in their resolution. Therefore the same images generated to a PDA and a thin client, may be generated with lower screen resolution in the case of the PDA than in the case of the thin client.
- the compression method may further be conditioned upon a type of the request.
- Compression of a graphical image may involve a loss, i.e. the image resulting after a compression decompression process is not identical to the image before the compression decompression process, such methods are normally referred to as lossy compression methods. Compression methods that involve a loss are usually faster to perform and the images may be compressed to a higher rate.
- the type of request may be taken into account in situations where it is important that the decompressed image is lossless, or in situations where a loss is unimportant.
- the type of the request may be such as: show an image, rotate an image, zoom in on an image, move an image, etc.
- the compression method may further be conditioned upon a type of the at least second device. Especially the computing power of the at least second device may be taken into account. If, e.g., the at least second device is equipped with a computing power so that the task of decompression is estimated to be too time consuming, a different and less demanding compression method may be used.
- the first device may comprise means for encrypting the screen image before it is sent to the at least second device.
- the at least second device may possess means for decrypting the received screen images before a screen image is generated on the screen of the at least second device.
- the system may include a feature where the user manually sets the level of encryption, or the system may automatically set an appropriate encryption level.
- the time it takes to decrypt the received screen images may depend on the processing means of the at least second device machine, especially handheld devices may be limited in processing power. In certain cases it may therefore be a limiting factor to use demanding encryption routines.
- the encryption routine used for encrypting the data may therefore be dependent upon the type of the at least second device.
- the applications for data analysis, data manipulation and data visualization may be stored on the first device, and may be run from the first device.
- the applications may also be stored on and may be run from a device that is connected to the first device via a computer network connection.
- a multitude of applications may be accessible from the first device.
- the application may include software which is adapted to manipulate both 3D graphical medical data such as data from: MRI, CT, US, PET, and SPECT, as well as 2D graphical medical data such as data from: CR and DR, as well as data from other devices that produce medical images.
- the manipulation may be any standard manipulation of the data such as rotation, zooming in and out, cutting an area, or subset of the data, etc.
- the manipulation may also be less standard manipulation, or it may be unique manipulation specially developed for the present system.
- compression methods may be used.
- the compression method may either be selected manually at session start or may be chosen automatically by the software.
- the different compression methods are applied according to the required compression rate.
- Compression methods may differ in compression time, compression rate as well as, which type of data they are most suitable for.
- a variety of compression method may be used, both standard methods, as well as methods especially developed for the present system.
- GCC Gray Cell Compression
- the GCC method is especially well suited for compressing images where a large fraction of the image is gray scale.
- the GCC method is therefore well suited for compression of medical images since many medical objects may often be imaged in gray scale.
- a session manager at the first device site may create and maintain a session between the at least second device machine and the first device and upload control components to the at least second device.
- the at least second device may be a computer without an operating system (OS), e.g. a thin client.
- OS operating system
- the at least second device may also be a computer with an OS, e.g. a PDA or a PC.
- OS is already functioning on the at least second device, and in this case it may be necessary only to upload a computer application to enable a session.
- a session may, however, also be created and/or maintained without uploading a computer application from the first device to the at least second device. For example, it may suffice to allow the at least second device to receive screen images from the first device. It is not necessary to run a computer application on the at least second device in order to receive, view and/or even manipulate screen images on an at least second device.
- a frame sizer may be present which sets the frame buffer resolution of the at least second device in accordance with the detected available bandwidth, and optionally also in accordance with specifications of the at least second device. That is, if the detected bandwidth is low, the frame buffer resolution may be set to a low value, and the screen image may be generated according to the frame buffer resolution. Setting the frame buffer to a low resolution is a fast way of compressing the data.
- the graphical hardware of most computer systems possess the functionality that if a screen image with a lower resolution than the screen resolution is received, the screen image will automatically be blown up to fill the entire screen. The final screen output on the at least second device is naturally limited in resolution in this case.
- the frame buffer resolution may be set to the screen resolution of the at least second device. In this case, more bandwidth is occupied, but full resolution is sustained.
- the specifications of the at least second device may be taken into account if the at least second device is, e.g. a PDA, since the screen resolution of PDA's which are available today is limited. It would be a waste of bandwidth to transfer an image with a resolution that is too high, only for it to be down sampled at the at least second device.
- An object subsampler which sets the visualization and rendering parameters in accordance with the detected available bandwidth, and optionally also in accordance with the specifications of the at least second device may be present.
- the color depth of the generated screen image may be varied, 8 bit colors may be used while the bandwidth is low, and 16, 24 or 32 bits may be used if the bandwidth permits it.
- the computing power of the at least second device may be taken into account. The time it takes to decompress the received screen images may depend on the processing means of the at least second device machine, especially handheld devices may be limited in processing power. In certain cases it may therefore be faster not to compress, or only slightly compress, the screen images.
- the sized, subsampled, compressed and possibly encrypted data is transferred by an I/O- manager at the first device side to an I/O-manager at the at least second device side, which also handles the transferring of the user-interactions to the first device.
- the requested screen image will only contain a small change from the screen image which is already present on the at least second device screen.
- the screen image generated at the at least second device side is either based on a screen image received from the first device, on the content of a frame buffer at the at least second device side, or on a combination of the received screen image and the contents of the frame buffer. That is, the received screen image contains changes to the previously sent screen image, so that the displayed screen image is a superposition of the previously displayed screen image available through the at least second device's frame buffer, and the received image changes.
- Most networks are shared resources, and the available bandwidth over a network connection at any particular instant varies with both time and location.
- the present available bandwidth is estimated and the rate with which the data is transferred is varied accordingly.
- the at least second device refreshes the screen from the frame buffer of the at least second device in this case. Therefore, the network connection occupies variable amounts of bandwidth.
- Many hospitals, clinics or other medical institutions already have a data network installed, furthermore the medical clinician may sit at home or at a small medical office without access to a high capacity network. It is therefore important that the at least second device and first device may communicate via a number of possible common network connections, such as an Internet connection or an Intranet connection, e.g.
- the second device and the first device may communicate through any type of network, which utilizes the Internet protocol (IP) such as the Internet or other TCP/IP networks.
- IP Internet protocol
- the second device and the first device may communicate both through dedicated and non-dedicated network connections.
- the graphical data may be graphical medical data based on data that conforms to the Digital Imaging and Communications in Medicine standard (DICOM standard) implemented on Picture Archiving and Communications Systems (PACS systems). Most medical scanners support the DICOM standard, which is a standard handling compatibility between different systems.
- Textual data may be presented in a connection with the graphical data. Preferably the textual data is based on data which conforms to the Health Level Seven (HL7) standard or the Electronic Data Interchange for Administration, Commerce and Transport (EDIFACT) standard.
- HL7 Health Level Seven
- EDIFACT Electronic Data Interchange for Administration, Commerce and Transport
- the interchange of graphical and/or medical data may be based on the International Health Exchange (IHE) framework for data interchange.
- IHE International Health Exchange
- a system for transferring graphical data in a computer-network system comprises:
- a second device equipped with means for registering a user input as well as visualization means for visualizing graphical data
- a first device equipped with:
- the first device may further comprise means for encrypting data to be sent via the computer connection between the first device and the at least second device, and the at least second device may comprise means for decrypting the received data.
- the at least second device and the first device may communicate via a common network connection.
- the first device may be a computer server system and the at least second device may, e.g., be a thin client, a workstation, a PC, a tablet PC, a laptop computer or a wireless handheld device.
- the first device may be, or may be part of, a PACS system.
- Fig. 1 shows a schematic view of a preferred embodiment of the present invention
- Fig. 2 shows a schematic flow diagram illustrating the functionally of the Adaptive Streaming Module (ASM);
- ASM Adaptive Streaming Module
- Fig. 3 shows an example of a rotation and the corresponding bandwidth of a data object
- Fig. 4 illustrates the correspondence between the compression time, the compression method used, and the obtainable compression rate for loss-less compression
- Fig. 5 illustrates the correspondence between the compression quality, the compression method used, and the obtainable compression rate for lossy compression.
- the present invention provides a method and system for transferring graphical data from a first device to an at least second device in a computer-network system.
- the invention is in the following described with reference to a preferred embodiment where the graphical data is graphical medical data, and where the computer-network system is a client-server system.
- a schematic view is presented in Fig. 1.
- Medical image data is acquired by using a medical scanner 1 that is connected to a server computer 2.
- a multitude of clients 3 may be connected to the server.
- the server is part of a PACS system.
- the acquired images 16 may automatically or manually be transferred to and stored on a server machine.
- the server may be a separate computer, a cluster of computers or computer system connected via a computer connection. Access to the images may be established at any time thereafter.
- the applications 15 for data analysis and visualization is stored on and may be run from the server machine.
- the server is equipped with the necessary computing power to be able to handle the demanding tasks of analyzing and manipulating large 3D data sets, such as 3D images of a human head, a chest, etc. All data and data applications 15 for visualization and analysis are stored, operated and processed on the server.
- the client 3 can be any type of computer machine equipped with a screen for graphical visualization.
- the client may, e.g., be a thin client 5, a wireless handheld device such as a personal digital assistant (PDA) 6, a personal computer (PC), a laptop computer, a workstation 7, etc.
- PDA personal digital assistant
- PC personal computer
- laptop computer a workstation 7 etc.
- An adaptive streaming module (ASM) 4 is used in order to ensure a continuous stream of data between the server and the client.
- the ASM is capable of estimating the present available bandwidth and vary the rate with which the data is transferred accordingly.
- the ASM 4 is a part of the server machine 2.
- the client may comprise an ASM 5, 6, 7 or it may not comprise an ASM 17.
- a client ASM is not necessary for the system to work.
- the ASM comprises a session manager 8.
- the session manager creates and maintains a session between the client machine and the server.
- the session manager 8 uploads control components to the at least second device. For example if the client is a thin client 5, first an operating system (OS) is uploaded, so that the thin client becomes capable of accepting and sending request actions, as well as receiving and showing screen images generated by the server.
- OS operating system
- the client is a PDA 6 or a PC
- an operating system is already functioning on the client, and in this case it may be necessary only to upload a computer program to enable a session.
- the ASM further comprises a bandwidth manager 9 that continuously measures the available bandwidth.
- a frame sizer 10 that sets the frame buffer resolution of the client.
- An object subsampler 11 that sets the visualization and rendering parameters.
- a compression encoder 12 that compresses an image.
- An encrypter 13 that comprise means for encrypting the data before it is sent to the client 3.
- the sized, subsampled, compressed and encrypted data is transferred by an I/O-manager 14.
- a schematic flow diagram illustrating the functionally of the ASM-module 20 is shown in Fig. 2.
- the user of the medical data may, e.g., be a surgeon who should plan an operation on the background of scanned 3D images.
- the user first establishes a connection from a graphical interface 21, such as a thin client present in his or her office.
- a graphical interface 21 such as a thin client present in his or her office.
- the user is presented with a list from which the user may request access to the relevant images that are to be presented on the computer screen 23.
- the user of the medical data is a clinician on rounds at a ward in a hospital.
- the clinician may carry with him a PDA, onto which he can first log on to the system, and subsequently access the relevant images of the patient.
- the user of the client is requesting an action, such as a specific image of a patient.
- the request 24 is sent to the server, which interprets the request in terms of a request for a specific screen image.
- the server obtains the relevant image data 25 from a storage medium to which it is connected.
- the present bandwidth 26 of the connection is estimated, and based on the detected available bandwidth and a multitude of other parameters, the screen image is compressed to a corresponding compression rate.
- two other parameters may be used for generating the screen image.
- the first parameter may be the color depth 27.
- the second parameter may be the client type 28. If the requesting client machine is a thin client a 19-inch screen may be used as the graphical interface. In this case an image with 768 times 1024 pixels may be generated. But if the requesting machine is a PDA, a somewhat smaller image should be generated, e.g. an image with 300 times 400 pixels, since most PDA's are limited with respect to screen resolution.
- the screen image is generated, compressed and encrypted 22.
- the image is transferred to the client machine, where it is first decrypted and decompressed 29 before it is shown on the screen 23 used by the requesting user.
- surgeon may use a multitude of 3D graphical routines, such as rotation, zooming, etc., for example to obtain insight into the location of the object to be operated on.
- 3D graphical routines such as rotation, zooming, etc.
- An example of a rotation and the corresponding bandwidth of a data object is given in Fig. 3.
- the user has by using the steps explained above in connection with Fig. 2, requested a 3D image of a cranium 30.
- a certain amount of bandwidth 34 has been used, but once the image has been transferred, no, or very little bandwidth, is occupied 35.
- the user now wants to rotate the image in order to obtain a different view 31, 32, 33.
- the user may, e.g., click on the image and while maintaining the mouse button pressed move the mouse in the direction of the desired rotation.
- the type of the request is thus a rotation of the object, and while the mouse button remains pressed, the software treats the request as a rotation. Compression of a graphical image is a tradeoff between resolution and rate.
- Two types of compression methods are used, loss-less compression methods and loss giving compression methods or lossy compression methods. Different compression methods of both types are used. The different compression methods are applied according to the required compression rate. Compression methods may differ in compression time, compression rate as well as which types of images for which they are most suited. The image compression is determined primarily upon the available bandwidth, but also the type of request is important especially with respect to whether a loss-less or a lossy method is used.
- An example of the correspondence between the compression time and the compression rate is given in Fig. 4 for three standard loss-less compression methods: PackBits (or Run length encoding), BZIP2 and Lempel-Ziv-Obenhumer (LZO).
- PackBits or Run length encoding
- BZIP2 or Lempel-Ziv-Obenhumer
- CCC Color Cell Compression
- XCCC Extended Color Cell Compression
- GCC Gray Cell Compression
- the methods may be used separately or one after the other to obtain a higher compression rate.
- CCC CCC: : LZO
- LZO LZO
- the compression time is compared with the obtainable compression size 40, or the compression rate for the PackBits compression method 41, the BZIP2 method 42 and the LZO method 43.
- the exact correspondence between compression time and rate depends upon the structure of the image being compressed. This is illustrated by a certain extension of the area occupied by each method.
- the image quality is compared with the obtainable compression size 50 for a variety of compression methods, single or combined.
- GCC Gray Cell Compression
- the Gray Cell Compression (GCC) method is an example of such a compression method.
- GCC is a variant of the standard CCC technique. It uses the fact that cells containing gray-scale pixels have gray-scale average cell colors. This is exploited for a more efficient encoding of the two average cell colors: In case the average cell color is a gray-scale color, 1 bit is used to mark the color as a gray-scale color and 7 bits are used to represent the gray-scale value. In case the average cell color is non gray-scale color, 1 bit is used to mark the cell as non-gray scale color and 15 bits are used to represent the color itself.
- the compression rate of the GCC method depends on how large a fraction of the image is gray-scale. In worst case, none of the average colors will be gray-scale colors. In this case, the compression rate is 1:8. In the best case, all average colors are gray-scale colors, yielding a compression rate of 1: 12.
- the advantage of the GCC method is that images containing large gray-scale areas may be transferred at a lower bandwidth and a higher image quality when comparing to the standard CCC method.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Graphics (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Claims
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/DK2003/000312 WO2004102949A1 (en) | 2003-05-13 | 2003-05-13 | Method and system for remote and adaptive visualization of graphical image data |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1634438A1 true EP1634438A1 (en) | 2006-03-15 |
Family
ID=33442589
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP03720290A Ceased EP1634438A1 (en) | 2003-05-13 | 2003-05-13 | Method and system for remote and adaptive visualization of graphical image data |
Country Status (5)
Country | Link |
---|---|
EP (1) | EP1634438A1 (en) |
CN (1) | CN100477717C (en) |
AU (1) | AU2003223929A1 (en) |
CA (1) | CA2566638A1 (en) |
WO (1) | WO2004102949A1 (en) |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102004038668A1 (en) * | 2004-08-09 | 2006-02-23 | Siemens Ag | Image data object transmitting method for use in medical area, involves classifying object-based data into safety and non-safety critical data, sending safety data after authorization and non-safety data to target device |
EP1844612B1 (en) * | 2005-02-04 | 2017-05-10 | Barco NV | Method and device for image and video transmission over low-bandwidth and high-latency transmission channels |
US7996495B2 (en) | 2006-04-06 | 2011-08-09 | General Electric Company | Adaptive selection of image streaming mode |
DE102006053261B4 (en) * | 2006-11-11 | 2015-04-16 | Visus Technology Transfer Gmbh | System for the reproduction of medical images |
JP4935385B2 (en) * | 2007-02-01 | 2012-05-23 | ソニー株式会社 | Content playback method and content playback system |
JP5451755B2 (en) | 2008-06-04 | 2014-03-26 | コーニンクレッカ フィリップス エヌ ヴェ | Apparatus, system and method for adaptive data rate control |
CN101606863B (en) * | 2008-06-19 | 2011-05-11 | 深圳市巨烽显示科技有限公司 | Method and system for controlling medical display network |
TWI364220B (en) | 2008-08-15 | 2012-05-11 | Acer Inc | A video processing method and a video system |
CN101686382B (en) * | 2008-09-24 | 2012-05-30 | 宏碁股份有限公司 | Video signal processing method and video signal system |
US8145001B2 (en) | 2008-10-01 | 2012-03-27 | International Business Machines Corporation | Methods and systems for proxy medical image compression and transmission |
US9083759B2 (en) | 2009-02-20 | 2015-07-14 | Citrix Systems, Inc. | Systems and methods for intermediaries to compress data communicated via a remote display protocol |
JP5067409B2 (en) * | 2009-09-28 | 2012-11-07 | カシオ計算機株式会社 | Thin client system and program |
FR2975556B1 (en) * | 2011-05-19 | 2014-05-16 | Keosys | SYSTEM FOR REMOTE RESTRUCTURE OF MEDICAL DATA PROCESSED |
CN102857791B (en) * | 2012-09-14 | 2015-07-08 | 武汉善观科技有限公司 | Method for processing and displaying image data in PACS system by mobile terminal |
CN103121324B (en) * | 2013-02-06 | 2015-09-16 | 心医国际数字医疗***(大连)有限公司 | A kind of medical imaging concentrates the system of printing |
CN104423783B (en) * | 2013-09-02 | 2019-03-29 | 联想(北京)有限公司 | The method and electronic equipment of information transmission |
CN103475953B (en) * | 2013-09-13 | 2017-11-17 | 华为技术有限公司 | A kind of media control method and equipment based on desktop cloud |
CN103914998A (en) * | 2014-04-21 | 2014-07-09 | 中国科学院苏州生物医学工程技术研究所 | Remote medical consultation training system with medical image processing function |
US11361861B2 (en) | 2016-09-16 | 2022-06-14 | Siemens Healthcare Gmbh | Controlling cloud-based image processing by assuring data confidentiality |
US10820151B2 (en) * | 2016-10-06 | 2020-10-27 | Mars, Incorporated | System and method for compressing high fidelity motion data for transmission over a limited bandwidth network |
CN106485079B (en) * | 2016-10-12 | 2019-06-07 | 南京巨鲨医疗科技有限公司 | A kind of medical image cloud processing method |
WO2020030488A1 (en) * | 2018-08-07 | 2020-02-13 | Signify Holding B.V. | Compressive sensing systems and methods using edge nodes of distributed computing networks |
CN111833788B (en) * | 2019-04-19 | 2023-08-04 | 北京小米移动软件有限公司 | Screen dimming method and device, terminal and storage medium |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1056273A3 (en) * | 1999-05-25 | 2002-01-02 | SeeItFirst, Inc. | Method and system for providing high quality images from a digital video stream |
CA2312333A1 (en) * | 2000-06-21 | 2001-12-21 | Kimihiko E. Sato | Multimedia compression, coding and transmission method and apparatus |
GB2367219A (en) * | 2000-09-20 | 2002-03-27 | Vintage Global | Streaming of media file data over a dynamically variable bandwidth channel |
WO2002051148A1 (en) * | 2000-12-18 | 2002-06-27 | Agile Tv Corporation | Method and processor engine architecture for the delivery of audio and video content over a broadband network |
US8107524B2 (en) * | 2001-03-30 | 2012-01-31 | Vixs Systems, Inc. | Adaptive bandwidth footprint matching for multiple compressed video streams in a fixed bandwidth network |
-
2003
- 2003-05-13 AU AU2003223929A patent/AU2003223929A1/en not_active Abandoned
- 2003-05-13 CA CA002566638A patent/CA2566638A1/en not_active Abandoned
- 2003-05-13 CN CNB038266539A patent/CN100477717C/en not_active Expired - Fee Related
- 2003-05-13 EP EP03720290A patent/EP1634438A1/en not_active Ceased
- 2003-05-13 WO PCT/DK2003/000312 patent/WO2004102949A1/en active Application Filing
Non-Patent Citations (1)
Title |
---|
See references of WO2004102949A1 * |
Also Published As
Publication number | Publication date |
---|---|
CA2566638A1 (en) | 2004-11-25 |
CN1839618A (en) | 2006-09-27 |
AU2003223929A1 (en) | 2004-12-03 |
WO2004102949A1 (en) | 2004-11-25 |
CN100477717C (en) | 2009-04-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20040240752A1 (en) | Method and system for remote and adaptive visualization of graphical image data | |
EP1634438A1 (en) | Method and system for remote and adaptive visualization of graphical image data | |
US6711297B1 (en) | Methods and apparatus for dynamic transfer of image data | |
US8508539B2 (en) | Method and system for real-time volume rendering on thin clients via render server | |
US6424996B1 (en) | Medical network system and method for transfer of information | |
US7602950B2 (en) | Medical system architecture for interactive transfer and progressive representation of compressed image data | |
EP1236082B1 (en) | Methods and apparatus for resolution independent image collaboration | |
US8422770B2 (en) | Method, apparatus and computer program product for displaying normalized medical images | |
USRE42952E1 (en) | Teleradiology systems for rendering and visualizing remotely-located volume data sets | |
CN101334818B (en) | Method and apparatus for efficient client-server visualization of multi-dimensional data | |
US20060122482A1 (en) | Medical image acquisition system for receiving and transmitting medical images instantaneously and method of using the same | |
US7492970B2 (en) | Reporting system in a networked environment | |
US8417043B2 (en) | Method, apparatus and computer program product for normalizing and processing medical images | |
US8068546B2 (en) | Method and apparatus for transmitting video signals | |
EP2669830A1 (en) | Preparation and display of derived series of medical images | |
US20070223793A1 (en) | Systems and methods for providing diagnostic imaging studies to remote users | |
US20080043015A1 (en) | Online volume rendering system and method | |
US7765109B2 (en) | Systems and methods for obtaining readings of diagnostic imaging studies | |
US20170228918A1 (en) | A system and method for rendering a video stream | |
Pohjonen et al. | Pervasive access to images and data—the use of computing grids and mobile/wireless devices across healthcare enterprises | |
CN1392505A (en) | Medical information service system | |
Stoian et al. | Current trends in medical imaging acquisition and communication | |
Haggag et al. | Interactive secure dissemination of medical images using JPEG 2000 | |
PRZELASKOWSKI | The JPEG2000 standard for medical image applications | |
Swarnakar et al. | Multitier image streaming teleradiology system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20051212 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: KJAER, NIELS, HUSTED Inventor name: OLSEN, MORTEN, SYLVEST Inventor name: KARAIVANOV, ALEXANDER, DIMITROV Inventor name: DOBBS, ANDREW, BRUNO |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20061128 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: MEDICAL INSIGHT A/S |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED |
|
18R | Application refused |
Effective date: 20080624 |