EP1710719B1 - Image transmission method, computer-readable image transmission program, and image transmission apparatus - Google Patents
Image transmission method, computer-readable image transmission program, and image transmission apparatus Download PDFInfo
- Publication number
- EP1710719B1 EP1710719B1 EP20060251953 EP06251953A EP1710719B1 EP 1710719 B1 EP1710719 B1 EP 1710719B1 EP 20060251953 EP20060251953 EP 20060251953 EP 06251953 A EP06251953 A EP 06251953A EP 1710719 B1 EP1710719 B1 EP 1710719B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- tile
- unit
- precinct
- image
- generating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Not-in-force
Links
- 238000000034 method Methods 0.000 title claims description 142
- 230000005540 biological transmission Effects 0.000 title description 68
- 230000004044 response Effects 0.000 claims description 36
- 238000004590 computer program Methods 0.000 claims 1
- 230000008569 process Effects 0.000 description 40
- 238000010586 diagram Methods 0.000 description 24
- 230000002452 interceptive effect Effects 0.000 description 10
- 238000010276 construction Methods 0.000 description 6
- 230000000750 progressive effect Effects 0.000 description 6
- VBRBNWWNRIMAII-WYMLVPIESA-N 3-[(e)-5-(4-ethylphenoxy)-3-methylpent-3-enyl]-2,2-dimethyloxirane Chemical compound C1=CC(CC)=CC=C1OC\C=C(/C)CCC1C(C)(C)O1 VBRBNWWNRIMAII-WYMLVPIESA-N 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
- H04N19/64—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by ordering of coefficients or of bits for transmission
- H04N19/647—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by ordering of coefficients or of bits for transmission using significance based coding, e.g. Embedded Zerotrees of Wavelets [EZW] or Set Partitioning in Hierarchical Trees [SPIHT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234327—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
- H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
- H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/16—Analogue secrecy systems; Analogue subscription systems
- H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
- H04N7/17309—Transmission or handling of upstream communications
- H04N7/17318—Direct or substantially direct transmission and handling of requests
Definitions
- the present invention relates to an image transmission method, a computer-readable image transmission program, a recording medium, and an image transmission apparatus.
- JPEG 2000 was standardized as a novel image encoding method in the end of 2000 (refer to Non-patent Documents 1 to 5 cited below, for example).
- JPEG 2000 Part9 referred to as JPIP is an international standard for transmitting video and still images encoded using JPEG 2000.
- JPIP includes a transmission method with a tile unit (JPT-stream method) and a transmission method with a precinct unit which is an area unit smaller than a tile (JPP-stream method) (refer to Patent Documents 1 to 5, for example).
- FIG. 1 is a configuration diagram of a JPIP system.
- the JPIP system includes a JPIP server and a JPIP client.
- the JPIP server returns an image area (or code words corresponding to the image area) to the JPIP client in response to a request from the JPIP client.
- the JPIP client includes an image name of a target and the image area in the request and transmits the request to the JPIP server.
- the JPIP server obtains data of the image area from the target image on the basis of information about the image name, the image area, and the like, and returns a response including the data of the image area to the JPIP client.
- FIG. 2 is a functional configuration diagram of the JPIP system in the JPT-stream method.
- a client side in the transmission method with a tile unit includes a request message generating unit, a response message decoding unit, and a tile part combining unit.
- a server side in the transmission method with a tile unit includes a request message decoding unit, a tile identifying unit, a tile part reading unit, and a response message generating unit.
- the request message generating unit generates a JPIP request message including a name of a target image, information about a view window, and the like.
- the request message decoding unit obtains the name of the target image, the information about the view window, and the like from the JPIP request message.
- the tile identifying unit selects a tile within an area of the requested view window on the basis of the information obtained by the request message decoding unit.
- t ⁇ x 0 w ⁇ x 0 ⁇ T 0 ⁇ x / T s ⁇ x
- t ⁇ y 1 w ⁇ y 1 ⁇ T 0 ⁇ y / T s ⁇ y
- the tile reading unit obtains a tile part and the response message generating unit generates a response message.
- the client side receives the response message and decodes the response message in the response message decoding unit. Then, the client side connects the decoded data and the like (tile parts, for example) in the tile part combining unit or the like and generates a code stream.
- the tile is a space unit for transmitting the code stream, so that the capability of transmission in space depends on the size of the tile.
- SOT Start-Of-Tile-part
- the client side is capable of readily receiving tile parts and simply connecting them, for example.
- JPT-stream method it is relatively easy to construct the JPIP system in the transmission method with a tile unit.
- JPT-stream method the transmission method with a tile unit
- JPT-stream method is problematic in that the capability of transmission in space is rough, since the tile is used as the transmission unit.
- FIG. 3 is a functional configuration diagram of the JPIP system in the JPP-stream method.
- a client side in the transmission method with a precinct unit includes a request message generating unit, a response message decoding unit, a tile part assembling unit, and a tile part combining unit.
- a server side in the transmission method with a precinct unit includes a request message decoding unit, a tile identifying unit, a precinct identifying unit, a packet reading unit, and a response message generating unit.
- the request message generating unit generates a JPIP request message including a name of a target image, information about a view window, and the like.
- the request message decoding unit obtains the name of the target image, the information about the view window, and the like from the JPIP request message.
- the tile identifying unit selects a tile within an area of the requested view window on the basis of the information obtained by the request message decoding unit.
- the precinct identifying unit selects a precinct of the tile positioned within the area of the requested view window and selected by the tile identifying unit.
- PPx and PPy indicate indexes of the width and length of the precinct, respectively. Further, (wx' 0 , wy' 0 ) and (wx' 1 , wy' 1 ) indicate a range of the view window on the reference grid of the subband domain enlarged using a wavelet filter.
- the packet reading unit collects (or reads out) packets concerning the precinct selected by the precinct reading unit. Then, the response generating unit generates a response message including the packets.
- the client side receives the response message, decodes the response message in the response message decoding unit, and assembles the packets and the like into a tile part in the tile part assembling unit. Thereafter, the client side connects the tile parts in the tile part combining unit and generates a code stream. An application in the client side decodes the code stream and displays an image on a display unit or the like of the client.
- the precinct which is relatively smaller than a tile, is used as the unit.
- interactive capability in space is improved in comparison with the JPIP system in the transmission method with a tile unit (JPT-stream method) and it is possible to efficiently transmit encoded data in the area of the object view window to the client side.
- JPT-stream method the JPIP system transmits an image with the improved efficiency.
- Non-patent Document 1 M. Boliek, C. Christopoulos and E. Majani, "J JPEG 2000 Part 1 020719 (Final Publication Draft),” ISO/IEC JTC I/SC 29/WG1 N2678, July 2002 .
- Non-patent Document 2 M. Gormish, D. Lee and M Marcellin, "JPEG 2000: Overview, Architecture, and Applications,” Int. Conf. on Image Processing, Vancouver, Canada, September 2000 .
- Non-patent Document 3 R. Prandolini, G. Colyer, and S. Houchin, "15444-9:2004 JPEG 2000 image coding system - Part 9: Interactivity tools, APIs and protocols - JPIP," Final Publication Draft Revision 3, ISO/IEC JTC 1/SC 29/WG 1 N3463, November 2004 .
- Non-patent Document 4 D. Taubmana and R. Prandolini, "Architecture, Philosophy and Performance of JPIP: Internet Protocol Standard for JPEG 2000," VCIP2003 SPIE volume 5150 pp. 649-663, July 2003 .
- Non-patent Document 5 M. Gormish and S. Banerjee, "Tile-based transport of JPEG 2000 images," VLVB03, Madrid, Spain, September 2003 .
- Patent Document 1 Japanese Laid-Open Patent Application No. 2004-274758
- Patent Document 2 Japanese Laid-Open Patent Application No. 2004-208266
- Patent Document 3 Japanese Laid-Open Patent Application No. 2004-349939
- Patent Document 4 Japanese Laid-Open Patent Application No. 2004-228717
- Patent Document 5 Japanese Laid-Open Patent Application No. 2004-40674
- the packet reading unit of the server side in the transmission method with a precinct unit must perform a complicated process.
- the tile part assembling unit must reconstruct (or assemble) a tile part rearranging packets in each precinct, for example, and generate a code stream, so that the process of the client side becomes complicated and thus poses a problem. Therefore, the JPIP system in the transmission method with a precinct unit (JPP-stream method) is problematic in that it requires high construction costs.
- the present invention has been made in view of the aforementioned problems.
- the coefficient data corresponds to wavelet coefficient (or wavelet coefficient data) or the like, for example.
- the level corresponds to a subband level or the like, for example.
- the coefficient areas correspond to precincts in JPEG 2000, for example.
- the code words correspond to encoded data, namely, encoded image data or the like, for example.
- the small image areas correspond to tiles or the like, for example.
- a computer-readable image transmission program, a recording medium, and an image transmission apparatus may be used so as to achieve the aforementioned objects.
- an image transmission method it is possible to provide an image transmission method, a computer-readable image transmission program, and an image transmission apparatus in which the transmission efficiency of an image is improved and the complexity of constructing a client side is reduced.
- FIG. 4 is a configuration diagram of hardware of a server.
- the server includes an input apparatus 11, a display apparatus 12, a drive apparatus 13, a memory apparatus 15, a CPU 16, and an interface apparatus 17.
- the input apparatus 11 comprises a keyboard and a mouse for operation by a user of the server and is used for inputting various types of operation information to the server.
- the display apparatus 12 comprises a display unit or the like for use by the user of the server and is used for displaying various types of information.
- the interface apparatus 17 is for connecting the server to a network and the like.
- a program of the server is installed on the memory apparatus 15 or the like before shipment, supplied to the server using a recording medium 14 such as a CD-ROM after shipment, or downloaded via a network and the like, for example.
- the recording medium 14 is set in the drive apparatus 13 and the program is installed on the memory apparatus 15 or the like from the recording medium 14 via the drive apparatus 13.
- the CPU 16 supplies functions as will be described in the following in accordance with the program while using the memory apparatus 15 as appropriate and performs processes as shown in a flowchart as will be described in the following, for example.
- the memory apparatus 15 stores image information and the like, for example.
- a client and the like have the same hardware configuration as that of the server.
- the CPU of the client performs processes and the like as will be described in the following in accordance with a program of the client while using as appropriate the memory apparatus and the like of the client, for example.
- FIG. 5 is a functional configuration diagram of the JPIP system in the JPT-stream method according to the present invention.
- a client side in the transmission method with a tile unit (JPT-stream method) according to the present invention includes a request message generating unit 21, a response message decoding unit 22, and a tile part combining unit 23.
- a server side in the transmission method with a tile unit (JPT-stream method) according to the present invention includes a request message decoding unit 31, a tile identifying unit 32, a precinct identifying unit 33, a packet reading unit 34, a tile part assembling unit 35, and a response message generating unit 36.
- the request message generating unit 21 generates a JPIP request message including a name of a target image, information about a view window, and the like.
- the request message decoding unit 31 obtains the name of the target image, the information about the view window, and the like from the JPIP request message.
- the tile identifying unit 32 selects a tile within an area of the requested view window on the basis of the information obtained by the request message decoding unit 31.
- the precinct identifying unit 33 selects a precinct of the tile positioned within the area of the requested view window and selected by the tile identifying unit 32.
- the packet reading unit 34 collects (or reads out) packets concerning the precinct selected by the precinct reading unit 33.
- the tile part assembling unit 35 assembles (or generates) a tile part including a tile header and packet data concerning the precinct within the area of the requested view window using the packets collected (or read out) by the packet reading unit 34 and position information about the packets.
- the response message generating unit 36 generates a response message including the tile part assembled by the tile part assembling unit 35.
- the response message decoding unit 22 decodes the response message received from the server.
- the tile part combining unit 23 connects tile parts included in the response massage and generates a code stream.
- the JPIP system in the JPT-stream method according to the present invention includes the tile part assembling unit, which is included in the client side in the JPIP system in the JPP-stream method shown in FIG. 3 , in the server side as the tile part assembling unit 35.
- FIG. 6 is a flowchart showing the processing in the server.
- step S10 the server judges whether the JPIP request message is received from the client side. If the server judges that the JPIP request message is received from the client side (YES in step S10), the process proceeds to step S11. If the server judges that the JPIP request message is not received from the client side (NO in step S10), the process returns to step S10.
- step S11 the server decodes the received JPIP request message and performs a process for obtaining the name of the target image, the information about the view window, and the like included in the JPIP request message.
- step S12 the server performs a process for selecting a tile within the area of the requested view window on the basis of the information and the like obtained in step S11.
- step S12 the process proceeds to step S13 and the server performs a process for selecting a precinct of the tile positioned within the area of the requested view window and selected in step S12.
- step S14 the process proceeds to step S14 and the server performs a process for collecting (or reading out) packets_concerning the precinct selected in step S13.
- step S14 the process proceeds to step S15 and the server judges whether the processes in steps S13 and S14 are performed on all precincts. If the server judges that the processes in steps S13 and S14 are performed on all the precincts (YES in step S15), the process proceeds to step S16. If the server judges that the processes in steps S13 and S14 are not performed on all the precincts (NO in step S15), the process proceeds to step S13.
- step S16 the server judges whether the processes in steps S12 through S15 are performed on all tiles. If the server judges that the processes in steps S12 through S15 are performed on all the tiles (YES in step S16), the process proceeds to step S17. If the server judges that the processes in steps S12 through S15 are not performed on all the tiles (NO in step S16), the process returns to step S12.
- step S17 the server judges whether the server is requested on a tile basis, namely, whether the server is requested to transmit the response message including the tile part. If the server judges that the server is requested on a tile basis (YES in step S17), the process proceeds to step S18. If the server judges that the server is not requested on a tile basis (NO in step S17), the process proceeds to step S19.
- the server judges that the server is requested on a tile basis. If the JPIP request message does not include information about a request on a tile basis, the server judges that the server is not requested on a tile basis.
- step S18 the server assembles (or generates) the tile part including a tile header and packet data concerning the precincts within the area of the requested view window using the packets collected (or read out) in step S14 and position information about the packets.
- step 19 the server generates the response message. For example, if the process proceeds to step S19 through step S18, the server generates a response message (JPT-stream message) including the tile part, and if the process proceeds from step S17 to step S19 without step S18, the server generates a response message (JPP-stxeam message) including the packets collected (or read out) in step S14.
- JPT-stream message a response message including the tile part
- JPP-stxeam message JPP-stxeam message
- step S19 the process proceeds to step S20 and the server transmits the response message created in step S19 to the client as a requestor, for example.
- step S20 the process proceeds to step S21 and the server judges whether to end the process (process concerning the JPIP system, for example). If the server judges that the process is to be ended (YES in step S21), the process is ended. If the server judges that the process is not to be ended (NO in step S21), the process proceeds to step S10.
- JPP-stream method In the transmission method with a precinct unit (JPP-stream method), it is not necessary to transmit unselected (or unrequested) precincts to the client, the precincts being outside the area of the view window requested from the client.
- JPT-stream method in JPEG 2000, although it is not necessary to send unselected tiles, information (or data) about an entire tile part is required even when the client requests only a specific portion of a tile.
- the tile part assembling unit 35 generates the tile part including a zero length packet (hereafter simply referred to as ZLP) with only one byte (or several bytes if needed) for indicating that the length of packet data concerning the precinct of the unselected area is zero, for example.
- ZLP a zero length packet
- FIG. 7 is a diagram showing the example of the tile including ZLPS in the subband level.
- the tile in the subband level has 25 precincts and the area of the view window requested from the client covers the area of the precincts from (iii, b) to (v, e).
- the precinct identifying unit 33 identifies the precincts of this area and the packet reading unit 34 obtains packet data concerning the precincts identified by the precinct identifying unit 33.
- the tile part assembling unit 35 generates a new tile part in which the area of precincts unrequested by the client is filled with ZLPS.
- the tile part assembling unit 35 When a frame size requested from the client is smaller than the size of a requested image in the server by L 2 in the length and width, the tile part assembling unit 35 does not have to send code data in L zones from the requested image size in the subband level. Accordingly, the tile part assembling unit 35 may generate a tile part without using ZLPS for the L zones of the requested image size. In addition, the tile part assembling unit 35 may generate the tile part filling the L zones with ZLPS. Instead of filling the precinct area unrequested by the client with ZLPS, the tile part assembling unit 35 may generate a new tile part including information about the nonexistence of packet data concerning the precincts of the unselected area.
- the tile part assembling unit 35 may generate a new tile part including information about the fact that the length of packet data concerning a code block of the unselected area is zero. Further, the tile part assembling unit 35 may generate a new tile part including information about the nonexistence of packet data concerning the code block of the unselected area.
- the method for constructing ZLPS may use a packet for indicating the nonexistence of packet data and a packet for indicating that packet data is not included in addition to the aforementioned methods.
- the following describes another example of the generation of the tile part including ZLPS and yet another example of the generation of the tile part using precincts necessary to send the precincts of the area of the view window requested from the client without using ZLPS.
- FIG. 8 is a diagram showing an example of a gray image divided into four tiles.
- FIG. 9 is a diagram showing precincts and numbers thereof when each tile shown in FIG. 8 is wavelet-transformed in three levels.
- FIG. 10 is a first diagram showing the area concerning the precincts to be returned to the client when the client accesses (or requests) the portion of the gray image in FIG. 8 reduced to 1/2.
- the server transmits only packet data concerning the precincts of a shaded portion of FIG. 10 , for example.
- JPT-stream method in JPEG 2000, it is necessary to transmit information (or data) about an entire tile part even when the client requests only a specific portion of an image (the portion of the gray image reduced to 1/2 in this example).
- FIG. 11 is a diagram showing the example of the generation of the tile part using ZLPS.
- the tile part assembling unit 35 generates the tile part using packet data concerning the precincts of shaded portions shown in FIG. 11 and the ZLPS of horizontal-striped portions shown in FIG. 11 , namely, 0, 2, 8, 10, 16, 18, 24, 26, 32, 34, 40, and 42 in FIG. 11 and 4 , 6 , 12 , 14 , 20, 22, 28, 30, 36, 38, 44, and 46 in FIG. 11 , respectively.
- FIG. 12 is a second diagram showing the area concerning the precincts to be returned to the client when the client accesses (or requests) the portion of the gray image in FIG. 8 reduced to 1/2.
- the server transmits only packet data concerning the precincts of a shaded portion in the left of FIG. 12 , for example.
- the server generates the tile part as shown in the right of FIG. 12 , using the packet data concerning the precincts included in the tile including the precincts shown in the shaded portion in the left of FIG. 12 .
- FIG. 13 is a diagram showing the example of the generation of the tile part using the precincts concerning the area requested from the client, the precincts being included in the tile.
- the tile part assembling unit 35 generates the tile part using packet data concerning the precincts of striped portions shown in FIG. 13 , for example.
- FIG. 14 is an illustration showing the capability of interactive transmission in space when the tile part is generated using ZLPS in the JPT-stream method according to the present invention.
- FIG. 14-(A) is an image when the normal JPT-stream method is used and FIG. 14-(B) is an image when the JPT-stream method according to the present invention is used.
- Squares in the middle of FIG. 14-(A) and (B) indicate the area of a view window and crossed lines indicate the boundary of tiles.
- the JPT-stream method according to the present invention is capable of constructing necessary tile parts and transmitting them to the client.
- FIG. 15 is a table showing the capability of interactive transmission in space when the tile part is generated using ZLPS in the JPT-stream method according to the present invention.
- the stream code size is 166582 bytes and the size of the frame is (512, 512).
- the JPT-stream method according to the present invention has the same capability of interactive transmission in space as that of the JPP-stream method.
- FIG. 16 is an illustration showing the capability of progressive transmission when the tile part is generated without using ZLPS in the JPT-stream method according to the present invention.
- progressive transmission can be readily performed also in the JPT-stream method according to the present invention.
- FIG. 17 is a table showing the capability of progressive transmission when the tile part is generated without using ZLPS in the JPT-stream method according to the present invention.
- the stream code size is 164962 bytes and the volume of transmission bytes in the normal JPT-stream is 168626 bytes.
- the JPT-stream method according to the present invention is capable of transmitting an image with improved resolution in comparison with the normal the JPP-stream method.
- the tile part assembling unit 35 generates the tile part including a tile header and packet data concerning the precincts within the area of the requested view window using the packets and position information about the packets.
- the tile part assembling unit 35 may generate a tile part in each subband level using packet data concerning the subband level of the requested view window, the tile part including the packet data.
- the tile part assembling unit 35 may generate a tile part in each color component using precincts concerning the color component or packet data concerning a subband level, the tile part including the packet data.
- the image transmission apparatus corresponds to the aforementioned server or the like, for example.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Computer Graphics (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Description
- The present invention relates to an image transmission method, a computer-readable image transmission program, a recording medium, and an image transmission apparatus.
- In recent years, in the fields of ubiquitous computing, home computer networking, and the like, it is important to transmit information other than sound such as video and still images. In many systems for transmitting video and still images, encoding techniques such as JPEG (Joint Photographic Experts Group), MPEG (Moving Picture Experts Group), and the like are used for efficient encoding.
- Transmission methods using encoding such as JPEG, MPEG, and the like are efficient in transmission in a predetermined format but are problematic in that they are not suitable (namely, not efficient) for interactive transmission.- In view of this,
JPEG 2000 was standardized as a novel image encoding method in the end of 2000 (refer to Non-patentDocuments 1 to 5 cited below, for example). - Due to the image encoding method of
JPEG 2000, it is possible to efficiently encode video and still images in interactive transmission and to process in accordance with an object without decoding code streams after the encoding. - JPEG 2000 Part9 referred to as JPIP is an international standard for transmitting video and still images encoded using
JPEG 2000.. JPIP includes a transmission method with a tile unit (JPT-stream method) and a transmission method with a precinct unit which is an area unit smaller than a tile (JPP-stream method) (refer toPatent Documents 1 to 5, for example). - The following describes an example of a configuration of a server/client system according to JPIP (hereafter simply referred to as a JPIP system) with reference to
FIG. 1. FIG. 1 is a configuration diagram of a JPIP system. As shown inFIG. 1 , The JPIP system includes a JPIP server and a JPIP client. The JPIP server returns an image area (or code words corresponding to the image area) to the JPIP client in response to a request from the JPIP client. The JPIP client includes an image name of a target and the image area in the request and transmits the request to the JPIP server. The JPIP server obtains data of the image area from the target image on the basis of information about the image name, the image area, and the like, and returns a response including the data of the image area to the JPIP client. - The following describes the functional configuration of the JPIP system in the transmission method with a tile unit (JPT-stream method) with reference to
FIG. 2. FIG. 2 is a functional configuration diagram of the JPIP system in the JPT-stream method. - As shown in
FIG. 2 , a client side in the transmission method with a tile unit (JPT-stream method) includes a request message generating unit, a response message decoding unit, and a tile part combining unit. A server side in the transmission method with a tile unit (JPT-stream method) includes a request message decoding unit, a tile identifying unit, a tile part reading unit, and a response message generating unit. - The request message generating unit generates a JPIP request message including a name of a target image, information about a view window, and the like. The request message decoding unit obtains the name of the target image, the information about the view window, and the like from the JPIP request message. The tile identifying unit selects a tile within an area of the requested view window on the basis of the information obtained by the request message decoding unit. In this case, the coordinates of the upper left corner (tx0, ty0) and the coordinates of the lower right corner (tx1, ty1) of the tile selected on a reference grid are represented as follows:
- The tile reading unit obtains a tile part and the response message generating unit generates a response message. The client side receives the response message and decodes the response message in the response message decoding unit. Then, the client side connects the decoded data and the like (tile parts, for example) in the tile part combining unit or the like and generates a code stream.
- The tile is a space unit for transmitting the code stream, so that the capability of transmission in space depends on the size of the tile. Thus, in the
standard JPEG 2000, Start-Of-Tile-part (SOT), a marker having the length of a tile and positional information of a start, is defined. Accordingly, the client side is capable of readily receiving tile parts and simply connecting them, for example. Thus, it is relatively easy to construct the JPIP system in the transmission method with a tile unit (JPT-stream method). However, the transmission method with a tile unit (JPT-stream method) is problematic in that the capability of transmission in space is rough, since the tile is used as the transmission unit. - Next, the following describes the functional configuration of the JPIP system in the transmission method with a precinct unit (JPP-stream method) with reference to
FIG. 3. FIG. 3 is a functional configuration diagram of the JPIP system in the JPP-stream method. - As shown in
FIG. 3 , a client side in the transmission method with a precinct unit (JPP-stream method) includes a request message generating unit, a response message decoding unit, a tile part assembling unit, and a tile part combining unit. A server side in the transmission method with a precinct unit (JPP-stream method) includes a request message decoding unit, a tile identifying unit, a precinct identifying unit, a packet reading unit, and a response message generating unit. - The request message generating unit generates a JPIP request message including a name of a target image, information about a view window, and the like. The request message decoding unit obtains the name of the target image, the information about the view window, and the like from the JPIP request message. The tile identifying unit selects a tile within an area of the requested view window on the basis of the information obtained by the request message decoding unit. The precinct identifying unit selects a precinct of the tile positioned within the area of the requested view window and selected by the tile identifying unit. In this case, the coordinates of the upper left corner (px0, py0) and the coordinates of the lower right corner (px1, py1) of the precinct selected on a reference grid in a subband domain are represented as follows:
- The packet reading unit collects (or reads out) packets concerning the precinct selected by the precinct reading unit. Then, the response generating unit generates a response message including the packets. The client side receives the response message, decodes the response message in the response message decoding unit, and assembles the packets and the like into a tile part in the tile part assembling unit. Thereafter, the client side connects the tile parts in the tile part combining unit and generates a code stream. An application in the client side decodes the code stream and displays an image on a display unit or the like of the client.
- In the JPIP system in the transmission method with a precinct unit (JPP-stream method), the precinct, which is relatively smaller than a tile, is used as the unit. Thus, interactive capability in space is improved in comparison with the JPIP system in the transmission method with a tile unit (JPT-stream method) and it is possible to efficiently transmit encoded data in the area of the object view window to the client side. Accordingly, by constructing the JPIP system using the transmission method with a precinct unit (JPP-stream method), the JPIP system transmits an image with the improved efficiency.
- Non-patent Document 1: M. Boliek, C. Christopoulos and E. Majani, "J JPEG 2000 .
- Non-patent Document 2: M. Gormish, D. Lee and M Marcellin, "JPEG 2000: Overview, Architecture, and Applications," Int. Conf. on Image Processing, Vancouver, Canada, September 2000.
- Non-patent Document 3: R. Prandolini, G. Colyer, and S. Houchin, "15444-9:2004 .
- Non-patent Document 4: D. Taubmana and R. Prandolini, "Architecture, Philosophy and Performance of JPIP: Internet Protocol Standard for .
- Non-patent Document 5: M. Gormish and S. Banerjee, "Tile-based transport of .
- Patent Document 1:
Japanese Laid-Open Patent Application No. 2004-274758 - Patent Document 2:
Japanese Laid-Open Patent Application No. 2004-208266 - Patent Document 3:
Japanese Laid-Open Patent Application No. 2004-349939 - Patent Document 4:
Japanese Laid-Open Patent Application No. 2004-228717 - Patent Document 5:
Japanese Laid-Open Patent Application No. 2004-40674 - However, the packet reading unit of the server side in the transmission method with a precinct unit (JPP-stream method) must perform a complicated process. Also, on the client side in the transmission method with a precinct unit (JPP-stream method), the tile part assembling unit must reconstruct (or assemble) a tile part rearranging packets in each precinct, for example, and generate a code stream, so that the process of the client side becomes complicated and thus poses a problem. Therefore, the JPIP system in the transmission method with a precinct unit (JPP-stream method) is problematic in that it requires high construction costs. In particular, when the process of the client side is complicated and the construction costs thereof are increased, vendors and the like are likely to construct a client and the like in the transmission method with a tile unit (JPT-stream method) so as to reduce the construction costs. As a result, the number of clients in the transmission method with a tile unit (JPT-stream method) having an inferior transmission efficiency is increased. Further, there is a problem of compatibility between the transmission method with a tile unit (JPT-stream method) and the transmission method with a precinct unit (JPP-stream method), for example.
- The present invention has been made in view of the aforementioned problems.
- It is general object of the present invention to provide an improved and useful image processing system and method, in which the above-mentioned problems are eliminated and preferably the transmission efficiency of an image is improved and the complexity of the construction of the client side is reduced.
- According to the present invention there is provided an image processing system and method as defined in the appended claims.
- In addition, the coefficient data corresponds to wavelet coefficient (or wavelet coefficient data) or the like, for example. The level corresponds to a subband level or the like, for example. The coefficient areas correspond to precincts in
JPEG 2000, for example. The code words correspond to encoded data, namely, encoded image data or the like, for example. Further, the small image areas correspond to tiles or the like, for example. - A computer-readable image transmission program, a recording medium, and an image transmission apparatus may be used so as to achieve the aforementioned objects.
- According to the present invention, it is possible to provide an image transmission method, a computer-readable image transmission program, and an image transmission apparatus in which the transmission efficiency of an image is improved and the complexity of constructing a client side is reduced.
- The present invention will be described below with reference to exemplary embodiments and the accompanying drawings, in which:
-
FIG. 1 is a configuration diagram of a JPIP system; -
FIG. 2 is a functional configuration diagram of a JPIP system in a JPT-stream method; -
FIG. 3 is a functional configuration diagram of a JPIP system in a JPP-stream method; -
FIG. 4 is a configuration diagram of hardware of a server; -
FIG. 5 is a functional configuration diagram of a JPIP system in a JPT-stream method according to the present invention; -
FIG. 6 is a flowchart showing processing in the server; -
FIG. 7 is a diagram showing an example of a tile including ZLPS in a certain subband level; -
FIG. 8 is-a diagram showing an example of a gray image divided into four tiles; -
FIG. 9 is a diagram showing precincts and numbers thereof when each tile shown inFIG. 8 is wavelet-transformed in three levels; -
FIG. 10 is a first diagram showing an area concerning precincts to be returned_to a client when a client accesses (or requests) a portion of the gray image inFIG. 8 reduced to 1/2; -
FIG. 11 is a diagram showing an example of the generation of a tile part using ZLPS; -
FIG. 12 is a second diagram showing an area concerning precincts to be returned to a client when the client accesses (or requests) a portion of the gray image inFIG. 8 reduced to 1/2; -
FIG. 13 is a diagram showing an example of the generation of a tile part using precincts concerning an area requested from a client, the precincts being included in a tile; -
FIG. 14 is an illustration showing the capability of interactive transmission in space when a tile part is generated using ZLPS in the JPT-stream method according to the present invention; -
FIG. 15 is a table showing the capability of interactive transmission in space when a tile part is generated using ZLPS in the JPT-stream method according to the present invention; -
FIG. 16 is an illustration showing the capability of progressive transmission when a tile part is generated without ZLPS in the JPT-stream method according to the present invention; and -
FIG. 17 is a table showing the capability of progressive transmission when a tile part is generated without ZLPS in the JPT-stream method according to the present invention. - In the following, embodiments of the present invention will be described with reference to the accompanying drawings.
FIG. 4 is a configuration diagram of hardware of a server. As shown inFIG. 4 , the server includes aninput apparatus 11, adisplay apparatus 12, adrive apparatus 13, amemory apparatus 15, aCPU 16, and aninterface apparatus 17. - The
input apparatus 11 comprises a keyboard and a mouse for operation by a user of the server and is used for inputting various types of operation information to the server. Thedisplay apparatus 12 comprises a display unit or the like for use by the user of the server and is used for displaying various types of information. Theinterface apparatus 17 is for connecting the server to a network and the like. - A program of the server is installed on the
memory apparatus 15 or the like before shipment, supplied to the server using arecording medium 14 such as a CD-ROM after shipment, or downloaded via a network and the like, for example. Therecording medium 14 is set in thedrive apparatus 13 and the program is installed on thememory apparatus 15 or the like from therecording medium 14 via thedrive apparatus 13. - The
CPU 16 supplies functions as will be described in the following in accordance with the program while using thememory apparatus 15 as appropriate and performs processes as shown in a flowchart as will be described in the following, for example. Thememory apparatus 15 stores image information and the like, for example. - In addition, a client and the like have the same hardware configuration as that of the server. The CPU of the client performs processes and the like as will be described in the following in accordance with a program of the client while using as appropriate the memory apparatus and the like of the client, for example.
- In the following, an example of the functional configuration of a JPIP system in a transmission method with a tile unit (JPT-stream method) according to the present invention is described with reference to
FIG. 5. FIG. 5 is a functional configuration diagram of the JPIP system in the JPT-stream method according to the present invention. - As shown in
FIG. 5 , a client side in the transmission method with a tile unit (JPT-stream method) according to the present invention includes a requestmessage generating unit 21, a responsemessage decoding unit 22, and a tilepart combining unit 23. A server side in the transmission method with a tile unit (JPT-stream method) according to the present invention includes a requestmessage decoding unit 31, atile identifying unit 32, aprecinct identifying unit 33, apacket reading unit 34, a tilepart assembling unit 35, and a responsemessage generating unit 36. - The request
message generating unit 21 generates a JPIP request message including a name of a target image, information about a view window, and the like. The requestmessage decoding unit 31 obtains the name of the target image, the information about the view window, and the like from the JPIP request message. Thetile identifying unit 32 selects a tile within an area of the requested view window on the basis of the information obtained by the requestmessage decoding unit 31. Theprecinct identifying unit 33 selects a precinct of the tile positioned within the area of the requested view window and selected by thetile identifying unit 32. - The
packet reading unit 34 collects (or reads out) packets concerning the precinct selected by theprecinct reading unit 33. The tilepart assembling unit 35 assembles (or generates) a tile part including a tile header and packet data concerning the precinct within the area of the requested view window using the packets collected (or read out) by thepacket reading unit 34 and position information about the packets. The responsemessage generating unit 36 generates a response message including the tile part assembled by the tilepart assembling unit 35. - The response
message decoding unit 22 decodes the response message received from the server. The tilepart combining unit 23 connects tile parts included in the response massage and generates a code stream. - In other words, the JPIP system in the JPT-stream method according to the present invention includes the tile part assembling unit, which is included in the client side in the JPIP system in the JPP-stream method shown in
FIG. 3 , in the server side as the tilepart assembling unit 35. By employing such a structure, interactive capability in space is improved, and it is possible to efficiently transmit encoded data within the area of the object view window to the client side and to assemble the tile part on the server side, thereby reducing the processing on the client side and the complexity of the construction of the client side. - In the following, an example of the processing on the server side is described with reference to
FIG. 6. FIG. 6 is a flowchart showing the processing in the server. In step S10, the server judges whether the JPIP request message is received from the client side. If the server judges that the JPIP request message is received from the client side (YES in step S10), the process proceeds to step S11. If the server judges that the JPIP request message is not received from the client side (NO in step S10), the process returns to step S10. - In step S11, the server decodes the received JPIP request message and performs a process for obtaining the name of the target image, the information about the view window, and the like included in the JPIP request message. Following step S11, the process proceeds to step S12 and the server performs a process for selecting a tile within the area of the requested view window on the basis of the information and the like obtained in step S11.
- Following step S12, the process proceeds to step S13 and the server performs a process for selecting a precinct of the tile positioned within the area of the requested view window and selected in step S12.
- Following step S13, the process proceeds to step S14 and the server performs a process for collecting (or reading out) packets_concerning the precinct selected in step S13. Following step S14, the process proceeds to step S15 and the server judges whether the processes in steps S13 and S14 are performed on all precincts. If the server judges that the processes in steps S13 and S14 are performed on all the precincts (YES in step S15), the process proceeds to step S16. If the server judges that the processes in steps S13 and S14 are not performed on all the precincts (NO in step S15), the process proceeds to step S13.
- In step S16, the server judges whether the processes in steps S12 through S15 are performed on all tiles. If the server judges that the processes in steps S12 through S15 are performed on all the tiles (YES in step S16), the process proceeds to step S17. If the server judges that the processes in steps S12 through S15 are not performed on all the tiles (NO in step S16), the process returns to step S12.
- In step S17, the server judges whether the server is requested on a tile basis, namely, whether the server is requested to transmit the response message including the tile part. If the server judges that the server is requested on a tile basis (YES in step S17), the process proceeds to step S18. If the server judges that the server is not requested on a tile basis (NO in step S17), the process proceeds to step S19.
- For example, if the JPIP request message received in step S10 and decoded in step S11 includes information about a request on a tile basis, the server judges that the server is requested on a tile basis. If the JPIP request message does not include information about a request on a tile basis, the server judges that the server is not requested on a tile basis.
- In step S18, the server assembles (or generates) the tile part including a tile header and packet data concerning the precincts within the area of the requested view window using the packets collected (or read out) in step S14 and position information about the packets.
- In
step 19, the server generates the response message. For example, if the process proceeds to step S19 through step S18, the server generates a response message (JPT-stream message) including the tile part, and if the process proceeds from step S17 to step S19 without step S18, the server generates a response message (JPP-stxeam message) including the packets collected (or read out) in step S14. - Following step S19, the process proceeds to step S20 and the server transmits the response message created in step S19 to the client as a requestor, for example. Following step S20, the process proceeds to step S21 and the server judges whether to end the process (process concerning the JPIP system, for example). If the server judges that the process is to be ended (YES in step S21), the process is ended. If the server judges that the process is not to be ended (NO in step S21), the process proceeds to step S10.
- By performing the process as shown in
FIG. 6 , it is possible to provide an image transmission method and the like in which the transmission efficiency of an image is improved and the complexity of constructing the client side is reduced. - The following describes the details of the process of the generation and the like of a new tile part in the tile
part assembling unit 35 and the like. In the transmission method with a precinct unit (JPP-stream method), it is not necessary to transmit unselected (or unrequested) precincts to the client, the precincts being outside the area of the view window requested from the client. However, in the transmission method with a tile unit (JPT-stream method), inJPEG 2000, although it is not necessary to send unselected tiles, information (or data) about an entire tile part is required even when the client requests only a specific portion of a tile. - Thus, the tile
part assembling unit 35 generates the tile part including a zero length packet (hereafter simply referred to as ZLP) with only one byte (or several bytes if needed) for indicating that the length of packet data concerning the precinct of the unselected area is zero, for example. In the following, an example of a tile including ZLPS in a certain subband level is shown inFIG. 7. FIG. 7 is a diagram showing the example of the tile including ZLPS in the subband level. - In the example of
FIG. 7 , the tile in the subband level has 25 precincts and the area of the view window requested from the client covers the area of the precincts from (iii, b) to (v, e). Theprecinct identifying unit 33 identifies the precincts of this area and thepacket reading unit 34 obtains packet data concerning the precincts identified by theprecinct identifying unit 33. The tilepart assembling unit 35 generates a new tile part in which the area of precincts unrequested by the client is filled with ZLPS. - When a frame size requested from the client is smaller than the size of a requested image in the server by L2 in the length and width, the tile
part assembling unit 35 does not have to send code data in L zones from the requested image size in the subband level. Accordingly, the tilepart assembling unit 35 may generate a tile part without using ZLPS for the L zones of the requested image size. In addition, the tilepart assembling unit 35 may generate the tile part filling the L zones with ZLPS. Instead of filling the precinct area unrequested by the client with ZLPS, the tilepart assembling unit 35 may generate a new tile part including information about the nonexistence of packet data concerning the precincts of the unselected area. Also, the tilepart assembling unit 35 may generate a new tile part including information about the fact that the length of packet data concerning a code block of the unselected area is zero. Further, the tilepart assembling unit 35 may generate a new tile part including information about the nonexistence of packet data concerning the code block of the unselected area. The method for constructing ZLPS may use a packet for indicating the nonexistence of packet data and a packet for indicating that packet data is not included in addition to the aforementioned methods. - The following describes another example of the generation of the tile part including ZLPS and yet another example of the generation of the tile part using precincts necessary to send the precincts of the area of the view window requested from the client without using ZLPS.
-
FIG. 8 is a diagram showing an example of a gray image divided into four tiles.FIG. 9 is a diagram showing precincts and numbers thereof when each tile shown inFIG. 8 is wavelet-transformed in three levels. - The following describes an example of an area concerning precincts to be returned to the client when the client accesses (or requests) a portion of the gray image in
FIG. 8 reduced to 1/2 with reference toFIG. 10. FIG. 10 is a first diagram showing the area concerning the precincts to be returned to the client when the client accesses (or requests) the portion of the gray image inFIG. 8 reduced to 1/2. - When the client accesses (or requests) the portion of the gray image in
FIG. 8 reduced to 1/2, preferably, the server transmits only packet data concerning the precincts of a shaded portion ofFIG. 10 , for example. However, as mentioned above, in the transmission method with a tile unit (JPT-stream method), inJPEG 2000, it is necessary to transmit information (or data) about an entire tile part even when the client requests only a specific portion of an image (the portion of the gray image reduced to 1/2 in this example). - The following describes an example of the generation of the tile part using ZLPS with reference to
FIG. 11. FIG. 11 is a diagram showing the example of the generation of the tile part using ZLPS. The tilepart assembling unit 35 generates the tile part using packet data concerning the precincts of shaded portions shown inFIG. 11 and the ZLPS of horizontal-striped portions shown inFIG. 11 , namely, 0, 2, 8, 10, 16, 18, 24, 26, 32, 34, 40, and 42 inFIG. 11 and4 ,6 ,12 ,14 , 20, 22, 28, 30, 36, 38, 44, and 46 inFIG. 11 , respectively. - The following describes another example of an area concerning precincts to be returned to the client when the client accesses (or requests) a portion of the gray image in
FIG. 8 reduced to 1/2 with reference toFIG. 12. FIG. 12 is a second diagram showing the area concerning the precincts to be returned to the client when the client accesses (or requests) the portion of the gray image inFIG. 8 reduced to 1/2. - When the client accesses (or requests) the portion of the gray image in
FIG. 8 reduced to 1/2, preferably, the server transmits only packet data concerning the precincts of a shaded portion in the left ofFIG. 12 , for example. However, as mentioned above, in the transmission method with a tile unit (JPT-stream method), in theJPEG 2000, it is necessary to transmit information (or data) about an entire tile part even when the client requests only a specific portion of an image (the portion of the gray image reduced to 1/2 in this example). Thus, the server generates the tile part as shown in the right ofFIG. 12 , using the packet data concerning the precincts included in the tile including the precincts shown in the shaded portion in the left ofFIG. 12 . - The following describes an example of the generation of the tile part using precincts included in the tile including the precincts concerning an area requested from the client with reference to
FIG. 13. FIG. 13 is a diagram showing the example of the generation of the tile part using the precincts concerning the area requested from the client, the precincts being included in the tile. - The tile
part assembling unit 35 generates the tile part using packet data concerning the precincts of striped portions shown inFIG. 13 , for example. - The following describes the capability of interactive transmission in space when a tile part is generated using ZLPS in the JPT-stream method according to the present invention with reference to
FIGS 14 and15 .FIG. 14 is an illustration showing the capability of interactive transmission in space when the tile part is generated using ZLPS in the JPT-stream method according to the present invention. -
FIG. 14-(A) is an image when the normal JPT-stream method is used andFIG. 14-(B) is an image when the JPT-stream method according to the present invention is used. Squares in the middle ofFIG. 14-(A) and (B) indicate the area of a view window and crossed lines indicate the boundary of tiles. As clearly shown inFIG. 14 , the JPT-stream method according to the present invention is capable of constructing necessary tile parts and transmitting them to the client. -
FIG. 15 is a table showing the capability of interactive transmission in space when the tile part is generated using ZLPS in the JPT-stream method according to the present invention. InFIG. 15 , the stream code size is 166582 bytes and the size of the frame is (512, 512). As shown inFIG. 15 , the JPT-stream method according to the present invention has the same capability of interactive transmission in space as that of the JPP-stream method. - The following describes the capability of progressive transmission in the JPT-stream method according to the present invention with reference to
FIGS. 16 and17 .FIG. 16 is an illustration showing the capability of progressive transmission when the tile part is generated without using ZLPS in the JPT-stream method according to the present invention. - As shown in
FIG. 16 , progressive transmission can be readily performed also in the JPT-stream method according to the present invention. -
FIG. 17 is a table showing the capability of progressive transmission when the tile part is generated without using ZLPS in the JPT-stream method according to the present invention. InFIG. 17 , the stream code size is 164962 bytes and the volume of transmission bytes in the normal JPT-stream is 168626 bytes. As shown inFIG. 17 , the JPT-stream method according to the present invention is capable of transmitting an image with improved resolution in comparison with the normal the JPP-stream method. - In the aforementioned embodiment, the tile
part assembling unit 35 generates the tile part including a tile header and packet data concerning the precincts within the area of the requested view window using the packets and position information about the packets. However, the tilepart assembling unit 35 may generate a tile part in each subband level using packet data concerning the subband level of the requested view window, the tile part including the packet data. - Further, the tile
part assembling unit 35 may generate a tile part in each color component using precincts concerning the color component or packet data concerning a subband level, the tile part including the packet data. - As mentioned above, according to the present invention, it is possible to provide an image transmission method, a computer-readable image transmission program, a recording medium, and an image transmission apparatus in which the transmission efficiency of an image is improved and the complexity of the construction of the client side is reduced. In this case, the image transmission apparatus corresponds to the aforementioned server or the like, for example.
- The present invention is not limited to the specifically disclosed embodiment, and variations and modifications may be made without departing from the scope of the present invention as defined by the appended claims.
Claims (11)
- An image processing system for processing images that are compressed by a method in accordance with JPEG 2000 and are divided into tiles and the tiles further divided into precincts, the system comprising:a client: anda server (1) connected to the client via a communication path, wherein the client includes:a request message generating unit (21) for generating a request message having size and/or area information of an image, andthe server includes:a request message decoding unit (31) for obtaining the size and/or area information of the image based on the request message;a tile identifying unit (32) for selecting at least one tile of a rectangular area of the image based on the obtained size and/or area information;a precinct identifying unit (33) for selecting at least one precinct of a selected tile;a reading unit (34) for reading packets corresponding to the at least one selected precinct;a tile part generating unit (35) for generating at least one tile part each having a tile header and packets corresponding to the at least one selected precinct based on the read packets and positional information on the packets; anda response message generating unit (36) for generating a response message having the generated tile part.
- An image processing system according to claim 1, wherein the client further includes:a response message decoding unit (22) for decoding the response message received from the server; anda tile part assembling unit (23) for connecting the decoded tile parts included in the response message generating a code stream.
- An image processing system according to claim 1 or 2, wherein the tile part generating unit (35) is arranged to include a zero length packet indicating zero data length for packets relating to a precinct of an area not selected in the precinct identifying unit.
- An image processing system according to claim 1, 2 or 3, wherein said image is divided into levels, each level being divided into tiles, and the tile identifying unit (32), precinct identifying unit (33) reading unit (34) and tile part generating unit (35) are arranged to operate on each level.
- An image processing system according to claim 1, 2, 3 or 4, wherein said image is divided into colour components, each colour component being divided into tiles, and the tile identifying unit (32), precinct identifying unit (33), reading unit (34) and tile part generating unit (35) are arranged to operate on each colour component.
- A method of processing images that are compressed by a method in accordance with JPEG 2000 and are divided into tiles and the tiles further divided into precincts, the method comprising:receiving at a server a request message from a client having size and/or area information of an image;decoding the request message to obtain the size and/or area information;selecting at least one tile of a rectangular area of the image based on the obtained size and/or area information;selecting at least one precinct of the selected tile;reading packets corresponding to the at least one selected precinct;generating at least one tile part each having a tile header and packets corresponding to the at least one selected precinct based on the read packets and positional information on the packets; andgenerating a response message having the generated tile part.
- A method according to claim 6, further comprising:decoding the response message at the client;connecting decoded tile parts included in the response message; andgenerating a code stream.
- A method according to claims 6 or 7, wherein the step of generating a tile part includes generating a zero length packet indicating a zero data length for packets relating to a precinct of an area not selected in the precinct selecting step.
- A method according to claims 6, 7 or 8 wherein said image is divided into levels, each level being divided into tiles, and the steps of selecting at least one tile, selecting at least one precinct, reading packets and generating a tile part are repeated for each level.
- A method according to claims 6, 7, 8 and 9 wherein said image is divided into colour components, each colour component being divided into tiles, and the steps of selecting at least one tile, selecting at least one precinct, reading packets and generating a tile part are repeated for each colour component.
- A computer program comprising program code means that, when executed by a computer system, instruct the computer system to perform a method according to any one of claims 6 to 10.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2005111037A JP4738869B2 (en) | 2005-04-07 | 2005-04-07 | Image transmission method, image transmission program, recording medium, and image transmission apparatus |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1710719A1 EP1710719A1 (en) | 2006-10-11 |
EP1710719B1 true EP1710719B1 (en) | 2010-05-05 |
Family
ID=36685613
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP20060251953 Not-in-force EP1710719B1 (en) | 2005-04-07 | 2006-04-06 | Image transmission method, computer-readable image transmission program, and image transmission apparatus |
Country Status (4)
Country | Link |
---|---|
US (1) | US7456844B2 (en) |
EP (1) | EP1710719B1 (en) |
JP (1) | JP4738869B2 (en) |
DE (1) | DE602006014046D1 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5326234B2 (en) * | 2007-07-13 | 2013-10-30 | ソニー株式会社 | Image transmitting apparatus, image transmitting method, and image transmitting system |
JP5194776B2 (en) * | 2007-12-21 | 2013-05-08 | 株式会社リコー | Information display system, information display method and program |
JP4918026B2 (en) * | 2007-12-21 | 2012-04-18 | 株式会社リコー | Image coding apparatus, image coding method, computer program, and information recording medium |
US9258530B2 (en) | 2010-04-09 | 2016-02-09 | Canon Kabushiki Kaisha | Method for accessing a spatio-temporal part of a compressed video sequence using decomposed access request |
FR2959636B1 (en) * | 2010-04-28 | 2012-07-13 | Canon Kk | METHOD FOR ACCESSING A SPATIO-TEMPORAL PART OF A VIDEO SEQUENCE OF IMAGES |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3706201B2 (en) * | 1996-07-18 | 2005-10-12 | 富士写真フイルム株式会社 | Image processing method |
US6674436B1 (en) * | 1999-02-01 | 2004-01-06 | Microsoft Corporation | Methods and apparatus for improving the quality of displayed images through the use of display device and display condition information |
US6891888B2 (en) * | 2000-03-23 | 2005-05-10 | Fuji Photo Film Co., Ltd. | Image processing method and image processing apparatus |
JP2002330951A (en) * | 2001-05-11 | 2002-11-19 | Canon Inc | Image encoding/decoding device and method, computer program and storage medium |
JP3768866B2 (en) * | 2001-11-29 | 2006-04-19 | キヤノン株式会社 | Method and apparatus for creating encoded data |
JP4125087B2 (en) * | 2002-01-31 | 2008-07-23 | キヤノン株式会社 | Image processing method |
JP4086728B2 (en) | 2002-07-03 | 2008-05-14 | 株式会社リコー | Image communication method and system |
JP2004040674A (en) * | 2002-07-08 | 2004-02-05 | Nec Engineering Ltd | Restoration method for jpeg2000 code data and restoration system for jpeg2000 code data |
JP3958198B2 (en) * | 2002-12-09 | 2007-08-15 | キヤノン株式会社 | Image processing method |
JP2004214828A (en) * | 2002-12-27 | 2004-07-29 | Ricoh Co Ltd | Image processing apparatus, image processing method, computer program for image processing, and storage medium for storing the image processing computer program |
JP4148462B2 (en) | 2003-01-20 | 2008-09-10 | 株式会社リコー | Image processing apparatus, electronic camera apparatus, and image processing method |
US7460724B2 (en) | 2003-03-07 | 2008-12-02 | Ricoh Co., Ltd. | JPP-stream to JPEG 2000 codestream conversion |
JP4143471B2 (en) | 2003-05-21 | 2008-09-03 | キヤノン株式会社 | Image encoding method, image encoding apparatus, and recording apparatus |
US7382922B2 (en) | 2003-07-11 | 2008-06-03 | Canon Kabushiki Kaisha | Creation of remarkable regions in a digital image |
-
2005
- 2005-04-07 JP JP2005111037A patent/JP4738869B2/en not_active Expired - Fee Related
-
2006
- 2006-04-06 DE DE200660014046 patent/DE602006014046D1/en active Active
- 2006-04-06 EP EP20060251953 patent/EP1710719B1/en not_active Not-in-force
- 2006-04-06 US US11/398,677 patent/US7456844B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
US20060228033A1 (en) | 2006-10-12 |
JP2006295386A (en) | 2006-10-26 |
DE602006014046D1 (en) | 2010-06-17 |
EP1710719A1 (en) | 2006-10-11 |
US7456844B2 (en) | 2008-11-25 |
JP4738869B2 (en) | 2011-08-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4538430B2 (en) | System and method in server client environment | |
US7206804B1 (en) | Methods and systems for transmitting digital images | |
EP1335561B1 (en) | Method for document viewing | |
JP4709493B2 (en) | Method and product for communicating compressed digital images | |
AU766874B2 (en) | Partial retrieval of images in the compressed domain | |
US7349585B2 (en) | Image processing system that internally transmits lowest-resolution image suitable for image processing | |
US7660476B2 (en) | Image processing method and image processing apparatus | |
EP1710719B1 (en) | Image transmission method, computer-readable image transmission program, and image transmission apparatus | |
JP2004274758A (en) | Method and apparatus for converting jpp-stream into jpeg-2000 code stream | |
US8577157B2 (en) | Conditional replenishment for motion JPEG2000 | |
KR20020087482A (en) | Object transfer method with format adaptation | |
JP2008244539A (en) | Client-server system | |
JP2009017527A (en) | Server-driven progressive image transmission and system | |
US7721971B2 (en) | Image processing system, image processing method and computer readable information recording medium | |
EP1341328A1 (en) | Data broadcast system, for broadcasting multimedia data; receiving terminal device for receiving said multimedia data | |
JP2006135792A (en) | Image processing apparatus, its control method, computer program, and computer readable storage medium | |
Hara | An implementation of JPEG 2000 interactive image communication system | |
US20080019599A1 (en) | System and method for compressing and scaling images | |
Lafruit et al. | View-dependent, scalable texture streaming in 3-D QoS with MPEG-4 visual texture coding | |
EP0959432A2 (en) | Image processing apparatus and method | |
JP4874170B2 (en) | Image processing apparatus and image processing method | |
JP2008147893A (en) | Client/server system and remote operation system | |
JP2006339972A (en) | Image processing apparatus and its control method, computer program, and storage medium | |
JP5187094B2 (en) | Image encoding device | |
JP2007142581A (en) | Image processing method and image processing unit |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20060508 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR MK YU |
|
17Q | First examination report despatched |
Effective date: 20070117 |
|
AKX | Designation fees paid |
Designated state(s): DE FR GB NL |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RTI1 | Title (correction) |
Free format text: IMAGE TRANSMISSION METHOD, COMPUTER-READABLE IMAGE TRANSMISSION PROGRAM, AND IMAGE TRANSMISSION APPARATUS |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): DE FR GB NL |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REF | Corresponds to: |
Ref document number: 602006014046 Country of ref document: DE Date of ref document: 20100617 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20100505 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20100505 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20110208 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602006014046 Country of ref document: DE Effective date: 20110207 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 11 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 12 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20180321 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20180420 Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20180420 Year of fee payment: 13 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602006014046 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G06F0017300000 Ipc: G06F0016000000 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 602006014046 Country of ref document: DE |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20190406 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191101 Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190406 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190430 |