WO2013018248A1 - Image transmission device, image transmission method, image receiving device, and image receiving method - Google Patents
Image transmission device, image transmission method, image receiving device, and image receiving method Download PDFInfo
- Publication number
- WO2013018248A1 WO2013018248A1 PCT/JP2012/002216 JP2012002216W WO2013018248A1 WO 2013018248 A1 WO2013018248 A1 WO 2013018248A1 JP 2012002216 W JP2012002216 W JP 2012002216W WO 2013018248 A1 WO2013018248 A1 WO 2013018248A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image data
- unit
- compression
- image
- compressed
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
Definitions
- the technical field relates to transmission and reception of video information.
- VESA DisplayPort
- HDMI High-Definition Multimedia Interface
- VESA Video Electronics Standards Association
- Patent Document 1 describes that “a non-compressed video signal or a compressed video signal obtained by compressing a non-compressed video signal with a compression system that can be supported by a receiving device”. For selective transmission, a video signal having a desired bit rate can be satisfactorily transmitted within the transmission bit rate of the transmission path ”(see Patent Document 1 [0048]), and for the compression method,“ data compression ”
- the units 121-1 to 121-n respectively compress the uncompressed video signal output from the codec 117 with a predetermined compression ratio, and output the compressed video signal.
- -n constitutes a video signal compression unit, and each of the data compression units 121-1 to 121-n performs a data compression process using a different compression method.
- the image data adopts a TMDS (Transition Minimized Differential Signaling (registered trademark of Silicon Image, Inc.)) system, and Patent Document 2 is shown as an example.
- TMDS Transition Minimized Differential Signaling (registered trademark of Silicon Image, Inc.)
- a color difference signal can be obtained by coordinate conversion using correlation with a luminance signal.
- a color difference signal processing unit that narrows the range; a compression processing unit that compresses image data including luminance color difference signals; and a data transfer unit that outputs image data compressed by the compression processing unit, the data transfer unit including: The compressed code data relating to the processing performed in the color difference signal processing unit and the compression processing unit is output together with the compressed image data.
- FIG. 1 illustrates an example of an image transmission device and an image reception device according to a first embodiment.
- An example of the compression process part of 1st Example An example of a general YCb space.
- An example of the error correction code generation part of 1st Example An example of the data transmission part of 1st Example.
- An example of the header of the compression code information packet of 1st Example An example of the data of the compression code information packet of 1st Example.
- An example of the header of the compression code information packet of 1st Example An example of the data of the compression code information packet of 1st Example.
- extension process part of 1st Example. An example of the timing of the expansion process of the first embodiment.
- a Cb or Cr color difference signal is obtained using a correlation with a Y luminance signal before the image data transmitted by the image transmission apparatus is compressed and transmitted.
- Pre-processing that improves the compression efficiency of image data by limiting the effective range of the Cb or Cr color difference signal by coordinate conversion, and compression based on the transmission clock synchronized with the pixel clock signal assumed by the original uncompressed image data
- the image data is transmitted, and the image receiving apparatus performs post processing for decompressing the compressed image data and restoring the Cb or Cr color difference signal limited by using the correlation with the Y luminance signal by coordinate transformation.
- FIG. 1 is a block diagram illustrating an image transmission system according to the present embodiment, in which an image transmission apparatus 100 and an image reception apparatus 200 are connected by a cable 300.
- the image transmission apparatus 100 is an image transmission apparatus that transmits image data.
- Examples of the image transmission device 100 include a recorder, a digital TV with a built-in recorder function, a personal computer with a built-in recorder function, a camcorder, and a mobile phone with a camera function.
- the image receiving apparatus 200 is a display device that inputs image data and outputs an image to a monitor using an HDMI cable or the like.
- Examples of the image receiving apparatus 200 include a digital TV, a display, and a projector.
- the cable 300 is a data transmission path for performing data communication such as image data between the devices of the image transmission device 100 and the image reception device 200.
- data communication such as image data between the devices of the image transmission device 100 and the image reception device 200.
- the cable 300 there is a wired cable corresponding to the HDMI standard or the DisplayPort standard, or a data transmission path for performing wireless data communication.
- the input units 101, 102, and 103 are input units for inputting image data to the image transmission apparatus 100.
- An example of image data input to the input unit 101 is digital broadcasting input as radio waves from a relay station such as a broadcasting station or a broadcasting satellite.
- the input unit 101 receives radio waves from a relay station such as this broadcasting station or broadcasting satellite.
- Examples of image data input to the input unit 102 include digital broadcasts distributed via a network and information content using an Internet broadband connection.
- image data input to the input unit 103 content recorded on an external recording medium connected to the input unit 103, or recorded on a recording medium 108 built in the image transmission apparatus 100.
- the external recording medium connected to the input unit 103 or the recording medium 108 built in the image transmission apparatus 100 include an optical disk, a magnetic disk, and a semiconductor memory.
- the tuner reception processing unit 105 is a reception processing unit that converts an input radio wave into a bit stream.
- the radio wave of the RF band (Radio Frequency) is frequency-converted to an IF band (Intermediate Frequency) and does not depend on the reception channel.
- a modulation operation applied to the demodulated bit stream for transmission as a fixed band signal is demodulated.
- bitstreams include an MPEG2 transport stream (hereinafter referred to as MPEG2-TS), a bitstream having a format conforming to MPEG2-TS, and the like.
- MPEG2-TS MPEG2 transport stream
- bitstream having a format conforming to MPEG2-TS bitstream having a format conforming to MPEG2-TS
- the following bit stream will be described with MPEG2-TS as a representative.
- the tuner reception processing unit 105 further detects and corrects a code error occurring during transmission, and after the descrambling of the error-corrected MPEG2-TS, a program for viewing or recording is multiplexed. One transponder frequency is selected, and the bit stream in the selected one transponder is separated into audio and video packets of one program.
- the MPEG2-TS from the tuner reception processing unit 105 is supplied to the stream control unit 111 through the data bus 181.
- the stream control unit 111 performs PTS (Presentation Time Stamp) that is time management information from the received packet, and MPEG system reference decoding
- PTS Presentation Time Stamp
- STC System Time Clock
- the STC System Time Clock
- the packet with the time stamp added is supplied to either one or both of the decoder 112 and the recording media control unit 107.
- the data path 184 to the decoder 112 is used for processing when viewing image data
- the data path 183 to the recording medium control unit 107 is used when recording image data on a recording medium.
- the MPEG2-TS input from the input unit 102 via the network reception processing unit 106 is input to the data bus 182 of the stream control unit 111.
- the data path 182 is an input unit that acquires digital broadcast or digital content distributed via a network.
- the stream control unit 111 selects at least one of these inputs and outputs it to the decoder 112.
- the decoder 112 decodes the MPEG2-TS input from the stream control unit 111 and outputs the generated image data to the display processing unit 113.
- the display processing unit 113 performs, for example, OSD (On Screen Display) superimposition processing or enlargement / reduction processing on the input image data, and then outputs the processed image data to the compression processing unit 114.
- OSD On Screen Display
- the compression processing unit 114 performs compression processing on the image data from the display processing unit 113 and outputs it to the data transmission unit 115.
- the data transmission unit 115 converts the image data into a signal in a format suitable for transmission and outputs it from the output unit 116.
- a signal in a format suitable for transmission a format suitable for transmission by cable is described in the HDMI standard.
- image data adopts a TMDS data transmission format.
- the input unit 104 is an input unit for inputting a signal for controlling the operation of the image transmission apparatus 100.
- An example of the input unit 104 is a remote control receiver.
- a control signal from the input unit 104 is supplied to the user IF 109.
- the user IF 109 outputs a signal from the input unit 104 to the control unit 110.
- the control unit 110 controls the entire image transmission apparatus 100 according to the signal from the input unit 104.
- An example of the control unit 110 is a microprocessor. Image data from the image transmission apparatus 100 is supplied to the image reception apparatus 200 via the cable 300.
- the input unit 201 receives a signal in a format suitable for transmission.
- the signal input to the input unit 201 is supplied to the data reception processing unit 205.
- the data reception processing unit 205 performs processing for converting a signal in a format suitable for transmission into predetermined digital data, and outputs the converted digital data to the expansion processing unit 206.
- the decompression processing unit 206 decompresses the image data compressed by the compression processing unit 114 in the image transmission apparatus 100 and outputs it to the display processing unit 207.
- the display processing unit 207 performs display processing on the input image data. Examples of display processing include OSD superimposition processing, enlargement / reduction processing for conversion to the resolution of the display unit 208, frame rate conversion processing, and the like.
- the output of the display processing unit 207 is output to the display unit 208.
- the display unit 208 converts the input image data into a signal suitable for the display method and displays it on the screen.
- Examples of the display unit 208 include a display unit such as a liquid crystal display, a plasma display, and an organic EL (Electro-Luminescence) display.
- the input unit 202 is an input unit for inputting a signal for controlling the operation of the image receiving apparatus 200.
- An example of the input unit 202 is a remote control receiver.
- a control signal from the input unit 202 is supplied to the user IF 203.
- the user IF 203 outputs a signal from the input unit 202 to the control unit 204.
- the control unit 204 is a control unit that controls the entire image receiving apparatus 200 in accordance with a signal from the input unit 202.
- FIG. 2 is a block diagram illustrating an example of the configuration of the compression processing unit 114.
- the input unit 130 is an input unit for inputting image data to the compression processing unit 114.
- the input image data is supplied to the YCbCr processing unit 131. If the input image data is an RGB primary color signal, the YCbCr processing circuit converts it into a YCbCr luminance color difference signal using a predetermined conversion formula based on colorimetric information added to the image data.
- format information such as the horizontal and vertical frequencies of the image signal is used, for example, ITU-R BT709 (hereinafter abbreviated as BT709) for HD images and ITU-R BT601 (hereinafter abbreviated as SD video).
- BT709 ITU-R BT709
- SD video ITU-R BT601
- Conversion processing is performed based on the prescribed colorimetry and conversion formula. If the input image data is a YCbCr luminance color difference signal, this conversion processing is not necessary.
- FIG. 3 is an explanatory diagram of the YCb space defined by BT709 applied to HD images.
- the values of Y and Cb indicate both the normalized value and the 8-bit quantized value, and these definitions are described in detail in the BT709 standard.
- the range 702 in which the normalization Y is 0 to 1 and the normalization Cb is -0.5 to +0.5 is used.
- the area 703 in which the normalization Y exceeds 1 is called Over White, and the area less than 0 is called Over Black. Yes.
- a parallelogram-shaped region 707 surrounded by points 708 and 709, 710, and 711 indicates a region in which the values of the normalized R, G, and B are 0 to 1 when converted into RGB primary color signals. .
- an area in which any one of the normalizations R, G, and B is negative is an area that is not normally used.
- the xvYCC standard defined in IEC 61996 2-4 which expands the color expression range by using the outside area 707, the area 705, and the area 706 that are not normally used, is also known.
- FIG. 4 shows a parallelogram-shaped area 717 obtained by converting the parallelogram-shaped area 707 by converting the left and right sides in parallel with the Y-axis.
- the arrows in the figure indicate movements by converting each coordinate.
- Normalization (Cb, Y) (0, 0.5) is taken as the center where the coordinates do not move by coordinate transformation.
- Y is not converted and only the Cb value is moved.
- By not converting Y there is also an advantage that the same Y can be used in common even in the case of performing Cr range reduction by similar coordinate conversion in the YCr space.
- the normalized Cb range 714 of the parallelogram-shaped region 717 is -0.2695 to 0.2695, and the normalized Cb range of the parallelogram-shaped region 707 before conversion is -0.5 to 0. Compared to 5, it can be reduced to about 54%.
- the subsequent compression encoding can be efficiently performed.
- the area defined by the xvYCC standard is not all, but the area around the parallelogram area 707 is maintained around the parallelogram area 717 after coordinate conversion. Therefore, transmission of the Cb area defined by the xvYCC standard is possible.
- FIG. 5 shows an example of coordinate conversion in the YCb space in which the coordinate conversion calculation at the time of 8-bit quantization is further simplified.
- the amount of movement of the quantized Cb is (quantized Y ⁇ 128) / 2, and the amount of computation in binary numbers is particularly reduced. Therefore, the quantized Cb after the coordinate conversion is (quantized Cb + (quantized Y ⁇ 128) / 2), and the decimal part is rounded down.
- the normalized Cb range of the parallelogram-shaped area 727 after the coordinate conversion is in the range of ⁇ 0.2861 to 0.2861, and the reduction effect is slightly reduced as compared with the embodiment in FIG. 4, but it can still be reduced to about 57%.
- the amount of movement of the quantized Cb may be (quantized Y-2048) / 2.
- the advantage of the simplified calculation method shown in FIG. 5 is not only the reduction of the coordinate conversion calculation amount but also the reversible calculation that can be completely restored in the restoration calculation to the original coordinates after the coordinate conversion because the movement amount is an integer. That is. If the movement amount is defined as an integer, the advantage of reversible operation can be realized. That is, not only the movement amount (quantization Y ⁇ 128) / 2 but also a mathematical expression such as (quantization Y ⁇ 128) ⁇ 5/8 may be processed as an integer.
- FIG. 6 shows the vertex coordinates of a parallelogram-shaped region in which the normalized RGB primary color signal is in the range of 0 to 1 before and after coordinate conversion to reduce the range of Cb and Cr in the BT709 standard YCb space and YCr space, respectively. It is shown.
- the coordinate conversion results obtained by the minimum Cb range method described in FIG. 4 and the simple processing method described in FIG. 5 are compared.
- the first line is normalized
- the second line is 8-bit quantization
- the third line is 12-bit quantization
- the fourth line is 12-bit quantization
- the fifth line is 16-bit quantized coordinates. From these conversion results, it can be seen that Cr can be compressed to a minimum Cr range of about 64% and a simple processing Cr range of about 72%.
- the output of the YCbCr processing unit 131 whose operation has been described above is supplied to the thinning processing unit 132, the horizontal compression unit 133, and the vertical compression unit 134.
- FIG. 7 and 8 are diagrams illustrating an example of image data input to the input unit 130.
- FIG. A luminance signal of n pixels in the horizontal direction and m lines in the vertical direction is shown.
- the color difference signal has the same format as the luminance signal.
- the unit block of the image data compressed by the compression unit 114 is assumed to be 1 pixel in the horizontal direction and k pixels in the vertical direction.
- the horizontal compression unit 133 compresses this image data unit.
- the vertical compression unit 134 compresses the unit block of the image data. That is, k1 and k2 (k1 ⁇ k2) image data unit blocks having different numbers of k pixels (lines) in the vertical direction are prepared, the horizontal compression unit 133 sets the image data unit block of k1, and the vertical compression unit 134 sets k2. Compress the image data unit block.
- a compression unit having a different k3 can be added, and the output of three or more compression units can be selected and used to further increase the compression efficiency.
- the Cb component and Cr component of the color difference signal become nested data for each pixel.
- the Cb component and Cr component of the color difference signal are data for one pixel with respect to four Y signal pixels.
- the horizontal compression unit 133 and the vertical compression unit 134 reduce the compression efficiency to the 422 format or 420 format if the predetermined compression rate does not reach the predetermined compression rate. May be increased.
- the 420 processing unit 132 When the 444 format or the 422 format is input, the 420 processing unit 132 performs a thinning process to the 422 format or the 420 format according to a predetermined compression rate.
- the horizontal compression unit 133 includes a compression circuit that compresses a plurality of image data in the horizontal direction.
- Wavelet transform is calculated in the horizontal direction, and the calculation result is configured by an encoded compression method or the like.
- a compression method Hadamard transform, run length coding, Huffman coding, differential coding, or the like may be applied.
- the vertical compression unit 134 includes a compression circuit that compresses a plurality of pieces of image data in the vertical direction.
- a compression method as a unit block of image data for compressing image data of 2 lines in the vertical direction and 16 pixels in the horizontal direction, a difference is first taken in the vertical direction, and then a difference is taken in the horizontal direction. .
- the result is configured by a compression method for encoding the result.
- the compression rate of the thinning processing unit 132, the horizontal compression unit 133, and the vertical compression unit 134 is, for example, 2/3, 1/2, or 1/3 of the compressed image data with respect to the original uncompressed image data. Any compression method that can compress at a predetermined compression rate may be used (however, the present invention is not limited to this compression rate). This predetermined compression rate is determined from the ratio of the original uncompressed image data and the amount of data that can be transmitted by the cable.
- the unit of image data to be compressed by the horizontal compression unit 133 and the vertical compression unit 134 is composed of the number of pixels that reduces the delay amount due to the compression processing.
- 32 pixels have been described as an example.
- a unit of 64 pixels or 128 pixels may be used.
- the selection unit 135 selects the output of the thinning processing unit 132, the horizontal compression unit 133, or the vertical compression unit 134 that satisfies a predetermined compression ratio and has a high image quality index, and supplies the selected one to the error correction code generation unit 136.
- the image quality index may be an index that indicates a better value as the difference between the image data restored from the compressed image data and the image data before compression is smaller, for example. The highest value is when lossless compression does not occur and lossless encoding is possible.
- an image quality index value may be prepared for each compression method.
- a case where lossless encoding can be performed after thinning out to 422 format may be defined as a higher image quality index than when compression loss occurs when compression is performed in the 444 format.
- a predetermined compression ratio can be achieved by thinning out to 422 format or 420 format and reducing the number of quantization bits. Set the appropriate image quality index.
- the error correction code generation unit 136 calculates an error correction code for each unit of image data compressed by the horizontal compression unit 133 or the vertical compression unit 134, adds the error correction code to the compressed image data, and a compression code information addition unit 137. Output to.
- One of error correction methods is a CRC (Cyclic Redundancy Check) method or a parity check method.
- the image data thinned out by the thinning-out processing unit 132 may not be given an error correction code.
- thinning processing even if an error occurs, it is an error of the thinning pixel adjacent to the corresponding pixel, there is little influence on the image quality when viewed from the whole screen, and furthermore, when the data transmission capacity of the cable is limited, This is because when an error correction code is added, more pixels or gradations are thinned out, leading to image quality deterioration of the entire screen. If the transmission capacity is sufficient, an error correction code may be added to the thinned-out image data to improve error tolerance.
- Whether or not error correction processing is to be performed may be determined on the reception side by transmitting together with metadata indicating whether or not error correction is added to the thinned image data. Further, the error resistance level may be changed by changing the error correction processing between the thinned image data and the compressed image data, and information indicating which error correction code is given may be added as metadata.
- the compression code information adding unit 137 outputs the compressed image data with the error correction code in the effective period 406 of FIG. 11 described later, and outputs the compression code information indicating the compression method in the immediately preceding horizontal blanking period 404.
- compressed image data with an error correction code for one line and compressed code information indicating the compression method are transmitted within an effective period 406 for one line to increase the transmission amount. Also good.
- the error correction code may be output in the horizontal blanking period 404 to improve the error correction reliability.
- the output unit 138 outputs the compressed image data with error correction code from the compression code information adding unit 137 and the compression code information indicating the compression method. Although not shown, the operation of each block in FIG. 2 is controlled according to the control signal of the control unit 110 in FIG.
- FIG. 9 is a block diagram illustrating an example of the configuration of the error correction code generation unit 136.
- the input unit 150 receives compressed image data.
- the compressed image data is input to the holding unit 155 and the error correction code calculation unit 153.
- the error correction code calculation unit 153 performs a cyclic calculation with the generator polynomial on the input compressed image data.
- An example of a generator polynomial is
- the input unit 151 receives a signal indicating a period during which the compressed image data is input, and supplies the signal to the timing generation unit 154.
- the timing generation unit 154 counts the effective period of the compressed image data, and outputs a signal indicating that the calculation for the unit block of the image data to be compressed has been processed to the data holding unit 155 as an error correction calculation result output timing signal. Output.
- the timing generation circuit 154 also outputs a timing signal indicating the input period of the compressed image data, a timing signal for outputting the compressed image data and the error correction code calculation result, and the like to the data holding unit 155.
- the data holding unit 155 temporarily stores the calculation result of the error correction code calculation unit 153 and the compressed image data by, for example, a memory, flip-flop, delay element, or the like according to the timing indicated by the timing generation unit 154, and sequentially stores them. To the output unit 157.
- FIG. 10 is a block diagram illustrating an example of the configuration of the data transmission unit 115.
- the input unit 170 outputs the compressed image data to the serializer 174.
- the input unit 172 receives a clock of image data and outputs the clock to the PLL 173 and the output unit 177.
- As the clock of the image data a clock synchronized with the pixel clock used in the standard timing format of the uncompressed image data is used.
- a clock obtained by dividing the pixel clock of uncompressed image data by two may be used.
- the uncompressed image data is 12-bit quantized image data
- the clock is 1/2 and the number of quantization bits is 8/12, so the predetermined compression rate is set to 1/3 or less. There is a need to.
- the clock may be multiplied by 3/4 or 2/3 in addition to dividing by 2.
- a clock synchronized with the pixel clock of the uncompressed image data for transmission of the compressed image data when restoring the uncompressed image data on the receiving side, it is 2 times, 4/3 times, 3/2 times the transmission clock.
- the multiplied clock By using the multiplied clock as the pixel clock, there is an advantage that the jitter of the restored data can be minimized.
- the PLL 173 generates a clock obtained by multiplying or dividing the input clock. Examples of multiplication include 5 times and 10 times the frequency of the input clock.
- the clock generated by the PLL 173 may be one type of clock or two types of clocks. An example of one type of clock is 10 times the input clock. Examples of the two types of clocks include a first clock speed that prioritizes the amount of data transmission and a second clock speed that is slower than the first clock speed that prioritizes lowering the frequency of error occurrence. is there. As an example of the speed, the first clock speed is multiplied by 10 of the input clock, and the second clock speed is multiplied by 5 of the input clock.
- the multiplied clock generated by the PLL 173 is output to the serializer 174.
- the serializer 174 serializes the compressed image data of the input YCbCr luminance color difference signal bit by bit with a clock multiplied by 10 and outputs the serialized data to the level conversion unit 175.
- TMDS transmission method that suppresses the DC component of the bit stream that is serialized by mapping the 8-bit data to 10 bits when there is 8 bits of data input to the input unit 170 for one clock input to the input unit 172 May be used.
- 24 bits of compressed image data per input clock can be sent by performing the serial processing for each cable.
- the level conversion unit 175 outputs a signal in a format suitable for cable transmission via the output unit 176.
- FIG. 11 is a diagram illustrating an effective region in which image data of one frame period is superimposed and a blanking period in which image data is not superimposed.
- a region indicated by 400 indicates a vertical period, and the vertical period 400 includes a vertical blanking period 401 and a vertical effective period 402.
- the VSYNC signal is a 1-bit signal in which 1 is set between the number of lines defined from the top of the vertical blanking period 401 and 0 is set between the other vertical blanking periods and the vertical effective period 402.
- An example of the prescribed number of lines is 4 lines.
- the HSYNC signal is a 1-bit signal in which 1 is set between the number of pixels defined from the head of the horizontal blanking period 404 and 0 is set between the other horizontal blanking periods and the horizontal effective period 405.
- An example of the prescribed number of pixels is 40 pixels.
- the effective period 406 indicates an area surrounded by a vertical effective period 402 and a horizontal effective period 405, and image data is allocated to this period.
- the blanking period 407 is an area surrounded by a vertical blanking period 401 and a horizontal blanking period 404.
- information indicating the compression transmission method is transmitted in the vertical blanking period, compressed image data is transmitted in the effective period 406, and each compression encoding is performed in the horizontal blanking period 404 of the line.
- the compression code information selected for each image data unit block is transmitted.
- the blanking period 407 data obtained by packetizing audio data and other attached data is transmitted.
- a method for sending a packet of voice data or the like in the blanking period 407 is disclosed in, for example, Japanese translations of PCT publication No. 2005-514873.
- the error correction code is included in the packet data in the blanking period, it is possible to correct an error occurring in the transmission path, and the error resistance is enhanced.
- the data for data transmission of the packet in the blanking period is configured to be transmitted to two physically different channels, and the channel to be transmitted is switched every certain time. Since the other channel is not affected by the error, the data error can be corrected.
- the error correction rate has an improvement effect of 10 -14 in the horizontal blanking period compared to 10 -9 in the horizontal effective period.
- a very high clock frequency of 891 MHz is required.
- the high clock frequency not only increases the cost of the transmission / reception unit, but also shortens the cable length capable of stably sending an image, resulting in poor usability.
- horizontal 3840 effective pixels and vertical 2160 effective lines for example, horizontal 560 blanking pixels and vertical 90 blanking lines.
- a horizontal 3840 effective pixel, a vertical 2160 effective line, a YCbCr luminance / chrominance signal 12 bits each in 444 format, and a video signal having a frame frequency of 60 Hz is compressed to 1/3 will be described.
- the compressed clock frequency is 297 MHz, and if the transmission is performed under the conditions of horizontal 1940 effective pixels, vertical 2160 effective lines, horizontal 280 blanking pixels, and vertical 90 blanking lines, the original clock 594 MHz, which is twice the frequency, is stabilized on the receiving side. Easy to play.
- the horizontal effective pixel period by compressing each 12-bit YCbCr luminance color difference 12 bits total 36 bits to 1/3 of 12 bits, it is possible to transmit 24 bits per compressed pixel, so transmission of 2 pixels of the original pixel is possible.
- the later clock frequency is half that of the original clock.
- each compression code system information of YCbCr luminance color difference signal for 120 blocks since it is a horizontal 3840 effective pixel, each compression code system information of YCbCr luminance color difference signal for 120 blocks, a total of 360 description spaces are required. If the compressed code information is expressed in 2 bits, for example, it becomes a description space of 90 bytes and can be transmitted in 4 packets, so that it can be compatible with large-capacity audio data transmission of 192 kHz and 8 ch. An example of a packet is shown in FIGS.
- FIG. 12 and FIG. 14 are examples of packet headers, and a common header type 0Bh is described in the first header block HB0, which indicates information related to the compression-encoding transmission system of the present invention.
- Each bit of HB1 and Bits 4 to 7 of HB2 are set to 0 for future expansion.
- Eco_Packet # shown in Bits 0 to 3 of HB2 indicates identification within the frame.
- FIGS. 13 and 15 are examples of 28-byte data transmitted following the header.
- Eco_Packet # is assigned 0h, indicating that the packet composed of the header of FIG. 12 and the data of FIG. 13 is common information in each frame.
- This packet is arranged during the vertical blanking period and is transmitted at least once for each image frame.
- the contents of the data in FIG. 13 will be described.
- Color_Sample indicates color sample information, for example, 0 is YCbCr444 format, 1 is YCbCr422 format, 2 is YCbCr420 format, 3 is RGB444 format, and 4 to 7 are for future expansion.
- a bit indicating CbCr sample position information may be additionally allocated.
- CD is Color Depth, for example, 4h is each YCbCr component 8bit total 24bit Color, 5h is each YCbCr component 10bit total 30bit Color, 6h is each YCbCr component 12bit total 36bit Color, 7h is each YCbCr component 16bit total 48bit Color is shown, others are for future expansion. This definition conforms to the definition of Deep Color Mode defined by HDMI.
- Eco_CbCr indicates application information of the coordinate conversion of Cb and Cr described in FIG. 4 and FIG. If 0, no coordinate transformation is applied, 1 is the simple coordinate transformation explained in FIG. 5, 2 is the coordinate transformation minimizing the Cb or Cr range explained in FIG. 4, and 3 is the CbCr range minimized. Further, coordinate conversion with coefficient compression is shown. 4 to 7 are for future expansion. When the Color_Sample specifies the RGB444 format, this coordinate conversion is invalid, and an unapplied 0 may be described. Next, coefficient compression will be described.
- the effective range region of Cb is a range of 68 to 188 at the time of 8-bit quantization, and the range width can be expressed by 7 bits, which is 1 bit, 121.
- the minimum Cr range in FIG. 6 cannot be expressed by 143 and 7 bits in the range of 57 to 199 and the range width at the time of 8-bit quantization.
- each reduction coefficient of Cb and Cr is indicated by a value obtained by dividing the 8-bit data of Cb_Factor and Cr_Factor in FIG.
- the 00h notation indicates a reduction factor of 1.00, that is, no factor compression.
- the number of quantized bits after the coefficient reduction is smaller than the number of quantized bits in the original space.
- Cb_Factor and Cr_Factor are 129 to 255, 0 is 1 bit, 65 to 128 is 2 bits, 33 to 64 is 3 bits, and 17 to 32 are It shall be reduced by 4 bits. Even if the number of gradations of the CbCr color difference signal is reduced with respect to the Y luminance signal, since the influence on the image quality is relatively small, the data amount compression by this method is effective.
- 64 is normalized CbCr 0.00 and quantization data 1 to 126 is used.
- 0 and 255 before coordinate conversion are transmitted as 0 and 127, respectively.
- Cb_Factor is 128.
- the data after coefficient compression is 10 bits less by 2 bits, and 512 is normalized Cb 0.00, and quantized data 1 to 1022 is used. 0 and 4095 before coordinate conversion are transmitted as 0 and 1023.
- the lower 2 bits of the 10-bit Y luminance signal may be assigned to the lower 1 bit of 8 bits, one bit at a time.
- Eco_FLM is set to 1 when the compression code system of all the blocks in the frame is the same, and is set to 0 when set for each block.
- the Y-, Cb-, and Cr-compression code systems are described in Eco-CD0, Eco-CD1, and Eco-CD2, which will be described later.
- the ratio (CK_N / CM_M) of CK_N and CK_M indicates a frequency ratio between a pixel clock of uncompressed image data and a clock of a communication path for transmitting the compressed data, for example, a TMDS clock.
- Eco_Block indicates the number of pixels of the compressed pixel unit block.
- Eco_CD0 to Eco_CD3 indicate four types of compression coding information candidates to be applied to each image data unit block. As shown in FIG. 16, four types are selected from compression code information Eco_Code for each image data unit to be compressed.
- One or more packets including the header of FIG. 14 and the data of FIG. 15 are transmitted in each horizontal blanking period.
- Eco_Packet # in the header of FIG. 14 indicates the serial number of this packet transmitted to each line, and starts from 1 and is incremented by 1.
- Code_0 to Code_111 shown in FIG. 15 describe the Y, Cb, and Cr components in order in each image data unit block, with numbers selected from the four types of compression encoded information described in Eco_CD0 to Eco_CD3 in FIG. . For example, if Code_0 indicating the compression code information of the Y component of the first image data unit block is 1, it indicates Eco_CD1. If Eco_CD1 indicates 10, it indicates that data obtained by compressing the 444 format Y component of the original image data by the differential encoding method from FIG.
- Code_0 of the first line is compression code information of the image data unit block 503
- Code_0 of the second line is compression code information of the image data unit block 504.
- Code_1 is Cb compression code information of the first image data unit block
- Code_2 is Cr compression code information of the first image data unit
- Code_3 is second image data.
- the Y compression code information of the unit block is shown.
- Color_Sampl indicates 420 format
- Code_1 is Y compression code information of the second image data unit block
- Code_2 is compression code information of Cb (Cr in even lines) of the first and second image data units
- Code_3 indicates Y compression code information of the third image data unit block.
- Code_1 is Cb compression code information of the first and second image data unit blocks
- Code_2 is Y compression code information of the second image data unit
- Code_3 is the first and second image data unit blocks.
- Cb compression code information of the second image data unit block is shown.
- Code_0 indicates Eco_CD0
- Eco_CD0 indicates 6
- the 444 format Y, Cb, and Cr component 12-bit data of the original image data is thinned out to 420 format 8 bits from FIG. It is shown that.
- the transmission form of Cb and Cr is determined only by Y component Code_0, information of Code_1 indicating the Cb component and Code_2 indicating the Cr component is unnecessary, and 0 may be described.
- the image receiving device 200 is equipped with a ROM that stores EDID (Enhanced Extended Display Identification Data) indicating the performance of the image receiving device 200.
- Information for determining whether or not the image receiving apparatus 200 supports compression / decompression may be added to the ROM.
- the image transmission apparatus 100 reads information for determining whether or not it supports compression / decompression from the ROM storing the EDID of the image reception apparatus 200, and if it is a compatible apparatus, the compressed image data If the device is non-compatible, the image can be transmitted in a conventional size without being compressed, and compatibility with an image reception device not compatible with compression processing can be maintained.
- the user can be notified by displaying on the display unit 208 that the image receiving apparatus is incompatible with compression and has a conventional image size.
- FIG. 17 shows a description example of this EDID.
- FIG. 17 shows an example of extension to an area called HDMI-VSDB.
- An Eco_transfer flag indicating whether or not the compression encoding transmission system of the present invention can be supported is provided in Bit 2 of the 6th byte. Since this area has been treated as a reserved area, it is described as 0 for non-compliant legacy devices, and backward compatibility can be maintained by describing only 1 for compatible devices. When the Eco_transfer flag is 1, the description of Byte 9 and Byte 10 is valid.
- Eco_CbCr of Bit 0 and 1 indicates whether or not the coordinate conversion of the CbCr color difference signal can be supported, and corresponds to the lower 2 bits of Eco_CbCr in FIG.
- Block_64 and Block_128 are flags indicating that the size of the image data unit block to be compressed corresponds to 64 pixels and 128 pixels, respectively.
- the image data unit block size of 32 pixels is defined as an indispensable mode in correspondence with the transmission of the compressed image data, and is not intentionally written in order to save EDID description space.
- Eco-Codes 1 to 4 are flags indicating that they are compatible with the compression coding method, Wavelet transform, run-length coding, Huffman coding, and differential coding, which are shown as examples in FIG.
- CLK_1, CLK_3 / 4, and CLK_1 / 2 are flags indicating that the mode corresponds to the mode in which the frequency of the TMDS transmission clock is 1 time, 3/4 time, and 1/2 time with respect to the uncompressed image data clock, respectively. is there.
- the image transmission apparatus 100 when used as a portable device, it becomes a battery-powered apparatus, so the power consumption of the image transmission apparatus 100 affects the continuous use time.
- image data can be compressed and transmitted to reduce the amount of data transmission and reduce power consumption.
- This effect is achieved by adding a function such as “power saving mode” as an operation mode of the image transmission apparatus 100, and when power is supplied from the outside, the image transmission apparatus 100 is transmitted by non-compressed image data and driven by a battery.
- the continuous use time can be set longer by compressing and transmitting the image data.
- FIG. 18 is a block diagram illustrating an example of the configuration of the data reception processing unit 205.
- the input unit 220 outputs the signal converted by the level conversion unit 175 of the image transmission apparatus 100 to the level conversion unit 222.
- the level conversion unit 222 converts the signal level-converted by the image transmission apparatus 100 into a digital signal and outputs the digital signal to the deserializer 223.
- An example of level conversion is conversion of a differential signal to a single-ended signal.
- the input unit 221 inputs the clock output from the image transmission apparatus 100 and outputs it to the PLL 224.
- the PLL 224 generates a clock 10 times the input clock and outputs the generated clock to the deserializer 223.
- the PLL 224 outputs a pixel clock used in the image receiving apparatus 200 from the output unit 226.
- the clock output by the PLL 224 multiplied by (CK_M / CK_N) times is output from the output unit 226 based on the packet data of FIG. To do.
- the deserializer 223 parallelizes the serialized data with the clock from the PLL 224 and outputs it from the output unit 225.
- the deserializer 223 parallelizes the 10-times clock data, and outputs the data from the output unit 225 as 8-bit parallel data by predetermined TMDS decoding, for example.
- FIG. 19 is a block diagram illustrating an example of the configuration of the decompression processing unit 206.
- FIG. 20 is a timing explanatory diagram showing a processing concept of the decompression processing unit 206.
- the input unit 250 is a data input unit of the decompression processing unit 206.
- Data input to the input unit 250 includes HSYNC (FIG. 20A) and VSYNC indicating a synchronization signal of image data, image data 512 and 514 compressed in the valid period 406, and compressed in the horizontal blanking period 404.
- the input unit 251 receives a pixel clock of uncompressed image data after restoration, a compressed image data clock, and the like.
- the timing generation unit 252 controls the counter based on the input HSYNC and VSYNC, and starts and expands the timing of the vertical blanking period 401, the vertical valid period 402, the horizontal blanking period 404, the horizontal valid period 405, the valid period 406, and the like. Generates and outputs timing necessary for control of each block in the processing unit.
- the compression code information extraction unit 253 extracts the compression code information of each image data unit block sent during the horizontal blanking period and stores it in the compression code information storage unit 254.
- the storage period is shown in FIG. Since the compression code information 511 is data corresponding to the subsequent compressed image data 512, a storage period 515 until the compression code information 513 of the second line comes is sufficient. However, for example, when two lines of image data are vertically compressed, it is necessary to hold only the compression code information 516 for vertical expansion until a period during which the compressed image data of the second line is expanded.
- Compressed image data sent within the horizontal effective period is subjected to transmission system error correction processing by the error correction unit 255.
- the error correction unit 255 calculates the same error correction code as the error correction code generation unit 136 for each unit of compressed image data.
- the calculation result is compared with the error correction code input from the code detection unit 253, and if the comparison result is different, an error correction process is performed.
- An example of error correction processing is CRC calculation. Alternatively, only error detection may be performed, and errors may be interpolated in subsequent processing.
- the thinned image data is directly input to the selection unit 258 without passing through the error correction unit 255 (arrow 263 in FIG. 19).
- the error is limited to the corresponding pixel and adjacent thinned-out pixels, and does not significantly deteriorate the image quality of the entire screen. This is because consideration is given to the point that thinning out pixels or gradations leads to image quality deterioration of the entire screen.
- error correction processing may be added to the thinned image data.
- the horizontal decompression unit 256 and the vertical decompression unit 257 decompress the image data and output it to the selection unit 258.
- the horizontal decompression unit 256 output (FIG. 20D) outputs horizontal decompressed image data 518 and 519 delayed from the compressed image data 512 and 514 by the time required for decoding.
- the output of the vertical extension unit 257 (as shown in FIG. 20E) is the vertical extension image data 520 and 521 of the first and second lines over a two-line period. Is output.
- the selection unit 258 appropriately selects the image data input to the input unit 250, the output of the horizontal expansion unit 256, and the output of the vertical expansion unit 257 based on the information in the compression code information storage unit, and outputs them to the thinning restoration unit 259. .
- the restored image data 522 is output with a delay of one line or more from the input compressed image data 512 in accordance with the vertical expansion unit output 521 that is output with the latest delay.
- the input image data of the selection unit 258 has a memory (not shown) for appropriately absorbing a processing delay due to each expansion method.
- the thinning restoration unit 259 restores the image data of the thinning unit from the thinned image data based on the information in the compression code information storage unit 254.
- the thinning restoration corresponds to the case where the horizontal compression unit 133 and the vertical compression unit 134 perform thinning processing from the 444 format to 422 in addition to the thinning processing unit 132 in FIG. Restore.
- the output unit 261 requests 422 or 420 format image data instead of the 444 format, the thinning restoration process may be stopped or the thinning format conversion may be performed.
- the thinning restoration unit 259 complements the left and right images, and if the signal is compressed in the vertical direction , Complementary processing may be performed from data one line before.
- the YCbCr processing unit 260 that receives the output of the thinning restoration unit 259 is in charge of color difference signal processing for restoring the reduced Cb and Cr based on the packet data information Eco_CbCr, Cb_Factor, and Cr_Factor shown in FIG. Further, when the output unit 261 requests an RGB primary color signal, a conversion process to an RGB primary color signal is performed. Thus, the YCbCr processing unit 260 is positioned to perform the reverse operation of the YCbCr processing unit 131 in FIG. 2 in the image transmission apparatus, and detailed description thereof is omitted.
- image data having a size larger than the currently specified image size is transmitted to the currently specified transmission path by compressing and transmitting the image data transmitted by the image transmission apparatus.
- image transmission with higher error tolerance by adding error detection and correction codes to a region with higher error resistance than the image data transmission region.
- the data transmission amount per predetermined time or the data transmission clock can be lowered, so that the frequency of occurrence of errors can be lowered, and A highly reliable system can be constructed against errors in the transmission path.
- the vertical compression image data unit block extends over two or more lines as shown in FIG. 8, and as a result, a delay of one horizontal cycle or more is generated with respect to the compressed image data as shown in FIG.
- the vertical compression image data unit block is also set to one line, which is the same as the horizontal compression image data unit block of FIG.
- the vertical compression is limited to an encoding method such as encoding of the difference with respect to the previous line, but there is an advantage that the processing delay time can be reduced as compared with the first embodiment.
- the compression code information for each image data unit block is transmitted using a packet during a highly reliable horizontal blanking period.
- FIG. It was necessary to keep close to the cycle.
- On the transmitter side in order to arrange the compression code information in the horizontal blanking period immediately before the compressed image data of the corresponding line, it is necessary to store the compressed image data in the line memory and output it with a delay of about one horizontal cycle.
- the circuit scale increases and the transmission delay time of the image signal increases. Therefore, in the third embodiment, a method of transmitting compression code information in units of pressure image data together with compressed image data within the horizontal effective period instead of the horizontal blanking period is employed.
- the compression rate of the compressed image data must be increased, and there is a concern that the image quality of the image at the time of restoration will increase greatly .
- the horizontal blanking period is reduced to such an extent that two audio data transmission packets can be sent, and the horizontal effective period is extended.
- the horizontal effective period 1920 and the horizontal blanking period 280 are used in the first embodiment. Since it is sufficient that the horizontal blanking period is about 96 including the transmission period of two voice packets and the guard bands before and after, the remaining 184 can be increased in the horizontal effective period. There is an effect that the horizontal effective period 1920 can be expanded by about 9.5%.
- the position and width of the horizontal synchronization signal deviate from the standard timing of predetermined uncompressed image data. Therefore, when the uncompressed image data is reproduced on the receiving side, SVD (Short Video added to the image data) Descriptor) The timing should be restored to the standard timing format with reference to the metadata.
- SVD Short Video added to the image data
- FIG. 21 is a block diagram illustrating an example of the configuration of the decompression processing unit 206. Blocks having the same functions as those in FIG. 19 are given the same numbers. The difference is that a second video code information extraction unit 263 is added to the output unit of the error correction circuit 255, and the input of the selection unit 258 (arrow 264 in FIG. 21) is not directly sent from the input 250 as the thinned image data but through the error correction unit 255. This is the point of reference).
- the compression code information extraction unit 253 includes the compression code information (packets in FIGS. 12 and 13) in the vertical blanking period and the compression code information (packets in FIGS. 14 and 15) in the horizontal blanking period. Was extracted. In this embodiment, the compression code information extraction unit extracts only the intra-frame common information within the vertical blanking period and stores it in the compression code information storage unit 254.
- the second compression code information extraction unit 262 uses the data after the error correction processing of the error correction unit 255 and the output of the compression code information storage unit 254 extracted in units of frames, for each compressed image data unit block.
- the compression code information is extracted and output to the horizontal expansion unit 256, the vertical expansion unit 257, and the selection unit 258.
- the compressed image data and the compressed code information of each compressed image data unit block are close in timing, the compressed code information storage for one horizontal period is excluded except for the compressed code information common in the frame. Since it becomes unnecessary, the circuit scale can be reduced.
- the horizontal effective period can be extended by 9.5%, so that an error correction code is additionally transmitted.
- the thinned image data is also input to the selection unit 258 as the output 264 of the error correction unit 255, errors in the thinned image data can be suppressed, and image data transmission with higher error resistance can be realized.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Color Television Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The purpose of the present invention is, when image data to be transmitted by an image transmission device is compressed and transmitted and the image data having a size larger than a currently specified image size is transmitted, to obtain, with a simple configuration, both a transmission delay time minimization of and a high compression of the image data.
An image transmission method for compressing and transmitting image data is characterized in that: color-difference signal processing is performed in which a Cb or Cr color-difference signal valid range is reduced by a coordinate transformation using a correlation with a Y brightness signal; compression processing is performed on the image data consisting of the resultant YCbCr brightness/color-difference signal; and the image data on which the compression processing has been performed is output in synchronization with a clock signal having a synchronous relationship with a clock of non-compressed image data.
Description
技術分野は、映像情報の送受信に関する。
The technical field relates to transmission and reception of video information.
近年、デジタル画像処理で扱う画素数は、放送のHD(High Definition:1920×1080画素)化、デジタルビデオカメラのイメージセンサやディスプレイの高画素化に伴い、年々増加している。
In recent years, the number of pixels handled in digital image processing has been increasing year by year as broadcasting HD (High Definition: 1920 × 1080 pixels) and digital video camera image sensors and displays have higher pixels.
これらの画像データを機器間で伝送する方式について、HDMI(High-Definition Multimedia Interface (HDMI Licensing,LLCの登録商標))規格やVESA(Video Electronics Standards Association)により策定されたDisplayPort(VESAの登録商標または商標)規格などがある。
Regarding the method of transmitting these image data between devices, DisplayPort (VESA registered trademark or VESA registered trademark) or HDMI (High-Definition Multimedia Interface (HDMI Licensing, LLC registered trademark)) standard or VESA (Video Electronics Standards Association) Trademark) standards.
前記のHDMIのデータ伝送方式において、特許文献1には、「非圧縮映像信号またはこの非圧縮映像信号に対して受信装置が対応可能な圧縮方式で圧縮処理を施して得られた圧縮映像信号を選択的に送信するものであり、伝送路の伝送ビットレート内で、所望のビットレートの映像信号を良好に送信できる」(特許文献1[0048]参照)こと、また圧縮方式について、「データ圧縮部121-1~121-nは、それぞれ、コーデック117から出力された非圧縮の映像信号を所定の圧縮比をもって圧縮処理し、圧縮された映像信号を出力する。データ圧縮部121-1~121-nは、映像信号圧縮部を構成している。データ圧縮部121-1~121-nは、それぞれ、互いに異なる圧縮方式でデータ圧縮処理を行う。例えば、圧縮方式としては、「RLE(Run Length Encoding)」、「Wavelet」、「SBM(Super Bit Mapping(ソニーの登録商標))」、「LLVC(Low Latency Video Codec)」、「ZIP」等が考えられる」(特許文献1[0077]参照)ことが記載されている。
In the above-mentioned HDMI data transmission system, Patent Document 1 describes that “a non-compressed video signal or a compressed video signal obtained by compressing a non-compressed video signal with a compression system that can be supported by a receiving device”. For selective transmission, a video signal having a desired bit rate can be satisfactorily transmitted within the transmission bit rate of the transmission path ”(see Patent Document 1 [0048]), and for the compression method,“ data compression ” The units 121-1 to 121-n respectively compress the uncompressed video signal output from the codec 117 with a predetermined compression ratio, and output the compressed video signal. -n constitutes a video signal compression unit, and each of the data compression units 121-1 to 121-n performs a data compression process using a different compression method. "RLE (Run Length Encoding)", "Wavelet", "SBM (Super Bit Mapping (Sony's registered trademark))", "LLVC (Low Latency Video Codec)", "ZIP", etc. are considered "(Patent Document 1) [0077]).
また、HDMIにおいて、画像データは、TMDS(Transition Minimized Differential Signaling(Silicon Image, Inc.の登録商標))方式のデータ伝送フォーマットが採用されており、その一例として特許文献2が示されている。
In HDMI, the image data adopts a TMDS (Transition Minimized Differential Signaling (registered trademark of Silicon Image, Inc.)) system, and Patent Document 2 is shown as an example.
しかし、いずれの先行技術文献においても、符号化による遅延時間を抑えながら、より大きなサイズの画像(「映像」とも言う。以下同じ。)データを伝送することは考慮されていない。
However, in any of the prior art documents, it is not considered to transmit image data of a larger size (also referred to as “video”; the same applies hereinafter) while suppressing delay time due to encoding.
上記課題を解決するために、例えば特許請求の範囲に記載の構成を採用する。
本願は上記課題を解決する手段を複数含んでいるが、その一例を挙げるならば、画像データを圧縮し伝送する画像伝送装置において、輝度信号との相関を用いた座標変換によって色差信号のとりうる範囲を狭める色差信号処理部と、輝度色差信号からなる画像データを圧縮する圧縮処理部と、前記圧縮処理部で圧縮された画像データを出力するデータ転送部を有し、該データ転送部は、圧縮された画像データと共に前記色差信号処理部及び前記圧縮処理部で行われる処理に関する圧縮符号情報を出力することを特徴とする。 In order to solve the above problems, for example, the configuration described in the claims is adopted.
The present application includes a plurality of means for solving the above-mentioned problems. For example, in an image transmission apparatus that compresses and transmits image data, a color difference signal can be obtained by coordinate conversion using correlation with a luminance signal. A color difference signal processing unit that narrows the range; a compression processing unit that compresses image data including luminance color difference signals; and a data transfer unit that outputs image data compressed by the compression processing unit, the data transfer unit including: The compressed code data relating to the processing performed in the color difference signal processing unit and the compression processing unit is output together with the compressed image data.
本願は上記課題を解決する手段を複数含んでいるが、その一例を挙げるならば、画像データを圧縮し伝送する画像伝送装置において、輝度信号との相関を用いた座標変換によって色差信号のとりうる範囲を狭める色差信号処理部と、輝度色差信号からなる画像データを圧縮する圧縮処理部と、前記圧縮処理部で圧縮された画像データを出力するデータ転送部を有し、該データ転送部は、圧縮された画像データと共に前記色差信号処理部及び前記圧縮処理部で行われる処理に関する圧縮符号情報を出力することを特徴とする。 In order to solve the above problems, for example, the configuration described in the claims is adopted.
The present application includes a plurality of means for solving the above-mentioned problems. For example, in an image transmission apparatus that compresses and transmits image data, a color difference signal can be obtained by coordinate conversion using correlation with a luminance signal. A color difference signal processing unit that narrows the range; a compression processing unit that compresses image data including luminance color difference signals; and a data transfer unit that outputs image data compressed by the compression processing unit, the data transfer unit including: The compressed code data relating to the processing performed in the color difference signal processing unit and the compression processing unit is output together with the compressed image data.
上記手段によれば、より大きなサイズの画像データをより少ない遅延時間で伝送することが可能となる。
According to the above means, it is possible to transmit image data of a larger size with less delay time.
従来、遅延時間を最小にする画像データ伝送方式として非圧縮画像データ伝送方式があるが、大きなサイズの画像データを送るには高速な伝送路を必要とする課題があった。その解決の為に、画像データを圧縮して伝送する方式が提案されているが、圧縮率を上げると遅延時間が大きくなる課題があり、さらには非圧縮画像データとのタイミング同期を実現しにくい課題があった。
Conventionally, there is an uncompressed image data transmission system as an image data transmission system that minimizes the delay time, but there is a problem that a high-speed transmission path is required to send large-size image data. In order to solve this problem, a method of compressing and transmitting image data has been proposed. However, there is a problem that the delay time increases when the compression rate is increased, and it is difficult to realize timing synchronization with uncompressed image data. There was a problem.
以下の実施例では、画像データを圧縮して伝送する伝送方式において、画像伝送装置が伝送する画像データを圧縮して伝送する前にY輝度信号との相関関係を用いてCb又はCr色差信号を座標変換することによってCb又はCr色差信号の有効範囲を制限して画像データの圧縮効率を向上させるプレ処理と、元の非圧縮画像データが想定する画素クロック信号と同期した伝送クロックに基づいた圧縮画像データを伝送し、画像受信装置が圧縮画像データを伸張後にY輝度信号との相関関係を用いて制限されたCb又はCr色差信号を座標変換によって復元するポスト処理を行う。以下、この実施例を、図面を用いて説明する。
In the following embodiments, in a transmission method in which image data is compressed and transmitted, a Cb or Cr color difference signal is obtained using a correlation with a Y luminance signal before the image data transmitted by the image transmission apparatus is compressed and transmitted. Pre-processing that improves the compression efficiency of image data by limiting the effective range of the Cb or Cr color difference signal by coordinate conversion, and compression based on the transmission clock synchronized with the pixel clock signal assumed by the original uncompressed image data The image data is transmitted, and the image receiving apparatus performs post processing for decompressing the compressed image data and restoring the Cb or Cr color difference signal limited by using the correlation with the Y luminance signal by coordinate transformation. Hereinafter, this embodiment will be described with reference to the drawings.
以下本実施例における画像伝送装置および画像受信装置について説明する。
Hereinafter, an image transmission apparatus and an image reception apparatus in this embodiment will be described.
図1は、本実施例の画像伝送システムを示すブロック図であり、画像伝送装置100と画像受信装置200をケーブル300で接続した構成である。
FIG. 1 is a block diagram illustrating an image transmission system according to the present embodiment, in which an image transmission apparatus 100 and an image reception apparatus 200 are connected by a cable 300.
画像伝送装置100は、画像データを伝送する画像伝送装置であり、デジタル放送を受信し視聴できるようにデコードした画像データや、カメラなどで撮影した画像データをHDMIケーブルなどにより、他の機器に画像データを出力する機器である。画像伝送装置100の一例として、レコーダや、レコーダ機能を内蔵したデジタルTVや、レコーダ機能を内蔵したパソコンや、カムコーダや、カメラ機能を搭載した携帯電話などがある。
The image transmission apparatus 100 is an image transmission apparatus that transmits image data. Image data that has been decoded so that digital broadcasts can be received and viewed, and image data that has been captured by a camera or the like can be imaged to other devices using an HDMI cable or the like. A device that outputs data. Examples of the image transmission device 100 include a recorder, a digital TV with a built-in recorder function, a personal computer with a built-in recorder function, a camcorder, and a mobile phone with a camera function.
画像受信装置200は、HDMIケーブル等を使用して、画像データを入力しモニタに画像を出力する表示機器である。画像受信装置200の一例として、デジタルTVや、ディスプレイ、プロジェクタなどがある。
The image receiving apparatus 200 is a display device that inputs image data and outputs an image to a monitor using an HDMI cable or the like. Examples of the image receiving apparatus 200 include a digital TV, a display, and a projector.
ケーブル300は、画像伝送装置100と画像受信装置200の機器間で画像データ等のデータ通信を行うデータ伝送路である。ケーブル300の一例として、HDMI規格や、DisplayPort規格に対応した有線ケーブルもしくは、無線方式のデータ通信を行うデータ伝送路などがある。
The cable 300 is a data transmission path for performing data communication such as image data between the devices of the image transmission device 100 and the image reception device 200. As an example of the cable 300, there is a wired cable corresponding to the HDMI standard or the DisplayPort standard, or a data transmission path for performing wireless data communication.
まず、画像伝送装置100の構成について説明する。
入力部101、102、103は、画像データを画像伝送装置100に入力するための入力部である。入力部101に入力される画像データの一例としては、放送局または放送用衛星などの中継局からの電波として入力されるデジタル放送がある。入力部101には、この放送局または放送用衛星などの中継局からの電波が入力される。 First, the configuration of theimage transmission apparatus 100 will be described.
The input units 101, 102, and 103 are input units for inputting image data to the image transmission apparatus 100. An example of image data input to the input unit 101 is digital broadcasting input as radio waves from a relay station such as a broadcasting station or a broadcasting satellite. The input unit 101 receives radio waves from a relay station such as this broadcasting station or broadcasting satellite.
入力部101、102、103は、画像データを画像伝送装置100に入力するための入力部である。入力部101に入力される画像データの一例としては、放送局または放送用衛星などの中継局からの電波として入力されるデジタル放送がある。入力部101には、この放送局または放送用衛星などの中継局からの電波が入力される。 First, the configuration of the
The
入力部102に入力される画像データの一例としては、インターネットのブロードバンド接続を利用して、ネットワーク経由で配信されてくるデジタル放送や、情報コンテンツなどがある。
Examples of image data input to the input unit 102 include digital broadcasts distributed via a network and information content using an Internet broadband connection.
入力部103に入力される画像データの一例としては、入力部103に接続された外部の記録メディアに記録されているコンテンツや、画像伝送装置100内に内蔵された記録メディア108に記録されているコンテンツがある。入力部103に接続された外部の記録メディア、もしくは画像伝送装置100内に内蔵された記録メディア108の一例としては、光ディスク、磁気ディスク、半導体メモリなどがある。
As an example of image data input to the input unit 103, content recorded on an external recording medium connected to the input unit 103, or recorded on a recording medium 108 built in the image transmission apparatus 100. There is content. Examples of the external recording medium connected to the input unit 103 or the recording medium 108 built in the image transmission apparatus 100 include an optical disk, a magnetic disk, and a semiconductor memory.
チューナー受信処理部105は、入力された電波をビットストリームに変換する受信処理部であり、ここでRF帯域(Radio Frequency)の電波はIF帯域 (Intermediate Frequency) に周波数変換され、受信チャネルに依存しない一定の帯域の信号として、復調されたビットストリームに伝送のために施された変調操作を復調する。
The tuner reception processing unit 105 is a reception processing unit that converts an input radio wave into a bit stream. Here, the radio wave of the RF band (Radio Frequency) is frequency-converted to an IF band (Intermediate Frequency) and does not depend on the reception channel. A modulation operation applied to the demodulated bit stream for transmission as a fixed band signal is demodulated.
ビットストリームの一例としては、MPEG2トランスポートストリーム(以降MPEG2-TSとする)や、MPEG2-TSに準じたフォーマットのビットストリーム等がある。以降のビットストリームは、MPEG2-TSを代表として説明を行う。
Examples of bitstreams include an MPEG2 transport stream (hereinafter referred to as MPEG2-TS), a bitstream having a format conforming to MPEG2-TS, and the like. The following bit stream will be described with MPEG2-TS as a representative.
前記チューナー受信処理部105は、さらに伝送途中で発生した符合の誤りを検出し訂正し、誤り訂正されたMPEG2-TSにつきスクランブルの解除を行った後、視聴もしくは記録を行うプログラムが多重化されている1トランスポンダ周波数を選択し、この選択した1トランスポンダ内のビットストリームを1プログラムのオーディオとビデオのパケットに分離化する。
The tuner reception processing unit 105 further detects and corrects a code error occurring during transmission, and after the descrambling of the error-corrected MPEG2-TS, a program for viewing or recording is multiplexed. One transponder frequency is selected, and the bit stream in the selected one transponder is separated into audio and video packets of one program.
前記チューナー受信処理部105からのMPEG2-TSは、データバス181を通してストリーム制御部111に供給される。ストリーム制御部111は、前記チューナー受信処理部105においてパケットを受信した時の間隔を保時する為に、受信したパケット内から時刻管理情報であるPTS(Presentation Time Stamp)と、MPEGシステムの基準復号器内部のSTC(System Time Clock)を検出し、検出結果により補正したタイミングでタイムスタンプを付加する。前記タイムスタンプを付加したパケットをデコーダ112もしくは、記録メディア制御部107のどちらか一方、または双方に供給する。デコーダ112へのデータパス184は画像データを視聴する時の処理に用いられ、記録メディア制御部107へのデータパス183は、画像データを記録メディアに記録する時に用いられる。
The MPEG2-TS from the tuner reception processing unit 105 is supplied to the stream control unit 111 through the data bus 181. In order to maintain the interval when the tuner reception processing unit 105 receives a packet, the stream control unit 111 performs PTS (Presentation Time Stamp) that is time management information from the received packet, and MPEG system reference decoding The STC (System Time Clock) inside the device is detected, and a time stamp is added at the timing corrected by the detection result. The packet with the time stamp added is supplied to either one or both of the decoder 112 and the recording media control unit 107. The data path 184 to the decoder 112 is used for processing when viewing image data, and the data path 183 to the recording medium control unit 107 is used when recording image data on a recording medium.
前記ストリーム制御部111のデータバス182は、入力部102からネットワーク受信処理部106経由で入力されるMPEG2-TSが入力される。前記データパス182は、ネットワーク経由で配信されてくるデジタル放送もしくはデジタルコンテンツを取得する入力部である。
The MPEG2-TS input from the input unit 102 via the network reception processing unit 106 is input to the data bus 182 of the stream control unit 111. The data path 182 is an input unit that acquires digital broadcast or digital content distributed via a network.
さらに前記ストリーム制御部111のデータバス183からは、入力部103に接続された外部の記録メディア、もしくは画像伝送装置100内に内蔵された記録メディア108に記録されているデジタル放送もしくはデジタルコンテンツを、記録メディア制御部107により読み出したMPEG2-TSが入力される。前記ストリーム制御部111は、これらの入力のうち少なくとも1つ以上を選択し、デコーダ112に出力する。
Further, from the data bus 183 of the stream control unit 111, digital broadcasting or digital content recorded on an external recording medium connected to the input unit 103 or a recording medium 108 built in the image transmission apparatus 100, The MPEG2-TS read by the recording media control unit 107 is input. The stream control unit 111 selects at least one of these inputs and outputs it to the decoder 112.
デコーダ112は、前記ストリーム制御部111から入力されたMPEG2-TSをデコードし、生成した画像データを表示処理部113に出力する。表示処理部113は、入力された画像データに対して、例えばOSD(On Screen Display)の重畳処理や、拡大もしくは縮小処理を施した後、圧縮処理部114に出力する。
The decoder 112 decodes the MPEG2-TS input from the stream control unit 111 and outputs the generated image data to the display processing unit 113. The display processing unit 113 performs, for example, OSD (On Screen Display) superimposition processing or enlargement / reduction processing on the input image data, and then outputs the processed image data to the compression processing unit 114.
圧縮処理部114は、表示処理部113からの画像データに圧縮処理を施し、データ伝送部115に出力する。
The compression processing unit 114 performs compression processing on the image data from the display processing unit 113 and outputs it to the data transmission unit 115.
データ伝送部115は、画像データを伝送に適した形式の信号に変換して出力部116から出力を行う。伝送に適した形式の信号の一例として、ケーブルによる伝送に適した形式がHDMI規格に記載されている。HDMIにおいて、画像データは、TMDS方式のデータ伝送フォーマットが採用されている。
The data transmission unit 115 converts the image data into a signal in a format suitable for transmission and outputs it from the output unit 116. As an example of a signal in a format suitable for transmission, a format suitable for transmission by cable is described in the HDMI standard. In HDMI, image data adopts a TMDS data transmission format.
入力部104は、画像伝送装置100の動作を制御するための信号を入力するための入力部である。入力部104の一例として、リモコンの受信部などがある。入力部104からの制御信号は、ユーザーIF109に供給される。前記ユーザーIF109は、入力部104からの信号を制御部110に出力する。制御部110は、入力部104の信号に従い、画像伝送装置100全体を制御する。制御部110の一例としては、マイクロプロセッサなどがある。画像伝送装置100からの画像データは、ケーブル300を介して画像受信装置200に供給する。
The input unit 104 is an input unit for inputting a signal for controlling the operation of the image transmission apparatus 100. An example of the input unit 104 is a remote control receiver. A control signal from the input unit 104 is supplied to the user IF 109. The user IF 109 outputs a signal from the input unit 104 to the control unit 110. The control unit 110 controls the entire image transmission apparatus 100 according to the signal from the input unit 104. An example of the control unit 110 is a microprocessor. Image data from the image transmission apparatus 100 is supplied to the image reception apparatus 200 via the cable 300.
次に画像受信装置200の構成について説明する。
入力部201は、伝送に適した形式の信号が入力される。前記入力部201に入力された信号は、データ受信処理部205に供給される。 Next, the configuration of theimage receiving apparatus 200 will be described.
Theinput unit 201 receives a signal in a format suitable for transmission. The signal input to the input unit 201 is supplied to the data reception processing unit 205.
入力部201は、伝送に適した形式の信号が入力される。前記入力部201に入力された信号は、データ受信処理部205に供給される。 Next, the configuration of the
The
データ受信処理部205は、伝送に適した形式の信号から所定のデジタルデータに変換する処理を施し、伸張処理部206に変換したデジタルデータを出力する。
The data reception processing unit 205 performs processing for converting a signal in a format suitable for transmission into predetermined digital data, and outputs the converted digital data to the expansion processing unit 206.
伸張処理部206は、前記画像伝送装置100内の圧縮処理部114で圧縮された画像データを伸張し、表示処理部207に出力する。
The decompression processing unit 206 decompresses the image data compressed by the compression processing unit 114 in the image transmission apparatus 100 and outputs it to the display processing unit 207.
表示処理部207は、入力された画像データに対して表示処理を施す。表示処理の一例としては、OSD重畳処理、表示部208の解像度に変換するための拡大もしくは縮小処理、フレームレート変換処理などがある。表示処理部207の出力は表示部208に出力する。
The display processing unit 207 performs display processing on the input image data. Examples of display processing include OSD superimposition processing, enlargement / reduction processing for conversion to the resolution of the display unit 208, frame rate conversion processing, and the like. The output of the display processing unit 207 is output to the display unit 208.
表示部208は、入力された画像データを表示方式にあわせた信号に変換し画面に表示する。表示部208の一例として、液晶ディスプレイや、プラズマディスプレイや、有機EL(Electro-Luminescence)ディスプレイなどの表示部がある。
The display unit 208 converts the input image data into a signal suitable for the display method and displays it on the screen. Examples of the display unit 208 include a display unit such as a liquid crystal display, a plasma display, and an organic EL (Electro-Luminescence) display.
入力部202は、画像受信装置200の動作を制御するための信号を入力するための入力部である。入力部202の一例として、リモコンの受信部などがある。入力部202からの制御信号は、ユーザーIF203に供給される。前記ユーザーIF203は、入力部202からの信号を制御部204に出力する。制御部204は、入力部202の信号に従い、画像受信装置200全体を制御する制御部である。
The input unit 202 is an input unit for inputting a signal for controlling the operation of the image receiving apparatus 200. An example of the input unit 202 is a remote control receiver. A control signal from the input unit 202 is supplied to the user IF 203. The user IF 203 outputs a signal from the input unit 202 to the control unit 204. The control unit 204 is a control unit that controls the entire image receiving apparatus 200 in accordance with a signal from the input unit 202.
図2は、圧縮処理部114の構成の一例を示すブロック図である。
入力部130は、圧縮処理部114へ画像データを入力するための入力部である。入力された画像データは、YCbCr処理部131供給される。YCbCr処理回路は、入力画像データがRGB原色信号であれば、その画像データに付与されたカラリメトリ情報に基づき所定の変換式を使用してYCbCr輝度色差信号に変換する。 FIG. 2 is a block diagram illustrating an example of the configuration of thecompression processing unit 114.
Theinput unit 130 is an input unit for inputting image data to the compression processing unit 114. The input image data is supplied to the YCbCr processing unit 131. If the input image data is an RGB primary color signal, the YCbCr processing circuit converts it into a YCbCr luminance color difference signal using a predetermined conversion formula based on colorimetric information added to the image data.
入力部130は、圧縮処理部114へ画像データを入力するための入力部である。入力された画像データは、YCbCr処理部131供給される。YCbCr処理回路は、入力画像データがRGB原色信号であれば、その画像データに付与されたカラリメトリ情報に基づき所定の変換式を使用してYCbCr輝度色差信号に変換する。 FIG. 2 is a block diagram illustrating an example of the configuration of the
The
カラリメトリ情報が付与されていない場合は、画像信号の水平や垂直周波数などのフォーマット情報から、例えばHD画像であればITU-R BT709(以降BT709と略す)、SD映像であればITU-R BT601(以降BT601と略す)規定のカラリメトリ、変換式に基づいて変換処理を行う。入力画像データがYCbCr輝度色差信号である場合はこの変換処理は不要である。
If colorimetry information is not given, format information such as the horizontal and vertical frequencies of the image signal is used, for example, ITU-R BT709 (hereinafter abbreviated as BT709) for HD images and ITU-R BT601 (hereinafter abbreviated as SD video). (Hereafter, abbreviated as BT601) Conversion processing is performed based on the prescribed colorimetry and conversion formula. If the input image data is a YCbCr luminance color difference signal, this conversion processing is not necessary.
さらに、CbとCrの可変域を制限して後の圧縮処理の効率を上げるために、下記の色差信号処理も行う。図3は、HD画像に適用されるBT709規定のYCb空間の説明図である。YとCbの値は正規化された値と8bit量子化された値を併記しているが、これらの定義はBT709規格に詳述されている。通常は正規化Yが0~1、正規化Cbが-0.5~+0.5の範囲702が使われ、正規化Yが1を超える領域703をOver White、0未満をOver Blackと呼んでいる。点708と709、710、711で囲まれた平行四辺形状の領域707は、RGB原色信号に変換した際に、各正規化R,G,Bの値が0~1になる領域を示している。領域707の外は各正規化R,G,Bのいずれかが負になる領域を示しており、通常は使われない領域である。しかしながら、通常は使われない領域707の外側領域と領域705と領域706を利用して色表現範囲を広げたIEC 61996 2-4で定義されたxvYCC規格も知られている。
Furthermore, in order to limit the variable range of Cb and Cr and increase the efficiency of the subsequent compression processing, the following color difference signal processing is also performed. FIG. 3 is an explanatory diagram of the YCb space defined by BT709 applied to HD images. The values of Y and Cb indicate both the normalized value and the 8-bit quantized value, and these definitions are described in detail in the BT709 standard. Usually, the range 702 in which the normalization Y is 0 to 1 and the normalization Cb is -0.5 to +0.5 is used. The area 703 in which the normalization Y exceeds 1 is called Over White, and the area less than 0 is called Over Black. Yes. A parallelogram-shaped region 707 surrounded by points 708 and 709, 710, and 711 indicates a region in which the values of the normalized R, G, and B are 0 to 1 when converted into RGB primary color signals. . Outside the area 707, an area in which any one of the normalizations R, G, and B is negative is an area that is not normally used. However, the xvYCC standard defined in IEC 61996 2-4, which expands the color expression range by using the outside area 707, the area 705, and the area 706 that are not normally used, is also known.
図4は、平行四辺形状の領域707を座標変換によって左右の辺をY軸と平行に変換した平行四辺形状の領域717を示している。図中の矢印は各座標の変換による移動を示している。座標変換によって座標が移動しない中心として正規化(Cb,Y)=(0,0.5)をとっている。座標変換を簡易化するため、Yは変換せず、Cb値のみ移動させている。Yを変換しないことによって、YCr空間でも同様な座標変換によるCr範囲縮小を施す場合でも、同じYを共通に使える利点もある。
FIG. 4 shows a parallelogram-shaped area 717 obtained by converting the parallelogram-shaped area 707 by converting the left and right sides in parallel with the Y-axis. The arrows in the figure indicate movements by converting each coordinate. Normalization (Cb, Y) = (0, 0.5) is taken as the center where the coordinates do not move by coordinate transformation. In order to simplify the coordinate conversion, Y is not converted and only the Cb value is moved. By not converting Y, there is also an advantage that the same Y can be used in common even in the case of performing Cr range reduction by similar coordinate conversion in the YCr space.
この座標変換の結果、平行四辺形状の領域717の正規化Cb範囲714は-0.2695~0.2695と、変換前の平行四辺形状の領域707の正規化Cb範囲-0.5~0.5に比べて約54%に低減できる。正規化Cb範囲を狭めることで、後段の圧縮符号化を効率よく働かせることができる。
As a result of this coordinate conversion, the normalized Cb range 714 of the parallelogram-shaped region 717 is -0.2695 to 0.2695, and the normalized Cb range of the parallelogram-shaped region 707 before conversion is -0.5 to 0. Compared to 5, it can be reduced to about 54%. By narrowing the normalized Cb range, the subsequent compression encoding can be efficiently performed.
尚、図4からもわかるように、xvYCC規格で定義される領域は全てではないが、平行四辺形状の領域707の周囲領域は、座標変換後の平行四辺形状の領域717の周囲に維持されており、xvYCC規格で定義されるCb領域の伝送ができる。この場合、正規化Cbの伝送範囲としては、視感効果の高い平行四辺形状の領域717の周囲に、領域705と領域706の幅に相当する範囲を付加することが広色再現範囲実現に効果であり、例えば(-0.2695-0.067=-0.3365)~(0.2695+0.063=0.3325)となる。この場合でも、Cb可変範囲-0.567~0.563に比べて約59%の範囲に低減できる。
As can be seen from FIG. 4, the area defined by the xvYCC standard is not all, but the area around the parallelogram area 707 is maintained around the parallelogram area 717 after coordinate conversion. Therefore, transmission of the Cb area defined by the xvYCC standard is possible. In this case, as a transmission range of the normalized Cb, it is effective to realize a wide color reproduction range by adding a range corresponding to the width of the region 705 and the region 706 around the parallelogram region 717 having a high visual effect. For example, (−0.2695−0.067 = −0.3365) to (0.2695 + 0.063 = 0.3325). Even in this case, it can be reduced to a range of about 59% compared to the Cb variable range -0.567 to 0.563.
図5は8bits量子化時の座標変換演算をより簡易化したYCb空間の座標変換の例を示している。座標の移動しない中心点は量子化(Cb,Y)=(128,128)であり、正規化(Cb,Y)=(0,0.5114)の点に相当する。さらに、量子化Cbの移動量は(量子化Y-128)/2として、特に2進数での演算量を削減している。従って座標変換後の量子化Cbは、(量子化Cb+(量子化Y-128)/2)となり、小数部分を切り捨てとする。座標変換後の平行四辺形状の領域727の正規化Cb範囲が-0.2861~0.2861範囲と、図4に実施例よりは縮小効果は若干減るがそれでも約57%に低減できる。尚、12bit量子化とした場合は、量子化Cbの移動量は(量子化Y-2048)/2とすればよい。
FIG. 5 shows an example of coordinate conversion in the YCb space in which the coordinate conversion calculation at the time of 8-bit quantization is further simplified. The center point where the coordinate does not move is quantization (Cb, Y) = (128, 128), which corresponds to the point of normalization (Cb, Y) = (0, 0.5114). Further, the amount of movement of the quantized Cb is (quantized Y−128) / 2, and the amount of computation in binary numbers is particularly reduced. Therefore, the quantized Cb after the coordinate conversion is (quantized Cb + (quantized Y−128) / 2), and the decimal part is rounded down. The normalized Cb range of the parallelogram-shaped area 727 after the coordinate conversion is in the range of −0.2861 to 0.2861, and the reduction effect is slightly reduced as compared with the embodiment in FIG. 4, but it can still be reduced to about 57%. In the case of 12-bit quantization, the amount of movement of the quantized Cb may be (quantized Y-2048) / 2.
図5で示した簡易化計算方法の利点は、座標変換演算量の低減だけでなく、移動量が整数である為に座標変換後に元の座標への復元演算において完全に復元できる可逆演算であることである。移動量を整数で定義しておけば、可逆演算である利点を実現できる。すなわち、上記の移動量(量子化Y-128)/2だけでなく、(量子化Y-128)×5/8などのような数式でもその結果を整数として処理すればよい。
The advantage of the simplified calculation method shown in FIG. 5 is not only the reduction of the coordinate conversion calculation amount but also the reversible calculation that can be completely restored in the restoration calculation to the original coordinates after the coordinate conversion because the movement amount is an integer. That is. If the movement amount is defined as an integer, the advantage of reversible operation can be realized. That is, not only the movement amount (quantization Y−128) / 2 but also a mathematical expression such as (quantization Y−128) × 5/8 may be processed as an integer.
図6は、BT709規格のYCb空間とYCr空間において、それぞれCbとCrの範囲を縮小する座標変換前後の、正規化RGB原色信号が0~1の範囲になる平行四辺形状の領域の頂点座標を示したものである。図4で説明した最小Cb範囲法と図5で説明した簡易処理法による座標変換結果を対比させている。各欄の1行目が正規化、2行目が8bit量子化、3行目が12bit量子化、4行目が12bit量子化、5行目が16bit量子化した座標をそれぞれ記載している。これらの変換結果によって、Crについても、最小Cr範囲が約64%、簡易処理のCr範囲が約72%に圧縮できることがわかる。
FIG. 6 shows the vertex coordinates of a parallelogram-shaped region in which the normalized RGB primary color signal is in the range of 0 to 1 before and after coordinate conversion to reduce the range of Cb and Cr in the BT709 standard YCb space and YCr space, respectively. It is shown. The coordinate conversion results obtained by the minimum Cb range method described in FIG. 4 and the simple processing method described in FIG. 5 are compared. In each column, the first line is normalized, the second line is 8-bit quantization, the third line is 12-bit quantization, the fourth line is 12-bit quantization, and the fifth line is 16-bit quantized coordinates. From these conversion results, it can be seen that Cr can be compressed to a minimum Cr range of about 64% and a simple processing Cr range of about 72%.
以上でその動作を説明してきたYCbCr処理部131の出力が、間引き処理部132、水平圧縮部133および垂直圧縮部134に供給される。
The output of the YCbCr processing unit 131 whose operation has been described above is supplied to the thinning processing unit 132, the horizontal compression unit 133, and the vertical compression unit 134.
図7と図8は、入力部130に入力される画像データの一例を示す図である。水平方向にn画素、垂直方向にmラインの輝度信号を示す。色差信号は、444フォーマットの場合、輝度信号と同じフォーマットとなる。画素数n及びライン数mの一例としては、n=1920、m=1080のいわゆるフルHD画像やn=3840、m=2160のいわゆる4k2k画像などがある。
7 and 8 are diagrams illustrating an example of image data input to the input unit 130. FIG. A luminance signal of n pixels in the horizontal direction and m lines in the vertical direction is shown. In the case of the 444 format, the color difference signal has the same format as the luminance signal. As an example of the number of pixels n and the number of lines m, there are so-called full HD images with n = 1920 and m = 1080 and so-called 4k2k images with n = 3840 and m = 2160.
ここで、圧縮部114で圧縮する画像データの単位ブロックを、水平方向l画素と垂直方向k画素とする。図7において、501および502はl=32、k=1の例を示し、同じライン内で連続した32画素のデータで構成する。この画像データの単位を水平圧縮部133が圧縮する。
Here, the unit block of the image data compressed by the compression unit 114 is assumed to be 1 pixel in the horizontal direction and k pixels in the vertical direction. In FIG. 7, reference numerals 501 and 502 denote examples where l = 32 and k = 1, and are constituted by data of 32 pixels continuous in the same line. The horizontal compression unit 133 compresses this image data unit.
図8において、503および504は、l=16、k=2の例を示し、上下する2ライン間で連続する16画素のデータで構成する。この画像データの単位ブロックを垂直圧縮部134が圧縮する。すなわち、垂直方向k画素数(ライン数)が異なるk1とk2(k1<k2)の画像データ単位ブロックを用意し、水平圧縮部133がk1の画像データ単位ブロックを、垂直圧縮部134がk2の画像データ単位ブロックを圧縮する。
8, 503 and 504 are examples of l = 16 and k = 2, and are constituted by data of 16 pixels continuous between two lines that move up and down. The vertical compression unit 134 compresses the unit block of the image data. That is, k1 and k2 (k1 <k2) image data unit blocks having different numbers of k pixels (lines) in the vertical direction are prepared, the horizontal compression unit 133 sets the image data unit block of k1, and the vertical compression unit 134 sets k2. Compress the image data unit block.
kが大きいとラインメモリ量が増えて回路コストが上がり、処理時間が増加して映像表示がと遅延する原因となる。以下の説明ではk1=1、k2=2としてコストと処理時間を抑える例を用いる。水平圧縮部133と垂直圧縮部134に加えてさらに異なるk3を持つ圧縮部を追加し、3個以上の圧縮部の出力を選択して用い、さらに圧縮効率を上げることもできる。
If k is large, the amount of line memory increases, the circuit cost increases, the processing time increases, and the video display is delayed. In the following description, an example in which cost and processing time are suppressed by using k1 = 1 and k2 = 2 will be used. In addition to the horizontal compression unit 133 and the vertical compression unit 134, a compression unit having a different k3 can be added, and the output of three or more compression units can be selected and used to further increase the compression efficiency.
入力された色差信号が422フォーマットの場合は、色差信号のCb成分とCr成分が1画素ごとに入れ子のデータとなる。例えば4k2k画像においてCb成分とCr成分を合わせてn=3840、m=2160の画像データとして扱ってもよい。一般にCb成分同士、Cr成分同士の相関が高いので、Cb成分とCr成分を分けて、n=1920、m=2160として扱うことにより、同一成分のみで圧縮する画像データの単位ブロックとすることによって、圧縮効率を高めることができる。
When the input color difference signal is in the 422 format, the Cb component and Cr component of the color difference signal become nested data for each pixel. For example, in a 4k2k image, the Cb component and the Cr component may be combined and handled as image data of n = 3840 and m = 2160. In general, since the correlation between Cb components and Cr components is high, the Cb component and the Cr component are separated and handled as n = 1920 and m = 2160, thereby forming a unit block of image data compressed with only the same component. , Compression efficiency can be increased.
入力された色差信号が420フォーマットの場合は、色差信号のCb成分とCr成分がY信号4画素に対して1画素分のデータとなる。例えば4k2k画像においてCb成分とCr成分を合わせてn=1920、m=2160の画像データとして扱ってもよい。一般にCb成分同士、Cr成分同士の相関が高いので、Cb成分とCr成分を分けて、n=1920、m=1080として扱うことにより、同一成分のみで圧縮する画像データの単位とすることによって、圧縮効率を高めることができる。
When the input color difference signal is in the 420 format, the Cb component and Cr component of the color difference signal are data for one pixel with respect to four Y signal pixels. For example, in a 4k2k image, the Cb component and the Cr component may be combined and handled as image data of n = 1920 and m = 2160. In general, since the correlation between the Cb component and the Cr component is high, the Cb component and the Cr component are separated and handled as n = 1920 and m = 1080, so that the unit of image data to be compressed only with the same component is obtained. Compression efficiency can be increased.
水平圧縮部133と垂直圧縮部134は、444フォーマットや422フォーマットの色差信号が入力された際に、所定の圧縮方式では所定の圧縮率に到達しない場合は422フォーマットや420フォーマットに間引いて圧縮効率を高めてもよい。
When the 444 format or 422 format color difference signal is input, the horizontal compression unit 133 and the vertical compression unit 134 reduce the compression efficiency to the 422 format or 420 format if the predetermined compression rate does not reach the predetermined compression rate. May be increased.
420処理部132は、444フォーマットや422フォーマットが入力された場合、所定の圧縮率に応じて422フォーマットや420フォーマットへ間引き処理を行う。
When the 444 format or the 422 format is input, the 420 processing unit 132 performs a thinning process to the 422 format or the 420 format according to a predetermined compression rate.
水平圧縮部133は、水平方向の複数の画像データに対して、圧縮を行う圧縮回路により構成される。圧縮方式の一例としては、水平方向にWavelet変換を演算し、その演算結果に対して、符号化した圧縮方式等で構成する。圧縮方式として、アダマール変換や、ランレングス符号化、ハフマン符号化、差分符号化などを適用してもよい。
The horizontal compression unit 133 includes a compression circuit that compresses a plurality of image data in the horizontal direction. As an example of the compression method, Wavelet transform is calculated in the horizontal direction, and the calculation result is configured by an encoded compression method or the like. As a compression method, Hadamard transform, run length coding, Huffman coding, differential coding, or the like may be applied.
垂直圧縮部134は、垂直方向の複数の画像データに対して、圧縮を行う圧縮回路により構成される。圧縮方式の一例としては、垂直方向に2ライン分、水平方向に16画素分の画像データを圧縮する画像データの単位ブロックとして、まず垂直方向に対し差分をとり、次に水平方向に差分をとる。その結果に対して符号化する圧縮方式等で構成する。
The vertical compression unit 134 includes a compression circuit that compresses a plurality of pieces of image data in the vertical direction. As an example of the compression method, as a unit block of image data for compressing image data of 2 lines in the vertical direction and 16 pixels in the horizontal direction, a difference is first taken in the vertical direction, and then a difference is taken in the horizontal direction. . The result is configured by a compression method for encoding the result.
間引き処理部132、水平圧縮部133および垂直圧縮部134の圧縮率は、元の非圧縮画像データに対して、例えば圧縮後の画像データを2/3や、1/2、1/3などの所定の圧縮率で圧縮可能な圧縮方式であればよい(しかし、本発明はこの圧縮率に限定されない)。この所定の圧縮率は、元の非圧縮画像データとケーブルで伝送できるデータ量の比から決まる。
The compression rate of the thinning processing unit 132, the horizontal compression unit 133, and the vertical compression unit 134 is, for example, 2/3, 1/2, or 1/3 of the compressed image data with respect to the original uncompressed image data. Any compression method that can compress at a predetermined compression rate may be used (however, the present invention is not limited to this compression rate). This predetermined compression rate is determined from the ratio of the original uncompressed image data and the amount of data that can be transmitted by the cable.
水平圧縮部133、垂直圧縮部134の圧縮する画像データの単位は、圧縮処理による遅延量を少なくする画素数で構成する。圧縮する画像データの単位の一例として、32画素を例にとって説明したが、64画素や128画素などの単位としてもよい。
The unit of image data to be compressed by the horizontal compression unit 133 and the vertical compression unit 134 is composed of the number of pixels that reduces the delay amount due to the compression processing. As an example of the unit of image data to be compressed, 32 pixels have been described as an example. However, a unit of 64 pixels or 128 pixels may be used.
選択部135は、間引き処理部132と、水平圧縮部133もしくは、垂直圧縮部134の出力の内、所定の圧縮率を満たしかつ画質指数の高いものを選択し、エラー訂正符号生成部136に供給する。画質指数とは、例えば、圧縮画像データを復元した画像データと圧縮前の画像データの差異が小さいほど良い値を示す指数であればよい。圧縮ロスが発生せず、可逆符号化ができた場合が最高値である。
The selection unit 135 selects the output of the thinning processing unit 132, the horizontal compression unit 133, or the vertical compression unit 134 that satisfies a predetermined compression ratio and has a high image quality index, and supplies the selected one to the error correction code generation unit 136. To do. The image quality index may be an index that indicates a better value as the difference between the image data restored from the compressed image data and the image data before compression is smaller, for example. The highest value is when lossless compression does not occur and lossless encoding is possible.
画質指数の計算を簡略化するため、圧縮方式別に画質指数の値を用意しておいてもよい。444フォーマットのまま圧縮して圧縮ロスが発生する場合よりも、422フォーマットへ間引き後に可逆符号化できた場合を高い画質指数に定義してもよい。圧縮によって逆にデータ量が増加してしまう場合は、422フォーマットや420フォーマットに間引いて圧縮や、量子化bit数を減らすことによって所定の圧縮率を達成させるが、その間引きや量子bit数減に応じた画質指数を設定する。
In order to simplify the calculation of the image quality index, an image quality index value may be prepared for each compression method. A case where lossless encoding can be performed after thinning out to 422 format may be defined as a higher image quality index than when compression loss occurs when compression is performed in the 444 format. Conversely, if the amount of data increases due to compression, a predetermined compression ratio can be achieved by thinning out to 422 format or 420 format and reducing the number of quantization bits. Set the appropriate image quality index.
エラー訂正符号生成部136は、水平圧縮部133や垂直圧縮部134で圧縮された画像データの単位ごとにエラー訂正符号を演算し、圧縮された画像データに付加して、圧縮符号情報付加部137に出力する。エラー訂正の方式の一つとして、CRC(Cyclic Redundancy Check)方式やパリティチェック方式などがある。
The error correction code generation unit 136 calculates an error correction code for each unit of image data compressed by the horizontal compression unit 133 or the vertical compression unit 134, adds the error correction code to the compressed image data, and a compression code information addition unit 137. Output to. One of error correction methods is a CRC (Cyclic Redundancy Check) method or a parity check method.
間引き処理部132で間引かれた画像データはエラー訂正符号を付与しなくてもよい。間引き処理の場合、エラーが生じても、該当画素と隣接した間引き画素のエラーであり、画面全体からみると画質への影響が少ないこと、さらにはケーブルのデータ伝送容量が限られている場合、エラー訂正符号を付与すると画素又は階調をより多く間引くことになり、画面全体の画質劣化につながるためである。伝送容量に余裕があれば間引き処理された画像データにもエラー訂正符号を付加して、エラー耐性を向上させてもよい。間引き画像データへのエラー訂正付加有無のメタデータを合せて伝送することによって、受信側でエラー訂正処理をするかどうかを判別させてもよい。また、間引き画像データと圧縮画像データでエラー訂正処理を変えてエラー耐性レベルを変えてもよいし、どのエラー訂正符号を付与したかを示す情報をメタデータとして加えてもよい。
The image data thinned out by the thinning-out processing unit 132 may not be given an error correction code. In the case of thinning processing, even if an error occurs, it is an error of the thinning pixel adjacent to the corresponding pixel, there is little influence on the image quality when viewed from the whole screen, and furthermore, when the data transmission capacity of the cable is limited, This is because when an error correction code is added, more pixels or gradations are thinned out, leading to image quality deterioration of the entire screen. If the transmission capacity is sufficient, an error correction code may be added to the thinned-out image data to improve error tolerance. Whether or not error correction processing is to be performed may be determined on the reception side by transmitting together with metadata indicating whether or not error correction is added to the thinned image data. Further, the error resistance level may be changed by changing the error correction processing between the thinned image data and the compressed image data, and information indicating which error correction code is given may be added as metadata.
圧縮符号情報付加部137は、エラー訂正符号付の圧縮画像データを後述する図11の有効期間406に出力し、その直前の水平ブランキング期間404に圧縮方式を示す圧縮符号情報を出力する。また、別の方式としては、1ライン分のエラー訂正符号付の圧縮画像データ、及び圧縮方式を示す圧縮符号情報を、1ライン分の有効期間406内に伝送して伝送量の増大を図ってもよい。また、エラー訂正符号を水平ブランキング期間404に出力してエラー訂正の信頼性を向上させてもよい。
The compression code information adding unit 137 outputs the compressed image data with the error correction code in the effective period 406 of FIG. 11 described later, and outputs the compression code information indicating the compression method in the immediately preceding horizontal blanking period 404. As another method, compressed image data with an error correction code for one line and compressed code information indicating the compression method are transmitted within an effective period 406 for one line to increase the transmission amount. Also good. In addition, the error correction code may be output in the horizontal blanking period 404 to improve the error correction reliability.
出力部138からは、圧縮符号情報付加化部137からのエラー訂正符号付圧縮画像データと、圧縮方式を示す圧縮符号情報が出力される。図示していないが、図2の各ブロックは、図1の制御部110の制御信号に従ってその動作を制御される。
The output unit 138 outputs the compressed image data with error correction code from the compression code information adding unit 137 and the compression code information indicating the compression method. Although not shown, the operation of each block in FIG. 2 is controlled according to the control signal of the control unit 110 in FIG.
図9は、エラー訂正符号生成部136の構成の一例を示すブロック図である。入力部150には圧縮された画像データが入力される。圧縮された画像データは、保持部155とエラー訂正符号演算部153に入力される。エラー訂正符号演算部153は、入力された圧縮された画像データに対して、生成多項式で巡回演算をする。生成多項式の一例としては、
FIG. 9 is a block diagram illustrating an example of the configuration of the error correction code generation unit 136. The input unit 150 receives compressed image data. The compressed image data is input to the holding unit 155 and the error correction code calculation unit 153. The error correction code calculation unit 153 performs a cyclic calculation with the generator polynomial on the input compressed image data. An example of a generator polynomial is
(数1)G(X)=X16+X12+X5+1がある。この生成多項式は、入力された画像データ中の各ビットにつき排他的論理和をとり巡回演算する。演算の単位は圧縮する画像データの単位とする。
(Expression 1) There is G (X) = X 16 + X 12 + X 5 +1. This generator polynomial takes an exclusive OR for each bit in the input image data and performs a cyclic operation. The unit of calculation is a unit of image data to be compressed.
入力部151は、圧縮された画像データが入力されている期間を示す信号が入力され、タイミング生成部154に供給される。タイミング生成部154は、圧縮された画像データの有効期間をカウントし、圧縮する画像データの単位ブロック分の演算が処理されたことを示す信号をエラー訂正演算結果出力タイミング信号としてデータ保持部155に出力する。
The input unit 151 receives a signal indicating a period during which the compressed image data is input, and supplies the signal to the timing generation unit 154. The timing generation unit 154 counts the effective period of the compressed image data, and outputs a signal indicating that the calculation for the unit block of the image data to be compressed has been processed to the data holding unit 155 as an error correction calculation result output timing signal. Output.
さらに、タイミング生成回路154は、圧縮された画像データの入力期間を示すタイミング信号や、圧縮された画像データとエラー訂正符号演算結果を出力するタイミング信号なども合せてデータ保持部155び出力する。
Furthermore, the timing generation circuit 154 also outputs a timing signal indicating the input period of the compressed image data, a timing signal for outputting the compressed image data and the error correction code calculation result, and the like to the data holding unit 155.
データ保持部155は、タイミング生成部154の示すタイミングに従って、エラー訂正符号演算部153の演算結果と圧縮された画像データを例えばメモリやフリップフロップ、遅延素子等によって一時的に記憶し、それらを順次、出力部157へ出力する。
The data holding unit 155 temporarily stores the calculation result of the error correction code calculation unit 153 and the compressed image data by, for example, a memory, flip-flop, delay element, or the like according to the timing indicated by the timing generation unit 154, and sequentially stores them. To the output unit 157.
図10は、データ伝送部115の構成の一例を示すブロック図である。
入力部170は、圧縮された画像データをシリアライザ174に出力する。また入力部172は、画像データのクロックが入力され、PLL173と出力部177に出力する。画像データのクロックには、非圧縮画像データの標準的なタイミングフォーマットで使われる画素クロックに同期したクロックを用いる。 FIG. 10 is a block diagram illustrating an example of the configuration of thedata transmission unit 115.
Theinput unit 170 outputs the compressed image data to the serializer 174. The input unit 172 receives a clock of image data and outputs the clock to the PLL 173 and the output unit 177. As the clock of the image data, a clock synchronized with the pixel clock used in the standard timing format of the uncompressed image data is used.
入力部170は、圧縮された画像データをシリアライザ174に出力する。また入力部172は、画像データのクロックが入力され、PLL173と出力部177に出力する。画像データのクロックには、非圧縮画像データの標準的なタイミングフォーマットで使われる画素クロックに同期したクロックを用いる。 FIG. 10 is a block diagram illustrating an example of the configuration of the
The
例えば、非圧縮画像データの画素クロックを2分周したクロックを用いてもよい。この場合、該非圧縮画像データが12bitsの量子化画像データである場合は、クロックで1/2、量子化bit数で8/12となるので、前記した所定の圧縮率は1/3以下を設定する必要がある。
For example, a clock obtained by dividing the pixel clock of uncompressed image data by two may be used. In this case, if the uncompressed image data is 12-bit quantized image data, the clock is 1/2 and the number of quantization bits is 8/12, so the predetermined compression rate is set to 1/3 or less. There is a need to.
クロックは2分周以外に、3/4逓倍や2/3逓倍としてもよい。非圧縮画像データの画素クロックに同期したクロックを圧縮画像データの伝送に使うことにより、受信側で非圧縮画像データを復元する場合に、伝送クロックの2倍や4/3倍、3/2倍などの逓倍したクロックを画素クロックとして使うことにより、復元データのジッタを最小化できる利点がある。
The clock may be multiplied by 3/4 or 2/3 in addition to dividing by 2. By using a clock synchronized with the pixel clock of the uncompressed image data for transmission of the compressed image data, when restoring the uncompressed image data on the receiving side, it is 2 times, 4/3 times, 3/2 times the transmission clock. By using the multiplied clock as the pixel clock, there is an advantage that the jitter of the restored data can be minimized.
PLL173は、入力されたクロックに対して逓倍化もしくは分周したクロックを生成する。逓倍化の一例としては、入力されたクロックの周波数に対して、5倍や10倍などがある。PLL173で生成するクロックは、1種類のクロックとしてもよいし、2種類のクロックとしてもよい。1種類のクロックの例としては、入力されたクロックの10逓倍がある。また2種類のクロックの例としては、データ伝送量を優先した第1のクロック速度と、エラーの発生頻度を下げることを優先した第1のクロック速度より遅い速度の第2のクロック速度のクロックがある。速度の一例としては前記第1のクロック速度を入力クロックの10逓倍、第2のクロック速度を入力クロックの5逓倍などがある。
The PLL 173 generates a clock obtained by multiplying or dividing the input clock. Examples of multiplication include 5 times and 10 times the frequency of the input clock. The clock generated by the PLL 173 may be one type of clock or two types of clocks. An example of one type of clock is 10 times the input clock. Examples of the two types of clocks include a first clock speed that prioritizes the amount of data transmission and a second clock speed that is slower than the first clock speed that prioritizes lowering the frequency of error occurrence. is there. As an example of the speed, the first clock speed is multiplied by 10 of the input clock, and the second clock speed is multiplied by 5 of the input clock.
PLL173で生成した逓倍したクロックは、シリアライザ174に出力される。シリアライザ174は、入力されたYCbCr輝度色差信号の圧縮された画像データを、10逓倍したクロックで1bitずつシリアル化し、レベル変換部175に出力する。入力部172に入力された1クロックに対して、入力部170に入力されるデータが8bitある場合、該8bitデータを10bitにマッピングしてシリアル化したビットストリームのDC成分を抑圧するTMDS伝送方法などを用いてもよい。出力部176に接続されるケーブルが3本ある場合は、各ケーブル毎に上記シリアル処理を行うことによって、入力クロック当り24bitの圧縮された画像データを送ることができる。
The multiplied clock generated by the PLL 173 is output to the serializer 174. The serializer 174 serializes the compressed image data of the input YCbCr luminance color difference signal bit by bit with a clock multiplied by 10 and outputs the serialized data to the level conversion unit 175. TMDS transmission method that suppresses the DC component of the bit stream that is serialized by mapping the 8-bit data to 10 bits when there is 8 bits of data input to the input unit 170 for one clock input to the input unit 172 May be used. When there are three cables connected to the output unit 176, 24 bits of compressed image data per input clock can be sent by performing the serial processing for each cable.
レベル変換部175は、ケーブル伝送に適した形式の信号を出力部176経由で出力する。
The level conversion unit 175 outputs a signal in a format suitable for cable transmission via the output unit 176.
図11は、1フレーム期間の画像データが重畳される有効領域と画像データが重畳されないブランキング期間を示す図である。
FIG. 11 is a diagram illustrating an effective region in which image data of one frame period is superimposed and a blanking period in which image data is not superimposed.
400で示す領域が垂直期間を示し、垂直期間400は、垂直ブランキング期間401と垂直有効期間402から構成される。VSYNC信号は、垂直ブランキング期間401の先頭から規定されたライン数の間を1とし、その他の垂直ブランキング期間と垂直有効期間402の間は0とした1bitの信号である。規定されたライン数の一例としては、4ラインなどがある。
A region indicated by 400 indicates a vertical period, and the vertical period 400 includes a vertical blanking period 401 and a vertical effective period 402. The VSYNC signal is a 1-bit signal in which 1 is set between the number of lines defined from the top of the vertical blanking period 401 and 0 is set between the other vertical blanking periods and the vertical effective period 402. An example of the prescribed number of lines is 4 lines.
403で示す領域が水平期間を示し、水平期間403は、水平ブランキング期間404と水平有効期間405から構成される。HSYNC信号は、水平ブランキング期間404の先頭から規定された画素数の間を1とし、その他の水平ブランキング期間と水平有効期間405の間は0とした1bitの信号である。規定された画素数の一例としては、40画素がある。
403 indicates a horizontal period, and the horizontal period 403 includes a horizontal blanking period 404 and a horizontal effective period 405. The HSYNC signal is a 1-bit signal in which 1 is set between the number of pixels defined from the head of the horizontal blanking period 404 and 0 is set between the other horizontal blanking periods and the horizontal effective period 405. An example of the prescribed number of pixels is 40 pixels.
有効期間406は、垂直有効期間402と水平有効期間405の期間に囲まれた領域を示し、この期間に画像データが割り当てられる。また、ブランキング期間407は、垂直ブランキング期間401と水平ブランキング期間404の期間に囲まれた領域である。
The effective period 406 indicates an area surrounded by a vertical effective period 402 and a horizontal effective period 405, and image data is allocated to this period. The blanking period 407 is an area surrounded by a vertical blanking period 401 and a horizontal blanking period 404.
本実施例においては、上記構成において、垂直ブランキング期間に圧縮伝送方式を示す情報を送信し、有効期間406に圧縮した画像データを送信し、そのラインの水平ブランキング期間404に各圧縮符号化の画像データ単位ブロック毎に選択された圧縮符号情報を送信する。ブランキング期間407は、音声データやその他の付属データをパケット化したデータを伝送している。
In this embodiment, in the above configuration, information indicating the compression transmission method is transmitted in the vertical blanking period, compressed image data is transmitted in the effective period 406, and each compression encoding is performed in the horizontal blanking period 404 of the line. The compression code information selected for each image data unit block is transmitted. In the blanking period 407, data obtained by packetizing audio data and other attached data is transmitted.
この音声データ等のパケットをブランキング期間407で信頼性のあるパケットを送る方法は、例えば特表2005-514873号公報に開示されている。
A method for sending a packet of voice data or the like in the blanking period 407 is disclosed in, for example, Japanese translations of PCT publication No. 2005-514873.
この構成で、ブランキング期間のパケットのデータに対しては誤り訂正符号が入っているため、伝送路で発生したエラーに対して補正ができエラー耐性が強くなる。また、ブランキング期間のパケットのデータ伝送用のデータは、物理的に異なる2つのチャンネルに伝送する構成とし、一定時間ごとに伝送するチャンネルを切り替えているため、片側のチャンネルでバースト的に発生したエラーに対して、他方のチャンネルが影響されないため、データエラーの補正を行うことができる。エラーの訂正率は、水平有効期間が10-9に対して、水平ブランキング期間は10-14の改善効果がある。
With this configuration, since the error correction code is included in the packet data in the blanking period, it is possible to correct an error occurring in the transmission path, and the error resistance is enhanced. In addition, the data for data transmission of the packet in the blanking period is configured to be transmitted to two physically different channels, and the channel to be transmitted is switched every certain time. Since the other channel is not affected by the error, the data error can be corrected. The error correction rate has an improvement effect of 10 -14 in the horizontal blanking period compared to 10 -9 in the horizontal effective period.
この先行技術文献では、有効期間406では1クロック当り24bitの画像データを伝送し、ブランキング期間407では3byteのヘッダと28byteのデータで構成される1パケットを32クロック期間で伝送している。
In this prior art document, 24-bit image data per clock is transmitted in the valid period 406, and one packet composed of a 3-byte header and 28-byte data is transmitted in a 32-clock period in the blanking period 407.
例えば、水平3840有効画素、垂直2160有効ライン、YCbCr輝度色差信号各12bitを444フォーマット、フレーム周波数60Hzで伝送する場合、891MHzと非常に高いクロック周波数が必要となる。高クロック周波数は送受信部のコストがかかるだけでなく、画像を安定に送ることができるケーブル長も短くなり使い勝手が悪くなる。
For example, when transmitting 12 bits each of horizontal 3840 effective pixels, vertical 2160 effective lines, and YCbCr luminance / chrominance signal in 444 format at a frame frequency of 60 Hz, a very high clock frequency of 891 MHz is required. The high clock frequency not only increases the cost of the transmission / reception unit, but also shortens the cable length capable of stably sending an image, resulting in poor usability.
本実施例によれば、2/3の圧縮で594MHz、1/3の圧縮で297MHzと、実用的なクロック周波数に近づけることができる。また、YCbCr輝度色差信号各8bitの444フォーマットや、各12bitの422フォーマットにおいては、1/2の圧縮で297MHzの実用的なクロック周波数にすることができる。
According to the present embodiment, it is possible to approach a practical clock frequency of 594 MHz with 2/3 compression and 297 MHz with 1/3 compression. In addition, in the YCbCr luminance / chrominance signal 8 bit 444 format or 12 bit 422 format, a practical clock frequency of 297 MHz can be obtained by 1/2 compression.
水平3840有効画素、垂直2160有効ラインにおいて、例えば、水平560ブランキング画素、垂直90ブランキングラインになる。以下、水平3840有効画素、垂直2160有効ライン、YCbCr輝度色差信号各12bitを444フォーマット、フレーム周波数60Hzの映像信号を1/3に圧縮する場合を例にとって説明する。
In horizontal 3840 effective pixels and vertical 2160 effective lines, for example, horizontal 560 blanking pixels and vertical 90 blanking lines. In the following, an example in which a horizontal 3840 effective pixel, a vertical 2160 effective line, a YCbCr luminance / chrominance signal 12 bits each in 444 format, and a video signal having a frame frequency of 60 Hz is compressed to 1/3 will be described.
圧縮後のクロック周波数は297MHzで、水平1940有効画素、垂直2160有効ライン、水平280ブランキング画素、垂直90ブランキングラインという条件で伝送すると、受信側で2倍の周波数である原クロック594MHzを安定に再生しやすい。水平有効画素期間では、1画素当りYCbCr輝度色差各12bit計36bitを1/3の12bitに圧縮すれば、圧縮後の1画素当り24bit伝送できるので原画素の2画素分の伝送が可能となり、圧縮後のクロック周波数は原クロックの半分となる。
The compressed clock frequency is 297 MHz, and if the transmission is performed under the conditions of horizontal 1940 effective pixels, vertical 2160 effective lines, horizontal 280 blanking pixels, and vertical 90 blanking lines, the original clock 594 MHz, which is twice the frequency, is stabilized on the receiving side. Easy to play. In the horizontal effective pixel period, by compressing each 12-bit YCbCr luminance color difference 12 bits total 36 bits to 1/3 of 12 bits, it is possible to transmit 24 bits per compressed pixel, so transmission of 2 pixels of the original pixel is possible. The later clock frequency is half that of the original clock.
32クロック当り1パケットが伝送できるので、圧縮後の水平280ブランキング画素では最大8パケットの伝送が可能となる。一方、音声データは1パケット当り24bit×8chの伝送ができる。画像データの水平周波数は135kHz(=60Hz×(2160+90))なので、192kHzサンプルの24bitリニアPCM音声を8ch伝送する場合は1水平ブランキング期間中に最大2パケット必要である。残りの6パケット、168byte(=28byte×6パケット)以内で圧縮符号方式情報を記述できることが望ましい。
Since one packet can be transmitted per 32 clocks, a maximum of 8 packets can be transmitted with the compressed horizontal 280 blanking pixels. On the other hand, audio data can be transmitted at 24 bits × 8 channels per packet. Since the horizontal frequency of the image data is 135 kHz (= 60 Hz × (2160 + 90)), when transmitting 24 channels of 24-bit linear PCM audio of 192 kHz samples, a maximum of 2 packets are required during one horizontal blanking period. It is desirable that the compression code method information can be described within the remaining 6 packets, 168 bytes (= 28 bytes × 6 packets).
一方、前述した32画素を単位ブロックとして圧縮する場合、水平3840有効画素なので120ブロック分のYCbCr輝度色差信号の各圧縮符号方式情報、計360個分の記述スペースが必要である。圧縮符号情報を例えば2bitで表現すると、90bye分の記述スペースとなり、4パケットで伝送できるので、192kHz、8chの大容量音声データ伝送と両立できる。図12~図15にパケットの一例を示す。
On the other hand, when the above-described 32 pixels are compressed as a unit block, since it is a horizontal 3840 effective pixel, each compression code system information of YCbCr luminance color difference signal for 120 blocks, a total of 360 description spaces are required. If the compressed code information is expressed in 2 bits, for example, it becomes a description space of 90 bytes and can be transmitted in 4 packets, so that it can be compatible with large-capacity audio data transmission of 192 kHz and 8 ch. An example of a packet is shown in FIGS.
図12と図14がパケットのヘッダの一例であり、最初のヘッダブロックHB0に本発明の圧縮符号化伝送方式に関する情報であることを示す共通のヘッダタイプ0Bhを記述する。HB1の各bitとHB2のBit4~7は将来拡張用として0としている。HB2のBit0~3に示すEco_Packet#がフレーム内での識別を示している。図13と図15がヘッダに続けて伝送される28byteのデータの一例である。
FIG. 12 and FIG. 14 are examples of packet headers, and a common header type 0Bh is described in the first header block HB0, which indicates information related to the compression-encoding transmission system of the present invention. Each bit of HB1 and Bits 4 to 7 of HB2 are set to 0 for future expansion. Eco_Packet # shown in Bits 0 to 3 of HB2 indicates identification within the frame. FIGS. 13 and 15 are examples of 28-byte data transmitted following the header.
図12のヘッダにおいて、Eco_Packet#は0hを割り当て、図12のヘッダと図13のデータから構成されるパケットが各フレーム内の共通情報であることを示す。このパケットは垂直ブランキング期間中に配置され、各画像フレームに少なくとも1回送信される。以下、図13のデータの内容を説明する。
In the header of FIG. 12, Eco_Packet # is assigned 0h, indicating that the packet composed of the header of FIG. 12 and the data of FIG. 13 is common information in each frame. This packet is arranged during the vertical blanking period and is transmitted at least once for each image frame. Hereinafter, the contents of the data in FIG. 13 will be described.
Color_Samplは、カラーサンプル情報を示し、例えば0がYCbCr444フォーマット、1がYCbCr422フォーマット、2がYCbCr420フォーマット、3がRGB444フォーマットを示し、4~7は将来拡張用である。422や420フォーマットではさらにCbCrのサンプル位置情報を示すBitを追加で割り当ててもよい。
Color_Sample indicates color sample information, for example, 0 is YCbCr444 format, 1 is YCbCr422 format, 2 is YCbCr420 format, 3 is RGB444 format, and 4 to 7 are for future expansion. In the 422 or 420 format, a bit indicating CbCr sample position information may be additionally allocated.
CDはColor Depthであり、例えば4hは各YCbCr成分8bitの計24bit Color、5hは各YCbCr成分10bitの計30bit Color、6hは各YCbCr成分12bitの計36bit Color、7hは各YCbCr成分16bitの計48bit Colorを示し、その他は将来拡張用である。この定義はHDMIが既定するDeep Color Modeの定義に準じている。
CD is Color Depth, for example, 4h is each YCbCr component 8bit total 24bit Color, 5h is each YCbCr component 10bit total 30bit Color, 6h is each YCbCr component 12bit total 36bit Color, 7h is each YCbCr component 16bit total 48bit Color is shown, others are for future expansion. This definition conforms to the definition of Deep Color Mode defined by HDMI.
Eco_CbCrは、図4や図5で説明したCbやCrの座標変換の適用情報を示している。0であれば座標変換未適用、1は図5で説明した簡易座標変換、2は図4で説明したCbやCrの範囲を最小化した座標変換、3はCbCr範囲を最小化にした上でさらに係数圧縮した座標変換を示す。4~7は将来拡張用である。Color_SamplがRGB444フォーマットを指定する場合は、この座標変換は無効なので未適用の0を記述すればよい。次に係数圧縮について、説明する。
Eco_CbCr indicates application information of the coordinate conversion of Cb and Cr described in FIG. 4 and FIG. If 0, no coordinate transformation is applied, 1 is the simple coordinate transformation explained in FIG. 5, 2 is the coordinate transformation minimizing the Cb or Cr range explained in FIG. 4, and 3 is the CbCr range minimized. Further, coordinate conversion with coefficient compression is shown. 4 to 7 are for future expansion. When the Color_Sample specifies the RGB444 format, this coordinate conversion is invalid, and an unapplied 0 may be described. Next, coefficient compression will be described.
図4に示すCb範囲最小化の座標変換を行った場合、Cbの有効範囲領域は8bit量子化時に68~188の範囲であり、範囲幅は121と1bit少ない7bitで表現できる。しかし、図6の最小Cr範囲では8bit量子化時に57~199の範囲と範囲幅で143と7bitでは表現できない。
When the coordinate conversion for Cb range minimization shown in FIG. 4 is performed, the effective range region of Cb is a range of 68 to 188 at the time of 8-bit quantization, and the range width can be expressed by 7 bits, which is 1 bit, 121. However, the minimum Cr range in FIG. 6 cannot be expressed by 143 and 7 bits in the range of 57 to 199 and the range width at the time of 8-bit quantization.
また、xvYCCなどの拡張色空間をある程度表現したい場合もあるので、係数縮小を行い1bit少ないデータで表現できるようにするものである。CbとCrの各縮小係数は、図13のCb_FactorとCr_Factorの各8bitデータを256で割った値で示す。但し、00h表記は縮小係数1.00すなわち係数圧縮しないことを示す。また、係数縮小後の量子化bit数は元空間の量子化bit数に比べて、Cb_FactorやCr_Factorが129~255と0は1bit、65~128は2bit、33~64は3bit、17~32は4bit減らすものとする。Y輝度信号に対して、CbCr色差信号の階調数を減らしても画質上への影響は比較的小さいので、本手法によるデータ量圧縮は効果的である。
Also, since there is a case where an extended color space such as xvYCC is desired to be expressed to some extent, the coefficient is reduced so that it can be expressed by 1 bit fewer data. Each reduction coefficient of Cb and Cr is indicated by a value obtained by dividing the 8-bit data of Cb_Factor and Cr_Factor in FIG. However, the 00h notation indicates a reduction factor of 1.00, that is, no factor compression. In addition, the number of quantized bits after the coefficient reduction is smaller than the number of quantized bits in the original space. Cb_Factor and Cr_Factor are 129 to 255, 0 is 1 bit, 65 to 128 is 2 bits, 33 to 64 is 3 bits, and 17 to 32 are It shall be reduced by 4 bits. Even if the number of gradations of the CbCr color difference signal is reduced with respect to the Y luminance signal, since the influence on the image quality is relatively small, the data amount compression by this method is effective.
例えば座標変換前の画像データが8bitであって、Eco_CbCr=”3”、Cb_Factorを00h(すなわち係数縮小なし)と設定した場合、RGB原色信号換算値がいずれも正となるCb範囲68~188(128が正規化Cb=0.0に相当)は、4~124(64が正規化Cb=0.0に相当)へ変換される。7bit化時において、64を正規化CbCrの0.00として量子化データは1~126を用いる。座標変換前の0と255はそれぞれ0と127として伝送する。座標変換前が12bitであって、Cb_Factorが128の場合は。係数圧縮後のデータは2bit少ない10bitとなり、512を正規化Cbの0.00として量子化データは1~1022を用いる。座標変換前の0と4095は0と1023として伝送する。
For example, if the image data before coordinate conversion is 8 bits, Eco_CbCr = “3”, and Cb_Factor is set to 00h (that is, no coefficient reduction), the RGB primary color signal conversion values are all positive in the Cb range 68 to 188 ( 128 corresponds to normalized Cb = 0.0) is converted into 4 to 124 (64 corresponds to normalized Cb = 0.0). At the time of 7-bit conversion, 64 is normalized CbCr 0.00 and quantization data 1 to 126 is used. 0 and 255 before coordinate conversion are transmitted as 0 and 127, respectively. When the coordinate conversion is 12 bits and Cb_Factor is 128. The data after coefficient compression is 10 bits less by 2 bits, and 512 is normalized Cb 0.00, and quantized data 1 to 1022 is used. 0 and 4095 before coordinate conversion are transmitted as 0 and 1023.
このように、CbやCr色差信号のbit数を圧縮すれば、後段の水平又は垂直圧縮部などを使わないでも、YCbCr輝度色差信号各8bitの合計24bitを1画素として非圧縮伝送の代わりに。Y輝度信号に10bit、CbCr色差信号各7bitの合計24bitを1画素として非圧縮伝送もできる。視感度の高いY輝度信号に2bitを追加して伝送することにより滑らかな階調表現が実現できる。この場合、従来のY輝度信号8bitには10bitのY輝度信号の上位8bitを、CbとCr色差信号各8bitの上位7bitには7bitに縮小したCbとCr色差信号を、CbとCr色差信号各8bitの下位1bitには10bitのY輝度信号の下位2bitを1bitずつ割り当てるとよい。この割り当てにより、万一、このbit割り当てを誤解釈して従来のYCbCr輝度色差信号各8bit割り当てと扱った場合でも、その画質への影響を軽減することができる。
In this way, if the number of bits of Cb and Cr color difference signals is compressed, a total of 24 bits of YCbCr luminance color difference signals of 8 bits are used as one pixel instead of uncompressed transmission without using a horizontal or vertical compression unit at the subsequent stage. Non-compressed transmission is possible with a total of 24 bits, 10 bits for the Y luminance signal and 7 bits for each CbCr color difference signal, as one pixel. Smooth gradation expression can be realized by adding 2 bits to the Y luminance signal having high visibility and transmitting it. In this case, the upper 8 bits of the 10-bit Y luminance signal in the conventional Y luminance signal 8 bits, the Cb and Cr color difference signals reduced to 7 bits in the upper 7 bits of each 8 bits of the Cb and Cr color difference signals, and the Cb and Cr color difference signals The lower 2 bits of the 10-bit Y luminance signal may be assigned to the lower 1 bit of 8 bits, one bit at a time. By this assignment, even if this bit assignment is misinterpreted and treated as a conventional YCbCr luminance color difference signal 8 bit assignment, the influence on the image quality can be reduced.
尚、上記は座標変換後に係数圧縮と言う2ステップ演算を示唆しているが、これを合せて1つの座標変換としてあつかうと演算量を軽減できる。さらにはRGB原色信号が入力される場合は、YCbCr輝度色差信号へ変換するマトリクス演算の係数に反映させれば1回の演算でよい。
Although the above suggests a two-step operation called coefficient compression after coordinate conversion, the combined amount can be reduced by using this as one coordinate conversion. Further, when RGB primary color signals are input, one calculation is sufficient if they are reflected in the matrix calculation coefficients for conversion into YCbCr luminance color difference signals.
Eco_FLMは、フレーム内の全ブロックの圧縮符号方式が同一である場合に1とし、ブロック毎に設定する場合を0としている。1の場合は、後述するEco-CD0とEco-CD1、Eco-CD2にそれぞれY、Cb、Crの圧縮符号方式を記述する。
Eco_FLM is set to 1 when the compression code system of all the blocks in the frame is the same, and is set to 0 when set for each block. In the case of 1, the Y-, Cb-, and Cr-compression code systems are described in Eco-CD0, Eco-CD1, and Eco-CD2, which will be described later.
CK_NとCK_Mは、その比(CK_N/CM_M)が、非圧縮画像データの画素クロックと、圧縮後のデータを伝送する通信路のクロック例えばTMDSクロックとの周波数比を示す。例えばCM_N=1でCK_M=2であれば、4k2kの場合の非圧縮画像データの画素クロック594MHzに対して伝送系のTMDSクロックは1/2の297MHzとなる。
The ratio (CK_N / CM_M) of CK_N and CK_M indicates a frequency ratio between a pixel clock of uncompressed image data and a clock of a communication path for transmitting the compressed data, for example, a TMDS clock. For example, if CM_N = 1 and CK_M = 2, the TMDS clock of the transmission system is 297 MHz, which is 1/2 of the pixel clock 594 MHz of uncompressed image data in the case of 4k2k.
Eco_Blockは圧縮画素単位ブロックの画素数を示している。Eco_CD0~Eco_CD3は、各画像データ単位ブロックに適用する圧縮符号化情報の候補を4種類示すものである。図16に一例を示すように、圧縮する画像データ単位の圧縮符号情報Eco_Codeから4種類を選択する。
Eco_Block indicates the number of pixels of the compressed pixel unit block. Eco_CD0 to Eco_CD3 indicate four types of compression coding information candidates to be applied to each image data unit block. As shown in FIG. 16, four types are selected from compression code information Eco_Code for each image data unit to be compressed.
図14のヘッダと図15のデータからなるパケットは、各水平ブランキング期間に1個以上伝送される。図14のヘッダ中のEco_Packet#は、各ラインに伝送される本パケットのシリアル番号を示すものであり、1から始まり順次1ずつ増やしていく。図15に示すCode_0~Code_111は、図13のEco_CD0~Eco_CD3で記述した4種類の圧縮符号化情報から選択した番号を、各画像データ単位ブロックにおいて、各Y、Cb,Cr成分を順番に記述する。例えば、最初の画像データ単位ブロックのY成分の圧縮符号情報を示すCode_0が1であれば、Eco_CD1を指す。Eco_CD1が10を示していれば、図16から原画像のデータの444フォーマットY成分を差分符号化方式で圧縮したデータであることを示している。
14 One or more packets including the header of FIG. 14 and the data of FIG. 15 are transmitted in each horizontal blanking period. Eco_Packet # in the header of FIG. 14 indicates the serial number of this packet transmitted to each line, and starts from 1 and is incremented by 1. Code_0 to Code_111 shown in FIG. 15 describe the Y, Cb, and Cr components in order in each image data unit block, with numbers selected from the four types of compression encoded information described in Eco_CD0 to Eco_CD3 in FIG. . For example, if Code_0 indicating the compression code information of the Y component of the first image data unit block is 1, it indicates Eco_CD1. If Eco_CD1 indicates 10, it indicates that data obtained by compressing the 444 format Y component of the original image data by the differential encoding method from FIG.
図8に示した垂直2ラインを画像データ単位ブロックとする場合は、1ライン目のCode_0が画像データ単位ブロック503の圧縮符号情報を、2ライン目のCode_0が画像データ単位ブロック504の圧縮符号情報を示す。
When the vertical two lines shown in FIG. 8 are image data unit blocks, Code_0 of the first line is compression code information of the image data unit block 503, and Code_0 of the second line is compression code information of the image data unit block 504. Indicates.
図13のColor_Samplが444フォーマットを示している場合は、Code_1は、第1画像データ単位ブロックのCbの圧縮符号情報、Code_2は第1画像データ単位のCrの圧縮符号情報、Code_3は第2画像データ単位ブロックのYの圧縮符号情報を示す。Color_Samplが420フォーマットを示している場合は、Code_1は、第2画像データ単位ブロックのYの圧縮符号情報、Code_2は第1と第2画像データ単位のCb(偶数ラインではCr)の圧縮符号情報、Code_3は第3画像データ単位ブロックのYの圧縮符号情報を示す。Color_Samplが422フォーマットを示している場合は、Code_1は、第1と第2画像データ単位ブロックのCbの圧縮符号情報、Code_2は第第2画像データ単位のYの圧縮符号情報、Code_3は第1と第2画像データ単位ブロックのCbの圧縮符号情報を示す。
When Color_Sampl in FIG. 13 indicates the 444 format, Code_1 is Cb compression code information of the first image data unit block, Code_2 is Cr compression code information of the first image data unit, and Code_3 is second image data. The Y compression code information of the unit block is shown. When Color_Sampl indicates 420 format, Code_1 is Y compression code information of the second image data unit block, Code_2 is compression code information of Cb (Cr in even lines) of the first and second image data units, Code_3 indicates Y compression code information of the third image data unit block. When Color_Sampl indicates the 422 format, Code_1 is Cb compression code information of the first and second image data unit blocks, Code_2 is Y compression code information of the second image data unit, and Code_3 is the first and second image data unit blocks. Cb compression code information of the second image data unit block is shown.
また、Code_0が0であれば、Eco_CD0を指し、Eco_CD0が6を示していれば図16から、原画像のデータの444フォーマット各Y、Cb、Cr成分12bitデータを420フォーマット8ビットに間引いていることを示している。この場合は、Y成分のCode_0のみでCbとCrの伝送形態が決まる為、Cb成分を示すCode_1やCr成分を示すCode_2の情報は不要であり、0を記述しておくよい。
If Code_0 is 0, it indicates Eco_CD0, and if Eco_CD0 indicates 6, the 444 format Y, Cb, and Cr component 12-bit data of the original image data is thinned out to 420 format 8 bits from FIG. It is shown that. In this case, since the transmission form of Cb and Cr is determined only by Y component Code_0, information of Code_1 indicating the Cb component and Code_2 indicating the Cr component is unnecessary, and 0 may be described.
さて、図1に図示していないが、画像受信装置200には画像受信装置200の性能を示すEDID(Enhanced Extended Display Identification Data)を格納したROMが搭載されている。このROMの中に画像受信装置200が圧縮伸張に対応しているか否かを判別する情報を付加してもよい。これにより、画像伝送装置100は、画像受信装置200のEDIDを格納したROMから、圧縮伸張に対応しているか否かを判別する情報を読出し、対応している装置であれば、圧縮した画像データを伝送し、非対応の装置であれば、圧縮せず従来のサイズで画像を送信することにより、圧縮処理に非対応の画像受信装置とも互換性を保つことができる。
Although not shown in FIG. 1, the image receiving device 200 is equipped with a ROM that stores EDID (Enhanced Extended Display Identification Data) indicating the performance of the image receiving device 200. Information for determining whether or not the image receiving apparatus 200 supports compression / decompression may be added to the ROM. As a result, the image transmission apparatus 100 reads information for determining whether or not it supports compression / decompression from the ROM storing the EDID of the image reception apparatus 200, and if it is a compatible apparatus, the compressed image data If the device is non-compatible, the image can be transmitted in a conventional size without being compressed, and compatibility with an image reception device not compatible with compression processing can be maintained.
また、画像受信装置が圧縮に非対応であり、従来の画像サイズであることを表示部208に表示することにより、ユーザーに通知することがきる。
Also, the user can be notified by displaying on the display unit 208 that the image receiving apparatus is incompatible with compression and has a conventional image size.
このEDIDの記述例を図17に示す。図17は、HDMI-VSDBと称される領域へ拡張子した例を示している。6Byte目のBit2に本発明の圧縮符号化伝送方式への対応可否を示すEco_transferフラグを設ける。本領域は予約領域として扱われてきたので、非対応のレガシー機器では0と記載されており、対応機器のみ1と記載することによって後方互換性を維持できる。このEco_transferフラグが1の場合、Byte9とByte10の記述が有効となる。Bit0と1のEco_CbCrは、CbCr色差信号の座標変換対応可否を示し、図13のEco_CbCrの下位2bitに相当する。0であれば座標変換未対応、1は簡易座標変換対応、2は簡易座標変換に加えてCbやCrの範囲を最小化した座標変換にも対応、3は簡易座標変換に加えてCbCr範囲を最小化にした上でさらに係数圧縮した座標変換する機能も3方式全部の座標変換にも対応していることを示している。
FIG. 17 shows a description example of this EDID. FIG. 17 shows an example of extension to an area called HDMI-VSDB. An Eco_transfer flag indicating whether or not the compression encoding transmission system of the present invention can be supported is provided in Bit 2 of the 6th byte. Since this area has been treated as a reserved area, it is described as 0 for non-compliant legacy devices, and backward compatibility can be maintained by describing only 1 for compatible devices. When the Eco_transfer flag is 1, the description of Byte 9 and Byte 10 is valid. Eco_CbCr of Bit 0 and 1 indicates whether or not the coordinate conversion of the CbCr color difference signal can be supported, and corresponds to the lower 2 bits of Eco_CbCr in FIG. If 0, coordinate conversion is not supported, 1 corresponds to simple coordinate conversion, 2 corresponds to coordinate conversion in which the range of Cb and Cr is minimized in addition to simple coordinate conversion, and 3 corresponds to CbCr range in addition to simple coordinate conversion This shows that the function of coordinate conversion with further coefficient compression after minimization is also compatible with coordinate conversion of all three systems.
Block_64とBlock_128は、圧縮する画像データ単位ブロックの大きさがそれぞれ64画素、128画素に対応することを示すフラグである。画像データ単位ブロックの大きさ32画素は本圧縮画像データ伝送対応における必須モードと定義して、EDID記述スペース節約のため、あえて表記していない。Eco-Code1~4は、それぞれ、図16に一例を示した圧縮符号化方式、Wavelet変換、ランレングス符号化、ハフマン符号化、差分符号化に対応していることを示すフラグである。CLK_1、CLK_3/4、CLK_1/2は、それぞれ非圧縮画像データクロックに対する、TMDS伝送クロックの周波数が1倍、3/4倍、1/2倍となるモードに対応していることを示すフラグである。
Block_64 and Block_128 are flags indicating that the size of the image data unit block to be compressed corresponds to 64 pixels and 128 pixels, respectively. The image data unit block size of 32 pixels is defined as an indispensable mode in correspondence with the transmission of the compressed image data, and is not intentionally written in order to save EDID description space. Eco-Codes 1 to 4 are flags indicating that they are compatible with the compression coding method, Wavelet transform, run-length coding, Huffman coding, and differential coding, which are shown as examples in FIG. CLK_1, CLK_3 / 4, and CLK_1 / 2 are flags indicating that the mode corresponds to the mode in which the frequency of the TMDS transmission clock is 1 time, 3/4 time, and 1/2 time with respect to the uncompressed image data clock, respectively. is there.
また、画像伝送装置100を携帯機器として使用する場合に、バッテリー駆動の装置となるため、画像伝送装置100の消費電力が連続使用時間に影響する。この場合、画像データを圧縮して伝送しデータ伝送量を下げ、消費電力を低減できる。この効果は、画像伝送装置100の動作モードとして、例えば「省電力モード」などの機能を付加し、外部から電源が供給されている場合は、非圧縮の画像データで伝送し、バッテリーで駆動されている場合は、画像データを圧縮して伝送することにより、連続使用時間を長く設定することができる。
Also, when the image transmission apparatus 100 is used as a portable device, it becomes a battery-powered apparatus, so the power consumption of the image transmission apparatus 100 affects the continuous use time. In this case, image data can be compressed and transmitted to reduce the amount of data transmission and reduce power consumption. This effect is achieved by adding a function such as “power saving mode” as an operation mode of the image transmission apparatus 100, and when power is supplied from the outside, the image transmission apparatus 100 is transmitted by non-compressed image data and driven by a battery. In this case, the continuous use time can be set longer by compressing and transmitting the image data.
図18は、データ受信処理部205の構成の一例を示すブロック図である。
入力部220は、画像伝送装置100のレベル変換部175にて変換された信号をレベル変換部222に出力する。レベル変換部222は、画像伝送装置100でレベル変換された信号をデジタル信号に変換しデシリアライザ223に出力する。レベル変換の一例としては、差動信号をシングルエンド信号への変換がある。 FIG. 18 is a block diagram illustrating an example of the configuration of the datareception processing unit 205.
Theinput unit 220 outputs the signal converted by the level conversion unit 175 of the image transmission apparatus 100 to the level conversion unit 222. The level conversion unit 222 converts the signal level-converted by the image transmission apparatus 100 into a digital signal and outputs the digital signal to the deserializer 223. An example of level conversion is conversion of a differential signal to a single-ended signal.
入力部220は、画像伝送装置100のレベル変換部175にて変換された信号をレベル変換部222に出力する。レベル変換部222は、画像伝送装置100でレベル変換された信号をデジタル信号に変換しデシリアライザ223に出力する。レベル変換の一例としては、差動信号をシングルエンド信号への変換がある。 FIG. 18 is a block diagram illustrating an example of the configuration of the data
The
入力部221は、画像伝送装置100から出力されたクロックを入力しPLL224に出力する。PLL224は、入力されたクロックの10倍のクロックを生成し、デシリアライザ223に出力する。またPLL224は、画像受信装置200内で使用する画素クロックを出力部226から出力する。原画像の非圧縮画像データの画素クロックを画像表示装置200内で用いる場合は、入力クロックを図13のパケットデータに基づき、PLL224が(CK_M/CK_N)倍に逓倍したクロックを出力部226から出力する。
The input unit 221 inputs the clock output from the image transmission apparatus 100 and outputs it to the PLL 224. The PLL 224 generates a clock 10 times the input clock and outputs the generated clock to the deserializer 223. The PLL 224 outputs a pixel clock used in the image receiving apparatus 200 from the output unit 226. When the pixel clock of the uncompressed image data of the original image is used in the image display device 200, the clock output by the PLL 224 multiplied by (CK_M / CK_N) times is output from the output unit 226 based on the packet data of FIG. To do.
デシリアライザ223は、シリアル化されたデータをPLL224からのクロックでパラレル化し、出力部225から出力する。デシリアライザ223は、10倍のクロックのデータをパラレル化して所定の例えばTMDS復号により8bitパラレルデータとして出力部225から出力する。
The deserializer 223 parallelizes the serialized data with the clock from the PLL 224 and outputs it from the output unit 225. The deserializer 223 parallelizes the 10-times clock data, and outputs the data from the output unit 225 as 8-bit parallel data by predetermined TMDS decoding, for example.
図19は、伸張処理部206の構成の一例を示すブロック図である。また、図20は、伸張処理部206の処理概念を示すタイミング説明図である。
入力部250は、伸張処理部206のデータ入力部である。入力部250に入力されるデータには、画像データの同期信号を示すHSYNC(図20(a))およびVSYNCと、有効期間406に圧縮された画像データ512と514、水平ブランキング期間404に圧縮符号情報511と513がある。入力部251には復元後の非圧縮画像データの画素クロックや、圧縮画像データクロック等が入力される。 FIG. 19 is a block diagram illustrating an example of the configuration of thedecompression processing unit 206. FIG. 20 is a timing explanatory diagram showing a processing concept of the decompression processing unit 206.
Theinput unit 250 is a data input unit of the decompression processing unit 206. Data input to the input unit 250 includes HSYNC (FIG. 20A) and VSYNC indicating a synchronization signal of image data, image data 512 and 514 compressed in the valid period 406, and compressed in the horizontal blanking period 404. There is code information 511 and 513. The input unit 251 receives a pixel clock of uncompressed image data after restoration, a compressed image data clock, and the like.
入力部250は、伸張処理部206のデータ入力部である。入力部250に入力されるデータには、画像データの同期信号を示すHSYNC(図20(a))およびVSYNCと、有効期間406に圧縮された画像データ512と514、水平ブランキング期間404に圧縮符号情報511と513がある。入力部251には復元後の非圧縮画像データの画素クロックや、圧縮画像データクロック等が入力される。 FIG. 19 is a block diagram illustrating an example of the configuration of the
The
HSYNC、VSYNC、圧縮画像データクロックと復元後の非圧縮画像データの画素クロックは、タイミング生成部252に供給される。タイミング生成部252は、入力されたHSYNC、VSYNCによりカウンタを制御し、垂直ブランキング期間401、垂直有効期間402、水平ブランキング期間404、水平有効期間405、有効期間406などのタイミングを始め、伸張処理部内の各ブロックの制御に必要なタイミングを生成し出力する。
HSYNC, VSYNC, the compressed image data clock, and the pixel clock of the decompressed uncompressed image data are supplied to the timing generator 252. The timing generation unit 252 controls the counter based on the input HSYNC and VSYNC, and starts and expands the timing of the vertical blanking period 401, the vertical valid period 402, the horizontal blanking period 404, the horizontal valid period 405, the valid period 406, and the like. Generates and outputs timing necessary for control of each block in the processing unit.
圧縮符号情報抽出部253は、水平ブランキング期間に送られてくる各画像データ単位ブロックの圧縮符号情報を抽出して圧縮符号情報記憶部254に記憶させる。その記憶期間を図20(c)に示す。圧縮符号情報511は続く圧縮画像データ512に対応したデータであるので、同情報を2ライン目の圧縮符号情報513が来るまでの記憶期間515で十分である。しかし、例えば2ラインの画像データを垂直圧縮させている場合は、垂直伸張用圧縮符号情報516だけは2ライン目の圧縮画像データを伸張させる期間まで保持させる必要がある。
The compression code information extraction unit 253 extracts the compression code information of each image data unit block sent during the horizontal blanking period and stores it in the compression code information storage unit 254. The storage period is shown in FIG. Since the compression code information 511 is data corresponding to the subsequent compressed image data 512, a storage period 515 until the compression code information 513 of the second line comes is sufficient. However, for example, when two lines of image data are vertically compressed, it is necessary to hold only the compression code information 516 for vertical expansion until a period during which the compressed image data of the second line is expanded.
水平有効期間内に送られてくる圧縮画像データはエラー訂正部255で伝送系のエラー訂正処理を行う。 エラー訂正部255は、圧縮された画像データの単位ごとに、エラー訂正符号生成部136と同じエラー訂正符号を演算する。前記演算結果と符号検出部253から入力されるエラー訂正符号とを比較し、比較結果が異なっている場合は、エラー訂正処理を行う。エラー訂正処理の一例としては、CRC演算がある。また、エラー検出のみを行い以降の処理でエラーを補間してもよい。
Compressed image data sent within the horizontal effective period is subjected to transmission system error correction processing by the error correction unit 255. The error correction unit 255 calculates the same error correction code as the error correction code generation unit 136 for each unit of compressed image data. The calculation result is compared with the error correction code input from the code detection unit 253, and if the comparison result is different, an error correction process is performed. An example of error correction processing is CRC calculation. Alternatively, only error detection may be performed, and errors may be interpolated in subsequent processing.
水平有効期間内に送られてくる圧縮画像データの内、間引き画像データはエラー訂正部255を通さず、直接選択部258へ入力している(図19の矢印263)。単なる間引き画像データの場合、エラーがあってもそのエラーは該当画素及び隣接した間引き画素に限定され、画面全体の画質を著しく劣化させるものではないこと、さらにはエラー訂正符号を付加するとさらに多くの画素又は階調を間引くことになって画面全体の画質劣化につながる点を考慮した為である。もちろん、データ転送帯域に余裕があれば、間引き画像データにもエラー訂正処理を付加してもよい。
Among the compressed image data sent within the horizontal effective period, the thinned image data is directly input to the selection unit 258 without passing through the error correction unit 255 (arrow 263 in FIG. 19). In the case of simple thinned-out image data, even if there is an error, the error is limited to the corresponding pixel and adjacent thinned-out pixels, and does not significantly deteriorate the image quality of the entire screen. This is because consideration is given to the point that thinning out pixels or gradations leads to image quality deterioration of the entire screen. Of course, if there is a margin in the data transfer band, error correction processing may be added to the thinned image data.
圧縮符号情報記憶部254の情報を基に、水平伸張部256や垂直伸張部257で画像データを伸張して選択部258へ出力する。水平伸張部256出力(図20(d))は圧縮画像データ512と514から復号にかかる時間だけ遅れた水平伸張画像データ518と519を出力する。垂直伸張部257出力(図20(e))は、例えば2ライン画像データ用いた圧縮符号化方式の場合、2ライン期間に渡って1ライン目と2ライン目の垂直伸張画像データ520と521として出力される。
Based on the information in the compression code information storage unit 254, the horizontal decompression unit 256 and the vertical decompression unit 257 decompress the image data and output it to the selection unit 258. The horizontal decompression unit 256 output (FIG. 20D) outputs horizontal decompressed image data 518 and 519 delayed from the compressed image data 512 and 514 by the time required for decoding. For example, in the case of a compression encoding method using two-line image data, the output of the vertical extension unit 257 (as shown in FIG. 20E) is the vertical extension image data 520 and 521 of the first and second lines over a two-line period. Is output.
選択部258は、圧縮符号情報記憶部の情報に基づき、入力部250へ入力される画像データと、水平伸張部256の出力、垂直伸張部257出力を適宜選択して間引き復元部259へ出力する。上記したように、最も遅れて出力される垂直伸張部出力521に合せて、復元画像データ522が入力圧縮画像データ512よりも1ライン以上遅れて出力することになる。このようなタイミング調整の為、詳細説明は省略するが、選択部258の入力画像データは各伸張方式による処理遅延を適宜吸収するためのメモリ(図示せず)を有する。
The selection unit 258 appropriately selects the image data input to the input unit 250, the output of the horizontal expansion unit 256, and the output of the vertical expansion unit 257 based on the information in the compression code information storage unit, and outputs them to the thinning restoration unit 259. . As described above, the restored image data 522 is output with a delay of one line or more from the input compressed image data 512 in accordance with the vertical expansion unit output 521 that is output with the latest delay. Although detailed description is omitted for such timing adjustment, the input image data of the selection unit 258 has a memory (not shown) for appropriately absorbing a processing delay due to each expansion method.
間引き復元部259は、圧縮符号情報記憶部254の情報を基に、間引き画像データから間引き部の画像データを復元する。間引き復元には、図2の間引き処理部132の間引き対応の他、水平圧縮部133や垂直圧縮部134が444フォーマットから422への間引き処理を施している場合にも対応し、間引き種別に応じた復元を行う。また、出力部261が444フォーマットではなく422や420フォーマット画像データを要求する場合は、間引きの復元処理を停止又は間引きフォーマット変換などを行ってもよい。
The thinning restoration unit 259 restores the image data of the thinning unit from the thinned image data based on the information in the compression code information storage unit 254. The thinning restoration corresponds to the case where the horizontal compression unit 133 and the vertical compression unit 134 perform thinning processing from the 444 format to 422 in addition to the thinning processing unit 132 in FIG. Restore. When the output unit 261 requests 422 or 420 format image data instead of the 444 format, the thinning restoration process may be stopped or the thinning format conversion may be performed.
間引き復元部259は、エラー訂正部255からの信号が該当画像データのエラー有を示しており、かつ水平方向に圧縮されていた場合は左右画像から補完し、垂直方向に圧縮されていた場合は、1ライン前のデータから補完する処理を施してもよい。
If the signal from the error correction unit 255 indicates that there is an error in the corresponding image data and is compressed in the horizontal direction, the thinning restoration unit 259 complements the left and right images, and if the signal is compressed in the vertical direction , Complementary processing may be performed from data one line before.
この構成により、エラーが発生した場合においても相関性が高い画像データにおきかえることができるため、伝送エラーによる画像への影響を少なくすることができる。また圧縮する画像データの単位が十数画素であるため、置き換える画素の範囲もすくなくすむため、伝送エラーの影響をすくなくすることができる。また、圧縮伸張による処理遅延も図20(f)に示すように1ラインで抑えることができる。
With this configuration, even when an error occurs, it is possible to replace the image data with high correlation, so that the influence on the image due to the transmission error can be reduced. In addition, since the unit of image data to be compressed is a dozen pixels, the range of pixels to be replaced is reduced, so that the influence of transmission errors can be reduced. Also, processing delay due to compression / decompression can be suppressed by one line as shown in FIG.
間引き復元部259の出力を入力とするYCbCr処理部260は、図13に示したパケットデータ情報Eco_CbCrとCb_Factor、Cr_Factorを基に、縮小されていたCbとCrを復元する色差信号処理を担当する。また、出力部261がRGB原色信号を要求する場合は、RGB原色信号への変換処理を行う。このように、YCbCr処理部260は、画像伝送装置における図2のYCbCr処理部131の逆動作を行う位置付けであり、詳細説明は省略する。
The YCbCr processing unit 260 that receives the output of the thinning restoration unit 259 is in charge of color difference signal processing for restoring the reduced Cb and Cr based on the packet data information Eco_CbCr, Cb_Factor, and Cr_Factor shown in FIG. Further, when the output unit 261 requests an RGB primary color signal, a conversion process to an RGB primary color signal is performed. Thus, the YCbCr processing unit 260 is positioned to perform the reverse operation of the YCbCr processing unit 131 in FIG. 2 in the image transmission apparatus, and detailed description thereof is omitted.
以上で説明した本実施例によれば、画像伝送装置が伝送する画像データを圧縮して伝送することにより、現在規定している伝送路に、現在規定されている画像サイズより大きなサイズの画像データを伝送することが可能になり、さらに、画像データの伝送領域よりも、エラー耐性を高めた領域にエラー検出や訂正符号を付加することによって、エラー耐性が高い画像伝送を行うことができる。
According to the present embodiment described above, image data having a size larger than the currently specified image size is transmitted to the currently specified transmission path by compressing and transmitting the image data transmitted by the image transmission apparatus. In addition, it is possible to perform image transmission with higher error tolerance by adding error detection and correction codes to a region with higher error resistance than the image data transmission region.
また、現在規定されている画像サイズの画像データを伝送する場合においては、所定時間当たりのデータ伝送量、もしくはデータ伝送クロックを下げることができるため、エラーの発生頻度を下げることができ、且つ、伝送路でのエラーに対して信頼性の高いシステムを構築することができる。
In addition, when transmitting image data of the image size currently specified, the data transmission amount per predetermined time or the data transmission clock can be lowered, so that the frequency of occurrence of errors can be lowered, and A highly reliable system can be constructed against errors in the transmission path.
また、伝送路でエラーが発生して完全なエラー訂正ができない場合においても、エラーによる画質劣化が目立たないエラー処理を行うシステムを実現することができる。
Also, even when an error occurs in the transmission line and complete error correction cannot be performed, a system that performs error processing in which image quality deterioration due to the error is not noticeable can be realized.
実施例1において、垂直圧縮の画像データ単位ブロックは図8のように2行以上複数ラインにまたがるため、図20に示すように圧縮画像データに対して1水平周期以上の遅延が生じている。この遅延時間を軽減する為に、垂直圧縮の画像データ単位ブロックも図7の水平圧縮の画像データ単位ブロックと同じ1ラインとする。この場合は、垂直圧縮は前ラインに対する差分の符号化などの符号化方式に限定されるが、実施例1に比べて処理遅延時間が軽減できる利点がある。
In the first embodiment, the vertical compression image data unit block extends over two or more lines as shown in FIG. 8, and as a result, a delay of one horizontal cycle or more is generated with respect to the compressed image data as shown in FIG. In order to reduce this delay time, the vertical compression image data unit block is also set to one line, which is the same as the horizontal compression image data unit block of FIG. In this case, the vertical compression is limited to an encoding method such as encoding of the difference with respect to the previous line, but there is an advantage that the processing delay time can be reduced as compared with the first embodiment.
実施例1では、画像データ単位ブロック毎の圧縮符号情報を信頼性の高い水平ブランキング期間中のパケットを用いて伝送したが、図20に示すように、受信機側で圧縮符号情報を1水平周期近く保持する必要があった。また、送信機側では、圧縮符号情報を該当するラインの圧縮画像データの直前の水平ブランキング期間に配置する為に、圧縮画像データをラインメモリに記憶させてほぼ1水平周期遅らせて出力させる必要があり、回路規模の増大と画像信号の伝達遅延時間が大きくなる課題があった。そこで、実施例3では、圧画像データ単位の圧縮符号情報を水平ブランキング期間ではなく、水平有効期間内の圧縮画像データと一緒に伝送する方式を採用する。
In the first embodiment, the compression code information for each image data unit block is transmitted using a packet during a highly reliable horizontal blanking period. However, as shown in FIG. It was necessary to keep close to the cycle. On the transmitter side, in order to arrange the compression code information in the horizontal blanking period immediately before the compressed image data of the corresponding line, it is necessary to store the compressed image data in the line memory and output it with a delay of about one horizontal cycle. There is a problem that the circuit scale increases and the transmission delay time of the image signal increases. Therefore, in the third embodiment, a method of transmitting compression code information in units of pressure image data together with compressed image data within the horizontal effective period instead of the horizontal blanking period is employed.
有効期間内に送るべきデータが圧縮画像データだけでなく圧縮符号情報も一緒に送ろうとすると、圧縮画像データの圧縮率を上げざるを得ず、復元時の画像の画質劣化が大きくなる懸念がある。その対策として、水平ブランキング期間を音声データ伝送パケット2個分を送れる程度まで縮小し、水平有効期間を広げる。
If the data to be sent within the effective period is to send not only the compressed image data but also the compression code information, the compression rate of the compressed image data must be increased, and there is a concern that the image quality of the image at the time of restoration will increase greatly . As a countermeasure, the horizontal blanking period is reduced to such an extent that two audio data transmission packets can be sent, and the horizontal effective period is extended.
4k2k画像データを非圧縮画像の画素クロック594MHzの半分の297MHzクロックで伝送する場合、実施例1では、水平有効期間1920、水平ブランキング期間280であった。音声パケット2個分の伝送期間と前後のガードバンドを含めて水平ブランキング期間は96程度あればよいので、残り184を水平有効期間を増やせる。水平有効期間1920を約9.5%程度拡大できる効果がある。
In the case where 4k2k image data is transmitted with a 297 MHz clock that is half of the pixel clock 594 MHz of the uncompressed image, the horizontal effective period 1920 and the horizontal blanking period 280 are used in the first embodiment. Since it is sufficient that the horizontal blanking period is about 96 including the transmission period of two voice packets and the guard bands before and after, the remaining 184 can be increased in the horizontal effective period. There is an effect that the horizontal effective period 1920 can be expanded by about 9.5%.
この実施例3の場合、水平同期信号の位置や幅などが所定の非圧縮画像データの標準タイミングからずれる為、受信側で非圧縮画像データ再生時に、画像データに付与されているSVD(Short Video Descriptor)メタデータを参考にして、標準タイミングフォーマットとなるようタイミングを復元するとよい。
In the case of the third embodiment, the position and width of the horizontal synchronization signal deviate from the standard timing of predetermined uncompressed image data. Therefore, when the uncompressed image data is reproduced on the receiving side, SVD (Short Video added to the image data) Descriptor) The timing should be restored to the standard timing format with reference to the metadata.
図21に、伸張処理部206の構成の一例を示すブロック図を示す。図19と同様な機能を持つブロックには同じ番号を付している。その差異は、エラー訂正回路255の出力部に第2の映像符号情報抽出部263が加わり、間引き画像データとして入力250から直接ではなくエラー訂正部255を通して選択部258の入力(図21の矢印264参照)になっている点である。
実施例1では、圧縮符号情報抽出部253は垂直ブランキング期間内の圧縮符号情報(図12と図13のパケット)と、水平ブランキング期間内の圧縮符号情報(図14と図15のパケット)を抽出していた。本実施例では、圧縮符号情報抽出部は垂直ブランキング期間内のフレーム内共通情報のみ抽出し、圧縮符号情報記憶部254で記憶する。 FIG. 21 is a block diagram illustrating an example of the configuration of thedecompression processing unit 206. Blocks having the same functions as those in FIG. 19 are given the same numbers. The difference is that a second video code information extraction unit 263 is added to the output unit of the error correction circuit 255, and the input of the selection unit 258 (arrow 264 in FIG. 21) is not directly sent from the input 250 as the thinned image data but through the error correction unit 255. This is the point of reference).
In the first embodiment, the compression codeinformation extraction unit 253 includes the compression code information (packets in FIGS. 12 and 13) in the vertical blanking period and the compression code information (packets in FIGS. 14 and 15) in the horizontal blanking period. Was extracted. In this embodiment, the compression code information extraction unit extracts only the intra-frame common information within the vertical blanking period and stores it in the compression code information storage unit 254.
実施例1では、圧縮符号情報抽出部253は垂直ブランキング期間内の圧縮符号情報(図12と図13のパケット)と、水平ブランキング期間内の圧縮符号情報(図14と図15のパケット)を抽出していた。本実施例では、圧縮符号情報抽出部は垂直ブランキング期間内のフレーム内共通情報のみ抽出し、圧縮符号情報記憶部254で記憶する。 FIG. 21 is a block diagram illustrating an example of the configuration of the
In the first embodiment, the compression code
本実施例では、各圧縮画像データ単位ブロックの圧縮符号情報がブランキング期間内でエラー訂正処理が施されているパケットで送る代わりに、他の圧縮画像データと同様なエラー訂正処理を加えている。このため、第2の圧縮符号情報抽出部262が、エラー訂正部255のエラー訂正処理後のデータと、フレーム単位で抽出された圧縮符合情報記憶部254の出力から、各圧縮画像データ単位ブロックの圧縮符号情報を抽出して、水平伸張部256や垂直伸張部257、選択部258へ出力している。
In this embodiment, instead of sending the compression code information of each compressed image data unit block in a packet subjected to error correction processing within the blanking period, error correction processing similar to that of other compressed image data is added. . For this reason, the second compression code information extraction unit 262 uses the data after the error correction processing of the error correction unit 255 and the output of the compression code information storage unit 254 extracted in units of frames, for each compressed image data unit block. The compression code information is extracted and output to the horizontal expansion unit 256, the vertical expansion unit 257, and the selection unit 258.
本実施例では、各圧縮画像データ単位ブロックの圧縮画像データと圧縮符号情報がタイミング的に近接しているので、1水平周期に渡る圧縮符号情報記憶は、フレーム内共通の圧縮符号情報を除いて不要となるので、回路規模を削減できる効果がある。
In this embodiment, since the compressed image data and the compressed code information of each compressed image data unit block are close in timing, the compressed code information storage for one horizontal period is excluded except for the compressed code information common in the frame. Since it becomes unnecessary, the circuit scale can be reduced.
さらに、本実施例では、実施例1に比べて前述したように、水平有効期間が9.5%拡張できるので、エラー訂正符号を追加して伝送している。このため、間引き画像データに対してもエラー訂正部255の出力264として選択部258へ入力されているので、間引き画像データのエラーを抑圧でき、よりエラー耐性の強い画像データ伝送が実現できる
。 Furthermore, in the present embodiment, as described above in comparison with the first embodiment, the horizontal effective period can be extended by 9.5%, so that an error correction code is additionally transmitted. For this reason, since the thinned image data is also input to theselection unit 258 as the output 264 of the error correction unit 255, errors in the thinned image data can be suppressed, and image data transmission with higher error resistance can be realized.
。 Furthermore, in the present embodiment, as described above in comparison with the first embodiment, the horizontal effective period can be extended by 9.5%, so that an error correction code is additionally transmitted. For this reason, since the thinned image data is also input to the
100 画像伝送装置
101,102,103,104,130,150,151,170,172,201,202,220,221,250,251 入力部
105 チューナー受信処理部
106 ネットワーク受信処理部
107 記録メディア制御部
108 記録メディア
109,203ユーザーIF
110,204 制御部
111 ストリーム制御部
112 デコーダ
113,207 表示処理部
114 圧縮処理部
115 データ伝送部
116,138,157,176,177,225,226,261 出力部
131 YCbCr処理部
132 間引き処理部
133 水平圧縮部
134 垂直圧縮部
135,258 選択部
136 エラー訂正符号生成部
137 圧縮符号情報付加部
153 エラー訂正符号演算部
154, 251タイミング生成部
155 データ保持部
173,224 PLL
174 シリアライザ
175,222 レベル変換部
200 画像受信装置
205 データ受信処理部
206 伸張処理部
208 表示部
223 デシリアライザ
253,262 圧縮符号情報抽出部
254 圧縮符号情報記憶部
255 エラー訂正部
256 水平伸張部
257 垂直伸張部
259 間引き復元部
260 YCbCr処理部
300 ケーブル
400 垂直期間
401 垂直ブランキング期間
402 垂直有効期間
403 水平期間
404 水平ブランキング期間
405 水平有効期間
406 有効期間
407 ブランキング期間
501,502,503,504 圧縮する画像データの単位
512,514 圧縮された画像データ
511,513,515,516,517 圧縮符号情報
518,519 水平伸張画像データ
520,521 垂直伸張画像データ
522 復元画像データ 100 image transmission equipment
101,102,103,104,130,150,151,170,172,201,202,220,221,250,251 Input section
105 Tuner reception processor
106 Network reception processor
107 Recording media controller
108 Recording media
109,203 user IF
110,204 Control unit
111 Stream controller
112 decoder
113,207 Display processing section
114 Compression processing section
115 Data transmission section
116,138,157,176,177,225,226,261 Output section
131 YCbCr processing section
132 Thinning processing section
133 Horizontal compression section
134 Vertical compression section
135,258 selector
136 Error correction code generator
137 Compression code information adding part
153 Error correction code calculation unit
154, 251 timing generator
155 Data holding part
173,224 PLL
174 Serializer
175,222 Level converter
200 Image receiver
205 Data reception processor
206 Decompression processing unit
208 Display
223 Deserializer
253,262 Compression code information extraction unit
254 Compression code information storage unit
255 Error correction section
256 Horizontal stretch
257 Vertical extension
259 Thinning restoration part
260 YCbCr processing section
300 cables
400 vertical period
401 Vertical blanking period
402 Vertical validity period
403 horizontal period
404 horizontal blanking period
405 Horizontal effective period
406 Validity period
407 Blanking period
501,502,503,504 Unit of image data to compress
512,514 compressed image data
511,513,515,516,517 Compression code information
518,519 Horizontally expanded image data
520,521 Vertically expanded image data
522 Restored image data
101,102,103,104,130,150,151,170,172,201,202,220,221,250,251 入力部
105 チューナー受信処理部
106 ネットワーク受信処理部
107 記録メディア制御部
108 記録メディア
109,203ユーザーIF
110,204 制御部
111 ストリーム制御部
112 デコーダ
113,207 表示処理部
114 圧縮処理部
115 データ伝送部
116,138,157,176,177,225,226,261 出力部
131 YCbCr処理部
132 間引き処理部
133 水平圧縮部
134 垂直圧縮部
135,258 選択部
136 エラー訂正符号生成部
137 圧縮符号情報付加部
153 エラー訂正符号演算部
154, 251タイミング生成部
155 データ保持部
173,224 PLL
174 シリアライザ
175,222 レベル変換部
200 画像受信装置
205 データ受信処理部
206 伸張処理部
208 表示部
223 デシリアライザ
253,262 圧縮符号情報抽出部
254 圧縮符号情報記憶部
255 エラー訂正部
256 水平伸張部
257 垂直伸張部
259 間引き復元部
260 YCbCr処理部
300 ケーブル
400 垂直期間
401 垂直ブランキング期間
402 垂直有効期間
403 水平期間
404 水平ブランキング期間
405 水平有効期間
406 有効期間
407 ブランキング期間
501,502,503,504 圧縮する画像データの単位
512,514 圧縮された画像データ
511,513,515,516,517 圧縮符号情報
518,519 水平伸張画像データ
520,521 垂直伸張画像データ
522 復元画像データ 100 image transmission equipment
101,102,103,104,130,150,151,170,172,201,202,220,221,250,251 Input section
105 Tuner reception processor
106 Network reception processor
107 Recording media controller
108 Recording media
109,203 user IF
110,204 Control unit
111 Stream controller
112 decoder
113,207 Display processing section
114 Compression processing section
115 Data transmission section
116,138,157,176,177,225,226,261 Output section
131 YCbCr processing section
132 Thinning processing section
133 Horizontal compression section
134 Vertical compression section
135,258 selector
136 Error correction code generator
137 Compression code information adding part
153 Error correction code calculation unit
154, 251 timing generator
155 Data holding part
173,224 PLL
174 Serializer
175,222 Level converter
200 Image receiver
205 Data reception processor
206 Decompression processing unit
208 Display
223 Deserializer
253,262 Compression code information extraction unit
254 Compression code information storage unit
255 Error correction section
256 Horizontal stretch
257 Vertical extension
259 Thinning restoration part
260 YCbCr processing section
300 cables
400 vertical period
401 Vertical blanking period
402 Vertical validity period
403 horizontal period
404 horizontal blanking period
405 Horizontal effective period
406 Validity period
407 Blanking period
501,502,503,504 Unit of image data to compress
512,514 compressed image data
511,513,515,516,517 Compression code information
518,519 Horizontally expanded image data
520,521 Vertically expanded image data
522 Restored image data
Claims (10)
- 画像データを圧縮し伝送する画像伝送装置において、
画像データを圧縮する圧縮処理部と、
前記圧縮処理部で圧縮された画像データを出力するデータ転送部を有し、
前記データ転送部は、圧縮された画像データと共に前記圧縮処理部で行われる処理に関する情報を出力することを特徴とする画像伝送装置。 In an image transmission device for compressing and transmitting image data,
A compression processing unit for compressing image data;
A data transfer unit that outputs the image data compressed by the compression processing unit;
The image transfer apparatus, wherein the data transfer unit outputs information related to processing performed in the compression processing unit together with compressed image data. - 請求項1の画像伝送装置において、
前記圧縮処理部は、輝度信号との相関を用いた座標変換によって色差信号のとりうる範囲を狭める色差信号処理部と、輝度色差信号からなる画像データを圧縮する圧縮部と、を有し、
前記圧縮処理部で行われる処理に関する情報は、前記色差信号処理部及び前記圧縮部で行われる処理に関する圧縮符号情報を含むことを特徴とする画像伝送装置。 The image transmission apparatus according to claim 1.
The compression processing unit includes a color difference signal processing unit that narrows a range that the color difference signal can take by coordinate conversion using a correlation with a luminance signal, and a compression unit that compresses image data including the luminance color difference signal,
The information related to processing performed by the compression processing unit includes compression code information regarding processing performed by the color difference signal processing unit and the compression unit. - 請求項1または2に記載の画像伝送装置において、
前記画像データに基づいてクロック信号を生成するクロック信号生成部を有し、
前記データ転送部は、前記信号生成部の出力するクロックに同期して前記圧縮画像データを出力することを特徴とした画像伝送装置。 The image transmission apparatus according to claim 1 or 2,
A clock signal generator that generates a clock signal based on the image data;
The image transfer apparatus, wherein the data transfer unit outputs the compressed image data in synchronization with a clock output from the signal generation unit. - 画像データを圧縮し伝送する画像伝送方法において、
画像データを圧縮するステップと、
前記圧縮された画像データを前記圧縮に関する情報と共に出力するステップと、を有することを特徴とする画像伝送方法。 In an image transmission method for compressing and transmitting image data,
Compressing the image data;
And outputting the compressed image data together with information related to the compression. - 請求項4の画像伝送方法において、
前記画像データを圧縮するステップは、輝度信号との相関を用いた座標変換によって色差信号のとりうる範囲を狭める色差信号処理を行うステップと、輝度色差信号からなる画像データを圧縮処理するステップと、を含み、
前記圧縮に関する情報は、前記色差信号処理と前記圧縮処理に関する圧縮符号情報を含むことを特徴とする画像伝送方法。 In the image transmission method of Claim 4,
The step of compressing the image data includes a step of performing color difference signal processing for narrowing a possible range of the color difference signal by coordinate conversion using a correlation with the luminance signal, a step of compressing the image data including the luminance color difference signal, and Including
The information relating to compression includes the color difference signal processing and compression code information relating to the compression processing. - 請求項4または5に記載の画像伝送方法において、
前記画像データに基づいてクロック信号を生成するステップを有し、
前記生成したクロックに同期して前記圧縮画像データを出力することを特徴とした画像伝送方法。 The image transmission method according to claim 4 or 5,
Generating a clock signal based on the image data;
An image transmission method comprising outputting the compressed image data in synchronization with the generated clock. - 圧縮された画像データを受信する画像受信装置において、
圧縮された画像データと当該圧縮された画像データの圧縮処理に関する情報を受信する入力部と、
前記入力部に入力された前記圧縮処理に関する情報に基づいて前記入力部に入力された圧縮された画像データを伸張する伸張処理部と、を有することを特徴とした画像受信装置。 In an image receiving device that receives compressed image data,
An input unit that receives compressed image data and information related to compression processing of the compressed image data;
An image receiving apparatus comprising: an expansion processing unit that expands compressed image data input to the input unit based on information related to the compression processing input to the input unit. - 請求項7の画像受信装置において、
前記伸長処理部は、前記入力部に入力された前記圧縮処理に関する情報に基づいて前記入力部に入力された圧縮された画像データを伸張する伸張部と、前記入力部に入力された前記圧縮処理に関する情報に基づいて、前記伸張部にて伸張された輝度色差形式の画像データに対して輝度信号との相関を用いた座標変換を行い、当該画像データを色差信号として出力する色差信号処理部とを有することを特徴とした画像受信装置。 The image receiving device according to claim 7.
The decompression processing unit includes: a decompression unit that decompresses compressed image data input to the input unit based on information related to the compression process input to the input unit; and the compression process input to the input unit. A color-difference signal processing unit that performs coordinate conversion using the correlation with a luminance signal on the image data in the luminance-color-difference format expanded by the expansion unit and outputs the image data as a color-difference signal, An image receiving apparatus comprising: - 圧縮された画像データを受信する画像受信方法において、
圧縮された画像データと当該圧縮された画像データの圧縮処理に関する情報を受信するステップと、
前記受信した圧縮処理に関する情報に基づいて前記受信した圧縮された画像データの伸張処理を行うステップと、を有することを特徴とした画像受信方法。 In an image receiving method for receiving compressed image data,
Receiving compressed image data and information relating to compression processing of the compressed image data;
And a step of performing decompression processing on the received compressed image data based on the information on the received compression processing. - 請求項9の画像受信方法において、
前記画像データの伸張処理を行うステップは、前記受信した圧縮処理に関する情報に基づいて前記受信した圧縮された画像データを伸張するステップと、前記受信した圧縮処理に関する情報に基づいて、前記伸張した輝度色差形式の画像データに対して輝度信号との相関を用いた座標変換を行い、当該画像データを色差信号として出力するステップと、を有することを特徴とした画像受信方法。 The image receiving method according to claim 9.
The step of decompressing the image data includes the step of decompressing the received compressed image data based on the information related to the received compression process, and the decompressed luminance based on the information related to the received compression process. An image receiving method comprising: performing coordinate conversion using correlation with a luminance signal on image data in a color difference format and outputting the image data as a color difference signal.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011166182A JP2013031024A (en) | 2011-07-29 | 2011-07-29 | Image transmission device, image transmission method, image receiving device, and image receiving method |
JP2011-166182 | 2011-07-29 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2013018248A1 true WO2013018248A1 (en) | 2013-02-07 |
Family
ID=47628804
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2012/002216 WO2013018248A1 (en) | 2011-07-29 | 2012-03-30 | Image transmission device, image transmission method, image receiving device, and image receiving method |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP2013031024A (en) |
WO (1) | WO2013018248A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016539550A (en) * | 2013-10-25 | 2016-12-15 | メディアテック インコーポレイテッド | Method and apparatus for controlling transmission of compressed pictures according to transmission synchronization events |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5232319B2 (en) | 2011-10-20 | 2013-07-10 | 株式会社東芝 | Communication apparatus and communication method |
JP5694412B2 (en) * | 2011-10-20 | 2015-04-01 | 株式会社東芝 | Transmission device, reception device, transmission method, and reception method |
JP5390667B2 (en) | 2012-06-11 | 2014-01-15 | 株式会社東芝 | Video transmission device and video reception device |
JP5808509B2 (en) * | 2015-01-22 | 2015-11-10 | 株式会社東芝 | Video receiver |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009098933A1 (en) * | 2008-02-04 | 2009-08-13 | Sony Corporation | Video signal transmission device, video signal transmission method, video signal reception device, and video signal reception method |
JP2010041636A (en) * | 2008-08-08 | 2010-02-18 | Sony Corp | Information processing device, method and program |
WO2010023884A1 (en) * | 2008-08-29 | 2010-03-04 | パナソニック株式会社 | Color signal converting apparatus, video displaying apparatus, color signal converting method, video displaying method and image data |
-
2011
- 2011-07-29 JP JP2011166182A patent/JP2013031024A/en not_active Withdrawn
-
2012
- 2012-03-30 WO PCT/JP2012/002216 patent/WO2013018248A1/en active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009098933A1 (en) * | 2008-02-04 | 2009-08-13 | Sony Corporation | Video signal transmission device, video signal transmission method, video signal reception device, and video signal reception method |
JP2010041636A (en) * | 2008-08-08 | 2010-02-18 | Sony Corp | Information processing device, method and program |
WO2010023884A1 (en) * | 2008-08-29 | 2010-03-04 | パナソニック株式会社 | Color signal converting apparatus, video displaying apparatus, color signal converting method, video displaying method and image data |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016539550A (en) * | 2013-10-25 | 2016-12-15 | メディアテック インコーポレイテッド | Method and apparatus for controlling transmission of compressed pictures according to transmission synchronization events |
US10038904B2 (en) | 2013-10-25 | 2018-07-31 | Mediatek Inc. | Method and apparatus for controlling transmission of compressed picture according to transmission synchronization events |
Also Published As
Publication number | Publication date |
---|---|
JP2013031024A (en) | 2013-02-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5868997B2 (en) | Image transmission device, image transmission method, image reception device, and image reception method | |
WO2013018248A1 (en) | Image transmission device, image transmission method, image receiving device, and image receiving method | |
WO2012147791A1 (en) | Image receiving device and image receiving method | |
JP5739262B2 (en) | Image transmission device, image transmission method, image reception device, and image reception method | |
JP6609074B2 (en) | Image output apparatus and output method | |
JP2013115456A (en) | Image transmission apparatus and image transmission method | |
JP6472845B2 (en) | Image receiving device | |
JP6200971B2 (en) | Image transmission apparatus and transmission method | |
WO2012147786A1 (en) | Image transmission device and image transmission method | |
JP2013115455A (en) | Image transmission apparatus and image transmission method | |
JP5041969B2 (en) | Video transmission method, system and program | |
JP5318992B2 (en) | Video transmission program | |
JP2013115453A (en) | Image transmission apparatus, image transmission method, image reception apparatus, and image reception method | |
AU2017367221A1 (en) | A decoder, encoder, computer program and method | |
JP6166401B2 (en) | Video data transmission / reception method, video transmission device, and video reception device | |
WO2020231893A1 (en) | Packing of subpixel rendered data for display stream compression | |
JP5883026B2 (en) | Video data transmission / reception method, video transmission device, and video reception device | |
JP2013115452A (en) | Transmitting/receiving method for video data, video transmitting device, and video receiving device | |
JP2013115454A (en) | Video transmitting device, video receiving device, video transmitting method, and video receiving method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12819372 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 12819372 Country of ref document: EP Kind code of ref document: A1 |