GB2316568A

GB2316568A - Coded image bit stream format converter

Info

Publication number: GB2316568A
Application number: GB9717923A
Authority: GB
Inventors: Ryoichi Kawada; Takahiro Hamada; Shuichi Matsumoto
Original assignee: Kokusai Denshin Denwa KK
Current assignee: KDDI Corp
Priority date: 1996-08-23
Filing date: 1997-08-22
Publication date: 1998-02-25
Anticipated expiration: 2017-08-22
Also published as: GB2316568B; JP3363036B2; JPH1066033A; GB9717923D0

Abstract

A format converter which receives a compressed image bit stream removes the requirement for separate motion vector calculation in the decoding and conversion steps. The bit stream (a) is input to an extracting unit (1a) for the extraction of coding control data (m), including motion vectors, used by the decoder unit (1b). The motion vectors from the coding control data extracting unit (1a) are used by a motion vector calculating unit (2a) in the format converter (2), together with interpolated field position data (p) to calculate the motion vectors (n) to be used by the format converter (2b). These motion vectors (n) are subsequently fed, together with coding control data (m) and field position data (p), to a coding controller (3a) to generate further motion vectors (q) for the final encoding unit (3b).

Description

MOTION PICTURE CODED-BITSTREAM CONVERTER The present invention relates to a motion picture coded-bitstream converter and particularly, to a motion picture coded-bitstream converter for converting the television format of a motion picture coded-bitstream at high quality with ease using various data such as motion information in a motion picture which are contained in the bit stream in a compressed form.

For transmission of TV signals between stations which are different in the TV format, the TV signals have to be converted from one format to another. The TV format conversion is commonly carried out for maintaining the quality of pictures by one of known motion compensated TV format conversion methods which utilize motion vectors determined through estimating the movements in a motion picture. Among such motion compensated TV format conversion methods is "Motion picture frame rate conversion method with motion compensation" by us, the inventors, depicted in Japanese Patent Publication No.H3-25119.

The conventional method is explained in brief. A common motion picture such as a TV picture is principally implemented by displaying a series of still images at a constant frame rate per second and is not yielded by a timely continuous signal. For example, the TV broadcasting format in Japan employs a frame rate of 30 frames per second but 25 frames per second in Europe. When TV programs are exchanged between the countries of which TV formats are different in the frame rate, a technology is needed for shifting the frame rate.

Fig. 8 is a diagram showing the principle of shifting the frame rate. As shown, the frame rate is shifted from 25 frames per second (yielding pictures a25, b25, c25, ...) to 30 frames per second (yielding a30, b30, c30, ...). When the frame a25 of a picture is synchronized with the frame a30, two different format frames overlap each other at equal intervals of six frames because a ratio of the frame rate is 25:30=5:6.

While synchronized ones of the original frames can directly be used, other frames have to be constructed by interpolation between two adjacent frames. For example, the b30 frame is yielded by interpolation between the frame a25 and the frame b25, and the frame c30 between the frames b25 and c25. For yielding an interpolated frame or field, it is essential to compensate the motion in an original picture.

The motion compensation in the process of field interpolation involves motion estimation in which uniformity of the motion in a picture is concerned, each field of the picture is divided into a plurality of blocks, and the motion in the blocks is estimated. More specifically, the motion is detected in each block between two adjacent fields to express as vectors and a field to be interpolated between the two fields is developed referring to the vectors.

Also, a digital data compression encoding technique such as MPEG-2, which is different in type from the TV format converting method, has been employed for ease of the motion picture data transmission.

The MPEG-2 is comprehensively described in ITU-T Recommendation H.262. In such a data compression encoding method as MPEG-2 the motion vectors given through estimation of the motion in a picture are employed for increasing the compression rate of data.

In the prior art, the motion compensated TV format conversion and the data compression encoding are carried out separately from each other but not in a single process. A conventional apparatus for performing the two different processes is now explained referring to Fig. 9.

When an incoming bit stream a of a TV picture in a coded form is input, it is decoded by a decoder 31 to reconstruct an original TV data b (referring to as input format image hereinafter). A motion compensated format converter 32 comprises a motion estimating unit 32a and a linelfield converter unit 32b. The motion estimating unit 32a calculates a motion vector from the input format image and transmits it as denoted by c to the line/field converter unit 32b. The line/field converter unit 32b converts the input format image b to a desired format image using the method disclosed in Japanese Patent Publication No. H3-25119 as well as the motion vector c. The converted format image (referred to as an output format image hereinafter) is then delivered as another TV format data d from the line/field converter unit 32b.

In a succession, the output format image d is fed into a data compression or motion compensated encoder 33 of e.g. MPEG-2. The motion compensated encoder 33 comprises a motion estimating unit 33a and an encoding unit 33b. The motion estimating unit 33a calculates motion vectors from the converted TV format image d. The encoding unit 33b encodes the format image d referring to the motion vectors e and delivers it as an output bit stream f.

As understood, the motion compensated format converter 32 and the motion compensated encoder 33 are isolated from each other in the function.

In each of the motion compensated TV format converter 32 and the motion compensated encoder 33, a section for calculating the motion vector occupies a considerable extension of the hardware size and cannot be omitted. If the estimation of the motion in a picture is not correct in the format converter 32, a reproduced TV image of the converted format will be declined in the quality. In other words, the correct estimation of the motion in a picture is a primary requirement for having a high quality TV reproduction. In the motion compensated encoder 33, the motion estimation is also essential for increasing the rate of data compression.

As set forth above, the prior art allows the motion compensated TV format conversion to be carried out separately of the motion compensated encoding and hence has the motion estimation repeated.

As the two functions are made with hardware apparatus, its overall dimensions will be increased. More precisely, the motion estimating unit 32a and 33a occupy a considerable extension of the hardware size of the motion compensated TV format converter 32 and the motion compensated encoder 33, thus increasing the overall size and cost of the apparatus.

Also, the motion compensated TV format converter 32 produces motion vectors from the image decoded by the decoder 31 as shown in Fig. 9. Even if the de.coded image contains adverse artifacts, it is directly subjected to the motion estimation of the motion estimating unit 32a. As the result, the motion estimation may include an error.

It is therefore desirable to provide a motion picture coded-bitstream converter which eliminates the foregoing disadvantages and converts a motion picture coded bit stream at high quality with ease by motion compensated format conversion and re-encoding with the use of encoding control information, including motion vectors, carried in the bit stream of a compressed form.

According to the present invention there is provided a nation picture ÇbitstreB aamrerter for converting an input bit stream coded from a video signal of a TV format to an output bit stream of another TV format, ccnprising: a coding control data extracting means for extracting from the input bit stream coding control data including motion vectors in each macro block, coding mode, and quantization step size; a decoding means for decoding the input bit stream in relation with the coding control data; a motion vector calculating means for converting extracted motion vectors to motion compensated vectors n in accordance with the coding mode; a format converting means for converting a decoded image data to an image of the another n/ format with reference to the motion vectors n; and an encoding means for calculating motion vectors q from the motion vectors n, the interpolated field location data p, and the quantization step size, and encoding the another TV format image with reference to the motion vectors q.

The present invention allows the motion vectors required for TV format conversion and the motion vectors used for re-encoding to be developed by means of the coding control data carried in the bit stream and extracted by the coding control data extracting means. The quality of a TV format converted image data will be increased and also, the overall dimensions of the TV format converter will remarkably be reduced.

Eitodiints of the invention will now be described by way of example with reference to the drawings, in which: Fig. 1 is a block diagram showing an embodiment of this invention.

Fig. 2 shows a diagram showing conception of a specific example of an interpolated field and preceding and succeeding fields in both side of the interpolated field.

Fig. 3A-D show diagrams showing methods for determining a motion vector of the interpolated field.

Fig. 4 shows a diagram showing specific example of motion vectors of a macroblock determined in Fig. 3A-D.

Fig. 5 shows a diagram showing method for compensating a vector of a block segment.

Fig. 6 shows a diagram showing vector prediction with 16 x 8 and field prediction mode.

Fig. 7 shows a diagram showing another vector prediction with 16 x 8 and field prediction mode.

Fig. 8 shows a diagram showing principle of shifting frame rate.

Fig. 9 is a block diagram of a conventional motion picture coded-bitstream converter.

Embodiments of the invention will be described in detail with reference to the drawings.

One embodiment of the present invention will be described in more detail referring to the accompanying drawings. Fig. 1 is a block diagram showing an embodiment of the present invention. From Fig. 1 to Fig. 9, like components are denoted by like numerals for ease of the description.

As shown, an incoming bit stream a of a TV picture signal of a TV format in a coded form is supplied to a decoder 1. The decoder 1 comprises a coding control data extracting unit la for extracting from the incoming bit stream a coding control information m, which includes motion vector data, coding control data (coding mode data) and quantizing step size, and a decoder unit 1 b for decoding the incoming bit stream a with the coding control information m.

A motion-compensated TV format converter 2 is provided comprising a motion vector calculating unit 2a for calculating a motion vector n, which is used for converting the TV format to another, from the motion vector data in the coding control information m as well as the interpolated field location data p fed from a line/field converter 2b, and a line/field converter unit 2b. A motion compensated encoder 3 is provided comprising a coding controller unit 3a for producing a motion vector q, which is used for re-encoding, from the coding control information m, the motion vector n, and the interpolated field location data p, and an encoder unit 3b for re-encoding with the motion vector q.

The operation of the decoder 1 is well known and its explanation will be omitted. The operation of the motion vector calculating unit 2a is now explained in detail. The motion vector calculating unit 2a calculates the motion vector for each interpolated field (such as denoted by a30, b30, c30, ... in Fig. 8) to be generated in changing the TV format. For example, the motion vector for an interpolated field 6 may be determined from the two motion vectors of the preceding 5 and the succeeding field 7 shown in Fig. 2.

More particularly, assuming that the motion vectors MV derived from an MPEG-2 bit stream have been registered for macro blocks of each field, i.e. the macro blocks A of the preceding field 5 and the macro blocks C of the succeeding field 7, the motion vector for the interpolated field 6 is determined by the following procedure: (1) As shown in Fig. 3A, when a backward vector vl and a forward vector v2 are attributed to the macro blocks A and C respectively, their corresponding prediction vectors should be orientated from the interpolated field 6. Since both the vectors are high in the reliability, the vector v2 which is closer in distance to the macro block B can be selected as the motion vector of the macro block B in the interpolated field 6. The location data p of the interpolated field 6 may be supplied from the line/field converter unit 2b or may have been saved as memory data in the motion vector calculating unit 2a.

(2) As shown in Fig. 3B, when a backward vector vl is attributed to the macro block A but no forward vector to the macro block C, the vector orientated from the interpolated field 6 is absent on the block C.

This indicates that the succeeding field 7 is discontinuous from the previous fields. As the vector vl in the macro block A is more reliable, it is selected as the motion vector of the macro block B.

(3) As shown in Fig. 3C, when a forward vector v2 is attributed to the macro block C but no backward vector to the macro block A, the vector v2 in the block C is selected as the motion vector of the macro block B by the same reason as in the paragraph (2).

(4) As shown in Fig. 3D, when a backward vector vl and a forward vector v2 are not attributed to the block A and C respectively, no prediction vectors from the interpolated field 6 stand in both the blocks A and C. In this case, it may be recognized that the motion in a picture varies in a complicated manner. It is hence more appreciated to assign a zero vector than to predict the motion of the block B. Accordingly, the zero vector is used as the motion vector of the macro block B. This is also eligible when the blocks A and C are intra-coded.

It is common that the motion vectors zl to zn in their respective blocks B1 to Bn in the interpolated field 6, calculated in the paragraphs (1) to (4), are orientated in different directions such as shown in Fig. 4. If the macro block vectors z1 to zn are directly subjected to the conversion action of the line/field converter unit 2b, block distortion may occur. For compensation, all the vectors in the interpolated field 6 are thus processed by order statistic filtering. An arrangement and its action of the order statistic filtering is disclosed in Japanese Patent Application No.H6-1 92253, "Motion vector processing apparatus", by us, the inventors. In brief, the order statistic filtering starts with measuring the magnitude of vectors in a target macro block as well as its neighbor blocks (a total of nine blocks), aligning the vectors in an order of the magnitude, from smallest or from largest, and selecting an intermediate vector in the middle of the alignment as the vector of the target macro block.

The single macro block B comprises 16 pixels by 16 lines and may be too large for use in the format conversion with motion compensation which handles true movements. The macro block has thus to be reduced to as a small size as reading the motion of a smaller object. The macro block in the interpolated field 6 is divided into segments, for example, each having 4 pixels x 2 lines. A vector in the block segment is determined by a known linear interpolation technique with the macro block vector which has been given by the order statistic filtering and referred as a representative vector.

The motion compensated format conversion may be impaired by perceiving a motion from still objects in a picture such as balls or captions. For eliminating such a drawback, the vector in each block segment is compared with the zero vector. It is implemented, as shown in Fig. 5, by projecting a motion vector MV of a block segment 10 in the interpolated field 6 on the preceding 5 and the succeeding field 7 to determine an absolute difference DFD, calculating absolute differences DFD of the preceding 5 and the succeeding field 7 with the zero vector, and selecting the smaller absolute difference DFD.

For correcting variations, the vectors determined from the absolute difference DFD are subjected to the order statistic filtering.

Using the motion vector n (Fig. 1) calculated in this manner, the line/field converter unit 2b performs a conversion of the TV format with motion compensation.

The operation of the motion compensated encoder 3 for re-encoding a format changed data output d of the line/field converter unit 2b is explained. The format changed data output d contains periodically spaced fields which have been produced only by line interpolation, such as denoted by (a)30 and (9)30 in Fig. 8. Those fields are free from deterioration caused by motion vector error and used for quality-thirsted I or P picture data in MPEG-2. In view of converting the TV format on a field-by-field basis and calculating the motion vector in each field, the data represents a field picture.

The encoding controller unit 3a of the motion compensated encoder 3 produces a motion vector q, which is transmitted to the encoder unit 3b, from the encoding control information m from the encoding control information extracting unit 1 a, the motion vector n from the motion vector calculating unit 2a, and the interpolated field location data p. More specifically, the type of the target macro block is determined by reviewing a distribution figure of the motion vectors n assigned to the block segments of the macro block while the motion vector q of the macro block is calculated from the motion vectors n of the block segments. The calculation for the motion vector q of the macroblock may be carried out by various manners. One of them is explained.

(1) For simplicity, the B picture is not used.

(2) Dispersion v2 of the motion vectors n in the segments of the macro block is calculated by using the motion vectors n, the encoding control information m, and the interpolated field location data p. Here, the encoding control information m and the interpolated field location data p are utilized for weight of the dispersion v2.

(3) Thresholds are Thl and Th2. When Th2 > v2 > Th1, variation in the vectors is medium. It is thus effective to divide the macro block into an appropriate number of the segments and the prediction with 16 x 8 is selected. When av2 > Th1, the vectors are rather uniform and the field prediction is employed. When srv2 > Th2, the vectors are not uniform and the efficiency of inter-coding will be declined. Thus, the intra-coding is selected.

(4) For prediction with 16 x 8, the following process has to be executed in each block of a 16 x 8 matrix and for field prediction, in each block of a 16 x 16 matrix.

(a) When the block is filled with mostly the zero vectors, the zero vector prediction from the same parity field is used as the motion vector q (See Fig. 6).

(b) When av22 < Th3 where crv22 is a dispersion of the vectors in the block (crv22= crv2 for the field prediction) and Th3 is a threshold, the motion is rather uniform and the reliability of the motion vectors is high. Hence, an average of the vectors is calculated and projected over the same parity field and a different parity field, as shown in Fig. 7.

Selected for the prediction is either field in which the indicating point of the average vector is closer to the semi-pixel location or pixel location and the motion vectors q are determined from the selected field. Referring to Fig. 7, the projecting point of the vector (A) is close to the semi-pixel location of the same parity field which is thus used for the prediction.

The vector (B) is opposite.

(c) When av22 > Th3, the vectors are not uniform. It is assumed that the motion in a picture is rapid and the different parity field which is closer in the distance is selected for the prediction. The motion vectors q are determined from the selected different party field.

It should be adjusted for the quantizing parameters that the quantizing scale code of each macro block is not rougher than the same in the two, preceding and succeeding, adjacent fields between which the interpolation is made.

As its coding controller unit 3a calculates the motion vector q used for the re-encoding, the encoder 3 needs to include not such a motion estimating unit as in a conventional apparatus thus contributing to the smaller hardware arrangement of the TV bitstream format converter.

As set forth above, the present invention permits the coding control information, such as motion vector data, carried in the incoming coded bit stream to be utilized for both the motion compensated format conversion and the re-encoding. Accordingly, the coded bit stream of a picture data can be converted to a desired TV format at high quality and the overall hardware arrangement of the TV bitstream format converter will notably be simplified.

Claims

CLAIM

1. A motion picture coded-bitstream converter for converting an input bit stream coded from a video signal of a TV format to an output bit stream of another TV format, comprising: a coding control data extracting means for extracting from the input bit stream coding control data including motion vectors in each macro block, coding mode, and quantization step size; a decoding means for decoding the input bit stream in relation with the coding control data; a motion vector calculating means for converting extracted motion vectors to motion compensated vectors in accordance with the coding mode; a format converting means for converting a decoded image data to an image of the another TV format with reference to the motion vectors; and an encoding means for calculating motion vectors from the motion vectors, the interpolated field location data, and the quantization step size, and encoding the another TV format image with reference to the motion vectors.

2. A motion picture coded-bitstream converter according to claim 1, wherein the motion vector calculating means selects either a forward vector or a backward vector whichever has been extracted and is oriented from the interpolated field and assigns it as the motion vector for use in the motion compensated format conversion.

3. A motion picture coded-bitstream converter according to claim 1, wherein the motion vector calculating means assigns a zero vector as the motion vector for use in the motion compensated format conversion when neither a forward vector nor a backward vector extracted is oriented from the interpolated field.

4. A motion picture coded-bitstream converter according to any of claims 1 to 3, wherein the motion vector calculating means performs an order statistic filtering process for the vectors in a macro block in order to eliminate a variation in the vectors in the macro block.

5. A motion picture coded-bitstream converter according to claim 4, wherein the motion vector calculating means calculates from linear interpolation a vector in a segment of the macro block and assigns it as the representative vector of the macro block.

6. A motion picture coded-bitstream converter according to claim 1, wherein the encoding means uses a field produced only by line interpolation only as an I or P picture data which is required for high quality reproduction in the data compression encoding such as MPEG-2.

7. A motion picture coded-bitstream converter according to claim 1, wherein the encoding means determines the type of a macro block through reviewing a profile of the motion vectors allocated in the segments of the macro block, and calculates a motion vector of the macro block from the motion vectors in the segments.

8. Motion picture coded-bitstream converter as hereinbefore described with reference to Figures 1 to 9 of the accompanying drawings.