CN115174927A - Method for mixing and de-mixing multi-component data stream in VLSI - Google Patents

Abstract

The invention discloses a method for mixing and unfreezing a multi-component data stream in a VLSI (very large scale integrated circuit), which comprises the following steps: the mixer stores each component code stream in a corresponding FIFO; when the mixer works in real time, the component output by the image encoder is dynamically selected, and the component corresponds to the maximum number of residual data in the FIFO at a certain moment; the mixer divides the component code stream into a plurality of data blocks, when the mixer puts the selected component code stream into the mixed stream, the mixer inserts the head information, and the head information is used for recording the index of the component and distinguishing the normal data block from the tail data block; a counter is arranged in a code stream selection module of the mixer and has three different value ranges to finish different functions; and the mixed flow reaches one end of an image decoder through a channel and is stored into an FIFO (first in first out), a de-mixing flow reads data from the FIFO, the head of each data block is found, each component code stream is separated out, and the image decoder reads the data to finish image decoding to obtain a reconstructed image.

Description

Method for mixing and de-mixing multi-component data flow in VLSI

Technical Field

The invention relates to an image processing VLSI algorithm, in particular to a method for performing flow mixing and flow mixing on a multi-component data stream in a VLSI.

Background

In compression and decompression of images or video, referring to fig. 1, an image encoder compresses an input video (image sequence) to obtain a plurality of single-component video streams. The time uncertainty of a multi-component video stream can be seen from fig. 2, where the code stream of a single component does not have data at every clock, and if there is (e.g. a small black block in fig. 2), the data at a certain clock is of a fixed bit length (e.g. an unsigned 32-bit data), and at another certain clock, it is possible that two or more component code streams have data at the same time. The single component video streams may be mixed into a data stream by stream merging (stream merging) before they pass through the channel, the data stream being a fixed bit length of data (e.g., an unsigned 32 bit data) at each clock. At one end of the image decoder, the mixed flow needs to be separated into component code streams, and the image decoder decodes the multiple single-component video streams to obtain a reconstructed image sequence (reconstructed video). In international standards such as MPEG2, MPEG4, etc., mixed flow is performed in a transport stream. The typical technical feature is to pack (package) the video stream output by the encoder, and the key is to add clock information (clock). That is to say that the synchronization of the plurality of elementary video streams is done by means of clock information. But this additional clock information carries a lot of extra data overhead.

Disclosure of Invention

In order to solve the problem of mixed flow and mixed flow de-mixing flow in the prior art, the invention provides a method for carrying out mixed flow and mixed flow de-mixing flow on multi-component data flow in VLSI (very large scale integrated circuit). The single-component data flow is dynamically detected, the component with the maximum flow rate is put into the mixed flow, and index information of the component is recorded by inserting a piece of header information so as to reduce data load.

In order to achieve the above purpose, the technical scheme adopted by the invention comprises the following steps: the mixer stores each component code stream in a corresponding FIFO; when the mixer works in real time, the component output by the image encoder is dynamically selected, and the component has the most residual data in the corresponding FIFO at a certain moment; the mixer divides the component code stream into a plurality of data blocks, when the mixer puts the selected component code stream into the mixed stream, the mixer inserts the head information, and the head information is used for recording the index of the component and distinguishing the normal data block from the tail data block; a counter is arranged in a code stream selection module of the mixer, and has three different value ranges to complete different functions; and the mixed flow reaches one end of an image decoder through a channel and is stored in an FIFO, a de-mixing flow reads data from the FIFO, the head of each data block is found, each component code stream is separated out and is read by the image decoder, and image decoding is completed to obtain a reconstructed image.

Furthermore, the mixer needs 3 FIFOs to cache 3 component code streams output from the image encoder, selects the component with the largest number of the residual data in the FIFO according to the number of the residual data in the 3 FIFOs, selects a normal data block with fixed number of data to be put into the mixed stream, and repeats the process until the number of the residual data of each component code stream is less than the number of the data in the normal data block (represented by an adjustable parameter PATCH _ LEN); at the end of each frame of image, after the normal data block is transmitted, the tail part of each component code stream is processed, and tail part data in the mixed flow is called a tail data block; the sequence of the data blocks of the specific component in the mixed flow is the same as the output sequence of the original image encoder; each normal data block needs a header to indicate the component index of the original code stream, and each normal data block contains PATCH _ LEN data; the head of each tail data block is 00, the tail _ length in the tail data block indicates the number of data in the tail, and the tail _ length data in the patch data block comprises the tail _ length data.

Further, an input line of the flow mixer includes:

(1) sys _ reset: resetting the system;

(2) start _ streams: the encoder indicates a frame encoding start;

(3) picture _ end: the encoder indicates that a frame is encoded to be finished;

(4) stream _ signals: an enable signal indicating an enable of the component code stream output by the encoder;

(5) stream _ values: multiplexing line, component code stream output by coder;

the output line of the mixer includes:

(1) mulstram _ sig: a mixed flow output enable signal;

(2) mulstram _ v: and (6) mixed flow output.

Further, the mixer comprises a code stream selection module and an insert _ patch _ mark module, wherein the code stream selection module selects which FIFO to read the component code stream from at a specific moment, and the insert _ patch _ mark module inserts header information into the selected data block;

the input line of code stream selection module includes:

(1) sys _ reset: resetting the system;

(2) start _ streams: the encoder indicates a frame encoding start;

(3) pic _ end: the encoder indicates that a frame is encoded to be finished;

(4) fifo _ cnt1: the number of the residual data in the ENC _ FIFO 1;

(5) fifo _ cnt2: the number of the residual data in the ENC _ FIFO 2;

(6) fifo _ cnt3: the number of the residual data in the ENC _ FIFO 3;

the code stream selection module is internally provided with a counter enc _ idx, and the counter enc _ idx has three different value ranges: the first range is a fixed value of 0 indicating the insertion of data block header information in the mixed flow output; the second range is 1 to PATCH _ LEN, which indicates the insertion of data block data in the mixed flow output; the third range is a fixed value PATCH _ LEN +1, which indicates that the mixer is in a special state; the specific implementation comprises the following steps:

(1) When enc _ idx is 0, the insert _ patch _ mark module indicates that a data block header is inserted;

(2) For normal data blocks, when the enc _ idx value is 1 to PATCH _ LEN, the insert _ PATCH _ mark module indicates insertion of normal data block data;

(3) For the tail data block, the enc _ idx value is 1 to the patch _ length, and the insert _ patch _ mark module inserts the tail data block data;

(4) When the enc _ idx value is PATCH _ LEN +1, the insert _ PATCH _ mark module is in a waiting phase;

enc _ idx is set to PATCH _ LEN +1 in one of the following cases:

(1) Resetting the system;

(2) When start _ streams is high;

(3) The enc _ id value exceeds PATCH _ LEN;

(4) After each component code stream of each frame is finished.

Further, the counter enc _ idx is set to 0 at the beginning of a normal data block or tail data block, and then is accumulated along with the VLSI clock until PATCH _ LEN +1;

at some point, a normal data block begins if the following conditions are simultaneously met:

(1) The enc _ idx value is PATCH _ LEN +1;

(2) The number of the remaining data in at least one FIFO in the three FIFOs is not less than PATCH _ LEN, and the FIFO with the most remaining data is selected as the FIFO to be read currently;

at some point, a tail data block starts if the following conditions are met:

(1) The enc _ idx value is PATCH _ LEN +1;

(2) At least one FIFO of the three FIFOs is not empty;

(3) The encoder is at the end of outputting a frame.

Further, the output line of the code stream selector further includes:

(1) is _ tail: when the level is high, indicating the tail data block currently processed by the insert _ patch _ mark module, otherwise, indicating the tail data block is a normal data block;

(2) num _ in _ tail: indicating the number of data in the insert _ patch _ mark module tail data block.

Further, the input line of the de-mixer comprises:

(1) sys _ reset: resetting the system;

(2) pic _ start _ sig: the decoder indicates the start of encoding a frame;

(3) mulstream _ valid: enabling signals for indicating read-in mixed flow enable;

(4) mulstram _ v: reading mixed flow;

(5) rd _ stream _ sigs: signals output from the decoder, indicating data read from the three DEC _ FIFOs;

(6) mulfifo _ cnt: the number of the remaining data in the MUL _ FIFO;

the output line of the flow breaker comprises:

(1) rd _ mulfifo _ sig: a signal to read data from the MUL _ FIFO;

(2) fifo _ cnts: the number of the residual data in the three DEC _ FIFOs is used by a decoder;

(3) stream _ valids: enabling signals of each component code stream;

(4) stream _ values: and (4) each component code stream.

Furthermore, the de-mixer comprises a delete _ patch _ mark module and 3 FIFOs, wherein the delete _ patch _ mark module scans mixed flow read from the MUL _ FIFO to find the head of each data block, separates each component code stream, stores the component code stream into the 3 FIFOs, and then reads the component code stream by an image decoder;

the detect _ patch _ mark module is internally provided with a counter dec _ idx, a cache mul _ buf and a cache nbits _ left, wherein the counter dec _ idx has three different value ranges: the first range is a fixed value of 0, which is used to look up the data block header information in the mixed flow; the second range is 1 to PATCH _ LEN, which is used for searching data block data in mixed flow; the third range is a fixed value PATCH _ LEN +1, which indicates a special state; the specific implementation comprises the following steps:

(1) When dec _ idx is 0, the detect _ patch _ mark module processes the data block header;

(2) For a normal data block, when the dec _ idx value is 1 to PATCH _ LEN, the detect _ PATCH _ mark module processes data in the normal data block;

(3) For the tail data block, when the dec _ idx value is 1 to patch _ length, the detect _ patch _ mark module processes data in the tail data block;

(4) When the dec _ idx value is PATCH _ LEN +1, the detect _ PATCH _ mark module is in a waiting stage, and the de-scrambler has no data output;

the counter dec _ idx is set to PATCH _ LEN +1 in one of the following cases:

(1) Resetting the system;

(2) The dec _ idx value exceeds PATCH _ LEN;

(3) After each component code stream of each frame is output.

Further, the counter dec _ idx is set to 0 at the beginning of a block and then accumulated with the VLSI clock, and at a certain time, a block starts if the following conditions are simultaneously satisfied:

(1) The dec _ idx value is PATCH _ LEN +1;

(2) MUL _ FIFO is not empty;

(3) There is sufficient space in the three DEC _ FIFOs.

Furthermore, the buffer mul _ buf is an internal buffer in the detect _ patch _ mark module, and is used for storing mixed flow data which is read into the detect _ patch _ mark module but is not processed, and the buffer nbits _ left is the bit number of valid data in the current buffer mul _ buf;

after the detect _ patch _ mark module scans the read mixed flow data, processing the mixed flow data according to the following steps:

(1) Supplementing the data of the reader behind the effective bit in the mul _ buf;

(2) If dec _ idx is 0, scanning from the beginning of mul _ buf, if the first two bit data are 00, then processing according to the tail data block, otherwise, processing according to the normal data block, wherein the two bit data are the indexes of the component code stream;

(3) If the data block is detected to be a normal data block in the step (2), dec _ idx is gradually accumulated from 1 to PATCH _ LEN, one data is taken out from the mixed flow in each step and sent to a corresponding FIFO, and then mul _ buf and nbits _ left are updated;

(4) If the step (2) detects that the data block is the tail data block, next to read the patch _ length, dec _ idx is gradually accumulated from 1 to the patch _ length, and each step takes out one data from the mixed flow, sends the data to the corresponding FIFO, and then updates mul _ buf and nbits _ left.

Compared with the prior art, the invention comprises the flow mixer and the flow de-mixer. The mixer mixes the multi-component code stream output by the image encoder into one code stream to be transmitted in the channel. At the image decoder side, the de-mixer separates the component code streams for further processing by the image decoder. The mixer of the invention stores each component code stream of each frame in the corresponding FIFO, and when the mixer works in real time, the mixer dynamically selects the component output by the image encoder, and the component is the one with the largest number of corresponding FIFO data at a certain moment. The mixer divides the component code stream into a plurality of data blocks. When the mixed flow device puts the component code flow into the mixed flow, a piece of head information is inserted, and the head information is used for recording the index of the component and distinguishing the normal data block from the tail data block. The core of the mixer is code stream selection, and a counter is arranged in the mixer and has three different value ranges to complete different functions. The mixed flow reaches one end of a decoder through a channel and then is stored into an FIFO, a de-mixing flow reads data from the FIFO, each component code stream is separated, and the decoder completes image decoding to obtain a reconstructed image. The mixed flow is scanned by the mixed flow eliminator, the head of each data block is found, each component code stream is separated out and stored in 3 FIFOs, and then the component code streams are read by an image decoder.

Drawings

FIG. 1 is a diagram of video compression and decompression architecture;

FIG. 2 is a schematic diagram of valid time of data of a component code stream output by an image encoder;

FIG. 3 is a schematic view of the flow mixer configuration of the present invention;

FIG. 4 is a schematic view of the code stream dynamics;

FIG. 5 is a data chunk (patch) format;

fig. 6 is a schematic view of the de-mixer structure.

Detailed Description

The present invention will be further explained with reference to the drawings and specific examples in the specification, and it should be understood that the examples described are only a part of the examples of the present application, and not all examples. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

The invention provides a method for mixing and de-mixing multi-component data streams in a VLSI (Very Large Scale Integration), which is a method for VLSI (Very Large Scale Integration) realization and is used for solving the problem of mixing and de-mixing multi-component data streams of image or video compression. Such application scenarios have the following characteristics or requirements: (1) the less additional data burden the better; (2) The video flow rate of a single component is dynamically variable (as is the case with video streams); (3) The flow rate ratio between the plurality of single component video streams is not fixed; (4) The delay between the image encoder and the image decoder is not fixed. The invention reduces the data load by dynamically detecting the single component data flow, putting the component with the maximum flow into the mixed flow and recording the index information of the component by inserting a head information.

The invention comprises a flow mixer and a flow de-mixer. The mixer mixes the code streams of a plurality of components output by the image encoder into one code stream to be transmitted in a channel. At the image decoder side, the de-mixer separates the component code streams for further processing by the image decoder. The flow mixer of the invention stores each component code stream of each frame in a corresponding FIFO, and when the flow mixer works in real time, the flow mixer dynamically selects the component output by the image encoder, and the component is the one with the largest number of corresponding FIFO data at a certain moment. The mixer divides the component code stream into a plurality of data blocks. When the mixer puts the selected component code stream into the mixing stream, a head information is inserted, and the head information is used for recording the index of the component and distinguishing the normal data block from the tail data block. The core of the mixer is a code stream selection module, a counter is arranged in the mixer, and the mixer has three different value ranges to complete different functions. The mixed flow reaches one end of a decoder through a channel and then is stored into an FIFO, a de-mixing flow reads data from the FIFO, each component code stream is separated, and the decoder completes image decoding to obtain a reconstructed image. The mixed flow is scanned by the mixed flow eliminator, the head of each data block is found, each component code stream is separated out and stored in 3 FIFOs, and then the component code streams are read by an image decoder.

The typical application of the invention is for image or video compression scenes, but the method of the invention can also be used for other application scenes which need to mix a plurality of dynamic code streams into one code stream. In these application scenarios, the mixed flow does not have a time stamp and the delay of the channel is not fixed. After the de-mixer separates the separated code stream, the subsequent processing needs to process the component code streams in order according to the logic of the application itself.

Specifically, the structure and the basic principle of the flow mixer are as follows:

the mixer structure is as shown in fig. 3, and First 3 FIFOs (First Input First Output) are needed to buffer 3 component code streams Output from the image encoder. According to the data quantity in the 3 FIFOs, the component with the largest number of the residual data is selected, a fixed number (represented by an adjustable parameter PATCH _ LEN) of data blocks (called normal data blocks) are taken and placed into the mixed stream, and the process is repeated until the number of data of each component code stream of each frame is less than PATCH _ LEN. The mixing process can be seen in fig. 4, where the mixed data blocks are placed in the mixing flow in the sequence ABCDEFHIG. At the end of each frame of image, after the normal data block is transmitted, the TAIL (TAIL 1, TAIL2, TAIL 3) of each component code stream is processed, and the TAIL data in the mixed stream is called TAIL data block (TAIL patch). In this mixed flow, the order of the data blocks of a particular component is unchanged from the order of the output of the original image encoder.

The format of the data block is shown in fig. 5, and the normal data block and the tail data block have different formats. Each normal data block needs a header (patch head) to indicate the component index of the original code stream, and for an image, the number of components is generally 3, and the index of the components can be represented by 2 bits, that is, 1 to 3. The data (PATCH data) in the normal data block is pieces of PATCH _ LEN data, for example, pieces of agent 32 (unsigned 32-bit bits). The head of each tail data block is 00, the tail _ length in the tail data block indicates the number of data in the tail, and the tail _ length data in the patch data block comprises the tail _ length data.

In the mixer, a stream _ selector selects which FIFO to read a component stream from at a particular time, and an insert _ patch _ mark module inserts header information (patch head) in the selected data block.

The input lines of the mixer are as follows:

(1) sys _ reset: resetting the system;

(2) start _ streams: the encoder indicates a frame encoding start;

(3) picture _ end, the encoder indicates that encoding of a frame is finished;

(4) stream _ signals: an enable signal (multiplexing) indicating the enablement of the component code stream output by the encoder;

(5) stream _ values: multiplexing line, component code stream output by coder.

The output line of the mixer is as follows:

(1) mulstram _ sig: a mixed flow output enable signal;

(2) mulstram _ v: and (6) mixed flow output.

Specifically, the working principle of the code stream selection module in the flow mixer is as follows:

the input lines of the stream _ selector are as follows:

(1) sys _ reset: resetting the system;

(2) start _ streams: the encoder indicates a frame encoding start;

(3) pic _ end: the encoder indicates that a frame is encoded to be finished;

(4) fifo _ cnt1: the number of the residual data in the ENC _ FIFO 1;

(5) fifo _ cnt2: the number of the residual data in the ENC _ FIFO 2;

(6) fifo _ cnt3: the number of remaining data in ENC _ FIFO 3.

The code stream selection module is provided with a counter enc _ idx which has three different value ranges to complete different functions: (a) The first range is a fixed value (e.g., 0) indicating that data block header information is inserted in the mixed flow output; (b) The second range is used for indicating the insertion of data block data in the mixed flow output; (c) The third range is a fixed value (e.g., PATCH _ LEN + 1) to indicate that the mixer is in a particular state.

For convenience of description, in the following description: (a) the first range is a fixed value of 0; (b) a second range from 1 to PATCH _ LEN; (c) the third range is PATCH _ LEN +1. The specific implementation is as follows:

(1) When enc _ idx is 0, the insert _ patch _ mark module indicates to insert a patch head (when the stream _ selector output line patch _ start is high, delayed to the input line insert _ start of the insert _ patch _ mark);

(2) For normal data blocks, when enc _ idx values are 1 to PATCH _ LEN, insert _ PATCH _ mark indicates to insert normal data block data (at this time, one line of rd _ stream1, rd _ stream2 and rd _ stream3 of stream _ selector is high, and is transmitted to stream _ sig1, stream _ sig2 and stream _ sig3 corresponding to insert _ PATCH _ mark after delay);

(3) For the tail data block, when the enc _ idx value is 1 to patch _ length (the patch _ length is the number of tail data of a specific component code stream), insert _ patch _ mark is inserted into the tail data block data;

(4) With the enc _ idx value PATCH _ LEN +1, insert _ PATCH _ mark is in a wait phase.

enc _ idx is set to PATCH _ LEN +1 in one of the following cases:

(1) Resetting the system;

(2) When start _ streams is high;

(3) The enc _ id value exceeds PATCH _ LEN;

(4) After each component code stream of each frame is finished.

enc _ idx is set to 0 at the beginning of a normal data block (normal PATCH) or tail PATCH, and then is incremented with the VLSI clock until path _ LEN +1. At some point, a normal data chunk (normal patch) begins if the following conditions are simultaneously met:

(1) The enc _ idx value is PATCH _ LEN +1;

(2) The remaining data number in at least one of the three FIFOs is not less than PATCH _ LEN, and the FIFO with the most remaining data is selected as the current FIFO to be read.

At some point, a tail chunk (tail patch) starts if the following conditions are met:

(1) The enc _ idx value is PATCH _ LEN +1;

(2) At least one FIFO of the three FIFOs is not empty;

(3) The encoder is at the end of a frame.

Other output lines of the code stream selector are as follows:

(1) is _ tail: when the level is high, indicating that the insert _ patch _ mark currently processes the tail data block, otherwise, indicating that the tail data block is a normal data block;

(2) num _ in _ tail: indicating the number of data in the insert _ patch _ mark tail data block.

Specifically, the structure and the working principle of the flow de-mixing device are as follows:

as shown in fig. 1, the mixed stream reaches one end of the decoder through the channel and is stored in a FIFO (MUL _ FIFO), the de-mixer reads data from the FIFO, separates the component streams, and the decoder performs image decoding to obtain a reconstructed image.

The de-mixer structure is shown in fig. 6. The de-mixing device mainly comprises a delete _ patch _ mark module and 3 FIFOs, wherein the delete _ patch _ mark scans mixed flow read from the MUL _ FIFO, finds the head of each data block, separates each component code stream, stores the component code stream into the 3 FIFOs, and then reads the component code stream by an image decoder.

The input lines of the flow breaker are as follows:

(1) sys _ reset: resetting the system;

(2) pic _ start _ sig: the decoder indicates the start of encoding a frame;

(3) mulstream _ valid: enabling signals for indicating read-in mixed flow enable;

(4) mulstram _ v: reading mixed flow;

(5) rd _ stream _ sigs: signals output from the decoder, indicating data read from the three DEC _ FIFOs;

(6) mufiffo _ cnt is the number of data remaining in the MUL _ FIFO.

The output lines of the flow de-mixing device are as follows:

(1) rd _ mulfifo _ sig: a signal to read data from the MUL _ FIFO;

(2) fifo _ cnts: the number of remaining data in the three DEC _ FIFOs (multiplexed lines) for use by the decoder;

(3) stream _ valids: each component code stream enable signal (multiplexing line);

(4) stream _ values: each component code stream (multiplexing line).

Specifically, the operation principle of the delete _ patch _ mark module is as follows:

the detect _ patch _ mark module has a counter dec _ idx and two buffers: mul _ buf, nbits _ left. The counter dec _ idx has three different value ranges to perform different functions: (a) The first range is a fixed value used to look up data block header information in the mixed flow; (b) The second range is used for searching data block data in the mixed flow; (c) The third range is a fixed value that indicates a particular state.

For convenience of description, in the following description: (a) the first range is a fixed value of 0; (b) a second range from 1 to PATCH _ LEN; (c) the third range is PATCH _ LEN +1. The specific implementation is as follows:

(1) When dec _ idx is 0, then detect _ patch _ mark module is processing the data block header:

(2) For the normal data block, when the dec _ idx value is 1 to PATCH _ LEN, detect _ PATCH _ mark is processing the data in the normal data block;

(3) For the tail data block, when the dec _ idx value is 1 to patch _ length (patch _ length is the number of data in the tail data block of the specific component code stream), detect _ patch _ mark processes the data in the tail data block;

(4) When the dec _ idx value is PATCH _ LEN +1, detect _ PATCH _ mark is in the waiting phase, and the de-scrambler has no data output.

dec _ idx is set to PATCH _ LEN +1 in one of the following cases:

(1) Resetting the system;

(2) The dec _ idx value exceeds PATCH _ LEN;

(3) After each component code stream of each frame is output.

dec _ idx is set to 0 at the beginning of a data block and then accumulated with the VLSI clock. At some point, a data block starts if the following conditions are simultaneously met:

(1) The dec _ idx value is PATCH _ LEN +1;

(2) MUL _ FIFO is not empty;

(3) There is sufficient space (e.g., at least half as much space) in the three DEC _ FIFOs.

mul _ buf is an internal buffer in the detect _ patch _ mark, and the purpose of the buffer is to store mixed flow data which is read in the detect _ patch _ mark but is not processed, and nbits _ left is the bit number of valid data in the mul _ buf at present.

The mixed flow data read in by the detect _ patch _ mark scanning is processed according to the following steps:

(1) Supplementing the data of the reader behind an effective bit (nbits _ left) in mul _ buf;

(2) If dec _ idx is 0, scanning from the beginning of mul _ buf, if the first two bit data are 00, then processing according to the tail data block, otherwise, processing according to the normal data block, wherein the two bit data are the indexes of the component code stream;

(3) If the normal data block is detected in the step (2), dec _ idx is gradually accumulated to PATCH _ LEN from 1, one data is taken out from the mixed flow in each step and sent to a corresponding FIFO, and then mul _ buf and nbits _ left are updated;

(4) If the tail data block is detected in step (2), next patch _ length is read out, then dec _ idx is gradually accumulated from 1 to patch _ length, and each step takes out one data from the mixed flow and sends it to the corresponding FIFO, and then mul _ buf and nbits _ left are updated.

The mixer of the invention mixes the code streams of a plurality of components output by the image encoder into one code stream to be transmitted in a channel. At the image decoder side, the component code stream is separated by the de-mixer for further processing by the image decoder. The invention divides each component code stream of each frame into a plurality of data blocks, each normal data block contains PATCH _ LEN (an adjustable parameter) data, wherein the bit number of each data is fixed. The tail of each component code stream of each frame may contain less than PATCH _ LEN data, and the tail data is placed in a tail data block. The component code streams output by the image encoder are placed in corresponding FIFOs, and the mixer dynamically selects the component output by the image encoder when working in real time, wherein the component corresponds to the most FIFO data at a certain moment. When the mixed flow device puts the component code flow into the mixed flow, a piece of head information is inserted, and the head information is used for recording the index of the component and distinguishing the normal data block from the tail data block. The core of the mixer is code stream selection, a counter is arranged in the mixer, and three different value ranges are provided for completing different functions: (a) The first range is a fixed value indicating the insertion of data block header information in the mixed flow output; (b) The second range is used for indicating the insertion of data block data in the mixed flow output; (c) The third range is a fixed value indicating that the mixer is in a particular state. The mixed flow reaches one end of a decoder through a channel and then is stored into a mixed flow FIFO, a de-mixing flow device reads data from the FIFO, each component code stream is separated, and the decoder completes image decoding to obtain a reconstructed image. The de-mixing device mainly comprises a delete _ patch _ mark module and other 3 component FIFOs, wherein the delete _ patch _ mark scans mixed flow read from the mixed flow FIFOs, finds the head of each data block, separates each component code stream, stores the component code stream into the 3 component FIFOs, and then reads the component code stream by an image decoder. The detect _ patch _ mark has a counter dec _ idx and two buffers: mul _ buf, nbits _ left. The counter dec _ idx has three different ranges to perform different functions: (a) The first range is a fixed value, and the data block header information is searched in the mixed flow; (b) The second range is used for searching data block data in the mixed flow; (c) The third range is a fixed value that indicates a particular state. mul _ buf is an internal buffer in the detect _ patch _ mark, and the purpose of the buffer is to store mixed flow data which is read in the detect _ patch _ mark but is not processed, and nbits _ left is the bit number of valid data in the mul _ buf at present.

Although the typical application of the present invention is directed to image or video compression scenarios, the method of the present invention can also be used in other application scenarios that require multiple dynamic code streams to be mixed into one code stream. In these application scenarios, the mixed stream has no timestamp, the delay of the channel is not fixed, and after the separated code stream is separated by the flow de-mixer, the subsequent processing needs to sequentially process the component code streams according to the logic of the application itself.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method for mixing and deframing a multi-component data stream in a VLSI, comprising: the mixer stores each component code stream in a corresponding FIFO; when the mixer works in real time, the component output by the image encoder is dynamically selected, and the component corresponds to the maximum number of residual data in the FIFO at a certain moment; the mixer divides the component code stream into a plurality of data blocks; when the mixer puts the selected component code stream into the mixing stream, inserting head information which is used for recording the index of the component and distinguishing a normal data block and a tail data block; a counter is arranged in a code stream selection module of the mixer and has three different value ranges to finish different functions; and the mixed flow reaches one end of an image decoder through a channel and is stored in an FIFO, a de-mixing flow reads data from the FIFO, the head of each data block is found, each component code stream is separated out and is read by the image decoder, and image decoding is completed to obtain a reconstructed image.

2. The method of claim 1, wherein the mixer needs 3 FIFOs to buffer 3 component code streams outputted from an image encoder, selects the component with the largest number of the remaining data according to the number of the remaining data in the 3 FIFOs, selects the normal data block with a fixed amount of data to be placed in the mixed stream, and repeats the process until the number of data in each component code stream is less than the number of data in the normal data block; at the end of each frame of image, after the normal data block is transmitted, the tail part of each component code stream is processed, the tail part data in the mixed flow is called tail data block, and the sequence of the data block with specific component in the mixed flow is the same as the output sequence of the original image encoder; each normal data block needs a header to indicate the component index of the original code stream, and the normal data block contains PATCH _ LEN data; the head of each tail data block is 00, the tail _ length in the tail data block indicates the number of data in the tail, and the tail _ length data in the patch data block comprises the tail _ length data.

3. A method of mixing and deswirling a multi-component data stream in a VLSI device according to claim 2, wherein the input lines to the mixer comprise:

(1) sys _ reset: resetting the system;

(2) start _ streams: the encoder indicates a frame encoding start;

(3) picture _ end: the encoder indicates that a frame is encoded to be finished;

(4) stream _ signals: an enable signal indicating an enable of the component code stream output by the encoder;

(5) stream _ values: multiplexing line, component code stream output by coder;

the output line of the mixer includes:

(1) mulstram _ sig: a mixed flow output enable signal;

(2) mulstram _ v: and (5) mixed flow output.

4. A method of mixing and decompressing multi-component data streams in VLSI according to claim 3, wherein the mixer comprises a bitstream selection module and an insert _ patch _ mark module, wherein the bitstream selection module selects from which FIFO the component bitstream is read at a specific time, the insert _ patch _ mark module inserts header information in the selected data block;

the input line of code stream selection module includes:

(1) sys _ reset: resetting the system;

(2) start _ streams: the encoder indicates a frame encoding start;

(3) pic _ end: the encoder indicates that a frame is encoded to be finished;

(4) fifo _ cnt1: the number of the residual data in the ENC _ FIFO 1;

(5) fifo _ cnt2: the number of the residual data in the ENC _ FIFO 2;

(6) fifo _ cnt3: the number of the residual data in the ENC _ FIFO 3;

the code stream selection module is internally provided with a counter enc _ idx, and the counter enc _ idx has three different value ranges: the first range is a fixed value of 0, indicating the insertion of data block header information in the mixed flow output; the second range is 1 to PATCH _ LEN, which indicates the insertion of data block data in the mixed flow output; the third range is a fixed value PATCH _ LEN +1 for indicating that the mixer is in a special state; the specific implementation comprises the following steps:

(1) When enc _ idx is 0, the insert _ patch _ mark module indicates that a data block header is inserted;

(2) For normal data blocks, when the enc _ idx value is 1 to PATCH _ LEN, the insert _ PATCH _ mark module indicates insertion of normal data block data;

(3) For the tail data block, the enc _ idx value is 1 to the patch _ length, and the insert _ patch _ mark module inserts the tail data block data;

(4) When the enc _ idx value is PATCH _ LEN +1, the insert _ PATCH _ mark module is in a waiting phase;

enc _ idx is set to PATCH _ LEN +1 in one of the following cases:

(1) Resetting the system;

(2) When start _ streams is high;

(3) The enc _ id value exceeds PATCH _ LEN;

(4) After each component code stream of each frame is finished.

5. A method of swizzling and deframing a multi-component data stream in VLSI applications as claimed in claim 4, characterized in that the counter enc _ idx is set to 0 at the beginning of a normal or tail data block and then incremented with the VLSI clock until PATCH _ LEN +1;

at some point, a normal data block starts if the following conditions are met simultaneously:

(1) The enc _ idx value is PATCH _ LEN +1;

(2) The number of the remaining data in at least one FIFO in the three FIFOs is not less than PATCH _ LEN, and the FIFO with the most remaining data is selected as the FIFO to be read currently;

at some point, a tail data block begins if the following conditions are simultaneously met:

(1) The enc _ idx value is PATCH _ LEN +1;

(2) At least one FIFO of the three FIFOs is not empty;

(3) The encoder is at the end of outputting a frame.

6. The method of mixing and deframing a multi-component data stream in a VLSI device of claim 5, wherein the output line of the codestream selector further comprises:

(1) is _ tail: when the level is high, indicating the tail data block currently processed by the insert _ patch _ mark module, otherwise, indicating the tail data block is a normal data block;

(2) num _ in _ tail: indicating the number of data in the insert _ patch _ mark module tail data block.

7. A method of mixing and unmixing a multi-component data stream in a VLSI application as claimed in claim 1, wherein the input line to the unmixing valve comprises:

(1) sys _ reset: resetting the system;

(2) pic _ start _ sig: the decoder indicates the start of encoding a frame;

(3) mulstram _ valid: enabling signals for indicating read-in mixed flow enable;

(4) mulstram _ v: reading mixed flow;

(5) rd _ stream _ sigs: signals output from the decoder indicating reading of data from the three DEC _ FIFOs;

(6) mulfifo _ cnt: the number of the remaining data in the MUL _ FIFO;

the output line of the flow breaker includes:

(1) rd _ mulfifo _ sig: a signal to read data from the MUL _ FIFO;

(2) fifo _ cnts: the number of the residual data in the three DEC _ FIFOs is used by a decoder;

(3) stream _ valids: enabling signals of each component code stream;

(4) stream _ values: and each component code stream.

8. The method of claim 7, wherein the defragmenter comprises a delete _ patch _ mark module and 3 FIFOs, the delete _ patch _ mark module scans the defragment read from the MUL _ FIFO, finds the head of each data block, separates each component code stream, stores in the 3 FIFOs, and then reads by the image decoder;

the detect _ patch _ mark module is internally provided with a counter dec _ idx, a cache mul _ buf and a cache nbits _ left, wherein the counter dec _ idx has three different value ranges: the first range is a fixed value of 0, used to find the data block header information in the mixed flow; the second range is 1 to PATCH _ LEN, which is used for searching data block data in mixed flow; the third range is a fixed value PATCH _ LEN +1, which is used to indicate a special state; the specific implementation comprises the following steps:

(1) When dec _ idx is 0, the detect _ patch _ mark module processes the data block header;

(2) For a normal data block, when the dec _ idx value is 1 to PATCH _ LEN, the detect _ PATCH _ mark module processes data in the normal data block;

(3) For the tail data block, when the dec _ idx value is 1 to patch _ length, the detect _ patch _ mark module processes data in the tail data block;

(4) When the dec _ idx value is PATCH _ LEN +1, the detect _ PATCH _ mark module is in a waiting stage, and the de-scrambler has no data output;

the counter dec _ idx is set to PATCH _ LEN +1 in one of the following cases:

(1) Resetting the system;

(2) The dec _ idx value exceeds PATCH _ LEN;

(3) After each component code stream of each frame is output.

9. A method of mixing and deframing a multi-component data stream in a VLSI circuit as claimed in claim 8, wherein the counter dec idx is set to 0 at the start of a block and then incremented with the VLSI clock, at a certain time, if the following conditions are met simultaneously, a block starts:

(1) The dec _ idx value is PATCH _ LEN +1;

(2) MUL _ FIFO is not empty;

(3) There is sufficient space in the three DEC _ FIFOs.

10. The method of claim 8, wherein the buffer mul _ buf is an internal buffer in the detect _ patch _ mark module for storing the mixed flow data that has been read into the detect _ patch _ mark module but has not been processed, and the buffer nbits _ left is a bit of valid data in the present buffer mul _ buf;

after the detect _ patch _ mark module scans the read mixed flow data, processing the mixed flow data according to the following steps:

(1) Supplementing the data of the reader behind the effective bit in the mul _ buf;

(2) If dec _ idx is 0, scanning from the beginning of mul _ buf, if the first two bit data are 00, then processing according to the tail data block, otherwise, processing according to the normal data block, and the two bit data are the index of the component code stream;

(3) If the data block is detected to be a normal data block in the step (2), dec _ idx is gradually accumulated from 1 to PATCH _ LEN, one data is taken out from the mixed flow in each step and sent to a corresponding FIFO, and then mul _ buf and nbits _ left are updated;

(4) If the step (2) detects that the data block is the tail data block, next to read the patch _ length, dec _ idx is gradually accumulated from 1 to the patch _ length, and each step takes out one data from the mixed flow, sends the data to the corresponding FIFO, and then updates mul _ buf and nbits _ left.

Images