WO2006072853A1

WO2006072853A1 - Method of and device for synchronizing multiple input streams

Info

Publication number: WO2006072853A1
Application number: PCT/IB2005/054354
Authority: WO
Inventors: Alphonsus A. J. De Lange; Freddy Snijder
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2005-01-04
Filing date: 2005-12-21
Publication date: 2006-07-13

Abstract

A method of synchronizing multiple input streams (In-I, In-2, In-3 ... In-N) fed to separate input channels, wherein the input streams comprising packets (IPlO, IPl 2; IP20; IP30; IPNO) that contain time codes (T+0, T+2; T+2; T+l; T+3), comprises the steps of: allocating memory space in a buffer array (2) for assembly packets (AP), wherein each assembly packet (AP) is identified by a specific time code (T+0 to T+3), which time code is associated with an address of the allocated memory space; for each input channel, reading the packets arriving at said input channel, checking their time codes and writing each packet into a specific part of a respective one of the assembly packets (AP), wherein the specific part is determined by the address associated with the time code of the packet and by an offset from said address, which offset depends on the channel number of the respective input channel; reading the assembly packets (AP) from the buffer array and outputting them in chronological order.

Description

Method of and device for synchronizing multiple input streams

The invention relates to a method of synchronizing multiple input streams of data that hold information of different parts or aspects of at least one original stream of data, wherein each of said input streams of data being fed to a separate input channel and the data of each input stream being provided in packets that contain time codes. The invention further relates to a multi- stream synchronizer for synchronizing multiple input streams of data that hold information of different parts or aspects of at least one original stream of data, wherein each of said input streams of data being fed to a separate input channel and the data of each input stream being provided in packets that contain time codes. The invention further relates to a computer program product directly loadable into the memory of a programmable device, comprising software code portions for performing the steps of a method according to the first paragraph when said product is run on the device.

The invention further relates to a data processing device comprising a multi- stream synchronizer according to the second paragraph.

Today's consumer electronics devices and/or systems perform many different signal processing operations, which are implemented as stream processes and processors that read data from system buffers, modify it and write the modified data to (other) system buffers. To obtain correct stream processing functionality, the reading and writing of streaming data from system buffers is protected by a streaming protocol that prevents the loss of streaming data and the insertion of erroneous data. All operations on streaming data, such as audio data and video data are done this way. An additional problem occurs when the data stream is split in different parts that are processed by different processing tasks in different streaming paths. An example of such is the difference in audio, video and data processing, for which different signal processing operations are required. Yet another example is the analysis of audio and video signal parameters to detect specific features in an audio / video data stream (A/V stream), e.g. the start of a commercial or the occurrence of a player scoring a goal in soccer match. These events are related to a specific time instant and duration in the original stream, but the delay paths in the stream processing for the different event detectors are different from each other and also different from the delays that occur in the video and audio processing.

In a system with mass storage, audio, video and data streams are preferably stored in a synchronized way, as to facilitate and speed up the retrieval process. This is even more important for streams that contain metadata for some other A/V stream, where each data item refers to a specific A/V packet / frame. Namely, when several data streams are generated by different A/V analysis processing elements in different stream processing chains, it is important that these are aligned with one another to enable data analysis algorithms to work on consistent sets of metadata that belong to the same time instant and/or interval. In general, synchronization is necessary wherever signal processing is done on multiple streams that hold information of different aspects of a single original stream. A standard way of doing this is to add time codes to the different streams as they get split and fed into different processing paths. To merge and synchronize streams, all but one of the streams are delayed such that packets of the different streams with the same time code become available at the same time and can be used at the same time. The principle of this known solution, where time-codes are used in combination with a delay to synchronize two or more streams, is outlined in Fig. 1. Each packet of the input streams arriving at the input channels is time stamped with a time code and packets are buffered to enable simultaneous play-out of packets of different streams with the same time code. In this approach all packets from each input channel must be buffered. An independent thread per input channel usually does the reading from an input channel and writing of packets to an internal buffer; this in order not to induce interference between the different input channels. Yet another 'output' thread checks the internal buffers to find packets with the same time code. As soon as the output thread has discovered such a packet in the internal buffer of each channel, then it selects these packets from all input channel buffers, assembles a new packet holding all the selected packets, attaches the same time code and outputs the assembly packet.

An implementation of this known principle is provided in the document US 2002/0080399 Al. This document discloses a data processing apparatus and method for separating and decoding a bit stream including object data of one or plural coded moving image and audio, in units of the object data, compositing the one or plural object data thus decoded, and outputting the result of composition. Time information is used for synchronization management of the moving image and audio from the object data.

The known solutions, however, suffer from the following disadvantages: - Performance is low, since the 'output' thread must poll all internal buffers to check for time codes. Depending on the data-rate of incoming streams, this may require high computational effort. Another option is to notify the "output" thread whenever a new packet is put into an internal buffer. This way, the "output" thread only checks internal buffers when a new packet has been written to one. In this approach, the output thread has to find the newly written packet in the internal buffers, check its time code and compare this with the time codes of packets written so far. This requires either searching on other internal buffers or in an internal administration of time codes, internal buffer identifiers and packet addresses within an internal buffer. Also this is time consuming and requires a complex administration.

Required memory space is high, since for each input channel, an internal buffer must be pre-allocated. Some of these buffers will only be used for a small part, because it may turn out that some streams need be delayed much less than other ones to achieve synchronization. Furthermore, internal buffers are kept big to store packets that can never be synchronized anyway, because other streams do not have packets with same time codes. A major reason for this is that some streams may be (re)started later, such that packets of certain streams only have time codes with a 'later' time code.

Robustness of the system is poor. Particularly, if one of the streams does not contain a particular time code, e.g. due to improper start-up behavior, graph reconfiguration or restarting of specific processing chains, then stream synchronization will fail.

It is an object of the invention to provide a method of the type defined in the opening paragraph, and a multi-stream synchronizer defined in the second paragraph, wherein the disadvantages of prior art mentioned above are avoided.

In order to achieve the object defined above, with a method according to the invention characteristic features are provided so that a method according to the invention can be characterized in the way defined below, that is:

A method of synchronizing multiple input streams of data, which input streams hold information of different parts or aspects of at least one original stream of data, wherein each of said input streams of data being received via an individual input channel and the data of each input stream being provided in packets that contain time codes, wherein the method comprises the steps of: allocating memory space in a buffer array for assembly packets, wherein each assembly packet is identified by a specific time code, which time code is associated with an address of the memory space allocated to the respective assembly packet in the buffer array; for each input channel, reading the packets arriving at said input channel, checking the time code of said packets and writing each packet into a specific part of a respective one of the assembly packets, wherein the specific part is determined by the address associated with the time code of the packet and by an offset from said address, which offset depends on the channel number of the respective input channel, so that each assembly packet is filled with one packet of each input stream having the same time code as the assembly packet; reading the assembly packets from the buffer array and outputting them in chronological order via at least one output channel.

In order to achieve the object defined above, with a multi-stream synchronizer according to the invention characteristic features are provided so that a multi-stream synchronizer according to the invention can be characterized in the way defined below, that is:

A multi-stream synchronizer for synchronizing multiple input streams of data, which input streams hold information of different parts or aspects of at least one original stream of data, wherein each of said input streams of data being fed to a separate input channel and the data of each input stream being provided in packets that contain time codes with a thread manager controlling multiple input threads and at least one output thread, and with a buffer array providing memory space allocatable to assembly packets, wherein each assembly packet is identified by a specific time code, which time code is associated with an address of the memory space allocated to the respective assembly packet in the buffer array, wherein one input thread is assigned to each input channel and each input thread is adapted to read the packets arriving at said input channel, to check the time code of said packets and to write each packet into a specific part of one of the assembly packets, wherein the specific part is determined by the address associated with the time code of the packet and by an offset from said address, which offset depends on the channel number of the respective input channel, so that each assembly packet is filled with one packet of each input stream with the same time code as the assembly packet, and wherein the at least one output thread is adapted to read the assembly packets from the buffer array and to output them in chronological order via at least one output channel. In order to achieve the object defined above, with a computer program product according to the invention characteristic features are provided so that a computer program product according to the invention is directly loadable into the memory of a programmable storage device and comprising software code portions for performing the steps of a method according to the invention when said product is run on the device.

In order to achieve the object defined above, with a data processing device according to the invention characteristic features are provided so that a data processing device according to the invention comprises a multi-stream synchronizer according to the invention. The characteristic features according to the invention provide the advantage that, compared with the prior art, much higher performance can be achieved, since the indexing process according to the invention is straightforward and requires very little computational effort. Further, multi-stream synchronization according to the invention reduces the memory requirements for the buffer. Finally, the invention provides high robustness of the synchronization process and offers various fallback options in the event that problems occur with the input packets fed to the various input channels.

The measures as claimed in claim 2, claim 3, claim 14 or claim 15, respectively, provide the advantage that the access to the buffer array for read and write operations are very straightforward and require very little computational effort. In order to assure outputting the assembly packets in strict chronological order, organizing of the buffer array as a ring buffer further reduces the computational efforts.

The measures as claimed in claim 4 provide the advantage that a well-defined beginning of synchronization can be achieved, avoiding any locking of the process or loss of data. The measures as claimed in claim 5 provide the advantage that the size of allocated memory can be adapted, so that long-term synchronization between the input streams is achieved without the necessity to skip data due to run-out of memory.

The measures as claimed in claim 6 provide the advantage that for each packet sent and received the size may be variable. This is of particular advantage when components are replaced 'on-the-fly', the new components resulting in different packet sizes.

The measures as claimed in claim 7 and 8 provide the advantage that the allocation of memory in the buffer array for storing packets can be handled dynamically. Basically, the required delay for each stream is detected, and with this knowledge the appropriate amount of memory can be allocated. The measures as claimed in claim 9 to claim 12 provide robustness and fallback options to the system, so that locking of the system or data loss can be prevented.

It should be noted that the features of the inventive method could be directly implemented in the multi-stream synchronizer.

The aspects defined above and further aspects of the invention are apparent from the exemplary embodiment to be described hereinafter and are explained with reference to this exemplary embodiment.

The invention will be described in more detail hereinafter with reference to an exemplary embodiment. However, the invention is not limited to this exemplary embodiment.

Fig. 1 shows the principle of prior art multi-stream synchronization in the form of a block circuit diagram. Fig. 2 shows a multi-stream synchronizer according to the invention in the form of a block circuit diagram.

Fig. 3 shows schematically a multi-stream synchronizer according to the invention with multiple input streams and one output stream.

Fig. 4 shows schematically a multi-stream synchronizer according to the invention with multiple input streams and multiple output streams.

Fig. 1 shows a multi-stream synchronizer 1 for synchronizing multiple input streams In-I, In-2, In-3 ... In-N of data that hold information of different parts or aspects of at least one original stream of data, which original stream is not depicted in the drawing, since it is not essential for the invention. It suffices to exemplarily mention that such an original stream can be an audio or video stream or some kind of other data stream that originates from an external source, like a mass storage system or a transmission system of a content provider, e.g. a computer network or a cableless network. It should also be mentioned that the present invention is also suited to internet streaming. Further, the invention is also suited to synchronizing streams that do not originate form a single source, e.g. for viewing applications based on different data sets that need be aligned (synchronized) in time for enhanced viewing. At some stage of processing the original stream has been split into the various input streams In-I, In-2, In-3 ... In-N, each of them is fed to a separate input channel and received via that individual input channel. The data of each input stream In-I, In-2, In-3 ... In-N being provided in packets IPlO, IP12; IP20; IP30; IPNO that contain time codes T+0, T+2; T+2; T+l; T+3. The multi- stream synchronizer 1 comprises a thread manager 3 controlling multiple input threads 1-N and at least one output thread, and a buffer array 2 providing memory space allocatable to assembly packets AP, wherein each assembly packet AP is identified by a specific time code, for instance time code T+l. Each assembly packet AP contains one packet from each input stream, but all having the same time code, for instance, in completely filled state the assembly packet AP(T+ 1) has to contain the input packets IPl 1 , IP21 , IP31 ... IPNl , each of which is assigned the time code T+l . The assembly packets AP are created directly by the input channel threads. Instead of writing to separate internal buffers (one for each input stream), an input packet IPlO; IP20; IP30; IPNO is directly written to a specific part of an assembly packet AP. To this purpose, each input thread 1-N uses the time code T+0; T+2, T+l, T+3 of the packet IPlO; IP20; IP30; IPNO to index the single internal buffer array 2. The time code identifies a single output assembly packet AP(T+x) where x=l, 2, 3 ..., which is associated with an address area in the internal buffer array 2, while the input stream In-I, In-2 ... In-N determines an offset within the assembly packet AP(T+x). This new approach is depicted in Figure 2. It shows the internal buffer array 2, where each input thread 1-N writes input packets IPlO; IP20; IP30; IPNO to a different row in the array at a column with a time code T+0; T+2, T+l, T+3 equal to the time code of the input packet. The output thread reads an assembly packet AP, e.g. AP(T+1), i.e. the combination of all input packets that have the same time code (single column in the internal buffer array 2). As can be seen in Fig. 2 buffer array 2 is organized in 2-dimensional manner.

It should be mentioned that the time codes T+0, T+l, T+2 and T+3, depicted in Fig. 2, only serve as a simplified example. In an actual implementation time-stamps are usually not incremented with only T, but with a fixed minimal number of milliseconds, e.g. 'Dmin'. So packets with time codes: (T + Dmin, T + 2 x Dmin, T+3 x Dmin, , T + M x Dmin) are created for 'M' simultaneously buffered assembly packets. Of course, it is possible to store more than the four assembly packets depicted in Fig. 2, if required. This only depends on the extend of required synchronization in a specific system.

In order to thoroughly enlighten the use of packets with incorporated time codes, a mathematically more precise denotation is given below: Packets Pjk arrive at input channels Ij, j=l, 2, ...N, at time T with time stamps Tk = T + k x Dmin, k = 1,2, ... M, where the difference Da,b = |Ta - Tb | between the time stamps Ta, Tb of any two packets Pja, Pib, i != j, is smaller than a certain maximum value Dmax ( I Ta - Tb | <= Dmax), and is a multiple of the time- stamp interval Dmin. The maximum number of assembly packets that can be buffered simultaneously is then M = Dmax / Dmin. The value of M can be determined by measuring the difference between the time-stamp- values of all incoming packets for all inputs. Once at least a single packet is received at all inputs, Dmax can be determined and subsequently M. Using this value and the size of each packet, the internal buffer array can be dimensioned. By taking just enough least significant bits from Tk to hold M x Dmin, the buffer array can be addressed as a ring buffer. The address of a particular assembly packet in linear memory is then given by JUST- ENOUGH-LSB(Tk) / Dmin x SIZEOF (Assembly-Packet).

The indexing process according to the invention is straightforward and requires very little computational effort. One embodiment of this indexing process is to use a 'buffer manager' that takes the time code and stream identifier as an input. It simply takes part of the least significant bits of the time code to index a circular buffer array holding assembly packets. By choosing a particular number of bits of the time code for indexing, a ring buffer is automatically established. The number of selected LSBs of the time code determines the number of entries of the ring buffer. It should be observed that the function of the 'buffer manager' can be fulfilled by the thread manager 3.

To write to a particular entry in the internal buffer array 2, the size of packets of each input stream must be known as well as the time code of each packet.

The output thread simply reads from the internal buffer and outputs the assembly packets in chronological order. As soon as such an assembly packet is output, the internal buffer space is available for reuse. As a result, the invention minimizes the performance requirements for synchronization.

To reduce memory requirements, the multi-stream synchronizer according to this invention can start up in "analyzing mode". This means that it first analyzes the incoming streams In-I, In-2, In-3, In-N before starting the synchronization. In most consumer and internet streaming applications this is perfectly acceptable, if the analysis takes only a short time, unnoticeable to the user.

The analysis consists of detecting packets in all of the incoming streams that have the same time code. Once these have been detected, the system can deduce the required delay for each stream and allocates the appropriate amount of memory for this. The detection is done with a simple algorithm: it simply checks the time code of packets read by the input threads. It reads packets and discards them. As soon as at least one packet IPlO; IP20; IP30; IPNO has been read from all input streams, then the time codes T+0; T+2, T+l, T+3 of these packets are compared. Of all the packets read so far from all input streams, the one with the highest time code (T+3) serves as the synchronization starting point. All packets from other streams with a lower time code are discarded, since they cannot be synchronized with the packets from the stream from which the packet with the highest time code was selected.

The following assumption must hold: all incoming streams In-I, In-2, In-3, In- N produce packets at the same rate. This is necessary to enable long-term synchronization between these streams. If this is not the case, skipping of data is necessary in order to keep the memory requirements within certain limits. What may differ per input stream is the bit rate. This depends on the size of packets transported by each of the streams.

Using the above assumption, it is easy to compute the required memory per stream for synchronization. Having read at least one packet for each stream, the difference in time code between the last read packet with the highest time code (T+3) and the last read packet with the smallest time code (T+0) can be determined. The same can be done for all other streams: determining the difference in time code for the currently read packet and the last read packet from some stream that has the lowest time code of them all. For each of these streams this difference determines the memory delay requirements for a stream to enable synchronization. By using an appropriate time code and using the size of packets for each stream, the exact (minimal) memory size can be determined and allocated.

Possible drawback of the new approach is data loss. However, this is not really relevant, if this data cannot be synchronized anyway. If some stream stops producing data, because some module in the system is stopped, needs more time to compute new data, or has crashed, then synchronization may fail. In this case, the synchronization module continues running, but will not be able to synchronize all streams at all time anymore. Several options are possible in this case:

1. Synchronization is done with the remainder of the streams, producing partly filled assembly packets;

2. All incoming packets are discarded and no assembly packets are produced. This option will turn the synchronizer back into "analyzing" mode, in which streams are analyzed for synchronization set-up as described above. This mode can be considered as a restart of the streaming system. 3. Do not output an assembly packet, when one or more incoming packets are still missing, but wait until they arrive. This waiting has a time-out. When a time-out occurs (or buffer overflow), then option 1 or 2 can be chosen as a fallback. 'Waiting' and 'time-out' need to be set beforehand for each input. 4. Similar to option 1, but now a pre-defined 'default' packet is substituted for the missing packet. This is only allowed if a default packet is specified beforehand for the particular input. If no default is specified, the system typically returns to option 2: re-start. Which of the options to choose is a system- level decision. For each packet sent and received, the size may be pre-defined or is variable. In most cases, the size of packets produced by a certain component will be constant, but variations may occur when components are replaced 'on-the-fly'.

A robust way of handling this is to explicitly pass a series of bytes Szl2 that denote the size of the packet IP 12 that is sent directly after the size information. If data communication is done this way, the synchronizer 1 can dynamically adapt the internal buffer array 2, the allocation of packets and the read-out of assembly packets AP. This will involve dynamic memory management.

The invention offers several output styles for assembly packets. An assembly packet can be output as a series of bytes, as follows: Send size of time code of assembly packet; Send time code;

For all packets ^Λi' of assembly packet Send size of packet ^Λi' ; Send packet ^Λi' ;

Another way is to simply start reading the assembly packet from its start in the internal buffer array:

Send size of time-code; Send time-code; Send size of assembly packet; Send assembly packet, byte per byte; The multi-stream synchronizer 1 can send assembly packets AP through one output channel Out, as is depicted in Fig. 3, or through different output channels Out-1, Out-2, Out-3, as is depicted in Fig. 4. In the latter case, the multi-stream synchronizer 1 has the same number of input and output ports. Its only purpose is to assure that all packets of all streams that leave the synchronizer at the same time have the same time code. The inventive multi-stream synchronizer may be embedded in form of either hardware or by the aid of software executable by the data processing device that processes data in form of data streams. Such a data processing device can be realized as an internet enabled appliance, e.g. a data server connected to the internet on which for example audio / video processing software is executed or as a portable or stationary stand alone consumer electronics device and / or system, such as an audio player and/or recorder or a video recorder and / or player or a video camera or the like and any combination thereof. Also a personal computer or a portable computer such as a personal digital assistant (PDA) or a mobile phone can realize the data processing device. In many situations a computer program product that comprises software code portions for performing the steps of the method according to the invention when the computer program product is run on the data processing device is already pre-stored in such a data processing device, e.g. in a ROM or EPROM or any other permanent memory. It may also be that the computer program product can be fed into the data processing device by the aid of a data carrier on which the computer program product is stored. In both cases the computer program product is typically loaded into the working memory of the device, e.g. the RAM, and software portions of the computer program are executed by a processor of the device.

Claims

CLAIMS:

1. A method of synchronizing multiple input streams (In- 1 , In-2, In-3 ... In-N) of data, which input streams (In-I, In-2, In-3 ... In-N) hold information of different parts or aspects of at least one original stream of data, wherein each of said input streams (In-I, In-2, In-3 ... In-N) of data being received via an individual input channel and the data of each input stream being provided in packets (IPlO, IP12; IP20; IP30; IPNO) that contain time codes (T+0, T+2; T+2; T+l; T+3), wherein the method comprises the steps of: allocating memory space in a buffer array (2) for assembly packets (AP), wherein each assembly packet (AP) is identified by a specific time code (T+0 to T+3), which time code is associated with an address of the memory space allocated to the respective assembly packet in the buffer array; for each input channel, reading the packets (IPlO, IP12; IP20; IP30; IPNO) arriving at said input channel, checking the time code (T+0, T+2; T+2; T+l; T+3) of said packets and writing each packet into a specific part of a respective one of the assembly packets, wherein the specific part is determined by the address associated with the time code of the packet and by an offset from said address, which offset depends on the channel number of the respective input channel (In-I, In-2, In-3 ... In-N), so that each assembly packet (AP(T+1)) is filled with one packet (IPl 1; IP21; IP31; IPNl) of each input stream (In-I, In-2, In-3 ... In-N) having the same time code (T+l) as the assembly packet; reading the assembly packets (AP) from the buffer array and outputting them in chronological order via at least one output channel.

2. A method as claimed in claim 1, wherein the buffer array (2) is organized as a 2-dimensional array having columns and rows, wherein the columns are associated with assembly packets (AP) and their specific time codes (T+0 to T+3), respectively, and the rows are associated with the offsets for each input channel (In-I, In-2, In-3 ... In-N).

3. A method as claimed in claim 1 or 2, wherein the buffer array (2) is organized as a ring buffer.

4. A method as claimed in claim 1, wherein a time code as a starting point for synchronization is determined by reading packets (IPlO; IP20; IP30; IPNO) from all input channels (In-I, In-2, In-3 ... In-N), comparing their time codes (T+0, T+2, T+l. T+3) and defining the highest time code (T+3) as the starting point for synchronization.

5. Method as claimed in claim 1, wherein, for each input channel (In-I, In-2, In-3 ... In-N), the bit rate is measured and the size of memory necessary to store packets of this input channel in the assembly packets is adapted according to the bitrate.

6. A method as claimed in claim 1, wherein packet size information (SzI 2) provided by the input channel (In-I) is read in advance of reading the packet (IP 12) and the size of memory that has to be allocated for storing the packet in an assembly packet is dynamically adapted according to the packet size information (SzI 2).

7. A method as claimed in claim 1 , wherein a required delay for the input streams

(In-I, In-2, In-3 ... In-N) is determined by calculating the difference in time code between the last read packet with the highest time code (T+3) of one of the input channels and the last read packet with the lowest time code (T+0) of one of the other input channels.

8. A method as claimed in claim 1, wherein a required delay for an input stream

(In-I, In-2, In-3 ... In-N) is determined by calculating the difference in time code between the packet currently read from this input stream and the last read packet from some stream that has the lowest time code of them all.

9. A method as claimed in claim 1, wherein in the event that a partly filled assembly packet (AP) should already be output in order to keep the chronological order, this partly filled assembly packet is output.

10. A method as claimed in claim 1, wherein in the event that a partly filled assembly packet (AP) should already be output in order to keep the chronological order, no assembly packet is output, but all assembly packets in the buffer array (2) are discarded, and a new time code as a starting point for synchronization is determined by reading packets from all input channels (In-I, In-2, In-3 ... In-N), comparing their time codes and defining the highest time code as the starting point for synchronization.

11. A method as claimed in claim 9 or 10, wherein outputting of the assembly packets (AP) is delayed, until the missing packets have arrived on the input channels, or a time-out has expired, or a buffer-overflow has occurred.

12. A method as claimed in claim 1, wherein in the event that a partly filled assembly packet (AP) should already be output in order to keep the chronological order, the missing packets in the assembly packet are substituted by deiault packets.

13. A multi-stream synchronizer (1) for synchronizing multiple input streams (In-

1, In-2, In-3 ... In-N) of data, which input streams (In-I, In-2, In-3 ... In-N) hold information of different parts or aspects of at least one original stream of data, wherein each of said input streams of data being fed to a separate input channel and the data of each input stream (In-I, In-2, In-3 ... In-N) being provided in packets (IPlO, IP12; IP20; IP30; IPNO) that contain time codes (T+0, T+2; T+2; T+l ; T+3), with a thread manager (3) controlling multiple input threads and at least one output thread, and with a buffer array (2) providing memory space allocatable to assembly packets (AP), wherein each assembly packet is identified by a specific time code (T+0 to T+3), which time code is associated with an address of the memory space allocated to the respective assembly packet in the buffer array, wherein one input thread is assigned to each input channel and each input thread is adapted to read the packets (IPlO, IP12; IP20; IP30; IPNO) arriving at said input channel, to check the time code (T+0, T+2; T+2; T+l ; T+3) of said packets and to write each packet into a specific part of one of the assembly packets, wherein the specific part is determined by the address associated with the time code (T+0 to T+3) of the packet and by an offset from said address, which offset depends on the channel number of the respective input channel (In-I, In-2, In-3 ... In-N), so that each assembly packet (AP(T+1)) is filled with one packet (IPI l; IP21; IP31; IPNl) of each input stream (In-I, In-2, In-3 ... In-N) with the same time code (T+l) as the assembly packet, and wherein the at least one output thread is adapted to read the assembly packets from the buffer array and to output them (AP(T+1) in chronological order via at least one output channel.

14. A multi-stream synchronizer (1) as claimed in claim 13, wherein the buffer array (2) is organized as a 2-dimensional array having columns and rows, wherein the columns are associated with assembly packets (AP) and their specific time codes (T+0 to T+3), respectively, and the rows are associated with the offsets for each input channel (In-I, In-2, In-3 ... In-N).

15. A multi-stream synchronizer as claimed in claim 13 or 14, wherein the buffer array (2) is organized as a ring buffer.

16. A computer program product directly loadable into the memory of a programmable device, comprising software code portions for performing the steps of a method according to claims 1 to 12 when said product is run on the device.

17. Data processing device comprising a multi- stream synchronizer (1) according to one of the claims 13 to 15.