WO2012054771A2

WO2012054771A2 - Synchronizing data streams over a local area network

Info

Publication number: WO2012054771A2
Application number: PCT/US2011/057164
Authority: WO
Inventors: Andre Vankov; Scott Makeig
Original assignee: The Regents Of The University Of California
Priority date: 2010-10-21
Filing date: 2011-10-20
Publication date: 2012-04-26
Also published as: WO2012054771A3

Abstract

In an aspect there is provided a method. The method may include receiving, from a first sender, a first plurality of packets, each of the first plurality of packets including at least a first time stamp representative of a first time local to the first sender of the first plurality of packets; retrieving a first correction factor determined for the first sender; correcting, at the receiver, each of the first time stamps based on the first correction factor, the corrected first time stamps providing a time local to the receiver; receiving, from a second sender, a second plurality of packets, each of the second plurality of packets including a second time stamp representative of a second time local to the second sender of the second plurality of packets; retrieving a second correction factor determined for the second sender; and correcting, at the receiver, each of the second time stamps based on the second correction factor, the corrected second time stamps providing the time local to the receiver, the corrected first time stamps and the corrected second time stamps being corrected to the same time local to the receiver. Related apparatus, systems, techniques, and articles are also described.

Description

SYNCHRONIZING DATA STREAMS OVER A LOCAL AREA NETWORK

REFERENCE TO PRIORITY DOCUMENT

[0001] This application claims priority to U.S. Provisional Patent Application Serial No. 61/405,598 entitled SYNCHRONIZING DATA STREAMS OVER A LOCAL AREA NETWORK, filed on October 21, 2010. The disclosure of the Provisional Patent Application is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

[0002] The subject matter described herein relates to data synchronization.

BACKGROUND

[0003] Computers typically include local clocks. However, the local clocks may not be synchronized to a master clock or a common clock, such as coordinated universal time (UTC). In some instances, synchronization may not be practical due to cost considerations and/or configuration control considerations (e.g., the computers may not be under the control of a single entity). At any instant in time, computers on a local area network may thus have local clocks set to different time values. These time differences can be problematic, especially when trying to correlate an event among the different computers.

SUMMARY

[0004] In an aspect there is provided a method. The method may include receiving, from a first sender, a first plurality of packets, each of the first plurality of packets including at least a first time stamp representative of a first time local to the first l sender of the first plurality of packets; retrieving a first correction factor determined for the first sender; correcting, at the receiver, each of the first time stamps based on the first correction factor, the corrected first time stamps providing a time local to the receiver; receiving, from a second sender, a second plurality of packets, each of the second plurality of packets including a second time stamp representative of a second time local to the second sender of the second plurality of packets; retrieving a second correction factor determined for the second sender; and correcting, at the receiver, each of the second time stamps based on the second correction factor, the corrected second time stamps providing the time local to the receiver, the corrected first time stamps and the corrected second time stamps being corrected to the same time local to the receiver. Related apparatus, systems, techniques, and articles are also described.

[0005] Articles of manufacture are also described that comprise machine executable instructions permanently stored on machine readable media (e.g., non- transitory computer readable media), which, when executed by a machine, causes the machine to perform the operations described herein. Similarly, computer systems are also described that may include a processor and a memory coupled to the processor. The memory may temporarily or permanently store one or more programs that cause the processor to perform one or more of the operations described herein.

[0006] The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, and from the claims. DESCRIPTION OF DRAWINGS

[0007] FIG. 1 depicts a block diagram of a system configured to provide data synchronization using time stamps;

[0008] FIG. 2 depicts a process for determining a correction factor;

[0009] FIG. 3 depicts a process for applying a correction factor to time stamps to enable data synchronization over a local area network;

[0010] FIG. 4 depicts examples of packets including time stamps;

[0011] FIG. 5 depicts an example of a table include data samples and time stamps; and

[0012] FIG. 6 depicts a block diagram of another system configured to provide data synchronization using time stamps.

[0013] Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

[0014] FIG. 1 depicts a system 100 including one or more processors, such as processors 105A-C, a local area network 150, and a receiving processor 1 10. Each of the processors 105A-C may send to the receiving processor 110 via local area network 150 a plurality of packets (also referred to as a stream of packets and/or packet streams)

120A-C.

[0015] Each packet of the packet streams 120A-C may include a time stamp having a time local to the processor sending the packet. Moreover, the local clocks at each processor 105A-C (and thus the corresponding time stamps applied by each processor 105A-C) are not synchronized among the processors 105A-C. And, the processors 105A-C do not share a common, master clock. For example, processor 105 A may apply to a packet a time stamp having a time local to processor 105 A; processor 105B may apply to a packet a time stamp having a time local to processor 105B; and processor 105C may apply to a packet a time stamp having a time local to processor 105C. When packets including time stamps are received at receiving processor 1 10, the receiving processor 110 determines correction factors for each packet stream (described further below) to enable the packet streams 120A-C from each of the processors 105A-C to be synchronized based on the one or more correction factors and times local to the receiving processor 110. Before providing additional details regarding the determination of the correction factors and the application of the correction factors to the packets in the packet streams, the following provides a general description of the system 100 of FIG. 1.

[0016] The processors 105A-C may include at least one processor and at least one memory (which may include code and/or data). The processors 105A-C may be configured to connect to, transmit to, and/or receive from receiving processor 1 10 via local area network 150. The processors 105A-C may also be configured to transmit and/or receive packets, such as Internet Protocol packets, as depicted by packet stream 120A. The processors 105A-C may each include a local area network interface card to couple to local area network 150, a local clock, and a generator for generating a packet, transmitting a packet, and/or receiving a packet via the local area network interface card and the local area network 150. In some implementations, the processors 105A-C may each comprise a device, such as a computer, a sensor, a device providing

electroencephalogram (EEG) data, a position sensor, and/or any other device capable of generating data, such as data samples. [0017] The packets of packet streams 120A-C may each include a time stamp and data, such as data samples generated, and/or provided, by the processor 105A. For example, the packets of packet stream 120A may each be configured as an Internet Protocol (IP) packet including a data portion and a header portion. Unlike a typical IP packet, the packets of packet stream 120A have a time stamp inserted into the data portion before being transmitted to the receiving processor 1 10. In some

implementations, the packets of packet stream 120 A may be further configured in accordance with the user datagram protocol (UDP).

[0018] In some implementations, a timestamp is generated based on a local clock of a computer, in which case the local clock may provide the time applied to the time stamp. For example, the time may correspond to a 24 clock, a simple counter, and/or a local clock providing a quantity of milliseconds that have elapsed since the computer was turned on (e.g., a data packet, carrying a timestamp 12,345,678 would indicate that the sample was acquired and time stamped 12,345,678 milliseconds after the computer was turned on).

[0019] The local area network 150 may be any type of wired and/or wireless local area network (LAN). Examples of local area networks include Ethernet, Gigabit Ethernet, and any other local area network type. The local area network 150 may connect computers in a room, home, school, building, or like limited geographic area, and the local area network 150 may have higher data transfer rates and thus lower delays, when compared to other network types.

[0020] The receiving processor 1 10 may include at least one processor and at least one memory (which may include code and/or data). The processor 1 10 may be configured to connect to, transmit to, and/or receive from each of processors 105A-C via local area network 150. The receiving processor 1 10 may also be configured to transmit and/or receive packets, such as Internet Protocol packets, UDP/IP packets, and the like. The receiving processor 1 10 may include a local area network interface card to couple to local area network 150, a local clock, and a generator for generating a packet, transmitting a packet, and/or receiving a packet via the local area network interface card and the local area network 150. The receiving processor 110 may be configured to receive IP packets from each of the processors 105A-C; determine correction factors for each of the processors 105A-C; and correct the time stamps included in the packet streams 120A-C to enable synchronization of the packet streams 120A-C at the receiving processor 1 10. Receiving processor 1 10 may further include a table 190 at which data from the packet streams 120A-C is synchronized according to the time stamps described herein.

[0021] FIG. 2 depicts a process 200 at the receiving processor 1 10. The process 200 may be used to determine one or more correction factors.

[0022] At 205, the receiving processor 110 receives a plurality of packets 120A from a processor, such as processor 105A. The packets 120A may each include a time stamp having a time local to the processor 105 A. As used herein, local to the processor refers to a clock that is not common to the plurality of processors 105A-C or is not shared with among the processors 105A-C. For example, a local clock at processor 105 A may generate a time stamp local only to processor 105 A, without the local clock being synchronized to a coordinated universal time (UTC) standard or without sharing a common, master clock, such as a clock provided by a global positioning system. Nor is the local clock slaved or synchronized to the local clocks of the other processors 105B-C. The time stamp is applied to a data sample at the processor when the data sample is obtained and/or measured by the processor.

[0023] At 210, the receiving processor 110 may determine a correction factor representative of a difference in time between the sender of the packet (which includes the time stamp) and the receiving processor 110. For example, when sending processor 105 A transmits to receiving processor 110 a packet stream 120 A in which each packet includes a time stamp having a time from a clock local to the sending processor 105 A, the difference in the time stamps for each packet of the packet stream 120A is a sum of the instantaneous values of the local clocks at the sending processor 105 A and the receiving processor 1 10 and the transmission delay, Dly. This relationship is expressed in the following equation:

Ts (recv) = Ts (send) + Cor + Dly,

wherein the Ts(recv) is the time stamp at the receiving processors, Ts(send) is the time stamp at the sending processor, Cor is the correction factor, and Dly is the transmission delay.

[0024] Assuming a delay, Dly, which is minimal (e.g., less than 1 millisecond is assumed to be about equal to zero), the correction factor, Cor, may be calculated in accordance with the following equation:

Cor = Ts (recv)— Ts(send).

[0025] Thus, for packet streams that possess some regularity, the correction factor may be obtained dynamically for each packet stream by tracking the minimum time delay over a period of time. [0026] At 215, the correction factor is provided to enable correction of packets sent by processor 105 A. For example, the correction factor may be provided to memory or to a central processing unit at receiving processor 1 10 to enable applying the correction factor to a packet stream 120A.

[0027] At 220, the receiving processor 1 10 may calculate correction factors for each of the other processors 105B-C. Moreover, the calculation of the correction factors for each of the processors 105A-C may be dynamically calculated (e.g., repeatedly over time) to provided updated correction factors as conditions change within the local area network 150, sending processor 105A-C, and/or receiving processor 1 10. When calculated dynamically, the dynamic recalculation of the correction factor may also compensate for drifts in the local clocks at each of the processors 105A-C.

[0028] FIG. 3 depicts a process 300, at receiving processor 1 10, for synchronizing each of the packet streams 120A-C.

[0029] At 305, the receiving processor 1 10 receives a plurality of packets 120A from processor 105A. Each of the plurality of packets 120A includes a time stamp representative of a time local to processor 105A.

[0030] FIG. 4 depicts a packet 405. The packet 405 is an example of at least one of the plurality of packets 120A. The packet 405 may include a header 412, which may include a source address and a destination address for the packet 405. The packet 405 may also include a data portion 410 (also referred to as a payload). The data portion may further include a time stamp 416 local to the processor 105 A sending the packet 405 A. The time stamp 416 may be inserted by for example processor 105 A before the packet is transmitted to receiving processor 1 10. In some implementations, the packet may be implemented as an UDP/IP packet, although other formats may be used as well.

[0031] Referring again to FIG. 3, the receiving processor 1 10 may retrieve, at 310, a correction factor for the first processor 105A. For example, the receiving processor 1 10 may retrieve from memory the correction factor determined at 210 and provided to memory at 215. In implementations where the correction factor is dynamically recalculated, the receiving processor 1 10 may select a correction factor having the greatest correction (which represents the greatest delay).

[0032] At 315, the receiving processor 110 may correct the time stamps of the packets based on a correction factor. The corrected times of the time stamps represent a time local to the receiving processor 110 rather than the sender 105 A of the packet. Referring again to FIG. 4, the correction factor may be applied to the time stamp 416 of packet 405A to yield a packet 405B having a corrected time stamp 420. The receiving processor 1 10 may thus use the correction factor to correct the time stamps of one or more of the packets in the packet stream 120A. The corresponding correction factors for processor 105B may be applied to the time stamps of the packets of stream 120B, and the correction factor determined for processor 105C may be applied to the time stamps of packets of stream 120C.

[0033] In some implementations, the correction factor is applied to any packet of streams 120A-C delayed via local area network 150 to determine the time when the packets should have arrived had it not been delayed, which is represented by the following equation:

Ts (corr) = Ts (send) + Cor, wherein Ts(corr) is the corrected time at receiving processor 1 10, Ts(send) is the value of the time stamp applied by a sending processor (e.g., processors 105A-C), and Cor is the correction factor. Thus, the corrected time represents the time when a packet should have arrived had it not been subject to network delays. The corrected time of a packet, rather than the actual time the packet arrive, may be used to synchronize with other packets from other streams, which have been corrected as described herein.

[0034] At 320, the corrected time stamp is used to enable synchronization with other corrected time stamps from other sending processors. Specifically, if each of the packet streams 120A-C are corrected as described herein, the packet streams 120A-C include a corrected time stamp, which is corrected so that the time stamps are relative to the receiving processor 1 10 rather than each of the sending processors 105A-C (which may also eliminate the effects of occasional network delay). As such, the corrected time stamps allow the packet streams 120A-C to be synchronized amongst each other based on the correction factor (e.g., based on the corrected time stamps in each packet). For example, each of the packets of the packet streams 120A-C may be synchronized relative to the time at the receiving processor 1 10.

[0035] In some implementations, the data from each packet and time stamp may be used to synchronize the data from each processor as depicted at table 190 of FIG. 5. Referring to FIG. 5, the receiving processor 1 10 extracts the data portion of each of the packets in streams 120A-C and then applies the correction factor to each of the time stamps. Because the corrected time is relative to the receiving processor 1 10, the "Data" from each of the processors 105A-C can be synchronized and/or correlated relative to the same time reference, which in this example is a time local to receiving processor 1 10. For example, row 1 at corrected time 1.0 depicts data obtained from each of the processors 105A-C; row 2 at corrected time 1.5 depicts data obtained from each of the processors 105A-C; and so forth. Thus, if an event occurs at corrected time 1.0 and each of the processors 105A-C transmits packets including data and a time stamp, the receiving processor 1 10 synchronizes the data portions as shown at row 1 of FIG. 5. The data in table 190 may include the actual data being synchronized or a pointer to the data being synchronized. The corrected time stamps described herein may thus be used to describe the temporal relationship of data samples and to permit co-registration of the data streams.

[0036] FIG. 6 depicts a system 600. The system 600 is similar to the system 100 but the system 600 includes an EEG processor 505 A as the source of packet stream 120A and a body motion capture processor 505B as the source of packet stream 120B. In some implementations, the devices 505A-B provide data samples representative of a measured quantity (e.g., EEG measurements and body position). For example, the data samples may represent EEG data samples of a patient taken at processor 505A, in which case the packet stream 120A includes the EEG data samples and the time stamps described herein. Moreover, processor 505B may provide data samples of the body position of the patient, in which case packet stream 120B includes data samples for the patient's body position and the time stamps described herein. In this example, the receiving processor 1 10 may synchronize using for example processes 200 and 300 the data samples, such that EEG data samples can be correlated to the body position of the patient. For example, when the patient raises a hand, the receiving processor 1 10 may use processes 200 and 300 to synchronize the EEG data samples and time stamps included in packet stream 120A to the body position data samples and time stamps of packet stream 120B representative of raising a hand. Thus, the receiving processor 1 10 can correlate the event of raising a hand with the corresponding EEG data samples and the body position of raising a hand. Although the previous example used body position and EEG, any other data samples may be used as well.

[0037] In some implementations, the subject matter described herein provides synchronization of data samples transmitted over a local area network based on the time stamps described herein. The processes described herein may account for, and minimize, timing errors, enabling a restoration of the temporal relationship between data sample streams created on different computers and streamed in real time. In some

implementations, the processes described herein may be operative with packet streams including data samples that are continuous and relatively uniform, although the processes are operative in other environments as well. Moreover, the processes described herein may be operative over a local network, such that the packet streams including data samples are transmitted using an Internet Protocol (IP) packet, such as a user datagram protocol (UDP). In some implementations, the UDP protocol provides a typical network delay (including delays from the operating system) which is less than about 1

millisecond.

[0038] The data samples and time stamps carried by the packet streams are buffered locally at the receiving processor for a given amount of time required for all of the delayed data packets of the stream to arrive at the receiving processor (e.g., buffered for a time greater that the longest network delay). This buffering allows for lossless (e.g., no dropped packages) synchronization of incoming packet streams, without using handshake based protocols, such as TCP. Moreover, the packet streams may be buffered at the receiving processor with a timeout. When this is the case, the buffered packets are flushed when the network delay exceeds a predetermined maximum network delay and the buffer duration is adjusted to the new length of time.

[0039] In some implementations, the time for time stamps have micro-second precision from the processor itself, which is then rounded to a millisecond.

[0040] In some implementations, the processes 200 are 300 include data samples encapsulated in UDP/IP as UDP enables packets to be sent directly to the local area network with little delay, when compared to other protocols, such as the transmission control protocol (TCP). The use of UDP, rather than TCP, enables network delays that are less than about 1 millisecond. Indeed, UDP enables unidirectional communication from the ending processor to the receiving processor rather than the bidirectional handshake protocol of TCP (which increases delay). UDP also enables sending relatively small data packets at a very high rate, thus creating a relatively continuous data stream, which, in its turn, may allow for very precise and continuous adjustment of the correction factor for each packet stream.

[0041] In some implementations, the subject matter described herein may exhibit one or more of the following: allow rapid and precise maintenance of the minimum network propagation delay (and a corresponding correction factor), and consequently, achieving a precision of less than about 1 millisecond; allow a precise adjustment of the correction factor to drifts in the relative local clock frequency of the sending computer (e.g., less than about 1 millisecond); and allow high data throughput (without handshaking). [0042] Although processes 200 and 300 are described with respect to packets, other data structures including blocks (which have a plurality of data samples) may be used as well. When a block is used, the last data sample in the block may include a time stamp, and the other data samples of the block may be extrapolated at the receive processor from the time stamp.

[0043] Various implementations of the subject matter described herein may be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations may include implementation in one or more computer programs that are executable and/or interpretable on a

programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0044] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the term "machine-readable medium" refers to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions. [0045] To provide for interaction with a user, the subject matter described herein may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user may provide input to the computer. Other kinds of devices may be used to provide for interaction with a user as well; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

[0046] Although a few variations have been described in detail above, other modifications are possible. For example, while the descriptions of specific

implementations of the current subject matter discuss analytic applications, the current subject matter is applicable to other types of software and data services access as well. Moreover, although the above description makes references to specific products, other products may be used as well. In addition, the logic flows depicted in the accompanying figures and described herein do not require the particular order shown, or sequential order, to achieve desirable results. Other embodiments may be within the scope of the following claims.

Claims

WHAT IS CLAIMED:

1. A method comprising:

receiving, from a first sender, a first plurality of packets, each of the first plurality of packets including at least a first time stamp representative of a first time local to the first sender of the first plurality of packets;

retrieving a first correction factor determined for the first sender;

correcting, at the receiver, each of the first time stamps based on the first correction factor, the corrected first time stamps providing a time local to the receiver; receiving, from a second sender, a second plurality of packets, each of the second plurality of packets including a second time stamp representative of a second time local to the second sender of the second plurality of packets;

retrieving a second correction factor determined for the second sender; and correcting, at the receiver, each of the second time stamps based on the second correction factor, the corrected second time stamps providing the time local to the receiver, the corrected first time stamps and the corrected second time stamps being corrected to the same time local to the receiver.

2. The method of claim 1 further comprising:

calculating the first correction factor based on a difference in time between a first local clock at the first sender and a local clock at the receiver.

3. The method of claim 2 further comprising:

recalculating the first correction factor for each of the first plurality of packets.

4. The method of claim 1, wherein at least one of the first time stamps is determined based on the following equation:

Ts (corr) = Ts (send) + Cor,

wherein Ts(corr) represents the at least one of the first time stamps corrected at the receiver, Ts(send) represents a value of a time stamp applied by the first sender, and Cor represents the first correction factor.

5. The method of claim 1 further comprising:

synchronizing data from each of the first and second senders based on the corrected first time stamps and the corrected second time stamps.

6. The method of claim 1, wherein the first plurality of packets comprise user data gram packets.

7. The method of claim 1, wherein each of the first plurality of packets include a header and a data portion having the first time stamp.

8. A system comprising:

at least one processor; and

at least one memory including code, wherein the at least one processor and the at one memory cause operations comprising: receiving, from a first sender, a first plurality of packets, each of the first plurality of packets including at least a first time stamp representative of a first time local to the first sender of the first plurality of packets;

retrieving a first correction factor determined for the first sender;

9. The system of claim 8 further comprising:

10. The system of claim 9 further comprising:

11. The system of claim 8, wherein at least one of the first time stamps is determined based on the following equation:

Ts (corr) = Ts (send) + Cor,

12. The system of claim 8 further comprising:

13. The system of claim 8, wherein the first plurality of packets comprise user data gram packets.

14. The system of claim 8, wherein each of the first plurality of packets include a header and a data portion having the first time stamp.

15. A non-transitory computer-readable medium comprising code, which when executed by a processor, causes operations comprising:

retrieving a first correction factor determined for the first sender; correcting, at the receiver, each of the first time stamps based on the first correction factor, the corrected first time stamps providing a time local to the receiver; receiving, from a second sender, a second plurality of packets, each of the second plurality of packets including a second time stamp representative of a second time local to the second sender of the second plurality of packets;

16. The non-transitory computer-readable medium of claim 15 further comprising:

17. The non-transitory computer-readable medium of claim 16 further comprising:

18. The non-transitory computer-readable medium of claim 15, wherein at least one of the first time stamps is determined based on the following equation:

Ts (corr) = Ts (send) + Cor, wherein Ts(corr) represents the at least one of the first time stamps corrected at the receiver, Ts(send) represents a value of a time stamp applied by the first sender, and Cor represents the first correction factor.

19. The non-transitory computer-readable medium of claim 15 further comprising:

20. The non-transitory computer-readable medium of claim 15, wherein the first plurality of packets comprise user data gram packets, and wherein each of the first plurality of packets include a header and a data portion having the first time stamp.