Remote Synchronization of Time Taggers
In high-precision applications such as distributed time monitoring, telecommunications, quantum communication, and quantum key distribution, synchronizing time-tagging devices across remote locations is crucial. When Time Taggers operate independently, their clocks run free, and even small differences in their frequencies accumulate over time, creating significant timing discrepancies across multi-device setups.
To overcome this challenge, a synchronization-agnostic and versatile approach is employed.
This method supports any synchronization technology that provides a frequency reference signal (e.g., 10 MHz)
and a 1PPS (Pulse Per Second) timing signal, as shown in the figure below.
For example, the White Rabbit is a widely adopted technology
that enables picosecond-level synchronization precision with sub-nanosecond accuracy across remote setups, as detailed in this
application note.
The synchronization process relies on the two timing signals combined with software-based solutions,
including functionalities such as the ReferenceClock, to unify the time bases of the distributed Time Taggers.
Users must ensure that their hardware infrastructure supports the required synchronization signals to enable this functionality.
Moreover, when using the ReferenceClock with Time Tagger 20, an inherent timing error of approximately 200 ps needs to be considered.
More details can be found in the setReferenceClock() documentation
and the related in-depth guide (Software-Defined Reference Clock).
This tutorial demonstrates how to achieve remote synchronized operation of Time Taggers.
It provides a step-by-step guide to synchronizing multiple Time Taggers across distributed locations,
unifying their time bases, and merging time tag streams from remote network nodes for real-time processing
using the TimeTaggerNetwork functionality, which supports simultaneous connections to multiple servers from version 2.18 onward.
Establishing a common time base across distributed locations
A critical step in synchronizing Time Taggers across remote locations is establishing a unified time base. Within our approach, this is achieved by associating the 1PPS signal fed into the Time Tagger, which is generated at each location with an external clock locked to a Grand Master (GM), with the Coordinated Universal Time (UTC) provided by the host PC. To ensure accurate synchronization, the PC clocks at all locations must remain aligned to UTC with a time shift of less than 0.5 seconds. Synchronizing the PC clock is therefore essential because it allows the system to associate the 1PPS signal at each location with the correct UTC second, ensuring that all timing data is unified under a common temporal framework.
The PC clock alignment to UTC can be achieved using reliable synchronization protocols such as Network Time Protocol (NTP) or Precision Time Protocol (PTP). Standard operating system synchronization tools (e.g., Windows Time Service, Linux’s ntpd or chrony) configured to synchronize with either publicly available or internal time servers, are sufficient to maintain synchronization shift below 0.5 seconds. For enhanced robustness, ease of use, or advanced configuration options, dedicated synchronization software solutions such as the Meinberg NTP software package are recommended.
Starting a Time Tagger Server at each location
Once the PC clocks at all locations are aligned to UTC, the next step is to configure each Time Tagger to operate within a unified time base. By default, timestamps generated by a Time Tagger are relative to when the device was initialized in software. This means that if multiple Time Taggers start at different times, identical physical events occurring simultaneously at different locations receive different timestamps.
To synchronize the Time Taggers, enabling measurements across time tag streams generated at remote locations, the ReferenceClock must be activated. This ensures that all Time Taggers apply a UTC-based offset to their time base, allowing simultaneous events at remote locations to receive identical timestamps, regardless of when each Time Tagger was started. The ReferenceClock achieves this by locking the internal clock to an external distributed frequency reference (e.g., 10 MHz) and using the 1PPS signal to establish absolute time alignment.
To set up each Time Tagger, a connection to the device is first established in software. Hardware settings such as trigger levels and dead time should be configured according to the specific measurement requirements. Then, the ReferenceClock is enabled, aligning the internal time base with the external synchronization signals. Finally, a server is started on the host PC, making the Time Tagger accessible over the network for real-time data collection.
import TimeTagger
# Connect to the Time Tagger via USB
tagger = TimeTagger.createTimeTagger()
# Declare the channel names and corresponding physical connections
frequency_channel = 1
PPS_channel = 2
# Define the hardware settings here, such as trigger level or dead time.
# Enable the ReferenceClock
tagger.setReferenceClock(clock_channel=frequency_channel,
clock_frequency=10e6,
time_constant = 1e-4,
synchronization_channel=PPS_channel,
wait_until_locked=true)
# Start the Server.
# TimeTagger.AccessMode sets the access rights for clients.
# Port defines the network port to be used
tagger.startServer(access_mode=TimeTagger.AccessMode.Control,port=41101)
Warning
Activating the ReferenceClock shifts and rescales the internal time base of the Time Tagger. As a result, all ongoing measurements on every channel will be affected. Data recorded after activating the ReferenceClock will no longer be comparable with data acquired beforehand from the same Time Tagger instance.
Connecting to multiple Time Tagger Servers over the network
Once the Time Tagger servers are running at different locations, a client PC can connect to them to retrieve and process synchronized measurement data. The connection process relies on network communication, where the client searches for available Time Tagger servers and establishes a link to them.
To automatically discover servers on the local network, the function scanTimeTaggerServers() is used.
This function sends multicast UDP messages to detect active Time Tagger servers and retrieve their IP addresses.
However, its effectiveness depends on the network configuration.
Since multicast messages typically remain within the same local subnet,
their ability to reach servers across different subnets depends on whether the network routers forward multicast packets.
If the function does not detect any servers, it is likely that multicast routing is either not enabled or unreliable in the network.
In cases where scanTimeTaggerServers() fails to find a server,
the user needs to know the IP addresses and network ports of the servers to connect to,
when calling createTimeTaggerNetwork() on the client PC.
# Use the scanTimeTaggerServers() function to search for Time Tagger servers in the local network
servers = TimeTagger.scanTimeTaggerServers()
print("{} servers found.".format(len(servers)))
print(servers)
# Create a TimeTaggerNetwork instance and connect to the desired servers
server_addresses = ["192.168.1.100:41101", "192.168.1.101:41101"] # Replace with the IPs of the servers to connect to
ttn = TimeTagger.createTimeTaggerNetwork(server_addresses)
Warning
When a TimeTaggerNetwork object is created and connected to multiple servers,
the time tag streams from different locations are merged into a unified data stream for measurements.
This merging process relies on a sorting algorithm and requires that all incoming time tag streams have similar timestamps,
meaning the Time Taggers must be properly synchronized. If the time bases are not aligned, stream merging fails.
Accessing individual servers
When using a TimeTaggerNetwork object to connect to multiple Time Tagger servers,
it is possible to access and control each individual server separately.
This is particularly useful when extracting detailed device-specific information
that is not channel-dependent, such as checking overflow states using methods like getOverflows().
The getServers() method of the TimeTaggerNetwork class returns a list of TimeTaggerServer objects,
each representing one of the connected servers.
These server objects act as proxies, providing access to all relevant control functions available in TimeTaggerBase and TimeTaggerHardware,
as long as the servers were created with AccessMode::Control privileges.
Unlike the TimeTaggerNetwork object, a TimeTaggerServer object cannot be used to perform measurements directly.
Namely, it is not possible to pass the TimeTaggerServer object on to any measurement creator.
For example, to retrieve overflow information from a specific server:
# Retrieve the list of connected Time Tagger servers
servers = ttn.getServers()
# Access a specific server
server_A = servers[0]
# Retrieve overflow information from the server
overflows = server_A.getOverflows()
It is also possible to adjust hardware settings using either the TimeTaggerServer object or the globally mapped channels
via the TimeTaggerNetwork object. Both approaches yield the same result:
# Adjust dead time via the server object
server_A.setDeadtime(1, 1000)
# Equivalent operation using TimeTaggerNetwork global channel mapping
ttn.setDeadtime(1001, 1000)
With the current Software, using TimeTaggerServer objects for configuration is primarily a convenience,
as they serve as proxies for direct interaction with specific servers.
However, TimeTaggerServer provides four specific functions that are not available in TimeTaggerNetwork:
getAddress(), getAccessMode(),
getClientChannel(), getReferenceClockState().
Verification of the synchronization technology using a single Time Tagger
To evaluate the precision of the synchronization technology, a single Time Tagger
can be used before considering a multi-device setup. The proposed verification consists of enabling the ReferenceClock
on one external reference, typically the master if there is a hierarchy, and using a second external reference
as an input to the FrequencyStability measurement class.
This allows direct analysis of the synchronization precision using built-in measurement tools.
The measurement provides stability metrics such as Allan Deviation (ADEV), Modified Allan Deviation (MDEV), and Time Deviation (TDEV), which quantify timing fluctuations over different timescales. These results characterize the synchronization performance and provide a reference before working with multiple Time Taggers.
The first external reference is connected to an input channel and used to enable the ReferenceClock.
Since all analyses are performed on the same hardware, there is no need for a PPS signal for time synchronization.
The second and additional external references are fed into other input channels and analyzed using the FrequencyStability measurement.
The measurement runs over a range of averaging times to extract stability metrics.
import numpy as np
import time
# Define synchronization channels
ch_master = 1
ch_slave = 2
# Enable the ReferenceClock using the first external reference
tt.setReferenceClock(clock_channel=ch_master,
clock_frequency=10e6,
synchronization_channel=None,
wait_until_locked=True)
# Define measurement steps (logarithmically spaced averaging times)
steps = np.unique(np.logspace(0, 7, 100, dtype=np.int64))
# Initialize Frequency Stability measurement on the second reference
fs = TT.FrequencyStability(tt, ch_slave, steps, average=1)
# Allow the measurement to collect data
time.sleep(100) # Adjust as needed
# Retrieve frequency stability results
obj = fs.getDataObject()
tau = obj.getTau()
ADEV = obj.getADEV()
TDEV = obj.getTDEV()
MDEV = obj.getMDEV()
Measuring synchronization precision across multiple Time Taggers
Before starting experiments where Time Taggers are deployed in remote locations, it is useful to verify the synchronization precision while the units are still physically close to each other. This ensures that the synchronization setup is working correctly before the Time Taggers are separated.
The verification follows the method described in Swabian Instruments’ application note on remote synchronization.
A common test signal is split using a power splitter or any other method that guarantees that identical copies of the signal
are fed into both Time Taggers. The signal is then connected to an input channel of each device.
The Correlation measurement class is used to analyze the arrival times of events recorded by both Time Taggers,
providing a direct measure of synchronization precision.
The measurement is performed using the TimeTaggerNetwork in a dual-server setup, as discussed in the previous sections.
import matplotlib.pyplot as plt
# Define the correlation measurement between input channels on both Time Taggers
# We assume the test signal is fed into the third input of each Time Tagger
corr = TimeTagger.Correlation(ttn, 1003, 2003, binwidth=1, n_bins=1000)
# Start the measurement and run it for a given time
corr.startFor(10e12)
corr.waitUntilFinished()
# Retrieve the correlation data
index = corr.getIndex()
counts = corr.getData()
# Plot the correlation result
plt.plot(index, counts)
plt.xlabel("Time Difference (ps)")
plt.ylabel("Counts")
plt.title("Correlation of Synchronized Time Taggers")
plt.grid()
plt.show()