How to Convert your TTM Box to a Stratum-1 NTP Server

The main reasons for investigating this are listed below:

• Lots of hosts found one of the most useful features of TTM was having a local stratum-1 NTP server
• The hosts have already gone to the effort of having the antenna installed on the roof
• The RIPE NCC card driver has already been added to the upstream NTP distribution
• It will allow these hosts to take over control of the systems and run them as an NTP server

Hardware Replacement

If you want to replace the hardware, the only real requirement is that the device you choose has an available PCI slot (not PCIe), an available hardware serial port (do not use a USB serial adaptor), and has hardware compatible with either FreeBSD or your chosen Linux distribution.

The TTM card does not interface with the system via the PCI bus - it is only there to obtain power from the bus (via the 12V and 5V power rails on the PCI bus). Therefore the requirement is for a PCI slot with active power rails.

Although most servers today only have PCI-Express ports on them, you may be able to find a model with a PCI slot with your chosen server provider. Alternatively, you could construct an external enclosure for the card which provides a power supply but no data lines.

Software Instructions

FreeBSD

The testing was done using FreeBSD 9.2-RELEASE (i386) as the latest stable version at time of writing

It should be installed with (or added later using sysinstall) the 'src' package, as a kernel recompile should be performed.

Compile a new kernel to enable the kernel PPS_SYNC option:

 [root@ttm-test ~]# cd /usr/src/sys/i386/conf
 

 [root@ttm-test /usr/src/sys/i386/conf]# cat TTM-PPSSYNC
 

  include GENERIC
 

 ident TTM-PPSSYNC
 

 options          PPS_SYNC

Depending on what modules you need for your hardware, you may choose not to bother recompiling all the modules. You should ensure that you compile the 'pps' module, plus any others you need:

 [root@ttm-test ~]# kldstat
 

 Id Refs Address    Size     Name
 

   1    7 0xc0400000 7e2ce4   kernel
 

   2    1 0xc0be3000 b928     ipmi.ko
 

   3    2 0xc0bf1000 5c484    acpi.ko
 

   4    1 0xc0c4e000 4f98     ida.ko
 

  
 

 [root@ttm-test ~]# cat /boot/loader.conf
 

  ipmi_load="YES"
 

 ida_load="YES"
 

 [root@ttm-test /usr/src]# cat /etc/make.conf
 

 MODULES_OVERRIDE = pps ipmi acpi ida

Now compile and install your new kernel:

 [root@ttm-test /usr/src]# make buildkernel KERNCONF=TTM-PPSSYNC
 

 <snip build process>
 

 [root@ttm-test /usr/src]# make installkernel KERNCONF=TTM-PPSSYNC
 

 <snip install process>

Update your ports tree:

 [root@ttm-test ~]# portsnap fetch
 

 [root@ttm-test ~]# portsnap extract
 

  

Now configure the build for NTP:

 [root@ttm-test ~]# cd /usr/ports/net/ntp
 

 [root@ttm-test /usr/ports/net/ntp]# make config

Enable the 'RIPENCC specific Trimble driver' and the 'ATOM PPS interface':

 │ │ [x] ATOM         Enable ATOM PPS interface                               │ │ 
 

 │ │ [x] RIPENCC      Enable RIPENCC specific Trimble driver                  │ │

Compile NTP:

 [root@ttm-test /usr/ports/net/ntp]# make

Install NTP:

 root@ttm-test /usr/ports/net/ntp]# make install

Create a log directory:

 [root@ttm-test ~]# mkdir /var/log/ntpstats

Put the following config in place:

 cat /etc/ntp.conf
 

 [root@ttm-test ~]# cat /etc/ntp.conf
 

 #
 

 # $FreeBSD: release/9.2.0/etc/ntp.conf 239600 2012-08-23 00:39:08Z delphij $
 

 #
 

 # Default NTP servers for the FreeBSD operating system.
 

 #
 

 # Don't forget to enable ntpd in /etc/rc.conf with:
 

 # ntpd_enable="YES"
 

 #
 

 # The driftfile is by default /var/db/ntpd.drift, check
 

 # /etc/defaults/rc.conf on how to change the location.
 

 

 # Enable this if you want statistics to be logged.
 

 statsdir /var/log/ntpstats/
 

 statistics loopstats peerstats clockstats
 

 filegen loopstats file loopstats type day enable
 

 filegen peerstats file peerstats type day enable
 

 filegen clockstats file clockstats type day enable
 

 logconfig =all
 

 logfile /var/log/ntpstats/logfile
 

 

 # RIPE NCC Trimble card driver to enable antenna to send PPS
 

 # signal, and provide the current timecode, and collect PPS via
 

 # the PPSAPI interface
 

 # 43.(X) -> X will be used to define:
 

 # the serial device to access to talk to the GPS receiver (/dev/gpsX)
 

 server 127.127.43.0 prefer      # use RIPENCC driver
 

 fudge 127.127.43.0 refid GPS                        # set the refid name
 

 fudge 127.127.43.0 flag3 1                          # enable kernel PPS sync
 

 # By default, exchange time with everybody, but don't allow configuration.
 

 restrict -4 default kod notrap nomodify nopeer noquery
 

 restrict -6 default kod notrap nomodify nopeer noquery
 

 # Local users may interrogate the ntp server more closely.
 

 restrict 127.0.0.1
 

 restrict ::1

Configure the system to use the port version of NTP, not the included system version:

 [root@ttm-test ~]# cat /etc/rc.conf
 

  <snip>
 

 ntpd_enable="YES"                    # Run ntpd Network Time Protocol (or NO).
 

 ntpd_program="/usr/local/bin/ntpd"   # path to ntpd, if you want a different one.
 

 ntpd_sync_on_start="YES"             # Sync time on ntpd startup, even if offset is high

Instruct devfs to put an appropriate symlink in place for your serial device on boot. Also, manually create this symlink just now:

 [root@ttm-test ~]# tail -1 /etc/devfs.conf
 

  link    cuau0    gps0
 

 [root@ttm-test ~]# cd /dev/
 

 [root@ttm-test /dev]# ln -s cuau0 gps0

Now you can start NTPd:

 [root@ttm-test ~]# /etc/rc.d/ntpd start
 

 Starting ntpd.

You should see antenna initialisation output, followed by periodic timecode updates in the clockstats log:

 [root@ttm-test /var/log/ntpstats]# cat clockstats
 

 56758 35170.036 127.127.43.0 FW Versions:  Nav Proc  3.06   3/11/105  Sig Proc 10.04   3/11/105
 

 56758 35170.037 127.127.43.0 Rcvr status1: Doing position fixes (00h); status2: No BBRAM, Ant OK (01h)
 

 56758 35170.038 127.127.43.0 Rcvr Machine ID: 97; Status3 = RTC OK, Alm OK (00h)
 

 56758 35170.038 127.127.43.0 Product ID 8F42
 

     extension:            0
 

     case serial # prefix: 0
 

     case serial #:        82643833
 

     prod. #:              39091
 

     premium options:      0h
 

     machine ID:           97
 

     key:                  B79h
 

 56758 35170.141 127.127.43.0   search range:          0
 

    board options:         0
 

    board serial #:        82643833
 

    build date/hour:       28/06/05 07:00
 

    osc offset:            0.000 PPM (0 Hz)
 

    test code:             0
 

 56758 35170.141 127.127.43.0 I/O Options: 1C  0  5  A
 

      MSL altitude output (Geoid height)
 

      MSL altitude input
 

      Double precision
 

      Time tags in UTC
 

      Fixes sent only on request
 

      PPS output during fixes
 

      Code-phase smoothed before output
 

      Signal Strength Output as dBHz
 

 56758 35170.142 127.127.43.0 Channel A Configuration
 

     Transmit speed: No output at  9600
 

     Receive speed: TSIP at  9600
 

     Character format (bits/char, parity, stop bits): 8-ODD-1
 

 56758 35170.142 127.127.43.0 Auto-Report Mask: 00 00 00 00
 

 Auto-Reports scheduled for Output:
 

 Auto-Reports NOT scheduled for Output:
 

        Almanac  , Ephemeris, UTC      , Iono     , GPS Msg  , Alm Hlth
 

        Time Fix , SV Select, Ext Event, Pos Fix  , Raw Meas
 

  56758 35170.239 127.127.43.0    operating mode:      OverDetermined Time
 

     dynamics:            Static
 

     elev angle mask:     10 deg
 

     SNR mask:            4 AMU
 

     DOP mask:            8
 

     DOP switch:          6
 

 56758 35170.239 127.127.43.0 PPS is         enabled
 

     timebase:   UTC
 

     polarity:   Positive
 

     offset:     0.0 ns,
 

     biasunc:    1000.0 ns
 

 56758 35170.239 127.127.43.0 Max # of position fixes for self-survey : 2000
 

 56758 35170.848 127.127.43.0 U1 11.4.2014 11:46:11 13 01
 

 56758 35170.893 127.127.43.0 L1 0 16 16 -1.86265e-09 -3.55271e-15 61440 1788 1694 7
 

 56758 35170.988 127.127.43.0 C1 11042014 114611 6  189255 110.2 556 6.7 16 5222.374603 N 0453.268272 E 57  -7 26 -21 8 5 -15 28 9
 

 56758 35171.033 127.127.43.0 S1 07 1 2 02 34.8  61.8 24.8
 

 56758 35171.060 127.127.43.0 S1 26 2 1 02 56.5 293.3 71.1
 

 56758 35171.097 127.127.43.0 S1 21 3 1 02 41.1 314.4 13.5
 

 56758 35171.133 127.127.43.0 S1 08 4 1 02 59.7  61.2 55.4
 

 56758 35171.161 127.127.43.0 S1 05 5 1 02 58.6 207.9 52.8
 

 56758 35171.198 127.127.43.0 S1 15 6 2 02 32.7 289.4 32.5
 

 56758 35171.225 127.127.43.0 S1 28 7 1 02 53.2 125.2 48.0
 

 56758 35171.258 127.127.43.0 S1 09 8 1 02 66.1  65.2 64.3

While the antenna locks onto the available satellites, it will output timecodes but it may take up to 1 hour before it has trained accurately enough to produce PPS pulses.

During this period, the NTP driver will fail to get a PPS signal, so you will see these messages in /var/log/ntpstats/logfile initially - these messages will stop once the PPS signal is received: 

 11 Apr 11:46:09 ntpd[69466]: 0.0.0.0 c016 06 restart
 

 11 Apr 11:46:09 ntpd[69466]: 0.0.0.0 c012 02 freq_set kernel 0.000 PPM
 

 11 Apr 11:46:10 ntpd[69466]: ripencc_receive(): ripencc_get_pps_ts returns failure
 

 11 Apr 11:46:10 ntpd[69466]: ripencc_receive(): !refclock_process
 

 11 Apr 11:46:26 ntpd[69466]: 0.0.0.0 c41c 0c clock_step +7200.865151 s
 

 11 Apr 13:46:27 ntpd[69466]: 0.0.0.0 c414 04 freq_mode
 

 11 Apr 13:46:27 ntpd[69466]: 0.0.0.0 c418 08 no_sys_peer
 

 11 Apr 13:46:27 ntpd[69466]: ripencc_receive(): ripencc_get_pps_ts returns failure

When it is ready to do so, the PPS pulses will arrive on the DCD pin of the serial port. When this happens, you should see the PPS LED on the PCI card flashing once per second.

You should now see your NTP peer begin to converge on a stable time - the offset and jitter should begin to reduce over time. This may take a few hours to reach full precision:

 [root@ttm-test /usr/local/etc]# ntpq -p
 

       remote           refid      st t when poll reach   delay   offset  jitter
 

 ==============================================================================
 

 *GPS_RIPENCC(0)  .GPS.           0  l    7   16  377    0.000    0.000   0.001

Linux distributions

CentOS

We did not develop instructions for CentOS 6 after we realised that some of the earlier TTM systems had only 128M RAM, which is not sufficient to load the Anaconda installer. The listed recommended minimum for a CLI CentOS6 install is 392M.

Debian

Debian was chosen to test the older hardware as it has much lower system requirements for a base installation - particularly memory. The instructions below were tested with Debian 7.4.0 (i386). Note that the earlier TTM hardware will not support the amd64/x86_64 distributions - you must use the i386/i686 variants.

Package needs rebuild to add RIPENCC driver:

 root@ttmbuild:~# apt-get install build-essential devscripts
 

 root@ttmbuild:~# apt-get build-dep ntp
 

 root@ttmbuild:~# apt-get install pps-tools
 

 root@ttmbuild:~# mkdir ripencc_build
 

 root@ttmbuild:~# cd ripencc_build/
 

 root@ttmbuild:~/ripencc_build# apt-get source ntp
 

 root@ttmbuild:~/ripencc_build# cd ntp-4.2.6.p5+dfsg/
 

 root@ttmbuild:~/ripencc_build/ntp-4.2.6.p5+dfsg# vi debian/rules

Edit the configure line, to add the flag to build the refclock_ripencc and the Atom/PPS driver:

                 --enable-all-clocks --enable-parse-clocks --enable-SHM --enable-RIPENCC --enable-ATOM\

You must now apply a patch to the refclock_ripencc.c driver file, which adds support for the LinuxPPS API. The driver can be downloaded from the LinuxPPS website:

Click here to get the driver patch

Once you have patched the driver, you can now build the package:

 root@ttmbuild:~/ripencc_build/ntp-4.2.6.p5+dfsg# debuild -i -us -uc -b

 
  You should now have packages:
 

 root@ttmbuild:~/ripencc_build# ls -al *.deb

 -rw-r--r-- 1 root root  588592 Apr 10 10:10 ntp_4.2.6.p5+dfsg-2_i386.deb
 

 -rw-r--r-- 1 root root   80606 Apr 10 10:10 ntpdate_4.2.6.p5+dfsg-2_i386.deb
 

 -rw-r--r-- 1 root root 1142300 Apr 10 10:10 ntp-doc_4.2.6.p5+dfsg-2_all.deb

Copy these packages to your TTM server. Do not install them yet.

On your NTP server, you need to install the LinuxPPS tools:

 root@ttm-test:~# apt-get install pps-tools

Next, install udev rules to create the appropriate symlinks, and to attach the PPS line discipline to the serial port (the one you connected the GPS loopback serial cable to):

 root@ttm-test:~# cat /etc/udev/rules.d/09.pps.rules
 

 KERNEL=="ttyS0", SYMLINK+="gps0", RUN+="/bin/setserial -v /dev/%k low_latency irq 4", RUN+="/usr/sbin/ldattach pps /dev/%k"
 

 KERNEL=="pps0", OWNER="root", GROUP="tty", MODE="0660", SYMLINK+="gpspps0"

Activate the rules by calling a udev trigger, and check your kernel messages to ensure they are loaded correctly:

 root@ttm-test:~# udevadm trigger
 

 root@ttm-test:~# dmesg
 

 [  365.664354] pps_core: LinuxPPS API ver. 1 registered
 

 [  365.664380] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <
 giometti@linux.it
 >
 

 [  365.665841] pps_ldisc: PPS line discipline registered
 

 [  365.706785] pps pps0: new PPS source serial0
 

 [  365.706846] pps pps0: source "/dev/ttyS0" added
 

 root@ttm-test:~# ls -al /dev/*[gp]ps*
 

 lrwxrwxrwx 1 root root      5 Apr 10 15:59 /dev/gps0 -> ttyS0
 

 lrwxrwxrwx 1 root root      4 Apr 10 15:59 /dev/gpspps0 -> pps0
 

 crw-rw---T 1 root tty  251, 0 Apr 10 15:59 /dev/pps0

Now put the following NTP configuration in place (in /etc/ntp.conf)

 root@ttm-test:~# cat /etc/ntp.conf
 

  
 

 driftfile /var/lib/ntp/ntp.drift
 

 # Enable this if you want statistics to be logged.
 

 statsdir /var/log/ntpstats/
 

 statistics loopstats peerstats clockstats
 

 filegen loopstats file loopstats type day enable
 

 filegen peerstats file peerstats type day enable
 

 filegen clockstats file clockstats type day enable
 

 logconfig =all
 

 logfile /var/log/ntpstats/logfile
 

 

 # RIPE NCC Trimble card driver to enable antenna to send PPS
 

 # signal, and provide the current timecode, and collect PPS via
 

 # the LinuxPPS interface
 

 # 43.(X) -> X will be used to define:
 

 # the serial device to access to talk to the GPS receiver (/dev/gpsX)
 

 # the PPS device which receives the timing pulses (/dev/gpsppsX)
 

 server 127.127.43.0 prefer    # RIPENCC driver
 

 fudge 127.127.43.0 refid GPS
 

 # By default, exchange time with everybody, but don't allow configuration.
 

 restrict -4 default kod notrap nomodify nopeer noquery
 

 restrict -6 default kod notrap nomodify nopeer noquery
 

 # Local users may interrogate the ntp server more closely.
 

 restrict 127.0.0.1
 

 restrict ::1
 

  

You can now install the custom-build package - which will automatically start the ntp server:

 root@ttm-test:~# dpkg -i ntp_4.2.6.p5+dfsg-2_i386.deb
 

 Selecting previously unselected package ntp.
 

 (Reading database ... 32562 files and directories currently installed.)
 

 Unpacking ntp (from ntp_4.2.6.p5+dfsg-2_i386.deb) ...
 

 Setting up ntp (1:4.2.6.p5+dfsg-2) ...
 

 [ ok ] Starting NTP server: ntpd.
 

 Processing triggers for man-db ...

You should also pin the package so a system update does not overwrite the custom-compiled version - of course this means you must be aware of any future security updates to this package and have a process to rebuild a new custom-compiled version each time:

 root@ttm-test:~# echo "ntp hold" | dpkg --set-selections
 

 root@ttm-test:~# dpkg --get-selections | grep ntp
 

 ntp                                                   hold

You should see antenna initialisation output, followed by periodic timecode updates in the clockstats log:

 56757 51930.862 127.127.43.0 FW Versions:  Nav Proc  3.06   3/11/105  Sig Proc 10.04   3/11/105
 

 56757 51930.867 127.127.43.0 Rcvr status1: Only 2 satellites usable (0Ah); status2: No BBRAM, Ant OK (01h)
 

 56757 51930.871 127.127.43.0 Rcvr Machine ID: 97; Status3 = RTC OK, No Alm (08h)
 

 56757 51930.899 127.127.43.0 Product ID 8F42
 

     extension:            0
 

     case serial # prefix: 0
 

     case serial #:        82643833
 

     prod. #:              39091
 

     premium options:      0h
 

     machine ID:           97
 

     key:                  B79h
 

 56757 51930.926 127.127.43.0   search range:          0
 

    board options:         0
 

    board serial #:        82643833
 

    build date/hour:       28/06/05 07:00
 

    osc offset:            0.000 PPM (0 Hz)
 

    test code:             0
 

 56757 51930.935 127.127.43.0 I/O Options: 1C  0  5  A
 

      MSL altitude output (Geoid height)
 

      MSL altitude input
 

      Double precision
 

      Time tags in UTC
 

      Fixes sent only on request
 

      PPS output during fixes
 

      Code-phase smoothed before output
 

      Signal Strength Output as dBHz
 

 56757 51930.945 127.127.43.0 Channel A Configuration
 

     Transmit speed: No output at  9600
 

     Receive speed: TSIP at  9600
 

     Character format (bits/char, parity, stop bits): 8-ODD-1
 

 56757 51930.954 127.127.43.0 Auto-Report Mask: 00 00 00 00
 

 Auto-Reports scheduled for Output:
 

 Auto-Reports NOT scheduled for Output:
 

        Almanac  , Ephemeris, UTC      , Iono     , GPS Msg  , Alm Hlth
 

        Time Fix , SV Select, Ext Event, Pos Fix  , Raw Meas
 

 56757 51931.009 127.127.43.0    operating mode:      3-D
 

     dynamics:            Land
 

     elev angle mask:     10 deg
 

     SNR mask:            4 AMU
 

     DOP mask:            8
 

     DOP switch:          6
 

 56757 51931.036 127.127.43.0 PPS is         enabled
 

     timebase:   UTC
 

     polarity:   Positive
 

     offset:     0.0 ns,
 

     biasunc:    1000.0 ns
 

 56757 51931.043 127.127.43.0 Max # of position fixes for self-survey : 2000
 

 56757 51931.633 127.127.43.0 U1 10.4.2014 14:25:31 9 01
 

 56757 51931.688 127.127.43.0 L1 0 16 16 0 1.77636e-15 589824 1787 1694 7
 

 56757 51947.635 127.127.43.0 U1 10.4.2014 14:25:47 9 01
 

 56757 51963.636 127.127.43.0 U1 10.4.2014 14:26:03 9 01
 

 56757 51979.634 127.127.43.0 U1 10.4.2014 14:26:19 9 01
 

 56757 51995.635 127.127.43.0 U1 10.4.2014 14:26:35 9 01
 

 56757 52011.637 127.127.43.0 U1 10.4.2014 14:26:51 9 01
 

 56757 52027.635 127.127.43.0 U1 10.4.2014 14:27:07 9 01
 

 56757 52043.633 127.127.43.0 U1 10.4.2014 14:27:23 9 01
 

 56757 52059.633 127.127.43.0 U1 10.4.2014 14:27:39 9 01

While the antenna locks onto the available satellites, it will output timecodes but it may take up to 1 hour before it has trained accurately enough to produce PPS pulses.

During this period, the NTP driver will fail to get a PPS signal, so you will see these messages in /var/log/ntpstats/logfile initially - these messages will stop once the PPS signal is received:

 10 Apr 16:25:31 ntpd[2243]: ripencc_receive(): ripencc_get_pps_ts returns failure
 

 10 Apr 16:25:31 ntpd[2243]: ripencc_receive(): !refclock_process
 

 10 Apr 16:25:47 ntpd[2243]: ripencc_receive(): ripencc_get_pps_ts returns failure
 

 10 Apr 16:25:47 ntpd[2243]: ripencc_receive(): !refclock_process

When it is ready to do so, the PPS pulses will arrive on the DCD pin of the serial port, be received by the line discipline LinuxPPS driver, and be presented on /dev/pps0. When this happens, you should see the PPS LED on the PCI card flashing once per second. You can also check that PPS is being correctly received by the kernel by using the pps-tools commands.

Before PPS is available:

 root@ttm-test:~# ppstest /dev/pps0
 

 trying PPS source "/dev/pps0"
 

 found PPS source "/dev/pps0"
 

 ok, found 1 source(s), now start fetching data...
 

 time_pps_fetch() error -1 (Connection timed out)
 

 time_pps_fetch() error -1 (Connection timed out)
 

 time_pps_fetch() error -1 (Connection timed out)
 

 time_pps_fetch() error -1 (Connection timed out)

In addition, your NTP server will not yet show a synchronised peer:

 root@ttm-test:~# ntpq -p
 

       remote           refid      st t when poll reach   delay   offset  jitter
 

 ==============================================================================
 

   GPS_RIPENCC(0)  .GPS.           0 l    -   16    0    0.000    0.000   0.000

After PPS becomes available:

 root@ttm-test:~# ppstest /dev/pps0
 

 trying PPS source "/dev/pps0"
 

 found PPS source "/dev/pps0"
 

 ok, found 1 source(s), now start fetching data...
 

 time_pps_fetch() error -1 (Connection timed out)
 

 time_pps_fetch() error -1 (Connection timed out)
 

 time_pps_fetch() error -1 (Connection timed out)
 

 time_pps_fetch() error -1 (Connection timed out)
 

 time_pps_fetch() error -1 (Connection timed out)
 

 time_pps_fetch() error -1 (Connection timed out)
 

 source 0 - assert 1397140681.267546064, sequence: 1 - clear  0.000000000, sequence: 0
 

 source 0 - assert 1397140682.267547022, sequence: 2 - clear  0.000000000, sequence: 0
 

 source 0 - assert 1397140683.267551381, sequence: 3 - clear  0.000000000, sequence: 0
 

 source 0 - assert 1397140684.267554021, sequence: 4 - clear  0.000000000, sequence: 0
 

Once the PPS inputs are being received, check your NTP server status - it may take some time to stabilise, but your stratum 0 peer should become active (as indicated by the '*' flag - this flags the main system peer):

 root@ttm-test:~# ntpq -p
 

       remote           refid      st t when poll reach   delay   offset  jitter
 

 ==============================================================================
 

 *GPS_RIPENCC(0)  .GPS.           0 l    2   16  377    0.000    0.000   0.002

Congratulations. You now have a Stratum 1 NTP server up and running for you to use as you wish.