Does anyone play with GPS based ntp server and have clients?

[tkamiya]

I have a GPS with PPS feeding Ubuntu Linux machine, and it is working just fine.  Time stream is picked up and PPS is picked up. 

Problem is the client side.  main.takadom.org is my internal domain.  As you can tell, .PPS. is picked up but not the time itself. 

ntp.conf looks like this

"server 192.168.1.102 prefer iburst"
"fudge 192.168.1.102 prefer"

Here's the screen shot....  main is the stratum 1 with GPS.  backup is client.
none of ip addresses listed there are not mine.

[Foxxz]

Remove your fudge line. You can only (confusingly) use fudge for time sources on the local system itself (strictly 127.x.x.x)

Otherwise it looks like it is using your ntp server as its primary time source so I'm not sure exactly what you mean by its not getting the time from it.

[tkamiya]

On my primary server, I have two lines indicating one is from GPS ascii string data and PPS.  On client, I only have PPS line from main.  I'd thought I should not get PPS on stratum 2 server since it is not even getting PPS pulses.  (According to ntpq -p)

I will remove fudge.

[ve7xen]

I'm not really sure what you're asking here, but it sounds like you are confused about the meaning of REFID. This is sent by the server to indicate which reference it is currently using. So if you are running ntpq on a stratum 2, you will see the stratum 1 server in your peers list, with its current clock source REFID (e.g. .PPS.) listed.

[tkamiya]

Yes, I know that much.  But it doesn't make sense to me.

Both servers are MINE.
MAIN server has stratum 0 source - gps and pps which are named .GPS. and .PPS.  Which makes MAIN server a stratum 1 server.  That much is easy.

Here's what I'm confused about.
However, BACKUP server, which is stratum 2 talks to stratum 1 above.  Why is it only talking to refid .PPS.?  At stratum 2 level, what we need is time.  If it gets from PPS, presumably containing second pulse, and other server containing time string, AND if time string happens to be all over the place, wouldn't I get exactly 1 second part but wrong time?

[ve7xen]

Here's what I'm confused about.
However, BACKUP server, which is stratum 2 talks to stratum 1 above.  Why is it only talking to refid .PPS.?

It's not. The peer is, and is reporting it to the server you are querying. The REFID comes from the peer. Ergo when you query the stratum 2 server it is telling that remote `main.takadom.org` is reporting its current refid is `.PPS.`. I'm still pretty sure you're confused about what refid actually means here. ntpq is giving you the list of peers, and each peer reports its current reference clock id, which is what is listed. So when you query the stratum 2 server, included in the list is the stratum 1 server (main.takadom.org) and the id of its current - stratum 0 - reference clock (.PPS.). If you were to query a stratum 3 server, you would see stratum 2 servers in the list, with their stratum 1 refids (which would be the IPs of the stratum 1 servers).

[tkamiya]

So in a nutshell, .PPS. server (refid) no longer means PPS symaphore as named on stratum 1 server.  It is now supplying time stream.  I was expecting .GPS. to show up.  It's on the same server and it is handling time stream.

Did I get it right?

I recently had a system crash and I had to rebuild it.  It has been so long since I built it first time.  Knowledge I had on ntp (and nfs) is totally gone.  I'm re-learning it.  Thanks for helping me out.

[ve7xen]

So in a nutshell, .PPS. server (refid) no longer means PPS symaphore as named on stratum 1 server.  It is now supplying time stream.  I was expecting .GPS. to show up.  It's on the same server and it is handling time stream.

Did I get it right?

I recently had a system crash and I had to rebuild it.  It has been so long since I built it first time.  Knowledge I had on ntp (and nfs) is totally gone.  I'm re-learning it.  Thanks for helping me out.

ntpd has marked the GPS refclock as falseticker (x), so it will never be selected as reference. I guess you can consider this as a bit of an oddity of ntpd - you can't combine the time of day and PPS sources (since PPS can't provide time of day) into a single time source, even if they are physically 'the same device'. So on boot, ntpd will learn the second from the GPS source, then lock on to PPS for synchronization, but once it syncs up, the NMEA source will usually be too unstable / have too much offset to be considered as a time source, especially if the network is up and you have some good peers online. It's needed to bootstrap and learn the correct second on boot if you don't have network, but after that doesn't do anything. It can actually be kind of dangerous to have it enabled, since some GPSes will happily serve NMEA sentences giving the GPS epoch as the current time before they establish a time solution, and if that is NTP's only time of day source, it will happily believe it and start serving time in the 1980s, and then get angry when the GPS time steps later. So unless you need your time server to work when isolated from the Internet, I would rely on other servers to bootstrap time of day - make sure your system time is stepped on boot too (ntpdate or similar) before ntpd starts, for the same reason.

You can fudge the PPS source to have the refid GPS if you prefer, as this will be visible to clients, the field is informational only, it's not used by the NTP algorithm, AIUI.

[tkamiya]

Thanks!

Just for testing, I removed all the "pool" references in stratum 1 box, so it can only pick gps time stream and pps.  Then restarted the stratum 2.  Now it sees GPS on former box.  PPS doesn't appear as I expect. 

Now the mystery is, why gps was checked out before.  This is a gps local to the box.  Weird.  I know there is a lot of history behind how gpsd works but algorithm is a bit weird.

[jav]

As GPS and PPS clock sources in "main" show as SHM, I'm assuming you're using gpsd, right? If so, please be aware that the gpsd version in Ubuntu (and upstream Debian) 3.22-4 has some bugs in the PPS module and, depending on the specific GPS unit you're using, also in its driver. If you're using a generic NMEA unit, it'll probably be fine, but for others, you may have stability problems as the "x" (falseticker) is indicating.

ntpd can use an integrated an pps+time source, and there's a few options to connect to gpsd that way (drivers 28 and 46, for instance) but each one has their own issues.

So, for the "main" unit, I suggest to keep GPS and PPS sources independent. If you're using kernel PPS (and if not, you should try it), a good config would be:

server 127.127.28.0 minpoll 4 maxpoll 4 prefer
fudge 127.127.28.0 time1 0.000 flag1 1 refid GPS

server 127.127.22.0 minpoll 4 maxpoll 4
fudge 127.127.22.0 flag3 1 refid PPS

The time1 value (in seconds) should be adjusted to offset for GPS message delay once you have it operational. It shouldn't fluctuate more than a few milliseconds in the "ntpq -np" output. In my case it's at 0.165 (Raspberry Pi). I also needed to add the "flag1 1" too (it doesn't have a RTC)

ntpq output:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*127.127.28.0    .GPS.            0 l    1   16  377    0.000   -1.955   1.605
o127.127.22.0    .PPS.            0 l   12   16  377    0.000   +0.000   0.002

For the "backup", as explained by Foxxz, you don't need the "fudge" lines.

You should keep the "pool" references as backup, anyway.

[tkamiya]

Double Thank You!

I just did ntpq -crv

associd=0 status=0115 leap_none, sync_pps, 1 event, clock_sync,
version="ntpd 4.2.8p4@1.3265-o Tue Jan  7 15:08:23 UTC 2020 (1)",
processor="x86_64", system="Linux/4.4.0-146-generic", leap=00, stratum=1,
precision=-23, rootdelay=0.000, rootdisp=3.115, refid=PPS,
reftime=e560d3ad.093248ff  Sun, Dec 12 2021 15:01:49.035,
clock=e560d43a.a2258084  Sun, Dec 12 2021 15:04:10.633, peer=28071, tc=4,
mintc=3, offset=0.028966, frequency=19.454, sys_jitter=0.970294,
clk_jitter=1.024, clk_wander=0.112

So the ntpd version is 4.2.8p4

I hope this version is a "*" bug free one.
My GPS is actually this unit:
https://www.tindie.com/products/nsayer/gps-clock/
The actual GPS module is SkyTraq Venus838LPx-T based, which outputs regular NEMA sentences.

I changed the lines referencing GPS to the one you suggested.  I'll let it settle down and see how they are doing.  I have few other GPS, so if this one doesn't do it, it is an easy matter to switch it out.

[jav]

So the ntpd version is 4.2.8p4

That version has some security issues. You'll be fine as long as you're only giving access to trusted clients. Don't open it up to the Internet.

Also, please note that you cannot use driver 46 (gpsd json) with that version. Not really needed, just don't spend time trying.

gpsd do support SkyTraq binary format, so if your run into any issue, make sure which protocol it's really using before trying to debug it. "cgps" can help you there.

[tkamiya]

I thought it was putting out NEMA.  "cgps" shows ascii string is some sort of tagged format.  Is this JASON? 

lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqklqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x    Time:       2021-12-12T20:44:36.000Z   xxPRN:   Elev:  Azim:  SNR:  Used: x
x    Latitude:    28.600000 N               xx  16    81    342    45      Y   x
x    Longitude:   81.400000 W               xx   4    53    322    46      Y   x
x    Altitude:   144.4 ft                   xx 138    46    225    44      N   x
x    Speed:      0.0 mph                    xx  26    44    038    46      Y   x
x    Heading:    244.4 deg (true)           xx  27    38    152    47      Y   x
x    Climb:      0.0 ft/min                 xx   3    34    240    43      Y   x
x    Status:     3D FIX (74 secs)           xx 133    29    246    46      N   x
x    Longitude Err:   +/- 8 ft              xx  22    24    217    40      N   x
x    Latitude Err:    +/- 9 ft              xx  31    22    075    40      Y   x
x    Altitude Err:    +/- 30 ft             xx   8    19    180    30      Y   x
x    Course Err:      n/a                   xx   9    17    319    37      Y   x
x    Speed Err:       +/- 12 mph            xx  32    01    138    00      N   x
x    Time offset:     1.005                 xx                                 x
x    Grid Square:     EL98gq                xx                                 x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqjmqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
{"class":"PPS","device":"/dev/ttyS0","real_sec":1639341870, "real_nsec":0,"clock_sec":1639341870,"clock_nsec":4865140,"precision":-20}
{"class":"TPV","device":"/dev/ttyS0","mode":3,"time":"2021-12-12T20:44:30.000Z","ept":0.005,"lat":28.600000000,"lon":-81.400000000,"alt":44.000,"epx"
:2.646,"epy":2.771,"epv":9.200,"track":244.4000,"speed":0.000,"climb":0.000,"eps":5.54,"epc":18.40}
{"class":"PPS","device":"/dev/ttyS0","real_sec":1639341871, "real_nsec":0,"clock_sec":1639341871,"clock_nsec":4866205,"precision":-20}
{"class":"SKY","device":"/dev/ttyS0","xdop":0.71,"ydop":0.74,"vdop":1.60,"tdop":1.30,"hdop":0.90,"gdop":2.51,"pdop":1.80,"satellites":[{"PRN":16,"el"
:81,"az":342,"ss":45,"used":true},{"PRN":4,"el":53,"az":322,"ss":46,"used":true},{"PRN":138,"el":46,"az":225,"ss":44,"used":false},{"PRN":26,"el":44,
"az":38,"ss":45,"used":true},{"PRN":27,"el":38,"az":152,"ss":47,"used":true},{"PRN":3,"el":34,"az":240,"ss":43,"used":true},{"PRN":133,"el":29,"az":2
46,"ss":45,"used":false},{"PRN":22,"el":24,"az":217,"ss":40,"used":false},{"PRN":31,"el":22,"az":75,"ss":41,"used":true},{"PRN":8,"el":19,"az":180,"s
s":28,"used":true},{"PRN":9,"el":17,"az":319,"ss":37,"used":true},{"PRN":32,"el":1,"az":138,"ss":0,"used":false}]}
{"class":"TPV","device":"/dev/ttyS0","mode":3,"time":"2021-12-12T20:44:31.000Z","ept":0.005,"lat":28.600000000,"lon":-81.400000000,"alt":44.000,"epx"
:2.646,"epy":2.771,"epv":9.200,"track":244.4000,"speed":0.000,"climb":0.000,"eps":5.54,"epc":18.40}
{"class":"PPS","device":"/dev/ttyS0","real_sec":1639341872, "real_nsec":0,"clock_sec":1639341872,"clock_nsec":4867045,"precision":-20}
{"class":"TPV","device":"/dev/ttyS0","mode":3,"time":"2021-12-12T20:44:32.000Z","ept":0.005,"lat":28.600000000,"lon":-81.400000000,"alt":44.000,"epx"
:2.646,"epy":2.771,"epv":9.200,"track":244.4000,"speed":0.000,"climb":0.000,"eps":5.54,"epc":18.40}

Right now, I'm showing this too:
# ntpq -np
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 0.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 1.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 2.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 3.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
x127.127.28.0    .GPS.            0 l    1   16  377    0.000  -215.86   6.976
x127.127.22.0    .PPS.            0 l   15   16  377    0.000   -4.836   0.043

----- rest omitted ----

I had this working for few months before I lost a hard disk.  So I'm trying to re-install this.  Something is awfully off.  How do you know NTP so much?  Are you part of development team?

By the way, this is inside my firewall.  I have no intent to open up to the whole world.

[jav]

No, I'm not part of the development team. I just put quite some work fine-tuning it and had to write some patches :-)

If you updated your system when your disk crashed, you may be dealing with new features (and bugs) that you didn't have before. I've upgraded from Debian buster to bullseye and took me some time to fine-tune it back again.

So, being so much off isn't the issue right now as it needs to settle down. You can use "gpsmon" for a more "raw" output. You'll get a line with device details. In my case:

{"class":"DEVICES","devices":[{"class":"DEVICE","path":"/dev/ttyAMA0","driver":"u-blox","subtype":"SW ROM CORE 3.01 (107888),HW 00080000","subtype1":"FWVER=SPG 3.01,PROTVER=18.00,GPS;GLO;GAL;BDS,SBAS;IMES;QZSS","activated":"2021-12-12T:30:14.575Z","flags":1,"native":1,"bps":9600,"parity":"N","stopbits":1,"cycle":1.00,"mincycle":0.02}]}

The "driver": "u-blox" is what you're looking for.

Two things to do now:

1.- Make sure that the pps signal is working. If you're using kernel pps, use "ppstest /dev/pps0"
2.- Keep looking at the jitter of the GPS line. It should keep at that level (or below).

If both of them are fine, ntp should remove the falseticker ("x") from both lines and begin steering the clock to the PPS. Once it's there, you can fix the GPS offset using the time1 parameter: Take a bunch of samples for the GPS offset, average them and put the resulting value in time1 (change sign and divide by 1000).

If the falseticker stays there, there's some issue with the data delivered from gpsd: Missing messages, fluctuating times or wrong data. In that case, "ntpshmmon" will help you understand the data delivered to ntp. It should be nicely periodic (don't worry about the -215 ms offset as long as it doesn't fluctuate much). If there's something wrong there, you'll need to analyze the messages from gpsd using "gpsmon".

[Halcyon]

Unless you specifically wanted to share your exact home address with the rest of the world, I've modified your last post (in bold) to remove your exact coordinates. Feel free to change it back if this was your intention.

[tkamiya]

Thank you.  I didn't even think of that.

[tkamiya]

With my previous setup, both GPS and PPS would go to "X" state.  Often times immediately, sometimes an hour or so later.

So, I made several changes. 
1) replaced GPS
2) shorten the pps and data serial cable
3) play with offset a little

12 hours later, I got this:
     remote           refid      st t when poll reach   delay   offset  jitter
===================================================

*SHM(0)          .GPS.            0 l    1   16  377    0.000   -9.225   5.857
oPPS(0)          .PPS.            0 l   15   16  377    0.000    0.000   0.001
<more sources here>

This is how it should be, correcct?

[ve7xen]

With my previous setup, both GPS and PPS would go to "X" state.  Often times immediately, sometimes an hour or so later.

So, I made several changes. 
1) replaced GPS
2) shorten the pps and data serial cable
3) play with offset a little

12 hours later, I got this:
     remote           refid      st t when poll reach   delay   offset  jitter
===================================================

*SHM(0)          .GPS.            0 l    1   16  377    0.000   -9.225   5.857
oPPS(0)          .PPS.            0 l   15   16  377    0.000    0.000   0.001
<more sources here>

This is how it should be, correcct?

Looks fine, assuming the offset to your other sources is reasonable and ntpd hasn't marked them all invalid because they don't agree with your prefer source.

[tkamiya]

Offset used for GPS (local)

server 127.127.28.0 minpoll 4 maxpoll 4 prefer
fudge 127.127.28.0 time1 0.032 flag1 1 refid GPS

fudge will require further tuning.

none of other sources are "X"'d out. 

[jav]

Looks great.

Don't bother too much with the GPS offset (and the fudge factor). As long as it's around those levels, it's fine. ntpd will use PPS as a reference, that's what is for.

Key here is that the values keep in that range so ntpd don't falsetick ("x") them out.

[tkamiya]

I just checked.  Still doing well.

Thank you very much for sharing your extensive knowledge.