"Laureline" embedded GPS NTP server

[gxti]

A simple but effective project for your perusal. Laureline is an embedded SNTP server that receives time from a GPS receiver in the form of a pulse-per-second (PPS) input as well as the usual serial data in whatever format it comes (NMEA, Oncore, TSIP, etc.). I call it SNTP and not NTP because time is served from GPS alone and no other NTP servers are consulted. However, laureline is perfectly suitable as a Stratum 1 reference for NTP and SNTP clients alike.

The processor is a STM32F107 which includes a built-in 100Mbit Ethernet MAC, which I have paired with an inexpensive transceiver to get a good Ethernet solution with predictable and minimal latency unlike the USB-based MACs found in e.g. Raspberry Pi. Also onboard is a small VCXO which is disciplined by the microcontroller to precisely 10MHz using the GPS pulses, in effect making laureline a low-grade GPSDO. The board is powered by a buck converter that accepts 8-40VDC, allowing for direct power from a 12V or 24V lead acid battery. Other than that, the rest of the magic is in software: observing the frequency of the local oscillator by way of incoming GPS pulses, calculating NTP time from the 70MHz timer when queried  and handling network administrivia.

Because almost all STM32 IOs are 5V-tolerant, the serial interface works with both 3.3V and 5V GPS receivers. I've designed but not yet fabricated some tiny adapters to connect two types of receivers to the 6-pin header on laureline, one being Oncore GT+/UT+ and probably other Motorola receivers as well, and the other being Trimble's Resolution T and Resolution SMT carrier boards. Both types are presently available used on Ebay for about $30, half that in the case of UT+. As you can see in the pictures, jumper wires also work perfectly well in lieu of an adapter. There's enough current at either 3.3V or 5V to power a typical receiver with active antenna, as well.

Here is laureline paired with an Oncore GT+, the board at lower left is an isolated USB UART I'm using for debugging:


A (blurry) closeup of the board:


Current status: rev 1 of the hardware works flawlessly, except for the power supply which needed extra capacitance. rev 2 will have a separate PSU module of my own design, so that when I screw it up again at least I can test it separately and/or swap it out.

The software is still in early development. All of the disciplining/GPSDO stuff is not written yet, although I have written such things before for an earlier project that actually was a GPSDO. Presently I hacked together a simple mechanism that just resets the counter each time a PPS arrives and does no disciplining but it's still performing quite well. It also needs to handle leap seconds, both internally and also for presenting the leap flag to NTP clients. I haven't added the code to the repository but I'll do so in the next day or two.

Repository (design docs, code, etc.): http://hg.partiallystapled.com/circuits/laureline/
In zip format: http://hg.partiallystapled.com/circuits/laureline/archive/tip.zip
Also download the library and unzip it to lib/: http://hg.partiallystapled.com/circuits/alib/archive/tip.zip
Schematic PDF: http://hg.partiallystapled.com/circuits/laureline/raw-file/tip/out/production.PDF

[gxti]

Forgot to post this yesterday -- actual NTP results:

Code: [Select]

[root@rei ~]# ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
-64.73.32.134    192.41.214.45    4 u  501 1024  377   65.940    7.184   0.689
-24.124.0.251    128.206.12.130   3 u  457 1024  377  106.168  -18.808  46.581
+64.73.32.135    18.26.4.105      2 u  443 1024  377   67.824    5.759   0.714
-169.229.70.95   128.32.206.55    2 u  629 1024  377  103.428    7.904   1.133
*172.24.0.6      .GPS.            1 u   19   64  377    2.483    0.777   0.346
-172.24.0.66     172.24.0.1       3 u  491 1024  377    0.169    1.038   0.369
x172.24.0.68     172.24.0.66      4 u  530 1024  377    0.202  109.953 170.476
+172.24.0.1      172.24.0.6       2 u  607 1024  377    0.129    1.566   0.049


Code: [Select]

[root@hanmyo ~]# ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*172.24.0.6      .GPS.            1 u    5   64  377    2.531   -0.583   0.291
 172.24.0.64     172.24.0.6       2 u  730 1024  377    0.135   -1.500   0.077
 172.24.0.66     172.24.0.1       3 u  889 1024  377    0.387   -0.636   0.329
x172.24.0.68     108.133.178.111  2 u  932 1024  377    0.314  157.627 123.609
-64.73.32.134    192.41.214.45    4 u  993 1024  377   66.944    4.189   1.518
-24.124.0.251    64.6.144.6       3 u  846 1024  357  106.791  -21.016  37.217
+64.73.32.135    18.26.4.105      2 u  831 1024  377   68.850    4.766   0.517
+169.229.70.95   128.32.206.55    2 u   27 1024  377  103.022    5.961   0.445

172.24.0.6 is laureline, 172.24.0.68 is a windows box that just can't seem to keep up :[

Note the not-fantastic 2.4ms delay. I will see what I can do about that, probably it is due to context switching. I am using ChibiOS and its example LWIP bindings, which ends up having one thread handling packets and another doing the NTP server stuff and on top of that some third thread ends up processing the receive interrupt to begin with. Which means there's a lot of handing off to other threads, which incurs additional delay. So reducing the entire network stack to a single thread will probably improve latency considerably.

[gxti]

Turns out those Oncore receivers are not suitable for timing purposes. The auction listed them as "GT+" but they seem to be some older revision and don't say "GT" at all in the receiver ID. Also "GT+" isn't even the timing-oriented model, "UT+" is. That part is my fault. In any case, the dealbreaker is that the pulse-per-second output jumps forward by exactly 1ms every 16-20 seconds. With enough smoothing it could be made to work since I presume that on average the pulses are still accurate, but for now I've switched to my known-good Trimble Resolution SMT receiver. I bought 3 of these stupid Oncore receivers too, for $10 each... not the most expensive mistake I've made, and at least they will work well for position applications. I always wanted to make a data logger for my car.

I also fixed the Ethernet latency issue by using a single thread to process network data. I talked to someone else who is using the same stack and was able to get good latency figures using the regular, multi-threaded arrangement but I was unable to determine why I was not. Either way, single-threaded works and doesn't make the code any more complex so long as I'm not trying to do TCP.

New NTP figures:

Code: [Select]

[root@rei ~]# ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
-138.236.128.36  216.218.254.202  2 u   21 1024  377   68.819    3.228  0.641
+108.59.14.130   209.51.161.238   2 u  298 1024  377   34.920    1.167  0.288
+108.61.73.244   96.47.67.105     2 u  300 1024  377   39.524    1.135  1.604
-64.6.144.6      128.252.19.1     2 u  772 1024  377   84.329    6.609  0.252
*172.24.0.6      .GPS.            1 u    2   16  377    0.320   -0.073  0.016
-172.24.0.66     135.34.116.162   3 u  237 1024  377    0.197    0.130  0.581
x172.24.0.68     172.24.0.6       2 u  291 1024  377    0.237  113.341 77.665
-172.24.0.1      172.24.0.6       2 u  287  512  377    0.123    2.382  0.876


Code: [Select]

[root@maruko ~]# ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
-67.18.187.111   129.7.1.66       2 u  850 1024  177   66.888    2.844   1.485
+174.129.25.69   158.227.98.15    2 u  639 1024  377   35.557    2.227   0.196
+2001:470:1:15b: 132.239.1.6      2 u  397 1024  377  107.438    1.415   0.126
 172.24.0.1      172.24.0.6       2 u  673 1024  377    0.175    2.176   0.446
*172.24.0.6      .GPS.            1 u   48   16  377    0.324    0.382   0.083
 172.24.0.64     172.24.0.6       2 u  795 1024  377    0.309   -0.067   0.073
 172.24.0.68     172.24.0.6       2 u  821 1024  377    0.837  193.564  57.882


[joe912]

Cool project.
What's the end use for it? Have you investigated the IEEE 1588 timestamping features of the uC?

Also taking a look at the schematic I would recommend 0.1uF caps from TCT and RCT of the magjack to ground.

[gxti]

Just for fun, mainly. I plan on building a few into enclosures and maybe selling them, but sourcing the GPS module is the limiting factor for making them in any volume. Probably I will try to find a module that's good enough for NTP but not "timing grade" since it seems that new timing receiver modules cost $70 and up. Buying more Resolution Ts from China on ebay is also an option but not ideal as I've seen similar stock dry up and disappear. I need to do some experiments with the Oncores that have a phase skip as that will likely be similar to what I see with new non-timing receivers. I don't foresee it being problematic for NTP due to less demanding precision needs and use of the VCXO as a "flywheel" to smooth out jitter. The PLL parameters will need to be slower and I will need to soak test everything to make sure they track well over time.

I may investigate IEEE 1588 in the future. A 1588 PHY is only another $5, but I would probably want to upgrade to a full ovenized oscillator once getting into that territory. Definitely worth investigating though.

I'll look at best practice for the Ethernet grounding. I don't recall if the one I'm using has builtin EMI caps but I didn't tie the shield to ground anywa.

[joe912]

Timing GPS's do not seem to be cheap, which might make be reasonable since you are getting microsecond accurate outputs from a satellite.  It's pretty impressive though that for $70 dollars you can receive a pulse with 1us of someone on the other side of the world!

You shouldn't need a 1588 PHY to get 1588.  I have used the internal 1588v1 function on a Freescale P2020 processor with a non-1588 PHY. The timestamping is handled when the packet leaves or enters the MAC.  I have been able to achieve +-30ns(-15ns mean with 7.16ns stdev) synchronization between two boards over a 100m cable using only 25MHz crystal, neither board was synchronized to any other reference though.  Pretty impressive considering it takes about 700ns for the signal to travel that 100m cable.  If you get a 1588 PHY you should be able to get better timestamping resolution because your timestamp point is closer to the cable so less uncertainty. You can even do software 1588, but that sortof defeats the purpose because then you have large amounts of uncertainty due to packet buffers and stuff.

There doesn't really seem to be a "right" way of doing it after looking around a bit, I've just always seen those pins decoupled.

[gxti]

Revision 2 hardware is finally assembled. I've been making slow progress on the software, it now has more error detection and can support multiple GPS receiver types. The hard part of the leap second handling is also done, but I misplaced my docs for the particular TSIP variant I have to use so right now nothing is actually arming the leap second code. Still quite a long ways to go in terms of supporting lots of different TSIP variants, Oncore, NMEA, ublox, etc. as well as baud rate detection, attempting to wake up a receiver that isn't auto-transmitting, and some sort of serial command-line for configuring static IPs. But probably first I should write a bootloader, that way I can get hardware in people's hands even if it's not complete yet. Performance of the NTP server itself is excellent and I've tuned the PLL so it locks up in less than 90 seconds.

I'm also looking for GPS receiver modules that I could "OEM" as part of a finished product, probably not a timing receiver as they are overkill for NTP and harder to source but something with a solid PPS output with no jumps is essential. There are quite a few modules of unknown provenance on ebay in the $10-$20 range but really I'd like something from a reputable manufacturer that I can buy a few pieces of without breaking the bank or selling my soul to a salesdroid, and then get a larger quantity later if needed. Probably a u-blox module, but again not the timing ones as they are rather difficult to find in small quantities it seems.


http://www.flickr.com/photos/suigetsu/8437505440/#

[marshallh]

Good job on that board. I am a fan of oshpark too, too bad the silkscreen sucks.

[Carpenter]

Good work, property damage use only VCO, for NTPserver  is maybe sufficient, but for use as  NTPserver and frequency normal this is not good choice.
I will try to combine this project with any as http://ve2zaz.net/GPS_Std/GPS_Std.htm

[talsit]

Have you looked at the uBlox modules? About a year ago I sourced some LEA-6H modules from their local distributor in Australia for $32 each. Those were the higher-end ones, I'm sure that the MAX ones are cheaper. They use these modules inside of master clock timers on movies, where all cameras have to be synchronised exactly to a frame.

[gxti]

Revision 3 is here and has a NEO-6M module which is <$10 from numerous sellers on aliexpress (yay, China) and gets the job done perfectly for NTP. It will, of course, also fit NEO-6T modules but those are harder to get hold of. I ought to figure out if I can buy a partial reel, both for myself and also to part out and sell to other interested parties. But for this project at least, NEO-6M does perfectly fine. It doesn't have "position hold" but it does have "sawtooth correction" / "estimated quantization error" which is a feature that only timing receivers typically have. I don't have any glamour shots just yet, will post them when I can.

[Carpenter]

Unfortunately, under $ 10 including shipping is the only dealer and when ordering min 5pcs
http://www.aliexpress.com/item/Free-Shipping-UBLOX-modules-NEO-6M-GPS-positioning-module-NEO-6M-0-000-NEO-6M-0/733044197.html

[talsit]

That's actually not bad at all...

[Carpenter]

Unfortunately, any Chinese on ebay or alliexpres no offer NEO-6T.
Fixed position is in time aplication very useful.
If you count the average position error 3m, it is about 10ns time error.

If at least already sold new NEO-7, has a 50% higher accuracy and time programmable pulse signal to 10MHz

[gxti]

They say a man with one clock knows what time it is, but a man with two clocks can never be sure. What about eight?



Of course they are all being fed from the same GPS, which makes it a little less impressive. Just soak testing everything to make sure it stays stable. I haven't decided yet whether to sell these with the GPS module. Perhaps it'll be optional. I'm very happy with the NEO-6M timing performance, especially since it uses a fraction of the power of the Trimble Resolution SMT. The whole board comes in at less than 1W.

After this batch, revision 5 is already in progress. 4 was a disaster as I bungled the Gerber file export, but instead of reattempting what was already a rushed patch-up to deal with a power supply issue that was frustrating me I'm starting over. 5 will be more of a finished product, with an extruded aluminium case and additional ESD and power protection. I'm also replacing the VCXO with a TCXO, and thus dropping the voltage control part of the design and opting for a software PLL. The temperature compensation should have quite a positive impact on stability, while moving to software control might actually also improve stability considering the difficulty of very precisely driving an oscillator with a PWM DAC.

[LapTop006]

It's a shame it doesn't run (directly) off PoE.

[gxti]

I discussed that a bit on the time-nuts list. It doesn't seem cost effective for me to try to do it on-board when you can get a passive (fake PoE) injector/splitter pair for $5 from Amazon that will plug right in. I also wouldn't really want to put the old design on a pole out in the snow and rain due to the slow temperature response, but perhaps the TCXO will fare better. Thermal stability helps a great deal in getting the best performance although of course some people are more interested in the convenience of running Cat-5 not coax.

[JBeale]

I'm also replacing the VCXO with a TCXO, and thus dropping the voltage control part of the design and opting for a software PLL. The temperature compensation should have quite a positive impact on stability, while moving to software control might actually also improve stability considering the difficulty of very precisely driving an oscillator with a PWM DAC.

Interesting project! How hard would it be for an end user to take out your TCXO and use a different 10 MHz clock source?  What are the signal requirements?

[gxti]

Thanks for reminding me, I want it to be easy to do exactly that but had forgotten about it among a pile of other new features. There won't be an external connector but there will be an internal one on the board. Thinking now about how best to buffer that and how easy it would be to auto-detect the presence of an external clock, without adding too many extra parts. An inverter and feedback resistor will square up and buffer anything you can throw at it, and a mux to select the clock would get the job done. But while I have the board area it would be nice to not have to add ICs.

[JBeale]

from my point of view, I wouldn't mind a cut-and-jumper solution.  Or you could place the footprints and just make it a stuffing option without adding cost to the base model.  If there isn't enough interest to ever produce the external-clock model, at least it would fairly easy to add it later (as long as it's not a BGA part!)

[veryevil]

Any update on this. Very interesting!

[ddavidebor]

i've used the lea-4t timing module from ublox

they're awesome, you can set up it in timing mode and they do everything is needed to guarantee accuracy. they've also a built in tcxo.

in the newest ones, you can set up a frequency up to some Mhz

[gxti]

Revision 5 is assembled! I'm quite happy with this one, much more of a "system" than the previous revisions. More input and power protection, designed to fit a box, etc. Of course that comes at a price. The PCB is 55% bigger to fit the box and add some breathing room, and it was already my most expensive single part. It raises some questions of whether this is the best approach or whether a BeagleBone addon or similar would be more cost-effective. But I'm still going to see it to completion and let the buyer decide.

There are a trio of issues with the clock input circuit, but it all turned out for the best. I'm glad I put that input circuit there, because the onboard TCXO only puts out 0.8V and I ended up greenwiring from the TCXO to the external input just to make it work.

This is my first time doing lead-free solder, it's quite different from Sn63Pb37. More difficult to work with but it seems to be less prone to bridging, which is beneficial for the QFN. As usual I hand-soldered all the surface-mount parts except the QFN, of which I tinned the pads and PCB and added paste flux then put the whole thing in the toaster oven to reflow. I also bought a tube of quality leadfree solder paste for when I'm ready to make a bunch of these. I've done stencils before and wasn't very pleased with the whole process but it probably had to do with the ultra low quality paste I was using.

On the software front, I have successfully integrated the PLL math from NTPns to replace my simplistic original implementation. I also wrote a simple software clock exactly like you would find in your OS kernel, to replace the now-gone VFC disciplining circuit. The input capture code has been simplified considerably and uses 1 timer peripheral instead of 4. Even using the revision 3 hardware which has no temperature compensation, I can't tell the difference between hardware and software clock so I'm calling this a big win.

[dfmischler]

I like it.  Can you get a disciplined 10MHz output from it, too?  It wouldn't bother me to add an external buffer for that (so you could drive a few meters of RG58 into 50 ohms).

Powering this from 802.3af POE would be nice, but I'm OK with using a splitter for that.

EDIT: after reading that you have switched to a software PLL I guess you can't provide a disciplined 10MHz output.  Oh well.

[gxti]

The rev 3 boards have 10MHz out but it's not extraordinarily high quality due to the lack of direct temperature compensation (let alone ovenizing) and due to the PWM DAC that's driving it. I don't have any rigorous ADEV evaluations of it but there will definitely be frequency wiggles on millisecond timescales. That said, if you're interested I do have a pile of the rev 3 boards that I need to get cleaned up and shipped out, and they'd be perfect for someone who just wants to experiment.