Sockets, keep-alive, and VPNs

[peter-h]

I have a couple of boxes (32F417, FreeRTOS, LWIP) running, interconnected via ETH, over a LAN or a GPRS/3G/4G WAN, or ADSL via a VPN. The VPN absolutely works; been used for years for remote working etc.

One is a server and one is a client. The data passed across the LAN/WAN is data generated internally, and/or arriving on a UART(s).

It is basically working, but there are some basic issues. If the client generates some data, that is obviously going to work even if the WAN has disconnected because the client will re-establish the connection (it has the server's IP configured). But the other way (the data is generated at the server end) won't re-establish because a server can't call up a client. The solution is keep-alive packets which make the client establish the connection after power-up and it stays connected.

TCP has a documented KA option, but googling suggests few people seem to know what it actually does and how to configure it. An extra variable is that some WANs may intentionally drop KA packets; in my case the packets are either missing (and I have no ETH debugger to check; one which I used years ago was incredibly difficult to use) or an IPSEC VPN (a fairly standard VPN between two routers) is disregarding the KA packets and disconnects.

I can disconnect the VPN manually and it stays disconnected for hours even though the KA interval is set to 5 secs.

I have this to configure KA where g_ethser_ka is the desired KA interval in seconds:

Code: [Select]

           
            int optval, optlen;

            optval = 1;
            optlen = sizeof(optval);
            setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &optval, optlen);

            optval = g_ethser_ka;
            optlen = sizeof(optval);
            setsockopt(fd, SOL_SOCKET, TCP_KEEPIDLE, &optval, optlen);

            optval = g_ethser_ka;
            optlen = sizeof(optval);
            setsockopt(fd, SOL_SOCKET, TCP_KEEPINTVL, &optval, optlen);

and, in LWIP, setsockopt is a macro:

Code: [Select]

#define setsockopt(s,level,optname,opval,optlen)  lwip_setsockopt(s,level,optname,opval,optlen)


Does this make sense?

I also wondered whether a better way to do KA is to send "normal" data packets but with zero data length. Those may be less likely to be dropped than something which some intermediate network hardware recognises as a KA packet. But then the same hardware may be dropping packets with zero data, too... So one could send data packets containing just one byte, and any real data would be sent after that 1 byte (so 500 bytes of data would be a packet containing 501 bytes and you chuck away the 1st one).

On top of that, it isn't clear that LWIP implements KA correctly. Extensive googling shows a lot of people having gone up this road but almost nobody is reporting solutions (but that is normal for social media, too).

[retiredfeline]

KA uses standard TCP messages, see here: https://www.cspsprotocol.com/tcp-keep-alive/ Certainly Linux implements KA. I don't know if LWIP implements KA. You may have to dig into the code.

[peter-h]

That much I found, but I wonder if anyone has knowledge of how KA is configured, and whether "WANs" (or even LANs) are likely to be discarding the packets.

LWIP definitely documents KA but whether it works, nobody seems to be sure.

Some KA implementations are plain bizarre e.g. you config it and it runs for some hours and then stops after some other interval which is configured elsewhere.

[ejeffrey]

Unfortunately, we can't count on TCP to be end-to-end reliable any more.  TCP is the new UDP and protocols based on TCP need to be built on the assumption that they will be dropped at any time.

In my experience it works best to build the keep alive into the application layer rather than the TCP KA option.  This isn't always an option of course with existing protocols, but most well designed protocols have a lightweight, nullilpotent request that can be used for this.

[peter-h]

Of course, it can't be if the connection is broken.

My Q is more whether KA packets are actually generally implemented.

Unfortunately this stuff is very hard to debug at low level.

[peter-h]

One thing I have found is that if say the server is rebooted during a connection, the connection is not re-established (by the client).

Maybe that is how it is supposed to work i.e. if one wanted to re-establish a connection in such a situation, the client should be sending out regular "real" packets, not KA packets.