But of course the client does need to store their own private key. How otherwise would you authenticate the client?
Obviously you know the answer
The client accesses the server by doing a DNS lookup. Now obviously the DNS could be poisoned. But the client can be disassembled anyway and any private key extracted from it. There is no perfect solution, unless you put in a secure crypto module.
However one can add additional security within the comms, and normally one would even if TLS is used. For example firmware updates would have a hash on them, although again this is not secure from client disassembly.
Ultimately any single client can be messed with, but that's OK. What you really do not want to happen is somebody trashing the installed base of say 100k of them, or any significant part of it. And that seems fairly easy to do so long as the server is not penetrated. The next thing to do is set up the server so it does
not contain a database of the installed clients. I think that is easy enough because the ultimate mutual authentication could be between the client and the customer's app trying to access it. The server's purpose can be just to facilitate the clients being, ahem, clients, all behind various NAT routers so they can't all be casually discovered.
I am sure many people have been around this before (it is a very common application) but they won't be posting the details here
But I also know that many don't bother; I recently stayed in accommodation where the heating was remotely controlled and the controller was a server and the router had port forwarding.