From a friend I found out that porting an existing "Linux" variant for their HW platform that was supported by the variant, including hardening such as killing all unneeded services, updating and employing security and making it stable (production ready), took them almost a year with four experienced software engineers and an architect. So his best guess for a mature development cycle would be around 3 man years at least. My question why it took so long: many many hidden time consuming issues if you want it to get right and stable. This is for devices that stay on 24/7 and are internet connected. After one month they had it up and running. The rest of the time was getting it lean, secure, and stable and the remote firmware update also took them quite a while.
Interesting, because this reflects my own experiences about three or four years ago, it seemed like a great idea to use something where everything was already implemented, including integrated WiFi with reasonable speed, and both Ethernet and USB stacks. What wasn't so great was finding bugs for various bits and pieces, particularly in the USB stack, slow boot times and being terribly power hungry. Getting fixes re-integrated into the build proved difficult because I wasn't in the clique. It was a case of the grass apparently being greener, but in the end it wasn't. Getting a PoC up and running was reasonably easy, but making it into something that would be reliable, lean and ready for production use proved to be just too demanding in terms of time and effort.
Yes, reliability and production readyness is my biggest issue. I am working with a custom linux SBC, without real network, and it is a pain. I had to implement a support for an extra USB feature, you know, the type you plug in in windows and it works.
OK, plug in, nothing happens. OK, is it supported in the kernel? Recompile the thing and the linux a dozen times to work (that is a few hours every time). Ok lets find a program that uses it. Ah, there is two dozen, but only one in the distro. Ok, lets try to add it. Few weeks later I realised two different version of the program is on the distro, and the files are strangely mixed. Ok, so unmix them, apply a bunch of patches from random internet forums. Compile the linux another dozen times.
The information about any Linux related stuff seems to be randomy organised. Forums and blogs and mailing lists going crazy. "ah your software is not working, just get another one with apt-get..." I have no network (by design), and I dont want to do it on every machine we make from now on, and I'm using a different distro.
And this was one issue. And I'm a hardware engineer.
This is the definition of not having fun at work.