I'm considering doing a rather large robotics project in the nearish future that involves managing multiple different cameras and I need some advice/constructive criticism. At bare minimum, my camera array will include either a 48MP, 50MP, or 64MP camera for still photos with it ideally being capable of recording 4K60fps video, optionally a second 4K60fps camera if I can't find any 64MP cameras outside of Arducam that can record 4K60 and switching it through a multiplexer to take stills, with an additional four fisheye 5-12MP cameras running through another multiplexer for object detection and avoidance. My main concern, aside from finding out where I can even find camera modules capable of that outside of Arducam (like from Samsung:
https://www.sem.samsung.com/global/product/module/camera-module/mobile-camera-modules.do), is how I should go about handling the data from the cameras. I originally considered using a Raspberry Pi 5, something I have more experience with, but decided against it when considering that it is going to have to both encode the video signal, take 64MP stills, take multiplexed stills or video from the fisheye cameras, and run the object detection/avoidance software.
Currently I'm considering a few paths involving either a Jetson Orin Nano or a Kria KV260 or KR260 (Xilinx Zynq Ultrascale+):
1 - Use a Jetson Orin Nano to both encode the video data and process the video from the fisheyes for object avoidance and send commands to the MCU controlling movement
2 - Use a separate FPGA for video and image encoding, and have the Jetson Nano do the rest.
3 - Use the FPGA/APU on the Kria for both object avoidance and encoding.
Pros
1 - I only need to buy a Jetson Nano, cameras, and multiplexers, and I should be able to program that somewhat easily.
2 - The Jetson won't have to deal with encoding the raw 4K60 video feed or the raw 64MP image data.
3 - I could use existing IP for the video encoding/decoding as well as for AI/object avoidance. Has 3 MIPI connectors over Jetson's 2 if I use the KV260 so if I can't find a 64MP camera that can do 4K60, I won't need a multiplexer for the 4K and 64MP cameras.
Cons
1 - I'm not sure how well the Jetson will actually handle that kind of a load, though when considering that people have been using Arducam 64MP modules for DSLR projects with the Pi 5, I tend to believe that the extra processing power the Jetson has will be sufficient for what I want to do.
2 - This is the most expensive option, though I could use a cheaper FPGA for it and it would require less development for the FPGA itself.
3 - The Kria is as expensive as the Jetson Nano, but has less RAM (4GB vs 8GB), has a slow APU compared to the Jetson (though I'm not sure it will matter if most of it is done in the logic cells), the carrier board doesn't (to my knowledge) break out the PCIE port on the devboard for SSD drives - not a total deal breaker, and I have little experience with FPGAs so I'd have to spend a lot more time getting acquainted with that in comparison to the Jetson before I could consider progressing with this project.
I have experience programming CPLDs, but haven't dealt with FPGAs more than once or twice, so I'm tending toward the Jetson.
I'm also considering using USB-C camera modules from Arducam, but they are very expensive in comparison with the MIPI modules ($250-320 vs $50-70) and it would only help with video encoding. It will be a last resort though if I do use USB cameras.