I saw Xylon demo on face detection, but they were using Zynq:
As you can see, they are also doing other detections: eyes, cheeks and lips. Original video claimed to be 1080p60, but captured at 30FPS.
I don't see any reason why you could not do anything beyond 1080p60, you just need more bandwidth with your DRAM.