Not sure why you don't like the papilio duo, care to share? since I did back it up and have one in my hands I would like to know what is so wrong with it before I have time to play with it.
Back to the OP and I might be oversimplifying:
640*480 16bit = 614,400 bytes per frame
Camera can do up to 30 fps that's 1,8432,000 bytes per second (54.2535 ns per byte)
and needs 2 frames, he has 33.33 ms to do his computations before the next frame is ready, so he will need a triple buffer and another two buffers to store the results to be displayed so that's 5 buffers worth of frames with two being written and two being read in 33.33ms.
So buffer size is a bit under 3MB (3,072,000) for all 5 buffers.
2 buffers need to be written and read in 33ms (2,457,600 bytes) that's 13.56ns per byte giving little margin.
I would put the output buffer of the result in the fpga fabric to drive the output, or send it out or you'll need 6 buffers.
The memory module can write at 10ns minimum so that's 100MHz
The Spartan in the duo is a grade -2C (XC6SLX9-2TQG144C) and can handle 200MHz easy, at least I didn't find any switching speed lower than that but I didn't look too hard
http://www.xilinx.com/support/documentation/data_sheets/ds162.pdfSo doable at the max speed of 30fps that camera can do, but the timing constrains are going to be a pain.
Now for 60fps, there is no way but good thing his requirements don't call for that