For the most part, high speed per-pin Select I/O SERDES on Xilinx devices is only useful if the send and receiver are deriving their timing from the same clock.
If your source sensor is sending data at 500Mb/s +/- 50ppm, and your FPGA's IO clock is 500MHz but a different +/- 50ppm, then you are doing it wrong. Have the FPGA generate the sensor's clock, or connect the sensors clock to your FPGA's clocking infrastructure.
You cannot implement line-speed Clock Data Recovery using a SERDES, even if you are using a CDR friendly coding scheme. At best you will end up clocking your FPGA design faster, and then need a Data Enable Strobe to identify which cycles have active data. Regenerating a nearly-jitter-free pixel clock for video data or an I2S clock for audio data is not simple, (and maybe not even possible).
Differences in relative phase at the IO pins can be trimmed out using IDELAY and ODELAY primatives, or by adjusting the phase in the clocking infrastructure (MMCM or PLL). You can detect phase dynamically by having two SERDES modules sampling the same pin but at different phases (effectively sampling at 2x), and trimming the phase so the "dodgy samples" (the ones that occur on the rising/falling edges of the data bits) are all seen in one SERDES, thus ensuring that the other SERDES is seeing clean samples taken within the eye.
The only other way is to oversample the data (at 3x or 4x faster than the data rate), and then identify the transitions yourself and follow as the relative phase creeps this way or that. This technique tops out at about 300 or 400Mb/s per pin (due to the 1.2Gb/s max for a SERDES), and is extremely expensive in area, power, time and complexity. It also cannot work if you have sparse transitions in your data stream (i.e long runs of zeros or ones), which could be likely for sampled video data.