Settings: OSR = 64, 16 MHz, prescaler = 1, data rate = 62.5 ksps, I can't validate for OSR = 40960, as my code runs on block structure, and filling 32768 samples at 60 sps would takes hours.
MUX = FF, both input jumped to Internal V CM.
This is what I have inside input buffer:
*0 -5 -5 -5 -4 -5 -6 -6 -5 -4 -5 -5 -4 -5 -4 -5 -6
16 -4 -5 -5 -5 -5 -4 -6 -3 -4 -3 -5 -4 -6 -4 -4 -4
32 -5 -4 -4 -5 -5 -7 -5 -4 -5 -4 -5 -3 -5 -5 -4 -3
48 -5 -5 -4 -5 -4 -6 -3 -4 -4 -3 -5 -4 -4 -6 -4 -4
64 -7 -6 -5 -5 -4 -6 -5 -6 -3 -6 -5 -5 -5 -3 -7 -5
80 -7 -5 -6 -4 -6 -3 -7 -5 -7 -5 -6 -5 -6 -6 -6 -5
Data "casting" to get AC:
int32_t tempr = adc[0][i];
tempr &= 0x00FFFFFF;
if(tempr & (1L<<23))
tempr |= 0xFF000000;
tempf = tempr;
tempf /= 64.0;
fft_r[i] = tempf;//tempr;
As you can see there is a division /64 to scale everything down to 18-bits (compatibility with AD & TI tests).
So, -5 x 64 = - 320, -> average DC offset (adc was not calibrated);
+-1 x 64 -> 64 "counts" - essentially noise, +-2 x 64 -> 128 peak-to-peak noise;
What I see from the picture, your hardware design is missing antialising filter at the inputs of the ADC. I have 10nF in between inputs, and 4.7nF each to ground.
The name "antialising" is actually misleading, better to call "OPA protector against ADC charge kickback". Microchip's adc's are not so aggressive like TI, but still there is switching capacitor load present.
Another possible issue - reference driver mcp6002 may oscillate.