1. We used to check ADC performance in ADC counts and PPM as well, not abstract mV w/o information about full-scale voltage. Please tell me how could I know - your 4040 mV is full-range or just 1/4 of whole range?
2. What components between reference and ADC? Any capacitor there as well? You must put low ESR capacitors close to every voltage input of ADC, including reference.
3. Put 100 .. 330 ohm series resistors on all digital (clock/data) lines between ADC and MCU. Maybe before that try slower data transfers and see - it changes results or not.
4. Something is wrong. Can't be. Double-check everything including software.
5. TI is not that kind of company that could fail with chips specs so miserably. In case you have option to consult TI field application engineer or just support - do it. Most likely there's something about your circuit like... dunno... ground loop or software error or something else. You just have to go through everything and double-check everything.
1. Scale is not relevant for linearity testing. I mention that "-" negative input connected to Ref. voltage, and MCP1501 is 4.096V.
2. 100pF capacitor to ground. DS for MCP1501 states, that IC doesn't require output decoupling capacitor at all.
3. Test results getting much worse with single 1k on clock line, see last column on second page in the attachment.
It's not reflected on image, to prevent re-scaling.
4. It's not my first adc on test bench. Same software has no issue with ads1232, nau7802, mcp34xx/35xx/
5. I agree here, could be fault on my part. ADC is on a beadboard, and something can easily goes wrong. At this point, I could only say, that ADC is not hobby friendly Any decent audio codec (wm8731, pcm1803 - tested in same breadboard condition) works much better, and I2S clock-data bus is no way less noisy.
Updates.
Hell, unbelievable. Ringing on both inputs line 50-100 mV, I just can't imagine that sigma-delta 24-bits adc could be so noisy. It's not some kind of RF 400 MHz sampling adc. So I put 47 OHm on both inputs, 0.1 uF ceramic in between, plus 0.1 uF on reference and non-linearity almost gone. At least down to 15- bits - enough for breadboard.
Thanks for advising, it was under number 2 in your list.