Sorry about the lack of schematic, I was at work so I couldn't post it. Here it is. As I said I tried a bunch of different values for the resistors.
Ah, I know. Highly probable the explanation lays with the
offset voltage and current bias.
You are assuming the Vout=alpha*Vin.
In reality, due to OpAmp's offset voltage and current, the Vout will have a Y intercept at non-zero coordinate.
Something
like this (exaggerated to make the things clearer), the red line being the actual Vout=f(V_in) and the blue one is the assumed Vout=f(V_in)
LM158/358 spec - page 5.
You can have a typical 2mV yAxis intercept due to the voltage offset (may be positive or negative).
Additionally, it can be compounded with a voltage difference caused by the current bias - the inverting input has 0R in input, but the non-inverting input will feel "those two halves of the potentiometer in parallel". Assuming you are close to the mid of your pot, it'll be 500K||500k=250k which will "steal" from the input voltage due to the current bias - with a "typical value" for current bias of 45nA, over 250k resistor this give a "typical value" of around 11mV - to be subtracted from the actual non-inverting input voltage.
Now, you've made a 1-point calibration based on the assumption that the axes intercept of Vout=f(Vin) is in origin. As such, your computed slope (blue line) will be smaller than the actual slope of the OpAmp, which will give you higher (than expected) measured values when your Vin is above the "calibration point" and lower when Vin is lower than the calibration point.
The good news is that your relation should be linear (thus easily invertible by the software), except that you'll need a two point calibration to get both the slope and the y intercept.
Or, you know, just simplify your design: put aside your OpAmp and make the connection straight into the ADC - CMOS, with an impedance in hundred MOhm range, is better prepared to do the job than a bipolar OpAmp.