What are the logic levels that are driving inputs of multiplexer? These CD4000s multiplexers require that the voltage of logic "1" level on the inputs must be close to VCC, as it is indicated in datasheet.
I had such problem in one of my previous designs, and after close inspection of the datasheet and botched level translator to the PCB it was fixed.
The 4052 inputs are drive with ULN2803 outputs with 12V as high and 0.6v as zero, and the ULN2803 inputs are driven with a MCU powered @ 3.3V, so the inputs are fine.
Note the OP07 can swing neg to pos, but the 4052 can only tolerate 0-12V as you have wired it.
The analog switch resistance should affect the lowest range the most, and then only some %, your higher ranges are 'off by miles'.
I know that, But as it can be seen the shunt resistors are driven with in the 0-5V range, the right side which is consist of Q6,Q7 etc pairs and Q16 Q17 etc pairs are set to 2.5V supply AKA VGND or virtual ground, and the left side is driven with the currentsense output and an external Function gen for now with a sine wave with a 2.5V DC bias, so every thing is held at mid 5V supply, so I think the 4052 supply maybe not an issue.
The on-resistance of the 4052 multiplexor in series with the feedback resistor might explain it.
I think it would mostly affect the 1X range,
There is a different circuit configuration where the switch is moved to be in series with the inverting input making its on-resistance irrelevant.
I do not get it fully, would you explain more?
What does "current sense" drive ? Also, how are you measuring the gain - with respect to ground, or V-hi ? And what value is VDD that supplies U3?
The current sense would drive the negative feedback, you can check it in this post schematic, U3 supply is 12V and GND, also I would measure the gain like this, 1 channel of the scope is connected to the VDAC, and the other channel is connected to the current sense, Also I have inserted an Ampere meter in P2, so I can measure the output current directly, So basically If I change the 4052 switch to different ranges I should expect the output current be lowered 10X or 100X from 1X, But the Ampere meter would only show 1x, 1.1X and 2.16X in practice, for different positions, also the VDAC and current sense outputs seems to regulate to the same value,