Interesting that you mention the method of offsetting the negative input by 26mV... I recall in the past I'd done just that for offsetting a non-zero initial CS voltage from an Allegro hall type current sensor (it was a model that swung both positive and negative current readings but the CS output was all positive voltages so I had to "subtract" the lower half - 2.5V). In that case, I used a wire off a 5V regulator for the ICL7107 based volt meter, setup a halving voltage divider to supply the offset voltage and connected it to the negative side of the the voltage inputs. I was surprised that it worked* even though the negative input was connected to the ground of the supply voltage (initially I wasn't sure if the offset voltage would actually be noticed by the ICL7107's differential input in such arrangement).
Since the volt meter has a TL431 (2.5V reference), I'm thinking I could attach a wire to it and add a 1:100 voltage divider to supply 25mV to the negative input of the differential input of the ICL7107. This, assuming it would work, would be simpler than using a separate op amp etc.
* Update: Way back when I had done the offset method, I initially encountered weird fluctuations(like drifting) of readings not only on one meter but all others that shared the same 5V supply source/ground. The 5V supply for the offset voltage and the 5V supply for the voltmeter(ICL7107) were 2 different sources, and so that may have caused a ground loop issue. I recall I had to use small isolated 5V DC-DC converters for the 5V supply voltages on the meters that implemented the subtractive offset voltage (on the differential input's negative terminals). I think the fluctuations were due to the 5V ground being affected by the volt meter PCB which links the differential input's negative terminal with the main ground of the 5V supply voltage input. At any rate, the isolated DC-DC converter used for 5V supply of the voltmeter resolved the fluctuation issues.
This time, even though I'll be using only one source of power (the main server PSU itself, as opposed to separate adapters) with a 5V regulator in between, I think I'll use the same isolated DC-DC converter on the meter to eliminate any possibility of the meter affecting the server PSU other than acting as a minimal load.