Depending on the load, the small offset might be enough. But at about 3 mV output the OP can not sink much current.
The carge pumps use relatively low resistance FET switches to charge the caps. So the current flow it not continineous, but in short pulses (e.g. 10 µs) at about 1 kHz clock. Peak current are thus somthing like 100 times higher than output current. There is also the bucket capacitor with a full 5 V rectangular swing, that can cause electrostatic coupling if not properly shieded.