I should have mentioned that this Fluke 77 was given to me with a mangled rotary switch. It is the original model 77 series 1 (7X-3001, Rev. K).
First thing I did was repair and clean those rotary switch contacts. The switch is working fine.
Turns out the issue was that tantalum capacitor C9 was installed backwards. On the PCB there is a + indicator for C9 and this marker is in the wrong position. And due to this mistake the factory installed C9 backwards with the + lead connected to ground.
I removed C9 and it tested OK, so I rotated it and installed it so that + went to R18 and - went to ground, as the schematic indicates.
Problem solved! Now switching to AC mode current from any other mode quickly settles down to zero as expected.
I also have a Fluke 75 series 1 (7X-3021 Rev. A) and I confirm that cap C9 is installed properly and the PCB marker is correct.
So it turns out that at least some of the very early 7x models had this manufacturing defect that was fixed in later revisions. Usually when a tant cap is installed backwards there's some fireworks, but in this case I think the currents were quite small and the cap was not damaged but rather just acted a bit strange. Kind of a.... fluke!
I've had many Fluke meters come through my shop but this one is the first time I've seen a very early model 7x series 1. Kind of an oddball meter with some questionable safety features, or lack of safety features I should say. For example, the high current input is completely UNFUSED. Um, okay.... Also, R1 which is the 1k protection resistor, is not fuseable/flameproof. As you can see in the picture it looks like a big old 2W carbon resistor. Glad that this stuff was fixed in later revisions.