Not unexpected actually, given mention of this filter failing in other equipment in the forum. Still, gave me a good scare (loud bang, horrendous stink) in the middle of making a measurement - thought I'd stuffed up a $$ meter.
Mess cleaned up with IPA - most on the chassis/cover, some on the digital board.
Replaced with the Schaffner modern equivalent, powered up and fortunately all seems OK. The old filter is fastened with two rivets; it looked difficult to drill out without leaving metal chips all over the place, so a sharp chisel wedged between chassis and the plastic housing of the filter, application of a hammer, and it eventually came out.
This HP3458A is nearly 20 years old. Filter has a HP part number on it, so the filter might also be that old.
So, for HP3458A owners who are have very old machines and are unaware of the tendency of this filter to fail, you now know :-)