In these designs that use this type of conversion scheme, IF feedthough is a problem on the 7.0 MHZ band, for one thing without a properly tuned IF trap somewhere after the mixer but before the PA input it becomes possible for an operator to tune up the transmitter to operate on the IF frequency, that would get said operator in trouble.
The ALC circuit is seeing more signal than it should, and doing what it does best, provide a signal to lower the gain of a stage or stages in the transmitter chain.
Before I had a spectrum analyzer and all the pads and directional couplers, sniffing loops and DC blocks you need to take meaningful measurements I would use a sniffing loop with an adjustable attenuator connected to a receiver tuned to the transmitter IF frequency and use the receiver as a selective volt meter.
Works great for tuning those traps and general troubleshooting.
Which is why I know what that coil is used for. (
notice the manual doesn't provide alignment instructions for that coil Yaesu manuals Suck deeply.)
And i will mention this again. None of those circuits internal to that radio are designed to work into a fifty ohm load such as your spectrum analyzer, which is why I suggested using a sniffing loop. Nobody back when that radio was being built was designing their subsystems to work into fifty ohms, nobody making amateur radio gear that is. So when you plug one of those cables into your SA you alter how that circuit works.
At this point you should be able to put that coil back in, place the radio in transmit on 40 meters and tune it through its range and observe the change it has on the transmitter's output. That coil does have to be tuned to 8.9MHZ to notch out the IF signal, you might find things working a whole lot better.
Good luck.