If I may ask, are you willing to share something of that design? Also regardless of the alternator application I would better understand opamp stability issues. Even if this circuit would be fine. But when looking around on the internet I can hardly find something with a transistor or FET in the opamp feedback loop. Every example that talks about poles, zero's and Laplace just has RLC components apart from the opamp but no transistor. I'm a bit stuck on this. I don't know where to start.
It was a long time ago and my notes and schematics are buried somewhere but I remember it well.
The operational amplifier measures the difference between the output voltage, and a reference, and applies that difference, multiplied by massive DC gain, to the modulator for the switching controller to produce a PWM signal to drive the high side power switching transistor which drives the field winding. (1) Internally the switching regulator controller has a ramp generator and comparator to make the modulator, which will be available at the "compensation" pin. I did it this way because I wanted to use an external operational amplifier, which made a 4 wire measurement of the battery voltage easy, but the internal one would have worked and is probably the way to go, but the design is the same.
I remember that this was the first time I got Unitrode to send me samples, but they were happy to do so.
A feedback network from the operational amplifier output to inverting input tailors the frequency response. The way I calculated it was knowing the field inductance which I measured, alternator "gain", in amps per amp, an estimate of the starter battery's ESR, and the modulator "gain", allows for drawing a bode plot and I worked from there to estimate the feedback network, just a series RC network, for stability. Then I ran load response tests to tune the frequency compensation network but it was very close. The alternator's field winding inductance is massive, like more than a henry, so it dominates.
The part of the circuit outside of the switching controller looked like that below, with the linear regulator replaced by the switching controller and high side power transistor, and the frequency compensation network across the operational amplifier. At the time I had recently used a circuit based on that to make some very high performance low noise precision power supplies at work. I would do it differently now with more safety features but with the same general idea.
(1) I used a fast recovery rectifier located at the regulator as a flyback diode across the field winding which is not ideal with the long lead length but it worked fine.
Was the purpose of the 'frequency compensation' to prevent oscillation or to decrease the rise time and improve transient response? I'm assuming the latter, but I don't know the application this was for--and I'm sort of curious Actual automotive applications have so many additional parameters nowadays that this function is almost universally done within the engine controller system. Precise voltage control is pretty far down the list of priorities.
It prevents over and undershoot with load changes and as I recall, I tuned it for slightly more than critical dampening but honestly it is difficult to get it wrong for the reason you identify; the inductance of the alternator's field winding is so high that it dominates the response. You could discard the PWM modulator and drive the field winding with a comparator output that has a little hysteresis and get fine results.
I remember worrying about integrator windup at startup, and I had a plan to fix it, but it was never a factor.