Why is such a circuit considered linear? The transistor will furiously switch on and off as the opamp tries to adjust. Isn't this the basic mechanism of switching power supplies?
Transistors do not switch on and off.
Ever.
They are a continuously variable device, under all conditions.
That is where your error lies.
A circuit is linear if the linearity property applies, over a useful range, and neglecting noise:
Vout is a function of Vin. Vout is a linear function, i.e.:
Vout(Vin) = gain * Vin + offset
Let Vin be the sum of two signals, Vin = Vin1 + Vin2. Then,
Vout(Vin1 + Vin2) = gain * Vin1 + gain * Vin2 + offset
That is, that the additive, associative and distributive properties hold.
A real amplifier will have some distortion, so that the output is not exactly as above, but there are additional terms involving both Vin1 and Vin2 (their products, and higher powers). A real amplifier will also have noise, which looks like a small random variation in the offset, and/or in the gain. A real amplifier also cannot amplify forever, so that at some point, Vout "saturates" (it cannot go higher or lower than some physical bound, such as supply voltage).
And of course, change Vin or Vout to Iin or Iout for current type (transresistance, transconductance, current) amplifiers.
Finally, for a constant input, the change in output shall tend towards zero (again, ignoring noise), over time. That is, it is an asymptotically stable system.
A switching circuit does not have all of these properties. A class D audio amplifier may exhibit linearity in its bandwidth, but no matter how well filtered it is, there will be a small residue of the switching frequency on its output: therefore, it is not asymptotically stable, but constantly moving around, however small the movement may be.
A comparator is the fundamental concept of a switching device: for an input strictly above threshold, the output is high; below, it's low. An ideal comparator switches instantly, but we can only approximate this behavior with real transistors (which are continuous). A real comparator must take some time to transition between high and low. Similarly, this is true of all logic circuits.
A more practical description of a switching circuit, is one which spends a large amount of time near the on and off states, with little time spent inbetween (but not zero time inbetween, which would be impossible). This is typical of switching power supplies, amplifiers, digital logic and so on.
This is a rather heavy weight description for a beginner, but I hope that these keywords will inspire you to read more on the topic, and to come to a fuller understanding of why these things are as they are.
Tim