That's probably true, but if you modeled each of those elements individually, the net result would probably be as close to the behavior of any given real tube amp as any other given tube amp is. I.e., it doesn't need to be a perfectly accurate reproduction of exactly one -- just a realistic emulation of any one.
I think Peavey's amp sim product tries to do this (simulate the components), but I'm not sure how well they pulled it off. I got a demo a couple years back, but it didn't work very well for some technical reason that I can't remember now. Many of the others (Waves, etc.) seem to follow the "complete signal path" modeling approach. There are, indeed, probably too many variables to make the complete model behave identically to the original, even if a particular demo clip sounds similar, and that might be OK if it sounds good in its own right.