Back in the 70s I think there were larger differences between equipment so getting a good amp might be audible.
Even then, con took a different form. Think about all variations of tube vs. solid state, bjt vs. mosfet, amplifier feet, ... However, some factors people thought were unimportant turned out to be quite important: open loop linearity, OPS symmetry, global vs. local feedback, deep negative feedback vs. low/no feedback, TIM, CFB vs. VFB, active loading vs. resistive loading, etc.
There are faults on both sides: audio, like anything else, isn't exactly 100% science, nor 100% art. The "subjectivists" (aka cable guys here) are pushing this into an all art situation; and the "objectivists" (aka measurement guys) are pushing this into an all science situation.
The truth, I suspect, is somewhere in between.