Some cars are definitely more fatiguing to drive than others, I just don't see it necessarily correlating with size. High noise level is part of it, vibration is part of it, how comfortable the seat is to sit in for a long period is significant, overall ergonomics such as the placement of the steering wheel and controls, armrests and such. And visibility plays a significant part in how much workload there is to drive it. In some cases larger cars have an advantage, but small cars typically have better visibility and it's easier to maintain spatial awareness. It's like a chair or a pair of shoes, what is comfortable to one person may not be for someone else.