A few random thoughts in no particular order:
1) Vivado is faster and better in pretty much every way than ISE, in my experience. That was not true a couple of years ago, but at this point I don't miss ISE at all. If your project takes longer to build in Vivado than in ISE, make sure you're using comparable settings and taking advantage of out-of-context IP synthesis.
2) Consider Hyperram if DRAM routing or controller complexity is a problem for any given reason. Sounds like you may have to do some interleaving, either way.
3) Start with an eval/demo board or trainer. Do not attempt to bring up hardware at the same time you're getting your first FPGA project going, especially not one as complex as this one sounds like.
4) No matter what you need to do, there is almost certainly either an existing tutorial somewhere for it, or a ready-made project somewhere on Github. Do not attempt to build complicated stuff from scratch when you're not entirely sure of what you're doing.
5) There will probably never be a time when FPGAs and their toolchains aren't stupefyingly complex, slow, and annoying. It is what it is. Again, your best defense is to start with an existing hardware platform, even if only for tutorial purposes.