I don't understand what it means to "reinvent UART/SPI/I2C drivers".
These simple interfaces can be easily used directly in the application, "driver" at most being maybe a helpfully named macro or a 5-line helper function; these may or may not follow along from project to another. In any case, these take tiny fraction of the development time.
The reason it works this way is:
* The simplest cases (for example, just initialize UART for a certain baudrate, then send bytes in a loop or give a buffer address for DMA and let it work freely, give an interrupt on received byte) that you could write a generic driver interface for, also are so simple to implement ad-hoc that such generic interface is not needed. This is the trivial case; it's not utterly important whether you use libraries or not, but in general, I recommend avoiding unnecessary complications.
* The difficult cases (where you utilize the more advanced features of said peripherals, so that they intervene in twisted ways with your complete application including other peripherals) are always hardware specific and coming up with an interface to catch all difficult use cases and HW features is basically impossible, or only makes things more difficult, so you don't want to add layers. This is where projects completely crash when people try to force the libraries which simply can't do the job. If they don't have the skills (or guts) to Do The Right Thing, they are stuck, and the project just dies.
So, basically no to extra layers. Only do work which can be proved beneficial and not hindrance.
Sure, it will be hardware dependent and migrating to another microcontroller requires some work, but in case of SPI or UART that would be something between zero to a few hours max (unless you hit a case where the original peripheral implemented some really important special HW feature you needed; in this case, no amount of layers or interfaces will fix the problem, quite the opposite, they will hide it), even in a large project. This is nothing compared to the tedious work of hardware redesign.
How ST solved this problem, for example? They created abstraction layers that abstract absolutely nothing. But now management people requiring these layers are happy. Almost all peripheral features are accessible because the layers do not abstract anything, it's just syntactic and semantic mapping interns write the in a rush with some errors.
Free tip of the day: open the documentation. Write in C, bare metal, using just the headers defining register names etc. The same old way it worked in PIC or AVR, since 1990's. It still works, and it still works well. You can do anything, use the part to the fullest, with least amount of code, which is easy to follow and modify later.
GNU toolchain is great, and the interface has been the same for decades. Don't add extra layers to it, just use the GNU tools, namely compiler, linker, binutils. make is good for automation.