DMA indeed bypasses any of the CPU logic - as it says, it's direct memory access, and typically glues directly to a dual port RAM block using its own bus. DMA isn't much of an issue here, although I use DMA to pull the data from the SPI flash into the cache block of the mapped SRAM for the overlay (on that custom design). But let's forget about DMA for now.
I think the bulk of the work in implementing the bus fault -> external (SPI?) RAM idea is that, if you want it to work with normal C, you need the bus fault handler to decode a whole range of possible instructions, because you will never know what the compiler will be using.
A much easier implementation would limit itself to specific instructions for reading and writing, but then it would not be code transparent. You would have to access the external memory via C functions which would contain inline asm.
The handler should not need to decode anything at all, that's already way more complex than a simple dynamic loader, I fear. Actually, I don't need 'dynamic' at all.
For a single function, it would just load the overlay in the exception handler, get the failure address from MMFAR, mask it for the physical address and jump, if I'm not mistaken (need to yet verify sane behaviour in the hardware simulator).
As far as I see, the MPU is lacking the feature to do a simple virtual/physical mapping, which in my caching logic is a simple and/or combinatorial, such that I don't need to do any compiler-specific trampoline stuff apart from the exception handler performing the data pulling from the SPI flash. All the rest of the app-linking and allocating in 'virtual' space is done by a linker file. I actually just realized I had this somewhat documented here:
https://section5.ch/index.php/2015/09/28/virtual-rom-on-small-fpgas/When not using the MPU, I assume that a BusFault is generated (provided the implementation specific bus logic covers that). So, this ends up just being a different exception handler, but the problem is the same, that an exception will be thrown on every subsequent absolute addressing within the virtual space, just lacking the fine grained control via the MPU.
One safe way out of the exception overhead situation would be a tool parsing the ELF and calculating the proper VMA/LMA offsets for the overlay 'app' like objcopy does, then no pc-relative or absolute confusion would get into the way. Haven't got a real concept yet, but this is likely the next thing I'd try. Then I just have to make sure none of the 'virtual' addresses ends up in a pointer elsewhere...