Figured it out, but just to close the loop, here's what I did:
To start off, I think most of my confusion here was that I forgot to define a route tool layer in the first place. Having done that, the rest becomes pretty obvious.
For the purposes of this nested design panel, I created a copy of the larger PCB that didn't have the cutout region. I then placed the area to be cutout on my route path layer. Simple as that. Now when I lay the designs on top of each other using the panelize tool, everything renders correctly. Now just to add some tabs/mouse bites...