If a bottom-side part has its courtyard primitives on layer 15, they will not automatically move to layer 16 when you configure the layer 15/16 pair, because Altium has no real way of knowing that those primitives represent the courtyard or that they should be on the top or bottom until you tell it so. It would be perfectly valid for a top side part to have a bottomside courtyard, or vice versa, or for a part to have both top and bottom courtyard (imagine something like an edge-mount connector).
Ideally, all of your library components should have a consistent set of layers, and those layers should all be set to the appropriate layer types before you add them to a PCB, and that PCB should also have all of its layers defined to match the parts. If your project didn't start out that way, then ideally you would correct the PCB layer settings, correct the layers settings in the library parts if necessary, and then update footprints in the PCB from libraries.
As a quick fix, you can select a component, unlock its primitives in the properties window, change whatever primitives to whatever layer, and then re-lock primitives -- but you will have to do that on each instance of each component, so might not be very 'quick' after all.