burying copper inside your PCB will effectively insulate it from the surrounding air, right?
What? You aren't insulating anything, but spreading the heat, thus increasing the power dissipation factor.
You can't insulate anything by adding thermally-conductive layers 
FR4 is a much worse heat conductor than cooper, adding a ton of vias and tinning/filling them will be a much better option.
The thermal performance improvement will be small or large, but will be better for sure.
And air (~0.01W/mk) is a much worse heat conductor than FR4 (0.25 with no copper present).
Common PCB design myth, that internal layers are somehow "super insulated" from the outside environment. When it is not the case.
Distributing heat, from where it's generated, to where it needs to go, is probably a somewhat underappreciated problem.
A basic calculation of, say, a transistor bolted in the middle of a wide sheet -- well, I wish it were "basic", the solution (for fairly basic/standard assumptions, that is) actually involves a Bessel function -- but the result is that heat spreads out in some radius (to within some margin of total dissipation e.g. 90% of power dissipated within some radius R), determined by sheet conductivity and dissipation rate (assuming linear convection, no chimney effect etc.), with the total temperature rise depending on these constants, and the source radius (that is, assuming heat is deposited within an isothermal patch of radius r).
The temp rise depends critically on r, because as r --> 0, the thermal resistance of the sheet diverges, as the circumference of layers carrying heat away from the point are approaching zero. This shouldn't be a controversial point; it's also why e.g. arcwelding, or laser cutting (or e-beam welding for that matter, for a somewhat exotic example), require relatively low power levels (from kW, down to 10s of W) to fully melt or even vaporize the base metal in a small spot, whereas doing that over a large area would take orders of magnitude more power.
You want a large starting radius, to keep device temperature low. Transistors use small dies on large copper tabs to facilitate this. But you don't want to put too much power through a tab of some size; hence why we might recommend TO-220 stay under 50W, TO-247 under 100W or so, etc. (particularly with thermal interface material in play; with greased joints and large or liquid-cooled heatsinks, double or triple these figures can be feasible).
There is value in heat spreaders. Instead of bolting a device to a heatsink, bolt it to another, heavier tab, to a heatsink. Or bolt it to multiple heatsinks.
Or, those CPU heatsinks with a copper slug pressed in place. Gimmicky? Maybe. Effective? Yes!
A similar case is that pyrolytic graphite sheet stuff -- terrible, awful thru-plane conductivity, it's basically plain graphite; but mind-boggling in-plane conductivity,
better than copper, approaching diamond even! Now, if you have a few 0.1mm's to spare, you get the same effect from a somewhat heavier sheet of copper -- but in the rare case you don't, this stuff can do wonders. You still need to spread out heat wide enough in the local area to ensure it's flowed through the sheet, but once there, it's spreading as far as it can. A lining of that might make a plastic enclosure thermally feasible, for example, whereas you might have to go with painstakingly machined aluminum otherwise.
It's a situation very similar to multilayer PCB: you need to spread the heat out a little ways first (expand the device footprint with pours; start getting heat into the board with vias), to get it into all layers; from there, it continues to spread out along inner planes, and since convection rate is lower than PCB thru-conductivity, the outgoing thru-FR4 conductivity is negligible, it only counts against you significantly near the source.
Tim