Hard paper, fishpaper, vulcanized paper, just regular old (thin) FR-4 / G-10, or polyester sheet, etc.
Lots of options will do.
Incidentally, standard "fishpaper" is UL-94 rated, but not V-0 (self extinguishing). It's more subtle than that: it must burn at a particular rate! I tested a strip of my own and it does indeed burn in a well controlled (but definitely not self-extinguishing) way.

A reminder that, just because something has UL ratings, doesn't mean it's magically safe, regardless of its composition! UL is about consistency, about putting together elements such that the whole will be safe to operate, even if some of the elements (for whatever reasons -- good old cost being a big one) aren't the safest possible option.
And yes, insulation and flash arresting are the likely goals for those insulating sheets around PCBs. In a pinch, you could just as well apply layer(s) of tape -- assuming good enough stuff, like polyester or polyimide, preferably 2+ layers (giving a reinforced layer even if the air gap falls to zero). I suppose that would be fine in production as well, it's just that you can get whole, self-adhesive sheets punched to size and ready to apply.

Tim