Does it hit the pipe because you don't want to build some standoffs to prevent it or because you're really running out of the mm of space to achieve it? The latter seems implausible. Given the application even a couple mm of airgap is better than any insulation which could fill those mm's.
Create some superstructure to keep the heatsink away from the pipe before you try to insulate anything ...
Implausible, but that is actually the case. The outermost fins of the heatsink are actually trimmed to get the assembly to fit in the pipe. It's not a perfect fit, so there's some space to work with. I could take a hacksaw to the fins to get more space, but if possible I'd like to explore other options first.
What might be a good material for a supporting superstructure? It would need to be heat resistant, and in our case, non-magentic. Oh, and easy to work with and cheap since we have no proper workshop and the budget is.... thin. (The joys of academia...

) I was looking at ceramic standoffs but I haven't figured out how to cut or mount them yet.
If the temperature of the heat source is quite high, some sheet mica between the heat shield and the heat source may be needed.
There MUST be air flow between the heat shield and the PVC pipe for this to work.
Isn't mica a thermal conductor? Which is why it's used when attaching TO-220 type packages to heatsinks? Only I've come across things like
this which claims to be a thermal insulator.
Also,
this. If that's a thermal insulator that might be what we're looking for. If it's not...
I'll look into CPVC, but finding "specialist" materials locally is hard. I once asked the local office of an international plastics supplier about availability of rods in various materials, particularly acetal. I was told they had none of what we were looking for in stock, only POM.
Except that POM - polyoxymethylene - IS acetal.

A colleague has suggested silicone sheets, and another says he might have some insulation tape used for their equipment. So that's more options to look at.