I don't know about all of CE specifically. Most EMC standards are only concerned about emissions if there's a source inside; an audio amp has no source, it's not supposed to oscillate (do check that it's stable for all load conditions, mind

) so that's easy.
Safety isn't going to work if you leave it wide open. Metal mesh cover or something like that, no worry.
Electrical safety rules are about what's touchable, and also single failure conditions. Glass I think will count as reinforced insulation, so that's fine, and a missing tube, the socket should be insulated (metal under the surface, can't reach it). You do need adequate insulation for the internal voltages, and a grounded metal enclosure is a good idea. (How to solve ground loop, is another matter.

)
I don't know what general touch safety rules there are, but that will set surface temperatures and such, and hence necessitate an enclosure. There may also be transportation rules for shock and vibe, handling a drop etc. More for mass production and merchantability than for one-off projects.
Immunity should be easy enough. Commercial EMI test levels are comparable to signal levels, and tubes don't have the aggressive exponential relationship that transistors do. A little RF filtering might still be a good idea, to keep the signal completely clean.
Surge and ESD basically don't matter. Surge is handled by the transformer and C-input rectifier. Tubes may wear due to ESD (causing internal breakdown maybe?) but rarely die of it.
Tim