It could have been wax impregnated after winding for extra environmental protection, which with the heat of the relay being shorted out ( probably the coil was operated with a too high voltage or it was duty cycle limited and operated at 100% for an extended period) reacted with the insulation to make this grease like substance.
If not impregnated then it was the result of the wire enamel being cooked without enough oxygen to burn it, making an organic compound that looks like foam.