Yes, it's shorted.
Are you sure that is the primary winding???? Usually the primary wires are black!!! Also the primary wires often have connections that seem to lead to the winding closest to the core. I swear it looks like you are connected to the secondary.
Yellow wires is certainly a secondary winding. Wasn't it obvious by resistance and inductance? Are you sure that transformer is rated for 230V, not 120V?
EI-57A-9
110/220V
FUSE 115°C
If the yellow wires are from the secondary winding, is it normal for it to have thinner wires?They do not look thicker. I see them crimped into thin ferrules. So wire itself should have quite small cross section. Don't look on outer thickness. Wires on primary side should have thicker insulation obviously.
Is the heating normal?Depends how hot. Small Chinese transformers often run quite hot even without any load. If it for example runs 50oC without any load, most likely this as intended.
I still don't understand how the result could be different from when I first tested it (heat up and smelled), but now it seemes that everything is fine.Either you connected blue-red or black-blue to the mains or some wires were shorted together.
If you connect a dual-voltage primary wrong, the inductance can cancel out in the two halves and the result looks like a short across the mains.Not possible in this case, since winding has a center tap, not 2 separate windings.