There should be a very high resistance (but some capacitance) between the secondary winding, the primary winding, the iron core, and any metal container. However, there is a maximum voltage rating for all of those insulation resistances. Vacuum-tube era “filament transformers” had explicit insulation ratings (sometimes in the kilovolts for tube rectifier circuits). In your voltage range, where the secondary insulation (except to the primary) will see < 50 V, they don’t bother to post the specification, but the UL specification for power adapters does call out voltage test values.
With a bridge rectifier and a non-CT secondary, you can choose to ground either the positive or negative node of the bridge, but not both, to obtain a negative or positive voltage, respectively.