Ok, quite a few things I see here that could be wrong.
- As mentioned, the magnetics. I have no idea what the cores are, but it all seems a bit too small for their size, especially it 50odd kHz.
- Working straight of the mains, not a good idea
- Add some current limiting, even if its just a latching to off state. The melted and charred croc clips suggest prolonged over current events have occurred in the past.
- Why are the diodes (the SA180 & P600M) there? The mosfet reverse diodes are fine for the operation of the LLC.
- Gate drive, check both channels to look for the proper deadtimes. An LLC needs a long enough dead time to discharge the mosfet output capacitances for the switching node to transition
from rail to rail. If this time is too short and you fail to do so you no longer achieve ZVS and the main point of the LLC goes out of the window, also you now have switching losses in your fets heating them up.
- From your scope shot it seems that you are driving the gates negative by a whole lot, I'm not sure they'll like that much. You can get nice gate drivers that are isolated and can be bootstrapped easily,
no need to mess with gate drive transformers.
- Heatsink+Fan, this suggests your fets are running hot. A properly designed LLC of 200Watts can run the fets without any heatsinking at all, so again this indicates something is awry.
- You don't have a rectifier on the secondary side, you need that for the current to behave properly in the resonant tank. The single resistor replacement trick only works for the FHA design technique, in a real circuit you need the rectifier + cap there for the converter to operate properly. See attached screenshots.
Legend of the simulation screenshots:
Red+Teal, Mosfet gate signals.
Blue: switching node voltage, notice how this voltage transitions from high to low and vice versa in the time between the two gate signals (both fets off), this ZVS and very important for an LLC converter.
Green: resonant current, note how its anything but sinusoidal when only a resistor is place on the secondary.