Try running your converter at 15Vin and 12V out. That will cut the core loss way down (8% of whatever you're getting at 42Vin). This should buy you some time to actually scope out the "SW" node, output voltage, etc. If you can get that scenario to work, then you can start bumping up the input voltage and you'll see what happens. Keep records of input and output voltage and current and you'll end up with a good idea of how the overall converter losses change based on those parameters.
Good idea. Note that the XAL parts are probably low loss as powder go, and the part in question here may be still higher loss. Combined with the small size, that pretty well explains the rapid temperature rise.
I think you're way more likely to have success if you reduce your switching frequency to 150-300kHz. You mentioned TI's Webbench designer, if you look at all their suggested designs for 42Vin and 12Vout they are mostly in that range. Core and MOSFET switching losses will be way more manageable. You will need to increase the inductance, get the 4.7µH and 10µH versions of that inductor and give them a try. You won't be able to get 20A out of them, but it should let you at least get this design to do something and will help you learn as much as possible for the next revision.
Higher inductance seems to be the top priority, and lower frequency can help as well. Of course the converter will get much larger in overall footprint, but if the pressure is to have something that works at all, this seems like a useful direction, and then it can be optimized further as time and budget allows.
The controller I guess is voltage mode (yuuuck!), and the application section pretty clearly recommends "30-40%" current ripple, but declines to explain why. If it were a peak-current-mode controller with slope compensation factor of 2 to 3, this would be a lower stable limit, but voltage-mode can simply be made arbitrarily low and it's fine, given the compensation is adjusted accordingly.
Even without adjusting Fsw, increasing L and adjusting compensation should help greatly.
A moral of the story: design tools are just that, tools to get started, or to get somewhere or do something, given that you know what you're doing.
Nothing can replace an engineer. Not yet anyway; maybe in a couple decades, but not right now. Tools will gladly give you ludicrous or nonsensical results, or results that are just a starting point based purely on limited parameters such as component values and ratings, and then after that, it's up to you to select components that can actually do that without melting themselves off the board, or to meet space or cost or other limitations.
I'm afraid there's no magic bullet that will save this design as is. You might be able to find a much taller inductor that will get you closer to the design goals, but my guess is that part will need to grow significantly.
You can also look into winding your own toroidal core inductors, that could be good for prototyping and figuring out what is required to reach your design goals and select an inductor for production/end use.
Nice thing about low ripple fraction, powdered iron toroids become practical, even quite high loss materials like #52. Though I wouldn't choose that material for Fsw over 100kHz, really, but other materials will do just fine.
Litz I think becomes useful somewhere north of 40% ripple; it depends what balance of losses you get (or need) in the thing, but it's definitely of use when high efficiency is required, and BCM or DCM is used. Around there or below, a few strands, or round wire of just somewhat larger than calculated DCR size, is adequate for the job. You never see litz on powdered iron of course, for obvious reasons.*
*Well, commercially speaking, lol. I did once make this,
https://www.seventransistorlabs.com/Images/Magamp_PSU2.jpga boost converter using a #2 powdered iron core (low mu, low loss) I had on hand, plus ~10AWG equiv. litz, which I also had on hand. Well, even running in DCM, it simply doesn't get hot. It's also running all of like 5A DC. Ridiculously overbuilt, as spare-parts solutions sometimes are, heh.
Tim