Temperature depends on Pdiss, which depends on Rdson, which depends on temperature again, so some iteration is needed to get realistic figures.
Your calculation doesn't account for the fact that Rdson increases with temperature, but it still nicely shows that the device cannot be used under the specifications that Rja are specified. Let's say we want to keep the junction temperature at 125 celcius, with a 25 celcius ambient. This leaves some margin, and limits the Rdson increase compared to running it closer to Tjmax. At this temperature, Rdson is 1.6 times the room temperature value. Also running the device at 5 V Vgs roughly doubles Rdson compared to the value at 10 V (page 3 in the datasheet). This gives us a realistic Rdson of around 60 milliohms, and a dissipation of 60 W at 100 A. Since Rth_jc is 0.4 k/w, and we want the die to not exceed 125 celcius, the case temperature needs to be kept below (125 C - 60 W * 0.4 k/W) = 100 C. With an ambient of 25 celcius, and a heasink temperature of 100 celcius, the heatsink needs to have a thermal resistance Rth_sa of ((100 C - 25 C) / 60 W) = 1.25 k/w. This is just an example based on some arbitrary but not unrealistic numbers.
Using a MOSFET with a lower Rdson and a lower Vgs_th would lower the losses and also simplify the cooling requirements. Using a package like a TO-220 or a TO-247 also makes it easier to heatsink. Lower voltage devices will also have lower Rdson for a given device size, so if you don't need the device to withstand 80 V in the off-state, using a 40 or 60 V device instead could also help.