Benta and Siwastaja, you guys are awesome!
I followed your tips methodically, and the problem is fixed. Just for your interest (and that of others struggling with the same problem, I'll show a few before and after shots. Here is the original noise on the 5V input to the uC. This plot is centered on the 5V power, with 1V grid. You can see nearly 7V p-p!
Siwastaja, you suggested:
Minimize the loop area of the motor mosfet source node (GND) and the +20V motor supply node. Then, add a combination of a big electrolytic cap (say, 470-1000uF 30V) and smaller ceramic caps (say, a few 1 uF 35V) between those nodes, physically as close as possible.
Note that while the motor may be further away, D1 should be in the tight loop, right next to the MOSFET and the capacitors. Although I understand that schematic is not a layout picture, the schematic shows the diode in the "wrong place", suggesting a possible layout issue here.
You are correct that the schematic didn't accurately represent the layout. The diode and MOSFET, 20V and ground were in a fairly tight arrangement, and the motor about 10 inches away. I took your suggestion to add some additional capacitors as close to the MOSFET and diodes as possible. I put a giant 3300 uF capacitor and the largest ceramic I had on hand, a 0.1 uF. The result on the 5V noise is shown below. This still shows around 6V p-p, a slight improvement, though the uC still crashed.
An additional elcap (such as 47uF) at the 7805 input would be a plus.
Before embarking on this morning's debugging, I had already placed a 100 uF on the 7805 input, so the above results include this capacitor.
Benta, you suggested:
Try with 22 nF across the motor and 2 x 10 nF from each terminal to the motor casing. They should be mounted directly at the motor.
This made a dramatic improvement. The largest noise transients on the 5V supply that I could find were about 2V p-p. I let the motor run for 10 minutes at 100% and the uC never crashed.
Thank you both! I've updated the schematic to show the changes I made. This now shows the motor as separate from the diode and MOSFET.