So I got around to throwing this circuit on the scope. It's the exact circuit in the OP.
The uC is issuing a 5ms pulse, driving one output pin high and the other low.
There indeed is inductive spiking going on, but it seems to actually cause the attiny to sink/source the inductive current down to ground/vcc.
This results in ~-2V potential at the uC output pins, which seems like a bad idea.
This current appears to (dis)charge the linear regulator's output cap, resulting in ringing in the LC tank circuit seemingly formed by the coil's inductance and the 78L05 output cap. At least that's my beginner-level analysis, I am probably wrong.
Here is a shot of one of the events in question:

Either way, the circuit seems to work but I still feel uncomfortable about the inductive behavior.