Drilling is kind of rough for a couple reasons:
- I'm guessing you can't get this under a drill press, so accuracy will be limited. (All the more reason to centerpunch!)
- Once it's pretty loose, or a big enough drill takes a big enough bite... it starts spinning in the panel.
- You can still drill out a spinning rivet (handheld is easier, rock it side to side so the drill still cuts in and out a little bit as it's spinning), but it'll make a mess of the panel, and you might lose precious thickness/diameter in the process. This is worse obviously if it's off-center too.
So, not great on thin panels or whatever. Looks like it's fine here, no worries.
Remember to use star washers on the replacement (assuming you're going to bolt it instead).

Tim