Yeah exactly, you can't harden it again, it's not a hardening alloy -- you could perhaps do that with 6061 or what have you, where you could anneal, quench from annealing temp, then age in the 200C ballpark for some hours (exact time/temp depends on alloy and condition) for a, T5 I think temper. But 1xxx, 3003, etc. don't have the alloying elements that allow this kind of hardening cycle to work -- your only option is to hammer/roll/etc. until work hardening gets it there. Which presumably, your plate already is, so it's as good as it's going to get.
That it doesn't take a set from being clamped flat, is encouraging -- it's not dead soft. Having strength, means it has to be bent past flat, to get into the plastic deformation range, before it will take a set. It's... not so useful of a property to you right now,
but it's also a necessary property in general.
Which is kinda where hammering comes in, because hitting it tends to over-bend it (material curves up from the anvil). So, hit it a few times against the curve, and at some point it should come flat. Start with light taps, and get harder until it's taking a set. Do it evenly over the curve, get it flatter; turn it over, correct the parts you over-corrected, etc. Also, use a smooth clean hammer, don't want to mar the surface too much. Can use plastic, cardboard, leather, etc. to soften and spread out the blow too. Soft-blow hammer, rubber mallet, etc.
Also, the fact that it's work hardened, means there's internal stress in it, too -- this is probably not what caused the bend in the first place, not from just drilling anyway; maybe you dropped it, or cranked down on it too hard while drilling, or something... well, whatever, it is how it is now. But this internal property is important when, for example, milling into the surface: relieving some of the stress from one side (by literally removing the material there) causes the other side to take over, and it warps towards or away from the cut. Again, likely not what's happened here, but important to know when doing stuff like that.
Tim