The M328 will withstand 3 passes through a normal leaded oven, I've run ~1000 PCB's with M328 with two passes , and then reworked several of those, never a problem.
I use a $20 Kmart toaster oven for prototyping at home, and the only time I've ever burnt M328 chips was when the big oven broke down , and we were pushing panels through the toaster oven, it was outside in the fresh air, and we forgot about two panels, so they were about 20mins at 220C, the PCB's were visibly darker, and the overlay was badly yellowed, and the solder was all grey, even then 5 of the M328's still worked and only 3 died, (we swapped out all 8 to be on the safe side, all the other components, including a blue LED were OK.
It's possible your DIY oven has too much infra-red heat, this causes black components to get hotter than they should.
For the toaster oven the following method is used. (This may be helpful to you)
(a) if there are no visible pads near the glass door, put some dobs of solder paste on the two corners in the front
(b) place the boards on the rack (with flywire underneath) , insert a thermocouple into any available mounting hole
(c) position the PCB's with about a 1" gap from the door (it's a bit cooler near the door)
(d) ensure you have a baffle under the PCB (i.e. the drip tray that comes with the oven)
(e) set oven timer to 5mins (in case you forget!), set temperature knob to "200C", turn on BOTTOM element only, this will give convection heating only, and will slowly warm up the oven and it's contents
(f) wait a max of two minutes in the 150-160C zone for preheat (this oven only does about 170C on the 200C setting, you may have to tweak the temperature knob.
(g) wind the temperature to maximum, turn BOTH elements on , (now you have IR heating from top, and convection from below)
(h) now be ready to turn off power and open door when
no reflow (you have missed it?) and temp goes past 210C
you see reflow, count 10 secs, and if now over 205C
you see reflow, count 10 secs, and if under 205C, wait for 205C or 30secs maximum.
(i) pull PCB's from oven at 160C, and place on large aluminium (or steel) sheets to cool.
Using the above method , I get about 0% to 2% of joints that are greyish (not quite reflowed) that need touch up, and that's about right for prototyping, as you may have to get in there with the hot air gun on several occasions later.
--------other things----
You may have a problem with solder balls, these can get stuck under the chip and give odd symptoms. Without a stencil, you invariably put too much paste on, try putting the M328 on the PCB first, then just run the thinnest line on the pcb all the way around the chip so you are just touching all pins; if you have some , place a drop of liquid flux on top of the m328, it should run out to all the legs.
PCB's are sometimes a bit "greasy" , try wetting a paper towel with liquid flux, and give it a good rub all over, this will give the copper pads that little extra tack, so the solder paste is better behaved when you dispense it.
Prototyping kit components (and probably many hobbyists parts) don't have good storage conditions, moisture pickup can make for problems like "head in pillow" and pop-corning , you need to keep your parts bagged, warm and dry.
Hope that's of some help.
Bob.