I have and use both.
1.Flexibility: PK2 can tolerate much more capacitance on the ISCP lines, which means
a. you can use a much longer ICSP cable
b. you can tolerate more types of circuitry on the ICSP pins, if they are shared.
2.Speed/complexity: PK2 software is simple and opens up close to instantly on my computer. And the software is more streamlined. The pk3 takes 3x as much clicking, even after the software loads and the device connects, which on my computer the pk3 software and hardware takes quite a bit of time to load and connect.
3.Reliability: PK2 is reliable and simple enough I have one built into my bench, pretty hard to access, glued underneath my bench somewhere. It's just plugged in at all times, 24/7. (And due to the ICSP drive and capacitance tolerance, the ICSP cable can be like 2 or 3 feet long). In 5 years it has only lost connection once. I can't even remember what plug to unplug/replug to cycle the power, so I just rebooted my computer. The PK3 is like a bad router that will have to be restarted on occasion. It is buggier. Even in P2G mode, I expect if I flash 100 chips, I will have to reboot the PK3 a few times. If I have to reboot the PK2 in P2G mode 1 in 1000, it's a shock. And the PK2 boots up in P2G mode in about half a second. The PK3 takes 5 or 6 seconds to reboot after a power cycle. Ironically, the type of issue the OP has posted about, this is what I expect with PK3, not PK2. There are many times, for instance, where PK3 will fail to update a chip. And it will also corrupt the previous flash, in the process. But hook it to the computer, and it works fine. I've even had it once verify a protected flash, but it left the code data unprotected.
PK3 is almost better compared to ICD3. The only thing it has in common with PK2 is name, form factor, and P2G. (And standalone software, if you want to include that failed (IMO) experiment). In terms of hardware, software, integration, support, debugger, it is a cheaper ICD3 and it compares very well, considering the cost and form factor. It is almost as fast as the ICD3 in debugging (although I don't use debugger but quite rarely). And compared to ICD3, the boot speed, the bugginess, and software interface is essentially the same, once you finally get the PK3 working (which can take a bit of initial fiddling). To be honest, I'm not even sure what features/devices the ICD has which the PK3 does not. I have all 3 of these programmers, and I have not yet had an occasion to pull out the ICD3 since getting a PK3.
If PK3 works fast and reliably on your computer system and you use MPLAB, and you do only dev and one-offs, then there's no reason to wonder. Just use the PK3. If you have any issues and you use devices supported by PK2, particularly in any sort of support role in a production/high quantity type environment, the PK2 has many advantages. And for $20 or $30.00, there is no reason not to have both. The question is... do you really need an ICD3, anymore?
If they had called it the "ICD-Lite", or something like that, and marketed is as an entry level ICD3 with full support and cross platform software, the response would perhaps have been overwhelmingly positive. Another part of the initial backlash was likely due to the buy-back program. Everyone who owned a PICKit2 received an offer for 50% off a PK3 if you send it back. They know you bought it, otherwise they wouldn't be sending you the offer. They could have just given you a discount on the PK3. (What are they going to do with used PICKit2's?) When you take this into consideration, along with the decision to ride on the name and form factor of the PK2, this sets up the PK3 for fail. It better do everything the older one does. And it better work out of the box... at least after following some official instructions. And it better be an nicely thought out interface. I bet you no one cared about the improved overcurrent protection and the multiple warnings added to the software interface. If anything, that part moved in the wrong direction. If the device is dangerous/expensive, that is up to me to make sure I don't brick it when I attempt to ICSP. I don't need popups (which, by the time I actually could have used a warning, I'll probably automatically click the wrong thing by habit, anyway). I have a computer to do things for me. Not to waste my time and attention to make me hit dialog buttons like a trained pigeon.
But if they hadn't wrapped "PICKit" all over this programmer, it might not have sold as much.