Thanks for the reply.
You were right. The PWM arduino code takes 0-1023, and the value used was a byte. As soon as you mentioned that, a lot of these symptoms made more sense.
The problem I think I have with a current limiting resistor is that if I have the LEDs strung up to around 12V, and the power supply is using 16V when there is no load, but down to who knows what with some load, how would I know what size resistor to use to limit the current to an acceptable range? If the voltage drop across the resistor is 3V, then I would want a 6 Ohm resistor. If it's 0.5V, then I'd want a 1Ohm resistor. The power disipated by the 1 Ohm is about 1/4W, which are the resistors I have, but if it's a 6Ohm, 3V, then I would need a 1.5W resistor, at least.
I did end up wiring it with the NPN controlling the NFET, and I ended up getting it to work, with 200mA measured for either 3 or 2 LEDs, which is good. But when I connected the 4th, the voltage wasn't enough, and it went down to about 20mA. That should have been higher, and I did tests with an open collector input to the circuit, and I was geting closer to 500mA, which is because of the PWM being set to 255, I'm sure.
I've also destroyed one of the NFETs, when I bent the pin trying to disconnect it from the PCB. But I think the original idea would have worked. The next thing I was going to try was to bypass the HC245 so I could control the pin as an open drain/open collector, but I bet you're right, I bet it's just not completely opening up the PWM.
Right now, I'm going to also test out driving it with an LM317, which I also happen to have one of. I'm hoping I can light at least 4 with one output. 5 would be better. Thanks for the tip.