I was originally using an ATtiny with a 74LS145to drive the rows, but switched to an Arduino in an effort to simplify everything as much as possible to find the problem, and am now powering it with 5 volts. The code I am using is here: https://gist.github.com/ElectronicsIdiot/3ef9cdecffb68426b64e3a87c9f0becc.
I don't have schematics or anything other than a rough idea of how things are connected.
The board:
The glowing LEDs are brighter towards the row that is pulsed first (bottom). If I reverse the row loop, the LEDs glow brighter at the top as the top row is now pulsed first:
The row that is pulsed first also seems to glow slightly, which I hadn't noticed before.
Thanks for all the extra information for everyone. I've looked through it and understand things better now. But can't put my finger on exactly what is wrong.
Ideally, if you have a decent Oscilloscope (I understand you don't have access to one, but are using some kind of utility program which does something like that), connect it up to an incorrectly glowing LED and see what it is being driven with. Then you can hopefully get clues as to what is going on. You could also look at the clocks and other things at the same time, to help diagnose what is wrong.
Another possible diagnostic experiment is to send all logic zeros (i.e. off) to all the rows, and all logic ones (i.e. on) to all columns, and check that all LEDs are off.
Similarly do the complete opposite, i.e. all rows on and all columns off, and again check that all the LEDs appear off.
N.B. I may have given you the wrong logic polarities. But I hope you can understand what I mean.
tl;dr
Turn on (active) all columns (but one at a time would probably do), but disable all rows.
Secondly, turn on (active) all rows (but one at a time would probably do), but disable all columns.
What is puzzling meThe LEDs would need two "faults" in order to glow when they are suppose to be off. I.e. both pins of the LED should NOT be seeing any connection to a voltage (current), so even if one side of the LED should be connected to a voltage, the other lead, should still be floating, so no current should flow and hence the LED should still NOT light up.