As a mostly metric animal, I think 20ths of an inch are occasionally useful, I think you may as well keep them there (at the very least, as the little minor ticks). I must point out that there are two separate sections of the ruler there marked "1/10" and "1/20", but they look identical. It's obvious that tenths are the big ticks and 1/20 are the small ticks, but it still irks me that they're labelled as if to imply that the two sections are different in some way.
I'd be very keen to keep the 32nds. So I guess I'm suggesting, drop the 1/50, and drop the 1/64. At least that's symmetrical, one off each side.
I really like the design in the 1/32 section, with the ticks of 3 different lengths -- there is some argument perhaps for just extending this along the entire ruler?