Observing how ganged push-button switches work, when you press a button, it resets all the other switches and then latches. So let's try to mimic this behavior electronically.

When you press a button, it will send a rising edge pulse to all the FFs. J and K' will be 0 for the buttons that are not pressed forcing their respective FFs to reset, at the same time J and K' will be 1 for the pressed button.
As with mechanical switches, if you press the same button again it'll remain "pressed". Nothing will happen. Likewise if you press two or more buttons at the same time they have a chance to get simultaneously latched. Pressing one of them again will release the others.
I haven't built nor simulated this circuit so I don't know if things like timing will affect its operation. It's just an idea. Good luck.