For the the criteria I first go 'special usage or not', then 'by voltage', i.e.:
first I collect together (if needed, divided in few classes) those adapters that have a special purpose, e.g. because of custom connector, etc.
then most of generic ones are sorted in different bags by voltage (5V, 9V, 12V, 24V),
last is the box of the odd ones (e.g. unusual voltages)
Then, when one of these boxes or bags gets overpopulated, I made a selection of its content e.g. by keeping only the ones with higher power rating.