Think solutions, not problems.

The polarity requirement that you just introduced can be taken care of with for example:
1 - using shrouded pin header with polarity.
2 - cut off one pin, and use a female connector that has the corresponding plug filled with your favorite filler material.
Option 1) is slightly more expensive, option 2) requires some extra work for every connector you do. For typical one off's I think I would just go with numero duo. Unless I can find equally cheap shrouded connectors on ebay, which is unlikely. The advantage of the unshrouded ones is that you can get longer strips rather cheap on ebay, and then cut them to size for this or that particular project.
The tight fit requirement ... dunno.

Those connectors are not all that flimsy IMO. It might help if you give a list of the actual signals you want to connect to your front panel. Without that it becomes a bit of a guessing game.