When the pot's at maximum, it's effectively bypassed - creating a short. When it's below maximum it's introducing series resistance, which equals hiss. The higher the source resistance, the higher the noise - simple. The way around this is to use an active volume control, where instead of acting as a voltage divider, the pot is in the feedback loop, varying gain with position. In this scenario, you get maximum hiss when the pot's at max. In a typical audio system that has several components, you will find that designers mix passive and active level controls, depending on application. The lower the value of the pot, the less noise when wired as a divider. But this comes at a price of higher loading for the circuit driving it. A 1K pot will have a lot less noise wired passively than 100K, but not all opamps will be happy to drive it.