You are on the right track. The load/resistance that you connect across the terminals of your voltage supply allows current to flow from one terminal to the other. This is the current drawn by the load.
So if you put a very low resistance between the terminals then you will get a very high current, from ohms law. I = V/R.
The current will never be infinate however, infact it will probably not be very high at all for two reasons.
First of all every power supply has what is called an internal impedance, this is a resistance built into the power supply, it comes from the physical way the supply is built, if there are copper wires in the supply then there is resistance in the copper, if it is a battery then there is resistance in the electrolyte, etc.
Secondly most power supplis have overload protection, this means that if too much current is drawn from them then the power supply will stop applying a voltge to the terminals. this can be done in many different ways, fuses, circuit breakers, altering PWM in switch mode power supplies.