A further observation on this.
In my previous example I used a 1000 ohm resistor. This gives the current source an impedance of 1000 ohms, which is quite high but still only an approximation of the ideal infinite impedance.
We could make a better approximation if we used a higher voltage like 1 megavolt in series with a 1 megohm resistance. Now the source impedance is 1 megohm which is getting closer to the ideal.
It occurred to me that a real example of a current source is available to play with. It is a Van de Graaff generator. Get yourself one of these and do some experiments.
Firstly discharge the dome and then measure the resistance between the dome and the ground terminal. You should hopefully find no current path and any typical resistance meter will show open circuit. So we have the first test of a current source satisfied. Infinite source impedance.
Next connect a wire from the dome, pass it through a microammeter, and then to ground. Turn on the generator and see what current flows through the meter. This is the nominal output of the current source.
Now put some conductive objects in the current path, such as resistors, small batteries, your hand, etc. (Make sure the generator is stopped and discharged when you do this.) In each case, turn the generator back on and look at the current on the microammeter. You should find that no matter what conductive item you put in the current path the current stays the same.
If you are curious, also measure the voltage difference across each conductive item when the current is flowing. Then you can see exactly what happens with batteries.