Sure, here's the website
And here's a pic of just the positive connected from that website.
Yeah, unfortunately the person who posted that picture is an idiot. There are many good resources on the web, but unfortunately many errors and much bad information too. In the picture shown he has the circuit sitting as an island with only the positive meter probe connected. It is working by chance due to capacitive coupling between circuit and probe, but as you found if you touch the circuit the added capacitance of your body can mess up the reading. He needs to have both probes connected in the picture to ensure a positive signal.
I guess the negative could be connected to some ground not seen in the picture that has nothing to do with the battery. If this is possible, where can I read about how that works? Having read the basics of electricity so far in my books, they don't seem to cover the whole negative vs external-ground relationship.
"Ground" is a human invention that seems to be very common in American circuit diagrams. If you forget the word ground and just think instead of a common return path for currents in the circuit you will be doing fine. The essential elements are that there must be a path, and it must return to the original source of power.
There are also protective earth systems (the third wire in mains plugs and sockets), which also called "ground", but this ground is there for a different reason and is not normally supposed to be carrying any current. The over frequent use of the word ground for different purposes and the use of liberally sprinkled ground symbols all over schematics is just done by electrical engineers to confuse people.
So to summarize, when you see a ground symbol on a schematic diagram, just think "there must be a circuit, and this marks the return path for the current in the circuit because the designer was too lazy to draw a line back to the origin".