An even better way to view an op-amp is as an integrator:
Real ones act more as an integrator (-20dB/dec and 90 degree phase shift for [almost] all frequencies).
The result from an [improper] integral is always "plus a constant". That constant is a free variable, and has to be defined by the feedback of your circuit.
(If you've taken integral calculus, this will be rather peculiar, because feedback puts the same variable (say, a node voltage) on either side of the integral. You actually get a differential equation, which might not have a simple integral solution. Generally, these equations have solutions with exponents. Which is what you'll observe in the circuit: RC time constants decay over time as the output settles. Assuming it does settle, of course.)
Without feedback, the output voltage is undefined: it could be anywhere, as long as it's within the supply rails of course. This is why trying to use an op-amp as a regular amplifier (signal to inputs, no feedback from the output), just sucks (nevermind that the offset, gain and distortion of an open loop op-amp are pretty awful, too).
So when you normally wire up a circuit, the output is referenced to some other node that you've put in the circuit. For example, the differential amplifier circuit senses a differential input, and supplies an output relative to the ground terminal: but this could be any arbitrary voltage reference, not necessarily the circuit ground.
For a precision circuit, such as a power supply's "remote sense" connection, you can use this principle to extend where the voltage is actually sensed. You connect one wire to that ground point (not just in the net -- that is, anything touching "ground" -- but exactly at that physical location), and then when you read that voltage at some distant location (which might not share the exact same ground, for a variety of reasons like voltage drop and AC noise), the voltage difference is still exactly what it should be.
Tim