Well here's an example video by Afrotechmods about regulators and he talks about how you can pull them off old circuit boards.
Basically if you're new to it, you'll want to get a soldering iron and a desoldering pump so you can melt the solder and suck it up quickly. Braided desoldering wick also works.
For identifying parts it's something you learn over time but googling the part numbers helps. Also Octopart.com is good for identifying things.
Here's an example of some kinds of capacitors:
There are a few other things that look like capacitors like:
Thermistors which are heat sensing resistors.
Metal Oxide Varistors (MOVs) these are used in power supplies to protect the circuit from too much electricity.
Transistors (Binary Junction and MOSFET):
Transistors can also look like temperature sensors and voltage regulators. So best to google the part number.
Resistors: Most have colored stripes on them. That can help calculate the value.
Diodes: (Usually have just 2 pins. Most will be black with white or grey stripe)
LEDs: These are easy
Integrated Circuits: (You'll have to google the part numbers)
Voltage Regulators: Many begin with LM78 or 78 followed by the voltage (not always)