Driving 8Ω speakers from a single AA battery may prove difficult.
For crystal earpieces, maybe something like that? They say it works
http://www.techlib.com/electronics/crystal.html#onetransistorYou are interested in the part between the 1µF capacitors. Maybe even 100nF would suffice (it's mainly about bass response).
For 1V operation, reduce 100k to 10k and 10Meg to 100k.
I would use even smaller transistors like 2N3904 or BC547 or similar 2SC stuff. It helps to have high beta at low collector current.
One such stage will have some
20~30x gain, depending mainly on battery voltage. Two or more may be needed.

Nope, a bit under 10x gain. Oh well, add more stages. Or increase the base resistor to 220k or 330k. It will never have more than 30x gain, though.
No idea how much noise your diode generates and the amp may prove to make more by itself. Maybe you don't even need the diode
