The noise is there when nothing is connected, only speaker. The noise is strongest when potentiometer is set at 50% and it lowers when set to 100% or 0%.
For the input diode, I guess I can simply short it to get rid of some heat.
The output diode, however, has a strange configuration, with capacitors behind it.