I have decreased R58 down to 3.3K and now I have 23.3V which I consider very good. Also regarding to the caps, the space and the volume of the box wouldn't allow me to add higher rated caps, so I will just leave them like that for now and observe any unusual behavior until I can have another caps with higher ratings and the same volume.
BTW would this affect the max. output current, this unit is capable of 4.5A @20V, and I will be using it around 3A @ 23V?