Any signal has a peak, average, RMS voltage, etc. Here you will find a very good explanation of the concepts. All the education section in that site is very good, i recommend that you read it from the beginning DC section, it's very good:
https://www.allaboutcircuits.com/textbook/alternating-current/chpt-1/measurements-ac-magnitude/Now, if you take the output of a transformer and pass it thru a full-wave rectifier, at the output you will have a wave similar to the sinewave, but the negative parts now are positive too:

Also, as the current has to travel across two diodes, you lose about 1.4v of voltage at the output of the bridge.
Now, if you add a big capacitor to the output of the diode bridge, the capacitor will charge to the peak value of the output voltage from the bridge, and almacenate chage, so that it will power your load during the "gaps" of the output wave from the bridge. And so, the output voltage will tend to be more like a DC value. But at any time that the capacitor is supplying current to the load, it's discharging, so your output voltage will get more and more ripple, and less like DC, at higher currents. Bigger capacitors discharge less under the same current.
That kind of PSU is an UNregulated one. A regulated PSU could be one built by adding a linear regulator like the LM7805 at the output of the UNregulated supply. The LM7805 will will output a 5v voltage if you supply it with a higher voltage (higher than the dropout of the LM7805). The capacitor will dischage the same than before, but as long as it doesn't discharges below the minimum voltage requirements of the 7805, the IC will continue to provide a stabilized 5v output.
As you can see, the output of the regulated PSU doesn't matches the peak or average values of the input voltage.