Ok, let's take it step by step.
You have a power transformer which outputs Vac = 12v and Iac = 10A.
AC voltage means the voltage of the transformer goes goes from a positive voltage to a negative voltage and back about 60 times a second... 60 because that's the mains frequency in your country.
A bridge rectifier converts this AC voltage into DC voltage by making the voltage always be above 0v, instead of voltage going below 0v, it now goes from 0 to a positive voltage. This is called full wave rectification :
As you can see, the number of "hills" is twice the mains frequency, or 120 in your case. The peak DC voltage is equal to 1.414 x Vac - 2 x Vdiode , where Vdiode is the foward voltage of an individual diode in your bridge rectifier.
Common values for this forward voltage are 0.7v to 1.1v, but you can look in the datasheet and get a more accurate value. Here's the datasheet :
http://www.futurlec.com/Datasheet/Diodes/MB1510.pdfOn the second page, you can see a graph called "Instantaneous forward voltage" so you can just follow with your eyes that curve and you can see that at 10A, the curve intersects the 0.9v point, so that means you should expect a forward voltage of 0.9v (but let's make our lives easier and just work with 1v)
So anyway, the peak DC voltage will be 1.414 x 12v - 2 x 1v = ~ 15v
The maximum current available won't be the same as the AC current (10 A), the DC current can be estimated using the formula Idc = 0.62 x Iac = 0.62 x 10 = 6.2A - but just to be safe let's round it down to 6A
So, after the bridge rectifier, you have a DC output with a peak voltage of about 15v and capable of up to 6 A , so a total of 15v x 6A = 90 watts.
But as you can see in the picture, lots of times a second the voltage goes down all the way to 0v, so you have to add a capacitor (or several) to get charged with energy when you're at the peak voltage, and then when the output from the transformer would be close to 0v, the capacitors can provide the energy instead.
There's a simple formula you can use to figure out about how much capacitance you would need so that the output voltage would always be above a certain threshold :
Capacitance (in Farads) = Maximum Current / [ 2 x AC Frequency x ( Vdc peak - Vdc min) ] where Vdc min is the minimum you want to see in your circuit.
So for example, we determined that the transformer can output up to 6A , we know the mains frequency is 60 Hz, and let's say you want at least 12v DC all the time. We put the numbers in the formula :
C = 6A / [2 x 60 x (15v - 12v) ] = 6 / 3 x 120 = 0.016666 Farads or 16'666 uF
The capacitors would have to be rated for at least 25v, because the peak DC voltage of 15v is very close to 16v (and if the mains voltage in your country goes up at some point, the peak dc voltage may also go up over 16v for brief periods)
You can use several capacitors in parallel, for example a bunch of 3300 uF or 4700 uF 25v capacitors, in order to reach at least that amount of capacitance. With capacitors in parallel you simply add the capacitance.
Now you have a DC voltage between 12v and 15v, and a maximum current of 6A.
Since your power supply has to work at up to 55 watts, you must design something that can output a combination of voltage and current up to that amount, for example let's say maximum 10v at 5.5A
Linear regulators produce an output voltage by dissipating the difference between the input voltage and the desired voltage as heat. Linear regulators also require that the input voltage must be a bit higher than the output voltage.. that voltage difference is called dropout voltage.
For the LM338, you have the dropout voltage specified in the datasheet, and it's a bit over 2v at more than 5A at normal operating temperatures (50-80c) - see Figure at the bottom of page 4 here:
http://www.ti.com/lit/ds/symlink/lm338.pdfSo basically if you want the power supply to absolutely be able to output 10v, you would have to raise the capacitance after the bridge rectifier a bit, to have at least 12.5v or more at the input.
Keeping in mind that the difference between input voltage and output voltage is dissipated as heat, you HAVE to put the linear regulator on the heatsink.
In the best case scenario where you have 12.5v at the input and 10v at the output and you output 5.5A , you're dissipating (12.5-10) x 5.5A = 2.5*5.5 = ~ 14 watts of power.
If you want to make it adjustable, the regulator would dissipate even more power as heat as high currents. It would be safe for example to adjust the output to 5v and connect something to it that uses up to 2A (because then the regulator would dissipate (12.5v - 5v ) x 2 = 15w which is still reasonable) but you couldn't make it output 5.5A at 3v for example, there would be too much power dissipated as heat.