I have a small step down transformer to step it down from 120V to 12V. The waveform looks the same as when I connected it directly. This time I use the digital scope so I can make some measurement. Compared to a sine wave with the same peak value it has the same RMS value as well.
Yes, that's exactly what you should expect to happen.
In my experience small mains transformers have a fairly flat frequency response up to a few tens of kHz, then there's a peak at nearly 100kHz. They can even be used for audio, if the voltage is reduced to account for the minimum frequency being 20Hz, rather than 50Hz. It will do nothing to smooth your modified sine wave into anything remotely resembling a proper sine wave.