if i go with the step up solution, would the frequency matter?
If you're referring to the charge pump, the frequency should be set so that you have enough on-time each cycle to fully charge the capacitor. This will depend on the RC time constant: the R being the source impedance of the Arduino output plus any series current limit resistance you choose to use. But understand that the object in this case is to produce a higher DC voltage for that RS Buzzer. It is not a discrete piezo speaker. It is a self-contained buzzer that includes the oscillator and the piezo element. You can assume that they are using the resonant frequency of the piezo element for maximum sound output.
The links that you provided in the first message are for transducers that need to be driven with an audio frequency in order to get sound. This is more complex than just providing DC to the RS buzzer. If the RS Buzzer will do the job for you, then go for that solution.
If you need greater sound level than the RS Buzzer will provide, it will take some work. You would probably want to use a rigid piezo element as in the seekic link. But note that in the schematic, they call it a beeper. It's only a beeper if you provide it with the correct audio frequency signal, though. The rigid elements tend to have a rather sharp resonance; they indicate 3.5 kHz, and this is a common resonant frequency. You will get much greater sound pressure level at the resonant frequency, so you may want to sweep it with a signal generator to make sure you find the best frequency, if it is not specified precisely. (There are also piezo elements with 3 terminals; the 3rd terminal is for feedback, and this is used to make a self-resonating oscillator in which the piezo element itself is actually part of the tank circuit. Properly designed, this circuit will always give you the resonant frequency of the piezo for maximum sound level output. I don't know how easy it is to find one of these elements, though.)
The piezo film speaker in the other link is a different kind of device. It probably has a wider frequency response, and it may not be capable of the same sound pressure level as the rigid piezo element because the film speakers don't have the sharp resonance. I'm guessing this would not be your best choice if your goal is a high sound pressure level.