A side benefit would be to know exactly how much output I am getting.
Not really, unless you characterize the amplifier over the full range of frequencies and create a lookup table or graph so you know what its gain will be at a specific frequency. I managed to find the specs for that $10 eBay device... the amplifier gain varies by 12dB over the full operating frequency range, pretty typical for such an amplifier since it has nothing in the way of feedback or gain compensation.
Gain will also change slightly based on temperature variations and any supply voltage variations.
Also it's only rated to a max of +10dBm output power, but you would likely never use it at such a high level. I can guarantee you the harmonics and IMD will be atrocious when pushing it anywhere near that maximum output.
If it sounds too good to be true, I'm afraid it probably is. Making a wideband amplifier with very good gain flatness, low IMD, low harmonics, is no easy feat. That is why such amplifiers usually fetch a pretty high price tag.
There do exist some amplifier hybrid modules made primarily for CATV applications that have some pretty good characteristics as far as gain flatness and linearity. Sometimes when I need a little more "oomph" from my signal generator I have an amplifier I put together from an MHW591 (for 1-250MHz) and another from a MHW6185 (for 40-600MHz). Both are able to output +20dBm (100mW) with ease over their whole frequency range with minimal distortion/harmonics (about -40dBc second harmonic) and the gain flatness is to within less than +/-1.5dB. Not perfect but good enough for my needs.
I pulled mine from junked CATV and analog TV broadcast equipment (free) but you can find these and similar other modules on the internet used and NOS (new old stock) for pretty reasonable prices. They are are simple to use as providing a regulated power source with a bit of bypassing (simple choke and cap suffices) and hooking up your input and output coax.
I've attached a pic of one of the boxes I made up using this. Don't mind the input attenuation resistors, I only put those in there to protect the hybrid in case I accidentally set my signal generator to its maximum of +13dBm, which would be above the +3dBm absolute maximum rating for input power without the attenuation. Ain't pretty but it works.