Thanks everyone for the many replies, much appreciated !
Don't have too much time to play with this tonight, will do this week-end maybe.
But the "mystery" is solved now, it was a measurement issue, not a defect in the DUT.
Sur enough, I didn't expect a nice clean square wave when I hastily plugged that oscillator in the breadboard, and used the long/standard ground lead, I obviously expected significant ringing and noise superimposed, but I expected to at least see consistent voltage level, and something vaguely resembling a square... I was not ready at all for a pure sine wave with silly voltage levels, both going negative and over the supply voltage...
Lesson learned.
Sure enough, I expeted the higher harmonics to be attenuated and distort the original square, but but the front-end of a scope being a first order low pass filter, the rate of decay is "only" 3dB per decade, so the most important harmonics, 3rd and 5th, would at least partially go through... so I naively expected to get on my 100MHz scope, something like I got on my 500MHz.
But well, the whole point of me shelling some cash on the TDS 544A was precisely to get a wide BW when needed.... glad I did not waste my money then, was indeed worth buying it !
Would love to have one of the later incarnation of that TDS 500 series, or 700 serie, the 'D' series, with their 1 GHz BW and 4GS/s sampling, but let's face it... it's not in my budget. A 784D is rare to come by in Europe, and they sell anywhere for 1,500 up to 4,000, not even tested nor repaired, just sold "as is" ! and then I would need 1GHz probes to go with it.... not going to happen in my life time ! I was already lucky enough to find that 544A at a very reasonable price, fully tested and repaired, as well as a set of 4 new/old stock 500MHz Tek probes for only 100 Euros/USD the set of 4 ! Normal price I see for new generic Tek probes, is 500 Euros for ONE 500MHz probe, 2,000 the set of 4, paid only 100, can't complain !!! Clearly the seller wasn't too clued on what he was selling... my luck !

So, I will just have to make do with my 500MHz 544A for a few years to come. That's alright... now that I know what to expect when measuring fast square waves, I am prepared...
To answer a few questions from people above :
- nope, I don't have a low impedance probe... not even sure where I would go about finding one. All I see for sale are the usual x1/x10, fixed x10, current probes, and active/differential probes, which cost an arm and a leg and a kidney and a liver, all wrapped in gold mixed with some platinum !

- The 2232 scope is 1Mohm only, but the TDS 544A being much fancier, allows to switch the input to 50ohm, no problem.
- datasheet :not sure which one you got (care to share ?), picked the first I found on Google, seems to match perfectly the markings I found on my parts, which means they are all 5V unless otherwise specified... but mine don't specify anything, so 5V it is. Will try lowering the voltage to 3.3V anyway, just out of curiosity... can't do much harm I assume, at worst it will start misbehaving, at first, then stop oscillating altogether when the voltage get really too low to its taste.
- rise time : indeed, the specified rise time hardly makes for a sharp edge, given the 20ns period. The edges I get I get on the 500MHz TDS are therefore perfectly consistent with the specs... so can't complain !
- AC coupling : nope, am using DC coupling so I get the full picture so to speak. I only switch to AC when the DC component gets in the way and and keeps me from looking at the part of the signal I am really interested in.
- measuring breadboard parasitic capacitance : sadly I can't do that, would love to though. For now all I have is my multimeter, with a resolution of 10pF, not good enough. A nice RLC bench bridge is on my short list of test gear left to acquire, but can't y afford it right now. I found a great deal on an old bench RLC meter from Fluke/Phillips, a PM630X.... only 150 Euros, but I was not quick enough ! :-/ I see a few on Ebay but for 500 Euros, up to 1500 ! not the same story, and all from the USA, none local to me in France, or even Europe at large. Grrrr....;
Funny thing is, I just noticed this week, that very same meter, in the workshop at work, in some weird place, unattended.... no idea what they use it for (I am don't work in an electronics/tech related company, not even remotely !) . Maybe I could try and ask around, if they are really still using it ?! Maybe I could get it.... worth a shot any how !
Will look into modern handheld RLC meters too, in case some of them are any good compared to "proper" full blown meters like that Fluke above.
Enough talking, thanks again for your comments, will try and play a bit more with the oscillator and scope, using your good advice, just to see if I can improve the waveform better than what's pictured above on the TDS 544A using the short ground "thick wire" thingy. Might try to DIY a "spring"/ultra short type connection, as per the video W2AEW (thanks for your well put together video !), as it indeed does quite a difference, though on ly on ringing, not on the overall shape, which is more the problem I am having it seems. That said, why bother... I seem to recall seeing ready made Tektronix items for sale on Ebay, bundled in packs of 10 or so, for a reasonable amount of money. Might go search for that...