Circlotron is on the right track, but rather than fumble around disconnecting the capacitor from its charging supply and transferring it to across the inductor, lets make it easy to work.
Charge the capacitor through a high value resistor so you don't have to disconnect the charging circuit. Anything up to 200nF will pretty much fully charge in under a second through a 1 Meg resistor, which will have negligible damping effect on the LC tank circuit. For your 1uF capacitor, give it five seconds. Connect a x10 scope probe to the non-grounded end of the inductor, so it doesn't load the charging circuit till you connect the inductor to the capacitor. Put a NE2 wire-ended neon bulb (without a resistor) directly across the inductor to protect the scope
* by clamping the back-EMF in case the inductor gets disconnected before the damped oscillation has died out. Assuming a positive supply,when you connect the inductor to the capacitor you'll get a sharp positive edge the scope can easily trigger on. The biggest problem will be contact bounce. The traditional way to avoid contact bounce would be to use a
mercury wetted relay, which can still be found NOS, but I wouldn't even entertain getting one unless it was a hermetically sealed miniature one.
* Caution: If your scope isn't rated for >150V max input, the NE2 neon bulb will NOT provide adequate protection if you use a x1 probe. With a x10 probe it will be OK as long as the scope is rated for >15V max input. Some cheap USB scopes and handheld scope kits have very limited input voltage ratings.N.B. High value small axial inductors have considerable DC resistance. Therefore, its likely that your LC circuit is heavily damped, possibly over-damped, hence the lack of visible ringing. Measure its DC resistance and post it, and given the inductance and capacitance, we can calculate if that is the case, from the equation for the series critical damping condition:
R2=4L/Cwhich gives fractionally over 63 ohms for your choice of a 1uF capacitor and a 1mH inductor.
So you can see what you are up against, here's a LTspice sim with 11.5 ohms inductor's DC resistance. You'll clearly see four cycles of ringing before it decays into insignificance. Change the DC resistance to 50 ohms and you'll only see a slight overshoot like you have. Change it to 63 ohms and the overshoot will vanish.