I would assume that the via pad size should not be larger than the prescribed pad size for the connector.
The proper way to do this, which is economically impossible for hobby use, is to get an (often encrypted) full 3D simulation model for the connector and simulate it with the PCB launch as on the final stackup.
Your starting point could be to take your via and imagine it being a circular coaxial transmission line. (Coax cable) You can easily find a inner and outer conductor diameter ratio that gives you 50 Ohm impedance for the PCB dielectric you're using.
What makes this estimation not all that useful, especially for short/stubby vias, is that the impedance discontunities when transitioning from the connector to the board, or to the trace on the other side, or between layers in case you have mixed dielectrics will likely have a larger negative effect.
What you can do is play with the pad sizes, the antipad sizes and the trace launch geometry. If you can't do full 3D EM simulation on the structure you have to run on guesses and the TDR plot helps a lot with that.

This is the same via test I posted in the previous post but shown in impedance. You can see it starts with perfect 50Ohm, then I have a bit of wiggle; those are coax adapters. The first dip to 47.5Ohm is the connector launch to microstrip, then the dip at 400ps is a via transition to the other side of the board.
The magic is that this is purely done from the capture S-Parameters via an inverse FFT and some simple additional processing.
If you have a VNA with this functionality built in you can look at this plot real time and poke at the circuit to accurately locate features. Typically you take your tweezers and short the trace somewhere and see graph plummet to 0Ohm at that location. If you can only do this in post you can still take such measurements and save the s-parameters to later confirm accurate feature location.
To some extent you can also calculate. Take the velocity factor of your transmission lines then you can estimate spatial location from the travel time. The farther features are separated in space on your board the easier it will be to tell what belongs to what.
To get back to my plot, what that tells me is both my connector launch at 300ps and my via at 400ps are overly capacitive (Z0 < 50Ohm), while my termination at ~480ps is grossly off. That was supposed to be a 50Ohm terminator but it might have a soldering defect or alike.
Let's ignore the termination for now.
Time gating comes in handy to see what to fix. See the image from my last post. You can mask out sections of your time domain response and see what the frequency domain would look like in that case. (Black traces are as is, red is the hypothetical).
If you confirmed that a particular section is influencing your overall response significantly you can try to compensate in the other direction. In my case both the connector launch and the via is overly capacitive. This means there is too much ground copper too close. I can try to make it more inductive by reducing trace width, reducing via diameter, increasing via antipad diameter or increasing coplanar wavegude ground gap, etc. (closer ground -> better capacitor -> lower Z0; wider trace -> bigger capacitor -> lower Z0; etc.)
It's a lot of guesswork even if you are able to simulate it, but some amount of improvement is certainly possible by iteration.
All that being said, your plots are not bad. In many real world cases I would take these results and be happy, but yes, probably it can be improved.