If it is acting as a thrust bearing for the weight of the antenna, and the only support, the connector is not very substantial and will eventually fail (as everything does). It probably isn't going to take side loads from winds very well.
But all this is conjecture. Try it. You'll learn a couple of things. The connector will or will not be satisfactory. You'll know better what the problem is which will lead to a more satisfactory solution. If you destroy a $1 BNC connector in the process, who cares?
Pack it full of grease. Figure a way to put a dust cap over it. Protect it from direct weather (rain, etc). See what happens. Nobody experiments with things we already know.
Eric KE6US