By definition, to measure means to compare with an etalon unit, so unless you have some other reference oscillator with very good (or at least very precisely characterized) phase noise, then there is no way to check an oscillator alone.
However, assuming you have some reference oscillator, or you need only to compare which oscillator is better in a lot, than you can use the fact that phase noise accumulates as jitter in time, so if you use two counters and look at the counts a second later or so, you could see which one is better by measure them repeatedly one against each other inside the lot of oscillators to be tested.
This is an oscilloscope example of how much time jitter accumulates at different periods of time because of the phase noise (on the upper trace there are 3 reference spikes, for 1 second, 0.5s and 0s of phase noise accumulation respectively, on the lower trace observe that pulses are very spread around their ideal location specified by the upper trace markers. How far the pulses jumps around their ideal reference location is dependent of the phase noise of the oscillator and the time delay, aka the number of counts if we use counters instead of an oscilloscope):
Not exactly what you asked, but good enough and trivially simple setup if you only need to know which one has the lowest phase noise.