Of course. The crystal oven has to thermally stabilise before the frequency will settle. If its got a status output, monitor it. Otherwise look at the supply current - it should either settle to a steady state or cycle with a regular period and duty cycle once the oven is up to temperature. Once its reached steady state its going to need some time for all parts in the oven to reach thermal equilibrium, and it needs to be protected against drafts and changes in ambient temperature. Suicidaleggroll has suggested the timescales involved. If it isn't then stable and within its specified tolerance when checked with a calibrated frequency counter of at least ten times greater accuracy than the OXCO tolerance, its got problems.
However it shouldn't be grossly off frequency even shortly after startup - if you are seeing more than 0.1% error its certain that either its broken or your measurement setup is defective. Most cheaper DMMs don't have high accuracy frequency ranges. Even a good bench DMM is likely to have vastly inferior frequency accuracy than a comparably priced dedicated frequency counter so check your DMM's specs and try it on another stable frequency source in the KHz range