The 10811As specified T.C. is < 2.5x10-9 over an environmental temperature range of 0..71°C. (see page 1-2 in manual of 10811A).
So your observations might give too high a T.C., but maybe can not be distinguished so well from other influences.
Also, the specification is given as a 'box method', not directly as ppm/°C.
Maybe the oven control is not ok. A monitoring of the oven power consumption over temperature may give more hints.
Its timely drift is specified < 5x10-10 /day. So your observations are inside specification.,
I suppose, that you let the counter plugged in all the time, so that the 10811A is continuously powered.
When I switch my 5335A on, I also observe a slight drift of <1x10-9 after a few hours of internal heating up.
I use the counter in a room at 21°C +/- 2°C over the whole year.
The timely drift of my 5335A is less than 1x10-9 over a year, meanwhile the 10811A is so old, that it has settled.
Again, whenever I unplug the 5335A, it takes > 48h to reach the old value within < 2X10-9.
I recommend to let the 5335A warm up for several hours, with the lid unscrewed, but closed.
By briefly shifting the lid open, trim the 10811A as close as possible to 1x10-9 deviation and observe again with closed lid, and repeat adjustment 1-2 times.
I think, my counter meanwhile displays about 2x10-9 deviation, several years after last adjustment.
That's totally fine, because that is really at the limit of what is possible with an OCXO.
You need a Rb reference to get even lower deviation, about 1/100 of that ballpark.
Frank