The AD9851 takes 18 clock cycles to change frequency, so I don't think that's every going to limit your system.
The noise floor rising is while your sweeping is going to be largely due to the discontinuous nature of the sweep (something that might be a showstopper using the DDS). If you really need low noise, then building an analog VCO maybe the simplest solution.