What kind of noise? It could be vibration effecting the contacts in the reed switch, but it could also be noise induced in the switch wiring from the powered up motor - if it's picked up noise, getting rid of it could be as simple as some shielding or using a different cable to go out to the switch.
Otherwise, yeah, debouncing is a good approach. If you can find that the noise has a pretty short period (maybe, 200ms or less?), you could put on an RC filter with a low cutoff frequency and run it through a schmitt trigger or a comparator and probably get good results, at least as long as the charge up time for the first impulse into the filter isn't too long as to be useful for the thing monitoring the relay.