The function of a watch is to count exact, whole seconds, for as long as possible using as little power as possible. The frequency chosen should be one which can be divided down to 1Hz by a chain of D-type latches, which are simple to construct and require very little power to operate.
The chosen frequency should be as low as possible, because the slower it oscillates the less power is required. 32k is a practical lower limit on what can be inexpensively manufactured.