There are different kinds of hall switches.
Some of them output an analog voltage depending on the strength of the magnetic field. This is the only kind that makes sense with a comparator.
Hall sensors often are also unipolar, they only generate generate a usable output with one polarity of the magnetic field, an they also have a limited detection range. If you use Nibidibibibium magnets, they may be too strong and seem to generate only an on/off because the linear range is small.
You can try soldering a wire directly to the output pin of the sensor, and measure the output voltage that goes to the comparator. This gives you a much better understanding than just seeing the output of the comparator turn on or off.
If you want to experiment on this level, I also recommend to buy a few loose Hall sensors, they are quite cheap.
Fans in PC's usually also have Hall sensors in them, but they are the switching kind with built in comparator.