I can't find any "4 Ohm at room temperature" from Vishay Dale on Digi-Key. In general, 4 Ohms seems a bit on the low side, typically the values are higher. E.g. 100 kOhms at room temperature is typical for 3D-printer thermistors.
When measuring the resistance with a DMM, you indeed get a current flowing through the thermistor. However, it's typically below 1 mA, which means the thermistor will barely heat up (also depends on thermal mass of thermistor), so the change will likely be negligible compared to actual temperature perturbation.