Sure, if you want to measure absolute temperatures (since a thermocouple only measures the temperature difference), you need cold junction compensation. But you should see changes in voltage when you change the temperature. Try something like ice water and boiling water to get some significant temperature differences. With my tests I got about 300µV/7K difference, from -80µV to 220µV or so.
An analog VOM wouldn't be my first choice, not sure how sensitive thermocouples are to loading impedance, I wouldn't expect them to deliver a significant amount of current. Seeing 100µV changes on a 75mV scale may also be hard, guess that's why they only suggest using it for large temperature differences. Note that the voltage may be positive or negative.
Type J will work similarly to type K, but the voltage will change slightly more per unit of temperature change (difference is around 25% I think).