Indeed, ideal CC supplies in this regard are rare; often custom designed in where pwm-by-shorting is needed.
Simply because most CC supplies have an output capacitor.
My point was, doing the fundamentally wrong and insane thing - equivalent to dead-shorting a CV supply such as a battery! - i.e., opening the LED chain, has no chances of working, not in theory nor in practice. It will just trig the protection, either latching or CV mode (or blow up a really badly designed CC supply if such exists).
Of course, doing the "ideally correct" thing won't work either, but wanted to mention it for reference.
So you are stuck with either using a supply with a PWM control input, or doing some unoptimal kludge (such as partial shunting with a power resistor).