Capacitors can be damaged if too much voltage is applied.
Can a capacitor get damaged if too little voltage is applied?
Yes.

There is a type of Li capacitors, a hybrid between a supercapacitor and a Li battery.
https://www.digikey.com/en/blog/lithium-ion-capacitors-can-help-you-provide-high-quality-powerThey come charged, and become permanently damaged if discharged less than 2.5V or so, usually they should be kept charged at about 3.7V or so. Often they come with a small BMS (battery management/protection), just like a rechargeable Li-Ion and alike use to have.
Found one a couple of years ago in a toy RC car, desoldered and charged it, and it was behaving about the same as a small Li-Ion battery. Then soldered it back and forgot about the RC toy car in the scrapbox.
Now the "super Li capacitor" was measuring only a few mV, tried to charge it but it only heats and barely goes above 0.3V-0.6V open circuit after charging. I've desoldered the battery protection PCB and pumped more voltage/current, but it only heats and started to bulge. I guess the damage get from self discharging is fatal for Li capacitors.

Sometimes called capacitor battery or alike, usually with an associated number, i.e. 13250, like the 16850 Li-Ion cells (16mm diameter 450mm height).
To me it was a surprise to learn that Li capacitors have both a maximum voltage and a
minimum voltage, and that they will be permanently damaged not only by overvoltage, but by undervoltage, too.
Li-ion capacitors include a cathode of electric double layer capacitors and an anode of Li-ion batteries. They feature longer operation, higher power output, and higher safety than Li-ion batteries, although the capacity is smaller. Besides, compared with electric double layer capacitors, Li-ion capacitors have larger capacity and higher output voltage.
However, a Li-ion capacitor needs the same protection as Li-ion batteries, for it also overcharges or overdischarges.
You can use Li-ion battery protection ICs for Li-ion capacitors as well, for the voltage level to protect is the same as Li-ion batteries.
Source:
https://www.nisshinbo-microdevices.co.jp/en/faq/091.html