I see some 300F super caps on digikey for ~$12. Thats a lot better than I thought, but still a huge cost difference compared to a lithium.I don't know what a CR2026 is, but you can get CR2032 for <15c.
From what I compared its not close to being worth it. Size is much larger, capacity is much lower, you will have to have some supporting circuitry (although not much). Of course some special applications it may be of use (high discharge/charge).
What is the power consumption of the product?
10 years is too long to worry about IMO.