Who is "we", and what's the intended purpose? Are you designing a product or is it as an end-user?
If you're designing a product, want to use a power supply for it (that you'd buy as a CE-marked end-product), and this power supply would be an integral part of your product, the CE mark of the power supply, which would then just be a component of your product, doesn't really matter. The CE mark of a product is not a simple addition of already CE-marked parts. So if this is your question, you're probably asking it wrong.
Whatever you do, you'd have to CE-mark your product as a whole. Power supply included. The fact that you'd be using it out of spec is a clear indicator, IMO, that you'd be using it as a component of your product and not just as an off-the-shelf accessory, so again its own CE mark is not the question. Its compliance to some standards (UL for instance) would certainly help in your CE mark process, but if you're using it out of spec, obviously you can't claim the standards it's compliant with. So not only will you have to prove your whole product is compliant, including the power supply and all testing concerned, but you would also lose any warranty or ability to get back to the manufacturer of this PS in case of any fault (even if not related to your using it out of spec...)
Overall, this all sounds like a very bad idea, because you'd take full responsibility for the PS and would have to have it re-tested with your own specs, which could be costly, but also you'd probably never be able to get complete compliance without having the technical documentation, include the schematics...
If, OTOH, you're asking this as an end-user, then this is a different question and yes that would invalidate any claimed compliance, thus ultimately the CE mark, and also any warranty. Possibly if that ever caused a fire, you could also get into nasty troubles with your insurance company.