Excuse my ignorance on the subject. Going from 32bit audio to 24bits should be easy by just dropping the 8 LSBs.
Optionally use them to decide if you want to round up the remaining MSBs.
That's sigma delta modulation in a nutshell, extremely simplified.
So I would start by sending every sample twice.
You just injected a ton of harmonics. Interpolation is easy, getting rid of the unwanted spectrum is the tricky part that needs tons of computation.
With Foxxz's comment, it sounds like simple resampling + a tuned low pass filter would be enough for him. This is not absolute junk, but it's a beginner's step and a properly tuned low pass filter will reject a chunk of the nasty band. Higher quality output filters would just begin to mimic existing DAC output analog reconstruction filters.
Unfortunately, even a 1.50$ 32bit stereo audio DAC from TI has a higher quality internal up-sampling when it takes in 16/24/32 bit 44.1KHz source and up-samples the source data internally driving it's dac at 384KHz. This price-to-up-sampling quality will not be matched on a 1$ FPGA, not to mention the needed bootprom, 32bit DAC and powersupply which doesn't exist for anything less than at least 10$.
This project should be taken as a FPGA learning experience and you should attempt to at least match the up-sampling quality of the bottom end TI DAC's internal one. This means full processing as blueskull has been touting and this also means an FPGA with some good strong core capabilities + a lot of work.
If you want just the up-sampling quality, there are components much cheaper and all the mathematics have already been done. If you want the learning experience, go with the FPGA and make sure you can have some PC connectivity to sniff the data & set some controls. IE, at lease make sure your FPGA board has at least and additional USB/RS232 interface.