What ADC are you using?
As Dan mentioned, the circuit you show is great for driving the ADC inputs, but it won't work well as a first stage after your transducer. The diff amp has a very low input impedance (only 1k ohm), so you'll want some sort of buffer in between it and the transducer itself. A simple voltage follower would probably work very well because it will present a high impedance to the transducer, and a low impedance to the diff amp. As Dan mentioned, a JFET input OP will have the best characteristics (noise, distortion) for this kind of application.
If you are biasing the transducer via R13 from the +9V supply, any noise on the +9V supply will be directly added to your signal. You may want to look at alternate biasing arrangements, or make sure that the +9 volts is very low noise.
If you want to determine if the 'noise' is coming from the diff amp circuit, disconnect the input and ground it (also disconnect R13 if possible). This will allow you to look at the noise of the diff amp circuit by itself.
The output cap (C33) is required for the ADC to work properly, so you do need it. The only way that it can create noise is if it's causing the circuit to oscillate, or if it's a ceramic cap that isn't a C0G/NP0 type. Ceramic caps with lower grade dielectrics exhibit a piezoelectric effect, so they can pick up vibrations in the environment and translate them to electrical signals.
All of the caps in the signal path (DC block, feedback, and output cap) should be of C0G/NP0 type if they are ceramic caps. In addition to exhibiting the piezoelectric effect I mention above, the lower grade dielectrics also change capacitance significantly with varying voltages, which causes distortion.
What frequency range are you looking for? This will help us understand what is required from the circuit.