Few ideas...
0.5-1V p2p is pretty large so its probably something simple, like a grounding issue.
Do you get noise when the signal connection is made or does the noise appear even before then when the cables ground connection is made?
Move your DSO probe ground clip to the arduino ground. (if its not there already). If your measuring the signal with reference to a ground point that is far away from the arduino (like mains earth) you will probably see lots of ground noise.
Try powering the circuit from something that's isolated. eg, if it's getting power from usb try an laptop running on battery.
Check you have the pin pullup disabled. When a pin is set as input it will only be highZ if the pullup is disabled (pretty sure its off by default but i'm not too sure about the arduino environment).
Double check you don't have some other code that's accidentally switching the port to output sometimes. As that would look like high frequency noise if it's happening in your main program loop. You could check this by putting the scope on the input pin without anything connected except a 10k resistor to arduino ground and another 10k to vcc. That will show if the pin is being accidentally switched to output or pullup enabled, (it should be half rail all the time).
Some pins have other functions, check your not trying to use one of these other functions as it might be changing the pin to output.
Also if you can post a better pic of your signal with and without this noise we might be able to tell what kind of noise it is and where its from.