It could also be the ground connection. Your oscilloscope is grounded by mains plug, your arduino could be in fact grounded too by its power source. There could be some cap from the output to the mains ground or just some parasitic capacity to the live or neutral. From that picture, I would say that you are getting 50Hz from somewhere. It could be whole bunch of things from bad connection (f.e. broken wire in probe), some loop that is going from the place of measurement back to osc. by mains, etc. Do you use separate power source, or 5V USB plug to the computer for powering that board up?