What load are you putting on the output? The datasheet suggests that proper regulation requires a minimum load current of about 1.25mA (basically it has to dump the current needed to operate the internals of the regulator - it does this through the output terminal.) In your circuit, the only load is the 10k feedback R1 - this is barely adequate to sink working current of the regulator. Try it with a load of at least 10mA.
Edit: Sorry, ignore that. I'd managed to interchange R1 and R2. Although, you might still have problems if R2 is too high in value - consider what happens to the LM317's operating current.