A real op-amp has finite (although very high) gain, and non-zero phase shift at high frequencies. A differentiator is a high-pass filter: it does not attenuate by itself at high frequencies. So it can happen that a signal felt at the inverting input is amplified by the 795 with the right phase shift that, when passing through the RC (100k/10nF) feedback loop, is fed back with enough phase shift to become positive feedbacked at that specific frequency, without being very attenuated. If the gain at that frequency is high enough, the circuit will oscillate. If the gain is not high enough, it will decay gradually after an impulse. In your case, when the circuit starts, the positive feedback mode is excited, and then it decays after a while: the gain at that frequency is close to one but lower.
In more specific terms, a very simplified transfer function for an inverting op-amp is \$-\frac{A}{s}\$, where A is the transition angular frequency of your op amp (this is called the "integrator model"). Your feedback loop introduces another pole, \$\frac{1}{1 + sRC}\$. Both together give a closed loop gain \$\frac{-A(1+sRC)}{s^2RC + s + A}\$. So you have resonance, more or less (solve the quadratic equation), at \$f = \frac{1}{2\pi}\sqrt{\frac{A}{RC}}\$. Looking at the datasheet of the 795, A=6.2832*1.6MHz = 10000000, and your RC=0.001. Substituting, f=16kHz. You should expect trouble about that frequency. Counting peaks, in your case the parasitic seems to be in the range of 14-15kHz or so, more or less in the ballpark of the computed range.
An expert will correct me if I'm wrong, since I'm not talking from extended experience, but I think this is your trouble.