The idea of an op-amp is that you treat it in the limit of infinite gain. If the output is finite (i.e., in range), the input difference must be zero. This part works whether you have the + / - terminals the right way around or not. Both versions are in 'equilibrium' -- the input to the amplifier is zero, so it doesn't change the output.
However, if the input terminals are reversed, you have an unstable equilibrium. Like a pencil standing on its point, any tiny fluctuation causes the error to continuously increase, and the device will 'hit the rail' (or oscillate). In your example circuit, if you flipped the + and - inputs, a slight increase to the signal input would increase the voltage on the positive input terminal. That would cause the output voltage to rise. This causes the amplifiers + terminal voltage to rise further, leading to ... and then the output is stuck outputing the maximum voltage possible. Likewise, if the fluctuation is negative, the output will end up at the most negative voltage possible.
So, regardless of whether your op-amp circuit is inverting or non-inverting with respect to the input signal, the feedback signal always has to go to the negative terminal.
Positive feedback is usually undesirable. At DC, it causes amplifiers to immediately slam into one rail or the other. If the feedback only happens at high frequency, perhaps due to unwanted capacitive coupling, it causes the amplifier to oscillate. So positive feedback is used when you want something to switch very fast and cleanly (the schmitt trigger) or when you want an oscillator -- the wien bridge oscillator uses a combination of positive and negative feedback to make an oscillator.