Your question is related more to physics than electronics. Esentially, an oscillating system is any physical system that stores some energy E, and where that energy periodically transitions between two manifestations. In a pendulum, total energy transitions from kinetic energy (pendulum speed) and gravitational potential (pendulum height). In a mass hanging from a spring, energy transitions from elastic energy (Hooke's law) and kinetic energy. In a typical electric oscillator, energy transitions from electrical energy stored in a capacitor, and electrical energy stored in an inductor.
So a battery is an energy storing system, but it is not an oscillator, because the energy in the battery never changes its nature: electrochemical energy.
There are many types of oscillators, but the simplest, linear ones, have the property that the frequency of oscillation (between the two forms of energy storage) is independent of the total quantity of energy stored.
Imagine that you take an oscillator and measure how much of the total energy is manifesting in one of the two states: for example, you take a pendulum, and measure its height (potential energy). You will measure a wave in time, as the total energy of the system transitions from potential to kinetic and back. The same with an electrical oscillator: if you measure its electric potential (energy stored in the capacitor), you will get a wave, as the cap stores and discharges the total energy of the system. If you measure the current (energy stored in the inductor) you will also get a wave, but 90 degrees out of phase, because when the capacitor is full of energy, the inductor must be empty, and so on.
Most physical system have losses, so your oscillator will slowly lose its stored energy. As this happens, the frequency of oscillation won't change, but the top energy measured in any of the two states will decrease. So if you measure the energy of one kind in an oscillator with losses, you will measure a decaying wave of constant frequency, that is, a damped wave. The decaying rate of the energy is closely related to the oscillator's Q factor.
Is damping desirable? It depends. If you are designing a car's suspension system, you want a gentle damping. When the car hits an obstacle, the suspension system stores that bump as energy, and slowly dissipates it. If the energy decays too fast, the driver will feel the whole bump. If the energy decays too slowly, the car will vibrate long after you passed the obstacle. On the other hand, if you are designing a clock oscillator, you want very slow dissipation, since in order to keep the oscillator energized, you need to 'tackle' it periodically, and that introduces harmonics, losses, noise, and other undesirable effects.
All this can be formalized with easy and universal physical equations (a Hamiltonian for energy conservation, leading to a linear second order differential equation, with a damping/forcing term added) that you can find in almost any first course physics mechanics course.