Thanks for everyone's help. In the end, I got this simple (component and connection wise) circuit to work by increasing the voltage.
I can't measure how stable it is, and since I didn't cut off the base lead, the transistor is collecting everything from the environment. Especially if there's a physical contact, e.g. by touching it with your finger, the oscillation frequency is increased.
While not being stable for sure, I guess this is one of the simplest oscillators component-wise. There's just one transistor, one capacitor, and one resistor. It worked with my transistor at around 13.6v circuit input voltage (minus whatever drop there might be from the current drawn). I'm sure that each transistor type will be different regarding the voltage required.
I know a 555 timer (tried one), or some other relaxation oscillator circuits with different components would work better (didn't try any, need the components), but sometimes you just want an LED to flash, and do so as simply as possible. The real challenge here would be getting a specific and stable frequency, which fortunately I don't need for the LED.
P.S. it might damage the transistor as mentioned in this thread, but I don't know by how much this is, and it's probably not something that can be simply measured (e.g. would such a LED flasher be able to run for years, would the transistor break before other components stop working properly, e.g. the capacitor or the LED itself).