Now I would like to understand how to make it work with the caps (in case I want to use a higher gain)
And what is the principle/reason that requires those resistors to be there?
In a nutshell: nothing's perfect! You need to consider what happens if, by
some means, those capacitors become charged. The resistors are there to "leak" away that unwanted charge - in this case to your 2.5V reference level. The LM324 is a bi-polar op-amp, its input transistors require a "bias" current - 20nA according to the data sheet. This is a very small current but you can't ignore it - it needs to be considered when you design the circuit.
Dave did a video a while back - it was primarily about offset errors, but it also explained why and where they arose. At the risk of being flamed to a crisp I'm going to suggest "The Art of Electronics" as a readable introduction to the basics (yes I know it lacks the rigorous mathematics that the pedants require
).
If you feel you must reduce the offsets consider where/which stages of the amplifier are going to contribute the most error and decouple them. Another trick that you might consider is that it is very easy to subtract offsets in software.