1. No designators or net names... so I can't even call things out for critique.
2. No battery protection. If this is a Li Ion or related battery, you don't want to discharge it below 3V. (Maybe a fuse too, in case the converter or load shits the bed, or something?)
3. Low battery indicator literally makes things worse, and if you don't see it, well... yeah.
A battery management IC would be safer. (Ideally, one with integrated charge controller so you can maybe run the whole thing from, like, a 5V USB charger cord or something?)
4. The LDO probably isn't necessary. The battery is almost noiseless, so you're only filtering against whatever noise the converter produces (which shouldn't be much worse than the LDO itself), plus switching noise. The switching noise is near 1MHz, which the LDO says will be around -40dB, but it looks like a capacitor up there. You could do just as well with an LC filter of 0.5uH and another 10uF, for example (which will be cheaper and smaller; and besides, you already have the other 10uF). Note that this does nothing about possible common mode noise, which you'll have to address in the layout (keep the switcher well away from the signal path).
5. Not just the input coupling caps, but all caps in the signal path (which includes the output caps, and the coupling caps for the inverting input pins) should be low-piezo types. Which really stinks for SMT construction, but if you don't mind using tantalum, that'd do a fine job. Otherwise, SMT films suck for a number of reasons (they're still large; they're expensive, and although it's not a problem here, they do absorb flux/moisture, a problem for sensitive circuits). But if you have to use through-hole films, that's hardly the end of the world.
6. The input and output should be referred to the "virtual ground", not power supply ground. This way you only need one divider, and you don't get a power-on/off "pop". Or you could reconfigure the switcher so it makes symmetrical e.g. +/-5V rails (which isn't hard to do, using a dual winding inductor), but that's dumb. As long as you don't need common ground between things*, it's fine.
*Which is actually a good point, if you add a e.g. USB charging connector -- charging it from the same computer it's connected to would short out the negative rail, making a bipolar supply better.
7. A make-before-break switch is preferred for the gain setting. But even better: you can swap the cap-to-GND with the resistor, and switch a resistor in parallel with the existing resistor, needing only a DPST switch.
As shown, with a break-before-make (most common) switch, you'll get a pop as it goes to unlimited gain for a few milliseconds and probably makes a nasty output voltage.
8. More gain settings would be better, too. 1.5 and 2 is hardly 2dB different, not even worthwhile for headphones. One or two 10dB steps (~3x gain per step) might be helpful, to deal with widely varying sources. Note that attenuation might even be desirable, which you can't implement on the feedback resistors. (You could with an inverting configuration.) A good old fashioned volume control would be the most universal, of course.
9. The op-amp only delivers ~40mA (as measured from 15V supplies; unknown if that's supply independent), which might not be enough for very loud signals into 32 ohm headphones. A buffer/driver/power amp device might be better suited, at least if high dynamic range is desired.
I think that covers about everything, HTH
Tim