Here are some initial thoughts, that came to my mind after looking over your design briefly:
Overall PCB-Artwork
1) You seem to have a solid 5V plane on the bottom and a solid ground plane on top. That makes it hard to get a good (good = solid, uncut) ground plane at all. Better have ground on both sides and use vias to "stitch" the planes together. Having different potentials on outer planes might also be dangerous if brought close to the mounting holes, as a screw could easily short the rails.
2) I would add more bypass capacitors to the digital ICs. 100n is normally a good starting point.
USB
1) USB tracks seem not to be layed out as differential pairs and there seems no impedance controlling. However, controlling impedance on a two layer board is difficult, why I would recommend four layers for any USB design. You should also have some distance between USB DM/DP pairs to adjacent tracks and adjacent ground planes.
2) I would strongly suggest adding a fuse at least at the USB input side.
3) Think about adding some ESD-Protection on the USB-lines and on the 5V power rails at the connectors.
4) The crystal oscillator seems to lack the typical load capacitors (depending on the crystal, typical values range somewhere from 12p-33p). I've not come across such a capacitor-less design before so check with the datasheet if that is really supposed to be like this!
Audio
1) I would suggest to have a separate ground plane for the audio amp - but the experts here will certainly comment on this in more detail. The chip has different ground pins for reasons.
2) You might add some EMI filters on the output according to the datasheet's suggestion (
https://www.diodes.com/assets/Datasheets/PAM8403.pdf Page 9)
3) Also, you might add a 1000uF cap on the supply is suggested on the same page in the datasheet. However, be careful with the inrush-current - that 1000uF will act like a dead short at first, before it is charged. I'm not sure if your upstream USB-Port likes this. Again, wait for the experts here to clarify.
4) Be careful to decouple the amplifier supply from the USB 5V lines. As your chip is a class D amp, it might create considerable noise on the supply lines if not filtered sufficiently.
Schematic:
Quite hard to read, when signals are drawn straight through components. Also the schematics in the picture is clearly not corresponding to the board as all connectors seem missing.
So yeah - even if that seems like a simple project, a lot of details might have to be considered ...