Never used the LTC3534. From the outline on your PCB it looks like you have used a small 5uH axial inductor and I bet it's saturating and then the chip goes into current limit when you turn on PWM. A small axial inductor won't have enough ferrite core area to store the energy, 1/2LI
2, the current will reach the right value but L is tiny because the core is saturated. If you can find an app note for the LTC3534 eval board then it will have a part number for the inductor, well at least I hope so rather than a list of phone numbers like on the data sheet. I had to look twice, a list of phone numbers what the... that's about as useful as shoes for snakes. Thanks amitchell you posted before me
bigger inductor. You could probably use a surface mount inductor mounted sideways with a pair of wires on the SMT pads. Easy mistake to make if your're not clued up on magnetics design and I've probably made the same mistakes in the dim and distant past, so don't worry.