VCCB is 3V to 5.5V while VCCA is 2.3V to 3.6V.
So in your case, you must use B as the "input", 5V side, and power VCCB with 5V, and use A as the output, 3.3V side, and power VCCA with 3.3V.
As you did, if you power VCCB with 3.3V, you're going to potentially damage the chip as the input voltage on the B side is spec'ed at VCCB max. It's in the datasheet.
For a similar use case as yours, I've used the SN74LVC16245A instead, which requires only one voltage rail and is 5V-tolerant. So you'd power its VDD with 3.3V and it will tolerate 5V inputs with no problem. (Of course, it would not translate from 3.3V to 5V, but that's not what you need here as far as I can tell.) I recommend using it, especially if you don't have a 5V rail available on your board.