Small correction, which probably is because my average english. Bang-bang controllers are nickname for on-off-controllers, iirc one description is hysteresis controller. You are (at least by manual) using PID-controller. Why I did refer to bang-bang controller at beginning is that I falsely (which quick reading of manual titles etc.) thought that it does have such included, now it seems that I was mistaken.
I think the best thing for you is play with the controller a few days and try to google some hands-on PID tuning instructions. No need to go full academic with mathematics, since you do not have a model of your system anyway (and I think the TEC2000 is also black-box, so actual algorithms on controller is not known) and probably it will be changed many times more.
The D-parameter try to resist the change what I&P terms do, it doesn't hurt you to tweak it, but I would imagine that it is still not fully optimal at all since you are saying there is steady (but small
oscillation going on).
The a graphed setpoint step is good way to see how system behaves.
You have a good setup to familiarize yourself for tuning of PID.
Like I said try to find some good (authorative) hands-on articles about PIDs. While this is TEC-controller it is still just a PID controller in the end. After also reading a bit about the control subject return to tec2000 manual, it most probably open differently what everything do mean on it.
You have a good learning setup there, also play a bit the box and sensor itself to see how it might effect to controller outcome. That is with different measurement point and added thermal mass (ie. Cold soda can to the box etc.).
Not so quickly from phone.
Ps. Graphs should have (if possible) setpoint, controller output and desired target (box or dut temperature at this case) with those three you can see how the controller behaves. For long term measurements add athmosperic temp as that is source of disturbamce for you (by not knowing exactly all the whys and wheres).