Do you use a device that is pre built or do something custom?
Depends on the volume:
I worked on an extremely low volume special purpose device (10pcs a year = complete and utter wohooo!). There was no test jig for the PCBs, only a set of testing boards and devices that I plugged in manually into all of the IO and then went by a step by step procedure, manually checking a lot of things, occasionally cooperating with really awful software (I programmed it). The things were complicated enough that automating them would be
possible, but would require too much time and money to implement to justify making it, so this was chosen. The logic was that if I need two days to test and calibrate the thing then it's better to do it manually then spend three months creating a test device. It was pretty fun actually

I worked on a small/medium volume device - I designed a test device where the operator just placed the device, pressed a button, waited some 20 seconds, read the terminal output and checked whether there was [PASS] or [FAIL] at the end of the readout and whether a happy beep or an annoyed beep sounded. But again, it was not a bed of nails, but rather a device that connected to the IO port of the device with a high quality DSUB 25.
For a larger volume I actually designed a custom PCB with pogo pins. It was 3mm thick, sturdy, supported by an aluminum frame and a press pressed the DUT onto it.
For a big company I designed a device that would be produced in really high volumes, but I did not design the testing device, but saw it - a monstrous behemoth that could test insane volumes, built on a standard platform with a customized test fixture.