The difficult bit isn't getting a few pogo pins together in the right pattern, it's holding them onto the board.
The Tag-Connect connectors are ridiculously overpriced, but the fundamental idea of them is right - some precise pins to locate into tooling holes on the board and some kind of clip to pass through other holes in the board to provide retention force for the pogo pins. Now, if they were ~$10 each instead of ~$60 each I'd probably already have a selection of them.