What kind of hardware do I need to make use of a 6-pin debug header?

[sad_blimp]

Hello, I'm currently reverse engineering the firmware of a consumer ARM-Cortex M device which has a 6-pin debug header on its board. The service manual clearly labels it as such. I'm a little unsure what kind of hardware I need to interface with it. My end goal is to be able to use some software like GDB to 'debug' the firmware. I guess I'll need a JTAG/SWD probe? Can anyone give me a good recommendation for one that won't break the bank? I've been looking online and the options are all bit overwhelming.

Background: I'm a professional software engineer who's somewhat new to embedded software development. I understand low-level programming well, and I have a good deal of experience using debuggers in userspace. However I'm less experienced in the EE-related aspects of embedded. I've been learning a lot so far from this project. The firmware reverse-engineering is going quite well. I'd just like to improve my methodology. I've reverse-engineered some other firmware in the past by hand successfully.

Apologies for what may be a silly question, however I'm a little overwhelmed with the amount of information online. Any help steering me in the right direction would be appreciated!

[capt bullshot]

There's a lot of cheap devices that you can use with OpenOCD. OpenOCD is the bridge from the hardware debugging device to gdb or other stuff. For STM32 it'll be an ST-Link v2, for example. These are available for maybe 20 Euros. Other hardware uses FTDI chips and works equally well with OpenOCD too.
For CortexM, you'll have to figure out the pinning of the connector first, afaik there's no standard for 6pin connectors. Then make an adapter to the standard 10pin or 20pin JTAG / SWD connector of your debugger.
BTW, when used with OpenOCD, the ST-Link should work with many other CortexM chips, too. It's not locked to STM devices.

[Twoflower]

I haven'r tried that for myself but it is on my list of things to do: You can use a RasPi for GDB JTAG debugging (e.g. https://blog.wokwi.com/gdb-debugging-esp32-using-raspberry-pi/). If my understanding is right the debugging works via LAN/WLAN or local on the RasPi. That LAN feature usually cost a fortune with commercial hardware and might be useful if the debugging needs to be in-situ at a remote location.

One downside is probably using a RasPi is that you will not reach the speed the expensive JTAG hardware provides (up to 100MHz or so if the hardware and wiring provides that). The page I mentioned above says 1kHz works for him. And you're limited to 3.3V I/O voltages or you need additional levelshifter.

[Doctorandus_P]

... however I'm a little overwhelmed with the amount of information online.

That is the main problem with ARM these days.
A small microcontroller has over a 1000 pages of datasheet, I once downloaded the datasheet for the Ti Sittara (used on BBB) which was something like 3500 pages.
And that's just the datasheets for the IC's.  Manual for GCC is also of similar size and then there is documentation for tools, application notes, tutorial websites and what not.

I quite like the ST-Link V2 clones from China Direct. Even though they are undocumented and can various brands of uC's can be inside it. I think the last I used had a uC from APM in it, but STM32Cube detected an "old firmware version", I clicked "OK" to update it and after that it still worked.
And apparently you can turn these into a "Black Magic Probe", which would turn it in an universal tool which can also be used for other JTAG based stuff, but I have not tried it myself.

Usually each chip manufacturer has some recommended programmers for their IC's, or have hardware that only works for their IC's. There are also companies that concentrate on making more generic tools such as Segger, but these (more powerful) tools tend to be much more expensive too.
Such companies can never compete with an USD 3 ST-Link Clone from China.

Any help steering me in the right direction would be appreciated!

So you have pretty much two directions to choose between. A generic programmer, (relatively expensive) or a simple programmer that works with your brand of uC (Pricing starts at USD 3)