Author Topic: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?  (Read 4204 times)

0 Members and 2 Guests are viewing this topic.

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Hi,

I currently spend a lot of time developing on Cortex-M0+ STM32s. (also use low power STOP1 mode a lot, but disable it for debugging)

I find that the debug tools are pretty buggy and have strange limitations that are not well documented.

I tend to directly use gdb in text mode to maximize my chances of having things work.

I would like find out if a J-Link probe and J-Link gdb server might be "better" than ST's gdb server, and probe-rs's gdb server.

I currently am using a ST-LinkV2 probe.

When I use ST's gdb server I encounter the following limitations:
  • step and next work fine, but...
  • I can't do any variation of "print *0x40000000" (x/x 0x40000000 works)
  • I can't do any variation of "set *0x40000000=123"
  • watchpoints don't really work (or at least, they can't be deleted)
  • I can't single step instructions into interrupts or exceptions
  • Monitor reset don't work reliably


When I use probe-rs's gdb server I encounter the following limitations:
  • step and next only step single assembly instructions (although it is useful for looking at interrupts/exceptions it makes it useless for regular debugging)
  • I can't do any variation of "print *0x40000000" (x/x 0x40000000 works)
  • I can't do any variation of "set *0x40000000=123"
  • watchpoints don't really work

Would a J-Link or some other gdb server software solve some of the above limitations?

edit: added black magic to subject line since it was mentioned below and did not show in search
edit: copying example here for visibility
Code: [Select]
//After connecting to stlink gdb server with target extended-remote :61234
//This command is accepted and several packets related to this command are sent to the gdb server
(gdb) set {int}0x20000000 = 123
(gdb) print {int}0x20000000
$1 = 123

//It Works!
//But only before the MCU executes its first instruction

(gdb) continue
//Then either interrupt to get the prompt back with control-C or wait until you hit a breakpoint

//Running continue breaks it for some reason
(gdb) set {int}0x20000000 = 123
unexpected token
(gdb) print {int}0x20000000
unexpected token

//These "unexpected tokens" were perfectly valid one or two commands ago...


edit:

gdb tries to autodetect the current language. The common casting syntax requires the current language to be set to "C".

For me it was not auto-detecting that it was debugging a C program.

For reference to the future reader the "show language" command will show the currently autodetected language in gdb
"set language c" is required to avoid "unexpected token" errors for set commands using the syntax that is shown in my posts.
« Last Edit: January 31, 2025, 11:22:15 pm by incf »
 

Online thm_w

  • Super Contributor
  • ***
  • Posts: 7664
  • Country: ca
  • Non-expert
Most of these functions work fine when using either stlinkv2 or jlink edu/pro, with visualgdb. I think the v2 was just a bit slower.

If you have a stlink clone you can convert it into a jlink and try that out:
https://www.eevblog.com/forum/testgear/alternative-to-segger-j-link-edu/
https://www.reddit.com/r/embedded/comments/165ls6k/would_you_recommend_a_segger_jlink_over_stlink/
http://www.mjblythe.com/hacks/2013/02/debugging-stm32-with-gdb-and-openocd/

But I'm sure this stuff should be working with gdb, maybe someone more familiar with command line debugging will comment.
Versions?
Profile -> Modify profile -> Look and Layout ->  Don't show users' signatures
 

Offline pcprogrammer

  • Super Contributor
  • ***
  • Posts: 4768
  • Country: nl
Don't they all use the same GDB engine under the hood.

With the small target MCU's it is the debugging hardware that allows one to do certain things, and for this the interfaces like the ST-link implement a protocol to talk to the debugging hardware on the MCU side and the "server" on the other side. It could be that there are differences in this between the various interfaces.

I tend to use openocd to make the connection with the target, via the st-link v2 clones I have. This opens up a session with GDB and allows me to also connect to it from for instance netbeans. Have used it to inspect and modify variables and it works, but not al the time. Sometimes it states that a value has been optimized out, what ever that may mean.

It is my believe that the other tools implement the same protocol and do the same as openocd does in my case. The fact that similar things don't work like you expect with two different methods seems to indicate that the target hardware is not up to the task.

What might be a factor in play here is the host OS. I'm using Linux which is a closer match to GDB than Windows.

Offline elektryk

  • Regular Contributor
  • *
  • Posts: 147
  • Country: pl
Sometimes it states that a value has been optimized out, what ever that may mean.

That's normal, try -o0 optimization level and you will se every variable, there's also -og which was made for good debugging experience, but still some things may be optimized out.

https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15984
  • Country: fr
Debugging optimized code is not for the faint of heart. Often line numbers are all over the place and breakpoints do not correspond to what you think you set them to. Not a ton of fun.
 

Offline pcprogrammer

  • Super Contributor
  • ***
  • Posts: 4768
  • Country: nl
Sometimes it states that a value has been optimized out, what ever that may mean.

That's normal, try -o0 optimization level and you will se every variable, there's also -og which was made for good debugging experience, but still some things may be optimized out.

https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html

That will be it. I mostly have it set to -o3 to get maximum performance.

Don't use the debugger that often. There are other techniques that can be used, like using a GPIO and an oscilloscope to check if an interrupted routine is used and how long it stays in it.

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Yes, under the hood all the GUIs use gdb plus a "gdb server" to connect to the debug probe. They often seem to send the same text commands that a terminal user would.

I started tracing/printing the extended-remote protocol packets going between gdb and the stlink gdb server.

By accident, I partially figured out one issue (it still does not work). In the case of setting memory and printing addresses with gdb and ST's gdb server, it turns out that ST's gdb server needs killed and restarted every single debug session. (and possibly every time the MCU is reset?) (this "fix" does not actually make the issue go away, it just delays it)

gdb can start with the ability to print and set memory addresses (if any only if the debug server is restarted each debug session), but gdb looses the ability to set and print addresses once the "continue" command is issued.

For some reason, communicating with the stlink gdb server changes the "mode/state" that gdb is in and cause it to start rejecting valid commands with "unexpected token" errors*
Code: [Select]
//After connecting to stlink gdb server with target extended-remote :61234
//This command is rejected and no packets related to this command are sent to the gdb server
(gdb) set {int}0x20000000 = 123
unexpected token
(gdb) print {int}0x20000000
unexpected token

whereas after restarting stlink gdb server it initially works, and then stops working after typing continue
Code: [Select]
//After connecting to stlink gdb server with target extended-remote :61234
//This command is accepted and several packets related to this command are sent to the gdb server
(gdb) set {int}0x20000000 = 123
//It Works!
(gdb) print {int}0x20000000
$1 = 123
//But only before the MCU executes its first instruction
(gdb) continue
//Then either interrupt to get the prompt back with control-C or wait until you hit a breakpoint
//This breaks it for some reason
(gdb) set {int}0x20000000 = 123
unexpected token
(gdb) print {int}0x20000000
unexpected token

it still does not work, but it is a small step in the right direction

gdb 14.2 compiled from source
stlink gdb server 7.9.0

Footnote, tips for debugging stlink gdb server -> gdb
  • in gdb set `set debug remote 1` - this will show the presence or lack of packets going from gdb to stlink gdb server
  • in stlink server's config.txt enable verbose mode and all logging and give it a valid path to write the logfile (not really needed if you "set debug remote 1" in gdb)

search tags: STM32 gdb server unexpected token error, target remote, extended-remote
« Last Edit: January 22, 2025, 05:12:10 pm by incf »
 

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
For completeness. With probe-rs it is just like with the st-link gdb server - except I don't even have to type in "continue" for it to start rejecting valid commands.
Code: [Select]
//After connecting to probe-rs gdb server with target extended-remote :1337
//This command is immediately rejected and no packets related to this command are sent to the gdb server
(gdb) set {int}0x20000000 = 123
unexpected token
(gdb) print {int}0x20000000
unexpected token

You can also send stlink gdb server packets via netcat while gdb is connected, but it only seems to work for read commands.
Probe-rs does not respond at all to gdb extended-remote packets sent via netcat when gdb is connected.
« Last Edit: January 22, 2025, 01:43:12 pm by incf »
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 4448
  • Country: gb
  • Doing electronics since the 1960s...
Usually setting a breakpoint on
asm "nop"
or some such works fine :)

I use -Og. I found -O3 does very little extra. -O0 makes the code some 30-40% bigger than -Og and in my case would overflow the FLASH area I have.

A while ago I spent some time testing Segger stuff and could not see any point over ST's STLINK V3 - apart from the dynanic FLASH patching i.e. breakpoints in FLASH capability.

But I use Cube IDE, not command line.
« Last Edit: January 22, 2025, 02:58:10 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online ajb

  • Super Contributor
  • ***
  • Posts: 2792
  • Country: us
There's an OSH project that implements the gdb server on the debug probe itself, called Black Magic Debug

I mostly debug with an IDE, but IME there is a notable difference in performance and reliability with J-link probes vs ST-LINKV2.  Faster erase/program/verify cycles  with the jlink make a big difference on large flash parts and it's also not prone to random debug session failures or needing to unplug/replug the USB connection multiple times a day like with the stlink.   I don't know how much of that difference is down to better IDE support vs probe firmware vs drivers etc, but I've never regretted switching to the jlink stuff. 
 
The following users thanked this post: voltsandjolts

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9495
  • Country: fi
Welcome to the world of debugging. Some like to claim that debuggers are the magical solution with easy transparent access to the code that big boys use after they grow out of "printf debugging". Could not be further from the truth - for starters, you need to compile with -O0 or -Og or similar optimization level to make source-level debugging make any sense, which affects your program timing, performance, and size MUCH more than a simple "intrusive" logging layer ("printf debugging" without printf - e.g. saving ID numbers and values of interest to a ring buffer) of your own.

Remember, with a debugger, you are really debugging the machine code, not C code. Great fit when you also write your program in assembly.

It's not easy - it's not fast - it's not even supposed to be. But sometimes you need to do it. Just sometimes, not very often - for example it seems I open up gdb every two years. If you started doing it because you were told that Real Programmers use debuggers, you listened to wrong people. Invest that time into adding visibility, tracing, error checking etc. to your own code.
« Last Edit: January 22, 2025, 04:28:27 pm by Siwastaja »
 

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
There's an OSH project that implements the gdb server on the debug probe itself, called Black Magic Debug

Has anyone used a Black Magic probe before?

It sounds promising - a much better way to implement a debug probe from a technical perspective.


I have to admit that am hesitant to spend a thousand dollars on a j-link when a black magic possibly might work.
 

Online uer166

  • Super Contributor
  • ***
  • Posts: 1030
  • Country: us
There's an OSH project that implements the gdb server on the debug probe itself, called Black Magic Debug

Has anyone used a Black Magic probe before?

It sounds promising - a much better way to implement a debug probe from a technical perspective.


I have to admit that am hesitant to spend a thousand dollars on a j-link when a black magic possibly might work.

I have: it could work flawlessly on one platform, and be utter garbage on another for reasons unknown (intermittent connections, dropping connection to debug session, no-connect, etc). I have had much better luck with the much cheaper ST-Link V3 overall. Instead of spending $1000 on a J-link, you can spend $35 on an ST-Link :)
 
The following users thanked this post: mwb1100, Dazed_N_Confused

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 4448
  • Country: gb
  • Doing electronics since the 1960s...
Quote
Welcome to the world of debugging. Some like to claim that debuggers are the magical solution with easy transparent access to the code that big boys use after they grow out of "printf debugging". Could not be further from the truth - for starters, you need to compile with -O0 or -Og or similar optimization level to make source-level debugging make any sense, which affects your program timing, performance, and size MUCH more than a simple "intrusive" logging layer ("printf debugging" without printf - e.g. saving ID numbers and values of interest to a ring buffer) of your own.

True to some extent unless -Og is actually fine for your product, and it most certainly will be unless your CPU load is right on the margin.

Some results on the finished project (11/2024), boot block (32k max) / main flash (1MB-32k max)

-O0          15.3k / 623k
-Og          10.3k / 440k  <- used for development and production
-O1          11.5k / 493k
-Os          9.4k / 391k
-Ofast       15.5k / 490k

The code difference between -Og and -O3 is minimal, because most of those optimisations are just GCC devs having nothing to do for the last 10 years, and moving stuff sideways, looking for and optimising increasingly esoteric cases :)


Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #14 on: January 22, 2025, 05:31:24 pm »
Code: [Select]
Blackmagic for ST-Link Adapters

With recent ST firmware, the ST-Link v2, ST-Link v2-1 and v3 can be used with Black Magic Debug App rather than having to flash this firmware to the adaptor.

Running the BMP firmware on ST-Link v2 and ST-Link v2-1 provides:

    built-in gdb server on the dongle
    VCP on ST-Link v2. Access to VCP lines needs additional wires to the STM32F103C
...

Perhaps I do not need to purchase a probe. I might be able to reflash my ST-Link with BlackMagic firmware to see if it is less buggy.
 

Online ajb

  • Super Contributor
  • ***
  • Posts: 2792
  • Country: us
I have: it could work flawlessly on one platform, and be utter garbage on another for reasons unknown (intermittent connections, dropping connection to debug session, no-connect, etc). I have had much better luck with the much cheaper ST-Link V3 overall. Instead of spending $1000 on a J-link, you can spend $35 on an ST-Link :)

Heh, I almost mentioned in my post that I have a BMP, but have not managed to use it successfully -- gdb couldn't seem to see the correct COM port number, which I chalked up to some sort of gdb-on-windows configuration thing.  I'm sure it's a solvable problem, but it was easier to go back to the J-Link.

I have to admit that am hesitant to spend a thousand dollars on a j-link
 
FWIW, the J-Link Base models start at half that, if their feature set will work for you, and they offer software upgrades to the higher level models if you need their features later.  Still not cheap, of course.  I'll admit I got the Plus model for 50% off a while ago, that made it a much easier sell!
 

Offline Silenos

  • Regular Contributor
  • *
  • Posts: 65
  • Country: pl
  • Fumbling in ignorance
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #16 on: January 22, 2025, 07:49:38 pm »
Stm32 debugging? Certain peripherals have state machines being triggered from the r/w bus operations, often susceptible to hidden time limits like usb or QSPI. And at least stlinks tend to spuriously read over large swaths of memory, even when not halted, so they cause "bugs". I could not find a solution to suppress this behavior, so I ditched debugging that time for years.
« Last Edit: January 22, 2025, 07:51:23 pm by Silenos »
 
The following users thanked this post: Siwastaja

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #17 on: January 22, 2025, 07:53:18 pm »
All I was looking to do was read and write a NVIC register from the debugger using its address (ie. without having modify my program to contain a pointer to it). However
Code: [Select]
unexpected token got the better of me.
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 4448
  • Country: gb
  • Doing electronics since the 1960s...
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #18 on: January 22, 2025, 07:55:12 pm »
It isn't possible to properly debug USB or ETH because a breakpoint buggers up the extremely complicated "protocol". About the only thing one can do is to use a breakpoint to display variable etc values, and then you have to restart the target.

The only way around this is a hardware trace buffer but I am not sure any current CPU had got that. 30+ years ago one had in circuit emulators (sometimes implemented in logic analysers) which had trace buffers and could do debugging without slowing down the target.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online thm_w

  • Super Contributor
  • ***
  • Posts: 7664
  • Country: ca
  • Non-expert
I have to admit that am hesitant to spend a thousand dollars on a j-link when a black magic possibly might work.

jlink conversion of a stlink clone ($3) is free as I mentioned above. stlink clone can also be converted to bmp: https://wiki.philpem.me.uk/elec/stlink-blackmagic

Neither will have the performance of a full jlink but it may not matter. Full jlink clone can be had for $20-30.
Profile -> Modify profile -> Look and Layout ->  Don't show users' signatures
 

Offline amymcneil

  • Contributor
  • Posts: 14
  • Country: us
  • AB1WH
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #20 on: January 23, 2025, 04:52:19 am »
I'm using the Black Magic Probe (BMP) for my debug on ARM Cortex series processors. Mainly ST, but I've played with TI as well. The BMP probe speaks natively to GDB, simplifying how it all works. I've had no issues in the years that I've owned them. I like it much better than using the stlink device.
Some mornings it's just not worth gnawing through the straps....
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 4448
  • Country: gb
  • Doing electronics since the 1960s...
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #21 on: January 23, 2025, 07:46:04 pm »
What does the BMP probe do that a STLINK does not? I had a look at their website but can't see clear advantages.

Very occassionally, STLINK needs the USB cable pulled/inserted after some days. Also, rather more often, the target bombs out of debug mode and goes off free-running (I think we have some previous thread on whether target software is capable of crashing a debugger, and I don't mean by messing with the GPIO pins used for SWD). The SWV ITM console (a high speed UART basically, which the target can "printf" to, etc) is not all that reliable.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online thm_w

  • Super Contributor
  • ***
  • Posts: 7664
  • Country: ca
  • Non-expert
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #22 on: January 23, 2025, 11:17:14 pm »
What does the BMP probe do that a STLINK does not? I had a look at their website but can't see clear advantages.

BMP supports many other devices from other vendors. Hardware is a bit cheaper.
If its just STM32 you are looking at, I don't think there is a major difference.
Profile -> Modify profile -> Look and Layout ->  Don't show users' signatures
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 4448
  • Country: gb
  • Doing electronics since the 1960s...
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #23 on: January 24, 2025, 08:45:01 am »
I can't understand why anyone would want a cheaper debugger than an STLINK, which costs just a few tens of €. It's not worth even an hour of my time, and it will be a lot more than an hour if there is any config to do.

Like I said, the pricey top end Segger models have value to those who want unlimited hardware breakpoints in FLASH, but that seems to be about it... apart from a cultural preference in Germany to buy German regardless of whether it is actually Chinese, or (in the case of some total junk I've just received) Turkish.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline voltsandjolts

  • Supporter
  • ****
  • Posts: 2603
  • Country: gb
Re: ST's gdb server, probe-rs, JLink, BlackMagic differences between them?
« Reply #24 on: January 24, 2025, 11:52:38 am »
Like I said, the pricey top end Segger models have value to those who want unlimited hardware breakpoints in FLASH, but that seems to be about it...

Unlimited hardware breakpoints in FLASH?

Anyway, I use j-links because they provide freedom...
...to choose mcu vendor
...to avoid vendor ide nonsense (see thread 'Is CubeIDE a buggy piece of crap' for someone who's grumpy about their ide)
...to use os of my choice

I also appreciate...
...excellent reliability
...RTT is excellent, particularly useful on mcu's without swo capability
...ozone
...a familiar toolset, and that saves time across different mcu projects

There are likely open tools that achieve much of the above, but vendor tools are more about lock-in than freedom.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf