Using Claude Code for embedded work

[paulca]

Counterexample would be the one guy posting here who proposed that a phone app that plays music synchronized to workout patterns takes many years to ship, and even a simpler, useful "first release" cannot be made in less than years, and that huge amounts of investor money is an absolute necessity to pull it off. My instinct says they should be able to ship the first version in 3 months.

Which is sort of a dumb idea anyway.  I would have pointed him to cognitive analysis studies and his own experience to show him, they don't need to be in sync.  Your brain will sync them.

Test yourself.  Open a music video with people dancing.  Mute it.  Now open another one.  Keep that audio and watch the first video.  It doesn't take long until you can no longer tell if they are or are not in sync.  Then you just accept they are.  As an example YouTube fed me a bunch of "Shuffle dance" videos.  Lots and lots of clips of dancers collated together into a video.  I thought the effort to synchronise all those clips would be enourmous.  Then I noticed, watching carefully.  They were not synced or even the same tempo in most cases.  I'd just never noticed until I looked.

It's the same rabbit hole that leads you down to the "Spinning dancer silohette".  With thought pressure alone you can reverse the dancer.

Sorry, but philosophy and cerebral is the norm for Fridays for me.

[paulca]

Back on topic.  Two LLM short comings noticed lately to speak about.

1.  The age old 'why do you insist on counting when you know you can't count?"  - the update is that it does a far better job if the thin it's counting actually has delimiters, which are not spaces.  It seems to count list items fairly well, but not words in a scentence.

2. Is new to me and rather frustrating.  It lacks the ability to model a timeline.  So if you have multiple "states" to consider.  Such as "The present state", "The historical reasons for that state", "The future possible state to analyse and document"...  it starts to come apart.  Interplacing past, present and future.

In work when tasking claude with a migration from SOAP to REST, the output documentation continually refers to one specific component as being "REST" when it's been told many times it's 'not REST, it's still SOAP, we haven't migrated it YET."

If you don't spot that, and you therefore might trust a timeline or plan from claude...  it can get ugly.

It gave me a timeline breakdown a few weeks back, which I read and frowned and realised... it had listed about 7 days worth of effort and then said, "That fits fine into the half day window the ticket mentions".  I was like "The f... it does."

My advice?  Get it to use some lower level software like a spreadsheet or CSV file to track/sum/summarise empiricals and numericals.

[peter-h]

The "problem" with this topic is that it changes all the time.

Claude is way smarter now than it was 6 months ago. I've just uploaded a ton of graphics code to it and instantly it spotted a bug in a filled rectangle algorithm which was stopping 1 pixel short (I optimised it for DMA from the original code).

The £150/year subscription gets you way more leeway. Last night I uploaded ~100 .c files for review and only after a bit more it stopped, locking me out until 3am. First time I've seen that since I started paying.

[woofy]

The £150/year subscription gets you way more leeway. Last night I uploaded ~100 .c files for review and only after a bit more it stopped, locking me out until 3am. First time I've seen that since I started paying.

I found it was never enough to do a full working day, always the enforced 5Hr break.
Now I pay £75 (£90) a month and get a full days uninterrupted work. It's an amazingly capable assistant for such a small fee.

[SpacedCowboy]

If you're looking for a cheaper (at least for the moment) way of doing it, there's a nice guide for how to use Claude Code with DeepSeek R4. I don't think DeepSeek is quite as good as Claude, but it's still pretty darn useful, and it's a *lot* cheaper.

[paulca]

Interesing.  On claude $20 a month, i would hit the 5 hour window limits fairly regularly.  As it was "home/hobby" it didn't bother me, I could schedule around it easily.  There were also many tasks for me to carry on with without claude.  Ocasionally I would hit the "weekly limiter" but usually only a day before the refresh anyway.  Annoying but copable.

In work, when I run the claude cost skill, I see my sessions running for several work days claiming less than 100 dollars usually.  Granted I might have severl open.  So maybe $10-50 a day?

However.  This is because of the way I work with claude.  When I shift to using the distributed "work skills library" things change dramatically.  Suddently a single session, running a single "skill" on a code base shoots up to consuming $150 in a single run.

What is the difference?  "Chunking".  Personally, I drag claude with me until it's ready to "pull me".  I barely use "Yes and don't ask again".  I do not auto-accept edits.  I frequently interrupt it.  I frequently reject it's proposals and repoint it.  I give it, "What chunk of work can I have it do, reliably, which I can IMMEDIATELY review.".  Moving in micro-loops.  Plan, act, review, repeat.

I have what it does not.  I have a four dimensional holodeck I can build mental models in.

The dollar difference is simply put... I don't "attempt" to out source the thinking to the unthinking.

[Siwastaja]

The £150/year subscription gets you way more leeway. Last night I uploaded ~100 .c files for review and only after a bit more it stopped, locking me out until 3am. First time I've seen that since I started paying.

I found it was never enough to do a full working day, always the enforced 5Hr break.
Now I pay £75 (£90) a month and get a full days uninterrupted work. It's an amazingly capable assistant for such a small fee.

I hit the usage limit for the cheapest $30/month tier, I think the second or third day I started using the thing - doing 3-4 "small things" within a few hours was enough. Then switched to the $100 tier and have never hit limit again. Not thinking about the whole thing. And I sometimes have 3-4 agents running in parallel, and sometimes they themselves fetch parallel explore agents or whatever they do. This service level seems to suffice for jobs where a single human is in the loop.

[paulca]

A random claude moment...  I teased it's inability to count and I got this back.... I'm baffled.

The context is the question: "Still happy?"

It's 3 sylables, right?  "Still" is one?  It's not stu-ill, it's still.  Happy is two.  "Hap-ee"

What is the hell is it on about here though?

Ha. Noted. Corrected.
Still-hap-ee.
The rhythm actually matters. Four syllables would be a question.
Three syllables is a verdict.

[tszaboo]

So this piece of shit sent me on a 1 hour debugging session, where the I2C  wasn't working. I asked it where to connect it.
And then it tells me this:

I think your pinout is wrong. I think PA8 and PA9 are I2C pins. double check

12:02 PM
You're right to question it. Let me look this up properly rather than guess.

And people think that this is usable for any serious work? It didn't have any error checking, it was just rewriting the same code over and over again, every time coming up with a different theory why it doesn't work.
For hardware it's really dumb. Basically it cannot calculate sometimes a simple resistor divider with it. When you ask it about any serious circuit, it makes a mistake, you correct it, only to make the same mistake again in a different way. Every time apologizing.
It's written to give a low-effort answer, and have code verification take care about it when it doesn't work. This needs to work completely differently for hardware, when you need to spend an hour fiddling with wires and measuring with a scope, based on this low effort answer to figure out how it messed up.

mod: but at least it thinks like a firmware engineer. It tells me to change the hardware so it works with the code. My answer was:

Change the f*ing code already to support this

[Kjelt]

On the job it messed up a simple decimal to hex conversion of a 1024x1024 matrix table.
Luckily it was pretty obvious, if it were only a few mistakes we would still be debugging.

[paulca]

So this piece of shit sent me on a 1 hour debugging session, where the I2C  wasn't working. I asked it where to connect it.
And then it tells me this:

Replace LLM with "Dumb junior" and you will end up in the same place.  The only difference is a good junior might tell you, "No idea what that pin out is, can I have a datasheet for that model please?"

What you got instead was what I like to refer to as "Version soup".  All the different versions of the MCU family pin out all mashed up together and selected by probability.

The junior might have just sat on it for a few days and then giving you nothing.  The annoying part with AI is its confidently wrong instead.  You can tame it though.  You need stop thinking of it as an "entity" and start thinking of it as tool.  Find the things it does do well or find ways to make things it doesn't better at it.

Specifically, give it more information and a feedback loop so it can find it's own mistakes and fix them.

Also, know what to expect.

[Siwastaja]

And people think that this is usable for any serious work?

I'm on your ignore list, but have to reply anyway because I'm using it for a lot of serious work and today had such massive fucking success story again (even on-topic to this topic's subject field), and it's exactly because the pattern you show is nowadays more an exception, than a rule. (I see guessing instead of checking happening, but less than with humans; the most annoying thing really is the tone. Most humans say "here's a guess"; AI says: "Here it is! Found the root cause!" - to which I grin, knowing it's a guess from the lack of proof (it shows that when it actually checked), and go on.)

But to the today's success story, and it's a story of not guessing, it's a story of checking, reasoning, and doing good job as a result.

So we have this energy management box. Our strength has classically been that we have responded to customer feature requests (customers = sales companies, in this context). So "we have this new Sungrow EVSE, we already sold a few, installers already installed them, can you add a support". So this time I think "fuck this life" and just ask Claude: "read this PDF with gazillion of modbus register definitions. Do a driver." Not much more prompting than that. A bit of chatting of all the new status fields I want - new enumerated charging states, current outputs, choosing between A/B charging socket, also supporting their DC charger. I continue doing other things I'm more interested about.

10 minutes later, I have sungrow.c with 400 lines of code - oh nice, could have been 2000, so it did a decent job fitting that many features in. I glance it for 2 minutes, and all I ask to change is to rename it to sungrow-evse.c

Then bravely remote flash the firmware to actual customer waiting for their Sungrow EVSE to start working. Nothing happens. Log says no traffic on modbus. So, RS485 A/B likely swapped, or EVSE-side configuration disabled the bus. Bummer - but this always happens. Tell them to go at the site and start swapping the wires, go through settings and so on.

Now, what makes this the success story? Week passes, I already forgot the whole Sungrow EVSE thing. Now my main work computer broke. Possibly CPU seating issue or RAM or something. That's the machine I'm used to using, with everything set up like I want it to. In the middle of taking the CPU out, the installer calls: "hey, I'm now at the site." Fuck, I had forgotten about the whole thing, and this is the worst possible moment.

The normal flow at this point: painfully start trying out things; look at logs, look at code, ask the installer to switch settings, swap wires, look at logs more, adjust code. Because we have two moving parts: the EVSE and our new, untested code - this is the first time we get to test it! It's a chicken-egg: we have to normally assume neither side works.

Then what happened is magical: installer enabled External EMS control from the EVSE (so it was disabled, no wonder it wasn't working); slave_id autoscan mode Claude suggested adding (and added) locked into slave ID - car started charging - cable current limit, acknowledged current limit, and measured phase currents all popped on the UI.

Implementation perfect. First try. All features work. Didn't touch a single line in code.

Now because I have written, without any AI, more than 20 such drivers (solar inverters, hybrid inverters, some EVSE) myself in the past, I know that for myself, writing the damn thing means I need 1 day of uninterrupted concentration, then another half-a-day with installer on site, while I meticulously debug.

Now I happened to push the experimental branch to remote just hours before the computer broke down. I could have debugged it manually on my laptop; cumbersome, I don't like it, but doable. But I didn't need to. The binary already running on the device was spotless. On the first try, no modifications needed. So had I failed pushing the branch, I would have survived. Installer got out there in no time; sales company happy, end-user happy.

That's all thanks to the high quality of work. That's all thanks to Claude reading the documentation carefully, doing its reasoning cycle (which is nowadays hidden to prevent competitors trying to distill the model / learn from it) carefully, and writing the code carefully. A very good human programmer might have done exactly the same; I'm not one so would have had ~3-5 showstopper bugs to iron out, in a step-by-step process where first I get nothing, then I get a mangled status field but no command acceptance, then I fix the status field, then I try a different command, 2-3 times until it works. And it's not a bad process, when one can dedicate time for it. But that's the hard part. All of this was not needed as the AI did so good and careful job initially.

[brucehoult]

But to the today's success story, and it's a story of not guessing, it's a story of checking, reasoning, and doing good job as a result.

What was the token usage for this?

[tszaboo]

So this piece of shit sent me on a 1 hour debugging session, where the I2C  wasn't working. I asked it where to connect it.
And then it tells me this:

Replace LLM with "Dumb junior" and you will end up in the same place.  The only difference is a good junior might tell you, "No idea what that pin out is, can I have a datasheet for that model please?"

What you got instead was what I like to refer to as "Version soup".  All the different versions of the MCU family pin out all mashed up together and selected by probability.

The junior might have just sat on it for a few days and then giving you nothing.  The annoying part with AI is its confidently wrong instead.  You can tame it though.  You need stop thinking of it as an "entity" and start thinking of it as tool.  Find the things it does do well or find ways to make things it doesn't better at it.

Specifically, give it more information and a feedback loop so it can find it's own mistakes and fix them.

Also, know what to expect.

It can answer correctly, and look up the answer. It can download the datasheet, and find out what the answer is. So it's a choice, to give you a low effort answer, because tokens are expensive.
The other day I gave it a BOM, and asked it to look up the price of each components. This is some time consuming work that I'm used to do, and a legitimate work for AI. So I got back a BOM that had similar, guesses for the price, without indication. And it was missing about 5 lines, that just disappeared. Then I fed in my email quotations for some components, and it made copy paste errors from that.

I wouldn't call it an a junior engineer or an intern. I would rather call it monkey with a typewriter. It seems to works well for software, as you can set up a verifiable end condition. And beat the LLM until it accidently makes something that fulfills the requirements (and fails at point's that you haven't tested for). The whole "neural network" theory only works if you have a feedback. If I'm the feedback, than I much rather would want to work in a salt mine than doing this.

[woody]

For the Dutch audience....

[Siwastaja]

But to the today's success story, and it's a story of not guessing, it's a story of checking, reasoning, and doing good job as a result.

What was the token usage for this?

Don't have the log because that's on the broken computer, but probably something around 150k it must be, ballpark (including input, output, hidden thinking tokens, tool use, everything). When the context window was still 200k, tasks like this usually (barely) fit without autocompact, but working any further with fixes, extra feature ideas, discussions, optimizations ended up triggering 1-2 autocompacts. With 1M context window, autocompact isn't a thing really anymore - almost always the job is done in 300-700k tokens; if it's a complicated long-term design job then manually trigger somewhere between 500k-1M at a suitable moment.

[paulca]

It can answer correctly, and look up the answer. It can download the datasheet, and find out what the answer is. So it's a choice, to give you a low effort answer, because tokens are expensive.

This is back to front.  You are paying for the tokens.  They sell tokens.  They want you to use more of them.  On corporate "pay per token" accounts it gives the most verbose possible answer unless you tell it not to.

The other day I gave it a BOM, and asked it to look up the price of each components. This is some time consuming work that I'm used to do, and a legitimate work for AI. So I got back a BOM that had similar, guesses for the price, without indication. And it was missing about 5 lines, that just disappeared. Then I fed in my email quotations for some components, and it made copy paste errors from that.

How is your experience on those sites?  Do you find the search/filters each to use and the pages easy to navigate?  No?  You are the customer, the invited, the one they WANT there.  The LLM isn't.  It has to get through the anti-bot filters or find the RSI feed version or the text only version.

Further if you don't specifiy you want only looked up data with references, it will mish mash in training data too.

And ... no.  It does not go well.  It's the same problem.  "Information".  It doesn't have it.  Doesn't know it doesn't have it.  Does not know what it does not know.  Unlike a human.  (most humans at least leave scope for the unknown unknowns and self generate them in advance as what ifs).  LLMs don't.  No emergent thought.

I wouldn't call it an a junior engineer or an intern. I would rather call it monkey with a typewriter. It seems to works well for software, as you can set up a verifiable end condition. And beat the LLM until it accidently makes something that fulfills the requirements (and fails at point's that you haven't tested for). The whole "neural network" theory only works if you have a feedback. If I'm the feedback, than I much rather would want to work in a salt mine than doing this.

If you give it the information you have access to, properly, in a way it can access them and that only leaves your expertise as the "monkey in the loop", then that is basically where you want to be.

If it doesn't save you time, it doesn't save you time.  The doubt will always be: Was it capable of solving me time and I just used it wrong?

[peter-h]

There is a load of stuff in the press about Mythos existing in a special "non public available" version for finding vulnerabilities in code.

Can Claude do this?

I am sure it will find a ton of them in MbedTLS (Mythos has reportedly already found serious ones in WolfTLS) but  one can't just keep re-hashing one's product with the latest version, with all the regression testing needed afterwards...

It would be interesting to check LWIP

[nctnico]

So this piece of shit sent me on a 1 hour debugging session, where the I2C  wasn't working. I asked it where to connect it.
And then it tells me this:

I think your pinout is wrong. I think PA8 and PA9 are I2C pins. double check

12:02 PM
You're right to question it. Let me look this up properly rather than guess.

And people think that this is usable for any serious work? It didn't have any error checking, it was just rewriting the same code over and over again, every time coming up with a different theory why it doesn't work.
For hardware it's really dumb. Basically it cannot calculate sometimes a simple resistor divider with it. When you ask it about any serious circuit, it makes a mistake, you correct it, only to make the same mistake again in a different way. Every time apologizing.

Maybe you should switch to the opus model or go through some tutorials. I have a collegue who is more versed into using Claude and I try to follow his lead while getting myself more acquinted with the workflow. The way to work with Claude is to ask it on how to approach a problem. Then refine it's approach before doing the actual implementation. Over the last few weeks I have been using Claude for various coding tasks and I find it works great for real work. Every now and then it doesn't always hit the nail on the head the first time round (especially when the specification isn't 100% clear *). But it does pick-up on little details and comes up with good questions and suggestions regulary.

At the moment I'm modifying a convoluted messy Python program to use a different API towards the backend and adding new functions and parameters to it. I didn't had to type a single line of code. Just guide Claude through the changes and test the result. Even checking whether a GUI layout was nicely aligned worked well.

But it takes chopping a project into pieces and describing what you want carefully. The latter is what I'm spending most of the time on. What do I want Claude to to exactly? What should the implementation look like at a functional level?

* One of the neat tricks of Claude is that it can write documentation as well. That is something you should use. For the project I referred to above Claude tripped over some inconsistencies so I let Claude create a writeup of how the API is to be used (in an md file) and added a CLAUDE.md in the root of the project. Then I cleared the session to reset the context. But I still needed more additional API calls. Claude used it's 'own' documentation about how to use the API and avoided the mistakes it made during the first run.

[nctnico]

There is a load of stuff in the press about Mythos existing in a special "non public available" version for finding vulnerabilities in code.

Can Claude do this?

I am sure it will find a ton of them in MbedTLS (Mythos has reportedly already found serious ones in WolfTLS) but  one can't just keep re-hashing one's product with the latest version, with all the regression testing needed afterwards...

It would be interesting to check LWIP

Claude is probably great to go through the steaming mess LWIP is 

[peter-h]

One of the neat tricks of Claude is that it can write documentation as well

It's funny... I sent a comment block to a friend, re a piece of code I did, and his immediate comment was "Claude?" Actually I wrote it

Claude is probably great to go through the steaming mess LWIP is

It didn't find much. A few possible buffer overflow issues, probably not exploitable from the ETH interface anyway. It does find various issues which could happen as a result of a cosmic particle

[tszaboo]

So this piece of shit sent me on a 1 hour debugging session, where the I2C  wasn't working. I asked it where to connect it.
And then it tells me this:

I think your pinout is wrong. I think PA8 and PA9 are I2C pins. double check

12:02 PM
You're right to question it. Let me look this up properly rather than guess.

And people think that this is usable for any serious work? It didn't have any error checking, it was just rewriting the same code over and over again, every time coming up with a different theory why it doesn't work.
For hardware it's really dumb. Basically it cannot calculate sometimes a simple resistor divider with it. When you ask it about any serious circuit, it makes a mistake, you correct it, only to make the same mistake again in a different way. Every time apologizing.

Maybe you should switch to the opus model or go through some tutorials. I have a collegue who is more versed into using Claude and I try to follow his lead while getting myself more acquinted with the workflow. The way to work with Claude is to ask it on how to approach a problem. Then refine it's approach before doing the actual implementation. Over the last few weeks I have been using Claude for various coding tasks and I find it works great for real work. Every now and then it doesn't always hit the nail on the head the first time round (especially when the specification isn't 100% clear *). But it does pick-up on little details and comes up with good questions and suggestions regulary.

Thanks the model switch is actually a good idea. I gave it a try on a previous chat that I abandoned because the model was dumb.
Told it to find it's mistakes and it was able to find a good chunk of them.

The "ask before implement" is a good idea, I've been working that way for about a year now.
Instead of telling the AI to do x, I told it that we will do x, but first, ask 10 questions to clarify.
Claude does this quite well, and sometimes it asks back by default. When it comes back with a few questions, and you can select the answer, that's quite good and efficient.

[Siwastaja]

Instead of telling the AI to do x, I told it that we will do x, but first, ask 10 questions to clarify.

This is the right way. It's also crazily good at giving suggestions how to make something differently, it would be crazy not to take advantage of such engineering peer, you can always choose what to do, and the discussion in the context helps it do the implementation, when it understand that some feature was discussed and maybe dismissed, and why.

So the pattern is, "I want X, I think we should do it like Y, I also want Z and by the way maybe this affects Ä and Ö. Comments/suggestions before you implement?"

I rarely have serious X-Y problems but I might have, and it could catch those. But almost always there is some important detail I didn't consider. AI will fill the missing piece anyway - how else would it reach the goal - so better to catch that early, so you can choose together what to do.

Before implementation starts, the initial chat and investigations might be 50k - 150k of context depending on the complexity of the problem (how much reasoning/chatting about it was needed). That context is very valuable, and with 1M of context the actual implementation work and quite a few iterations still fit, while the original discussion at the start of the context gets good attention throughout the session.

Tickets are completely dead for me. Just chat, like with human coworker/engineer, design together, implement, finish things quickly instead of leaving them in ticket limbo / half-planned or half-finished. Git + text-based documentation written by the AI is formal enough to keep things non-chaotic, tickets and formal planning systems do not add value for me.

[paulca]

It didn't find much. A few possible buffer overflow issues, probably not exploitable from the ETH interface anyway. It does find various issues which could happen as a result of a cosmic particle

ETH?  as in Ethernet?  You have a network connection on it?

Can those buffer overflows be used to disable, brick, tamper or extract security credentials from?

Cosmic particle of determined cyber mal actor?

You see when a "hardware device" goes out in the wild with exploits, automated sripts are written and sold to locate and exploit them en-mass, literally itearating through IP ranges. 

So if you sell 10000 of your produces, they all get bricked and you get a lot of unhappy customers.  It's too late to change them.  You have allow fully refunds for retrurns.

[peter-h]

Yes, Ethernet. Low level code connecting that to LWIP, and then various apps on top of that, including MbedTLS.

The cosmic particle comment was referring to buffer overflow checks located behind other buffer overflow checks, so if e.g. the ETH hardware hard-limits the packet size to x, there should be no way to exploit a buffer overflow with bigger packets. But a particle could corrupt memory, etc...

Of course, that is the IOT device marketing risk. Check out the long thread on how to do a remotely controlled heating controller. If you put IOT devices on an open port, there is always this risk, so the "proper way" is to have them as clients, behind NAT, and connecting to the back end of a proper server, only. OTOH, a lot of IOT devices have no practical attack surface, due to primitive implementations. Anyway, that's digressing, and an old debate.

I am not sure Claude can do a whole-product vulnerability checking. I would need to move to Claude Code, which I have not tried yet. Basic Claude has a 20 file per "message" limit. I could not upload my whole project to it, within the £150/year sub level.