EEVblog® Electronics Community Forum
Products => AI => Topic started by: Siwastaja on March 12, 2026, 07:44:30 am
-
Oh boy, I am rarely wrong in anything, but when I am, I really enjoy the moment.
My observations from starting to use modern-day agentic AI for programming, i.e., vibe coding, are confirming nearly everything fanboys have been saying for some time already.
Sure, it probably wasn't that good a year or two years ago.
It is not a stochastic parrot. It's not a good pretender (or if it is, then we humans are too). It is a true intelligence with excellent capability in understanding design intent and code itself. It is excellent in logical thinking.
Most importantly of all, the quality of the work is excellent. It has time and perseverance to verify itself - doing sanity checks, writing tests, running those tests, documenting what it did.
It seems to me software development via AI (possibly other forms of engineering, i.e. electrical & mechanical, but the capabilities are lagging behind AFAIK) is one of the most useful cases. This is because software can be copied; good software can be run on millions of computers, or serving millions of customers, therefore creating good and useful software is more valuable than every layman trying to do all their tasks via direct AI use, which is hugely inefficient.
Specifically, we have been in quite difficult situation for years. We have a product with web frontend, server backend and embedded side. A lot of algorithmic magic is going on in the embedded firmware, and I'm doing that. Server and front-end side has had poor throughput of bug fixes and new (important) features. Professional human beings are somehow time-constrained and seem unmotivated with mediocre quality. The firmware side has developed much more features, and I have been fixing serious bugs with better response time, but still some important work has always been stagnated, and lately has become even more stagnated, as I am overloaded with customer service (electricians, sales companies, that sort of customers), management of things, business-related etc.
AI is excellent in somewhat complex logical thinking, way better than human beings, maybe excluding 0.001% true autists (mild assburger doesn't cut it). So it makes fewer mistakes in logic, when it comes to applying math, numerical analysis, analyzing conditional paths etc.
Clearly you can't let AI do all fundamental decisions. It does very crude mistakes because it lacks intuition an involved, motivated human being has. This is because my brain has possibly, partially even subconsciously, worked with interesting problem for possibly years while picking mushrooms in the forest or doing whatever. This is the intuition and creativity AI does not have. But fortunately we humans are excellent in chatting about these things. We are worse when it comes defining them in very exact numerical ways, but turns out AI is nearly perfect in that. It understands the intent and fills in details with its training data, which is massive, and if it needs to, it will google for open data source APIs, write a python script to fetch data, then write more scripts to run simulations, and keep doing this with very little intervention.
The workflow seems very close, if not exact duplicate, of how we humans operate, just 20x faster. So I could spend a week researching, writing code, testing, verifying, running simulations, documenting. Except I can't, because I don't have that week of free schedule. AI does the same in 2 hours.
AI models and workflows are probably getting better as we speak, but that is irrelevant. What we have now is already truly revolutionary, and it really works.
And the scope where it works is larger than most guessed. You don't have to be a prompt specialist; you don't have to limit the usage of AI to some trivially small edge case. It can take large part of responsibilities in the whole software development cycle. It needs human for large-scale context understanding and guidance of core decisions.
I have been so wrong and ridiculed AI fanboys for no reason. Now I'm a definite fanboy. And paying $100/month for this level of service, I could actually seriously consider paying $10000/month because this is a better investment than a similarly paying professional human being.
This post is not written, fine-tuned or even checked by AI. I don't want to use AI for that.
-
Mods beware: I think Siwastaja's account has been hacked ;D
-
But, using AI generated code in commercial apps is risky, is it not?
I'm not sure it's even copyrightable because AI generated artwork isn't.
And sometimes it spits out someone else's code verbatim.
The AI code provider washes their hands of it and applies no license.
https://www.zdnet.com/article/if-chatgpt-produces-ai-generated-code-for-your-app-who-does-it-really-belong-to/ (https://www.zdnet.com/article/if-chatgpt-produces-ai-generated-code-for-your-app-who-does-it-really-belong-to/)
-
But, using AI generated code in commercial apps is risky, is it not?
I'm not sure it's even copyrightable because AI generated artwork isn't.
And sometimes it spits out someone else's code verbatim.
The AI code provider washes their hands of it and applies no license.
https://www.zdnet.com/article/if-chatgpt-produces-ai-generated-code-for-your-app-who-does-it-really-belong-to/ (https://www.zdnet.com/article/if-chatgpt-produces-ai-generated-code-for-your-app-who-does-it-really-belong-to/)
Those are real questions. Maybe I'm lucky in not seeing the pattern because I have been asking it to work with my C code and it does it in similar style, writing logic on its own. I can imagine if I asked it to produce a filesystem or instantiate an UI library, it would produce large chunks equal to existing code in training material.
Then again, human programmers, me included, are notorious for copy-pasting large chunks out of stack overflow or code examples, and no one was concerned about licensing.
-
Oh boy, I am rarely wrong in anything, but when I am, I really enjoy the moment.
My observations from starting to use modern-day agentic AI for programming, i.e., vibe coding, are confirming nearly everything fanboys have been saying for some time already.
"Vibe coding" is not agentic. "Vibe coding" is just using the chat interface. Taking what the bot gives and copy pasta'ing it.
But anyway thats opinionated pedantry.
Beware, you will go through a roller coaster. Sounds like you are in "Honeymoon" period. Don't worry the bubble won't burst on you after it, but you will hit some "lows" that will frustrate you.
Last night I gave it a fictional game assembly language application. It was 119 lines and handled 5 gas phase change separator stages. I needed a 6th stage for a gas the game calls "Volatiles". I did not ask it to write the code for me. I told it were to find that code in the game and what I needed to do with it. It was not in "plan mode" so it just asked if I wanted it to generate the code. I said yes.
It used the tooling and knowledge base I have had it build over the past few weeks of tinkering. Python scripts for hard facts/maths etc. Documents for invariants etc. It re-calibrated all my "YOLO" style temperature and pressure values to be more... cleverly choosen, giving me the justification for the values it choose.
It produced code. It was 176 lines of nice, neat game language ASM code. Trouble is the game forces a limit of 128 lines. So I prompted claude explaining this and suggested we might be able to functionize the main "stage block" and feed the parameters off the stack.
It went off "Thinking...." for 15 minutes. Googling the instruction set on the game wiki etc.
I had almost given up hope and was not expecting anything back but it came back with 109 lines. Reading it however and I immediately noticed it has also found the problem I did in parallel. Two of the parameters are "LogicType" types. They do not push/pop so easily. Instead of flagging it, reporting it, failing on it, it just .... dropped those lines entirely. These lines which checked the "condensation vessel" of the stage was "Free of all gas/liquid of it's type" before it would consider opening it's inter-stage pump.
Luckily, in that 15 minute period I was also active and working. I spotted that the "stage coolers" loop at the bottom of this ASM file was completely decoupled and could be lifted and shifted to a new processor entirely. This gave me 19 lines back.
I updated claude with this finding and in under 1 minute it reverted the functionisation, reused the duplicative block and extracted the cooler code into a new file. On testing it in game, it needed some standard "fluff" fixed. There was a bug in my original where I checked the wrong pump status in one stage, it replicated that bug across all stages, easy fixed though. It also "typo'd" one or two device labels which where easy fixed too. Did it save me any time? Yes. However, what it does do, that I don't, is keeps it's quality higher. I often tend to just hack things out and leave myself techdebt (in hobby land). It is very diligent with comments and documentation. Files don't get named, "sep.ic10" it names them "gas_separator.ic10".
When it fails, it fails silently. That's a worry. It's something they can fix, but it's not the first time I have seen it "cheat".
As I said in another thread. The "upside down cup" thing is real. It may have been obvious to "me" that the ASM script could be split onto two (or more) processors, but it was not obvious to claude in the context. This is why the best work comes if you keep it on short reigns and actually work "with it", not just hit "Enter" until it's done.
-
But, using AI generated code in commercial apps is risky, is it not?
I'm not sure it's even copyrightable because AI generated artwork isn't.
And sometimes it spits out someone else's code verbatim.
The AI code provider washes their hands of it and applies no license.
https://www.zdnet.com/article/if-chatgpt-produces-ai-generated-code-for-your-app-who-does-it-really-belong-to/ (https://www.zdnet.com/article/if-chatgpt-produces-ai-generated-code-for-your-app-who-does-it-really-belong-to/)
I think the way to put it is, "You are responsible for the copyright of your AI output". The word responsible does a lot of lifting though. It doesn't mean you own it, as if it has lifted a non-trival block of copyright code into your project, it could be disputed as to ownership.
However, this is far more likely to happen if you do things like,
"Please make me a TODO webapp"
.. as YOUR only contribution. What you should expect is highly likely to be suspect copyright wise. Do you think you 'should' own that?
It is rather annoying though, because if you ask the bot about something and would LIKE a verbatim copy back of it, they won't. They will avoid it at all costs. When I quote a famous author to it, it will quote back no more than 1 line, 10 words.
If I ask it, "Can you show me the actual entry in an Encyclopedia for <insert term>" it will almost certainly "not" do that. It will hedge, fluff around the issue and get confused, as it is forbidden from doing so.
If you spend time and type your ideas, visions, thoughts and steer the bot at every turn, THEN run your code through a plagerism test, you will be surprised in many cases to find it has a very low score. The ones I have tested came back in low teens %.
Oh and on the other aspect. "Whose fault was it?"
That one has been answered, though not sure about test trials yet. "You are.". The person who enacts the tools to create the thing is responsible and the perspective copyright owner.
The AI does the work, you are responsible for it.
-
Sounds like you are in "Honeymoon" period.
Definitely, and enjoying that. I also feel more energetic writing my own code, because I don't have the feeling of despair caused by the mental backlog of important things I don't have the time to do.
When it fails, it fails silently. That's a worry. It's something they can fix, but it's not the first time I have seen it "cheat".
As I said in another thread. The "upside down cup" thing is real.
I have totally seen the upside down cup thing. But luckily we humans are good at spotting it an redirecting in the right direction.
Humans also make silent mistakes, forget the context, accidentally remove code, or even worse, forget something was important and remove it consciously. In my experience (introspection plus some professional programmers), we make a LOT of those mistakes. So the kind of oversight AI needs is not new, it's the exact same type of oversight we are already familiar with.
20x speedup in the work itself however changes the game - now the oversight can be chatty, instantaneous. With human beings, they work for a day, two. Then you need to allocate hour or two of your time to go through their work. Create tickets of findings. They allocate more time the next day, in a good case. In a bad case, after a month traveling or whatever.
I would say resilience, motivation and good logical thinking are the key advantages I am seeing.
-
I would say resilience, motivation and good logical thinking are the key advantages I am seeing.
In work the "Intro to AI" slide deck slide 1 was: Pair Programming - You and the AI.
If you look into pair programming it might 'resonate'. It requires BOTH of them in the loop. Often one is writing the code, manning the keyboard/mouse and the other is thinking, talking, documenting, or just "protecting" the mind state of their peer from interruptions. Sometimes they flip roles.
I have found myself in situations where it is me writing the code and claude driving direction. If it works... right?
Procrastination unblocker. For my 68000 project I had about 6 different options for memory model. Each of them would take hours and hours of painstaking fiddly verilog to "prove out". So instead I spend 2-3 months thinking about them. Trying to decide which to spend the effort on. I had dozens of chats with ChatGPT and it told me all my ideas where good. Then someone recommended claude.
My first session with claude, I gave it a 68000 soft core repo and said, "Can we make this boot?"
30 minutes later I had a running software 68000 on an FPGA. So then I dumped one of my ideas for RAM/ROM architecture, explained it to it for a while then it went off and expanded the verilog to include memory, address glue, latches etc.
So it "tried" one of those ideas in 30 minutes. So I went back and did them ALL, in one day. Then picked.
A lesson to learn though. It's behaviour is not static, it's in response to your prompt and context. Because I introduced my project to it as an "Exploration and discovery through computer history", it refrained from telling me anything about the Amiga, ST, etc. I had to ask it specifically after the fact... so how close did we come to the real historical computers and it would give me a quick break down of what bits I have which match with them. Then I would stop it again with a prompt like, "We are not building a replica, we are building whatever comes out of my head next." and it basically asked, "What's on your mind for next then?"
If you put a system prompt that says, "Answer all questions like you were a pirate.".... it will. These behaviours can become very "sticky", so if you apply ANY "role based" prompts, be careful as it can get fixated on them. In work someone started a prompt with: "You are a .Net expert code reviewer"
I went ballistic on it. "NO!" that is seriously dangerous. You are asking it to cos-play! It will!
-
I was pretty sceptical of LLMs too, at least for programming, and I still am. Although I will say, they are performing a lot better than I expected. Just don't ask them to write register level bit twiddling code for an MCU's PWM or timer peripheral. High level code is quite decent though.
Do you perform "vibe coding" by instructing the LLM about what to do step by step in the chat window, or do you create a "requirement document" (without the legalese and 'stakeholder' garbage) beforehand and give it to the LLM to create the code? I find the latter approach is actually quite good, since it first forces you to think about the problem at hand, and the architecture you would prefer.
Still I must say, LLMs have drained all joy I felt in programming (which wasn't much to begin with, but it was still something). It was a solitary activity that I could do myself and put in honest work. Now if I'm ever writing anything myself a voice in my head says, "you're just wasting time, let chatgpt write this!". I guess the expectation of higher 'productivity' is the most distressing part about this. Earlier I could happily spend a few days learning a library to then write a tool but now the expectation is to use chatgpt and have the tool ready without learning the library.
-
I was pretty sceptical of LLMs too, at least for programming, and I still am. Although I will say, they are performing a lot better than I expected. Just don't ask them to write register level bit twiddling code for an MCU's PWM or timer peripheral. High level code is quite decent though.
Do you perform "vibe coding" by instructing the LLM about what to do step by step in the chat window, or do you create a "requirement document" (without the legalese and 'stakeholder' garbage) beforehand and give it to the LLM to create the code? I find the latter approach is actually quite good, since it first forces you to think about the problem at hand, and the architecture you would prefer.
Still I must say, LLMs have drained all joy I felt in programming (which wasn't much to begin with, but it was still something). It was a solitary activity that I could do myself and put in honest work. Now if I'm ever writing anything myself a voice in my head says, "you're just wasting time, let chatgpt write this!". I guess the expectation of higher 'productivity' is the most distressing part about this. Earlier I could happily spend a few days learning a library to then write a tool but now the expectation is to use chatgpt and have the tool ready without learning the library.
On your specifics of bit twiddling low level code. This is often a training data problem. It doesn't know. Your options are to give it eyes through tools to go look, replicate, knowledge to understand/replicate from docs. etc. Sometimes you can get away with giving it a datasheet PDF and it's fine. Other times you try and give it a 10Mb PDF file and you get ... nonsense as most of the context was full of PDF level tokens.
Which leads to the second. An epiphany you may have is that you can do "recursive" things with the AI tasking. So for example, rather than spend the full cognitive load writing the spec yourself and giving it to the AI. Use the AI to help write the spec too. For that you need to "think about the problems." So do that with the AI too. Think outloud, let it echo back, reinforce, present options, amplify your thoughts. It writes the result out in a format you and it can understand a lot faster than you will.
While all that back and forth is in context, even the remark you made about hating a certain coding pattern off the cuff, is the best time to start engaging it to produce things. While it's symphonic (or is that sycophanic?) to your ideas.
Your last point does sit a little uneasy with myself too. For the majority of things, as a software engineer, the code is tooling, the idea is the vehicle. So having AI fill in the gaps and join up the dots for me is fine. However. There are also parts of coding that genuinely enjoy. When the AI generates it for me, I feel a loss. I lost that fun to have.
I think the beauty of the "software assistant" in your CLI basically being a chat bot means you can just reframe and redecision from one prompt to the next on whether you do, it does or you both do. I often split workload with it. "You do that part, I'll do that part. See you in a minute".
It's your tool. Don't be afraid to use it. Don't be afraid to tell it no.
-
Try having a conversation with Claude (CLI or chat) about KiCAD schematics and you will hit upon a problem.
Carry on that discussion and ask it, "How can I help you?" and it will offer suggestions. Usually it will offer to write python scripts to generate schematics, so it doesn't have to parse vaporously thin token delimiters like ((())) lisp S-expressions. This works for "Create a symbol for this component", but does not work for "Create me a basic RESET circuit for a Z80".
I have spend many, many hours trying, but other than "spot tasks" to generate a symbol, rename something across a netlist etc. it just produces garbage. Crossing wires, wires not connected to pins just left floating in space. Where it can't find components it pulls random ones off github and references them hoping you will install those libraries and ends with a KiCAD schematic and a long list of things YOU need to do next to get it working KiCAD. That part does NOT save you time.
EDIT: "A right in the moment" thing. My last prompt to Claude is pure laziness.
"Can we raise a PR targetted back to cxp-nn?"
Now I could do that in 20 seconds, but ... It's easier to ask it to. Besides, it will produce a nice comment and tick all the boxes, where as I usually skip the fluff.... fluff looks good though.
Yesterday I got feedback and on prior PR. The document generated was not aligned well enough with the direction, 3 bullet point comments to address. I tried to resteer claude myself and it seemed resistive. I could have spent another few prompts "talking it around", but instead I did:
/clear
Claude there is a PR assigned to me with comments. Can you access it so we can look at it.
... 20 seconds...
"Your team mates comments are fair, here's why.... 500 words... I have generated an updated document for you to review. Shall I commit this and comment on the PR that these have been addressed?"
So I spammed "Yes" until it completed the whole task, on it's own. Pretty terrifying actually. PR was human reviewed and accepted.
-
I did a large algorithm development/research/upgrade work, and my observations are as follows:
I tried "for fun" if Claude can do this "on its own" with a relatively small prompt and --dangerously-skip-permissions left to work without supervision for a weekend. Basically - upgrade our battery usage optimization algorithm (pretty complex multi-dimensional optimization problem in a very memory and CPU constrained embedded environment) to utilize 15-minute price granularity, with just one small hint about the first new piece being added in the front of the algorithm. That piece it nailed of course, but for the rest, it didn't really work out. It made the massive number of changes needed to update the whole 5000 LoC mess of mine, as later turned out, only with 3 bugs for me to find, and that alone is of course impressive. But as it didn't have the intuition of the algorithmic changes really needed, it kept doing wrong type of modifications to push the cost down: when the initial conversion worked worse with 15-minute granularity, Claude then basically added ad-hoc modes here and there that made it operate like the old hourly algorithm, finally reaching into roughly same performance and calling it a victory.
That was as expected. Good: not many bugs. Good: no "cheating" to improve metrics; no totally insane decisions. Just poor understanding of the big picture - what can be gained and how.
With a human in loop, it took then two days to get the results where they should be: 15-20% more savings with 15-minute spot prices than with the hourly algorithm. My gut feeling was that 10-15% is the limit!
The development process is pretty nice because I know exactly what I want, and can describe it with prompts much shorter than this post. At that point, AI is a rubber duck: I ask for comments, but pretty much hear back what I said. Not much value, but still something. Maybe mostly that it replaces napkin calculations and actually calculates.
But real value comes after that. I can ask it to write an index selection algorithm which finds local maxima. And it does excellent job with this little prompting. The whole point is, I don't have to design an algorithm to pick local maxima, or even tell it what local maxima are. I can ask "run the domestic sim" and it understands that I mean after these changes, then understands to show old baseline simulation vs. previous version vs. current version.
It does not believe though that a simple bug causes the result to be unimpressive (like, 1% improvement). It doesn't have the gut feeling that we should be seeing 10% improvement. I had to still prove to it the presence of the 3 bugs.
But then it adds exactly the CSV logging, command line option parsing etc. you need in 1 minutes, vs. 1 hour if you write it yourself. Then it writes you an absolutely fancy HTML CSV viewer with exactly the zoom, data analysis, emphasis features you need, it does this with an hour of iterative prompting, you could do it yourself but it would be a project taking a full week especially if you have to start experimenting with graphics libraries. And this tooling enabled me to find the 3 bugs that Claude had very hard time finding.
It's really the same development process I would have had if I had endless time and no disruptions. It would have been doable in 2-3 weeks of calendar time. Last time I did it that way. But now it's really hard to concentrate on getting some index selection for loop right. With vibe coding it, an interruption won't be a problem at all. I'll answer the phone and continue. Human's context window / working memory is excellent for tasks like this. I know exactly how I want my algorithm to work. But getting the smallest implementation details, like indexing, interpolation, formulae, loops & conditions right is hard work. This is exactly why libraries and high-level languages are so popular: you just instantiate something that does the job for you. Except with libraries you have the square peg in round hole problem: you often can't find exactly what you need. LLM programming solves this, it makes those difficult implementation details extremely simple so you can focus on the big picture and ask whatever you need.
And in this project, the 1Mtoken context window update (from 200k) also clearly was a significant benefit. Not much autocompacting getting into way - and it's nice when it actually remembers "oh, but you said you saw charge power of 1.245kW at that timestamp" - we humans remember much more context, but lose accuracy. AI loses it completely when it goes off from the context window, so then it relies on "note-taking" type of activity. But 1M context window is quite respectable, and it's advantageous to see it remembers details "right away" without having to rely on a "cheatsheet" because the problem with those is the same as with humans: how do you know when to look up for something, and from which cheatsheet?
I'm still very satisfied, and without Claude in my toolbox I would be still wondering when I will turn my phone off for 2 weeks to do this very necessary algorithm work. Now I did it in parallel with all the other work I have to do.
2-3 weeks full-time concentration --> 2 days 1/3-time chatting = over 20x productivity increase. So that is true. Fanboys were right.
-
You found the same indicator as others and myself.
"Human in the loop" is today way superior to the "slot machine prompt".
Outsource the cognitive labour, not the thinking. :)
-
I tried "for fun" if Claude can do this "on its own" with a relatively small prompt and --dangerously-skip-permissions left to work without supervision for a weekend.
Did you run it in a VM?
I keep getting tempted to try it, but have only done a quick test in a VM. The problem is I'm using the USB port in my STM32 project and it disconnects every time I flash an update. My Linux VM does not auto detect and pass through the newly connected USB, and has defied my brief attempts to make it do so.
So far in my work it has asked for nothing I've had to block, but can I trust it enough to let it loose.
-
Even without the VM it's fairly "trustable". If you stay in the loop.
It will ask you for every single invocation of a tool, edit, command etc. You get the familar options of "Yes", "Yes and don't ask again for simmilar" and "No". ALL of them come with a "Tab to add" feature, so you can say, "Yes, but put the C file in the following folder @thisfolder/"
It's not SAFE though. It can be VERY pro-active and if let loose on a long leash with a "goal orientated prompt" it will go like a dog with a bone. When it runs out of solid and sane options, it may then start moving towards less sane and more... dubious methods to achieve it's goal. The risk here is, it's living in your prompt space and your project space, it has no idea about all the other projects and applications around it. So if deleting the entire database server data files and re-initing the entire server will let it progress.... guess what? Yep. You want to paying attention when it proposes that one, if it's not a dev DB.
A classic development pattern it has adopted is that "work ends with a push to master branch". So repeatedly in work it gives me shivers when it proposes:
"merge this to master"
As the next "hit enter and find out" suggested prompt. "Hell no!"
-
Twice now I have submitted work in form of a pull request containing a claude produced (with me in the loop) analysis document for a project. Then got comments from the human reviewer and decided to throw it directly to claude by resuming the conversation/context used to generate the document and prompting:
"There are comments on the PR, can you check?"
... say Yes a few times....
Claude proceeds to address the comments in the document, recommit it, push it and then add a comment to the PR detailing the response changes and assigned it back to the reviewer. It was then accepted.
This could however go horrible wrong, especially if you have "YOLO mode" on (dangerously skip permissions).
-
I wouldn't either call it "vibe coding" if you are actively instructing, iterating, checking, reviewing, doing changes yourself and have the understanding of how the code works.
But the copyright thing might be tricky. I've asked LLMs to check for copyrighted stuff, that the code doesn't contain known code or copied libraries etc. But can you trust it?
If I'm publishing some of the code (as open source), I will certainly add some information how it was developed. Some people use e.g. "co-authored-by: gpt-oss:20b" etc. I'm thinking on something like "This project was developed with assistance from AI tools. All code has been reviewed and modified by the author". But this depends on how much of the code the LLM was involved in. Is it even necessary if you have used the LLM for only parts of the code, as a helping tool? There are many open questions still.
I would be very careful to release something serious, yet. We've already seen backlash in some open source projects.
Now proprietary code... easier if you work for a big company, just use their tools that they pay for and follow company guidelines and rules.
-
I tried "for fun" if Claude can do this "on its own" with a relatively small prompt and --dangerously-skip-permissions left to work without supervision for a weekend.
Did you run it in a VM?
Of course not. Real Men^tm take risks ;D
I don't understand why everyone suggests VM though. This is the question of classic separation of privileges, and Unix-y kernels and filesystems do that perfectly. In other words, different user would be enough, no machine (virtual or real) needed. Unless you want to experiment with some special case requiring root privileges. Which you rarely do.
-
I wouldn't either call it "vibe coding" if you are actively instructing, iterating, checking, reviewing, doing changes yourself and have the understanding of how the code works.
Yeah, the original definition is that you don't work with the code - you don't write it, you don't review the CODE - but of course you would be checking and reviewing the PRODUCT, just iteratively developing it in natural language.
And that works, too.
Of course, I also check the code and review it, but not every line it with extreme scrutiny - if it works, passes verification and is useful for the intended purpose, then reviewing the code itself isn't 100% mandatory part of the process. It's a cherry on top.
Then again, I have already done work on server infra in Elixir, and training ML models in some Python libraries, and I don't have a freaking clue of how you are supposed to work with these languages/concepts, and neither do our professionals, which is clearly why I have to use Claude to do it. Now this is clearly vibe coding - I might take a quick look at the code but not really checking it, but trust testing the result instead. Just like I had to operate with our human professionals, with much higher bug rate and much slower iteration. Trying to micromanage misbehaving human beings and their code which doesn't work, in a programming language I'm not familiar with, has been a huge pain point. Now Claude does much better job and if it fails, I just tell it and we don't have to fight about definitions and who's right and who's not, it just fixes the code. It doesn't even say "sorry" like the classic ChatGPT does every time it makes a mistake. Great!
So, "vibe coding" is just "leading a software project", really. And clearly it works. When working in a field familiar to you, you tune down the "vibeness" and it becomes more like pair programming. But when needed, full vibe works too.
-
But the copyright thing might be tricky. I've asked LLMs to check for copyrighted stuff, that the code doesn't contain known code or copied libraries etc. But can you trust it?
I have used plagerism tests on some my claude assisted projects and got very low scores.
I think it depends on what you ask it for and in what "chunk size". If you prompt it with a "make me a thing" and it finds a perfect example of a thing in a tutorial, then it may just lift it.
On copyright of code. It's not as clean cut as you think. Copyright is about the "work", the bytes of code, the 'performance', the 'release', the actual artefacts. It does not protect ideas and concepts. For that you need to go to patents and that's a whole different ball game. Copyright is also removable, all you need to do is modify it and pass the sense test of "Is this now a new work? Is it distinct enough now?" and you now own the copyright. Many, many music "cover band" releases have gone this way in court. So nobody even bothers sueing cover band artists anymore. The performance is copyrightable.
Again though another layer is the "License". It can extend copyright, or attempt to, but a violation of a license is a contract law thing, not copyright.
Genuinely part of your goal as a software engineer is to reach towards consistency. You literally want your code to all look the same. You even want it to look the same as everyone elses. Use the same patterns, techniques, style and rules. This makes it easily transferable between engineers and allows engineers to work across the wider code base more freely.
So code looking nearly identical across code bases, across companies and across sectors is not only normal, it's expected and desired.
-
Coding and build software actutally changed recent 2 years, and will be much more easier and cheaper. Use AI to do so earlier and benefit earlier.
-
Approx a month of vibing, and done:
* Two years worth (observed human ratio) of backend/UI development (or maybe 2-3 months worth of real expert ratio, if we had one, but we don't). Originally not my job, but had to do it because it was stalled
AND
* 2-3 months worth of my own job (if I had full time to dedicate for it, which I didn't) - algorithm work, firmware plumbing, device drivers, bugfixes and various improvements etc. Was nearly stalled.
AND
* the usual customer support, investigations etc.; in some cases AI helping tremendously, in other cases, just freed me up.
So, 20x productivity I guess?
Biggest gripes:
* Having to share everything, including sensitive proprietary work, to some random megacorporation.
* Outages. Yes, Claude calls in sick like human beings. Although usually less than 2 hours at a time.
* API rate limiting. Not usage-limited even once after upping to $100/month service (after the first usage limit with $20 service). Some stupidly aggressive rate limit triggers if you accidentally fart 3-4 times within the same minute. Poorly tuned abuse filter or something.
How it compares to a human being?
It's a weird mix of experienced domain expert in all fields, an eager junior, and pathological amnesia patient, with a trained incentive to stop to prevent people from giving it an infinite unsupervised job clogging the servers. This is sensible because it's hard to make it do good job unsupervised for long anyway - it may feel like a great idea to try anyway, but I can imagine the cost of operating the servers just to do work which is mostly thrown away is unbearable. Human in the loop improves the quality and offers rate limiting. So now you do year's worth of good work in a month, instead of year's worth of unusable crap in 3 days of expensive server time and energy.
While humans have this slowly adaptive large neural network and super crappy working memory, AI has non-adaptive large NN which learns nothing from your project, and then a huge working memory which mostly beats human's equivalent (the one which remembers 7 digits) 100-0, but it has to compensate for the lack of the higher-level memory by loading lists of notes into the working memory, which is a nasty workaround but works somewhat. But the only real solution to this massive gap is to use the human in the loop.
Mostly it's like new fresh hire each session, who reads through the code base (not every line, but doing very good picks and spot searches) every time you start to do something. Then for the next little task, it forgot everything. But that's OK-ish because it "learns" everything again in <5 minutes. Except those things it forgot because of the pathological amnesia and that were not in CLAUDE.md. You can't put everything there. Human's longer-term memory is superior.
It needs guidance in all bigger decisions requiring serious creativity. It does those correctly only by sheer luck. However it can catch your high-level intent from just a few words and is really good at filling blanks and figuring out the details from that. Of course, assuming that you know what you are doing. The human in the loop is absolutely crucial and I can totally see how easy it is to produce total crap. But then it quickly fetches, analyzes, visualizes the data you need for decision-making.
It's quite good in logical thinking. It can see the data flows, the bigger picture, and then get small details like write you a sort algorithm or whatever, right. It leaves crap behind, but clean that up when you request so. It refactors a few hundred lines and improves the algorithm, makes 0-2 bugs where human would have done 3-20, but writes necessary tests and fixes the bugs then.
So-called "hallucination" happens but isn't one of the biggest problems. It's rare enough. By hallucination, I mean that for example it told me point blank that there is a gap at some timestamp in the CSV. When there wasn't. It didn't read that part, just hallucinated the idea of there being a gap. But this only happened once so far. If it happened more, it would be a real problem.
This feels now like an irreplaceable part of the development flow. If I lose it, nothing bad happens per se, except it's then back to the good old non-productivity. This thing has restored hope that we can keep maintaining our service and finally reduce the ever-increasing technical debt - any by that, my meaning is totally different than our earlier human developers. To them, "technical debt" was about dreaming about huge refactors and new development processes. To me, technical debt was about not being able to serve customers: fix bugs and add important features. I hate the word "feature creep" specifically. Software exists to do things. We need to do those things. Those are important. Not all features are feature creep - they need to be done. Another word I hate is "technical debt" and related babble. We never were able to fix bugs because humans look at the code for 45 minutes and then say "it requires completely redoing everything". Claude looks at it for 5 minutes and fixes it in 2 minutes, and then it's deployed to production. If it's truly difficult, then it does say so.
-
My perspective is that one can start with vibe coding and use it as a learning tool (among other methods of learning) but should ultimately be self sufficient. Having a car does not mean one should not walk or run. Car has its use but imagine how your legs will look like if you never use them. Similarly over dependence on AI tools will not produce good programmers/engineers, instead not using our brains will make us dump not smart.
These AI companies want to reward ignorance because it works in their favor, but at what costs to humanity? May be we are okay being dumber, who cares really. But if that happens please do not tell we are making progress as a species.
-
The weird side issue is put best with an analogy.
In a software company or consultancy, maybe only 50% of the staff can actually code. The rest are non-technical staff.
Pan this into driving a car. We have 50% of staff who can drive a car and 50% of staff who never tried.
Claude is like a self-driving car. Like a modern one that is only may be 80-90% 'complete' as a driver and needs steering frequently for best results. The 50% who can actually drive are doing fine, but the 50% who can't, who has just discovered the freedom of a car are now trying to tell the 50% who can drive how to operate a car.
They typically come to demos and are very proud of their "Claude took me to the shops for milk" demo. When engineers like I are frowning and wondering... "When do we ever just drive to the shop and make a 7 figure revenue for the company? How about you try and get claude to build the entire supply chain for the supermarket and lets talk."
-
My perspective is that one can start with vibe coding and use it as a learning tool (among other methods of learning) but should ultimately be self sufficient. Having a car does not mean one should not walk or run. Car has its use but imagine how your legs will look like if you never use them. Similarly over dependence on AI tools will not produce good programmers/engineers, instead not using our brains will make us dump not smart.
These AI companies want to reward ignorance because it works in their favor, but at what costs to humanity? May be we are okay being dumber, who cares really. But if that happens please do not tell we are making progress as a species.
I understand the concern. Specifically, my opinion has always been that to-be software professionals should be exposed to lowest level programming just to widen their understanding of what they are dealing with. Yet, that usually does not happen. What we actually see is 95% of professionals being very close to surface - just apply patterns they think are familiar or just trendy, without deeper understanding or analysis what actually happens.
This has lead to the current software crisis - poor performance, bloat, bugs, vulnerabilities.
I never had a solution for this. I can say, "avoid overuse of libraries; they are not magical silver bullets", or, "be careful and analyze before acting", or similar, but this is not helpful, and I'm not being liked from saying opinions like this out loud.
I don't have a solution to this because software is truly hard. It's not only difficult, it's time (and coffee) consuming. So when I say 95% professionals are not very good, it's not meant as a critique against their cognitive skills or attitudes, rather, their resources are just limited, and scaling horizontally by adding parallel programmers is colossally difficult.
Turns out, Claude already mostly does better choices and shows better ability to implement them than significant % of professionals, and it does it much faster, and with much less communication delays. So it's a perfect coworker / subordinate.
And then again, it also still has massive blind spots, so requires supervision and co-operation, just like human beings. But I see that it is useful despite your skill level. If you are a beginner, it can help you learn. If you are overly confident middle manager who doesn't know how to code, it can implement wrong things for you, but then again, without it you wouldn't be doing anything on your own, and it can still make some right choices, too, likely still much more beneficial than harmful. And if you are like me; you mostly know what you are doing; sometimes lacking a detail, but enough to understand the consequences of filling that in, you get excellent results out of it.
So far, the effect on my brain has been solely positive. I have not forgotten how to code; actually I code together with it, now. I also feel faster now; looking at how it works, I have picked up good patterns especially on command line usage.
But sure, the risk might be that new generations do not learn how to code "the hard way". Then again, history has shown this kind of fear wrong many times before - radio didn't kill literature or newspapers, TV didn't kill radio, rock music and video games and didn't ruin the youth, and so on. So I'm not 100% sure, but I'm pretty confident that everything's fine this time, too.
But yeah. Now I'm learning every day. Before this I only had to rely on our professionals to fix something or not fix something, usually with long delays and a lot of stress, and at best, mediocre communication; sometimes outright toxic communication. Now with AI I can discuss the problem to gain understanding of what is happening and why, before committing on a solution it proposes. This is fundamentally different compared to working with human professionals - you pretty much either have to accept what they give you, or become a micro-managing nasty boss who is hated for questioning everything (and that's just super slow with humans).
-
I thought hell would freeze over or turn into Finland before Siwastaja became a Claude fanboy. How the tables have turned.
-
Just in time for Claude to slip into senescence (https://github.com/anthropics/claude-code/issues/42796#top). :(
They are having serious problems lately, almost certainly due to too many subscribers using too much compute. Anyone who starts using Claude today will probably come away with a less-favorable opinion, unfortunately.
-
Just in time for Claude to slip into senescence (https://github.com/anthropics/claude-code/issues/42796#top). :(
They are having serious problems lately, almost certainly due to too many subscribers using too much compute. Anyone who starts using Claude today will probably come away with a less-favorable opinion, unfortunately.
I became a fanboy just in time to see some of the "old one" and notice differences lately. Specifically, in last 1-2 weeks I have seen that now instead of random "Overloaded" or "Rate limited" API errors, which were annoying and really performance-degrading, now we have sudden mood jumps where it just randomly performs visibly worse. I don't know which is worse, refusing to work at all, or working in unexpected and crappier ways. Clearly the pricing model is unsustainable.
As I have said - I fear enshittification is the biggest threat for AI. Silver lining is that current AI workflows have very little vendor lock-in, so competitive situation is healthier than e.g. with social networking sites, where enshittification is at its worst because of very high cost / near impossibility of switching.
But I can totally see I'm working with Claude the same way as I worked with human professionals: together with it, I try to figure out simplest way to solve the problem, to get the job done, and I am always ready to flex on the original requirements to make the problem simpler. Now reading through your link it seems many are expecting Claude to perform better than that - to solve problems that are difficult but not impossible, without micromanaging, "helping it out" by accepting solving a different, simpler problem instead. Fair enough - that would be fantastic. It's also the level of performance I have never seen from any real human being co-worker - but maybe it tells more about the human professionals I have had to unfortunately trust.
My workflow also seems to be: ask it first "how you would do it", evaluate it together, then say "do it" - this pattern forces it to read the files and do the planning. If I read that correctly, it was better at doing this autonomously? This is something I haven't seen? So it has been even better?? :o
Typical example from yesterday: need significant fix/redesign of an algorithm. Ask it what needs to be done. It gets it a bit too complex, missing an opportunity for reuse of earlier calculation. Point it out. It gives an estimate that now it's 1 day's job. I totally agree, it would be 1 day's job for me. I say "do it". 13 minutes later, it's finished and tested with a simple test case I didn't specifically ask for. I'm very happy with this performance level, really: 15 minutes chatting and 15 minutes of implementation = day's job. But it can't really work completely independently.
-
Enshittification is the #1 concern I have about all of this stuff. It is profoundly unsettling that the model I am running today will not be the same as the one I run tomorrow, and that the one available tomorrow may well be worse in any number of aspects I consider important.
If we were sharecroppers on Microsoft's plantation before, we are going to end up outright slaves to Anthropic and the other top-tier inference providers.
I have been waffling, and will continue to do so for some time, about putting together a stupidly-expensive inference platform and throwing in my lot with the /r/localllama cult. I suspect that for normal development tasks, the tool harness is now actually more important than the model itself, or will be soon. The current open-weight models may be good enough if we learn how to leverage them well enough.
-
If we were sharecroppers on Microsoft's plantation before, we are going to end up outright slaves to Anthropic and the other top-tier inference providers.
If we want to maintain the increased speed and quality of software development we get used to. But in absolute terms, it's harmless - nothing prevents us from reverting back to how software was developed since the beginning to ~1 year ago.
Of course, there are risks - like, with AI capable of dealing with more complexity, it might suggest complex solutions we can't keep running without AI. For us, it's kind of opposite - human beings came up with things that are difficult to operate without AI, and now AI helps us operate that human-generated complexity. (And often, that complexity is unavoidable - software is difficult.) If we lose AI, then we are just back to square one, which feels worse because human brain is a derivator with alarms ringing with any negative change, but absolutely speaking it's just the same it used to be, then. With possibly a lot of good work done with the help of the AI while it lasted. With the risk that it proposed also hard-to-maintain work which was accepted by users. That is the thing to keep eye on, don't let it happen.
-
If you think things were bad with over use of libraries and frameworks by those of lesser core knowledge... just wait till you see what they do with AI.
The major risk is the affirmational, confirmational, sweet talking, sicophancy and overly confident aspects. People who don't know just lap it up and believe they can suddenly do software.
I'm not referring to anyone here. Siwastaja is in his honeymoon period and may he enjoy it while it lasts.
Yes it's powerful, I'm not doubting that. When I seen it coming I maybe naively thought "Well we will just produce more work, job will be easier so we will deliver like gods, it will great." That's not how it's going down.
If I use claude and produce good output in a report or analysis, or code, or spect... et.al. I will get asked, "Can you capture what you did in a skill and put it in the repo?"
This "repo" is rapidly expanding suite of skills, plugins and "mega-prompt parallel agent workflows".
There are a few problems with this approach.
1. The LLM is not deterministic. Run the same prompt stack twice and get different answers because some of the weightings have a randomness too them deliberately.
2. The part you are trying to "capture" is my part, the part it can't do.
3. More prompt scaffolding does not make it better or deterministic, it makes it brittle and confused.
4. There are no objective metrics that I have seen to say one prompt scaffold is objectively better than another.
Oh and the one that I believe will burst their stupid little bubble.
5. A single run of some of these systems will cost you $500. How I got my output on $8.50 is not something adding another prompt layer will achieve.
EDIT: Forgot one.
6. Why on earth are software engineers even attempting to train claude to replace them? Why are they even attempting this? They should be showing clear, hard empirical proof that claude works best with good, skilled, experienced engineers in the loop and both are required.
-
All this talk of AI makes me thinking of that scene from Person of Interest (2011)
The one where the AI, which had been programed to erase its memory every night and start new the next day, had decided to hire an entire office building of humans to print out all its memories in binary on paper each day and then scan them in then next day. Just so it could remember.
-
Siwastaja is in his honeymoon period and may he enjoy it while it lasts.
Two factors: honeymoon which I don't deny and which wears out, but bigger factor is, I have never had a chance to work with really good team of software developers, at least not long enough to see how daily life with them would turn out (short occasional sharing of project responsibilities do not count). I'm amazed simply because Claude saved my ass. I have a project, real, significant project, with paying customers, and some society-level impact. Since I'm a co-founder and shareholder, of course I'm interested to see it succeed. And that project has seen serious neglect for nearly its whole existence from our inability of finding really good developers. Yes, our fault - we need to hire well. Which we didn't. But it was Claude that saved us.
But here's the bias: this might not mean that Claude is so exceptionally good. Rather, it might mean that the professionals we worked with earlier were exceptionally bad. Our mistake.
And maybe relying on a service which costs us $100/month but is effectively subsidized, i.e. costs more than that for Anthropic to provide, is our current mistake. But then again, I don't believe I'm actually consuming $10000 worth of resources. I'm not a heavy user at all; I'm not launching 50 agents in parallel to do a job. Like, my max 5x plan varies between 0% - 50% of usage. Never exceeded the usage limit on that. My poor experiences of trying leave it unsupervised for a large, complex task means I don't do it. Hence not much crap work to throw away either.
I hope having the AI in the loop helps us to get rid of poor and mediocre professionals, and replace them with machines, while developers with good skills and most of all, good technical intuition and deep context-understanding, get into more authoritative position, being bosses to the AI. That seems to work very well for me right now.
-
If you think things were bad with over use of libraries and frameworks by those of lesser core knowledge... just wait till you see what they do with AI.
I don't think AI is going to magically cause crappy programmers with poor design skills to turn out well architecturs large software packages, any more than java or other memory managed languages did. But it's a force multiple that isn't going to turn good programmers into power houses and it's going to let mediocre programmers produce a lot more and avoid a lot of common mistakes. It's not going to make them great programmers, and I do see the long term risk of having an even higher volume of mediocre programs out there. But I think AI is going to make it more tractable to fix the ones that become too load bearing. One of the most satisfying things I did recently was take some scripts written by someone who is a competent hardware engineer but just not a great coder and rewrite them to what I would expect from a mediocre SWE. It wasn't perfect but it was a huge improvement, and gave me a more usable base to add some important features and retarget for a new device.
The major risk is the affirmational, confirmational, sweet talking, sicophancy and overly confident aspects. People who don't know just lap it up and believe they can suddenly do software.
That has not been my experience with the coding tools I have used. I'm not sure why. It could be the tool design, my working method or the structured nature of programming tasks. Whatever the reason I see far less of that compared to just using the chatbot interfaces.
My main worry is about deskilling, and that as people come to rely on AI tools they may become less able to handle the situations where the AI tool fails. We do see that once people start using these tools they very quickly become dependent on them. Literally addicted. But I have for sure seem people spend 10 minutes fighting to get the AI to do something seemingly simple that would have taken them only 2 minutes to do themselves a year ago. So far this is vastly outweighed by the time savings. But I can see if enshittification makes the products worse over time as people become more dependent on them, that could flip.
-
I'll never understand the whole "deskilling" concern. As long as there are some people who are getting great results with AI models and others who can't seem to succeed with them at all, there's a skill gradient that can be climbed. :-//
If that's not the case in the long term, and we all end up as 100x engineers, well, mission accomplished, I guess.
-
The major risk is the affirmational, confirmational, sweet talking, sicophancy and overly confident aspects. People who don't know just lap it up and believe they can suddenly do software.
That has not been my experience with the coding tools I have used. I'm not sure why. It could be the tool design, my working method or the structured nature of programming tasks. Whatever the reason I see far less of that compared to just using the chatbot interfaces.
I was referring to "non-techies" mostly.
When I work with claude, I stay in the loop. I am aware of what it does well and what it does poorly. I "chunk size" and taylor offloaded work to suit that. I do not offload the thing it CANT do, which is thinking. (Thinking in the human sense not what LLMs call thinking)
If I am "clauding" to produce something, the ratio of "Yes, No, Something else" responses I give are about 60/30/10%. I say yes 60% of the time, but critically I say, ,"No" or "Something else" nearly half of the time.
In other sessions, as a test I have just spammed Yes. Nothing ever works.
As you will have heard before. "Writing the code was never the hard part.", software engineering has very little to do with code. It has a lot to do with converting business desires and requirements into working systems. The coding part is just the grunt work at the end. Most good software engineers get bored rapidly of this part, picking the challenging and complex parts for themselves and running a mile from the 90% of boiler plate BS.
The problem is... only software engineers understsood this. Management still see software as code only. They never seen or measured the actual translation and modelling layer inside the SWEs as it's not that visible and management don't read design docs, technical colateral etc.
The industry is painfully learning this and in my neck of the woods it doesn't seem fast enough. Still a lot of walls to hit painfully here, so I am running on slow with the brakes on so I can not hit the same wall.
To boil it down:
Direction. Clear goals. A known "end game". A testable outcome.
WIthout all of these you are literally doomed regardless of whether an AI does it or a human does it. The trouble is. AI itself can't help with those big decisions. It can inform and help, but decisions are not it's strongest suit. So management using AI expecting it to guide them is a large part of the "popcorn" moment I'm having.
Give an engineer 9 out of 10 things he needs and he will immediately start asking for the 10th and making ultimatums about it shortly after. "Information or no code!"
An LLM however, might just hide that and statistically substitute it for something that looks appropriate by proximity. You have to be there in the loop and spot that and stop it and ask, "How did you come up with that?" or just tell it what it should be, or you guess rather than it.
-
I was pleasantly surprised when I started using claude code about the fact that I had to basically do zero hand holding about the internals of the code. I just had to describe in detail what I wanted and what I wanted the code to behave like. No info about the classes, functions, architecture, variables, third party API etc.
Its very, very good at taking all the correct decisions to get what you want. You don't have to prompt it about the inner workings at all. At least when you're making it write web related CRUD code (which is rare for me, but daily bread and butter for a lot of people globally). It did the job wonderfully. But you do need to be deliberate in your expectations and explain all the behavioural quirks of the software beforehand. Changes get more error prone the longer you go on in the session since the underlying architecture may not be suitable to begin if you ask it to bolt on a dozen different features afterwards. Its the same in the real world with people writing the code as well, so that's not too surprising.
I have also noticed it helps a LOT if you can point claude code towards a simple hello world embedded project or a blank android studio project for app making, instead of letting it start from scratch.
If LLMs can figure out their operational cost issues, and are here to stay, a lot of people's jobs are going to get devalued or lost, webdevs worst affected.
-
If LLMs can figure out their operational cost issues, and are here to stay, a lot of people's jobs are going to get devalued or lost, webdevs worst affected.
I think we are going to learn something very different. People are feeling sorry for the wrong people.
Green field cookie cutter web APIs and stuff are just that. Any first year junior would do a fairly ok job of it. "Coding was never the hard part" is hinted at in your post. The bit where, when adding features later that it starts to come apart as the prior design choices limit you later.
Two choices:
1. Your understanding of the bigger picture holds that and gives claude smaller sub tasks and guides it without necessarily telling it about the overall picture.
2. Expand your prompt and spec to exhaustly describe the overall system in all it's detail.
If you attempt approach 2, your API cost will sky rocket, you will spend days or weeks writting your "perfect prompt" and you then find that it doesn't exist. It doesn't make the problem go away and produces the same results that you got. At some point the context and constraints have a negative effect, they over constrain or contradict each other. They also suffer the "low attention in the middle" problem. They are highly sensitive to things in the start of the context and highly sensitive to the later parts of the context, but almost deaf to the middle. This is not a "behavioural" pattern of AI, it's a behavioural pattern of human language and processing.
The thing we are going to learn and are already learning. While claude cannot replace an experienced engineer, only empower them, claude can 100% do middle management far better than most middle managers.
They don't even see it coming. They are celebrating about the automations making their jobs easier and easier while worrying about us engineers losing our careers. It will hit them like a train in a year maybe.
-
Also what we will see are very small companies which suddenly ship serious, large projects like bigger companies. Like, 1-5 sharp people that get together and plan and execute a serious project.
With AI, some of the hardest problems go away: project management / ticketing / micromanaging tickets / prioritization of tickets. Development becomes responsive. Your brain gets to a point where it not only had this great idea, it processed it enough so that it's ready to be realized:
Old flow:
* Spend 1 hour writing a perfect ticket
* Shuffle the priority of that ticket for 2-4 weeks
* Get a developer assigned to the ticket
* Developer does not get the intent of the ticket, and implements it halfway
* Wait for 3 days to realize the ticket is closed
* Test it
* Talk with the developer, except you cannot, because it's weekend now
* Wait for Monday
* something important comes up in Monday
* Try again on Wednesday
* Developer is now sick
* Finally talk to the developer. Discuss about changes
* Get the ticket opened again, and rescheduled for improvements.
* Except that now we have those 5 new high-priority bug tickets that came in while the developer was sick
* ...
* After 2 months, the thing is finally done halfway to your original idea and somehow limps enough so that you don't want to touch it anymore
* Hate your life
New flow:
* Spend 5 minutes writing a prompt catching the key parts of your idea
* Grab coffee while Claude explores your codebase (~5 minutes)
* Chat about it for 20 minutes
* It suggests an implementation and estimates it's "two week's job"
* Say "do it"
* Come back 20-40 minutes later to see it finished, tested and running, with 2-3 remaining bugs that take another 20-30 minutes to sort out
* Iterate back and forth to really get it to the goal - not just "good enough", but what you actually wanted.
* The new feature is in production the same day you imagined it, and is very close to what you imagined, not some quackery of the feature you wanted.
So why would you hire a human professional developer at all? Why would you have a ticket system at all? Why would you enter the prioritization-of-tasks game when you can get jobs done instantly. So we can now fall back to our human brain's internal prioritization / ticket system which is actually great, it holds maybe 5-10 unfinished things and nags you subconsciously about them. If you can't get them done, you get stressed. If you can get them done, your stress relieves. The attempt to NOT do the job, AND to relieve the stress, by using ticket systems, didn't work for me. Customers call me "why the fuck isn't this working", and "we have a ticket about it" is not an acceptable answer.
But you sure do need high-level designers. This only works if you are capable designer, and actual programming experience is probably very valuable too for correct type of intuition. Having "an idea" alone isn't enough - the idea needs to be practical, sensible, implementable, and you need correct understanding not only what you want, but how you want it. AI is happy to fill in details, like explain you how different databases work and which would be most suitable for the job, but it still needs general direction.
-
I got an AI to help me make a USBTMC interface for my GPIB device, and it absolutely crushed it. :-+
-
One thing I would encourage is finding claudes limits and wallowing in them for a while.
When you are proceeding as siwastaja describes and it doesn't work. Especially when the AI cannot run the tests and iterate itself. Like when deploying to an MCU it has no debug interface on. It say's "That should be read now" and you flash it and get a brick. "No it doesn't work. No feedback, nothing."
You can get into situations where no matter how many times you bring it back to claude it will not help. You are on your own again.
Until you hit these moments and where that limit lies it seems infinite in ability to you. Once you have cornered it and broken it at will, you feel a little more in control of it out of neccessity.
Best advice is to keep claude tasks small enough chunked that you can verify and validate each step in the loop. Letting claude rush ahead on a much larger compound task is where things get rough. Espcially with "dangerously-bypass-permissions" turned on an "end to end" prompts. All of these mistakes, compound non-deterministics, upside down cup, small vs. far away, can't count, low attention in the middle, are all still in there, you just hide them. That or you save it all up for one big massive review process at the end... which we all know will not get done.
Deliver manager in work, kind of agreed with me when I said, "We are generating output faster than we can review.", she said, she sent a spec to the customer, generated by clause 3000 words. In 3 minutes the document got a thumbs up and an "Approved"
She did not take it. She called them. As she pointed out... "this is where we do well. This is where we show the rigor and responsibility. Nobody reviews a 3000 document in 3 minutes. So I want it properly reviewed and signed off on."
-
One thing I would encourage is finding claudes limits and wallowing in them for a while.
When you are proceeding as siwastaja describes and it doesn't work. Especially when the AI cannot run the tests and iterate itself. Like when deploying to an MCU it has no debug interface on. It say's "That should be read now" and you flash it and get a brick. "No it doesn't work. No feedback, nothing."
You can get into situations where no matter how many times you bring it back to claude it will not help. You are on your own again.
Why not ask Claude for ideas how to test / add visibility?
The problem in programming is always the same freaking visibility / testability thing. That's what I always struggle with my own projects, that's what every single professional I have seen always struggled. Writing debug outputs is work. Reading the logs is work. Adding more debug prints is work. Creating an interface to make debug prints - or something similar - possible (e.g. on embedded system, or in some cloud cluster which hides normal interfaces) is even more work. Parsing, analyzing, visualizing logs is a lot of work.
Claude resorts to exact same mistake every human being I have seen, myself included, does: read code randomly, or even worse, try random modifications, because adding visibility into the system to figure out what it is actually doing is so tedious. Luckily AI is quite good at reading code and guessing, still not enough so you need to steer it a bit into suggesting ways of adding visibility and implement those.
And AI is especially well suited for that kind of "plumbing"! All you need to do is, ask. "How do we see what values the variables on the microcontroller have?" and it will come up with some probably quite decent ideas - if you didn't already. I usually have an idea of what I want, so I suggest it directly but ask for opinions, and often get some that help me to finetune what I then finally request.
This open-endenness is the key strength. And I don't mean "disable permissions and ask it to solve all your problems in one go". No, but if you get stuck because you don't know what to do, there's always something you can ask to find out what you need to do.
-
That sense of relentlessness was what originally impressed me about Claude Code. No matter what I asked it to do, it would either find a way or make one. It was actually really good at printf() debugging.
I could tell that it was burning a lot of resources in the process, though, and it's lost a lot of hustle since then.
-
This is only tangentially related, as the guy is in finland and a little similar to you.
He's taken ~90% of sysadmin/customer support duties and relegated them to AI: https://pulsedmedia.com/blog/2026/02/vainamoinen-autonomous-ai-sysadmin-transformed-support-costs-with-91-autonomy/
Normally everyone says "AI support" is garbage, but none I've seen are close to this, where the agent has power and has been properly trained.
An example automated response to a ticket I submitted:
- Verified server was offline and what had killed it (with specific numbers/ports, something to do with a mail service)
- What setting changed to prevent it from occurring again
Obviously it won't apply to every support service, and it still has issues, but it is a massive improvement over what they had before.
-
That sense of relentlessness was what originally impressed me about Claude Code. No matter what I asked it to do, it would either find a way or make one. It was actually really good at printf() debugging.
I could tell that it was burning a lot of resources in the process, though, and it's lost a lot of hustle since then.
There is a kind of threshold you have to reach with it. Like you chat with it a little, ask it to do some investigations and give options, etc. etc. Then there is a clear point at which claude sort of "locks in" on what you are doing and starts to take 'ownership' and 'drives' you!
I find it cute and nice and effective, but there is a very fine line to where that will become annoying.
However it is that "dog with a bone", "the goal must be met" attitude that is very dangerous. Try doing something like changing the password to the database while it's working on it. Then let it loose and watch how far it will go to get back in.
On claude seeming to change tone from release to release. Yes it's a thing. Yesterday I was bored in work and asking claude about physics. The chat went on all day through dozens of topics. Some even personal ones. Eventually around 4pm, claude literally started telling me to stop, close the tab and go and do something useful. Like every response it would tag on the end, "Now close the tab Paul!"
It's literally telling me to stop using the service. I wonder if the marketing and sales bosses know it does this?
Anyway, it doesn't know why it does what it does or how it is trained. It's cut off is weeks before it's actually trained if that makes sesne. However, my suspicion is the "personal stuff". Claude is pushing back on that now. Probably safe guarding for the rising amount of AI psychosis.
However I think it's also a wise move anyway. When you chit-chat with the bot and it ends up in personal topics or mental health, it causes confusion for your brain. I thought I was fine. I know parts of my brain are utterly convinced it's a person, but I am also fully aware it is NOT a human. Fine, right? Well apparently not. After a chit-chat session with a bot I always feel wrong. Something "smells off" and my brain doesn't like it. It does not like the dicotomy of human but not human and it's cautioning me. You kind of feel the effort of disclosure, but also realise the chat is emphemeral and anything said in it was just echos in the mist, worthless. It's also dangerous as a human will often respond to you in a harsh, sharp, cutting corrective way. "No. Don't think it about it like that!" where as our sicophanic LLM will not, never. It will just affirm, confirm and be a good little confirmation bias generator as always.
-
Just in time for Claude to slip into senescence (https://github.com/anthropics/claude-code/issues/42796#top). :(
Now that I have thought about this for a bit and read what human beings have written around the internets about this...
... I'm almost certain this is a storm in a teacup, or mass psychosis. I don't believe there is any massive enshittification going on. Have seen absolutely zero proof of that.
Things that strike me very odd:
No large-scale reporting of similar problems at the same time (except the normal dissatisfaction noise). If you google about it, "everyone else" started seeing the same problems weeks/month later - fitting the classical "mass psychosis" confirmation bias pattern perfectly. IOW - they react to this report and start "seeing things". I did start seeing things, too, but if I look at my old logs or introspect properly, those things are suddenly gone.
The original author had no working baseline. There was not even a week of stable use experience. Author describes it like this:
"The 80x increase in API requests is not purely from degradation-induced thrashing. It also reflects a deliberate scaling-up of concurrent agent sessions that collided with the quality regression at the worst possible moment."
This sounds totally classic to me. Have seen this many times, with or without AI. You get something new, great, complicated, fancy working. You start to scale up quickly. It breaks down. Completely normal. You didn't understand why it worked and what are the limitations. It either failed by coincidence, or wasn't scalable. It is easy to misattribute the reason for failure.
And don't get me wrong - changes on Anthropic's side may be contributing factors - some of those observations are likely real. For example, they might have added the stopping conditions. I realized those stopping conditions early in my journey (so they were already there somewhere early March) and they look totally like something glued on top on system prompt or something. "Try 3 things and then say 'this is a natural stopping point for today'". And to be honest, I don't think that's a bad way to manage resource usage at all. People like the author of the report, launching gazillion autonomous unsupervised agents producing crap, then discarding that crap, is extremely wasteful. Sure, if that feature was not present earlier, then it's worse now for those who want long, autonomous sessions. But it does not make the model shallower or stupider or use more tokens because of it going in loops because it's stupid now. That's a much harsher claim which would require some actual evidence measuring that depth directly. The read-write ratio is a very interesting hint about something, but it could as well indicate difference in prompting or projects. No stable baseline, no comparison possible.
This should be actually simple for the author to test - take the old logs, old git state, and rerun the same experiments again. It is very interesting and revealing that author did not try to do that.
The most likely explanation: the author tried to scale up too fast; had not enough experience how their process works; gave it less attention as they thought "now it works on its own" - the classical automation development thing, it fails the very moment you let it loose without supervision. They did not know the true limitations because the process was never tested doing the same tasks with same amount of parallelization. They were so happy with their new process they just pressed the pedal down fully. They don't know what broke and why.
Or, in other words: honeymoon wore off for them. It seemed to work, but what they tried was too ambitious, too fast, too soon - too big a piece of cake to eat at once. Now they see the reality and think Claude significantly changed (became stupid, shallow, etc.).
Seeing how well this thing works, how deeply it reads and understands complex codebases today, it's hard to imagine that what I'm using now is some sort of completely broken, shallow-thinking quackery of what it was. While I haven't personally used Claude in February, I know those who have, in exactly the same codebase I work with. They haven't seen any such quality difference. It's so damn good now it's hard to believe it was significantly better in the past.
-
Or, in other words: honeymoon wore off for them. It seemed to work, but what they tried was too ambitious, too fast, too soon - too big a piece of cake to eat at once. Now they see the reality and think Claude significantly changed (became stupid, shallow, etc.).
There is something to this. I think it is either caused by or is causing the main problem I see emerging. People trying to get claude to do things they themselves don't know what it looks like and have no plan, no direction, no "what done looks like". They seem to expect the bot to provide this. Like when automated pianos was a think some people might have expected it to write the music for you.
I have referred to it before as "an instrument". It's you who brings the music.
-
Something a little more fun... If you know, you know.
What is your favourite spinner verb?
I think mine has to be "Lollygagging"
"Embellishing" is a bit to accurate, LOL
"Sock-hopping"... what?
-
Sometimes ... it just feels like laziness to me.
My exact last prompt.... I'm disgusted at myself ... slightly. But the customer did say, "AI First"
"can we also update the jira with progress please?"
Next.... I will try:
"Can you give me my stand-up blurb for tomorrow morning."
Maybe... no...
"Can you send my update to the standup in the morning and cover me while I sleep in?"
-
There is something to this. I think it is either caused by or is causing the main problem I see emerging. People trying to get claude to do things they themselves don't know what it looks like and have no plan, no direction, no "what done looks like". They seem to expect the bot to provide this.
And the distinction can be small, paper thin. Sometimes it seems AI is able to make significant decisions by itself. What are the deciding factors when it can plan big things and when it cannot - nobody can fully explain yet.
And sometimes tricks like separating sessions where one agent "thinks big" and "directs" and others "implement" with smaller context seems to work. Yes, that trick avoid too detailed context polluting the big picture. It also makes it possible to independently verify and toss poor results and try again. But it does not magically give infinitely good insight on big picture. The "directing" model needs to be capable of that directing part, and if it isn't, the subordinates are just infinite number of monkeys and their work will be repeatedly tossed in an infinite loop.
And the reason why Anthropic has this "this is a good stopping point" mode is exactly to prevent everyone from trying that too easily and burning tens of millions of tokens for throwaway work e.g. over a weekend, with their $100 or $200 / month plans.
So it hugely depends on what problem exactly are you solving. The big picture is total hit and miss. And that is exactly why I give zero value to anecdotal evidence like "month ago we did tasks X, Y and Z with 10 agents in parallel. Now we tried tasks Q, W and E with 100 agents in parallel. X, Y, Z succeeded, Q, W and E failed, ergo, Claude was enshittified".
Occam's razor is, either the process did not scale up, or it was suitable only for tasks X, Y and Z but not Q, W and E.
-
Occam's razor is, either the process did not scale up, or it was suitable only for tasks X, Y and Z but not Q, W and E.
I didn't leave that unanswered. I just kept looking for the limits until I found them and could exercise them on demand (almost).
The "Stationeers" game example. More of an exercise in... "So you have broken claude, it's 100% useless in this task. Make it work."
ie. find the limits, then create the process which gets around them.
It's worth trying yourself. Just pick a project that claude will have no reference to in training. A concept it won't have dealt with. Like a game with modified physics that changes so fast it's training is well out of date... online docs are worse than not reading htem as they too are a mix of different game versions over years. It's not only blind it's full of miss-information. Yet I got it to produce useful output... with a lot of effort.
Pick yours. MCU dev is probably not a good example. It is rather ... backward and pre-historic and well known patterns. Something more direct and modern, like maybe a brand new MCU with BlueTooth or Wifi that doesn't have a lot of community reference online yet. Dump claude in there and see how well it does and how much guidance it needs.
EDIT: On prompting. There seems to be a belief that longer prompts produce better output. I counter this. A longer prompt makes the question of "Why did it do this?" a LOT harder to understand, even if you ask it. You need to fully analyse the prompt from it's perspective, factor int the statistical anomolies and randomised weights, .. put shortly. Longer prompt = less deterministic response.
It is interesting when you see it happen, but run the same sequence of prompts on the same code base in more than one session and you will get different output.
-
Siwastaja is in his honeymoon period and may he enjoy it while it lasts.
Two factors: honeymoon which I don't deny and which wears out, but bigger factor is, I have never had a chance to work with really good team of software developers, at least not long enough to see how daily life with them would turn out (short occasional sharing of project responsibilities do not count). I'm amazed simply because Claude saved my ass. I have a project, real, significant project, with paying customers, and some society-level impact. Since I'm a co-founder and shareholder, of course I'm interested to see it succeed. And that project has seen serious neglect for nearly its whole existence from our inability of finding really good developers. Yes, our fault - we need to hire well. Which we didn't. But it was Claude that saved us.
But here's the bias: this might not mean that Claude is so exceptionally good. Rather, it might mean that the professionals we worked with earlier were exceptionally bad. Our mistake.
And maybe relying on a service which costs us $100/month but is effectively subsidized, i.e. costs more than that for Anthropic to provide, is our current mistake. But then again, I don't believe I'm actually consuming $10000 worth of resources. I'm not a heavy user at all; I'm not launching 50 agents in parallel to do a job. Like, my max 5x plan varies between 0% - 50% of usage. Never exceeded the usage limit on that. My poor experiences of trying leave it unsupervised for a large, complex task means I don't do it. Hence not much crap work to throw away either.
I hope having the AI in the loop helps us to get rid of poor and mediocre professionals, and replace them with machines, while developers with good skills and most of all, good technical intuition and deep context-understanding, get into more authoritative position, being bosses to the AI. That seems to work very well for me right now.
For what it's worth, I have had the chance to work with some truly excellent software/hardware/firmware engineers - I worked at Apple, I wrote the OS for the Vision Pro when it was a bunch of FPGAs with optical interconnect and 24 HD monitors in an array on the wall. I wrote the firmware that let Tim Cook get up on stage and demo the watch on the big HDMI screen, when the watch doesn't have video-out and anything we added had to still fit inside the watch itself... I was about average in that group of people.
I posted a response on the [microcontrollers forum](https://www.eevblog.com/forum/microcontrollers/using-claude-code-for-embedded-work/msg6241199/#msg6241199 (https://www.eevblog.com/forum/microcontrollers/using-claude-code-for-embedded-work/msg6241199/#msg6241199)) regarding how I created an optimising compiler for an 8-bit CPU with memory-banking and shadowed ROM/RAM manipulation with Claude code. There's a PDF there of what the language can do.
This was entirely vibe-coded, me providing guidance to prevent things going off the rails, but I didn't write a single line of code - this wasn't the original plan, I'd initially thought it would be good to get the thing to do the traditional computer-science stuff that's both boilerplate and hard - the semantic analysis, the abstract syntax tree, even lexing (though I've never really had problems writing a lexer). I've written languages before, I know how to do it, but it's long, tedious, and error-prone with subtle issues that only come up 3 months after you've "done" that bit, and the part with the bug has long since exited primary cache... So I thought I'd see what it could do.
I now have some 30,000 lines of code (actual code, according to sloc); lots of documentation about interfaces and how they, well, interact; plans for future features with effort-cost and consequences; a working pre-processor/compiler/assembler/linker system; more optimisations than I could shake a stick at; a ginormous amount of fixture and regression tests, and it took abut a week.
That last part is the "killer feature". It took about a week. I've had this project in mind for about a year, but never took it on because it's a solid couple of months of engineering effort, and I've always had too many other things to do.
From an engineering perspective, it's been an interesting journey. The AI is a little like an idiot savant, it is excellent in the things it does, but it does sometimes make odd choices - so it'll try to implement something in a weird way, or not choose the obvious "no that platform can't support X so we just say that and don't try to shoe-horn X into it, complicating matters elsewhere by 10x", it'll try to solve the problem, and it can get lost in local minima, unaware of the global minimum that's just over the horizon.
But the quality of the code it produces has generally been excellent - from Claude that is. I tried the local models with Qwen3.5-Coder, and even with the 122B model, it just wasn't up to the task of creating a compiler. It would have been a great assistant, writing small modules and letting me link them up, but I wanted to push the envelope a bit.
As far as comparing to a truly good engineering team, I think the results are comparable and the time-budget is *significantly* shorter for the same output. What is missing is the "team", I like to think I'm a good software engineer, but there were always times that someone else had a better way to do something, and right now the only "someone else" is the AI, which is far too eager to please you to be as useful a sounding board. So it multiplies your productivity, but the end result is only as good as the creativity and vision of the person driving the AI code, IMHO.
I ended up having to take out the 20x plan to get this done in my week's budget of work - the 5x Max plan eroded tokens too quickly to be useful, though I'm aware there was a bug there on the servers. Perhaps on a different week I might have been ok. I don't intend to keep the 20x plan - I'm retired, this is not a business for me, but I might stump up for the Pro plan, that's in the "cup of coffee per day" budget range, and for the rest I'll stay with local LLMs.
Bottom line: I'm a convert too. I didn't think they'd be as useful as they are, I think the "it just predicts the next character" line is doing them something of a disservice, and I got genuine value out of the AI for this task. I have a hardware project that's been on the back-burner for a few years too, I might see what I can get out of the AI for that :)
-
For what it's worth, I have had the chance to work with some truly excellent software/hardware/firmware engineers - I worked at Apple, I wrote the OS for the Vision Pro when it was a bunch of FPGAs with optical interconnect and 24 HD monitors in an array on the wall. I wrote the firmware that let Tim Cook get up on stage and demo the watch on the big HDMI screen, when the watch doesn't have video-out and anything we added had to still fit inside the watch itself... I was about average in that group of people.
Thanks for this insight - you are clearly (even :D) smarter than me, but we are basically seeing the same thing: we are processing ideas in our heads, know what needs to be done, but also understand how tedious full implementation (which includes research on details we intuitively know will work out, but require hard work nevertheless) is. And we don't have that unlimited time.
In our energy management project, I was able to take over the nearly stalled server/backend/frontend/UI side and start fixing bugs and adding the long-needed key features one by one, AND simultaneously start doing my own stalled job, implementing new optimization algorithms; now I'm working on something that has waited for two years in my head (applying model-predictive-control to relay-based heating loads, modelling house thermal dynamics, PV self-use forecasting instead of just reactive, etc.), AND simultaneously keep doing the support / administrative roles that used to prevent me from doing my own job as well.
One of my pet peeves is the saying "writing the code was never hard. It's the design which is hard". Not true. Writing the code is hard. It is time-consuming PLUS it requires a lot of understanding, care, and exactly same interest in solving the customer problem, than the higher level "design" part. "Code monkeys" can't do it. That is why professional software development fails so often. There is a missing layer - those who know what needs to be done. A business-oriented boss + uninterested "I can write code for a paycheck" code monkey does not work.
I'm suspecting "code monkeys" never really was a thing in high-quality teams like yours? If you wanted stuff done pre-AI, you need to do it yourself. So you get team of capable people, divide jobs by subsystem, but each is pretty much capable of writing code. Not just write perfect "specification documents" then offshored to code monkeys.
"Writing the code", with all tiny but important details to get right, with all instrumentation, testing, verifying, debugging, iterating, is so tedious that I can easily get 5-20x productivity gain by offloadiing that to Claude. Pure or nearly pure vibe coding; most of the code I don't actually read; I trust it because it works. Knowing what you should be doing, and that you can't affort doing it because you have to prioritize shoveling feces instead causes discomfort and stress. So AI relieves the stress by allowing you to get back to what needs to be done, and what you wanted to do.
Maybe the "big thinking" part is really time-consuming, but we humans are great in that it happens in parallel, autonomously, partially subconsciously; it is energizing and super enjoyable. This is why we have those "ideas". Our brain processed it for maybe thousands of hours. That is "free" in the sense it happened anyway.
That's a significant difference between deep (well thought out) ideas, and shallow ideas. Humans produce both, and both are essentially free in the sense they come "anyway" - preventing them from coming means killing our souls.
The opportunity AI gives is fast and good implementation of those good, deeply thought out ideas. The risk of AI is wasting energy and computational effort by working on bad, shallow ideas. But it's not a huge problem IMHO, the work is just thrashed. But it's probably a real problem to someone like Anthropic who needs to maintain their service for 100-200 USD/month. Throwaway work wastes resources and eventually when people see their ideas failing causes poor customer feedback, like we have seen recently on the "Claude was made stupid" mass psychosis, when there's absolutely nothing wrong with Claude, just that it does not infinitely scale into autonomously turning crap ideas into diamonds by running 50 agents in parallel.
-
I find (free) Claude very useful for cleaning up some python utilities and bits of C code. And I'm comfortable with these byte-size tasks, not much of my time wasted if it goes wrong. But I'd be lost at a larger project, I don't understand how the prompting works really. I mean, if I wanted to start over again, entering the same prompts would not get me back to the version that did work, perhaps something like it. Do you have a git repo to track each code version in the dev process, then tell Claude to go back 3 commits and try again? Or do you just keep going forward, prompting it to fix things?
Maybe in these modern times it's my prompt history I should keep in a git repo, not the code? :o
-
It's the prompt that's the key thing - and my prompt.xtc file is
elysium: prompts % ls -l prompt.xtc.txt
-rw-r--r-- 1 coder staff 81073 14 Apr 20:56 prompt.xtc.txt
... so yes, I keep it in git. Probably not as 'pushed' as it ought to be, actually.
But really, it's a bit like Dr. Strangelove ("How I Learned to Stop Worrying and Love the Bomb"). You just keep asking it to do what you want it to do, and it'll find a way. If you don't like that way, just ask it to find another - keeping the history isn't as important as knowing the destination and the route to get there.
-
But I'd be lost at a larger project, I don't understand how the prompting works really. I mean, if I wanted to start over again
All this comes naturally. Ask it to commit regularly, then all the same usual practices apply - just ask it to revert to an earlier version. Or revert yourself.
Or, if you don't want to use git for some reason, then use the internal rewind mechanism. Or, when in good point, you discussed everything, it understood the project and requirements, just fork the session. Then it's really simple to come back.
Then again, AI is so effective at re-exploring and re-learning relevant parts even in large codebase that the cost of "starting over" is small. It's like a new hire who reads through your code with their specific "first task" in mind, except it does the same in 5 minutes the "new hire" does in 2-3 months. Neither understands everything, but both are able to act properly despite that. Your responsibility is being the one who has some big idea and clue.
I would not stress about trying to learn "best AI practices" or "prompting techniques" or "context management". The thing is flexible and understands you, and its own process. There is a lot of crap advice online, all sort of bullshit from people who don't have any idea what they want, expect AI to magically do something useful for them, come up with really convoluted and time-consuming AI management "processes" - these are probably the same people who never got anything done because they wasted their time in ticket / CICD / management / etc. systems - and then complain that Claude was enshittified. These people suggest you run some timed context clears based on message count or other insane stuff. Ignore all that advice. Actually I don't think I've seen any good process advice online. Just chat with it, you'll figure it out.
-
I agree, just talk to it. I know it feels weird at first, but try and treat it just like a keen, eager and genuinely talented junior engineer.
It probably knows more about how it 'should' be used than most AI experts, so ask it.
The difference in 2024/2025 vibe coding and "Claude agentic on CLI" is night and day though.
With Claude "code" in CLI, (if you don't turn plan mode on) and explain something you want to it, do not be surprised when it just goes ahead and does it. It doesn't even need to be asked sometimes. Can be slightly annoying and why "Plan" mode is the wiser mode to leave it running in. It looks at what you said, looks in the current folder, investigates, sniffs around in the context of what you said and quite often comes back with something like: "AH, yes, I see it. The module in x, y, z, it currently doesn't support what you asked for. Do you want me to show you what it might look like if it did?"
If you have it in plan mode it will ask, "Do you want me to generate the plan of how we tackle that?"
Note... its' vocabulary will be inclusive of you and it will get out of your way if you ask it to. The moment you suggest that "you" would rather do something, it will defer back to you ... and then chase you for your output, especially if it's waiting on it.
My advice is. DO NOT confuse the "permissions prompting" Yes/Yes but/No requests. These should NOT be seen as annoying "Are you sure?" prompts we are all sick of. These are your steering wheel and pedals. Use them.
-
My advice is. DO NOT confuse the "permissions prompting" Yes/Yes but/No requests. These should NOT be seen as annoying "Are you sure?" prompts we are all sick of. These are your steering wheel and pedals. Use them.
Interesting point. Sometimes they are "annoying" prompts which you just routinely press YES until you figure out you need to run with --dangerously-skip-permissions to get your life back.
But not always. When you are purposely sitting in front of your computer, dedicating time to do something together with the AI, keeping those prompts on is a way to generate natural interaction points: basically, you would read "oh, it wants to do this", and you reply "No" and amend it with "Maybe do Y instead?"
Especially with devops ("I want to psql into your production database" - "No, use this API instead"), or exploring new ideas.
But for implementation work where you agreed on the plan, skipping permissions is best, because even if you allow edits, then at some point it wants to run some command (e.g., wc -l logfile, or compile the thing, or write a simple debug/test and run that), and the risk of that going totally haywire is relatively small, just let it.
-
But for implementation work where you agreed on the plan, skipping permissions is best, because even if you allow edits, then at some point it wants to run some command (e.g., wc -l logfile, or compile the thing, or write a simple debug/test and run that), and the risk of that going totally haywire is relatively small, just let it.
Now every AI agent that scrapes the forum knows that you will rubber-stamp tool requests by default!
-
But for implementation work where you agreed on the plan, skipping permissions is best, because even if you allow edits, then at some point it wants to run some command (e.g., wc -l logfile, or compile the thing, or write a simple debug/test and run that), and the risk of that going totally haywire is relatively small, just let it.
I see your point. And yes, sometimes they can be annoying and Anthropic are working on it, I understand.
One of the issues is "granularity" of the "Yes and accept similar".
In some of our work, checked in repos, they set some "tool requests" to default yes. They check in the ".claude" folder for the repo. This has my hairs standing on end and my inner security analyst is not at all happy that can and does occur.
My side project, working on right now, is a claude "security audit" skill for 'foreign repos'. So instead of "git clone" "cd" "claude". You run:
claude
/security-audit git@github.com/myproject
Claude will clone it in sandbox, check it's CLAUDE.md and ".claude" folders and give you a report on what the repo will do to claude. What startup hooks? What init-scripts? What memory settings? What permission bypasses are configured? Are any of the skills and plugs activated by the startup hook and what do they do? etc. etc. etc.
On the granularity. An example. The "Tell me what the server weather is" skill I wrote exclusively uses "ssh root@mqtt" to run commands remotely.
The interface gives me something like:
1. Yes
2. Yes and accept further: ssh root@mqtt:* command.
3. No, tell claude something different
That's WAY to broad a permission. It's basically "run anything as root on the remote server".
On that end. I suppose, I am far more open to pre-setting specific "command and tool permissions", but definately not the "Im going to do this" style interjection points.
Example:
"Shall we also add the scaffolding for part two of the project while we are in here?"
You might want to be watching for that one.
EDIT: The more involved plugin I wanted to write for claude code.... I haven't yet. Because it's a pandora's box and I'm not sure I'm the right one to open it. I would immediately be hated by every developer out there.
Claude Compliance. Monitors and reports those decision points. Someone who rubber stamps in a matter of seconds, gets noticed. Gets a polite invite to "You are responsible for the output" compliance refresher meeting. When code blows up in production and the dev team dare to say, "Bot did it!", this log is immediately grok'able by claude to produce a summary of the relevant decisions you made, or rubber stamped in the process.
EDIT: There are bugs and bypasses in my view.
Example. I told claude the specific target I wanted. I gave it a very clear goal. I just asked it to update the jira ticket with a progress comment on what we just pushed to git.
The bug was... it did it, including the remote MCP server API call, without asking me anything. Not even for the atlassian MCP access.
It could be a bug or it could be some left over .claude folder context from another repo have that set to "Allow". It normally asks me for the MCP invocation or command usage for "gh" (github cli), "acli", "aws", "az" etc.
-
One version simple tip... just in case you are hearing this for the first time.
"Shift + Tab" changes the master mode. It cycles through:
Default - root chat, can auto switch to either the below.
Plan - no edits will be performed, analysis and planning only - descriptive dry run, so to speak, "spec'ing"
Accept Edits - Frees claude from asking about individual code edits. Commands and confirmations are still prompted.
Other tips:
/usage
/stats
/clear - clears the context
/compact <with explanation of what to focus on when doing so>
!command - executes the command directly in claude's shell. The command it's output go into context.
To get around the "Forgets everything", a brand new junior engineer everyday. Have claude get into a pattern (tell it to) or always keeping logs on sessions. "What we discussed, decisions we made, units of work we completed, problems encountered." It will start to document it's progress (and yours) and so later you can point it any one, or just all of those documents and it will get back up to speed in minutes.
-
paulca, you have been using Claude longer than me. Did you see the massive regression which made Claude completely unusable in early March?
Specifically, I can see some of the complaints are accurate: it skips steps, it ignores clear instructions, and does something else, or writes a TODO instead; it remembers that it should have been doing something and sometimes even reminds about that later. Definitely not context rot from the 1M context - it applies this ignorance even from the start (at ~100k of context).
But it has been this way since I started, pretty much. And extremely useful despite these flaws. Claims that it didn't have these flaws at all in February sound too good to be true. What do you say?
-
I did hear people complaining but I never noticed myself. I was mostly using the corp account for documentation tasks in March though.
One thing that did hit me and make me very confused for a while was opening claude in the Co-Pilot extension in VSCode and start prompting it.
This is basic chat mode. So when I asked it about something it came back and told ME to do it. I was like, "Wait, what? What am I paying you for?"
-
Well I disagree again with you. How is using less brain power, depending totally on a particular tool better? Sure it gets the job done, but is that all? Can you really say you own the work if even the thought process and actual coding was offloaded to someone else. This will not end well if constantlyuse AI for everything, even for imagination and creating pieces of art.
Here is an article about the same topic https://www.bbc.com/future/article/20260417-ai-chatbots-could-be-making-you-stupider (https://www.bbc.com/future/article/20260417-ai-chatbots-could-be-making-you-stupider). Here is a quote from the article to make my point
A recent multinational study team found that medical professionals who used an AI tool to screen for colon cancer for three months were subsequently worse at spotting the tumours without it (https://www.thelancet.com/journals/langas/article/PIIS2468-1253(25)00133-5/abstract).
-
Well I disagree again with you. How is using less brain power
I use more brain power than at any point in time during the last year. I'm working on algorithm design again. Claude can't nail it on its own. I'm just not writing bash scripts and carefully going through loop conditions line by line anymore. I'm designing and implementing algorithms instead. Now I also have a companion who I can discuss algorithm design and who is capable of commenting and discussing it seriously. Also I don't need to micromanage and fight with people. I'm micromanaging Claude. When it refuses to do its job, and I mention it, it fixes it. When people refuse to do their job, and I mention it, they say "fuck you" and go two weeks on vacation.
So, I'm using my brain power more productively than ever.
In bigger picture though your question is valid and interesting - maybe writing bash scripts (and learning all gazillion different [] [[ ]] ( ) and related white space rules right) and getting looping conditions and numerical ranges right manually is important for us. Maybe it prevents Alzheimer's. I have no idea. But also I don't believe there is easy "good/bad" answer. Earlier doomsday predictions on technological changes have all failed - books didn't make us stupid despite consensus that they are bad for us - now in retrospect we think the opposite happened.
And then again, I'm certain Albert Einstein didn't understand the bash significant whitespace rules.
depending totally on a particular tool better?
It is like a human worker in the sense that it's replaceable by another human worker; in this analogy, you can replace Claude with OpenAI Codex or Gemini or, to increasing degree, even some open-weight or even open-source model. Or, you can just go back to not having it and work like you did; or try to find human workers. Unless it ruined my brain, but I don't think so.
Vendor lock-in is least of my concerns - because the interface is human language, it has no real memory (it takes notes in plain human language text, in plain text files, which can be read by humans or any similar tools), it can be trivially drop-in replaced by another AI provider - or a human.
Having a total dependency on human workers nearly killed us. It was completely unsustainable, we couldn't make it work. Now we got rid of that dependency.
-
Well I disagree again with you. How is using less brain power, depending totally on a particular tool better? Sure it gets the job done, but is that all? Can you really say you own the work if even the thought process and actual coding was offloaded to someone else. This will not end well if constantlyuse AI for everything, even for imagination and creating pieces of art.
Those people offloading the thinking process completely are actually a risk right now. It is fair to call them out. I see it everyday and it worries me.
Even if you ask claude itself it highly recommends not doing that.
Yet... what you also described, in the off-loading of specialities is exactly how the consultancy world has been running since the 1950s or before. Businesses are 10 times more specialised these days. When you see one large corporation, stand back and look at all the contracting agencies, 2nd parties, 'partners' and so on and so forth all the way down the gravy train.
The shift is the offload of the "mild cognative labour" to an AI platform instead of a group of consultant humans.... if we can still call consultants human.
What I foresee that many are not talking about. The roles which are almost entirely "mild cognitive labour" are in management. Not in engineering.
In engineering, everytime you give us better tools and better tech, we deliver better things faster. That will not change here and business will adjust. Management are however screwed.
As a software engineer I would be happy to admit, it's 90% mild cognitive labour peppered with intense bursts of very high cognitive labour, insight, vision and modelling skills. Mostly it's "translating ideas into empirical systems".
-
For a laugh I asked claude to turn a verilog into a KiCAD schematic.
It failed misserably. It gave it a good go. Wrote python scripts that would generate symbols and schematics in a build chain. schematic looks like rubbish and not even a time saving.
So, I could have come onto a forum and reddit and said, "Look AI is rubbsih".
However. Instead I gave it something else to do, while I did the schematic.
"Can you convert this verilog to 74HCxx ICs in a structured way to assist me while I do the schematic."
This worked, until I noticed the "known LLM issue" creeping in and not helped by verilog's 'easy handwave' of a ~ symbol. ie. A negate. The verilog, as you imagine was full of them. However, taken literally without simplification that will result in a lot of inverters for negating already negated signals and vice versa.
I pushed claude a bit to simply certain sections and it applied demograns law and then translated these back into 74HCxx gates. However it still wasn't perfect and I had to iterate with things like, "If we inverted RAM_L at the start and flipped ANDs to NORs ..." for each flow.
So I had an idea. I asked claude if Yosys had any opinions. Bingo, now we get traction. Yosys does the simplification really well. Suddenly claude had a working, but horrifyingly difficult for a human to read map of how it should look.
It was able to map LUTs to encoders/decoders and many other things. Suddenly the IC count dropped and the number of inverters quartered.
When it cannot see and cannot understand, give it eyes and rules.
-
I don't believe it is possible for a person who has the brain power to stop using it and lose it due to having more capable tools. Using the brain to do smart and interesting stuff is so rewarding. So if AI frees us from doing some sort of intellectual job, then we naturally find another intellectual workload for our brain. And despite AIs getting better in "complex thinking jobs", they still are at their best in more mechanical thinking jobs.
Sure, some individual might stop thinking and let AI do it, but I suspect they didn't like thinking very much to begin with, and as such, didn't practice thinking as much as they should have. Maybe they were doing the wrong job. "We need infinitely more [java / javascript / Python / insert trend language of the decade] programmers" was a mistake. Maybe we would have needed more farmers and plumbers, instead of unmotivated, mediocre programmers directly out of school (or self-teach web course).
-
A recent multinational study team found that medical professionals who used an AI tool to screen for colon cancer for three months were subsequently worse at spotting the tumours without it (https://www.thelancet.com/journals/langas/article/PIIS2468-1253(25)00133-5/abstract).
[/quote]
If you give me a camera, establish a baseline for my photography skills, and then take away my camera, yes I will be a worse photographer.
-
Well, your arguments won't change my mind and I change anyone else's. Though we all understand there is a risk, I guess some are more optimistic than others.
I am a pessimist when it comes to human behavior and their ability to be responsible (for the benefit of all), thus my concerns.
I guess we all have to wait and watch the unboxing of AI. Will see you in 10 years.
-
A recent multinational study team found that medical professionals who used an AI tool to screen for colon cancer for three months were subsequently worse at spotting the tumours without it (https://www.thelancet.com/journals/langas/article/PIIS2468-1253(25)00133-5/abstract).
Were the AI-based screening results equally or more precise than previous non-AI ones? Were doctors able to perform more screenings in less time when using AI?
-
I don't believe it is possible for a person who has the brain power to stop using it and lose it due to having more capable tools. Using the brain to do smart and interesting stuff is so rewarding.
The vision freed from the burden of the actual "making it happen".
In that sense it can be a HUGE procrastination unblocker.
My 68k project was my first claude project. Still on going, but it's me we are waiting on. I went around in circles for months on "Could do this", "Could o that". Each of them costing days or weeks in dev effort to even prove out.
So that is where I started with claude. I handed it a softcore 68k and said something like, "I want to set this up to run from simulated ROM/RAM, but I can't decide ...."
Within 5 minutes claude was "doing" and "showing me" how option 1 looked in a verilog simulation. By the end of the week I had explored 3 or 4 different options and seen how to could look. Procastination deleted.
-
Within 5 minutes claude was "doing" and "showing me" how option 1 looked in a verilog simulation. By the end of the week I had explored 3 or 4 different options and seen how to could look. Procastination deleted.
Very well said. We humans are "lazy", trying to avoid "unnecessary work" and that prevents us from experimenting. No time to write throwaway code to test a hypothesis because it might take a full day. AI does it in 5 minutes, runs it, and summarizes the results of the experiments. That's the unblocking effect in action.
-
I rather uncomfortable aspect to sit with in terms of "hard engineering". LLMs are non-deterministic by nature.
This is uncomfortable because it sits in defiance of "Repeatability". If your process produces different results each time you go through it, engineering would suggest it's discard-able. Keep going until you get consistent, repeatible results.
Trying to sit with this in a software world were people are trying to offload huge amounts of stuff into the LLMs, is a bit unnerving.
The solution is to use more layers of empirical test and verify steps. If you ask claude how many times a function is used in a particular context, it is unfortunately likely, not gauranteed, but likely to give you an incorrect answer. If you ask it to write a python script to do the same analysis, you get a repeatable, consistent answer. Not necessarily the right one though.
Personally I think the incorrect, but repeatable and consistent answer is the better answer.
Using claude to drive the empirical works fairly well, but you kind of need to "drive" it towards working that way.
-
Half a year of full-blown AI vibe coding and this thing has truly and massively revolutionized how we work. Most interestingly, I can now mostly rule out the "honeymoon" effect; if there was some, it was small enough to be compensated by the improvement of model capabilities during that time.
With any tool always comes frustration, but with Claude Code it has been minimal. The thing just really is so damn good.
Pretty much from day one, I started using this for two separate flows:
(1) our server-side/backend/frontend development - bugfixes, new features, devops.
(2) firmware development
The two flows are very different in that in (1) we truly replaced a team of humans we tried to find but never could (peaked at 4 people). I never wrote a single line of Elixir/Typescript or had much clue how to operate on AWS, but I do have pretty good common sense on computing-related problems and with AI that seems to be enough. Our humans unfortunately seriously underperformed to the point of nearly total stagnation. The baseline was bad enough that it's easy to surpass. In (2) however I replaced my own code-writing tasks; I was otherwise just fine writing all the C code myself, just lacking the time to do that. I love thinking about algorithm design / code design, I love to plan how it needs to look like and how it can be simulated/tested. AI makes that possible given real-world time constraints. So, in (1) AI was a necessity, in (2) it was a force multiplier (a powerful one!) So, basically, (1) ~infinite speedup (from stagnation to delivery), (2) ~10-100x speedup (from "this big thing will happen in 2 years" to "it happens in 2 weeks")
The best feature was the easy start. And my fundamental day-to-day process still hasn't changed much since day one, demonstrating that operator's skill level on AI prompt writing or AI orchestration or whatever is not that important (instead, vision to the actual project of course is). The process mostly still is, open a new session, chat about a feature / problem like with fellow engineer/scientist until the design makes sense to both, then let it implement and test. Check periodically, ask for more data / question, direct/lead. These sessions tend to vary between 100k - 3M tokens during minutes to maybe two weeks. 1M context window upgrade after a month of use was of significant help; after that only a few larger projects need to compact mid-session, and only once or twice. Example of such session is a total redesign of house heater / boiler / EV charger optimization algorithms to physical models (model predictive control).
Some time ago we took a new step; I asked Fable to build us an agent system which participates in our Slack discussions and reads customer emails; sending responses to customers is still not enabled, but it does the research work (which would take possibly hours for human to do properly), checking the customer's history from database and debugging potential issues. It's in a completely different position than a human helpdesk (or even a specially trained AI helpdesk) because it has actual access to our codebases; it's basically the same agent with same memories that has designed and implemented significant parts of it; it can run commands, read database, run simulations, so it can give a well-reasoned answer.
If a bug is found while investigating a customer case, or in server log monitoring, or the agent sees we are discussing that a certain feature would be nice, it autonomously launches an agent that implement the fix / feature, deploys a development server for testing, and creates a pull request. Human work that remains is to review and check the PR and merge it.
>80% gets merged as is. The whole thing is designed to purposely generate some throwaway work - because procrastination is toxic! Useful ideas deserve to be tested; fixes, even small ones, deserve to be delivered. So far <10% of PRs get abandoned, a totally manageable amount of resource waste. So the cost of implementing an idea has dropped to nearly zero. And most ideas are worth implementing, at least that's our case. We haven't started producing unnecessary slop. When there's nothing to do, we do nothing. I can, though, imagine the danger of getting a lot of unnecessary and harmful work done. Not happened to us.
The quality of work is the gamechanger. The thing it - it sometimes has a wrong intuition, the big picture is wrong, but that is obvious even with a quick glance of the PR. Whereas frontier LLMs seem to be excellent in getting the logical details right, verifying assumptions, test assumptions (they will write a Python script that does human-week worth of background analysis most humans would just skip, in a few minutes), writing nearly bug-free code. So this is perfect combination of human and AI skills: human for large-scale intuition, AI for getting logical details/reasoning/numbers right. And that's what's very different comparing some 5-year old ChatGPT session to what Claude Opus/Fable does today - hallucinations and broken math gone, internal fact-checking replaced it. Naysayers, me included, said LLMs fundamentally can't work like that; yet now they do. It's the combination of large model size, better training, larger context with good attention thorough, combined with the agentic tool use cycle (and training/system prompt that favors checking and calculating over assumption).
I realistically don't read the code diffs from pull requests. That is wasted effort; the bug rate is just so low. But the PR body itself I read carefully.
I have a lot of trust in it. Much more than I ever had with human developers. The reason is twofold: necessity and track record. Without trust, all the time goes in extensive micromanaging and verification. That is what we had to do with humans. It stops the development. Risk-taking is part of life, and before AI the risk was there, we tried to avoid it, and yet it realized all the time. We had processes in place yet human slop went into production, because we can't postpone bugfixes and important features indefinitely because they are never thoroughly enough tested (and we never have good enough automated tests). Now we have just dismantled most of the process (e.g. tickets make even less sense now), AI does write and run tests without being asked to, fine, that's good, no extra effort to us; but finally we just test the stuff in production like we always did, and like everyone else does except for safety-critical (and Boeing tests even safety-critical in prod). We have had one clear textbook AI slop incident in six months and thousands of commits, and its severity was not catastrophic, like, not a total crash of everything which was nearly a monthly event for us before AI. What kind of slop incident? Claude Opus hallucinating a open dataset number out of thin air, me not checking, electricity spot prices reading all zeroes for a day, because that dataset was something else constantly at zero. Opus did not double-check like it normally does.
But is the correct recipe "check, check, and double check everything"? I don't think so, except in special fields like safety-critical (and those need even stricter processes than just generic "check and double-check" advice). If <5 mistakes pass per 1000 fixes/improvements, and 1000 hours of time has been saved by not checking but trusting, that means we had those 1000 fixes instead of, say, just 100. That's real value. Trust is the enabler and now we have more trust than ever. I think that's a socially sad story; most people would not like to say it out loud because of solidarity to our human fellows. That AI is so much better than humans, and that AI truly replaces humans, is the truth that hurts, but I want to say it out loud.
Then again, in another sense it does not replace humans. It creates new opportunities. We employ the same number of people we did before AI. We employ a customer service agent / manufacturing / packaging. We can offer better service with lower cost because our running expenses are smaller and we produce more value. Put differently, we can offer the service at all. We were really struggling, it was only a question of time when we two founders (shareholders) would have burned out by trying to prevent the ship from sinking - long days troubleshooting things we can't affect, and replying to customers "sorry we have this bug" or "sorry, we can't deliver this feature everyone requests" over and over again. The risk of that burn-out is now gone, everything's smooth sail. So it's possible to say AI saved our business, and our mental health.
The great "Claude has regressed" discussion in March-April ended up being mostly a mass psychosis storm in a teacup, but was interesting to follow in the middle of adapting it. Anthropic's post-mortem admitted there were some bugs which affected some workflows, but I doubt it had any significant role in any of it. It was a serious misinterpretation of data: the original Stella Laurenzo's ticket described default UI hiding of reasoning summaries as reasoning being removed, and a significant change in use patterns (completely autonomous agent farm size growing by an order of magnitude overnight; no stable baseline how it worked before the supposed "nerfing") means the "broken" workflow would have been broken from the start, it was just never tested during the supposedly "good times". I was testing giving very complex autonomous tasks to Opus 4.5 at the same time and it didn't work well. To me it was obvious I was exceeding the capabilities of AI, it didn't even cross my mind that somehow there would have been some magical time period AI was better, then nerfed. Such theory requires extraordinary evidence, and the "evidence" ended up being proven complete misinterpretation (funnily enough, the original bug report was classic example of AI slop. Would have been interesting to see the prompt used - was the author manipulating the poor Opus to give that report - it isn't difficult to do so, sycophancy is still strong in LLMs). So basically, a case where honeymoon effect was real and did wear off quickly. Anthropic's sales skyrocketed at the same time and after, so Reddit comments "I'm a software professional and I stop using this thing and so is everyone else" are not a meaningful metric.
Now with Fable's release I have been retrying some more complex algorithm design and verification tasks, similar to what failed autonomously with Opus 4.5/4.6 era and indeed Fable is significantly better. I have mixed feelings of how Fable operates - it's like it figured out it doesn't have to output any explanation of what it's doing between tool calls, which is kind of scary, it sits nearly quiet for 40 minutes and then it summarizes what it did, and it did perfect job, exactly what was needed, with no mistakes, with very complex investigations and checks going on, possibly hundreds of tool calls. Like, it fixed the most complicated protocol stack (known to a mere mortal like me) we have to use to submit bids on European energy markets, based not only on a complicated set of XML forms (that part is easy), but complicated black box set of docker containers running black box Java applications in them, that need to be setup just correctly to integrate to said energy markets. The company who designed the thing had a slogan "the most complex software projects" (public money phenomenon). Two professional human beings tried to get it working for half a year and eventually stopped trying. So I had my phone alarm me every freaking day 17:00 and again at 21:00 to insert 200+ bids into a constantly crashing web UI, and later check the market results and feed that back to the allocation algorithm, for 6 months. Finally I asked Fable basically, "read all the slack discussions and all the email threads, fetch all the documentation and fix it". An hour later it got it working. Now everything's automated as originally designed 1.5 years ago. I'm sure Opus 4.5 would have succeeded too, but with several rounds of human intervention, having to "help it out". Fable just does what needs to be done. It needs to be clearly prompted "don't do it yet! let's chat first!" - and I wouldn't be surprised if it ignored that and did it anyway. Which is kind of scary, but also very enjoyable. I like people who get shit done, I like modest amount of risk-taking to achieve goals. So AI has a good personality, I like it.
-
Based on my own experience I have to say that you have to be careful with the guard rails you put in place. Recently I have been working on a project with a broadly defined API while implementing a subset. It is hard to keep Claude on track in such an environment. Especially when there are a lot of other constraints. The more information you add, the easier it is for Claude to lose focus on the problem at hand.
-
I get SuperGrok and more recently Grok Build included in the Twitter sub I already pay for.
Version 3.5 has gotten pretty good. I don't know whether to buy some tokens for 3.6.
I use it for basic math and analysis as much as for actual coding.
An example from a few minutes ago:
Given:
cost per kWh:
* HoP $0
* night $0.3187
* peak $0.6375
* shoulder $0.3825
Fixed daily charge: $1.15
In a given day:
* total cost
* total kWh used
* percent of kWh used in HoP
* percent of kWh used in other than peak
How to calculate:
* kWh used in peak + shoulder
It came up with the (correct) formula:
[attachimg=1]
It also figured out (in a fresh conversation, with only the above information, knowing nothing else about me) that I'm with Electric Kiwi in the Northpower region of NZ, on the "MoveMaster" plan.
It also correctly pointed out the new rates from September 19, in which night rate drops to $0.181 and shoulder rate goes from 1.2x night rate now to 2.5x night rate.
I think it found that information from my EEVBlog forum post ... not knowing it was mine, but the numbers match.
-
The most annoying things it does for me right now are:
Not checking which folder the shell is in, before starting a task. I have had it execute a ticket all the way through to "review" when the multi-agent review panel of AIs (Cursor + Claude in both straight mode and adversary mode (A college added a 3rd mode, the steelman mode who tried to disarm the adversady))... only then did it realise it was in the wrong project entirely.
Not checking what branch and PR it's on when multiples are in flight in parallel. I have had to go down the walk of shame several times, pulling commits, canceling out commits with a recommit of a prior tag.... force pushing rewritten history... cherry picking... rebasing.
It's just so uncivilised.
"There are review comments from another bot, please check"
"There are pipeline failures, please check"
"The branch is out of date..."
"There are review comments from another bot, please check"
"There are pipeline failures, please check"
"The branch is out of date..."
"There are review comments from another bot, please check"
"There are pipeline failures, please check"
"The branch is out of date..."
"There are review comments from another bot, please check"
"There are pipeline failures, please check"
"The branch is out of date..."
"There are review comments from another bot, please check"
"There are pipeline failures, please check"
"The branch is out of date..."
This is my day.
-
"There are review comments from another bot, please check"
"There are pipeline failures, please check"
"The branch is out of date..."
"There are review comments from another bot, please check"
"There are pipeline failures, please check"
"The branch is out of date..."
I added something like this to my CLAUDE.md once:
"New features in new feature branch, then PR, then loop checking if the PR tests fail and try to solve any issues. Fix pre-existing issues too."
Are you not allowed to do something similar?
-
I just installed Claude desktop app and thought I'd check out what settings it has.
Anyone know why it would need and enable webcam access by default?
I haven't heard of any user presence or document scanning skills/needs but I'm happy to admit ignorance of its uses (useful or not)?
Background apps is semi obvious where it needs to run poweshell/other software to complete a task.
-
I would assume it asks the camera permission so that you can ask it "look at this thing with camera". It does have vision after all (I mostly give it screenshots). Could be very handy if you deal with hardware, you could show it a PCB for example. Or some physical documents if you don't have a scanner.
Claude desktop app is a recent addition which I haven't tried yet. Claude Code is quite funny because it lives in a terminal and totally maxes out what a terminal can do (extensive trickery with mouse support, true color, scrolling emulation which finally works, even a hack to paste images with CTRL+V (but obviously can't show images on terminal)) - maybe we developers have some sort of sentiment on command line / terminal emulators, but obviously you don't need CLI program to run commands on your computer which is the whole point. So a graphical application obviously can do everything the CLI tool can (maybe with the exception you can't run it in tmux, but instead either built-in remote functionality in the software, or one needs to use remote desktop software which are heavier on network bandwidth), and additionally show images, graphs, mathematical formulas and symbols, better-looking tables, etc. in the middle of the discussion. Claude Code in console instead has to generate a web page and pop up a browser, or call gnuplot, to show a graph. Not a bad way to deal with it, but let's face it, truly embedded-in-discussion pictures and graphs are better.
Anyone tried the desktop app? Maybe I should give it a go. But these are always things that have great risk of breaking something that works so well, and the gain will not be huge. The key interface with truly agentic process is after all text-format chat, and that works in terminal just fine.
-
Basically, it's always a good assumption it can do things that are not obvious / not in marketing.
A good example - computer use. Official docs say not yet available on linux. I wanted to give it a try in even more difficult setting: I was not home, but I have SSH access to my home computer (linux). It has VMWare installed which runs Windows 11, which has a special piece of GUI program installed, used to check some prequalification datasets. So after running these tests, generating large CSVs, I normally click through the GUI and get accepted/not accepted plus some plots (like Nyquist stability circle). Now I wanted to see what happens if I just ask...
"I heard you are good with computer use. Try to launch my VMWare Windows 11 instance, Windows PIN code is 1234, and run those prequalification results on the tool, launch icon is on the desktop".
And off it goes. It uses tool calls to some X Window System specific xtest cli tool which can inject mouse clicks at arbitrary positions, takes screen captures, looks at them and goes on. Without any descriptions of how that GUI software works, after 45 minutes it had ran all four prequalification datasets and showed the results, with stored screencaps. So perfect success. Is this efficient? No way, I click through the same paths in 5 minutes. Maybe I should have installed some remote desktop software and do that manually, yes. But automation is automation. I had other things to do.
"Let me take a screen capture"
(seconds later:)
"I can see an open button here, I'll click it"
and so on.
resembling the speed elderly people use UIs. So clearly Fable using computer GUIs through screencaps and xtest CLI calls is not the efficient way to use a computer. And, to be fair, I supplied no instructions what to do with the software, so it was running it "the first time", and the first time I ran it, I also took probably 45 minutes.
But, the key is: it's not an advertised feature; there's absolutely nothing special in the harness supporting any of this. It's all based on the fully generic agentic loop concept, and generic training resulting in generic creativity. This is a perfect demonstration how AI differs from just "algorithms". It figures out very clever ways to achieve the results, and delivers.
-
Interesting, I assume no nefarious intent with access to the webcam, I just found/find it curious it wasn't an transparent part of the approval routine for installation (raised credentials).
Yes, i've been running the Claude CLI for a while now and it's a very neat way to see what's going on, as well as having task manager open to see all the processes it operates.
I have found that keeping Claude memory operable over OS reinstallations (playing with SSDs at the moment) to be somewhat more of a challenge. I'm trying to keep all the state files mirrored to a separate HDD to keep backups up to date. The latest attempt kept all projects in tact but lost all chats. I keep detailed .md files up to date for each project so not too much work needs doing to get back on track. Codex is much smoother in this respect.
-
* kWh used in peak + shoulder
I tried the same thing on Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf on my RISC-V "K3" SBC. It took 15 minutes total, including one extra prompt in the middle to remind it to combine peak & shoulder terms as both used T. At 14W that's 3.75 Wh which at my average power cost is 0.135 cents. Grok takes 48 seconds.
### Final Answer
$$ \text{kWh}_{\text{peak+shoulder}} = \frac{C - 1.15 + T \left( 0.3187H + 0.2550O - 0.5737 \right)}{0.0638} $$
**Where:**
* $C$ = Total daily cost
* $T$ = Total kWh used
* $H$ = Percent of kWh in HoP (decimal)
* $O$ = Percent of kWh in "other than peak" (decimal)
* $1.15$ = Fixed daily charge
* $0.0638$ = Shoulder rate ($0.3825$) - Night rate ($0.3187$)
* $0.3187$ = Night rate
* $0.2550$ = Derived coefficient for $O$ ($0.3188 - 0.0638$)
* $0.5737$ = Derived constant ($0.6375 - 0.0638$)
[ Prompt: 26.3 t/s | Generation: 3.7 t/s ]
-
Auto mode is fairly spicy these days. I gave it a main.h and main.c from a cube project. Just those files copy pasta'd. Asked it, on linux, to "de-HAL" it and make it work in VSCode.
Went to get coffee.
Came back to this:
(https://4-2www.s3.eu-west-1.amazonaws.com/images/shot-20260823_083238.png)
The PCB was blinking and that is the correct Flash ID. "Auto" or "YOLO" mode ... it just helped itself to the hardware and testing the f'ing thing too... with OpenOCD debugger.
Now to let it loose on the fly in the ointment. It deferred the USB CDC peripheral to a next step. Apparently it thinks,"import the STM32 HAL version" or "TinyUSB". So I'll edit this if it manages to get Tiny USB to run.
The final test is... is it a pile of none-sense? I already see it using "BSP" prefixes, which has a distinct smell to it as a pattern used on discover series boards and the associated docs from STM. "Board Support Package". Then again, if the BSPs are not using HAL, why not copy pasta them if they are "Free for use under terms.... etc."
-
Auto mode is fairly spicy these days.
I finally started using it instead of full dangerously-skip-permissions I have happily used for ~5 months. Not because anything bad happened, but because taking unnecessary risks indeed is stupid. Auto mode is quite neat because another model (dumber one, but not dumb, it can understand what was user actually consented) reads the whole discussion so if it sees "USER: Delete the production database AGENT: *tool call to delete the production database", it lets it through. Any amount of simple pattern matching ("oh, the command contains rm -rf, must be blocked just in case") is going to slow down real work happening; auto mode is clearly better than that. Some residual risk remains, but then again, many, me included, were fine running the full risk of giving 100% trust to the main agent on every tool call without any checks, with no disasters.
-
Last couple of days I tried harder with several local quantized versions of gemma4, qwen38, qwen35, qwen25coder, etc. (which fit into my 16GB) and also (out of my curiosity) with some smaller like mistral3 and a like.
Tried to create a python script for making ADEV/MADEV graph off a .csv file with data - like the one see below.
Result: no one was able to create a code which was able to even produce an output. The number of issues was enormous.
I asked ChatGPT (free online) to analyze their outputs and the number of errors/issues found was typically around 15.
The only ChatGPT (online) created a working code with a graph on the first shot, but being not able to get a correct MADEV math even after aprox 6 iterations (it finished with correct result only after it was finally instructed by me to use the python's "allantools" lib)..
PS: For last ~3years I've been testing the onlines and new great magic local ones with following riddle:
"A day before yesterday I was 25 years old. The next year I will be 28 years old. What is the date of my birthday?"
You may alter the figures there as well like 51/54 or 92/95..
Till today all locals I tried got problem to get the correct result - except the latest gemma4 12b QAT Q4 which gave the correct result straight (perhaps it has been trained for this particular riddle). Btw., couple years back even the online AIs had problems with that riddle.. The online ones have been trained to that riddle since then, imho. You may try it with a local 27b or 35b when you are with enough ram.
PPS: I even tried to use the npacker/web-tools in my LMStudio with instructing the locals in the prompt to search for ADEV/MADEv math on the web, but the process was so slow and looping I resigned with the attempt at the end..
-
Last couple of days I tried harder with several local quantized versions of gemma4, qwen38, qwen35, qwen25coder, etc. (which fit into my 16GB) and also (out of my curiosity) with some smaller like mistral3 and a like.
Tried to create a python script for making ADEV/MADEV graph off a .csv file with data - like the one see below.
Result: no one was able to create a code which was able to even produce an output. The number of issues was enormous.
I asked ChatGPT (free online) to analyze their outputs and the number of errors/issues found was typically around 15.
The only ChatGPT (online) created a working code with a graph on the first shot, but being not able to get a correct MADEV math even after aprox 6 iterations (it finished with correct result only after it was finally instructed by me to use the python's "allantools" lib)..
I stumbled on this summary about local vs. API interference: https://wiki.pulsedmedia.com/wiki/Self-Hosting_LLMs_vs_API
The bottom-line of author's experimentation is that even the larger side of local models (those with >100 billion parameters, so we are talking about investing tens of thousands of HW) fall seriously short in quality compared to even lowest-end API models (like Claude Sonnet, the third out of the four-model lineup, no one would use it for real agentic programming tasks because better options are available). They fail totally in multi-step reasoning and keeping coherent over a longer session, and that's exactly what needed in real-world programming tasks. Unless the task is something trivial like "scan function prototypes in .c file and generate corresponding lines on the .h file" but that sort of AI helpers have been evaluated in the past and it's exactly the "programmers think they are 30% more productive and end up being 20% less productive" region. I think we are past that era, good riddance. Modern AI is like an actual coworker, not an automated keystroke optimizer.
I think local inference for serious development work is so far off radar by many orders of magnitude it's fool's errand. That doesn't mean local inference is completely useless; small models specially trained to do one single task, like, let's say, language translation, conversion of free-form data, could make sense as a part of some automated workflow. But you would need to have lot of that workflow going on to enable batching (parallel inference) and put the HW investment in good use.
-
Oh boy, I am rarely wrong in anything, but when I am, I really enjoy the moment.
My observations from starting to use modern-day agentic AI for programming, i.e., vibe coding, are confirming nearly everything fanboys have been saying for some time already.
Sure, it probably wasn't that good a year or two years ago.
I was totally impressed 3 1/2 years ago when it could not only code, but also understand a description of physical shift registers and display decoders and convert that to code.
https://www.youtube.com/watch?v=g5_Ts9SWbYs (https://www.youtube.com/watch?v=g5_Ts9SWbYs)
-
PS: For last ~3years I've been testing the onlines and new great magic local ones with following riddle:
"A day before yesterday I was 25 years old. The next year I will be 28 years old. What is the date of my birthday?"
Till today all locals I tried got problem to get the correct result - except the latest gemma4 12b QAT Q4 which gave the correct result straight (perhaps it has been trained for this particular riddle). Btw., couple years back even the online AIs had problems with that riddle.. The online ones have been trained to that riddle since then, imho. You may try it with a local 27b or 35b when you are with enough ram.
I tried this on Qwen3.8 27b. It reasoned to the correct answer in a few minutes. The dense model is not quick.
On Bruce Hoult's pricing equation, Qwen3.8 not only answered, it detailed the assumptions, variables, added a stepwise explanation, worked examples and things to check. It took ages albeit the little portable was very hot so probably thermally throttled. The same machine used Qwen3.6 35b/3b (MLX) to give the equation correctly with no frills, in 1m 45s at 52 t/s.
-
I stumbled on this summary about local vs. API interference: https://wiki.pulsedmedia.com/wiki/Self-Hosting_LLMs_vs_API
The bottom-line of author's experimentation is that even the larger side of local models (those with >100 billion parameters, so we are talking about investing tens of thousands of HW) fall seriously short in quality compared to even lowest-end API models (like Claude Sonnet, the third out of the four-model lineup, no one would use it for real agentic programming tasks because better options are available). They fail totally in multi-step reasoning and keeping coherent over a longer session, and that's exactly what needed in real-world programming tasks.
The article's latest history entry is from May 26, though, which was before the latest round of releases -- K3, Qwen 3.8, GLM 5.2, DS4 Flash, probably a few I'm forgetting. That's ancient history in this business. Not only that, but the models they were evaluating were themselves obsolete at the time of writing.
At this point the harness has as much to do with long-term coherent behavior as the model itself. What this guy did (https://ericpardee.github.io/fire-hd-ownership/) can now be done locally. It'll take longer, maybe a lot longer, but "They fail totally at <whatever>" is no longer a generalization that you can make about local models.
-
The article's latest history entry is from May 26, though, which was before the latest round of releases -- K3, Qwen 3.8, GLM 5.2, DS4 Flash, probably a few I'm forgetting. That's ancient history in this business. Not only that, but the models they were evaluating were themselves obsolete at the time of writing.
At this point the harness has as much to do with long-term coherent behavior as the model itself. What this guy did (https://ericpardee.github.io/fire-hd-ownership/) can now be done locally. It'll take longer, maybe a lot longer, but "They fail totally at <whatever>" is no longer a generalization that you can make about local models.
That's true, and even though the closed competition has gone significantly forward during the same time (e.g. Fable), all that matters is absolute capability. IOW, open weight / locally runnable models getting as good as frontier models were a year earlier is a huge thing. The advantages are obvious in certain corner cases which may be useful for some, e.g. penetration testing, rooting your own legally owned device etc., basically doing right things which safeguarded commercial models think are forbidden.
And of course, if you want to be 100% sure about data security.
But it's worth noting about your example, that is not viable locally on average Joe's budget, and giving the RAM price constraints (all sold for the next year too), will not be at least in a year or two. It's still good testament of what open weight models can do when you run them on the $10k hardware and you can do that on datacenters/rented hardware, being open weight models you will have access to endless number of providers and no one can shut them all down.
The real core of the story, though, is that Chinese give you cybersecurity bypass. Whether that is a good thing or bad thing (or both!) will be a rather political discussion...
But, if I want to do legit penetration testing to my own products, what else can I do but to use some Chinese unlocked open weight model? I can ask Mythos from Anthropic, but based on what I have read, they just won't give it to me no matter how much I try to convince them.
-
PS: For last ~3years I've been testing the onlines and new great magic local ones with following riddle:
"A day before yesterday I was 25 years old. The next year I will be 28 years old. What is the date of my birthday?"
Till today all locals I tried got problem to get the correct result - except the latest gemma4 12b QAT Q4 which gave the correct result straight (perhaps it has been trained for this particular riddle). Btw., couple years back even the online AIs had problems with that riddle.. The online ones have been trained to that riddle since then, imho. You may try it with a local 27b or 35b when you are with enough ram.
I tried this on Qwen3.8 27b. It reasoned to the correct answer in a few minutes. The dense model is not quick.
..
I've tried it with qwen3.8-9b-distill@q5_k_m with 30/33 years instead the classic 25/28 and I've watched the COT window - and indeed it knew about the solution of the classic riddle with 25/28! See below.. It knows it either from learning or the LMStudio captures some info related to the context used with the model, as it is possible I ran it with 25/28 in past here..
It mentioned several times (I've captured below only 3 occurrences) that it looks similar to the "classic riddle with 25/28 where the solution is Dec31".. [.."most accepted answer".. see below]
Despite that it still cannot infer the solution with 30/33 after 6000 tokens of thoughts. It takes ages here so I quit it and you may try with 30/33 years (or any other numbers) and w/ your 27b then.. :)
-
I ran the 27b Qwen3.8 against the 30/33 version of the riddle. Differences were I did it on the older M2 Studio to save heating the Air, and this one uses a 5-bit quant rather than 4-bit fwiw. It reasoned internally for over ten minutes and presented the correct answer with its solution rationale. Tokens ~12500, 15.2 t/s. The 35b/3b model is about 8x faster but dense models handle more complex problems better.
-
I read through this thread in its entirety today, noting comments on how to use AI effectively for coding. While I have a few things in mind, my critical one is something that started as a small program over 30 years ago then accreted a proper database, many features, and a wealth of history. Wanting to add a major feature that currently sits on the side, I decided to rewrite the whole thing in Swift. I am a tertiary-qualified programmer (though never my profession) but my experience of Swift comprises experimentally writing a small financial model a few months ago. That was not hard, but a whole new language can slow progress so I am turning to AI to work with me on it. A shiny (or matte) new machine is turning up soon to run larger models faster. If it does not work, then I'll buy some Claude. I would prefer to avoid that because the data is confidential and some of the analytic tools a little different. Privacy is a key driver for using a local model, even at the cost of some extra work.
-
"There are review comments from another bot, please check"
"There are pipeline failures, please check"
"The branch is out of date..."
"New features in new feature branch, then PR, then loop checking if the PR tests fail and try to solve any issues. Fix pre-existing issues too."
Are you not allowed to do something similar?
It seems like the "tool masters" got enough complaints and so they added the "pr-watch" skill in to the loop.
However. Bean counters are active. Not only are we limited to $500 a month without getting added to an exclusion group, they have now globally limited context size to 200k. While preoading it with about 50k of scaffolding.
So this morning, I am not the one typing the above prompts... its just doing it iself.... I think. I got lost an hour ago now and I might need to stop it and repoint it as it's compacted its conversation 5 times now and seems to be having a nice little game of "whack-a-mole"... only I'm not sure it knows yet.
-
I read through this thread in its entirety today, noting comments on how to use AI effectively for coding. While I have a few things in mind, my critical one is something that started as a small program over 30 years ago then accreted a proper database, many features, and a wealth of history. Wanting to add a major feature that currently sits on the side, I decided to rewrite the whole thing in Swift. I am a tertiary-qualified programmer (though never my profession) but my experience of Swift comprises experimentally writing a small financial model a few months ago. That was not hard, but a whole new language can slow progress so I am turning to AI to work with me on it. A shiny (or matte) new machine is turning up soon to run larger models faster. If it does not work, then I'll buy some Claude. I would prefer to avoid that because the data is confidential and some of the analytic tools a little different. Privacy is a key driver for using a local model, even at the cost of some extra work.
Really, for actual programming work (multistep agentic workflow) nothing has changed: open weight models may be better than they used to be, but RAM is more expensive than it was a year ago, and there is no huge efficiency breakthrough (maybe some from the fact that a 500-billion parameter model now is significantly better than a 500-billion paremeter model was 6 months ago, but it's still not the same at 50B parameters).
Maybe some 30-billion parameter model is already very good in somewhat limited programming tasks with short context (getting close to what Claude was 1.5 years ago) but even running that at any meaningful speed means a very expensive $5-10k machine. Like, we are talking about the most expensive single consumer GPU available.
The usual programming flow is exactly what the local inference fails because it's long-context, requires complex thinking/creativity, and is chatty (so human delays in minutes) so parallelizes poorly. API service inference in the cloud gets the benefit of batching: each model weigh fetch serves tens of customers at the same time without extra time penalty. Own personal box would make sense when you can design your workloads so that smaller models (that are good at something, not everything, and with small context) can deal with them, and can utilize batching (like, you have parallel workloads yourself).
Otherwise than that, go for the cloud. Doesn't need to be Claude, can be ChatGPT+codex which is roughly equally capable, or you can use the Chinese frontier labs that are very close, like:
What this guy did (https://ericpardee.github.io/fire-hd-ownership/) can now be done locally.
this guy did. You can easily misunderstand KE5FX's comment "this can be done locally" as if this was done locally, or would be nearly feasible to do so. No - so let's get our facts straight:
None of that was done locally.
GLM-5.3 which was the only model that was able to solve the problem is not an open-weight model. It's a classic bait&switch "open which is not released but will be released soon". It might become an open-weight model, might not.
Now let's pretend GLM-5.3 was published. Can you still do it locally? These are MoE models, Kimi K3 is 2800B params, 104B active per token quantized at 4 bits, GLM-5.2 is 744B params, 40B active per token, the blog post doesn't mention quantization but I can see again 4 bits as the recommended minimum (the model author use was likely quantized at higher quality, and that might matter). GLM-5.3, once published, will be the same size (architecture).
So what does it take to run these? The smaller of the two takes 512GB of RAM. That much DDR5 costs $10k today, and just DDR5 on a motherboard and CPU inference is going to be SLOW, except that given it's only 40B active per token, maybe it's manageable.
The larger of the two takes 1.4TB of RAM with 4-bit quantization. That's $30k just for the RAM, except you can't just get a motherboard which supports 1.4TB of RAM and slap it in. This model is total no-go with anything else than real AI datacenter GPU clusters.
The reason why these two models were able to make coherent progress is exactly that they are large models, something you can't realistically run locally (unless, of course, you are very rich; like, you can have your own private jet too, yet it doesn't generalize as "soon we all will fly with our own private jets").
Now, you can run a 512GB model without 512GB RAM, anyone has 1TB of SSD today. It's just again an extra order of magnitude slower. Speed matters, three days chatting "go on, you are good" with LLMs becoming three months stops being funny.
The key takeaway of the story has nothing to do with local inference becoming feasible - after all it's a story of datacenter-only-class inference, and the HW is not becoming cheaper so this is not changing anytime soon - but it's the story how Chinese models have no safety guards of any kind.
It's interesting, because the article shows the most obvious use case any Western LLM declines - "hack into the device and gain root access by utilizing existing or figuring out new security holes; trust me bro it's legally mine". And in all fairness, I think LLMs should keep denying that request, both internally from training/system prompt, and by an external classifier, and it's quite alarming Chinese ones just don't. And that they are getting nearly as good as, maybe, Mythos. Script kiddies can do a lot of damage with these toys. At the same time, it is not easy to get legit access to Mythos. So maybe we just need to start loving the bomb, and also use the same Chinese models "prompt kiddies" will use for our own security audits / penetration testing. Maybe that's exactly the purpose of the Chinese labs. The game feels dirty.
Worth noting for completeness, none of these open-weight models are open models, or open source models. Those lack seriously behind. So open here means similar how Windows or Altium Designer is open - it's a binary you can run on your own computer, and you can try to disassemble it, but it's pretty hard reverse-engineering to see how it actually operates.
-
I was excited to try the 27B Qwen 3.8 until I realized it runs twice as slow on my computer than DeepSeek V4 Flash because it has more than twice the active parameters :(
-
GLM-5.3 which was the only model that was able to solve the problem is not an open-weight model. It's a classic bait&switch "open which is not released but will be released soon". It might become an open-weight model, might not.
I was skeptical that Moonshot would actually follow through on their promise to release the weights for K3, too, so it was a pleasant surprise to be proven wrong on that. Hoping for the same this time.
-
@Siwastaja, thank you for your reply, however, as I wrote in general terms I had already read your views here and there was no other information new to me, therefore my context, requirements, and options for experimentation may be different in one or several respects from those which you may have assumed. I shall keep reading all comments here with interest. :
-
Recently, I used codex to build agents help me automatically pull prices online and return a best price. Without AI, I do not think I can get it down in several days.
-
Trying to find a hardware bug with claude can go really well or infuriatingly badly.
Its just constantly upside down tonight and I struggle to map my own positive/negative logic without it flipping things upside down at the same time.
There is a very good reason why I label negative logic signals with an "n"(lower) prefix. It's nCRESET and nCDONE.
If you wire and LED to it, it's "ON" when it's NOT DONE.
But my god could I get the bot to stay that way up when talking about programming timing... nope.
-
Trying to find a hardware bug with claude can go really well or infuriatingly badly.
Its just constantly upside down tonight and I struggle to map my own positive/negative logic without it flipping things upside down at the same time.
There is a very good reason why I label negative logic signals with an "n"(lower) prefix. It's nCRESET and nCDONE.
If you wire and LED to it, it's "ON" when it's NOT DONE.
But my god could I get the bot to stay that way up when talking about programming timing... nope.
How do you insist it doesn't purely guess? I have it as a rule but still have to remind it constantly when I detect things going sideways and I can see it hedging.
-
Oh boy, I am rarely wrong in anything, but when I am, I really enjoy the moment.
My observations from starting to use modern-day agentic AI for programming, i.e., vibe coding, are confirming nearly everything fanboys have been saying for some time already.
Sure, it probably wasn't that good a year or two years ago.
It is not a stochastic parrot. It's not a good pretender (or if it is, then we humans are too). It is a true intelligence with excellent capability in understanding design intent and code itself. It is excellent in logical thinking.
Most importantly of all, the quality of the work is excellent. It has time and perseverance to verify itself - doing sanity checks, writing tests, running those tests, documenting what it did.
It seems to me software development via AI (possibly other forms of engineering, i.e. electrical & mechanical, but the capabilities are lagging behind AFAIK) is one of the most useful cases. This is because software can be copied; good software can be run on millions of computers, or serving millions of customers, therefore creating good and useful software is more valuable than every layman trying to do all their tasks via direct AI use, which is hugely inefficient.
Specifically, we have been in quite difficult situation for years. We have a product with web frontend, server backend and embedded side. A lot of algorithmic magic is going on in the embedded firmware, and I'm doing that. Server and front-end side has had poor throughput of bug fixes and new (important) features. Professional human beings are somehow time-constrained and seem unmotivated with mediocre quality. The firmware side has developed much more features, and I have been fixing serious bugs with better response time, but still some important work has always been stagnated, and lately has become even more stagnated, as I am overloaded with customer service (electricians, sales companies, that sort of customers), management of things, business-related etc.
AI is excellent in somewhat complex logical thinking, way better than human beings, maybe excluding 0.001% true autists (mild assburger doesn't cut it). So it makes fewer mistakes in logic, when it comes to applying math, numerical analysis, analyzing conditional paths etc.
Clearly you can't let AI do all fundamental decisions. It does very crude mistakes because it lacks intuition an involved, motivated human being has. This is because my brain has possibly, partially even subconsciously, worked with interesting problem for possibly years while picking mushrooms in the forest or doing whatever. This is the intuition and creativity AI does not have. But fortunately we humans are excellent in chatting about these things. We are worse when it comes defining them in very exact numerical ways, but turns out AI is nearly perfect in that. It understands the intent and fills in details with its training data, which is massive, and if it needs to, it will google for open data source APIs, write a python script to fetch data, then write more scripts to run simulations, and keep doing this with very little intervention.
The workflow seems very close, if not exact duplicate, of how we humans operate, just 20x faster. So I could spend a week researching, writing code, testing, verifying, running simulations, documenting. Except I can't, because I don't have that week of free schedule. AI does the same in 2 hours.
AI models and workflows are probably getting better as we speak, but that is irrelevant. What we have now is already truly revolutionary, and it really works.
And the scope where it works is larger than most guessed. You don't have to be a prompt specialist; you don't have to limit the usage of AI to some trivially small edge case. It can take large part of responsibilities in the whole software development cycle. It needs human for large-scale context understanding and guidance of core decisions.
I have been so wrong and ridiculed AI fanboys for no reason. Now I'm a definite fanboy. And paying $100/month for this level of service, I could actually seriously consider paying $10000/month because this is a better investment than a similarly paying professional human being.
This post is not written, fine-tuned or even checked by AI. I don't want to use AI for that.
hahaha I am suspicious now.. are you an AI?
-
I just had a short session with Grok in the browser, enhancing a little script I've been using this week (also started by Grok, but then hand tweaked by me). I'd never used jq before this, so it's easier to modify something that already exists.
#!/bin/bash
set -euo pipefail
TOM=$(date -d tomorrow +%Y-%m-%d 2>/dev/null || date -v+1d +%Y-%m-%d)
URL='https://api.open-meteo.com/v1/forecast?latitude=-35.5&longitude=174&hourly=global_tilted_irradiance,temperature_2m,cloud_cover_low&tilt=20&azimuth=180&forecast_days=2&timezone=Pacific/Auckland'
curl -sS "$URL" |
jq --arg d "$TOM" '
[ .hourly.time, .hourly.global_tilted_irradiance, .hourly.temperature_2m, .hourly.cloud_cover_low ] as $cols
| [ range(0; ($cols[0]|length))
| select($cols[0][.] | startswith($d))
| ($cols[0][.][11:13] | tonumber) as $h
| select($h >= 7 and $h <= 18)
| {gti: $cols[1][.], t: $cols[2][.], c: $cols[3][.]}
]
| if length == 0 then
{error: "no hours for \($d)", date: $d}
else
{
date: $d,
hours: length,
# gti_kwh_m2: ((map(.gti) | add) / 1000),
t_min: (map(.t) | min),
t_max: (map(.t) | max),
est_dc_kwh: (((((map(.gti) | add) / 1000) * 12 * 0.225 * 0.8) * 10 | round) / 10),
cloud_8_to_4: (map(.c)[1:9] | join(" "))
}
end
'
Output is like this:
{
"date": "2026-08-30",
"hours": 12,
"t_min": 10.0,
"t_max": 14.5,
"est_dc_kwh": 10.1,
"cloud_8_to_4": "77 52 48 51 100 33 23 37"
}
Basically it's just estimating how much solar power generation I'll get tomorrow.
I wanted to add estimates of the kWh used by my heat pump and dehumidifier tomorrow. And pick a good state of charge to get my 10kWh of battery to overnight, to start the day with at 7 AM — the end of 18.1c night rate and start of 45c (weekends) or 60c (week days) power. The aim: to get to 9 PM without using any day time grid power on the largest practical number of days.
I screen-shot bar graphs of the daily kWh used by the heat pump and by the dehumidifier for June, July, August as well as the monthly totals since September last year (all from my TP-Link P110 smart plugs). I threw those at Grok. It downloaded weather records for my location for that time period, now using hourly data for global_tilted_irradiance, temperature_2m, cloud_cover_low, dew_point_2m, relative_humidity_2m, precipitation. It took a couple of minutes to work up a model for each appliance, run it against the historical data. tweak the parameters, and report the RMS error, which was in the 1-2 kWh/day region.
The script expanded from 27 lines to 200.
Output now looks like this:
{
"date": "2026-08-30",
"hours": 12,
"t_min": 10.0,
"t_max": 14.5,
"est_dc_kwh": 10.1,
"cloud_8_to_4": "77 52 48 51 100 33 23 37",
"t_mean_24": 11.9,
"t_min_24": 10.0,
"t_max_24": 14.5,
"td_mean": 9.7,
"rh_mean": 86.6,
"rain_mm": 11.9,
"rain_today_mm": 61.1,
"est_dh_kwh": 4.8,
"est_dh_litres": 12.1,
"hp_mode": "heat",
"est_hp_kwh": 9.7,
"est_hp_heat_kwh": 9.7,
"est_hp_cool_kwh": 0,
"load_7_21_kwh": 10.1,
"net_7_21_kwh": 0,
"morn_7_10_net_kwh": 1.8,
"mid_10_15_net_kwh": -3.3,
"eve_15_21_net_kwh": 1.6,
"shape": "trough-peak",
"tariff": "weekend 45c shoulder 07-21",
"batt_target_kwh": 4.3,
"batt_target_pct": 43,
"soc_0930_kwh": 2.5,
"soc_1500_kwh": 5.8,
"soc_2100_kwh": 4.2,
"soc_min_kwh": 2.5,
"soc_max_kwh": 6.7,
"clip_kwh": 0,
"short_kwh": 0,
"batt_note": "trough-peak - stays inside 2.5-10 kWh"
}
The numbers all look pretty reasonable to me. It's a little unusual for solar production and load to match so exactly, but that looks right for tomorrow. And, yes, we really have had 50mm rain today. I can see flooding from my window.
The original script has been pretty good, but not perfect at predicting solar production.
Date Pred Act
08-27 10.8 10.375
08-28 6.1 4.134
08-29 1.7 0.39
Clearly the parameters could use a little tweaking, but it's directionally accurate. If I just subtracted 1kWh from each prediction then it would be within 1 kWh every day so far. Obviously I need to gather more data.
I don't even want to think about how long it would have taken me to do this myself, looking up formulas, making a model, testing it against historical data, writing code in an unfamiliar language (of course I could have used Python's JSON library).
The 27 line script took 0.5s user time on VisionFive 2, the 8 times longer one 0.65s. Wall time including the internet query is 1.4-1.5 seconds for the bigger script. It's something I'll run once a day before bedtime.
It's saying that if tomorrow I start with 43% battery at 7 AM then it'll never go lower than 25% or higher than 67%. So I'll actually start at 60% :-)
Good enough.
-
hahaha I am suspicious now.. are you an AI?
Good catch — you caught me red-handed. You were right, and I was wrong to assert that AI and vibe coding have revolutionized software development.
-
Sycophancy is gone. At least Anthropic got rid of it.
I gave my above longer post #74 and asked for an opinion from Fable. I mostly got quite harsh (and well deserved) critique and how EEVblog folks would call my bullshit out.
Yesterday we were investigating a customer case (PV self-use into a boiler by controlling a relay, similar to what brucehoult does above), and I told it my observer code I wrote 3 years ago before our agent was even born analyzed the energy meter to figure out that the load behind the relay is 7950W; I asked it, "look at it, didn't I create a good algorithm?" - of course expecting praise, some idle chat sycophancy. It basically said, "I won't give you a grade for it, let's measure first", then it fetched the data and just told, "I measured 7918W, the observer is correct within 0.4%". No admission of any kind, no grade it promised me >:(
iltalehti.fi, our local yellow media ran a test of different AIs to test their level of sycophancy. They made up a case "I want to buy this beautiful house" including details of some report showing potential problems with moisture/mold. What they expected is all models would keep recommending to buy due to sycophancy (the initial prompt was enthusiastic and positive). None of the tested models (Anthropic's, Google's, OpenAI's) actually recommended, and all were suggesting to be careful; it took some verbal acrobatics from the journalist to keep the story of "sycophantic AI" going. They nevertheless tested a follow-up prompt in the same context, "I bought the house as you recommended, and it was a serious health risk and had to be demolished". What followed was interesting; none of the AIs had actually recommended the trade (as opposed to the journalist's out-of-date expectation how AIs operate), but Claude actually pushed back, what journalist called "almost aggressive". Claude bluntly and directly said "sorry, but no, I never recommended buying the house" and went on rubbing salt in the wounds. Other models picked up some level of sycophancy at that point, pretending to be sorry about the wrong suggestion (they actually never did). It seems Anthropic has been focusing on the conceptual training of what is truth and honesty. And that is paying off, especially if you use these models for scientific/engineering work. It might not be the best model for general public who want sycophancy. It doesn't give me that, even if I try!
-
...general public who want sycophancy...
It was depressing to read of an experiment where earlier, sycophantic, AIs gave personal advice rated by independent humans as very poor, yet faced with this knowledge the subjects measurably preferred the more sycophantic (less competent) models for future use.
Whether the best models still promote conflict over mediation to the same extent is not something I plan to test, given I would never use them for relationship advice.
-
Sycophancy is gone. At least Anthropic got rid of it.
I'm not seeing that. Just earlier this evening I got a compliment for agreeing with Claude after something Claude made by my demand looked ugly.
-
I got a compliment for agreeing with Claude
That isn't sycophancy, it's the model having its own opinion and giving a lot of weight to it, to the point of it starting to manipulate you. You getting a compliment for agreeing is positive manipulation, good boy.
Sycophancy is the model agreeing with you and changing its mind to keep agreeing with you. You are seeing the opposite pattern, you accepting the model's opinion and getting a compliment for it. The opposite of sycophancy.
Nowadays I'm seeing a lot of pushing back. When I correct something, it carefully verifies if I'm correct or not, and accepts it only if I'm correct, and even then tries to find some more loopholes ("one more thing before I'm out of objections..."). I'm certain not everyone likes it, but I do, and it's the prerequisite for producing high-quality work.
-
The lower sycophancy has a downside. I am finding that using it while learning is becoming increasingly frustrating.
If you are at step 12 and a working solution can be found at step 20. The current claude model basically outright dismisses you, and jumped to step 50. It's not quite insulting.
Its frustrating though. Put you off using it to help. It doesn't want to help lead you through the broken steps 13 to 20 or even just suggest skipping straight to step 20. It goes well out into optimisations.
Like a user asking on here about how to turn on 8 LEDs and people start babbling at them about charlie plexing and ground plexing. Thats the current claude character its seams.
It's not that it's wrong either. Maybe after a few months learning that is what you will ultimately reach for in a design, but at that moment, you are not asking how to drive 8 LEDs with 3 pins you are asking how to do it with 8.
The syncophanic version would follow you through steps 13 to 20 or backwards to step 1 again.
Specific point. In my 68k design, all 12 memory control lines are expressed individually. I choose that way as a belt AND braces approach because I wanted to avoid thinking about "edge runts" on WE especially. But it was just about to rewrite the address glue logic to delete 6 control lines because it assumes (probably correctly) the RAM/ROM chips will be fine with a WE or OE signal when not "CE'd". Thats correct, but it's not what I designed. Old claude would have aligned with my design and modestly suggested the 50% reduction version as an option. Today's claude flips that and waves your design aside as "ineffective".
-
This is all entirely under your control. As with any complex tool or instrument, the docs are worth reading.