Is ST Cube IDE a piece of buggy crap?

[rsjsouza]

As an example.  Many of our build scripts include a git hook to find tagged collateral and use it to generate code.  However the way that is presently implemented causes it to ask for credentials.  "ENTER, ENTER" is all that is required.  However if you accidentally let InteliJ build the application "first" it will run this build context and 100% hang the process with no way to recover it, except to restart the whole IDE.  There is no way to inject those 2 line feeds into standard in for the build and InteliJ has not enough balls in it's process control mechanisms to even kill it.  Windows mandatory locks also prevent the normal build for running concurrently.  The work around is to quickly build the application from it's own build script (maven) so that those files are generated and the InteliJ auto build systems don't "declaratively" look to building it themselves.  There are many others.

No tool is perfect, but at least in Eclipse you have the option of running pre- and post-build steps to handle these cases.

My first contact with Eclipse was around 2004~05. Its changing perspectives and very similar UI woes of other nightmarish tools of my then recent past (Mentor and Java Swing applications on SunOS systems, for example) made me absolutely despise it. The bugs and terrible performance of these earlier versions did not help as well. A few years later I was forced to use it (our tools were overhauled to use it) and, despite its many flaws, I learned to appreciate its vast improvements, especially in the editor and code navigation fronts. I nowadays look at other tools that use a very similar interface as our pre-Eclipse versions, and it only has the advantage of performance - otherwise it is quite limited in functionality.

Nowadays the "powers that be" are moving away from Eclipse and going the VSCode-like route - I despise the new thing, but perhaps it might be simply an adjustment phase. Or perhaps I am getting too old for this. Only time will tell.

[paulca]

And I bet that 1k IDE has a license / checks it online every time you start, etc, so a project cannot be archived, revisited 10 years later, etc.

Well, yes and no.

All the projects I have created in my career are IDE agnostic.  The reason is simple.  We do not permit developers to do builds.  Builds get done by infra and by automation in secure environments that not even the devs are allowed to tamper in.  Easter eggs could be a criminal offence.

Where IDE integration and dependency does emerge is in developers local environments.

I have been in several projects where Eclipse was the defacto standard.  However the team tackling it now are using InteliJ.

We found a lot of the "tooling" used to spawn local instances, to init DBs, to run test harnesses and stubs was all based on Eclipse plug in configs.  All coupled up into complex "build.bat" scripts behind the scenes.  A right Fing mess.

In projects that got this "better/right".  They created a seperate repo called "projectname-sdk" and put all the non-IDE specific harnesses et. al. in there.

[paulca]

Nowadays the "powers that be" are moving away from Eclipse and going the VSCode-like route - I despise the new thing, but perhaps it might be simply an adjustment phase. Or perhaps I am getting too old for this. Only time will tell.

Yes.  There is a massive lure towards VSCode-like interfaces.  One HUGE advantage that IDE has over most others is that it's very easy to wrap it into a web application and serve the entire IDE remotely from the infra.

It has popped up in many places from "Code academy" style learning/training sites to "Code interview" environments for recruitment.

I did an OWASP security deep dive course and the practicals blew my mind.  You have, via a website, access to an actual running version of the full application stack via an online IDE (VSCodeWeb) ... a Kali Linux VM (By remote desktop web) and the application itself via an independant virtual browser.  They required you reprodue the vulneribiity by hacking the REST communications with Kali, then fix the code to remove the vulnerbility and prove it worked.  It was then able to independantly verify you had fixed it.

I even tested how smart it was by fixing only the vul in a very unexpected way to see if it was just looking for code keywords.  No it wasn't.  It passed the vul test, but failed the unit tests.

[rsjsouza]

Nowadays the "powers that be" are moving away from Eclipse and going the VSCode-like route - I despise the new thing, but perhaps it might be simply an adjustment phase. Or perhaps I am getting too old for this. Only time will tell.

Yes.  There is a massive lure towards VSCode-like interfaces.  One HUGE advantage that IDE has over most others is that it's very easy to wrap it into a web application and serve the entire IDE remotely from the infra.

Quite interesting and surely meets the current trends for web/cloud model.  Given the SW development for me is done locally with the traditional collaborative tools (Git, bitbucket, etc.), this is not an edge at this time. But perhaps it could change - who knows.

We already have another tool that is completely based on node.js, which is still used locally but there is a fully functional web version as well.

Quite interesting times.

[peter-h]

One HUGE advantage that IDE has over most others is that it's very easy to wrap it into a web application and serve the entire IDE remotely from the infra.

One can run Cube IDE over RDP perfectly well. What would be the application for having a dev kit as a website?

[paulca]

One HUGE advantage that IDE has over most others is that it's very easy to wrap it into a web application and serve the entire IDE remotely from the infra.

One can run Cube IDE over RDP perfectly well. What would be the application for having a dev kit as a website?

When you have 10+ users?

[peter-h]

Sure, but then what are you doing?

You are running a hosted editor/compiler kit, basically. Which, with 10 users, running on a high end PC, will be bloody slow. You will need a linux version and set it up on a fairly fast virtual server. People will upload their sources, edit them online (there are major challenges in achieving a decent hosted editor, but it can be done), build them, and download the code back locally to run on their target. The target could be remote also, with (this is tongue in cheek but not wholly) a webcan looking at the LCD display.

It's a viable setup for something, but I am not sure anybody wants it.

Think of security issues, for a start...

[paulca]

It's kinda complicated to explain how it's nowhere near as resource intensive as 100 people trying to use the same VM.  There are many flavours and customisations on the components.  I believe GitLab now includes an "Edit in IDE" button for files in the webUI.

On security.  Whose?

Consider this allows people to bring the developers to the code rather than the code being distributed to developers.  Thats orders of magnitude more secure ... in "that" particular sense.

Running a test environment behind the IDE is more just standard modern CI/CD where we create and dispose of environments constantly.  Every branch or every commit can have an environment spun up for it. 

Can get expensive. 

[peter-h]

OK, VMs are slow. 100 VMs actually running needs a massive amount of power and memory. I wasn't thinking of 100

Anybody can do screenshots... so the security would be in blocking untrusted coders from parts of the project.

Who would want this?

[paulca]

Not VMs.  Containers.

Containers can be crudely described as "light weight VMs".  They share the running kernel using the various supervisor modes available to virtualise the hardware.

So while a VM typically needs several Gb of memory to even boot (A standard Ubuntu Server distro will fail with under 2Gb) a container only needs the memory the actual application allocates.  Other than some memory filesystems and caching of course.

Running "GUI" components within containers and then hooking that to a remote web "head" in a browser is where the magic happens that makes this architecture viable.

On a busy Friday afternoon you might have 100 devs banging at these containers and your ops team will be "buying" spare capacity instanes off the cloud provider to expand the resource pools.

However on Saturday morning you won't find a single pod running an IDE, except those unlucky enough to be in "ops" or on a call out.

Come Monday morning the infra expands again.

It facilitates extreme "Thin clienting".

Here is a demo of one showing that the bulk of the IDE is actually just running in your browser, in the case of VSCode in this flavour.  Running a Kali linux desktop with limited functionality over the web is a bit different.
https://vscode.dev/

[NorthGuy]

Fantastic. First you create an IDE which is incapable of maintaining the state of your project across versions.  To solve this problem, we now want to run it in VM to keep the entire IDE intact. Then we get a problem with too many VMs, so we move to the cloud with containers, browsers, and paid subscriptions. Now we need 5G to sustain all the traffic, and perhaps AI to manage accesses by different developers in real time would be the next step.

Or just manually write a Makefile or a build script and use it in place of the crappy IDE. Take your pick.

[Siwastaja]

Fantastic.

Human beings are very creative. There appears to be no limit to the complexity and difficulty we are capable of getting ourselves into.

I sometimes joke about interfaces where data is base-64 encoded into XML, printed on a paper, to be telefaxed, OCRd and turned back into original binary. But I have to be careful where to suggest something like this. Not everybody understand it as a joke, it's a real risk it's treated as a serious suggestion.

[betocool]

Fantastic.

Human beings are very creative. There appears to be no limit to the complexity and difficulty we are capable of getting ourselves into.

I sometimes joke about interfaces where data is base-64 encoded into XML, printed on a paper, to be telefaxed, OCRd and turned back into original binary. But I have to be careful where to suggest something like this. Not everybody understand it as a joke, it's a real risk it's treated as a serious suggestion.

This is something to tell your line manager... he might actually approve it if you explain that as "encryption" 

Cheers,

Alberto

[SiliconWizard]

Fantastic.

Human beings are very creative. There appears to be no limit to the complexity and difficulty we are capable of getting ourselves into.

I think we're able to go to great lengths just to stay in our comfort zone. That IMO is the key factor.

[peter-h]

"Comfort zone" is also same as "protecting business investment"

Sure; new tools put bread on the table at home for salaried devs, but the company keeps paying for it, in man-hours and often in projects which cannot be maintained.

This can be quite serious. I can give you an example of an autopilot - the KFC225 - whose sources cannot be rebuilt, since about 2003, and which contains serious bugs.

[NorthGuy]

"Comfort zone" is also same as "protecting business investment"

One man's "protecting business investment" is another man's "putting good money after bad"

[peter-h]

Of course that is true, but equally are an infinite number of one-liners

The criteria can be quite complex. Just picking one aspect, I would choose Cube IDE for the ability to archive a project and rebuild it 10, 20 years from now, possibly in a VM. Obviously the tool has to be actually usable but this one certainly is.

[paulca]

On modern software infrastructure and scaling.... the later word you just won't get from MCU dev.

Lets look at it on it's bottom line.  You are developing in a fixed eco system (fixed in hardware over decades) which is SO SIMPLE, you can start from complete, bare bones, scratch every time.  A SINGLE developer can hold the scope of nearly any MCU project in his head and you can contain it all in a single project in a single IDE with a single repo.

Pause and consider how lucky you are.

The last time the "real world" software industry had the luxury of starting again from scratch was about 1970 somewhere when C and Unix were born.

The scale at which we develop software in enterprise is many orders of magnitude more complex and more costly.

When you get to 100 times more complex than a single developer can handle or that should even be attempted in a single component, you begin to understand a lot of it comes down to a "people problem".  You could start with 100 developers.  Now, now we start to get into software engineering.  100 developers will not deliver 100 times the complexity in the same amount of time.  "9 women can't make a baby in 1 month."  On top of that for 100 developers you will need nearly as many "support staff" like management, HR, sales, etc. etc. etc.  So now you have 200 people in the mix.  Costs soar.

The engineering is in how do you divide and conqueror that complexity and form it on one side into a sustainable, efficient workstream for 100s of developers and on the other end have it coherently come together and function as intended, be resilient, scalable, secure and maintainable as usually the next guys alone will NOT be starting from scratch because your code was too hard to read.  They will have no choice but to inheret and proceed.  "We don't want to do this anymore, we need a rewrite!" cries take quite a while to be heard and are very often fought at great length due to costs.  So while all software reaches end of life and retires eventually the costs involved make it something that's hard to sell in anything but a small project.

Smartly divided software of course, by design, tries to facilitate "give up and rewrite" in affordable chunks.  As long as you did your "contract" and "integration" design well enough to facilitate "drop in component replacement".  "Replacability".

On scaling tooling....

If you have 100 developers all building the application in their IDEs, this leaves you open to artifacts caused by slight variances in those IDEs, compiler version etc.

So, we don't build releases on a developers PC.  That goes away when you step up to "maturity level 2".  Now you have dedicated build servers which build all releases and put them into a readonly secure repository for deployment.

Now your company wants to scale from 10 devs to 100 devs.  The Devops guys immediately notice the build servers are out of resources and all hell breaks loose.  You fix it by adding a build queue, but devs are complaining at busy times they can wait 15 minutes on a build to start.

Yet at 6pm there isn't a single build running anywhere.

Do you
(a) add enough physical servers to handle 100 devs banging it full steam
(b) virtualise the build servers and only have "as many as you need" running at anyone time?
(c) forget all that manging instance BS, just spawn the build pod, build the application, publish to repo and then delete the entire build pod and forget it.

I'd be going for C and that is how it'd done in most "scaled" dev houses.   

If there is a Sev 1 CVE vuleribilty like the log4j vul that hit 2 years ago, you might have all of your teams rebuilding all of  their services the next morning.  Your entire infra will be teaming with build pods.  Yet by lunch time you are back to one or two build pods at a time.  No broken build servers to fix, they all get mercilessly killed and deleted for their diligent success every time.  Everytime the build server you get is freshly booted.

To say it again.  Until you start to develop software with groups of people you will not see the way you go about it changes dramatically.

MCU land your non-functional constraints are 99% of your design and you are thus shielded from actual complexity as you don't have the capacity to even model it or the memory to contain the state.

[peter-h]

If you really mean 100 devs, or anything over say 10, then it is a very big project. It will be done in C++ because that is what C++ was intended for

It would have to be split up, into functional modules, so each guy can work on something which can be tested, etc.

Cube is not for this kind of work. But what is?

Also with STM32 you would not start from zero and bare-metal. This is 100x the complexity of the Z80 (which I knew extremely well). You would pick up the previous STM32 project and chop out the bits not used. Basically, chop out unused stuff from main.c and chop out unused RTOS tasks. So you preserve nearly all the hard stuff and can get productive in a few days. In my last project, if I remove #define ETHERNET, then all of ETH, TLS, basically 80% of the code I have, falls away.

[NorthGuy]

Now your company wants to scale from 10 devs to 100 devs.  The Devops guys immediately notice the build servers are out of resources and all hell breaks loose.  You fix it by adding a build queue, but devs are complaining at busy times they can wait 15 minutes on a build to start.

Instead of coping with the chaos, you need to create order. Give each developer its own relatively independent task and let him work on it locally. Someone also (not a developer) can gather code and do builds as needed and distribute them between testers.

[Siwastaja]

Now your company wants to scale from 10 devs to 100 devs.  The Devops guys immediately notice the build servers are out of resources and all hell breaks loose.  You fix it by adding a build queue, but devs are complaining at busy times they can wait 15 minutes on a build to start.

Instead of coping with the chaos, you need to create order. Give each developer its own relatively independent task and let him work on it locally. Someone also (not a developer) can gather code and do builds as needed and distribute them between testers.

This, plus reduce unnecessary complexity. Nearly every project has some. It is natural, it builds up during project lifetime and needs to be cleaned.

It is easy to talk about 100-programmer software projects, but really there should be only a handful of those on this planet.

Really! 99% of projects that employ 100 programmers and 200 managers to manage them (these "inverted pyramids" are surprisingly common, I know a few) would be all doable with 10 motivated and well-communicating programmers and 1 manager. Remaining 1% are truly so large projects they do require 100 programmers.

"That does not scale"? Everything does not need to scale. Besides, infinitely adding more developers does not scale either. Also it does not work.

It is quite well known that even in a decently organized project the work output goes roughly sqrt(n) of workers (including programmers and their managers). This means that 100 people do just 2 times the output of 25 people. The important consequence is that if you are able to shave off 50% of the complexity (e.g., with habit of regular re-thinking, refactoring and cleanup), you just dropped 75 unnecessary people. And it's not just cost saving in wages. Such smaller team is easier to manage.

[SiliconWizard]

Yes. Adding more people yields way more complexity than the inherent complexity of the project. Of course there's the seminal "the mythical man-month", freaking published in 1975. Fifty years later, we're still doing the same sh*t, so I'm assuming few current managers have ever read it. Or maybe they've been told that "Agile" would solve all these problems and would make anything written in this book completely obsolete. Yeah, right.

And the other point is that yes, most software projects are made overly complex for no reason other than justifying software positions.

But the main issue is handling software development and engineering in general as you would handle production, using the same approach and metrics. That never works.

[paulca]

Agile failed for people because people failed to do Agile.

Honestly, this will get me angry because of the pain this point and this point alone has caused me this past 5-10 years.  Not just this morning we had another set of "made up" processes inserted into our methodology.  All made up garbage that will not fix any of the problems, but when you point this out, you get gas lit in more management BS.  We had 2 actual end users invite themselves into a UAT hand-over to the business.  They promptly spent an hour and a half saying this and that and the other thing are wrong and should be this or that and clarifying the god damn basic processes involved in the system.  I was livid.  "This is not 1973!  FFS!  We learnt this already"?

Agile is just a collection of tools.  It's an integrated development management "tool".  The tools are each designed to facilitate and aid one aspect of development.

These tools and techniques and policies when utilised together in concert provide more gains together than they would individually.  The process is more powerful than the sum of it's parts.

The most key principles of Agile are there to do EXACTLY as you want SiliconWizard.  They are there to isolate the dev team from management BS and inefficiencies.

The trouble lies with Agile, while brilliant for software development with small teams, is completely incompatible with business methodologies and practices in nearly every company on the planet.

Those for which Agile works, they take the full package and get training and go forward with the FULL agile setup for long enough to see it deliver well.  At that point they understand and just start working their business processes around Agile.

For the 95% of companies the business just cannot tolerate "Time boxed" work streams as the only billing/costing method they have for that is "Fixed rate" and customers do not like being charge fixed rate per hour without knowing IN ADVANCE how many hours.

Agile REFUSES point blank to provide an answer for both When and What will be delivered.  Its basically the development uncertianty principle.  The more accurate you are on WHAT will be delivered the less accurate you can be about WHEN... and vice versa.

Business can't handle this.  The customer wants to know what they are getting and when. 

What happens is they overlay fixed "phases" with fixed "deliverables" and try and play costing games with sprints, should they go a sprint over they negotiate.

That is the first wound to Agile's mechanisms.  It's a deep one though.  Others tend to follow.  Springs become interruptible, the customer gets to redirect work in flight etc. etc.

Fast forward 2 years and the project is over time and over budget and a failure.  They blame Agile.

You cannot understand how angry this makes me.  Please don't do the same.  If you have ever actually worked on a proper full adherence Agile project, then I will listen to you.  If all you have is whining from developers who said Agile failed them, I'm not interested.  Agile didn't fail them simply because they didn't use Agile.  What failed them was their made up, Frankenstein/hybrid methodology conceived by non-technical, IT-illiterate management with no formal education in software development.

For every 1 team I have been in that was actually agile, I've been in 9 which have been some messed up, failing, concoction of Agile + business methodology that doesn't work.

The main issue is simple.  The software industry has expanded so massively in the past 5-10 years and in order to do so we have had to "water down the skills pool" by hirering lesser and lesser skilled and qualified people.  This then needs more and more "good" management.  There is an even bigger shortage of skilled development managers so they hire "any old manager".

Because the engineering mass in the teams is now stretched so thin and most of the management is managing unskilled "coder drones"...  Management start believing they CAN manage a software project, without any skills, education, or even having read the Wikipedia page on the same.  They push their management down onto the team.  The team struggle and suffer.

There is positive though.  I think the lower echelons are already being cleared out somewhat.  AI.  While I have always said that AI can write your code, but you will need a skilled software engineer to tell you if it will work or not and then fix it.  The exact same thing can be said of most "self taught", "I taught myself C/Java/Python" coders.  Literally ChatGPT can produce more complete work than half of them.

The COVID boom is now well behind us and people are shouting about lay-offs "across the big tech industry".  No. No. No.  That is not what is happening.  The industry is shedding all the dead weight in two-cent, one trick ponies.  YouTube was once full of "A day in the life as software engineer", on like Day 3 of their first role, with ZERO qualifications to use that "engineer" term.  Those channels are now full of tears and whining as their lovely 40k position where they do nothing all day has been cancelled.

Hopefully we can return to something where the critical mass off engineer oversight regains power.

[paulca]

Really! 99% of projects that employ 100 programmers and 200 managers to manage them (these "inverted pyramids" are surprisingly common, I know a few) would be all doable with 10 motivated and well-communicating programmers and 1 manager. Remaining 1% are truly so large projects they do require 100 programmers.

I think we both use a different definition of "project".

A project at my level might be "Modernise the ____ government's driver licensing system".

That might end up amounting to 10 years with 100+ people.  On the development side there might be 4 small teams with 4 small test teams.  Circa 40-50 people.  You might have another 10-20 business analysts, managers, DBAs, support, devops, platform, infra, architecture, PMs, etc. etc.

On the "Project" anatomy there are likely to be several large scale central DBs to handle, a sizable platform of servers and infra in multiple locations, probably a dozen or two different services and several/many UIs (Public and Internal UIs).  It's going to involve lots of fiddly integrations like receiving physical letters in the post, receiving bulk form batches from other agencies.  Receiving things like enforcements, disqualifications, fines, points etc.  coming in from multiple agencies.  Printing and automating sending letters.  Ad.inf.

You would not believe how complex these things can be made by governments and courts.

However, "down in the trenches" as we call it, due to the "isled" desks we would typically sit in that look like each small group of desks is a trench and team is fighting from it.  Lots of analogies about sticking your head up over the edge and such.  In that context you have a small dev team of maybe 4 or 5 with 4 or 5 testers, a BA or two and at least one manager.  Often a manager over the whole dev+test+aux and maybe a dev manager and test manager across 2 or more projects.

Those small teams would be tasked with isolated tasks.... or that is the ideal, not always achieved.  Ideally,  "one component, one owner".  While a team can "own" multiple components, no component shall be owned by more than one team at a time.  The complexities in synchronizing the development across multilpe teams into a coherent output ... on time.. is always the challenge.

Now to scale that up again, in 2020-2022 I was in one of these little small teams, but the "bigger picture" project was 1050 poeple.

Why so many?  A tier one investment bank, when asked by regultors what their total markets risk exposure was that day, the bank said, "We'll get back to you on that."  Weeks passed before the report was finally delivered.  The regulator fined them 10 billion dollars for not having a clear picture of their total risk exposure for "that day" or "the day ahead".  They were also told they would be asked each year to produce the same, on a random day and if they cannot provide it they will be fined another 5 billion for each year they fail.

It was stipulated that all data relating to risk exposure should be centrally controlled under the engineering departments and NOT spread across the company in dozens of different business managed applications, databases and departments.

The last "birds eye" map I seen of the project was sitting at 1450 individual data feeds to be migrated to central data storeage with retention, catalogs, inventries and query access.  Over 500 databases and Excel sheet stores to be migrated to actual databases.  We had Excel sheets, MS Access databases, Sybase, DB2, everything.

When the stick is billions of pounds of fines, banks will happily spend 100s of millions on WAY over populating things in order to stand a chance of delivery.  They know full well that a team of 100 might achieve 70% efficiency and a team of 200 might only be 40% efficient, they are not stupid, but if they add another 200 developers to the existing 800, it will still bring in the final delivery dates.  So they do that.

I quit investment banking because of the mess that resulted.  No matter how hard, how long and how loud I tried to help them not make a mess, they over ruled all engineering best practices and started stacking broken copy and pasted projects ontop of broken copy and pasted projects.  At one point I could 205 different copy and pastes from a feed importer.  All had bespoke changes, but 90% duplicated code.  I was looking at production issues and pointing out, "If that feed has that bug, they all do.".  They had plenty of people around the world who were "Yes" men.  "Please hurry and make this look like it works.  Code it as dirty as you like, leave as much technical debt as you like, we just need to tick the box that we delivered it only 4 months late!".   "Ok".  <-- this person and those like them need flogged.

[paulca]

Cube is not for this kind of work. But what is?

Why not?  Open the team menu and have a browse.  Configure it.  Whats the problem?  It will work perfectly with a team.  I have used it across multiple concurrent environments.  I use a VM sometimes and native others, same code base, same project, works fine.

You want a secure build server?  Write your build script for it.  Want it automated?  Use a build pipeline manager like "Jenkins", trigger off commit or branch name.

None of these thing are done in the IDE, because in the real world different developers use different IDEs within the same project.  IDE is up to the dev.  It's their tooling, but everyone is responsible on making sure the build script works.  Literally if you say, in the morning, stand up, "My build is broken.", you will get something sarc'y like "Awww, boo hoo, good luck with that.".  However if you say, "I broke the build.", then you get the "Breaking build" dunce cap for the day (not really), but you will know what your first task of the day is and you are holding everyone up.

What automated deploy and test?  Just set up those environments with the correct programmers, attached devices and test harnesses.  You can even use SerialTCP servers if you must.