Author Topic: A version control system for embedded work, which doesn't need a PhD  (Read 12281 times)

0 Members and 1 Guest are viewing this topic.

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 2575
  • Country: nz
  • Formerly SiFive, Samsung R&D
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #175 on: August 24, 2021, 01:37:38 am »
If you really hate git for whatever reason, use at least Subversion (SVN). It fixes both the lack of atomic commits (the entire commit either passes or fails, it is never partial as can happen with CVS which operates file-by-file), the repository is much more stable than anything CVS ever had and merging is more sane. And it is actually still maintained.
As an SVN user: it was a learning curve but I like git much better nowadays. The integration into Eclipse (which is what the OP is basically using) is really good. Either way in my experience it is not a good idea to do merges blind; I always do these semi manual by using the GUI from Eclipse to select which chunks use from a different branch and which not.

It's probably been a while since I said this, so here goes again.

If you have to merge branches that both have significant development on them (and a high probability of clashes somewhere) then use Micheal Haggerty's git-imerge script.



It rapidly merges the (initial) things from each branch that *don't* conflict, and then shows you the exact pairs of commits from each branch that *do* conflict, with all previous non-conflicting changes from both branches already applied.

You get to resolve the conflicting changes in their proper context, which makes it much much easier. And you do it once.

Conflicting changes to the same bit of code are not always easy to resolve and this is NECESSARILY SO, completely independent of any RCS. One person decided that code needs to be modified to do something different, and someone else modified it to work in a 3rd way. Sometimes you can find a way to achieve both of the objectives, but sometimes they are simply at odds with each other.

For example someone in your company modifies a buffer size from 1 KB to 256 bytes to decrease memory usage on some small machine, while someone in the upstream project modifies the same buffer from 1 KB to 16 KB for better performance.

That's an utterly trivial example. In this case you'd no doubt decide size beats speed in your application and keep the 256 byte change. There are far more complex examples which are hard to understand what is happening and the ramifications, let alone the right solution. But having the ACTUAL COMMIT that conflicts in each branch in front of you, along with the commit messages, makes it much easier to figure out what is going on. And without the 2000 later commits in the upstream code that might have modified the same lines of code again later for another reason.

I've used this to keep private forks of LLVM (for RISC-V and for a secret CPU at Samsung) up to date with upstream development and it's an absolute life-saver.

git is amenable to 3rd party tools like this, because it exposes the plumbing and everything is local until you decide to push your dozens or thousands of commits (whether to your colleagues or to upstream). SVN isn't.
 
The following users thanked this post: mrflibble

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 22035
  • Country: nl
    • NCT Developments
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #176 on: August 24, 2021, 08:20:12 am »
If you really hate git for whatever reason, use at least Subversion (SVN). It fixes both the lack of atomic commits (the entire commit either passes or fails, it is never partial as can happen with CVS which operates file-by-file), the repository is much more stable than anything CVS ever had and merging is more sane. And it is actually still maintained.
As an SVN user: it was a learning curve but I like git much better nowadays. The integration into Eclipse (which is what the OP is basically using) is really good. Either way in my experience it is not a good idea to do merges blind; I always do these semi manual by using the GUI from Eclipse to select which chunks use from a different branch and which not.

It's probably been a while since I said this, so here goes again.

If you have to merge branches that both have significant development on them (and a high probability of clashes somewhere) then use Micheal Haggerty's git-imerge script.

It rapidly merges the (initial) things from each branch that *don't* conflict, and then shows you the exact pairs of commits from each branch that *do* conflict, with all previous non-conflicting changes from both branches already applied.

You get to resolve the conflicting changes in their proper context, which makes it much much easier. And you do it once.
That is exactly what Eclipse can do as well. The problem however is that when multiple people are working on the same code, not all changes made are good ones. So those need to be checked / filtered out. And then there are conflicting changes and stuff that needs to stay instead of being thrown out during merging. Again: in my experience with working on the same code by multiple people is that you really want to do merges manually / closely supervised. After such a merge I check the result against the branch(es) that got merged AND the previous version to see if all changes made it in an nothing got deleted.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 2575
  • Country: nz
  • Formerly SiFive, Samsung R&D
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #177 on: August 24, 2021, 09:25:25 am »
I don't think Eclipse helps with merging different branches with dozens or thousands or commits against each other. One commit, sure.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 22035
  • Country: nl
    • NCT Developments
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #178 on: August 25, 2021, 09:07:34 am »
I don't think Eclipse helps with merging different branches with dozens or thousands or commits against each other. One commit, sure.
I do use it to compare versions of large projects (like the Linux kernel) against various branches / commits and cherry pick what I want to keep or remove. The 'Team synchronising' perspective is quite useful. I'm not saying it is the best tool out there but it pretty much does what the guy outlines in the video you linked to.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 1290
  • Country: gb
  • Doing electronics since the 1960s...
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #179 on: August 25, 2021, 02:02:52 pm »
I didn't notice that Eclipse (Cube IDE in my case) offers version control, and a search suggests it is done with plug-ins for Git or others.

Funnily enough the main issues I have with Eclipse are with it not rigorously checking datestamps before it does a build. I am often doing edits on a second machine (at home v. at work). I now make a point of treating the work one as "throw-away / test only" and keeping the master at home (it's my own business so I work at home a lot anyway, 7 days a week) and I back-edit changes tested at work to the home machine over RDP. If you edit files on another machine and drop them into the project directory (in or outside of Cube) then Cube gets confused, doesn't reliably notice the later datestamps, and does weird things, so after dropping in anything edited elsewhere one has to do a Clean and then Build.

Same of course if I am doing any edits on a laptop while on holiday (you can tell it's my own business; no normal employee would be doing any company stuff on holiday ;) ) and then again dropping files back in is fine so long as you do a Clean and Build.

Oh and a re-Index is generally a good idea anyway.

And I keep a totally rigorous regime of backups (the whole 150MB project directory, not incremental like a VCS does) after every substantial bit of work, and certainly every day, to a separate physical device. Plus a 3am auto backup to an off site machine but that one keeps just the one copy.

Of course if I break something then I can't easily just step back versions until it works again. I can only go back to the last whole-project-backup.

I have seen the compare tool in use and it is really impressive but have never worked out how to use it. So far I use the Compare 2 files macro in notepad++ :)
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 90S1200 32F417
 

Offline dunkemhigh

  • Super Contributor
  • ***
  • Posts: 3700
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #180 on: August 25, 2021, 02:28:00 pm »
SCM would help with syncing both machines. Check in at one, update at the other, bosh. You can ignore any intermediate files that get built (*.o and the like) and just do a full build after the update. Only thing to remember is to check in before you leave home, but you can make that become a habit and it's good for you anyway :)

The compare tools... I hate them. It can be difficult to figure out what you're getting from the inputs. (I have the same issue with Word's track changes - my missus has the thing displaying all the time and easily knows what it's all saying, whereas it's just a confusing mash to me.) My preferred compare tool, which plays nicely as a plug-in to other apps) is Beyond Compare:

https://www.scootersoftware.com/features.php

Too many features and uses to list here. It's my go-to compare/sync tool and a mere right-click away.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 22035
  • Country: nl
    • NCT Developments
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #181 on: August 25, 2021, 03:50:36 pm »
I didn't notice that Eclipse (Cube IDE in my case) offers version control, and a search suggests it is done with plug-ins for Git or others.

Funnily enough the main issues I have with Eclipse are with it not rigorously checking datestamps before it does a build. I am often doing edits on a second machine (at home v. at work). I now make a point of treating the work one as "throw-away / test only" and keeping the master at home (it's my own business so I work at home a lot anyway, 7 days a week) and I back-edit changes tested at work to the home machine over RDP. If you edit files on another machine and drop them into the project directory (in or outside of Cube) then Cube gets confused, doesn't reliably notice the later datestamps, and does weird things, so after dropping in anything edited elsewhere one has to do a Clean and then Build.
You can tweak these settings from the project specific settings (internal or external builder, number of parallel processes) and from the workspace settings (how files are tracked, automatically saved before a build, etc)
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 
The following users thanked this post: peter-h

Offline peter-h

  • Super Contributor
  • ***
  • Posts: 1290
  • Country: gb
  • Doing electronics since the 1960s...
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #182 on: August 26, 2021, 08:23:50 am »
Do you mean these

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 90S1200 32F417
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 22035
  • Country: nl
    • NCT Developments
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #183 on: August 26, 2021, 09:41:25 am »
No, you can choose these in the build configurations for the project as these can be target & OS dependant (for example using the internal builder on Windows and the externel makefile builder on Linux).
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline mrflibble

  • Super Contributor
  • ***
  • Posts: 2052
  • Country: nl
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #184 on: September 08, 2021, 07:07:34 pm »
Point is, whatever approach you use, do not use version control as a backup, and an 'automated' backup as version control. Two different things. Just my 2 cents.
Agreed. Although ... Devil's advocate mode what's the worst that could happen? No seriously, what is the worst thing that could happen if you use version control as backup, and automated archives as version control?  >:D

Quote
With that said, considering version control, whatever you use for this, is a good step. Many "lone" developers build a bad habit of NOT tracking changes for what they develop. They just add features, fix bugs, and move on, considering that the last version will always be the best one anyway. So starting to keep track of changes is certainly a good thing. You can start with something as simple as Changelog text file listing the changes from one version/revision to the next.
Guilty as charged, especially in the past. :-[ Typically it would go something like "Mmmh, I wonder if this would work..." followed by a bunch of small scripts/experiments, finding/reading/misplacing papers, making some notes, and before you know it you have a collection of stuff that really could do with some version control. Especially since this kind of experiment more often than not either fizzles, gets interrupted, or runs into a roadblock. So it can be on the backburner for days, months, sometimes even years... Lets just say that I have never had a regret of this form: "Gee, this repo sure is useless. Eating up all this precious diskspace. I sure wish I never created a repository for this." On the other hand the "#^%@!! why didn't I just create a repo for this EARLIER?" variety has been known to happen every now and then. |O |O |O

As for the original question, yet another git user here. Fairly typical path taken: CVS -> SVN -> GIT for the most part, with some mercurial here and there. For new projects preference is git, but mercurial is fine too. Subversion is a most definite maaaaaybe, and CVS is a hard pass.

 

Offline dunkemhigh

  • Super Contributor
  • ***
  • Posts: 3700
Re: A version control system for embedded work, which doesn't need a PhD
« Reply #185 on: September 08, 2021, 07:24:11 pm »
Quote
what is the worst thing that could happen if you use version control as backup

You lose everything.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf