A version control system for embedded work, which doesn't need a PhD

[SiliconWizard]

As some have said, source backups are still a workable approach if you're a single user. Heck, that's what even teams used before VCS started to become popular.
If you're familiar with 'tar', it's pretty easy to generate archives customized with the file types you want to archive (avoiding object files and binaries, for instance), etc. Avoid using useless storage for files that on top of it compress poorly.

If you're doing that, I really DO NOT recommend automating it if you want to use that as any kind of rough version control. You need to at least specifiy a version/revision number for each backup you trigger, along with a list of changes (the first part can just be in the archive name, the second in a text file included in the archive). Otherwise, that's pretty much just a regular backup. Of course, you can further automate the backup of your source archives and of the current source files as they are.

Point is, whatever approach you use, do not use version control as a backup, and an 'automated' backup as version control. Two different things. Just my 2 cents.

With that said, considering version control, whatever you use for this, is a good step. Many "lone" developers build a bad habit of NOT tracking changes for what they develop. They just add features, fix bugs, and move on, considering that the last version will always be the best one anyway. So starting to keep track of changes is certainly a good thing. You can start with something as simple as Changelog text file listing the changes from one version/revision to the next.

[PlainName]

at least specifiy a version/revision number

Speaking of which, has this been solved in git yet?

With svn, and others similar, you can take the revision number, which increments sequentially. All my svn projects have the revision number (and mod indicator) automatically embedded, so recognising that the version giving this report is an earlier version than the bug fixed in that version is a no-brainer. No need to dig out the source or fire up any tool.

You can't do that with git so the only alternative seems to be to manually apply a revision reference, but you can't do that for every check-in. If you're running every version of the project (which you probably are to test it) then that means you have to manually sort a version number every check-in. It really needs to be automated anyway, so the act of applying a version reference doesn't change the version.

[voltsandjolts]

Shit program, though. Installs to Appdata without asking or warning, and when uninstalled doesn't actually remove any of the shit that it's filled the system with.

Hehe, the former I really don't care about (lots of big players put software in Appdata) and the latter I haven't got to yet
I think it's safe to say we have different requirements for software

[DiTBho]

You can't do that with git

The kernel Linux has a "dirty" label. Never investigated how it's generated but when you compile an experimental kernel it takes the revision number from Git 

[PlainName]

we have different requirements for software

That's one aspect. The other is the demonstrated mindset of the developer.

[PlainName]

The kernel Linux has a "dirty" label.

Sounds like it might be useful!

Of course, one way would be to have a revision number server, which dishes up sequential numbers on a first-come first-served basis, but that involves a centralised server which defeats the point of distributed versioning.

[emece67]

.

[PlainName]

Maybe this can help.

It's might do! Thanks

[magic]

The kernel Linux has a "dirty" label. Never investigated how it's generated but when you compile an experimental kernel it takes the revision number from Git 

Nah, it gives you the commit number which is some random gibberish. It's not sequential like SVN revision numbers.

Pretty sure it's the build system running git to figure out what is being compiled. It also appends a + if there are uncommitted changes.

[DiTBho]

The kernel Linux has a "dirty" label. Never investigated how it's generated but when you compile an experimental kernel it takes the revision number from Git 

Nah, it gives you the commit number which is some random gibberish. It's not sequential like SVN revision numbers.

Pretty sure it's the build system running git to figure out what is being compiled. It also appends a + if there are uncommitted changes.

Code: [Select]

kernel-5.12.0-mips64-dirty # nano tools/power/cpupower/utils/version-gen.sh

# First check if there is a .git to get the version from git describe
# otherwise try to get the version from the kernel makefile
if test -d ../../../.git -o -f ../../../.git &&
        VN=$(git describe --abbrev=4 HEAD 2>/dev/null) &&
        case "$VN" in
        *$LF*) (exit 1) ;;
        v[0-9]*)
                git update-index -q --refresh
                test -z "$(git diff-index --name-only HEAD --)" ||
                VN="$VN-dirty" ;;
        esac
then
        VN=$(echo "$VN" | sed -e 's/-/./g');
else
        eval $(grep '^VERSION[[:space:]]*=' ../../../Makefile|tr -d ' ')
        eval $(grep '^PATCHLEVEL[[:space:]]*=' ../../../Makefile|tr -d ' ')
        eval $(grep '^SUBLEVEL[[:space:]]*=' ../../../Makefile|tr -d ' ')
        eval $(grep '^EXTRAVERSION[[:space:]]*=' ../../../Makefile|tr -d ' ')

        VN="${VERSION}.${PATCHLEVEL}.${SUBLEVEL}${EXTRAVERSION}"
fi

I think they were these lines 

[brucehoult]

It's the contents of the commits that matter.

To a point. Knowing that the commit fixes some specific bug or implements some specific feature is perhaps as important when going through history. You might be trying to determine if this version has that fix, and knowing that some weirdo code has changed to some other weirdo form doesn't help that along much.

Sure, and that's down to information embedded in the commit messages, often in a stylised way in the first line "Fixes: 3416" or similar, which tools can pick up. The name of the branch on which the fix was developed is not known and not relevant.

CVS doesn't support merges as a concept at all.

SVN does merges, but what is recorded afterwards (shown by SVN MERGEINFO) is the revision numbers that were merged, not the branch they come from.

[brucehoult]

If you use git like that, why not just use zips and a backup folder?

Because git calculates differences between files in old snapshots and files in new snapshots and doesn't store unchanged bytes again and again, saving space.

As an example, a git checkout of LLVM -- which contains the entire history of the project in your checkout (compressed) -- is smaller than an SVN checkout of LLVM.

The SVN checkout contains just two uncompressed copies of the current files, so you can do diffs of your uncommitted changes locally. For history you have to talk to the server.

[brucehoult]

The kernel Linux has a "dirty" label. Never investigated how it's generated but when you compile an experimental kernel it takes the revision number from Git 

Nah, it gives you the commit number which is some random gibberish. It's not sequential like SVN revision numbers.

Pretty sure it's the build system running git to figure out what is being compiled. It also appends a + if there are uncommitted changes.

Sequential revision numbers make no sense outside of a single organisation with a shared central server with a great big shared lock.

If you do something as simple as fork a project, make a series of local changes, and then later decide to upstream your changes -- your revision numbers will clash with the upstream revision numbers.

The SVN model gets unworkable at global scale. It will be simply unusable if people on Mars want to contribute to your project, as them doing a commit will lock up your SVN server for some multiple of the 6 minute to 45 minute ping time.

[PlainName]

The SVN model gets unworkable at global scale.

A car is pants for continent hopping, but an awful lot of people find them useful because they don't continent hop. Similarly, the OP isn't working at global scale, is unlikely to have a Mars presence, etc.

Sequential revision numbers make no sense outside of a single organisation with a shared central server with a great big shared lock.

Apart from the big shared lock (what's that about?) that identifies many of us.

If you do something as simple as fork a project, make a series of local changes, and then later decide to upstream your changes -- your revision numbers will clash with the upstream revision numbers.

Yes, if you use distributed repos. Not if you use a master repo. Even if you fork off to a separate repo, when you merge back to the first one the revision number will be coherent because it applies only to that repo, which is the only repo for the project.

[brucehoult]

Not everyone is collaborating with Mars. But even on single-person projects you can often be out of communication with your SVN server. If it's on the internet you can be with your laptop on a plane or at the beach, with no internet. If it's on your desktop PCs on your home network, you can be simply not at home with a laptop (and not have a static IP at home).

If you're out of communication with your SVN server you can edit code and prepare one future commit. But you can't commit something and start another commit, and another, and sync up when you get back, the way you can with git. With git you can trivially send a series of commits by email, on a floppy disk (er, sd card), etc.

It's just vastly more flexible and supports whatever workflow you want to use instead of one inflexible way.

[PlainName]

If it's ...

The possible "if"s are pretty much unlimited and subject only to one's imagination. Not all of them are even remotely (sorry!) relevant. What is an important factor is what "if"s apply to one's own situation and expectations.

It's just vastly more flexible and supports whatever workflow you want to use

Hello? Have you just turned up? The fact is it doesn't support my workflow, otherwise I'd be using it instead!

Where git is more appropriate I will use it, but where it isn't I prefer a tool which works better for my process.

[Nominal Animal]

Of course, one way would be to have a revision number server, which dishes up sequential numbers on a first-come first-served basis, but that involves a centralised server which defeats the point of distributed versioning.

That's exactly it; you just use git for that too.

You could write a simple shell/Perl/awk/Python script in a dedicated git tree (versions.git?) that manages version numbers, authors, and short descriptions in simple text files, say
     YYYY-MM-DD HH:MM:SS major.minor.revision git-commit-tag author@email Description
If you name the files according to their project, you now have an organization-wide database of version numbers, git commit tags (relevant hash), timestamps, authors, and internal descriptions.  (Date would be obtained automatically, and author would be taken from current git configuration.)
Note that I chose the field order above so that sorting it will yield date and time order.

Since the script would pull its own HEAD first, then construct the new line, append it, and commit it, it would not be "instant"; it would also have to do a re-pull to verify the chosen version number is still unique in the head (or revert the commit and fail), to be robust.  (Because of this, it is better to implement it in parts: say first part a shell script to do the pull, then exec the actual update/modification script.)
However, with a little care, the command line interface could be very simple.  In my experience, such "registering" of a version number might take 3-10 seconds per run, depending on the bandwidth to and load of the git server you're using.  Not instant, but not aggravatingly slow either.

Hopefully, you'll see how git sits right in to the modular Unix philosophy; that to maintain version numbers in an organization, a separate git repository for it is just about perfect match.  (Consider, for example, that git will then maintain a separate log of the commits; something not provided by a straight-up database.  This can be useful if you have an occasional asshat among your workforce who tries to find others to blame for their own mistakes.  Updates and fixes, even done by hand directly to the database file, can be allowed, as long as no collisions are generated by such edits.  Again, the commits will leave a log that can be consulted to see if something odd ever occurs.)

As I've mentioned before, you are still looking for a single tool to accomplish the task at hand; but really, with Linux – and Unix in general – is better geared towards splitting the task into parts, each solvable using one or more separate tools.  git, or rather its parts, are very much such separated modular tool-lets.

(Note: No snide or denigrating tone is intended, I'm aiming at helpful, only.  Here I've tried to show a pattern of solutions to this kind of problems, hopefully showing the "mindset" behind the tool.  I'm highlighting the difference, because in my (teaching/tutoring/mentoring) experience, it is this "mindset" or paradigm or approach, that is the hardest to acquire.  When one "groks" it, pieces just start to fall into place.  I also like the term "grok", because this has nothing to do with intelligence or such; it really is something more akin to social or cultural details.  The more experience one has, the harder these can be to "grok", no matter how smart or intelligent or capable one is.  It is strange, and I don't understand why it is; but it is what I have observed across dozens of very smart, capable people.  Myself very much included; I regularly make a silly ass of myself by not grokking a thing before I stuff my opinion into the subject matter, even though I know better.)

[PlainName]

That's an interesting approach. Doesn't the version repo conflict with the project repo (that is, the working folder belongs to a single repo)? And if you're using a single master repo (which you would need to so you don't end up with duplicate revisions), why not just have a svn-like repo or database (sqlite might work) which will dish up unique incrementing numbers on demand without needing a script to check for updates and do re-requests?

On a practical level, how would you embed the version number in the project? Important aspects are:

1. Each commit needs a unique (incrementing) version number.

2. The version number for the commit needs embedding in the project.

3. The version number needs to be in the repo as a label so it can be (ideally) viewed in the revision graph or (at least) searched for. I think your suggestion would almost achieve this, but the scripting would have to figure if there is going to be an actual commit before asking for a reference, so it's a bit complicated.

[PlainName]

As I've mentioned before, you are still looking for a single tool to accomplish the task at hand

I don't think that's appropriate. For one, because we are talking, presumably, a part of a build system that will comprise many tools.

What I am actually looking for is a means to a specific end, but what I'm being pointed to is a different end that will allow a specific means to be used.

[Nominal Animal]

That's an interesting approach. Doesn't the version repo conflict with the project repo (that is, the working folder belongs to a single repo)?

No, because you put your local copy of the version repository in a separate folder:
    $HOME/projects/versions/
    $HOME/projects/foo/
    $HOME/projects/bar/

You could also make the version git a submodule of each project,
    $HOME/projects/foo/
    $HOME/projects/foo/versions/
    $HOME/projects/bar/
    $HOME/projects/bar/versions/
but the repetition wastes some disk space, when a developer works on many projects at the same time.

In both cases, since the version repo is separately maintained, the version information is available to all developers, even those without access to the projects themselves, so interactions between projects should be easier to plan.

And if you're using a single master repo (which you would need to so you don't end up with duplicate revisions), why not just have a svn-like repo or database (sqlite might work) which will dish up unique incrementing numbers on demand without needing a script to check for updates and do re-requests?

Because with the separate version repository, you have more information, and any errors can be corrected later, as the "database" is just a text file (per project).

On a practical level, how would you embed the version number in the project?

I'd tag the commit.  See e.g. Teensyduino git at github.

1. Each commit needs a unique (incrementing) version number.

Each git commit has an unique hash already.  These are not incrementing, though.

2. The version number for the commit needs embedding in the project.

Internally, use the commit hash.  Externally, use the tag.

(For example, you can use git pull repository tag tag to pull everything up to that specific tag.)

3. The version number needs to be in the repo as a label so it can be (ideally) viewed in the revision graph or (at least) searched for.

Do tags not fulfill exactly that purpose?

In a very real sense, the version repo duplicates tags, really, except being "central", it reduces the number of conflicts between local git repositories – for example, if one group works on feature X and another on feature Y, they can work on them separately, with (initially locally minimally tested) tagged versions merged into an "upstream" repo; with designated persons working on merge conflicts.  This works fine even when submodules are used.

As I've mentioned before, you are still looking for a single tool to accomplish the task at hand

I don't think that's appropriate. For one, because we are talking, presumably, a part of a build system that will comprise many tools.

What I am actually looking for is a means to a specific end, but what I'm being pointed to is a different end that will allow a specific means to be used.

Let me rephrase, then.

In #118 you said "[git] doesn't support my workflow", and I am trying to show the pattern of how it does.

The difference (as far as I can tell) is that it does not provide a built-in feature (atomically increasing version numbers) you expect from other version control systems, nor monotonically increasing identifiers for individual commits.

The solution in "git-land" is to use multiple repositories, and a "master" one for the atomic/monotonic/cross-project information; here, the database is a plain text file (so it is managed similarly to source code, including commit messages et cetera), typically – but not necessarily – accessed via helper scripts.  For the version case, it is somewhat an overview of changesets across the projects in an organization.

The one thing that gives me pause is the need for monotonically increasing commit identifiers, as I don't see why –– and I mean, I am puzzled and want to know more, as there is evidently something about your workflow I am missing or misunderstanding.  Isn't a commit hash and the timestamp of the commit sufficient, or even better?  Note that timestamps of form YYYY-MM-DD HH:MM:SS sort correctly as text; perhaps this is relevant here?  It is close enough to ISO 8601 (YYYY-MM-DDTHH:MM:SSZ or YYYY-MM-DDTHH:MM:SS±hh:mm) to be portable as well, while still being somewhat human-readable.

[PlainName]

In #118 you said "[git] doesn't support my workflow", and I am trying to show the pattern of how it does.

Well, thanks for working through all that to show me how the pattern would work. However, it falls over at these two places:

Quote from: dunkemhigh on Today at 12:21:38

    1. Each commit needs a unique (incrementing) version number.

Each git commit has an unique hash already.  These are not incrementing, though.

(I made a goof there and used 'incrementing' where I should probably have used 'sequential'.)

It needs the sequential version numbering for each commit.

Quote from: dunkemhigh on Today at 12:21:38

    2. The version number for the commit needs embedding in the project.

Internally, use the commit hash.  Externally, use the tag.

(For example, you can use git pull repository tag tag to pull everything up to that specific tag.)

I'm not sure I follow, sorry. Internal to what? Suppose the reported version is 1.0.123 where 123 is the revision reference. It seems to me that you're suggesting that to find the commit I need to cross-reference that 123 using the other repo to find the hash which I can then do a search on in the project repo. It's doable, yes, but hardly optimal.

The difference (as far as I can tell) is that it does not provide a built-in feature (atomically increasing version numbers)

Yep, that.

nor monotonically increasing identifiers for individual commits.

Almost. Each commit needs a version reference which is higher than the previous one (but don't need to be monotonic). 1, 2, 7, 12, 15 - that kind of sequence is fine.

... you expect from other version control systems ...

Don't care how it's done, that's the aim. It can be made to happen with git, but it's messy and it relies on there being a single repo. If you're going to have a single repo then git has some downsides and version control that is designed to work this way is just simpler and easier to use.

The one thing that gives me pause is the need for monotonically increasing commit identifiers, as I don't see why

Because it tells you this thing came after that thing. A git hash has no meaning outside the repo, so you can't tell from a hash whether this version came before or after that version without looking it up in the repo.

Is this important? I think so. When you release some product it will be version x.y, with increasing numbers. The end user will know that 1.2 came before 1.3. If he is on 1.2 then he knows he should upgrade to 1.3, there might be a bug fix (or a new bug), etc.

Suppose you want to run an Apple OS: do you install Mojave or Capitan or what? I would have no idea at all,  but I can tell you that Mojave is the one because it is version 10.14 whereas Capitan is 10.11.

Even where just names are used, typically they will be in alpha order so they are not completely random (but you need to be hip to the scheme to realise). But even if you don't twig that Gingerbread came after Froyo, you will know that 2.3 is later than 2.2.

So increasing version numbers are important to identify progression easily. No-one wants to have to google names (or 7-digit hex codes) to find out relative age. And then do it for every one of a series. But... for some reason all that goodness stops at writing the code. It's still there - we manually applied tags or labels, which are version references, on release versions, but for some reason the intermediate versions aren't important in that way. What's sauce for people up the line isn't for us before we hand it off to them. It's just a coincidence that that's where the version control has a problem, I'm sure. It certainly isn't because we suddenly don't need to know the temporal order of stuff.

Isn't a commit hash and the timestamp of the commit sufficient, or even better?

No. "When you select 'about', what does the version number say?" It needs to be something short and simple, so memorable. A date stamp and hash is essentially a GUID, and you know how simple they are to trot off the tongue!
Although, come to think of it, 210812191135 isn't too bad. Still tricky to spot when scrolling though it would benefit from being broken up: 210812-1911-35 maybe. But then you need the rest of the version number as well, so we're back to a forgettable 20-digit number.

Like I said before above, it's doable but smacks of taking the scenic route just to avoid an unliked road.

[SiliconWizard]

It's amusing to see that the OP explicitely asked for a simple, not requiring a PhD solution, and the thread is getting increasingly hard to follow. It will soon take a PhD to even understand what people are saying.

[brucehoult]

The one thing that gives me pause is the need for monotonically increasing commit identifiers, as I don't see why

Because it tells you this thing came after that thing. A git hash has no meaning outside the repo, so you can't tell from a hash whether this version came before or after that version without looking it up in the repo.

Is this important? I think so. When you release some product it will be version x.y, with increasing numbers. The end user will know that 1.2 came before 1.3. If he is on 1.2 then he knows he should upgrade to 1.3, there might be a bug fix (or a new bug), etc.

So we hit the root of your problem.

Product release numbers are almost completely unconnected to version control commits. There will usually be dozens or thousands of the latter for every one of the former.

Commits happen often many times a day.

Product releases happen something from once a week to once a year or longer. They are very deliberate things. Usually extensive testing is done. A maintenance branch is made for version 1.3 so development can continue on the main branch while bugs found in the release are fixed without mixing in untested unannounced new features.

Usually there will be a plain text file called "Version" in the top level of the project. This is often incremented and committed by hand just after a version branch is created. If the old was "1.3-devel" then the release branch will be updated to "1.3-rc1" and the trunk to "1.4-devel" (or similar).

This is such a small amount of work compared to everything else involved in doing a release that no one cares. But you might script it.

[brucehoult]

It's amusing to see that the OP explicitely asked for a simple, not requiring a PhD solution, and the thread is getting increasingly hard to follow. It will soon take a PhD to even understand what people are saying.

Because someone -- not the OP -- is insisting on replicating incidental features of their preferred tool in order to consider any other tool.

Simple use of the tool that has been by far the most popular for the last dozen years (with nothing else seriously on the horizon) is just that ... simple.

[PlainName]

So we hit the root of your problem.

Finally, yes!

Product release numbers are almost completely unconnected to version control commits.

Yes, but you are missing the wood for the trees.

Commits happen often many times a day.

Yes. So?

Why are  sequential version numbers NOT applicable to developers when they are to everyone else? Why don't developers want to know this came before that without having to look it up? A cynic would suggest it's because it's too much hassle to achieve, so therefore they're convinced they don't want or need the ability.

Product releases happen something from once a week to once a year or longer. They are very deliberate things. Usually extensive testing is done.

Yes, but irrelevant. Having version numbers, and having them increase is in no way related to 'deliberate' things or tested things. Or regular or infrequent things. Think about it: you would never say "Oh, we made 20 commits today, and all before lunch so they are all the same".

Usually there will be a plain text file called "Version" in the top level of the project.

Sure, that's how you get it into the project.

This is often incremented and committed by hand just after a version branch is created

Sure, that's how you define the major parts of the version number.

So are you saying that when an error pops up, or you look at the version output of the product, it is sufficient to know it is one of the very many - remember, you said there are lots of commits - versions of this branch? Knowing which particular commit is unimportant?