Perception of Git outside of softdev

[golden_labels]

While inspired by EEVblog video, this thread is not about the video. Therefore I put it Computers and not in EEVblog Specific.

In the “LIVE HELP with Github/Gitlab” video^(youtube:1) Dave experienced some trouble with using GitHub. For technical reasons I only seen the latter half of it: from about where he stated the line must be drawn. Some weird image emerged from that. An image, which makes me think, how people from outside of the softdev world perceive the topics touched there.

Topic #1
Dave does not himself use Git. Why did someone suggest him to put the project on GitHub? What was the motivation? The goals? To me, as a software developer, this idea sounds absurd. The only thing I could came up with, is the ability to clone or GitHub-fork and maintain the connection with Dave’s version. But I feel this is a far-fetched explanation. The VCS interaction would be almost entirely missing on Dave’s end and, where present, it would require him to put a lot of pointless effort to fit into the workflow. IMO it would be enough  to put an archive on eevblog.com and using forum for bug reports and FRs.

Topic #2
There seemed to be confusion between Git and GitHub, and misunderstanding the purpose of GitHub. I felt as if Dave thought GitHub is either a standalone service offering VCS, or even a file hosting service with bells and whitles. In my life I encountered really outlandish views regarding the areas I myself know: both from talking with people, as reported by friends and from reading research. I wonder, how far it goes with Git and, independently of that, with services like GitHub.

^(youtube:1)

[JohanH]

In the years I've worked with version control software, people outside of software and embedded development are totally ignorant of version control. It's a hard concept and most assume it to be file storage. It's a lack of education, but still, it's mostly used within software. Dave belongs to a generation where version control wasn't mandated by companies he worked for (I assume) and in late 90's and beginning of 2000, version control was very basic. CVS just started to get around and the proprietary systems were expensive (I've experience of two different ones).

Git is a difficult concept even if you come from an older system. The best thing to do is to learn git from scratch. Learn the command line and basic commands. Then they will also understand github and similar online web interfaces.

[SiliconWizard]

Already said it, but I find git a bit too complex to use past the very basic stuff, which is straigthforward. But that's nothing unexpected - Linus never expected it to take off as it did, it was mainly for the needs of the Linux kernel maintenance when he designed it.

I personally do recommend Mercurial and that's what I use when I'm not required to use git. Similar, but easier to tame and avoids shooting yourself in the foot.
Certainly something to look at for those looking for an alternative (there are others too.)

But as Johan said, I'm guessing the real question is about using version control in engineering outside of software dev, rather than about a particular tool.
And yes, this is still pretty uncommon, but it's getting there.

Of course one "hurdle" - not a dealbreaker - is that these tools are adapted to pure text. They work perfectly fine with any file format, but any diff or merge will work only on text and so is most adapted to software code. For the rest, it will just be a glorified backup system, and that's probably why it's still not that well perceived outside of software. It's useful, but not game-changing as it has been for software.

[DiTBho]

I do prefer Mercurial too, which, talking about u-boot and the Linux kernel, I massively use integrated with my own patch-manager.

Code: [Select]

macmini-intel /src/machine/mips/tigra/kernel/6.02.008 # ./mybuild-2023 5
  4 - list all files to be patched
  5 - apply patches
  6 - make patches
  7 - revert patches
  8 - list all files in the hacked list
  9 - generate the hacked list

Mercurial does all the version control job, and I keep current stuff "in trunk", this way my patch-manager can operate faster, and it's very easy to migrate patches from a kernel to an other: you can comment a patch in a special "note" section, this way the patch manager can check if a patch can be applied to a different project version.

Code: [Select]

# lspatch output/generated_patches/my-mm-util-c.patch
mm/util.c.note
mm/util.c

Mercurial consumes resources, a lot of ram when it's used for a large project like the Linux kernel, so as trick ... Nano has been modified and when you start a new branch and then you start editing stuff, Nano can now take tracks of the original files and the modified ones; this way the patch-manager can automatically generate patches based on diff between original and current.

It consumes only 64Kbyte of ram instead of several tens of Mbyte, and since everything is always under the control of Mercurial, the patch-manager can always query Mercurial to generate the hacked file list.

Code: [Select]

macmini-intel /src/machine/mips/tigra/kernel/6.02.008 # ./mybuild-2023 5 8
~ arch/mips/Kconfig
~ init/main.c
~ drivers/net/ethernet/Makefile
+ drivers/crypto/hcry.c
~ mm/util.c
~ net/core/dev.c
~ net/bridge/br_stp.c
(this consumes a few Kbyte of ram)

Code: [Select]

macmini-intel /src/machine/mips/tigra/kernel/6.02.008 # ./mybuild-2023 5 9
### generating ... from Mercurial db
~ arch/mips/Kconfig
~ init/main.c
~ drivers/net/ethernet/Makefile
+ drivers/crypto/hcry.c
~ mm/util.c
~ net/core/dev.c
~ net/bridge/br_stp.c
(this consumes up to 40Mbyte of ram)


I can do the same with Git, but ...
- Git consumes more resources (especially when it uses external tools, including python)
- Mercurial has less dependencies, and its code is more concise
- Mercurial offers a better interface for my patch-manager

The new version offers a ncurses-based menu, so you select one/more patches to be applied or reverted.

Weird way to operate, I know, but it's very very useful for me when I have to bi-sect/hack a large project with thousand files.

[brucehoult]

In the years I've worked with version control software, people outside of software and embedded development are totally ignorant of version control. It's a hard concept and most assume it to be file storage. It's a lack of education, but still, it's mostly used within software. Dave belongs to a generation where version control wasn't mandated by companies he worked for (I assume) and in late 90's and beginning of 2000, version control was very basic. CVS just started to get around and the proprietary systems were expensive (I've experience of two different ones).

I'm pretty sure Dave is a good 5 years younger than me. Certainly his first "experience" listed on LinkedIn  (1990) is 5 years after I graduated.

CVS was first released to the public in 1986 and was put under the auspices of the FSF in 1990.

I personally didn't have anything at home that could run CVS until I started using Linux around 1997 or 1998. Before that at home I used Apple's "Projector" in MPW, and also from CodeWarrior. At work I was using things such as Visual SourceSafe and PVCS.

I started using cvs in open source projects in 1998, and in a company in 2002.

Companies were switching from cvs to svn in 2003-2006. And then to git in 2006 and on.

I've also used sccs, rcs, Perforce, and probably others my brain has blocked memories of.

I have no idea what they were doing in electrical engineering, but as a 60 year old programmer I can't remember a time after student days that I *wasn't* using version control software.

Git is a difficult concept even if you come from an older system. The best thing to do is to learn git from scratch. Learn the command line and basic commands. Then they will also understand github and similar online web interfaces.

git is *worse* if you come from another system.  But git is actually very simple. It's just taught very badly, and by people demonstrating bad practices.

[JPortici]

I am still a VCS novice, i was never taught about VCS and used to copy everything manually every time (well, i still do that for most things)
it helps that i am the sole developer (and the graphic is the sole graphic, and so on) but it's no way to mantain things in the long run.
At some point i decided to get inversted in learning VCS and as you can imagine most, if not all, the supposed information you can get from googling is the usual BS:
-comparing SVN with Git, git is better
-and of course don't use mercurial because it's dead
-actually svn is dead too
-git this, git that, you must use git
-actually git is github, no now it's gitlab because github is evil because microsoft is evil
as you can imagine, a lot of highly ranked low quality search result, blog posts copying each other.
I could find nothing about better practices, how to set up a workflow, the things you may really want to know.

Anyway, in the meantime i decided to use SVN as i had to decide between what the IDEs i use supported and the choice initially went to SVN with Tortoise svn and a bunch of local repositories (sole developer, everything on my pc which is of course backed up) . SVN as i saw the problem in git not handling binary files well and besides source code everything else starting from PCBs is binary (though i'm changing to other formats as well, but old projects..)

However lately i decided to switch to git. The reason is not because i think git is better in any way, but it is in the github experience.
I always saw a parallel between git -> github and svn -> sourceforge. To me github is way better, has a better UI and UX, i like the "Issue" and the wiki parts.
This is relevant because i wanted to convince other people at work to use VCS, if anything for the project that are not done by one person (for example, very old software i'm mantaining, another person has the software for making the installer and all the final steps on their PC and every time is did you update X on my? is Y ok?) i hate manual copies, it's so easy to keep the wrong copy around. and i hate incrementing a version number for every comma, as we use major.minor in version numbers you can see how it's inconvenient.
So i was looking at self hosted git and i stumbled upon Gitea, you get an UI/UX very simillar to github but you can self host. And i can see the possibilities, do a repo for every project, let it be software, hardware*, photo*, video* (*yes, i know, binary files), documentation/procedures, then for everything you have the issue to report problems without telling anybody by voice or email or stupid postit, you can edit your problem and add images and meaningful description and keep trace of everything, all in a local server accessed by everybody in the company. Same for the wiki part, documentation on the project online, yes please. The project planning features are basic but still useful, at least to put a date on the calendar. When something is done or changed you get the notification.
The fact that it uses git forces me to adopt git, i wonder if a simillar system is available with SVN

[JohanH]

CVS was first released to the public in 1986 and was put under the auspices of the FSF in 1990.

The concept is old. But the company I work for didn't start using any form of VCS before around 1998. And we produced a lot of code to be a hardware company. VCS wasn't even mentioned during my studies in the latter half of 90's. Now the awareness is better.

git is *worse* if you come from another system.  But git is actually very simple. It's just taught very badly, and by people demonstrating bad practices.

True.

[brucehoult]

as i saw the problem in git not handling binary files well

That is absolutely not true.

Git in fact treats ALL files as binary, and and stores byte-by-byte binary diffs between revisions or between similar files, not line-by-line diffs.

What git doesn't like is binary files where any trivial change to the file changes the entire file, so it CAN'T be diffed. That generally means any kind of pre-compression or encryption.

Types of binary files that git loves: disk images, databases, bitmapped images without compression such as (some) TIFF files, tar archives (NOT .tgz).

git stores everything compressed AFTER it diffs it. Compressing things before giving them to git prevents diffing.

You can still store such files, but every version has to be stored in full.

It's hard to imagine any other RCS that does better -- unless they specifically recognise certain types of compressed files and automatically uncompress them.

[JohanH]

That's why file types like pdf and similar are not that good. They do not diff well. There are solutions for this. One of our projects use Git LFS (Large File Storage) where they store files that are hard to diff, but they really want to be handled by git. There are some disadvantages to this, but in all it works well.

[JohanH]

Binary file changes is interesting from VCS and archiving point. IMHO, all software should consider making as little change as possible to the original file when when saving to disk and overwriting. This would be beneficial for a lot of things.

A different case, but that touches this issue with binary changes is the Debian project with reproducible builds (https://wiki.debian.org/ReproducibleBuilds). It's incredible (and important) how they are able to repeat builds in different systems and ending up with the same final binary with same checksum.

[DiTBho]

I've also used sccs, rcs, Perforce, and probably others my brain has blocked memories of.

list+="Doors"

 

[DiTBho]

A different case, but that touches this issue with binary changes is the Debian project with reproducible builds (https://wiki.debian.org/ReproducibleBuilds). It's incredible (and important) how they are able to repeat builds in different systems and ending up with the same final binary with same checksum.

Good find! I need something similar!
Thanks! 

[mapleLC]

I have a 20 year career in software development and do not understand Git, Github, and even where the damn downloads on the page are 

Its quite confusing in the way GNU was/is and other things like a Java Bean.

The whole system could do with a Steve Jobs for a few weeks to clean up the story.

[brucehoult]

I have a 20 year career in software development and do not understand Git, Github, and even where the damn downloads on the page are 

Its quite confusing in the way GNU was/is and other things like a Java Bean.

The whole system could do with a Steve Jobs for a few weeks to clean up the story.

Steve Jobs, about iPhone 4 antenna being masked by the user's hand: "You're holding it wrong"

[JohanH]

I have a 20 year career in software development and do not understand Git, Github, and even where the damn downloads on the page are 

That's because Github isn't primarily a "download page". It's a software source repository with some added project management features. You are meant to copy the source by running "git clone <address>" on your own computer. That some projects provide binary downloads is something that came later and is an added bonus and a feature of the github web pages. Github is trying to become an online integrated software source, development and project management system with issue tracking etc. It works well at least for smaller, open source projects.

[Halcyon]

My view of GitHub outside software development is that it's irrelevant.

[shapirus]

My view of GitHub outside software development is that it's irrelevant.

It, like any VCS, may be used for version-controlled work on any digital data, which is not limited to software source code.

Git is great, but the more years I use it, the more I, sometimes, think that it was created by aliens for aliens. It may take time and forgetting of what you already know to grasp and get it right.

For a complete VCS beginner, especially for a solo developer beginner, a non-distributed VCS (such as Subversion; forget about CVS) may be a better and easier to understand tool. For an easier start in distributed systems, Mercurial is a good choice, however git has become an industry-standard tool these days, so if you expect to collaborate with others, you will need git.

Does a solo developer need to use VCS? Yes, absolutely. It pays off in the long run and is well worth the initial effort.

Why do people have trouble understanding VCS? Because they start at the wrong entrypoint. The right way to start is to learn what problems VCS are made to solve.

[tszaboo]

For HW development, I've been using SVN because that's what I was shown. Plus, Altium supports that. I've tried git for it, but it wasn't really good. Binary files made the performance bad, plus it was difficult to explain the concept to others. Or to remember it myself sometimes. This leads to issues ("did you push it?"), that can be avoided by using SVN so we just go with the path of least resistance.
On the other hand github is free, so I use it for personal projects.

[Karel]

I started to use git ten years ago. We were still using Eagle V6 & V7 and I must say, using git  for backup & versioning (not merging!) always worked very wel because
the xml files of Eagle were pretty much "human readable" so "git diff" was usable.
Now, with KiCad, not so much because in my opinion the files are harder to read and understand and also, KiCad has the nasty behaviour to reorder elements that didn't change
but ofcourse it is a change as far as git is concerned. Still, for backing up and versioning only, git works fine also for KiCad.

[JohanH]

Now, with KiCad, not so much because in my opinion the files are harder to read and understand and also, KiCad has the nasty behaviour to reorder elements that didn't change

This is what I hate with some software. I remember a proprietary one that when you saved, it reordered some variables in a random fashion even when there were no changes done to files belonging to the project. So when checking in changes to VCS, it always detected changes in all files. Very annoying.

If KiCAD does the same, it should be reported as a bug. I haven't checked this myself.

[mapleLC]

I have a 20 year career in software development and do not understand Git, Github, and even where the damn downloads on the page are 

Its quite confusing in the way GNU was/is and other things like a Java Bean.

The whole system could do with a Steve Jobs for a few weeks to clean up the story.

Steve Jobs, about iPhone 4 antenna being masked by the user's hand: "You're holding it wrong"

Technically, he was right. 

[brucehoult]

My view of GitHub outside software development is that it's irrelevant.

Software developers are not the only people who can benefit from being able to track N changes to a document or set of documents, see who changed what when, revert to old versions if you made a mistake, and do all this using much much less than N * document_size disk storage.

The LLVM compiler project currently has 453000 commits (changes) over its lifetime. A git checkout that contains that entire development history, checkable without any further internet connection, is SMALLER (2635 MB, of which 1048 MB is the .git database and 1587 MB is the checkout)  than an svn checkout of it (3174 MB) that has only the current version and needs a live internet connection to do even "svn log".

But git is just as useful if you are writing the Great American Novel.

[JohanH]

As of KiCAD 6, the last bytes of each line's timestamps are slightly updated when files are saved. However, you can choose to add or not add specific schematic sheets if you don't want files to be changed. I've used KiCAD with git on a number of projects without issues.

There's even a tool for visual KiCAD git diffs, https://forum.kicad.info/t/diff-tool-for-schematics-and-layout-using-git/28559

Great! Seems to be in order.

[AntiProtonBoy]

I'm predominately a software dev by profession, so git seems natural to use for me. So I ended using git for a whole heap of things. For example, I collect recipes for cooking and use git to track revisions, edits, etc. I've also been using git and git LFS for graphics design projects. Also use it for my web site, scripting experiments, schematics, and so forth. Super useful.

[mapleLC]

I'm predominately a software dev by profession, so git seems natural to use for me. So I ended using git for a whole heap of things. For example, I collect recipes for cooking and use git to track revisions, edits, etc. I've also been using git and git LFS for graphics design projects. Also use it for my web site, scripting experiments, schematics, and so forth. Super useful.

This seems like what immutability on the block chain was for.