EEVblog Electronics Community Forum

Electronics => PCB/EDA/CAD => KiCad => Topic started by: phil from seattle on October 21, 2020, 06:07:18 pm

Title: design versions?
Post by: phil from seattle on October 21, 2020, 06:07:18 pm
I am trying to make new/different versions of my design.  In one case, I want to change the connectors.  I have 32 of them on the board and want to switch to a different design that has slightly different spacing. This leads to different board dimensions.  For documentation and support purposes, I want to keep both versions around.  I don't need fork and (re)join capabilities, just a starting point to make changes that don't affect the current design.

What is the best way to do this? I tried creating different file names for the different versions within a pro directory but that leads to confusion. I tried copying the entire directory but lose local footprint libs. Guess I should have made them global, eh? These are not intractable but clumsy enough that I think there must be a better way and I am not yet one with KiCAD. Googling lead to a laundry list of files to copy with lots of cavaets.

Uh, there must be a better way. Any ideas?
Title: Re: design versions?
Post by: nfmax on October 21, 2020, 06:37:30 pm
I have just this minute been bitten by the same problem. I made a copy of the project directory and started work in that, only to find that KiCad is modifying the files in the original directory, not the copy. It seems the project file holds absolute pathnames.

There ought to be a 'clone this project' function in project manager!
Title: Re: design versions?
Post by: Benta on October 21, 2020, 07:06:56 pm
I've had the same issue.

Caveat: I'm running Linux, which means multiple (but not simultaneous) users, so this might not be relevant to a Win installation.

I tried declaring user-defined parts as "Project" library, but the limitations and implications were just too great.

Now, all my user-defined parts are defined as "Global", but are still stored in my local user directory. This means they're available to the current user at any time.

If I want to clone and modify a design, I open it and do a "Save as" (with another name) to a new .pro file. In this way, it's a new project that still has access to all the libraries and will not muck around with the original design.
Having different .sch or .pcb file versions within the same .pro file is a really bad idea.

Let us know how you get on or if you find a better way. Cheers.
Title: Re: design versions?
Post by: phil from seattle on October 21, 2020, 07:14:39 pm
Quote from: Benta on October 21, 2020, 07:06:56 pm
Having different .sch or .pcb file versions within the same .pro file is a really bad idea.

Uh, yeah, found that out the hard way.  In Eagle, a new version is as simple as doing a save-as with a different name.  And is intuitive.  KiCAD in this respect is anti-intuitive.

Must become one with KiCAD.

Edit: By the way, just copying a directory is a fail.  If you open both a schematic and PCB when you try to switch fromone to the over via the buttons on the menu bar at the top it tries to reopen the file even though it is already open. This is very frustrating - something that should be bone head easy.
Title: Re: design versions?
Post by: Benta on October 21, 2020, 08:12:30 pm
I just revisited and tested this.

There's indeed no "Save as" option in the current 5.1.7 KiCAD. It's been some time since I had to clone a design, sorry. I've refound the way I did it back then:

From the schematic editor, select "Save current sheet as" and then create a new folder and file with new names.
From the file manager, open the new .sch file (double-click), this will open KiCad and at the same time create a new .pro file.

Non-intuitive indeed!

The switch between schematic and layout works fine for me, I can use the the buttons at the top with no problems. You'll have three windows open, though: Project, Eeschema and Pcbnew.
Title: Re: design versions?
Post by: phil from seattle on October 21, 2020, 09:48:26 pm
That works for a single sheet schematic.  When you have hierarchical schematics, you need to save not only the individual sheets (_1, _2, ..) but also the top level sheet.  unintuitive, indeed!

Next problem is "local" libraries.  Sigh...

Working my way through it.  I think there is a light at the end of tunnel.  Hope it's not a train...
Title: Re: design versions?
Post by: Benta on October 21, 2020, 10:34:01 pm
Quote from: phil from seattle on October 21, 2020, 09:48:26 pm
Next problem is "local" libraries.  Sigh...

Local libraries are not a problem, that just means they are stored somewhere in your user environment ( = think about them when doing backups).
The issue is, that you've probably declared your devices as "Project" instead of "Global". I've found that this is a bad idea.

What's your OS? With Win I can only offer limited help, with Linux/Ubuntu more.
Title: Re: design versions?
Post by: Doctorandus_P on October 22, 2020, 01:10:40 am
As you've discovered it's not a straight forward process, and it also depends on how your project is configured. For example, do you use relative or absolute path names to custom libraries?

On top of that there are also some gotchas when you try to mix projects, for example during updates later. The first I think of is that KiCad by default uses "timestamps" or "UUID's" to make the match between schematic symbols and PCB footprints, instead of the RefDes (such as R32 for a resistor or U5 for an IC).

The best direction is probably to make a complete copy of the existing project into another directory, but because of the path issue (and probably other) issues it's not a straight forward copy.

There are a few (party) automated ways to do it. First there is a "KiRename" script for renaming projects. You can start by looking at that script and what it does:
https://html.duckduckgo.com/html/?q=%22kirename%22&norw=1 (https://html.duckduckgo.com/html/?q=%22kirename%22&norw=1)
https://github.com/bobc/KiRename (https://github.com/bobc/KiRename)
Apparently this script has not been updated for a few years, so it may be (partially) out of sync with current KiCad versions.

Another option is to create a template from your existing project.
The steps roughly are:
1). Set up a custom directory for all your future personal templates.
2). Copy your existing project to the template directory.
3). Add some html, some graphics such as an icon. (See the existing templates on your system as example).
4). In KiCad's project manager select: File / New / Create Project from Template.

The last option I want to mention is to:
1). Copy the whole project to another directory.
2). Delete all files which can be generated by KiCad (netlists, backup files, gerbers, etc)
3). Open the remaining files in a text editor, and look for path names and directory names, and mend as appropriate.
With this last option you will probably do the work manually that can be done by the KiRename script.
Title: Re: design versions?
Post by: phil from seattle on October 22, 2020, 03:03:39 am
Thanks. That is helpful  I'm still a little surprised this was not part of Kicad from V1. Having to know KiCAD under the covers to make a new version seems kind of user hostile.

First thing I did when I realized the path problem was to open all the files up with NP++ just to see what was in them.  Saw a few paths but not a lot.

I'm on Windows. And yes, everything is going to be global from now on. 
Title: Re: design versions?
Post by: phil from seattle on November 01, 2020, 06:15:03 am
I think I've found the way to do this. At least it seems to work for me.

In your first version of the design, do not use version numbers in files, use it in the directory/folder name only. So if your project is called foo, all your schematic, pcb, rescue, ... files are called foo.sch, foo.kicad_pcb, ... Anywhere it asks you absolute or relative path, take relative.  Anywhere it asks global or project, take global. 

When you go to make foo version 2, you create a foo_v2 directory, copy the folder contents of foo into it and work from there. If all the paths are relative, you are home free. Your files are still called foo.sch, foo.kicad_pcb, ...  And if you need to tweak a symbol or footprint, don't. create a new library entry as you could break older versions of your design.

And, never use a space in file names.  Never. Ever. Ever.  Use _ or - or anything but a space. I think a lot of Kicad can handle it but it looks like there are parts that can't. Once I purged spaces, I stopped getting strange error messages.

As long as I am at it, if you have an Eagle design that you want to move to Kicad you can import the schematic and pcb files fairly easily.  But, you should not use the imported files as the basis for making changes.  Rebuild your design with kicad symbols and footprints.  AND remake any custom symbols or footprints with the Kicad tools.  This may seem like unnecessary work but you will be in hell fixing all the problems that arise from imported symbols and footprints.  Better to just create a clean slate.  I imported a 120 part design and had a ton of clean up to do. Along the way I learned a lot, A LOT, about the library systems of Kicad so it's not completely wasted. I'm not an expert by any stretch though it is finally making sense to me. I spent 3 frustrating days fighting Kicad (er, my ignorance of Kicad) and fixing all sorts of problems with the board. 

By the way, I like the template idea but each of my versions is built on the previous version of the design.
Title: Re: design versions?
Post by: pierreraymondrondelle on November 01, 2020, 11:18:54 am
Doctorandus_P,
Quote
When you go to make foo version 2, you create a foo_v2 directory, copy the folder contents of foo into it and
For long, I was basically doing the same but I always appended the version number to every copied file. This didn't seem to cause any issue and all the files were version tagged. Your solution looks interesting however but I can't manage to understand why:
Quote
And if you need to tweak a symbol or footprint, don't. create a new library entry as you could break older versions of your design.
may cause any problem since the libraries didn't move. Please, may you elaborate ?
regards
Title: Re: design versions?
Post by: bson on November 01, 2020, 11:12:51 pm
Quote from: nfmax on October 21, 2020, 06:37:30 pm
I have just this minute been bitten by the same problem. I made a copy of the project directory and started work in that, only to find that KiCad is modifying the files in the original directory, not the copy. It seems the project file holds absolute pathnames.

There ought to be a 'clone this project' function in project manager!
Yeah, the whole thing with storing absolute file paths screams amateur hour.  It causes all kinds of problems with multiple computers when you keep your projects on github or gitlab.  Fortunately the project files are text, so fairly easily fixed, though it shouldn't be necessary.
Title: Re: design versions?
Post by: phil from seattle on November 01, 2020, 11:27:33 pm
Quote from: pierreraymondrondelle on November 01, 2020, 11:18:54 am
Doctorandus_P,
Quote
When you go to make foo version 2, you create a foo_v2 directory, copy the folder contents of foo into it and
For long, I was basically doing the same but I always appended the version number to every copied file. This didn't seem to cause any issue and all the files were version tagged. Your solution looks interesting however but I can't manage to understand why:
Quote
And if you need to tweak a symbol or footprint, don't. create a new library entry as you could break older versions of your design.
may cause any problem since the libraries didn't move. Please, may you elaborate ?
regards
I can't answer the question.  But I will fix a library entry while having one project open.  When I go back to the the other project that uses the same library entry I get the dreaded ?? symbol instead of the library symbol. Perhaps I am doing something stupid but can't (yet) say what it is.
Title: Re: design versions?
Post by: Doctorandus_P on November 02, 2020, 01:57:45 am
@pierreraymondrondelle
Those quotes are not from me!

Quote from: bson on November 01, 2020, 11:12:51 pm
Yeah, the whole thing with storing absolute file paths screams amateur hour.
This is probably mostly user error.
KiCad has default locations for default libraries, and you can use both absolute or relative paths to custom libraries, depending on you you want to manage and use personal libraries.

Especially for beginners or people who do not have much experience with KiCad it is easy to set up your personal libraries "wrong". One of the things that help is t use $(KIPROJMOD) which is a variable that always points to the root of the current project. Each project also has a "sym-lib-table" file for project specific schematic symbol libraries, and a "fp-lib-table" file for project specific footprint libraries.

Quote from: phil from seattle on November 01, 2020, 11:27:33 pm
I can't answer the question.  But I will fix a library entry while having one project open.  When I go back to the the other project that uses the same library entry I get the dreaded ?? symbol instead of the library symbol. Perhaps I am doing something stupid but can't (yet) say what it is.

KiCad has different ways of managing libraries. First there are global libraries, that are the same for all projects, and then there are Project specific libraries, that can (and often are) different for each project.

There is no need to dread the [??] symbols.
It is just a sign that KiCad can not find the graphics for that symbol, and the most common cause of that is some error in the name of the library, or the library itself is not available to the project.
To look into it, hover the mouse over such a [??] symbol and press e for edit, and then look at the library reference of that schematic symbol. It has both a library name and a symbol name.
The library name must also be present in one of the library tables which you can find under:
Eeschema / Preferences / Manage Symbol Libraries

Another common cause for the [??] symbols is a missing "[projectname]-cache.lib" in combination with a change in the default libraries. It's easy to think of this file as "unimportant" as it usually gets recreated after you deleted it, But if you bacup or archive your project, you should add this file to the archive too. It has a backup of all library symbols used in the project, and if you restore a project without this file you may have a problem if KiCad's libraries changed in the mean time. The way this works now is a hack and it has already been removed in the nightlies. KiCad V6 will have a brand new schematic format and have copies of all used schematic symbols in the schematic itself in a similar way that Pcbnew now has copies of used Footprints.

Currently the most up to date information on KiCad's workings are in the FAQ section on the KiCad forum, which is also searchable for subjects such as:
https://forum.kicad.info/search?q=library%20management%20category%3A19 (https://forum.kicad.info/search?q=library%20management%20category%3A19)
Title: Re: design versions?
Post by: phil from seattle on November 02, 2020, 06:14:54 pm
Yes, I learned how to fix the ?? symbols but it shouldn't have happened in the first place.  I had a schematic that was correct. Closed it and reopened it to to find the ?? symbols.  Ugh. Probably related to the rescue library (which seems like a real band aide thing in the first place).

I figured out the problem and wound up using a text editor to make the fixes so it went pretty fast.

In fact, I'd say that once you understand how kicad libraries, symbols and footprints work, Notepad++ is your best friend.  For example, I decided I wanted to change how designators were printed on the board (size and thickness), I was able to change all of them (>100) in about 2 minutes.
Title: Re: design versions?
Post by: Doctorandus_P on November 03, 2020, 07:14:38 am
I used to use a text editor too to get around some limitations in KiCad, and it's still good to know it can be used as a "backup method", but I've been using it less and less while KiCad evolves.

For your example, changing the RefDes on the PCB, use:
Pcbnew / Edit / Edit Text & Graphics Properties

Then set "Scope" to "Footprint references", and then modify those text properties to your liking.
Title: Re: design versions?
Post by: pierreraymondrondelle on November 03, 2020, 02:43:12 pm
Quote
    Quote phil from seattle

        And if you need to tweak a symbol or footprint, don't. create a new library entry as you could break older versions of your design.

    may cause any problem since the libraries didn't move. Please, may you elaborate ?
    regards

I can't answer the question.  But I will fix a library entry while having one project open.  When I go back to the the other project that uses the same library entry I get the dreaded ?? symbol instead of the library symbol. Perhaps I am doing something stupid but can't (yet) say what it is.

I think I have understood: I just modified a symbol to correct a clerical error and then opened an older project using this symbol. I got the ?? of the hell, ignored it closed eeschema without saving. Reopening eeschema I accepted to fix the error and it created a rescue.lib. After reflexion, the modified symbol probably had a different time stamp. eeschema saved the original symbol in the resue lib for further reference skiping the corrected symbol forever (until the next disk crash!). However, renaming the rescue.lib I'm presented again with the ?? that I can associate with the corrected symbol to definitely fix the issue.
The lesson I learnt is that every time I have to modify an existing symbol or footprint or... I shall add a version number to its filename and keep the older versions.
Title: Re: design versions?
Post by: nfmax on November 03, 2020, 02:50:15 pm
And yet...

...I can open the symbol editor from the eeschema window, modify a symbol used in the drawing I am working on, save it, and the circuit diagram updates to the new version without a murmur! Why should it behave differently in the two cases?
Title: Re: design versions?
Post by: pierreraymondrondelle on November 04, 2020, 01:15:56 pm
was your kicad session still open when you modified the symbol ?
If so, it was saved when you closed it, re-opening the project, it loaded the corrected symbol.
In my case, the project was closed for some weeks before I modified the symbol.
Further on, when I opened it, a message poped up, telling something like the symbol was unknown or modified, asking to fix it or continue.
The difference is that my old project was refering to the unmodified symbol until I re-allocated the modified one.

What is not clear for me yet is the "time stamp". I don't know if it applies to symbols or footprints or both and how exactly it operates.
Title: Re: design versions?
Post by: phil from seattle on November 04, 2020, 05:30:38 pm
Quote from: nfmax on November 03, 2020, 02:50:15 pm
And yet...

...I can open the symbol editor from the eeschema window, modify a symbol used in the drawing I am working on, save it, and the circuit diagram updates to the new version without a murmur! Why should it behave differently in the two cases?

Why indeed! This morning I got the rescue dialog on a project that I had not changed in the previous session.  Huh?  The whole library system seems way too complex.  I'm not going back to eagle but it is something to keep an eye on. I'm sure all of the behavior is explainable but I don't have it in my head yet.
Title: Re: design versions?
Post by: Doctorandus_P on November 04, 2020, 06:47:26 pm
The way Eeschema works with it's libraries is a bit of kludge, but will be vastly improved with the next KiCad version. (There are rumours it's intended to be out before FOSdem (end of February) next year).

I already posted a link to the KiCad FAQ on KiCad's user forum, which describes lots of aspects of library management.
How it works in short:

Each schematic symbol has a "timestamp", and this is how KiCad itself keeps track of different schematic symbols. In KiCad V6.0 this is extended to an: "UUID".
This timestamp is normally used to sync between Eeschema and Pcbnew. So you can for example update RefDes numbers without loosing synchronization (This default behavior can be overridden)

In the next Eeschema File format, the graphics of schematic symbols will be embedded in the schematic file itself.
Currently (KiCad V5) there is only a link to a "LibName:SymbolName" combination.
If this link does not exist (or the file exist, but the library itself is not in the library table), then Eeschema looks in the "...-cache.lib" file, (where it keeps a copy of all used schematic symbols). You should never delete the "...-cache.lib" file. Especially when archiving a project!

If a schematic symbol is not found in the location pointed to by the schematic, but it is found in the ...-cache.lib file, then the "rescue" system is triggered.
The "rescue" system copies symbols from the ...-cache.lib file to the ...-rescue.lib file, and also modifies the schematic itself to use the ...-rescue.lib symbols in the future, and it adds the ...-rescue.lib file itself to the project specific libraries.

The ...-rescue.lib is a normal Eeschema library, and it can be renamed to any name you want, but it needs a few steps:
1. Rename the ...-rescue.lib itself.
2. Add the renamed library file to the library table: Eeschema / Preferences / Manage Symbol Libraries / (tab page) Project Specific Libraries
3. Change all library references in the Eeschema that point to ...-rescue.lib to the new library.

This last step can be done in a few different ways.
A quick (hackish) way is to open the schematic in a text editor, search for the ...-rescue.lib references and replace it with the new library name.
Another way is to use: Eeschema / Tools / Edit Symbol Library References
Title: Re: design versions?
Post by: phil from seattle on November 07, 2020, 06:21:29 pm
I've used my technique to make new versions now on 3 different projects and it works pretty well.  In summary:
While this is kind of wasteful of space, it seems to minimize errors and confusion.

However, I am still getting the occasional missing rescue library errors when starting up.  Not all the time, though. Never on the first run in the new directory, it happens after I've had several error free sessions previously on the same project. It seems that ignoring them causes no harm. I've even taken care to close everything down each evening (as opposed to leaving it running overnight like I did with Eagle) but it seems to still happen. c'est la vie

I hope V6 smoothes out these rough edges.  For someone like me with 20+ years of doing PCB design, it's just a hiccup but for someone starting out, it has to be pretty daunting. It doesn't help that the messages are pretty cryptic to someone not familiar with the inner workings of Kicad.
Title: Re: design versions?
Post by: mx-yh on December 03, 2020, 07:50:39 am
Version control system can be used

For example, I use git

installation in ubuntu, mint
Code: [Select]
sudo apt update
sudo apt install git

creating a project repository
Code: [Select]
cd my_project
git init

Code: [Select]
git add .
git commit -m "version 0.01,  first commit"