Author Topic: Footprint association tool  (Read 6491 times)

0 Members and 1 Guest are viewing this topic.

Online SiliconWizardTopic starter

  • Super Contributor
  • ***
  • Posts: 14303
  • Country: fr
Footprint association tool
« on: June 14, 2018, 05:40:41 pm »
Kicad's footprint association tool is VERY slow. Tried it with Kicad 4.0.7 and 5.0.0 nightly builds, both on Windows and Linux - same thing.

It seems stuck while "loading footprint libraries". Kicad uses some CPU while doing this, but it's not even at 100% of one thread. So I have no clue what it does. It's just unusable.
Does this for any schematic edited with older versions of Kicad. Not sure yet if this is the case with newly created schematics.

Have you run into the same problem?
 

Offline bd139

  • Super Contributor
  • ***
  • Posts: 23018
  • Country: gb
Re: Footprint association tool
« Reply #1 on: June 14, 2018, 11:07:29 pm »
Probably disk IO. Do you have an SSD? If not get one.
 

Offline Whales

  • Super Contributor
  • ***
  • Posts: 1899
  • Country: au
    • Halestrom
Re: Footprint association tool
« Reply #2 on: June 14, 2018, 11:42:28 pm »
Old versions of Kicad used to fetch all the footprints from the web whenever you opened these, which was an absolute pain.  Could this be somehow re-enabled?  Possibly by you opening old docs?

> Do you have an SSD? If not get one.

I've never needed an SSD to run this software comfortably.

Offline Aeternam

  • Supporter
  • ****
  • Posts: 97
  • Country: lu
    • Schartz Engineering
Re: Footprint association tool
« Reply #3 on: June 15, 2018, 07:02:24 am »
cv2pcb does take a bit of time to load the libraries. But once you're in the program there shouldn't be any more delays. Do you have the modules stored locally or on a network drive? I used to keep mine on a NAS, this slowed things down quite a bit.

Also, are you aware that you can assign footprints in the schematic editor also? It's not as comfortable as using cv2pcb but if the latter is unusable, you could go that way...

 

Offline awallin

  • Frequent Contributor
  • **
  • Posts: 694
Re: Footprint association tool
« Reply #4 on: June 15, 2018, 09:53:38 am »
It seems stuck while "loading footprint libraries". Kicad uses some CPU while doing this, but it's not even at 100% of one thread. So I have no clue what it does. It's just unusable.
Does this for any schematic edited with older versions of Kicad. Not sure yet if this is the case with newly created schematics.
Have you run into the same problem?

AFAIK it could be checking for updates via git? Do you have a slow network connection?
 

Online SiliconWizardTopic starter

  • Super Contributor
  • ***
  • Posts: 14303
  • Country: fr
Re: Footprint association tool
« Reply #5 on: June 15, 2018, 03:32:31 pm »
Happens both on an SSD and a conventional HDD, with new installs of KiCad. The only thing "oldish" are the schematics. And I mean, like 2 years ago. So I think those were made with Kicad 4.0.x?
Again, on different machines (both pretty fast), different OSs, local. An my network connection is not slow. I just know that the libaries that come installed with recent versions of KiCad are kinda big. And when this happens, there is no significant disk activity either. Maybe it's some kind of GUI bug though, as when it does this, the GUI becomes randomly unresponsive.

I know we don't really need cv2pcb anymore (not sure this is called this anymore, it seems a whole part of the KiCad executable now). I'm just evaluating KiCad again (as I do like every once in a year or two). But this tool has the benefit of listing the entire list of parts with references, symbols and footprints. That's what I was interested in. If there is any other way of getting this kind of listing, I haven't found it yet.
« Last Edit: June 15, 2018, 03:35:25 pm by SiliconWizard »
 

Offline donotdespisethesnake

  • Super Contributor
  • ***
  • Posts: 1093
  • Country: gb
  • Embedded stuff
Re: Footprint association tool
« Reply #6 on: June 16, 2018, 07:01:27 am »
KiCad now loads all footprints into cache for eeschema and the dialog formerly known as cvpcb, and several users report very slow operation. The KiCad devs tend to say "works fine on my machine, must be your system" but that is a cop out. I suspect the method of parsing or the fact of reading many small files causes slowdown depending on file system, virus checkers or other reasons. It may be a feature of the wx framework that is not optimal in some conditions.

The devs have added a progress window, which is a band aid, but some users report a long delay with no apparent progress, so there is some operation the developers think is quick which actually hits a bottleneck. I think the only way it will get investigated is if it is reproducible by a developer who is motivated to fix it...

For me there is a noticeable delay (20-30 seconds), which is manageable.
Bob
"All you said is just a bunch of opinions."
 
The following users thanked this post: SiliconWizard

Online SiliconWizardTopic starter

  • Super Contributor
  • ***
  • Posts: 14303
  • Country: fr
Re: Footprint association tool
« Reply #7 on: June 16, 2018, 08:47:27 pm »
Thanks for confirming the issue. I second the "long delay with no apparent progress". Actually it starts at about half the progress bar, which corresponds to the footprints loading stage.

It's indeed related to footprints loading and happens in any situation when footprints have to be loaded, such as selecting a footprint library.
There's obviously a patent bug since it's not just a matter of heavy CPU/HDD load - during those hang ups, both are not at all heavily loaded. The number of small files seems to be the trigger, so it's even worse in 5.0.0 with its gigantic base library. And again, my main workstation is a beast, so this is absolutely NOT a performance issue. At least not one that should happen. There is something else.

The old "it works on my own machine" drives me crazy. It's a joke in the software dev world. Yet developers still use it on occasion.  |O
https://blog.codinghorror.com/the-works-on-my-machine-certification-program/  :-DD You're welcome to use the certificate badge in KiCad forums next time.

Even 30 seconds would drive me crazy, but here it gets virtually stuck. The library management in KiCad was once its worst feature, and I was hoping for a lot of improvement, but apparently I'll have to wait for a few more years.

On the evaluation side, another thing that drives me nuts is the way selection works in KiCad. I guess you can get used to it, but it's pretty infuriating most of the time.

Too bad, I think the interactive router of KiCad is pretty good (except for the GUI quirks), and I would have liked to switch to it.

So I guess this will be yet another failed yearly evaluation time. :-DD
« Last Edit: June 16, 2018, 08:49:52 pm by SiliconWizard »
 

Offline donotdespisethesnake

  • Super Contributor
  • ***
  • Posts: 1093
  • Country: gb
  • Embedded stuff
Re: Footprint association tool
« Reply #8 on: June 17, 2018, 07:25:54 am »
I find the loading a bit slow on first startup, after that it's ok. I would love to get to the bottom of the problem, but I'm not very familiar with the KiCad source code, simply building it was a struggle.

I also hope the library management will improve..

Quote
another thing that drives me nuts is the way selection works in KiCad

Do you mean the old method or the new method? The old method is definitely weird. The new method implemented only in the OpenGL canvas follows a more conventional, click or lasso to select, then do operation on selected items.

The KiCad dev team is small and not well resourced. CERN are providing some sponsored development which has added some valuable contributions such as the new OpenGL canvas and push and shove router. I guess unless KiCad becomes better financed and organized as a non-profit foundation then progress will continue at a slow pace.
Bob
"All you said is just a bunch of opinions."
 

Online SiliconWizardTopic starter

  • Super Contributor
  • ***
  • Posts: 14303
  • Country: fr
Re: Footprint association tool
« Reply #9 on: June 17, 2018, 10:15:51 pm »
Well, the OpenGL mode is definitely much better IMO, but apparently is only implemented in the Layout tool? In the schematics editor, it's still the old way and I find it a royal pain to use. I have used a lot of different CAD tools and KiCad's schematics editor is one of the worst I've used usability-wise. It's full of features but very unproductive. YMMV.

I have no real clue what means KiCad now really has for development. CERN had advertised its involvement pretty loudly, so I assumed it got some serious means and a lot of traction. DigiKey also got involved: https://www.digikey.com/en/resources/design-tools/kicad , so that must mean they have detected the potential and the significant user base.

Yes maybe it's still a small team. But they should still listen to their users once in a while. The badge is still up for grabs: :-DD


Rant off.
 

Offline Bassman59

  • Super Contributor
  • ***
  • Posts: 2501
  • Country: us
  • Yes, I do this for a living
Re: Footprint association tool
« Reply #10 on: June 18, 2018, 04:36:37 am »
Well, the OpenGL mode is definitely much better IMO, but apparently is only implemented in the Layout tool? In the schematics editor, it's still the old way and I find it a royal pain to use. I have used a lot of different CAD tools and KiCad's schematics editor is one of the worst I've used usability-wise. It's full of features but very unproductive. YMMV.

Dunno, I’m used to the Kicad schematic editor and it’s fine. This is the sort of “not what I’m used to” comment that accompanies every CAD thread. I mean, LTSpice’s capture sucks, right?

As for OpenGL — redoing the schematic editor for that is in process.

Quote
I have no real clue what means KiCad now really has for development. CERN had advertised its involvement pretty loudly, so I assumed it got some serious means and a lot of traction. DigiKey also got involved: https://www.digikey.com/en/resources/design-tools/kicad , so that must mean they have detected the potential and the significant user base.

Yes maybe it's still a small team. But they should still listen to their users once in a while. The badge is still up for grabs: :-DD

If you’re actually interested in understanding what the developers are actually doing, instead of whinging and making assumptions, why not follow the developers’ mailing list? https://launchpad.net/~kicad-developers
 

Offline donotdespisethesnake

  • Super Contributor
  • ***
  • Posts: 1093
  • Country: gb
  • Embedded stuff
Re: Footprint association tool
« Reply #11 on: June 18, 2018, 10:38:53 am »
But they should still listen to their users once in a while.

They do, and the users basically want everything for nothing and don't care who is writing the code.

It's really quite strange how people become incredibly entitled when it comes to using Open Source code. You should be asking how to return the favor and how to help KiCad, rather than stomping feet and demanding changes.

It's like lending a lawnmower to a neighbour, who then demands that you also cut their lawn for them.


Bob
"All you said is just a bunch of opinions."
 

Online SiliconWizardTopic starter

  • Super Contributor
  • ***
  • Posts: 14303
  • Country: fr
Re: Footprint association tool
« Reply #12 on: June 18, 2018, 07:54:20 pm »
Reminds me a bit of this discussion with Dave: https://www.eevblog.com/forum/eda/so-what_s-the-future-standard-cad-tool-for-open-source-hw/msg696833/#msg696833

Evaluating and pointing out what we think are weak points is contributing. If I didn't care I wouldn't even bother seriously evaluating it with the goal of switching to it eventually. And as Dave said, this kind of forum is IMO a good source of "external" opinions, much more so than any internal mailing list. We are not KiCad developers, only experienced users.

A lot of us don't necessarily have the time to invest for actively contributing. Even so, those familiar with open source projects of this size know that it's often not a very democratic process. The maintainers do decide what the priorities should be and have no particular incentive to take other points of view into account, nor any particular direct contribution. Gnome has been a very good example of this in the last years. Of course we could always start our own fork. That happens *very* often in the open source world, usually for this very reason. This takes a lot of time, a scarce resource, unfortunately.

@donotdespisethesnake: you're saying they listen to the users yet gave me a good example (which was the whole topic of this thread) that they don't always. I'm familiar with open-source projects and many devs can be very stubborn; I was pointing out that the "works on my machine" is definitely not a sane attitude. As for the additional (a bit off-topic) GUI rant, that was just a quick rant. Point in case was that improving usability, along with improving library management, would definitely make KiCad a very serious contender IMO. As it is, not quite so.

@Bassman59: I admit a lot of CAD software are very quirky (even the big names) and that you get used to them eventually. I'm not sure we, as end-users, should just buckle. There are so many CAD tools that look like they were designed in the 60's. May be time to embrace GUI concepts that are a bit more usable. :-+
Actually, I find LTSpice editor much more productive than KiCad's one. Admittedly it's rather primitive but does the job for editing circuits for simulation and is reasonably intuitive. I wouldn't use it for general-purpose schematics design, because it lacks features and its look is rough, but it's not that bad usability-wise.

I just find it sad that a lot of sofware (and obivously not just KiCad for that matter) tends to be plagued with feature creep, yet often fails to address long-standing issues. This can be seen in as many (or more) commercial products as open source projects. Just a thought.

As an additional thought, and to be completely fair, since KiCad got funds from the CERN, we can't really consider this is totally "free" anymore, at least not for those of use who live in countries that are active financial contributors of the CERN. This is actually a (admittedly pretty small) chunk of our tax money. ;)

Anyway, this is just my views and opinion. Take it or leave it. No offense intended to anyone.
« Last Edit: June 18, 2018, 08:04:01 pm by SiliconWizard »
 

Offline Bassman59

  • Super Contributor
  • ***
  • Posts: 2501
  • Country: us
  • Yes, I do this for a living
Re: Footprint association tool
« Reply #13 on: June 19, 2018, 06:30:54 pm »
A lot of us don't necessarily have the time to invest for actively contributing. Even so, those familiar with open source projects of this size know that it's often not a very democratic process. The maintainers do decide what the priorities should be and have no particular incentive to take other points of view into account, nor any particular direct contribution.

All I am saying is this: if you followed the developers' email listserv, or logged into their IRC channel, you would see that Kicad development is more democratic than you state. Wayne takes suggestions seriously, and when there are decisions about what to implement and how, there is a lot of discussion. Sometimes things are held off from implementation because it would seriously break existing code and existing designs. What might seem an easy feature to implement may not be so easy to do!

The developers actually understand the PCB design process (something other commenters seem to not understand).

I follow the list, occasionally make suggestions ("... from a user's perspective, I think ...") and I run nightlies on macOS to test the program on that platform. I mention that because I'm impressed by the amount of discussion that does go on, and impressed by the amount of code contribution from non-core developers.

I will say, again, that complaining on this forum doesn't get results. If you have suggestions, enter them into the Launchpad bug reporter (as wishlist item). Ask on the mailing list if your suggestion is reasonable. It's a responsive group. But manpower is what it is, and the more that people can contribute, the better.

Quote
@donotdespisethesnake: you're saying they listen to the users yet gave me a good example (which was the whole topic of this thread) that they don't always. I'm familiar with open-source projects and many devs can be very stubborn; I was pointing out that the "works on my machine" is definitely not a sane attitude.

I've been testing nightlies of Kicad for a long time, and when I hit a bug, I report it as comprehensively as possible. I've never gotten a "works on my machine!" response. Sometimes the problems are platform-specific, so the developers who know my platform (macOS) best have to chase down the bug. But in every case, the response has been excellent, especially considering time zone differences.

Quote
As for the additional (a bit off-topic) GUI rant, that was just a quick rant. Point in case was that improving usability, along with improving library management, would definitely make KiCad a very serious contender IMO. As it is, not quite so.

If you were following the developer listserv, you would know that they really do recognize the library management issue as a serious problem, and there is a lot of work going on to improve it. And they want to do that without breaking existing designs. GUI usability is improving, too, and the rewrite of the schematic capture will go a long way towards making people happy (things like consistent hotkeys across the programs, etc etc).

Quote
Actually, I find LTSpice editor much more productive than KiCad's one. Admittedly it's rather primitive but does the job for editing circuits for simulation and is reasonably intuitive. I wouldn't use it for general-purpose schematics design, because it lacks features and its look is rough, but it's not that bad usability-wise.

I think it sucks, but since it's only used for simulating small circuits, I deal with it. I'm not going to go on the LTSpice forum and demand that it be changed -- the inertia is too great and users will complain.

Quote
I just find it sad that a lot of sofware (and obivously not just KiCad for that matter) tends to be plagued with feature creep, yet often fails to address long-standing issues.

When saying "yet often fails to address long-standing issues," it's helpful to detail those issues. Maybe the issues have been addressed (maybe in a nightly build)?

For the record: I defend Kicad because it actually works. I've designed a dozen boards with it, and I've never had an issue with getting the boards fabbed, and they all work. Any mistakes on the boards were mistakes I made, not because there was a problem with Kicad. Are there quirks and issues with it that I wish didn't exist or that it worked better? Sure. Is Altium better? Probably. Is Altium $13,000 better for the kinds of boards I design? Not at all.
 

Offline bson

  • Supporter
  • ****
  • Posts: 2265
  • Country: us
Re: Footprint association tool
« Reply #14 on: June 22, 2018, 06:34:35 am »
Actually, I find LTSpice editor much more productive than KiCad's one. Admittedly it's rather primitive but does the job for editing circuits for simulation and is reasonably intuitive. I wouldn't use it for general-purpose schematics design, because it lacks features and its look is rough, but it's not that bad usability-wise.
I guess there's a first for everything... I mean KiCAD's capture is mediocre, but LTspice is absolutely horrible.  No way to move components and have it automatically stretch wires is the big missing function in LTspice.  (No equivalent of the G or TAB functions.)  No duplication.  I don't particularly care about color schemes, but blue and black on medium gray?!  Seriously?!

I mean this is the main reason I'm stoked to see simulation getting added to KiCAD - to go from horrible to mediocre is a pretty big step forward.
 

Online SiliconWizardTopic starter

  • Super Contributor
  • ***
  • Posts: 14303
  • Country: fr
Re: Footprint association tool
« Reply #15 on: June 22, 2018, 04:01:03 pm »
I guess there's a first for everything... I mean KiCAD's capture is mediocre, but LTspice is absolutely horrible.

This is where we can see that habits are so powerful, especially when it comes to CAD software, that we have a hard time being really objective (me included of course), and we even miss features that are actually there.

No way to move components and have it automatically stretch wires is the big missing function in LTspice.

Yes there is and it's been there for as long as I've used LTSpice AFAIR, when it was called SwitcherCAD. There is a 'Drag' tool  near the 'Move' tool, in the toolbar as well as in the Edit menu, where you can additionally see keyboard shortcuts (which is F8 for dragging). Have you never seen it or does it not do what you want? It does allow dragging objects while retaining connections. You may not like its "accessibility" but at least it's not hidden since you can find it in both the toolbar and a menu. I'd prefer a more common way of selecting objects and a contextual menu, but the functionality is there.

No duplication.

Yes there is, again you can find it in the Edit menu (but not in the toolbar for some reason). Keyboard shortcut is F6. The selection mechanism is the same as for the Move and Drag tools (select the tool, then use the mouse to select the objects).

I don't particularly care about color schemes, but blue and black on medium gray?!  Seriously?!

The default color scheme may be dubious (but there is nothing more subjective than colors), but you can change colors to your liking using the Tools/Color Preferences menu.
 

Online PCB.Wiz

  • Super Contributor
  • ***
  • Posts: 1476
  • Country: au
Re: Footprint association tool
« Reply #16 on: June 24, 2018, 10:21:33 pm »
Checking the PCB side (pcbnew) Library handling, I can see that

Add-Footprint Search by keyword, does a full-trawl, and that's ~ 30 seconds in this PC. (not SSD)

Add-Footprint select by browser, brings up a libs-dir instantly, (must have cached that at some stage), and you can pick any item in under 1 second.
So that gives some idea of cache tables times, versus a keyword trawl time, for libraries.
 

Online SiliconWizardTopic starter

  • Super Contributor
  • ***
  • Posts: 14303
  • Country: fr
Re: Footprint association tool
« Reply #17 on: July 09, 2018, 06:42:43 pm »
Just a quick follow-up. Willing to be fair and give KiCad 5 a real shot, I finally got what triggered the ultra-long footprints loading (which prevented any useful work in KiCad), and that's probably what other people who get the same behavior run into. I think this point should be emphasized for users updating KiCad at any point.

It comes from the 'fp-lib-table' file that's in the user's config. directory (on Windows, it's 'C:\Users\XXX\AppData\Roaming\kicad'). If you happen to have installed older versions of KiCad (even when you may have forgotten about it completely) and update, this file doesn't get updated in any way and may contain links that are not valid anymore. In my case, it contained references of GIT URIs (which isn't the default anymore) which presumably were incorrect and would make updating time out for every footprint without any error message, thus the gigantic overall duration. I finally understood this while after a last try, out of curiosity, I left it "load" footprints and it lasted over 8 hours... and when it was finally done, I realized there were seemingly no new file after this and the next load was going to be as long. So after some searching, I found the culprit was this file, deleted it and it solved the issue. The first loading took just ~20 s. Anyway, there are definitely things to improve in this footprint loading code, error handling being one of them. The following chunk of code needs some rewrite, and also explains why the GUI is mostly unusable while the loading is in progress: http://docs.kicad-pcb.org/doxygen/classFOOTPRINT__LIST__IMPL.html#a8ec395955feb57f414c82ec9e375eeb6

So, with that "fixed", I tested the layout tool and as I said earlier, I think it's pretty good now. Too bad I still kind of hate the schematics editor (usability issues, very low readability at low zoom factors), but if it got the same overhaul as the layout editor, with an OpenGL or Cairo-based rendering, it could definitely become a winner.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf