Database library problems

[JohnG]

Folks,

Perhaps someone can lend some insight here.

My company been trying to move to a database library for Altium, since we have increased the number and complexity of designs in the last few years, and with it an increased number of people who use Altium. Our designs are generally low-volume (100s to 1000s) of demo and development boards. We have done a lot of work cleaning up the libraries, and have chosen to start with an Excel database rather than MS Access to allow our assembly house to update the database (which will be checked and is under version control).

The problem is this: Our company uses 32 bit MS 365 based on MS recommendations. Furthermore, 64-bit Office/365/whatever breaks macros. Apparently, Altium uses a database component from MS Office, 365 or whatever they call it, and the 32-bit version is not compatible. They have instructions to install the 64-bit MS Access component, but some members of our team have recently had new computers, and these instructions no longer work https://www.altium.com/documentation/altium-designer/using-database-libraries-with-32-bit-and-64-bit-altium-design-software-on-the-same-computer. It turns out that if you had followed Altium's instructions before the last 365 (Version 14), they did work, but at some point Office has broken these instructions (Version 15 or 16). Mine works.

Altium's suggestion is that we no longer use the macros and install 64-bit office, end of story. Unfortunately this affects a fair number of people, and is not a viable solution right now. Anyone dealt with this or have a workaround?

We have a total of 6 Altium seats, a mix of fixed and floating, and have stayed current on our subscriptions the last several years. Apparently this makes us a bottom tier customer based on customer service for this and other requests for help. No such problems from their sales and marketing, who pester us endlessly.

If anyone has a workaround or suggestions, I would really appreciate it. In the meantime, I will take another look at Kicad, but I have been following it and am not sure it's quite where we need it to be. But who knows, maybe it's close enough.

Thanks,
John

[Pseudobyte]

I strongly recommend against excel (and even access) for doing databases in Altium.

Move your data to AWS, it feels faster, is less buggy, and is redundant.

https://aws.amazon.com/rds/?hp=tile&so-exp=below

Micro instances are cheap and would likely cost your team on the order of 20 dollars a month to run.

You can use access to connect with it or use a third party database tool. Access can be nice for creating forms for part creation and the like though.

The important thing is that Altium's connection is direct and the database is not local file based.

[temperance]

Why would you as a company choose AWS? You probably have your own server if you employ a team.

What you wrote seems inconsistent. For access you will need the office component you wrote about but you are using excel.

Altium will read the excel files and there is no office component involved. I've made my excel files for AD with Libre office. You just have to take care each part has a unique identifier. A database would be more practical. But you mention macro's. Are there macro's in the excel files?

[olkipukki]

If anyone has a workaround or suggestions, I would really appreciate it.

Do you have somebody there to manage a server DB?

Literally,  it can be any database where you can connect via ODBC / OLE DB.

I would stay away from desktop DB (aka Excel, Access, etc).
It only make sense if somebody working alone / off-line / WFH and do not have a permanent access to a company server.

[Pseudobyte]

Why would you as a company choose AWS? You probably have your own server if you employ a team.

What you wrote seems inconsistent. For access you will need the office component you wrote about but you are using excel.

AWS because the reach is global and the experience is consistent no matter if you are in the office or at home. Sure you can host your own PostgreSQL or MySQL database onsite if you prefer.

Access in this situation just becomes a tool to modify the external database. You can connect access to remote databases and use it purely as a GUI for modifying data.

"Altium will read the excel files and there is no office component involved."

This isn't true. Otherwise why do you have to have office installed as 64bit. The odbc connection has to literally load the whole excel file into RAM every time it is queried, I am guessing it launches a headless version of excel to manage the queries.
Bottom line is it is very slow for large databases. I tried excel when we were starting up, I seeded the parts db with 10000 resistors and capacitors and it just couldn't do it.

[JohnG]

We don't have a lot of parts, and we don't have database experience. We will never go into large production of boards and designs because that is not our main product. Our Altium database is about 2000 items, and growth rate of new components has tapered off to about 100/year. For those of us who can install the Access component, it works flawlessly. It's not the cost of AWS, it's that fact that we have NO idea how to do it and we want control of what we have. Excel easily handles what we have, and it is easy for one of our team to make a new part using it. Going to a full database is way overkill, and we have no other use for any database skills developed, and won't use it enough to stay sharp with it.

Requiring on-line access is a non-starter for us (assuming the world starts to travel again), because many of us work remotely or while we travel. Believe it or not, there are still quite a few places where you cannot get a good, reliable connection whenever you want.

What it sounds like is that there is no known fix, and maybe we will be stuck going to some other solution like an actual database, which will cost us a bunch more time and resources. It's just galling that we pay for both products (and Altium is quite expensive), and get the brushoff.

John

[olkipukki]

Did you try create ODBC 32-bits Excel datasource with your Excel workbook and set up 'Use Connection String' in Altium (rather than 'Select Database Type' Excel) via Microsoft OLE DB Provider for ODBC Drivers?

[JohnG]

Did you try create ODBC 32-bits Excel datasource with your Excel workbook and set up 'Use Connection String' in Altium (rather than 'Select Database Type' Excel) via Microsoft OLE DB Provider for ODBC Drivers?

I have barely an idea what any of this means, much less how to go about it. But, there are a lot of specific clues here that I can follow up on, so I appreciate it. It's at least 10x more useful than Altium's response.

Cheers,
John

[lluchiari]

Hi there...
Well...I'm somehow new to Altium, but I've been studying it a lot. My company has recently bought two Altium Licenses and in the migration process (Proteus to Altium) I confess I'm a little bit disappointed the way that Altium threat library management. We are having serious problems with shared development and library (we are here in two engineers) and the best solution Altium offers you is to buy Altium 365 and Concord Pro.

Well...I tried all those method (IntLib, DbLib, SVNDblib) and DBLib would be amazing if that works correctly. I submitted a lot of cases on Altium for bugfix! And another drawback is that it's not easy to setup and operate. But I made it works! (Still buggy, but it's Altium's bug).

Well...my suggestion...watch FEDEVEL tutorial () and try to apply this for Excel or Access file! I suggest you Access, it's way easier to setup. I left db file on my local server and I use SVN to checkout a repository locally containing my Footprints and Symbols. I used a structure very similar to an Altium's example project (you can download here: https://techdocs.altium.com/display/ADOH/Download+Examples+and+Reference+Designs). I guess inside there's a project called SVNDblib example or something like this and they show the exact structure! It works!

But...besides this, I talked to a lot of my dev friends and they suggested me to pay (it's not what I want  ) or to make an SVN repository (could be GIT, it doest matter!) containing an IntLib and all my Footprints and Symbols all integrated and install this lib in Altium an work with commits.

Best Regards,
Lucas.

[conelec]

Folks,

Perhaps someone can lend some insight here.

Hi John,
I am new to this forum but have had a lot of personal experience with Altium starting with Protel 3.2 back in the early days when they still used "dbase". Frankly I don't know how they stay in business. After over 30 years this is the best bug ridden crap they can come up with.

You can run a 32 bit Access back end database if you install the 64 bit Access runtime. There is a guide on how to do it on the Altium website. I tried it with AD20 after discovering that AD20 no longer supported 32bit and it worked for me. Unfortunately AD20 was too bug ridden to even consider upgrading to from my AD15 version. I will have to wait another 5 years I am guessing. I spent hundreds of hours developing a full component database management front end and was relieved that I could still use it without having to do a rewrite.

I can see on this forum that most of you are looking at component management and databases too simplistically. First of all the .DBlib database connection works very well in connecting a back end database to the Altium library manager (one of the few things they got right). The confusion comes about in that you need a front end database to manage the back end data. All this other crap about library version control is unnecessary in my opinion. Everything is controlled by the component you purchase. It is fixed and never changes. When you or your purchasing department purchase a part they assign it a unique part number That part number then attaches a schematic symbol, a footprint and a 3D model. It's the job of the front end database to manage all this. This is where the crap starts. and I won't go into detail about it here suffice to say that little thought has gone into accessing Altium libraries in a way to make them available directly to a front end database.

Wile the FEDEVEL tutorial is a good start and lets you know that any free database like MySQL etc will work, it's too simplistic an approach. Open office is a great contender to access in my opinion. It lets you create the back end database which you can import from most other databases and gives you the power to create a great front end as well. And it's free.

Planning the database is the key to having a parts management system that will expand and grow along with your business.

I think that's enough for now.
Cheers
Pete

[olkipukki]

Everything is controlled by the component you purchase. It is fixed and never changes. When you or your purchasing department purchase a part they assign it a unique part number That part number then attaches a schematic symbol, a footprint and a 3D model.

Are you purchase same, as example, 10K 0603 1% resistor or 0.1uF 16V 0603 capacitor each and every-time?
What about if you want to place same part (again, for simplicity - resistor or capacitor), into a high-density board or just some prototype one - do you use same footprint for both?

[conelec]

Are you purchase same, as example, 10K 0603 1% resistor or 0.1uF 16V 0603 capacitor each and every-time?
What about if you want to place same part (again, for simplicity - resistor or capacitor), into a high-density board or just some prototype one - do you use same footprint for both?

[/quote]

A 10K 0603 1% resistor footprint should never change. Even if you buy this resistor from different vendors.
The database library allows you to link different footprints to the same part so long as they are listed in the back end table. For instance if you want to use footprints for wave soldering or you want to use footprints for re-flow. I don't see there is any difference using the same footprint for both prototype or high density.

[JohnG]

Thank you. We resolved the problem with Access. Now we are cleaning up our library.

We will not be moving to a true database unless someone else at our company does it. I am at my wit's end with Altium, anyways. I have just had to update to 19 from 17. Altium 17 proved to be quite stable, but 19 suffers from crashes, pauses, and hangs. We see them most when working on design rules, FWIW.

John

[conelec]

Hi John,
Even if you don't want to develop a full blown frontend for your database some careful planning is required to structure your backend data to future proof it.
Well we all vote with our wallets. It's become very evident to me that no matter how many updates you do with Altium you just get buried in an ever increasing hole of bugs. They have no interest in fixing the product you purchased and are happy to take your money for maintenance under false pretenses, so in reality you might have well just flushed the money down the toilet. Stop paying them and they will have no choice but to respond or go broke. I am seriously considering a look at Orcad. From what I read there seem to be far fewer issues.

Cheers
Pete

[JohnG]

We looked at switching to OrCAD. A problem for us was that the cost-comparable version was limited to one project open at a time (this was about 3-4 years ago). As a business, we tend to have numerous, low volume, simple designs, heavy on power and analog, with a large researchy vibe to them. If we switch, it will be in a few years at the earliest, and I hope KiCAD is far enough along at that time.

John

[conelec]

Hi John,
I have predominantly a repair business and mostly use AD for reverse engineering of existing designs. From that I have learnt a lot about the many variations in design, components and footprints from one manufacturer to another. From the knowledge I gained I created a database design that would adapt. The database drives Altium and not the other way round. For instance all the datasheets for components are available through the database with the database referencing the stored locations on a drive. Same goes with all the schematic symbols, footprints and component models. They all have images with names that mirror the Altium library names. So for me any other IDE would have to be able to work in the same way with database libraries. Over time you will find the investment in your data will far out weigh the IDE. Complaining to the sales people at Altium has had little effect over 6 years and really the CEO should dig a hole and bury himself in it in shame for the poor job he is doing. There is a good saying that comes from one of your president's that goes something like"You can fool some of the people some of the time......" For all of Altium's failures to provide a unified structure it still can create good looking documents. You just need plenty of bug spray and lots of patients. I will still take a look at OrCAD in case they are doing it better now.

Cheers
Pete

[Pseudobyte]

I will still take a look at OrCAD in case they are doing it better now.

OrCAD is what you use when you want to have all your information in many different places, oh and you can't open two boards or footprints at the same time. I would say Altium truly is the most unified model for board design, maybe falling short just behind Zuken products. Having had the opportunity to use many different tools Altium has consistently been my favorite to work in. There are definitely some quirks but if you put a modest amount of effort into tweaking settings, and tuning it to the way you work, you will be happy with the result.

[conelec]

Thanks Pseudobyte for the comment on OrCAD. I have not had a chance to give it a try but I wouldn't want the sales team at Altium to get a big head by telling them their product is better than someone else's. They will completely stop improving if that's what you can call what they are doing. Every version since AD15 that I originally bought has landed a new set of problems that makes upgrading unusable. I customized my schematic grid to be in units of 1mm, 2mm and 4mm and also schematic symbols to match. You would think this should work but no. One wire connected between two points can have up to four vertices. Some places on the sheet you can't even get a tie dot to form. Other places a tie dot will appear weather you want it or not. Unfortunately I can't upload a video in MP4 but if you took 1-2 mins to make connections you would cry.  Zuken Looks very interesting.

[voltsandjolts]

I customized my schematic grid to be in units of 1mm, 2mm and 4mm

Now that is just crazy talk, maybe you are making problems for yourself.

[conelec]

@voltsandjolts

I did this so I could fit the majority of my schematics on an A4 sheet and the more complex ones fit on an A3 sheet if needed. What's crazy about that? Who wants to use Altiums crappy outdated symbol library anyway!

[voltsandjolts]

You miss out on not just Altiums vault components but everything thats out there.
Your on your own, but that's fine if you are happy to make everything yourself.
Also, why not just use a bigger sheet, 10 thou standard grid, then scale to fit page when printing.

[alexwhittemore]

I've tried DBLibs a few times and never liked them. A few years ago on a team of 3, we settled on just keeping IntLibs under Git control and telling each other if we made any edits to avoid merge conflicts.

These days, especially if the entire team is on up to date Altium subscriptions, I'd say just use A365. A365 is, among other things "team library management you just don't have to worry about anymore." Concord Pro is a set of features ON TOP OF that around lifecycle and release management that you can decide whether or not you care about (but you personally don't, so don't worry about it).

I know the Altium subscription I just (finally) bought myself includes A365 for free, and it sounds like that's probably not a special, limited-time thing: I think the whole point of A365 is to give people who don't really care about updates or support a set of reasons they might want to maintain a yearly subscription.

If your team's subscriptions don't come with A365 automatically right now (which I think they should and probably do), I'm betting you can just call up your sales rep and say "hey we don't seem to have the 365 features that we're supposed to" and make them turn it on.

Seriously, A365 AFAIK has literally two main features: turnkey data management, and third-party data access with granular permissions, for use-cases like "show the vendor this whole design without emailing .zips." It's not earth-shattering, or crazy, or anything. It's just a way to solve this specific problem you seem to have right now, and keep you paying the yearly subscription fee you already are.

If you're rigorous and nerdy with data management and git and libraries and all the rest of it, you probably don't need A365. If, as it sounds is your case, you just don't want to DEAL with that, and just want a system that works well without effort, A365 is probably exactly what you want.

Best part: if you decide it's not enough to justify paying yearly, you can seamlessly export all your data, and relatively easily. Projects are JUST git repos hosted on 365 instead of github, and with a good altium-specific viewer in the web interface. Libraries are vaults, but can be exported to schlib/pcblib/intlib as you please (and are versioned as well).