Electronics > Open Source Hardware
Open bus architecture for instrumentation/control (VME, VXI). Feedback wanted
msat:
Hello, everyone! Occasional lurker, but first time poster.
There's been something that's been in the back of my mind for several months that occasionally springs to the front, which I've been gestating a bit more as of late. This has come about after finding myself creating bespoke circuits for instrumenting and controlling experiments. Sometimes I have to add additional capabilities which becomes somewhat unwieldy, or requires redesigns and/or firmware hackery. Worse still, these are often not reusable for other projects. Sure, it can in part be blamed by lack of foresight or good practices/discipline. This is partly the goal of this project. Yes, VME, VXI, etc already exists but there's reasons I don't want to go that route which I'll elaborate more on later. Currently, this project is nothing more than a collection of thoughts, and may very well stay that way, but in any case, I'd like to get feedback on the design choices that I'm currently considering.
First things first: On my part, I intend for this to be completely open source. I have no intention of making money off this, but really just want to benefit from people creating open source modules that are useful to me.
So, it shouldn't take much imagination to get a visual idea of what this project is. Either it'll consist of stackable modules with board-to-board connectors, or modules with something like card edge connectors that slide in a rack and connect to a backplane (though it certainly could be both, though I'm leaning towards the latter for tidiness reasons).
There are 3 items that are the core of the system which I see every unit containing:
1) A global voltage reference module. This provides a Vref for all additional modules that require it. Since this is a module, it can be as fancy or cheap as you need.
2) Global clock ref module for sync and time keeping/stamping
3) Bus to USB bridge module, likely by was of the Cypress FX3 USB 3.0 peripheral controller.
That last one requires the most elaboration because it's in a way the most critical component, and inherently differentiates this from the other bus architectures. The FX3 offers quite a bit of flexibility in terms of interfacing via a customizable parallel bus, good bandwidth, some inter bus-to-usb processing using the 200MHz ARM9 core, as well as SPI and I2C controllers, which are all things that make it amenable to low cost and DIY-able rack modules. I see a lot more drawbacks than benefits in creating a bus that is designed to support computing modules, while the FX3 potentially allows for much more flexibility and future proofing by giving someone the ability to interface with the computer of their choice. No, it can't compete with PCI-Express in terms of bandwidth and latency, but it's not meant to. This is really targeting the home/small/school lab. It's just one of those engineering trade-offs and I have no illusions of it being a one size fits all type of thing.
Current thoughts on a parallel bus design: The FX3 allows for an up to 100MHz 8/16/32-bit wide bus with configurable control lines, which I don't think is intended to be reconfigured in real-time (but possibly might be). In terms of bandwidth, 32-bit would be the most desirable, but I personally think chews up too many IO pins of a uC each module is likely to contain. 16-bit is probably a happy medium while simpler modules could use SPI/I2C. The FX3 parallel interface can be driven from an external clock up to 100MHz and best I can tell, no lower limit. The idea here is to let each module control the clock to set the bus speed so that even relatively slow microcontrollers can be used. That's not great for maximum bus throughput, but the ability to use potentially simple low-cost modules is more important, IMO. Again, this isn't mean to compete with VXI in terms of performance, but should still be plenty fast for many uses.
PC-side drivers: The FX3 has 15 in + 15 out usable USB endpoints. For a rack that probably would support ~8 modules, that would allow at least one in+out endpoint per module. While that could certainly be handy, I'm personally leaning towards it enumerating as a mass storage device and interfacing taking place via virtual files (inspired by other projects that used this method). The main reason for this is that it allows for the use of OS-agnostic driver libraries and the benefits that brings. That said, since this will be fully open source (and hopefully everyone's custom modules as well), endpoints can be enumerated however a person wants.
Since the current vision is that all communication takes place only between a module and the FX3, I don't think it's necessary that trace lengths between each slot and FX3 be the same, especially since each module will drive the bus clock. However, I do intend that the trace lengths match for the global ref sync clock.
Originally I was thinking about additional bus lines to be used for inter-module communication but I've since shied away from it. That requires a bit of forethought that I really don't want to spend time on, especially since I'm having a hard time seeing the value in it, and I definitely don't want to try to accommodate every edge case (which is hopeless). Best I can do is either spec a card edge connector with plenty of extra pins for potential future use, or place the used card edge in a location where another one can be added down the road.
Ultimately, I would like to see people not only developing a sharing their module designs, but potentially seeing them being picked up and produced from places like China, similar to what has been done with other useful gadgets (the LCR and component testers come to mind).
I don't want this to eat up too much of my time. I'm hoping that my initial investment offsets the time I'd spend down the road making custom and non-reusable circuits for my experiments. Though I don't want this to end up being completely hampered by poor design choices either, which is why I'm seeking feedback. There's probably a handful of details I've considered that I forgot to mention, but I think I outlined the bulk of them. Any illumination of major flaws would be appreciated.
-Mark
T3sl4co1l:
Nice thing about standards, there's so many of them...
What would you really be using this for?
Also, why parallel? Peripherals are in ever shorter supply, and it's such a pain to breadboard a half a hundred connections (or if we're talking 32 bits here, maybe hundreds).
Assuming this gets advertised enough for people to be aware of it -- what would they find in it, that isn't offered by, say:
- PCI: thousands of expansion cards to choose from (some still being made..?); require drivers (which are mostly x86/Windows??)
- ISA, in slot or PC104 form: still standard in industry, I think? Low bandwidth, may require various compatibility strategies (setting port/memory windows, drivers, etc.).
- USB: just toss a bunch of hubs on it, she'll be right. Plug-and-play, may need drivers but also many things can just be used as character-mode devices, and structures can be built on top of that (e.g. mounting ttyusbN via some filesystem driver in Linux must be a thing?).
- PCIe: like USB, it's serial and it's fast, just not (usually) cable-oriented. Tons of peripherals and expansion cards, more bandwidth than you can shake a lane at; plug-and-play; drivers compatible with current computers; etc.
- VME and all those: unless you're supporting ancient cards or doing vintage computing sorts of things, I don't get it, and again don't see how you're differentiating from it..? (You mentioned VME once but didn't refer back to it later; not sure if we're supposed to understand VME and draw that conclusion ourselves?)
- Or also from the vintage era, GPIB: slow, moderately addressable, cable-oriented.
- Or if bandwidth really isn't much, then any of the embedded serials buses, like I2C, or SPI with multi-master addressing and arbitration, or RS-485 with arbitration.
- Or Ethernet for a similar experience (lots of serial point-to-point links) but way more bandwidth, and again, standard hardware. VREFs can't be shared, but time is regularly synchronized over it!
- Or for an emphasis on analog, whatever Eurorack synths use..? Maybe that's just power, I never looked it up...
I guess what you're really after, is some manner of metrology playground, given the emphasis on VREF and CLK? That already puts it into a very narrow space, and like, if all you're after is some community adoption of something, I guess that could work? But I don't see much reason for industry to consider it.
Note that most of these buses include a clock already, which may not be all that high quality in the original application, but there's no reason you can't make it so. You could take a 10MHz disciplined ref and PLL it to the 8MHz of ISA, or 33 or 66MHz of PCI, nicely buffered at each card connector; or just abuse it for your own purposes and dump 10MHz through the pin regardless of the standard. Likewise, the -5 or -12V supplies (which at a glance, VME has as well?) could be used for REF, equipped with modest current capacity to remain compliant, or who cares just make it a modest -5 or -10V ref and don't use any cards that draw much current from it (which is most of them, AFAIK?).
Tim
Doctorandus_P:
This is also a long standing idea of mine, but it never got much further then that.
Join this thread:
https://www.eevblog.com/forum/projects/diy-instrumentation-bus-(or-dib)/
I think it's mostly fizzled out over the years, but you may find it interesting to read.
Here's a list:
http://www.interfacebus.com/Interface_BackPlane_Buses.html
There are also a few open source bus based systems you can re-use. Some time ago I even found an open source project which made it into a real-life product and you can buy chassis with power supply and several plugin boards commercially for reasonable prices.
=============
My latest idea is to use a bus architecture that is no bus architecture at all.
The idea a box with:
* Front accepts pieces of angle aluminum, which can have connectors, switches, etc, and also a PCB can be screwed to it.
* The boards with angle aluminum, are all connected to some frame to keep relative positions solid.
* The whole frame can be lifted out of the box, so 3 sides of all PCB's are accessible.
* All PCB's have connectors that fit their function, such as USB, Ethernet, CAN, Whatever.
* I think I'd define a fixed location for an IDC connector for power and some common signals such as I2C, SPI, UART, and then use it either with flatcable or a ridgid PCB.
The main idea though is to have simple way to connect different future projects together in a flexible way so any sort of connector / standard connection can be used for your future projects.
In some ways it's similar to the idea of: "Use DIN rails", on which you can fit almost anything, but then for a relatively neat and flat front panel.
If you look closer at a system like:
https://linuxgizmos.com/modular-open-source-automation-controller-runs-on-raspberry-pi-cm3/
Then (from memory, it was some time ago) it is not much more then a small Linux computer, with a power supply and an USB3 hub, and a fancy box which makes it incompatible with anything else. Is that worth USD350?
I'd rather grab an USD10 USB hub, glue it to a piece of angle aluminum and then use it with any small linux computer that is not based on broadcom chips. But whatever board you want. Olinuxino, Odroid, BBB, etc, or some other board such as some form of microcontroller development board. As long as it's got some mounting holes it's easy to fix to a piece of angle aluminium to stow it away in a box so cables etc do not get dislodged if you want to clean your desk and stow away your project for some time.
And even if it does not have mounting holes, you can still glue it to the aluminum profile :)
Those low volume projects can never compete on price with mass produced standard products.
msat:
--- Quote from: T3sl4co1l on November 09, 2020, 03:06:36 am ---Nice thing about standards, there's so many of them...
What would you really be using this for?
Also, why parallel? Peripherals are in ever shorter supply, and it's such a pain to breadboard a half a hundred connections (or if we're talking 32 bits here, maybe hundreds).
--- End quote ---
You're right, there's no real shortage of standards, but it's a reasonable question to ask why there are so many. Sometimes the most ideal standard to use is one that doesn't exist yet, perhaps unfortunately.
Just to be clear, when I say parallel, I don't mean the PC-type parallel port - I just mean the bus is more than 1-bit wide. The reason is for bandwidth purposes for a given clock rate. Plus, it's often fewer instructions on a uC to bit-bang 8 or 16 bits in parallel than it is to shift in/out one bit at a time. Every uC can do this as long as it has sufficient pins. The current thought is 16 bits + few control lines. But this is all tentative and you have me questioning it, which is actually helpful. So, thanks!
--- Quote ---Assuming this gets advertised enough for people to be aware of it -- what would they find in it, that isn't offered by, say:
- PCI: thousands of expansion cards to choose from (some still being made..?); require drivers (which are mostly x86/Windows??)
- ISA, in slot or PC104 form: still standard in industry, I think? Low bandwidth, may require various compatibility strategies (setting port/memory windows, drivers, etc.).
- USB: just toss a bunch of hubs on it, she'll be right. Plug-and-play, may need drivers but also many things can just be used as character-mode devices, and structures can be built on top of that (e.g. mounting ttyusbN via some filesystem driver in Linux must be a thing?).
- PCIe: like USB, it's serial and it's fast, just not (usually) cable-oriented. Tons of peripherals and expansion cards, more bandwidth than you can shake a lane at; plug-and-play; drivers compatible with current computers; etc.
- VME and all those: unless you're supporting ancient cards or doing vintage computing sorts of things, I don't get it, and again don't see how you're differentiating from it..? (You mentioned VME once but didn't refer back to it later; not sure if we're supposed to understand VME and draw that conclusion ourselves?)
- Or also from the vintage era, GPIB: slow, moderately addressable, cable-oriented.
- Or if bandwidth really isn't much, then any of the embedded serials buses, like I2C, or SPI with multi-master addressing and arbitration, or RS-485 with arbitration.
- Or Ethernet for a similar experience (lots of serial point-to-point links) but way more bandwidth, and again, standard hardware. VREFs can't be shared, but time is regularly synchronized over it!
- Or for an emphasis on analog, whatever Eurorack synths use..? Maybe that's just power, I never looked it up...
--- End quote ---
While I'd be happy if others adopted it, I don't expect it to be the "next big thing". My hope is that perhaps enough other people would use it to create modules that I would find useful for my own needs, but mainly I want something to conveniently control and instrument my experiments. If I'm the only user, that's fine too. After all, I've already done that with previous experiments - and those would probably be almost entirely useless to anyone else.
Yes, plenty of devices exist with various interfaces, but with a home lab budget, you'll mainly be scouring ebay and you'd have to resort to settling with a hodge-podge assortment of equipment requiring a variety of different interfaces, many probably dated (and drivers not supported by newer OSes). That's one of the big things I'm trying to avoid. I'd prefer to have as many tools in one box as possible. I also want the interfacing to be simple and flexible for cost reduction reasons. If you want to make your own PCI compliant card, that's Big Boy stuff. If you want to make a PCIe card, then you're in grey beard territory.
As for mention of VME and VXI in the thread title, I was just using that as an example of what I mean by instrument bus architecture, and not that this necessarily shares any technical specifications with them. I couldn't make that more clear in the title since I reached the character limit.
When I first read your reply last night, you had me thinking of just using USB w/hub as a data bus (though still on a backplane to include some of the other things I want), and it certainly could be a workable idea. After all, plenty of microcontrollers across the performance spectrum support it. There is something about it that I don't like, though, and it's all related to PC-side drivers and enumeration, and the hassle you might have pointing the software to the correct module. I did just have a thought of a potential acceptable solution. I need to ruminate on this some more.
--- Quote ---I guess what you're really after, is some manner of metrology playground, given the emphasis on VREF and CLK? That already puts it into a very narrow space, and like, if all you're after is some community adoption of something, I guess that could work? But I don't see much reason for industry to consider it.
Note that most of these buses include a clock already, which may not be all that high quality in the original application, but there's no reason you can't make it so. You could take a 10MHz disciplined ref and PLL it to the 8MHz of ISA, or 33 or 66MHz of PCI, nicely buffered at each card connector; or just abuse it for your own purposes and dump 10MHz through the pin regardless of the standard. Likewise, the -5 or -12V supplies (which at a glance, VME has as well?) could be used for REF, equipped with modest current capacity to remain compliant, or who cares just make it a modest -5 or -10V ref and don't use any cards that draw much current from it (which is most of them, AFAIK?).
Tim
--- End quote ---
I suppose you can consider it a metrology playground in some ways. A global vRef just seems like a worthwhile thing to add in which all modules utilizing ADCs/DACs/comparators/etc can use. It also means you only need to calibrate one reference voltage in a manner potentially much easier than finding offsets for all those devices using internal references. I see no good reason not to have one, especially since a simple one is cheap (but that doesn't stop someone from going fancy if they need/want it)
The global clock is mainly for synchronization, which modules can reference for things like sequencing and timestamping. As with everything else, I haven't completely thought it through, but the idea is to not just be a pulse train, but also have a data line transmitting the current counter value. Once again, the quality of the clock is up to the user.
-Mark
msat:
--- Quote from: Doctorandus_P on November 09, 2020, 10:07:32 am ---This is also a long standing idea of mine, but it never got much further then that.
Join this thread:
https://www.eevblog.com/forum/projects/diy-instrumentation-bus-(or-dib)/
I think it's mostly fizzled out over the years, but you may find it interesting to read.
--- End quote ---
Thanks for bringing this to my attention! I suspected that I wasn't the only one interested in something like this. I will be looking through that thread for ideas and inspiration, or perhaps even try to revive someone else's good ideas.
--- Quote ---Here's a list:
http://www.interfacebus.com/Interface_BackPlane_Buses.html
There are also a few open source bus based systems you can re-use. Some time ago I even found an open source project which made it into a real-life product and you can buy chassis with power supply and several plugin boards commercially for reasonable prices.
--- End quote ---
I didn't come across a bus that I was satisfied with, mainly because I figured I'd end up creating the modules I wanted anyways, so I figured speccing my own bus would end up making my life easier and thriftier in the future.
--- Quote ---=============
My latest idea is to use a bus architecture that is no bus architecture at all.
The idea a box with:
* Front accepts pieces of angle aluminum, which can have connectors, switches, etc, and also a PCB can be screwed to it.
* The boards with angle aluminum, are all connected to some frame to keep relative positions solid.
* The whole frame can be lifted out of the box, so 3 sides of all PCB's are accessible.
* All PCB's have connectors that fit their function, such as USB, Ethernet, CAN, Whatever.
* I think I'd define a fixed location for an IDC connector for power and some common signals such as I2C, SPI, UART, and then use it either with flatcable or a ridgid PCB.
The main idea though is to have simple way to connect different future projects together in a flexible way so any sort of connector / standard connection can be used for your future projects.
In some ways it's similar to the idea of: "Use DIN rails", on which you can fit almost anything, but then for a relatively neat and flat front panel.
--- End quote ---
That's certainly an effective way of making things more tidy, but it's limited in what it can do about cable chaos. A data bus doesn't have to be super complex, but then obviously you're limited to equipment that supports it. Tade-offs, trade-offs, trade-offs.
--- Quote ---If you look closer at a system like:
https://linuxgizmos.com/modular-open-source-automation-controller-runs-on-raspberry-pi-cm3/
Then (from memory, it was some time ago) it is not much more then a small Linux computer, with a power supply and an USB3 hub, and a fancy box which makes it incompatible with anything else. Is that worth USD350?
--- End quote ---
I think you summed it up. I can't say it doesn't look cool, but otherwise it's a lot of things I don't particularly want. There's one low-performance interface (I2C) while the rest are vastly more high performance (and thus complex). Plus, while I don't have anything against a RasPI, I strongly dislike being anchored to any particular computer [architecture] or OS. It's also one of the things I dislike about some of the other buses that are intended to utilize a computing module. Why waste time designing a compatible SBC when you can plug in almost anything you want via ethernet/USB (unless you require the performance a higher bandwidth interface provides?)
--- Quote ---I'd rather grab an USD10 USB hub, glue it to a piece of angle aluminum and then use it with any small linux computer that is not based on broadcom chips. But whatever board you want. Olinuxino, Odroid, BBB, etc, or some other board such as some form of microcontroller development board. As long as it's got some mounting holes it's easy to fix to a piece of angle aluminium to stow it away in a box so cables etc do not get dislodged if you want to clean your desk and stow away your project for some time.
And even if it does not have mounting holes, you can still glue it to the aluminum profile :)
Those low volume projects can never compete on price with mass produced standard products.
--- End quote ---
Yep! It's both simpler and cheaper!
So I'm going to go through that thread you linked, and if it's appropriate to continue this discussion there, then I'll do so.
Navigation
[0] Message Index
[#] Next page
Go to full version