"DIY instrumentation bus" (or DIB)

[mubes]

Folks,

Sory I'm slightly late to this party, but has anyone considered Automotive Ethernet? http://www.opensig.org/ or https://en.wikipedia.org/wiki/OPEN_Alliance_SIG.  I don't know so very much about it, but it seems to have multidrop capability and supports different classes of transport.  Any open instrumentation bus is going to need to be capable of doing that if its going to be generally applicable.

Regards

DAVE

[prasimix]

I really like the idea of having an open-source mainframe style instrumentation architecture, but I really don't foresee it taking off as a decentralized, ad-hoc pseudo-standard as you've laid it out. 

Thanks for another batch of valuable inputs. Not really understand what decentralized means. Thanks also for label "pseudo" that is a reminder that it does not come from establishment. But unlike establishment I'm not ready to start another "religious war" to defend my position but want to hear an learn from you guys. So, lets move on...

In order for a backplane system to be successful it must:

- Have a defined way of managing interconnect resources and resolving conflicts (ie, I2C address collisions, allocation of cross-connect channels).

That’s right, but I didn’t even came to this point. I’m aware for possible issues with mentioned I2C. Situation with e.g. SPI could be a little bit better but still it’s possible that two modules allocates the same CS signal. Hopefully we can borrow something from VXI and similar specifications: on the backplane level they don’t have anything more then multiple CS signals. Therefore some kind of CS on the module level must be present (jumpers, or something else). Your input are welcome.

- Assure that modules do not interfere with each other--the massive switching converter on someone's DIY power supply module shouldn't be allowed to introduce a bunch of noise that will interfere with someone else's precision voltmeter module, for example.

Here we can also follow VXI practice: they are recommending shielding on the module level for sensitive circuits. That mean that whole module can be shielded or part of the PCB can carry shielding.

- Have a defined way of managing power budget.
- Have maximum bandwidth, line loading, and voltage level characteristics defined for all bus connections.

Ok, that should be clearly stated in specification. Also e.g. max. capacitance on the each input power line has to be specified.

- Once you've defined all of the requirements above (and many more), in order to achieve reliable interoperation you must ensure that all developed modules comply with those requirements.

Don’t see a way how to insure idiot-proof solution. Did anyone succeed with that so far? It’s possible to some extent to make some testing into firmware/libraries, but even that cannot ensure a lot. Ok, “corporate specification” usually offer or even demand some certification procedures. I think that in our case the voice of community would be the best judge does one should include certain design/module into the lab or not.

The more complicated the interface is, the less interest there's going to be in developing modules for it, and the harder it's going to be to ensure that those modules are compliant.

Exactly! That is true on all levels starting from backplane or other way of electromechanical module interconnection. That’s why 15th specification is proposed that could serve DIYers/makers better then VXI or some other .

Regarding the 'CPU card', I'm wary of this idea.  Maybe if it's a raspberry pi, but this is another area where the software support must be developed and behaviors defined...it's also an additional cost and amount of effort that may not be necessary if the instruments can simply be connected to an existing lab PC instead.

For the sake of specification simplicity (what we already addressed above) I think that DIB should be CPU/MCU agnostic. It could be any ready-to-go board such as Raspi, Ardiuno, Beaglebone, Launchpad, etc. Obviously such boards require some kind of “piggyback” on the hypothetical “CPU Slot #0” what is not necessary so bad, because on the same board/module one can put some additional circuity that is not available on the selected board itself (e.g. more memory, RTC, Ethernet, etc.).

The bottom line is, the more complicated this is to implement the less participation you're going to get.  The more options and loopholes and behaviors are left undefined, the less interoperability you will get.  If you want any chance of this succeeding:

- Fully define the behavior of devices on the bus.
- Minimize the hardware requirements for instrument devices so that defining requirements (and then implementing them) isn't an insurmountable amount of work. 
- Provide portable, easy-to-customize libraries for the interface.
- Provide working reference designs for a few basic devices.

I’m agree and can provide in the foreseeable future at least one working reference design: a multi-channel power supply with local (TFT touch-screen) and remote control (SCPI over USB and Ethernet).

If you really want to stick with a backplane/mainframe system, I don't see that being successful until someone designs and makes available for sale a complete mainframe system and several instrument modules that work out of the box (which means doing everything in the list above first!) and doesn't cost an arm and a leg.  Unfortunately I think the cost aspect is what really makes the mainframe idea a non-starter.  Low volume means it's going to be expensive to manufacture even before you try to recoup the engineering time to do all of the hardware and software development, and no one's going to want to spend $500 on an empty mainframe that they now have to spend hundreds of hours and hundreds of dollars developing hardware to plug into it.

Thanks for mentioning this and round figure of $500. I heard in some other discussions a similar figure (€500) here in Europe. My stance is if entry level mainframe cost more then €80 then something is wrong. Lets say that entry level is 4-slot mainframe. I’ll continue with “horizontal” backplane introduced in my latest post but it also can be a “vertical” on like in we’d like to use Eurocard or similar form factor. The backplane could looks like this:



The cost even in qty of 2 using European manufacturer (not Chinese one) is €30 per backplane (2-layer, 35u Cu, no silk screen):



On top of that we’ll need 8 connectors (about €14) and visit to local hardware store that carry steel or Alu plates (e.g. 1.5-2 mm thick) and L-profiles. I believe that can be get for not more then additional €20. Of course you have to cut on size plates to get up to 4 front panels, and rest of the chassis. But that is a “business as usual” for DIYer/marker, isn’t it? And, yes some hardware stores also offers for a little bit more already perforated plates that can be used for e.g. top side for better cooling.

8-slot backplane doesn’t cost much more: according to the same manufacturer calculator it is about €40:

Again, I really like the mainframe idea in principle, I just don't see it being successful as a framework for DIY instruments.  I think maybe it makes more sense to consolidate around an existing interface with maybe a thin wrapping that adds one or two really useful features that requires minimal hardware and can be used to link multiple instruments directly to a PC.  Build up some nice open source libraries for it and a few reference designs, then focus on developing a nice software suite to run it, and make sure it's easy for people to integrate their own instruments into the software.  Maybe require compliance with SCPI standard instruments, and have a nice profile builder tool for custom instruments.

I’d like to join any initiative that could save benchtop space, money and time. I don’t have neither interest, time nor knowledge to build up a whole lab. Hopefully this topic will not die even before it starts to present that is possible to bridge the gap between DIY and commercial world in an efficient way.

[prasimix]

After long time I came back to this topic that is still living in my mind. I'd like to continue it but this time with another component that should shape future DIB progress, and that is control or MCU board.
I was started my journey in placing on my benchtop equipments that is tailored in line with my wishes using Arduino as a control board. Despite the fact that my first device, a bench power supply, is rather trivial in sense of needed processing power it became obvious pretty soon that is not a case if you'd like to make it fully programmable and going beyond 7-segment LED or 2x16 LCD display as primary user interface. Therefore the PSU ended up controlled with 32-bit ARM based Arduino Due board that is actually not a real representative of Arduino world and Arduino team itself gave up long time from its production and with just dozen new software commits in previous year in its GitHub it can be said that is pretty dead (not to mention that even in its active phase many things were not supported as in case of AVR MCUs).
A next step in using more capable solution could be to choose between few very potent candidates, where e.g. STM32 with their Discovery evaluation boards looks like excellent choice. But, again when I start thinking about a little bit more demanding tasks where e.g. multi-Mbps of data have to be transfered and processed and that some of processes should we done in parallel for more acceptable end result then it's easy to came up into position to take into account SoC and FPGA to assist MCU.
FPGA is a different world, that is still waiting that both my colleague (software developer) and I discover, and nobody of us are eager to do that any time soon. So, one possibility is to try to use SoC or cheap SBC like Beaglebone, RasPi (or similar fruit variants like BananaPi, OrangePi, itc.) to take care about "hi-level" taks such as host communication, display control, media/USB control, etc. But, even with RTOS in one part of the solution it's questionable how some time critical operation can be handled, i.e. without measurable execution "jitter". But, there is another candidate that is still pretty exotic (i.e. "risky") and well known between digital audiophiles: XMOS multicore MCUs. It looks like something between MCU and FPGA, that can be programmed in C or XC that comes with extended set of commands for handling tasks in parallel. Parallel is the keyword for this MCU family, something that make it closer to FPGA then ordinary MCU. Although it really lack some basic features of top-grade MCU, processing power and possibility to process more tasks in parallel with resolution below 10 ns (!) make it really interesting for me. Additionally, even if number of resources on single MCU is not enough, you can easily make a true grid with e.g. 2, 4, 16 MCU that compiler could easily access. Also possibility to works with different version of firmwares and update firmware from single place (if more then on MCU reside on single or multiple boards) promise easier maintenance.
A list of features is amazing, but I'll continue to talk about things that could be placed anywhere from questionable to possible show stoppers:

• xCORE MCUs comes from single fabless company XMOS. They have a great experience with parallel processing and was involved with things like Transputers from 80's.
• Limited number of suppliers, three in total: Digikey, Farnell, Future, no my favorite TME
  
• Unit price and bulk price is not small but neither too high if it can deliver what is advertised
• One cannot expect a great community for something this is still exotic, but they have forum (xcore.com). My current experience with it is quite different from this forum or some others: it can be descried as "slow" with very small number of active members and many observers. I'm still wondering how to receive an "official" or reputable answers to my questions and doubts
• No flash (memory) even if flash is there! That really confused me on the beginning but, I start to live with it and as previously mentioned that could make firmware uploading task very flexible and easy. In short, MCU comes in two flavors with and without flash memory (has F letter in the name, e.g. XEF216 instead of XE216). But, flash cannot be reached directly and must be loaded into working/internal RAM!
• Limited and segmented available RAM: xCORE comes with max. 256KB of RAM per tile (one tile carry up to 8 logical cores). That's all! If 512KB is advertised that is in reality 2 x 256KB, and cores from one tile cannot access RAM from another one. When you take into account previous "flash feature" then it seems that one could be very careful about what to put into firmware. Just for comparison, top-class MCUs comes with up to 4MB of flash memory (per core) and I do believe that is not just a pure luxury but a real world necessity.
• Most of activities on official web site, GitHub and forum (app. notes, libraries, discussion that include XMOS team, etc.) seems that are dated a few years ago. Therefore one could expect a lot of missing link and information that is related to EOL (or soon to be) products. That can be understand to some extent - company crew had more time on the beginning for support and presentation of what they are offering.

Anyway, despite all that I decided to make a try with xCORE MCUs simply because I can (still) afford such luxury . It could be at the end a complete failure or something that cannot be easily labeled with "just another xyz MCU based (and equally limited!) solution"! I have no illusion that it could replace usage of FPGA, but it definitely should over-perform MCU on many fronts. One of them is "boring" debugging. Its free and open source IDE (xTIMEcomposer based on Eclipse) offers non-intrusive debugging in real-time. Crazy stuff, all what you need is a xTAG debugger (decently priced) connected (if board has XSYS 20-pin IDC connector). An introduction in so-called xScope can be seen in the following video:



I was thinking how to start with XMOS evaluation. They started few years about with startKIT board that is now obsolete and replaced with eXporerKIT that comes with xCORE-200 (2nd generation) MCU. There are many others that can be expanded with sliceCARD modules. Mentioned eXporerKIT alone is not cheap by evaluation board standards (106 GBP), but include 15 GBP xTAG debugger, Gigabit Ethernet and USB 2.0 port. What is really missing is something that can be found on sliceCARD: LCD controller and SDRAM needed for frame buffering. For that one should go with even more expensive board, and I need everything multiplied by 2. At the end if we decided to continue working with it, I need do make my own PCB as part of DIB project. Therefore I decided to make my own evaluation board following schematics of existing boards and do that for the very first time on the 4-layer PCB.
What I made so far is from today available on the Github here, and I'll try to explain my intentions and design in more posts that follows (before I continue to start talking about DIB again).

[prasimix]

Here is the first proposal of pin mappings for backplane that I'd like to use to connect more modules within a single chassis.

The foundation of everything is a digital control based on the principle of a single processor (master) module that would take care of more peripheral (slave) modules. Although a good specification should be "architectural agnostic" this one is, as you will see, inspired or suggestive of using a XMOS parallel processing MCU, but that does not mean it could not be usable with any other MCU family.

Power lines
3.3 V is used as the main power supply for peripheral modules. It is also backed up with  +3.3VAUX and Vbat outputs that should be used for standby mode. The additional power supplies are +5V, +12V and -12V. In addition, several pins for GND are provided. There is currently no PE (Protective Earth) that may need to be added to avoid providing additional wiring out of the modules if necessary. A possible advantage of using the separate wire is higher current rating as perhaps better/easier arrangement of the star structure.
The power supply section can be placed on the processor module itself or brought from the outside.

System signals
It is envisaged to bring a "master clock" signal of 100 MHz to all peripheral modules. Because of the speed, it might be mandatory to be provided over differential lines. For this reason, I predicted two pins for it: MASTER_CLK+ and MASTER_CLK-. It should not be confused with the SPI clock signal, since it is provided through SCLK1 to SCLK7 pins.
#RESET is the next important system signal generated by a reset generator/power supervisor on processor module #0.
Optional, or maybe better to put them as mandatory from the beginning, are five signals for JTAG debugging (TDI, TDO, TCK, TMS, #TRST).

Communication between modules
The basic communication between processor and peripheral modules is SPI where for each module a separate SPI channel is dedicated to enable parallel operation, assuming that the CPU/MCU is capable of doing something like this, as in case of the mentioned XMOS xCORE. If other MCU is to be used then it will be necessary on the processor module to merge multiple SPI channels to what the MCU (at least on the physical level or pins) offers separately. Plenty of MCUs have several SPI channels, the other thing is that there is still just one processor core that have to deal sequentially and not in parallel with the connected peripherals.
SPI channels have three signals SCLK, MOSI and MISO. The question remains what about CS (Chip Select) which is also needed if we have more than one SPI device on the same SPI channel. For this purpose, pins #CS1 to #CS7 are foreseen. Now, this would mean that it is possible to have only one SPI device per module. However, with simple trick it become possible to address x peripherals. We have to deploy a SIPO register such as '595 for addressing up to 8 peripherals, of more if they are daisy-chained (16, 24, etc). Of course, the greater the number of SPI peripherals on the module, the access time could be slower. However, since the '595 has access speed that largely exceed the usual SPI speeds, it would in this way greatly or wholly neutralize the delay of such addressing. E.g. if the SPI device with max. speed of 4 MHz is used, and there are up to 8 of them, if the '595 is accessed with more than 30 MHz first, and then proceeds with SPI communication at 4 MHz, the end result should be the same as if '595 is not present at all. Furthermore, I assumed that we could have a very fast SPI devices on the first two slots (closest to the MCU!) and want them to access with, say 20 MHz. In that case, the '595 signaling could also go over 100 MHz. For this reason it is envisaged that CS1 and CS2 can be differential, so we have #CS1 Diff+, #CS1 Diff-, CS2 Diff+ and #CS2 Diff-.

Additionally, for slower functions, I2C can be used, which is shared between all peripheral modules (slot #1 to #7).

Finally, if using xCORE, they allow multiple MCUs to be interconnected for e.g. more processing power and I/O lines. This means that main processor resources are no longer necessary resides on slot #0 only, but may be on other slots, too. For such a connection, a 2-wire xCONNECT bus is used, which is slower than a 5-wire variant but consumes fewer pins on both MCUs and connectors (4 instead of 10, or 8 instead of 20 if differential). xCONNECT would be theoretically (and probably practically as advertised by XMOS) used for fast serial communication that should be at least 10 times faster than the usual SPI.
The processor module has an xCONNECT channel that can be used to connect to the remaining modules, but also to connect to another chassis that will have its own processor module and associated peripheral modules.

Triggers and analogue switches
For the purpose of synchronizing activities between multiple peripheral modules, trigger inputs and outputs are used. Star topology is implemented, i.e. all inputs and outputs are terminated at one end on the processor module. The MCU is therefore in charge of waiting for the trigger (on inputs) and sending the trigger to one or more modules. In the case of a xCORE MCU with a predictable resolution of less than 10 ns, a precise synchronization of multiple modules can be expected.

In addition, I took from the VXI specification the use of cross-point switching that allows the redirection of (analog) signals from x to y module. Depending on the speed required, there are chips that do this for decent price in range from 45 (MT8808) to 300 MHz (ADG2188).

______________________________________________________

Below (and in attached PDF) is shown the pin mappings of the #0 connector that has 96 pins (3x32) and other 48-pin (3x16) connectors/slots. The connector suggestion is at the bottom of the text.

As usual, your comments and suggestions are very welcome .

[awallin]

Looking at your drawing there are two connectors on a 114 mm high board, that's 57mm or less of the moduble PCB towards the 'front-panel' if I understand correctly.
For something like a PID-controller you want multiple coax-connectors on the front panel (set-point, sensor in, actuator out, monitor out, etc).
For example normal 90-deg BNC connectors are ~16mm wide and can be spaced ~16mm apart, so there would be room for only 3 BNC's on a 57mm edge.

Did you consider 100mm high 'eurocard' cards that would also fit a standard 3U rack/subrack format?

Given the EEZ naming of your bus - are the EEZ powersupply parts in the bus format?   

[prasimix]

Almost all of the previous drawings are now obsolete. I'm going to make another backplane proposal and dimensions related to the chassis/subrack that can carry a) up to 4 (MCU + 3 modules) or b) up 8 modules that is max. for one chassis. Yes, 3U height is planned and min. width for "slave" modules will be a little bit larger: 7 HP (35.56 mm) or 8 HP (40.64 mm). Width of MCU module could be larger to carry e.g. 3.5" or 4.3" TFT display + other stuff (encoders, trigger I/Os, etc.). Therefore 100 mm high card is planned but not necessarily in any of "Eurocard" formats or at least I'm not going to put DIN connector on the rear but on the bottom (like cards in a PC).

I'd like to make power supply module as first example that use this bus. But this time such module will carry end-to-end functionality, i.e. AC/DC SMPS pre-regulation and linear post-regulation. Therefore 3rd party AC/DC adapter (currently it is Mean Well) and DC-DC SMPS pre-regulator should go out of the picture.

[jbb]

I like the thought of an accessible, (largely) open-source instrument spec, and I'm glad to see someone having a go. I believe that the engineering school at EPFL (Lausanne, Switzerland) did something with ARM host processor and USB rack backplane, but can't seem to find any info on it (maybe died of neglect when critical students finished their PhDs).

Here are some thoughts:

• I expect DIYers like DIYing. So it's important that people feel that they get some benefit from the spec rather than just building whatever.
• People will be tempted to skip 'difficult' or 'annoying' bits of the spec. There may need to be an 'official' test scheme to say that yes, such-and-such is sufficiently compatible (note: this could be a self-test and the developer fills in a report card. No need to spend mega-bucks at a lab.). Writing such test procedures is, unfortunately, difficult and time-consuming.
• Sorting out a connection or racking system is really important.  If it's a racking system, the not-at-all simple act of saying 'here are the card dimensions, here's a comprehensive parts list, you can buy them from vendors X Y and Z' is critical.  There are probably some good Aliexpress options, but someone needs to research, acquire and check out some samples.
• Great example projects would be important as a springboard for widespread adoption.

On the electronics/communications side, I think that:

• Having some kind of card ID system (e.g. using I2C) with well-defined data structures is a good idea.  That way the master can either a) use the right driver or b) complain.  Could also store calibration info.
• Facilities for isolation should be available (e.g. 5V/12V DC supply sufficient to run isolator systems) and easy for module designers to use.
• Low bandwidth modules (e.g. DC power supply) don't need much bandwidth, so they should be able to use a cheap connection. (We don't want to pay for an FPGA in a basic DC supply...)
• Medium/high bandwidth modules (e.g. analog capture or DAC) need more bandwidth (and probably clock + trigger distribution), so they will need a more capable connection (and there people might be willing to pay for an FPGA).
• The module designer should be able to choose what goes into their module, so the comms should be fairly generic and vendor-independent.

I would suggest a tiered comms system, where the designer picks the appropriate level:

• Card ID (e.g. I2C or SPI bus) + basic low speed comms
• Clock and trigger distribution
• High bandwidth

I'd like to make power supply module as first example that use this bus. But this time such module will carry end-to-end functionality, i.e. AC/DC SMPS pre-regulation and linear post-regulation. Therefore 3rd party AC/DC adapter (currently it is Mean Well) and DC-DC SMPS pre-regulator should go out of the picture.

Err, I love power electronics, but I'd be a bit nervous about promoting built-it-yourself power supplies with large quantities of 380V DC inside (after PFC stage).  If you look through my post history I'm the often saying 'safety safety safety!' to people, 'cause I don't want anyone to get killed. Maybe part of the spec should include an 'external' type connection so the scary power stuff can be put in a separate enclosure?

[jbb]

Also, I'd like to be involved :-)

[prasimix]

I like the thought of an accessible, (largely) open-source instrument spec, and I'm glad to see someone having a go. I believe that the engineering school at EPFL (Lausanne, Switzerland) did something with ARM host processor and USB rack backplane, but can't seem to find any info on it (maybe died of neglect when critical students finished their PhDs).

If you talk about Easy ? it is still around.

Here are some thoughts:

• I expect DIYers like DIYing. So it's important that people feel that they get some benefit from the spec rather than just building whatever.
• People will be tempted to skip 'difficult' or 'annoying' bits of the spec. There may need to be an 'official' test scheme to say that yes, such-and-such is sufficiently compatible (note: this could be a self-test and the developer fills in a report card. No need to spend mega-bucks at a lab.). Writing such test procedures is, unfortunately, difficult and time-consuming.

You're right. I believe that a sort of test procedures will come out of our development and be published as all other stuff.

• Sorting out a connection or racking system is really important.  If it's a racking system, the not-at-all simple act of saying 'here are the card dimensions, here's a comprehensive parts list, you can buy them from vendors X Y and Z' is critical.  There are probably some good Aliexpress options, but someone needs to research, acquire and check out some samples.
• Great example projects would be important as a springboard for widespread adoption.

I spent some (but possibly not enough) time to find out what kind of connectors are available with good price/performance. It seems that DIN 41612 are very attractive with price around 2 USD, good pin density and mechanical strength. All your suggestions are highly welcomed.

On the electronics/communications side, I think that:

• Having some kind of card ID system (e.g. using I2C) with well-defined data structures is a good idea.  That way the master can either a) use the right driver or b) complain.  Could also store calibration info.

Ok, I can see here two approach: a very simple reading out 8-bit register that has hard wired ID number (e.g. up to 254 devices, if 00 and FF are reserved/forbidden) and another with on-board cheap EEPROM (started today about 0.25 USD) where ID number and some other data (like calibration that you mentioned below) can be stored. Issue with later is how to program it conveniently for the first time.

• Facilities for isolation should be available (e.g. 5V/12V DC supply sufficient to run isolator systems) and easy for module designers to use.

I think that specified power lines can remain and we can introduce another one e.g. 12, 15 or 24 V that can be used for supplying module isolated power stage.

• Low bandwidth modules (e.g. DC power supply) don't need much bandwidth, so they should be able to use a cheap connection. (We don't want to pay for an FPGA in a basic DC supply...)
• Medium/high bandwidth modules (e.g. analog capture or DAC) need more bandwidth (and probably clock + trigger distribution), so they will need a more capable connection (and there people might be willing to pay for an FPGA).
• The module designer should be able to choose what goes into their module, so the comms should be fairly generic and vendor-independent.

At the beginning I was thinking about "mandatory" and "auxiliary" connectors. Now, I tried to put as much as possible on mandatory/primary connector to cover work with different modules/functionalities. For extra ("FPGA-class") stuff maybe we should introduce over the time another connector what is actually viable to do when backplane/connectors are placed at the chassis bottom not rear side.

I would suggest a tiered comms system, where the designer picks the appropriate level:

• Card ID (e.g. I2C or SPI bus) + basic low speed comms
• Clock and trigger distribution
• High bandwidth

Yes, something like that.

I'd like to make power supply module as first example that use this bus. But this time such module will carry end-to-end functionality, i.e. AC/DC SMPS pre-regulation and linear post-regulation. Therefore 3rd party AC/DC adapter (currently it is Mean Well) and DC-DC SMPS pre-regulator should go out of the picture.

Err, I love power electronics, but I'd be a bit nervous about promoting built-it-yourself power supplies with large quantities of 380V DC inside (after PFC stage).  If you look through my post history I'm the often saying 'safety safety safety!' to people, 'cause I don't want anyone to get killed. Maybe part of the spec should include an 'external' type connection so the scary power stuff can be put in a separate enclosure?

I'm not going to promote anything in that area that is not heavily tested on my side and didn't review from someone that is an expert in the field just because of safety: my safety in the first place and then safety of anyone alse. That was the reason why EEZ H24005 power supply comes with Mean Well AC/DC adapters. If I succeed in making a robust and safe AC/DC section, great, I'll share it with others, otherwise I'll keep my mouth shut .

[prasimix]

Also, I'd like to be involved :-)

Actually you already are, keep posting your comments and suggestions  .

[awallin]

If you talk about Easy ? it is still around.

this one might be worth a look also:
https://openhpsdr.org/atlas.php

[void_error]

Communication between modules
The basic communication between processor and peripheral modules is SPI where for each module a separate SPI channel is dedicated to enable parallel operation, assuming that the CPU/MCU is capable of doing something like this, as in case of the mentioned XMOS xCORE. If other MCU is to be used then it will be necessary on the processor module to merge multiple SPI channels to what the MCU (at least on the physical level or pins) offers separately. Plenty of MCUs have several SPI channels, the other thing is that there is still just one processor core that have to deal sequentially and not in parallel with the connected peripherals.
SPI channels have three signals SCLK, MOSI and MISO. The question remains what about CS (Chip Select) which is also needed if we have more than one SPI device on the same SPI channel. For this purpose, pins #CS1 to #CS7 are foreseen. Now, this would mean that it is possible to have only one SPI device per module. However, with simple trick it become possible to address x peripherals. We have to deploy a SIPO register such as '595 for addressing up to 8 peripherals, of more if they are daisy-chained (16, 24, etc). Of course, the greater the number of SPI peripherals on the module, the access time could be slower. However, since the '595 has access speed that largely exceed the usual SPI speeds, it would in this way greatly or wholly neutralize the delay of such addressing. E.g. if the SPI device with max. speed of 4 MHz is used, and there are up to 8 of them, if the '595 is accessed with more than 30 MHz first, and then proceeds with SPI communication at 4 MHz, the end result should be the same as if '595 is not present at all. Furthermore, I assumed that we could have a very fast SPI devices on the first two slots (closest to the MCU!) and want them to access with, say 20 MHz. In that case, the '595 signaling could also go over 100 MHz. For this reason it is envisaged that CS1 and CS2 can be differential, so we have #CS1 Diff+, #CS1 Diff-, CS2 Diff+ and #CS2 Diff-.

Do you mean something like this? By using line decoders you can even ditch the 595s since you're looking at 7 #CS pins anyway since there shouldn't be more than one Chip Select line pulled low on the bus at any given time anyway. Also, why the odd number (7), no pun intended? In the schematic section below I found it a lot simpler to use a 3-to-8 decoder instead of another 595 and it also comes with inverted outputs. It's all down to how many SPI devices you'll have on the bus.

[prasimix]

Maybe using CS (Chip select) label is confusing. It's module (or card) select in the first place. Therefore only one CS line reach each of 7 modules, and if you have more then one SPI device on one module you have to "decode" their chip selects. A star-topology is used (1-to-many) to connect MCU with each module, and due to that peripheral modules needs smaller connector then processor module. In case of xCORE you can simultaneously (in parallel) communicate with all seven boards via dedicated SPI channels at once!

Why number seven? Because it's 8 minus 1 (processor board on slot #0) .

[jbb]

I spent some (but possibly not enough) time to find out what kind of connectors are available with good price/performance. It seems that DIN 41612 are very attractive with price around 2 USD, good pin density and mechanical strength. All your suggestions are highly welcomed.

Yeah, connectors are hard. Do they have some pins which are a bit longer than others and engage early?

Ok, I can see here two approach: a very simple reading out 8-bit register that has hard wired ID number (e.g. up to 254 devices, if 00 and FF are reserved/forbidden) and another with on-board cheap EEPROM (started today about 0.25 USD) where ID number and some other data (like calibration that you mentioned below) can be stored. Issue with later is how to program it conveniently for the first time.

I was thinking some of the connector pins could be used to encode the backplane slot #, and feed them to address lines on the I2C EEPROM.  The EEPROM can be programmed by plugging it into the backplane and using some 'special' master commands (maybe with a removable jumper for write-protection.  Reference design material should include the appropriate (low cost but reliable) EEPROM, addressing scheme, scripts (Python?) for generating configuration descriptor etc.

I think that specified power lines can remain and we can introduce another one e.g. 12, 15 or 24 V that can be used for supplying module isolated power stage.

Yes, that's probably good.  Final voltage TBA. I would suggest a separate power ground return as well to keep power return currents out of the signalling.

At the beginning I was thinking about "mandatory" and "auxiliary" connectors. Now, I tried to put as much as possible on mandatory/primary connector to cover work with different modules/functionalities. For extra ("FPGA-class") stuff maybe we should introduce over the time another connector what is actually viable to do when backplane/connectors are placed at the chassis bottom not rear side.

Mandatory + auxiliary (or 'Core' + 'Expansion' or whatever) is a good concept.  Some people looking at this will be highly cost sensitive.

Also, I suggest starting with the Mandatory part, because you will likely at some point say 'this isn't quite right, I need to make some big changes.'  That will probably break some compatibility, and is one of the costs of good engineering. You have to be willing to do a big revision to capture lessons learnt, or you end up with the Arduino pin header offsets.

Arduino probably should have said, 'oh, this is a mistake, we'll change the spec to make sense and redo a few shield designs,' but instead they kept going and now all Arduinos have a funny pin spacing.  When people make items like this, the specification is flawed.

Maybe using CS (Chip select) label is confusing. It's module (or card) select in the first place. Therefore only one CS line reach each of 7 modules, and if you have more then one SPI device on one module you have to "decode" their chip selects. A star-topology is used (1-to-many) to connect MCU with each module, and due to that peripheral modules needs smaller connector then processor module. In case of xCORE you can simultaneously (in parallel) communicate with all seven boards via dedicated SPI channels at once!

Why number seven? Because it's 8 minus 1 (processor board on slot #0) .

Maybe the master should spit out 7 Module Select signals (one per module) (or even 7 separate SPI busses) and a few Chip Select signals (maybe 4??, could be shared across all modules). That way some simple cards (e.g. relay output channel / basic ADC / basic DAC) could be built with SPI chips and no micro.

Or maybe a cheap & cheerful bunch-of-UARTs would be the solution? That way developers wouldn't have to make SPI slave firmware, which can be a bit tricky. (But then you would almost always need a micro.)

[prasimix]

Yeah, connectors are hard. Do they have some pins which are a bit longer than others and engage early?

Not ones that I have/suggested. What do you have in mind a hot-swap capability?

I was thinking some of the connector pins could be used to encode the backplane slot #, and feed them to address lines on the I2C EEPROM.  The EEPROM can be programmed by plugging it into the backplane and using some 'special' master commands (maybe with a removable jumper for write-protection.  Reference design material should include the appropriate (low cost but reliable) EEPROM, addressing scheme, scripts (Python?) for generating configuration descriptor etc.

That sounds good.

I would suggest a separate power ground return as well to keep power return currents out of the signalling.

Ok, make sense but I didn't find so far such separation in other specifications (i.e. VXI). Maybe they compensate that with few dozen of evenly placed GND pins?

Mandatory + auxiliary (or 'Core' + 'Expansion' or whatever) is a good concept.  Some people looking at this will be highly cost sensitive.

Also, I suggest starting with the Mandatory part, because you will likely at some point say 'this isn't quite right, I need to make some big changes.'  That will probably break some compatibility, and is one of the costs of good engineering. You have to be willing to do a big revision to capture lessons learnt, or you end up with the Arduino pin header offsets.

Arduino probably should have said, 'oh, this is a mistake, we'll change the spec to make sense and redo a few shield designs,' but instead they kept going and now all Arduinos have a funny pin spacing.  When people make items like this, the specification is flawed.

I believe that mandatory/core specification and connector has to be wisely defined that we don't have a need to change anything there in the future. Further modifications and enhancements should be done by introducing "Expansion/AUX" connector. There is a good chance that we do not come even closer to Arduino situation .

Maybe using CS (Chip select) label is confusing. It's module (or card) select in the first place. Therefore only one CS line reach each of 7 modules, and if you have more then one SPI device on one module you have to "decode" their chip selects. A star-topology is used (1-to-many) to connect MCU with each module, and due to that peripheral modules needs smaller connector then processor module. In case of xCORE you can simultaneously (in parallel) communicate with all seven boards via dedicated SPI channels at once!

Why number seven? Because it's 8 minus 1 (processor board on slot #0) .

Maybe the master should spit out 7 Module Select signals (one per module) (or even 7 separate SPI busses) and a few Chip Select signals (maybe 4??, could be shared across all modules). That way some simple cards (e.g. relay output channel / basic ADC / basic DAC) could be built with SPI chips and no micro.

Or maybe a cheap & cheerful bunch-of-UARTs would be the solution? That way developers wouldn't have to make SPI slave firmware, which can be a bit tricky. (But then you would almost always need a micro.)

I draw a simple schematic that should help you to visualize proposed pinouts (only three peripheral modules are shown for the sake of simplicity):



You can see above that signaling that comes to each modules is almost completely dedicated to that module, i.e. a star-topology is proposed driven by idea that on the processor module (#0) a multicore CPU/MCU can be employed for parallel processing. That could be XMOS xCORE, Parallax Propeller, a bunch of single-core MCUs or multi-core CPU/SoC. If multi-core/parallel processing resources are not available, designer has to merge on the processor module SPI buses into one or more (depending of MCU/CPU/SoC pinout). There is few shared signals only (i.e. RESET, MASTER_CLOCK, I2C, and non-isolated power lines). Finally only two buses are daisy-chained and not mandatory: JTAG and xCONNECT (or that could be something else if xCORE is not used).

I don't see a way how to efficiently share more CS lines between more modules. Please keep in mind that some of modules should be completely isolated (e.g. power supply, electronic load, etc.) that sharing could make even more difficult (if possible at all).
With current proposal, if signaling isolation is needed for min. functionality one have to implement only one 4-in/2-out digital isolator (or even 4-in/1-out if IRQ is not needed), and has to derive powering from proposed "hi-voltage" source that could be +24 V (or TBA).

[jbb]

Do they [connectors] have some pins which are a bit longer than others and engage early?

Not ones that I have/suggested. What do you have in mind a hot-swap capability?

Good reply! I hadn't really been thinking of hot swap, so there's not much point.  (Maybe a look at options with early mating connectors are in order so that those pins can be ground from the get go and the door to hot swap isn't closed.)

I was thinking some of the connector pins could be used to encode the backplane slot #, and feed them to address lines on the I2C EEPROM.  ...

That sounds good.

You'll need to reserve a couple of pins for card address, then.

I would suggest a separate power ground return as well to keep power return currents out of the signalling.

Ok, make sense but I didn't find so far such separation in other specifications (i.e. VXI). Maybe they compensate that with few dozen of evenly placed GND pins?

Perhaps overkill then - just dedicate some connector pins for power return and use a single big ground plane on the backplane?  (I was thinking that the totals may go up to some amps).

The reason for having lots of ground pins is usually signal integrity.  In fact, I think that your proposed pinouts may need more ground pins.  What kind of data rates do you expect on the SPI, and how fast do you expect to drive the STAR system (i.e. will you use high speed buffers)?

I believe that mandatory/core specification and connector has to be wisely defined that we don't have a need to change anything there in the future. Further modifications and enhancements should be done by introducing "Expansion/AUX" connector. There is a good chance that we do not come even closer to Arduino situation .

Obviously you'll try to get everything right the first time, but the simple act of sticking 'Version 0.1, Provisional' on all the reference material leaves the option open to make a big change if something serious becomes apparent.

(Possible AUX connector: DIN41612 power connection)

I draw a simple schematic that should help you to visualize proposed pinouts ... signaling ... is almost completely dedicated to that module, i.e. a star-topology is proposed.

The star-based architecture (with some MUXing on the master) could be very effective. Some care will be required to manage signal integrity (I'm thinking about stubs and reflections and multiplexers...)

Do you think that the XCONNECT and JTAG connections are a good idea?  I've got four concerns:

• XCONNECT is for XCORE only, and a lot of module implementers will use something else (AVR ... PIC ... FIGHT!)
• How often will an end user use JTAG to connect to a module?  I would think that this is more for module developers, who presumably can just put their own JTAG header on the module and connect to it directly.
• Some JTAG devices (MSP430 I think, and others...) don't play well when daisy-chained for anything other than boundary scan duties.  So the JTAG may mysteriously stop working when some other module is swapped.

[/li][/list]A JTAG diasy chain would require completing links, i.e. dummy modules, to complete the connection.[/li]
[/list]

I don't see a way how to efficiently share more CS lines between more modules.

It's not easy, no.  There are limits on pin count, after all.  However, I would like to pose the following question: if I have two SPI chips on a module (e.g. an ADC and a relay driver), how do I address them?

Please keep in mind that some of modules should be completely isolated (e.g. power supply, electronic load, etc.) that sharing could make even more difficult (if possible at all).
With current proposal, if signaling isolation is needed for min. functionality one have to implement only one 4-in/2-out digital isolator (or even 4-in/1-out if IRQ is not needed), and has to derive powering from proposed "hi-voltage" source that could be +24 V (or TBA).

Yeah, isolation makes things more difficult.  From experience, I can say that sometimes you just have to drop a whole lot of channels.  Alternatively, you can deploy something like a UART (1 in / 1 out) but that has challenges too, especially with timing.

Once again (and I can see you're doing it carefully), it's important that the module designer gets choices about what features they use and how much they pay.

[jbb]

I've been thinking about this more (please let me know if it gets annoying)...

Star topology

• I like the concept, because it means that module designers don't have to muck about with weird and wonderful comms stuff, time slots etc.
• It does use up a few pins on the master
• In order to allow for a hypothetical 'cheap' master, it should be possible to connect all the SPI buses on the master card to a smaller number of SPI masters. So modules should release the MISO line when they aren't selected.
• Given there are SPI connections to each module, maybe the module ID system should be an SPI EEPROM?  That way I2C lines and slot address pins aren't required. (This would involve adding a couple of addressing lines to each module.) Modules with a microcontroller could even emulate the SPI EEPROM if someone is desperate to save some cents.

Synchronisation
Is a great idea.

• I saw a 100MHz clock distribution pencilled in - were you thinking this should come from the master only, or should it allow for the choice a more accurate timing module?
• Isn't the usual clock distribution frequency of choice 10 MHz?  A lot of cheap micros can't handle 100 MHz.
• A synch signal which strobes at a lower rate is also very helpful.
• The synch signal could be sourced from the master clock (fixed period strobe), or maybe it could be a bidirectional pin, allowing a (suitably configured) slave to trigger everything.

Triggers and higher speed signalling

• I think a lot of applications won't need hardware triggering or high speed communications. Maybe these should be an optional part of the spec?
• If the STAR bus is designed for high speed signalling, maybe it could be dual function: either advanced triggering or high speed comms (imaging 100 MHz DDR: 200 Mbit/s is quite a lot of bandwidth...)

[prasimix]

Hi jbb, of course that your posts are not annoying at all, please keep posting. I also believe that more people are going to join us soon to have better idea how useful is what we are talking about. I'll start with another schematic to clarify "chip select" demux/decoding for multiple SPI device on the same module. In example below I also added min. digital isolator what should be added if module has to be isolated/"floating" what will be for sure a case for my first module (power supply):



A SPI EEPROM is also added, that can be used for storing info about module/card ID and some other useful data (settings, calibration data, working hours, last state, etc.). That is in line with what you already realized that instead of I2C a SPI EEPROM could be more convenient. That is especially a case if isolation is needed when we can spare one isolator (for I2C).
As you can see in this configuration only one Card/chip select (CS#N) is needed for each module regardless of number of deployed SPI devices. To keep access time constant with adding/addressing more on-board devices you have to increase the SCLK frequency for sending data (MISO to SER) to SIPO register.
Of course, firmware has to take care of sending data that include only one 0 (e.g. 0111 1111 to select EEPROM, 1011 1111 to select CS1, etc.). A more "firmware bug-proof" solution could be to deploy 1-to-8 demux or sequencer instead of '595 (something like '138 but with only "CLK" input). I didn't spent any time to find one, and I'm not sure that cheap one exists either, so your input is welcome here.

Grounding
Currently we have 11 GND on #0 connector and 4 on peripherals. GND pins are "strategically" placed close to SPI channels and some other places. Don't know if that is enough, but situation with peripheral connectors with only 4 GND pins is possibly not good. That can be improved with using two currently unassigned pins (B5, B6) and with removing JTAG and/or xCONNECT from "mandatory" connector specification.

SPI bus speed
The max. SPI bus really depends of what we are going to exchange between #0 and other modules. I think that we need sooner or later to identify few use case that can help us to set a boundaries. I'm not going to mention in this post any use case because they need more prominent place/presentation (that could be a separate doc file available for review).

AUX connector

(Possible AUX connector: DIN41612 power connection)

Do you have anything in particular, since DIN41612 include dozen of power connector?

xCONNECT and JTAG
Your comments about such signaling are right and I can agree that they are not deserve to occupy pins on the primary/mandatory connector. That is true especially for JTAG (I've borrowed it from the PCI Express, but that is a quite different type of beast). By removing them we have more pins for GND and possibly some other stuff (e.g. promoting some signaling to differential due to expected higher bandwidth, etc.).

Synchronization, triggers, hi-speed signaling, UART...

I saw a 100MHz clock distribution pencilled in - were you thinking this should come from the master only, or should it allow for the choice a more accurate timing module?
Isn't the usual clock distribution frequency of choice 10 MHz?  A lot of cheap micros can't handle 100 MHz.

That is a good point. Even VXI has both 100 and 10 MHz clocks. In our case 10 MHz sounds more reasonable. Therefore I'm going to make that change. 100 MHz could find its place later on the AUX connector.

A synch signal which strobes at a lower rate is also very helpful.
The synch signal could be sourced from the master clock (fixed period strobe), or maybe it could be a bidirectional pin, allowing a (suitably configured) slave to trigger everything.

I'm not sure how to provide bidirectional master clock, but for slave triggering STRGIN#N and also STARX#N, STARY#N cross-switch matrix could be used.

Yeah, isolation makes things more difficult.  From experience, I can say that sometimes you just have to drop a whole lot of channels. Alternatively, you can deploy something like a UART (1 in / 1 out) but that has challenges too, especially with timing.

UART imply using a MCU on peripheral module too. In my view that shouldn't be a case/requirement. Basic configuration should imply using CPU/MCU on the module #0 (i.e. master) only. Also if we are going to assign to pins per module that will increase demand for pins on #0 too (up to 2 x 7 pins!). I presume here that no shared UART topology is possible. Eventually, we can offer existing MOSI#N, MISO#N to be used for UART communication with selected modules, but in that case we cannot use SPI EEPROM as default mean of module identification .


I hope that everything is addressed from your latest two posts. Please let me know if something require more elaboration.

[cat87]

    Now, this thread is a nice surprise. I'm planning on making my own modular case which will take 100 x 160mm  cards, connected with the DIN4162 connectors. Because I don't want to have digital lines in my backplane, because they're a pain to route properly, I planned on having the SPI or I2C or what have you communications interface coming out from the front of the module. Kinds half-assed, but doable.
 
But now, looking at this thread, I think I might give the backplane a once-over and try to incorporate some comm.  interfaces.


Thanks guys

Oh, and I've got the frame of the case almost complete. I'll be picking up some card  guides tomorrow and maybe post some pics of the case one of these days.

[xani]

One little question, is there are reason for 3.3VAUX to be 3.3V? Using 5V there would allow for direct use of ATX power supply (at least to bootstrap it, as they have 5 volt standby voltage), at least for testing as providing  +3.3, +5V, +12V and  -12V is at least 2 bench power supplies worth of power rails and that's a bit of a pain for testing.

[prasimix]

One little question, is there are reason for 3.3VAUX to be 3.3V? Using 5V there would allow for direct use of ATX power supply (at least to bootstrap it, as they have 5 volt standby voltage), at least for testing as providing  +3.3, +5V, +12V and  -12V is at least 2 bench power supplies worth of power rails and that's a bit of a pain for testing.

No it's not. I was guided here with the idea that nowadays +3.3 V is pretty standard and choose it as a main power. Consequently AUX has the same voltage. But, your remark about ATX is valuable. Also since 5 V is easier (and less noisier) to step-down then other way around we can change AUX to be 5 V.

[jeremy]

I have an Agilent smu with a backplane connector, and the pinout is basically just power, USB and trigger/sync.

I throw my vote in with the earlier SCPI/vxi11 folks. Why reinvent the wheel  works fine for keysight/tek/etc

Just to add my 2c 

[prasimix]

The main reason for such (modular) approach is to save, reuse and use, to the greatest extent what is already done or spent (in time and money). My stance is that is exactly the quite opposite from hardcore consumerism (fulled by debt "money") which is a main mantra for so-called "sustainable development" (that is in fact sustainable domination!) and publicly listed corporations as that what you're mentioned cannot afford a luxury to ignore that. That is a default/"winners" game, and it's presented so far as the best what humanity can get (and buy!) and consequently deserve. A "litmus test" for such worldview among other is mentioning of open source . It's usually trigger a bunch of rationalizations (and attacks).

[jeremy]

Right, but scpi/vxi11 is free, and USB/Ethernet chips are extremely cheap and widely available. I don’t think keysight uses these protocols because they are trying to lock you in, the opposite actually. My current “backplane” on my desk is just a 7 port USB hub, and it works great for connecting PSUs, multimeters, oscilloscopes, motion controllers, etc. The only issue is that cabling is a mess. I write python code to control them all, and I certainly spend the most time reading the manuals to understand all of the functionality available rather than fiddling with driver installs.

Keysight even sells a big fancy USB hub mainframe: https://www.keysight.com/en/pc-1890327/usb-mainframes-controllers?nid=-33269.0&cc=AU&lc=eng. You can use the modules inside the mainframe, or outside using just a USB cable.

[prasimix]

Ok, but USB as a "backplane"/bus imply a sort of "intelligence" (CPU/MCU/SoC) on both side. Yes, USB-to-SPI/I2C bridges exists and that was one of my earlier considerations but I gave up from it. A nice detail on mentioned Keysight solution is, if I understood picture correctly, a possibility to use the same module as part of the rack or as stand-alone (possibly USB powered) solution. That's something that I had in mind, too.

EDIT: I forgot to comment SCPI: that should be a "basic requirement" and my first project support SCPI pretty well, too.