GeniX - an I2C protocoll protocoll?

[Informatic0re]

Hello, world outside of my Home Office!

A non-technical friend recently approached me with a simple yet profound question: why can't assembling sensors be as straightforward as building with Lego bricks? This got me thinking: what exactly is needed to achieve such simplicity? We have a plethora of sensors, microcontrollers (MCUs), CPUs, communication protocols like WiFi, LTE, and BLE, but it's the sensor aspect that truly intrigued me. Why isn't there a standardized, plug-and-play solution for sensors akin to Lego blocks? I was certain someone must have pondered this before and devised a solution, but my search came up empty-handed.

Perhaps such solutions exist within closed systems. Nevertheless, I couldn't shake the complexity inherent in sensor integration. Sensors utilize various communication protocols like SPI or I2C, some employ analog data, while others use unique methods like one-wire communication. Building a sensor requires not just creating a PCB but also implementing core functionalities such as drivers and communication protocols. Often, this entails deciphering convoluted manufacturer code, which obscures functionalities rather than simplifying them.

I2C and SPI are commendable protocols for sensor communication, so why the inconsistency across sensors? While direct register addressing might explain some of the variation, it still doesn't justify the chaos. After all, what do we expect from a temperature sensor? Temperature data. And from a pressure sensor? Pressure readings. Yet, the available sensors demand calibration, data reading, and interpretation, often involving complex calculations just to obtain the desired value. While I understand the inherent complexities, why must users grapple with them? I simply want to focus on my application and use-case without delving into the intricacies of sensor calibration and data interpretation.

So, I pondered a solution. Although I'm neither an expert electronics engineer nor an expert in any field, I believed there had to be a simpler approach. In essence, I devised a prototype "language" built upon the I2C protocol to facilitate sensor discovery and communication.

Utilizing the I2C protocol, I defined registers for every conceivable action and encapsulated them within a protocol I've named GeniX (Generic inter-integrated Circuit Communication Protocol). I created a prototype to test my idea, and it functions as intended.

For a more comprehensive description, you can visit: https://github.com/open-thngs/genix.

This project is entirely open-source and also WIP, and I'm eager to hear your thoughts. As I primarily work alone, I lack sufficient brainstorming partners. While I've sought feedback from a few individuals, their understanding of the topic was limited, and their responses were lukewarm at best. Therefore, I invite you to scrutinize and critique this idea thoroughly!
I will add more informations and code and PCB Designes and stuff into the repo during the next days if people are interested.

P.S.:  if this text sounds super strange it is because I wrote it in my bad english and throwed it into ChatGPT to make it sound more eloquent 

[tooki]

A non-technical friend recently approached me with a simple yet profound question: why can't assembling sensors be as straightforward as building with Lego bricks? This got me thinking: what exactly is needed to achieve such simplicity? We have a plethora of sensors, microcontrollers (MCUs), CPUs, communication protocols like WiFi, LTE, and BLE, but it's the sensor aspect that truly intrigued me. Why isn't there a standardized, plug-and-play solution for sensors akin to Lego blocks? I was certain someone must have pondered this before and devised a solution, but my search came up empty-handed.

Because sensing is complex, and sensors can be complex.

Sensors differ in their features (hardware interface, protocol, etc) because different sensors exist to meet different needs. A 1wire sensor is ideal for longer distances than I2C; SPI lets you transmit data much faster, allowing more frequent measurements, for example. Industrial sensors typically use interfaces that are far more resistant to interference. Automotive products have to tolerate large temperature ranges and vibration. And so on and so forth.

Complexity doesn’t exist because someone set out to create a barrier for beginners, the complexity exists because it’s needed for some use case, and different types are optimized for different use cases.

I2C and SPI are commendable protocols for sensor communication, so why the inconsistency across sensors? While direct register addressing might explain some of the variation, it still doesn't justify the chaos. After all, what do we expect from a temperature sensor? Temperature data. And from a pressure sensor? Pressure readings. Yet, the available sensors demand calibration, data reading, and interpretation, often involving complex calculations just to obtain the desired value. While I understand the inherent complexities, why must users grapple with them? I simply want to focus on my application and use-case without delving into the intricacies of sensor calibration and data interpretation.

Well, cost might be one reason: keep the “smarts” in the sensor to the minimum, since you already have a microcontroller anyway, which can do those calculations easily. Just have the sensor do the bare minimum, so that it can be cheap and reliable.

You think calibration is something you can just ignore? Those sensors have calibration registers for a reason. What good is your oversimplified sensor protocol if it can’t handle basic functionality like this? Without calibration, the sensor data is useless.

I think you maybe need to design more real-world projects with real-world sensors. You may realize why they have this complexity.

[Siwastaja]

Obviously manufacturers are not going to follow any common protocol, especially one coined by a hobbyist, but I kinda applaud the oversimplified design.

Opposite case: take solar inverters, energy storage battery systems etc. Problem: each manufacturer has their own interface to receive data such as solar production, or apply commands such as "charge battery at 3000W" "discharge battery at 1500W". Enter SunSpec, a committee-driven common standard which, if everyone followed it, would offer perfect interoperability with no custom per-device driver layers.

End result - committee specifying every imaginable complexity, to the point that if you want to receive solar production data, you have to implement six different interfaces implementation each of which is optional for the manufacturers. Yesterday I spent hours trying to figure out how to control the battery power level bidirectionally using SunSpec, and it appears this is just impossible to do, and there is no documentation how it's supposed to work; while the actual manufacturer of the battery system, thank god, offers their own custom interface, over which it's a single register write, understood in 5 minutes. So even though SunSpec has existed for... over a decade? ... it is hard to find any products that actually implement it. And the few that do, are actually not compliant even though they say so, but require understanding quirks (for example, a device reporting "not implemented" for a mandatory field which cannot be "not implemented", when they want to say "not available").

Enter alternative reality: SiwastajaSpec, written ad-hoc here in 2 minutes:
registers 40001-40002: power, in W, int32, positive = to grid, negative = from grid
registers 50001-50002: battery real power setpoint, in W, int32, positive = to grid, negative = from grid
registers 50003-50004: battery reactive power setpoint, in W, int32, positive = to grid, negative = from grid
... and so on. Pretty similar to Informatic0re's short list of 0x01 = temperature in degC, 0x02 = pressure in Pa...

If such thing existed, it would be followed, because implementing it would not be worth a full man-year or two. With no royalties and compliance programs, it would just benefit everyone. Sure, manufacturer specific alternative protocol would still be needed for those use cases for which the generic protocol is too simplified. But at least the generic protocol would be usable. Maybe for 95% of the cases.

Of course, genix has no way to configure gyro sensitivity or acceleration filters or temperature sensor sampling rate or whatever. There are three ways to deal with it:
(1) come up with perfectly generic and elegant set of configuration options
(2) leave room for "manufacturer specific" messages which can ride along on the same bus without conflicts
(3) make it a mode selection, "genix mode" or "manufacturer's custom mode", force users to use the latter when they need more features

1 does not work. 2 is the best. 3 is the safest option.

This is the takeaway: complexity and genericity do not play along at all. When you choose to do an abstracted, generic interface, you are going to throw away flexibility and options, and choosing "good defaults" for the end-users. This is fine, do it with pride. This is why Arduino is successful: you get the simple AnalogWrite() or whatever which gives you crappy 500Hz PWM, but it's simple to understand and it works. One can always choose to bypass the simplicity and use the custom solution.

This is how all Perfectly Elegant Generic Solutions fail: because of genericity, they make simple cases difficult. Because of genericity, they are too expensive to implement. And in case of SunSpec, it is wrong that the inverter manufacturer can choose which of the six information models they implement, but as data user, I have to implement every single one of them, when I only wanted a few simple numbers, the same subset everyone wants.

[Informatic0re]

You think calibration is something you can just ignore?

I do not ignore calibration, of course there is a reason for that and the protocoll should be capable of handling that. As @Siwastaja mentioned there can be a way.

Sensors differ in their features (hardware interface, protocol, etc) because different sensors exist to meet different needs. A 1wire sensor is ideal for longer distances than I2C; SPI lets you transmit data much faster, allowing more frequent measurements, for example. Industrial sensors typically use interfaces that are far more resistant to interference. Automotive products have to tolerate large temperature ranges and vibration. And so on and so forth.

I understand that each sensor was made for a purpose and for a variety of use-cases but often this use-cases overlap in functionality. But why does this matter the communication with a sensor? Why don't they all speak the same language at the end? Why do I need to implement that 1-Wire communication or care about if it is SPI or I2C or maybe simply an analog signal.
Why can't this be abstracted? The differences are valid, like why you would want a 1-Wire communication or SPI but why can't this be abstracted in a way that I don't need to know?

Maybe imagine this like the abstraction layer in software (thats where this is coming from). I write the software in a way that I have a communication interface which defines what I can do for example sending temperature data. Me as a user of this interface I don't care about if this is done via MQTT or HTTP or if this is done via WiFi or LTE.
Of course someone will implement this once but from then, nobody needs to know.

Same with sensors. I need to know WHAT a sensor can do and I am using an interface which abstracts the HOW for me. The Sensor can still be a long distanced sensor with a 1-wire-communication but how it communicates with a MCU is defined by that interface.
I try to apply software abstraction onto the electronics world. Maybe I2C isn't the right choice here but you could use the same "language" on SPI or any other protocoll as long as it is capable of sending bits and bytes somehow.

Cost for sure is a problem because wrapping any sensor into a system with a tiny MCU is more costly but thats happening already more and more and small MCUs are getting less expensive like a ATTiny or whatever. You might reduce the flexibility of a sensor down to only fit the 90% use-cases but you gain a lot. Less time for layouting/designing, less time for programming which saves a lot of money.

@Siwastaja you are right but you also show how such a thing is needed and made an industry standard - it will not be my solution for sure.
I mean of course - if manufacturers do not follow along it makes no sense! Thread an Matter is a good example of this (not yet, but the idea behind it). They created a new netork protocoll which works like WiFi and they but the application language ontop of it (Matter), thats exactly how this should be done.
Lets not talk about how they go ahead with that - but I kind of see the same possibility for electronics.

Having protocolls like I2C, SPi and what not but speaking the same language is the goal. I see that my GeniX solution maximum would become something similar to Arduino where you can plug in sensors and exchange them and there is no need to programm them. But I guess nothing more tho.

[tooki]

Well in the original post you dismiss calibration, and don’t mention it in the documentation, so forgive me for not thinking you considered it important… 

The most you can hope to do is abstract away the communication layer. But that was never where the complexity lay to begin with.

The complexity is in the data you move to and from the sensor. The best you can ever hope for in a generic system is to support some baseline functionality. But for advanced functionality, as a manufacturer you have to support your own protocol anyway, so why bother implementing the hobbled generic protocol?

[HarryDoPECC]

[url][https://xkcd.com/927//url]

[HarryDoPECC]

https://xkcd.com/927/

[ajb]

Utilizing the I2C protocol, I defined registers for every conceivable action and encapsulated them within a protocol I've named GeniX (Generic inter-integrated Circuit Communication Protocol).

I won't give you too hard a time for your word choice here (chatGPT or no, your english is certainly better than my german!), but your table hardly covers "every conceivable action".  Even sensors that measure the same basic phenomenon, different units, scaling factors, numeric representations, etc, can be significant choices in sensor design and selection. 

A self-discovering and self-describing protocol is cool, and has a lot of utility in certain use-cases, but it's really hard to make the argument that it's a net benefit at the level where I2C, 1-wire, SPI, etc are dominant.  It really isn't that complicated to look at a datasheet, implement maybe a handful of reads/writes for initialization, and then a simple single read to get sensor data.  Those can get wrapped up in simple `sensor_init()` and `sensor_read()` type functions (maybe a handful more if needed), and then you never have to think about the communication protocol again.  You could do this dozens or hundreds of times with dozens or hundreds of different sensors, and end up having done less work in total than an equally robust GeniX implementation would require, because many of these sensors and the ways they are used simply aren't that complicated. 

If you have a system where sensors are constantly getting added or removed or exchanged, or you need to add new types of capabilities without requiring software updates, then something like this makes more sense.  But for the most part, these sensors are chosen once during development, implementation specifics (device or register addresses) are abstracted by a couple of #defines or wrapper functions that take a couple of minutes to write, and that's the end of it.  If you need to change to a different sensor, you change a couple of lines of code, and then get back to the rest of the project. 

Of course there are sensors that are much more complex (IMUs come to mind), but then your protocol has to become even more complex to properly generalize and describe their behavior, so you still haven't really tipped the balance.  Remember that the more complex a thing is, the more ways it can be used, and the proper way to abstract what something is depends on how it's used, so the complexity of generalization increases ~exponentially.  Furthermore, if you are generalizing for a higher level of complexity than an application truly needs, or to support future capabilities that no one offers yet, then almost by definition you end putting more poorly- or un-tested software into production.  Even if you fully mock out those capabilities in unit tests of the GeniX implementation, the API is also more complex, and it becomes increasingly difficult to ensure test coverage across combinations of sensor sets and application usecases (again, IMUs come to mind...).  A sensor implementation that does exactly what the application requires and no more will always be easier to write, easier to debug, and more reliable than one that tries to do everything. 

This is all before looking at this from a business perspective.  Say it takes an engineer 8hrs at $100/hr to directly implement an I2C sensor that costs $0.30, or with GeniX it only takes 1hr, but now the sensor costs $1, and potentially requires more processing.  The NRE has decreased by $700, but BOM has increased by $0.70, so if you produce more than 1000 units, GeniX isn't saving you any money.  This cuts in both directions: I mostly design for production in the hundreds, so it literally isn't worth my time to fuss about $1-2 of BOM cost, but companies that produce in the millions might spend thousands on NRE to save a few pennies on BOM. 

So while it's a neat idea, it's hard to see that it really solves a meaningful problem.

[Informatic0re]

@ajb That fits what came into my mind as well when thinking about it. I got the protoype running and the idea is pretty nice but I could not think of a problem which this is solving.
Thats exactly the reason I posted this here because I wanted this kind of input to confirm my thoughts. Thank you very much for this very clean answer. (not like the reply with "yet another standard" )

But in a dream world sensors should "talk" one language to simplify its usage - but I guess that will stay a dream forever.

I think this sentece:

If you have a system where sensors are constantly getting added or removed or exchanged

is pretty strong. Most of the system do not really need a sensor change and if so - there will be a new revision which might be done relatively quickly anyway (if it is just about exchanging a sensor)
The idea actually came from a Data Logging System - more like a product - where someone can say "Hey I need to track temperature, humidity and X in my supply chain" and then you could use such a modular system to easily and quickly just plug a device together like lego. But this would be more like a whole product already for this specific use-case.
I just tried to generalize it and see if it might makes sense for other use-cases as well.

Thank you very much