GUI with gauges in python

[Simon]

Well my first job is to get a basic project going that will allow me to display a CAN bus message on the screen, then I know that I have (hopefully) got the whole tasking thing sorted out.

[paulca]

A little late to the party.  Have you considered a "web ui"?

It's where all the cool kids are these days, snow boarders and such. jk.  Well, it does have easy to use components, can create full "GUI" apps quickly.

In that model you would build the UI in mostly Javascript and HTML5, you can run 90% of things in the browser.

The interface to be backend is, traditionally, HTTP with JSON REST-like.  For "streaming" data you have web sockets or just keep alive connections.

The browser opens you to features like dynamic SVG XML which also support animation, there are a multitude of "widget" libraries for things like guages, including ones where momentum, "springiness" etc can be modelled like "steam punk gauges" and the like.

The interface over REST gives you a solid UI/Backend separation, allowing both to exist.

If you are sending data from an MCU like an ESP32, REST-like HTTP API is easy.  Just try and avoid needing HTTPS and full OAUTH.

I spent 2 years writing a front (and back) end for a monitoring system for ePON head-end "OLT"s for ISPs while in Pace.  I got the fun task of doing the live gauges showing system load and a few other key metrics.  Was a lot of fun.  Out lib at the time only did basic animations, but I extended them to have inertia and acceleration to smooth them out and stop thme just jittering on each update on the websocket.

Sounds like a lot of code, but "Flask" in python will give you a REST API in 2 lines and your handler function.  Creating basic "bootstrap like" web UI is an afternoons tutorial type thing.

UI JS Layer:  Angular, React or higher level wrapped frameworks based on them.
BE REST Layer: Python+Flash or node.js (serverside JS, I know, yuk)
MCU layer: An ESP32 will do HTTP REST-like calls with ease, it should handle websockets for streaming, or the alternative is a serial gateway which takes the MCU requests/data updates as basic serial data and converts them into REST calls.  Python has direct wrappers around C struct and union expansion and packing (hint).

Each is replacable, isolated and have additional benefits which may come in handy.  Like multiple simultaneous clients.  Authentication.  Remote access.  Security.

MCU(*) <-> RaspberryPI(HTTP Server) <-> User's browser <-> Angular application

Here is an example of an SVG JS Library at work:
https://svgjs.dev/docs/3.0/tutorials/
Mores specific to guages.
https://github.com/naikus/svg-gauge


Note....  If you have that HTTP layer, it's a very easy step to send updates to InfluxDB and point Grafana at it.  You can then import big boy grafana charts into your front end.

[Simon]

Yes I has considered doing it as a full screen web page a long time ago but it felt more like a lot of work.

So how exactly would I be doing any logic and running the CAN bus? My understanding of CAN bus handling from what I have read of the standard CAN bus drivers / modules so far is that the ID filtering for different functions is done in the program and that filters in the controller are not used (I actually am considering this on my MCU based implementation of CAN Open).

So there is a fair bit of logic to be dealing with and as soon as that is all sorted out I'll be creating log files too and eventually reports with graphs in PDF. So I need something solid running the logic that the web browser can talk to.

[paulca]

I am not familiar with CAN bus, but I expect it will be some what "message" based?  You sent messages with IDs to invoke something on a device?  The device can also send updates when it chooses?

This is a bi-directional producer/consumer pattern, aka a message bus.

So, an option for your logic is to receive those messages and file them by ID into an in memory map.  So you keep a "snapshot" of the last message for each ID.

canbus_cache[canbus_id] = canbus_payload


The rest request could just be:

GET /canbus/123456789

and the response would be a JSON serialised copy of the canbus message.  Or you could just hard core it and send the binary, although that means importing binary definitions into JS.

To send a message you would use the opposite:

POST/PUT /canbus/123456789
Payload: the can message serialised or not.

On the other end the RPI is receiving the stream of CAN messages and updating the snapshot cache.

If you need "time range capture" then you can make the key a combination of "CAN_ID + TIMESTAMP" and a finite ring buffer to hold N previous records.

For live gauges on a web-ui updating every second or at full data rate, if you dare, can be done with an "Observer pattern".  Basically each "can bus topic" of interest has a handler (the observer) passed to it.  If it gets updated it calls that handler.  That handler emits the data down the existing HTTP keep alive connection or down the web-socket TCP connection.  It retains the minimal delay update to "NRT" near-realtime.

The observer pattern is out of the MVC textbook, (sort of).  Model View Controller.  If your case the "Model" is the map of CAN topics and last-seen payloads.  The Views are anything you want to display that data, they work best with direct observers to the data.  Data updates, view updates.  The controllers are the glue in the middle which takes requests (Chart View) and responds with the correct "View" populated via the "Model".... or given it's own callback observer so it updates all on it's own without the need for a controller.  In practice the "Controller" ends up being the REST API class which takes the requests and orchestrates them.  It gets a bit funky with Angular et. al. as the MVC itself tends to exist in both the Front End angular app and again in the backend REST API component.  Both use a type of MVC although today the backend normally just sends JSON data and it's only view is "JSON View" basically.  So it's just cancelled out.

A queue might be helpful in places.  If you have message flows where you must not drop a message, then a queue is essential and the cache idea above not sufficient and will not do anything special about order or lost messages.  Python's Queue import is perfectly useful. 

Python hint.  Python is "multi-threaded", but single execution locked.  This means you can take short cuts with concurrency as there is NO parallel execution, even on multiprocessor archs.  It should be VERY familiar to you from MCU land as that (ESP32 asides) is the basic model there too.  You can still get race conditions of course, but you get to take short cuts as you can get close enough to atomic operations on variables.

For producing PDF reports, again, something like Ghostscript with SVG graphics lib.  SVG Scaleable Vector Graphics.  You basically define shapes as their mathematical primitives which are drawn real time at an infinite range of resolutions.  So no matter how much you scale your gauge up it will be at full resolution of the browser.  More popular in print than web due to the infinite resolution aspect allows you to design on a 4K screen and print at 1000dpi.

EDIT:  SerDes.  SerDes between different formats tends to be tedious and repetitive.  For instance.  If you want to deserialise a can bus message with 47 fields into a meaningful JSON message you will need a lookup table in your parser to give them all names.  These translation lookups should not be treated without concern, they grow like cancer and will become unweilding.  So get "something" in place there as quickly as possible.  A "master" CSV file or a C header file defining them all in a single record of truth and then auto generate the translation code at build time... is one popular approach.

In your case, if you consider the fancy modern web front end (angular/react) option to explore...  If you can get the JS browser layer to interrupt the binary messages, I would just send the binary.  It depends on the data and how you want to interrupt it really as to whether this would be faster or not.  It can avoid the round trip ser-des which is technically only there so that the REST API is "Human Readable" which is the common convention.

[Marco]

For example, here’s a GUI with an interactive map showing online radio servers all around the world and a real-time computed solar terminator that I put together in just one evening. The screenshot was taken while running on a Raspberry Pi 4, and the code isn’t even optimized — it just renders everything directly on every frame 75 times per second.

That's the whole point though. Occlusion culling will hardly do anything for this, the big optimisation is retained mode rendering ... and that would kinda defeat the purpose. Ideally all application AND almost all UI library code can just sleep when there is no interaction and updates, practically that causes immense complexity.

[radiolistener]

Yes, but the map allows zooming into specific areas, so in principle many primitives could be culled during rendering. However, even without such optimizations it runs smoothly on a Raspberry Pi 4. And yes, ImGui usually assumes rendering every frame, which may lead to higher CPU usage than a static UI with no updates, but for simple interfaces the difference is negligible. For more complex UI, frameworks like Qt can themselves contribute significantly to CPU load. Technically, if the interface is static and doesn’t require constant updates, you could redraw only when needed, but implementing this adds extra complexity that usually isn’t worth it given the already low load.

[Simon]

I probably need to deal with understanding threading first and getting some CAN bus code working with another thing going on, then test out the GUI libraries.

I guess ImGUI works like LVGL on Linux (even if that is not it's intended use) they they draw directly in the screen buffer.

[Simon]

There seems to be two main methods of using the python can module.

busABC that would then need to used the notifier system with asynio.

The other method is the ThreadSafeBus one that works in threads.

I'm still not sure of which to use and why. I suppose I should be setting up multiple threads anyway so that the CAN bus can work without blocking any GUI. I'd rather not tie the CAN logic up in the GUI if I can help it. This will make it easier to change the GUI.

[paulca]

I wouldn't thread anything until you know where to do so.  That depends on your model.

Consider that the CAN Bus itself is not going to wait on your code.  Therefore whatever is sending receiving that isn't a queue has to be real time scheduled at least to the point of your buffer size.  You would like to hope that the CANBus module provides this low level queueing.  If it doesn't then I would put a queue on it to split the threads so that there is always a thread immediately going back to tend to the bus.

More threads does not mean better.  More threads means more competition and more conflict to resolve. 

For example, we have the "executor pattern", or "worker threads" where by new requests (messages) spawn new threads which live concurrent to the main receiver thread for the duration of the processing and then get recycled.  To stop catasrophic run-aways the "Executors" would typically be pooled, so that a fixed quantity are available but when they are all allocated requests for one block.  This is the model used by many web-servers, Apache for instance used to spawn whole processes and now spawns kernel threads IIRC.  The Server socket accepts new connections, spawns a worker thread and goes back to listening for new connections/messages.

This is all very well if you need that kind of concurrent power, but again I remind you that Python is "single execution" with a global interrupter lock.  One thread at a time.  They are therefore not suitable for increasing through put.  They are only useful for decouplin producer/consumer IO relationships and "waiting on many things" while idle.

A more lean and practical model is a 2 thread model with a queue in between.  The low level messages handling is done in one fast thread and puts messages/connections/events onto a queue.  Another thread, the processing thread, pops them off as fast as it can.

The simplicity means you can be more lax with your state as you know pretty much which thread is where and they tend not to share code, unlike the full async-async with async-io which is free form multi-threading with anonymous threads spawned for call backs everywhere.

For the UI.  My UI work is ancient and probably too low level for a modern Python thing.  They tend to be "event based", execution in the handler's thread.  So when "CLICK" happens from the main GUI thread interfacing with the OS, the execution that enters your click handler for the component is that GUI thread.  Thus if you take a while that GUI thread is also incapable of serving the OS "Redraw" calls.  The window is "crashed" and windows will even prompt the user "Wait or kill".  There an executor pattern or just spawned disposable threads for single tasks comes in handy.

If you are working in a "single address space" and the GUI can directly access to backend, then you can just reuse the UI thread, remembering, if you permit many of them to exist that access is then concurrent.  Again, add threads only when and where you need them, they have cost and teeth.

If you can keep each thread to it's own code and avoid them sharing code paths it makes things a lot simplier.  Writing code is not the hard part.  Not writing code is the hard part   Think it through first and you will have to write half as much.

[radiolistener]

I would not recommend trying to make a multithreaded GUI, as this is quite complex and often not fully supported by frameworks. While it is technically possible, you rarely see it in real-world code.

A better approach is to run the GUI in a dedicated thread responsible for rendering and handling user events, while background threads, for example handling CAN bus communication, process data without blocking the interface. To synchronize UI updates from background threads, it is convenient to use a command queue for the GUI thread, which can be as simple as a queue of lambdas or delegates. When a background thread needs to update the UI, it simply adds a lambda to the queue that performs the update, capturing any necessary values and continuing execution. The GUI thread, during its next render cycle, goes through the queue and executes the lambdas, safely updating the UI from the correct thread. This way you achieve asynchronous UI updates without introducing multithreading issues in the GUI itself.

[tggzzz]

More threads does not mean better.  More threads means more competition and more conflict to resolve. 

Conflicts of many sorts; deadlock and livelock are the most visible.

A more lean and practical model is a 2 thread model with a queue in between.  The low level messages handling is done in one fast thread and puts messages/connections/events onto a queue.  Another thread, the processing thread, pops them off as fast as it can.

There are added benefits...

You can debug each half independently by having a (unit) test harness that injects messages. That's really valuable for the GUI, since it can be run anywhere convenient. It is also a really convenient way of testing the "hardware" interface.

You can see what's happening in the complete system by logging the messages and observing the queue lengths. Sometimes it is possible to leave the logging in the delivered system, which is really convenient for field fault-finding and deflecting responsibility onto other companies.

The simplicity means you can be more lax with your state as you know pretty much which thread is where and they tend not to share code, unlike the full async-async with async-io which is free form multi-threading with anonymous threads spawned for call backs everywhere.

The half-sync half-async is a design pattern that is useful in many situations.

[paulca]

The half-sync half-async is a design pattern that is useful in many situations.

Annoyingly in work I am many miles from these layers now.  Our threading model is based on HTTP REST API events, 1 thread per request with all manor of DB executor pools ala Spring/JPA.  Around that the services, all 2 to 3 dozen of them, are not only their own processes, but the only process on their micro OS container.  So all component interactions are async and concurrent over the wire.

Again all very well for when you need to handle a million requests a minute or 1000 users concurrently and thus need to scale out horizontally.  We do the whole 18 holes of it and have about 30 transaction a week....  customer is paying though. (sic)

Middleware.  When you stop dealing with 1 or 2 components and start (or design to) split things up into dozens of components.  Then middleware, usually message buses become very powerful.  While they do make some things harder, like "Request/Response" and "transactions", their strict async model kinda sets a nice flow for most things.  Transactionality can be a pain, but not that bad.

[tggzzz]

HTTP REST is an interaction pattern. Processing can be implemented using half-sync half-async.

One thread per request is usually an antipattern. Much more horizontally scalable to have a pool of worker threads, one thread[1] per core. Plays nicely with high availability requirements. The processing in a worker thread can be the "half-sync" part, encompassing ACID transactions. Works very nicely for telecoms servers.

[1] for computations with a low latency

[Nominal Animal]

For the UI.  My UI work is ancient and probably too low level for a modern Python thing.  They tend to be "event based", execution in the handler's thread.  So when "CLICK" happens from the main GUI thread interfacing with the OS, the execution that enters your click handler for the component is that GUI thread.  Thus if you take a while that GUI thread is also incapable of serving the OS "Redraw" calls.  The window is "crashed" and windows will even prompt the user "Wait or kill". 

The event based approach is good, but synchronously doing the work is problematic.

The easiest way to get around that is to use a single worker thread, that communicates with the GUI thread using two thread-safe queues: one for ui-to-worker requests, and one for worker-to-ui responses.

It is not a 1:1 mapping, though, because the UI thread is still responsible for all transitions, popping up new windows, busy animations, and so on; only work that can potentially take longer than a fraction of a second on a slow computer is pushed to the worker thread — things like opening new files, processing complex operations, and so on.  So, not all UI actions cause a request to be sent.  For example, if you have a toolbar with many tools to choose from, selecting the active one is completely an UI thing, and would not cause any requests to the worker threads.  For longer tasks, like applying a filter or performing computation, I like the request to keep the action element "active", with the completion request making it inactive (like a button popping back up, for example), so that the UI reflects the state of processing at all times, even though internally the two are decoupled.

Python 3 has a built-in Queue perfect for this, although windowing toolkits may have even better ones (specifically, to inject responses from worker-to-ui directly to the ui event queue in a thread safe manner: in Qt, as a "signal" sent to a specific widget "slot", which can be done across threads.)

[paulca]

To steer it back to topic.

The reason we speak of "isolation", "layers" and "decoupling" via queues is not just the obvious "threading" model, but "state responsibility and concern".

Without any effort to isolate the various components you WILL 100% be tempted to access things from the GUI within the backend and vice versa.  This leads to things being reused here, there, everywhere.  Then you need to change something and the house of cards falls and turns into spaghetti mess.

"Isolation of concern" is the important one.  Don't permit components to reach inside others and manipulate them directly.  Instead define an interface for the component.  Make all communications/integrations via that interface.  The interface is the "contract".  A bit like the opening pages of a datasheet.  It says what it will do, what you must do to invoke it but nothing of how it does it.

If you are going to work in a single address space, which in python means a "single python script process", you can if you choose access most things without hinderance.  "private" is not really a thing in Python, there do exist annotations, but they are wet paper and routinely got around.  So it becomes purely "convention" to define an interface on components and use those and only those.

In MCU land an analogy might be that your boards by convention have an MCU with a UART interface for interboard communications.  However, a clever junior realises his IC has access to the SPI bus, so short cuts the UART and goes straight to the IC directly.  In review he gets scolded for coupling the two boards and if someone later changes that board to use IC2 instead the whole show crashes.

There are advantages to splitting things into more than one process and separating them via network, or just a socket/fifo.  The primary one is that now you simple CANT just reach in to the other component and help yourself to a value it happens to have that you want.  It's simply not available.  Your ONLY option is to go via the interface.

Doesn't have to be complex either.

In a very simple, non-dynamic or pull only UI you can just provide interfaces in the backend to "getData", "putData" or "takeAction", "processEvent".

If you want something more responsive you can just take the above and poll "getData" with a GUI timer.... or you can do "push" and add an "update()" call to the UI interface.

In code you have something like:

be = Backend( config )
ui = UI( be )
ui.runBlocking()
fail

When the user changes a drop down selected item the UI gets a "updated()" event from it.  It works out what data it needs and calls it's "be" backend reference to get it.

Backend in this case I made it a class, is just the "facade".  It only holds "interface" methods.  Everything it actually does it deligated to actual backend code.  The idea is to put out of reach what the client does not need to know.

[paulca]

One thread per request is usually an antipattern. Much more horizontally scalable to have a pool of worker threads, one thread[1] per core.

It is limited to a pool or maximum count underneath when requests will block.  Not by core though, "pods" are allocated cores on a decimal basis.  A pod might have access to 0.5 cores, but make no doubt that does NOT halve your concurrency problems.  It does NOT mean single threaded and it does not mean a single core.  At that stage though, in a pod, it's literally like being on a whole different computer where "you" are the only process.

[paulca]

Python 3 has a built-in Queue perfect for this, although windowing toolkits may have even better ones (specifically, to inject responses from worker-to-ui directly to the ui event queue in a thread safe manner: in Qt, as a "signal" sent to a specific widget "slot", which can be done across threads.)

I use the python built in Queue in a few places.  It works quite well for 1:1 offloads.  I don't think I've tried it under any kind of concurrent duress though.

I use it to batch DB updates for the time series database.  Components can spam the queue with one update at a time, but send 30 or so in a short burst.  This would result in 30 or so rapid fire REST requests.  So instead I have a 1 second timer which empties the queue into a batch update and send 1 request per second.  With the system  doing 3000+ updates a minute it makes a difference.

The other place is in processing a message topic which is "must process each".  So if a burst of messages arrive and 5 new "Requests" arrive before the processor can grab the first, then the next 4 are still in the queue unactioned, so every message is processed one at a time and none are missed.  In most other places, all 5 requests will overwrite the single state cache and still call the update handler 5 times anyway.  Idempoently of course.  A nice way to deal with "out of order" and "collisions" is to just design them into the "normal flow" from the start.

[paulca]

The easiest way to get around that is to use a single worker thread, that communicates with the GUI thread using two thread-safe queues: one for ui-to-worker requests, and one for worker-to-ui responses.

In a banks Quote Request/Response gateway I worked on they used Solice message bus and for "request response" style interactions we would spawn entirely new queues.

PUSH "Requests":  DoXAndConfirm - Request ID <UUID>
PULL "Responses_UUID"

But solice had queue retention settings so that these, single use queues where expired and garbage collected automatically.

When running on a normal day there were 100s of thousands of topic/queues.  Topics and queues were not the same thing either.  All very complicated.  The average was 60,000 quotes per "symbol" per day.  However they support 1600 symbols.  Each client would receive at least those 60,000 messages for each symbol they subscribed to.  Additionally they could send direct quote requests, order requests and receive individual order execution reports.  Of course, if they disconnected late in the day and reconnected they could request a full resend of the full day for all symbols.  Millions of messages in a batch.

This was basically:
I watch the price that others get by watching the Quote feeds for stocks.  However, if I want to place an order, I need my own unique quote ID.  So I send a quote request with an ID and I get a Quote response that the bank will honour for a brief period before the quote expires.  With my "Quote ID" I can then raise a order request and recieve an execution report.  The "end customer" is a webpage on a brokerage or the banks own websites for people to buy stocks/shares online.  You may have used these interfaces.

The worst anti-pattern I even seen though was the "ready()" endpoint used by the cluster to determine if machines where still up or dead would, as it should take a long time to execute when gateways were overloaded.  This caused gateways to be marked offline when they weren't actually dead.... yet.  How did they fix this?  They put the ready() endpoint into it's own dedicated thread.  DOH! Completely neutralising the entire f'ing purpose of it as a health indicator.  The idea is, that thread is meant to be heavily contended by "actual work", such that when the gateway is lagging behind that thread responds slowly to the cluster and the cluster can add a new gateway to the list.  However, budget had already been reached and no more gateways where available.

[tggzzz]

One thread per request is usually an antipattern. Much more horizontally scalable to have a pool of worker threads, one thread[1] per core.

It is limited to a pool or maximum count underneath when requests will block.  Not by core though, "pods" are allocated cores on a decimal basis.  A pod might have access to 0.5 cores, but make no doubt that does NOT halve your concurrency problems.  It does NOT mean single threaded and it does not mean a single core.  At that stage though, in a pod, it's literally like being on a whole different computer where "you" are the only process.

I don't know what you mean by "pod".

The way I've architected and implemented servers is that each worker thread is kept continually "occupied" (i.e. executing), with another few threads for general purpose housekeeping and a bit of I/O. In such cases, one worker thread per core is an excellent starting point.

It doesn't matter whether they are heavy-duty cores or lightweight SMT cores as found in Sun's Niagara T-series. Either work very well, but you can get more SMT cores on a chip than heavy-duty cores

[paulca]

Pod = container = virtual kernel = very light weight "native" VM.

Say you have a backend and a front end.  Instead of putting them into a "single server executable", you would instead run each in it's own mini-OS.   For 99.99999% of cases the software running in the pod thinks it is on a small linux system on it's own.

To communicate between them an integration layer like REST or message bus is used.  The components are stateless as best they can be and work on "message scope" only.  This means that when the UI component is getting hammered by lots of concurrent users, you can simple spawn a whole new UI component to take up 50% of the load.  HTTP->Gateway/LoadBalancer->Service->Pod* and the pods are spread across an opague cluster so the actual hardware nodes are practically irrelevant to the applications.

It avoids one of the biggest challenges to "get to production" and that is deployments.  Often not the deployment of the application itself, but its requirements of the OS.  If all the applications are being manged on a single (or clustered) server then that server much support, concurrently all the different requirements from those applications.  When applications start requiring different versions of core libraries all hell breaks loose.  In the "pod", "container" model, the dev team create the OS that each application runs on and each one can, though often isn't, bespoke to that application.  If the application doesn't need JPEG support you could literally remove libjpeg from it's OS.  All the platform has to do is honor it's side of the contract to the pods and provide networking, persistence etc.

EDIT:  In some less regulated environments, if your application needs a database because "reasons" and that database is not considered "core", then instead of requesting that someone create your database in the shared infra and that entails, you just add another pod to your deployment and run you own Maria DB instance and use it. 

Is it wasteful?  It depends on how you look at it.  Consider cooking a meal in a kitchen at home and then looking at how much waste you produced compared to the amount of food you produced.  Also how long it took you to prepare.  Then look a factory doing the same thing making 10,000 meals a day.  While their processes them selves are very wasteful, especially around batches and excess, their total waste per output is smaller than yours.  If that made as much sense as it hoped.

You could end up with 25 applications deployed and a total of 14 different DB server instances and 16 different webservers.  However, when compared to how long it would take to set them all up and maintain them in concert to share a webserver and a single DB server....  isn't practical efficient either in terms of time and money. 

That is until the dynamic change which I fear isn't coming anytime soon.  Currently it is extremely more expensive to write highly efficient code than it is to buy more server and compute power.

If that changes and the hardware starts to be the bottle neck again, like it once was, "high performance" coding and "efficient footprints" will become a thing again.  Today it is religated to specific usecases where low latency or high through put is required.  Vanilla business apps are like Russian Dolls in layers these days.

[tggzzz]

"Container" I understand.

My applications have been a single application running across multiple servers, with a distributed key-value store and trad database for compliance purposes. Hence containers didn't offer sufficient advantages to the client.

[paulca]

"Container" I understand.

My applications have been a single application running across multiple servers, with a distributed key-value store and trad database for compliance purposes. Hence containers didn't offer sufficient advantages to the client.

It's not a different contentment.  More a generational evolution.

In 2014 the architecture you describe was what NYSE used for their "front ends".  The distributed key-value store is very popular in many forms.  Then is was as Oracle Coherence.  Since then, the number of options for key-value, cache and or unstructured data distributed storage have multiplied.  Today things like Kasandra, KDB, Elastic Search, bare low level Kafka, MQ, JMX, or MongoDB, Redis or that Java one I can never remember... GemFire!  If you feel really cruel, EJB.

All of them are a pain in the butt to deal with and tend to expand with no mercy to overtake any and all environments like weeds, if not carefully maintained.  You will fall asleep in meetings with people debating how to "reshard the cluster" without "dropping all the indexes" and usually "how the order for more nodes is going"

It's likely today your DB connections might get funnelled through a single point service.  Maybe, maybe not, depends.

I moved a personal app from "Central key-value" store to MQTT message bus.  The model isn't very different, except:

* topic = value is the primary mechanism of storage
* topics are hierarchical like subscriptions to them.  subscriptions are multicast
* basic QoS and Queuing
* limited though useful redundancy
* less actual code.

I haven't looked back fondly.

[tggzzz]

In 2014 the architecture you describe was what NYSE used for their "front ends".  The distributed key-value store is very popular in many forms.  Then is was as Oracle Coherence.  Since then, the number of options for key-value, cache and or unstructured data distributed storage have multiplied.  Today things like Kasandra, KDB, Elastic Search, bare low level Kafka, MQ, JMX, or MongoDB, Redis or that Java one I can never remember... GemFire!  If you feel really cruel, EJB.

All of them are a pain in the butt to deal with and tend to expand with no mercy to overtake any and all environments like weeds, if not carefully maintained.  You will fall asleep in meetings with people debating how to "reshard the cluster" without "dropping all the indexes" and usually "how the order for more nodes is going"

I created and used that architecture a decade earlier, c2004, with Tangosol Coherence. Cameron Purdy was a guy with the Right Attitude, and had created a good team around him. We mentioned an application problem we had noticed due to known flaky hardware; they were all over it like a shot and quickly released a significant improvement which worked on the flaky hardware

Then Tangosol was borged.

EJB is indeed heavyweight. There is - or was - a telecoms grade equivalent to J2EE, JAIN. It was simpler and more focussed, but came with respectable management facilities.

The real killer with HA distributed applications is the "split brain problem", since that has no known general solutions.

[Simon]

OK, so a practical example of where I am at:

Code: [Select]

import os
import can
import time

os.system('sudo ip link set can0 type can bitrate 500000')
os.system('sudo ifconfig can0 up')
os.system('sudo ip link set can1 type can bitrate 500000')
os.system('sudo ifconfig can1 up')

can0 = can.Bus(channel = 'can0', bustype = 'socketcan')
can1 = can.Bus(channel = 'can1', bustype = 'socketcan')

while True:

    can0.send(can.Message(arbitration_id = 0x123, data = [0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88], is_extended_id = False))
    print(f"Sent message on can0")

    msg = can1.recv()
    print(f"Received message on can1: {msg}")

    time.sleep(1)


So this loops every second but, if the message were not to arrive, it would hang.

OK so I give the message reception a timeout, but that means that the code will need to loop much more often. I am using the MCP2515 controllers, I do not know how deep their buffer is or how many of the buffers are used or if the OS has any buffer.

I will run at 1Mb/s on the bus, this means that a can message with no payload will take around 64µs to arrive and one with a full 8 bytes of data will take up to 144µs.

So no I'd need to run the loop every 64µs to guarantee that I get every message, I assume that the recv() method just grabs the top message in the buffer.

Now obviously this is not the way to do it before I even fuss about a GUI system running alongside this. I guess a separate thread that checks for messages could have a really short timeout in the µs and then sleep for 60ish µs.

[paulca]

For a start:

Code: [Select]

        try:
            bus.send(msg)
            print(f"Message sent on {bus.channel_info}")
        except can.CanError:
            print("Message NOT sent")

Might help with the hang.

Using system() especially with sudo is very taboo. 

This looks more practical:
https://python-can.readthedocs.io/en/stable/notifier.html