Author Topic: How Do You Decide Which Protocol to Use in an IoT Application?  (Read 16397 times)

0 Members and 5 Guests are viewing this topic.

Offline EVblog1Topic starter

  • Regular Contributor
  • *
  • Posts: 145
  • Country: 00
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #175 on: August 12, 2026, 08:30:48 am »
MQTT is nice and simple when it is used as "shoot and forget", i.e. qos 0.
Otherwise good luck with all those pubrels and pubcomps. Suddenly it is bloated and pain in the ass to implement correctly. Some brokers (e.g. aws) just refuse to implement qos2 because - it is bloated.
Have you actually implemented all three MQTT QoS levels in a real project, or is your understanding mainly theoretical? If you have practical experience with them, could you explain a little more about the differences and when you would choose QoS 0, 1, or 2?
 

Offline tellurium

  • Frequent Contributor
  • **
  • Posts: 322
  • Country: ua
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #176 on: August 12, 2026, 09:00:40 am »
Have you actually implemented all three MQTT QoS levels in a real project, or is your understanding mainly theoretical? If you have practical experience with them, could you explain a little more about the differences and when you would choose QoS 0, 1, or 2?

I suggest you do your homework first, then ask specific questions and not a general purpose tutorial.

Ask GPT, read HimeMQ's tutorial on QoS. Spend 20-30 min, implement a simple PoC with all QoS. Then revert and ask.
Open source embedded network library https://mongoose.ws
TCP/IP stack + TLS1.3 + HTTP/WebSocket/MQTT in a single file
 
The following users thanked this post: nctnico, Siwastaja, Kilrah, eutectique

Offline EVblog1Topic starter

  • Regular Contributor
  • *
  • Posts: 145
  • Country: 00
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #177 on: August 12, 2026, 09:35:14 am »
I suggest you do your homework first, then ask specific questions and not a general purpose tutorial.

Ask GPT, read HimeMQ's tutorial on QoS. Spend 20-30 min, implement a simple PoC with all QoS. Then revert and ask.
Thank you, I appreciate suggestion 
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 6386
  • Country: gb
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #178 on: August 12, 2026, 11:30:20 am »
If a packet gets lost TCP will resend it, and will keep re-sending until it succeeds or the whole thing times out. You have probably seen this many times when you browse the web and a site does not load. So, if the connection breaks you will not actually know whether the server received something or not. You would have to start over and re-send. Therefore the server may receive duplicates which it has to deal with. This is similar to email where the server will remove duplicates if it receives multiple messages with the same id.

5xx would mean a server error, such as server run out of disk space.

In this particular case, none of these protocols (like MQTT, HTTP, SMTP, or whatever) is needed because they don't add anything to what TCP already does. But you would want to use a protocol anyway. You have learned how to use a protocol and now if you don't use it all your learning will be in vein. You simply must use it. That's how the bloat begins ...

Both ends know the connection broke though.  The connection ends with IIRC a 2 way handshake FIN FIN or RST RST

Either way or others the TCP/IP stack will report it to the code.  If you carefully wrap your own transactional code around that state machine, then there shouldn't be any dirty writes which can't just be rolled back.... thats if you care.

getHandler() {
  mqttClient.publish("your ma!")
  throw( Whoops )
}

And you would break that rule.

try{
  // the whole shebang
  // as the LAST failable thing
  publish_stuff();
} catch {
  // phew!  "probably no mutation or emission"
} finally {
  // The either or.
}

Is getting better.  This is what tends to happen all the way down through the layers at each layer.  None of them want to be "that layer!"

The "probably" comment though is another reason why the layers recursively check this pattern... or it's that one that doesn't.

Whatever "publish_stuff" does is being trusted to obey the pattern, "DO or Throw.  Never both, never neither. OR"

The actual problem however is basically mooted in much the same way that "Parallel buses" was in almost everything.

Hard synchronous transactions have been overlaid with bisteringly fast async ones.... and return "notes".  "I promise to pay the bearer of this request an answer.  Sometime... at some point." If doesn't need to be an absolute record of truth, then it can be "eventually consistent".  Lastest and best wins.

If you want a hard record of truth from such a system, you basically need to snapshot it at a moment of time and write it to an archive DB.

If you pay particular attention to your online banking statement every day for a few months, you will notice that even the international banking system is "Eventually consistent" and inconsistencies can exist in the "records" for quite a long time.  In some cases 2 weeks.  In cases of protest, years.

You see things like duplicate transactions where a "HOLD" on a card crossed in time and space with the "DEBIT" request.  The system is out of sync.  Invalid.  But it gets corrected by an other process which detects and adjusts the ledger.  No transactions are erased or even reordered, the ledger is adjusted append only to realign it.  On your statement it can "appear" that they rewrote it by adjusting the balance in the past.  Thats just an illusion formed by the query on the effective date rather than actual date.
« Last Edit: August 12, 2026, 11:37:46 am by paulca »
"What could possibly go wrong?"
Current Open Projects:  68000 Self Build computer + OS.
 

Online NorthGuy

  • Super Contributor
  • ***
  • Posts: 3519
  • Country: ca
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #179 on: August 12, 2026, 01:03:55 pm »
TCP already does "chunking" - it packetizes the stream, gets feedback from the server, re-sends what was lost. I don't think duplicating this would make it any better.

If the connection loss is a problem (such a NAT node in-between constantly times out your connections), then the server may retain what was already received and you need to deal with this in your protocol as a part of the handshake, such as:

Code: [Select]
station: hey, I'm sending you record xyz
server: I already have 548320 bytes of xyz, continue from that point
station: Ok, sending.

This is very simple and I'm sure it is used in many different protocols (for example HTTP can do this if you make the picture taking station a server and the central unit a client), but it doesn't mean you have to find all such protocols and select one of them. It means you need to organize a sensible handshake which is appropriate for your particular situation, which you need to do anyway.

The OP has very beefy device which is capable of constant parsing of the video stream with image recognition, encryption, storage of large image files somewhere - most likely on SD card (so that they're retrievable in case of failure). It is highly unlikely that there may be resource shortage for TCP operations.

If network is bad, it may be beneficial to send records without pictures quickly, then use spare time to transfer pictures.
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 6386
  • Country: gb
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #180 on: August 12, 2026, 02:10:26 pm »
Restart/replay/resume is usually done via sequence numbers rather than byte numbers.  As it is in TCP.

FIX protocol for stock exchanges uses sequence numbers on messages and the protocol requires they remain in sync at all times.  It also states exactly how each party should behave under a desync.  It's basically "stop normal processing", realign the sequence numbers on the lacking party with replay (back fill) or blanks, "blank fill.  The later is MUCH faster if you don't care for what you lost.

Most async message bus systems have similar and the contract is usually.  "A client is assigned a sequence number." ... stuff happens that the message bus cares little for ... "A client posted a new offset consuming that message".

If you throw a bucket of water over half the servers at the same time, it recovers.  The assignments get flushed.  All clients are notified to expect their offsets to change.  The correct "next unhandled" message is picked up by the next available consumer.  But again, the full mechanics, not botched my me are a textbook.


EDIT:  The HTTP mechanism, to resume a download I believe is a "webserver" layer feature which appeared only around 2000 widely?  Before then if your 56k dial up dropped... there was no resume on your 5Mb MP3 download.  I believe the client passes offsets or where it would like to start from.

It should be noted that it is NOT the same as an HTTP keep-alive resume.  It's a new transaction.  The old one is gone.  Literally the instance of server which served you have been evicted out of RAM long ago (or reinited and reused).

Thats the point.  And under transactional integrity the client also rolls back on error.  It doesn't try and "resume" at that protocol layer.  It doesn't support it.  It creates a new transaction, but asking for an offset.
« Last Edit: August 12, 2026, 02:16:58 pm by paulca »
"What could possibly go wrong?"
Current Open Projects:  68000 Self Build computer + OS.
 
The following users thanked this post: nctnico

Online NorthGuy

  • Super Contributor
  • ***
  • Posts: 3519
  • Country: ca
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #181 on: August 12, 2026, 03:59:01 pm »
Restart/replay/resume is usually done via sequence numbers rather than byte numbers.

You receive a picture. You have received a part of it, then the connection breaks. TCP timed out and stopped. You need to restart and open a new connection. You know how many bytes you have already received. You don't want to receive these bytes again. Where the hell would you get a sequence number from?
 
The following users thanked this post: Siwastaja

Online PlainName

  • Super Contributor
  • ***
  • Posts: 8790
  • Country: 00
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #182 on: August 12, 2026, 05:46:35 pm »
Quote
Where the hell would you get a sequence number from?

Should all these be timestamped and/or have a unique identifier? Real world car park cameras suggest missing a car (or seeing it twice) isn't unknown and causes a right problem (financial, to the car owner). If the unique number were a sequence number for that camera, you'd know if a) you've seen it already and b) you've missed one. Timestamp is a verification of a) and also useful when someone insists they weren't there at the time.
 

Online NorthGuy

  • Super Contributor
  • ***
  • Posts: 3519
  • Country: ca
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #183 on: August 12, 2026, 06:13:12 pm »
Quote
Where the hell would you get a sequence number from?

Should all these be timestamped and/or have a unique identifier? Real world car park cameras suggest missing a car (or seeing it twice) isn't unknown and causes a right problem (financial, to the car owner). If the unique number were a sequence number for that camera, you'd know if a) you've seen it already and b) you've missed one. Timestamp is a verification of a) and also useful when someone insists they weren't there at the time.

These all are part of the record which is timestamped when the picture was taken. You can read the requirements a few pages back. The record is probably encrypted by the time it gets transmitted. Therefore it's largely irrelevant what's in it for the transmission purposes, except that the record exists and has some unique id which can be used by the system to distinguish between records.

The problem is that the record contains a somewhat large jpeg image of the plate, and therefore cannot be transmitted "atomically". We're now discussing how to handle the situation where a TCP connection breaks in the middle of the transmission of such record.
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 30156
  • Country: nl
    • NCT Developments
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #184 on: August 12, 2026, 06:38:20 pm »
Restart/replay/resume is usually done via sequence numbers rather than byte numbers.

You receive a picture. You have received a part of it, then the connection breaks. TCP timed out and stopped. You need to restart and open a new connection. You know how many bytes you have already received. You don't want to receive these bytes again. Where the hell would you get a sequence number from?
This is a communication optimisation problem, not a transaction problem. Think about where you get the information from which tells you where the picture belongs to. That higher level information comes when a new transaction is being started. The half of the image can be lingering in a cache somewhere as part of a transaction in limbo. But to identify the transaction, you'll need the sequence numbers. A picture which is part of something never travels without metadata. So as Paulca already noted: a new transaction can have a handshake which says: give me the rest of the image.
« Last Edit: August 12, 2026, 06:41:06 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 8790
  • Country: 00
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #185 on: August 12, 2026, 07:09:35 pm »
We're now discussing how to handle the situation where a TCP connection breaks in the middle of the transmission of such record.

Yes, I realise that. But if your high level can figure out it's already seen a particular photo (or missed one) it doesn't matter that the lower level is in a tizzy. The only thing that really needs to do is best effort get this data to that receiver, but it's a mistake to think it can be perfect.

I would refer you to FTP, which can resume a transfer if the connection goes wobbly. OK, it's a communications app, but the point is it's a high level and runs over TCP/IP, but accepts that TCP isn't going to be perfect.
« Last Edit: August 12, 2026, 07:12:47 pm by PlainName »
 

Online NorthGuy

  • Super Contributor
  • ***
  • Posts: 3519
  • Country: ca
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #186 on: August 12, 2026, 08:34:20 pm »
So as Paulca already noted: a new transaction can have a handshake which says: give me the rest of the image.

Exactly. Just as I said here:

Code: [Select]
station: hey, I'm sending you record xyz
server: I already have 548320 bytes of xyz, continue from that point
station: Ok, sending.
 

Online NorthGuy

  • Super Contributor
  • ***
  • Posts: 3519
  • Country: ca
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #187 on: August 12, 2026, 08:37:00 pm »
But if your high level can figure out it's already seen a particular photo (or missed one) it doesn't matter that the lower level is in a tizzy. The only thing that really needs to do is best effort get this data to that receiver, but it's a mistake to think it can be perfect.

What make you think that I thought it would be perfect?
 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 8790
  • Country: 00
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #188 on: August 12, 2026, 09:32:33 pm »
But if your high level can figure out it's already seen a particular photo (or missed one) it doesn't matter that the lower level is in a tizzy. The only thing that really needs to do is best effort get this data to that receiver, but it's a mistake to think it can be perfect.

What make you think that I thought it would be perfect?

I probably didn't phrase that appropriately, for which I apologise. What I meant was that the focus seems to be on the transport dealing with data recovery, and ISTM that the higher level should be doing that anyway (because the transport may change, the sending unit may get confused, etc).
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 6386
  • Country: gb
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #189 on: August 12, 2026, 09:41:27 pm »
My comment on how this is normally handled was much wider than HTTPoTCP.

Your best approach to duplicate and replay data is impotency. This is genuinely how large distributed messaging works in "fault scenarios".  The idea is, if you replay the message logs from the last "snapshot point" to all consumers.  The overall system will arrive back in exactly the same state it was before it crashed.  (Mutative actions are normally muted during replay).

On a smaller scale.  If I update 20% of a jpeg, reset my PC and then update 90% of the same jpeg, lose my internet connection and upload the full 100% finally the recieving end, ends up with one jpeg.  Nobody gets harmed.  The cost is just waste.

A large jpeg or MJPeg, as might be produced by a CCTV capture system from an RTSP or similar stream, might be a few Mb.  Do you really need to resume anything?  Why would you introduce the extra headache when bandwidth is cheap.

Pick your poison.  Its 90% of the task.

If you are deploying something like a Raspberry PC or a PC104 board and it has an SSD and Multiple giga bytes of RAM, you can get away with a LOT of misshaps and not lose data.

Most basic CCTV capture appliances for such platforms have a local disk which is "tail erased" to create a loop cache.  That cache of local files is then synced to the receiver.  At least the ones I played with worked like that.  "Local ring buffer of captures" period (or event) syncing to home base.  The later is usually in the cloud on such off the shelf systems.  Once uploaded and verified, the client can be flagged for their removal.

That RTSP stream itself from the camera.  It will 99% guarateed have an extremely small buffer and operate under "early drop" principles.  ie.  if it's late, don't bother sending it, get the next one on time and get back in sync.
"What could possibly go wrong?"
Current Open Projects:  68000 Self Build computer + OS.
 

Offline John B

  • Super Contributor
  • ***
  • Posts: 1073
  • Country: au
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #190 on: August 12, 2026, 10:59:31 pm »
It has other features like retaining messages, and QoS that make it suitable for potentially sporadic connection, eg with a WiFi ESP32.

Have you ever tested if any of these features actually work? Genuinely asking.

To depend on these, library recommendations would be appreciated......

Currently it looks like I will be focusing on Rust going forward, and for MQTT specifically I have been using the rumqttc library, which is a new pure Rust implementation of MQTT. I use an async client on top of the Tokio async runtime.

The easiest implementation is to run the eventloop in a separate async task. It returns any possible event type as an enum carrying a value (eg more enums or Strings), eg connection errors, incoming or outgoing packets, which could be acknowledgements, publish messages etc.

This is where Rust's enum pattern matching really shines. For your re-subscription issue, I would be first matching for a connection error type.  You can go through exhaustive specific connection error types, or just a general error type. For my purposes a simple 5 second retry interval on pinging the broker is sufficient.

Once there is a successful connection I would pattern match on a ConnAck event type and call some subscriptions() function. You can do further checks on a SubAck event if you need to confirm successful subscriptions.

It can function in a synchronous context as well, but it's most suitable for server side / desktop side OS environments. There are other libraries that target embedded environments but I haven't tried them yet.
« Last Edit: August 12, 2026, 11:11:32 pm by John B »
 

Online NorthGuy

  • Super Contributor
  • ***
  • Posts: 3519
  • Country: ca
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #191 on: August 13, 2026, 12:13:05 am »
On a smaller scale.  If I update 20% of a jpeg, reset my PC and then update 90% of the same jpeg, lose my internet connection and upload the full 100% finally the recieving end, ends up with one jpeg.  Nobody gets harmed.

Sure. But imagine there's a problem - network congestion, DoS attack, cable failing, router constantly rebooting, whatever. You have much more luck pushing through small pieces than trying the whole 3MB picture over and over again. That's reliability.

The cost is just waste.

There's no cost. You just write your code a little bit differently, that's all.
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 11182
  • Country: fi
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #192 on: August 13, 2026, 05:57:07 am »
If this was some kind of server-to-server, I would not bother with resume, but I agree with NorthGuy that in case of IoT traffic cameras spread over India, resuming image file transfers (maybe of several megabytes) instead of retrying fully would be good design and not expensive in amount of extra code. The reason: I assume these boxes would connect through whatever cellural network, some of them could be far from the tower, some just congested by other cellural users. It's entirely possible transferring an image could be taking tens of seconds to even minutes. Frequent TCP socket teardowns will happen in some relevant corner cases. A full resend of a, say, 2MB image could get stuck in a loop wasting precious bandwidth of other images that need to be dealt with.

Then again, another important design question, not related to protocols, is what is the maximum sustained rate of cars being detected and photos being sent? Can the link maintain that bandwidth? And if the "rush hour" if nearly non-stop 24/7, no amount of local memory suffices for evening out the bandwidth. This becomes a question of image processing more than protocol design: cropping of images to contain just what's needed, finding the correct jpg compression level so that information does not suffer, or maybe using something else than jpg. And maybe the end result is that "images need to be 50KB max and we still need a reliable always>10 Mbps link", in which case the resume of a single file would be unnecessary after all.
« Last Edit: August 13, 2026, 05:59:05 am by Siwastaja »
 

Offline nfmax

  • Super Contributor
  • ***
  • Posts: 1687
  • Country: gb
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #193 on: August 13, 2026, 08:01:24 am »
Your best approach to duplicate and replay data is impotency. This is genuinely how large distributed messaging works in "fault scenarios".  The idea is, if you replay the message logs from the last "snapshot point" to all consumers.  The overall system will arrive back in exactly the same state it was before it crashed.  (Mutative actions are normally muted during replay).

I think - I hope - you mean idempotency. Blame it on the autocucumber 😉
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 6386
  • Country: gb
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #194 on: August 13, 2026, 09:31:02 am »
Your best approach to duplicate and replay data is impotency. This is genuinely how large distributed messaging works in "fault scenarios".  The idea is, if you replay the message logs from the last "snapshot point" to all consumers.  The overall system will arrive back in exactly the same state it was before it crashed.  (Mutative actions are normally muted during replay).

I think - I hope - you mean idempotency. Blame it on the autocucumber 😉

LOL!  I type it and waited for red lines, there were none so I moved on.  Classic spell check reliable failure.

 :o
"What could possibly go wrong?"
Current Open Projects:  68000 Self Build computer + OS.
 

Offline EVblog1Topic starter

  • Regular Contributor
  • *
  • Posts: 145
  • Country: 00
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #195 on: August 13, 2026, 12:24:41 pm »
I started with a DHT11 connected to an ESP32 and first verified that I could read the temperature and humidity values correctly. After that I connected the ESP32 to Wi-Fi and used MQTT to publish the DHT11 values to an MQTT broker running on my PC using Mosquitto.

On the PC, I wrote a small Python program using the Paho MQTT library to subscribe to the esp32/dht11 topic and display the received values.

I also had a Raspberry Pi with an IR obstacle sensor connected to GPIO17. I first tested the IR sensor locally using Python and verified that the GPIO reading was working. Then I added MQTT publishing to the Raspberry Pi and published the obstacle status to another topic, raspberrypi/ir.

So the overall setup is basically:

ESP32 + DHT11 → MQTT → Mosquitto on PC → Python subscriber

and

Raspberry Pi + IR sensor → MQTT → Mosquitto on PC → Python subscriber

The PC Python application subscribes to both topics, so it can receive the DHT11 data from the ESP32 as well as the IR sensor status from the Raspberry Pi.




Code: [Select]
Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : esp32/dht11
Payload: Temperature=26.9, Humidity=95.0

Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : esp32/dht11
Payload: Temperature=26.9, Humidity=95.0

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : esp32/dht11
Payload: Temperature=26.9, Humidity=95.0

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : esp32/dht11
Payload: Temperature=26.8, Humidity=95.0

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : esp32/dht11
Payload: Temperature=26.8, Humidity=95.0

Received message
Topic  : raspberrypi/ir
Payload: Obstacle detected

Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : esp32/dht11
Payload: Temperature=26.8, Humidity=95.0

Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : raspberrypi/ir
Payload: No obstacle

Received message
Topic  : raspberrypi/ir
Payload: No obstacle
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 6386
  • Country: gb
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #196 on: August 13, 2026, 03:00:09 pm »
Interest angle to give you.

When your PC app receives an update for the IR sensor.  Technically all it has directly in scope of that event, is the IR sensor data.

Vice versa when it receives a temperature value, it does not have the IR 'state' in scope.

The moment you do:

ir_data = blah;
or
temp_data = blah;

As "shared" state between the two event handlers, you have not only introduced possibly concurrency problems, but have added "shared state". 

Now that isn't necessarily wrong.  It has implications though.

A few hints:

Calling "print" from the actual event handler will block it's thread on IO wait.  Probably fine, but if that terminal or whatever it is goes dead, it will block the entire "paho" client thread, if that's what you are using.

On, "paho" and "Python".  If you read the docs you will find it uses a single dispatch thread which pops the queue.  So if you block it, you block all events.  The "upside" for ease of use and "downside" for performance is that it's a single dispatch thread and not a pool of them. 

Treat the "on_message()" a but like an ISR if you will.

As an example.  I have a python service which subscribes to a MASSIVE swaith of topics and publishes the data to InfluxDB.  It does not make that network call in the Paho dispatch thread.  It pushes it onto a queue of metrics.  A while(running){} loop in a different thread watches the queue and either flushes it to influx every second or if it hits 1000 metrics (50% queue size).

Oh and good job.  Hope it was fun.  For polish, try "ESP Home", with or without homeassistant.  It gives you "remote code and logging access" to all your ESP32s.... on one page.
« Last Edit: August 13, 2026, 03:06:15 pm by paulca »
"What could possibly go wrong?"
Current Open Projects:  68000 Self Build computer + OS.
 

Online NorthGuy

  • Super Contributor
  • ***
  • Posts: 3519
  • Country: ca
 

Offline 5U4GB

  • Super Contributor
  • ***
  • Posts: 1735
  • Country: au
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #198 on: September 01, 2026, 04:43:27 am »
TCP already does "chunking" - it packetizes the stream, gets feedback from the server, re-sends what was lost. I don't think duplicating this would make it any better.

 

Online PlainName

  • Super Contributor
  • ***
  • Posts: 8790
  • Country: 00
Re: How Do You Decide Which Protocol to Use in an IoT Application?
« Reply #199 on: September 01, 2026, 09:29:46 am »
 :)
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf