uC remote Data Storage

[Gitch]

Hey All,

So I'm looking into starting my next project... This will be my first web connected one and I am trying to decide on how best to store variables and how often to be checking them.

I'll break it down to two projects initially, so my first question is in regards how often you would check a network connected device for it's current state. I have an a/c controller that provides a JSON output when requested and I am looking to add a better interface (part of my whole home integration) I'm think 500ms should be ok? Or should it be more or less often - remembering I would also be able to write back to the a/c from several devices.

Second one, is how people best store variable for their connected devices. Initially I will be using a Pi as my central unit (allow for html control and logging of certain parameters) and I am thinking of using a MySQL (or other db) and either an update statement or add statement depending on if it is or isn't a logged variable i.e. Temperature logging vs live state

Any feedback appreciated
Cheers

[jbb]

In industrial settings, you would tend to choose a sample rate with reference to the time constants of the process.
In this case the process is room temperature control? That won't change fast, so maybe every minute?

Additionally, be aware that long term log systems get really large. So even if you have plenty of bandwidth, storage is an issue.

Alternatively you can do some mixes sampling. For example, you could use a local node to poll the air con unit every 500ms, do some summary statistics (e.g. Min, max, average) and then send aggregate results off for logging every 2 minutes. Events like aircon start and stop could be logged as time stamped events to preserve time accuracy.

Also FYI, I hear that Rapberry Pis are a little prone to having their file systems wrecked if they're writing and the power fails, so make sure you back up often. (Or maybe try out one of those UPS shields with backup battery.)

[viperidae]

Ideally your devices should push their state over the network. That's what protocols like mqtt are for. Did you build the a/c controller or can you reprogram it?

[3db]

As @viperidae  mentioned there's MQTT. This is often used together with NodeRed.
You can find lots of information on their deployment and use  on a Raspberry PI on yYutube.
Peter Oakes channel might be a good starting point.

3db.

[xani]

Try SQLite for storage. It will save a bit of RAM on rPi and the "data" is just single file so easy to backup/copy/manage

You can also fire the "big guns" and go InfluxDB + Grafana for pretty graphs

[Gitch]

Ideally your devices should push their state over the network. That's what protocols like mqtt are for. Did you build the a/c controller or can you reprogram it?

No, at this stage it is a pre-existing controller that has given me access to it's api. So at this stage unfortunately it is a request only...

Try SQLite for storage. It will save a bit of RAM on rPi and the "data" is just single file so easy to backup/copy/manage

You can also fire the "big guns" and go InfluxDB + Grafana for pretty graphs

I Hadn't looked too deep into the DB yet, but yeah SQLite is looking to be the pick of the bunch - as you said better suited for embedded options.

Cheers