Deterministic PID(?) tracker algos.

[paulca]

I have written a few "trackers" in my time, the simplest probably being a solar MPPT algo.  They come up from time to time.

One aspect of them is their "stability" if that is the right term.  Like an opamp, for example, an MPPT tracker algo will "hunt" and keep hunting until it gets the response it expects or it hits it's limiter.

Today I'm bored and trying to solve a control algo which cannot behave that way.  If the metric in question is offset from target a single response should be made. That response should be deterministic.

Ideally, but throwing a gantlet for myself, it should also be stateless, idempotent and thread safe.

Problem:
In a game, "Nucleares", I have a super-simplified nuclear reactor.  A key metric is "Iodine Generation".  It is the closest thing we have on the game webservice to actual "power factor" or "flux level".  Iodine generation is proportional to reactivity.  Layman... it's the power the engine is delivering.  For purposes here, if iodine generation is above target, the reactor is reacting too much and the algo should respond to lower it and vice versa.

The throttle, in this specific case, is the control rods.  Fairly unsurprising that.  There are alternatives, but the only thing that would change, in context of the algo with those are the "feedback delay" time.

Solution:
For a given target iodine generation rate the algo will compare this target to the current generation rate.  It shall then produce a "single response" with a corresponding rod movement to correct the difference.    It should not "chase" or "hunt".  A single movement of "Iodine generation" should result in a single move of the rods.

Scrap book attempt 1:

Code: [Select]

def calculate_rod_delta(iodine):
    offset = iodine - iodine_tgt
    offset = offset * 5  # rods move 0.1% for each 0.02 iodine offset
    return offset

This gives the correct rod delta.  When looped however it highlights a problem.  As it deals in "rod delta" only and there is a lag time from commanding the rod move, to them moving, to the reactor responding, to the iodine increase being detected... it will continue to send "-0.1"..."-0.1"..."-0.1" until the reactor lag time completes, by which time it will have moved the rods 2 steps too far.  It will recoil back by "+0.2"... "+0.2" and ... we have a rapidly accelerating oscillation.  It's an opamp turns RF emitter bouncing rail to rail.

The practicalist in me says to just increase the period to at least twice the maximum lag time.  Make the algo wait for 20 seconds following any action.  I call this "termporal hysteresis" and it's a cheap cheat I have used many times.  I want to explore other ways.

I figured here would be a good place to get advise as people here understand things like "signal theory" and "impulse response", "damping" etc from EE degrees.

I believe I am looking for a highly damped algo or is that "critically damped" in that it will attempt to correct without any oscillation within a single "period".

Anyway.  One approach is to split the algo into three phases.  "Wait offset", "Make change", "Await response"

So if the iodine drops by the minimum amount for a control rod step (0.02), the algo sends the "rods-0.1%" command and then goes into a new loop awaiting change.  If that change takes longer than a timeout or if that change occurs in the wrong direction, flow will be released back to "Make Change".  If it occurs in the expected magnitude (required rod delta goes to 0) flow is returned to "Wait offset".

Pigeon code:

Code: [Select]

BEGIN
WHILE offset = 0:
   read iodine
   offset = iodine - iodine_target * scale_alignment_value
WRITE rod_movement with offset
WHILE offset != 0:
   read iodine
   offset = iodine - iodine_target * scale_alignment_value
LOOP

It is making the assumption its at 0 offset when started, but that can be addressed.

Now... is there a fancy way to do this mathematically instead of iteratively.... or a way to combine the two loops into one with some fancy FIR/IIR filter wizardry?

[brucehoult]

I think the problems are twofold:

1) you often don't have a precise model of the system being controlled, in order to be able to calculate the optimal input and output and how to get there. Solar MPPT would probably be one of the closest to precise model that exits, if it wasn't for things like clouds ...

2) your measurements of the current and past states of the system are noisy, so even if you completely understand 1) you can't work out the precise change needed

This might be a place to start:

https://en.wikipedia.org/wiki/Kalman_filter

[paulca]

For a laugh I conversed with ChatGPT and sought out my comfirmation bias.

In evaluation of my approach it says:

Yes, that’s a well-structured, dynamic approach. It effectively combines event-driven control with a simple state machine to manage lag time. Here’s how it works:

✅ Three-Step Approach:
1. Parameters in Range — Idle State:
Monitor the system parameters continuously.

If the parameters remain within the acceptable range, no action is taken.

Transition to Step 2 only if a parameter moves out of the range.

2. Parameters Out of Range — Take Action:
Take corrective action (adjust control variable).

Store expected response: Calculate the expected response based on the control action.

Transition to Step 3, where the response is monitored.

3. Await Response or Timeout:
Monitor the parameter to see if it reaches the expected response.

If the response meets the expectation, return to Step 1.

If not, wait for a maximum of 2 × max lag time, then re-evaluate and potentially adjust the control action.

✅ Advantages:
Deterministic Response: Reduces oscillations since the controller only responds after observing the full effect of a control action.

Adaptive: Adjusts to varying lag times without needing a fixed sample time.

Event-Driven: Minimizes unnecessary control actions during steady states.

✅ Potential Enhancements:
Introduce a dead-band or hysteresis to prevent minor fluctuations from triggering unnecessary actions.

Implement a recovery state for cases where the expected response is not achieved, allowing for more nuanced handling of unexpected conditions.

But it's just massaging my ego.

[paulca]

2) your measurements of the current and past states of the system are noisy, so even if you completely understand 1) you can't work out the precise change needed

It is "deterministic" as it is just a simulation.  There is reciprocal code algos in-game responding in the same way to things like Xenon concentration rising which will lower the iodine generation.  Noise is minimal at a steady state.  Swinging values around there are many lags and sinks and sources through the whole system a big change could take hours.. no days to reach equilibrium.

The algo in essence is to remove the click fest it is pulling rods through the first 2 to 3 days of operation due to rising xenon.  Then the click fest of pushing htem back in when peek xenon passes... if it ever passes.

In theory if my algo is a mirror to theirs it should just "lock in".  Of course all of this assumes "steady running".  If I trip a generator, shut off the wrong pump, or like last night "fat finger" a rod change to 44 and actually command 4. ... well I hope to enjoy stress testing my algo lets say.

What I did was run my super simple PID and low pass filter it in my head.

It said 0.0.... repeated this every second and when it said -0.1 I moved the rods -0.1.  It continued to say -0.1 for about 10 seconds, then dropped back to 0.0

In my "3 part" statemachine this progresses fine.

Of course. I am working on a very small set of test data so far.  I know that for iodine movements of <0.1 a corresponding linear rod movement works in about 10 seconds.  I have no idea how it will respond if I move iodine production by 1 or 2 or 10.

Increasing "t" in the PID process to 2xMax Lag time would remove almost all chance of oscilations, it's basically a low pass filter with t set to 20 seconds.

The trouble with that, if you go too far it won't respond fast enough to faster changing values.

[paulca]

Before over complicating things, give the first, most simple, iteration a test.  Even though it looks flawed, it might just work out fine with a few adjustments.

A simple limited PID with a "factor" of 5 between iodine and rod delta.  A t=20 seconds, a step size limiter of 1% and a resolution of 0.01% movement.

It ran it rock stable, 3.40 idoine +/- 0.02.  A lot better job than I can do.

"Go go sticks" = rods.
"Daughters" trace shows iodine generation rate.

This is a reactor start up with 3kppm boronic acid (Prozac for reactors).  Target, 3.0 iodine generation. Algo controlled rods, no intervention.

Evaluation.  Rods perfect a slight "tick" caused by 'step rounding'.  Target power of 3.0 for a "New game" scenario... way too high.

Anyway.  Next is to the next most annoying click-fest.  Balancing the steam generators.  For it, a different set of problems arise. 

Input condensate(kL/min) ~= Output steam(ton/min)

With a nice side order of non-negligible variables like pressure and temperature to ... code around.