Measuring time span very precise

[magic]

AVR has the CLI/SEI instructions to temporarily disable/enable interrupts.
Interrupts are automatically disabled when the core starts executing an interrupt handler and re-enabled when it ends.
SEI can enable interrupts within an interrupt handler, but I doubt that Arduino would do it behind your back and of course you don't need to do it either if it's undesirable.

[HendriXML]

There're 2 ways to hook on interrupts one using ISR(vector) and one using AttachInterrupt.

One is (I think) replacing the low level interrupt handler, one is adding something to be called by the standard low level handler.

It wouldn't surprise me if the latter has interrupts disabled and the other one not. If it's time critical the first option is probably the better choice.

It is good to know only one interrupt of a kind can be queued, so staying in them with interrupts disabled (for long) can have side effects. So if one knows what he's doing it's best to have only a small portion of code in CLEI/SEI. Besides that is getting harder to understand interrupts within interrupts within ... this is probably the main idea behind the common advice to keep the interrupt code very short.

But if one had to do some lengthy calculation (of lets say 500 ticks) every 1000th of a timer interrupt then it can still work fine. It will be interrupted many times (500).

I haven't tested this, but that is how I expect it to function.

[Ian.M]

The key to handling software extended capture registers with a software extended timer is: right at the start of the ISR, to grab the timer high bit, timer overflow interrupt flag, and all capture interrupt flags, and pack them into a byte or word for further processing.  Then do it again and check they haven't changed.  Repeat until you get two consistent sets of high bit and flags, and you then avoid all race conditions due to captures on different channels happening too close to each other or to the timer rollover. All the possible cases encoded in the flag set, including  incrementing the software timer extension if the stored overflow flag is set and doing a soft capture of the timer extension if any capture flag is set can then be handled without races or severe timing constraints.   

If multiple close captures on a single channel are required, that capture channel's capture register must be read in the same loop that gets a consistent flag set, and also checked for consistency.

On MCUs with vectored interrupts like the AVRs, point the timer overflow and all the capture interrupt vectors at the same ISR as described above.  Arduino attachInterrupt() is not useful here as it only supports interrupt capable digital pins, and one needs to route the timer overflow to the same ISR.

However that's a *LOT* of work and is only really appropriate when you have multiple input events (and possible generated output events) that you need to relate to a common long period timebase, so if you just want to measure an interval (or a frequency), its much simpler conceptually to use an internally or externally hardware gated timer that can simply count up from zero, with or without software extension, and only needs to be read when the gate signal has ended and the timer is thus stopped.  I described this for a PIC18 Timer 0 in my previous post. 

The original method of clocking a prescaler under program control to extract the count in it that you cant read directly comes from Microchip AN592.

[HendriXML]

There're 2 ways to hook on interrupts one using ISR(vector) and one using AttachInterrupt.

One is (I think) replacing the low level interrupt handler, one is adding something to be called by the standard low level handler.

It wouldn't surprise me if the latter has interrupts disabled and the other one not. If it's time critical the first option is probably the better choice.

It is good to know only one interrupt of a kind can be queued, so staying in them with interrupts disabled (for long) can have side effects. So if one knows what he's doing it's best to have only a small portion of code in CLEI/SEI. Besides that is getting harder to understand interrupts within interrupts within ... this is probably the main idea behind the common advice to keep the interrupt code very short.

But if one had to do some lengthy calculation (of lets say 500 ticks) every 1000th of a timer interrupt then it can still work fine. It will be interrupted many times (500).

I haven't tested this, but that is how I expect it to function.

Hmm what if a time critical interrupt was called and the bulky one immediately steps in... That would not be so great. Starting with interrupts disabled would have benefits then..

I've checked this, the interrupts are disabled by HW when entering an interrupt. Thus when doing a bulky operation one should enable them.

These things also show why using external stuff (the counters are kind of external too) can be advantageous.

[HendriXML]

Very interesting to think about interrupts and their order of execution again. That's how I started programming in DOS years ago. In the more modern OS's these problems got shifted to multi threading.

There critical sections, mutex's are used to be sure that access to data is synchronized: no simultaneous writing of data (or reading of dynamic data). In the DOS days one had to CLI (clear interrupts) to stop another "thread" from interrupting. Arduino has the same functionality. (On the 386 there were interrupts that could not be stopped, I recall). B.t.w. on PC's even the writing of a multiple byte (word, dword etc.) value can be interrupted, maybe for Arduino's as well.

What makes me uncomfortable with the overflow interrupt is that there's also stuff going on beyond  code (which can be protected). Like overflowing (going to zero) and the setting of the overflow flag should be atomic from the perspective of the code. Between one and the other instruction both should be updated, not only one. This I guess is the case.
However reading them will not be atomic.
It would have make me then more comfortable if there where higher order functions that would give both of them, and do a clear of the flag atomically. The caller that gets the overflow flag is the one that increments the higher bits. (Those higher bits would be read from / written to in a protected way.)

I don't think such a function can be written without interpretation of the counter, because of the external nature of updating the counter and flag.

So the next best thing would be to first read the counter, and only read the flag if the counter is less the 50% of max. (If it's more than that the flag is always 0. In reality it could be 1, but that wouldn't match the counter) If the flag is set then clear it and update the high bit counter. The same should be done in the overflow interrupt. The reason for this is only to not have the overflow happen twice without the execution of this code in between. (Anything happening more frequent than that would be usable - it should however be called before the counter reaches 50%)
99.99% of the time the overflow interrupt will be updating the higher bits. But the interrupt can't be the only one.

As said by Ian above reading stuff repeatedly is also a solution, but its good to know that this is because they are written atomically (at least practically). Otherwise one could theoretically read inconsistent values twice.
(Just to point out the difference between externally changed stuff, and for instance updating internal memory, which can be interrupted and be inconsistent a lot longer.)

[HendriXML]

TDC7200, easy to use, libs for arduino..

PS: with that chip (works fine) you may upgrade from the nerf darts gun to a railgun later on..

The time range it can measure is a bit narrow for a multi usable timer circuit.

[magic]

Not sure why you are trying to read anything "atomically"?

Kleinstein showed how to do it.
Check the overflow flag. If it's zero, no problem. If it's one, you need to figure out if the event has been captured just after the overflow or just before the overflow. You do it by checking if the captured timestamp is just above 0 or almost 0xffff.
Alternatively, compare it against the current counter value. That enables you to block interrupts for almost entire counter period and still avoid overflow errors. Figuring out the details is left as an exercise for the reader. Because it's pointless, just don't block interrupts for longer than a few dozen cycles.

By the way, the value of TCNTn or ICRn is one of the few things that can actually be read atomically; consult the datasheet.

[HendriXML]

Not sure why you are trying to read anything "atomically"?

Kleinstein showed how to do it.
Check the overflow flag. If it's zero, no problem. If it's one, you need to figure out if the event has been captured just after the overflow or just before the overflow. You do it by checking if the captured timestamp is just above 0 or almost 0xffff.
Alternatively, compare it against the current counter value. That enables you to block interrupts for almost entire counter period and still avoid overflow errors. Figuring out the details is left as an exercise for the reader. Because it's pointless, just don't block interrupts for longer than a few dozen cycles.

By the way, the value of TCNTn or ICRn is one of the few things that can actually be read atomically; consult the datasheet.

I was not trying to make a point to the very experienced. I mostly write with starters like myself in mind.
Off course you can always read the flag, but why do so if you act on its value only if the counter is low (overflow happened in counter as well).
From my perspective the "thread" that reads a positive flag takes responsibility to also do the high bit increment and clearing the flag. It's showing a way of thinking about this stuff. It is mostly about "atomic" read and writes, having stuff consistent with each other. Think about it that way leads to solid solutions in other problems as well, that why I like to hammer on it. 

[HendriXML]

You do it by checking if the captured timestamp is just above 0 or almost 0xffff

In my posts I compare it to the 50% value, 0x7fff
- just to explain the 50% terminology

[magic]

Off course you can always read the flag, but why do so if you act on its value only if the counter is low (overflow happened in counter as well).

I'm not sure if I understand what you are doing and I'm not sure if you are doing it right.

You don't need to check if TCNT1 is low or high to detect overflows. The overflow flag tells you that an overflow has just happened and hasn't been handled yet and that's all you need to know about overflows.

But if you use input capture, you need to check if ICR1 is low or high, if you discovered that there is a recent unhandled overflow.
The value of ICR1 tells you whether the captured event happened before (ICR1>50%) or after (ICR1<50%) the unhandled overflow.
This information is required to decide whether to increment the high bits before processing the captured event or leave it to be incremented later by the overflow ISR.

edit
Well, if you want to, of course you don't need to leave that job to the overflow ISR, you can increment it yourself and clear the flag.
But it needs to happen after a copy of ICR1 and the old high bits has been recorded by the input capture ISR as the event timestamp.

Again, I don't know. Maybe I'm saying the obvious, maybe you are making a mistake.

[HendriXML]

By the way, the value of TCNTn or ICRn is one of the few things that can actually be read atomically; consult the datasheet.

Good info! A good example of how they took extra effort of making something atomic  ..

So first read low byte and then high byte.

However reading both the counter and the flag together cannot be done consistent (atomically). That was something I was referring to. So my brain gets to think how to make it atomic? Why not take a derived overflow flag, which is always 0 when no overflow has happened in the counter. And is only 1 when the captured counter is low, and the real flag is still 1. It would even be possible that the flag would have been cleared HW after reading it (I think this kind of "tricks" are done), so hence the "practice" of not touching the flag if not needed. (You touch it, you eat it) In this case the flag probably won't change when reading (haven't checked it) but it could have been a way to only have one "thread" made responsible for it.

A side note, this atomic stuff is relative to the HW changing things. There still the requirement that only one thread reads the low and the high counter value. That is not atomically by it self, so interrupts should be disabled.

Normally api would try to handle these kind of pitfalls. So for young players this discussion might be of use.

[Kleinstein]

There is no need to make reading the overflow flag and ICP value atomic. The check is done inside the ICP ISR and thus with disabled interrupt flag. If the ICP value is changing due to a 2 nd event, one has a problem anyway.

With the AVR there is no problem reading the interrupt flag (no "You touch it, you eat it").

When the overflow happens just before the ICP, it can happen that the ICP ISR is called first. The case has to be handled separately and is checked with the overflow interrupt flag and ICP value. When actually handling this special case one can just do the jobs normally done in the overflow ISR and clear the flag or only correct the one reading and let the overflow ISR run (just after the ICP ISR is done). The 2 nd case is preferred if there are more jobs done in the timer overflow ISR.

Reading and writing low and high byte in the correct sequence is handled by GCC. However it does not use  CLI/SEI to disable interrupts.
Quite often the ISRs don't write to 16 bit registers that would effect the 16 bit access. In this case no CLI/SEI needed to make the access atomic.

[Ian.M]

The problem is, on many MCUs you need a snapshot of the flags and timer high bit state at a single moment in time to unequivocally guarantee that you are dealing with the capture extension and timer overflow extension soft registers correctly without inadvertant race conditions if they are too close together that might result in out of order processing and a 65536 count error in the full extended capture time stamp.  As the interrupt flags and the timer high bit are in different registers and are hardware volatile, so can never be read atomically, repeating till you get no changes in the snapshot is needed.  The same approach will work on *ANY* MCU that doesn't have clear on read interrupt flags, or even by polling in your main loop if you cant hook the necessary interrupts.

On some MCUs with vectored interrupts at the same priority level you *MAY* be able to get away without a snapshot and use separate ISRs, but to confirm that requires a deep understanding of the specific MCU's interrupt structure and worst case latencies, and if any library code can hook the same interrupts or runs at a higher interrupt priority, is horribly fragile.

[Kleinstein]

The correction inside the ISR depends on the ISR priorities and HW details. So it is hardware specific (should still work for all AVRs with ICP so far).
The AVRs don't have interrupt priorities in the sense of a higher priority interrupt interrupting another ISR. There is only a sequence in which the pending flages are checked. When doing only the correction (still keep the overflow flag active), there should be no problem if the lib code / Arduino environment would add extra code to the ISRs (as long as it would not enable nested interrupts before the timer code). Unless extremely long interrupt latency does not matter.

Repeated reading of capture register and software extension is quite tricky and would add other limitations form the loop speed. It does not really solve the problem and one would still have to handler the race condition case separate.
So I would not consider repeated reading in the main loop a viable solution. In the ISR there is no need to repeat the register readings - it's more like the opposite to read the ICP register only once, so that in case of 2 events one gets at least the correct time of one of the events and not in rare cases a wrong mixed value.

[Ian.M]

To get a consistent state for further processing you only need two identical repeated reads of the interrupt flags + the timer high bit.   Initialise the old_state variable with a spare bit set and the state variable with the spare bit cleared to guarantee an initial mismatch  and it simply becomes:

Code: [Select]

while(state!=old_state){
   old_state=state;
   state=pack_flags_and_timer_high_bit(); //replace with inline bitwise stuff
}
//Smash old_state here if in main loop

That cant hang for more than four passes unless you have a storm of short pulses on either capture trigger pin as only three independent event sources contribute to the state. (Timer overflow is a dependent event - it can only happen coincident with a high bit 1 to 0 transition). Therefore, if in each timing run you disable each capture unit after a single acquisition or if you are making do with a single capture unit, disable it after the second acquisition, then execution time is strictly bounded.

There is absolutely no need to 'blind read' or repeatedly read any capture registers, or reread the timer or keep any part of the timer value except its high bit.   

If you handle the various cases in the main loop, then you don't use any ISRs, and don't enable the relevant interrupts.   AVR peripherals set their interrupt flag when their trigger condition is met even when that interrupt is masked or otherwise disabled, so one would simply poll the interrupt flags to grab the state as described above then handle the various cases, as if in an ISR except with worse latency (i.e. the superloop execution time).  With an AVR @16MHz, you've got 2ms for the full superloop including the worst case capture and timer handling code

[HendriXML]

I'll try to summarize the things that have been said and are important:

Is the counter high/low byte and flag written consistent?
I assume it's atomic between 2 instructions

Can the counter be read consistent?
Yes by reading low byte before high byte. The high byte is kept safe.

Can this reading be interrupted?
Not in an interrupt, otherwise disabling interrupts is needed.

Is reading the counter and the flag always consistent?
No, when a overflow happened, reading the first one  of them can be old news. Reading it repeatedly can assure one self of the consistency. How ever because the counter can be read consistent and there's a lot of time between overflows, one also could assume there isn't a overflow when counter is larger then x7fff. And do a check is it's less than that. This way the counter is always consistent with the flag. (Only if the counter is xffff (or a bit less) and the flag is 1, there's inconsistency)

How about counting the 16+ bits?
On every overflow one should be added to the high word of the 32 bit counter. Its best to take the overflow flag for that, it is like a 17th bit. But the 16 bit value can only overflow once in that bit so it has to be cleared.

How to ensure the overflow isn't counted double?
The code that updates the counter, should also be the one that clears the flag. This operation must not be interrupted. Within a interrupt this is ok, outside an interrupt interrupts should be disabled.

How to ensure every overflow is counted?
Before the counter reaches x7fff the flag should be checked, counted and cleared. The overflow interrupt is very useful for that.

How to ensure that the counter and overflow flag are related?
Because the captured value is old news and the overflow flag is actual, the capture interrupt must be executed before they are not related anymore. The flag is checked when the counter is less than x7fff and might be unrelated if this is done x7fff ticks after HW capturing. So interrupts should not be disabled more than a portion of that.

There're different implementations possible but i think this lists what to consider. If something's missing or wrong I'd like to hear it.

[HendriXML]

To get a consistent state for further processing you only need two identical repeated reads of the interrupt flags + the timer high bit.   Initialise the old_state variable with a spare bit set and the state variable with the spare bit cleared to guarantee an initial mismatch  and it simply becomes:

Code: [Select]

while(state!=old_state){
   old_state=state;
   state=pack_flags_and_timer_high_bit(); //replace with inline bitwise stuff
}
//Smash old_state here if in main loop

There is absolutely no need to 'blind read' or repeatedly read any capture registers, or reread the timer or keep any part of the timer value except its high bit.   

If you handle the various cases in the main loop, then you don't use any ISRs, and don't enable the relevant interrupts.   AVR peripherals set their interrupt flag when their trigger condition is met even when that interrupt is masked or otherwise disabled, so one would simply poll the interrupt flags to grab the state as described above then handle the various cases, as if in an ISR except with worse latency (i.e. the superloop execution time).

As a general solution one should ensure that reading 2x an inconsistent state is not possible. (In this case that's ensured)

[magic]

This seems correct.

Writes to 16 bit counter registers are atomic if you write the high byte first and then the low byte. The high byte is remembered and applied later in the same clock cycle when the low byte is written. Interrupts must not touch the timer in the meantime.

I'm not sure if this clears the overflow flag. It may need to be cleared manually; in such case you better do it fast before the timer overflows for the first time

To get a consistent state for further processing you only need two identical repeated reads of the interrupt flags + the timer high bit.
[...]

That's all nice until you need to deal with an 8 bit timer running at full core clock speed

[HendriXML]

The captured counter and the current overflow flag can also be unrelated.  That would be the case when the capture interrupt was delayed very long. I'll have to put that in the list as well.

[Ian.M]

Consider the ATmega case with two capture units.  Their timers are synchronised so there is only one overflow event, but there are two independent capture events.  If the events are well spaced apart (and not too close to the timer high bit toggling) the loop only takes two passes. However if the start, stop and overflow events happen *VERY* close together, each of the later ones can invalidate the state match and force another pass of the loop.

This seems correct.

Writes to 16 bit counter registers are atomic if you write the high byte first and then the low byte. The high byte is remembered and applied later in the same clock cycle when the low byte is written. Interrupts must not touch the timer in the meantime.

I'm not sure if this clears the overflow flag. It may need to be cleared manually; in such case you better do it fast before the timer overflows for the first time

Once set up you *never* need to write to the timer. 

You've got half the timer rollover period before the state you've grabbed becomes useless.  As I edited  in above, that's fractionally over 2ms on an AVR @16MHz.  Use two capture units and complete the whole multi-state processing ISR including any flag clearing that needs to be done in under 2ms and you can guarantee not to miss any even if they are coincident and bang on top of a timer rollover, unless the same event recurs within your ISR execution dead time before you've read the event's capture register and cleared its flag.   If you need to handle fast repeating events put grabbing any active captures from their registers immediately after the consistent state loop.   You'll still be in trouble if a third event on the same channel occurs within the total ISR execution time of the second one.

To get a consistent state for further processing you only need two identical repeated reads of the interrupt flags + the timer high bit.
[...]

That's all nice until you need to deal with an 8 bit timer running at full core clock speed

Which is why you don't try to do this sort of s--t on a 12 bit core PIC, which as they generally don't have interrupts, also dont have an interrupt flag for their 8 bit timer 0, so you have to treat it as a 7 bit timer with a (self-resetting) overflow flag in the 8th bit that worst case will only be valid for 128 instruction cycles.   
BTDTGTTS 

[HendriXML]

There's something missing in my list of things to consider.
At high (near xffff) captured counters  it might be possible to add a high word that is new (and has an overflow in it). This would be the case if the overflow interrupt executes before the capture interrupt but after the capture has happened. I don't know whether that's actually possible. But i think the only way to counter attack that situation is to delay the increment of the high word of the counter.

[magic]

This scenario shouldn't be possible.

When interrupts are enabled and multiple IRQs are waiting for execution, the one with lower number will be executed first. In all AVRs I have seen, input capture comes before overflow. Check the list of interrupt vectors in mega2560 datasheet to confirm.

[Kleinstein]

Not using repeated reads and no interrupts still leaved the task to decide if a new capture and an overflow occur in the same loop to decide if capture data are before or after the overflow. So the repeated reading still need to catch and separate the specific case of capture just before overflow. It is not much more than avoiding the µC internal interrupt handling and do essentially the same tasks under software control.
Usually the hardware interrupts should be faster, when watching for more than 1 signal.

[HendriXML]

That would be great. I don't know whether the following is possible. Have a compare A and a compare B event set up. One increments HiWordA, one HiWordB.
Which HiWord to take is dependent on the captured counter value. That is shown below the X axis. This solution doesn't use the overflow flag anymore.
I know the compare A and B stuff can work, but can they work together with capture events?

[Kleinstein]

The system with 2 counters for software extension should also work. The counter used for large ICP values could still use the overflow interrupt.
Compare A and B work together with input capture - unless used as the counter upper limit, the ICP function is always active - there is no specific way to turn it off (except using the register as upper limit). One normally just does not look at the ICP registers.