Fluke PM6666 counter freezing after 3 seconds from turning on.

[thinkfat]

Don't go too crazy on the resistors. If the counter doesn't boot any more, that's a clear signal to stop what you're doing. If you damage the ASIC you have a paperweight.

The first attempt more or less proves that there is a problem either with the output driver or with the CPU (IC105) loading the clock signal too much. I'd now try recovering the clock with a buffer only for the GPIB card.

[0xdeadbeef]

Could be something totally unrelated though. Like due to aging of some component the GPIB card takes longer to boot but the CPU already started the I2C communication before the card is able to properly react on that. So it missed the request to answer and thus the CPU is blocked since it waits for an answer. Or something like that.

[thinkfat]

Could be something totally unrelated though. Like due to aging of some component the GPIB card takes longer to boot but the CPU already started the I2C communication before the card is able to properly react on that. So it missed the request to answer and thus the CPU is blocked since it waits for an answer. Or something like that.

The GPIB card pulls the SCL line low doing "clock stretching". The CPU is blocked waiting for the I2C bus to return to idle. It continues normally once you disconnect the SCL line to the GPIB card.

[0xdeadbeef]

Well, it's not really clock stretching in its usual meaning since it obviously continues forever. And one reason for this might be that the card's I2C state machine is confused because of timing issues.

[thinkfat]

I think so as well. What I imagine happening is that the I2C part of the GPIB CPU has a hardwired state machine driven by SCL. As soon as it receives its slave address it pulls SCL low to stall the master, then issues an irq. The processor part would then act on the interrupt and carry out whatever reaction is programmed. This part does not happen and so the bus stays blocked.

[0xdeadbeef]

My main point was: if this happens in lots of PM6666 counters (and it seems it does) and it is a process of deterioration (which it is in my case), there should be a common, typically aging related root cause. And a change of timing is something that caused similar problems in other cases. Programmers tend to rely on the timing that they saw during development. Things like performing a worst case analysis and explicitly waiting for the maximum time before trying to communicate with another IC is rarely done. Also adding an emergency break condition from a wait loop (that waits for some HW acknowledge) is something that is frequently forgotten.

[thinkfat]

There is probably an aging related defect in the clock output, it looks like the high side mosfet of the output driver has deteriorated (electron migration?). That's why you see it improve with the pullup resistor connected. Would be interesting to see the clock waveform in your counter.

[0xdeadbeef]

Didn't look into the schematics but is there actually a highside at all? That's not necessarily the case. Many designs just use a lowside output with passive pullup (i.e. "open drain"). Doesn't matter if the active edge is the falling edge - which it usually is.

[pizzigri]

There is probably an aging related defect in the clock output, it looks like the high side mosfet of the output driver has deteriorated (electron migration?). That's why you see it improve with the pullup resistor connected. Would be interesting to see the clock waveform in your counter.

So essentially this is non repairable, at least not easily. Are you asking Oxdeadbeef for his waveform, or me for the original 10 mhz  clock from MTCXO?
Can the 500 ohm resistor create a potential damage or speed up deterioration ? I was going to Implement it ito increase reliability but I may actually get the opposite!

[thinkfat]

There is probably an aging related defect in the clock output, it looks like the high side mosfet of the output driver has deteriorated (electron migration?). That's why you see it improve with the pullup resistor connected. Would be interesting to see the clock waveform in your counter.

So essentially this is non repairable, at least not easily. Are you asking Oxdeadbeef for his waveform, or me for the original 10 mhz  clock from MTCXO?
Can the 500 ohm resistor create a potential damage or speed up deterioration ? I was going to Implement it ito increase reliability but I may actually get the opposite!

I wanted to see the 5MHz clock from his counter.

The 500 Ohm resistor is not really a good solution. I meant it mostly to check a theory of mine. Buffering the signal would certainly be better, but we still haven't found the actual reason for the hang. We know what causes it but not why.

You mentioned the GPIB card worked fine in another counter. Did you manage to use it for PC communication or was it just that it didn't hang the other PM6669 during boot?

Since we don't have a lot of options here, it's best to play the elimination game to isolate the fault. The IC105 (CPU) is socketed as well? Could you pull it and check if the 5MHz looks better then?

[khs]

Now I've learned a device can stop I²C communication by holding SCK low.

So here my suggestion:
What happens when we insert a SCL buffer between the SCL output of the microprocessor and the GPIB controller , so the I²C bus cannot stopped by the GPIB device holding SCL low?

Not to destroy the SCL input/output of the GPIB device I would insert about 100 Ohm between the output of the buffer and the SCL input/output of the GPIB device.

 

[thinkfat]

Now I've learned a device can stop I²C communication by holding SCK low.

So here my suggestion:
What happens when we insert a SCL buffer between the SCL output of the microprocessor and the GPIB controller , so the I²C bus cannot stopped by the GPIB device holding SCL low?

Not to destroy the SCL input/output of the GPIB device I would insert about 100 Ohm between the output of the buffer and the SCL input/output of the GPIB device.

 

Violation of the I2C bus protocol. A slave must be able to stall the master when it needs more time for processing the access. This is called "clock stretching". If the GPIB card stops the counter from working I'd just remove it. It doesn't work anyway so it doesn't make any sense to have it.

[0xdeadbeef]

Dunno when/if I'll find the time but I'll consider it. Is this Pin19 of the IC marked L5899MC?

[pizzigri]

Well it's pin 19 of the custom ASIC labeled 0Q0501P, but yes, it's the same chip.
It's interesting to note markings, this IC was built in week 34 of 1997 in Nijmegen Holland, then sent to Taiwan for assembly and again tested in Holland.

[pizzigri]

Violation of the I2C bus protocol. A slave must be able to stall the master when it needs more time for processing the access. This is called "clock stretching". If the GPIB card stops the counter from working I'd just remove it. It doesn't work anyway so it doesn't make any sense to have it.

Thinkfat, therefore I think that this will place a nice nail in the coffin. I am wasting too much of the precious time of all EEVB members, and I am extremely grateful for all your help, at this point I will just remove the GPIB card for good and close the meter.

Maybe I'll use the option connector on the M/B as convenient way to power a OCXO inside the unit, simply routing the output externally using a 4cm BNC patch cable, the meter will see it as a External ref, and that's it.
But it has been a great ride, and I have learned a lot!

Just for info, I will remove the CPU once I get back home and see what the clock looks like without it...  All my best, Franco

[0xdeadbeef]

Did some quick measurements. Currently, the counter stops reacting to key presses after ~3s about every 2nd start.
So I did a measurement while frozen and while running. They look the same to me though.
I didn't invest any time to get rid off environmental crosstalk, but the measurements were made with a low capacitance active differential probe and a scope with ~750MHz bandwidth.
[EDIT]
The frequency measurement at the bottom of the screen is using the rising edge (no way to change that AFAIK). The falling edge frequency is pretty stable though if you look at the upper (trigger) frequency.

[thinkfat]

Thanks. If your IC105 is socketed, could you pull it out (carefully) and measure again? The counter will not boot (IC105 is the main CPU) but I'd like to see if that changes anything about the waveform.

PS: if you have a GPIB option, maybe pull it as well and try if the reliability improves?

[0xdeadbeef]

Not today or tomorrow. But I'm not convinced that this signal is the root cause anyway.

[shakalnokturn]

@0xdeadbeef: Does your counter also have the VTCXO option?

@pizzigri: Did your sold PM6669 have the VTCXO or not?

[0xdeadbeef]

I'm not at home right now, but I have that TCXO thing that can be calibrated by pushing some internal button when an external 10MHz reference is connected. Wasn't this called MTCXO?

[shakalnokturn]

Yes sorry my bad it is MTCXO.
Is it only the counters with this option that freeze after 3 seconds?

[0xdeadbeef]

Well, from what was concluded in this thread, the MTCXO option doesn't seem to be the issue but the GPIB option.

[pizzigri]

My PM6666 had the MTCXO option inside, the PM6669 a standard oscillator. But, I used also an external ref and it froze all the same. It only locks up with the GPIB option.

[shakalnokturn]

A little far fetched maybe...
What if the main code jams on an unexpected situation when GPIB is present due to a defect on MTCXO?
I'm thinking of a bad EEPROM or corrupt data, would be curious to know if keeping the temperature sensor well above or below ambient temperature (most accessed compensation values) has any effect.

[Ordinaryman1971]

I guess you can easily check it...  Pizzigri has working PM6669... that would be actually interesting... he didn't mention if he moved the oscillator over to the other machine..