Author Topic: Why won't it go wrong again?  (Read 8141 times)

0 Members and 1 Guest are viewing this topic.

Offline KJDS

  • Super Contributor
  • ***
  • Posts: 2442
  • Country: gb
    • my website holding page
Why won't it go wrong again?
« on: November 25, 2015, 04:22:45 pm »
I hate intermittent faults
I hate intermittent faults
I hate intermittent faults
I hate intermittent faults
I hate intermittent faults
I hate intermittent faults

especially ones that appear to have cured themselves, but I just know that if I sell the unit then it will come back the moment the customer turns it on

I hate intermittent faults
I hate intermittent faults
I hate intermittent faults

Offline AndyC_772

  • Super Contributor
  • ***
  • Posts: 3610
  • Country: gb
  • Professional design engineer
    • Cawte Engineering | Reliable Electronics
Re: Why won't it go wrong again?
« Reply #1 on: November 25, 2015, 04:48:22 pm »
I hate intermittent faults too
I hate ntermittent faults too
I hate intermittent faults too
I hate intermttent faults too
I hate intermittent faults too
I hate intermittent faults too

Offline nfmax

  • Super Contributor
  • ***
  • Posts: 1047
  • Country: gb
Re: Why won't it go wrong again?
« Reply #2 on: November 25, 2015, 04:50:14 pm »
Ah yes. You need to ensure the Critical Job Detector Circuit is active for the fault to occur. Been there, done that
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15388
  • Country: za
Re: Why won't it go wrong again?
« Reply #3 on: November 25, 2015, 04:57:27 pm »
ii
 

Offline Cubdriver

  • Supporter
  • ****
  • Posts: 2872
  • Country: us
  • Nixie addict
    • Photos of electronic gear
Re: Why won't it go wrong again?
« Reply #4 on: November 25, 2015, 05:26:23 pm »
It will go wrong again.  It just won't do so when you have it apart or are otherwise in a position to fix or troubleshoot it.  The random fault generator has built in sensors that disable it if tools are too close or the enclosure is open and the device under observation by a knowledgeable individual.

 |O

-Pat
If it jams, force it.  If it breaks, you needed a new one anyway...
 

Offline AF6LJ

  • Supporter
  • ****
  • Posts: 2903
  • Country: us
Re: Why won't it go wrong again?
« Reply #5 on: November 25, 2015, 05:30:18 pm »
You have to know how to work the intermittent.
Sitting there, watching it won't make it fail.
Sue AF6LJ
Test Equipment Addict, And Proud Of It.
 

Online xrunner

  • Super Contributor
  • ***
  • Posts: 4775
  • Country: us
  • hp>Agilent>Keysight>?
Re: Why won't it go wrong again?
« Reply #6 on: November 25, 2015, 05:36:05 pm »
I hate intermittent faults
I hate intermittent faults
I hate intermittent faults

Ke p at t un il you su ceed! D n't give p!
I am a Test Equipment Addict (TEA) - by virtue of this forum signature, I have now faced my addiction
 

Offline AndyC_772

  • Super Contributor
  • ***
  • Posts: 3610
  • Country: gb
  • Professional design engineer
    • Cawte Engineering | Reliable Electronics
Re: Why won't it go wrong again?
« Reply #7 on: November 25, 2015, 06:23:06 pm »
Whenever I'm confronted with a problem like this, th6 absolU9& guar*eed wa to fix it i/ to a8&b fqpnUOIUFE 7s8
")b ` ` , wh^ch makes the fault occur every single time, then it's easy to debug it using standard lab equipment.

Offline gibbled

  • Regular Contributor
  • *
  • Posts: 102
  • Country: ca
  • VE7mumble
Re: Why won't it go wrong again?
« Reply #8 on: November 25, 2015, 06:55:59 pm »
Bob Pease book "Troubleshooting Analog Circuits" has a page on Troubleshooting Intermittent Problems.

One section reads:

"Try  to  make something happen. Applying heat  or  cold may give you a clue. Adding
some vibration  or  mechanical shock could cause a marginal connection to open permanently,
thus leading to identification of the problem and its solution."

Great book...
 

Offline Neilm

  • Super Contributor
  • ***
  • Posts: 1455
  • Country: gb
Re: Why won't it go wrong again?
« Reply #9 on: November 25, 2015, 07:08:12 pm »
Just get a senior manager / customer to test - almost guaranteed it will go wrong.

Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe. - Albert Einstein
Tesla referral code https://ts.la/neil53539
 

Offline AF6LJ

  • Supporter
  • ****
  • Posts: 2903
  • Country: us
Re: Why won't it go wrong again?
« Reply #10 on: November 25, 2015, 08:08:33 pm »
Bob Pease book "Troubleshooting Analog Circuits" has a page on Troubleshooting Intermittent Problems.

One section reads:

"Try  to  make something happen. Applying heat  or  cold may give you a clue. Adding
some vibration  or  mechanical shock could cause a marginal connection to open permanently,
thus leading to identification of the problem and its solution."

Great book...


This is good basic advise; sometimes you have to look for that noisy voltage source or flaky logic levels, but you need to be on the ball when it fails in order to do some ten second troubleshooting.

As for intruducing heat;
Two of the best methods are the 100W incandescent bulb and fixture above the device under test or the towel placed over said device.
Every repair tech should have a goose neck desk lamp and one of those forbidden bulbs to insert in it. for this task. This is gentile on the device in question where a heat gun can do more harm than good in some cases.

Above all if you cannot get the problem to reappear by mechanical shock then check those voltage sources for noise and marginal ripple.
Sue AF6LJ
Test Equipment Addict, And Proud Of It.
 

Offline DimitriP

  • Super Contributor
  • ***
  • Posts: 1118
  • Country: us
  • "Best practices" are best not practiced.© Dimitri
Re: Why won't it go wrong again?
« Reply #11 on: November 25, 2015, 08:19:25 pm »
Faults are permanent. Symptoms are intermittent. :)
They are more fun when you are in R&D mode than production.











   If three 100  Ohm resistors are connected in parallel, and in series with a 200 Ohm resistor, how many resistors do you have? 
 

Online Tomorokoshi

  • Frequent Contributor
  • **
  • Posts: 870
  • Country: us
Re: Why won't it go wrong again?
« Reply #12 on: November 25, 2015, 08:51:42 pm »
Will you be going to an EMC lab for evaluation? Even if you aren't, you could run it there and get test data. Budget permitting.

Sweep with a Variac?

Switch various loads on and off on the same circuit?

Flex the board?

As said above, heat and cold. Humidity?
 

Offline grumpydoc

  • Super Contributor
  • ***
  • Posts: 2740
  • Country: gb
Re: Why won't it go wrong again?
« Reply #13 on: November 25, 2015, 09:25:28 pm »
Just sell it then the fault is sure to come back  >:D

You might have to travel to the buyer's location to fix, of course.

Is it anything especially sexy?
 

Offline AF6LJ

  • Supporter
  • ****
  • Posts: 2903
  • Country: us
Re: Why won't it go wrong again?
« Reply #14 on: November 25, 2015, 09:39:39 pm »
I have a reputation for fixing intermittents; fly me over and back and I will fix it.
Sue AF6LJ
Test Equipment Addict, And Proud Of It.
 

Offline KJDS

  • Super Contributor
  • ***
  • Posts: 2442
  • Country: gb
    • my website holding page
Re: Why won't it go wrong again?
« Reply #15 on: November 25, 2015, 10:00:34 pm »
Just sell it then the fault is sure to come back  >:D

You might have to travel to the buyer's location to fix, of course.

Is it anything especially sexy?

It's one of a pile of radio test sets.

I've monitored the supply rails and they aren't moving when the fault is there.

I think now it's fully warmed up the fault has gone, so I'll leave it in a cold part of the house and see what happens in the morning.

Offline fivefish

  • Frequent Contributor
  • **
  • Posts: 440
  • Country: us
Re: Why won't it go wrong again?
« Reply #16 on: November 25, 2015, 10:54:45 pm »
Use the Shaolin Fists of Hammer method.



 

Offline DimitriP

  • Super Contributor
  • ***
  • Posts: 1118
  • Country: us
  • "Best practices" are best not practiced.© Dimitri
Re: Why won't it go wrong again?
« Reply #17 on: November 25, 2015, 11:47:22 pm »
Just sell it then the fault is sure to come back  >:D

You might have to travel to the buyer's location to fix, of course.

Is it anything especially sexy?

It's one of a pile of radio test sets.

I've monitored the supply rails and they aren't moving when the fault is there.

I think now it's fully warmed up the fault has gone, so I'll leave it in a cold part of the house and see what happens in the morning.

Ok, since no one has asked specifically, what make model and symptoms do you have? 
   If three 100  Ohm resistors are connected in parallel, and in series with a 200 Ohm resistor, how many resistors do you have? 
 

Online vk6zgo

  • Super Contributor
  • ***
  • Posts: 5366
  • Country: au
Re: Why won't it go wrong again?
« Reply #18 on: November 26, 2015, 12:29:34 am »
Take it up in a helicopter---works every time!

I had a small monitor which would fail as soon as it went into service in the "News chopper".
On the ground,you could heat it,freeze it,bash it,& it wouldn't fail!

I finally fixed it by the "scattergun" joint rework technique. ;D
 

Offline Radio Tech

  • Frequent Contributor
  • **
  • Posts: 937
  • Country: us
  • KC4UMO Buddy
    • Hobby Forum
Re: Why won't it go wrong again?
« Reply #19 on: November 26, 2015, 12:47:38 am »
And I thought this type of fault was normal, at least most of the stuff I work on are like this.

Offline MarkF

  • Super Contributor
  • ***
  • Posts: 1695
  • Country: us
Re: Why won't it go wrong again?
« Reply #20 on: November 26, 2015, 01:17:10 am »
I don't understand the problem.

    It's job security!     ;D
 

Offline EEVblog

  • Administrator
  • *****
  • Posts: 31346
  • Country: au
    • EEVblog
Re: Why won't it go wrong again?
« Reply #21 on: November 26, 2015, 02:00:39 am »
Murphy.
 

Offline TerraHertz

  • Super Contributor
  • ***
  • Posts: 3707
  • Country: au
  • Why shouldn't we question everything?
    • It's not really a Blog
Re: Why won't it go wrong again?
« Reply #22 on: November 26, 2015, 02:01:15 am »
You just need to understand the psychology of the intermittent fault gremlin. They are wily, hostile and ingenious, but may respond to threats. I find walking into the room with a grim expression and a large sledgehammer in my hand sometimes works.


Darnit. My 'm' key is going interittent.

Collecting old scopes, logic analyzers, and unfinished projects. http://everist.org
 

Offline tonyarkles

  • Regular Contributor
  • *
  • Posts: 116
Re: Why won't it go wrong again?
« Reply #23 on: November 26, 2015, 03:57:28 am »
I do more software debugging than hardware debugging, but I've found this technique seems to be pretty useful: make a list of inputs, and turn them up to 11.

Software-wise, if you have a race condition that occasionally manifests when you've got two threads running... Run 20! A web server that starts having hiccups at 20 requests/second? Hit it with 200 req/sec! A database that occasionally runs out of RAM? Run something alongside it that takes a pile of RAM.

In hardware, it's the same deal. If your inputs are CMOS levels, run them as close to the switching thresholds as possible. Sag your voltage rails. Crank your voltage rails. Add noise to your rails and inputs. Runs your clocks fast. Hot. Cold. 

And write down your hypothesis, along with what experiments you're doing to prove/disprove the hypothesis! Hopefully, you'll figure out what conditions take it from intermittent to consistent.
 

Offline Ian.M

  • Super Contributor
  • ***
  • Posts: 8597
Re: Why won't it go wrong again?
« Reply #24 on: November 26, 2015, 04:46:59 am »
We used to take stubborn intermittants out on a road trip.  Stick it in the van for the round trip while doing a repair pick-up or drop-off for another customer, then carefully ease it back onto the bench and hope the fault has reappeared.
 

Offline calexanian

  • Super Contributor
  • ***
  • Posts: 1878
  • Country: us
    • Alex-Tronix
Re: Why won't it go wrong again?
« Reply #25 on: November 26, 2015, 05:04:26 am »
Ah yes. That moment when you would give just about anything to have your product fail in a nice predictable manner.
Charles Alexanian
Alex-Tronix Control Systems
 

Offline cobbler

  • Supporter
  • ****
  • Posts: 31
  • Country: us
Re: Why won't it go wrong again?
« Reply #26 on: November 26, 2015, 07:18:25 am »
Is it plugged in?
 

Offline Zucca

  • Supporter
  • ****
  • Posts: 2620
  • Country: it
  • EE meid in Itali
Re: Why won't it go wrong again?
« Reply #27 on: November 26, 2015, 07:24:19 am »
I have a reputation for fixing intermittents; fly me over and back and I will fix it.

Gentle women hands can do miracles...
Can't know what you don't love. St. Augustine
Can't love what you don't know. Zucca
 

Offline Rerouter

  • Super Contributor
  • ***
  • Posts: 4616
  • Country: au
  • Question Everything... Except This Statement
Re: Why won't it go wrong again?
« Reply #28 on: November 26, 2015, 07:31:45 am »
Few things i normally try,

Hit it, repeatably,
Get out a hair-drier and heat up the box to something quite uncomfortable
Add some ground, or supply resistance for its power supply
Load down its outputs
provide a pull up or pull down on inputs or outputs
walk away yelling,
quickly race back and catch the thing off guard
pick up, violently shake object and event bang on bench
Call working, bill for time.
 

Offline crispy_tofu

  • Super Contributor
  • ***
  • Posts: 1124
  • Country: au
Re: Why won't it go wrong again?
« Reply #29 on: November 26, 2015, 07:40:58 am »
I usually give my broken electronics some encouraging words... it's worked a few times!  :-+
 

Offline Paul Moir

  • Frequent Contributor
  • **
  • Posts: 894
  • Country: ca
Re: Why won't it go wrong again?
« Reply #30 on: November 26, 2015, 07:50:48 am »
My favourite are the bugs exposed by human factors.  You have some process that works just fine, but then some new operator comes along and things start failing intermittently.   It's a lesson in humility to sit there watching what they do to expose a bug you created because you couldn't conceive of them doing it "that way" and what the consequences were.
** Bonus humility for creating some standard operating procedures which you think will prevent the bug, but then learn it's still present.   Extra bonus humility for blaming the operator (even if only in your mind) for not following the SOPs when in fact they did. **

Electronics and mechanics are easy.  Just regularly approach them with a screwdriver.  Let them know you are not afraid of taking them apart! :)
 

Offline crispy_tofu

  • Super Contributor
  • ***
  • Posts: 1124
  • Country: au
Re: Why won't it go wrong again?
« Reply #31 on: November 26, 2015, 07:52:52 am »
My favourite are the bugs exposed by human factors.  You have some process that works just fine, but then some new operator comes along and things start failing intermittently.   It's a lesson in humility to sit there watching what they do to expose a bug you created because you couldn't conceive of them doing it "that way" and what the consequences were.
** Bonus humility for creating some standard operating procedures which you think will prevent the bug, but then learn it's still present.   Extra bonus humility for blaming the operator (even if only in your mind) for not following the SOPs when in fact they did. **

Electronics and mechanics are easy.  Just regularly approach them with a screwdriver.  Let them know you are not afraid of taking them apart! :)

Reminds me of the 'Antennagate' 'You're holding it wrong' fiasco...  ;D
 

Offline Paul Moir

  • Frequent Contributor
  • **
  • Posts: 894
  • Country: ca
Re: Why won't it go wrong again?
« Reply #32 on: November 26, 2015, 08:03:52 am »
That's exactly it Mr. Crispy.  If they had only included a little piece of paper that said "For your proper Apple(C) IPhone(C) experience(C), hold your phone like this:" then you could conceivably blame the user.  You would know in your heart it was really your fault for not providing a robust design, but you can use a SOP to weasel your way out of it.
 

Online Gribo

  • Frequent Contributor
  • **
  • Posts: 456
  • Country: ca
Re: Why won't it go wrong again?
« Reply #33 on: November 26, 2015, 08:13:05 am »
Fluorescent lamps are also useful in such cases, mainly as a noise source at 50 and 100Hz .
I am available for freelance work.
 

Offline KJDS

  • Super Contributor
  • ***
  • Posts: 2442
  • Country: gb
    • my website holding page
Re: Why won't it go wrong again?
« Reply #34 on: November 26, 2015, 07:43:33 pm »
I forgot to put it somewhere cold last night, so it's been fine whenever I've tested it today. It's now in a cold room, so will see what it does in the morning.


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf