Author Topic: Things you hope you don't hear...  (Read 14963 times)

0 Members and 1 Guest are viewing this topic.

Offline aargee

  • Frequent Contributor
  • **
  • Posts: 726
  • Country: au
Things you hope you don't hear...
« on: April 05, 2017, 06:50:38 am »
So, I do a lot of phone support for field engineers, as somewhat of a grey-beard type (without the facial hair)... the most knee knocking thing you don't want to hear in a critical phase of instruction is..

Field guy: "Uh-oh ... " 
to which the conversation quickly escalates to..
Me: "Uh-oh? .. What the (insert expletive) do you mean "Uh-oh?"

Nothing like that conversation to send your imagination off on a riotous romp.
Not easy, not hard, just need to be incentivised.
 

Offline NANDBlog

  • Super Contributor
  • ***
  • Posts: 4485
  • Country: nl
Re: Things you hope you don't hear...
« Reply #1 on: April 05, 2017, 07:16:45 am »
My favorite phone call quote, from a service technician:
"There was a small fire, but it seems to be working now"
 
The following users thanked this post: Kilrah

Offline gnif

  • Administrator
  • *****
  • Posts: 1119
  • Country: au
Re: Things you hope you don't hear...
« Reply #2 on: April 05, 2017, 07:23:04 am »
I had a computer "Technician" try to tell me that my sound card didn't work because my powered speakers were not impedance matched with the card :palm:.
HostFission - Full Server Monitoring and Management Solutions.
https://hostfission.com/
https://twitter.com/HostFission

I volunteer my time to manage this server, if you would like to support this work I have a patreon here:
https://www.patreon.com/gnif
 

Offline Brumby

  • Supporter
  • ****
  • Posts: 9415
  • Country: au
Re: Things you hope you don't hear...
« Reply #3 on: April 05, 2017, 08:00:03 am »
I had a computer "Technician" try to tell me that my sound card didn't work because my powered speakers were not impedance matched with the card :palm:.

Ahhh .... hmmmm ...

Nope - I'm lost for words.
 

Offline grumpydoc

  • Super Contributor
  • ***
  • Posts: 2682
  • Country: gb
Re: Things you hope you don't hear...
« Reply #4 on: April 05, 2017, 08:19:50 am »
I had a computer "Technician" try to tell me that my sound card didn't work because my powered speakers were not impedance matched with the card :palm:.

Ahhh .... hmmmm ...

Nope - I'm lost for words.


A little learning is a dangerous thing ;
Drink deep, or taste not the Pierian spring :
There shallow draughts intoxicate the brain,
And drinking largely sobers us again. [Pope]

Although a little learning combined with over enthusiasm/confidence +/- a seasoning of Dunning-Kruger is the real killer  >:D
 

Offline Rerouter

  • Super Contributor
  • ***
  • Posts: 4402
  • Country: au
  • Question Everything... Except This Statement
Re: Things you hope you don't hear...
« Reply #5 on: April 05, 2017, 08:29:01 am »
My favorite is when they straight up ignore the carefully written instructions and then call up in a mood where they refuse to check themself and just say "yep already done that"

Turns out the new model of a vehicle had a similar plug to one of our looms, and he had spent 2 hours cutting our looms plug to hell to make it fit and blow up a $2000 controller, even though the instructions never made mention of it and we sent the kits up labeled and already fitted in the correcg location.

The vehicle dealership then tried to bill us for the smell if burnt electronics that needed to be cleaned from the vehicle.
 

Offline george graves

  • Super Contributor
  • ***
  • Posts: 1259
  • Country: us
Re: Things you hope you don't hear...
« Reply #6 on: April 05, 2017, 08:35:39 am »
The worst is....

Question: Ok, well.... Check part A, and then part B. and report back.

Answer: "Yep part A is fine"

WHAT ABOUT PART B!?!?!?  Do we have to do this dance every single time?!?!?

Offline Halcyon

  • Super Contributor
  • ***
  • Posts: 3735
  • Country: au
Re: Things you hope you don't hear...
« Reply #7 on: April 05, 2017, 08:53:41 am »
I had a computer "Technician" try to tell me that my sound card didn't work because my powered speakers were not impedance matched with the card :palm:.

One of the better ones I've heard.

My VDSL2 modem blew up during a storm not long ago (it still powered on and appeared to do everything it should, except get DSL line sync). Confirmed it was the modem with both a Fluke network tester and another crappy VDSL modem I had laying around (except I couldn't use that one because it was hard-coded to another ISP and refused to get a WAN DHCP address).

Anyway, phoned up my service provider who shipped a new modem out. No questions asked. However then she said to me "While you're waiting for your new modem to arrive, lets go through some things so you aren't without internet in the mean time..."

Uhhh... the reason I have no internet is because the modem died. Did it not occur to her that if it was just a software glitch or something else I could fix, that I wouldn't be requiring a new modem!?

She proceeded to try and get me to do factory resets etc... etc... I politely stopped her and explained to her I was a network engineer in my past career and understood IT very well and that everything what she was about to suggest to me (including resetting the software to default config) has already been tried and tested and that it was a hardware issue.

Her reply was "What do you mean software?........ What lights are on at the moment?". I think I even said to her, "You know, software, that's running on the modem?... The configuration you're trying to get me to reset?... The thing where you stick the pen in the hole and such?".

 :palm:

Just ship me my damn modem!
« Last Edit: April 05, 2017, 08:57:52 am by Halcyon »
 

Offline grumpydoc

  • Super Contributor
  • ***
  • Posts: 2682
  • Country: gb
Re: Things you hope you don't hear...
« Reply #8 on: April 05, 2017, 09:17:46 am »
Her reply was "What do you mean software?........ What lights are on at the moment?". I think I even said to her, "You know, software, that's running on the modem?... The configuration you're trying to get me to reset?... The thing where you stick the pen in the hole and such?".

 :palm:

Just ship me my damn modem!
First line support often have little domain knowledge, their main ability/task is to run down a flow-chart of standard questions & responses.

Thing is - it works for most punters who have even less knowledge about how stuff works.
 
The following users thanked this post: TheDane

Offline tautech

  • Super Contributor
  • ***
  • Posts: 16372
  • Country: nz
  • Taupaki Technologies Ltd. NZ Siglent Distributor
    • Taupaki Technologies Ltd.
Re: Things you hope you don't hear...
« Reply #9 on: April 05, 2017, 09:23:19 am »
Her reply was "What do you mean software?........ What lights are on at the moment?". I think I even said to her, "You know, software, that's running on the modem?... The configuration you're trying to get me to reset?... The thing where you stick the pen in the hole and such?".

 :palm:

Just ship me my damn modem!
First line support often have little domain knowledge, their main ability/task is to run down a flow-chart of standard questions & responses.

Thing is - it works for most punters who have even less knowledge about how stuff works.
Yep, I get the same standard BS when I call in a ISP network fault. Please reset your modem, use these settings....  ::)
Yep, done that, please check my file about the ongoing faults I've reported.
Oh yes, you do seem to have some trouble with your connection.

You ain't kidding. ::)
« Last Edit: April 05, 2017, 09:25:49 am by tautech »
Avid Rabid Hobbyist
 

Offline Halcyon

  • Super Contributor
  • ***
  • Posts: 3735
  • Country: au
Re: Things you hope you don't hear...
« Reply #10 on: April 05, 2017, 09:25:32 am »
Yep, I get the same standard BS when I call in a ISP network fault. Please reset your modem, use these settings....  ::)

I'm on Australia's wonderful new National Broadband Network. I must have logged maybe 30 faults between August 2016 and now.

I generally only get the response "Wow! You've clearly been through this process before... let me just go ahead and log a fault on the NBN Co. portal for you".

 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 16372
  • Country: nz
  • Taupaki Technologies Ltd. NZ Siglent Distributor
    • Taupaki Technologies Ltd.
Re: Things you hope you don't hear...
« Reply #11 on: April 05, 2017, 09:29:57 am »
Yep, I get the same standard BS when I call in a ISP network fault. Please reset your modem, use these settings....  ::)

I'm on Australia's wonderful new National Broadband Network. I must have logged maybe 30 faults between August 2016 and now.

I generally only get the response "Wow! You've clearly been through this process before... let me just go ahead and log a fault on the NBN Co. portal for you".
Lucky you, you've got VDSL, try DSL1. (1.7 Mbps  >:( )
Avid Rabid Hobbyist
 

Offline Halcyon

  • Super Contributor
  • ***
  • Posts: 3735
  • Country: au
Re: Things you hope you don't hear...
« Reply #12 on: April 05, 2017, 09:57:33 am »
Lucky you, you've got VDSL, try DSL1. (1.7 Mbps  >:( )

I'd love 1.7Mbps if it meant that it wouldn't drop out everyday or just simply loose sync for weeks at a time. Until late last week I had no internet for almost 3 weeks. It's not just a once-off cable fault either, it has happened several times before and I've had this service for 8 months. I've often bitched to gnif about it. He lives near me but he is one of the lucky ones!

I hate you gnif (not really) :-)
« Last Edit: April 05, 2017, 10:00:43 am by Halcyon »
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 30135
  • Country: au
    • EEVblog
Re: Things you hope you don't hear...
« Reply #13 on: April 05, 2017, 11:05:16 am »
Field guy: "Uh-oh ... " 
to which the conversation quickly escalates to..
Me: "Uh-oh? .. What the (insert expletive) do you mean "Uh-oh?"
Nothing like that conversation to send your imagination off on a riotous romp.

Depends what the guy was doing at the time  ;D

 
The following users thanked this post: WattSekunde, SeanB, N2IXK, ManateeMafia, AF6LJ, Dave92F1, Cyberdragon, BrianHG, yuzuha

Offline WattSekunde

  • Regular Contributor
  • *
  • Posts: 79
  • Country: de
Re: Things you hope you don't hear...
« Reply #14 on: April 05, 2017, 11:24:26 am »
Droptest - Checked  :-+
 

Offline Harb

  • Regular Contributor
  • *
  • Posts: 244
  • Country: au
Re: Things you hope you don't hear...
« Reply #15 on: April 05, 2017, 11:30:14 am »
Field guy: "Uh-oh ... " 
to which the conversation quickly escalates to..
Me: "Uh-oh? .. What the (insert expletive) do you mean "Uh-oh?"
Nothing like that conversation to send your imagination off on a riotous romp.

Depends what the guy was doing at the time  ;D



Man that was lucky........missed the morning tea table by about 3"
 

Offline lwatts666

  • Supporter
  • ****
  • Posts: 60
  • Country: au
Re: Things you hope you don't hear...
« Reply #16 on: April 05, 2017, 11:34:23 am »
The (I-NOGO)  Issac Newton Orbital Gravitational Observatory makes its first important discovery. Gravity exists.
 
The following users thanked this post: Dave92F1, TheDane, yuzuha

Offline Harb

  • Regular Contributor
  • *
  • Posts: 244
  • Country: au
Re: Things you hope you don't hear...
« Reply #17 on: April 05, 2017, 11:45:25 am »
Its a bit late to secure it now lol


The causes of the NOAA N-PRIME mishap are summarized below.

Proximate Cause: The NOAA N-PRIME satellite fell because the LMSSC operations team failed to follow procedures to properly configure the TOC, such that the 24 bolts that were needed to secure the TOC adapter plate to the TOC were not installed.
« Last Edit: April 05, 2017, 11:46:59 am by Harb »
 
The following users thanked this post: TheDane

Offline gnif

  • Administrator
  • *****
  • Posts: 1119
  • Country: au
Re: Things you hope you don't hear...
« Reply #18 on: April 05, 2017, 01:05:54 pm »
I hate you gnif (not really) :-)

Hahaha, sorry man, I really do feel for you, I have had one outage since I got NBN, it lasted about 20 minutes, and it wasn't NBN, my ISP had a fault with their CG-NAT DHCP provider. Outside of that I have had 100% consistent 10MB/s download rates.
HostFission - Full Server Monitoring and Management Solutions.
https://hostfission.com/
https://twitter.com/HostFission

I volunteer my time to manage this server, if you would like to support this work I have a patreon here:
https://www.patreon.com/gnif
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 30135
  • Country: au
    • EEVblog
Re: Things you hope you don't hear...
« Reply #19 on: April 05, 2017, 01:20:25 pm »
Hahaha, sorry man, I really do feel for you, I have had one outage since I got NBN, it lasted about 20 minutes, and it wasn't NBN, my ISP had a fault with their CG-NAT DHCP provider. Outside of that I have had 100% consistent 10MB/s download rates.

My current lab ISP (at $400/month) is trying to scare me out of moving to the new NBN connection, because, you know, the lack of service level agreement boogieman
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15181
  • Country: za
Re: Things you hope you don't hear...
« Reply #20 on: April 05, 2017, 07:24:03 pm »
How about the one today. "Can you change the ribbon in the laser printer, it is not printing right".

With the satellite, the guy who put the bolts in was off that day, so the others just carried on without his work being done.
 
The following users thanked this post: AF6LJ, TheDane

Offline Rbastler

  • Frequent Contributor
  • **
  • Posts: 286
  • Country: it
  • Wörk Wörk
    • Rbastlers Blog
Re: Things you hope you don't hear...
« Reply #21 on: April 05, 2017, 07:36:20 pm »
Her reply was "What do you mean software?........ What lights are on at the moment?". I think I even said to her, "You know, software, that's running on the modem?... The configuration you're trying to get me to reset?... The thing where you stick the pen in the hole and such?".

 :palm:

Just ship me my damn modem!
First line support often have little domain knowledge, their main ability/task is to run down a flow-chart of standard questions & responses.

Thing is - it works for most punters who have even less knowledge about how stuff works.
Yep, I get the same standard BS when I call in a ISP network fault. Please reset your modem, use these settings....  ::)
Yep, done that, please check my file about the ongoing faults I've reported.
Oh yes, you do seem to have some trouble with your connection.

You ain't kidding. ::)

Same here.
When I moved in my new appartment, I had some big issues regarding internet. I have to use UPC, since theyre the only one around. The send me a decent Fritzbox modem+router. I installed everything and it worked. So far so good.
Than the problems began. After a update of the firmware nothing worked. The update was made via the web interface andthe file pulled directly from FitzBox'es website. I called then explaining what I did and what I did to undo that. Nothing worked. I told the person on the phone more than once I did waht they told me but they went through with theyre procedure anyway. In the end I was told that I can't update the firmware of my Fritzbox ?!? I call bullshit on that.
In the end I got myself a conceptronics router+modem, because I didn't wanna deal with those people anymore.
It was also a very bad timing for that fail to happen. I just started university and needed internet acess from home,to sign into lecture, do homework and stuff.
http://rbastlerblog.jimdo.com/
Gamma spectrometer works. Now some yellow crystals need regenerating and testing.
 

Offline Red Squirrel

  • Super Contributor
  • ***
  • Posts: 2322
  • Country: ca
Re: Things you hope you don't hear...
« Reply #22 on: April 05, 2017, 07:40:05 pm »
We had sent a tech to one of our sites as there was a power outage and the batteries were getting low.  For some reason that site did not have a proper plug and a transfer switch.  The tech had called us to tell us that but that he'd check is truck and see what he can do. 

The tech calls in shortly after we notice that the rectifiers are back online and he's like "uhhh so I got it working.  You don't want to know what I did, but it works".   I always found that one amusing.  Never did find out what he did but I am pretty sure it involved jumper cables from his truck and opening up the electrical panel.  :-DD  The guy was an electrician so I presume it was safe, just not conventional.  Sometimes when you're in the middle of nowhere and have to get something done you use what you have.
 
The following users thanked this post: AF6LJ

Offline james_s

  • Super Contributor
  • ***
  • Posts: 9880
  • Country: us
Re: Things you hope you don't hear...
« Reply #23 on: April 05, 2017, 07:57:14 pm »
While not exactly engineering, my dentist started to drill out a cavity in one of my teeth and said "oh boy". Turned out to not be a huge deal but the decay was significantly worse than he was expecting. My heart certainly jumped when he said that though.
 

Offline CJay

  • Super Contributor
  • ***
  • Posts: 3423
  • Country: gb
  • M0UAW
Re: Things you hope you don't hear...
« Reply #24 on: April 05, 2017, 08:10:54 pm »
While I was the tech with the uh-oh, I can legitimately roll out the 'I was only following orders' defence.

Called out of hours to a hospital server which was flagging a predictive fail on one disk in the RAID array (I forget the exact disk size, either 36 or 72GB)

It was a HP DL380 G4 which had 3.5" SCSI disks installed and one was indeed flashing amber to indicate predictive failure.

On the phone to the support team I was told they'd down the server for me so I could replace the failed disk (it's not necessary, they're hotswap, it's actually better to hotswap them), I agreed and proceed to boot the thing from a HP utility disk which allows me to check the RAID config, disk serial numbers, model numbers etc. (it wasn't unknown for disk caddies to be re-used by parts brokers and have incorrect markings compared to the actual disk in the caddy)

I found that some genius had configured the disks as a RAID0/JBOD, essentially a bunch of disks with a container spanned across them all to give a disk the size of the sum of parts, in this instance they'd done something I didn't realise was possible and had allocated the first 10GB of all the disks to the JBOD to give a disk size of 40GB for the OS (server 2003 IIRC)

They'd then allocated the rest of the disk space to a RAID5 set to hold the data volume (Might have been SQL Server, possibly Exchange, critical data anyway) so notionally the data volume was fault tolerant but not OS as removal of any one disk from a JBOD will kill it.

So, I explained this to the support team wonk who insisted he knew better than I, after all I was only a lowly out of hours tech (who happened to be HP qualified to work on Superdome, EVA, Alpha etc. etc.) and he was third line who knew all and requested I 'just shut up and replace the disk' or words to that effect, he was however happy for me to hotswap the disk so we booted the machine up, he connected remotely (genius things those ILOs) to 'monitor' my actions.

I protested, he demanded, getting quite stroppy and threatening to escalate it to his management. I suggested he dial their number and put us on a conference call so he did. Once I'd been introduced and explained why I didn't want to do it, his manager also insisted I replace the disk as he trusted his tech.

Obviously I then pulled the disk and watched the machine bluescreen instantly.

After what must have seemed an eternity to the remote parties but a few seconds to me, I asked 'would you like me to fix that for you?" and got a quiet 'yes, please, if you can'.

Turns out every server in the business had been built the same way.
M0UAW
 
The following users thanked this post: TheWelly888, AF6LJ, tautech, Ampera, Kilrah, jimdeane, TheDane, BrianHG, yuzuha, W2NAP

Offline Brumby

  • Supporter
  • ****
  • Posts: 9415
  • Country: au
Re: Things you hope you don't hear...
« Reply #25 on: April 06, 2017, 03:59:55 am »
So much for "hot-swap".
 

Offline Mr.B

  • Supporter
  • ****
  • Posts: 1043
  • Country: nz
Re: Things you hope you don't hear...
« Reply #26 on: April 06, 2017, 04:28:35 am »
So much for "hot-swap".

“Hot-Stop” capability.
Very common in devices configured by morons.
I had some gear at my work once that was configured in a similar way.
We changed infrastructure support partners shortly after discovery.
Time is the overseer of all things.
 
The following users thanked this post: Kilrah

Offline BradC

  • Super Contributor
  • ***
  • Posts: 1649
  • Country: au
Re: Things you hope you don't hear...
« Reply #27 on: April 06, 2017, 04:47:36 am »
Turns out every server in the business had been built the same way.

If its any consolation when I was considerably younger and not quite as experienced I did a similar thing. RAID0 for the OS & RAID1 for the data. Luckily for me it was the first test system in development and 3 days in one of the drives suffered an early life failure. Taught me a valuable lesson with no harm done.

The thing I heard at the end of the phone that I never really wanted to hear was the scream of a fully functioning server room descend into silence when the tech hard selected a redundant UPS selector to the wrong UPS and then opened the output breaker. Still, we all make mistakes.
 

Online noidea

  • Supporter
  • ****
  • Posts: 202
  • Country: au
Re: Things you hope you don't hear...
« Reply #28 on: April 06, 2017, 05:53:32 am »
So, I do a lot of phone support for field engineers, as somewhat of a grey-beard type (without the facial hair)... the most knee knocking thing you don't want to hear in a critical phase of instruction is..

Field guy: "Uh-oh ... " 
to which the conversation quickly escalates to..
Me: "Uh-oh? .. What the (insert expletive) do you mean "Uh-oh?"

Nothing like that conversation to send your imagination off on a riotous romp.
So I'm not a greybeard, more a long grey haired hippy type that spends a big part of his day doing a similar thing. Actually I think it can be more accurately described as "Playing drive a human ROV by playing twenty questions" than technical support but I digress. The products I support are mains connected both single and three phase (240/415VAC) with working DC voltages in the 300-500VDC range with decent sized capacitors in them.

I actually think the worst is when the conversation goes along the lines of this:

Me: "Can you please tell me what the voltage is between testpoint X and Y?"
Field guy: "It's.........." then utter silence and the call drops out
Me: "Oh F$%^" whilst I frantically try to ring back and find out what happened.

So far fingers crossed it's always been sorry the phone went flat or something along those lines, but it does make you think about liability with the way our current society is heading.
 
The following users thanked this post: yuzuha

Offline Halcyon

  • Super Contributor
  • ***
  • Posts: 3735
  • Country: au
Re: Things you hope you don't hear...
« Reply #29 on: April 06, 2017, 11:35:42 am »
My current lab ISP (at $400/month) is trying to scare me out of moving to the new NBN connection, because, you know, the lack of service level agreement boogieman

Run! Run as fast as you can! Unless of course you're like Geoff who happened to win "Node lotto". Although you should be getting fibre to pretty much your door even on NBN where you are? In that case, you should be good.

Worst case, a few hundred bucks of Ubiquiti gear and Geoff will host your connection and beam it down the mountain for you. ;-)

You're right about lack of SLA however. A mate of mine in Cranebrook has 100Mbps (downstream) FTTH, unlimited plan with a large RSP (I won't mention the name because I'm not 100% sure which one it is). But they cut him off temporarily due to "abuse" of his account when he decided to queue up 1200 files to download (no, it wasn't a dodgy torrent or anything like that). Apparently they didn't expect FTTH customers to actually make use of their "unlimited" accounts... funny that.

That said, a few choice words to them over the phone soon got his account reactivated.
« Last Edit: April 06, 2017, 11:41:12 am by Halcyon »
 

Offline gnif

  • Administrator
  • *****
  • Posts: 1119
  • Country: au
Re: Things you hope you don't hear...
« Reply #30 on: April 06, 2017, 01:28:29 pm »
My current lab ISP (at $400/month) is trying to scare me out of moving to the new NBN connection, because, you know, the lack of service level agreement boogieman

Run! Run as fast as you can! Unless of course you're like Geoff who happened to win "Node lotto". Although you should be getting fibre to pretty much your door even on NBN where you are? In that case, you should be good.

Worst case, a few hundred bucks of Ubiquiti gear and Geoff will host your connection and beam it down the mountain for you. ;-)

You're right about lack of SLA however. A mate of mine in Cranebrook has 100Mbps (downstream) FTTH, unlimited plan with a large RSP (I won't mention the name because I'm not 100% sure which one it is). But they cut him off temporarily due to "abuse" of his account when he decided to queue up 1200 files to download (no, it wasn't a dodgy torrent or anything like that). Apparently they didn't expect FTTH customers to actually make use of their "unlimited" accounts... funny that.

That said, a few choice words to them over the phone soon got his account reactivated.

For the record, Geoff is me :P.
HostFission - Full Server Monitoring and Management Solutions.
https://hostfission.com/
https://twitter.com/HostFission

I volunteer my time to manage this server, if you would like to support this work I have a patreon here:
https://www.patreon.com/gnif
 
The following users thanked this post: SeanB

Offline rrinker

  • Super Contributor
  • ***
  • Posts: 1921
  • Country: us
Re: Things you hope you don't hear...
« Reply #31 on: April 06, 2017, 01:55:52 pm »
 Bunch of years back I was working on some large database recovery at a client. They had several racks of Windows servers, plus a big HP mini. There was also a tech from the UPS company working in the data center. There was some maintenance required, so he went to put the UPS in bypass mode. Only he turned the wrong switch, instantly depowered the entire data center. Oops.

 Then there's the other client, who at the time wasn't a client. They had an HP storage array with a failing disk (amber light). The previous support company sent out a tech to replace the drive. The tech CLAIMS he pull the correct drive, but the best thing we can come up with was he pulled a different one that wasn't failed, and the array being configured as RAID 5 was not tolerant of TWO bad disks. They lost their entire Exchange database as well as about 6 other VMs from their ESXi farm. The VMs they got back fairly quickly, but the Exchange data took a month. I happened to be there when the president of the support company was on a call wth the IT directory. He told her, and I quote, that the whole reason for the outage was "a glitch". Needless to say, cleaning up the aftermath of their "glitch" was the last thing that company ever did for this client, and now we are their vendor of choice.



 

Online Kilrah

  • Supporter
  • ****
  • Posts: 1770
  • Country: ch
Re: Things you hope you don't hear...
« Reply #32 on: April 06, 2017, 02:21:50 pm »
While I was the tech with the uh-oh, I can legitimately roll out the 'I was only following orders' defence.
[...]

That is VERY worthy of https://www.reddit.com/r/talesfromtechsupport/ !
 

Offline CJay

  • Super Contributor
  • ***
  • Posts: 3423
  • Country: gb
  • M0UAW
Re: Things you hope you don't hear...
« Reply #33 on: April 06, 2017, 05:04:33 pm »
The tech CLAIMS he pull the correct drive, but the best thing we can come up with was he pulled a different one that wasn't failed, and the array being configured as RAID 5 was not tolerant of TWO bad disks. They lost their entire Exchange database as well as about 6 other VMs from their ESXi farm.

To be fair, even if he did pull the wrong drive that's not a common failure with a 'failing' drive left in the array on a HP, they generally survive and often powering down the connected server(s) then replacing the drive will often get you back to the start even if you've not got the ACU boot CD.

I've seen lots of RAID arrays go down because someone decided reseating a predictive fail drive was a good idea, you'd usually get away with it on a HP but it was absolute poison to some of  the Dell machines of the era and utterly pointless as the machines were under support and usually had four hour parts response.

One of the worst fails I saw was on a partitionable SCSI shelf which had been misconnected to two Dell servers after a weekend data centre move and some genius had decided to 'accept new configuration'.

I managed to get it back by powering it all down, correcting the cabling errors, powering back up, deleting the configured arrays and then recreating the original configuration, it was then simple to get to the data but the ESEUTIL Exchange check took almost a day and a half with ~3000 users champing at the bit to get their email back.

While that ESEUTIL scan was working I'd configured two replacement servers, a new shelf and was restoring data from 24 hours before the 'crash' so there was a fallback position.

The single worst fail I saw was caused by a contractor who reinitialised a ~200 disk EVA5000 from the controller front panel because he 'thought the controller had crashed' (it was a single FC disk which had deadlocked the loop and hung the array, seen it happen 2 maybe 3 times in three years).

That one took six days to snapshot/replicate over a high speed fibre link from their head office.

It was always amusing to 'Identify' the whole array from the EVA management appliance and leave it ID'd, would scare the proverbial out of a lot of people to see ~150 Amber LEDs flashing...

M0UAW
 

Online Kilrah

  • Supporter
  • ****
  • Posts: 1770
  • Country: ch
Re: Things you hope you don't hear...
« Reply #34 on: April 06, 2017, 10:03:20 pm »
To be fair, even if he did pull the wrong drive that's not a common failure with a 'failing' drive left in the array on a HP

Except you can be sure the clueless tech noticed he fucked up and removed the wrong drive, plugged it back in to "fix his mistake" then promptly plugged the "real" bad one out... jsut after a rebuild attempt started upon reconnection of the first one. You now have 2 bad drives and you're SOL.

Any basic human who doesn't really understand the system (which he seems to be) will think the best thing to do is try to cover up/fix his mess ASAP instead of asking for help, and will do that when the correct answer would be to let the thing spend a day rebuilding before touching anything else (and check/prepare your backups so you're ready to restore while it happens, jsut in case...) but that would mean owning up to a mistake when you think there's still a way to get out of it...
« Last Edit: April 06, 2017, 10:16:37 pm by Kilrah »
 
The following users thanked this post: TheDane

Offline CJay

  • Super Contributor
  • ***
  • Posts: 3423
  • Country: gb
  • M0UAW
Re: Things you hope you don't hear...
« Reply #35 on: April 07, 2017, 05:35:44 am »
To be fair, even if he did pull the wrong drive that's not a common failure with a 'failing' drive left in the array on a HP

Except you can be sure the clueless tech noticed he fucked up and removed the wrong drive, plugged it back in to "fix his mistake" then promptly plugged the "real" bad one out... jsut after a rebuild attempt started upon reconnection of the first one. You now have 2 bad drives and you're SOL.
Quote

Ah, yes, I didn't factor in the arse covering
M0UAW
 

Offline Housedad

  • Frequent Contributor
  • **
  • Posts: 512
  • Country: us
Re: Things you hope you don't hear...
« Reply #36 on: April 07, 2017, 06:34:12 am »
"Put him out!  Put him out!  God damn it!!  NOW!!"

What I remember from waking up laying on my stomach gagging on the airway while trying to scream during surgery to fix my spine that was shattered in a elevator accident.  38 years ago
At least I'm still older than my test equipment
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 30135
  • Country: au
    • EEVblog
Re: Things you hope you don't hear...
« Reply #37 on: April 07, 2017, 09:15:25 am »
 :palm:

 
The following users thanked this post: TheDane

Online Kilrah

  • Supporter
  • ****
  • Posts: 1770
  • Country: ch
Re: Things you hope you don't hear...
« Reply #38 on: April 07, 2017, 10:16:54 am »
Damn, was so much hoping he'd be putting the sleeves on, tried the remote again - with no change at all, and a loooong painful scream as a result  >:D

Alas no, jsut a stupid shill  |O
 

Offline HwAoRrDk

  • Frequent Contributor
  • **
  • Posts: 636
  • Country: gb
Re: Things you hope you don't hear...
« Reply #39 on: April 07, 2017, 12:38:27 pm »
I've seen lots of RAID arrays go down because someone decided reseating a predictive fail drive was a good idea, you'd usually get away with it on a HP but it was absolute poison to some of  the Dell machines of the era and utterly pointless as the machines were under support and usually had four hour parts response.

Ugh, that brings back a bad memory of RAID controllers on early '00s Dell servers that were a terrible pain in the arse.

I remember on one PowerEdge 4000-something (I forget which exactly) we'd configured a RAID 5 array using some old 10GB drives as temporary scratch space. But, after a few months one of the drives failed, and it wasn't worth it to get a matching replacement drive (the choice was either pay a king's ransom for a new one from Dell or buy a used one). So we decided to just get rid of that array. Except on this particular RAID controller's BIOS, there was literally no facility to delete an array when it was in degraded state. Dell's tech support confirmed it - such an option didn't exist! :wtf:

A further kick in the balls was that the RAID controller didn't remember its setting for the alarm buzzer, so from that point on, every single time the server was restarted, the alarm would sound (which was bloody loud) and you'd have to go into the RAID controller BIOS and turn the alarm off. We had to run that server for 3 years like that, all because nobody had the common sense foresight to realise someone might want to delete an array when it was in a degraded state. :palm:
 

Online macboy

  • Super Contributor
  • ***
  • Posts: 1981
  • Country: ca
Re: Things you hope you don't hear...
« Reply #40 on: April 07, 2017, 02:57:28 pm »

The tech CLAIMS he pull the correct drive, but the best thing we can come up with was he pulled a different one that wasn't failed, and the array being configured as RAID 5 was not tolerant of TWO bad disks. They lost their entire Exchange database as well as about 6 other VMs from their ESXi farm.

...

One of the worst fails I saw was on a partitionable SCSI shelf which had been misconnected to two Dell servers after a weekend data centre move and some genius had decided to 'accept new configuration'.

I managed to get it back by powering it all down, correcting the cabling errors, powering back up, deleting the configured arrays and then recreating the original configuration, it was then simple to get to the data but the ESEUTIL Exchange check took almost a day and a half with ~3000 users champing at the bit to get their email back.

..

This is why I love ZFS. You can survive a soft double failure like disconnecting the wrong disk. The array (pool) will go down of course, but just plug that disk back in, and the pool will re-silver within minutes, and you will be back in business. You can also reseat a disk, and it won't cause issues because the pool will re-silver that disk back into the pool (while remaining online), fixing new or changed data within seconds or minutes instead of requiring a full offline re-build of the array. You can move disks around at will, and the system will identify their new locations/connections, and adjust accordingly. All data is checksummed so if a data error (e.g. unreadable sector) occurs, even a silent one, the system figures out which combination of primary and redundant data to re-combine into the original file, then attempts to re-write the bad data (which should trigger the disk to reallocate the bad sector). In this way, the simple act of reading data can improve the health of it.

It's really easy to play/experiment with these failure scenarios by creating a pool from several vdevs that are files instead of disks. Then you can create a filesystem or two on the pool, put some test data on there, and start screwing around. My favorite: write random data to a vdev to create silent data corruption and marvel at how the pool remains online, silently heals the data and does it while never delivering a single incorrect byte. Or disconnect a vdev, write some data to the pool, reconnect the vdev, and marvel at how quickly redundancy is restored (instantly for all old data, quickly for new data). I just can't stomach the idea of old school RAID any longer.  This is so far ahead in every way.
 
The following users thanked this post: evb149

Offline CJay

  • Super Contributor
  • ***
  • Posts: 3423
  • Country: gb
  • M0UAW
Re: Things you hope you don't hear...
« Reply #41 on: April 07, 2017, 05:22:45 pm »

Ugh, that brings back a bad memory of RAID controllers on early '00s Dell servers that were a terrible pain in the arse.

I remember on one PowerEdge 4000-something (I forget which exactly) we'd configured a RAID 5 array using some old 10GB drives as temporary scratch space. But, after a few months one of the drives failed, and it wasn't worth it to get a matching replacement drive (the choice was either pay a king's ransom for a new one from Dell or buy a used one). So we decided to just get rid of that array. Except on this particular RAID controller's BIOS, there was literally no facility to delete an array when it was in degraded state. Dell's tech support confirmed it - such an option didn't exist! :wtf:

A further kick in the balls was that the RAID controller didn't remember its setting for the alarm buzzer, so from that point on, every single time the server was restarted, the alarm would sound (which was bloody loud) and you'd have to go into the RAID controller BIOS and turn the alarm off. We had to run that server for 3 years like that, all because nobody had the common sense foresight to realise someone might want to delete an array when it was in a degraded state. :palm:

Hell yes, I remember those too, horrible things, some arcane command line interface to the RAID controller that seemed to be so poorly documented as to be undocumented and they had the amazing 'facility' where it was a crapshoot dependant on which BIOS version was on them as to what 'scrubbing' meant, on one it was checking and cleaning up, on the other it was wiping the array.

The nasty continued a couple of generations up, onto PERC4 I think (maybe PERC5?) and it was again often a crapshoot as to what happened when a disk was replaced, it was possible to get them into a state where it complained bitterly about a disk failing but absolutely and resolutely refused to allow you to replace the damn thing and have it rebuild, the only way to resolve it was to completely trash the system, rebuild the system and restore the data.

More than one occasion where Dell support got confused and had given customers instructions for disk replacement which trashed their arrays, on one occasion I sat with the IT director of an NHS trust as Dell Support talked the IT tech through that process and took down their live webserver.

I developed a deep and abiding love for HP RAID after a short time working on Dell equipment, nowadays I believe the Dell gear is better (and to be fair, it wasn't too bad by the PE2900 and PERC5 SAS controllers).

M0UAW
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 9880
  • Country: us
Re: Things you hope you don't hear...
« Reply #42 on: April 07, 2017, 06:48:40 pm »
"Put him out!  Put him out!  God damn it!!  NOW!!"

What I remember from waking up laying on my stomach gagging on the airway while trying to scream during surgery to fix my spine that was shattered in a elevator accident.  38 years ago

Gah!
*cringe*
 

Online Howardlong

  • Super Contributor
  • ***
  • Posts: 4788
  • Country: gb
Re: Things you hope you don't hear...
« Reply #43 on: April 07, 2017, 07:57:39 pm »
Two quite recent ones at a Wintel hosting provider who is tasked with managing everything up to and including the OS, i.e., SAN, Virtualisation etc.

During a scheduled DR test, we couldn't access the high availability database server at the DR hosting site, but we could access everything else. As the production and hosting sites are on a stretched VLAN we have no visibility of physically where the hoats are. After three hours, I suggested that the symptoms, as far as I could see from verious primary and secondary indications, strongly implied that both nodes were in fact located at the now offline site rather than being geographically disparate as specified in the design. Two minutes later, we had an embarrassing call from the hosting provider.

Only this week, on Monday, we had an outage of about 20 clusters after a scheduled change on some Xsigo fabric by our esteemed hosting provider on a shared SAN that "shouldn't affect us". Most of the clusters failed over and recovered correctly in a matter of seconds, but three didn't because... all those cluster nodes resided on the same physical hosts on the same failed fabric.

Then there was the one many years ago where an erstwhile colleague plugged both redundant PSUs of a production box into the same PDU strip, and a few months down the line the innevitable happened. He learned.
« Last Edit: April 07, 2017, 07:59:59 pm by Howardlong »
 

Online Cerebus

  • Super Contributor
  • ***
  • Posts: 3673
  • Country: gb
Re: Things you hope you don't hear...
« Reply #44 on: April 07, 2017, 09:00:18 pm »
This is why I love ZFS. You can survive a soft double failure like ...

...  This is so far ahead in every way.

Yup, ZFS works very well. The one thing wrong with it is the arcane, random, scattered mess that is the command line interface. ZFS works so well that by the time you have a fault that actually requires any human intervention you've forgotten the peculiarities of the command line interface and have to re-learn it all over again.

I had two servers at home (retired kit from the office) and had one be essentially an identical copy of the the other. At regular intervals each day the live one would take some ZFS snapshots, remotely power up the standby machine, copy the snapshot changes across, bring the filesystems on the standby system up to date and the standby machine would power itself off. This worked faultlessly for ages, including failover tests. Then I needed to make some changes and it took me almost as long as it took when I first set it up just to re-work out how to do all the ZFS stuff. Great filesystem, lousy human interface.
Anybody got a syringe I can use to squeeze the magic smoke back into this?
 

Offline Red Squirrel

  • Super Contributor
  • ***
  • Posts: 2322
  • Country: ca
Re: Things you hope you don't hear...
« Reply #45 on: April 08, 2017, 12:20:23 am »
I use mdadm raid and I've gotten to know the commands pretty well and also made myself a cheat sheet, but I might look at ZFS at some point too.   One thing nice about Linux/command line is that it tends to be easier to automate stuff if you want to, or to write a front end to it.  For some reason though it seems no one in the Linux world seems to want to write front ends for anything or the existing ones are garbage.  Though Linux is so fragmented when it comes to GUI tools, I don't really blame people.   Web based is the way to go IMO though.
 

Offline timb

  • Super Contributor
  • ***
  • Posts: 2528
  • Country: us
  • Pretentiously Posting Polysyllabic Prose
    • timb.us
Re: Things you hope you don't hear...
« Reply #46 on: April 08, 2017, 12:31:40 am »
Her reply was "What do you mean software?........ What lights are on at the moment?". I think I even said to her, "You know, software, that's running on the modem?... The configuration you're trying to get me to reset?... The thing where you stick the pen in the hole and such?".

 :palm:

Just ship me my damn modem!
First line support often have little domain knowledge, their main ability/task is to run down a flow-chart of standard questions & responses.

Thing is - it works for most punters who have even less knowledge about how stuff works.

About 8 or 9 years ago I had an issue with my cable internet, so I called up and, while on hold, went ahead and completely unplugged the modem, as I knew they'd have me do that. So I tell the girl my problem and she proceeds to run a "remote diagnostic" on my modem. I let her. After about 30 seconds she tells me that my modem appears to be connected to the network, so she wants me to reboot my computer. I tell her, "That's very interesting, because my modem is currently unplugged, so I'm curious how you ran a diagnostic on it?"

A bit flustered, she explains that power is sent to certain parts of the modem over the cable line, so even if I've got the power cord unplugged she can still talk to it. I replied, "That's even more interesting, because I've had the modem *completely* disconnected during this entire call..." She goes silent and then asks me to hold.

A minute later I get a level 2 technician who dispatches a worker bee. They end up finding a problem with the signal booster on the pole at the end of my street. I still have no idea what the purpose of lying to the customer about a remote modem diagnostic (which is possible, they can see SNR and signal dB from their terminals) is supposed to accomplish.

Oh, a few months back I was staying at my parents place for a week. One of the DirecTV remotes was having an issue with the power button. I took the remote apart, cleaned it and put it back together, but the issue persisted (I was hoping the carbon contact on that button was just dirty, but it was apparently worn out). So I tell my dad to call them up and they'll send a new remote. He calls and I leave for the grocery store.

45 minutes later I get back and he's *still* on the phone with him. The tech is trying to get him to test *every* freaking button on the remote, in a procedure that takes about 25 seconds per button. Seriously. I'm not making this up or exaggerating. If I hadn't been there I wouldn't believe it myself.

So, I motion to him and he gives me the phone, I politely interrupt the tech and explain that I'm the son, the remote is working except for the power button, blah blah blah and the guy tries to get me to test the buttons! I explain it again, but the guy keeps trying to complete his flow chart...

Finally, losing my patience a bit, I explain that I'm an EE, I've taken the remote apart, the contact on that button is worn, we just need a new remote. So the tech pushes back a bit, asking how we know it's the remote and not the box, the test he's trying to get us to do will tell him (it won't). I tell him that remotes from other boxes in the house work fine on this box, yet this remote won't work on other boxes so it's clearly the remote! Finally, after 10 minutes of wearing him down with logic and a stern voice, he relents and orders a new remote for us. Finally! But I'm not out of the woods just yet...

Then the up-sell starts! He says, "I'm waiting for a confirmation code on the order for the new remote, but hey, while we're waiting let me tell you about this exciting new offer!" Bullshit, he should have that order number right away, it doesn't take 5 minutes. He's basically holding me hostage. So, I have to let him get through his spiel, decline twice and explain why before he'll give me the damn code. Ugh.

It took an hour of my time and their time to get a new remote control that in and off itself costs maybe $1 to make. *Bangs head on desk.*
 
Any sufficiently advanced technology is indistinguishable from magic; e.g., Cheez Whiz, Hot Dogs and RF.
 

Offline blueskull

  • Supporter
  • ****
  • Posts: 12480
  • Country: cn
  • Power Electronics Guy
Re: Things you hope you don't hear...
« Reply #47 on: April 08, 2017, 12:37:46 am »
"This phone call is to tell you your credit card..."
I never heard the complete sentence as I will just hang up at that moment.
I don't even have a credit card, how can anyone tell me my credit card has an issue?
« Last Edit: April 08, 2017, 02:22:43 am by blueskull »
 
The following users thanked this post: SeanB, TiN

Offline evb149

  • Super Contributor
  • ***
  • Posts: 1666
  • Country: us
Re: Things you hope you don't hear...
« Reply #48 on: April 08, 2017, 02:20:39 am »
mdadm based can be handy for a quick setup, maybe a non critical mirroring setup or something.
ZFS is so much better though with several "gotchas" that have popped up about risk factors.
IIRC there's some risk when you turn on deduplication that you'll "overflow" the physical storage and have some issues around that.  Maybe something like having lots of stored files with partially deduplicated contents that fit in your pool but then modifying some of them to now be different and suddenly a file of the same size that just fit on your disc is now not able to be written because it'd make the pool full etc.  Maybe a problem with snapshots etc. too, not sure.
IIRC there are some memory tuning possible problems as well where it uses in some ways / cases a proportional amount of memory to the dataset sizes (not surprising) and anyway you can get into cases where the recommended amount of ram is higher than most boxes actually have (>= 8-16GBy) given the sizes of discs that are easily available.
They never finished the security layer for it unfortunately.
And IIRC there were said to be some vulnerabilities to corruption during the host's processing of data in host RAM, e.g. exacerbated greatly by lack of ECC or whatever and then corruption happening despite the hashed on disc checksumming.  Too bad there don't seem to be "here's a buffer of data, and also here's the checksum for it" APIs all the way from the high level file I/O APIs all the way down to the disc so that it gets checked for corruption as soon as the write is queued and thereafter optionally as a confirming step of the writing and of course with every read operation.

Seems like the best way to set up ZFS for casual use is still a mirror rather than something too fancy with a raidz since less can go wrong no matter how unstable your host / ram or discs are.  You are able to set up a pool including multiple mirrored sets so that's a good way to get simple extra fail safe redundancy and a large pool.
Quote
Virtual devices are specified one at a time on the command line, separated by whitespace. The keywords "mirror"   and "raidz" are   used to distinguish where a group ends and another begins. For example, the following creates two root vdevs, each a   mirror of two disks:
# zpool create mypool mirror da0   da1 mirror da2 da3

I think there might be some potential gotchas about device replacement along the lines of what happens to create problems with other RAID systems.  If you have multiple "identical" discs in a mirror or RAIDZ and one fails and you go to get a replacement "of the same model" or a different model "of the same size" I'm not sure that will always work since due to tiny actual capacity differences between two units (e.g. bad sector table size or very slightly different formatted capacity) the resilvering could fail to find enough space on the replacement device.  Seems like that might be an easy way to get any existing mirror drive to become bad as well just due to a small error that does not exist on the paired mate.  I wonder if it only cares about the "used storage" on the drives so if you leave at least say 5% free then you're OK.


The problem with GUI front ends is that almost nobody wants to run a windowing system on their server, so as you suggested a web based GUI would be particularly effective since you could do that over ssh / https or whatever and then manage the server's disc stuff from another machine.  Maybe freenas has some nice zfs admin tools these days, IIRC they may have been working on such way back when they started supporting ZFS.



I use mdadm raid and I've gotten to know the commands pretty well and also made myself a cheat sheet, but I might look at ZFS at some point too.   One thing nice about Linux/command line is that it tends to be easier to automate stuff if you want to, or to write a front end to it.  For some reason though it seems no one in the Linux world seems to want to write front ends for anything or the existing ones are garbage.  Though Linux is so fragmented when it comes to GUI tools, I don't really blame people.   Web based is the way to go IMO though.
 

Offline Brumby

  • Supporter
  • ****
  • Posts: 9415
  • Country: au
Re: Things you hope you don't hear...
« Reply #49 on: April 08, 2017, 03:09:48 am »
Her reply was "What do you mean software?........ What lights are on at the moment?". I think I even said to her, "You know, software, that's running on the modem?... The configuration you're trying to get me to reset?... The thing where you stick the pen in the hole and such?".

 :palm:

Just ship me my damn modem!
First line support often have little domain knowledge, their main ability/task is to run down a flow-chart of standard questions & responses.

Thing is - it works for most punters who have even less knowledge about how stuff works.

About 8 or 9 years ago I had an issue with my cable internet, so I called up and, while on hold, went ahead and completely unplugged the modem, as I knew they'd have me do that. So I tell the girl my problem and she proceeds to run a "remote diagnostic" on my modem. I let her. After about 30 seconds she tells me that my modem appears to be connected to the network, so she wants me to reboot my computer. I tell her, "That's very interesting, because my modem is currently unplugged, so I'm curious how you ran a diagnostic on it?"

A bit flustered, she explains that power is sent to certain parts of the modem over the cable line, so even if I've got the power cord unplugged she can still talk to it. I replied, "That's even more interesting, because I've had the modem *completely* disconnected during this entire call..." She goes silent and then asks me to hold.

A minute later I get a level 2 technician who dispatches a worker bee. They end up finding a problem with the signal booster on the pole at the end of my street. I still have no idea what the purpose of lying to the customer about a remote modem diagnostic (which is possible, they can see SNR and signal dB from their terminals) is supposed to accomplish.

Oh, a few months back I was staying at my parents place for a week. One of the DirecTV remotes was having an issue with the power button. I took the remote apart, cleaned it and put it back together, but the issue persisted (I was hoping the carbon contact on that button was just dirty, but it was apparently worn out). So I tell my dad to call them up and they'll send a new remote. He calls and I leave for the grocery store.

45 minutes later I get back and he's *still* on the phone with him. The tech is trying to get him to test *every* freaking button on the remote, in a procedure that takes about 25 seconds per button. Seriously. I'm not making this up or exaggerating. If I hadn't been there I wouldn't believe it myself.

So, I motion to him and he gives me the phone, I politely interrupt the tech and explain that I'm the son, the remote is working except for the power button, blah blah blah and the guy tries to get me to test the buttons! I explain it again, but the guy keeps trying to complete his flow chart...

Finally, losing my patience a bit, I explain that I'm an EE, I've taken the remote apart, the contact on that button is worn, we just need a new remote. So the tech pushes back a bit, asking how we know it's the remote and not the box, the test he's trying to get us to do will tell him (it won't). I tell him that remotes from other boxes in the house work fine on this box, yet this remote won't work on other boxes so it's clearly the remote! Finally, after 10 minutes of wearing him down with logic and a stern voice, he relents and orders a new remote for us. Finally! But I'm not out of the woods just yet...

Then the up-sell starts! He says, "I'm waiting for a confirmation code on the order for the new remote, but hey, while we're waiting let me tell you about this exciting new offer!" Bullshit, he should have that order number right away, it doesn't take 5 minutes. He's basically holding me hostage. So, I have to let him get through his spiel, decline twice and explain why before he'll give me the damn code. Ugh.

It took an hour of my time and their time to get a new remote control that in and off itself costs maybe $1 to make. *Bangs head on desk.*

I've had the same myself.  It's very frustrating when the bull they roll out just does not fir the symptoms.

A slightly different experience of mine was a couple of years ago When I was trying to find out about broadband availability for an address that a friend was considering moving to.  I had done some basic research and had a little bit of an idea of what was in the area before I called and asked.  I was told "No" - but then I asked an impossible question: "Why?"

The response was underwhelming.

Over a period of 5 days, it took me four calls, several transfers and a total of nearly 3 hours to finally find someone that was prepared to talk to me - and I'm sure that only happened because I was asking technically answerable questions.  In the end I got an answer.  It was still "No", but the reasoning made sense.  (It wasn't so much a technical impossibility, but one of administrative inertia.)

My friend did eventually move, but to a different property - one that already had an Optus coax drop to the house.  That was an easy one to answer.
« Last Edit: April 08, 2017, 03:11:24 am by Brumby »
 

Online rdl

  • Super Contributor
  • ***
  • Posts: 2676
  • Country: us
Re: Things you hope you don't hear...
« Reply #50 on: April 08, 2017, 04:09:30 am »
I set up a little FreeNAS machine not too long ago. It works pretty well, so far no problems. It's managed through a web interface. The OS runs off a couple of flash drives and the data is on two WD Blue 2TB drives. I've been happy with it so far and was able to retire yet another Windows based machine.
 

Offline radar_macgyver

  • Frequent Contributor
  • **
  • Posts: 386
  • Country: us
Re: Things you hope you don't hear...
« Reply #51 on: April 08, 2017, 05:01:43 am »
Then the up-sell starts! He says, "I'm waiting for a confirmation code on the order for the new remote, but hey, while we're waiting let me tell you about this exciting new offer!" Bullshit, he should have that order number right away, it doesn't take 5 minutes. He's basically holding me hostage. So, I have to let him get through his spiel, decline twice and explain why before he'll give me the damn code. Ugh.

Much as everyone hates Comcast, I found that going to their brick-and-mortar office is loads better than trying to call them about anything. Calling always involves guilt-trips, up-sell and (if a service is discontinued) getting handed off to 'customer retention'. By contrast, the people at the local offices are courteous, and seem to actually care about getting what you need.


ZFS: whoa, where do I start? After accidentally hosing a hardware raid setup for the last time (Coraid, grumble grumble. They were OK when they started out, then became a bunch of money grubbers who charge huge sums for firmware updates. Also, I really hate Plan 9), I got a new Supermicro 2P chassis and threw CentOS + ZFS on Linux. Started with a raidz2 array of eight 3 TB SAS nearline disks, then added another eight 6 TB disks. With an SSD slog device, NFS just flies. I re-purposed the old Coraid hardware to serve as a backup storage (also running ZFS), and snapshots + send/receive make incremental backups a breeze. I recently set up another similar server, with eight 4 TB disks. Testing with bonnie++ revealed about 4 GB/s write speed, and about 5.5 GB/s read speed. Not too shabby for $5k of hardware. I got around the hard-to-remember command line syntax by writing some shell scripts for common tasks.
 

Offline tatus1969

  • Super Contributor
  • ***
  • Posts: 1231
  • Country: de
  • Resistance is futile - We Are The Watt.
    • keenlab
Re: Things you hope you don't hear...
« Reply #52 on: April 08, 2017, 11:14:06 am »
The thing where you stick the pen in the hole and such?".
You can be glad that she didn't report you for sexual harassment :o
We Are The Watt - Resistance Is Futile!
 

Offline Avacee

  • Supporter
  • ****
  • Posts: 297
  • Country: gb
Re: Things you hope you don't hear...
« Reply #53 on: April 09, 2017, 09:13:40 am »
Once had a clearly nervous support tech ask "How do you undelete the database?"  :-//
The idiot had managed to run "rm -rf /" on a client's server whilst logged in as root.  |O
It was a doctor's surgery and 2 months medical and financial records vanished as the person who did the backups was on maternity leave   :palm:
I had them pull the plug and ship it. The idiot then turned up with it and proudly said "I re-installed the OS to save you some time" - Yep, some of the database's disk sectors had been overwritten  |O |O |O |O |O |O.
At least I never had to work with that particular idiot again.

At the other end of the scale was an incident where I wanted to hear something.
I was working late in the server room when two firemen in breathing apparatus came in.
Turns out there was a fire and they were checking all the rooms and there wasn't a siren in the server room and the servers drowned out the ones outside :palm:
A lot of brown stuff hit the whirly thing  :-DD
 

Offline grumpydoc

  • Super Contributor
  • ***
  • Posts: 2682
  • Country: gb
Re: Things you hope you don't hear...
« Reply #54 on: April 09, 2017, 09:55:48 am »
Once had a clearly nervous support tech ask "How do you undelete the database?"  :-//
The idiot had managed to run "rm -rf /" on a client's server whilst logged in as root.  |O

There's a story from the 80's about doing this to a running server and recovering  :phew: everything without powering the system down.

http://www.lug.wsu.edu/node/414
« Last Edit: April 09, 2017, 09:59:55 am by grumpydoc »
 
The following users thanked this post: daveshah

Offline Avacee

  • Supporter
  • ****
  • Posts: 297
  • Country: gb
Re: Things you hope you don't hear...
« Reply #55 on: April 09, 2017, 10:12:24 am »
That's an impressive recovery - thanks for the link.  :-+
Confess that would have been beyond me then and definitely is now :p
Moral: You can never have too many backups - only too few.
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15181
  • Country: za
Re: Things you hope you don't hear...
« Reply #56 on: April 09, 2017, 03:04:44 pm »
Reminds me of the time I heard a muffled bang outside. Open the door, and see the bomb squad outside, looking at the cloud of cement dust wafting up. Our plumber had offloaded the bag of cement mix to dix a leaking pipe and reinstate the wall, and left it outside while he went off to park the van away from there. In the 15 minute interim some paranoid called the bomb squad and reported this suspicious parcel, and the bomb squad came and blew it up. Plumber arrived around 5 minutes later, to find he had to walk to the hardware around the corner and buy yet more cement again.

Ah well, better than watching a policeman shoot a bus that tried to run him over, while i was on the phone to his office, saying they really needed more than one officer there to enforce the road closure.  I did bring him a cup of tea afterwards, reckon he needed it. Bus driver arrested for failing to obey, officer went for stress leave for a while.
« Last Edit: April 11, 2017, 04:54:33 pm by SeanB »
 
The following users thanked this post: yuzuha

Offline CJay

  • Super Contributor
  • ***
  • Posts: 3423
  • Country: gb
  • M0UAW
Re: Things you hope you don't hear...
« Reply #57 on: April 10, 2017, 09:41:13 am »
Once had a clearly nervous support tech ask "How do you undelete the database?"  :-//
The idiot had managed to run "rm -rf /" on a client's server whilst logged in as root.  |O

There's a story from the 80's about doing this to a running server and recovering  :phew: everything without powering the system down.

http://www.lug.wsu.edu/node/414

When I first got involved with SCO Unix/Xenix, I was sent on some training in Birmingham, the first couple of days were configuration, administration etc, then a week of management/admin etc, the last couple of days were spent trying to think of ways to break the system and how to recover it, one of the things we discussed was rm -rf / and how you'd recover from it, we worked out (and tried) that you can recover it relatively easily as long as you don't reboot, it's arse clenching to do but it's not as disasterous as it seems at first glance.

I count myself very lucky to have never had to do it for real though.
M0UAW
 

Offline moz

  • Regular Contributor
  • *
  • Posts: 87
  • Country: au
Re: Things you hope you don't hear...
« Reply #58 on: April 11, 2017, 03:17:27 am »
I used to work in freezing works/abbatoirs, back when automation was moving from giant boards of discrete components to new 8 bit microprocessors. It was very exciting.

Worst call ever from from a pelting machine that wasn't working properly. Those machines pull the skins off cattle, they're not bothered if people get in the way. This particular unit had started occasionally pausing mid-stroke and they decided it was probably my hardware causing the problem. So they rang me, and after a bit of talking to the electrician and taking things apart we decided to cycle the machine (with some of the guards off and no cow inserted). The electrician was supposed to be next to the control panel and phone, not within touching distance of the moving bits:

"ok, it's running... {crunch}"
(silence)
"hello? Dave?"
(more silence)
"shit, mate, there was a big bone jammed down the back and it fell in front of the ram".

Shit, all right.
 

Offline boffin

  • Supporter
  • ****
  • Posts: 901
  • Country: ca
Re: Things you hope you don't hear...
« Reply #59 on: April 11, 2017, 03:50:18 am »
I had a computer "Technician" try to tell me that my sound card didn't work because my powered speakers were not impedance matched with the card :palm:.

I remember a salesman telling me I needed a "Bi-directional" printer cable.  I asked him if they specifically put diodes in the cheaper uni-directional ones.  He didn't get it..

 
The following users thanked this post: TheDane

Offline CJay

  • Super Contributor
  • ***
  • Posts: 3423
  • Country: gb
  • M0UAW
Re: Things you hope you don't hear...
« Reply #60 on: April 11, 2017, 06:29:16 am »
I had a computer "Technician" try to tell me that my sound card didn't work because my powered speakers were not impedance matched with the card :palm:.

I remember a salesman telling me I needed a "Bi-directional" printer cable.  I asked him if they specifically put diodes in the cheaper uni-directional ones.  He didn't get it..

Some parallel cables didn't have all the status lines wired, thus preventing full duplex bi-directional communication within the definition of the IEEE1284 bidirectional standard. A parallel host should be able to 'fall back' to compatible mode if that's the case and still work but will be slower.
M0UAW
 
The following users thanked this post: TheDane

Offline grumpydoc

  • Super Contributor
  • ***
  • Posts: 2682
  • Country: gb
Re: Things you hope you don't hear...
« Reply #61 on: April 11, 2017, 06:32:00 am »
I had a computer "Technician" try to tell me that my sound card didn't work because my powered speakers were not impedance matched with the card :palm:.

I remember a salesman telling me I needed a "Bi-directional" printer cable.  I asked him if they specifically put diodes in the cheaper uni-directional ones.  He didn't get it..

Ah, well, such things existed.

Originally the "Centronics" interface could only send data from the computer to the printer, it used 8 data lines, three handshake lines (STROBE, ACK and BUSY) and some status lines for signalling errors such as out of paper.

Crucially the status lines were often not connected on cheap cables.

When the interface was enhanced to include communication back from the printer (and faster data rates) I'm pretty sure it involved use of these lines so an older, cheap cable would not work (or would cause the computer to fall back to the original unidirectional mode).

So a "bi-directional" centronics cable is one with all the pins connected.

Not that would necessarily have been the fix to whatever problem you were having at the time.

EDIT: Rats :) CJay beat me to it.
 
The following users thanked this post: TheDane

Offline aargee

  • Frequent Contributor
  • **
  • Posts: 726
  • Country: au
Re: Things you hope you don't hear...
« Reply #62 on: April 11, 2017, 06:59:23 am »
Way back in the mid 80s, the company I worked for sold Aussie designed/built Unix boxes. Hard drives were fragile beasties back then.
One customer had written into their agreement as part of acceptance testing that the power be pulled at a random time from the box and it had to suffer nothing more than a reboot.
Some competitors replaced numerous drives in their offerings. We came pretty close to the desired result and were accepted.

Probably the worst is talking to someone calling from their car, hands free, and an ensuing accident. Thankfully minor injuries and not their fault.
Not easy, not hard, just need to be incentivised.
 

Offline LaserSteve

  • Frequent Contributor
  • **
  • Posts: 827
  • Country: us
Re: Things you hope you don't hear...
« Reply #63 on: April 11, 2017, 03:51:15 pm »
Non IT related post, so...

Dental Surgeon talks me into removing all four wisdom teeth in one day, with full anesthetic.  I note with exception of head nurse at his office, all nurses except one  are blond, petite, matched in height, weight etc... I was thinking, he hired them that way for a reason... And Chuckled... :-DD

When I wake up after surgery in the chair, I'm nearly suffocating as one nurse has her leg across my throat, one is setting on left arm, one on left leg, one on right arm, and one on right leg... Plus one I didn't see before was leaning on my chest, coming from a nearby Doctor's  office.  Being a single guy, and having a serious Geek gene, any other time in my life, that would have been quite a fantasy... 

"Hi Steve, we're done They are all out but half of the lower left one" .. UM, how much of that do you remember?" Gurggling against the weight of the combined nurses, I sputtuer  "Remember what Doc? I don't remember a thing"  "Good" With a huge look of relief on his strained face..

So my Mom and my neighbor go off and get the  post surgical prescriptions after picking me up, and go AWOL shopping for about four hours before bringing the meds back...  Twenty minutes after they leave, the shots wear off... ROFSIP is my new abbreviation after this.  means "Rolling on Floor Screaming in Pain". Shot was supposed to be good for four hours.. So I can only guess, but I'm thinking in-effective anesthesia.

To this day I don't know what happened, but all I do know is I had weeks of post surgical infections and a cluster of something gooey  the size of a golf ball rolling around in my left cheek, for about a month...

My normal dentist, asks me every time I visit, how much I remember, and how I feel about Doctor X... Then mutters something about how strong you are for a little guy...

Steve
« Last Edit: April 11, 2017, 04:12:00 pm by LaserSteve »
"I've Never Heard of a Nuclear Meltdown Caused by a Buffer Overflow"  filssavi
 

Offline boffin

  • Supporter
  • ****
  • Posts: 901
  • Country: ca
Re: Things you hope you don't hear...
« Reply #64 on: April 11, 2017, 04:40:49 pm »
I had a computer "Technician" try to tell me that my sound card didn't work because my powered speakers were not impedance matched with the card :palm:.

I remember a salesman telling me I needed a "Bi-directional" printer cable.  I asked him if they specifically put diodes in the cheaper uni-directional ones.  He didn't get it..

Ah, well, such things existed.

Originally the "Centronics" interface could only send data from the computer to the printer, it used 8 data lines, three handshake lines (STROBE, ACK and BUSY) and some status lines for signalling errors such as out of paper.

Crucially the status lines were often not connected on cheap cables.

When the interface was enhanced to include communication back from the printer (and faster data rates) I'm pretty sure it involved use of these lines so an older, cheap cable would not work (or would cause the computer to fall back to the original unidirectional mode).

So a "bi-directional" centronics cable is one with all the pins connected.

Not that would necessarily have been the fix to whatever problem you were having at the time.

EDIT: Rats :) CJay beat me to it.

Actually no, not quite.  Right from the beginning the port never changed much.  The difference were the cheap ass cables would simply skip a couple of wires.
A printer cable connecting from the DB25 to the 57-30360 (I remember the i/f well enough to remember the part number off the top of my head.  Could probably wire a printer cable in my head still) had 8 data lines (from the PC), plus a few lines (STB, AUTOFD, INIT, SLCTIN) that were bi directional, and a few more that went through the logic to be input only.  It was a weird design.  For a few pennies more they could have replaced the 244 with a 245 and made the main (8) data lines bidirectional.  Probably they should have made the whole interface 2 or 3 LS245s, and had a bank of 8 and two banks of 4 bidirectional. That would have allowed everything they wanted.

You can see a full schematic here: http://www.baltissen.org/images/lpt.gif



 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15181
  • Country: za
Re: Things you hope you don't hear...
« Reply #65 on: April 11, 2017, 05:07:27 pm »
Non IT related post, so...

Dental Surgeon talks me into removing all four wisdom teeth in one day, with full anesthetic.  I note with exception of head nurse at his office, all nurses except one  are blond, petite, matched in height, weight etc... I was thinking, he hired them that way for a reason... And Chuckled... :-DD

When I wake up after surgery in the chair, I'm nearly suffocating as one nurse has her leg across my throat, one is setting on left arm, one on left leg, one on right arm, and one on right leg... Plus one I didn't see before was leaning on my chest, coming from a nearby Doctor's  office.  Being a single guy, and having a serious Geek gene, any other time in my life, that would have been quite a fantasy... 

"Hi Steve, we're done They are all out but half of the lower left one" .. UM, how much of that do you remember?" Gurggling against the weight of the combined nurses, I sputtuer  "Remember what Doc? I don't remember a thing"  "Good" With a huge look of relief on his strained face..

So my Mom and my neighbor go off and get the  post surgical prescriptions after picking me up, and go AWOL shopping for about four hours before bringing the meds back...  Twenty minutes after they leave, the shots wear off... ROFSIP is my new abbreviation after this.  means "Rolling on Floor Screaming in Pain". Shot was supposed to be good for four hours.. So I can only guess, but I'm thinking in-effective anesthesia.

To this day I don't know what happened, but all I do know is I had weeks of post surgical infections and a cluster of something gooey  the size of a golf ball rolling around in my left cheek, for about a month...

My normal dentist, asks me every time I visit, how much I remember, and how I feel about Doctor X... Then mutters something about how strong you are for a little guy...

Steve

I had the left side removed by a dentist, who in his spare time was a Springbok rugby player. He lifted me out of the dental chair by the tooth each time, despite having a nurse ( built like a tank) holding me to the chair, each time the only parts touching the chair were the back of my head and my feet. Those 2 out under local, he put in 2 dissolving stitches on each hole, he proceeded to pack my mouth with swabs, and wrote out a prescription for some generic pain tablets, basically the mildest Tylenol you can get. Then said, in his very broken english, that he would see me in 3 months, after the holes had healed, to take the other 2 out. Vowed never to go back to him.

When you have a pair of hands, where the fingers are thicker than some peoples forearms, shoved into your mouth, you tend to worry, especially when there already is a suction tube, a scalpel, a tooth vice and sundry other metal parts in there as well.

Went to my current dentist around 20 years later for the one on the other side, and the last around 2 years ago, but for that I needed to go to a maxillofacial surgeon to have it cut out, as the roots were bent.
 

Offline LaserSteve

  • Frequent Contributor
  • **
  • Posts: 827
  • Country: us
Re: Things you hope you don't hear...
« Reply #66 on: April 11, 2017, 08:00:04 pm »
Sean,

Yikes,

 After reading your post, this might be the first time in my life I can really type "I feel your pain" and mean it.
  My guy was supposedly the best in North Ohio..  I'm not so sure now. He too, was "all thumbs" with big hands.

Going back to have the infected area drained  on was equally scary, a quick shot to numb it, which he never gave time to work, and then stabbed with a horse sized suction needle to remove the infectious material.

I now tell people one tooth at a time and check the Doctor's references...

Steve
"I've Never Heard of a Nuclear Meltdown Caused by a Buffer Overflow"  filssavi
 
The following users thanked this post: TheDane

Offline timb

  • Super Contributor
  • ***
  • Posts: 2528
  • Country: us
  • Pretentiously Posting Polysyllabic Prose
    • timb.us
Re: Things you hope you don't hear...
« Reply #67 on: April 11, 2017, 09:53:29 pm »
Non IT related post, so...

Dental Surgeon talks me into removing all four wisdom teeth in one day, with full anesthetic.  I note with exception of head nurse at his office, all nurses except one  are blond, petite, matched in height, weight etc... I was thinking, he hired them that way for a reason... And Chuckled... :-DD

When I wake up after surgery in the chair, I'm nearly suffocating as one nurse has her leg across my throat, one is setting on left arm, one on left leg, one on right arm, and one on right leg... Plus one I didn't see before was leaning on my chest, coming from a nearby Doctor's  office.  Being a single guy, and having a serious Geek gene, any other time in my life, that would have been quite a fantasy... 

"Hi Steve, we're done They are all out but half of the lower left one" .. UM, how much of that do you remember?" Gurggling against the weight of the combined nurses, I sputtuer  "Remember what Doc? I don't remember a thing"  "Good" With a huge look of relief on his strained face..

So my Mom and my neighbor go off and get the  post surgical prescriptions after picking me up, and go AWOL shopping for about four hours before bringing the meds back...  Twenty minutes after they leave, the shots wear off... ROFSIP is my new abbreviation after this.  means "Rolling on Floor Screaming in Pain". Shot was supposed to be good for four hours.. So I can only guess, but I'm thinking in-effective anesthesia.

To this day I don't know what happened, but all I do know is I had weeks of post surgical infections and a cluster of something gooey  the size of a golf ball rolling around in my left cheek, for about a month...

My normal dentist, asks me every time I visit, how much I remember, and how I feel about Doctor X... Then mutters something about how strong you are for a little guy...

Steve

I had the left side removed by a dentist, who in his spare time was a Springbok rugby player. He lifted me out of the dental chair by the tooth each time, despite having a nurse ( built like a tank) holding me to the chair, each time the only parts touching the chair were the back of my head and my feet. Those 2 out under local, he put in 2 dissolving stitches on each hole, he proceeded to pack my mouth with swabs, and wrote out a prescription for some generic pain tablets, basically the mildest Tylenol you can get. Then said, in his very broken english, that he would see me in 3 months, after the holes had healed, to take the other 2 out. Vowed never to go back to him.

When you have a pair of hands, where the fingers are thicker than some peoples forearms, shoved into your mouth, you tend to worry, especially when there already is a suction tube, a scalpel, a tooth vice and sundry other metal parts in there as well.

Went to my current dentist around 20 years later for the one on the other side, and the last around 2 years ago, but for that I needed to go to a maxillofacial surgeon to have it cut out, as the roots were bent.

Yeah, seeing an oral surgeon to have the wisdom teeth removed is the way to go. I finally did it (all four at once, bottom ones were impacted) about 8 years ago. It was a piece of cake. He called in a prescription for 3 Xanax the day before, I took one in the morning and two more when I got to the office. By the time I was in the chair I was already *very* pliable. Then came the injection of what I assume was some stronger benzo and something else. The next thing I remember is being in the car on the ride home and picking up the Vicodin from the pharmacy. The next 24 hours were spent basically nodding in and out of sleep with headphones on, in my recliner.

I can honestly say I felt nothing. He really knew what he was doing and everything went very smoothly. I had been putting it off for *years* because of all the horror stories you hear, but I had a good dentist who knew his limits and sent me to a good oral surgeon instead.
Any sufficiently advanced technology is indistinguishable from magic; e.g., Cheez Whiz, Hot Dogs and RF.
 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 16372
  • Country: nz
  • Taupaki Technologies Ltd. NZ Siglent Distributor
    • Taupaki Technologies Ltd.
Re: Things you hope you don't hear...
« Reply #68 on: April 12, 2017, 03:43:47 am »
When I had a few out some years ago the first thing I did was express my anxiety to my dentist....why he asked ?
While I might go home bearing less fangs I was worried about long lasting imprints from his boots on my forehead from him obtaining the purchase required. We got on well after that.  :)

The post match prescribed pain relief was of concern though.....Ibuprofen, that I wouldn't consider as it's bloody useless for the odd headache that I might have but he assured me that it was magic for postoperative dental pain relief and much to my surprise it damn well was excellent. No misery at all.  :-+
Avid Rabid Hobbyist
 

Offline German_EE

  • Super Contributor
  • ***
  • Posts: 2314
  • Country: de
Re: Things you hope you don't hear...
« Reply #69 on: April 12, 2017, 07:06:42 am »
"Have you been saved by Jesus?"
Should you find yourself in a chronically leaking boat, energy devoted to changing vessels is likely to be more productive than energy devoted to patching leaks.

Warren Buffett
 

Offline tautech

  • Super Contributor
  • ***
  • Posts: 16372
  • Country: nz
  • Taupaki Technologies Ltd. NZ Siglent Distributor
    • Taupaki Technologies Ltd.
Re: Things you hope you don't hear...
« Reply #70 on: April 12, 2017, 07:19:28 am »
"Have you been saved by Jesus?"
Easily neutralised with: I'm sorry but I'm already a practising heathen....try it some time.  ;)

Avid Rabid Hobbyist
 

Offline TerraHertz

  • Super Contributor
  • ***
  • Posts: 3593
  • Country: au
  • Why shouldn't we question everything?
    • It's not really a Blog
Re: Things you hope you don't hear...
« Reply #71 on: April 12, 2017, 09:56:02 am »
The crunch of a largish piece breaking off a molar tooth. My first experience of this was a couple of nights ago. Such are the trials of getting old.
Collecting old scopes, logic analyzers, and unfinished projects. http://everist.org
 

Offline Rerouter

  • Super Contributor
  • ***
  • Posts: 4402
  • Country: au
  • Question Everything... Except This Statement
Re: Things you hope you don't hear...
« Reply #72 on: April 12, 2017, 10:10:29 am »
when i was a small child, still on my first set of teeth, went to the dentist to remove one he didnt like (about a year before they where meant to come out),

He yanks, and yanks, and that tooth and the 2 next to it come out in his recoil, turns out the roots where hooked around each other, dentist then says, "well shit" to 5 year old me. kinda funny in retrospect as i cannot remember if it hurt anymore.

Kinda put me off dentists for a while

Then in my 20's, had a wisdom tooth removed, had the dentist walk off after 5 minutes of yanking the tooth every which way with no change, says "just going to find my bigger pliers", leaves the room, 4 minutes later walks back in carrying what i can best describe as multi-grip pliers, which he steralises, and has another go at, about 30 seconds in i hear (and felt) a loud crack, and out came the tooth, (crack was from him pressing so hard with the pliers that a chip shot off)
 

Offline LaserSteve

  • Frequent Contributor
  • **
  • Posts: 827
  • Country: us
Re: Things you hope you don't hear...
« Reply #73 on: April 12, 2017, 02:10:24 pm »
"Welcome to the Apple store."

WINNER!

Steve
"I've Never Heard of a Nuclear Meltdown Caused by a Buffer Overflow"  filssavi
 

Offline timb

  • Super Contributor
  • ***
  • Posts: 2528
  • Country: us
  • Pretentiously Posting Polysyllabic Prose
    • timb.us
Re: Things you hope you don't hear...
« Reply #74 on: April 12, 2017, 10:14:58 pm »
"Have you been saved by Jesus?"


Unfortunately, like most people, Jesus saves but doesn't perform backups. I'd hate to be his IT guy in heaven!
Any sufficiently advanced technology is indistinguishable from magic; e.g., Cheez Whiz, Hot Dogs and RF.
 
The following users thanked this post: Kilrah

Offline GreyWoolfe

  • Super Contributor
  • ***
  • Posts: 3290
  • Country: us
  • NW0LF
Re: Things you hope you don't hear...
« Reply #75 on: April 12, 2017, 10:24:42 pm »
"Have you been saved by Jesus?"


Unfortunately, like most people, Jesus saves but doesn't perform backups. I'd hate to be his IT guy in heaven!

His hard drives have a MTBF of 100,000 years instead of hours.  ;D
I am of the age that my brain no longer says "maybe I shouldn't say that" but "what the heck, let's see what happens"
 

Online rdl

  • Super Contributor
  • ***
  • Posts: 2676
  • Country: us
Re: Things you hope you don't hear...
« Reply #76 on: April 12, 2017, 10:26:17 pm »
I distinctly remember being "saved by Jesus" when I was 12 years old. It doesn't come with Lifetime Warranty?
 

Online Cerebus

  • Super Contributor
  • ***
  • Posts: 3673
  • Country: gb
Re: Things you hope you don't hear...
« Reply #77 on: April 12, 2017, 11:05:08 pm »
"Have you been saved by Jesus?"


Unfortunately, like most people, Jesus saves but doesn't perform backups. I'd hate to be his IT guy in heaven!

His hard drives have a MTBF of 100,000 years instead of hours.  ;D

Which is ~138,000 times they've failed since the beginning of the universe. An MTBF of 100,000 years sounds great, until you start thinking on a cosmological scale. Because of the "eternal warranty" that Heaven's Purchasing Department always insist on, that drive supplier was driven out of business after only about 10 million years.
Anybody got a syringe I can use to squeeze the magic smoke back into this?
 

Offline blueskull

  • Supporter
  • ****
  • Posts: 12480
  • Country: cn
  • Power Electronics Guy
Re: Things you hope you don't hear...
« Reply #78 on: April 13, 2017, 12:10:19 am »
I smelled something, like a thread to be locked.
 

Online Vgkid

  • Super Contributor
  • ***
  • Posts: 2520
  • Country: us
Re: Things you hope you don't hear...
« Reply #79 on: April 13, 2017, 12:15:33 am »
I was working on my first power supply. Heard arcing, realized that the ground lead of my o-scope probe had shorted the main filter cap. I quickly pulled the power, and then un-welded the ground probe. This only lasted a total of 10 seconds, from start to finish.
If you own any North Hills Electronics gear, message me. L&N Fan
 

Offline CJay

  • Super Contributor
  • ***
  • Posts: 3423
  • Country: gb
  • M0UAW
Re: Things you hope you don't hear...
« Reply #80 on: April 13, 2017, 06:43:42 am »
I smelled something, like a thread to be locked.

*nods, grabs popcorn*
M0UAW
 

Online Zucca

  • Supporter
  • ****
  • Posts: 2196
  • Country: it
  • EE meid in Itali
Re: Things you hope you don't hear...
« Reply #81 on: April 13, 2017, 07:37:44 am »
One day when I was living in USA I came back from work and turned the light on in my living room. The light was strange, more bright then usual and kind of variating with the time.
5 sec. later my EE self survive instinct kicked in:

1) turned the light off
2) Pulled out my Fluke 85V
3) AC Main were at 200-220VAC, ahhhhhhh
4) Run to the main switch and cutted the power of the entire flat.

The switching ISP (Charter) modem PSU was aldready melted, nothing else seems to be cooked. I call Charter and it took me 15min to get through that stupid automatic answering robot customer support. Finally, after my boold pressure got high enough, I got the usual smart ass real person, and he started with check this, check that. After the third time I said: "The power supply is just melted, that's why I don' t have internet", he got it and told me to go to the Charter shop. I felt stupid, I should had go there without calling the monkey phone customer support.

The next day after checking the mains before the main switch, I got them back to 110Vac. Main Switch on and back to normal life, without WIFI internet. CRAP.
Can't know what you don't love. St. Augustine
Can't love what you don't know. Zucca
 

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4310
  • Country: us
  • KE7GKP
Re: Things you hope you don't hear... 1/4: TIC-WHOOSH
« Reply #82 on: April 13, 2017, 09:54:45 am »
Things you don't want to hear: "Tic-Whoosh"

Back in the mid 1970s, the state of the art broadcast color TV camera technology still used vacuum tubes for imaging (while everything else was solid-state).  So we still had to deal with these fragile (and frightfully expensive) things.  It was critical that you never allowed them to face down because of fear that any possible debris inside the tube could land on the imaging surface and mar the picture.  They were shipped in large cube shape boxes with an internal gimbal to keep the tube always facing up regardless of the orientation of the cardboard carton.



One evening the engineer at Los Angeles TV station KTLA had the task of replacing one of the tubes in the servo-mount camera on the helicopter. He unplugged the socket and unscrewed the retaining ring at the base of the tube. That allowed you to pull the tube out from behind. Then he took the new tube from the carton and carefully inserted it into the deflection yoke and tightened down the retaining ring.  What he heard was the sickening sound of tiny "tic" of the glass envelope being breached, followed soft "whoosh" of the air rushing into the (formerly) vacuum tube.  Upon removing the now-ruined tube, he observed that sure enough, it had that look of a tube that lost its vacuum.

Now, these things cost somewhere around $2000 each back in the 1970s, so this isn't a trivial problem.  But the engineer shrugged and went downstairs and got another spare tube.  This time, he very carefully inserted the new tube into the camera and tightened down the retaining ring, only to hear exactly the same "tic-whoosh" of another multi-thousand dollar tube instantly ruined.  Thoroughly demoralized, he left a note for the chief engineer and proceeded to his favorite social establishment to drown his sorrows.

The next day, it was discovered that there was a manufacturing defect in the retaining ring. It had a small metal burr that broke the glass envelope when tightened.  Since the camera was under warranty, and they had been using Norelco plumbicon tubes in their Norelco camera, the company replaced the ruined tubes and the faulty retaining rings.  I remember getting the notice from Norelco to check all the cameras for this problem. Fortunately, our cameras didn't have this problem.

Bonus story 1:
At the big university hospital we had two Norelco 3-tube color cameras which consisted of a camera head approaching 1/2 cubic meter and a support rack full of panels bristling with nearly 200 adjustment knobs. One afternoon, I had to take the camera across the campus to shoot something in the medical center.  I loaded the camera head into the back of the little electric golf-cart and got in to drive it up the ramp and out onto the sidewalk. Alas. because of the weight distribution of the camera head and the minimal support from the golf-club bag platform, the nose-up attitude of the take-off, and the sudden jerk from the pre-PWM motor control technology, the cart jerked when I pressed the accelerator, and the camera head tumbled out of the back of the golf-cart and onto the hard concrete floor.

I was mortified when I had to tell my manager what happened.  We didn't have the budget to keep spare tubes and I feared I was completely stuffed.  The metal enclosure of the camera head was significantly damaged, but I discovered when I carefully removed the three (Red Green Blue) tubes, they were perfectly fine.  They don't make optical systems like that anymore.  So I removed all the electrical sub-assemblies from the camera head shell, and we took it over to the body-shop where they repaired it like-new (except that they couldn't match Norelco's special gray hammertone paint color).

When we got the shell back from the body-shop, I replaced all the components inside and re-terminated the TV-81 cable. The cable was around 15mm diameter, with a 300mm diameter military-style circular connector for the 75 wires plus six 75 ohm coax cables.  Alas, I got everything re-connected and it looked like everything was working again.  Except that I forgot to put the connector shell over the cable before terminating it, and I had to disconnect all the wires, thread the cable through the back-shell, and re-terminate the cable.  I have almost never forgot to thread the cable through the connector shell in the several decades since that incident.
 
The following users thanked this post: TheWelly888, SeanB, TheDane

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4310
  • Country: us
  • KE7GKP
Re: Things you hope you don't hear... 2/4: Would you like to Hold?
« Reply #83 on: April 13, 2017, 10:12:30 am »
Things you don't want to hear: "Would you like to hold?"

I had the great satisfaction of being on the delivery end of this one.  Back in the late 1980s, our large corporation was deploying PCs and usinng cc:Mail for online communications (pre-POP/IMAP).  The "IT" organization was still in a primitive level of evolution and cc:Mail had a bad habit of the server-based database locking up.  IT's solution was simple, just re-boot the server and everything would be great again. Never mind if anyone had any document open, delivering a presentation, etc.

IT had not yet deployed any servers on our R&D fab campus, but we had lots of computers in the server room which supported the factory. Of course, semiconductor fab down-time even back in those days cost something around $1M per hour, so, ultra reliability and redundancy was common practice in our environment.  The fab operates 7x24x365 with scheduled "warm-downs" only once a year (typically between Christmas and New Year).

In order to deploy cc:Mail on our campus, IT asked if we could set up some file shares on one of our VAX servers to handle the cc:Mail databases.  We accommodated their request and set up four or five file shares for the database files.  One warm day in August, I got a call from the corporate IT help desk.  They said that database ALCCM3 was locked up and cc:Mail was failing for several hundred customers.  "Could you re-boot the computer and clear up the fault?" they asked.  After putting them on hold for several seconds while I laughed uncontrollably, I calmly returned to the phone and said:  "The next scheduled re-boot of that server is Christmas-eve.  Would you like to hold?"   They discovered a different way of clearing up the problem without re-booting the server.
 
The following users thanked this post: TheWelly888

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4310
  • Country: us
  • KE7GKP
Re: Things you hope you don't hear... 3/4: Ready when you are, C.B.
« Reply #84 on: April 13, 2017, 10:42:39 am »
Things you don't want to hear: Ready when you are, C.B.

This is an old yarn from the "golden era" of Hollywood production.  Typical feature film production (even today) typically takes several hours to properly construct, light, and "block" (establish where everyone, including the camera, will be). A single camera is used and all the shots from that angle are filmed together (out of story-edit sequence).  Then they break for lunch and re-set everything for the "reverse-angle" shots in the same location, etc.

But when filming a large-scale scene, or especially if something destructive is happening (like blowing up the fort), they will use several concurrent cameras (more TV-style). Coincidentlally, one of my favorite movie scenes. Peter Sellers in "The Party" as a hapless extra...

https://youtu.be/DKwC6X7_JU0?list=PLZbXA4lyCtqqcuS1GjH0uVV2CA9O4dGyW

The story goes that when legendary Cecil B. Demille was directing Charlton Heston as Moses in "The Ten Commandments" there is the scene where he leads the Hebrews out of slavery in Egypt. The scene was epic, perhaps the origin of the marketing term "cast of thousands". All the extras were dressed up in period costumes, they had live animals, rolling stock, children running around, etc. And they had four cameras to capture the epic scene.

The cameras back in the day were large contraptions that used 35mm film, and the production was shot in "VistaVision" which ran the 35mm film horizontally for a larger image area without the enormous expense of 70mm equipment.  So Mr. Demille and his principal cinematographer were seated on the end of the Chapman camera crane.



And there were three other cameras around the area also capturing the scene.  When everyone (and everything) was finally in place, they were ready to proceed.  Mr. Demille shouted down to his Assistant Director on the ground that they could begin.  The A.D. shouted into his megaphone for the "background" motion to start (meaning the cast of thousands with their animals, carts and kids, etc.)  Then he called to start the cameras.  The oxen pulled, the carts rolled, the people shuffled, the herds of goats, sheep started moving, the kids ran around, etc. until they were out of the shot of all the cameras.  At which point, Mr. Demille signaled to the A.D to call "Cut!"

Mr. Demille turned to his cinematographer and said: "Well how did it go, did we get it?"  At which point the camera operator said "well, the film broke about half-way through, so I'm not sure how much we got.  Well, that is one reason they use multiple cameras.  So they ask the second camera operator who replies with an equally disappointing status: "Some of the sand has got into the gears of the camera and it doesn't run anymore.  I don't think I got anything." 

So on to the third camera.  Now those big cameras didn't have the nice viewfinders of more modern designs. The cameras had a lateral "rack" system where you could rack the camera over to the right to see through the eyepiece directly through the shooting lens. This was perfect for blocking, aiming, focusing, etc.  But, when it was time to expose the film, you had to rack the camera back over so that the film gate was behind the lens.  Alas, the 3rd cameraman forgot to rack the camera back over before starting to shoot, and his cinematography career in Hollywood came to an abrupt end.

Of course, by this time Cecil was rather frantic that they had got something usable from this every expensive scene.  They had the fourth camera up on the hill to get the "wide shot" of the epic scene. So the Assistant Director got his megaphone and called up to Camera 4. "Hey, Bill, did you get it?"  No response.  So he yelled louder: "BILL! DID YOU GET THE SHOT?"  And the very faint response barely came back through the distance: "Ready when you are, C.B."  :palm:
« Last Edit: April 13, 2017, 12:51:04 pm by Richard Crowley »
 
The following users thanked this post: TheWelly888, TiN, samnmax, TheDane

Offline WaveyDipole

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: gb
Re: Things you hope you don't hear...
« Reply #85 on: April 13, 2017, 10:59:55 am »
"I am calling from PC Support. We have detected a new virus on your PC and are calling on behalf of IBM and Microsoft to..... :blah:"

You know the types - they offer to remote to your PC and "correct" the problem, but instead install malware and charge your credit card for the privilege! I know of at least one elderly person that was  :scared: and conned out of 300GPB (so-called annual support charge) with this tactic although I'm glad to say I haven't heard of it being used for some time. We actually had a locally based outfit doing this - which was shut down and ringleaders hauled before the courts.
« Last Edit: April 13, 2017, 11:45:20 am by WaveyDipole »
 
The following users thanked this post: SeanB

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4310
  • Country: us
  • KE7GKP
Re: Things you hope you don't hear...
« Reply #86 on: April 13, 2017, 11:21:36 am »
The Blue Banana

Back in the earliest days of commercial color television in the US, only RCA was making production color TV cameras.  They were as big as a dog house and weighed several hundred pounds. Bristling with tubes, with a cable as big as your arm, and 3-4 full-height racks full of support circuits (with several hundred more tubes) and a couple racks of control panels. (For EACH camera!)   The cameras also required an enormous amount of light because the imaging tubes (image orthicons) were not all the sensitive.  And then add the poor (by today's stanards) lenses, and the R/G/B beam splitting/filtering optical path with its significant light loss, etc.



So, NBC (owned by RCA) had set up one of their studios in 30 Rockefeller center as the "color studio" with huge lights, and massive air-conditioning (to mitigate the heat from the huge lights, etc.)  Alas, because of space consumed by all those racks of support circuits (for EACH camera), they couldn't put the control room on the same floor of the building.  So they carved out a place several floors up and on the other side of the building for the support/control gear.

Now, it took several hours to "dial in" four vintage color cameras.  I could appreciate that as one of my early jobs was to perform regular setup of our two (much more modern) Norelco color cameras.  Even those cameras, which were relatively stable and used all solid-state circuits, could take an hour or more to get all three (Red Green Blue) deflection geometry to match perfectly, and then start on the signal path, adjusting offset ("pedestal") and gain, and "knee", etc. There were literally hundreds of user-adjustment knobs on each camera.  People who use modern video cameras have no appreciation for all the complexity that is now baked into a small, single-chip camera.

So the camera technician would travel up to the studio after lunch to start setting up everything in preparation for an evening live broadcast (there was no video tape recording at the time).  He would power up all the cameras, fire up the lights, and set up the various test and alignment image cards.  Starting with the "ball chart" to dial in the geometry of all three tubes to scan exactly in sync.



And then the "chip chart" where you adjust each color channel (Red Green Blue) to match exactly to produce a pure gray stairstep.



And then, once the geometry was synchronized, you could start on the video signal path adjustments.  Once you were done with the basic adjustments, it was common practice to set up a more "natural" looking subject to tweak the colors of the different cameras to match. One common practice was to use a bowl of plastic fruit to see the more subtle shades of real life.

So, our camera tech took the subway from New Jersey one afternoon and up to the control room to turn on the cameras, then down to the studio to turn on the lights (and the HVAC) and set up the test charts and aim and focus all the cameras.  Then back up to the control room to start hours of tweaking the four cameras for broadcast-quality performance.  When he finally got the geometry of all the cameras tweaked, back down to the studio to strike the test pattern easel and replace it with his bowl of plastic fruit.

Back up to the control room to tweak the colors on the cameras to match.  But no matter what he did he could get all the colors of the fruit to look great, except that the banana was blue.  But he could adjust the cameras so that banana was the proper yellow color, but all the other fruit were horribly wrong.  Back down to the studio to see if he could see anything wrong, but everything looked perfect.  Back and forth all afternoon trying to get the cameras to work. But then the director and the cast and crew started arriving ahead of the broadcast. The director asked the camera tech, "Aren't the cameras ready yet?  We have to start blocking and rehearsal soon."

It turned out that the camera tech's "friends" were secretly substituting a blue-painted banana into the fruit-bowl every time the camera tech went over to the elevator to go up to the control room.  I suspect that the camera tech formed a new circle of friends, that is if he didn't end up in the metal ward.   :scared:
« Last Edit: April 13, 2017, 11:57:44 am by Richard Crowley »
 
The following users thanked this post: TheWelly888, SeanB, TiN, Melt-O-Tronic, Cubdriver, TheDane

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4310
  • Country: us
  • KE7GKP
Re: Things you hope you don't hear...
« Reply #87 on: April 13, 2017, 11:28:46 am »
The "PC support" scammers are as active as ever (if not more so).  We hear of some being rounded up by the authorities every once in a while, but for each operation that is busted, there are likely dozens (or scores) of them out there scamming people every hour of the day. There is a whole sub-channel of "scammer baiting" on YouTube.  When I searched for scammer baiting, I got 34,200 hits. 

The scammers don't care (or can't tell) whether you are using a virtual machine with a fresh build, or even a Mac or Linux system). All they know is their pathetic script.  Some of them have actually broken down and admitted the scam and how it works, etc.  A few have even been tricked into bricking their own computers.

And it is not just computer support. There are many scammers out there pretending to be from the tax authorities (Internal Revenue Service in the USA) and demanding immediate payment of thousands of $$$ to avoid imminent arrest and imprisonment ("the sherrif is on his way to come and arrest you").  And the ones who fake calls from utility companies threatening immediate shut-off if payment isn't rendered on-demand.

No offense to our Indian (Bangladesh, etc.) friends, but the accents are so thick many of them are barely understandable. And their scam scripts are simply laughable. I am embarrassed that so many people are conned by these fools.

If I ever get a call from one of these fools, I will demand someone who speaks English.  That should stop them before they can even begin.
« Last Edit: April 13, 2017, 12:40:54 pm by Richard Crowley »
 

Online Zucca

  • Supporter
  • ****
  • Posts: 2196
  • Country: it
  • EE meid in Itali
Re: Things you hope you don't hear...
« Reply #88 on: April 13, 2017, 11:55:01 am »
Thanks Richard, very interesting. would like to drink a beer with you and hear some other stories before the PC age.... when everything was analog and noisy...
Can't know what you don't love. St. Augustine
Can't love what you don't know. Zucca
 

Offline Luminax

  • Regular Contributor
  • *
  • Posts: 159
  • Country: my
    • Electronesk
Re: Things you hope you don't hear...
« Reply #89 on: April 13, 2017, 12:11:18 pm »
Her reply was "What do you mean software?........ What lights are on at the moment?". I think I even said to her, "You know, software, that's running on the modem?... The configuration you're trying to get me to reset?... The thing where you stick the pen in the hole and such?".

 :palm:

Just ship me my damn modem!
First line support often have little domain knowledge, their main ability/task is to run down a flow-chart of standard questions & responses.

Thing is - it works for most punters who have even less knowledge about how stuff works.

Here in Malaysia our biggest ISP support line just sucks...
It was quite good until a few years back and, courtesy of knowing someone in their main outsourced call center, I get to know the 'secret handshake' to get me passed to second line to (usually) reset the port or drop the hang session, etc.
Monopoly, though... we ain't going to be getting better ISP anytime soon...

Anyway, things I hope I don't hear in my daily job... one of them being "Dude, the boss just overridden the test parameter again" because that's usually a precursor to something breaking and requiring me to troubleshoot down a rabbit hole trying to trace back what he fizzed up
 |O |O
Jack of all trade - Master of some... I hope...
 

Offline timb

  • Super Contributor
  • ***
  • Posts: 2528
  • Country: us
  • Pretentiously Posting Polysyllabic Prose
    • timb.us
Re: Things you hope you don't hear...
« Reply #90 on: April 13, 2017, 12:11:48 pm »
Things you don't want to hear: "Tic-Whoosh"
~Snip~
It was critical that you never allowed them to face down because of fear that any possible debris inside the tube could land on the imaging surface and mar the picture.
~Snip~
One evening the engineer at Los Angeles TV station KTLA had the task of replacing one of the tubes in the servo-mount camera on the helicopter.

If you were using the camera in a helicopter, wouldn't it be pointing down? Pointing down and vibrating even. What keeps all the debris from falling down once it's installed?
Any sufficiently advanced technology is indistinguishable from magic; e.g., Cheez Whiz, Hot Dogs and RF.
 

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4310
  • Country: us
  • KE7GKP
Re: Things you hope you don't hear...
« Reply #91 on: April 13, 2017, 12:31:31 pm »
If you were using the camera in a helicopter, wouldn't it be pointing down? Pointing down and vibrating even. What keeps all the debris from falling down once it's installed?
No. The servo mechanism was designed to prevent tilting the camera  more than 45 degrees down.  In situations where they used straight-down shots, they used mirrors. (They could flip the scan horizontally and/or vertically by simply reversing the deflection yokes.)

I suspect that last generation of tubes weren't nearly as vulnerable to the "internal debris" problem, but, by then the tradition had been established, and the things were so darned expensive, they didn't want to take any chances.
 

Online Zbig

  • Frequent Contributor
  • **
  • Posts: 857
  • Country: pl
Re: Things you hope you don't hear...
« Reply #92 on: April 13, 2017, 02:43:45 pm »
No offense to our Indian (Bangladesh, etc.) friends, but the accents are so thick many of them are barely understandable. And their scam scripts are simply laughable. I am embarrassed that so many people are conned by these fools.

If I ever get a call from one of these fools, I will demand someone who speaks English.  That should stop them before they can even begin.

Few years ago I was reading user comments under the PC support scam article on "The Register" and came across the story of a guy who told his preferred way of dealing with these. Important thing to note is they, despite their unmistakable accent, apparently like to introduce themselves using fake common English first names. So, it went something along these lines:

Support scammer, in thick Indian accent: "Hello my friend, my name is John and I am calling from Microsoft Support..."
The guy being called, in perfect British English: "Oh, hello, that's brilliant! My name's Deepak."
« Last Edit: April 13, 2017, 02:45:56 pm by Zbig »
 
The following users thanked this post: TheWelly888, Richard Crowley

Offline AF6LJ

  • Supporter
  • ****
  • Posts: 2903
  • Country: us
Re: Things you hope you don't hear...
« Reply #93 on: April 13, 2017, 02:57:59 pm »
Field guy: "Uh-oh ... " 
to which the conversation quickly escalates to..
Me: "Uh-oh? .. What the (insert expletive) do you mean "Uh-oh?"
Nothing like that conversation to send your imagination off on a riotous romp.

Depends what the guy was doing at the time  ;D


No Problem, that will buff out....
Sue AF6LJ
Test Equipment Addict, And Proud Of It.
 

Offline AF6LJ

  • Supporter
  • ****
  • Posts: 2903
  • Country: us
Re: Things you hope you don't hear...
« Reply #94 on: April 13, 2017, 03:06:52 pm »
While I was the tech with the uh-oh, I can legitimately roll out the 'I was only following orders' defence.

Called out of hours to a hospital server which was flagging a predictive fail on one disk in the RAID array (I forget the exact disk size, either 36 or 72GB)

It was a HP DL380 G4 which had 3.5" SCSI disks installed and one was indeed flashing amber to indicate predictive failure.

On the phone to the support team I was told they'd down the server for me so I could replace the failed disk (it's not necessary, they're hotswap, it's actually better to hotswap them), I agreed and proceed to boot the thing from a HP utility disk which allows me to check the RAID config, disk serial numbers, model numbers etc. (it wasn't unknown for disk caddies to be re-used by parts brokers and have incorrect markings compared to the actual disk in the caddy)

I found that some genius had configured the disks as a RAID0/JBOD, essentially a bunch of disks with a container spanned across them all to give a disk the size of the sum of parts, in this instance they'd done something I didn't realise was possible and had allocated the first 10GB of all the disks to the JBOD to give a disk size of 40GB for the OS (server 2003 IIRC)

They'd then allocated the rest of the disk space to a RAID5 set to hold the data volume (Might have been SQL Server, possibly Exchange, critical data anyway) so notionally the data volume was fault tolerant but not OS as removal of any one disk from a JBOD will kill it.

So, I explained this to the support team wonk who insisted he knew better than I, after all I was only a lowly out of hours tech (who happened to be HP qualified to work on Superdome, EVA, Alpha etc. etc.) and he was third line who knew all and requested I 'just shut up and replace the disk' or words to that effect, he was however happy for me to hotswap the disk so we booted the machine up, he connected remotely (genius things those ILOs) to 'monitor' my actions.

I protested, he demanded, getting quite stroppy and threatening to escalate it to his management. I suggested he dial their number and put us on a conference call so he did. Once I'd been introduced and explained why I didn't want to do it, his manager also insisted I replace the disk as he trusted his tech.

Obviously I then pulled the disk and watched the machine bluescreen instantly.

After what must have seemed an eternity to the remote parties but a few seconds to me, I asked 'would you like me to fix that for you?" and got a quiet 'yes, please, if you can'.

Turns out every server in the business had been built the same way.
OH GOD  :palm: :palm:
Sue AF6LJ
Test Equipment Addict, And Proud Of It.
 

Offline AF6LJ

  • Supporter
  • ****
  • Posts: 2903
  • Country: us
Re: Things you hope you don't hear...
« Reply #95 on: April 13, 2017, 04:22:50 pm »
Sounds you don't want to hear.....

For a short time I was working for a semiconductor company here in SanDiego (now long gone)
I had a bad feeling about working there when upon my first day there I overheard a conversation between two cal lab techs. The equipment they were discussing was a plasma vapor disposition device. The gear used a large water cooled triode vacuum tube. (3CW35000)
The tech working on the gear was complaining the high voltage was kicking out and he couldn't find out why... A week later I entered into the conversation since I have a fair amount of vacuum tube experience. So I ask what the problem was, I got the same story how the high voltage kept kicking out and he couldn't find the problem....
So I ask; What have you done up to this point...
The reply was well there was a high voltage power supply problem (blown diodes bad filter caps) I fixed that had had high voltage. I drained the coolant and replaced it. that's when the power supply kept kicking out. we replaced the tube (at $1500.00USD) it still kept kicking out the high voltage...
I asked what did you use for coolant........................
Wait for it....................................................................................
He replied in front of his supervisor.... I used antifreeze.  :palm: :palm:
I said (in front of his supervisor) "you know that is a conductor of electricity right?".
His supervisor went back and purged the cooling system, replaced the antifreeze with some of our own DI water. The tech was terminated that afternoon....

A second story from the same place.
My boss there was not the Cal Lab supervisor but the QA department supervisor, I build fixtures and maintained some of QA's HI-Rel gear. (leak detectors, centrifuges, vibe sets etc...)
He hired a young tech who's job was to take readings on items that we were doing life testing on, among other things...
We had a bridge rectifier that was under full ratings life test, and this new hire was responsible for taking readings on it at the beginning of each day.
I am sitting in my little office working on something... when I hear....
BANG I got this sick feeling in the pit of my stomach.. I got up and walked int our burn-in room just in time to see the last of the smoke that was released from this nearly priceless bridge rectifier...I looked at him, he looked at me......
I said you have to go tell W***.....
I went back my bench and heard the yelling, the door to W***'s office slam, and the next pay period end, W***, and the new hire were let go...

BONUS Story;
When I worked at Loral Corp. there was a period of time when I was responsible for teaching a new tech the ropes. This guy had NO RF experience. Ex-Nave Submarine sonar tech...
Young'ish kid, seemed to always be distracted. I'm suppose to be training this guy so they stick him two rows away from me... I asked my boss "how am I suppose to train him when he is all the way over there?" He said we will move him soon.....
Okay, I did the back N fourth, thing... The guy wouldn't take notes and always kept forgetting to check the zero on his power meter before powering up the transmitter he was supposed to be working on. And yes he always seemed to be distracted.... (he had animated conversations with himself)  :scared:

I got him through some of the basics of our modules and he even managed to tune them up correctly.  (I checked everything he did) so it was time to turn him loose with real RF test equipment and a transmitter to tune up and temp comp...
He gets everything he needs, frequency counter,
Bench receiver, distortion analyzer, audio osciallator, an HP-8559A spectrum analyzer with a diode limiter attached to protect the analyzer. power meters, I set up the plumbing and plumed in the SA, power meter, bench receiver, and counter.
He was instructed NOT to disconnect any of the gear at this time, just the transmitter...

All went well for two days, then he got board...........
I am at my bench and a coworker walked up and said "Why is J***'s spectrum analyzer smoking". My heart stopped.
YES the diode limiter was smoking and smoke was coming out of the reference level cal access hole in the front of the 8559A...
"J**** what the F... are you dong?
"I wanted to see if I could see audio on the spectrum analyzer"...
I looked closer..
The HP audio oscillator was on it's highest range, the level meter was just off the scale, and it was connected to the input of the analyzer... 
I looked at J**** and said in a calm voice, "you are putting thirty volts peak to peak into the input of an instrument that is designed to take a fraction of a volt maximum based on where you have the input attenuator set, do yo see what you did wrong?"
Out of the corner of my eye I saw my supervisor walking up just as J*** was answering my question. "I couldn't get a full scale signal on the analyzer" he said.
I replied. "That is because the lower limit is 10MHZ, I see you have a manual open to the specifications, didn't you notice that?"
"No...." was all he said...

They found something for him to do and I reminded my supervisor this was why I needed to have him next to me to oversee what he was doing. ...
He was transferred to our instrumentation division where his skills better fit the job at hand.

My boss did remind me of an Ex Marine who worked over on another product line who connected a 40W S-band transmitter directly to the input of another 8559A. All the front in that analyzer was reduced to carbon all the way back to and including the YIG, YIG filter, and First IF. 



Sue AF6LJ
Test Equipment Addict, And Proud Of It.
 
The following users thanked this post: TiN, TheDane

Offline timb

  • Super Contributor
  • ***
  • Posts: 2528
  • Country: us
  • Pretentiously Posting Polysyllabic Prose
    • timb.us
Re: Things you hope you don't hear...
« Reply #96 on: April 13, 2017, 09:15:47 pm »
If you were using the camera in a helicopter, wouldn't it be pointing down? Pointing down and vibrating even. What keeps all the debris from falling down once it's installed?
No. The servo mechanism was designed to prevent tilting the camera  more than 45 degrees down.  In situations where they used straight-down shots, they used mirrors. (They could flip the scan horizontally and/or vertically by simply reversing the deflection yokes.)

I suspect that last generation of tubes weren't nearly as vulnerable to the "internal debris" problem, but, by then the tradition had been established, and the things were so darned expensive, they didn't want to take any chances.

Ah, okay that makes a lot of sense, thanks!

When I hear "cameras" and "helicopters" I think of the modern stabilized cameras that are so ubiquitous. I guess we really do take this stuff for granted these days (they're on every news and police chopper, some with FLIR even).

Your stories are really interesting, I'd love to hear more!
Any sufficiently advanced technology is indistinguishable from magic; e.g., Cheez Whiz, Hot Dogs and RF.
 

Offline Gregg

  • Frequent Contributor
  • **
  • Posts: 605
  • Country: us
Re: Things you hope you don't hear...
« Reply #97 on: April 13, 2017, 10:06:39 pm »
The biggest heart stopping didn’t want to hear happened to me while in the Navy on an aircraft carrier in the South China Sea early 1970s.  It was supposed to be a stand-down day for Thanksgiving when at 6 AM the announcement came:

“General Quarters, General Quarters, all hands man your battle stations.  This is NOT a drill.”

It turned out to be a major fire in one of the engine rooms, but over 5000 crew me members probably all exclaimed the F word.
 

Offline Cubdriver

  • Supporter
  • ****
  • Posts: 2454
  • Country: us
  • Nixie addict
    • Photos of electronic gear
Re: Things you hope you don't hear...
« Reply #98 on: April 14, 2017, 04:33:22 am »
It turned out that the camera tech's "friends" were secretly substituting a blue-painted banana into the fruit-bowl every time the camera tech went over to the elevator to go up to the control room.  I suspect that the camera tech formed a new circle of friends, that is if he didn't end up in the metal ward.   :scared:

 :-DD :-DD :-DD :-DD :-DD :-DD

THAT was hysterical!  Though I'd suspect he needed a new circle of friends because he'd strangled the old ones!

-Pat
If it jams, force it.  If it breaks, you needed a new one anyway...
 
The following users thanked this post: AF6LJ

Offline Harb

  • Regular Contributor
  • *
  • Posts: 244
  • Country: au
Re: Things you hope you don't hear...
« Reply #99 on: April 15, 2017, 08:35:08 am »
The Blue Banana


Pretty funny story........I have a couple of old RCA TK76 chains that I am going to restore one day.....was lucky enough to pick them up with Peds , CCU's and all.....even a set of lenses.......including some long OB ones......great for the mancave ;)

I wonder how many people know this is the sort of Makeup the actors etc needed to wear to make up for the lousy response of the old Camera Tubes........they used all sorts of colors to get the picture looking right at the TV end....

http://cosmeticsandskin.com/aba/max-and-the-tube.php

« Last Edit: April 15, 2017, 12:09:44 pm by Harb »
 

Online Kilrah

  • Supporter
  • ****
  • Posts: 1770
  • Country: ch
Re: Things you hope you don't hear...
« Reply #100 on: April 15, 2017, 10:44:19 am »
Wow, learned something.
Would be interesting to see real life vs television comparison of both with and without makeup...
 

Offline Harb

  • Regular Contributor
  • *
  • Posts: 244
  • Country: au
Re: Things you hope you don't hear...
« Reply #101 on: April 15, 2017, 12:11:59 pm »
Wow, learned something.
Would be interesting to see real life vs television comparison of both with and without makeup...

Yes "Painting" the picture is a term thats still used now days, but its done electronically now to get all the colours and levels in order...
 

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4310
  • Country: us
  • KE7GKP
Re: Things you hope you don't hear...
« Reply #102 on: April 15, 2017, 04:11:56 pm »
Camera "shading" is a critical part of a broadcast-quality TV production.  The shader has a control panel for each camera where s/he can adjust lens iris, "pedestal" (a.k.a. DC offset in electronic terms), gain, offset, "gamma" (linearity) and "knee" (compression) for red and blue (green is the fixed reference).  The shader typically gets the best "broadcast reference" picture monitor to see the picture, as well as waveform and vectorscope to monitor the video waveform.  The shader has final responsibility for the quality of the video images.  The director "only" instructs the camera operators what shots to get and calls when each camera is switched into the program feed.

This shows a typical configuration for shading 16 cameras.  It is split in half because a single person can't handle shading 16 cameras by themselves.  The big knob at the bottom of each camera panel is a 2-axis "joystick". Forward-back controls the lens iris, and the knob on top rotates to adjust the "pedestal" (DC offset).



 

Offline Harb

  • Regular Contributor
  • *
  • Posts: 244
  • Country: au
Re: Things you hope you don't hear...
« Reply #103 on: April 15, 2017, 05:44:25 pm »
I would have a riot on my hands if I had only 2 operators to paint 16 camera's in one of my trucks........ the most I ask any single operator to handle is 4 camera's at a time for sport, maybe 6 for studio stuff, but there is also an Engineer on a master setup panel doing an overall setup across all of the camera's and he/she is tasked with making sure all of the various operators is keeping up and doing the right thing maintaining a standard.
We don't call them "Shaders" here in Australia, (although a couple of people I have hired might have been a bit shady, lol) they are simply CCU operators.  :-+
« Last Edit: April 15, 2017, 05:49:07 pm by Harb »
 
The following users thanked this post: Richard Crowley

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 10095
  • Country: us
  • DavidH
Re: Things you hope you don't hear...
« Reply #104 on: April 15, 2017, 06:55:36 pm »
I understand that doctors are trained to say "there" instead of "uh-oh" or "oops".
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 10095
  • Country: us
  • DavidH
Re: Things you hope you don't hear... 1/4: TIC-WHOOSH
« Reply #105 on: April 15, 2017, 07:32:10 pm »
Back in the mid 1970s, the state of the art broadcast color TV camera technology still used vacuum tubes for imaging (while everything else was solid-state).  So we still had to deal with these fragile (and frightfully expensive) things.  It was critical that you never allowed them to face down because of fear that any possible debris inside the tube could land on the imaging surface and mar the picture.  They were shipped in large cube shape boxes with an internal gimbal to keep the tube always facing up regardless of the orientation of the cardboard carton.

Tektronix made the same recommendation; do not store CRTs and do not transport oscilloscopes with the CRT face down.  CRTs which are otherwise in perfect shape can have debris inside which will damage the phosphor coating.

 

Online Cerebus

  • Super Contributor
  • ***
  • Posts: 3673
  • Country: gb
Re: Things you hope you don't hear...
« Reply #106 on: April 15, 2017, 09:22:00 pm »
We don't call them "Shaders" here in Australia, (although a couple of people I have hired might have been a bit shady, lol) they are simply CCU operators.  :-+

It's the cameramen who are shady, inside every cameraman there's an amateur pornographer. You only have to be in an OB truck, or on the bridge, watching the raw camera feeds to know that I'm right.  :)
Anybody got a syringe I can use to squeeze the magic smoke back into this?
 
The following users thanked this post: SeanB

Offline SNGLinks

  • Regular Contributor
  • *
  • Posts: 133
  • Country: gb
Re: Things you hope you don't hear...
« Reply #107 on: April 16, 2017, 08:28:44 am »
In the U.K. We call them rack operators and they rack the cameras.

I was in a news OB outside the MOD during the Faulklands war. There was not much happening and the cameramen were just panning along the street looking at the girls walking by and zooming in on their upper bodies.

The female director got annoyed with this and said over the talkback 'Haven't you seen a pair of breasts before?' To which a cameraman replied 'Not those, no!'
 

Offline German_EE

  • Super Contributor
  • ***
  • Posts: 2314
  • Country: de
Re: Things you hope you don't hear...
« Reply #108 on: April 16, 2017, 09:08:48 am »
I grew up with a pair of religious parents and every Sunday we used to watch a BBC programme called 'Songs of Praise' where they transmitted a church service from a different location each week. One Sunday we noticed that a camera kept on showing a pretty woman in a brightly coloured top and my father remarked "the cameraman likes that one". Fast forward ten minutes and the director switched cameras by accident to show the same woman's breasts in close-up.

Standard operating procedure for my father was to turn off the TV for the night if he saw anything that offended him, and for the first and only time 'Songs of Praise' was switched off mid programme.

I also remember a visit to Granada TV Studios in Manchester, and this was a technical tour rather than the tourist stuff that the general public gets to see. Two things that I recall about the 1980's cameras were a) their large size and b) the fact that Granada TV kept them all running 24/7 to avoid recalibrating them each time they were used. The power bill must have been substantial.
Should you find yourself in a chronically leaking boat, energy devoted to changing vessels is likely to be more productive than energy devoted to patching leaks.

Warren Buffett
 

Offline Harb

  • Regular Contributor
  • *
  • Posts: 244
  • Country: au
Re: Things you hope you don't hear...
« Reply #109 on: April 16, 2017, 09:15:19 am »
I grew up with a pair of religious parents and every Sunday we used to watch a BBC programme called 'Songs of Praise' where they transmitted a church service from a different location each week. One Sunday we noticed that a camera kept on showing a pretty woman in a brightly coloured top and my father remarked "the cameraman likes that one". Fast forward ten minutes and the director switched cameras by accident to show the same woman's breasts in close-up.

Standard operating procedure for my father was to turn off the TV for the night if he saw anything that offended him, and for the first and only time 'Songs of Praise' was switched off mid programme.

I also remember a visit to Granada TV Studios in Manchester, and this was a technical tour rather than the tourist stuff that the general public gets to see. Two things that I recall about the 1980's cameras were a) their large size and b) the fact that Granada TV kept them all running 24/7 to avoid recalibrating them each time they were used. The power bill must have been substantial.

In the studio's we leave everything running 24/7 for a number of reasons, but the most important are failures due to thermal cycling and inrush currents and the other of course is reduced setup time.
 

Offline razberik

  • Frequent Contributor
  • **
  • Posts: 265
  • Country: cz
Re: Things you hope you don't hear...
« Reply #110 on: April 16, 2017, 09:32:31 am »
"Have you been saved by Jesus?"
Once met Mormons in public transport. A dialogue like this happened:
Mormon: "Good evening to you. Do you have some faith ?"
Me: "Yes, I have a faith. I believe in non-existence of God."
Mormon was clearly nervous about this answer: "Wait, what ?"
Me: "I believe that no God exists."
Mormon: "But we have proves that God and Jesus act good."
Me: "Me belief is stronger than your proves."

Last two sentences were repeated in 3 cycles with no escape. It was ended by my terminus station. I hope they learned something. ;D
 

Online Kilrah

  • Supporter
  • ****
  • Posts: 1770
  • Country: ch
Re: Things you hope you don't hear...
« Reply #111 on: April 16, 2017, 09:37:56 am »
Me: "Me belief is stronger than your proves."

That's magnificent, reverse powers and get them at their own game  :-DD
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15181
  • Country: za
Re: Things you hope you don't hear...
« Reply #112 on: April 16, 2017, 10:16:23 am »
In the studio's we leave everything running 24/7 for a number of reasons, but the most important are failures due to thermal cycling and inrush currents and the other of course is reduced setup time.

Biggest cost in a studio used to be the lights and the AC to keep the rest of the set from catching fire. The power for the cameras would be incidental, as those lights started at 1kW, and went up from there, and you might have had 15 or more on a set for general lighting, plus some spots that were used for fill in so the talent had a decent face shot without shadowing. With 30kW of light, and about the same for airconditioning to get the sensible heat out, who would notice a 2kW load for the cameras, and with a 5 hour or so warm up period before starting cal it would be just noise.

Just had some massmeters done, and took them to the lab the day before and plugged all in to warm up for 24 hours before the cal check, so they get stable.
 

Offline rrinker

  • Super Contributor
  • ***
  • Posts: 1921
  • Country: us
Re: Things you hope you don't hear...
« Reply #113 on: April 17, 2017, 01:10:47 pm »
We don't call them "Shaders" here in Australia, (although a couple of people I have hired might have been a bit shady, lol) they are simply CCU operators.  :-+

It's the cameramen who are shady, inside every cameraman there's an amateur pornographer. You only have to be in an OB truck, or on the bridge, watching the raw camera feeds to know that I'm right.  :)

 As someone who, in HS as part of the AV crew got to film the practices of the girl's gymnastics team, I refuse to divulge further information on the grounds that it may incriminate me. I will say there were many things that never actually got to the tape  :-DD  That was before VHS and Beta, open reel video tape and a rather large B&W only camera. I did eventually get to use a color system, videoed the school plays with that one.
 

Offline GK

  • Super Contributor
  • ***
  • Posts: 2600
  • Country: au
Re: Things you hope you don't hear...
« Reply #114 on: April 17, 2017, 02:28:36 pm »
Bzzzzt. No longer care, over this forum shit.........ZZzzzzzzzzzzzzzzz
 

Offline Red Squirrel

  • Super Contributor
  • ***
  • Posts: 2322
  • Country: ca
Re: Things you hope you don't hear...
« Reply #115 on: April 18, 2017, 02:23:13 am »
Something you don't want to hear when checking out a new EEVblog channel:  "This account has been terminated".   :P



 
The following users thanked this post: AF6LJ

Offline RobK_NL

  • Frequent Contributor
  • **
  • Posts: 250
  • Country: nl
Re: Things you hope you don't hear...
« Reply #116 on: April 18, 2017, 03:26:52 pm »
"Welcome aboard this United Airlines flight to ..."
Tell us what problem you want to solve, not what solution you're having problems with
 
The following users thanked this post: AF6LJ, beenosam

Offline GreyWoolfe

  • Super Contributor
  • ***
  • Posts: 3290
  • Country: us
  • NW0LF
Re: Things you hope you don't hear...
« Reply #117 on: April 18, 2017, 10:01:38 pm »
"Welcome aboard this United Airlines flight to ..."

"We need 4 volunteers........"
I am of the age that my brain no longer says "maybe I shouldn't say that" but "what the heck, let's see what happens"
 

Offline Cyberdragon

  • Super Contributor
  • ***
  • Posts: 2141
  • Country: us
Re: Things you hope you don't hear...
« Reply #118 on: April 19, 2017, 12:25:14 am »
*unidentified sizzling*

Need I say more?
*BZZZZZZAAAAAP*
Voltamort strikes again!
Explodingus - someone who frequently causes accidental explosions
 
The following users thanked this post: Red Squirrel

Offline Richard Crowley

  • Super Contributor
  • ***
  • Posts: 4310
  • Country: us
  • KE7GKP
Re: Things you hope you don't hear...
« Reply #119 on: April 19, 2017, 02:57:12 am »
"Welcome aboard this United Airlines flight to ..."
"We need 4 volunteers........"
You think that the recent incident will make people more willing to volunteer in the future?
 

Offline Cyberdragon

  • Super Contributor
  • ***
  • Posts: 2141
  • Country: us
Re: Things you hope you don't hear...
« Reply #120 on: April 19, 2017, 03:01:16 am »
"Welcome aboard this United Airlines flight to ..."
"We need 4 volunteers........"
You think that the recent incident will make people more willing to volunteer in the future?
*BZZZZZZAAAAAP*
Voltamort strikes again!
Explodingus - someone who frequently causes accidental explosions
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf