Author Topic: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?  (Read 2102 times)

0 Members and 1 Guest are viewing this topic.

Offline EEVblog

  • Administrator
  • *****
  • Posts: 29263
  • Country: au
    • EEVblog
A deep dive into how and why Apple's post 2016 MacBook Pro's fail after water ingress, and the poor design decisions that lead to it. Which engineer at Apple is to blame? Or is anyone actually to blame?
The PCB and product design process, the often siloed design jobs, and failure mode analysis
Includes a demo of water hydrolysis on a PCB.



Louis Rossmann on the Apple botched design:
Water ingress in a MacBook:

 
The following users thanked this post: johnlsenchak

Online james_s

  • Super Contributor
  • ***
  • Posts: 8716
  • Country: us
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #1 on: June 20, 2019, 12:30:12 am »
I don't really care about vulnerability to liquid because I've never expected a computer to tolerate having liquid spilled on it. The MacBook Pro is a flawed design though, they sacrificed far too much to make the machine much thinner than it needs to be. The keyboard sucks, the expandability is zip, the battery life is marginal and you can't swap in another one. Also the touchpad is absurdly huge, so big that it gets in the way, and the touch bar is a neat but virtually useless gimmick.

I say this all as a daily user of an employer issued 2017 MBP.
 

Online blueskull

  • Supporter
  • ****
  • Posts: 11925
  • Country: cn
  • Power Electronics Guy
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #2 on: June 20, 2019, 01:14:03 am »
My MacMini 2018 has its fan connected to the motherboard with a 5P connector, makes sense.
What is stupid is the pad design for the fan. This thing has 5 0.5mm*0.25mm pads for 5 contacts, and two equally wide, maybe a bit longer (0.8mm?) mechanical pads on the sides.

So the total area supporting the connector, and its retention force during removal is only 0.125mm2*5+0.2mm2*2=1.025mm2. With 5N of retention force, that's almost 5MPa of pressure on those pads.
Also, the cross section width for the pads is 0.25mm*7=1.75mm. At 5N, the peel strength is 2.85N/mm, while typical FR4 boards are only rated 1.25N/mm.

Apparently not only Apple engineers were not thinking, whatever company making the connector was not thinking as well. That is to say, unless they are intentionally trying to mess up any unauthorized service attempts.
 

Online rsjsouza

  • Super Contributor
  • ***
  • Posts: 3265
  • Country: us
  • Eternally curious
    • Vbe - vídeo blog eletrônico
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #3 on: June 20, 2019, 01:42:35 am »
I don't understand the whole liquid spill brouhaha as well. A computer has fans, jacks, other openings, etc. If you dropped your computer on water or spilled cranberry juice on it (like my wife did to her old Toshiba laptop) don't expect it to survive graciously and a 0.5mm distance between an HV and a data pin will not make much difference - it may only be the weakest link.

Not to mention that "liquid" may also mean a loaded toilet - thus a much worse scenario than Dave's simple tap water.  :scared:
(although that may be much more significant for cellphones)
Vbe - vídeo blog eletrônico http://videos.vbeletronico.com

Oh, the "whys" of the datasheets... The information is there not to be an axiomatic truth, but instead each speck of data must be slowly inhaled while carefully performing a deep search inside oneself to find the true metaphysical sense...
 
The following users thanked this post: thmjpr, aram

Offline isometrik

  • Contributor
  • Posts: 17
  • Country: ca
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #4 on: June 20, 2019, 02:25:05 am »
If a device is not specified to be protected against water ingress (IP65, IP66, IP67, etc), how can you blame the manufacturer.

Sure, any device can be improved. Always. Even my own designs have known compromises dictated by economic/requirements/costs/timeline constraints.

Any device used outside its normal/specified operating envelope is at risk of failing. Liquid ingress into a computer is definitively outside a normal operating envelope.
 
The following users thanked this post: thmjpr, aram

Online maginnovision

  • Frequent Contributor
  • **
  • Posts: 900
  • Country: us
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #5 on: June 20, 2019, 02:59:14 am »
I don't even like Apple or their products and I'm not going to blame them for users spilling water, or their liquid of choice, on their laptops. If you want protection you can get insurance, or a laptop that is rated for some level of ingress. Ideally you stop treating expensive laptops like they're a phone which is actually rated for that kind of contact. Besides for the price they'll charge you to protect against these sorts of issues you may as well buy insurance, it'll be cheaper.
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2248
  • Country: 00
    • My random blog.
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #6 on: June 20, 2019, 03:02:49 am »
The issue is not water ingress breaking the motherboard, that always happened if you were unlucky to get water on the connector.
The issue is this time your CPU blows up and repair cost spikes up close to uneconomical. No difference to Apple, they bill $1200 for motherboard replacement every time anyway with strong suggestion of up-sell.

Tim Cook to Investors: "People Bought Fewer New iPhones Because They Repaired Their Old Ones"
https://www.vice.com/en_us/article/zmd9a5/tim-cook-to-investors-people-bought-fewer-new-iphones-because-they-repaired-their-old-ones
what a coincidence!
« Last Edit: June 20, 2019, 03:04:25 am by Rasz »
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Offline floobydust

  • Super Contributor
  • ***
  • Posts: 2722
  • Country: ca
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #7 on: June 20, 2019, 03:12:52 am »
What about dust, dirt, breadcrumbs on the keyboard? Is that "outside a normal operating envelope" ?

MacBook butterfly switch design problematic since 2015 and in 2019 they announce some extended keyboard service program. After the class action lawsuits filed.

Apple's downplaying the problem like an ostrich sticking it's head in the sand, such a bullshit response ensured I will never buy Apple products. Years of people bitching and they modified the keyboard design and still problematic.

Place your bets if the keyboard's design is fixed in 2019 models.

If Apple had a spirit of repair and getting your value out of the product, made their stuff repairable instead of this planned obsolescence and disposable mandate, they could increase revenue having more (repeat) customers instead of raping them with gross overpricing, like that $1000/$200 Vesa Pro Display stand.
 

Offline IanB

  • Super Contributor
  • ***
  • Posts: 9489
  • Country: us
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #8 on: June 20, 2019, 03:33:46 am »
If a device is not specified to be protected against water ingress (IP65, IP66, IP67, etc), how can you blame the manufacturer.

Except I have disassembled more than one desktop keyboard and found it completely immune to coffee spills. Upon disassembly the design has had a continuous plastic membrane between the keys and the switch pads underneath. Any spilled liquid stays above the membrane and the circuit board stays dry. I see no difficulty designing a laptop keyboard in a similar manner.

In fact, a few months back I took the top half of my keyboard to the sink and scrubbed it clean under running water with a brush. You should have seen the looks I got when people saw me washing a keyboard under running water! Don't worry folks, no electronics were harmed in this exercise  ;D
I'm not an EE--what am I doing here?
 

Offline thmjpr

  • Regular Contributor
  • *
  • Posts: 107
  • Country: ca
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #9 on: June 20, 2019, 05:38:36 am »
Except I have disassembled more than one desktop keyboard and found it completely immune to coffee spills. Upon disassembly the design has had a continuous plastic membrane between the keys and the switch pads underneath. Any spilled liquid stays above the membrane and the circuit board stays dry. I see no difficulty designing a laptop keyboard in a similar manner.

In fact, a few months back I took the top half of my keyboard to the sink and scrubbed it clean under running water with a brush. You should have seen the looks I got when people saw me washing a keyboard under running water! Don't worry folks, no electronics were harmed in this exercise  ;D

So what we are getting at here, is what Dave gets to at about the 35min mark of the video (great video btw), this can be a mechanical design related oversight if anything.
IF apple cared about water ingress not damaging the board, the mechanical design would attempt to shield the board. Or of course conformal coating of the board and sealing of the connector is an option as well, but generally not used on laptops from what I've seen (could be costly or labor intensive).

You can have a ground "guard" trace if you like, but you are not solving anything, the laptop will still fail. The PCB/plastic carbonizes as Dave shows, creates a conductive path and overheats, maybe makes some smoke too. Once you get the laptop under warranty -> sign of liquid damage -> not covered under warranty. Therefor not a problem for apple.

If a device is not specified to be protected against water ingress (IP65, IP66, IP67, etc), how can you blame the manufacturer.

Sure, any device can be improved. Always. Even my own designs have known compromises dictated by economic/requirements/costs/timeline constraints.

Any device used outside its normal/specified operating envelope is at risk of failing. Liquid ingress into a computer is definitively outside a normal operating envelope.

Yes. This can happen if there is clear demand for it and competition.
Many recent phones have water ingress mitigation (iphone X is IP67 rated). Its basically a "must have" feature these days.
 

Offline johnlsenchak

  • Frequent Contributor
  • **
  • Posts: 255
  • Country: us
  • js@antihotmail.com
    • paypal.me/johnsenchak
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #10 on: June 20, 2019, 12:58:46 pm »

It would have been nice to see you  dump  some water on a  motherboard  from one of "those dumpster  dive  computers" you end  up always
finding down in the  garbage  room. Maybe pour some water on  the motherboard  power  connector and then  turn the power supply on  to
see if you get corrosion  8)
John Senchak "Daytona  Beach  Florida "
 jls (at)  antihotmail.com   http://www.antihotmail.com
https://www.facebook.com/john.senchak.1
 

Online james_s

  • Super Contributor
  • ***
  • Posts: 8716
  • Country: us
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #11 on: June 20, 2019, 03:49:23 pm »
If a device is not specified to be protected against water ingress (IP65, IP66, IP67, etc), how can you blame the manufacturer.

Except I have disassembled more than one desktop keyboard and found it completely immune to coffee spills. Upon disassembly the design has had a continuous plastic membrane between the keys and the switch pads underneath. Any spilled liquid stays above the membrane and the circuit board stays dry. I see no difficulty designing a laptop keyboard in a similar manner.

In fact, a few months back I took the top half of my keyboard to the sink and scrubbed it clean under running water with a brush. You should have seen the looks I got when people saw me washing a keyboard under running water! Don't worry folks, no electronics were harmed in this exercise  ;D

Those membrane keyboards are mushy crap though. When I have the luxury of a desktop PC I refuse to suffer with anything less than a proper clicky mechanical keyboard.
 

Online David Hess

  • Super Contributor
  • ***
  • Posts: 9807
  • Country: us
  • DavidH
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #12 on: June 20, 2019, 05:29:27 pm »
The black stuff is copper(II) oxide or cupric oxide from the electrolysis and not carbon from the substrate or other source.  Other common products are copper hydroxide (blue) and copper chloride (blue and green).

Ultimately Apple just blew it.  The high voltage pin should have had a much higher separation from ground and any other signals.  The connector specifications only apply under close to ideal conditions with little surface contamination which based on real world results, is not realistic in this application.
 

Offline TheDane

  • Regular Contributor
  • *
  • Posts: 189
  • Country: dk
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #13 on: June 20, 2019, 06:24:12 pm »
It costs a lot of money to redesign the motherboard AND the display (possibly only motherboard and the display cable itself needs to be modified) - as 'just' moving the connections around on the motherboard won't make the correct connections on the display unit.

Another thing that seems strange is why the UL hasn't checked/required testing of the circuit (I haven't checked, but I presume the item is UL certified) High voltage and large currents is a safety hazard and can ignite a fire. The burnt connector is proof of this. Safety hazard - UL - Responsibility  :scared: :-BROKE  :palm:

It would be interesting to know if a ground guard can protect the MUX/CPU-GPU chip. When you're doing the bread board electrolysis demo (with a single trace removed) could you try do the same - just with a 'non-removed' trace now connected to ground and then measure the voltage on the next trace in relation to the grounded trace. 56V, Ground, Measure.  The ground trace should work as a lightning/leakage protector, channeling the energy safely to a known potential (ground in this case) - though there will be some coupling to the 'signal' trace next to it.
Is this 'simply enough' to protect the chip, or is an extra effort needed - such as ESD/Tranzorb protection? - which seems to be missing in general  :-//

And, the Ground-Differential_Coupling strategy isn't fully implemented on the top of the connector - the much needed ground pin is needed  :-DD

Who's to blame - Customers, who buy this :horse: imho. Also Apple staff:
Management for not investing properly in design/testing and focusing on profit. Design Engineers for a poor product. Testing facility for not doing climatic chamber/salt mist tests. Production Engineers/facility for not conformally coating.
Insurance agencies for ignoring safety hazards - hopefully they're not employed Apple staff, but you never know (Boeing wants to do computer simulations and not physical tests - https://www.theregister.co.uk/2019/06/17/boeing_certification_simulation_tests/ - and they were just exposed from having misled the F.A.A. on tests and certifications by having tooooo close relations - https://tech.slashdot.org/story/19/06/01/2334205/nyt-deadly-misguided-assumptions-were-built-into-boeings-737-max  :o)
 

Online David Hess

  • Super Contributor
  • ***
  • Posts: 9807
  • Country: us
  • DavidH
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #14 on: June 20, 2019, 06:57:40 pm »
Another thing that seems strange is why the UL hasn't checked/required testing of the circuit (I haven't checked, but I presume the item is UL certified) High voltage and large currents is a safety hazard and can ignite a fire. The burnt connector is proof of this. Safety hazard - UL - Responsibility  :scared: :-BROKE  :palm:

UL is concerned with safety and not functionality or reliability.  The plastics used for the connector and the board materials are self extinguishing.

The applicable standard has to do with creepage and clearance in different environments.  The connector is designed for a sealed environment but the actual environment is an exposed indoor or even outdoor environment where the creepage and clearance requirements are much greater.
« Last Edit: June 21, 2019, 09:54:01 am by David Hess »
 

Offline floobydust

  • Super Contributor
  • ***
  • Posts: 2722
  • Country: ca
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #15 on: June 20, 2019, 08:13:56 pm »
It's just another example of engineering being rushed and pushed. Not be confused with shoddy engineering.

Design for EMC, liquid and dirt ingress, vibration, flex, drop, serviceability, manufacturability, low cost, small size and add fake innovation for marketing.
You're basically frying engineers with an endless list of requirements that could be met - if the team had the resources, support and sufficient time.

It doesn't matter whether it's a Boeing 737 Max-8 or a MacBook- the engineering team got pushed and forced to roll out something half-assed. They don't have the luxury of time and people to do an excellent job.
Problems become evident months or years after the release date, and by then the corporation has forgotten what caused the design to be substandard in the first place.

Mr. Rossmann is pointing out how Apple is blatantly failing at making their products better. There's 1,000 reasons why engineering does a shit job despite having great talent. The best chef in the world will cook up something terrible if rushed and short of staff.

We can all focus on the spacing of a crappy connector's pins that shouldn't get wet in the first place, that Apple can command a custom part with decent spacings, that could limit damage by keeping HV from the mux.
I would say it's not the real problem - it's a corporate culture that has its values misplaced, probably onto the death spiral of "adding value to shareholders is priority #1".
 

Offline Doctorandus_P

  • Frequent Contributor
  • **
  • Posts: 573
  • Country: nl
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #16 on: June 20, 2019, 09:28:09 pm »
When water ingress becomes a problem, decent manufactures put in rubber seals and conformal coating.
Apple puts in a piece of paper that discolors when wet and then void's your warranty.

'nuf said.
« Last Edit: June 20, 2019, 09:39:09 pm by Doctorandus_P »
 

Offline SilverSolder

  • Frequent Contributor
  • **
  • Posts: 538
  • Country: 00
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #17 on: June 20, 2019, 09:57:15 pm »

If Apple built their laptops to 1960's or 70's Hewlett Packard standards...   few would be able to afford one! 

But they would be works of art, for sure.

 

Offline mariush

  • Super Contributor
  • ***
  • Posts: 3660
  • Country: ro
  • .
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #18 on: June 20, 2019, 09:58:36 pm »
It wouldn't surprise me if Apple went with that connector simply because it's from a less known connector manufacturer with whom maybe they made a deal to manufacture that model only for them, making it harder for repair shops to service laptops. Louis has to buy faulty boards just to desolder a good connector from them, along with mux chips when needed.

Looks to me like there's plenty of room on the pcb to use a more common flat flex connector and maybe have separate connector for backlight a bit further away from the data connector.

Maybe such a move would also allow them to use that filler material they usually put  under/around bga chips to prevent water ingress in phones... around the power connector, or they could use a tiny piece of kapton/whatever adhesive tape over the smaller led backlight connector to reduce the risk of water going inside the connector.
 

Online james_s

  • Super Contributor
  • ***
  • Posts: 8716
  • Country: us
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #19 on: June 20, 2019, 11:04:23 pm »
I still don't get it. There is a whole laundry list of design flaws in these machines but vulnerability to water is not one of them. I would not want to compromise feel or increase cost in order to get liquid resistance. I have never spilled liquid on a computer in my life. I have repaired several (non-Apple) that were damaged by spills though. I don't understand why anyone would expect a laptop to be liquid resistant unless it's specifically marketed as a ruggedized machine.
 
The following users thanked this post: aram

Offline KaneTW

  • Frequent Contributor
  • **
  • Posts: 258
  • Country: de
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #20 on: June 20, 2019, 11:23:22 pm »
I still don't get it. There is a whole laundry list of design flaws in these machines but vulnerability to water is not one of them. I would not want to compromise feel or increase cost in order to get liquid resistance. I have never spilled liquid on a computer in my life. I have repaired several (non-Apple) that were damaged by spills though. I don't understand why anyone would expect a laptop to be liquid resistant unless it's specifically marketed as a ruggedized machine.

Condensation can happen, and [eople use laptops outside.

Production cost is a tiny factor in Apple product price; any possible water ingress protection cost will not be significant.

And to be fair, most phones are fairly resistant to liquid ingress. It's easy to generalize from that.
 

Online james_s

  • Super Contributor
  • ***
  • Posts: 8716
  • Country: us
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #21 on: June 21, 2019, 12:27:01 am »
So what? Most laptops are not water resistant, why the focus on Apple here? If you work outside or deal with condensation there are laptops on the market specifically designed for such use.

What I am hearing here is a bunch of people moaning that an expensive sports car is flawed because it becomes damaged if you drive it offroad on a 4x4 trail because it's possible to design a vehicle that can survive being driven offroad.
 
The following users thanked this post: aram

Offline thm_w

  • Super Contributor
  • ***
  • Posts: 1210
  • Country: ca
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #22 on: June 21, 2019, 12:44:58 am »
Condensation can happen, and people use laptops outside.

If it were solely a condensation issue, then it would be a bigger deal in terms of design (because they specify it will work at 90-95% humidity), but its a liquid spill issue as far as we can see.

Quote
Production cost is a tiny factor in Apple product price; any possible water ingress protection cost will not be significant.

Production assembly time, component cost (if glue/conformal coating is used), NRE cost of additional engineering, time cost of product being delayed. Its hard to know what the true cost is.

Quote
And to be fair, most phones are fairly resistant to liquid ingress. It's easy to generalize from that.

Then hopefully the trend will progress to laptops, but it has not yet. Plus liquid resistance in phones has nothing to do with which signal goes where on an internal connector.
 

Offline TheDane

  • Regular Contributor
  • *
  • Posts: 189
  • Country: dk
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #23 on: June 21, 2019, 11:05:52 am »
Areas like Greenland is often very cold and quite dry.
Areas like Florida (in the United States) is often quite hot and very humid.
If it is also windy (not thinking hurricanes and floods in general), and it is near the ocean with saltwater, salt tends to migrate into the air - forcing people in the vicinity to wash their windows more often (if they want to look outside, as the otherwise would turn white from the accumulating salt).
Any device not designed/tested to withstand this, is going to fail - sooner or later. Electric or not.
 

Offline TheDane

  • Regular Contributor
  • *
  • Posts: 189
  • Country: dk
Re: EEVblog #1222 - Apple's MacBook Design FAIL - Who's To Blame?
« Reply #24 on: June 21, 2019, 12:16:31 pm »
Information as of 2019-06-20 - https://support.apple.com/15-inch-macbook-pro-battery-recall
15-inch MacBook Pro Battery Recall Program
Apple has determined that, in a limited number of older generation 15-inch MacBook Pro units, the battery may overheat and pose a fire safety risk.
Affected units were sold primarily between September 2015 and February 2017 and product eligibility is determined by the product serial number.

Source from which I found out: https://ing.dk/artikel/brandfare-apple-tilbagekalder-macbook-pro-computere-226833
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf