Author Topic: What did we learn from the "open source ventilator" mess.......  (Read 5277 times)

0 Members and 1 Guest are viewing this topic.

Online tggzzz

  • Super Contributor
  • ***
  • Posts: 21225
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: What did we learn from the "open source ventilator" mess.......
« Reply #50 on: July 08, 2020, 07:37:02 am »
One perspective, from comp.arch.embedded

On 08/07/20 04:36, Rick C wrote:
> On Tuesday, July 7, 2020 at 10:02:45 PM UTC-4, Paul Rubin wrote:
>> Rick C <gnuarm.deletethisbit@gmail.com> writes:
>>> Wow!  Designing in a group is strange.  I'm helping with the design of a
>>> ventilator and it is a unique experience.
>>
>> Sounds like fun, but why a ventilator?  Is the ventilator shortage still
>> happening and are there more urgent shortages?  I thought there were
>> already a bunch of ventilator projects.
>>
>> What is the software going to be like, and is it being taken care of? This
>> sounds like a good opportunity to write some Ada code.
>
> Yeah, that was one of my first observations that there are tons of ventilator
> projects.  They nearly all are based on squeezing an Ambu-bag to push air
> into the patient.  The details of not harming the patient are tricky.
> Ventilating a patient while conscious is harder and very few are doing that.
> They seem to be having problems with the plumbing at the moment, getting all
> the right valves in the right places to do the right things.  At the moment
> there is a lot of hardware right at the patient's mouth.  I'm working to
> understand what is needed and what isn't.
>
> There's a valve called a Ruben valve which allows the patient to be connected
> to the inspiratory limb (branch from the ventilator) or to the expiratory
> limb (the path for exhaling) without a leak to the outside via a straight
> through path.  It's not entirely clear just how the details of this valve
> work.  I have concerns that the valve can be stuck in the inhale position by
> the pressure in the line.  It doesn't seem to have a way to leak out once
> pressurized.
>
> I'm not sure the software is too hard at the higher levels, but there are
> interface procedures that have to convert differential pressure to flow rate
> and monitor other marginal sensors.  The O2 sensor is only ~18% accurate.
> The guy running the project doesn't seem to understand the difference between
> accuracy, resolution and noise when we were talking about increasing the
> resolution of a reading by averaging many samples.  Not that this was a big
> deal.  The real issue is I'm not sure improving the resolution of the
> measurement will give him what he wants.  The airflow measurement is very
> sensitive at the low end.  I'm wondering if there is an issue with the flow
> being more or less laminar at the low end so that it is not going to be an
> accurate reading.  The interesting part is they  verified the home brew flow
> rate sensor by using a commercial flow rate sensor that uses the same
> technique.
>
> Whatever.  I'm trying to help without being a drag.  I think sometimes my
> questions are not what they want to hear.  Maybe I'm too pessimistic.
>
> The software is C I believe.  I don't think anyone in this crowd is going to
> use Ada.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline SmokeyTopic starter

  • Super Contributor
  • ***
  • Posts: 3042
  • Country: us
  • Not An Expert
Re: What did we learn from the "open source ventilator" mess.......
« Reply #51 on: July 08, 2020, 11:51:15 am »
I'm a little surprised how casually some of you are suggesting Medtronic should have just "ramped up", especially given that I know some of you actually have manufacturing experience.  I would expect the market for ventilators was pretty stable and small volume so why would they have that extra capacity in place already?  I can think of a few companies that failed because they had an unexpected bump in demand and botched the "ramp up". 

Another big question for me would be how many of the components used in the Medtronic ventilator were outsourced to China or India or one of the other countries that was effectively shut down at the beginning of the pandemic.  Given modern parts sourcing and volume manufacturing strategies, I'm going to guess a significant number of parts (both custom and off the shelf) were not made locally and out of their direct control.  So now you can't get your overseas factories to make higher number of parts you need and you can't get the parts shipped quickly anyway since the bulk transport was all messed up.  At that point your only option would be to go through the pain and suffering of finding and qualifying a new local vendor at a time when the businesses are shut down and people are afraid they will die if they leave their house.  The medical device certifications probably prohibit you from arbitrarily swapping vendors anyway, which could be waved in the emergency but that's one more hoop that needs jumping.  Good luck with all that. 

/speculation/
In my opinion its plausible that Medtronic realized that what the world needed in that moment was essentially what it looks like the JPL team made, which was a high quality but stripped down machine built from parts that were actually available at that moment.  To facilitate this, they released their full proved specification which as I've said elsewhere is essentially the highly valuable roadmap and up front investment to quickly making a functional device for anyone that's capable.  They got that out first because that's the most important.  Having the BOMs or the mechanical assemblies may be helpful from a reference standpoint, but if you are making things from scratch based on what you can get now they aren't necessary so they release those later for whatever reason.  Since the expectation was that the result would be a new custom device anyway, they protected their specific machine design elements with a restrictive license long term.
Now you may be asking why Medtronic didn't just make the new stripped down thing themselves, which is a legit question too.  It takes a certain type of engineer and organization to function well in the skunkworks type environment.  Unless that's what you do, you will probably fail if you try to sprint through a design.  Developing things super quickly is a different skill and process, and I'm going to guess a big medical device company with multi-year design cycles is not set up that way.
Of course, their evil open-source hating lawyers may have been scheming about how to trap unsuspecting do good engineers with subtle license clauses, but I'm not sure I'd go there first when there are other possibilities.  With the spec being the investment, you don't just give that away hoping to get some publicity and eventual license royalties and especially not from a mega-corp.  Maybe they were just actually trying to do good, and they still got shit on.
« Last Edit: July 08, 2020, 12:24:21 pm by Smokey »
 

Offline EEVblog

  • Administrator
  • *****
  • Posts: 39025
  • Country: au
    • EEVblog
Re: What did we learn from the "open source ventilator" mess.......
« Reply #52 on: July 09, 2020, 07:35:16 am »
I'm a little surprised how casually some of you are suggesting Medtronic should have just "ramped up"

I'm saying that if the government put in a huge order Medtronics (and others) could have ramped up production, and done so vastly easier and quicker and to certification than anyone dicking around with the source files they released.
 
The following users thanked this post: langwadt, sokoloff

Offline EEVblog

  • Administrator
  • *****
  • Posts: 39025
  • Country: au
    • EEVblog
Re: What did we learn from the "open source ventilator" mess.......
« Reply #53 on: July 09, 2020, 07:40:03 am »
/speculation/
In my opinion its plausible that Medtronic realized that what the world needed in that moment was essentially what it looks like the JPL team made, which was a high quality but stripped down machine built from parts that were actually available at that moment.  To facilitate this, they released their full proved specification which as I've said elsewhere is essentially the highly valuable roadmap and up front investment to quickly making a functional device for anyone that's capable.  They got that out first because that's the most important. Having the BOMs or the mechanical assemblies may be helpful from a reference standpoint, but if you are making things from scratch based on what you can get now they aren't necessary so they release those later for whatever reason.  Since the expectation was that the result would be a new custom device anyway, they protected their specific machine design elements with a restrictive license long term.
Now you may be asking why Medtronic didn't just make the new stripped down thing themselves, which is a legit question too.  It takes a certain type of engineer and organization to function well in the skunkworks type environment.  Unless that's what you do, you will probably fail if you try to sprint through a design.  Developing things super quickly is a different skill and process, and I'm going to guess a big medical device company with multi-year design cycles is not set up that way.
Of course, their evil open-source hating lawyers may have been scheming about how to trap unsuspecting do good engineers with subtle license clauses, but I'm not sure I'd go there first when there are other possibilities.  With the spec being the investment, you don't just give that away hoping to get some publicity and eventual license royalties and especially not from a mega-corp.  Maybe they were just actually trying to do good, and they still got shit on.

Jones's Razor: Never attribute to genuine helpful intention that which can be adequately explained by a publicity opportunity.
 
The following users thanked this post: pardo-bsso, schmitt trigger

Offline donotdespisethesnake

  • Super Contributor
  • ***
  • Posts: 1093
  • Country: gb
  • Embedded stuff
Re: What did we learn from the "open source ventilator" mess.......
« Reply #54 on: July 09, 2020, 03:49:19 pm »
Jones's Razor: Never attribute to genuine helpful intention that which can be adequately explained by a publicity opportunity.

 :-+ Good one, probably right in this case. I expect someone in the boardroom though it a good idea and made the order, after being filtered through the rest of the company (enginering, business, legal) they publish the design for the most obsolete product they have which has the least impact on their sales, and with the most restrictive license possible (CYA).

But rather than rehashing old arguments about motivations, did any of these open designs actually get deployed in real hospitals?

I guess the real lesson is that there rarely is a quick techno-fix for this type of problem, despite how much people would like one. What works is monitoring and tracking disease vectors before they become human-human transmissible, and contact tracking when they do. All of which is boring but well-known by the WHO, who try to encourage countries to do it, despite lack of funding or other political agendas getting in the way.

Unfortunately, the interaction with humans and wild animals is generally increasing, as humans continue to expand into wild habitats. Nature has a way of dealing with "too many humans" (like any other excess population). Expect more pandemics. Build all the ventilators you want, it won't really help much.
Bob
"All you said is just a bunch of opinions."
 

Offline schmitt trigger

  • Super Contributor
  • ***
  • Posts: 2431
  • Country: mx
Re: What did we learn from the "open source ventilator" mess.......
« Reply #55 on: July 09, 2020, 04:06:08 pm »
Daqq;
From the sentence in your post: "All they needed was this balloon, a motor and a lot of duct tape.".............

You forgot an Arduino. Indeed, definitively an Arduino.  ;D
 

Offline Sal Ammoniac

  • Super Contributor
  • ***
  • Posts: 1764
  • Country: us
Re: What did we learn from the "open source ventilator" mess.......
« Reply #56 on: July 09, 2020, 06:49:59 pm »
if all open source software stopped working today, we would be thrown back to the late 1800 basically

How did you come up with this ridiculous assertion? Do you think open source software was running the world in, say, 1950? 1970?

So what is all of this open source software that runs the world? Linux? Apache? Sure, these are widely deployed open source applications, but you do realize, don't you, that there are other things out there that could take their place? Sure, you may think things like Windows Server and IIS are products of the Evil Empire, but they are perfectly functional and if open source alternatives didn't exist they would fill the gap.

Sorry about the rant, but I'm getting tired of people claiming open source is the savior of the world and the only possible solution...
"That's not even wrong" -- Wolfgang Pauli
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15794
  • Country: fr
Re: What did we learn from the "open source ventilator" mess.......
« Reply #57 on: July 09, 2020, 07:04:50 pm »
Daqq;
From the sentence in your post: "All they needed was this balloon, a motor and a lot of duct tape.".............

You forgot an Arduino. Indeed, definitively an Arduino.  ;D

 :-DD
 

Offline ZaphodBeeblebrox

  • Contributor
  • Posts: 23
  • Country: be
Re: What did we learn from the "open source ventilator" mess.......
« Reply #58 on: July 09, 2020, 07:57:06 pm »
I'm a little surprised how casually some of you are suggesting Medtronic should have just "ramped up"

I'm saying that if the government put in a huge order Medtronics (and others) could have ramped up production, and done so vastly easier and quicker and to certification than anyone dicking around with the source files they released.

With this I wholeheartedly agree.

The effort put into trying to manufacture the same ventilator (or a similar design) but without the actual knowlegde and/or resources to do so is not something to take lightly. A company already having said knowlegde and a supply chain in place only requires incentive to ramp up production (hey, capitalism has its advantages too!).

Yes, I know, "we don't have the production capacity" or "we are unable to obtain parts". The former can actually be adressed by outsourcing production and the latter, well, the same thing applies: put in a huge order and watch the magic happen. Besides, the open source community didn't seem to have this "unable to obtain parts" thing (correct me if I'm wrong here). Does this mean it is easy for a company to "simply ramp up production": no, far from it. But is easier for a company having the experience than for a group of volunteers new to the task.

Also, don't get me wrong: I think it was a great idea from the community to help out where needed by making ventilators. Alas, the execution of that thought went wrong somewhere along the road. I too must admit that there are open-source designs out there where I would rather take my chances without - because all you need is one single ESD event nearby and your lungs are ruptered.

Sorry for the rant.
 

Offline DrG

  • Super Contributor
  • ***
  • !
  • Posts: 1199
  • Country: us
Re: What did we learn from the "open source ventilator" mess.......
« Reply #59 on: July 09, 2020, 08:33:31 pm »
I'm a little surprised how casually some of you are suggesting Medtronic should have just "ramped up"

I'm saying that if the government put in a huge order Medtronics (and others) could have ramped up production, and done so vastly easier and quicker and to certification than anyone dicking around with the source files they released.

With this I wholeheartedly agree.

The effort put into trying to manufacture the same ventilator (or a similar design) but without the actual knowlegde and/or resources to do so is not something to take lightly. A company already having said knowlegde and a supply chain in place only requires incentive to ramp up production (hey, capitalism has its advantages too!).

Yes, I know, "we don't have the production capacity" or "we are unable to obtain parts". The former can actually be adressed by outsourcing production and the latter, well, the same thing applies: put in a huge order and watch the magic happen. Besides, the open source community didn't seem to have this "unable to obtain parts" thing (correct me if I'm wrong here). Does this mean it is easy for a company to "simply ramp up production": no, far from it. But is easier for a company having the experience than for a group of volunteers new to the task.

Also, don't get me wrong: I think it was a great idea from the community to help out where needed by making ventilators. Alas, the execution of that thought went wrong somewhere along the road. I too must admit that there are open-source designs out there where I would rather take my chances without - because all you need is one single ESD event nearby and your lungs are ruptered.

Sorry for the rant.

With regard to:

Yes, I know, "we don't have the production capacity" or "we are unable to obtain parts". The former can actually be adressed by outsourcing production and the latter, well, the same thing applies: put in a huge order and watch the magic happen. Besides, the open source community didn't seem to have this "unable to obtain parts" thing (correct me if I'm wrong here).

Here is an interesting video featuring Erika Earl, whom I *think* could be described as an audio engineer (and more) https://www.namm.org/nammu/contributors/erika-earl As I understand it, she was enlisted by the JPL VITAL team with a specific function for parts procurement (and some other things I guess, but that is the part I am focusing on). She speaks in the video of the Supplyframe platform https://supplyframe.com/ and yes, it almost sounds like a commercial.

She found it valuable and they did get it done and there certainly were requirements. Any opinions on that aspect? - I mean that literally, I am not an EE and could probably only design a runny nose if I had a cold, but not much else. I do find  find the thread an interesting one.

« Last Edit: July 09, 2020, 08:38:34 pm by DrG »
- Invest in science - it pays big dividends. -
 

Offline SmokeyTopic starter

  • Super Contributor
  • ***
  • Posts: 3042
  • Country: us
  • Not An Expert
Re: What did we learn from the "open source ventilator" mess.......
« Reply #60 on: July 09, 2020, 09:55:12 pm »
So most of the arguments being made are assuming pre Corona virus lockdown supply chain and vendor availability options.  "Just order more parts, what's the problem?". That completely ignores the fact that whole factories were shut down, whole supply chain shipping options were shut down, and whole global economies were shut down.  That video with the descriptions of parts sourcing issues from the JPL supply chain person really makes that clear.  They needed to do a new design based on stuff they could actually get.  Whether the assembly has five parts or thousands of parts, The unavailability of just one part is a show stopper.  Halt production.  And I'm not sure if you caught that part at the end about the BOM freeze.  Even for non-life critical applications oftentimes you can't just make part and vendor substitutions at will.  Being design engineers I know for sure that we should appreciate the fact that just because the data sheets look the same on first pass does not mean the parts are interchangeable in a specific application.  In this case, the consequences of part incompatibility isn't another BOM revision and maybe some RMAs, it's dead people.
 

Offline EEVblog

  • Administrator
  • *****
  • Posts: 39025
  • Country: au
    • EEVblog
Re: What did we learn from the "open source ventilator" mess.......
« Reply #61 on: July 10, 2020, 08:30:20 am »
So most of the arguments being made are assuming pre Corona virus lockdown supply chain and vendor availability options.  "Just order more parts, what's the problem?". That completely ignores the fact that whole factories were shut down, whole supply chain shipping options were shut down, and whole global economies were shut down.  That video with the descriptions of parts sourcing issues from the JPL supply chain person really makes that clear.  They needed to do a new design based on stuff they could actually get.  Whether the assembly has five parts or thousands of parts, The unavailability of just one part is a show stopper.  Halt production.  And I'm not sure if you caught that part at the end about the BOM freeze.  Even for non-life critical applications oftentimes you can't just make part and vendor substitutions at will.  Being design engineers I know for sure that we should appreciate the fact that just because the data sheets look the same on first pass does not mean the parts are interchangeable in a specific application.  In this case, the consequences of part incompatibility isn't another BOM revision and maybe some RMAs, it's dead people.

If you can say that about production stuff, then you can say the same, ever more so, about hastily hacked together solutions.
And therein lies a possible problem with the entire concept in the first place.
Perhaps it was simply never possible to develop something like this so quickly when a lot of supply chains and production systems of any type were shut, and the product was required to save lives and not take them.
 

Online tszaboo

  • Super Contributor
  • ***
  • Posts: 8217
  • Country: nl
  • Current job: ATEX product design
Re: What did we learn from the "open source ventilator" mess.......
« Reply #62 on: July 10, 2020, 09:10:11 am »
About ramping up production: My experience is, that ramping up some parts of production is not hard. The PnP machine easily can handle larger quantities, more shifts. Same goes for injection moulding. And the manual parts of assembly can be augmented by more people, or smarter organisation. This is usually not the hard part.
What I've seen is that the bottleneck is usually the testing of the device. They very likely functionally test each and every ventillator before shipping it. And the test bench of this is going to be the bottleneck. For example, if is possible that they have an artificial lung, government approved, calibrated, specifically designed to test ventilators. And then you need to connect your device to this, and do a x hour burn in test, to avoid early failures, and to prove your device is assembled correctly. Could be two hours, could be 24 hours, who knows? The test length probably was designed to reduce the chance of early failure to the ppm level. And that lung with all the sensors and equipment connected to it is very time consuming to put together, very expensive, and requires exotic parts, that you need to order yourself. And when you order them, they have a lead time. And very expensive. So if you want to just 2x or 3x your throughput, you might just pull in more shifts.
But you want x10? Invest hundreds of thousands in equipment, that will just sit there afterwards? Hire people in the middle of the pandemic for a new job to work at night? What if you just run out of factory floor space, because all these require space. Pull up a building next to it? Or assemble medical equipment in tents?
 
The following users thanked this post: Smokey

Online tggzzz

  • Super Contributor
  • ***
  • Posts: 21225
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: What did we learn from the "open source ventilator" mess.......
« Reply #63 on: July 10, 2020, 09:29:25 am »
So most of the arguments being made are assuming pre Corona virus lockdown supply chain and vendor availability options.  "Just order more parts, what's the problem?". That completely ignores the fact that whole factories were shut down, whole supply chain shipping options were shut down, and whole global economies were shut down.  That video with the descriptions of parts sourcing issues from the JPL supply chain person really makes that clear.  They needed to do a new design based on stuff they could actually get.  Whether the assembly has five parts or thousands of parts, The unavailability of just one part is a show stopper.  Halt production.  And I'm not sure if you caught that part at the end about the BOM freeze.  Even for non-life critical applications oftentimes you can't just make part and vendor substitutions at will.  Being design engineers I know for sure that we should appreciate the fact that just because the data sheets look the same on first pass does not mean the parts are interchangeable in a specific application.  In this case, the consequences of part incompatibility isn't another BOM revision and maybe some RMAs, it's dead people.

If you can say that about production stuff, then you can say the same, ever more so, about hastily hacked together solutions.
And therein lies a possible problem with the entire concept in the first place.
Perhaps it was simply never possible to develop something like this so quickly when a lot of supply chains and production systems of any type were shut, and the product was required to save lives and not take them.

It might have been possible, iff the right skills were available. That means a working team with the specific domain knowledge,  implementation skills, manufacturing experience, supply line experience.

A group of amateurs was doomed to failure. The American good-ole-boy "its just like shooting swamp rats" attitude is no more than a fantasy.

OTOH, the better amateurs were able to help with making things like face masks/visors. Even there I've seen people want to do something without having any clue whether the result would be acceptable and/or usable.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 
The following users thanked this post: Smokey

Offline EEVblog

  • Administrator
  • *****
  • Posts: 39025
  • Country: au
    • EEVblog
Re: What did we learn from the "open source ventilator" mess.......
« Reply #64 on: July 10, 2020, 09:37:12 am »
So most of the arguments being made are assuming pre Corona virus lockdown supply chain and vendor availability options.  "Just order more parts, what's the problem?". That completely ignores the fact that whole factories were shut down, whole supply chain shipping options were shut down, and whole global economies were shut down.  That video with the descriptions of parts sourcing issues from the JPL supply chain person really makes that clear.  They needed to do a new design based on stuff they could actually get.  Whether the assembly has five parts or thousands of parts, The unavailability of just one part is a show stopper.  Halt production.  And I'm not sure if you caught that part at the end about the BOM freeze.  Even for non-life critical applications oftentimes you can't just make part and vendor substitutions at will.  Being design engineers I know for sure that we should appreciate the fact that just because the data sheets look the same on first pass does not mean the parts are interchangeable in a specific application.  In this case, the consequences of part incompatibility isn't another BOM revision and maybe some RMAs, it's dead people.

If you can say that about production stuff, then you can say the same, ever more so, about hastily hacked together solutions.
And therein lies a possible problem with the entire concept in the first place.
Perhaps it was simply never possible to develop something like this so quickly when a lot of supply chains and production systems of any type were shut, and the product was required to save lives and not take them.

It might have been possible, iff the right skills were available. That means a working team with the specific domain knowledge,  implementation skills, manufacturing experience, supply line experience.

You mean like Medtronics and other makers that already had and produced existing working certified designs?  ;D
 

Offline SerieZ

  • Regular Contributor
  • *
  • Posts: 191
  • Country: ch
  • Zap!
Re: What did we learn from the "open source ventilator" mess.......
« Reply #65 on: July 10, 2020, 09:46:14 am »
About ramping up production: My experience is, that ramping up some parts of production is not hard. The PnP machine easily can handle larger quantities, more shifts. Same goes for injection moulding. And the manual parts of assembly can be augmented by more people, or smarter organisation. This is usually not the hard part.
What I've seen is that the bottleneck is usually the testing of the device. They very likely functionally test each and every ventillator before shipping it. And the test bench of this is going to be the bottleneck. For example, if is possible that they have an artificial lung, government approved, calibrated, specifically designed to test ventilators. And then you need to connect your device to this, and do a x hour burn in test, to avoid early failures, and to prove your device is assembled correctly. Could be two hours, could be 24 hours, who knows? The test length probably was designed to reduce the chance of early failure to the ppm level. And that lung with all the sensors and equipment connected to it is very time consuming to put together, very expensive, and requires exotic parts, that you need to order yourself. And when you order them, they have a lead time. And very expensive. So if you want to just 2x or 3x your throughput, you might just pull in more shifts.
But you want x10? Invest hundreds of thousands in equipment, that will just sit there afterwards? Hire people in the middle of the pandemic for a new job to work at night? What if you just run out of factory floor space, because all these require space. Pull up a building next to it? Or assemble medical equipment in tents?

You are correct. Every single PCB inside the Ventilator is ICT ,Functionally and/or burn in tested and the final assembled machine as well undergoes several testing. Every single machine is signed off and not just shipped like a regular product either. This happens in several facilities (supplier and the company who builds the final assembled machine as well).

Just think about it: The Global market leader used to produce maybe 1500 Ventilators a Year and now the USA alone wants 100000 ASAP.

Regarding the rest of the Thread: a noble pursuit those DIY Ventilators and developed in a month or two by Nasa/Tesla... Ventilators but not quite feasible IMHO for many many reasons.
As easy as paint by number.
 

Online tggzzz

  • Super Contributor
  • ***
  • Posts: 21225
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: What did we learn from the "open source ventilator" mess.......
« Reply #66 on: July 10, 2020, 09:59:48 am »
So most of the arguments being made are assuming pre Corona virus lockdown supply chain and vendor availability options.  "Just order more parts, what's the problem?". That completely ignores the fact that whole factories were shut down, whole supply chain shipping options were shut down, and whole global economies were shut down.  That video with the descriptions of parts sourcing issues from the JPL supply chain person really makes that clear.  They needed to do a new design based on stuff they could actually get.  Whether the assembly has five parts or thousands of parts, The unavailability of just one part is a show stopper.  Halt production.  And I'm not sure if you caught that part at the end about the BOM freeze.  Even for non-life critical applications oftentimes you can't just make part and vendor substitutions at will.  Being design engineers I know for sure that we should appreciate the fact that just because the data sheets look the same on first pass does not mean the parts are interchangeable in a specific application.  In this case, the consequences of part incompatibility isn't another BOM revision and maybe some RMAs, it's dead people.

If you can say that about production stuff, then you can say the same, ever more so, about hastily hacked together solutions.
And therein lies a possible problem with the entire concept in the first place.
Perhaps it was simply never possible to develop something like this so quickly when a lot of supply chains and production systems of any type were shut, and the product was required to save lives and not take them.

It might have been possible, iff the right skills were available. That means a working team with the specific domain knowledge,  implementation skills, manufacturing experience, supply line experience.

You mean like Medtronics and other makers that already had and produced existing working certified designs?  ;D

Amongst others, and possibly competent new entrants into the field.

Established manufacturers ought to be looking at preserving their reputation after the immediate crisis has passed. They might take actions to ensure that "panic products" aren't associated with their name.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline SmokeyTopic starter

  • Super Contributor
  • ***
  • Posts: 3042
  • Country: us
  • Not An Expert
Re: What did we learn from the "open source ventilator" mess.......
« Reply #67 on: July 10, 2020, 12:33:30 pm »
So most of the arguments being made are assuming pre Corona virus lockdown supply chain and vendor availability options.  "Just order more parts, what's the problem?". That completely ignores the fact that whole factories were shut down, whole supply chain shipping options were shut down, and whole global economies were shut down.  That video with the descriptions of parts sourcing issues from the JPL supply chain person really makes that clear.  They needed to do a new design based on stuff they could actually get.  Whether the assembly has five parts or thousands of parts, The unavailability of just one part is a show stopper.  Halt production.  And I'm not sure if you caught that part at the end about the BOM freeze.  Even for non-life critical applications oftentimes you can't just make part and vendor substitutions at will.  Being design engineers I know for sure that we should appreciate the fact that just because the data sheets look the same on first pass does not mean the parts are interchangeable in a specific application.  In this case, the consequences of part incompatibility isn't another BOM revision and maybe some RMAs, it's dead people.

If you can say that about production stuff, then you can say the same, ever more so, about hastily hacked together solutions.
And therein lies a possible problem with the entire concept in the first place.
Perhaps it was simply never possible to develop something like this so quickly when a lot of supply chains and production systems of any type were shut, and the product was required to save lives and not take them.

I think we are converging.  I do think the hacked together solutions were doomed to failure for a whole number of reasons including parts issues, complexity, and especially specialized knowledge and experience, which all should have obviously put the project out of reach of your average garage maker-gineer but didn't for a lot of people who still had delusions of saving the world. 

But from what I can tell about the JPL solution, it doesn't look very hacked together.  It looks more mars rover like than an ambubag squeezed by a windshield wiper motor between two pieces of plywood controlled by an Arduino.  They apparently had the kind of team that could do the work and the understanding that they needed to source parts they could still get even with the jacked up supply chain.

The Medtronic guys certainly know how to make a ventilator, but that doesn't mean they know how to make a new model ventilator super quickly.  That's sort of like asking Tektronix to make a new scope model super quickly.  That's not how that company works.  They take their damn sweet time, and everyone knows it.  Instead pass the proven spec to the skunkworks teams and let them do what they do best with the implementation. 
« Last Edit: July 10, 2020, 12:41:43 pm by Smokey »
 

Offline tooki

  • Super Contributor
  • ***
  • Posts: 13156
  • Country: ch
Re: What did we learn from the "open source ventilator" mess.......
« Reply #68 on: July 10, 2020, 01:28:00 pm »
You're kidding right? You have the most litigious society in the world.  ::)

By total cost of litigation, the US does lead (followed by Canada and the UK). But in terms of litigiousness (number of lawsuits per capita), the US is only #5. The UK is #6. Places 4 through 1 are Austria, Israel, Sweden, and Germany!!

Source: https://www.jurorsrule.com/10-most-litigious-countries-in-the-world/

What I find very interesting is that the US is fifth place by a pretty wide margin. Another page I found listed these numbers (cases/1000 people):

1) Germany: 123.2/1,000

2) Sweden: 111.2/1,000

3) Israel: 96.8/1,000

4) Austria: 95.9/1,000

5) U.S.: 74.5/1,000

6) U.K.: 64.4/1,000

7) Denmark: 62.5/1,000

8 ⁠) Hungary: 52.4/1,000

9) Portugal: 40.7/1,000

10) France: 40.3/1,000

(The source of those numbers is said to be  ‘Exploring Global Landscapes of Litigation’ by Christian Wollschlager, a 1998 book. Bear in mind that IMHO, frivolous lawsuits in USA peaked in the 80s and 90s, but were never as big a problem as the media made it out to be.)
« Last Edit: July 10, 2020, 01:30:24 pm by tooki »
 
The following users thanked this post: Gyro

Offline EEVblog

  • Administrator
  • *****
  • Posts: 39025
  • Country: au
    • EEVblog
Re: What did we learn from the "open source ventilator" mess.......
« Reply #69 on: July 10, 2020, 02:55:52 pm »
The Medtronic guys certainly know how to make a ventilator, but that doesn't mean they know how to make a new model ventilator super quickly.

Nobody is saying they had to make a new design, just make more of what they already produced.
Let's be serious here, if it was that important, like war-time important, like everyone-dying important, the government could have made that production happen.
There was literally nothing stopping that production happening but government mandated shutdowns and fear.
Let's be practical, whether those mythical 100,000 units came from backyard labs or from existing manufacturers, it still requires very similar levels of local and global logistics to make that happen.

Remember, even during the worst of this whole thing, people were still working keeping society running, by several accounts up to 40% of the workforce were still out there doing their thing. This could have included emergency manufacture of medical equipment if it was so mandated.
 

Offline CatalinaWOW

  • Super Contributor
  • ***
  • Posts: 5569
  • Country: us
Re: What did we learn from the "open source ventilator" mess.......
« Reply #70 on: July 10, 2020, 03:17:32 pm »
I've been through production ramp up on equipment with quality controls.  The analogy often used is making babies.  You want 100,000 babies it is pretty easy to line up the means of production.  But there is this one critical nine month process step.  No matter how big you make the pipe or how hard you push it, it takes most of a year before product starts flowing.

In the real case the process steps are usually not as long, nor as impossible to change, but there are many more of them.  A moment's inattention and the whole thing slams to a halt.
 
The following users thanked this post: Smokey

Offline DrG

  • Super Contributor
  • ***
  • !
  • Posts: 1199
  • Country: us
Re: What did we learn from the "open source ventilator" mess.......
« Reply #71 on: July 10, 2020, 04:02:36 pm »
The Medtronic guys certainly know how to make a ventilator, but that doesn't mean they know how to make a new model ventilator super quickly.

Nobody is saying they had to make a new design, just make more of what they already produced.
Let's be serious here, if it was that important, like war-time important, like everyone-dying important, the government could have made that production happen.
There was literally nothing stopping that production happening but government mandated shutdowns and fear.
Let's be practical, whether those mythical 100,000 units came from backyard labs or from existing manufacturers, it still requires very similar levels of local and global logistics to make that happen.

Remember, even during the worst of this whole thing, people were still working keeping society running, by several accounts up to 40% of the workforce were still out there doing their thing. This could have included emergency manufacture of medical equipment if it was so mandated.

Well, a couple of things come to mind and I may be wrong. BLUF:Sometimes companies and governments are too big to get the job done quickly and efficiently. It is sometimes hard to get out of their own way. Sad as that is.

I have always enjoyed reading about the Manhattan project. I remember seeing a Government person (it may have been military, I can't remember) talking about it in a documentary. This particular documentary featured the amazing coalescence of effort that was required. The individual noted that today, we could probably not get the environmental impact statement completed in the time that the entire project took - sobering.

Ventilator shortage was recognized well before Covid and at the time of SARS and through EBOLA, the US was apparantly trying to find a solution see: https://www.nytimes.com/2020/03/29/business/coronavirus-us-ventilator-shortage.html

Medtronic did, in fact, ramp up production (by 40% according to this press release in March - http://newsroom.medtronic.com/news-releases/news-release-details/correction-medtronic-continuing-increase-ventilator-production).
- Invest in science - it pays big dividends. -
 

Offline SparkyFX

  • Frequent Contributor
  • **
  • Posts: 676
  • Country: de
Re: What did we learn from the "open source ventilator" mess.......
« Reply #72 on: July 10, 2020, 09:05:03 pm »
I'm saying that if the government put in a huge order Medtronics (and others) could have ramped up production, and done so vastly easier and quicker and to certification than anyone dicking around with the source files they released.
It was a move that got them out of the line of fire, because they (as all medical device manufacturers) probably have or see a very limited ability to ramp up and still need to supply customers in the future. At the same time they avoid allegations to work by market rules (law of supply and demand) in times of a crisis. Money might not even be a thing when a government is involved, unless someone tries to make it "a deal".

I guess they could make more test equipment themselves and ship it, unless calibration is involved the type of part required is basically in their machines anyway or because they have spare parts to keep the few they have running. Unfortunately outsourcing this kind of work is also a typical thing to do in the industry.

Maybe the ventilators even have a built in self test to be used while in service. Two machines might even be able to proof each other during a burn in phase to take that out of the equation, but that was probably never in the spec sheet. But yeah, without a proper test jig even the whole license build is too risky to do.

There might be more limiting factors, like fixed times to dissipate chemicals involved (plastic softening agents, glue) or certain parts built to order (e.g. injection molded plastic). I don't think hiring employees or standard parts are a limiting factor as during lock down unemployment went up anyway, although the request was before that.

That NASA can pull their own device off is no wonder, they know how to supply oxygen to people in a vacuum, have experience with these parts and their complications and most medical requirements.
Support your local planet.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf