anyone good at statistics or experimental design?

[engineheat]

Let's say you want to optimize a manufacturing process to minimize defect, by doing an experiment with different settings of a certain variable. Suppose for each setting, you product 100 parts, and make note of the defect rate. Let's say your experiment is composed of 10 different settings, and after all it's done, you make use of the setting that gives the least defect. But is there a rigorous statistical test to truly tell whether the outcome didn't happen due to chance?

And here's a related scenario. Suppose you want to optimize the current process. Based on the history of the previous 10000 parts made, you know the defect rate is x%. You use a new process, but due to expense, you only make an experimental run with 500 parts, and you get a defect rate of y%. What statistical test can you use to determine if the new process truly had an effect?

Thanks

[ogden]

First you have to properly understand everything about your defects. Unit that do not work may have more than single defect. When you get data about your defects right - THEN you think how to fix (causes of) them. Good place to start: https://www.itl.nist.gov/div898/handbook/index.htm.
[edit] Defects differ as well: 1) visual: like paint scratched 2) functional: like helper system for lazy do not work as expected 3) safety: like car "autopilot" kills everybody onboard