Quality during Design
Quality during Design is a production of Deeney Enterprises, LLC. It is a podcast for product designers, engineers, and anyone else who cares about creating high-quality products. In each episode, we explore the principles of quality design, from user-centered thinking to iterative development. We introduce frameworks to make better design decisions and reduce costly re-designs. We explore ways to co-work with cross-functional teams. We also talk to experts in the field about their experiences and insights.
Join host Dianna Deeney in using quality thinking throughout the design process to create products others love, for less. Whether you're a seasoned designer or just starting out, looking to improve your existing designs or start from scratch, Quality during Design is the podcast for you.
Quality during Design
QDD Redux: Choosing a Confidence Level for Test using FMEA
Ever wondered if you're wasting resources by setting unnecessarily high confidence levels for your reliability requirements? You're not alone. Many engineering teams default to 95% or 99% confidence without considering the downstream impact on testing timelines and resources.
This episode tackles a question that's been coming up frequently from listeners: how to choose appropriate confidence levels for reliability requirements and test methods. Rather than making arbitrary decisions, I share a practical approach using your existing Failure Mode and Effects Analysis (FMEA) as a guide. This risk-based method helps you match your confidence levels to the actual risks associated with potential failures.
When you connect your testing strategy to your risk analysis, you create a logical framework for deciding where to invest more testing resources and where you might reasonably accept lower confidence levels. I walk through exactly how to do this by examining the severity of potential failures, the number of possible effects, and what other controls might already be in place. The beauty of this approach is that it leverages work your cross-functional team has already done during FMEA development, providing an objective foundation for your test planning decisions.
For those new to the Quality During Design podcast, this episode exemplifies our philosophy that emphasizes using quality tools early in the development process to make better decisions. Whether you're struggling with reliability testing or just looking to optimize your design process, you'll find practical insights to help you create better products with fewer resources. Subscribe to the podcast, visit qualityduringdesign.com for additional resources, or sign up for our monthly newsletter to stay informed about the latest quality design methodologies.
**BI-WEEKLY EPISODES**
Subscribe to this show on your favorite provider and Give us a Rating & Review to help others find us!
**ONLINE COURSE**
FMEA in Practice: from Plan to Risk-Based Decision Making is enrolling students now. Join over 200 students in taking your FMEA to the next level. Click Here
**MONTHLY DIGEST**
Subscribe to the free monthly e-newsletter at www.qualityduringdesign.com.
About me
Dianna Deeney is a quality advocate for product development with over 25 years of experience in manufacturing. She is president of Deeney Enterprises, LLC, which helps organizations optimize their engineering processes and team performance by promoting the use of reliability and quality methods during design. She offers consulting services for managers and directors, training for engineers through the Quality During Design program, and other practical resources.
Welcome to the Quality During Design podcast. I'm your host, diana Deeney. Just a few weeks ago I pulled an episode back to the forefront from the archive having to do with reliability requirements and since then I've gotten some feedback, input and messages about choosing a confidence level for our reliability requirement and I covered that in another previous episode of the Quality During Design podcast. So to address these ponderings and questions, I'm pulling another episode back from the archive having to do with choosing a confidence level for our reliability requirement or for our test method, and what we're going to use is we're going to use our FMEA failure mode and effects analysis Instead of blindly setting whatever confidence level we want failure mode and effects analysis. Instead of blindly setting whatever confidence level we want, we can base it off the risks of failure using our FMEA. I'm going to share that episode with you in a minute, but if you are a repeat listener of the Quality During Design podcast, welcome back. If you're new to the Quality During Design podcast, welcome. We talk a lot about product development and engineers working to create new products. Quality During Design is a philosophy that emphasizes the benefits of cross-functional team involvement in design. It's also a methodology. We use quality tools to refine design concepts early. So if you're involved in designing stuff and want or need to know how to do it better, if you want to avoid surprises during tests, design what your customers really want and have shorter design cycles, and also if you feel like you just need to do more with less and still create the best, we have some resources for you. I invite you to visit and bookmark the website qualityduringdesigncom. On that website, you can access and search through the podcast library. There's also additional training links and other offerings available that you can access for free. If you want to stay on top of what's the latest and greatest, then please sign up for our monthly newsletter. All of this can be done at qualityduringdesigncom. So, without further delay, I'll share this Quality During Design Archive episode about choosing confidence levels for tests and requirements using FMEA Enjoy. We're developing requirements for our product, including setting reliability requirements and its confidence levels, or we're setting acceptance criteria for our test plans. What confidence levels do we choose? We don't have to blindly set them. We can base it off the risks of failure. I'll tell you how. After this brief introduction, how. After this brief introduction, hello and welcome to Quality During Design, the place to use quality thinking to create products others love for less. My name is Diana. I'm a senior level quality professional and engineer with over 20 years of experience in manufacturing and design. Listen in and then join the conversation at qualityduringdesigncom.
Speaker 1:Before testing anything, we need to choose what confidence level we want to have in the results. We need to do this because there's variation in everything in the way that we measure and test. The way that we manufacture introduces variation, including the raw materials that we're using and the tools we're using to make it, including the raw materials that we're using and the tools we're using to make it. Setting a confidence level accounts for the variability we're going to see in our test data. A confidence level is used in determining the sample size to test. If we want to make statements about how the design will perform in the field, then we need to test a sample size that's statistically relevant, where we can use statistics to help us predict the performance in the field from a few tested in the lab. Usually, confidence levels are 90%, 95% or 99%. Why don't we take the most conservative approach and just pick a 99% confidence level when that may save us time in having to think about it? It wastes a lot of time and resources later. The higher the confidence level, the more likely we'll need to test lots of samples. The higher the confidence level, the more likely we'll need to test lots of samples, and that means making units for test, testing them all in the lab and then having a more complex analysis Instead. One way we can choose a confidence level that we want for test is to correlate it with the risk of failure associated with it.
Speaker 1:Our product requirement is likely a control for a potential failure. What was the origin of our requirement? Why did we set it in the first place? What performance or characteristic of the final design is it controlling? If our product doesn't meet this requirement, what are the ways that it can fail? If we have an FMEA, we can find the place in the table where our requirement is a control or where it's associated with a potential failure mode and cause. When we find it, then we have a lot of metrics we can use to help us decide on a level of confidence to test based on risk, and if we've done our FMEA earlier, then we would have had it populated with information from our cross-functional team. In a time of cool heads, without the pressures of project management. It will be an objective input into what confidence level we should require for our test.
Speaker 1:What are the potential effects of this failure mode? In other words, what type of harm to the user, environment or performance of the product is possible? Are there many effects listed or just one? If there are many effects, we may want a higher confidence level. What is the severity ranking of the effect? Is it high or is it low? The higher the severity ranking, the more likely we should choose a higher confidence level. What other controls are in place besides our requirement and what is the detection ranking? If this requirement is the only control or if it's the strongest control, then we may want to choose a higher confidence level. We could also use this information to justify a lower confidence level. If we have a requirement that's associated with a failure that has one effect, that effect is not severe and there are two other controls associated with that same cause, then maybe we'll choose a lower confidence level.
Speaker 1:What's today's insight to action? We should choose a confidence level for our requirements or their test plans. We can associate that confidence level with the level of risk of our product. Fmea is a great tool to refer to to help us choose a relevant confidence level for our tests. If this episode is helpful to you, I recommend two other previous Quality During Design episodes. Episode 27, how Many Controls Do we Need to Reduce Risk? Talks more about the controls that we use in an FMEA to control a risk. Episode 31, five Aspects of Good Reliability Goals and Requirements talks about why we want a confidence level associated with our requirement. Please go to my website at qualityduringdesigncom. You can visit me there and it also has a catalog of resources, including all the podcasts and their transcripts. Use the subscribe forms to join the weekly newsletter where I share more insights and links In your podcast app. Make sure you subscribe or follow Quality During Design to get all the episodes and get notified when new ones are posted. This has been a production of Dini Enterprises. Thanks for listening.
