Define Kill Criteria to Avoid Zombie Projects

When pursuing aggressive benchmarks, engineers must employ portfolio thinking, running multiple design projects simultaneously. But choosing winners requires a decisive way to eliminate projects that are not feasible to continue innovating, often referred to as a "project killer". 

In this episode, we analyze Tesla's battery development as a case study. We delve into their use of five clear-cut constraint categories that define failure conditions upfront: the Economic filter, Performance filter, Scalability filter, Resource filter, and System filter. 

We discuss the challenges engineers face in letting go of projects due to the sunk cost fallacy, where prior investments irrationally influence future choices, leading to the creation of "zombie projects". 

Learn why defining explicit kill criteria before development begins is a vital, often overlooked exercise that saves resources and ensures rational decision-making.

Blog for this episode: https://deeneyenterprises.com/qdd/podcast/define-kill-criteria-to-avoid-zombie-projects/

Episode with Dianna's review of Annie Duke's "Quit": Exploring Product Development and AI Through Literature: Insights from 'Loonshots', 'AI 2041', 'Quit', and "How Big Things Get Done' (QDD Book Cast) - Deeney Enterprises

Are your teams struggling with poor communication and rushed timelines? Is your product vision clouded by a lack of clarity? It's time to find your way through the confusion and build products that truly resonate with users.

Introducing "Pierce the Design Fog" by Dianna Deeney, the essential guide to turning abstract ideas into high-quality products. This book offers a proven playbook with practical frameworks and tools to help you foster team synergy, lead with vision, and ma

JOIN ME ON SUBSTACK Subscribe today.

GET THE BOOK Pierce the Design Fog

ABOUT DIANNA
Dianna Deeney is a quality advocate for product development with over 25 years of experience in manufacturing. She is president of Deeney Enterprises, LLC, which helps organizations and people improve engineering design.

Chapters

• 0:00 Framing Tesla’s Battery Targets
• 4:05 Portfolio Thinking And Parallel Chemistries
• 6:48 Five Clear Kill Criteria
• 11:48 Avoiding Zombie Projects And Sunk Costs
• 14:57 Psychology Of Quitting And Annie Duke

Transcript

SPEAKER_00: 2:43

Welcome back to Quality During Design. Today we're looking at Tesla's aggressive battery targets. To manage the risk of pursuing multiple battery chemistries simultaneously, a necessary element of portfolio thinking, Tesla developed a framework to identify winners and eliminate losers. As engineers, the hardest thing we do is kill a project, often falling victim to the sunk cost policy, where prior investments of our resources influence our future choices, risking the creation of a zombie project. Tesla proactively defined failure conditions up front using five constraint categories, including economic, performance, and system filters, to measure project viability. Join me as we explore why defining these explicit kill criteria before development begins is the most valuable and often overlooked exercise in product engineering. Welcome to Quality During Design, the place to use quality thinking to create products others love, for less time, less money, and a lot less headache. I'm your host, Diana Deaney. I'm a senior quality engineer with over 20 years in manufacturing and product development and author of Pierce the Design Fog. I help design engineers apply quality and reliability thinking throughout product development, from early concepts through technical execution. Each episode gives you frameworks and tools you can use. Want a little more? Join the Substack for monthly guides, templates, and QA where I help you apply these to your specific projects. Visit qualityderingdesign.com. Let's dive in. Welcome back. I've been reading up on Tesla's battery development program and looking at it from a case study point of view, starting with their 2020 battery day. All sorts of versions of this are on YouTube that you can go and view it yourself. In 2020, they announced some pretty aggressive Go criteria, like 56% reduction in battery cost per kilowatt hour, and a 54% increase in vehicle range. To meet these kind of targets, they started to self-develop their own battery, the 4680 cell, a custom design, custom manufacturing. They not only had these types of benchmarks that they wanted to meet, but they also had to keep producing and meeting other targets for their existing product line. So they started running multiple battery chemistries simultaneously. And we can all sort of guess why. When you're in development, you're not sure about the sure thing. You don't want to put all your eggs in one basket, so to speak. You don't want to focus all of your resources onto one idea. In case it fails, you have a backup plan. That's the portfolio thinking of this case study, right? You run parallel options, and then throughout you systematically choose the winners. So the company has these targets they want to achieve, and they're doing design and innovation to be able to meet that. And those are a good criteria to have because they are benchmarks, your goal, what you're trying to achieve. And the closer you can get to that, the better. But there's a whole lot of other things going on too. We also need a project kill criteria. If we're developing multiple things at once to be able to reach the same end goal, you need to be able to decide when to stop pursuing some of the other options. Not only where to focus with what's performing the best, but what's not going to be feasible to continue to innovate. Some of the constraint categories that I'm seeing that Tesla used were cost, this is the economic filter. For example, this is the 56% target they announced on their battery day. So if the innovation raises the pack price or fails to hit the cost curve, it's rejected regardless of density. And speaking of density, that's their performance filter. They wanted to significantly improve the volumetric and gravimetric density over existing cells. So if the innovation path that they're developing only offers marginal improvements or compromises safety for density, it's rejected. So those are two constraint categories or kill criteria for this portfolio management, economics and performance. But I'm seeing that they had maybe three other constraint categories. One was scalability, which is related to manufacturability. Whatever they're developing must have been compatible with rapid, high volume production. And if the process required exotic materials, complex wet processing, or it can't be implemented on the dry battery electrode line, they abandoned it. The next to last category was a resource filter related to the supply chain. They wanted to utilize the globally abundant and environmentally stable materials. For example, they wanted to move away from the high cobalt materials. So if their development relies on the scarce, geopolitically sensitive, or environmentally damaging materials, they rejected it. Finally, the last constraint category that I saw was the system filter. They wanted to make sure that whatever they were developing integrated seamlessly into the vehicle structure, the cell to pack or the cell to vehicle. And if the design couldn't function as a structural element, the technology was rejected as it compromises the overall vehicle efficiency and cost. So if you think about this, this is Tesla's battery innovation development kill criteria, economics, manufacturability, performance, resources, and the system filters. And I think they're really well-defined and clear-cut criteria against which to measure whatever it is that they're developing, whatever it is they're managing. So think about your current projects that you're working on. You definitely have Go criteria. Those are the user needs, requirements, and other performance requirements that you're trying to meet. What is your kill criteria? Do you have any? You might have some, but you don't know what it is. How can you find out? And how can you use that to help you make decisions? When we're developing kill criteria, the project kill criteria, we're actually defining the failure conditions up front. Doesn't that make things clearer? You see this in non-engineering and product development communities also, talking a lot about New Year's resolutions right now. Sometimes people will give themselves a cutoff time. You know, I'm going to start this new business. I'm going to give myself I'm going to give myself two years to have this many customers. And if I don't, I'm going to pivot and do something different. The hardest thing to do when you've invested so much time and energy, research and resources into a project is to kill it. You're invested in it and you want to see it succeed. You want to see it through to the end. I think many of us engineers think that way. It may have something to do with how we were trained as engineers to find the problem and then keep digging or evolving or innovating until you find the solution. We might find a solution, but it may not be feasible against all the other criteria because it's not just performance, is it? It's also cost and manufacturability and what the customer wants, what the business needs. I sometimes refer to some of these other aspects of product design and engineering as our internal customers. Yes, we need to meet some outward external goals for our customers or whoever's going to be buying our product, but we also have a responsibility to engineer for many other people and divisions and things. So kill criteria is something that we want to be able to define for our projects. Project manager or some of the people that are deciding what projects are pursued in the business probably have this kill criteria. Maybe you don't know what it is. It might be worth finding out. If there is no kill criteria and you're just going to keep developing until you get it, your project might turn into a zombie project where it never really gets done, it never sees the light of day, and it either just fizzles out or eventually somebody pulls the plug. There can be sunk cost fallacies in our engineering projects, also, where we have an emotional attachment to our design or our project. The sunk cost fallacy is a cognitive bias where we continue investing in a decision based on our prior investments. So we've already put so much time, energy, and effort into whatever we're developing that it doesn't make sense for us to not continue developing it. Whereas really we're eliminating or not factoring in the future benefits or future non-benefits. If we continue investing in this innovation or this development, is it going to pay out in the end, really? Or is it better worth our time and resources to be working on something different? When we're being rational about our decision making, these costs that we've already incurred in our project should not influence our future choices. They're irretrievable. During product development, we may be working under our sunk cost fallacy. It's something we need to be aware of and recognize and pivot our own decision making if we need to when we recognize it. So in addition to the success thresholds of our projects, which we need, we should also define and get alignment on the kill criteria of our projects. And this is an often overlooked exercise. In her book, Quit: The Power to Know When to Walk Away, author Annie Duke shows us the value of knowing when to quit. She offers a lot of examples from different points of view, different industries. She even looks at the sport of boxing. She also gets into some of the psychological and cognitive biases that we have when it comes to quitting. And she promotes defining kill criteria in the beginning, quit criteria. I recommend reading it. It will help you identify when people are ignoring the quit signals or avoid setting criteria in the first place. I reviewed this book in a previous podcast episode. I'll link to it in the show notes. As engineers, we're also used to running tests with acceptance criteria, especially if they're late-stage designed tests. Our product has to perform within these windows or it's a failure. Defining kill criteria for projects is sort of like that. You're defining criteria for your project up front before you get into it. And that'll help you define when to quit so that your project doesn't become a zombie project. Define your kill criteria before you start testing, and for your projects, even before you start developing. In the next episode, I will talk about the expected value calculation that helps end the debate of whether to keep going or not. Join me on Substack for the Ask Me Anything post. This month it's called Choose It: How to Decide Between Viable Options when you'll never have 100% confidence. And then later in the month, I'll be publishing another post, including some of the information that I've discovered about Tesla's battery development and what we can derive from that and how we can use it for ourselves. Find me on Substack at quality during design. And if you're just finding me on this podcast, please subscribe to your favorite podcast provider. This has been a production of Deanny Enterprises. Thanks for listening.