Reliability Goal and Confidence
We establish reliability goals and measure reliability performance.
They are not the same thing. Goals and measures, while related, are not the same nor serve the same purpose.
Recently, I’ve seen a few statements that seem to confuse the role of statistical confidence when establishing a goal. Thus, I would like to relate how I think about the difference of goals and statistical confidence along with how they are related.
The Purpose of a Reliability Goal
Setting any goal provides tangible direction or a meaningful target for the team. A reliability goal is a balance of
- what the customer expects,
- what is technically possible,
- and an expression of the business objectives.
A reliability goal establishs the probability a function will successfully operate over a specific duration given a specific use and environment. For example, my smart phone will make and receive calls in Northern California with a 99% probability of successful operation over 2 years.
The purpose of the goal to define to the engineering team, to the marketing team, to customers, the item’s expected or desired reliability performance.
It is not the actual reliability performance. It is what we would like to have happen. It’s a goal, objective, target, etc.
The purpose to provide guidance, alignment, and requirements for the creation of a new product or system.
The Purpose of Statistical Confidence
Statistical confidence is a statement of the acceptable risk we are willing to accept. The risk is based on the ability of a specific sample to accurately represent the entire population.
The chance that a single sample’s reliability performance will be the same as the population’s average reliability performance is very low. As we measure more samples from the population the ability of the sample to estimate the population’s performance increases.
There is some statistical law about this phenomenon.
Some organizations set a policy concerning the amount of risk they are willing to accept when using a sample to create an estimate. Other times we determine the statistical confidence based on how many samples we have evaluated.
The purpose of a statistical confidence is to manage the risk of the sample providing misleading information.
The goal is often to minimize sample size related risk within the constraints that exist limiting sample sizes.
The Relationship Between Goals and Confidence
Reliability goals are a desired state of performance. Confidence reflects the risk a sample based results is close to being true.
Stating a reliability goal that includes a confidence in effect defines the desired sample size to measure the reliability performance. That is not a bad relationship. Yet stated we want a reliability goal of 90% reliability over 2 years with 90% confidence, is not, in my mind, a goal.
We would like the actual reliability of the item to have a 90% chance of surviving 2 years. We may want to minimize sample risk by measure the reliability performance with a 90% confidence.
The estimate from a sample of the actual reliability performance will always be an estimate. The confidence provides a figure of risk that the results we are viewing actually reflect the real and unknown reliability performance.
If we use a one-sided lower bound for the confidence, we are looking for evidence that the reliability is at least above the lower bound value. We could set the threshold for the lower bound at the goal value. With a 90% confidence we are accepting the risk the actual population reliability is less then 90% and the sample results end with a value higher than the goal, thus we believe the product’s reliability performance is better than actual.
The relationship between a goal and confidence is they are both providing information about the unknown actual reliability performance. The former is the desired state, the later is a reflection of sample risk during measurement.
Let me know if this makes sense, or if there is a better way to describe the relationship between goals and confidence. Please comment and join the conversation.