How to Estimate Reliability Early in a Program

In a few discussions about the perils of MTBF, individuals have asked about estimating MTBF (reliability) early in a program. They quickly referred to various parts count prediction methods as the only viable means to estimate MTBF.

One motivation to create reliability estimates is to provide feedback to the team. The reliability goal exists and the early design work is progressing, so estimating the performance of the product’s functions is natural. The mechanical engineers may use finite element analysis to estimate responses of the structure to various loads. Electrical engineers may use SPICE models for circuit analysis.

Customers expect a reliable product. If they are investing in the development of the product (military vehicle, custom production equipment, or solar power plant, for examples) they may also want an early estimate of reliability performance.

Engineers and scientists estimate reliability during the concept phase as they determine the architecture, materials, and major components. The emphasis is often on creating a concept that will deliver the features in the expected environment. The primary method for reliability estimation is engineering judgement.

With the first set of designs, there is more information available on specific material, structures, and components, thus it should be possible to create an improved reliability estimate.

Is testing the true way to estimate MTBF?

Early in a program means there are no prototypes available for testing, just bill of materials and drawings. So, what is a reliability engineer to do?

One could argue that without prototypes or production units available for testing (exercising or aging the system to simulate use conditions) we do not really know how the system will respond to use conditions. While it is true it is difficult to know what we do not know, we often do know quite a bit about the system and the major elements and how they individually will respond to use conditions.

Even with testing, we often use engineering judgement to focus the stresses employed to age a system. We apply prior knowledge of failure mechanism models to design accelerated tests. And, we use FMEA tools to define the areas most likely to fail, thus guiding our test development.

Creating a reliability estimate without a prototype

Engineering judgement is the starting point. Include the information from FMEA and other risk assessment methods to identify the elements of a product that are most likely to fail, thus limit the system reliability. Then there are a few options available to estimate reliability, even without a prototype.
First, it is rare to create a new product using all new materials, assembly methods, and components.

Often a new product is approximately 80% the same as previous or similar products. The new design may be a new form factor, thus mostly a structural change. It may includes new electronic elements – often just one or two components, where the remaining components in the circuit regularly used. Or, it may involve a new material, reusing known structures and circuits.

Use the field history of similar products or subsystems and engineering judgement for the new elements to create an estimate. A simple reliability block diagram may be helpful to organize the information from various sources.

For the new elements of a design, base the engineering judgement on analysis of the potential failure mechanisms, employ any existing reliability models, or use simulations to compare known similar solutions to the new solution.

Second, for the elements without existing similar solutions and without existing failure mechanism models, we would have to rely on engineering judgement or component or test coupon level testing. Rather than wait for the system prototypes, early in a program it is often possible to obtain samples of the materials, structures, or components for evaluation.

The idea is to use our engineering judgement and risk analysis tools to define the most likely failure mechanisms for the elements with unknown reliability performance. Let’s say we are exploring a new surface finish technique. We estimate that exposure to solar radiation may degrade the finish. Therefore, obtain some small swatches of material, apply the surface finish and expose to UV radiation. While not the full product using fully developed production processes, it is a way to evaluate the concept.

Another example, is a new solder joint attachment technique. Again, use your judgement and risk analysis tools to estimate the primary failure mechanisms, say thermal cycling and power cycling, then obtain test packages with same physical structures (the IC or active elements do not have to be functional) and design appropriate tests for the suspected failure mechanisms.

Estimate combine the available knowledge

With a little creativity we can provide a range of estimates for elements of a design that have little or no field history. We do not need to rely on a tabulate list of failure rates for dissimilar product created by a wide range of teams for diverse solutions. We can draw from our team’s prior designs actual field performance for the bulk of the estimate. Then fill in the remaining elements of the estimate with engineering judgement, comparative analysis, published reliability models, or coupon or test structure failure mechanism evaluations.

In general, we will understand the bulk of the reliability performance and have rational estimates for the rest. It’s an estimate and the exercise will help us and the team focus on which areas may require extensive testing.