MTBF is a Statistic, Not the Only One

We often face just a sample of life data with the request of estimating the reliability of the system. Or, we have a touch of test results and want to know if the product is reliability enough, yet. Or, we gather repair times to grapple with spares stocking.

We need to know the reliability. We need to know the number.

MTBF (or close cousin MTTF) is just that number. It is easy to calculate. A higher number means the system is more reliable. And, the metrics are in the units of time, often hours, which is easy to understand (and misunderstand).

In early chapters of reliability engineering books, or in introduction to reliability, we learn about the exponential distribution and the population parameter, theta. We also learn about the sample statistics which provides an unbiased estimated for the population parameter. In both cases, MTBF, or the mean time between failure, is the one value we have to master.

Other Statistics

Reliability is pretty easy using just one statistic. One calculation, one number, and we’re done.

Then a couple of things start to happen.

First, we notice that the actual time to failure behavior is not predicted, nor follows, the expected pattern when using just MTBF and the exponential distribution. The average time to fail changes as the system ages. We find that we run out of spares based on calculations using MTBF as the parts fail more and more often.

Second, we learn just a little more. We turn the page in the book or attend another webinar. We hear about another distribution commonly used in reliability engineering. The Weibull distribution. But, wait, hold on there. The Weibull has two and sometimes three parameters. I’ll need to learn about plotting, censored data, regression analysis, goodness of fit, confidence intervals, and a bunch of statistical methods.

Life was good with just one statistic.

We didn’t sign up to be reliability statisticians.

Well, too bad.

Actually, when using even just the one statistic, MTBF, we also should have been

• Checking assumptions
• Fitting the data to the exponential distribution function
• Evaluating the goodness of fit
• Calculating confidence bounds
• And, using those other statistical methods

In order to understand and use our sparse and expensive datasets, we need to use the tools found in the statistics textbooks.

Yes, the Weibull distribution has two or three parameters, thus we need to evaluate how well our statistics describe the data in a more rigorous way. And, we learn so much more. For example, we can model and predict a system with decreasing or increasing failure rates over time. We can estimate the number of required spares next year with a bit more accuracy then using just MTBF.

There are more benefits. Have you advanced past the basic introduction and embraced the use of reliability statistics? How’s it going and what challenges are you facing?

When to Use MTBF as a Metric?

I will not say ‘never’, which is probably what you expect. There are a rare set of circumstances which may benefit with the use of MTBF as a metric. Of course, this does not include being deceitful or misleading with marketing materials. There may actually be an occasion where the MTBF metric works well.

As you know, MTBF is often estimated by tallying up the total hours of operation of a set of devices or systems and dividing by the number of failures. If no failures occur we assume one failure to avoid dividing by zero (messy business dividing by zero and to be avoided). MTBF is essentially the average time to failure.

Expected Value as Metric

The metric we select should be measurable and of a measure we have an interest. We would like to detect changes, measure progress, and possibly make business decisions with our metrics. If we are interested in the expected value of the time to failure for our devices, then MTBF might just be useful.

When making a device we often hear of executives, engineers and customers talk about how long they expect the product to last. An office device may have an expected life of 5 years, a solar power system – 30 years, and so on. If by duration we all agree that we expect 5 years of service on average, then using the average as the metric makes sense.

Before starting the use of MTBF, just make sure that a 5 year life implies half or two thirds of the devices will fail by the stated duration of 5 years. Yes, if the time to failure distribution is actually described by the exponential distribution (and a few other distributions) it means that two third of the units are expected to failure by the MTBF value. Thus if we set the goal to 5 years MTBF we imply half or more of the units will fail by 5 years.

Product Testing Advantages

Having a goal helps the design and development team make decisions and eventually conduct testing to prove the design meets the reliability objectives. Setting the goal a the expected value allows the fewest number of samples for testing. Testing for 99% reliability over 5 years is much tougher. We may require many samples to determine a meaningful estimate of the leading tail (i.e. first 1% or 5% of failures) of the time to failure distribution.

If the time failure pattern fits an exponential distribution, then testing becomes simplified. We can test one unit for a long time, or many units a short time, and arrive at the same answer. The test planning can maximize our resources to efficiently prove our design meets the objective. When the chance of failure each hour is the same, every device-hour of testing provide an equal amount of information.

Unlike products that wear out or degrade with time, when the design and device exhibit an exponential distribution we do not need any aging studies. We can just apply use or accelerated stress and measure the hours of operation and count the failures. Also any early failures are obviously quality issues and most likely do not count toward failures that represent actual field failures. Or do they?

Metrics Should Have a Common Understanding

When the industry, organization, vendors, and engineering staff already use MTBF to discuss reliability, then management would be wise to establish a metric using MTBF. Makes sense, right? The formula to calculate MTBF is very simple. Even the name implies the meaning (no pun intended). MTBF is the mean time between (or before) failure. It’s an average, which calculators, spreadsheets, smart phones, and possibly even your watch can calculate.

While the spread of the data is often of importance when making comparisons, estimating a sample set of data’s confidence bounds, or estimating the number of failures over the warranty period, if we assume the data actually fits an exponential distribution, we find the mean equals the standard deviation. Great! One less calculation. We have what we need to move forward.

Nearly every reliability or quality textbook or guideline includes extensive discussions about MTBF the exponential distribution and a wide range of reliability related calculations. Our common understanding generally is supported by the plentiful references.

Ask a few folks around you when considering using MTBF. What do they define MTBF as representing? If you receive a consistent answer, you may just have a common understanding. If the understanding is also aligned with the underlying math and assumptions, even better.

When to Use MTBF Checklist

In summary all you need is:

• A business interest in the time till half or more of product fail
• A design with a fixed chance to failure each hour of operation
• A well educated team that understands the proper use of an inverse failure rate measure

I submit we are rarely interested in the time till the bulk of devices fail, rather interested in the time to first failures or some small percentage fail

I suggest that very few devices or system actually fail with a constant hazard rate. If your product does, prove it without grand waves of assumptions.

I have found that engineers, scientists, vendors, customers, and manager regularly misunderstand MTBF and how to properly use an MTBF value.

So back to the opening statement, it is possible though not likely you will find an occasion to effectively use MTBF as a metric. Instead use reliability: the probability of successful operation over a stated period with stated conditions and definition of success. 98% of office printers will function for 5 years without failure in a office…. Pretty clear. Sure we can fully define the function(s) and environment, and we need to do that anyway.

Looking Forward to the MTBF Report

On social media the other day ran across a comment from someone that took my breath away. They were looking forward to starting a new reliability, no, MTBF report. They were tasked with creating a measure of reliability for use across the company and they choose MTBF.

Sigh.

Where have we gone wrong?

I certainly do not blame the person. They have read about MTBF in many textbooks. Studied reliability using MTBF and related measures, plus found technical papers using the same. They may have seen industry reports and standards also.

MTBF is prevalent and no wonder someone tasked with setting a metric would select MTBF. It’s easy to calculate. Just one number and bigger is better.

On the other hand

MTBF is roundly criticized across any reliability related forum or discussion group. There is progress in books, papers and standards. And, it’s not reaching those new to reliability engineering.

This note will be short and have one request. Please tell those just getting started in reliability engineering to please not consider using MTBF. To not request MTBF from vendors. And, to actually do some thinking before selecting MTBF as their organizations metric.

Better yet, challenge those using MTBF to explain in a coherent and rational manner why they are doing so. Ask them to validate their assumed constant failure rate or similar assumptions. Working together we can start a ripple that may help build the wave of knowledge to improve the state of reliability engineering.

Fixing Early Life Failures Can Make Your MTBF Worse

Let’s say we 6 months of life data on 100 units. We’re charged with looking at the data and determine the impact of fixing the problems that caused the earliest failures.

The initial look of the data includes 9 failures and 91 suspensions. Other then the nine all units operated for 180 days. The MTBF is about 24k days. Having heard about Weibull plotting and using the beta value as a guide initially find the blue line in the plot. The beta value is less than one so we start looking for supply chain, manufacturing or installation caused failures, as we suspect early life failures dominant the time to failure pattern.

Initial Steps to Improve the Product

Given clues and evidence that some of the products failed early we investigate and find evidence of damage to units during installation. In fact it appears the first four failures were due to installation damage. The fix will cost some money, so the director of engineer asks for an estimate of the effect of the change on the reliability of the system.

The organization uses MTBF as does the customer. The existing MTBF of 24k days exceeds the customers requirement of 10K days, yet avoiding early problems may be worth the customer good will. The motivation is driven by continuous improvement and not out of necessity or customer complaints.

Calculation of Impact of Change on Reliability

One way to estimate the effect of a removal of a failure mechanism is to examine the data without counting the removed failure mechanism. So, if the change to the installation practice in the best case completely prevents the initial four failures observed we are left with just the 5 other failures that occurred over the 6 months.

Removing the four initial failures and calculating MTBF we estimate MTBF will change to about 300 days.

Hum?

We removed failures and the MTBF got worse?

What Could Cause this Kind of Change?

The classic calculation for MTBF is the total time divided by the number of failures. Taking a closer look at time to failure behavior of the two different failure mechanisms may reveal what is happening. The early failures have a decreasing failure rate (Weibull beta parameter less than 1) over the first two months of operation. Later, in the last couple of months of operation, 5 failures occur and they appear to have an increasing rate of failure (Weibull beta parameter greater than 1).

By removing the four early failures the Weibull distribution fit changes from the blue line to the black line (steeper slope).

Recall that the MTBF value represents the point in time when about 63% of units have failed. With only 9 total failures out of 100 units we have only about 10% of units failed so the MTBF calculation is a projects to the future when most of have failed, it does not providing information about failures at 6 months or less directly.

In this case when the four early failures are removed the slope changed from about 0.7 to about 5, it rotated counter clockwise on the CDF plot.

If only using MTBF the results of removing four failures from the data made the measured MTBF much worse and would have prevented us from improving the product. By fitting the data to a Weibull distribution we learned to investigate early life failures, plus once that failure mechanism was removed revealed a potentially serious wear out failure mechanism.

This is an artificial example, of course, yet it illustrates the degree which an organization is blind to what is actually occurring by using only MTBF. Treat the data well and use multiple methods to understand the time to failure pattern.

The Reliability Metric

A Quick and Valuable Improvement Over MTBF

Finished it. 130 pages long and packed with advice on why and how to switch from MTBF to reliability.

Based in large part on comments, feedback, discussions and input from you, my peers in the NoMTBF tribe. Thanks for the encouragement and support.

The Reliability Metric book is available here. Continue reading The Reliability Metric Book Announcement →

Determine MTBF Given a Weibull Distribution

First off, not sure why anyone would want to do this, yet one of the issues I’ve heard concerning abandoning the use of MTBF is client ask for MTBF. If they will not accept reliability probabilities at specific durations, and insist on using MTBF, you probably should provide a value to them.

Let’s say you have a Weibull distribution model that described the time to failure distribution of your product. You’ve done the testing, modeling, and many field data analysis and know for the requestor’s application this is the best estimate of reliability performance. You can, quite easily calculate the MTBF value.

As you know, if theβ parameter is equal to one then the characteristic life, η, is equal to MTBF. If β is less than or greater than one, then use the following formula to determine the mean value, MTBF, for the distribution.

You’ll need the Gamma function and the Weibull parameters. The further β is from one, the bigger the difference between η and MTBF.

You can find a little more information and background at the article Calculate the Mean and Variance on the accendoreliability.com site under the CRE Preparation article series.