The probabilistic statistical predictions based on broad assumptions of the underlying physical causes begin with the first electronics reliability prediction guide  begin November 1956, with the publication of the RCA release TR-1100, “Reliability Stress Analysis for Electronic Equipment”, which presented models for computing rates of component failures. This publication was followed by the “RADC Reliability Notebook” in October 1959, and the publication of a military reliability prediction handbook format known as MIL-HDBK-217.

It still continues today with various software applications which are progenies of the MIL-HDBK-217. Underlying these “reliability prediction assessment” methods and calculations is the assumption that the main driver of unreliability is due to components that have intrinsic failure rates moderated by the absolute temperature. It has been assumed that the component failure rates follow the Arrhenius equation and that component failure rates approximately doubles for every 10 °C.

MIL-HDBK-217 was removed from the military as reference document in 1996 and has not been updated since that time; it is still being reference unofficially by military contractors and still believed to have some validity even without any supporting evidence.

Much of the slow change in the industry is due to the fact that electronics reliability engineering has a fundamental “knowledge distribution” problem in that real field failure data, and the root causes of those failures can never be shared with the larger reliability engineering community. Reliability data is some of the most confidential sensitive data a manufacturer has, and short of a court order will never be published. Without this real data and information being disseminated and shared, one can expect little change in the beliefs of the vast majority of the electronics reliability engineering community.

Even though the probabilistic prediction approach to reliability has been practiced and applied for decades any engineer who has seen the root causes of verified field failures will observe that most all failures that occur before the electronic system is technologically obsolete, are caused by 1) errors in manufacturing 2) overlooked design margins 3) or accidental overstress or abuse by the customer.  The timing of the root causes of these failures, which many times are driven by multiple events or stresses, are random and inconsistent. Therefore there is no basis for applying statistical or probabilistic predictive methods. Most users of predictions have observed the non-correlation between estimated and actual failure rates.

It is long past time that the electronics design and manufacturing organizations to abandon these invalid and misleading approaches, acknowledge that reliability cannot be estimated from assumptions and calculations, and start using “stress to limits” to find latent failure mechanisms before a product is released to market.  It is true that you cannot derive a time to failure for most systems, but then no test can provide an actual field “life” estimate for a complex electronic system nor do we need to. There is more life than needed in most electronics for most applications.

Fortunately, there is an alternative. A much more pragmatic and effective approach is to find to put most engineering and testing resources to discovery of  overlooked design margins or a weakest link  early in the design process (HALT) and then use that strength and durability to  quickly screen (HASS) for errors during  manufacturing.  HALT and HASS have little to do with a specific type of chamber or chamber capabilities. It is a fundamental change in the frame of reference for reliability development, moving instead  from time metrics to stress/limit metrics. Many have already realized this new frame of reference. Since they have found these methods much more efficient and cost effective for developing robust electronics systems, it gives them a competitive advantage. They are not about to let the world or their competitors know of how successful these methods are.

Here is the opening elements of the work by Wayne, followed by a link to the full paper.

Appendix M: Repair Data Analysis of Abernethy, R.B. (2006), The New Weibull Handbook, 5th ed., available from Dr. R.A. Abernethy, weibull@worldnet.att.net, 536 Oyster Road, North Palm Beach, FL 33408. May 5, 2006

AN APPLICATION OF GRAPHICAL ANALYSIS OF REPAIR DATA

Wayne Nelson, consultant
WNconsult@aol.com, 739 Huntingdon Drive, Schenectady, NY 12309, USA

SUMMARY. This expository article presents a simple and informative non-parametric plot of repair data on a sample of systems. The plot is illustrated with transmission repair data from cars on a preproduction road test.

KEY WORDS: repair data; reliability data; graphical analysis.

1. INTRODUCTION

Purpose. This article presents a simple and informative plot for analyzing data on numbers or costs of repeated repairs of a sample of systems. The plotting method provides a non-parametric graphical estimate of the population mean cu¬mulative number or cost of repairs per system versus age. This estimate can be used to:

1. Evaluate whether the population repair (or cost) rate increases or decreases with age (this is useful for sys¬tem retirement and burn-in decisions),
2. Compare two samples from different designs, production periods, maintenance policies, environ¬ments, operating conditions, etc.,
3. Predict future numbers and costs of repairs,
4. Reveal unexpected information and insight, an impor¬tant advantage of plots.

Overview. Section 2 describes typical repair data. Section 3 de¬fines the basic population model and its mean cumulative function (MCF) for the number or cost of repairs. Sec¬tion 4 shows how to calculate and plot a sample estimate of the MCF from data from systems with a mix of ages. Section 5 explains how to use and interpret such plots.
Dr. Wayne Nelson is a leading expert on analysis of reliability and accelerated test data. He consults and gives training courses for companies and professional societies. For 24 years he consulted across the General Electric Co. and received the Dushman Award of GE Corp. R&D for developments and applications of product reliability data analysis. He was elected a Fellow of the Amer. Statistical Assoc. (1973), the Amer. Soc. for Quality (1983), the Institute of Electrical and Electronics Engineers (1988) for his innovative developments. He was awarded the 2003 Shewhart Medal and the 2010 Shainin Medal of ASQ and the 2005 Lifetime Achievement Award of IEEE for outstanding developments of reliability methodology and contributions to reliability education. He authored three highly regarded books Applied Life Data Analysis (Wiley 1982, 2004), Accelerated Testing (Wiley 1990, 2004), Recurrent Events Data Analysis (SIAM 2003), two ASQ booklets, and 130 journal articles. He can be contacted via WNconsult@aol.com.
Dr. Robert B. Abernathy, www.bobabernethy.com

When an organization uses one reliability metric predominately, there is a propensity to subsequently force organizational behavior in a certain direction, which in turn becomes the social norm. If we fail to question the underlying rationale that motivates our organization to rely on certain metrics in favor of all others, then we also fail to fully comprehend our organization’s collective mindset. Such comprehension is crucial not only as the foundation to any potential change management initiative but also to break into management’s decision making cycle which drives every tier of an organization. Getting inside our executives decision making cycle is the first step in value adding at the core business level, rather than being relegated as a second tier employee that provides little more than a process compliance function. Reliability engineering is often viewed as an “add-on” mandated process by many middle to executive management staff and I proffer that it is not their shortcomings that lead to such a mindset….. it is ours. The reliability professional has the task of not only providing highly skilled analysis but also to educate our organization on how we can do business more efficiently, carry less risk forward and use existing resources more effectively by simply understanding how to integrate our skills into core business processes rather than as an “add-on”.

So, how is any of this related to something as seemingly innocuous as MTBF?

Well, if we have to ask ourselves that question then quite simply, we are not there yet. We are not at the level of professional and organizational understanding that allows us to instantly realize a potential disconnect between what we can truly offer and what we are currently doing. I concede, MTBF may be the right metric however, unless you “know” why it is the right metric then it should be questioned.

The real concern is not necessarily whether MTBF is being used but rather how its use is shaping organizational behavior that in turn potentially manifests into redundant activity, wasted resources, extended schedules, increased risk taking and more importantly…. our own inability to truly step into our management’s decision making cycle and ply our craft in a meaningful, effectual and professional manner.

So, are you exposing your organization to professional errors of commission or omission by not questioning the metrics being used?   How is this then shaping your organizational behavior and more importantly…. what can “we” do to treat ill-informed behavior?

Giving a presentation last week and asked if anyone uses an 85/85 type test, and a couple indicated they did. I asked why?

The response was – just because. We have always done it, or it’s a standard, or customers expected it. The most honest response was ‘I don’t know’.

They why is the test being done? Who is using the information for a decision? What is the value of the test results? If ‘just because’ is the best you can say about a test, why do it?

The same applies to MTBF. Why is it being used and for what purpose and with what value? If the response you find is basically, ‘just because’. Stop using MTBF!

The basic question that then arises is what should we use instead. The answer is or should be obvious – what matters to your customer and your business. If you customer wants uptime – use availability. If you customer wants durability then use reliability.

Reliability is the probability of successfully operating over a stated period of time. As you may know from my previous posts, some confuse MTBF as meaning the same thing. And, as you know, MTBF is a statement about the failure rate, and not a couplet of probability and time. It’s really only have of what’s needed.

Use Reliability. State the probability or percent that survive and state the period of time. 98% survive one year. Easy.

No assumptions about distributions or statistics. No simplifications or distortions. And, it’s straight forward to understand. It means what it means. 98 out of 100 units operate successfully for one year. Easy.

Based on this metric, we can determine or assume life distributions and answer all manner of queries. It’s just a start, yet directly useful and meaningful.

Why? Not just because. Reliability is a measure of what the customer or business needs. It directly relates the number of units that work over a period of time. For example, if we have an one year warranty period and want about 2% or fewer failures during the warranty period. Then saying 98% reliable over 1 year (a bit more positive statement then 2% failures) works just fine.

Sure this could be converted to MTBF – and again I would ask why?

I’ve often railed on and on about the inappropriate use of MTBF over Reliability. The often cited rationale is, “it simpler”. And, I agree, making simplifications is often required for any engineering analysis.

It goes to far when there isn’t any reason to knowingly simply when the results are misleading, inaccurate or simply wrong. The cost of making a poor decision based on faulty analysis is inexcusable.

Using even just a better fit make a big difference. I suggest that instead of using total time divided by total failures – use the time to failure information which you probably already have available for the analysis.

Recently I’ve been working with a few clients with this simple recommendation. And, most opt for a data analysis and plotting package that is relatively inexpensive and very easy to use – Reliasoft’s Weibull++ is one such package.

Another path open to a bit more series student is the freely available, gnu license software package called R. You can learn more and download the software at

http://cran.r-project.org/

While this is primarily a statistics programing/scripting language – it is fully functional for reliability statistics, too.

This morning I ran across a package that creates Weibull plots on the appropriate Weibull scales (just like Weibull plotting graph paper). It also permits all the graphical and analysis control of R. Very powerful and flexible.

I will say the learning curve is a bit steep. It is after all a programing language. And, there are plenty of books, documents, websites, courses, etc. available to get you up to speed.

I used the single line of code

plot.wba(Surv(wbparams.to.ft(5,2,2000)),col=”red”)

to create the plot above. plot.wba is a slightly modified version of the command, plot.wb contained within the weibulltoolkit package. You can learn more about the toolkit from the paper by Jurgen Symynck and Filip De Bal, titled, “Weibull Analysis Using R, in a Nutshell”. you can find the paper at

http://mechanics.kahosl.be/fatimat/images/papers-books/paper-weibull_analysis_using_r_in_a_nutshell.pdf

Which ever software you use – avoiding the simplest route is worth the effort.

During RAMS this year, Wayne Nelson made the point that language matters. One specific example was the substitution of ‘convincing’ for ‘statistically significant’ in an effort to clearly convey the ability of a test result to sway the reader. As in, ‘the test data clearly demonstrates…’

As reliability professionals let’s say what we mean in a clear and unambiguous manner.

As you may suspect, this topic is related to MTBF. Simply saying ‘reliability’ instead of ‘MTBF’ would convey what we really mean. If the message requires specific values, instead of ’50,000 hour MTBF’, say ’98% reliable over two years’. And, if you absolutely have to use MTBF, always add the duration over which the failure rate (1/MTBF) is relevant.

During a panel at RAMS a few few panelist spoke of the state of various reliability related international standards and mentioned the continued use of MTBF. When challenged, which I’ve been known to do on the topic, they defended the continued use of MTBF due to it’s widespread use. They also acknowledged that the common misunderstanding and misuse of MTBF, and agreed that using ‘reliability’ is more meaningful. Yet, they contended that the overall widespread use of MTBF warranted the continued use in the standard’s language.

It is my contention that the standard’s establish and reinforce the language that we as a profession use. We should expect that any standards’ language is clear and easy to understand. While MTBF, in itself is a perfectly meaningful expression, when used correctly, it does not currently communicate the intended message. Changing the term ‘MTBF’ to ‘Reliability’ in standards would encourage our profession and those that rely on reliability standards to elevate the discussion; to speak and write clearly; and, to avoid the communication errors surrounding MTBF.

The acknowledged widespread misunderstanding is further propagated by the repeated appearance in standards. Let’s change that.

One of the objections voiced is the ‘we’ve always done it that way’ type argument. This is related to the early medical profession’s use of Latin, as a means to cloak the professional in an aura of professionalism (educated elite members reinforced with arcane language). A major objective of the reliability profession is to enable the design and management teams to make good decisions while considering the full reliability impact. Using ‘arcane’ or difficult to understand language does not serve our profession nor provide a service to the design and management teams.

We already facing the daunting task of clearly explaining the range of statistical tools we routinely use to solve problems. Adding the term ‘convincing’ will help in that area. Let’s continue to improve our collective language by avoiding the use of MTBF also.

I am a rock climber. Climbing relies on skill, strength, knowledge, a bit of luck, and good gear. Falling is a part of the sport and with the right gear the sport is safe.

I do not know, nor want to know, the MTBF (or MTTF) of any of my climbing gear. Not even sure this information would be available. And, all of the gear I use does have a finite chance of failing every time the equipment is in use. Part of my confidence is the that probability of failure is really low.

The equipment manufacturers tend to stress the robustness of the equipment, the strength of the design to shock loading and so on. The equipment is very strong relative to the load a falling climber may cause. There is very little overlap between the stress and strength distributions – meaning the equipment has a large safety margin or safety factor.

The equipment does wear out with use and time. The literature and climbing community stresses the use of equipment that is in good condition. As harnesses and ropes age the strength reduces and we tend to retire this equipment while it still has potentially years of useful life remaining. Again, we use a safety margin (if paying attention).

The expectation is the equipment will not fail during a reasonable duration of use and stress. This isn’t the only industry that enjoys this goal. AND, they do not use MTBF.

In the climbing industry, a failure tends to lead to death or very serious injury. Again, this isn’t the only industry where this is true. Yet, in the medical equipment, aerospace and transportation industries we commonly see the use of MTBF. That is unfortunate.

The classic formula for availability is MTBF divided by MTBF plus MTTF. Standard. And pretty much wrong most of the time.

Recently working for a bottling plant design team we pursued the design options to improve availability and throughput of the new line. The equipment would remain basically the same, filler, capper, labeler, etc. So we decided to gather the last 6 months or so of operating data which included up and down time. Furthermore the data included time to failure and time to repair information.

The plant traced availability using the classic formula, simply the total operating or up time by the total run time. We learned during plant visits that the longer runs tended to get better availability. And the data, looking at short runs (1 day) versus long runs (5 days) did show a marked difference in availability. It also showed the MTBF went up with longer runs. The MTTR remained pretty constant or went up just a little.

Recall we have the ‘time to’ data. A little work with a spreadsheet and we fit distributions to the data. Weibull, Lognormal and Exponential clearly fit different equipment results. And, we had a wealth of data, permitting very good data analysis and distribution fit decisions. For sake of this article, let’s assume the equipment operating time is well described by a Weibull distribution with a beta less than 1. And the time to repair data is well described by the lognormal distribution. While this wasn’t universal across all the types of bottles and equipment, it will work for the purpose of this article

So why do we commonly use MTBF with the assumption of an underlying exponential distribution? There are many reasons: ignorance, “always done that way”, lack of ‘time to’ data or in rare cases the assumption is valid. Let’s remove the ignorance excuse and ‘always….’ and make the case to get the right data at the same time

Why is using MTBF so bad for availability calculations? Let’s look at one example, the design of the bottling line. There are up to 10 steps in the high volume bottling process and each step involve highly complex and expensive equipment. The design team was balancing the line availability and throughput per shift with the cost of the equipment. The other part of the design consideration was the cost of storing finished goods with the change over time between flavors and bottle sizes. The idea was to add just enough redundant equipment that the line change over time and line availability permitted frequent line changes which significantly reduce finished goods storage costs.

In a perfect plant, each flavor and bottle size combination would have a dedicated line. In reality the expected run times of the flavor and bottle size combinations ranged from a few hours per month to two weeks a month, with most running for less than a week. Also, consider the equipment cost a few million dollars per unit.

Therefore, we built a availability model of the existing line configuration and the various proposed line configurations. The model would permit the simulation of various expected line management policies to determine ability to reduce finished good storage costs. The model would significantly influence the million dollar decisions in the project.

Back to why we want to look at the underlying and very common assumption – MTBF and MTTR often assume the exponential distribution. This distribution does not account for changes in the failure or repair rate. The first hour and any hour of operation after that have exactly the same average failure rate or repair rate.

Recall the observation and supporting data that suggest neither MTBF nor MTTR are constant, both seem to depend to some degree on the length of the run. Well, I’m a statistician and the plant had years worth of data on the equipment. Happiness.

First get the data and determine the best fitting distribution. This is basic regression analysis. Here is an example of the difference between assuming the exponential and fitting a distribution.

<insert graphic of fit showing Weibull and Exponential>

Second, determine how to calculate availability given the fitted distributions. This second step had me hitting the books.

The general formula to calculate availability is based on expected values and not MTBF and MTTR based on exponential distributions. ‘Expected Values’ if you are like most engineers you many only have a vague recollection of this statistical term. Most associated this phrase with the mean or average value, which is mostly true. The availability formula changes from

<insert formula A = mtbf/(mtbf+mttr)>

to this

<insert formula A(t) = Eu[t] / (Ed[t] + Eu[t])>

If your are familiar with the Weibull distribution you recall if beta is equal to 1, the characteristic life is the theta. The same as for an exponential distribution. When the beta is not equal to one, the characteristic life is not equal to theta. The characteristic life is another way to say expected value.

For the exponential distribution the expected value calculation is very commonly used to calculate MTBF.

<insert expected value formula for exponential>

Whereas, for the Weibull distribution the formula is

<insert expected value formula for Weibull>

and, for the Lognormal distribution the formula is

<insert lognormal expected value>

Therefore, with the data properly described with the appropriate distribution we calculate the expected values and determine the availability. Easy.

A bolted hanger along a rock climbing route is often a very welcome site. It provides the climber with safety (by clipping the rope to the bolt); with direction (this is the way); and, with confidence. As climbers we count on the bolts to provide support in case something goes wrong or we need to rest along the route.

A reliability metric is often used in the same way as a climbing bolt. The measure, whether MTBF or Reliability or Failure Rate, provides a sense of assurance that the product in question is performing as expected. The organizations profits are or will be safe. The development team uses the measures as a guide for design and supply chain decisions. And, the measure provides confidence to the organization related to meeting customer expectations around reliability.

To extend the analogy a bit further, consider the image of the bolt closely. The strength relies on the bolt attachment within the rock. It is not visible. Once the bolt is set, the climber trusts the integrity of the attachmentA reliability metric, likewise, often hides the underlying data. The strength of the measure or summary of the data relies on the design of the metric, the analysis, the assumptions, and the underlying data. The more accurately the measure conveys the data the better the ‘attachment to the rock’.

In conversation we often desire to be true, beneficial and timely. With metrics the same is often at play. Metrics that are false, harmful or late are of little value. I suspect you agree.

Then why are we still clipping our reliability discussions on MTBF?

True? Rarely is the underlying data accurately modeled using the constant failure rate  assumption.

Beneficial? As you undoubtedly experienced, the metric itself leads to confusion and misunderstanding.

Timely? While it is possible to make predictions quickly, that’s only 1 of 3 of criteria.

Designing and manufacturing a product requires strong and helpful metrics. MTBF is like a loose bolt that is off route. Take a look at your use of MTBF and critically assess the truth it conveys, and how it is understood among your team.